Incremental Updates in the Importer

After the initial build, the Layer 3 knowledge graph has to keep up with documents that are added, removed, or replaced. It contains the documents, chunks, entities, communities, and relationships that the Importer owns.

Apart from the import calls, the Importer offers one endpoint for this. Recluster rebuilds the community layer of a single partition without importing anything again. The Importer has no delete and no update endpoint. Removing and replacing documents is handled by AutoGraph, see Deleting a document and Updating a document.

Inserting a document

Inserting is a normal import into an existing partition_id:

• POST /v1/import for a single file.
• POST /v1/import-multiple for one or more files. It returns a job_id that you can poll with GET /v1/jobs/{job_id}.

The Importer builds the graph data for the files and writes Documents, Chunks, Entities, Communities, and Relations to {project}_kg, depending on the rag_mode. Every record gets the partition_id of the request and an import_number that identifies the batch. If you import into the same partition again, additional batches are created. Reclustering consolidates these batches before it rebuilds the communities.

Deleting a document

The Importer has no delete endpoint. Documents are removed from the knowledge graph through AutoGraph, which cleans up Layers 1 and 2 as well as the Layer 3 data of the document:

Call it with the File Manager IDs in file_ids, or with doc_names as a fallback, and the category the documents belong to. The call is synchronous and removes the Layer 3 data as well, using AQL queries of its own. No Importer job is spawned and there is nothing to poll: the response is the final result and reports what was removed, along with the affected Layer 3 partitions. For the request and response fields, see Delete documents.

What a deletion removes from the Layer 3 collections of {project}_kg:

• The Documents vertices of the file
• Chunks that are left without a document
• Entities that are left without a MENTIONED_IN chunk
• Communities that are left empty, including parent communities that become empty through SUB_COMMUNITY_OF
• SemanticUnits that are left without a document
• The related Relations edges

Entities and communities that other files still use are kept. See Knowledge graph collections for the collections involved.

A deletion leaves the community layer of a full_graphrag partition behind the entities it now contains. AutoGraph recalculates the divergence of every affected partition and flags the ones that should be reclustered.

Updating a document

The Importer has no endpoint for updating a document in place either. Replacing the content of a document that is already in the graph is an AutoGraph operation:

It takes the new version from the File Manager by file_id and accepts no inline content. It removes the old version, including its Layer 3 data, and adds the new version to Layers 1 and 2. The new content only reaches Layer 3 once you run a targeted orchestration with that file_id, which submits an import to the Importer. AutoGraph resolves the partitions itself, so you do not name one. For the phases of an update and its response fields, see Update documents.

Divergence

The divergence shows how far the community layer of a full_graphrag partition has drifted from its current entities and relationships after inserts, deletes, and updates. It is calculated by AutoGraph, not by the Importer. No Importer endpoint returns a divergence score.

If the divergence gets above the threshold of the partition, AutoGraph flags the partition as needing a reclustering. It does not start one on its own. You decide whether the refresh is worth the cost and then start it, which in turn calls POST /v1/recluster for the affected partition_id. For the formula, the threshold, and the lifecycle, see Partition divergence and reclustering.

The divergence does not apply to vector_rag partitions because they have no Entities or Communities.

Reclustering

Rebuilds the community layer of a single full_graphrag partition without importing the documents again. The call is asynchronous. It returns a job_id right away, and you can poll GET /v1/jobs/{job_id} until is_terminal is true.

Only full_graphrag partitions have a community layer. A vector_rag partition has no Entities or Communities, see Knowledge graph collections, so there is nothing to consolidate, cluster, or replace.

Request

{
  "partition_id": "legal_0_a"
}

What the Importer does

1. Consolidate the batches, if needed. If the partition has more than one import batch, or is not already a single batch with import_number=1, the Importer groups the entities whose names match after normalization and picks a winner. Names are compared with the surrounding quotes stripped, whitespace trimmed, and the rest upper-cased, so "ACME Corp" and acme corp end up in the same group. The winner is the entity with the most RELATED_TO edges to other entities. Ties are broken by the highest import_number and then by the entity _key, so the outcome is deterministic. Its content is kept as it is and moved to import_number=1 if necessary. The other entity vertices are removed and their RELATED_TO and MENTIONED_IN edges are redirected to the winner. Self-references are removed and RELATED_TO edges that end up between the same two entities are folded into one.

   Consolidation does not merge content. The winner keeps its own description and embedding, and the descriptions and embeddings of the entities that are removed are discarded, not combined. The same is true on the edges: only weight and source_id accumulate when duplicate RELATED_TO edges are folded together, and weight is summed per distinct source_id rather than per edge. Nothing else on the edge is merged.

2. Load the graph. The Entities and RELATED_TO edges of the partition are loaded into an in-memory graph, keyed by entity name, with the source_id rebuilt from MENTIONED_IN. Entities without a name are skipped without an error. Graphs that are not fully connected are accepted.

3. Cluster and summarize. The Leiden clustering runs and the community reports are generated, in the same way as during a full import.

4. Swap in the new communities. The old Communities and the community edges (IN_COMMUNITY, SUB_COMMUNITY_OF) are only replaced once the new ones are ready. New communities, embeddings, and cluster assignments for the entities are written with import_number=1.

What is kept: Documents, Chunks, Entities, and the relations that are not community-related (PART_OF, MENTIONED_IN, RELATED_TO). Reclustering updates which community an entity belongs to. It does not recreate the relations that are kept.

Duration and performance impact

Reclustering is not a full import. It skips the extraction of documents, chunks, and entities, and only rebuilds the community layer. Most of the runtime goes into generating the community reports with the LLM, which is the same work as in the community stage of a full import, plus the optional consolidation and the Leiden clustering.

• Expect longer jobs for partitions with many entities, relationships, or communities.
• Partitions with multiple batches need to be consolidated first.
• A reclustering holds the import lock for the whole job. Until it is done, further recluster calls return HTTP 503 (gRPC UNAVAILABLE), and import calls return "success": false.
• The old communities are only removed once the new ones are ready, so if the job fails halfway through, the previous community layer is still there.

HTTP example

curl -X POST https://<EXTERNAL_ENDPOINT>:8529/graphrag/importer/<SERVICE_ID_POSTFIX>/v1/recluster \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <your-jwt-token>" \
  -d '{
    "partition_id": "legal_0_a"
  }'

{
  "success": true,
  "job_id": "<uuid>",
  "message": "Recluster started for partition legal_0_a. Use GET /v1/jobs/<uuid> to monitor progress."
}

Troubleshooting

Next steps

• Incremental Graph Updates in AutoGraph: The workflow that inserts, deletes, updates, and reclusters documents across all three layers.
• Architecture: The knowledge graph collections and the lifecycle of asynchronous jobs.
• Import Files: Single-file and multi-file import workflows.
• Error Handling: Synchronous error codes and markers for asynchronous failures.