Integration with AutoGraph

When to use the Importer standalone vs. with AutoGraph

The Importer supports two distinct usage patterns depending on your needs.

Standalone Importer (without partitioning)

If you are building a single knowledge graph from your documents and do not need multiple partitions or automated domain discovery, use the Importer directly. You can call it through:

• AutoGraph Studio, which guides you through configuring and running the Importer step by step.
• The Import API (POST /v1/import or POST /v1/import-multiple), which gives you full control over all parameters.

In this mode, you manage the Importer lifecycle yourself: you install it, submit import requests, and monitor the results.

Automated via AutoGraph (multi-partition, strategy-aware builds)

When you need to process large or heterogeneous document collections, AutoGraph manages the Importer for you. AutoGraph automatically discovers knowledge domains in your data, assigns the optimal RAG strategy (full_graphrag or vector_rag) per domain, and spawns Importer workers to build partitioned knowledge graphs.

In this mode, you do not call the Importer directly. AutoGraph handles replica creation, job submission, partition_id and rag_mode assignment, status polling, and teardown.

The rest of this page explains how this automated flow works.

Three-layer architecture

AutoGraph organizes data across three layers. Each layer has a clear owner and purpose:

The Importer builds Layer 3 only. All Layer 3 collections carry a partition_id field so that data from different clusters coexists in the same collections.

For full details on Layers 1 and 2, see the AutoGraph Architecture documentation.

How AutoGraph spawns import jobs

The same module string flows from files to clusters to strategies to Importer partitions:

1. You import documents into AutoGraph with a module label (e.g., "legal").
2. AutoGraph’s corpus build clusters documents within each module using Leiden community detection.
3. The RAG strategizer analyzes each cluster and assigns either FullGraphRAG or VectorRAG.
4. Each assignment gets a rag_partition_id derived from the cluster key. For example, cluster cluster_legal_0 becomes legal_0_a (FullGraphRAG) or legal_0_b (VectorRAG).
5. Orchestration (POST /v1/orchestrate) spawns Importer worker replicas and submits one import job per partition.
6. Each job payload includes partition_id (set to the rag_partition_id) and rag_mode (set to the assigned strategy).

flowchart LR
  module["`Module
    **legal**`"]

  module --> cluster0["cluster_legal_0"]
  module --> cluster1["cluster_legal_1"]
  module --> cluster2["cluster_legal_2"]

  cluster0 -->|full_graphrag| part0["`Partition
    **legal_0_a**`"]
  cluster1 -->|full_graphrag| part1["`Partition
    **legal_1_a**`"]
  cluster2 -->|vector_rag| part2["`Partition
    **legal_2_b**`"]

  part0 --> orch["Orchestration"]
  part1 --> orch
  part2 --> orch

  orch --> job0["Importer job 1"]
  orch --> job1["Importer job 2"]
  orch --> job2["Importer job 3"]

Under normal operation you do not call the Importer directly; AutoGraph handles replica creation, job submission, status polling, and teardown. Call the Importer yourself only for standalone imports or advanced scenarios (e.g., re-running a single partition with custom settings).

After the initial build, AutoGraph also uses the Importer for Incremental Graph Updates in Layer 3. It submits imports for new and changed files, and it can recluster a partition whose communities have drifted, but only if you ask for it. AutoGraph flags the drift and never reclusters on its own. Removing a document from Layer 3 is not an Importer call, AutoGraph handles it as part of its own delete and update operations. See Incremental Updates.

How partition_id maps to the Corpus Graph

• AutoGraph’s rag_partition_id (cluster key + strategy suffix _a/_b) is passed as partition_id in the import request payload.
• The Importer stores this value on every document, chunk, entity, community, and relation in the Knowledge Graph.
• Multiple partitions coexist in the same ArangoDB collections; filter by partition_id when querying.
• When inspecting Layer 3 data, the partition_id traces back to a specific Leiden cluster and its RAG strategy in the Corpus Graph.

When using the Importer standalone (without AutoGraph), you can set partition_id to any string to logically separate different import batches within the same collections. See the partition_id parameter reference for details.

Related resources

• AutoGraph overview: What AutoGraph is and why to use it.
• AutoGraph Architecture: The three-layer knowledge graph architecture and ArangoDB collections.
• AutoGraph Design Guide: How to structure your data with modules and layers.
• Corpus Build: Create and monitor corpus builds for document clustering.
• RAG Strategizer: Analyze clusters and assign RAG strategies.
• Orchestration: Spawn Importer workers and execute pipeline builds.