AutoGraph Architecture

Three-Layer Knowledge Graph

AutoGraph organizes data in ArangoDB across three layers. Each layer has a clear owner, a set of collections, and a specific purpose.

All collection names are prefixed with your project name. For example, if the project is myapp, collections will be myapp_sources, myapp_domains, and so on.

An AutoGraph project is made of two graphs that work as a pipeline: the Corpus Graph ({project}_CorpusGraph) organizes the documents you feed in, and the Knowledge Graph ({project}_kg) is the GraphRAG graph built inside each partition. They meet only at the partition, through a shared-property join between rags.rag_partition_id and Documents.partition_id — not a stored edge, so you cannot traverse from one graph to the other. Together they make up the Context Graph, the complete output of AutoGraph for the project.

Collections per layer

graph TD
  subgraph "`**Layer 1** — Categories  (defined by you)`"
    modules["`**modules**
      (vertex: one per category)`"]
  end

  subgraph "`**Layer 2** — Corpus Graph  (built by AutoGraph)`"
    sources["`**sources**
      (vertex: one per document)`"]
    similarities["`**similarities**
      (edge: source ↔ source, label SIMILAR_TO)`"]
    domains["`**domains**
      (vertex: one per Leiden cluster)`"]
    corpus_relations["`**corpus_relations**
      (edge: labels IN_DOMAIN · HAS_CLUSTER · INGESTED_AS)`"]
    rags["`**rags**
      (vertex: strategy profiles — added by strategizer)`"]
  end

  subgraph "`**Layer 3** — Knowledge Graph  (built by Importer)`"
    Documents["`**Documents**
      (vertex: original documents)`"]
    Chunks["`**Chunks**
      (vertex: text chunks)`"]
    Entities["`**Entities**
      (vertex: extracted entities — FullGraphRAG only)`"]
    Communities["`**Communities**
      (vertex: entity clusters — FullGraphRAG only)`"]
    Relations["`**Relations**
      (edge: all relationships)`"]
  end

  modules -->|HAS_CLUSTER| domains
  sources ---|SIMILAR_TO| similarities
  sources -->|IN_DOMAIN| domains
  domains -->|INGESTED_AS| rags
  rags -->|orchestration| Documents
  Documents --- Chunks
  Chunks --- Entities
  Entities --- Communities

Layer 1 and 2

AutoGraph builds the corpus by creating collections in Layers 1 and 2. These collections are organized into a named graph called {project}_CorpusGraph.

Edge labels in the Corpus Graph

AutoGraph assigns semantic labels to edges in the Corpus Graph to distinguish different relationship types:

• SIMILAR_TO: Applied to edges in the similarities collection connecting semantically similar documents. These edges include a similarity_score field (0.0-1.0) computed via vector similarity, BM25 lexical search, and Reciprocal Rank Fusion.
• IN_DOMAIN: Applied to membership edges in the corpus_relations collection, linking documents from the sources collection to their cluster vertex in the domains collection.
• HAS_CLUSTER: Edges in the corpus_relations collection connecting module vertices to their clusters. Links from the modules collection to the domains collection.
• INGESTED_AS: Edges in the corpus_relations collection connecting clusters to their RAG strategy profiles. Links from the domains collection to the rags collection.

Labels are stored in the label field on each edge document. AQL queries can filter by label to select specific relationship types (e.g., FILTER edge.label == "SIMILAR_TO").

Layer 3

The Importer constructs Layer 3 by processing documents into a detailed knowledge graph stored in the named graph {project}_kg. This layer contains the actual document content, text chunks, and optionally extracted entities and communities, depending on your chosen RAG strategy.

Layer 3 collections share the same {project}_ prefix. Each document in Layer 3 carries a partition_id field so data from different partitions coexists in the same collections.

The named graph {project}_CorpusGraph ties Layers 1 and 2 together. It contains two edge definitions:

• similarities (connecting sources to sources),
• corpus_relations (connecting sources, domains, modules, and rags).

Replication factor

AutoGraph creates every collection and graph of both layers with a replication factor of 3, or with the cluster’s minReplicationFactor if that is higher. The value is fixed and cannot be configured; a single-server deployment is unaffected.

Where collections inherit their replication factor from a prototype collection through distributeShardsLike, as in a OneShard database, the inherited value stands and AutoGraph leaves it alone.

The Importer uses the same replication factor for the Knowledge Graph collections it creates.

Node fields

When you open a node in the Graph Visualizer (right-click a node and choose View node), a Properties panel lists that node’s fields. The tables below show the fields you can expect on one node of each type, with example values. The system fields _id, _key, and _rev exist on every node and are omitted. Collection names use the {project}_ prefix (the examples below come from a project whose prefix is SG_).

Corpus-graph nodes

sources — one ingested source document

domains — a topic cluster of similar sources

rags — the RAG-partition configuration (written by the RAG Strategizer)

The divergence fields are only used for FullGraphRAG partitions. The divergence is calculated from the entities of a partition, and a reclustering rebuilds its communities. A VectorRAG partition has neither collection, see Layer 3, so it is never flagged for reclustering.

modules — the top-level corpus container

Knowledge-graph nodes

Documents — a full document inside a partition (provenance)

Chunks — a token-sized passage, the retrieval unit

Entities — an extracted entity, a node in the graph

The set of entity_type values you see across a corpus is the union of every partition’s ontology, so a multi-cluster corpus shows more distinct types than any single partition’s 8–12.

Communities — a cluster of related entities

Edge fields

Each edge collection records its relationship name in a different field, plus a few extras. This is the distinction that is easiest to misread, so it is worth checking the field name before writing AQL filters.

Knowledge-graph relationships in Relations also line up one-to-one with the collections each edge connects:

Things people misread

• Two graphs, not one. The Corpus Graph and the Knowledge Graph are separate; they meet only at the partition.
• The relationship type lives in different fields. corpus_relations and similarities use a label field; Relations uses a type field. Both also match the _from/_to collections.
• Arrow direction is not build order. Arrows show ownership/provenance, not the sequence in which the pipeline created the nodes.
• rags → Documents is a property link, not a stored edge. They are joined by the rags side’s rag_partition_id matching Documents.partition_id, not by an edge in any collection.
• module is both a vertex and a property. It is a node in the modules collection and also a field stored on sources, domains, and rags.
• The corpus embedding is not the retrieval index. The sources embedding is a single-vector clustering signature (from the first chunk); the full text is chunked and indexed separately in the Knowledge Graph.

Complete Pipeline

The diagram below shows the full end-to-end API flow across all three layers. Solid arrows are the sequential pipeline steps; dashed arrows are polling and inspection calls that you can make at any time.

flowchart TD

    Client["Client / HTTP REST"]

    %% Entry points
    Client -->|Step 0| HEALTH
    Client -->|Step 1| FM
    Client -.->|legacy alternative| IMP
    Client -->|Step 2 - 202| BUILD
    Client -.->|poll anytime| STATUS
    Client -->|Step 3 - 202| STRAT
    Client -.->|poll job progress| STRATJOB
    Client -.->|inspect anytime| GETSTRAT
    Client -.->|optional override| PATCHSTRAT
    Client -->|Step 4 - 202| ORCH
    Client -.->|poll anytime| ORCHSTATUS

    %% Health
    HEALTH["`<code>GET /v1/health</code>
      Confirm service status is <code>SERVING</code>`"]

    %% Layer 1
    subgraph L1 ["`**Layer 1** - Categories`"]
        FM["`File Manager RAG inputs
          Upload under scope
          <code>[project, category]</code>`"]
        IMP["`<code>POST /v1/import-multiple</code> (legacy)
          Upload documents to the service
          Attach a <code>module</code> label
          Files stored on disk
          No document-level updates later`"]
    end

    %% Layer 2
    subgraph L2 ["`**Layer 2** - Corpus Graph`"]

        %% Build Pipeline
        subgraph BUILD_PIPE [Corpus Build Background Task]
            BUILD["`<code>POST /v1/corpus/builds</code>
              Returns <code>corpus_build_id</code> + <code>graph_name</code>`"]
            STATUS["`<code>GET /v1/corpus/builds/{id}</code>
              Status, progress, counts, error_code`"]

            B1["`Resolve inputs
              (<code>categories</code> via File Manager,
              or legacy imported files on disk)`"]
            B2["`Extract text
              (File Parsing Service - preview scope
              first 4800 chars, no images;
              imported files parsed in-process)`"]
            B3["`Generate embeddings
              (first 1200 tokens)`"]
            B4["Insert document nodes"]
            B5["`Build similarity edges
              (Vector + BM25 + RRF)`"]
            B6["Store similarity edges"]
            B7["Leiden clustering per category"]
            B8["Create cluster nodes"]
            B9["Link docs to clusters"]
            B10["Build corpus relations"]
            B11["Register named graph"]

            BUILD --> B1 --> B2 --> B3 --> B4 --> B5 --> B6 --> B7 --> B8 --> B9 --> B10 --> B11
        end

        %% Strategizer
        subgraph STRAT_PIPE [RAG Strategizer Background Task]
            STRAT["`<code>POST /v1/rag-strategizer/analyze</code>
              Returns <code>strategize_job_id</code>`"]
            STRATJOB["`<code>GET /v1/rag-strategizer/jobs/{id}</code>
              Live progress + cluster counts`"]
            GETSTRAT["`<code>GET /v1/rag-strategizer/strategy</code>`"]
            PATCHSTRAT["`<code>PATCH /v1/rag-strategizer/strategy/{cluster_id}</code>
              Override strategy for a specific cluster`"]

            S1["Read clusters"]
            S2["`Generate per-cluster ontology
              (8-12 entity types via LLM)`"]
            S3["Compute complexity score"]
            S4{"Rank clusters by complexity"}
            S5["Assign FullGraphRAG"]
            S6["Assign VectorRAG"]
            S7["Store strategy profiles"]

            STRAT --> S1 --> S2 --> S3 --> S4
            S4 -->|Top N%| S5 --> S7
            S4 -->|Remaining| S6 --> S7
        end
    end

    %% Layer 3
    subgraph L3 ["`**Layer 3** - Knowledge Graph`"]

        subgraph ORCH_PIPE [Orchestration Background Task]
            ORCH["`<code>POST /v1/orchestrate</code>
              Returns <code>orchestration_id</code>
              and <code>unmatched_file_ids</code>`"]
            ORCHSTATUS["`<code>GET /v1/orchestrate/{id}</code>
              Counters + per-partition jobs`"]

            O0{"`Any stale partitions?
              (strategies exist and
              not already in the KG)`"}
            O1["Load jobs for stale partitions only"]
            O2["Spawn Importer replicas"]
            O3["`Submit one job per partition
              with its strategy
              (<code>FullGraphRAG</code> or <code>VectorRAG</code>)`"]
            O4["Poll Importer until done"]
            O5["Tear down workers"]

            ORCH --> O0
            O0 -->|none - 409| ORCHREJECT["Nothing to orchestrate"]
            O0 -->|stale found| O1 --> O2 --> O3 --> O4 --> O5
        end
    end

    %% Cross-layer connections
    FM -->|storage locations| B1
    IMP -->|files on disk| B1
    B11 -->|Corpus ready| S1
    S7 -->|Strategies ready| O0

Project-level operations

These endpoints sit outside the sequential pipeline. You can inspect and configure a project at any time. Both delete endpoints are guarded against work that is already running and return 409 while it is: a corpus build, an orchestration, or a document delete for a category delete, and additionally a strategizer run for a project delete.

flowchart LR

    Client["Client / HTTP REST"]

    Client -.->|inspect anytime| OVERVIEW
    Client -.->|configure| MODELCFG
    Client -.->|maintenance| DELCAT
    Client -.->|teardown| DELPROJ

    OVERVIEW["`<code>GET /v1/projects/{project}/overview</code>
      Corpus + KG cards, categories,
      read-time staleness`"]
    MODELCFG["`<code>PUT /v1/projects/{project}/model-config/credentials</code>
      Chat + embedding provider, model,
      secret profiles - validated on write`"]
    DELCAT["`<code>DELETE /v1/projects/{project}/categories/{category}</code>
      409 while a build, an orchestration,
      or a document delete runs`"]
    DELPROJ["`<code>DELETE /v1/projects/{project}</code>
      Removes every project-owned resource,
      then the service tears itself down
      409 while a build, an orchestration,
      a strategizer run, or a document delete runs`"]

    L2R[("`**Layer 1-2 - Corpus Graph**
      modules, sources, similarities,
      domains, corpus_relations, rags`")]
    L3R[("`**Layer 3 - Knowledge Graph**
      {project}_kg partitions`")]
    FMR[("`**File Manager**
      RAG inputs`")]
    META[("`**Project metadata**
      provider, model, secret profile ids`")]

    OVERVIEW -.->|reads| L2R
    OVERVIEW -.->|reads| L3R
    OVERVIEW -.->|reads counts| FMR

    MODELCFG -->|"`validates key with provider
      then persists`"| META

    DELCAT -->|1 - remove KG partitions| L3R
    DELCAT -->|2 - remove corpus graph data| L2R
    DELCAT -->|3 - delete_files true only| FMR
    DELPROJ -->|drops graphs and collections| L3R

    DELPROJ -->|drops graphs and collections| L2R
    DELPROJ -->|delete_files true only| FMR
    DELPROJ -->|removes the project node| META