How automatic indexing works

TerminusDB 12.1 can embed and index your documents by itself, so you can search them with VectorLink. But it only does this when three things are in place. If one of them is missing, nothing is indexed, and search returns no hits. TerminusDB does not tell you why.

This page explains the three things, what happens automatically after that, and what to expect from the first search in the release candidate.

The three things you need

1. The server runs with the VectorLink backend

Nothing is indexed unless the server runs with this environment variable:

TERMINUSDB_INDEXER_BACKEND=vectorlink

The default is none. With none, commits work as normal, but no document is ever sent for indexing.

The docker-compose.yml in the vectorsearch folder of the TerminusDB repository sets this for you. If you start TerminusDB another way (for example with a plain docker run or the CLI), set the variable yourself.

2. The database turns indexing on

At database level, you turn indexing on in the schema @context document, under @metadata.terminusdb.options:

{
  "@type": "@context",
  "@base": "terminusdb:///data/",
  "@schema": "terminusdb:///schema#",
  "@metadata": {
    "terminusdb": {
      "options": ["store_indices"]
    }
  }
}

• store_indices turns on automatic indexing on every commit.
• store_clustering is a sibling option. It adds the second, clustering embeddings. See Clustering Embeddings.

A commit that changes the schema reads these options and turns indexing on or off for the database.

With the embedded VectorLink backend, the engine also indexes classes that have embedding metadata (see the next step). We still suggest you set store_indices, so that your intent is clear in the schema.

3. Each class says what to embed

At class level, each class that you want to search needs an @metadata.embedding block with two parts:

• query — a GraphQL query that fetches the fields of one document.
• template — a Handlebars template that turns those fields into the text that is embedded.

A class without this block is never embedded. Here is the Person class from the AI agent prompt, together with a @context that turns on store_indices:

[
  {
    "@type": "@context",
    "@base": "terminusdb:///data/",
    "@schema": "terminusdb:///schema#",
    "@metadata": { "terminusdb": { "options": ["store_indices"] } }
  },
  {
    "@type": "Class",
    "@id": "Person",
    "name": "xsd:string",
    "friends": { "@type": "Set", "@class": "Person" },
    "@metadata": {
      "embedding": {
        "query": "query($id: ID){ Person(id: $id) { name } }",
        "template": "{{name}}"
      }
    }
  }
]

Send it to the schema graph with full_replace=true, because it includes the @context:

curl -u admin:$PASS -X POST \
  'http://localhost:6363/api/document/admin/agent_demo?graph_type=schema&author=me&message=Add+schema&full_replace=true' \
  -H 'Content-Type: application/json' \
  -d @schema.json

Who adds this block? If you write your schema through the TerminusDB MCP server, it adds the embedding metadata for you (you can also pass an indexing option to control it). If you write the schema through the plain HTTP document API, or a client library, nothing is added for you. You must put @metadata.embedding in the schema yourself.

For help with good templates, see the Embeddings Field Guide.

What happens automatically

When all three things are in place:

• Every new commit is queued for indexing. The work runs in the background, so your commit returns at once. The index catches up a moment later.
• The first search catches up the index. If the index is behind the head commit, the search asks the indexer to catch up. This also indexes documents that you committed before the setup was complete. The first search may return no results, or only some. Wait a short time and search again.

You can see the state of the index at any time:

curl -u admin:$PASS http://localhost:6363/api/index/admin/agent_demo

Look at searchable_documents. It should match the number of documents you expect to find.

What to know about the first search

There is one small gap in the release candidate:

• The first search can be early. Existing documents are indexed automatically on the first search. Until the catch-up is done, that search can return no results, or only some. Wait a short time and search again.

• You can index ahead of time. A manual reindex is optional. Use it if you do not want to wait for the first search, for example before a demo:

  Example: Bash

  Copy

  curl -u admin:$PASS -X POST http://localhost:6363/api/index/admin/agent_demo

  The general form is POST /api/index/{org}/{db}, with an empty body.

• Indexing catches up by itself. The indexer works through commits one at a time. If a commit is not indexed yet, for example because the process stopped, indexing picks it up again at the next search (when auto-indexing is on) or the next time indexing is triggered. Nothing needs to be redone by hand.

A later release may start the catch-up when the indexer starts, so that the first search is ready at once.

Troubleshooting: search returns nothing

Go through this list in order:

1. Backend. Does the server run with TERMINUSDB_INDEXER_BACKEND=vectorlink? With Docker Compose, check with docker compose exec terminusdb env | grep INDEXER.
2. Database option. Does the schema @context have "options": ["store_indices"] under @metadata.terminusdb? Get the schema with GET /api/document/{org}/{db}?graph_type=schema and look.
3. Class metadata. Does each class you search have an @metadata.embedding block with query and template?
4. Reindex. If search still returns nothing after a retry, run POST /api/index/{org}/{db}, then check GET /api/index/{org}/{db}. searchable_documents should not be 0.
5. Embeddings model. On first start, the local model can take a few minutes to load. While it loads, search returns 503. Wait and try again.

See also

• Set it up with an AI agent — a prompt that sets up the whole stack and runs a first search.
• Versioned Search — overview of VectorLink.
• TerminusDB Push Indexing — a worked example across three commits.
• Embedded Engine — run VectorLink inside TerminusDB.
• Searching — query modes, filters and the response.