RAG Tool

Give your agents access to document knowledge bases with background indexing, multiple retrieval strategies, and hybrid search.

Overview

The rag toolset lets agents search through your documents to find relevant information before responding. Knowledge bases are declared once at the top of the config under rag: and then referenced from any agent via type: rag, ref: <name>. Docker Agent supports:

• Background indexing — Files are indexed automatically and re-indexed on change
• Multiple strategies — Semantic embeddings, BM25 keyword search, and LLM-enhanced search
• Hybrid search — Combine strategies with result fusion for best results
• Reranking — Re-score results with specialized models for improved relevance

RAG is the strategy to reach for when a document collection is too large to inline directly, or gets queried repeatedly across turns/sessions — see Choosing a Large-Input Strategy for how it compares to @//attach attachments and prompt files.

Quick Start

rag:
  my_docs:
    tool:
      description: "Technical documentation"
    docs: [./documents, ./some-doc.md]
    strategies:
      - type: chunked-embeddings
        embedding_model: openai/text-embedding-3-small
        database: ./docs.db
        vector_dimensions: 1536

agents:
  root:
    model: openai/gpt-4o
    instruction: |
      You have access to a knowledge base. Use it to answer questions.
    toolsets:
      - type: rag
        ref: my_docs

Retrieval Strategies

Chunked Embeddings (Semantic Search)

Uses embedding models to find semantically similar content. Best for understanding intent, synonyms, and paraphrasing.

strategies:
  - type: chunked-embeddings
    embedding_model: openai/text-embedding-3-small
    database: ./vector.db
    vector_dimensions: 1536
    similarity_metric: cosine_similarity
    threshold: 0.5
    limit: 10
    embedding_batch_size: 50
    chunking:
      size: 1000
      overlap: 100

Semantic Embeddings (LLM-Enhanced)

Uses an LLM to generate semantic summaries of each chunk before embedding, capturing meaning and intent. Best for code search and understanding implementations.

strategies:
  - type: semantic-embeddings
    embedding_model: openai/text-embedding-3-small
    vector_dimensions: 1536
    chat_model: openai/gpt-4o-mini
    database: ./semantic.db
    ast_context: true # include AST metadata
    chunking:
      size: 1000
      code_aware: true # AST-aware chunking

BM25 (Keyword Search)

Traditional keyword matching using the BM25 algorithm. Best for exact terms, technical jargon, and code identifiers.

strategies:
  - type: bm25
    database: ./bm25.db
    k1: 1.5 # term frequency saturation
    b: 0.75 # length normalization
    threshold: 0.3
    limit: 10
    chunking:
      size: 1000
      overlap: 100

Hybrid Search

Combine multiple strategies for best results. Strategies run in parallel and results are fused together:

rag:
  hybrid:
    docs: [./docs]
    strategies:
      - type: chunked-embeddings
        embedding_model: openai/text-embedding-3-small
        database: ./vector.db
        vector_dimensions: 1536
        limit: 20
        chunking: { size: 1000, overlap: 100 }
      - type: bm25
        database: ./bm25.db
        limit: 15
        chunking: { size: 1000, overlap: 100 }
    results:
      fusion:
        strategy: rrf # Reciprocal Rank Fusion
        k: 60
      deduplicate: true
      limit: 5

Fusion Strategies

Reranking

Re-score retrieved documents with a specialized model to improve relevance:

results:
  reranking:
    model: openai/gpt-4o-mini
    top_k: 10 # only rerank top 10
    threshold: 0.3 # minimum score after reranking
    criteria: |
      Prioritize official documentation over blog posts.
      Prefer recent information and practical examples.
  limit: 5

Supported reranking providers: DMR (native /rerank endpoint), OpenAI, Anthropic, Gemini.

Code-Aware Chunking

For source code, enable AST-based chunking to keep functions and methods intact:

chunking:
  size: 2000
  code_aware: true # Uses tree-sitter for AST-based chunking

Indexing failures, retries and backoff

When a knowledge-base fails to start — because the embedding provider is rate-limiting your requests or returning a transient server error — Docker Agent spaces out retry attempts with bounded exponential backoff instead of hammering the provider on every agent turn.

What triggers backoff

Backoff applies only to HTTP 429 rate-limit responses from the embedding or model provider — the one signal that reliably reaches the toolset gate. Other errors (5xx, 408) are handled per-file within the indexing run and do not arm the gate. These are the current gate triggers:

Retry policy and parameters

The backoff is bounded exponential with additive jitter:

• Base delay: 15 seconds
• Maximum delay: up to ~6 minutes (5-minute cap plus up to 20% additive jitter)
• Growth: doubles after each consecutive retryable failure (15s → 30s → 1m → 2m → 4m → 5m)
• Jitter: each wait is a random value in [nominal, 1.2×nominal] (additive 0–20%) so concurrent knowledge-base sources spread their retries and avoid hammering the provider together
• Retry-After override: if the embedding provider responds with a Retry-After header, that hint overrides the computed delay (capped at the 5-minute maximum, with the same additive jitter applied to spread concurrent retries)

The gate is a lightweight wall-clock check — it creates no background threads or timers. A Stop command or agent shutdown takes effect immediately regardless of how much of the backoff window remains.

Operational impact

Before: a rate-limited knowledge base was re-indexed on every agent turn — max_indexing_concurrency × max_embedding_concurrency concurrent provider calls could relaunch within milliseconds, easily tripping rate limits for both the knowledge base and the agent's own model calls.

After: retries are spaced out and jittered so the provider has room to recover before the next attempt. The agent continues working with any other toolsets that are not affected.

What you will see

• Docker Agent logs a single warning when a knowledge base first fails to start. Repeated failures in between are logged at debug level only, so you are not flooded with alerts on every turn. Recovery is intentionally silent — the tool appearing in the agent's tool list is the signal that indexing succeeded.
• The knowledge-base tool does not appear in the agent's tool list until indexing succeeds. A successful start is silent — the tool is listed and the agent uses it.

Troubleshooting repeated 429 errors

If you see persistent 429 errors in the logs:

1. Check provider rate limits. Your embedding API key may have a low requests-per-minute quota. Upgrading the plan or using a different API key can help.

2. Reduce concurrency. The chunked-embeddings and semantic-embeddings strategies accept max_indexing_concurrency (default 3) and max_embedding_concurrency (default 3) parameters. Lowering these reduces simultaneous requests:

   rag:
     docs:
       docs: [./knowledge-base]
       strategies:
         - type: chunked-embeddings
           max_indexing_concurrency: 1
           max_embedding_concurrency: 1
   

3. Use a model with a higher quota. Some providers offer higher rate limits on specific embedding model tiers.

Debugging RAG

Enable debug logging to see retrieval details:

$ docker agent run config.yaml --debug --log-file debug.log

Look for log tags: [RAG Manager], [Chunked-Embeddings Strategy], [BM25 Strategy], [RRF Fusion], [Reranker].

Permanent model errors abort early. If the embedding model, semantic-LLM model, or reranking model returns a permanent error (HTTP 400, 401, 404, or 429 — invalid config, bad auth, unknown model, or rate limit), Docker Agent treats the model configuration as invalid and stops immediately rather than retrying doomed requests:

• Indexing — the entire indexing run is aborted after the first permanent failure (including 429). The error is surfaced in the logs so you know immediately if a model name or API key is wrong, rather than silently producing incomplete results.
• Reranking — a permanent error (including 429) permanently disables the reranker for the lifetime of the manager. Subsequent queries fall back to un-reranked results. Only transient errors (5xx, timeouts) fall back and retry on the next query.

Configuration Reference

Top-Level RAG Fields

Chunked-Embeddings Strategy

Semantic-Embeddings Strategy

BM25 Strategy

Results (Post-Processing)