Google Gemini

Use Gemini 2.5 Flash, Gemini 3.1 Pro, and other Google models with Docker Agent.

Setup

Docker Agent reads the first credential it finds from these environment variables (see pkg/model/provider/gemini/client.go):

Explicit provider_opts.project or provider_opts.location selects Vertex AI regardless of the flag.

On the Gemini Developer API, a model or custom provider that sets token_key reads its key from that variable instead of GOOGLE_API_KEY / GEMINI_API_KEY. The Vertex AI backends use Application Default Credentials and ignore token_key.

# Gemini Developer API
export GOOGLE_API_KEY="AI..."   # or GEMINI_API_KEY

# Vertex AI (no API key; uses Application Default Credentials)
gcloud auth application-default login
export GOOGLE_GENAI_USE_VERTEXAI=1
export GOOGLE_CLOUD_PROJECT="my-gcp-project"
export GOOGLE_CLOUD_LOCATION="us-central1"

Configuration

Inline

agents:
  root:
    model: google/gemini-3.8-flash

Named Model

models:
  gemini:
    provider: google
    model: gemini-3.8-flash
    temperature: 0.5

Available Models

Generated Images

Some Gemini models (e.g. gemini-2.5-flash-image) are designed to generate an image directly as part of their reply, not just describe one. When the model's image output capability resolves as enabled — see Output capabilities for the output_capabilities.image / models.dev precedence rules — Docker Agent requests that image output on supported Google surfaces: the models gateway, direct Gemini API, and Vertex AI. Each eligible ordinary chat request asks for text and image output. Vertex AI has deterministic guard/predicate coverage; live image-generation validation is deferred.

models:
  gemini-image:
    provider: google
    model: gemini-2.5-flash-image

output_capabilities.image can be omitted for models that models.dev lists with image output; set it explicitly for custom models or to override incorrect catalogue data. Capability is never guessed from the model name.

Session-title and compaction requests omit image response modalities and bypass the guard even for image-output-capable models. They do not explicitly force TEXT-only output. Ordinary image-output requests with custom function tools or structured output are rejected locally before any request is sent. Google Search, Maps, and code-execution built-ins remain available. When custom tools conflict, Docker Agent uses models.dev's tool_call capability to clarify whether the model cannot call tools at all or supports tools only outside an image-output request. Unknown catalogue data keeps the conservative generic message.

A few provider-side behaviors to know:

• The provider decides the image format (typically PNG). Asking for a .gif or .svg filename does not transcode anything — the saved file's extension is corrected to match the data actually returned.
• An image is not guaranteed. Even a correctly configured image model can answer with text only and generate no image; that is provider behavior, so reword or repeat the prompt. At a text-only stop, Docker Agent may add a nonfatal phrase-based warning to the turn — see Generated Media for the exact check and its limitations.

Docker Agent attempts to save each returned image into the session workspace, keep a portable copy in the session database, and display it in the TUI. See Generated Media Files for persistence and portability, and Generated Media for file naming, collision handling, and rendering.

Thinking Budget

Gemini supports two approaches depending on the model version:

Gemini 2.5 (Token-based)

models:
  gemini-no-thinking:
    provider: google
    model: gemini-2.5-flash
    thinking_budget: 0 # disable thinking

  gemini-dynamic:
    provider: google
    model: gemini-2.5-flash
    thinking_budget: -1 # dynamic (model decides) — default

  gemini-fixed:
    provider: google
    model: gemini-2.5-flash
    thinking_budget: 8192 # fixed token budget

Gemini 3 (Level-based)

models:
  gemini-pro:
    provider: google
    model: gemini-3.1-pro-preview
    thinking_budget: high # default for Pro: low | high

  gemini-flash:
    provider: google
    model: gemini-3.8-flash
    thinking_budget: medium # default for Flash: minimal | low | medium | high

Built-in Tools (Grounding)

Gemini models support built-in tools that let the model access Google Search, Google Maps, and public URLs, or execute code directly during generation. Enable them via provider_opts:

models:
  gemini-grounded:
    provider: google
    model: gemini-2.5-flash
    provider_opts:
      google_search: true
      google_maps: true
      code_execution: true

URL Context

Enable url_context to let Gemini read links supplied in your prompt without a separate fetch tool:

models:
  reader:
    provider: google
    model: gemini-3.8-flash
    provider_opts:
      url_context: true
      google_search: true # Optional: discover sources as well as read links

Then ask, for example, Summarize https://docs.docker.com/ai/docker-agent/. Google fetches the content on its servers; it cannot read local files, private network URLs, or pages requiring your browser's authentication. The option is opt-in and accepts a YAML boolean, not the string "true".

Docker Agent forwards this tool on the Gemini Developer API, models gateway, and native Gemini Vertex AI paths. Model, backend, region, and tool-combination support still depend on Google and the gateway. On supported models it can be combined with Google Search. Mixing built-in and custom function tools currently uses a flag that the Go SDK accepts only on the Gemini Developer API path (including gateways forwarding to that API), not Vertex AI. This is not a general browser or an alternative to the local fetch tool for authenticated requests.

See Google's URL Context guide for supported content types, limits, and models, and examples/gemini_url_context.yaml for a complete agent.

Embeddings

Gemini embedding models can back the RAG chunked-embeddings and semantic-embeddings strategies. Text is embedded natively through the Gemini API (batchEmbedContents) or Vertex AI, and vectors come back in input order. On Vertex AI, Gemini Embedding 2 models accept one input per request, so batches are sent one text at a time there.

models:
  gemini-embed:
    provider: google
    model: gemini-embedding-2
    provider_opts:
      output_dimensionality: 768   # optional; defaults to the model's native size

rag:
  docs:
    docs: [./docs]
    strategies:
      - type: chunked-embeddings
        embedding_model: gemini-embed
        database: ./docs.db
        vector_dimensions: 768       # must match output_dimensionality (or the native size)

gemini-embedding-001 works the same way. Vectors from different models (or different output_dimensionality values) live in different vector spaces, so rebuild the index after changing either.

Only text is embedded, and no task_type is sent: queries and documents share one vector space with no automatic task prefixes, and the same configuration works with Gemini Embedding 2, which rejects task_type. The Gemini Developer API does not report token usage for embeddings, so RAG cost statistics stay at zero; Vertex AI per-input token statistics are summed into the input token count when present.

Service Tier

Set provider_opts.service_tier to pick a Gemini API Flex or Priority inference tier:

models:
  gemini-flex:
    provider: google
    model: gemini-2.5-flash
    provider_opts:
      service_tier: flex

The value is forwarded unchanged as the top-level serviceTier of every generateContent request on the Gemini Developer API, Vertex AI, and the Docker models gateway; the endpoint must support it. It applies to all requests using the configured model, including reranking, title generation, and compaction. Other values are passed through for the API to validate. When omitted or empty, no serviceTier is sent, leaving the API's default behavior unchanged. Non-string values are ignored. Vertex AI Model Garden models do not use this option.

Flex requests can sit in Google's queue for up to 15 minutes before the first byte arrives. Docker Agent normally treats a stream that stays silent for 5 minutes as stalled; for google models with service_tier: flex it waits 15 minutes instead. Other tiers and providers keep the 5 minute limit. Rate-limit (429) and overloaded (503) responses are retried like any other Gemini request; Docker Agent never upgrades a request to a different tier on your behalf.

See examples/gemini_service_tier.yaml for a complete example.

Vertex AI Model Garden

You can use non-Gemini models (e.g. Claude, Llama) hosted on Google Cloud's Vertex AI Model Garden through the google provider. When a publisher is specified in provider_opts, requests are routed through the appropriate Vertex AI endpoint instead of the Gemini SDK:

• Anthropic Claude (publisher: anthropic) uses the Anthropic-native :rawPredict / :streamRawPredict endpoints. Claude models on Vertex AI do not support the OpenAI /chat/completions path.
• Other publishers (e.g. meta, mistral) use Vertex AI's OpenAI-compatible /chat/completions endpoint.

Authentication

Vertex AI uses Google Cloud Application Default Credentials (ADC). Make sure you are authenticated:

gcloud auth application-default login

Configuration

models:
  claude-on-vertex:
    provider: google
    model: claude-sonnet-5
    provider_opts:
      project: my-gcp-project       # GCP project ID (or set GOOGLE_CLOUD_PROJECT)
      location: us-east5             # GCP region (or set GOOGLE_CLOUD_LOCATION)
      publisher: anthropic           # Model publisher (anthropic, meta, etc.)