These docs track the main branch and may describe unreleased features. The stable documentation lives at docs.docker.com.

Groq

Use Groq models with docker-agent.

Overview

Groq serves open-weight models on its LPU inference engine through an OpenAI-compatible API, with a focus on very low latency. docker-agent includes built-in support for Groq as an alias provider.

Setup

Create an API key from the Groq Console.
Set the environment variable:
```
export GROQ_API_KEY=your-api-key
```

Usage

Inline Syntax

The simplest way to use Groq:

agents:
  root:
    model: groq/llama-3.3-70b-versatile
    description: Assistant using Groq
    instruction: You are a helpful assistant.

Named Model

For more control over parameters:

models:
  groq_model:
    provider: groq
    model: llama-3.3-70b-versatile
    temperature: 0.7
    max_tokens: 8192

agents:
  root:
    model: groq_model
    description: Assistant using Groq
    instruction: You are a helpful assistant.

Available Models

Groq hosts a rotating catalogue of open-weight models. Check the Groq models documentation for current model IDs, context limits, and rate limits.

Model	Description
`llama-3.3-70b-versatile`	Llama 3.3 70B, reliable general-purpose chat and tool calling
`llama-3.1-8b-instant`	Llama 3.1 8B, fastest and cheapest
`openai/gpt-oss-120b`	GPT-OSS 120B, strong reasoning and tool calling
`openai/gpt-oss-20b`	GPT-OSS 20B, compact reasoning model
`qwen/qwen3-32b`	Qwen3 32B, reasoning and tool calling
`meta-llama/llama-4-scout-17b-16e-instruct`	Llama 4 Scout MoE

Model IDs are case-sensitive and must be passed exactly as the catalogue lists them.

How It Works

Groq is implemented as a built-in alias in docker-agent:

API Type: OpenAI-compatible (openai_chatcompletions)
Base URL: https://api.groq.com/openai/v1
Token Variable: GROQ_API_KEY

Example: Code Assistant

agents:
  coder:
    model: groq/llama-3.3-70b-versatile
    description: Code assistant using Llama 3.3
    instruction: |
      You are an expert programmer.
      Write clean, well-documented code and follow language best practices.
    toolsets:
      - type: filesystem
      - type: shell
      - type: think

← Previous Google Gemini Next → Hugging Face