Fetch Tool

Read content from HTTP/HTTPS URLs.

Overview

The fetch tool lets agents retrieve content from one or more HTTP/HTTPS URLs. It is read-only — only GET requests are supported. The tool respects robots.txt, limits response size (1 MB per URL), and can return content as plain text, Markdown (converted from HTML), or raw HTML.

ℹ️ GET only

The fetch tool does not support POST, PUT, DELETE or other methods, and does not expose request bodies or custom headers. To call REST endpoints with other verbs, use the API tool or an OpenAPI toolset.

Configuration

toolsets:
  - type: fetch

Options

Property	Type	Default	Description
`timeout`	int	`30`	Default request timeout in seconds (overridable per tool call).
`allowed_domains`	array[string]	none	Allow-list of hosts the tool may fetch. When set, every URL whose host is not in the list is rejected before any network call is made. Mutually exclusive with `blocked_domains`.
`blocked_domains`	array[string]	none	Deny-list of hosts the tool must not fetch. URLs whose host matches one of these patterns are rejected before any network call (including `robots.txt`) is made. Mutually exclusive with `allowed_domains`.

Domain matching

Domain patterns in allowed_domains and blocked_domains use the following rules (case-insensitive):

Bare domain — example.com matches the host example.com and any subdomain such as docs.example.com. It does not match unrelated hosts that share a suffix (e.g. badexample.com).
Leading dot — .example.com matches only strict subdomains (docs.example.com, a.b.example.com), not the apex example.com.
Wildcard glob — *.example.com is an alias for the leading-dot form; the apex is excluded. The * is only valid as a leading *. token (entries like foo.*, *.*.example.com, or a bare * are rejected at config-load time).
IP literal — IP addresses are matched exactly (169.254.169.254).
CIDR range — 169.254.0.0/16, 10.0.0.0/8, ::1/128, fc00::/7. Matches when the URL’s host parses as an IP inside the network. Hostname hosts never match a CIDR pattern. Malformed CIDRs are rejected at config-load time.
Trailing dots in FQDN-form URLs (http://example.com./) are stripped before matching, so they cannot bypass a deny-list entry.

The lists are mutually exclusive: a single fetch toolset may set either allowed_domains or blocked_domains, but not both.

When a list is configured, every redirect target is re-checked against the same list. A request to an allowed origin that redirects to a forbidden host is rejected before any data is read from the redirect.

⚠️ Limitations

Matching is purely string-based on the URL host. It does not perform DNS resolution and does not normalise alternative IP encodings (decimal 2852039166, hex 0xa9.0xfe.0xa9.0xfe, octal, etc. IPv4-mapped IPv6 addresses ARE normalized to their IPv4 form). If you need to deny access to a specific IP, also list its alternative encodings, or block at the network layer.

Custom Timeout

toolsets:
  - type: fetch
    timeout: 60

Restrict to specific domains

toolsets:
  - type: fetch
    allowed_domains:
      - docker.com          # docker.com and *.docker.com
      - github.com          # github.com and *.github.com
      - .githubusercontent.com  # only subdomains, e.g. raw.githubusercontent.com

Block sensitive hosts

toolsets:
  - type: fetch
    blocked_domains:
      - 169.254.169.254       # cloud metadata endpoint (literal IP)
      - 169.254.0.0/16        # entire link-local range (CIDR)
      - 10.0.0.0/8            # RFC1918 private range
      - "*.internal.example.com"  # any subdomain (wildcard)
      - internal.example.com  # internal corporate hostname

Tool Interface

The toolset exposes a single tool, fetch, with the following parameters:

Parameter	Type	Required	Description
`urls`	array[string]	✓	One or more HTTP/HTTPS URLs to fetch (all via `GET`).
`format`	string	✓	Output format: `text`, `markdown`, or `html`. HTML responses are converted to text/markdown when requested.
`timeout`	integer	✗	Per-call request timeout in seconds. Overrides the toolset default. Valid range: `1`–`300`.

Responses are capped at 1 MB per URL. Hosts that disallow the agent’s user-agent via robots.txt are skipped with a clear error.

💡 Fetch vs. API Tool

Use fetch when the agent needs to read arbitrary public URLs at runtime. Use the API tool to expose specific, structured HTTP endpoints (including non-GET verbs) as named tools.

← Previous Tasks Next → Script