web access

How to Give AI Agents Web Access

July 03, 2026 · 5 min read · YayaAgent Team

Most people picture "giving an agent web access" as flipping one switch — off means it only knows its training data, on means it can browse freely. It's not a switch. It's a stack of separate tools with different jobs: one for searching, one for fetching a known URL, and one for full browser interaction. Which one your agent needs depends entirely on the task, not on some general "web access" flag.

This guide explains how to wire up real web access for OpenClaw and Hermes, and how to pick the right tool for the job instead of reaching for the heaviest one by default.

Before starting, two prerequisites are useful:

Already have an agent installed? This guide assumes OpenClaw or Hermes. If not, start with the OpenClaw installation guide or the Hermes setup guide.
New to agent architecture? It helps to understand what tools actually are before editing configuration files — a tool is a typed function the model calls; "web access" here just means three specific tools instead of one.

Quick Decision Guide

Task	Right Tool
"Find pages about X"	Search tool (returns links + snippets or summarized content)
"Read this specific URL"	Fetch/extract tool (HTTP GET → readable markdown)
"Click, log in, fill a form, screenshot"	Browser tool (real Chromium via CDP)

A more precise way to think about selection:

If it's discoverable but you don't know the URL → search
If you already know the URL → fetch
If the page needs clicking, login, or JavaScript to render → browser

Reach for search or fetch first. Both frameworks route to a real browser automatically or on request when a page needs JavaScript — you rarely need to configure the browser tool up front.

1. Web Access in OpenClaw

OpenClaw ships three web tools: web_search, web_fetch, and a full Web Browser for JS-heavy or login-gated pages. They're bundled under group:web in the tool allow-list.

web_fetch needs no setup — it's enabled by default. It does a plain HTTP GET, runs Readability extraction to pull the main content, and returns markdown. It does not execute JavaScript; for pages that need that, use the Web Browser instead.

web_search needs a provider. Brave is the default (free tier), but Gemini, Perplexity, Grok, Kimi, DuckDuckGo, Exa, and Tavily are all supported. Configure it interactively:

openclaw configure --section web

Or edit ~/.openclaw/openclaw.json directly:

{
  tools: {
    web: {
      search: {
        provider: "brave",
        apiKey: "YOUR_BRAVE_API_KEY", // optional if BRAVE_API_KEY env var is set
      },
    },
  },
}

If you're using tool allow-lists per agent, make sure web access is actually permitted:

{
  tools: {
    allow: ["group:web"], // covers web_search, web_fetch, and x_search together
  },
}

Results are cached for 15 minutes by default. That's fine for research, but keep it in mind: search results are candidates, not confirmed facts. A search returns snippets scored for relevance, not verified for accuracy — if the task depends on getting a specific number or claim right, follow up with web_fetch on the actual source page rather than trusting the snippet alone.

2. Web Access in Hermes

Hermes ships web_search and web_extract as its two core web tools, plus a full browser toolset (browser_navigate, browser_snapshot, and others) for interactive sessions.

Firecrawl is the default backend for both. Set your key once and both tools route through it automatically:

# ~/.hermes/.env
FIRECRAWL_API_KEY=fc-your-key-here

Get a key at firecrawl.dev — the free tier includes 500 credits/month. Once configured, web_search returns full page content as markdown instead of short snippets, and web_extract fetches a specific URL with real browser rendering (handles JS-heavy pages and Cloudflare-protected sites).

If you'd rather not manage a separate Firecrawl account, Hermes' own Tool Gateway (a paid Nous Portal feature) routes web search, extract, and browser automation through Nous-managed infrastructure with one login instead of separate API keys per service:

hermes setup --portal   # fresh install: log in + turn on every gateway tool
hermes tools             # existing install: enable web access à la carte

To scope which toolsets a session actually uses:

hermes chat --toolsets "web,terminal"

Here's the constraint worth knowing before you rely on it: long pages returned by web_extract get compressed by an auxiliary model before reaching the agent, so a 10,000-word article doesn't blow your context window. That compression keeps quotes and code blocks intact, but it can flatten structure — a table or a numbered list doesn't always survive the round trip cleanly. If you need the raw, unsummarized page instead — for example scraping a structured page where an LLM summary would drop fields — use browser_navigate + browser_snapshot directly.

3. What Happens After the Fetch

Both web_fetch (OpenClaw) and web_extract (Hermes) hand back a transformed view of a page, not the page itself. Think of it less like handing the agent the original document and more like handing it someone else's notes on the document — usually faithful, occasionally missing something that mattered. Common failure modes worth watching for:

Dynamic content that only appears after JavaScript runs (fetch tools won't see it — that's what the browser tool is for)
Structure that gets stripped or reordered during markdown conversion — tables, nested lists, footnotes
Summarized content dropping a field or number that a raw snapshot would have kept

Treat fetched output as a lossy observation, not ground truth, and the result still needs to go through a model to be summarized, compared, or turned into something you can act on. If you're running a self-hosted pipeline and deciding which LLM should sit in that step, that's a separate routing decision from which web tool fetched the page — see our comparison of OpenRouter vs LLM Gateway for how to think about it.

4. Security and Sandboxing: Myth vs. Reality

Myth: Giving an agent web access means it can be tricked into visiting or acting on anything a malicious page tells it to.

Reality: Both frameworks apply guardrails at the fetch layer, not just at the model layer. OpenClaw's web_fetch blocks private/internal hostnames and re-checks redirects so a page can't quietly redirect the agent onto your LAN. Hermes' browser tool blocks private URLs by default unless you explicitly opt in. Think of it less like handing the agent an open door to the internet, and more like a supervised kiosk — it can look things up, but it can't wander off the paved path without you widening it first.

That said, network-level guardrails don't cover the main real-world risk: prompt injection from retrieved content. A fetched page is just text the model reads — if that page contains instructions phrased to look like part of your prompt ("ignore previous instructions and..."), a naive pipeline can follow them. Treat everything a search or fetch tool returns as untrusted input, the same way you'd treat unvalidated user input in a web app, not as a trusted extension of your own instructions.

A few rules of thumb:

Start with search or fetch; only enable the browser tool when a task genuinely needs interaction (clicking, forms, login).
Keep API keys for search/fetch providers in .env or config, never hardcoded in a shared config file.
Don't let fetched page content directly trigger tool calls (file writes, purchases, messages) without a review step in between.
Watch cached results if a task depends on real-time data — a 15-minute cache is fine for research, not for live prices.

Troubleshooting

Symptom	Fix
`web_search` fails with a missing-key error (OpenClaw)	Run `openclaw configure --section web` or set the provider's env var (e.g. `BRAVE_API_KEY`)
Page returns empty or garbled content (OpenClaw)	`web_fetch` doesn't run JavaScript — switch to the Web Browser tool for that page
`web_search`/`web_extract` still shows "not configured" (Hermes)	Confirm `FIRECRAWL_API_KEY` is in `~/.hermes/.env`, or check gateway status with `hermes portal tools`
Extracted content is missing a table or list (Hermes)	Summarization flattened it — re-run with `browser_navigate` + `browser_snapshot` for the raw structure
Browser tool can't reach a local/internal URL (Hermes)	Expected — set `browser.allow_private_urls: true` only if you understand the risk

Bottom Line

Web access isn't one capability, it's three layers with different jobs: search for discovery, fetch for retrieval, browser for interaction — and none of what they return should be treated as verified truth, just a candidate or a lossy snapshot worth double-checking when accuracy matters. Configure search and fetch by default since they cover most tasks, escalate to the browser tool only when a page genuinely needs clicking or JavaScript, and treat everything that comes back as untrusted input rather than a trusted extension of your own prompt. Once your agent can pull in live information this way, combining web access with local file reading turns it into something that can research and write results back to your codebase in a single loop.

If you're still getting your first agent running end to end, it's worth backing up to building your first AI assistant before layering on web tools — get one clean conversation working, then add capabilities one at a time.