spicrawlspicrawlDocs

OpenCode

Set up Spicrawl in OpenCode: add the hosted MCP server to opencode.json with OAuth turned off, install the agent skill, and add AGENTS.md rules so the agent fetches web pages cheaply and handles errors correctly.

Three steps give OpenCode Spicrawl's tools, the knowledge to write Spicrawl code, and rules for using both well. Set your key first, in the environment OpenCode starts from:

export SPICRAWL_API_KEY=spicrawl_live_...   # add to your shell profile, then restart OpenCode

1. Add the MCP server

The shortest way is the opencode-spicrawl plugin (npm, source in Spicrawl/agent-plugins). OpenCode installs it from npm on start, and it adds the server below with OAuth turned off:

opencode.json
{
  "$schema": "https://opencode.ai/config.json",
  "plugin": ["opencode-spicrawl"]
}

Without SPICRAWL_API_KEY in the environment the plugin adds nothing and logs a warning, so a missing key never leaves a failing server behind. A spicrawl entry you write under mcp yourself always wins over the plugin.

To configure the server by hand instead, add a remote server to opencode.json in the project, or to ~/.config/opencode/opencode.json for every project. OpenCode replaces {env:SPICRAWL_API_KEY} with the variable's value, so the file is safe to commit:

opencode.json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "spicrawl": {
      "type": "remote",
      "url": "https://mcp.spicrawl.com/mcp",
      "enabled": true,
      "oauth": false,
      "headers": { "Authorization": "Bearer {env:SPICRAWL_API_KEY}" }
    }
  }
}

"oauth": false matters. The Spicrawl MCP server authenticates with an API key in the Authorization header and does not support OAuth. OpenCode tries OAuth discovery on remote servers by default, so without this key a 401 sends it down an OAuth flow that cannot succeed.

Run opencode mcp list to see each server and its status. spicrawl should show as connected with 25 spicrawl_* tools. See MCP server for each tool.

OpenCode's MCP docs warn that MCP servers add to your context. The 25 Spicrawl tool definitions count against it. To keep them out of most sessions, disable them globally and enable them for one agent: set "tools": { "spicrawl*": false } at the top level of opencode.json, then "agent": { "<agent-name>": { "tools": { "spicrawl*": true } } } for the agent that needs web data.

2. Install the skill

OpenCode loads skills from .opencode/skills/, .claude/skills/ and .agents/skills/ in the project, and from ~/.config/opencode/skills/, ~/.claude/skills/ and ~/.agents/skills/ for every project. The Spicrawl CLI writes the .agents/skills/ location:

spicrawl skill install --client codex

This writes .agents/skills/spicrawl/SKILL.md. The --client value names the CLI's own target, not the agent that reads it. Without the CLI:

mkdir -p .agents/skills/spicrawl
curl -fsSL https://app.spicrawl.com/skill.md -o .agents/skills/spicrawl/SKILL.md

See Agent skill.

3. Add rules to AGENTS.md

Paste this section into AGENTS.md at the project root. Put it in ~/.config/opencode/AGENTS.md to apply it to every session. OpenCode reads CLAUDE.md only when no AGENTS.md exists.

AGENTS.md
## Web data (Spicrawl)

Use Spicrawl to read web pages: the spicrawl_* MCP tools in chat, or
POST https://api.spicrawl.com/v1/scrape with `Authorization: Bearer $SPICRAWL_API_KEY` in code.
Docs: https://docs.spicrawl.com/llms.txt (append .md to any page URL for Markdown).

- Ask for markdown: `response_format: "markdown"` (API default is html); on spicrawl_scrape, `format: "markdown"`.
- Check the site's status, not only the HTTP status: `X-Target-Status` header, or `status`
  in the JSON envelope (spicrawl_scrape with `format: "json"`). 200 is the page; 404/410
  mean it does not exist; 403/429/503 mean the site refused, so escalate.
- On an error, switch on `code`. Retry only when `retryable` is true, after
  `retry_after_seconds`. Read `diagnostics.hint` and change what it names first.
  Never retry ERR::REQUEST::*, ERR::AUTH::* or ERR::LIMIT::QUOTA_EXCEEDED.
- Escalate one step at a time and stop at the first that works:
  plain fetch (1 credit) -> `js_render: true` (3) -> add the user's own `proxy`
  if they have one. Empty or skeleton content means render; ERR::UPSTREAM::CHALLENGE
  or a 403 target status means retry once, then the user's own proxy.
- Set `max_cost` on every request to the price of the step you intend.
- Keep the cache on (default). Set `cache: false` only for prices, stock or other live data.
- Trim tokens with `main_content_only` (on by default for markdown), `include_tags`, `exclude_tags`.
- For more than 20 URLs, use a batch job (spicrawl_batch_submit / POST /v1/batch).
  Batch items return raw HTML and apply only render, proxy and block_resources settings.
- Never print or commit SPICRAWL_API_KEY. Log `X-Request-Id` for failures.

A first task to try

Start opencode in the project and ask:

Use Spicrawl to read https://example.com/pricing as markdown. List each plan with its monthly price.
If the page comes back empty, retry with rendering.

The agent should call spicrawl_scrape with url and format: "markdown", and add render: true only if the first result is empty. For a coding task:

Add a fetchPage(url) function to this project that calls the Spicrawl API for markdown,
following the Web data rules in AGENTS.md. Read the key from process.env.SPICRAWL_API_KEY.

Compare the result with the reference implementation in Best practices.

Troubleshooting

SymptomFix
opencode mcp list shows spicrawl as failed or needing authenticationCheck "oauth": false is set, and that SPICRAWL_API_KEY was exported in the shell that started opencode. An empty variable sends Bearer with no key, which the server answers with 401.
Check the key and endpoint outside OpenCodecurl -i https://mcp.spicrawl.com/mcp answers 401 Unauthorized: send your Spicrawl API key. Add -H "Authorization: Bearer $SPICRAWL_API_KEY" to see the authenticated response.
The agent writes js_render on spicrawl_scrape and gets "unknown argument"The tool's argument is render. The API field is js_render.
Usage tools return ERR::AUTH::INSUFFICIENT_SCOPEGrant the read scope to the key in the dashboard.

On this page