Zed

Set up Spicrawl in Zed: add the hosted MCP server to context_servers in settings.json, install the agent skill, and add AGENTS.md instructions so the agent fetches web pages cheaply and handles errors correctly.

Three steps give Zed's agent Spicrawl's tools, the knowledge to write Spicrawl code, and rules for using both well. The Spicrawl CLI has no Zed target (--client accepts claude, cursor, vscode, codex and all), so you do each step by hand. Have your key ready:

export SPICRAWL_API_KEY=spicrawl_live_...   # for the skill's curl examples and your own code

1. Add the MCP server

Open your settings file with the zed: open settings file command and add a context_servers entry. A remote server takes url and headers:

settings.json

{
  "context_servers": {
    "spicrawl": {
      "url": "https://mcp.spicrawl.com/mcp",
      "headers": { "Authorization": "Bearer spicrawl_live_..." }
    }
  }
}

Replace spicrawl_live_... with your key. Zed's docs do not describe environment-variable substitution in headers, so the key is written into the file: keep settings.json out of version control and out of screenshots.

Always set the Authorization header. When a remote server has no headers, Zed starts an OAuth flow, and Spicrawl does not support OAuth, so the connection fails.

You can also add it in Settings → AI → MCP Servers (or the agent: open settings command). A green dot next to spicrawl means the server is active. Open the Agent Panel and check that it lists 25 spicrawl_* tools. See MCP server for each tool.

By default Zed asks before each tool call (agent.tool_permissions.default is "confirm"; Zed 0.224.0 and later). Each successful Spicrawl call spends credits, so leave the default on until you trust the agent's max_cost habits.

2. Install the skill

Zed's agent loads skills from .agents/skills/ in the project (trusted worktrees only) and from ~/.agents/skills/ for every project. Each skill is a direct child folder of that directory:

mkdir -p .agents/skills/spicrawl
curl -fsSL https://app.spicrawl.com/skill.md -o .agents/skills/spicrawl/SKILL.md

spicrawl skill install --client codex writes the same file, because Codex reads .agents/skills/ too. The agent picks the skill from its name and description; you can also type / in the message editor and choose it. See Agent skill.

3. Add instructions

Zed reads instructions from these files in the project root, in this priority order: .rules, .cursorrules, .windsurfrules, .clinerules, .github/copilot-instructions.md, AGENT.md, AGENTS.md, CLAUDE.md, GEMINI.md. Use AGENTS.md, which other agents read too. If a file earlier in that list already exists (for example .rules), it takes priority: put the section there instead. A personal ~/.config/zed/AGENTS.md (Linux and macOS) applies to every project; project instructions win on conflict.

AGENTS.md

## Web data (Spicrawl)

Use Spicrawl to read web pages: the spicrawl_* MCP tools in chat, or
POST https://api.spicrawl.com/v1/scrape with `Authorization: Bearer $SPICRAWL_API_KEY` in code.
Docs: https://docs.spicrawl.com/llms.txt (append .md to any page URL for Markdown).

- Ask for markdown: `response_format: "markdown"` (API default is html); on spicrawl_scrape, `format: "markdown"`.
- Check the site's status, not only the HTTP status: `X-Target-Status` header, or `status`
  in the JSON envelope (spicrawl_scrape with `format: "json"`). 200 is the page; 404/410
  mean it does not exist; 403/429/503 mean the site refused, so escalate.
- On an error, switch on `code`. Retry only when `retryable` is true, after
  `retry_after_seconds`. Read `diagnostics.hint` and change what it names first.
  Never retry ERR::REQUEST::*, ERR::AUTH::* or ERR::LIMIT::QUOTA_EXCEEDED.
- Escalate one step at a time and stop at the first that works:
  plain fetch (1 credit) -> `js_render: true` (3) -> add the user's own `proxy`
  if they have one. Empty or skeleton content means render; ERR::UPSTREAM::CHALLENGE
  or a 403 target status means retry once, then the user's own proxy.
- Set `max_cost` on every request to the price of the step you intend.
- Keep the cache on (default). Set `cache: false` only for prices, stock or other live data.
- Trim tokens with `main_content_only` (on by default for markdown), `include_tags`, `exclude_tags`.
- For more than 20 URLs, use a batch job (spicrawl_batch_submit / POST /v1/batch).
  Batch items return raw HTML and apply only render, proxy and block_resources settings.
- Never print or commit SPICRAWL_API_KEY. Log `X-Request-Id` for failures.

A first task to try

Open the Agent Panel and ask:

Use Spicrawl to read https://example.com/pricing as markdown. List each plan with its monthly price.
If the page comes back empty, retry with rendering.

Naming the server in the prompt helps the model choose it. The agent should call spicrawl_scrape with url and format: "markdown", and add render: true only if the first result is empty. For a coding task:

Add a fetchPage(url) function to this project that calls the Spicrawl API for markdown,
following the Web data section of AGENTS.md. Read the key from process.env.SPICRAWL_API_KEY.

Compare the result with the reference implementation in Best practices.

Troubleshooting

Symptom	Fix
Zed opens a browser or asks you to sign in to the server	`headers` is missing or misspelled, so Zed tried OAuth. Add the `Authorization` header and restart the server.
The dot next to `spicrawl` is not green, or the server returns 401	The key is wrong, revoked or has no `Bearer` prefix. Fix the header and reload.
The agent does not use the tools	Mention `spicrawl` in the prompt, and check that your agent profile does not disable the server's tools.
The skill does not load	The project is not a trusted worktree, or the folder is nested. Trust the worktree, or move it to `~/.agents/skills/spicrawl/SKILL.md`.
The agent writes `js_render` on `spicrawl_scrape` and gets "unknown argument"	The tool's argument is `render`. The API field is `js_render`.
Usage tools return `ERR::AUTH::INSUFFICIENT_SCOPE`	Grant the `read` scope to the key in the dashboard.

1. Add the MCP server

2. Install the skill

3. Add instructions

A first task to try

Troubleshooting

On this page