Kilo Code

Set up Spicrawl in Kilo Code: add the hosted MCP server to kilo.jsonc, install the agent skill, and add AGENTS.md rules so the agent fetches web pages cheaply and handles errors correctly.

Three steps give Kilo Code Spicrawl's tools, the knowledge to write Spicrawl code, and rules for using both well. Kilo Code (VS Code extension and kilo CLI) keeps MCP servers in its kilo.jsonc config. Set your key first, in the environment Kilo starts from:

export SPICRAWL_API_KEY=spicrawl_live_...   # add to your shell profile, then restart VS Code

Roo Code is archived. Kilo Code is the maintained alternative, so use this page in its place.

1. Add the MCP server

Add Spicrawl under the top-level mcp key of kilo.jsonc: .kilo/kilo.jsonc (or kilo.jsonc in the project root) for one project, or ~/.config/kilo/kilo.jsonc for every project. Project config wins over global. Kilo replaces {env:SPICRAWL_API_KEY} with the variable's value, so the file is safe to commit:

.kilo/kilo.jsonc

{
  "mcp": {
    "spicrawl": {
      "type": "remote",
      "url": "https://mcp.spicrawl.com/mcp",
      "headers": { "Authorization": "Bearer {env:SPICRAWL_API_KEY}" },
      "enabled": true
    }
  }
}

Remote servers try Streamable HTTP first and fall back to SSE. Kilo starts an OAuth flow when a remote server supports it. Spicrawl does not use OAuth, only the bearer key, so if you see an OAuth prompt, add "oauth": false to the server.

You can also open Kilo's settings (gear icon in the sidebar toolbar), choose Agent Behaviour, then MCP Servers, and click Add Server. Choose Remote (HTTP) and enter the URL and headers. The UI writes the same file. Check with kilo mcp list in the CLI, or in that settings panel: spicrawl should list 25 spicrawl_* tools. See MCP server for each tool.

2. Install the skill

Kilo loads skills from .kilo/skills/, .agents/skills/ and .claude/skills/ in the project, and from ~/.kilo/skills/ and ~/.agents/skills/ for every project. .agents/skills/ is what the CLI's codex client writes:

spicrawl skill install --client codex

This writes .agents/skills/spicrawl/SKILL.md. Without the CLI:

mkdir -p .agents/skills/spicrawl
curl -fsSL https://app.spicrawl.com/skill.md -o .agents/skills/spicrawl/SKILL.md

See Agent skill.

3. Add rules to AGENTS.md

Kilo reads AGENTS.md at the project root. Paste this section into it:

AGENTS.md

## Web data (Spicrawl)

Use Spicrawl to read web pages: the spicrawl_* MCP tools in chat, or
POST https://api.spicrawl.com/v1/scrape with `Authorization: Bearer $SPICRAWL_API_KEY` in code.
Docs: https://docs.spicrawl.com/llms.txt (append .md to any page URL for Markdown).

- Ask for markdown: `response_format: "markdown"` (API default is html); on spicrawl_scrape, `format: "markdown"`.
- Check the site's status, not only the HTTP status: `X-Target-Status` header, or `status`
  in the JSON envelope (spicrawl_scrape with `format: "json"`). 200 is the page; 404/410
  mean it does not exist; 403/429/503 mean the site refused, so escalate.
- On an error, switch on `code`. Retry only when `retryable` is true, after
  `retry_after_seconds`. Read `diagnostics.hint` and change what it names first.
  Never retry ERR::REQUEST::*, ERR::AUTH::* or ERR::LIMIT::QUOTA_EXCEEDED.
- Escalate one step at a time and stop at the first that works:
  plain fetch (1 credit) -> `js_render: true` (3) -> add the user's own `proxy`
  if they have one. Empty or skeleton content means render; ERR::UPSTREAM::CHALLENGE
  or a 403 target status means retry once, then the user's own proxy.
- Set `max_cost` on every request to the price of the step you intend.
- Keep the cache on (default). Set `cache: false` only for prices, stock or other live data.
- Trim tokens with `main_content_only` (on by default for markdown), `include_tags`, `exclude_tags`.
- For more than 20 URLs, use a batch job (spicrawl_batch_submit / POST /v1/batch).
  Batch items return raw HTML and apply only render, proxy and block_resources settings.
- Never print or commit SPICRAWL_API_KEY. Log `X-Request-Id` for failures.

To keep the rule in its own file, save it as .kilo/rules/spicrawl.md and list it in kilo.jsonc. Kilo also still reads existing .kilocode/rules/ folders.

.kilo/kilo.jsonc

{
  "instructions": [".kilo/rules/spicrawl.md"]
}

A first task to try

Open a Kilo Code chat and ask:

Use Spicrawl to read https://example.com/pricing as markdown. List each plan with its monthly price.
If the page comes back empty, retry with rendering.

The agent should call spicrawl_scrape with url and format: "markdown", and add render: true only if the first result is empty. For a coding task:

Add a fetchPage(url) function to this project that calls the Spicrawl API for markdown,
following the Web data rules in AGENTS.md. Read the key from process.env.SPICRAWL_API_KEY.

Compare the result with the reference implementation in Best practices.

Troubleshooting

Symptom	Fix
The server shows an error or 401	`SPICRAWL_API_KEY` was not set when VS Code started, so the header was empty. Export it and restart VS Code from that shell.
Kilo asks you to authenticate with OAuth	Add `"oauth": false` to the `spicrawl` entry. Spicrawl authenticates with the bearer header only.
The server is listed but has no tools	Toggle it off and on in Agent Behaviour → MCP Servers, then open a new chat.
The agent writes `js_render` on `spicrawl_scrape` and gets "unknown argument"	The tool's argument is `render`. The API field is `js_render`.
Usage tools return `ERR::AUTH::INSUFFICIENT_SCOPE`	Grant the `read` scope to the key in the dashboard.