Kiro

Set up Spicrawl in Kiro: add the hosted MCP server to .kiro/settings/mcp.json, install the agent skill, and add a steering file so the agent fetches web pages cheaply and handles errors correctly.

Three steps give Kiro's agent Spicrawl's tools, the knowledge to write Spicrawl code, and rules for using both well. Set your key first, in the environment Kiro starts from:

export SPICRAWL_API_KEY=spicrawl_live_...   # add to your shell profile, then restart Kiro

1. Add the MCP server

Kiro reads MCP servers from .kiro/settings/mcp.json in the workspace and ~/.kiro/settings/mcp.json for every workspace. The workspace file wins when both define a server. In the IDE, open the command palette and run Kiro: Open workspace MCP config (JSON) or Kiro: Open user MCP config (JSON). Kiro's docs list the same two files for Kiro CLI.

.kiro/settings/mcp.json

{
  "mcpServers": {
    "spicrawl": {
      "url": "https://mcp.spicrawl.com/mcp",
      "headers": { "Authorization": "Bearer ${SPICRAWL_API_KEY}" }
    }
  }
}

Kiro expands only environment variables you have approved. When you save the file, it shows a security popup listing SPICRAWL_API_KEY. Approve it, or add it in Kiro settings under Mcp Approved Env Vars. Saved changes apply without restarting the session, and only changed servers restart.

Check that spicrawl is connected and lists 25 spicrawl_* tools. See MCP server for each tool.

2. Install the skill

Kiro supports Agent Skills in the IDE and the CLI. It loads them from .kiro/skills/ in the workspace and ~/.kiro/skills/ for every workspace. spicrawl skill install has no Kiro target, so install manually:

mkdir -p .kiro/skills/spicrawl
curl -fsSL https://app.spicrawl.com/skill.md -o .kiro/skills/spicrawl/SKILL.md

Kiro matches the skill's description against your request, and you can also invoke it with a slash command. See Agent skill.

3. Add a steering file

Steering files give Kiro persistent instructions. Save this as .kiro/steering/spicrawl.md (or ~/.kiro/steering/spicrawl.md for every workspace). With inclusion: auto and a name and description, Kiro loads it when a task involves web data. Use inclusion: always to load it in every chat.

.kiro/steering/spicrawl.md

---
inclusion: auto
name: spicrawl
description: Fetching, scraping or extracting data from web pages with Spicrawl
---

# Web data (Spicrawl)

Use Spicrawl to read web pages: the spicrawl_* MCP tools in chat, or
POST https://api.spicrawl.com/v1/scrape with `Authorization: Bearer $SPICRAWL_API_KEY` in code.
Docs: https://docs.spicrawl.com/llms.txt (append .md to any page URL for Markdown).

- Ask for markdown: `response_format: "markdown"` (API default is html); on spicrawl_scrape, `format: "markdown"`.
- Check the site's status, not only the HTTP status: `X-Target-Status` header, or `status`
  in the JSON envelope (spicrawl_scrape with `format: "json"`). 200 is the page; 404/410
  mean it does not exist; 403/429/503 mean the site refused, so escalate.
- On an error, switch on `code`. Retry only when `retryable` is true, after
  `retry_after_seconds`. Read `diagnostics.hint` and change what it names first.
  Never retry ERR::REQUEST::*, ERR::AUTH::* or ERR::LIMIT::QUOTA_EXCEEDED.
- Escalate one step at a time and stop at the first that works:
  plain fetch (1 credit) -> `js_render: true` (3) -> add the user's own `proxy`
  if they have one. Empty or skeleton content means render; ERR::UPSTREAM::CHALLENGE
  or a 403 target status means retry once, then the user's own proxy.
- Set `max_cost` on every request to the price of the step you intend.
- Keep the cache on (default). Set `cache: false` only for prices, stock or other live data.
- Trim tokens with `main_content_only` (on by default for markdown), `include_tags`, `exclude_tags`.
- For more than 20 URLs, use a batch job (spicrawl_batch_submit / POST /v1/batch).
  Batch items return raw HTML and apply only render, proxy and block_resources settings.
- Never print or commit SPICRAWL_API_KEY. Log `X-Request-Id` for failures.

Kiro also reads AGENTS.md, and always includes it (no inclusion modes). If you already keep agent instructions there, paste the same section into it instead.

A first task to try

Start a chat in the Kiro IDE and ask:

Use Spicrawl to read https://example.com/pricing as markdown. List each plan with its monthly price.
If the page comes back empty, retry with rendering.

The agent should call spicrawl_scrape with url and format: "markdown", and add render: true only if the first result is empty. For a coding task:

Add a fetchPage(url) function to this project that calls the Spicrawl API for markdown,
following the Web data rules in the Spicrawl steering file. Read the key from process.env.SPICRAWL_API_KEY.

Compare the result with the reference implementation in Best practices.

Troubleshooting

Symptom	Fix
The server shows an error or 401	`SPICRAWL_API_KEY` is not approved or not set. Approve it in Mcp Approved Env Vars, export it, and restart Kiro from that shell.
Check the key and endpoint outside Kiro	`curl -i https://mcp.spicrawl.com/mcp` answers 401 `Unauthorized: send your Spicrawl API key`. Add `-H "Authorization: Bearer $SPICRAWL_API_KEY"` to see the authenticated response.
The agent writes `js_render` on `spicrawl_scrape` and gets "unknown argument"	The tool's argument is `render`. The API field is `js_render`.
Usage tools return `ERR::AUTH::INSUFFICIENT_SCOPE`	Grant the `read` scope to the key in the dashboard.

1. Add the MCP server

2. Install the skill

3. Add a steering file

A first task to try

Troubleshooting

On this page