# Agent quickstart

> Connect your coding agent to Spicrawl in one command with spicrawl init, or by pasting one self-contained setup prompt into the agent.

Source: https://docs.spicrawl.com/agents/quickstart

You need a Spicrawl API key in the `SPICRAWL_API_KEY` environment variable. Create one at [app.spicrawl.com](https://app.spicrawl.com/onboarding); keys look like `spicrawl_live_…` or `spicrawl_test_…` (test keys never spend live credits).

```bash
export SPICRAWL_API_KEY=spicrawl_live_...   # add this to your shell profile
```

Then pick one of the two paths below. Both leave your project with the hosted MCP server configured, the Spicrawl agent skill installed, and no key written into a committed file.

## Option 1: one command

From your project's root directory:

```bash
spicrawl init
```

`spicrawl init` detects the agent tools in use (Claude Code, Cursor, VS Code, Codex) in the project and your home directory, shows the files it would create or change, and after you confirm, adds the hosted MCP server `https://mcp.spicrawl.com/mcp` to each client's config and writes the Spicrawl agent skill (`SKILL.md`). The config references `$SPICRAWL_API_KEY` instead of copying the key wherever the client supports it.

| Flag            | Meaning                                                                                                                  |
| --------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `--client LIST` | `claude`, `cursor`, `vscode`, `codex` or `all`, comma-separated. Default: the detected clients.                          |
| `--yes`, `-y`   | Apply without asking. Required when stdin is not a terminal; without it, `init` prints the plan to stderr and exits `2`. |
| `--global`      | Configure your user-level client settings instead of the project.                                                        |
| `--agents-md`   | Also add a Spicrawl section to `AGENTS.md`.                                                                              |
| `--dir DIR`     | Project directory. Default: the current directory.                                                                       |

An agent running the command for you should use:

```bash
spicrawl init --client claude --yes
```

To do the two halves separately, or to see what would be written first:

```bash
spicrawl mcp install --client cursor --print     # print the MCP config instead of writing it
spicrawl skill install --client claude           # write only the skill
```

Install the CLI first if you do not have it: see [Install the CLI](https://docs.spicrawl.com/cli/install.md). Every flag and the exact file per client are on [Agent setup](https://docs.spicrawl.com/cli/agent-setup.md).

## Option 2: paste a prompt into your agent

If you do not want to install the CLI, give your agent this prompt. It is self-contained: the agent needs no other page to follow it.

```text title="Setup prompt"
Set up Spicrawl (web data API for agents) in this project. Follow these steps exactly.

1. Check that the environment variable SPICRAWL_API_KEY is set (do not print its value).
   If it is not set, stop and ask me to run: export SPICRAWL_API_KEY=spicrawl_live_...
   Never write the key itself into any file in this repository.

2. Add the hosted Spicrawl MCP server for the agent tool you are running in.
   URL: https://mcp.spicrawl.com/mcp (MCP Streamable HTTP)
   Auth header: Authorization: Bearer <SPICRAWL_API_KEY>
   Use the tool's environment-variable syntax so the key stays out of the file:
   - Claude Code: in .mcp.json at the project root, under "mcpServers":
     "spicrawl": {"type": "http", "url": "https://mcp.spicrawl.com/mcp",
                "headers": {"Authorization": "Bearer ${SPICRAWL_API_KEY}"}}
   - Cursor: in .cursor/mcp.json, under "mcpServers":
     "spicrawl": {"url": "https://mcp.spicrawl.com/mcp",
                "headers": {"Authorization": "Bearer ${env:SPICRAWL_API_KEY}"}}
   - VS Code: in .vscode/mcp.json, add to "inputs":
     {"type": "promptString", "id": "spicrawl-api-key", "description": "Spicrawl API key", "password": true}
     and under "servers":
     "spicrawl": {"type": "http", "url": "https://mcp.spicrawl.com/mcp",
                "headers": {"Authorization": "Bearer ${input:spicrawl-api-key}"}}
   - Codex: in ~/.codex/config.toml (or .codex/config.toml in a trusted project):
     [mcp_servers.spicrawl]
     url = "https://mcp.spicrawl.com/mcp"
     bearer_token_env_var = "SPICRAWL_API_KEY"
   Merge into existing files; do not remove other servers.

3. Install the Spicrawl agent skill. Download https://app.spicrawl.com/skill.md and save it as
   SKILL.md in a folder named spicrawl inside the skills directory:
   - Claude Code: .claude/skills/spicrawl/SKILL.md
   - Cursor: .cursor/skills/spicrawl/SKILL.md
   - Codex: .agents/skills/spicrawl/SKILL.md
   - VS Code: .github/skills/spicrawl/SKILL.md

4. Append this section to the project's agent instructions file (CLAUDE.md for Claude Code,
   AGENTS.md otherwise; create it if missing):

   ## Web data (Spicrawl)
   - Fetch web pages with the spicrawl_* MCP tools, or POST https://api.spicrawl.com/v1/scrape
     with Authorization: Bearer $SPICRAWL_API_KEY. Docs: https://docs.spicrawl.com/llms.txt
   - Ask for markdown (response_format "markdown"; the spicrawl_scrape tool's format "markdown").
   - The site's own status is X-Target-Status (or "status" in the JSON envelope), not the
     HTTP status. 200 is the page; 404 or 410 means the page does not exist; anything
     else (403, 429, 503) means the site refused you, so escalate as below.
   - On an error, switch on "code" and retry only when "retryable" is true, after
     retry_after_seconds. Act on diagnostics.hint before retrying.
   - Escalate one step at a time: plain fetch, then js_render, then your own proxy
     if you have one. Set max_cost on every request.
   - For more than 20 URLs use a batch job (spicrawl_batch_submit or POST /v1/batch). Batch
     items come back as the raw page (HTML), not markdown.

5. Verify: run this and confirm the HTTP status is 200 and X-Target-Status is 200:
   curl -sS -D - -o /dev/null -X POST https://api.spicrawl.com/v1/scrape \
     -H "Authorization: Bearer $SPICRAWL_API_KEY" -H "Content-Type: application/json" \
     -d '{"url": "https://example.com", "response_format": "markdown"}'
   A 401 means the key is wrong or revoked. Then tell me which files you changed and that I
   need to restart or reload the agent tool for the MCP server to appear.
```

## Check that it works

Restart your agent tool, then ask it:

```text
Use Spicrawl to read https://example.com/pricing as markdown and list the plans with their prices.
```

The agent should call `spicrawl_scrape` (MCP) or write a `POST /v1/scrape` request (skill). If the tools do not appear, see [MCP troubleshooting](https://docs.spicrawl.com/agents/mcp.md#troubleshooting).

## Next steps

* Per-client details: [Claude Code](https://docs.spicrawl.com/agents/claude-code.md), [Cursor](https://docs.spicrawl.com/agents/cursor.md), [Codex](https://docs.spicrawl.com/agents/codex.md).
* Every MCP tool and its inputs: [MCP server](https://docs.spicrawl.com/agents/mcp.md).
* Error handling and cost control for agents you build: [Best practices](https://docs.spicrawl.com/agents/best-practices.md).