# GitHub Copilot and VS Code > Set up Spicrawl in VS Code agent mode, the Copilot CLI and the Copilot cloud agent: add the hosted MCP server, install the agent skill, and add repository instructions. Source: https://docs.spicrawl.com/agents/github-copilot Three steps give GitHub Copilot Spicrawl's tools, the knowledge to write Spicrawl code, and rules for using both well. Copilot runs in several places, and each reads its own MCP config: pick the surface you use in step 1, then do steps 2 and 3 once. Set your key first: ```bash export SPICRAWL_API_KEY=spicrawl_live_... # add to your shell profile ``` `spicrawl init --client vscode` does steps 1 and 2 for VS Code (see [What the CLI writes](#what-the-cli-writes)). ## 1. Add the MCP server The endpoint is `https://mcp.spicrawl.com/mcp` (Streamable HTTP) with `Authorization: Bearer `. It does not use OAuth, so send the key as a header. See [MCP server](https://docs.spicrawl.com/agents/mcp.md) for each of the 25 `spicrawl_*` tools. ### VS Code agent mode Create `.vscode/mcp.json` in the project. VS Code uses the `servers` key (not `mcpServers`). An `inputs` entry with `password: true` makes VS Code ask for the key once and store it, so the file is safe to commit: ```json title=".vscode/mcp.json" { "inputs": [ { "type": "promptString", "id": "spicrawl-api-key", "description": "Spicrawl API key", "password": true } ], "servers": { "spicrawl": { "type": "http", "url": "https://mcp.spicrawl.com/mcp", "headers": { "Authorization": "Bearer ${input:spicrawl-api-key}" } } } } ``` Run **MCP: List Servers** from the Command Palette and start `spicrawl`, or use the **Start** action above the server in the file. Enter the key when prompted. Open the Chat view, switch to **Agent** mode, and open the tools picker: `spicrawl` lists 25 `spicrawl_*` tools. > **Warning:** Use `${input:...}` for the key, not `${env:SPICRAWL_API_KEY}`. Substitution of `${env:...}` in `headers` is a reported VS Code bug ([microsoft/vscode#336232](https://github.com/microsoft/vscode/issues/336232)): the server receives the literal text and answers 401. For every workspace, run **MCP: Open User Configuration** and put the same `inputs` and `servers` there. VS Code also reads a workspace `.mcp.json`, but that file uses the `mcpServers` key instead. **From the command line.** `code --add-mcp` adds a server to your user profile. The JSON takes `name`, the server fields and an optional `inputs` array (bash or zsh quoting): ```bash code --add-mcp '{"name":"spicrawl","type":"http","url":"https://mcp.spicrawl.com/mcp","headers":{"Authorization":"Bearer ${input:spicrawl-api-key}"},"inputs":[{"type":"promptString","id":"spicrawl-api-key","description":"Spicrawl API key","password":true}]}' ``` **From a link.** `vscode:mcp/install?` followed by the same JSON, URL-encoded, installs the server after a confirmation. Open it in a browser, or with `xdg-open` on Linux: ```text vscode:mcp/install?%7B%22name%22%3A%22spicrawl%22%2C%22type%22%3A%22http%22%2C%22url%22%3A%22https%3A%2F%2Fmcp.spicrawl.com%2Fmcp%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7Binput%3Aspicrawl-api-key%7D%22%7D%2C%22inputs%22%3A%5B%7B%22type%22%3A%22promptString%22%2C%22id%22%3A%22spicrawl-api-key%22%2C%22description%22%3A%22Spicrawl%20API%20key%22%2C%22password%22%3Atrue%7D%5D%7D ``` Use `vscode-insiders:` for VS Code Insiders. Keep the key out of the link: the `inputs` entry makes VS Code ask for it. ### Copilot CLI Add the server with the `copilot` command. The shell expands `$SPICRAWL_API_KEY`, so the key is written into `~/.copilot/mcp-config.json`: ```bash copilot mcp add --transport http \ --header "Authorization: Bearer $SPICRAWL_API_KEY" \ spicrawl https://mcp.spicrawl.com/mcp ``` Or edit the file yourself: ```json title="~/.copilot/mcp-config.json" { "mcpServers": { "spicrawl": { "type": "http", "url": "https://mcp.spicrawl.com/mcp", "headers": { "Authorization": "Bearer spicrawl_live_..." }, "tools": ["*"] } } } ``` That file is on your machine only; keep it out of version control. A repository can also hold `.mcp.json` or `.github/mcp.json`, but do not commit a key into either. Check the server with `copilot mcp list`, or `/mcp` inside a session. ### Copilot cloud agent The cloud agent (the one you assign issues to) runs on GitHub, so it takes its MCP config from the repository, not from your editor. A repository administrator sets it up in two places. **Step 1: Store the key as a secret** In the repository, open **Settings → Security → Secrets and variables → Agents**, choose **Secrets**, and click **New repository secret**. The name must start with `COPILOT_MCP_`; use `COPILOT_MCP_SPICRAWL_API_KEY`. Only names with that prefix reach MCP configs. **Step 2: Add the server** Open **Settings → Copilot → MCP servers** and paste this JSON. Click **Save MCP configuration**; GitHub validates the syntax. ```json { "mcpServers": { "spicrawl": { "type": "http", "url": "https://mcp.spicrawl.com/mcp", "tools": ["*"], "headers": { "Authorization": "Bearer $COPILOT_MCP_SPICRAWL_API_KEY" } } } } ``` `type` and `tools` are required on every server. `["*"]` enables all 25 tools; list names such as `["spicrawl_scrape"]` to allow fewer. Three things to know: * The agent calls these tools on its own and does not ask for approval first, and each successful call spends credits. Restrict `tools` to what the task needs, and use a key with only the scopes you want it to have. * The cloud agent does not support OAuth servers. That does not affect Spicrawl, which uses a bearer key. * Only tools are supported. Spicrawl exposes tools only, so nothing is lost. The agent's firewall does not apply to MCP servers, so `mcp.spicrawl.com` needs no allowlist entry for the tools to work. It does apply to commands the agent runs in its own shell. If the agent calls the API from code or with `curl`, add `api.spicrawl.com` under **Settings → Copilot → Internet access → Custom allowlist** (a domain entry also covers its subdomains). ## 2. Install the skill VS Code, the Copilot CLI and the cloud agent all load skills from `.github/skills/`, `.claude/skills/` and `.agents/skills/` in the repository, and from `~/.copilot/skills/` and `~/.agents/skills/` for every project (the cloud agent reads the repository ones): ```bash spicrawl skill install --client vscode ``` This writes `.github/skills/spicrawl/SKILL.md` (add `--global` for `~/.copilot/skills/spicrawl/SKILL.md`). Without the CLI: ```bash mkdir -p .github/skills/spicrawl curl -fsSL https://app.spicrawl.com/skill.md -o .github/skills/spicrawl/SKILL.md ``` Commit the file so the cloud agent sees it. See [Agent skill](https://docs.spicrawl.com/agents/skill.md). ## 3. Add instructions Save this as `.github/copilot-instructions.md`. Copilot in VS Code, the Copilot CLI and the cloud agent all include it in relevant requests, so it applies to every task in the repository. If you already keep this file, paste the section into it. ```markdown title=".github/copilot-instructions.md" # Web data (Spicrawl) Use Spicrawl to read web pages: the spicrawl_* MCP tools in chat, or POST https://api.spicrawl.com/v1/scrape with `Authorization: Bearer $SPICRAWL_API_KEY` in code. Docs: https://docs.spicrawl.com/llms.txt (append .md to any page URL for Markdown). - Ask for markdown: `response_format: "markdown"` (API default is html); on spicrawl_scrape, `format: "markdown"`. - Check the site's status, not only the HTTP status: `X-Target-Status` header, or `status` in the JSON envelope (spicrawl_scrape with `format: "json"`). 200 is the page; 404/410 mean it does not exist; 403/429/503 mean the site refused, so escalate. - On an error, switch on `code`. Retry only when `retryable` is true, after `retry_after_seconds`. Read `diagnostics.hint` and change what it names first. Never retry ERR::REQUEST::*, ERR::AUTH::* or ERR::LIMIT::QUOTA_EXCEEDED. - Escalate one step at a time and stop at the first that works: plain fetch (1 credit) -> `js_render: true` (3) -> add the user's own `proxy` if they have one. Empty or skeleton content means render; ERR::UPSTREAM::CHALLENGE or a 403 target status means retry once, then the user's own proxy. - Set `max_cost` on every request to the price of the step you intend. - Keep the cache on (default). Set `cache: false` only for prices, stock or other live data. - Trim tokens with `main_content_only` (on by default for markdown), `include_tags`, `exclude_tags`. - For more than 20 URLs, use a batch job (spicrawl_batch_submit / POST /v1/batch). Batch items return raw HTML and apply only render, proxy and block_resources settings. - Never print or commit SPICRAWL_API_KEY. Log `X-Request-Id` for failures. ``` Copilot also reads `AGENTS.md` at the repository root (the nearest one wins). To scope the rules to some files, save them as `.github/instructions/spicrawl.instructions.md` with frontmatter `applyTo: "**/*.ts,**/*.py"` (a glob, comma-separated). ## What the CLI writes `spicrawl init --client vscode` and its two parts, `spicrawl mcp install --client vscode` and `spicrawl skill install --client vscode`, target VS Code with Copilot: | Command | Writes | | ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `spicrawl mcp install --client vscode` | `.vscode/mcp.json` with the `servers.spicrawl` entry and the `spicrawl-api-key` `promptString` input shown above. It keeps other entries in the file, and needs a file without comments. | | Same, with `--global` | `mcp.json` in VS Code's user profile directory. | | `spicrawl skill install --client vscode` | `.github/skills/spicrawl/SKILL.md`, or `~/.copilot/skills/spicrawl/SKILL.md` with `--global`. | It does not write the Copilot CLI or cloud agent config, or the instructions file. Do those by hand as above. ## A first task to try Open Copilot Chat in **Agent** mode and ask: ```text Use Spicrawl to read https://example.com/pricing as markdown. List each plan with its monthly price. If the page comes back empty, retry with rendering. ``` The agent should call `spicrawl_scrape` with `url` and `format: "markdown"`, and add `render: true` only if the first result is empty. For a coding task: ```text Add a fetchPage(url) function to this project that calls the Spicrawl API for markdown, following the Web data instructions. Read the key from process.env.SPICRAWL_API_KEY. ``` For the cloud agent, open an issue with the same text and assign it to Copilot. Compare the result with the reference implementation in [Best practices](https://docs.spicrawl.com/agents/best-practices.md). ## Troubleshooting | Symptom | Fix | | ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | VS Code shows the server as errored or 401 | The key was empty or wrong. Restart the server from **MCP: List Servers** and re-enter the key. If you used `${env:...}` in `headers`, switch to `${input:...}`. | | VS Code shows no tools | Switch the Chat view to **Agent** mode, open the tools picker and enable the `spicrawl` tools. | | The CLI fails with "must be plain JSON" on `.vscode/mcp.json` | The file has comments. Remove them, or add the entry by hand. | | The cloud agent gets 401 from Spicrawl | The secret is not named `COPILOT_MCP_*`, or the JSON references a different name. The secret must exist under **Agents**, not **Actions**. | | The cloud agent cannot use the server | `type` or `tools` is missing in the JSON. Add `"type": "http"` and `"tools": ["*"]`. | | Its shell cannot reach `api.spicrawl.com` | Add the domain to the custom allowlist under **Settings → Copilot → Internet access**. | | The agent writes `js_render` on `spicrawl_scrape` and gets "unknown argument" | The tool's argument is `render`. The API field is `js_render`. | | Usage tools return `ERR::AUTH::INSUFFICIENT_SCOPE` | Grant the `read` scope to the key in the dashboard. |