spicrawlspicrawlDocs

Authentication

Authenticate Spicrawl API calls with a Bearer API key, choose live or test keys, and grant the scopes each route needs.

Send your API key in the Authorization header on every request. Read it from the SPICRAWL_API_KEY environment variable; never write it into code.

curl -sS "https://api.spicrawl.com/v1/scrape" \
  -H "Authorization: Bearer $SPICRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/products/42", "response_format": "markdown"}'

A missing key is 401 ERR::AUTH::MISSING_KEY. A key the server does not recognise is 401 ERR::AUTH::INVALID_KEY. See Errors.

Key format

PrefixEnvironmentUse
spicrawl_live_…liveProduction. Spends your organization's credits.
spicrawl_test_…testDevelopment and CI. A test key can never spend live credits.

After the prefix comes 52 characters from a-z and 2-7, so a key survives URL-encoding and case-normalising log pipelines unchanged. A key's environment is fixed when it is created and cannot be changed.

Get a key

  1. Sign in at app.spicrawl.com.
  2. Open API Keys and create a key. Choose live or test and the scopes it needs.
  3. Copy the key. The full secret is shown once; afterwards the dashboard shows only its first 16 characters.
export SPICRAWL_API_KEY="spicrawl_live_…"   # paste your key; keep it out of shell history files you share

The workspace default key

Every workspace has one live key named Default. The dashboard creates it for you and uses it for its own calls, including the Playground and the Usage page. It carries all five scopes, including browser and read. Members can reveal it in the dashboard, at most 10 times a minute per user.

Use the default key to try things out. For a production service, create a dedicated key with only the scopes it needs, so you can revoke it without affecting the dashboard.

Scopes

A key can call a route only if it carries that route's scope. A missing scope is 403 ERR::AUTH::INSUFFICIENT_SCOPE, and detail names the scope.

ScopeRoutesOn new keys by default
scrapeGET and POST /v1/scrapeyes
batchevery /v1/batch route, reads includedyes
sessionsevery /v1/sessions routeyes
browserGET /v1/browser (CDP WebSocket) Coming soonno, grant it deliberately
read/v1/usage, /v1/usage/summary, /v1/usage/reconciliation; GET /v1/requests?all_projects=true and another project's GET /v1/requests/{id}no, grant it deliberately
noneGET /v1/requests and GET /v1/requests/{id} for the key's own project, GET /v1/workersany valid key
  • browser is off by default because a CDP socket is a browser the holder can drive freely for the whole session, a larger capability than a scrape call.
  • read is off by default because it exposes your organization's spend and volume. A leaked scraping key should not reveal that too.
  • The request log of the key's own project needs no scope: a key that can scrape has already seen every response it fetched. Every other project's requests (all_projects=true, or one request by id) need read: that key has seen none of them.

Handle INSUFFICIENT_SCOPE

403 ERR::AUTH::INSUFFICIENT_SCOPE
{
  "type": "https://docs.spicrawl.com/errors#AUTH_INSUFFICIENT_SCOPE",
  "title": "API key lacks the required scope",
  "status": 403,
  "code": "ERR::AUTH::INSUFFICIENT_SCOPE",
  "detail": "This API key does not carry the `read` scope. Mint a key with it, or use one that has it.",
  "retryable": false,
  "doc_url": "https://docs.spicrawl.com/errors#AUTH_INSUFFICIENT_SCOPE",
  "target_status": null
}

Do not retry. Create a key with the named scope in the dashboard, or use the workspace default key, then repeat the call with the new key.

Remote browser Coming soon

The browser scope and GET /v1/browser are for remote browser sessions over CDP, which are not available yet. See CDP browser.

Keep keys out of code and logs

  • Read the key from SPICRAWL_API_KEY or a secret manager. Do not commit it, and do not paste it into prompts or issue trackers.
  • Log X-Request-Id, not the key. To identify a key in logs, use its first 16 characters (spicrawl_live_abcd), which is what the dashboard shows.
  • Give each service its own key with the smallest set of scopes, so revoking one does not break the others.
  • Use spicrawl_test_… keys in CI.
  • If a key leaks, revoke it in the dashboard. A revoked key returns 401 ERR::AUTH::REVOKED_KEY within 60 seconds.

Auth errors

CodeHTTPWhat to do
ERR::AUTH::MISSING_KEY401Send Authorization: Bearer $SPICRAWL_API_KEY.
ERR::AUTH::INVALID_KEY401Check for a truncated or quoted key; copy it again.
ERR::AUTH::REVOKED_KEY401Create a replacement key.
ERR::AUTH::EXPIRED_KEY401Create a replacement key.
ERR::AUTH::INSUFFICIENT_SCOPE403Use a key with the scope named in detail.
ERR::AUTH::FORBIDDEN403The organization may not do this, for example it is suspended. Contact support.

None of these are retryable, and none cost credits.

On this page