Response headers

Every header Spicrawl returns on /v1/scrape: request id, credits, engine, target status, proxy, cache state, limits and X-Warning codes.

A /v1/scrape response tells you what ran, what it cost and what the site said, in headers, whatever the body format. Read them with curl -D - (dump headers) or from your HTTP client's response object.

curl -sS -D - -o page.md "https://api.spicrawl.com/v1/scrape" \
  -H "Authorization: Bearer $SPICRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/products/42", "response_format": "markdown"}'

HTTP/2 200
content-type: text/markdown; charset=utf-8
x-request-id: 01M0HF5WFWE7PRE8KZHDTNETWN
x-credits-charged: 1
x-request-cost: 1
x-credits-remaining: 987
x-engine: fetch
x-proxy-source: direct
x-target-status: 200
x-final-url: https://example.com/products/42
x-ratelimit-limit: 10
x-ratelimit-remaining: 9
concurrency-limit: 5
concurrency-remaining: 4
cache-state: miss

Two statuses

The HTTP status line is the platform's. The site's status is X-Target-Status.

HTTP status	`X-Target-Status`	Meaning	Credits
`200`	`200`	The site returned the page.	charged
`200`	`404` or `410`	The site said the page does not exist. The call succeeded.	charged
`200`	`403`, `500`, …	The site refused or failed. You get its body. The call still succeeded.	0
`4xx`/`5xx` with `application/problem+json`	absent	Spicrawl could not complete the request. Read `code`.	0

Always check X-Target-Status (or envelope status) before trusting the body. A 200 with X-Target-Status: 403 is a block page, not your content. original_status=true moves the target's status onto the HTTP status line; avoid it unless a client requires it, because a target 404 then looks like an API error.

Header reference

Header	Values	Meaning
`X-Request-Id`	ULID	This request's id, on every response including errors. Quote it to support; pass it to `GET /v1/requests/{id}` for the trace.
`X-Credits-Charged`	integer	Credits actually billed. `0` on any failure, cache hit or non-billable target status.
`X-Request-Cost`	integer	Price of the engine and proxy tier that ran. `0` on a cache hit. Differs from `X-Credits-Charged` exactly when nothing was billed.
`X-Credits-Remaining`	integer	Credits left in your organization's monthly allowance after this request's charge, rounded down. Counts credits held by running batch jobs and open browser sessions. Never negative. On a `402` it can be above `0`: less than the request needed. Absent when your organization has no monthly limit, when the request was refused before the credit check (a validation error, a rate or concurrency limit), or when the balance could not be read. See Credits.
`X-Engine`	`fetch`, `obscura`, `chromium`	Engine that ran. Under `mode=auto`, the rung that produced the result.
`X-Proxy-Source`	`custom`, `direct`, `pool`	Whose exit carried the request: your `proxy`, no proxy, or the managed pool Coming soon.
`X-Proxy-Endpoint`	string	Identifier of the managed pool exit that served (managed pool Coming soon). Absent for `custom` and `direct`.
`X-Target-Status`	integer	The site's HTTP status. Absent when no document was retrieved.
`X-Final-Url`	URL	URL after redirects, when known.
`X-RateLimit-Limit`	integer	Requests per second allowed for this key.
`X-RateLimit-Remaining`	integer	Tokens left in the rate-limit bucket.
`X-RateLimit-Reset`	integer	Seconds until a retry is allowed. Sent only when the limiter imposed a wait.
`Concurrency-Limit`	integer	Concurrent in-flight requests allowed for your organization.
`Concurrency-Remaining`	integer	Concurrent slots left.
`Cache-State`	`hit`, `miss`, `bypass`	Result-cache outcome. Always present on a completed scrape.
`X-Warning`	`CODE: message`	A non-fatal decision the platform made for you. Repeated, one header per warning.
`Retry-After`	integer	On retryable errors: seconds to wait before retrying.
`Content-Type`	see below	Shape of the body.

Content-Type follows response_format: text/html (html, the default), text/markdown (markdown), text/plain (text), application/json (json, and any request that forces the envelope), application/pdf (pdf). Errors are always application/problem+json.

Cache-State

Value	Meaning	Credits
`hit`	Served from a stored result younger than `cache_ttl`. Nothing was fetched.	0
`miss`	The request was eligible for the cache, nothing fresh was stored, so it was fetched now. A billable success is stored for next time.	normal price
`bypass`	The request was not eligible: `cache=false` or `cache_ttl=0`, a `session_id`, `actions`, a `method` other than `GET`, or caching is off on this deployment.	normal price

The default cache_ttl is 172800 seconds (48 hours). Set cache: false for time-sensitive data. See Caching.

X-Warning codes

Each X-Warning header is CODE: message. Switch on the code; the message is for humans. The JSON envelope's warnings array carries the messages only.

Code	When you see it	What to do
`PROXY_FLAG_IGNORED`	You set `proxy` together with a managed-pool flag (`premium_proxy` or `proxy_country` Coming soon). Your own proxy wins and the pool flags were ignored.	Remove the ignored flags.
`FORMAT_COERCED`	You asked for `html`, `markdown` or `text`, but `extract`, `autoparse`, `links`, `ai_extract`, `network_capture` or `screenshot` needs the JSON envelope, so `response_format` became `json`.	Send `response_format: "json"` explicitly, or drop the flag. Markdown is in the envelope's `content`.
`CACHE_TTL_CLAMPED`	`cache_ttl` was above the deployment's maximum (48 hours by default) and was lowered to it.	Send a smaller `cache_ttl`.
`ENGINE_SUBSTITUTED`	You asked for a browser render without pinning `engine`, and this deployment served it on a different render engine than the default. `X-Engine` names the engine that ran, and `X-Request-Cost` its price.	Pin `engine` if you need a specific one; a pinned engine is never substituted.
`RENDER_DEGRADED`	The browser returned the page but could not deliver something you asked for, for example a `wait_for` selector that never appeared.	Check the selector, or raise `wait_for_timeout`.
`ESCALATED`	A cheaper rung was tried and abandoned before the one that answered: under `mode=auto`, or when a bot challenge on your own `proxy` moved the request to Camoufox Coming soon. One header per abandoned rung, sent on success too, so you can see why the price rose.	Under `mode=auto`, send the engine that worked (`js_render=true`) next time to skip the failed rung.

Warnings also appear in an error body's warnings array when a request fails.

Read the headers in code

curl -sS -D headers.txt -o page.md "https://api.spicrawl.com/v1/scrape" \
  -H "Authorization: Bearer $SPICRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/products/42", "response_format": "markdown"}'
grep -i -E '^(x-target-status|x-credits-charged|x-engine|cache-state|x-warning):' headers.txt