Render JavaScript pages

Run a page in a browser with js_render, pick an engine, wait for content with wait and wait_for, and let mode=auto escalate only when it has to.

Use this when the page you fetched is an empty shell: a <div id="root"></div>, a "Loading…" placeholder, or markdown with no article in it. The default engine (fetch) downloads the HTML and runs no JavaScript. js_render=true loads the page in a browser, lets its scripts build the content, then captures it. Rendering is also required for wait, wait_for, actions, block_resources, network_capture, headless, screenshot and response_format=pdf; setting any of those on the fetch engine is a 400, never silently ignored.

Minimal request

curl https://api.spicrawl.com/v1/scrape \
  -H "Authorization: Bearer $SPICRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/products/42",
    "js_render": true,
    "wait_for": ".price",
    "response_format": "markdown"
  }'

What comes back

The rendered page, in the format you asked for. Check two headers:

HTTP/1.1 200 OK
Content-Type: text/markdown; charset=utf-8
X-Engine: obscura
X-Target-Status: 200
X-Credits-Charged: 3

X-Engine is the engine that actually ran.
X-Warning appears when something was adjusted. RENDER_DEGRADED means wait_for never matched and you got the page as it stood. ENGINE_SUBSTITUTED means a different render engine served the request (see below).

Engines

Engine	How to select	Credits	Use it for
`fetch`	default	1	Server-rendered HTML. No JavaScript.
`obscura`	`js_render=true`	3	Most JavaScript sites. The volume render engine.
`chromium`	`engine=chromium`	8	Real Chromium. The only engine with `headless=false`. Screenshots and PDF. Never chosen for you.

engine pins the engine exactly: no escalation, no substitution. Your plan must include a pinned engine, otherwise the request fails with 403 ERR::AUTH::ENGINE_NOT_ENTITLED at 0 credits.

ENGINE_SUBSTITUTED

js_render=true without an engine pin asks for "a browser", not for Obscura specifically. If Obscura is not deployed, the request is rendered on Chromium, billed at that engine's price, and flagged with X-Warning: ENGINE_SUBSTITUTED. Pin engine=obscura if you would rather fail than pay more.

Options that matter

Field	Limits	Effect
`wait`	0–30000 ms, default 0	Fixed delay after load, before capture. Use `wait_for` instead when you can name an element.
`wait_for`	CSS selector	Capture as soon as the element appears. If it never does, you get the page as it stood plus `X-Warning: RENDER_DEGRADED`.
`wait_for_timeout`	0–120000 ms, default 0 (engine default)	How long to wait for `wait_for`. Requires `wait_for` (400 `ERR::REQUEST::INCOMPATIBLE_FLAGS` otherwise).
`block_resources`	`image`, `font`, `media`, `stylesheet`, `script`, `xhr`, `fetch`, `websocket`, `document`, `other`, or `none`	Subresources the browser does not load. Plural and short aliases (`images`, `css`, `js`) are accepted.
`headless`	boolean	`false` runs on a real display (1920x1080 instead of 800x600) and adds about 1 s. `engine=chromium` only.
`mode`	`auto`	Escalate `fetch` → `obscura` until a rung returns a usable page.

block_resources defaults

When you omit block_resources, renders block images and fonts. They are most of a page's bytes and do not change the text or any extraction. The default is lifted for screenshot, response_format=pdf, actions, and a network_capture that records images or fonts. An explicit list replaces the default rather than adding to it; ["none"] blocks nothing. Block script only when the content is already in the HTML, or the page will not build.

mode=auto

mode=auto tries fetch first, moves to obscura when the result is unusable (a bot challenge, or a target status that is not billable). You pay only for the rung that succeeded, and 0 if all fail. Two things to know:

max_cost and your monthly allowance are checked against 25 credits, the price of the ladder's top rung, even when the request stops at fetch.
When the request escalated, the response carries X-Warning: ESCALATED and the JSON envelope includes diagnostics.attempts listing each rung and why it was abandoned.

mode=auto cannot be combined with js_render or engine (400 ERR::REQUEST::INCOMPATIBLE_FLAGS), and it never reaches chromium.

Failure modes

Code	HTTP	What to do
`ERR::REQUEST::INCOMPATIBLE_FLAGS`	400	A browser-only flag without a browser engine, `wait_for_timeout` without `wait_for`, or `mode=auto` with an engine flag. The `detail` names the flag.
`ERR::REQUEST::CAPABILITY_UNSUPPORTED`	400	`headless=false` on an engine other than `chromium`. Add `engine=chromium`. 0 credits.
`ERR::AUTH::ENGINE_NOT_ENTITLED`	403	Your plan cannot pin that engine. The `detail` lists the engines you can pin.
`ERR::ENGINE::UNAVAILABLE`	503	No capacity, or the engine is not deployed. Retryable after `Retry-After`.
`ERR::ENGINE::RENDER_FAILED`	502	The browser could not render the page. Retryable.
`ERR::UPSTREAM::TIMEOUT`	504	The render timed out, often while waiting on `wait_for`. Raise `wait_for_timeout` or fix the selector.
`ERR::LIMIT::MAX_COST_EXCEEDED`	400	The engine (or the top `mode=auto` rung) costs more than `max_cost`.

All of these cost 0 credits.

Cost

Per successful request: fetch 1, obscura 3, chromium 8. wait, wait_for and block_resources add nothing. Failures cost 0. Read X-Credits-Charged for the amount billed. See Credits.

Anti-bot when a render returns a challenge page
Browser actions to click, scroll and fill before capture
Network capture to read the page's own JSON API calls
Screenshots and PDF

Render JavaScript pages

On this page