Stay logged in with sessions

Use this when a site needs state across requests: you log in once and then read ten account pages, or a site sets a consent cookie you do not want to fight every time. A session is a sealed cookie jar and web-storage snapshot, tied to one browser, which Spicrawl provisions for it automatically, and, by default, one exit IP. You create it, pass its id as session_id on /v1/scrape, and cookies the page sets are written back to it after each render.

Requires an API key with the sessions scope to create and manage sessions, and scrape to use one.

Minimal flow

curl https://api.spicrawl.com/v1/sessions \
  -H "Authorization: Bearer $SPICRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"ttl_seconds": 7200}'

{
  "url": "https://example.com/login",
  "session_id": "01J9ZQ4M7R3T8VX2K5N6P0B1CD",
  "js_render": true,
  "actions": [
    {"fill": {"selector": "input[name=email]", "value": "ada@example.com"}},
    {"fill": {"selector": "input[name=password]", "value": "correct-horse-battery", "secret": true}},
    {"click": "button[type=submit]"},
    {"wait_for": ".account-menu"}
  ]
}

curl https://api.spicrawl.com/v1/scrape \
  -H "Authorization: Bearer $SPICRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/account/orders", "session_id": "01J9ZQ4M7R3T8VX2K5N6P0B1CD", "js_render": true, "response_format": "markdown"}'

curl -X POST https://api.spicrawl.com/v1/sessions/01J9ZQ4M7R3T8VX2K5N6P0B1CD/release \
  -H "Authorization: Bearer $SPICRAWL_API_KEY"
# CLI: spicrawl sessions release 01J9ZQ4M7R3T8VX2K5N6P0B1CD

Rules that trip people up

One request at a time per session. A session is single-writer. While a render holds its lease, a second scrape with the same session_id gets 409 ERR::SESSION::BUSY with Retry-After. Serialise requests per session, or create one session per concurrent worker.

Managed exits for sessions. Coming soon Pinning a session to a Spicrawl-managed exit (sticky_key, rotate_ip) and session proxy tiers (premium_proxy, proxy_country, region_pool) are not available during the beta. Many sites challenge a login whose IP moves, so if you route through your own proxy, use one with a fixed exit.

Release destroys the state. POST /v1/sessions/{id}/release marks the session released and deletes its cookies and storage in the same transaction, along with the browser's own copy. It is not "give back the lease, keep the login". A later /v1/scrape with that session_id is refused with 410 ERR::SESSION::RELEASED at 0 credits. The session row stays readable with GET /v1/sessions/{id}; the context does not. DELETE /v1/sessions/{id} also purges the context and removes the row, so the id stops resolving (a 204 is returned even for an unknown id). Both answer 409 ERR::SESSION::BUSY while a render runs; add force=true to end the session anyway and fail the running task.

Sessions are never cached. A request with session_id bypasses the result cache (Cache-State: bypass) and is always fetched fresh.

The rendering is fixed. Spicrawl provisions the session's rendering tier (fetch, js or browser) when you create it, and every scrape on the session runs there, whatever else the request says. The session object's engine field reports it. Rendering is never chosen per request (engine and headless on /v1/scrape are deprecated and ignored).

An unknown session_id is an error. /v1/scrape checks the session before doing anything. An unknown, malformed or deleted id, or another organization's, is 404 ERR::SESSION::NOT_FOUND at 0 credits. The request never runs without the session.

Use is counted. Each successful scrape adds 1 to usage_count, sets last_used_at and moves expires_at to that time plus ttl_seconds, never past hard_expires_at. Failed scrapes and /context reads do not count.

Save and restore a login

GET /v1/sessions/{id}/context is the only endpoint that returns the cookies, local storage, session storage and IndexedDB. It works only while the session is active, does not slide the TTL, and is sent with Cache-Control: no-store, private.

spicrawl sessions context 01J9ZQ4M7R3T8VX2K5N6P0B1CD > login.json

Store that object yourself before you release the session. To clone the login later, pass its session_context to POST /v1/sessions. The serialised context is limited to 1 MiB (413 ERR::REQUEST::PAYLOAD_TOO_LARGE).

Create options

Failure modes

A session a worker has taken out of service shows a retired field (expired, usage_exhausted or explicit); replace it.

Cost

Creating, reading and releasing sessions costs nothing. Each scrape on a session is billed like any other scrape, at the price of the rendering it used (see Credits). A session's browser does full-browser rendering, so a scrape costs 8 credits (32 on a premium_proxy exit). Failures, including a 409 or 410, cost 0. See Credits.

Related

• Browser actions for the login steps
• Proxies and geo for using your own proxy
• CDP browser
• Caching