spicrawlspicrawlDocs
Batch

Get a batch job and its progress

Returns the job with live progress counters. Cheap to call: progress is read from sharded counters, not by counting items.

Requires scope: batch

GET
/v1/batch/{batchID}

Returns the job with live progress counters. Cheap to call: progress is read from sharded counters, not by counting items. Poll until status is completed, failed or cancelled; a reasonable interval is 2-5 s for jobs under ~1,000 items and 10-30 s for larger ones. If open is true the job will never finish on its own — call POST /v1/batch/{batchID}/close when you are done appending.

Authorization

bearerAuth
AuthorizationBearer <token>

Authorization: Bearer <key>. Read the key from the SPICRAWL_API_KEY environment variable; never hard-code or log it. spicrawl_test_… keys can never spend live credits. Scopes: scrape, batch, sessions (granted by default), browser and read (granted deliberately). A missing scope is 403 ERR::AUTH::INSUFFICIENT_SCOPE naming the scope.

In: header

Path Parameters

batchID*string

The job id (26-character ULID as returned). The 32/36-character UUID form of the same id is also accepted. A malformed id and another tenant's id both answer 404.

Length26 <= length <= 36

Response Body

application/json

application/problem+json

application/problem+json

application/problem+json

application/problem+json

curl -X GET "https://example.com/v1/batch/01J9Z6T3W9E21T5TZARVJRVN5C"
{  "id": "01J9Z6T3W9E21T5TZARVJRVN5C",  "project_id": "01J8X2M4C7Q1H5V9P3R6T8W0YB",  "name": "catalog-2026-09-22",  "status": "running",  "status_url": "/v1/batch/01J9Z6T3W9E21T5TZARVJRVN5C",  "results_url": "/v1/batch/01J9Z6T3W9E21T5TZARVJRVN5C/results",  "params": {    "js_render": true,    "proxy_country": "us",    "credit_budget_micro": 8000000  },  "total_items": 2,  "concurrency": 20,  "priority": 100,  "max_attempts": 3,  "estimated_credits": 8,  "estimated_credits_micro": 8000000,  "progress": {    "total": 2,    "completed": 1,    "succeeded": 1,    "failed": 0,    "cancelled": 0,    "skipped": 0,    "remaining": 1,    "percent_complete": 50,    "credits_charged": 4,    "credits_charged_micro": 4000000,    "bytes": 326000  },  "submitted_at": "2026-09-22T06:41:12Z",  "started_at": "2026-09-22T06:41:13Z",  "results_expire_at": "2026-09-25T06:41:12Z"}

Submit a batch job POST

Queues many URLs as one asynchronous job and returns `202` with the job object immediately.

Page through a job's finished items (JSONL) GET

Streams one page of FINISHED items (status succeeded, failed, cancelled or skipped) in `seq` order as JSON Lines: one `BatchResultLine` object per line, each terminated by `\n`, no array, no trailing pagination object.