# Logs, usage and status > Inspect recent requests with spicrawl logs, read billed usage with spicrawl usage, and check the API end to end with spicrawl status. Source: https://docs.spicrawl.com/cli/logs-and-usage Three read-only command groups tell you what happened, what it cost, and whether the API is reachable from this machine. ```bash spicrawl logs --errors --limit 20 # recent failures spicrawl logs get 01M0FK1EP2E7FBMWAR7W10SMXW spicrawl usage summary # credits used against the plan spicrawl status # API reachable, workers ready, key accepted ``` ## `spicrawl logs` Lists recent API requests in the key's project, newest first (`GET /v1/requests`). With `--all-projects`, every project in the key's organization instead, which needs a key with the `read` scope. Records are kept for the organization's retention window (24 hours by default); older ones are gone. ```bash spicrawl logs spicrawl logs --errors --limit 20 spicrawl logs --all-projects --errors spicrawl logs --status blocked --all --jsonl | jq -r .url ``` | Flag | API parameter | Default | Meaning | | ---------------- | --------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--errors` | `only_errors=true` | off | Every request whose status is not `success`. Cannot be combined with `--status` (exit `2`). | | `--status S` | `status` | all | One of `success`, `failed`, `timeout`, `blocked`, `rejected`, `cancelled`. Any other value exits `2`. | | `--limit N` | `limit` | `50` | Page size, 1-200. | | `--all` | `before`, `before_id` | off | Follow the cursor until every retained record is read. | | `--jsonl` | none | off | One compact JSON record per line, streamed page by page. | | `--all-projects` | `all_projects=true` | off | Every project in the key's organization, not just its own. Needs the `read` scope (`403 ERR::AUTH::INSUFFICIENT_SCOPE` without it). Sent on every page. | Output: * Human mode: a `TIME STATUS HTTP ENGINE CREDITS TOTAL_MS URL` table, with a `PROJECT` column after `TIME` under `--all-projects`. When more pages exist and you did not pass `--all`, stderr says `more records exist; pass --all to fetch every page`. * JSON mode: `{"data": [...], "page": {...}}`. `page` carries `retention_hours`, `retained_since`, `has_more`, `next_before` and `next_before_id`. * `--jsonl`: one `RequestLogEntry` per line, no wrapper. Each record has `id`, `created_at`, `engine`, `status`, `http_status`, `error_code`, `error_detail`, `attempts`, `url`, `proxy`, `spans`, `blocked`, `credits_micro` and `features`. Credits are in micro-credits (1 credit = 1,000,000). ```bash # Error codes of the last day's failures, most frequent first spicrawl logs --errors --all --jsonl | jq -r .error_code | sort | uniq -c | sort -rn ``` ## `spicrawl logs get` Shows one request: outcome, error, latency spans, proxy and bot-block details (`GET /v1/requests/{id}`). The id is the `X-Request-Id` response header, the `request_id` in a scrape's JSON output, or the `request_id` of a problem document. Every failed command prints it in human mode as `request id: (spicrawl logs get )`. ```bash spicrawl logs get 01M0FK1EP2E7FBMWAR7W10SMXW spicrawl logs get 01M0FK1EP2E7FBMWAR7W10SMXW --json | jq .spans ``` Spans are measured by different processes and do not sum to `total_ms`. A span shown as `—` was not measured, which is not the same as `0`. An id older than the retention window returns `ERR::REQUEST::NOT_FOUND` or `ERR::REQUEST::BEYOND_RETENTION` (exit `4`). ## `spicrawl usage` Billed usage for the key's organization, broken down by one dimension (`GET /v1/usage`). Needs a key with the read scope; otherwise `ERR::AUTH::INSUFFICIENT_SCOPE` (exit `3`). ```bash spicrawl usage spicrawl usage --from 2026-08-01 --to 2026-09-01 --group-by engine spicrawl usage --group-by project --metrics credits,requests spicrawl usage --group-by key --metrics requests,client_cli ``` | Flag | API parameter | Default | Meaning | | ------------------- | ------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--from YYYY-MM-DD` | `from` | 29 days ago | First UTC day, inclusive. | | `--to YYYY-MM-DD` | `to` | tomorrow | End UTC day, **exclusive**. One day, 2026-08-01, is `--from 2026-08-01 --to 2026-08-02`. At most 400 days per call. | | `--group-by G` | `group_by` | `day` | `day`, `project`, `engine`, `feature` or `key`. | | `--metrics LIST` | `metrics` | all | Comma-separated: `requests`, `credits`, `engine_ms`, `bytes_egress`, `bytes_ingress`, `proxy_bytes_dc`, `proxy_bytes_resi`, `extractions`, `ai_extractions`, `batch_items`. | | `--project ID` | `project_id` | every project | Only this project. | JSON mode prints the API's `UsageBreakdown` unchanged: `period`, `group_by`, `groups`, `totals`, `unattributed`, `usage_as_of`, `stale_seconds`, `warnings`. `totals` is the billed figure; trust it over the sum of `groups`. Credits are in micro-credits in JSON; the human table shows whole credits. `credits` is total spend, batch items included, in every grouping; with `--group-by feature`, the batch share of it is under `batch`. With `--group-by key` the body is instead `{period, group_by: "key", keys[]}`. Each key has `key_id`, `name`, `prefix`, `project_id`, `revoked`, `days[]` (`{day, metrics}`) and window `totals`, read from the per-key rollup. Requests made without a key (the dashboard) are not in it, and a project-scoped key only sees its own project's keys. No caller IPs or countries are returned. The human table shows one row per key with window totals. Besides the billing metrics, `--metrics` accepts `requests_failed`, `requests_timeout`, the per-feature counters (`feature_screenshot`, `feature_pdf`, ...) and the per-client counters (`client_cli`, `client_mcp`, `client_sdk`, `client_other`). ### `spicrawl usage summary` Current-period usage against the plan's credit allowance (`GET /v1/usage/summary`). When no subscription covers today, the organization is on the free plan (`plan.code` `free`, `subscription_status` `none`) and the period is its allowance month, which runs from the day of the month the organization was created. That is the window the monthly allowance is enforced over, not a billing period, so it is not a bill preview. ```bash spicrawl usage summary spicrawl usage summary --json | jq .allowance.remaining_micro ``` `allowance` is the monthly allowance as the API enforces it: `limit_micro`, `limit_source` (`default` or `override`), `used_micro`, `remaining_micro`, `window_start` and `resets_at`. Its `used_micro` includes credits held for work in flight (open browser sessions, unfinished batch items), so `allowance.remaining_micro` is what you can still spend. `credits` is metered usage for the period: `included_micro`, `used_micro`, `remaining_micro`, `overage_micro` and `used_basis_points`. It includes batch spend, but not credits held for work still running. ### `spicrawl usage reconciliation` Compares billed usage for one UTC day with usage re-derived from the request log (`GET /v1/usage/reconciliation`). Report only; nothing is repaired. ```bash spicrawl usage reconciliation spicrawl usage reconciliation --day 2026-09-21 --json | jq .drift ``` | Flag | Default | Meaning | | ------------------ | --------- | --------------------- | | `--day YYYY-MM-DD` | yesterday | UTC day to reconcile. | Read `interpretation` before `drift`: when the request log holds no rows for the day, every billed row shows as drift, which is not evidence of overbilling. Batch usage is not reconciled here, because batch items are not in the request log: `batch_items` never appears, and a `credits` row compares only the non-batch part of the day's credits. ## `spicrawl status` Checks the whole path from this machine to a working API, in three calls: | Call | Checks | | -------------------------- | ------------------------------------------------------------------------------------------------ | | `GET /readyz` | The API is reachable and ready, and whether the cloud browser (`cdp`) is enabled. No key needed. | | `GET /v1/workers` | Which worker kinds are present and ready. | | `GET /v1/requests?limit=1` | The key is accepted. | ```bash spicrawl status spicrawl status --json | jq .workers spicrawl status --base-url http://localhost:8080 --api-key "$SPICRAWL_API_KEY" ``` ```json { "api_reachable": true, "ready": true, "checks": { "postgres": "ok", "redis": "ok" }, "cdp": "disabled", "key_valid": true, "key_source": "env", "base_url": "https://api.spicrawl.com", "workers": { "browser": { "present": 2, "ready": 2 }, "egress": { "present": 1, "ready": 1 } } } ``` `cdp` says whether `GET /v1/browser` ([remote browser](https://docs.spicrawl.com/guides/cdp-browser.md) (coming soon)) can serve a session on this deployment: `enabled`, `disabled` (the operator did not install the CDP browser; `/v1/browser` answers `501` or does not exist, and every other endpoint still works) or `unknown` (the browser worker could not be asked). Human output prints it as a `cdp:` line. `key_valid` is `true`, `false`, or `null` when the check could not tell (for example a network error mid-check). `key_error` explains a rejected or missing key; `key is valid but lacks the read scope` still counts as valid. `workers` is read only once the key is accepted; `workers_error` is set when that call failed. | Exit code | When | | --------- | ------------------------------------------------------------------------------------------------------------------------- | | `0` | The API was reached and the key works. A reachable API that reports not ready still exits `0`; read `ready` and `checks`. | | `3` | The key is missing or rejected. | | `10` | The API could not be reached. |