CLI authentication and config

The CLI needs a Spicrawl API key (spicrawl_live_… or spicrawl_test_…). Create one at app.spicrawl.com, then either save it or export it:

spicrawl login --api-key "$SPICRAWL_API_KEY"   # validate and save to the config file
# or, with no file at all:
export SPICRAWL_API_KEY=spicrawl_live_...

How the key and base URL are resolved

Each command picks the key and base URL from the first source that has one:

With no key from any source, a command that calls the API fails with exit code 3 and error: no API key: run "spicrawl login", set SPICRAWL_API_KEY, or pass --api-key.

spicrawl login

Validates a key against the API and saves it to the config file.

spicrawl login --api-key spicrawl_live_...
SPICRAWL_API_KEY=spicrawl_live_... spicrawl login
spicrawl login --api-key spicrawl_live_... --base-url http://localhost:8080
spicrawl login                      # interactive: paste the key

• The key comes from --api-key, then $SPICRAWL_API_KEY. With neither, and only when stdin is a terminal, login asks you to paste it. The pasted key is visible as you type.
• Without a terminal and without a key it fails with exit code 2. It never waits for input an agent cannot give.
• Before saving, it makes one call (GET /v1/requests?limit=1). A rejected key exits 3; an unreachable API exits 10. Nothing is saved on failure.
• --base-url, when given, is saved too.

JSON output:

{
  "base_url": "https://api.spicrawl.com",
  "key": "spicrawl_live_2Hf8…9QaZ",
  "saved": "/home/dev/.config/spicrawl/config.json"
}

Keys are always masked in output: the first 16 characters, …, and the last 4.

spicrawl logout

Removes api_key from the config file. A saved base_url is kept.

spicrawl logout

{ "path": "/home/dev/.config/spicrawl/config.json", "removed": true }

logout does not touch $SPICRAWL_API_KEY or --api-key. If SPICRAWL_API_KEY is still set, it prints warning: SPICRAWL_API_KEY is still set in the environment and will keep being used on stderr.

spicrawl auth status

Shows the key (masked) and base URL the CLI would use, where each came from (flag, env, file, default), the config file path, and whether the API accepts the key.

spicrawl auth status
spicrawl auth status --offline           # do not call the API
spicrawl auth status --json | jq .key_valid

{
  "api_key": "spicrawl_live_2Hf8…9QaZ",
  "api_key_source": "env",
  "base_url": "https://api.spicrawl.com",
  "base_url_source": "default",
  "config_path": "/home/dev/.config/spicrawl/config.json",
  "key_valid": true
}

key_valid is true, false, or null when not checked (--offline, or no key). When the check fails, check_error holds the reason.

The status object is printed to stdout in every case, so you can read it even when the exit code is non-zero.

spicrawl config

Reads and writes the config file directly. Keys: api_key, base_url. Values here are the lowest-precedence source; spicrawl auth status shows what is actually in use.

spicrawl config set base_url http://localhost:8080
spicrawl config get base_url
spicrawl config unset base_url

An unknown key, or an empty value for set, exits 2.

Config file

Set SPICRAWL_CONFIG=/path/to/config.json to use another file. The file is JSON:

{
  "api_key": "spicrawl_live_...",
  "base_url": "https://api.spicrawl.com"
}

The CLI writes it atomically with mode 0600 (owner read and write only) and creates its directory with mode 0700. A missing file is treated as empty, not as an error.

CI and agents

Do not run spicrawl login in CI. Pass the key through the environment from your secret store; nothing is written to disk.

- name: Scrape the pricing page
  env:
    SPICRAWL_API_KEY: ${{ secrets.SPICRAWL_API_KEY }}
  run: |
    spicrawl auth status
    spicrawl scrape https://example.com/pricing --format markdown -o pricing.md

spicrawl auth status as the first step fails the job with exit code 3 if the secret is missing or revoked, before any real work runs.

To keep separate keys per project on one machine, point SPICRAWL_CONFIG at a per-project file, or pass --api-key for one call.