n8n

The n8n-nodes-spicrawl community node adds a Spicrawl node to n8n with two operations, Scrape URL and Batch Scrape. It returns a page as markdown, HTML, text, a PDF or extracted fields, and an AI Agent node can call it as a tool. Each call runs under your API key, with the same scopes, limits and credits as a direct API call.

Install

n8n-nodes-spicrawl

Example workflow

Import this with Workflow menu → Import from URL or clipboard, select your credential on the Spicrawl node, and run it. It reads a page as markdown and extracts two fields:

{
  "nodes": [
    {
      "name": "When clicking Test workflow",
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [0, 0],
      "parameters": {}
    },
    {
      "name": "Spicrawl",
      "type": "n8n-nodes-spicrawl.spicrawl",
      "typeVersion": 1,
      "position": [220, 0],
      "parameters": {
        "operation": "scrape",
        "url": "https://example.com/products/42",
        "responseFormat": "markdown",
        "rendering": "fetch",
        "simplify": true,
        "options": {
          "extract": "{\n  \"title\": \"h1\",\n  \"price\": \".price\"\n}",
          "maxCost": 3
        }
      }
    }
  ],
  "connections": {
    "When clicking Test workflow": {
      "main": [[{ "node": "Spicrawl", "type": "main", "index": 0 }]]
    }
  }
}

The output item carries the page in content and the fields in data:

{
  "url": "https://example.com/products/42",
  "status": 200,
  "credits": 1,
  "content": "# Walnut Desk\n...",
  "data": { "title": "Walnut Desk", "price": "$349.00" },
  "empty_fields": []
}

Scrape URL

Fetches one page. It runs once per input item, so five input items make five calls.

Add these under Options:

Check status in the output: a blocked or missing page is still a successful node run, and status is what the target site answered with.

Extract fields

Set Extract Fields (JSON) to a map of field name to CSS selector, or to a JSON Schema whose properties each carry a selector. The result is returned under data, and empty_fields lists every selector that matched nothing, which is the first sign a site changed its markup. Extraction costs no extra credits. For script-built pages, set Rendering to JavaScript and add Wait For Selector.

{
  "title": "h1",
  "price": ".price",
  "images": { "selector": ".gallery img", "kind": "list", "output": "attr", "attribute": "src" },
  "next_page": "a.next@href"
}

a.next@href reads an attribute; "kind": "list" returns every match. See Extract structured data for the schema form and type coercion.

Screenshots and PDFs

Binary output is attached to the item, not placed in json:

Pass the property to a Write Binary File, Google Drive or Send Email node. Both use full-browser rendering, which costs more (8 credits); see Credits and Screenshots and PDF. A screenshot or PDF is always binary, whatever Simplify is set to.

Batch Scrape

Submits a list of URLs as one job, waits for it to finish, then returns one item per URL. It runs once for all input items, not once per item. Requires the batch scope.

Options: Block Resources, Concurrency (1 to 50, default 10), Credit Budget, Custom Headers, Failure Threshold, Job Name, Main Content Only, Max Attempts, Max Cost Per Page (Credits), Max Wait (Minutes, default 30), Poll Interval (Seconds), Proxy URL, Wait After Load (Ms) and Wait For Selector.

Each output item has seq, url, status (succeeded, failed, cancelled or skipped), http_status, content, credits and, for a failed URL, error.code and error.retryable. Failed URLs are returned too, so filter on status with an IF or Filter node. If the job outlives Max Wait, the node stops with an error naming the job ID; the job keeps running, finished pages are still charged and results stay readable for 72 hours.

Use Spicrawl as an AI Agent tool

The node is marked usable as a tool, so an AI Agent node can call both operations and fill in the URL and options itself.

N8N_COMMUNITY_PACKAGES_ALLOW_TOOL_USAGE=true

Troubleshooting

Errors appear in n8n with their stable code, the API's hint and the request ID. Use Retry On Fail only for errors marked retryable: scraping is not idempotent, so a repeated successful call is billed again, even when the cache serves it. The full list is in Errors.

Links

• n8n-nodes-spicrawl on GitHub (source, issues, changelog)
• n8n-nodes-spicrawl on npm
• n8n: install community nodes
• Credits, Batch jobs, Extract structured data