Skip to Content

n8n node

the crawlbrulee node for n8n is a verified community node, n8n-nodes-crawlbrulee. it is listed on n8n’s integrations page: crawlbrulee on n8n . open source under MIT: source on GitHub .

install

  • n8n Cloud: open the node panel, search for “crawlbrulee” and pick the node. n8n installs it for you.
  • self-hosted: go to Settings → Community Nodes → Install, enter n8n-nodes-crawlbrulee and confirm.

credential

create an api key in the dashboard (see get a key), then add a crawlbrulee API credential in n8n:

fieldwhat to put in it
API Keyyour key, starting with cwbl_. required.
Webhook Signing Secretoptional. starts with whsec_. only the trigger uses it.

saving the credential checks the key, at no credit cost.

operations

pick a resource, then an operation:

resourceoperationwhat it does
ScrapeScrape URLfetches one url and returns the content you asked for
ScrapeScrape URL (Async)starts a background job and returns its job id
ScrapeGet Scrape Statustells you whether a job is pending, running, done or failed
ScrapeGet Scrape Resultreturns the content of a finished job
MapMap Websitediscovers a site’s urls from its sitemap and its homepage links
AccountGet Credit Usagecredits used and left in this billing cycle, plus your concurrency limit
AccountGet Account Infothe organization and token behind the api key

every field has a hint in n8n. the item you get back is the api’s json, unchanged. see scrape and map for what each option does.

use it as an AI Agent tool

the node works as a tool for an AI Agent in n8n, so an agent can scrape a page on its own.

on self-hosted n8n, an agent can use a community node as a tool only when the env var N8N_COMMUNITY_PACKAGES_ALLOW_TOOL_USAGE is true.

the page’s own status

Scrape URL and Get Scrape Result put the status the site answered with in page_status_code. a page the site served is a normal item, whatever its status: a 404 page comes back with its content, and “On Error” never sees it.

when the status matters to your workflow, check it before you use the content. for example, add an If node on {{ $json.page_status_code }} is less than 400.

screenshots as files

screenshot links expire 24 hours after the scrape. turn on Download Screenshot on Scrape URL or Get Scrape Result, and the node attaches the image to the item as binary data. do this when you need the image later in the workflow.

trigger

the crawlbrulee Trigger starts a workflow when a background scrape finishes, so you don’t have to poll for it:

  1. add a crawlbrulee Trigger, give it the same credential and copy the webhook url it shows.
  2. paste that url into the Webhook URL field of a Scrape URL (Async) operation.
  3. when the job finishes, we send one scrape.complete event to that url and the trigger fires.

turn on Fetch Result and the trigger also fetches the page content of a finished job.

to have the trigger check the signature on every delivery, put your webhook signing secret into the credential’s Webhook Signing Secret field. see webhook verification for where to find the secret.