n8n node
the crawlbrulee node for n8n is a verified community node,
n8n-nodes-crawlbrulee. it is listed on
n8n’s integrations page: crawlbrulee on n8n . open
source under MIT: source on GitHub .
install
- n8n Cloud: open the node panel, search for “crawlbrulee” and pick the node. n8n installs it for you.
- self-hosted: go to Settings → Community Nodes → Install, enter
n8n-nodes-crawlbruleeand confirm.
credential
create an api key in the dashboard (see get a key), then add a crawlbrulee API credential in n8n:
| field | what to put in it |
|---|---|
| API Key | your key, starting with cwbl_. required. |
| Webhook Signing Secret | optional. starts with whsec_. only the trigger uses it. |
saving the credential checks the key, at no credit cost.
operations
pick a resource, then an operation:
| resource | operation | what it does |
|---|---|---|
| Scrape | Scrape URL | fetches one url and returns the content you asked for |
| Scrape | Scrape URL (Async) | starts a background job and returns its job id |
| Scrape | Get Scrape Status | tells you whether a job is pending, running, done or failed |
| Scrape | Get Scrape Result | returns the content of a finished job |
| Map | Map Website | discovers a site’s urls from its sitemap and its homepage links |
| Account | Get Credit Usage | credits used and left in this billing cycle, plus your concurrency limit |
| Account | Get Account Info | the organization and token behind the api key |
every field has a hint in n8n. the item you get back is the api’s json, unchanged. see scrape and map for what each option does.
use it as an AI Agent tool
the node works as a tool for an AI Agent in n8n, so an agent can scrape a page on its own.
on self-hosted n8n, an agent can use a community node as a tool only when the env var
N8N_COMMUNITY_PACKAGES_ALLOW_TOOL_USAGE is true.
the page’s own status
Scrape URL and Get Scrape Result put the status the site answered with in
page_status_code. a page the site served is a normal item, whatever its status: a 404 page
comes back with its content, and “On Error” never sees it.
when the status matters to your workflow, check it before you use the content. for example, add
an If node on {{ $json.page_status_code }} is less than 400.
screenshots as files
screenshot links expire 24 hours after the scrape. turn on Download Screenshot on Scrape URL or Get Scrape Result, and the node attaches the image to the item as binary data. do this when you need the image later in the workflow.
trigger
the crawlbrulee Trigger starts a workflow when a background scrape finishes, so you don’t have to poll for it:
- add a crawlbrulee Trigger, give it the same credential and copy the webhook url it shows.
- paste that url into the Webhook URL field of a Scrape URL (Async) operation.
- when the job finishes, we send one
scrape.completeevent to that url and the trigger fires.
turn on Fetch Result and the trigger also fetches the page content of a finished job.
to have the trigger check the signature on every delivery, put your webhook signing secret into the credential’s Webhook Signing Secret field. see webhook verification for where to find the secret.