Skip to Content

zero data retention

with zero_data_retention: true, we don’t store or log scraped page content, urls or extracted data. async results and screenshots are deleted 24 hours after the request, instead of being kept for up to 90 days.

the result is also not added to the shared cache, so it is never served to another customer. reading from the cache is still allowed: a request with zero data retention can be served a result that is already in the cache. to get a fresh result every time, also set cache.max_age: 0.

the same is written into section 6 of our terms  and annex 2 of our dpa .

what we still keep

  • usage and billing records: which organization made the request, when, and what it cost. these are how we bill you and show your usage.
  • the domain name in our logs, for up to 90 days, for abuse prevention.

how to use it

add zero_data_retention: true at the top level of the request body, next to url (not inside cache). it works on POST /api/scrape, POST /api/scrape/async and POST /api/map — see the scrape request body and the map request parameters. the default is false.

curl -X POST https://api.crawlbrulee.com/api/scrape \
  -H "Authorization: Bearer $CRAWLBRULEE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "extract": { "markdown": true },
    "zero_data_retention": true
  }'

availability

zero data retention comes with our enterprise plan. we can also turn it on for other plans if you need it. it is off until we turn it on for your organization: write to sales@crawlbrulee.com.

we can also turn it on for every request in your organization. then every scrape and map request runs with zero data retention, even one that sends false or leaves the field out. we suggest this if you need it on every request, so a request that forgets the field is still covered.

costs

zero data retention adds 1 credit to each result we bill that did not come from the cache. a cache hit adds 0, and so does a page we don’t bill.

for example, an http request on the advanced proxy costs 5 + 1 = 6 credits. the usage object shows it as response_meta.usage.zero_data_retention_credit_cost.

see credits & pricing for the full rules.

what to know

  • fetch results or screenshots within 24 hours. fetch an async result, and download a screenshot, within 24 hours. once they are deleted, GET /api/scrape/result/{job_id} returns 404 and the screenshot url no longer works.
  • it must be enabled first. sending zero_data_retention: true before it is enabled for your organization returns 403 zero_data_retention_not_enabled, and you are not billed. sending false, or leaving the field out, always works. see errors.
credits & pricingproxies & location