Connecting Firecrawl to n8n: scrape the web into LLM-ready markdown for your RAG pipelines and AI agents

A RAG pipeline or an AI agent is only as good as the content it gets to read. The trouble is that the web isn't written for LLMs: a product page carries three times more menu, cookie banner, and script than useful text, and a good chunk of the content only appears after JavaScript runs — invisible to a plain HTTP Request node. Firecrawl is a scraping service built specifically for this problem: it renders the page, strips the noise, and returns clean markdown ready to feed a language model. Since 2026 it has shipped a native node in n8n. This guide covers installation, the available operations, and two concrete use cases: feeding a RAG pipeline and equipping an AI Agent with an autonomous web-search tool.

Why a dedicated scraper instead of HTTP Request + HTML Extract

Our guide to web scraping with n8n covers the generic approach: HTTP Request fetches raw HTML, then HTML Extract or an LLM pulls out the useful fields. That combination still works fine for a simple static page, but it runs into three common walls: JavaScript-rendered sites (the raw HTML is empty of content), anti-bot protections that block automated requests, and leftover noise — navigation, ads, footers — that pollutes the context sent to the LLM.

That last point isn't cosmetic: Christian Kohlschütter and co-authors show in Boilerplate Detection using Shallow Text Features (WSDM 2010) that navigation and template text degrades the accuracy of downstream processing whenever it isn't separated from the main content — a finding that carries over directly to an LLM answering from scraped context: every token spent on a menu or a banner is one less token for actual information, and one more source of confusion. Firecrawl handles this cleanup upfront: JavaScript rendering, reasonable anti-bot handling, and conversion to markdown that strips out most of the boilerplate — Firecrawl claims roughly 67% fewer tokens than sending the raw HTML straight to the model.

Installing the Firecrawl node in n8n

On n8n Cloud

Since the partnership announced in March 2026, the integration is native: add the Firecrawl node to your canvas, click Connect, and create (or link) your Firecrawl account without ever leaving the editor — no API key to copy and paste.

On a self-hosted instance

The official community node is called @mendable/n8n-nodes-firecrawl. An instance admin first needs to enable and install community nodes:

1. Settings > Community Nodes, then Install a community node.
2. Enter the package name (@mendable/n8n-nodes-firecrawl) and confirm.
3. Once installed, create a Firecrawl API credential with a key from the firecrawl.dev dashboard — treat it like any sensitive key, following our advice on securing API credentials in n8n.

Check your instance version: using the node as an AI Agent tool (covered below) requires n8n 1.79.0 or later.

The node's operations: scrape, crawl, map, search, extract

The node exposes several operations, to be combined as needed:

• Scrape: fetches a single URL and returns its content as markdown, HTML, structured data (via LLM-based extraction), or a screenshot. The baseline operation for one identified page.
• Map: lists every discoverable URL on a site without fetching its content — useful for spotting the relevant pages before scraping them one by one, rather than crawling blindly.
• Crawl, with its follow-up Crawl Status, Crawl Active, and Crawl Errors: walks a site (or a section of it) by following links and scrapes every page it finds. The job is asynchronous — you kick it off, then poll its status until completion.
• Batch Scrape, with Batch Scrape Status and Batch Scrape Errors: scrapes a known list of URLs in a single job, more efficient than looping individual Scrape calls when you already have the URL list.
• Search: runs a web search and can fetch the content of the results in a single operation.
• Extract: asks a model to pull structured data out of one or more pages according to a defined schema, rather than raw text you'd have to post-process yourself.
• Team Usage Metrics: reports the team's credit consumption, handy for keeping an eye on cost right from n8n instead of switching to the Firecrawl dashboard.

The node doesn't cover the entire Firecrawl API: the /agent endpoint, for instance, isn't exposed, and stays reserved for a direct HTTP Request call for anyone who needs it.

Use case 1: feeding a RAG pipeline with fresh web content

For a knowledge base that needs to reflect up-to-date public content (product documentation published on a third-party site, competitor pages tracked as part of competitive monitoring, articles from a partner blog), the typical pipeline looks like this:

1. A weekly Schedule Trigger or a manual run.
2. Firecrawl — Map on the target site to list URLs, optionally filtered by pattern (/docs/*, /blog/*).
3. Firecrawl — Batch Scrape on the selected URLs, with markdown output.
4. Chunking of the resulting markdown, following the same principles as for a PDF — see our guide to chunking for RAG in n8n — with the advantage that markdown is already structured by headings, which makes section-aware splitting easier.
5. Embeddings, then storage in your vector store, following the architecture described in our RAG with n8n and Supabase guide.

The RAG Assistant Pack (€119) already ships the ingestion, vectorization, and citation-chatbot pieces: just swap the PDF source for this Firecrawl pipeline upstream to ingest web content instead of internal documents.

Use case 2: Firecrawl as an AI Agent tool

Instead of a scheduled pipeline, Firecrawl can also become a tool the agent invokes on its own mid-conversation — the same principle as any custom tool wired into an AI Agent, covered in our AI Agent node guide. Concretely, attach the Firecrawl node in tool mode to the agent's Tool connector, with a precise description of when to use it ("search the web for recent information when the question is about a fact after your knowledge cutoff, or about a specific external site the user provides").

The agent then decides on its own when to search: a monitoring assistant can scrape a competitor's product page mentioned mid-conversation on demand, without a scheduled pipeline having had to anticipate it. It's a natural complement to agentic RAG, where the Vector Store Tool queries an already-built knowledge base while the Firecrawl Tool goes and fetches what isn't in it yet.

Credits and costs: what to watch

Firecrawl bills by usage, in credits. Scrape, Crawl, Map, and Monitor each cost 1 credit per page; heavier operations cost more (Interact: 2 credits per browser-minute; Stealth Mode, for heavily protected sites: 5 credits per page). A free tier of 1,000 credits per month is enough to test the node or run a small occasional pipeline; paid plans start around $16/month. One thing to watch: a page that returns an error still consumes a credit, and unused credits don't roll over month to month — a poorly scoped Crawl on a site with thousands of pages can burn through a plan fast if Map wasn't used upstream to filter down to the relevant URLs.

Legality and good practice

Firecrawl handles the technical mechanics of scraping, not the legal framework around it. The guidelines laid out by Vlad Krotov and co-authors in Tutorial: Legality and Ethics of Web Scraping (Communications of the AIS, 2020) still apply regardless of the tool: check the target site's robots.txt and terms of use, keep the Crawl scope to what's actually needed, and be especially careful whenever personal data shows up in the fetched content. Firecrawl makes the content easier to reach — it doesn't exempt you from any of these checks.

Summary

Firecrawl fills a real gap between raw HTTP Request calls and industrial-grade scraping: JavaScript rendering, common anti-bot handling, and above all clean markdown you can feed straight into a RAG pipeline or an agent, with no CSS selectors to maintain. On n8n Cloud, connecting takes one click; self-hosted, the @mendable/n8n-nodes-firecrawl community node installs in minutes. What's left is picking the right mode: scheduled Scrape/Crawl to keep a vector store continuously fed, or Firecrawl as an agent tool for on-demand web search — both pair well with the RAG Assistant Pack (€119) as a foundation.