Agentic RAG with n8n: when the AI agent decides how and where to search

Classic RAG in n8n follows a rigid pipeline: the user's question goes straight into a vector search, the closest passages get stuffed into the prompt, the model answers. One search, always executed, never rewritten. That pattern breaks down as soon as a question requires cross-referencing several documents, the user's phrasing doesn't match the corpus vocabulary, or the answer lives in one knowledge base rather than another. Agentic RAG flips the logic: you give an AI Agent node one or more Vector Store Tools, and the model decides whether to search, how many times, and with which queries. This guide shows how to set it up in n8n, when it justifies the extra cost, and how to avoid the trap of infinite search loops.

Pipeline RAG vs agentic RAG: what actually changes

Pipeline RAG descends directly from the architecture described by Patrick Lewis and co-authors in Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (NeurIPS 2020): one retrieval, one context, one generation. In n8n, that is the retrieve → stuff → answer pattern covered in our RAG with Supabase guide: the search runs unconditionally, with the raw question.

Agentic RAG builds on the reason-act loop popularized by Shunyu Yao and co-authors in ReAct: Synergizing Reasoning and Acting in Language Models (ICLR 2023): the model alternates between reasoning and tool calls, observes the results, and decides what to do next. Concretely:

• Optional retrieval: if the message is small talk ("thanks, that's clear"), the agent replies without hitting the store.
• Rewritten queries: the agent turns "it crashes when I export" into "PDF export timeout error", closer to your documentation's vocabulary.
• Multiple searches: for "what's the warranty difference between product A and product B?", the agent runs two targeted searches instead of one averaged query that retrieves neither well.
• Routing across indexes: with several tools, the agent picks the relevant base instead of searching everything everywhere.

Akari Asai and co-authors formalized this on-demand retrieval in Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (ICLR 2024): a model that decides for itself when to retrieve outperforms always-on RAG on question answering and fact verification tasks.

Configuring the AI Agent with a Vector Store Tool

The setup rests on three building blocks: an AI Agent node (see our complete AI Agent node guide), an already-populated vector store, and the tool connecting them.

1. Prepare your vector store: Supabase/pgvector, Pinecone or Qdrant — our Qdrant vs pgvector comparison covers that choice. Ingestion (chunking, embeddings, insertion) is identical to classic RAG.
2. Add an AI Agent node with its Chat Model (plus memory if the use case is conversational).
3. Attach the vector store as a tool on the agent's Tool connector. Two options depending on your n8n version: the Vector Store Question Answer Tool sub-node (which links a vector store and a model that summarizes the retrieved passages), or the vector store node itself used in "Retrieve Documents (As Tool for AI Agent)" mode, which returns raw passages to the agent.
4. Wire up the same embeddings model used at ingestion time — a mismatch here silently breaks retrieval.
5. Set the result limit (top K): 4 to 6 passages per search is usually enough, since the agent can run another search if something is missing.

The tool description: the parameter that matters most

The agent never "sees" your document base: it only sees the tool's name and description. That is the sole basis on which it decides to use it, exactly as with custom tools for the AI Agent. A vague description ("searches documents") produces an agent that queries the base randomly or not at all. A good description states the content, the scope, and the expected query format:

Name: support_docs
Description: Searches the Flowkit product's technical documentation:
installation guides, error messages, troubleshooting procedures,
release notes. Use this tool for any technical question about how
the product works. Phrase the query with precise technical keywords,
not the user's raw question. Contains NO pricing and NO contractual
terms.

The "does not contain" part is as useful as the "contains" part: it stops the agent from grinding away at the wrong index.

Multi-index: one tool per document base

This is where agentic RAG earns its keep. Instead of a single index where product sheets, terms of service and support tickets pollute each other at search time, create one vector store (or collection) per domain and attach one tool per domain: products_db (specifications, compatibility), legal_db (terms of service, contracts, GDPR notices), support_docs (technical documentation, troubleshooting).

Faced with "can I cancel if module X isn't compatible with my setup?", the agent queries products_db, then legal_db, and combines both answers — where a single-index pipeline would have retrieved a mediocre blend of both topics. This description-based routing is easier to maintain than metadata filtering on a single index when the domains are genuinely disjoint, and the two approaches combine well.

A complete workflow example

An internal support assistant reachable from Slack or a chat widget:

1. Chat Trigger (or Webhook) receives the question.
2. AI Agent with a system prompt along the lines of: "You are [company]'s support assistant. Answer only from documents retrieved through your tools. If two searches are not enough to find the information, say you don't know and suggest contacting support. Cite the source of every claim."
3. Three Vector Store Tools (products, legal, support), each with its scoped description.
4. Memory (Window Buffer Memory or equivalent) for conversational follow-ups.
5. On the output side, a formatting node, then delivery to Slack or the webhook response.

The execution data shows every cycle (reasoning, tool call with the rewritten query, observation) — a readable trace that is a genuine debugging advantage over a silent pipeline.

The trap: infinite search loops

A poorly framed agent can keep firing variants of the same search when the corpus simply does not contain the answer — with every iteration being a billed LLM call. Three guardrails:

• Max Iterations: in the AI Agent node's options, cap the number of cycles (3 to 5 is plenty for most assistants). Beyond that, the execution stops instead of burning tokens.
• An explicit give-up instruction in the system prompt: "after two searches with no relevant result, answer that the information is not available". Without that exit door, the model would rather keep searching than admit ignorance.
• Cost tracking: log the number of calls and tokens per execution to spot the questions that send the agent off the rails.

When it's worth the cost — and when to stick with pipeline RAG

Agentic RAG is not a replacement; it is a step up that you pay for. Go agentic when questions are multi-hop (cross-referencing several documents or bases), when the corpus is heterogeneous (tool-based routing beats a single index), or when user phrasing is far from the corpus (query rewriting recovers what a raw search misses).

Stay with pipeline RAG when:

• questions are simple and the corpus homogeneous: a product FAQ does not need an agent;
• latency matters: every iteration adds an LLM round trip; a pipeline answers in one call;
• determinism matters: a pipeline always does the same thing, which simplifies testing and RAG quality evaluation; an agent may take different paths for the same question;
• budget is tight: two to four LLM calls per question instead of one.

And before going agentic because results are mediocre, check that the real problem is not retrieval quality itself: hybrid search and reranking often fix things at a far lower cost.

Key takeaways

Agentic RAG replaces the pipeline's fixed retrieval step with a model decision: search or not, rewrite, repeat, route across several indexes. In n8n, everything fits in the trio of AI Agent + Vector Store Tool(s) + carefully written tool descriptions — the description being the real quality lever. Cap the iterations, add a give-up instruction, and reserve this architecture for multi-hop questions and heterogeneous corpora: for everything else, the classic pipeline remains faster, cheaper and more predictable. To start from an already-assembled foundation, the RAG Assistant Pack (€119) provides a base that extends directly into agentic multi-index setups.