The Question and Answer Chain Node in n8n: Turnkey RAG

The Question and Answer Chain node (@n8n/n8n-nodes-langchain.chainRetrievalQa) is n8n's most direct RAG building block: a question comes in, a retriever pulls the relevant passages out of a vector store, a model writes the answer, done. No loop, no tools, no orchestration decision. That simplicity is its strength — one LLM call, reproducible behavior — and its limit, as soon as you ask it for a conversation or several chained searches. This guide covers how it actually works, its exact parameters, and the criteria that tip the balance toward it rather than an AI Agent. If you are new to n8n's AI toolkit, start with our introduction to n8n's AI nodes.

What the node actually does

The mechanics come down to three steps, always in the same order:

1. Retrieval: the question goes to the connected retriever, which queries the vector store and returns the N semantically closest passages.
2. Assembly: n8n concatenates those passages and injects them into the system prompt, at the location of the {context} variable.
3. Generation: the full prompt (context + question) goes to the chat model, which produces the answer. One single call.

The node's main parameter is Query, documented as "The question you want to ask". It is fed automatically by a connected Chat Trigger, or set manually ("Define below") with an expression such as {{ $json.question }}.

One key point: the search always happens. Even if the question is "hello", the retriever goes and fetches documents and the model receives off-topic context. The node has no way to decide that retrieval is unnecessary — which is exactly what separates it from an agent.

The sub-connections: Model and Retriever

The node exposes two side connectors, both required.

Model expects a chat model (OpenAI, Anthropic, Google Gemini, Ollama, Mistral...) that writes the answer from the context. Your choice affects the available context window: if you retrieve many passages, a short-context model will truncate. The node's common issues page points to the model node's Maximum Number of Tokens parameter when answers come out too short.

Retriever expects a retrieval sub-node, most often Vector Store Retriever, itself wired to a vector store (Supabase/pgvector, Qdrant, Pinecone, the in-memory Simple Vector Store...). The connection chain therefore looks like: Question and Answer Chain → Vector Store Retriever → Vector Store. Forget that branch and n8n raises the explicit error "A Retriever sub-node must be connected".

The Vector Store Retriever sub-node has a single parameter: Limit, "Enter the maximum number of results to return". That is your top-k, and the most structural setting in the whole build — more on it below. For the storage layer itself, see our RAG guide with Supabase and pgvector.

The System Prompt Template, the only real control lever

In the node's Options, the System Prompt Template field replaces the default prompt. n8n's default is deliberately minimal:

Use the following pieces of context to answer the users question.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
----
{context}

The field description states the constraint: the template "should include the variable {context}" — and, for text completion models, {question} as well. Without {context}, the retrieved documents never reach the model and the answer becomes purely generative. A production template looks more like this:

You answer sales team questions from the internal documentation.
Rules:
- Use ONLY the excerpts below. Do not invent anything.
- If the excerpts do not allow an answer, reply exactly: "I could not find this information in the documentation."
- Cite the title of the document used, in square brackets, at the end of the sentence.
- Answer in English, in three sentences maximum.

Excerpts:
{context}

That abstention instruction is not a detail. The benchmark "Benchmarking Large Language Models in Retrieval-Augmented Generation" by Jiawei Chen, Hongyu Lin, Xianpei Han and Le Sun (AAAI 2024) evaluates LLMs in RAG settings along four axes — noise robustness, negative rejection (knowing when to say you don't know), multi-document integration, and resistance to misinformation — and concludes that models tolerate some noise but fail notably at explicit refusal. Without an abstention instruction spelled out, your chain will answer anyway.

Q&A Chain vs AI Agent vs Basic LLM Chain

Criterion	Basic LLM Chain	Question and Answer Chain	AI Agent + Vector Store Tool
Document retrieval	None	Always, once	Optional, decided by the model
LLM calls per execution	1	1	1 to N
Sub-connections	Model (+ output parser)	Model + Retriever	Model + Tools + Memory (optional)
Conversation memory	No	No	Yes
Multiple / multi-hop searches	No	No	Yes
Cost	Fixed	Fixed	Variable
Reproducibility	High	High	Lower
Typical use case	Summarize, classify, rewrite	FAQ over a fixed corpus	Multi-source, multi-turn assistant

The reading is simple: the Q&A Chain is a Basic LLM Chain with a mandatory retrieval step welded onto it. It inherits the same strengths — known cost, single-call latency, trivial debugging, detailed in our Basic LLM Chain vs AI Agent comparison — and the same limits: no decision made at runtime.

When to pick it over an AI Agent

Three signals argue for the chain:

• Every question targets the same corpus. Product FAQ, internal procedures, technical documentation: there is nothing to decide, you always have to search. Handing that choice to an agent means paying for a reasoning turn on a decision you already know.
• One search is enough. "What is the return window?" is answered by a single passage, without cross-referencing two documents or rewording after a failed attempt.
• Budget and latency matter. One LLM call per question: it is the only RAG setup in n8n whose monthly cost you can estimate before deploying it.

Conversely, move to the AI Agent as soon as one of these appears: the question may require no search at all; several indexes or sources must be queried (CRM, API, documentation); a failed search must trigger a rewrite; or the user talks across several turns. The multi-index case is covered in our agentic RAG guide.

The pitfalls worth knowing

Miscalibrated top-k. This is the setting most often left at its default, and the one with the heaviest consequences. The study "The Power of Noise: Redefining Retrieval for RAG Systems" by Florin Cuconasu and co-authors (SIGIR 2024) shows that the composition of the context matters as much as its size: the position of relevant documents and the nature of the irrelevant passages measurably change the answer, and adding documents only loosely related to the question degrades performance. So do not push Limit to 20 "just in case": test 3, 5 and 8 against a set of real questions, using the method described in our article on evaluating RAG quality.

The out-of-context answer. If the retriever returns nothing relevant, the model receives useless context — and answers anyway, often confidently. Two cumulative fixes: the abstention instruction, and work upstream. Our guide to document chunking and the one on metadata filtering address the two most frequent sources of noise; a reranker lets you retrieve broadly then keep only the best.

No citations by default. The native prompt asks for no sources: the user gets an unverifiable statement, which is a deal-breaker for legal, HR or support use. The only fix is to add the instruction to the System Prompt Template and make sure the useful metadata (title, URL, page) actually appears in the text of the indexed chunks, since that text, and only that text, lands in {context}.

The follow-up question. "And for the other client?" is sent to the retriever as-is, with no idea who the previous client was: the search fails and the answer is incoherent. This is not a bug, it is the absence of a memory input. The problem is well identified in research: the QReCC dataset introduced by Raviteja Anantha, Svitlana Vakulenko and co-authors in "Open-Domain Question Answering Goes Conversational via Question Rewriting" (NAACL 2021) rests on exactly this observation — a conversational question must first be rewritten into a self-contained one for retrieval to work. In n8n: either a Basic LLM Chain upstream rewrites the question from the history, or you switch to an AI Agent with conversation memory.

The empty prompt. The "No prompt specified" error happens when the expression feeding Query produces nothing, or when a connected Chat Trigger sends a null value. Add an IF node upstream rather than letting the node fail in production.

Key takeaways

The Question and Answer Chain does one thing and does it well: one question, one search, one answer, for one LLM call. Three settings determine its quality — the Vector Store Retriever's Limit, the System Prompt Template with its abstention instruction and citation requirement, and the quality of the indexed chunks. Switch to the AI Agent only when a genuine need for runtime decisions shows up: several sources, several rounds of search, or an ongoing conversation.

Going further

To avoid starting from a blank canvas, the RAG Assistant Pack (€119) ships the pipeline already assembled — document ingestion, embeddings, pgvector storage and a chat interface with citations — and this node slots straight into it. If your corpus is mostly emails and attachments, the Inbox AI Pack (€79) covers the collection step.