Chunking: how to split your documents properly for a RAG in n8n

When a RAG chatbot gives an off-target answer, the reflex is to blame the model or the prompt. In practice, the cause very often sits one step earlier: at the moment the documents were split into pieces before being vectorized. A chunk that's too large drowns the useful information in filler; a chunk that's too small deprives the model of the context it needs to understand what it's reading. This chunking step is the least visible part of a RAG pipeline — and one of those that most determines answer quality. Here's how to get it right in n8n, as a companion to our complete guide to RAG with n8n and Supabase.

Why splitting determines answer quality

A RAG pipeline rests on a simple principle, formalized in the foundational paper by Lewis et al. presented at NeurIPS 2020 ("Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" — Google Scholar): rather than relying solely on the model's parametric memory, you retrieve relevant passages from a document base and inject them into the context at generation time. The whole chain therefore depends on the quality of those passages — in other words, of the chunks.

Two degradation mechanisms, symmetrical:

• Chunk too large: the embedding of a 3,000-character fragment covering four different topics is a blurry average — vector search retrieves it poorly, and when it does retrieve it, it injects a lot of off-topic text into the prompt around the one useful sentence. The signal is diluted.
• Chunk too small: an isolated sentence like "this deadline is extended to 30 days" is perfectly retrievable but unusable without knowing which deadline, which contract, and under which conditions. The context is lost.

The intuition of "give the model maximum context, it will sort things out" doesn't hold either. The study by Liu et al. published in TACL ("Lost in the Middle: How Language Models Use Long Contexts" — Google Scholar) shows that LLM performance degrades markedly when the relevant information sits in the middle of a long context, following a U-shaped curve: what's at the beginning and end is used well, what's buried in the middle much less so. Retrieving a few relevant, concise chunks beats flooding the prompt with massive context.

The text splitters available in n8n

In n8n, splitting is configured on a Text Splitter sub-node attached to the Data Loader that feeds the vector store — the full mechanics are described in our PDF ingestion workflow into Supabase pgvector. Three splitters are available:

Character Text Splitter

The simplest: it cuts text on a single separator (the double line break by default) while targeting a given size. Predictable and fast, but rigid — if your document doesn't contain the separator in the right place, chunks can be very uneven. Reserve it for texts whose format you control.

Recursive Character Text Splitter

The recommended default. It tries a list of separators in order — paragraphs first, then single line breaks, then sentences, then words — and only drops to the next level if the fragment still exceeds the target size. The result: chunks that respect the text's natural boundaries as much as possible, with no sentence cut mid-flow when it can be avoided.

Token Splitter

It measures size in tokens rather than characters. This is useful when you must guarantee that each chunk fits within a precise budget — the embedding model's input limit, or a context budget you want to control down to the token. Counting in tokens is more faithful to what the model actually "sees" than counting characters, especially for text mixing natural language and technical terms.

Chunk size and chunk overlap: the usual orders of magnitude

Two parameters to set on every splitter:

• Chunk size: typical values sit around 500 to 1,000 characters (or 256 to 512 tokens) for precise question-answering use cases, and go higher for summarization tasks where the model needs longer passages. The denser your documents (contracts, technical documentation), the more short, focused chunks pay off.
• Chunk overlap: the shared text between the end of one chunk and the beginning of the next, typically 10 to 20% of the chunk size. Without overlap, an idea straddling the boundary between two chunks exists in full in neither of them — and becomes unfindable. Excessive overlap, conversely, bloats the store and surfaces near-duplicates.

These values are starting points, not truths: the right setting is tested (see the last section). The vector store itself — for instance Supabase pgvector, whose setup is detailed in our n8n + Supabase guide — is indifferent to chunk size; it's retrieval and generation that bear the consequences.

Structured documents: headings, tables, FAQs

A generic splitter treats text as a uniform stream. Yet many real documents have structure that carries meaning:

• PDFs with a heading hierarchy: a chunk that starts mid-section, without its heading, loses a precious piece of context. A robust approach is to split by section first (using the extracted headings), then apply the splitter within each section — and carry the section title into the chunk's metadata.
• Tables: split line by line by a naive splitter, they become unreadable (orphan cells without headers). Better to extract them separately and convert them to explicit text — "row: product X, price column: €42" — before vectorizing. Our article on extracting data from PDF invoices with AI shows how to turn this kind of semi-structured content into usable data, with a Structured Output Parser to make the output reliable.
• FAQs and glossaries: the natural unit is the question-answer pair (or term-definition), never an arbitrary size. A well-split FAQ is in fact the content that performs best in RAG: every chunk is self-sufficient by construction.

The general rule: the more structure a document has, the less automatic the splitting should be — ten minutes of preparation upstream pay for themselves many times over downstream.

Adding metadata to chunks for citations

A chunk isn't just text: n8n's Data Loader lets you attach metadata that is stored alongside the embedding in the vector store. The three fields that change everything:

• source: the name or identifier of the original document;
• page: the page number for a PDF;
• section: the title of the section the chunk comes from.

This metadata serves two purposes. First, filtering searches (only looking within a given document or document type). Second — and above all — citing sources: when a chunk is retrieved, its metadata comes with it, and the model can phrase "according to section 4.2 of the master agreement, page 12…". That's exactly the mechanism behind our RAG chatbot with citations: a sourced answer is verifiable, which radically changes the trust a team places in the tool.

Evaluating your chunking empirically

Chunking is not tuned by intuition. The method that works:

1. Build a test question set — 20 to 30 realistic questions for which you know both the answer and the document passage that contains it.
2. Inspect the retrieved chunks, not just the final answer: for each question, did the vector search bring back the right passage in the top 3-5? If the right chunk doesn't surface, no prompt will save the answer. An endpoint like our RAG question-answering API makes this test easy to script.
3. Vary one parameter at a time: chunk size, then overlap, then splitting strategy. Re-index, replay the same question set, compare.
4. Tool up the comparison with n8n's evaluation features, described in our article on evaluations for testing AI workflows: a replayable test set turns a subjective tweak into a measured decision.

This protocol fits in an afternoon and avoids the classic trap: changing three parameters at once, seeing an improvement on two questions, and never knowing what actually helped.

Key takeaways

• Chunking determines RAG quality more than the choice of generation model: diluted signal if too large, lost context if too small.
• The Recursive Character Text Splitter is the right default; the Token Splitter when the token budget must be guaranteed.
• Overlap of 10 to 20%, short chunks for precision, longer for synthesis — then validation on a test question set.
• Structured documents (headings, tables, FAQs) deserve splitting that respects their structure, not an arbitrary size.
• Metadata (source, page, section) turns an opaque chatbot into an assistant that cites its sources.

The RAG Assistant Pack (€119) ships this entire chain ready to use — ingestion with splitting and metadata, Supabase pgvector vector store, chatbot with citations and a question-answering API — leaving you only to tune chunk size and overlap to your documents with the method above.