Semantic caching in n8n: cutting AI cost and latency with Redis

A support chatbot answering 500 questions a day usually rests on a simple reality: most of those questions look alike. "How do I reset my password?", "I forgot my password, what do I do?" and "Lost password, how do I fix this?" are three phrasings of the same intent — yet without anything special in place, that's three billed calls to an LLM, with three times the latency. Semantic caching solves exactly this: instead of comparing questions word for word, it compares their meaning, via embeddings, and returns an already-generated answer as soon as a close-enough question has already been asked.

Classic cache vs semantic cache

A classic cache (key-value) only works on an exact match: the same string produces the same result. That's perfect for a deterministic API response, but useless against natural language — two users almost never phrase a question in exactly the same words.

Semantic caching flips the logic:

1. The incoming question is turned into an embedding (a vector capturing its meaning) via a model like OpenAI's text-embedding-3-small.
2. That vector is compared by cosine similarity against vectors already in the cache.
3. If the best score clears a threshold (often 0.95+), the associated answer is returned directly — zero calls to the LLM.
4. Otherwise, the LLM is called as usual, and the new question/answer pair is added to the cache for next time.

A 2024 study by Regmi and Phakami Pun, GPT Semantic Cache: Reducing LLM Costs and Latency via Semantic Embedding Caching, formalizes precisely this architecture with Redis as an in-memory embedding store, and reports a meaningful cut in redundant API calls across real query workloads. On the same ground, the foundational paper GPTCache: An Open-Source Semantic Cache for LLM Applications (Bang, 2023) measures cache hit rates above 60% across varied question sets, with answers returned many times faster than a fresh call to the model. Both converge on the same finding: a large share of production LLM traffic is redundant in intent, even when it isn't redundant word for word.

Why Redis rather than Supabase/pgvector here

This blog's RAG guide with Supabase and pgvector uses Postgres to store thousands of document chunks, queried occasionally. Semantic caching has a different usage profile: very frequent reads, a demand for low latency (you want to save time, not lose more of it in the lookup), and data that can expire. Redis, in memory, with its vector search module (RediSearch, bundled in Redis Stack), is built for exactly this profile: millisecond-range latency, and crucially a native per-key TTL — data self-expires with no cleanup job to write. If your n8n infrastructure already runs in queue mode with Redis, the instance already exists: you just add this use case to it, with no new service to operate.

Building the workflow in n8n

The pipeline sits between receiving the question (chat webhook, Slack message, API request) and calling the LLM:

1. Webhook / Trigger — receives the user's question.
2. Embeddings node (OpenAI or n8n's native node) — turns the question into a vector.
3. Redis node (or HTTP Request to the Redis API) — runs an FT.SEARCH command against the vector index, retrieves the best candidate and its similarity score.
4. If node — tests the score against the chosen threshold:
   • Above threshold → "cache hit" branch: the stored answer is returned directly, no model call.
   • Below threshold → "cache miss" branch: the AI node (Agent, LLM chain, or a direct call to Claude/GPT as covered in our guide to connecting Claude or GPT to n8n) generates the answer as usual.
5. Redis node (write), on the miss branch only — stores the new (embedding, answer) pair with a TTL matched to how volatile the content is (a few hours for product support, a few minutes for fast-changing data).

Concretely, the search command looks like:

FT.SEARCH idx:cache_qa "*=>[KNN 1 @embedding $vec AS score]"
  PARAMS 2 vec <embedding_binaire>
  SORTBY score
  DIALECT 2

and writing a new entry:

HSET cache:qa:<uuid> question "..." reponse "..." embedding <bytes>
EXPIRE cache:qa:<uuid> 21600

The score returned by FT.SEARCH is a distance (smaller = closer) rather than a direct similarity, depending on the metric chosen at index time — check whether your index uses COSINE, L2, or IP, and adjust the comparison direction in the If node accordingly.

Sizing the similarity threshold

This is the setting that decides whether the cache saves money or breaks answer quality:

• Too permissive (low threshold, e.g. 0.85): surface-similar but actually different questions get the same answer — risky whenever nuance matters (pricing, eligibility, a safety procedure).
• Too strict (very high threshold, e.g. 0.99): almost no rephrasing ever matches, the cache stays empty and adds nothing.

A reasonable starting point is 0.95-0.97 for a support or documentation use case, validated against a real sample of past questions before going live — the same testing discipline recommended in our article on n8n evaluations for AI workflows. For high-stakes cases (legal, medical, financial advice), simply disable the cache on those intents rather than risk a mismatched answer.

What this actually changes

• Cost: every cache hit fully avoids the LLM call — on a chatbot with a high volume of recurring questions (product FAQ, first-line support), the avoidable share of traffic often exceeds 40-60% once the cache has warmed up, in the same range measured by the studies cited above.
• Latency: an in-memory Redis read answers in a few milliseconds, versus several hundred milliseconds to a few seconds for a round trip to an LLM — immediately noticeable to the user.
• Resilience: during a rate limit or 429 error on the AI provider's side, an already-warm cache keeps answering frequent questions even while new calls are temporarily failing.

Common pitfalls

• Caching personalized answers: if the answer depends on user context (their account, their history), caching the question alone produces off-target answers for other users — fold the variable elements into the key, or exclude those intents from the cache.
• Too-long TTL on content that changes: a price or availability figure cached without expiry becomes a source of silent errors; match the TTL to how often the information actually changes.
• Never purging: without a size cap on the index (MAXMEMORY plus an allkeys-lru eviction policy on the Redis instance), the cache grows without bound; an eviction policy removes the need to think about it.
• Forgetting to log hits/misses: without a hit-rate metric, there's no way to know whether the cache is worth maintaining, or to tune the threshold with any real basis.

Going further

This caching mechanism drops in naturally upstream of the AI workflows in the RAG Assistant Pack (€119), whose citation-backed chatbot gains on both cost and responsiveness as soon as questions start repeating — common on a stable documentation base (procedures, FAQ, internal policy). If your n8n instance isn't running with Redis yet, our guide to queue mode with Redis covers the initial setup — reusable afterward both for this cache and for scaling your AI workflows.