Ask a large language model about your company’s refund policy and it will answer fluently, confidently, and quite possibly wrong. Not because the model is bad, but because your refund policy was never in its training data. A model’s knowledge is frozen at its training cutoff and averaged over the public internet. Your internal wiki, your contracts, your ticket history, yesterday’s incident report — none of it is in there, and no amount of clever prompting can conjure facts the model never saw.
Retrieval-augmented generation — RAG — is the standard fix. Instead of hoping the model knows the answer, you search your own documents at query time, take the most relevant passages, and paste them into the prompt so the model can answer from evidence instead of memory. The term comes from a 2020 paper by Lewis et al., but the idea is older and simpler than the acronym suggests: it is a search engine bolted onto a text generator.
The pipeline is easy to demo and genuinely hard to run well. This post walks through how it works end to end — embeddings, vector search, chunking, hybrid retrieval, reranking — and then spends real time on the parts the demos skip: how to evaluate it, how it fails, and when you should not build it at all.
The problem RAG solves
Two properties of LLMs create the gap. First, knowledge is frozen: whatever happened after the training cutoff does not exist for the model. Second, knowledge is generic: the model learned from public text, so it knows what refund policies typically look like, not what yours says. When you ask about specifics it was never trained on, it does what autocomplete does — produces the most plausible-sounding answer, which is exactly the failure you cannot tolerate in production.
RAG attacks both problems at once. Because retrieval happens at query time, the model sees whatever is in your index right now — so freshness becomes an indexing problem, not a retraining problem. And because the retrieved text is yours, the answer is grounded in your data instead of the internet’s average. The model supplies reading comprehension and fluent synthesis; your corpus supplies the facts.
The pipeline, end to end
Every RAG system has two phases. Ingestion runs offline: you collect documents, split them into chunks, run each chunk through an embedding model, and store the resulting vectors in an index — a dedicated vector database, or an extension like pgvector inside the Postgres you already run. Query time runs on every request: embed the user’s question with the same model, find the stored vectors nearest to it, pull the corresponding chunks, and assemble a prompt that contains both the question and the evidence.
System: Answer only from the context below. If the
context does not contain the answer, say so.
Context:
[1] "Refunds are issued to the original payment
method within 5-7 business days of approval."
[2] "Requests made more than 90 days after purchase
require manager review."
User: How long does a refund take?That is the whole trick. The model never queries a database and never “looks anything up” itself — retrieval happens outside the model, and generation is an ordinary completion over a prompt that happens to contain your documents. Everything that makes RAG good or bad lives in the quality of what you put in that context block.
Embeddings: the geometry of meaning
The step that makes semantic search possible is the embedding model. It maps a piece of text to a list of numbers — a vector, typically hundreds to a few thousand dimensions — such that texts with similar meaning land near each other in that space. “How do I get my money back” and “refund eligibility rules” share almost no words, but a good embedding model places them close together, because it was trained on enormous numbers of examples of which texts belong together. Distance in this space (usually cosine similarity) becomes a stand-in for relevance: nearest neighbors to the query vector are the passages most likely to be about the same thing. At scale you rarely compare against every vector; approximate nearest-neighbor indexes such as HNSW trade a sliver of recall for orders-of-magnitude faster lookups.
Chunking: the unglamorous decision that matters most
You cannot embed a 200-page PDF as one vector — the embedding would average away everything specific. So you split documents into chunks, and how you split turns out to matter more than which vector database you pick.
- Fixed-size chunks with overlap — every N tokens, with some shared tail between neighbors — are the baseline. Simple and fast, but they cheerfully cut sentences, tables, and code blocks in half.
- Structure-aware chunking splits on headings, paragraphs, and code fences so each chunk is a coherent unit. More work per document format, meaningfully better retrieval.
- Small chunks retrieve precisely but strip context. A chunk that reads “Yes, but only within 90 days” is useless when it arrives without the question it was answering.
- Large chunks keep context but dilute the embedding — a chunk about five topics is near none of them — and they burn prompt budget at query time.
Two refinements are worth knowing. Small-to-big retrieval embeds small chunks for precision but returns the surrounding section to the model for context. And contextual retrieval prepends a short generated summary of the document to each chunk before embedding, so the chunk carries its own context — Anthropic published a detailed writeup of the technique. Either beats tuning chunk size by trial and error.
Hybrid search and reranking
Embeddings are good at paraphrase and bad at exact strings. Ask for error code E4021, a part number, or a function name, and semantic search may return passages that are thematically related while missing the one chunk that literally contains the identifier. Classic keyword scoring — BM25, the workhorse of pre-LLM search — has the opposite profile: exact matches score highly, paraphrases score zero.
Production systems therefore run hybrid search: both a vector query and a keyword query, with results merged (reciprocal rank fusion is the common recipe). On top of that, a reranker — a smaller model that reads the query and each candidate chunk together and scores actual relevance — reorders the top candidates before they enter the prompt. Retrieve fifty candidates cheaply, rerank to the best five, and precision improves substantially for modest latency. If a RAG system is underperforming, hybrid retrieval plus a reranker is usually the highest-leverage upgrade available.
Why long context windows did not kill RAG
As of August 2026, flagship models from all three major vendors offer context windows around a million tokens. A million tokens is several novels — so why not skip the pipeline and paste the whole corpus into every prompt? Four reasons, none of which are going away:
- Cost per query. You pay for input tokens on every single request. At Claude Fable 5 rates ($10 per million input tokens as of August 2026), filling the window costs on the order of ten dollars per question. Prompt caching softens this for a stable corpus but does not eliminate it — the arithmetic is in the pricing explainer, and you can run your own numbers in the LLM cost calculator.
- Latency. Processing a giant prompt takes real time before the first output token appears. Retrieval hands the model a few thousand relevant tokens instead.
- Corpus size and freshness. Most real corpora do not fit in any window, and they change constantly. Updating an index row is cheap; re-stuffing and re-caching a mega-prompt on every edit is not.
- Access control. Different users are allowed to see different documents. With retrieval you filter chunks by permission before they reach the prompt. With one giant shared prompt there is no clean way to do that at all.
There is also a quality argument: models are noticeably better at answering from a few relevant passages than at finding and combining scattered facts inside an enormous prompt. Long context and RAG are complements — big windows let you retrieve more generously, not skip retrieval.
Evaluation and the failure modes demos never mention
A RAG system fails at two independent layers, and you must measure them separately. Retrieval: did the right chunk come back at all? Build a gold set of real questions paired with the passages that answer them, and track hit rate — how often the correct passage appears in the top k. Generation: given the right chunks, did the model produce a faithful answer? That is a groundedness check, often scored by a second model comparing the answer against the retrieved evidence. If you only measure end-to-end answer quality, you cannot tell whether to fix your index or your prompt — and no prompt engineering will rescue a query whose answer was never retrieved.
The failure modes that surface in production, ranked by pain:
- Retrieval misses that become confident hallucinations. The worst case. Retrieval returns plausible-but-wrong chunks, the model answers from its priors anyway, and the output arrives wrapped in citation-shaped confidence. Instruct the model to say when the context does not contain the answer, and test that it actually does.
- Bad chunking. Tables split mid-row, code split mid-function, pronouns orphaned from their antecedents. The model faithfully summarizes a fragment and the answer is subtly wrong.
- A stale index. Someone edits the policy document; nobody re-embeds it. The system now cites the old version with perfect confidence. Deleted documents that linger in the index are the same bug with worse optics. Re-indexing needs to be a pipeline, not a one-time script.
- Prompt-context conflicts. When retrieved text contradicts the model’s training data, which one wins is inconsistent — sometimes the document, sometimes the prior. Be explicit in the system prompt that context overrides memory.
- Untrusted content in the prompt. Retrieved documents are input from whoever wrote them, which makes a RAG index a delivery channel for prompt injection if your corpus includes anything user-submitted or scraped.
When not to use RAG
RAG is infrastructure, and infrastructure has carrying costs. Skip it when the corpus is small: if everything fits comfortably in the context window — a manual, a codebase’s docs, a contract — just put it in the prompt, cache it, and be done. A pipeline you did not build cannot go stale. Skip it when the problem is behavior rather than knowledge: if you want a particular tone, format, or domain dialect, that is fine-tuning territory, and stuffing style guides into context is the wrong tool. The full decision framework is in fine-tuning vs RAG vs prompting. And when users need a document rather than an answer, plain search with good ranking beats a paraphrased summary — not every query deserves a generation step.
The short version
- RAG grounds a model in your data: chunk, embed, index; then embed the query, retrieve neighbors, and generate from evidence.
- Chunking quality and hybrid retrieval with a reranker move the needle more than the choice of vector database ever will.
- Long context complements RAG; per-query cost, latency, freshness, and access control keep retrieval necessary.
- Evaluate retrieval and generation separately — the scariest failure is a retrieval miss dressed up as a confident answer.
- If the corpus fits in context, skip the pipeline; if the goal is style, fine-tune instead.