How retrieval actually works
Four steps. The first one decides the quality of everything after it.
The pipeline, in order
Chunking
Your documents are split into passages small enough to be precise and large enough to make sense alone. Split mid-table or mid-clause and the retrieved passage is useless; split too coarsely and the model gets three pages to find one sentence in. This is the step that decides whether the system is good, and the step most tutorials spend two lines on.
Embedding
Each passage is turned into a vector - a numerical position that places similar meanings near each other - so a question about "shipping to the Gulf" can find a passage that says "delivery to UAE and Saudi" without sharing a word with it.
Retrieval
The question is embedded the same way and the nearest passages are fetched. In practice this is usually hybrid: vector similarity for meaning plus keyword search for the exact product code or name that vectors handle badly.
Generation, with a refusal
The passages go to the model with an instruction to answer only from them and to say so when they do not contain the answer. That last instruction is what separates a sourced answer from a confident invention.
Where RAG systems go wrong
Four failures we see most, three of which are decided before any code is written.
What actually breaks
Chunks that cut through meaning
A fixed 500-character split through the middle of a price table, a numbered clause or a step-by-step procedure produces passages that are individually meaningless. Splitting on the document's own structure - headings, clauses, rows - is more work and is most of the difference in answer quality.
Documents that contradict each other
Two versions of a policy, one of them superseded, both indexed. The system will retrieve one with complete confidence. Deciding which source is authoritative is not a technical task and cannot be skipped.
A stale index
Prices change, policies get revised, products are discontinued - and the index still holds last quarter. A re-indexing path has to exist from the start, or the system silently becomes a record of how things used to be.
No evaluation
Without a set of real questions and known-correct answers, nobody can tell whether a change made it better. Teams end up adjusting prompts on instinct. The question set is tedious to build and it is the only thing that makes improvement measurable.
RAG and chunking: common questions
Related work
What retrieval feeds, and where it runs.
- AI chatbot development — The most common thing built on retrieval.
- AI agent development — When the system must act on what it retrieved.
- AI development services — How a project runs end to end.
- Cloud servers with root access — Where a vector store and index live.
- AI company in Kashmir — Who we are, and why ask us.
Start with the documents you have
A folder of forty PDFs where three contradict each other is a normal starting point. Tell us what is in yours.