Retrieval & RAG Systems
RAG is the skill of feeding a language model the right context from your own data at the right moment. It is assessed on retrieval quality diagnosis — most RAG systems fail at retrieval, not generation, and candidates who can tell the difference are the ones worth hiring.
Also called: RAG, retrieval augmented generation, vector search
What is retrieval & rag systems?
Retrieval systems cover chunking strategy, embedding and index choice, hybrid and re-ranked retrieval, context assembly, and the evaluation of retrieval quality independently from generation quality.
RAG is the most common architecture in enterprise AI and the most commonly broken. The failure is nearly always upstream of the model, and diagnosing that requires a specific discipline that no degree teaches.
What do the levels of retrieval & rag systems mean?
Four levels, each defined by observable behaviour rather than by years. This is the definition employers set their bar against, and the one every proof is scored to.
- L1Aware
Can build a basic embed-and-search pipeline that returns relevant documents.
Evidence: A working retrieval pipeline over a small corpus.
- L2Working
Tunes chunking and retrieval parameters against observed failures rather than by default settings.
Evidence: Before/after retrieval quality on a fixed query set.
- L3Independent
Measures retrieval quality separately from answer quality, and uses hybrid or re-ranked retrieval where the data demands it.
Evidence: A retrieval eval with recall metrics, separate from end-to-end accuracy.
- L4Leading
Designs retrieval architecture for heterogeneous corpora at scale and sets the quality bar the organisation measures against.
Evidence: A retrieval platform serving multiple products with tracked quality.
| Level | Label | People proven here |
|---|---|---|
| L1 | Aware | 0 |
| L2 | Working | 0 |
| L3 | Independent | 2 |
| L4 | Leading | 0 |
How is retrieval & rag systems assessed?
Candidates are given a RAG system producing wrong answers and must locate the failure. Scoring rewards separating retrieval failure from generation failure before proposing a fix, and measuring retrieval quality on its own.
Which roles require retrieval & rag systems?
| Role | Level required | Status |
|---|---|---|
| AI Engineer | L2 Working | Nice to have |
How do you prove retrieval & rag systems?
Retrieval & RAG Systems: questions people ask
- Is RAG obsolete now that context windows are large?
- No. Large context windows changed the trade-off, not the need — cost, latency, and precision still make retrieval the right answer at scale. Knowing when a long context beats retrieval is itself Level 3 judgment.
- Do I need vector database experience?
- Helpful but not required. The rubric scores diagnosis and measurement; someone who can prove retrieval is the failure point scores above someone who has used three vector databases without measuring any of them.
- How long does the RAG challenge take?
- Around 90 minutes. You are given a broken system and asked to locate the failure and justify a fix, not to build a pipeline from scratch.