Home▸Free interview questions▸RAG interview questions
free · no sign-up · no loginRAG interview questions, answered
20 questions with short answers, picked from the Q&A in the AI Engineer course. Tap a question to see the answer.
What is RAG (retrieval-augmented generation)?
RAG searches your own documents, puts the best passages into the prompt, and lets the model answer from them, with sources. Answers stay current and grounded in your data, without retraining the model. Most RAG problems are search problems, so interviewers ask a lot about chunking and retrieval.
- Split documents into chunks
- Turn chunks into embeddings
- Find the best chunks for the question
- Answer from them, with citations
No question matches that search. Try a shorter word.
RAG interview questions
The full set, with labs and mock interviews, is in the AI Engineer course.
Embeddings & Vector Search
1What is an embedding?
An embedding is a list of numbers that stands for the meaning of a text. Texts with similar meaning get similar numbers, even if they use different words.
2What does cosine similarity measure?
Cosine similarity checks how close two vectors point. Close to 1 means similar meaning. Close to 0 means not related.
3Why does keyword search miss things embeddings find?
People say the same thing in different words. 'My parcel never came' and 'delivery not received' share no words, so keyword search misses it. Embeddings match them by meaning.
4What is a vector database, and when don't you need one?
It stores embeddings and quickly finds the closest ones, usually with filters. For a few thousand documents, a simple in-memory search or your normal database with a vector extension is enough. A dedicated vector database pays off at large scale or with heavy filtering.
5When do you use metadata filters vs pure vector search?
First filter by fixed facts, like country, date or who can see it. Then search by meaning inside what's left. Filters keep answers correct and safe; vectors find the meaning.
6Dense vs sparse embeddings?
Dense vectors capture meaning. Sparse vectors capture exact words, codes and names. Many systems use both together, called hybrid search.
7What is a bi-encoder vs a cross-encoder?
A bi-encoder turns the question and each document into vectors separately. It is fast, so it's used for the first search. A cross-encoder reads the question and document together. It is more accurate but slow, so it's used to re-rank the top few.
8Why do vector databases use approximate search?
Comparing the query with every vector is too slow when you have millions. Approximate search finds very close matches in milliseconds, with only a tiny loss in accuracy.
9How does HNSW work at a high level?
HNSW builds layers of connections, like a map with highways and local roads. The search starts at the top layer with big jumps, then moves down layer by layer to find the closest match.
10Search quality collapsed after an upgrade, with no errors. Likely cause?
Most likely the questions used a new embedding model, but the documents still used the old one. Vectors from different models don't match. Re-embed all documents with the same model.
11How do you choose an embedding model?
Use public leaderboards only to make a shortlist. Then test the models on your own real questions and pick the best one, also checking cost, speed and language support.
12What role can knowledge graphs play next to vector search?
A knowledge graph stores facts as connections, like 'Product A is made by Supplier B'. It helps with questions that need links across many facts, which separate text chunks often miss. Teams usually use it alongside normal vector search, not instead of it.
RAG
13What is RAG?
RAG means: search your own documents, add the best parts to the prompt, and let the model answer from them. So answers use real, current information and can show sources.
14What are the two phases of a RAG system?
First, indexing, done ahead of time: split documents into chunks, turn them into vectors, and store them. Second, querying, for every question: find the best chunks, add them to the prompt, and generate the answer.
15Why not just fine-tune on the documents?
Fine-tuning freezes facts inside the model. Facts change, fine-tuning can't show sources, and it can't respect who is allowed to see what. RAG updates as soon as a document changes.
16What is agentic RAG?
In agentic RAG, an agent decides how to search. It can rewrite the question, search several places, check the results, and search again before answering.
17Your RAG system retrieves the right documents, but the model still answers wrongly. Why, and what do you change?
Retrieved is not the same as used. The right chunk may sit low in the list, get cut off by the context limit, or be buried in the middle, where models pay less attention. Vector search alone can also miss exact terms, like a drug name or an error code. Fix the ranking: combine keyword search (BM25) with vector search, merge the two lists with Reciprocal Rank Fusion, re-rank the top results, put the best chunks first, and ask the model to cite them.
18How do you choose a chunk size?
Aim for one complete idea per chunk. Split on headings, with a small overlap between chunks. Then test a few sizes on your own questions.
19What does a reranker add?
Vector search is fast but rough. A re-ranker reads the question with each of the top results and puts the best ones first. It often moves the right answer to the top.
Embeddings and Vector Search
20When is vector search the wrong tool?
When the answer is an exact fact in a system: an order status, a balance, today's price. Use a database or API lookup for those. Vector search is for finding text by meaning. Many good systems use both: a lookup for exact facts and search for policy text.
More free interview questions by topic
All 91 free questionsAgentic AIMCPLLMAI system designLLM evaluation and guardrailsFDE
Now test yourself
Play the free quiz, or try a mock interview where you answer out loud and get a score.