Home▸Free interview questions▸RAG interview questions

free · no sign-up · no login

RAG interview questions, answered

20 questions with short answers, picked from the Q&A in the AI Engineer course. Tap a question to see the answer.

20 questionsBy Samir KumarUpdated

What is RAG (retrieval-augmented generation)?

RAG searches your own documents, puts the best passages into the prompt, and lets the model answer from them, with sources. Answers stay current and grounded in your data, without retraining the model. Most RAG problems are search problems, so interviewers ask a lot about chunking and retrieval.

No question matches that search. Try a shorter word.

20questions

RAG interview questions

The full set, with labs and mock interviews, is in the AI Engineer course.

See the AI Engineer course ▸

Embeddings & Vector Search

1

What is an embedding?

An embedding is a list of numbers that stands for the meaning of a text. Texts with similar meaning get similar numbers, even if they use different words.

2

What does cosine similarity measure?

Cosine similarity checks how close two vectors point. Close to 1 means similar meaning. Close to 0 means not related.

3

Why does keyword search miss things embeddings find?

People say the same thing in different words. 'My parcel never came' and 'delivery not received' share no words, so keyword search misses it. Embeddings match them by meaning.

4

What is a vector database, and when don't you need one?

It stores embeddings and quickly finds the closest ones, usually with filters. For a few thousand documents, a simple in-memory search or your normal database with a vector extension is enough. A dedicated vector database pays off at large scale or with heavy filtering.

6

Dense vs sparse embeddings?

Dense vectors capture meaning. Sparse vectors capture exact words, codes and names. Many systems use both together, called hybrid search.

7

What is a bi-encoder vs a cross-encoder?

A bi-encoder turns the question and each document into vectors separately. It is fast, so it's used for the first search. A cross-encoder reads the question and document together. It is more accurate but slow, so it's used to re-rank the top few.

9

How does HNSW work at a high level?

HNSW builds layers of connections, like a map with highways and local roads. The search starts at the top layer with big jumps, then moves down layer by layer to find the closest match.

10

Search quality collapsed after an upgrade, with no errors. Likely cause?

Most likely the questions used a new embedding model, but the documents still used the old one. Vectors from different models don't match. Re-embed all documents with the same model.

11

How do you choose an embedding model?

Use public leaderboards only to make a shortlist. Then test the models on your own real questions and pick the best one, also checking cost, speed and language support.

RAG

13

What is RAG?

RAG means: search your own documents, add the best parts to the prompt, and let the model answer from them. So answers use real, current information and can show sources.

14

What are the two phases of a RAG system?

First, indexing, done ahead of time: split documents into chunks, turn them into vectors, and store them. Second, querying, for every question: find the best chunks, add them to the prompt, and generate the answer.

15

Why not just fine-tune on the documents?

Fine-tuning freezes facts inside the model. Facts change, fine-tuning can't show sources, and it can't respect who is allowed to see what. RAG updates as soon as a document changes.

16

What is agentic RAG?

In agentic RAG, an agent decides how to search. It can rewrite the question, search several places, check the results, and search again before answering.

17

Your RAG system retrieves the right documents, but the model still answers wrongly. Why, and what do you change?

Retrieved is not the same as used. The right chunk may sit low in the list, get cut off by the context limit, or be buried in the middle, where models pay less attention. Vector search alone can also miss exact terms, like a drug name or an error code. Fix the ranking: combine keyword search (BM25) with vector search, merge the two lists with Reciprocal Rank Fusion, re-rank the top results, put the best chunks first, and ask the model to cite them.

18

How do you choose a chunk size?

Aim for one complete idea per chunk. Split on headings, with a small overlap between chunks. Then test a few sizes on your own questions.

19

What does a reranker add?

Vector search is fast but rough. A re-ranker reads the question with each of the top results and puts the best ones first. It often moves the right answer to the top.

Embeddings and Vector Search

20

When is vector search the wrong tool?

When the answer is an exact fact in a system: an order status, a balance, today's price. Use a database or API lookup for those. Vector search is for finding text by meaning. Many good systems use both: a lookup for exact facts and search for policy text.

More free interview questions by topic

ready to practise?

Now test yourself

Play the free quiz, or try a mock interview where you answer out loud and get a score.

Your next AI interview is closer than you think

Interview-ready. From ₹999.

Four courses: FDE, AI Agent Developer, AI Engineer and Claude Certified Architect. Narrated lessons, labs where you write the code, and every interview answer written out. Pay once and start today.

Enrol now ▸ Take the free quiz