Home▸Free interview questions▸LLM interview questions
free · no sign-up · no loginLLM interview questions, answered
20 questions with short answers, picked from the Q&A in the AI Engineer course. Tap a question to see the answer.
What is a large language model (LLM)?
An LLM is a neural network, usually a transformer, trained on a huge amount of text to predict the next token. That one skill lets it answer questions, summarise, write code and follow instructions. It does not look facts up, so it can sound sure and still be wrong. That is why RAG and evaluation matter.
- Text becomes tokens
- Attention relates every token to the others
- The model predicts the next token
- Sampling settings shape the output
No question matches that search. Try a shorter word.
LLM interview questions
The full set, with labs and mock interviews, is in the AI Engineer course.
LLM Fundamentals
1What is a token, and why does it matter?
A token is a small piece of text, often part of a word. The model reads and writes tokens, not words. Price, speed and the context limit are all counted in tokens.
2How does an LLM generate an answer?
The model guesses the next token, adds it, and repeats. It does not plan the whole answer first. It writes one token at a time, until it decides to stop.
3What does temperature do?
Temperature controls how random the answer is. At 0, the model picks the most likely word, so you get the same answer every time. Higher temperature gives more varied answers. Use low for facts, higher for ideas.
4What is a context window, and what happens when you go over it?
The context window is the most tokens one request can hold, counting the input and the answer together. If you go over it, the call fails or older text gets cut off. Agents reach it quickly because they resend the whole history and every tool result on each step, so they need summaries, trimming or retrieval.
5What does top-p do, and how is it different from temperature?
Both control how random the answer is. Temperature reshapes the chances of every possible next word. Top-p keeps only the most likely words until their chances add up to p, for example 0.9, and picks from those. For tool calls and JSON, keep both low so answers stay predictable.
6What is the difference between a base model and an instruction-tuned model?
A base model only continues text, because it was trained to predict the next word. An instruction-tuned model was trained further on examples of following instructions and chatting, so it answers questions, follows rules and uses tools. Apps almost always use instruction-tuned models.
7What are system, user and assistant messages?
The system message sets the role and the rules, for example 'You are a support agent for Acme'. User messages are what the person types. Assistant messages are the model's earlier replies. The model reads all of them, in order, on every call.
8How is LLM API usage usually priced?
You pay per token, with one price for input and a higher price for output, often several times more. Cached input is much cheaper, and batch jobs are often about half price. In agents the input grows with every step, so long runs add up quickly.
9Why can the same question cost far more tokens in Hindi than in English?
Tokenizers learned mostly from English text. So Hindi words break into more, smaller pieces. More tokens means more cost, slower answers and less space in the context window.
10Why are LLMs stateless, and how do chat apps remember?
The model remembers nothing between calls. The chat app sends the whole conversation again every time. That is why long chats get slower and cost more.
11What is self-attention, in one sentence?
Each word looks at all the other words and decides which ones matter most to it. For example, in 'the cat sat because it was tired', the word 'it' pays most attention to 'the cat'.
12What is the KV cache and what does it cost?
The KV cache saves work the model already did on earlier tokens. So each new token is fast to make. But it uses GPU memory, and that memory grows with every token and every user.
13Why can answers change even at temperature 0?
Small rounding differences on GPUs, different batching on the provider's servers, and silent model updates can all change the result. So never trust a single run. Test over several runs, and design the app so small changes in wording don't break it.
14What are rate limits, and how do you design around them?
Providers cap how many requests and tokens you can send per minute. Go over it and you get a 429 error. Design around it with a queue, retries that wait a little longer each time, smaller prompts, caching, and spreading load across keys, regions or providers.
15Why does attention get expensive for long prompts?
In attention, every token is compared with every other token. So if the prompt is twice as long, the work is about four times more. That is why long prompts make the first word slow.
16What does mixture of experts change, and what does it not?
Mixture of experts has many small expert networks, but only a few work on each token. So each token needs less compute and runs faster. But all the experts must still be loaded, so memory stays high.
17Why do models hallucinate, and what actually reduces it?
The model writes text that sounds likely, not text it has checked. So it can say something wrong very confidently. To reduce this, give it real documents, ask it to quote them, and let it say 'I don't know'.
Model Selection
18How do you compare models fairly on cost?
Use your real traffic: input and output tokens, caching, retries and answer length. Then compare the cost per successful task, not price per token.
19A cheaper-per-token model ended up costing more. Why?
It wrote longer answers or needed retries, so it used more tokens per task. Always measure cost per task, not per token.
How Large Language Models Work
20Your prompt works in English but costs far more in Hindi. Why?
Cost and limits are counted in tokens. The tokenizer saw mostly English, so an English word is often one token while a Hindi word splits into several. The same meaning uses more tokens, costs more and fills the context window sooner. Measure tokens per language before you set a budget.
More free interview questions by topic
All 91 free questionsAgentic AIRAGMCPAI system designLLM evaluation and guardrailsFDE
Now test yourself
Play the free quiz, or try a mock interview where you answer out loud and get a score.