Home▸Free interview questions▸LLM interview questions

free · no sign-up · no login

LLM interview questions, answered

20 questions with short answers, picked from the Q&A in the AI Engineer course. Tap a question to see the answer.

20 questionsBy Samir KumarUpdated

What is a large language model (LLM)?

An LLM is a neural network, usually a transformer, trained on a huge amount of text to predict the next token. That one skill lets it answer questions, summarise, write code and follow instructions. It does not look facts up, so it can sound sure and still be wrong. That is why RAG and evaluation matter.

No question matches that search. Try a shorter word.

20questions

LLM interview questions

The full set, with labs and mock interviews, is in the AI Engineer course.

See the AI Engineer course ▸

LLM Fundamentals

1

What is a token, and why does it matter?

A token is a small piece of text, often part of a word. The model reads and writes tokens, not words. Price, speed and the context limit are all counted in tokens.

2

How does an LLM generate an answer?

The model guesses the next token, adds it, and repeats. It does not plan the whole answer first. It writes one token at a time, until it decides to stop.

3

What does temperature do?

Temperature controls how random the answer is. At 0, the model picks the most likely word, so you get the same answer every time. Higher temperature gives more varied answers. Use low for facts, higher for ideas.

4

What is a context window, and what happens when you go over it?

The context window is the most tokens one request can hold, counting the input and the answer together. If you go over it, the call fails or older text gets cut off. Agents reach it quickly because they resend the whole history and every tool result on each step, so they need summaries, trimming or retrieval.

5

What does top-p do, and how is it different from temperature?

Both control how random the answer is. Temperature reshapes the chances of every possible next word. Top-p keeps only the most likely words until their chances add up to p, for example 0.9, and picks from those. For tool calls and JSON, keep both low so answers stay predictable.

6

What is the difference between a base model and an instruction-tuned model?

A base model only continues text, because it was trained to predict the next word. An instruction-tuned model was trained further on examples of following instructions and chatting, so it answers questions, follows rules and uses tools. Apps almost always use instruction-tuned models.

7

What are system, user and assistant messages?

The system message sets the role and the rules, for example 'You are a support agent for Acme'. User messages are what the person types. Assistant messages are the model's earlier replies. The model reads all of them, in order, on every call.

8

How is LLM API usage usually priced?

You pay per token, with one price for input and a higher price for output, often several times more. Cached input is much cheaper, and batch jobs are often about half price. In agents the input grows with every step, so long runs add up quickly.

9

Why can the same question cost far more tokens in Hindi than in English?

Tokenizers learned mostly from English text. So Hindi words break into more, smaller pieces. More tokens means more cost, slower answers and less space in the context window.

10

Why are LLMs stateless, and how do chat apps remember?

The model remembers nothing between calls. The chat app sends the whole conversation again every time. That is why long chats get slower and cost more.

11

What is self-attention, in one sentence?

Each word looks at all the other words and decides which ones matter most to it. For example, in 'the cat sat because it was tired', the word 'it' pays most attention to 'the cat'.

12

What is the KV cache and what does it cost?

The KV cache saves work the model already did on earlier tokens. So each new token is fast to make. But it uses GPU memory, and that memory grows with every token and every user.

13

Why can answers change even at temperature 0?

Small rounding differences on GPUs, different batching on the provider's servers, and silent model updates can all change the result. So never trust a single run. Test over several runs, and design the app so small changes in wording don't break it.

14

What are rate limits, and how do you design around them?

Providers cap how many requests and tokens you can send per minute. Go over it and you get a 429 error. Design around it with a queue, retries that wait a little longer each time, smaller prompts, caching, and spreading load across keys, regions or providers.

15

Why does attention get expensive for long prompts?

In attention, every token is compared with every other token. So if the prompt is twice as long, the work is about four times more. That is why long prompts make the first word slow.

16

What does mixture of experts change, and what does it not?

Mixture of experts has many small expert networks, but only a few work on each token. So each token needs less compute and runs faster. But all the experts must still be loaded, so memory stays high.

17

Why do models hallucinate, and what actually reduces it?

The model writes text that sounds likely, not text it has checked. So it can say something wrong very confidently. To reduce this, give it real documents, ask it to quote them, and let it say 'I don't know'.

Model Selection

18

How do you compare models fairly on cost?

Use your real traffic: input and output tokens, caching, retries and answer length. Then compare the cost per successful task, not price per token.

19

A cheaper-per-token model ended up costing more. Why?

It wrote longer answers or needed retries, so it used more tokens per task. Always measure cost per task, not per token.

How Large Language Models Work

20

Your prompt works in English but costs far more in Hindi. Why?

Cost and limits are counted in tokens. The tokenizer saw mostly English, so an English word is often one token while a Hindi word splits into several. The same meaning uses more tokens, costs more and fills the context window sooner. Measure tokens per language before you set a budget.

More free interview questions by topic

ready to practise?

Now test yourself

Play the free quiz, or try a mock interview where you answer out loud and get a score.

Your next AI interview is closer than you think

Interview-ready. From ₹999.

Four courses: FDE, AI Agent Developer, AI Engineer and Claude Certified Architect. Narrated lessons, labs where you write the code, and every interview answer written out. Pay once and start today.

Enrol now ▸ Take the free quiz