//
Search across all documentation pages
When Claude answers a question, it is not searching a database for a pre-written response.
It is generating text one small piece at a time, choosing each piece based on patterns it learned during training.
Understanding this single mechanism, called next-token prediction, explains a lot about why Claude behaves the way it does.
It explains why responses can vary slightly between runs, why Claude sometimes states something incorrect with total confidence, and why longer conversations can change how it responds.
This page walks through what a token actually is, how Claude picks the next one, and why that process is fundamentally different from looking something up.
A token is the basic unit of text Claude reads and writes.
A token is often a whole short word, but it can also be part of a longer word, a punctuation mark, or a few characters.
The word "documentation" might be split into two or three tokens, while "the" is usually a single token.
Claude does not think in letters or in whole sentences. It thinks in these token-sized chunks.
When you send Claude a message, that message is first broken down into a sequence of tokens.
Claude then generates its response the same way: one token, then another, then another, until it decides the response is complete.
Each new token is chosen by looking at everything that came before it, including your original message and every token Claude has generated so far in its own reply.
This is why the process is called next-token prediction. At every step, the model is solving one narrow question: given everything so far, what token is most likely to come next?
A simple way to build intuition for this is to think about how you might finish the sentence "The capital of France is ___". Most people would immediately think "Paris."
That is not because you looked it up in that instant. It is because your brain has seen that pattern so many times that the completion feels automatic.
Claude's next-token prediction works on a similar principle, except it was trained on a vastly larger and more varied set of text, and its "instinct" is a learned mathematical function rather than a human memory.
Underneath next-token prediction is a probability distribution.
At each step, Claude does not pick just one candidate token. It calculates a likelihood score for every token in its vocabulary, which can be tens of thousands of possible tokens.
Most of those tokens get a near-zero score, because they would make no sense given the text so far.
A small number of tokens get high scores, because they fit the pattern well.
Claude then samples from that distribution to actually choose the next token, using controls (like temperature) that affect how often it picks the single most likely token versus a slightly less likely but still reasonable one.
This is why asking Claude the same question twice can sometimes produce slightly different wording, even though the underlying reasoning is similar.
The chosen token is then added to the sequence, and the whole process repeats to choose the token after that.
This sequential, one-token-at-a-time loop is why generation takes time proportional to the length of the response. A longer answer means more prediction steps, not one big lookup.
It also means that everything Claude has already written becomes part of what it considers when writing the next piece. If Claude's response starts down a particular line of reasoning, later tokens are influenced by that earlier text, for better or worse.
This is different from how a search engine or a rule-based chatbot works. A search engine retrieves an existing indexed document. A rule-based chatbot matches your input against pre-written scripts.
Claude, by contrast, constructs its response fresh, token by token, guided by patterns it absorbed from training, not by retrieving a stored answer that already existed somewhere.
Input so far: "The mitochondria is the"
Claude scores candidate next tokens (simplified):
"powerhouse" -> very likely
"membrane" -> somewhat likely
"banana" -> extremely unlikely
Claude samples "powerhouse", then repeats the process for the next token.The patterns Claude uses to predict tokens come from training on an enormous amount of text and code, learned before a fixed knowledge cutoff date.
This means next-token prediction is fundamentally a pattern-matching process grounded in what the model absorbed during training, not a live connection to current facts.
In a plain chat conversation, Claude has no default live access to the internet unless a research or browsing feature is explicitly turned on, so its predictions reflect training-time patterns plus whatever context you've given it in the conversation.
This is also the root cause of hallucination. Because Claude is always predicting a plausible next token rather than checking a fact against a source, it can generate a confident, fluent, well-formed sentence that happens to be wrong, especially for obscure facts, exact numbers, or citations.
The model has no separate "I don't actually know this" signal built into the generation mechanism itself. Fluency and correctness are not the same thing, and next-token prediction only guarantees the former.
Different Claude models apply this same core mechanism with different amounts of computation and different training. Claude Haiku 4.5 predicts tokens quickly and cheaply, suited to high-volume or latency-sensitive tasks.
Claude Sonnet 5 balances speed and depth for most everyday and coding work. Claude Opus 4.8 and Claude Fable 5 apply deeper reasoning, including visible step-by-step "thinking" tokens, before producing a final answer, which tends to improve accuracy on harder, multi-step problems.
| Approach | Strength | Weakness | Best Fit |
|---|---|---|---|
| Fast next-token prediction (e.g. Haiku 4.5) | Low latency, low cost | Less depth on complex multi-step reasoning | High-volume, simple, latency-sensitive tasks |
| Prediction with extended thinking (e.g. Fable 5, Opus 4.8) | More reliable on complex reasoning | Slower, more expensive per response | Research, analysis, and high-stakes reasoning tasks |
| Retrieval-augmented prediction (Claude plus search/browsing tools) | Grounds predictions in current, external sources | Requires the feature to be explicitly enabled and available | Questions needing current or verifiable facts |
No. Claude generates one token at a time, and each new token is chosen using everything written so far, including the tokens it has already produced in that same response.
Because next-token generation involves sampling from a probability distribution rather than always selecting one fixed answer, small variations in wording are normal and expected.
No. A database lookup retrieves a stored, pre-existing record. Next-token prediction constructs new text by predicting likely continuations based on learned patterns, with nothing being retrieved from storage.
Fluent, confident-sounding phrasing comes from the same prediction mechanism regardless of whether the underlying content is accurate. There is no separate certainty check built into basic generation.
Not in a fixed sense. Each token is chosen based on what came before it, so the response takes shape progressively rather than existing as a complete plan from the first token, although extended thinking modes add a visible reasoning pass before the final answer.
It's the set of likelihood scores Claude assigns to every possible next token at a given step, with far more likely tokens scoring much higher than implausible ones.
No. In a plain chat conversation, Claude relies on patterns learned before its knowledge cutoff plus whatever context is in the conversation, unless a research or browsing feature is explicitly enabled.
The core mechanism is shared, but models differ in depth of computation. Faster models like Claude Haiku 4.5 predict quickly with less depth, while models like Claude Opus 4.8 and Claude Fable 5 apply extended reasoning before finalizing an answer.
Because tokens are produced sequentially, one prediction step per token. A longer response requires more prediction steps, not a single larger lookup.
Yes. Since every prediction depends on the exact preceding text, changing your wording changes the context Claude is predicting from, which can shift which tokens seem most likely at each step.
Stack versions: Written against the Claude model lineup current as of ~June 2026 - Claude Fable 5, Claude Opus 4.8, Claude Sonnet 5 (the default), and Claude Haiku 4.5. Model names, pricing, and product features move quickly - verify current specifics at platform.claude.com/docs before relying on them.