Generative AI Terms Every Tech Team Should Know
This glossary lists the core vocabulary that comes up constantly when a team starts working with Claude or any large language model. It is organized as a generic reference list, grouped by theme rather than alphabetically, so related terms sit near each other.
How to Use This List
- Skim the group headings first to find the term you need.
- Read definitions in order within a group. Later terms in a group sometimes build on earlier ones.
- Bookmark this page. It is meant to be a lookup reference, not a one-time read.
- When a term appears elsewhere in this section (like "hallucination" or "context window"), it is used the same way it's defined here.
Core Mechanics
- Token - The basic unit of text a model like Claude reads and writes. A token can be a whole short word, part of a longer word, or a punctuation mark. Pricing and limits are measured in tokens, not characters or words.
- Context Window - The maximum amount of text, measured in tokens, that a model can consider at once in a single conversation. It includes your messages, the model's replies, and any attached documents.
- Inference - The process of a trained model actually generating a response to a given input. Inference is what happens every time you send Claude a message, as distinct from training, which happens beforehand.
- Next-Token Prediction - The core mechanism behind text generation: the model predicts the most likely next token, one at a time, based on everything written so far.
- Attention Mechanism - The part of a transformer architecture that lets the model weigh how relevant every other piece of text is when processing a given token, rather than reading strictly in order.
- Transformer - The neural network architecture underlying Claude and most modern large language models, built around the attention mechanism.
- Parameters - The internal, learned numerical values inside a model that were adjusted during training. Roughly speaking, more parameters can mean more capacity to capture complex patterns, though it is not the only factor in quality.
- Latency - The time it takes for a model to start or finish responding. Smaller, faster models like Claude Haiku 4.5 generally have lower latency than larger, deeper-reasoning models.
Using Claude Day to Day
- Prompt - The text you send to a model to get a response, including instructions, questions, and any context you provide.
- System Message - An instruction, usually set by the app or account rather than typed mid-conversation, that shapes how the model behaves for an entire conversation, such as setting a persona or ground rules.
- User Message - Your input in a conversation: questions, instructions, and pasted content.
- Assistant Message - The model's generated reply in a conversation.
- Temperature - A setting that controls how much variety is allowed when the model samples its next token. Lower temperature favors the single most likely token; higher temperature allows more varied, creative phrasing.
- Top-p / Top-k - Related sampling controls that limit which candidate tokens the model is allowed to choose from at each step, used alongside or instead of temperature to shape response variety.
- Zero-Shot Prompting - Asking a model to perform a task with no examples included in the prompt, relying entirely on what it learned during training.
- Few-Shot Prompting - Including a small number of examples directly in the prompt to show the model the pattern you want it to follow.
- Extended Thinking (Adaptive Reasoning) - A visible, step-by-step reasoning process a model works through before producing its final answer. It is standard on Claude Fable 5 and available on other higher-tier models, and tends to improve accuracy on multi-step problems.
- Hallucination - Confident, fluent, but factually incorrect output. It happens because generation is pattern-based prediction, not a database lookup, so the model has no separate mechanism to flag "I'm not actually sure" by default.
- Knowledge Cutoff - The fixed date after which a model's training data stops. A model has no default knowledge of events after that date unless a research or browsing feature is explicitly used.
- Retrieval-Augmented Generation (RAG) - An architecture where a model's response is grounded by first retrieving relevant external documents or data, then generating an answer using that retrieved material. This is not something a plain Claude chat does automatically; it requires a specific tool, connector, or application built around it.
How Claude Is Built
- Pretraining - The initial, large-scale phase of training a model on a broad corpus of text and code, before any task-specific or safety-specific tuning happens.
- Fine-Tuning - Additional training on a narrower dataset after pretraining, used to adjust a model's behavior, style, or focus for a more specific purpose.
- Constitutional AI - Anthropic's approach to training Claude to be helpful, harmless, and honest, using a set of guiding principles the model is trained against, combined with safety testing before release.
- Model Family / Model Tier - The set of variants of a model released together with different trade-offs. Claude's lineup spans Claude Haiku 4.5 (fastest, cheapest), Claude Sonnet 5 (the default, balanced), Claude Opus 4.8 (flagship reasoning), and Claude Fable 5 (top tier, largest context window, always-on extended thinking).
- Usage Policy - Anthropic's published rules restricting certain uses of Claude, part of the broader safety and responsible-use framework surrounding the model.
FAQs
What is the single most important term to know before anything else?
Token. Nearly every other concept on this page, context window, pricing, latency, is defined in terms of tokens.
Is "inference" the same thing as "training"?
No. Training (including pretraining and fine-tuning) happens beforehand to produce the model. Inference is what happens every time the already-trained model generates a response to your input.
What's the difference between a prompt and a system message?
- A prompt is broadly the text you send to get a response.
- A system message is a specific kind of instruction, usually set by the app or account, that shapes behavior for an entire conversation rather than being typed as a one-off user message.
Why does hallucination happen if the model was trained on so much accurate information?
Because generation is prediction of the next plausible token, not a fact-check against a source. Fluent, confident phrasing can occur regardless of whether the specific fact stated is correct.
Does Claude use retrieval-augmented generation (RAG) by default?
No. Plain Claude chat does not automatically retrieve external documents. RAG requires a specific connector, tool, or application built to fetch and inject relevant material before generation.
What's the practical difference between zero-shot and few-shot prompting?
Zero-shot gives the model no examples and relies on what it already learned during training. Few-shot includes a small number of examples in the prompt itself to demonstrate the exact pattern you want.
How does temperature affect my results?
Lower temperature makes responses more predictable and repetitive, favoring the single most likely next token each time. Higher temperature allows more varied, sometimes more creative, phrasing by sampling from a wider range of plausible tokens.
Why does context window size matter for a tech team?
It determines how much conversation history, code, or documents can be considered at once. Running out of context window mid-task means older content is no longer visible to the model.
What does "extended thinking" actually add?
A visible, step-by-step reasoning pass before the model commits to a final answer, which tends to improve reliability on complex, multi-step problems compared to a direct answer with no such pass.
Is a model with more parameters always better?
Not necessarily. Parameter count is one factor among training data quality, training method, and reasoning depth. A smaller, well-trained model can outperform a larger, poorly-tuned one on a given task.
What is Constitutional AI, briefly?
Anthropic's training approach for shaping Claude's behavior around a set of guiding principles aimed at being helpful, harmless, and honest, combined with safety testing before a model is released.
Why does the glossary group terms instead of listing them alphabetically?
Grouping by theme, core mechanics, day-to-day usage, and how Claude is built, keeps related concepts near each other, which makes the list easier to learn from on a first read, not just to look up later.
Related
- Claude & LLM Fundamentals Basics - a first walkthrough that uses many of these terms in context.
- How Large Language Models Actually Generate Text - a deeper look at next-token prediction.
- Inside the Transformer: How Neural Networks Power Claude - a deeper look at attention and transformers.
- Why Claude Sometimes Gets Things Wrong - a closer look at hallucination.
Stack versions: Written against the Claude model lineup current as of ~June 2026 - Claude Fable 5, Claude Opus 4.8, Claude Sonnet 5 (the default), and Claude Haiku 4.5. Model names, pricing, and product features move quickly - verify current specifics at platform.claude.com/docs before relying on them.