Model Tiering: Routing Simple Tasks to Haiku, Hard Tasks to Opus

Sending every request in a pipeline to the same model is the simplest thing to build and, at any real volume, one of the most expensive choices you can make.

A router that classifies task complexity first and only escalates to a stronger, pricier model when the task actually needs it can cut blended cost sharply without touching output quality on the tasks that matter.

This page builds that router: a cheap first-pass classifier on Haiku 4.5, an escalation path to Opus 4.8 for anything flagged as hard, and the guardrails that keep a bad classification from silently degrading quality.

Recipe

Quick-reference recipe card, copy-paste ready.

import anthropic
 
client = anthropic.Anthropic()
 
def classify_difficulty(client, task_text: str) -> str:
    response = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=10,
        messages=[{
            "role": "user",
            "content": f"Classify this task as SIMPLE or HARD. Reply with one word.\n\nTask: {task_text}",
        }],
    )
    return response.content[0].text.strip().upper()
 
def route(client, task_text: str) -> str:
    difficulty = classify_difficulty(client, task_text)
    model = "claude-opus-4-8" if difficulty == "HARD" else "claude-sonnet-5"
    response = client.messages.create(
        model=model,
        max_tokens=500,
        messages=[{"role": "user", "content": task_text}],
    )
    return response.content[0].text

When to reach for this:

• Your pipeline handles a mix of task difficulty, some genuinely simple (classification, extraction, formatting), some genuinely hard (multi-step reasoning, ambiguous requirements).
• Volume is high enough that a flat per-call cost gap compounds into a real budget concern.
• You can tolerate a small classification overhead (one cheap extra call) in exchange for routing accuracy.
• You want a defensible, auditable reason for which model handled a given request, not an arbitrary default.

Working Example

import anthropic
from dataclasses import dataclass
 
client = anthropic.Anthropic()
 
MODELS = {
    "haiku": "claude-haiku-4-5",
    "sonnet": "claude-sonnet-5",
    "opus": "claude-opus-4-8",
}
 
 
@dataclass
class RoutedResponse:
    text: str
    model_used: str
    difficulty: str
 
 
def classify_difficulty(client, task_text: str) -> str:
    """Cheap first-pass classification on Haiku. Returns SIMPLE, MODERATE, or HARD."""
    response = client.messages.create(
        model=MODELS["haiku"],
        max_tokens=10,
        messages=[{
            "role": "user",
            "content": (
                "Classify the following task's difficulty as exactly one word: "
                "SIMPLE, MODERATE, or HARD.\n"
                "SIMPLE: classification, extraction, formatting, lookup.\n"
                "MODERATE: summarization, drafting from clear requirements.\n"
                "HARD: multi-step reasoning, ambiguous requirements, trade-off analysis.\n\n"
                f"Task: {task_text}"
            ),
        }],
    )
    label = response.content[0].text.strip().upper()
    return label if label in {"SIMPLE", "MODERATE", "HARD"} else "MODERATE"
 
 
def route_task(client, task_text: str) -> RoutedResponse:
    """Classify difficulty, then dispatch to the matching model tier."""
    difficulty = classify_difficulty(client, task_text)
    model = {
        "SIMPLE": MODELS["haiku"],
        "MODERATE": MODELS["sonnet"],
        "HARD": MODELS["opus"],
    }[difficulty]
 
    response = client.messages.create(
        model=model,
        max_tokens=800,
        messages=[{"role": "user", "content": task_text}],
    )
    return RoutedResponse(
        text=response.content[0].text,
        model_used=model,
        difficulty=difficulty,
    )
 
 
result = route_task(client, "Tag this support ticket: 'My package arrived damaged.'")
print(result.model_used, result.difficulty)
 
result2 = route_task(
    client,
    "We need to decide between event sourcing and a simple audit log for our "
    "order pipeline. Weigh the trade-offs given we have 3 engineers and a 6-week timeline.",
)
print(result2.model_used, result2.difficulty)

What this demonstrates:

• A three-tier classification (SIMPLE/MODERATE/HARD) maps cleanly onto the three-tier model lineup used here (Haiku/Sonnet/Opus), rather than a binary simple-or-hard split.
• The classifier prompt gives concrete examples of each category, which matters more for classification accuracy than the model tier doing the classifying.
• Classification itself runs on Haiku, the cheapest tier, since classifying difficulty is itself a SIMPLE-shaped task.
• The RoutedResponse dataclass carries model_used and difficulty alongside the answer, so every routed response is auditable after the fact.
• A default fallback (MODERATE) handles the case where the classifier returns something outside the expected three labels, rather than crashing the pipeline.

Deep Dive

How It Works

• The router adds exactly one extra call per request: a short Haiku classification call with a tiny max_tokens (10 in this example), which costs a small fraction of a cent even repeated at volume.
• That classification overhead is the price of the routing decision, and it needs to stay cheap relative to the savings it enables, which is why the classifier itself always runs on the cheapest tier.
• The escalation path only pays Opus-tier pricing for requests actually flagged HARD, so total spend tracks your workload's true difficulty mix rather than its worst case.
• Classification accuracy is the single biggest risk in this pattern: a HARD task misclassified as SIMPLE gets routed to a model that may produce a worse answer, silently, with no error to catch it.

Guarding Against Misclassification

def route_with_confidence_check(client, task_text: str) -> RoutedResponse:
    """Escalate automatically if the SIMPLE-tier response looks too short or hedges."""
    difficulty = classify_difficulty(client, task_text)
    model = {"SIMPLE": MODELS["haiku"], "MODERATE": MODELS["sonnet"], "HARD": MODELS["opus"]}[difficulty]
 
    response = client.messages.create(model=model, max_tokens=800, messages=[{"role": "user", "content": task_text}])
    text = response.content[0].text
 
    # A SIMPLE-tier answer that hedges heavily is a signal the classifier under-shot difficulty.
    hedge_markers = ("it depends", "i'm not sure", "this is complex", "unclear")
    if difficulty == "SIMPLE" and any(marker in text.lower() for marker in hedge_markers):
        response = client.messages.create(model=MODELS["opus"], max_tokens=800, messages=[{"role": "user", "content": task_text}])
        return RoutedResponse(text=response.content[0].text, model_used=MODELS["opus"], difficulty="HARD (escalated)")
 
    return RoutedResponse(text=text, model_used=model, difficulty=difficulty)

This is a cheap heuristic, not a rigorous confidence score, but it catches the specific failure mode where a task looked simple in isolation but the model itself signals uncertainty. A production system doing this at scale would typically log escalations and periodically review them to refine the classifier's category definitions.

Cost Comparison: Routed vs Flat

The exact savings depend on your task mix and token shapes per task, but the direction holds broadly: the more skewed your workload is toward simple, high-volume tasks, the more a router saves relative to any flat single-model policy.

Parameters & Return Values

Gotchas

• Letting classification errors silently degrade output quality. A HARD task misrouted to Haiku doesn't throw an error, it just returns a worse answer that looks superficially fine. Fix: add a lightweight confidence check (see above) or periodically sample-audit routed outputs against a human or stronger-model baseline.
• Making the classifier prompt too vague. "Is this task simple or hard?" with no examples produces inconsistent labels. Fix: give the classifier concrete category examples, as in the Working Example, tuned to your actual task distribution.
• Forgetting the classification call itself has a cost. At extremely high volume with very cheap per-task costs, the classification overhead can approach the savings it's meant to produce. Fix: measure the classifier's own cost against the savings it generates; for a workload where every task is already known to be uniform difficulty, routing may not be worth the overhead.
• Hardcoding the difficulty categories to match only two model tiers. A binary SIMPLE/HARD split forces a false choice when a task is genuinely MODERATE. Fix: use as many tiers as your model lineup supports, mapping category to model one-to-one.
• Not logging which model handled each request. Without model_used (or equivalent) in your output, you can't audit whether routing decisions were sound after the fact. Fix: always carry the routing decision alongside the response, as RoutedResponse does here.
• Treating the router as a one-time build. Task distributions shift as a product evolves, and a classifier tuned for last quarter's task mix can drift out of date. Fix: periodically review a sample of routing decisions against actual difficulty, and retune the classifier's examples.

Alternatives

FAQs

Related

• Sonnet 5 vs Opus 4.8 vs Fable 5: A Cost-Per-Task Comparison - the per-task cost math this router is designed to exploit.
• Building a Pre-Request Cost Calculator with the Token Counting API - estimating cost for whichever tier the router picks.
• ADR Template: Choosing a Default Model Tier for Your Team - documenting the routing policy as a team-wide decision.
• Batch API Economics: When 50 Percent Off Beats Real-Time Calls - a second cost lever that layers on top of tiering.

Stack versions: Written against the Claude model lineup current as of ~June 2026 - Claude Fable 5, Claude Opus 4.8, Claude Sonnet 5 (the default), and Claude Haiku 4.5 - and the official anthropic Python SDK (latest 0.x release). Model names, pricing, and SDK versions move quickly - verify current specifics at platform.claude.com/docs before relying on them.