Choosing the Right Model

Story: The $12,000 Mistake

Marcus ran engineering at a SaaS company building an AI-powered customer support platform. When it came time to choose a Claude model, he did what most engineers do: he picked the best one. "We're building a premium product," he told his team. "Our customers deserve the best. We're going with Opus for everything."

Three months later, Marcus was staring at an API bill of $12,000 for a single month. His platform handled about 50,000 support tickets per month. Each ticket went through three stages: classification (routing to the right team), response generation (drafting a reply), and quality check (reviewing the draft before sending). Every stage used Opus.

When his team actually benchmarked the stages, the results were humbling:

Stage	Opus Accuracy	Sonnet Accuracy	Haiku Accuracy
Classification	97.2%	96.8%	95.1%
Response Generation	94.5%	92.1%	78.3%
Quality Check	96.0%	95.2%	88.7%

For classification, Haiku was within 2% of Opus -- and 20x cheaper. For quality checks, Sonnet matched Opus closely. Only response generation showed a meaningful gap. Marcus restructured: Haiku for classification, Sonnet for quality checks, Opus only for the most complex response generation (about 15% of tickets that Sonnet flagged as difficult).

<ConceptCard

front="Choosing the Right Model"

back="Choose Haiku for speed and cost on simple tasks, Sonnet for balanced everyday use, and Opus when maximum reasoning quality justifies the higher cost."

New monthly bill: $2,100. Quality metrics barely changed.

The lesson? Choosing the right model isn't about picking the "best" one. It's about matching capability to need, task by task.

---

Concept: A Decision Framework for Model Selection

The Three Dimensions

Every model selection decision involves three trade-offs:

                    Quality
                      /\
                     /  \
                    /    \
                   /      \
                  /________\
              Speed -------- Cost

No model wins on all three. Your job is to decide which dimension matters most for each specific task.

Dimension 1: Task Complexity

The single most important factor in model selection is how complex the task is. Here's a practical complexity scale:

Level 1 -- Pattern Matching (Haiku)

Tasks with clear inputs and deterministic-ish outputs:

Binary classification (spam/not spam)
Entity extraction (pull names, dates, amounts from text)
Sentiment analysis (positive/negative/neutral)
Format conversion (restructure data from one format to another)
Simple Q&A (lookup-style questions with clear answers)

Use Choosing the Right Model in a low-risk branch or scratch project first. That keeps the lesson concrete without making your first attempt carry production pressure.

</Callout>

Level 2 -- Reasoning Required (Sonnet)

Tasks that need understanding and judgment:

Code generation (writing functions, debugging)
Content creation (marketing copy, technical writing)
Summarization (distilling long documents into key points)
Multi-step instructions (following complex workflows)
Conversational AI (maintaining context, natural dialogue)

Level 3 -- Deep Analysis (Opus)

Tasks requiring extended reasoning or creative judgment:

Legal/medical analysis (high-stakes, nuanced interpretation)
Architectural decisions (trade-off analysis, system design)
Creative writing (nuanced fiction, persuasive essays)
Research synthesis (combining multiple sources into insights)
Agentic tasks (autonomous multi-step work)

Python

import anthropic
from enum import Enum
from typing import Optional

class Complexity(Enum):
    SIMPLE = 1    # Pattern matching
    MODERATE = 2  # Reasoning required
    COMPLEX = 3   # Deep analysis

def select_model(complexity: Complexity,
                 latency_ms: Optional[int] = None,
                 budget_per_call: Optional[float] = None) -> str:
    """Select the optimal Claude model based on requirements."""

    # Start with complexity-based selection
    model_map = {
        Complexity.SIMPLE: "claude-haiku-3-5-20241022",
        Complexity.MODERATE: "claude-sonnet-4-20250514",
        Complexity.COMPLEX: "claude-opus-4-20250514",
    }

    selected = model_map[complexity]

    # Override for latency constraints
    if latency_ms and latency_ms < 1000:
        selected = "claude-haiku-3-5-20241022"

    # Override for budget constraints
    if budget_per_call and budget_per_call < 0.001:
        selected = "claude-haiku-3-5-20241022"
    elif budget_per_call and budget_per_call < 0.01:
        if complexity != Complexity.COMPLEX:
            selected = "claude-sonnet-4-20250514"

    return selected

# Examples
print(select_model(Complexity.SIMPLE))
# claude-haiku-3-5-20241022

print(select_model(Complexity.MODERATE))
# claude-sonnet-4-20250514

print(select_model(Complexity.COMPLEX, latency_ms=500))
# claude-haiku-3-5-20241022 (latency override)

typescript

import Anthropic from "@anthropic-ai/sdk";

enum Complexity {
  SIMPLE = 1,    // Pattern matching
  MODERATE = 2,  // Reasoning required
  COMPLEX = 3,   // Deep analysis
}

function selectModel(
  complexity: Complexity,
  latencyMs?: number,
  budgetPerCall?: number
): string {
  const modelMap: Record<Complexity, string> = {
    [Complexity.SIMPLE]: "claude-haiku-3-5-20241022",
    [Complexity.MODERATE]: "claude-sonnet-4-20250514",
    [Complexity.COMPLEX]: "claude-opus-4-20250514",
  };

  let selected = modelMap[complexity];

  // Override for latency constraints
  if (latencyMs && latencyMs < 1000) {
    selected = "claude-haiku-3-5-20241022";
  }

  // Override for budget constraints
  if (budgetPerCall && budgetPerCall < 0.001) {
    selected = "claude-haiku-3-5-20241022";
  } else if (budgetPerCall && budgetPerCall < 0.01) {
    if (complexity !== Complexity.COMPLEX) {
      selected = "claude-sonnet-4-20250514";
    }
  }

  return selected;
}

console.log(selectModel(Complexity.SIMPLE));
// claude-haiku-3-5-20241022

Dimension 2: Latency Requirements

Response time matters more than most developers realize. Here's what each model typically delivers:

Model	Time to First Token	Full Response (500 tokens)
Haiku 3.5	~200ms	~1.5s
Sonnet 4	~400ms	~3s
Opus 4	~800ms	~8s

These are approximate values. Actual latency depends on prompt length, output length, server load, and region.

Choose one task you repeat often.
Run it with the model, cost, or performance setting discussed in this lesson.
Record latency, quality, and cost so you can choose intentionally next time.

</TryThis>

Latency-Critical Use Cases:

Autocomplete/typeahead -> Haiku (users expect <500ms)
Chat interfaces -> Sonnet with streaming (users tolerate 1-3s for first token)
Background processing -> Opus is fine (users aren't waiting)
Real-time moderation -> Haiku (must be faster than the content it's moderating)

Python

import anthropic
import time

client = anthropic.Anthropic()

def benchmark_model(model: str, prompt: str,
                     runs: int = 3) -> dict:
    """Benchmark a model's latency over multiple runs."""
    times = []

    for _ in range(runs):
        start = time.time()
        response = client.messages.create(
            model=model,
            max_tokens=500,
            messages=[{"role": "user", "content": prompt}]
        )
        elapsed = time.time() - start
        times.append(elapsed)

    return {
        "model": model,
        "avg_seconds": round(sum(times) / len(times), 2),
        "min_seconds": round(min(times), 2),
        "max_seconds": round(max(times), 2),
    }

# Benchmark all three models
prompt = "Explain recursion in 3 sentences."

for model in [
    "claude-haiku-3-5-20241022",
    "claude-sonnet-4-20250514",
    "claude-opus-4-20250514"
]:
    result = benchmark_model(model, prompt)
    print(f"{result['model']}: avg={result['avg_seconds']}s")

Dimension 3: Cost Analysis

Understanding cost requires understanding tokens. Here's the current pricing:

Model	Input (per 1M tokens)	Output (per 1M tokens)
Haiku 3.5	$0.80	$4.00
Sonnet 4	$3.00	$15.00
Opus 4	$15.00	$75.00

To make this concrete, let's calculate costs for a real scenario: processing 10,000 customer support tickets per day, where each ticket averages 500 input tokens and 300 output tokens.

Python

def calculate_daily_cost(
    tickets_per_day: int,
    avg_input_tokens: int,
    avg_output_tokens: int
) -> dict:
    """Calculate daily API cost for each model."""

    pricing = {
        "Haiku 3.5": {"input": 0.80, "output": 4.00},
        "Sonnet 4": {"input": 3.00, "output": 15.00},
        "Opus 4": {"input": 15.00, "output": 75.00},
    }

    results = {}
    for model, prices in pricing.items():
        total_input = tickets_per_day * avg_input_tokens
        total_output = tickets_per_day * avg_output_tokens

        input_cost = (total_input / 1_000_000) * prices["input"]
        output_cost = (total_output / 1_000_000) * prices["output"]

        results[model] = {
            "daily": round(input_cost + output_cost, 2),
            "monthly": round((input_cost + output_cost) * 30, 2),
            "per_ticket": round(
                (input_cost + output_cost) / tickets_per_day, 6
            ),
        }

    return results

costs = calculate_daily_cost(
    tickets_per_day=10_000,
    avg_input_tokens=500,
    avg_output_tokens=300
)

for model, cost in costs.items():
    print(f"{model}:")
    print(f"  Daily: ${cost['daily']}")
    print(f"  Monthly: ${cost['monthly']}")
    print(f"  Per ticket: ${cost['per_ticket']}")

# Output:
# Haiku 3.5:
#   Daily: $16.00
#   Monthly: $480.00
#   Per ticket: $0.0016
# Sonnet 4:
#   Daily: $60.00
#   Monthly: $1800.00
#   Per ticket: $0.006
# Opus 4:
#   Daily: $300.00
#   Monthly: $9000.00
#   Per ticket: $0.03

typescript

function calculateDailyCost(
  ticketsPerDay: number,
  avgInputTokens: number,
  avgOutputTokens: number
) {
  const pricing: Record<string, { input: number; output: number }> = {
    "Haiku 3.5": { input: 0.80, output: 4.00 },
    "Sonnet 4": { input: 3.00, output: 15.00 },
    "Opus 4": { input: 15.00, output: 75.00 },
  };

  const results: Record<string, {
    daily: number; monthly: number; perTicket: number;
  }> = {};

  for (const [model, prices] of Object.entries(pricing)) {
    const totalInput = ticketsPerDay * avgInputTokens;
    const totalOutput = ticketsPerDay * avgOutputTokens;
    const inputCost = (totalInput / 1_000_000) * prices.input;
    const outputCost = (totalOutput / 1_000_000) * prices.output;
    const total = inputCost + outputCost;

    results[model] = {
      daily: Math.round(total * 100) / 100,
      monthly: Math.round(total * 30 * 100) / 100,
      perTicket: Math.round((total / ticketsPerDay) * 1e6) / 1e6,
    };
  }
  return results;
}

const costs = calculateDailyCost(10_000, 500, 300);
for (const [model, cost] of Object.entries(costs)) {
  console.log(`${model}: $${cost.daily}/day, $${cost.monthly}/month`);
}

The Decision Flowchart

Here's the practical flowchart I use for every new task:

START
  |
  v
Is the task simple classification,
extraction, or routing?
  |
  +-- YES -> Haiku
  |
  v
Does the task require deep reasoning,
creativity, or high-stakes judgment?
  |
  +-- YES -> Can you afford Opus pricing?
  |          |
  |          +-- YES -> Opus
  |          |
  |          +-- NO -> Sonnet + extended thinking
  |
  v
Sonnet (default for everything else)

Benchmarking Your Specific Use Case

Generic benchmarks are useful for orientation, but your specific use case may behave differently. Here's how to run your own evaluation:

<ComprehensionCheck

question="What is the main benefit of using Choosing the Right Model well in Claude Code?"

optA="Treat Choosing the Right Model as a one-time setup and assume it will keep working without any further review."

optB="Choose the model or workflow for Choosing the Right Model based on the task's actual complexity, latency needs, and cost."

optC="Use Choosing the Right Model mainly to avoid reading the project itself, since direct context is less important than fast output."

correctId="b"

explanation="Choose the model or workflow for Choosing the Right Model based on the task's actual complexity, latency needs, and cost. That is the pattern the lesson is pushing you toward."

Python

import anthropic
import json
import time
from typing import Callable

client = anthropic.Anthropic()

def evaluate_models(
    test_cases: list[dict],
    evaluator: Callable[[str, str], float],
    models: list[str] = None
) -> dict:
    """Evaluate multiple models against test cases.

    Args:
        test_cases: List of {"input": str, "expected": str}
        evaluator: Function(response, expected) -> score (0-1)
        models: Models to test (defaults to all three)

    Returns:
        Results per model with quality and performance metrics
    """
    if models is None:
        models = [
            "claude-haiku-3-5-20241022",
            "claude-sonnet-4-20250514",
            "claude-opus-4-20250514",
        ]

    results = {}

    for model in models:
        scores = []
        latencies = []
        total_input_tokens = 0
        total_output_tokens = 0

        for case in test_cases:
            start = time.time()
            response = client.messages.create(
                model=model,
                max_tokens=1024,
                messages=[{
                    "role": "user",
                    "content": case["input"]
                }]
            )
            elapsed = time.time() - start

            response_text = response.content[0].text
            score = evaluator(response_text, case["expected"])

            scores.append(score)
            latencies.append(elapsed)
            total_input_tokens += response.usage.input_tokens
            total_output_tokens += response.usage.output_tokens

        results[model] = {
            "avg_quality": round(sum(scores) / len(scores), 3),
            "avg_latency": round(
                sum(latencies) / len(latencies), 2
            ),
            "total_input_tokens": total_input_tokens,
            "total_output_tokens": total_output_tokens,
        }

    return results

# Example: Evaluate sentiment classification
test_cases = [
    {"input": "I love this product!", "expected": "positive"},
    {"input": "Terrible experience.", "expected": "negative"},
    {"input": "It works as described.", "expected": "neutral"},
    {"input": "Best purchase ever!!!", "expected": "positive"},
    {"input": "Waste of money.", "expected": "negative"},
]

def sentiment_evaluator(response: str, expected: str) -> float:
    """Score 1.0 if the expected sentiment appears in response."""
    return 1.0 if expected.lower() in response.lower() else 0.0

results = evaluate_models(test_cases, sentiment_evaluator)

for model, metrics in results.items():
    print(f"\n{model}:")
    print(f"  Quality: {metrics['avg_quality']}")
    print(f"  Latency: {metrics['avg_latency']}s")
    print(f"  Tokens: {metrics['total_input_tokens']}in / "
          f"{metrics['total_output_tokens']}out")

---

Apply: Building a Model Selection Matrix

For a real project, create a spreadsheet or document that maps every API call in your application to a model. Here's the template:

Python

# Model selection matrix for a customer support platform

SELECTION_MATRIX = {
    "ticket_classification": {
        "model": "claude-haiku-3-5-20241022",
        "reason": "Simple categorization, high volume, latency-sensitive",
        "expected_volume": "50,000/day",
        "quality_threshold": 0.95,
        "fallback_model": "claude-sonnet-4-20250514",
    },
    "response_drafting": {
        "model": "claude-sonnet-4-20250514",
        "reason": "Needs good writing quality, moderate complexity",
        "expected_volume": "50,000/day",
        "quality_threshold": 0.90,
        "fallback_model": "claude-opus-4-20250514",
    },
    "escalation_analysis": {
        "model": "claude-opus-4-20250514",
        "reason": "Complex cases needing nuanced judgment",
        "expected_volume": "2,000/day",
        "quality_threshold": 0.95,
        "fallback_model": None,
    },
    "sentiment_tracking": {
        "model": "claude-haiku-3-5-20241022",
        "reason": "Simple classification, background job",
        "expected_volume": "50,000/day",
        "quality_threshold": 0.90,
        "fallback_model": None,
    },
    "knowledge_base_search": {
        "model": "claude-sonnet-4-20250514",
        "reason": "Needs to understand query intent and rank results",
        "expected_volume": "10,000/day",
        "quality_threshold": 0.85,
        "fallback_model": None,
    },
}

def get_model_for_task(task: str) -> str:
    """Get the configured model for a specific task."""
    config = SELECTION_MATRIX.get(task)
    if not config:
        # Default to Sonnet for unconfigured tasks
        return "claude-sonnet-4-20250514"
    return config["model"]

def estimate_monthly_cost() -> float:
    """Estimate total monthly cost across all tasks."""
    pricing = {
        "claude-haiku-3-5-20241022": {"input": 0.80, "output": 4.00},
        "claude-sonnet-4-20250514": {"input": 3.00, "output": 15.00},
        "claude-opus-4-20250514": {"input": 15.00, "output": 75.00},
    }

    # Assume average 500 input, 300 output tokens per call
    avg_input = 500
    avg_output = 300
    total = 0.0

    for task, config in SELECTION_MATRIX.items():
        volume_str = config["expected_volume"].replace(",", "").split("/")[0]
        daily_volume = int(volume_str)
        monthly_volume = daily_volume * 30

        model = config["model"]
        prices = pricing[model]

        input_cost = (monthly_volume * avg_input / 1_000_000) * prices["input"]
        output_cost = (monthly_volume * avg_output / 1_000_000) * prices["output"]

        task_cost = input_cost + output_cost
        total += task_cost
        print(f"  {task}: ${task_cost:,.2f}/month ({model.split('-')[1]})")

    return total

print("Monthly Cost Breakdown:")
total = estimate_monthly_cost()
print(f"\nTotal: ${total:,.2f}/month")

Build your own selection matrix for a project you're working on:

List every place your app calls the Claude API (or will call it).
For each call, classify the complexity (Level 1/2/3 from above).
Note the expected daily volume and latency requirements.
Calculate the monthly cost for all-Opus vs. optimized model selection.
The difference is your potential savings -- often 60-80%.

</TryThis>

Same classification task, different models:

Prompt: "Classify this email as spam or not-spam: 'Congratulations! You've been selected for a $1000 gift card. Click here to claim.'"

Haiku (0.3s, $0.0004): "spam" -- Fast, correct, minimal.

Sonnet (1.1s, $0.003): "This email is spam. It uses urgency tactics ('Congratulations!'), an unrealistic offer ($1000 gift card), and a suspicious call to action ('Click here to claim')." -- Correct with explanation.

Opus (3.2s, $0.015): Same quality as Sonnet but 3x slower and 5x more expensive. For this task, Sonnet and Haiku are clearly better choices.

The extra capability of Opus adds nothing for well-defined classification tasks.

</PromptCompare>

---

Reflect: Making the Decision

The golden rule of model selection: Start with Sonnet for everything. Benchmark. Then promote to Opus only where quality measurably improves, and demote to Haiku only where quality doesn't meaningfully drop. Let data drive the decision, not assumptions.

</Callout>

Beware the "Opus for everything" trap. It's comfortable to use the most capable model because you never have to worry about quality. But at scale, this comfort costs 5-20x more than necessary. And in many cases, Opus isn't actually better -- it's just slower and more expensive. Always benchmark your specific tasks.

</Callout>

Questions to ask during your next architecture review:

Have I classified every API call by complexity? Many applications have a mix of simple and complex tasks. Treating them all the same wastes money.

Do I have fallback models configured? If Opus is down or rate-limited, can your system gracefully fall back to Sonnet?

Am I measuring quality per model per task? Without metrics, you're guessing. Build automated evaluations and run them weekly.

Is my team aligned on the decision framework? If different engineers pick models based on gut feel, you'll end up with inconsistent costs and quality.

Model selection has three dimensions: quality, speed, and cost -- no model wins on all three
Classify tasks by complexity: Level 1 (Haiku) for pattern matching, Level 2 (Sonnet) for reasoning, Level 3 (Opus) for deep analysis
Start with Sonnet as your default -- promote to Opus or demote to Haiku based on benchmarks, not assumptions
Build a model selection matrix that maps every API call to a specific model with documented reasoning
Always benchmark your specific use case -- generic benchmarks don't capture your task's unique characteristics
Configure fallback models for resilience -- if your primary model is unavailable, the system should degrade gracefully
The cost difference between all-Opus and optimized model selection is typically 60-80% at scale
Review quarterly -- as models improve, a task that needed Opus last quarter might work fine with Sonnet today

</KeyTakeaways>

---

Next lesson: We'll dive into the financial side -- pricing tiers, rate limits, token economics, and how to estimate and control your API costs.