Detail Levels and Quality

A startup building an automated inventory system learned about detail levels the hard way. They were processing 5,000 warehouse shelf photos daily at the default auto detail setting. The model consistently miscounted items on densely packed shelves -- reporting 23 items when there were 28, or missing small labels entirely. Their API bill was $180/day. When they switched to high detail, accuracy improved dramatically -- but the bill jumped to $620/day. When they audited their image set, they realized 60% of the photos were simple barcode scans that only needed low detail. By routing images intelligently -- barcodes at low, shelf overviews at high -- they got better accuracy at $290/day.

Detail level is not a quality slider you set once and forget. It is an engineering decision that affects accuracy, latency, and cost in ways that compound at scale.

The Four Detail Levels

Low Detail

The image is resized to fit within 512x512 pixels. The model receives the entire image as a single tile -- approximately 85 tokens.

{
  type: "image_url",
  image_url: {
    url: "https://example.com/icon.png",
    detail: "low"
  }
}

What it sees: A thumbnail-quality representation. Good enough to identify objects, colors, general composition, and large text. Not sufficient for small text, fine details, or dense content.

Token cost: ~85 tokens regardless of original image size. A 50KB icon and a 20MB photo both cost ~85 tokens at low detail.

Best for: Image classification, object identification, color analysis, simple scene description, thumbnail preview analysis, emoji/icon recognition.

High Detail

The image is resized so its longest edge is at most 2048 pixels (while maintaining aspect ratio). Then it is divided into 512x512 tiles, and each tile is encoded separately.

{
  type: "image_url",
  image_url: {
    url: "https://example.com/document.png",
    detail: "high"
  }
}

What it sees: Each tile gets individual attention, allowing the model to read small text, identify fine patterns, and analyze detailed visual content. A 2048x1536 image produces 12 tiles.

Token cost: 85 base + 170 per tile. A 12-tile image costs 85 + (12 x 170) = 2,125 tokens.

Best for: Document analysis, OCR tasks, screenshot interpretation, detailed photo analysis, chart reading, form processing.

Original Detail (gpt-5.4+ models)\n\nThe newest detail level for maximum visual fidelity. The image is scaled so its longest edge is at most 6,000 pixels, with up to 10,000 patches. Available only on GPT-5.4 and newer models.\n\ntypescript\n{\n type: \"image_url\",\n image_url: {\n url: \"https://example.com/dense-diagram.png\",\n detail: \"original\"\n }\n}\n\n\nWhat it sees: The highest-fidelity representation available. Ideal for dense technical diagrams, satellite imagery, high-resolution medical images, or any content where fine spatial detail matters at pixel level.\n\nToken cost: Significantly higher than high. The 10,000-patch budget means token costs can be substantial -- use only when high detail is insufficient.\n\nBest for: Dense technical diagrams, architectural blueprints, satellite/aerial imagery, high-resolution medical imaging, any task requiring maximum spatial precision.\n\n### Auto Detail

The model decides which detail level to use based on the image content and your prompt.

{
  type: "image_url",
  image_url: {
    url: "https://example.com/photo.jpg",
    detail: "auto"
  }
}

What it sees: Depends on the model's assessment. For simple images, it may choose low. For complex images with text, it typically chooses high.

Token cost: Unpredictable. This makes cost forecasting difficult for production systems.

How Tiling Works

Understanding the tiling algorithm helps you optimize image preprocessing:

Cost Comparison: A Real Scenario

Let's calculate costs for analyzing 1,000 product photos per day, each 3000x2000 pixels:

At 1,000 images/day, the difference is modest. At 100,000 images/day, it is $21 vs $361 -- and that is input tokens only, before counting the model's response tokens.

When to Use Each Level

Optimization Strategies for Production

Strategy 1: Route by Task

Different endpoints in your application likely need different detail levels. Configure them independently:

const DETAIL_BY_TASK = {
  classification: "low",
  content_moderation: "low",
  document_ocr: "high",
  screenshot_analysis: "high",
  product_categorization: "low",
  defect_detection: "high",\n  blueprint_analysis: "original",  // gpt-5.4+ only\n  medical_imaging: "original"     // gpt-5.4+ only
} as const;

Strategy 2: Preprocess Images

If you control the images, resize them before sending to avoid paying for unnecessary tiles:

// For low detail: resize to 512x512 max
// For high detail: resize to 2048px longest edge
// Never send a 6000x4000 raw camera photo
import sharp from "sharp";

async function preprocessForVision(
  buffer: Buffer,
  detail: "low" | "high"
): Promise<Buffer> {
  const maxDim = detail === "low" ? 512 : 2048;
  return sharp(buffer)
    .resize(maxDim, maxDim, { fit: "inside" })
    .jpeg({ quality: 85 })
    .toBuffer();
}

Strategy 3: Two-Pass Analysis

For complex workflows, use a cheap first pass to decide if a detailed second pass is needed:

// Pass 1: Low detail classification
const triage = await analyzeImage(imageUrl, "low",
  "Does this image contain text that needs to be read? Answer yes or no."
);

// Pass 2: Only use high detail when needed
if (triage.includes("yes")) {
  const detailed = await analyzeImage(imageUrl, "high",
    "Extract all text visible in this image."
  );
}

This two-pass approach can reduce costs by 40-60% in mixed workloads where only a subset of images require detailed analysis.