What is Chunking in AI? — Plain-Language Definition

What is Chunking?

Chunking is the process of breaking a large document into smaller sections before storing it for search or retrieval. Instead of treating a long PDF, handbook, or article as one giant block, the system divides it into manageable pieces called chunks.

Chunking is a core step in most RAG pipelines because language models work better when they receive small, relevant passages instead of entire documents full of unrelated material.

Why It Matters

If your chunks are too large, retrieval becomes noisy. A chunk may contain the right answer, but also a lot of unrelated text. If your chunks are too small, important context can get lost.

Good chunking helps a system do three things well:

• retrieve the most relevant section
• preserve enough context to make the section understandable
• reduce wasted tokens in the final prompt

Common Chunking Strategies

• Fixed-size chunking: split every N tokens or characters
• Sentence or paragraph chunking: break on natural text boundaries
• Section-aware chunking: preserve headings, lists, and document structure
• Overlapping chunks: repeat a small amount of text between chunks to avoid cutting context too aggressively

Real-World Example

A 40-page employee handbook might be split by section so that a search for "parental leave eligibility" returns the leave-policy passage instead of an entire chapter on benefits.

Key Takeaway

Chunking is not just a technical preprocessing step. It directly affects search quality, prompt efficiency, and answer quality in AI retrieval systems.