4.1.2. Embeddings and Chunking Strategy
💡 First Principle: Chunk size is a tradeoff, not a constant. Chunks too large dilute embedding precision and waste context; too small lose the surrounding meaning a passage needs. The right size depends on the content structure and the query type — which is why Azure offers both fixed-size and structure-aware chunking.
Embeddings convert text to vectors using an embedding model (e.g., a text-embedding deployment) — the same model must embed both the indexed chunks and the query, or the vectors aren't comparable. On chunking, Azure AI Search supports two skill-based approaches: Text Split skill (fixed-size chunking by token/character count with configurable overlap — simple, predictable, cheap) and Document Layout skill (structure-aware chunking that uses the Document Intelligence layout model to chunk on headings, paragraphs, and semantic coherence — better relevance for structured documents). Overlap between chunks preserves context that would otherwise be cut at a boundary.
| Approach | How it splits | Best for |
|---|---|---|
| Text Split (fixed-size) | Token/character count + overlap | Uniform prose, cost-sensitive, predictable |
| Document Layout (structure-aware) | Headings, paragraphs, semantic units | Structured docs (manuals, reports, forms) |
⚠️ Exam Trap: "Bigger chunks give the model more context, so they're better." Oversized chunks dilute the embedding (one vector trying to represent too many ideas) and crowd the context window, lowering retrieval precision. Chunk size is tuned to the data, not maximized — and using the wrong embedding model for the query versus the index breaks retrieval entirely.
Reflection Question: Why must the query be embedded with the same model used to embed the indexed chunks, and what happens to retrieval if they differ?
