2.4.5. Preparing Documents for RAG: Parsing, Chunking, and Metadata
💡 First Principle: A chunk is the unit of retrieval. It must be small enough that its embedding represents one idea precisely, and large enough that the passage still answers a question on its own. Every chunking strategy is a different way of balancing those two pressures.
Chunking strategies in Amazon Bedrock Knowledge Bases:
| Strategy | How it splits | Choose when | Trade-off |
|---|---|---|---|
| Default | About 300 tokens, honouring sentence boundaries | General prose with no special structure | Simple, may split related sentences across chunks |
| Fixed-size | You set maxTokens and an overlap percentage | Uniform documents where you want predictable chunk sizes | Overlap repeats text (more storage) to protect context at boundaries |
| Hierarchical | Parent and child chunk sizes plus overlap tokens; retrieves precise child chunks, then returns their broader parent chunks | Long structured documents where answers need surrounding context | Can return fewer results than requested; not recommended with S3 Vectors |
| Semantic | Splits where meaning changes, using a maximum token size, a buffer size (neighbouring sentences considered) and a breakpoint percentile threshold | Documents whose topics shift irregularly (transcripts, reports) | Uses a foundation model during ingestion, which adds cost |
| No chunking | Each file is one chunk | Files already pre-split upstream, or FAQ-style one-answer files | You lose page-number citations and page-number filtering |
| Custom Lambda | A transformation Lambda function (POST_CHUNKING step) writes your own chunks, or adds chunk-level metadata to a built-in strategy, using an intermediate S3 bucket | Domain rules no built-in strategy supports (split legal contracts by clause) | You own and maintain the code |
Two facts worth remembering: the chunking strategy cannot be changed after a data source is connected (changing it means creating a new data source and re-ingesting), and for parsed content the chunker respects logical boundaries such as pages or sections even when a larger token size would allow merging across them.
Tuning semantic chunking. A higher breakpoint percentile threshold requires sentences to be more dissimilar before a split, which produces fewer, larger chunks. A larger buffer size considers more neighbouring sentences when measuring dissimilarity — more context, but potentially more noise.
Metadata extraction. Metadata lets retrieval filter before or alongside similarity ranking (only this product line, only documents after 2025, only the caller's department). For an S3 data source you attach metadata with a sidecar file:
- Named after the source file with
.metadata.jsonappended (policy.pdf.metadata.json), stored in the same folder as the source, and no larger than 10 KB. - Attribute types:
STRING,NUMBER,BOOLEAN,STRING_LIST. includeForEmbedding: trueprepends the key-value pair to the chunk text before embedding, so queries mentioning that value score higher; withfalse(or the simplified{"metadataAttributes": {"tag": "value"}}format) the metadata is stored for filtering only.- For CSV files, a record-based
fileName.csv.metadata.jsondeclares one content field to chunk and embed and which other columns become metadata; each row becomes its own chunk, and CSV-derived metadata values are stored as strings. - Chunk-level metadata written by a custom transformation Lambda overrides file-level metadata when keys collide.
Metadata can be extracted automatically during preparation: document properties (author, date), folder or S3 prefix conventions (department, region), entities detected with Amazon Comprehend, or classifications produced by an FM parser. Extracting it once at ingestion is far cheaper than asking the generating model to infer it on every query.
⚠️ Exam Trap: When users complain that answers cite outdated or out-of-scope documents, the fix is usually metadata plus a retrieval filter, not a bigger chunk size or a different generating model. When answers are correct but missing surrounding context, look at hierarchical chunking or overlap. When table values come back garbled, look at the parser (BDA or FM parser), not at chunking.
