4.4.4. Amazon Bedrock Knowledge Bases and Retrieval Pipelines
💡 First Principle: RAG quality is decided at two moments: ingestion (how documents are parsed, chunked and embedded) and query time (how many chunks you fetch, whether you filter or rerank them, and how you phrase the search). Settings chosen at ingestion are expensive to change, so they deserve the most thought.
Vector store choice. A customer-managed knowledge base writes embeddings into a store you choose:
| Vector store | Why pick it | Notable limits |
|---|---|---|
| Amazon OpenSearch Serverless | Quick-create default; hybrid search; binary vectors | Index must use the faiss engine for metadata filtering |
| Amazon OpenSearch Service managed clusters | Existing domain, binary vectors | Domain must allow public access (VPC-only domains are not supported) |
| Amazon S3 Vectors | Lowest cost for infrequent queries at scale | Float vectors only; 1 KB custom metadata and 35 keys per vector; hierarchical chunking not recommended |
| Amazon Aurora PostgreSQL (pgvector) | Relational team, hybrid search | Cluster must be in the same account; GIN index needed on the text column |
| Neptune Analytics | GraphRAG over entity relationships | Vector index dimension set at graph creation |
| Pinecone, Redis Enterprise Cloud, MongoDB Atlas | Existing third-party investment | Credentials held in AWS Secrets Manager |
AWS also offers a Bedrock Managed Knowledge Base, where Bedrock provisions and scales the data store, embeddings and reranking for you; choose a customer-managed knowledge base when you need a specific vector database or direct access to it.
Ingestion settings you cannot change later. The chunking strategy is fixed once the data source is created, and the embeddings model and storage configuration are fixed once the knowledge base is created. Changing either means creating a new data source or knowledge base and re-ingesting.
| Chunking strategy | Behaviour | Choose when |
|---|---|---|
| Default | About 300 tokens, respects sentence boundaries | Getting started |
| Fixed-size | Your max tokens and overlap percentage | Uniform documents, tunable recall |
| Hierarchical | Child chunks are matched, parent chunks are returned | Precise matching but the model needs surrounding context |
| Semantic | Splits on meaning using buffer size and breakpoint percentile (uses an FM, extra cost) | Documents whose topics shift mid-section |
| None | Each file is one chunk | You pre-split files yourself |
Query-time configuration (Retrieve and RetrieveAndGenerate):
numberOfResultsdefaults to 5 chunks. With hierarchical chunking it counts child chunks, so you may receive fewer results once children collapse into parents.overrideSearchType:HYBRIDcombines vector and keyword search;SEMANTICuses vectors only. Hybrid works only on Amazon RDS, OpenSearch Serverless and MongoDB stores with a filterable text field; elsewhere the query falls back to semantic.- Metadata filtering (
equals,in,greaterThan,andAll/orAlland more) narrows results before generation; implicit filtering lets a model derive the filter from the question. - Reranking rescores retrieved chunks with a reranker model (Amazon Rerank 1.0 or Cohere Rerank 3.5) so the most relevant chunks reach the prompt.
- Query decomposition splits a multi-part question into sub-queries before retrieval.
- Structured data stores (for example Amazon Redshift) let a knowledge base turn natural language into SQL instead of searching vectors.
⚠️ Exam Trap: When answers cite the right document family but the wrong product version, the fix is metadata filtering, not a bigger numberOfResults. When the right chunk is retrieved but ranked sixth and never reaches the prompt, the fix is reranking. When exact part numbers or error codes are missed, the fix is hybrid search.
