3.6.2. Retrieval-Augmented Generation with Amazon Bedrock Knowledge Bases
💡 First Principle: RAG grounds a general model in private, current data at query time instead of changing the model. Answer quality is decided by ingestion (parsing, chunking, embeddings, metadata) and retrieval (search type, filters, reranking) as much as by the model.
Scenario: An insurer wants an assistant that answers from 2 million policy PDFs and claims manuals. Users often search for exact form numbers such as HO-3 2019, each business unit may see only its own documents, and the budget for the vector store is tight because queries are infrequent.
Ingestion. "Amazon Bedrock Knowledge Bases" connects to data sources (Amazon S3, a web crawler, Confluence, SharePoint, Salesforce and others), parses documents, splits them into chunks, converts each chunk to a vector with an embeddings model and writes it to a vector store. Syncing the data source re-ingests what changed. Advanced parsing (Bedrock Data Automation or a foundation model as parser) handles tables, figures and scanned pages that plain text extraction mangles. A "Bedrock Managed Knowledge Base" option (GA June 2026) also runs the vector storage for you.
| Chunking | Behavior | Use when |
|---|---|---|
| Default / fixed-size | ~300-token chunks, or a size and overlap you set | General documents |
| Hierarchical | Searches small child chunks, returns their larger parent | Precise matching but answers need surrounding context |
| Semantic | Splits where meaning changes (uses a model, extra cost) | Long documents with uneven structure |
| None + custom Lambda | You pre-split or transform content yourself | Specialized formats |
| Vector store | Strength | Limitation to remember |
|---|---|---|
| OpenSearch Serverless | Default quick-create, hybrid search, metadata filters | Capacity-based cost even when idle |
| Aurora PostgreSQL (pgvector) | Hybrid search, reuse existing relational skills | Must be in the same account as the knowledge base |
| Neptune Analytics | GraphRAG for relationship-heavy content | Graph capacity to size |
| Amazon S3 Vectors | Lowest cost, durable, sub-second queries for infrequent workloads | Semantic search only, float embeddings only, 1 KB / 35 keys of metadata per vector, hierarchical chunking not recommended |
| Pinecone, Redis Enterprise Cloud, MongoDB Atlas | Reuse an existing third-party store | Credentials in Secrets Manager, third-party terms |
Retrieval. Retrieve returns ranked chunks for your own prompt; RetrieveAndGenerate also generates a cited answer. Hybrid search combines vector similarity with keyword matching, so exact identifiers are found; it is supported only on Aurora/RDS, OpenSearch Serverless and MongoDB Atlas stores with a filterable text field. Metadata filtering (from .metadata.json files next to S3 objects) scopes results by business unit, tenant or date, and is how a pooled multi-tenant knowledge base keeps tenants apart. Reranking reorders the top results with a reranker model, and query decomposition splits a multi-part question into sub-queries.
⚠️ Exam Trap: When users search for exact codes, part numbers or error IDs, pure semantic search is the weak point. S3 Vectors is the cheapest store but semantic-only, so the right design for that requirement is a hybrid-capable store (OpenSearch Serverless or Aurora PostgreSQL). Picking S3 Vectors there trades away the stated requirement for cost.