3.4.3. RAG Architecture Patterns and Retrieval Optimization
💡 First Principle: In a RAG system the model can only be as good as what it is handed. Most "hallucination" complaints in RAG are retrieval failures — the right passage was never retrieved, or it was buried among irrelevant ones. So you design and tune the retrieval pipeline (chunking, embeddings, search type, filtering, reranking) as carefully as you would design features for a traditional model.
Choosing a RAG pattern:
| Pattern | How it works on AWS | Use when |
|---|---|---|
| Fully managed RAG | Amazon Bedrock Managed Knowledge Base — Bedrock runs the datastore, embedding and reranking models, with native connectors (S3, SharePoint, Confluence, Web Crawler, Google Drive, OneDrive, custom); query it with Retrieve (always hybrid search) or with agentic retrieval, which can also generate a grounded answer with citations | You want the least operational overhead and many enterprise sources |
| Customer-managed knowledge base | Amazon Bedrock Knowledge Bases on a vector store you choose and operate (OpenSearch, Aurora PostgreSQL, S3 Vectors, Neptune Analytics, third-party stores) | You need control over the vector store, chunking strategy or index configuration |
| Retrieve-only + your own generation | Knowledge Bases Retrieve API, then your own prompt to any model | You need control over the prompt, the model or post-processing |
| Custom pipeline | Your own chunking and embedding code writing to OpenSearch Service, Aurora PostgreSQL (pgvector) or S3 Vectors | Unusual parsing, custom ranking logic, or an existing search platform |
| GraphRAG | Knowledge Bases with an Amazon Neptune Analytics graph | Questions span relationships between entities across documents |
| Structured data retrieval | Knowledge Bases connected to a structured data store, generating SQL | Answers live in tables, not prose |
| Multimodal RAG | Multimodal embeddings or a parser that converts audio, video and images to text first | The corpus includes images, scanned pages, audio or video |
Managed vs. customer-managed: a Managed Knowledge Base removes the vector-store decision entirely — Bedrock provisions and scales the datastore, supplies managed embedding and reranking models by default, and always retrieves with hybrid (keyword plus semantic) search. A customer-managed knowledge base exposes every knob below, and its RetrieveAndGenerate API returns generated answers with citations. From 30 September 2026, new Confluence, SharePoint, Salesforce and Web Crawler connectors can no longer be created on customer-managed knowledge bases (existing ones keep working); AWS points those use cases to the Managed Knowledge Base.
Choosing a vector store for a customer-managed knowledge base is a trade-off between query performance, features and cost. Hybrid search is supported on Amazon OpenSearch Serverless, Amazon RDS (Aurora PostgreSQL) and MongoDB vector stores that contain a filterable text field; OpenSearch Serverless and OpenSearch Service managed clusters are the only stores that hold binary vectors; Amazon Aurora PostgreSQL uses pgvector; Amazon S3 Vectors gives low-cost, durable vector storage with sub-second queries and is aimed at workloads queried infrequently.
Retrieval optimization levers (the full set applies to customer-managed knowledge bases; a Managed Knowledge Base offers default or fixed-size chunking and manages embedding and reranking unless you override them):
| Lever | Options in Bedrock Knowledge Bases | Effect |
|---|---|---|
| Chunking | Default (~300 tokens, sentence boundaries), fixed-size (max tokens + overlap %), hierarchical (parent/child), semantic (meaning boundaries), none | Small chunks embed precisely; large chunks carry context. Hierarchical retrieves precise child chunks and returns their broader parents |
| Embedding model | Titan Text Embeddings V2 (1,024 default, 512 or 256 dimensions; up to 8,192 tokens), Cohere Embed, multimodal embeddings | Domain fit and language coverage drive recall; fewer dimensions cut storage and cost at some accuracy cost |
| Search type | Semantic (vectors only) or hybrid (vectors + raw text) | Hybrid helps queries with exact identifiers, product codes or rare terms |
| Number of results | numberOfResults (default 5) | Too few misses evidence; too many adds noise and tokens |
| Metadata filtering | Manual filters (equals, greaterThan, in, ...), implicit filtering generated from the query | Restricts retrieval to the right product line, date range or tenant |
| Reranking | Reranker model (for example Cohere Rerank 3.5) applied after retrieval | Reorders chunks by relevance so fewer, better chunks reach the model — lower cost and latency |
| Query decomposition | Breaks a multi-part question into sub-queries | Improves recall on compound questions |
Embedding-model rules that break systems when forgotten: the vector index dimension must match the embedding model's output dimension, and documents and queries must be embedded with the same model. Changing the embedding model means re-embedding and re-indexing the whole corpus.
⚠️ Exam Trap: Semantic chunking and query decomposition cost more — semantic chunking invokes a foundation model during ingestion, and decomposition can run several queries per question. If the stem optimizes for least cost, they are rarely the answer; if it optimizes for retrieval accuracy on complex documents or questions, they often are.
⚠️ Exam Trap: When users search for exact part numbers or error codes and pure vector search misses them, the fix is hybrid search (keyword + semantic), not a bigger generator model. The generator cannot use a chunk that was never retrieved.
