3.2.3. Search Architecture: Semantic, Keyword, and Hybrid

💡 First Principle: Semantic search finds meaning-similar content; keyword search finds exact term matches. Neither is universally superior — semantic search dominates for conceptual queries; keyword search dominates for entity lookups (specific product codes, proper names, regulatory citation numbers). Hybrid search combines both to handle mixed query patterns.

Search type performance by query category:

Hybrid search implementation in OpenSearch:

# Step 1 (once): PUT /_search/pipeline/hybrid-pipeline
# BM25 and k-NN scores sit on different scales, so a normalization processor
# rescales each sub-query's scores, then combines them using these weights.
hybrid_pipeline = {
    "phase_results_processors": [
        {
            "normalization-processor": {
                "normalization": {"technique": "min_max"},
                "combination": {
                    "technique": "arithmetic_mean",
                    "parameters": {"weights": [0.3, 0.7]}  # keyword, semantic; same order as queries, sum = 1.0
                }
            }
        }
    ]
}

# Step 2 (every search): POST /<index>/_search?search_pipeline=hybrid-pipeline
hybrid_query = {
    "query": {
        "hybrid": {
            "queries": [
                {"match": {"content": {"query": user_query}}},  # BM25 keyword scoring
                {"knn": {"embedding": {"vector": query_embedding, "k": 10}}}  # Dense vector
            ]
        }
    }
}

The keyword/semantic mix is set by the pipeline's weights, not by a boost inside each sub-query: min_max normalization rescales every sub-query's scores to the 0–1 range before combining them, so a boost applied to a whole sub-query is normalized away. The hybrid query depends on that pipeline to combine the sub-query scores into one.

Boosting a specific field: the keyword side can also weight fields differently. Store code snippets or identifiers in their own text field and give that field a higher BM25 boost (for example, multi_match over ["code_content^3", "content"]), so an exact-syntax query such as an API name matches the code block rather than prose that only mentions it.

Reranking — the retrieval quality multiplier: After initial retrieval (which returns top-k candidates), a reranker model re-scores the candidates based on their actual relevance to the specific query — not just their general similarity. The usual pattern is to retrieve a generous k for recall, then rerank down to the few best chunks to restore precision. Reranking is a second model call, so whether its added latency is acceptable depends on the application's SLA. There is no universal threshold. Bedrock provides managed reranker models:

# Reranking retrieved chunks using Bedrock reranker
reranked = bedrock_agent_runtime.rerank(
    rerankingConfiguration={
        'type': 'BEDROCK_RERANKING_MODEL',
        'bedrockRerankingConfiguration': {
            'modelConfiguration': {
                'modelArn': 'arn:aws:bedrock:us-east-1::foundation-model/amazon.rerank-v1:0'
            },
            'numberOfResults': 3  # Return top 3 after reranking
        }
    },
    sources=[{'type': 'INLINE', 'inlineDocumentSource': 
              {'type': 'TEXT', 'textDocument': {'text': chunk}}} 
             for chunk in retrieved_chunks],
    queries=[{'type': 'TEXT', 'textQuery': {'text': user_query}}]
)

⚠️ Exam Trap: Hybrid search is not always better than pure vector search. For purely conceptual queries where no specific terms matter, hybrid search adds BM25 noise that can actually reduce precision. The exam tests whether you understand when to use hybrid (mixed entity + concept queries) versus when pure semantic search is sufficient.

Reflection Question: Your RAG system retrieves excellent results for general questions like "how does our leave policy work?" but fails for specific queries like "is policy HR-2024-Q3 still in effect?" What search architecture change would you implement, and how would you configure the weighting between the two search components?