4.2.1. Vector, Keyword, Hybrid, and Semantic Ranking
💡 First Principle: The three strategies operate at different points and answer different questions. Keyword (BM25) asks "do the exact terms appear?" Vector asks "is this conceptually similar?" Hybrid runs both and fuses results (Reciprocal Rank Fusion). Semantic ranking then re-scores the top results with a cross-encoder that reads query and chunk together. Each adds a distinct kind of relevance.
The key architectural distinction: vector search uses a bi-encoder (query and document embedded separately, then compared), which is fast but can miss nuance like negation. The semantic ranker uses a cross-encoder (query and chunk processed together), which captures subtler relationships but is applied only to a small re-ranking set because it's costlier. Hybrid search via RRF gives maximum recall by combining keyword and vector candidates; semantic ranking improves precision at the top of the list.
| Strategy | Matches on | Mechanism | Catches |
|---|---|---|---|
| Keyword (BM25) | Exact terms | Lexical | Codes, IDs, acronyms |
| Vector | Meaning | Bi-encoder similarity | Paraphrase, concepts |
| Hybrid | Both | RRF fusion | Maximum recall |
| Semantic ranking | Re-scored top set | Cross-encoder | Precision, nuance, negation |
⚠️ Exam Trap: Don't pick pure vector search for a RAG requirement that includes exact identifiers (part numbers, employee IDs). Vector similarity can miss exact tokens; the requirement-correct answer is hybrid (keyword + vector), and semantic ranking on top if precision at the top matters. Choosing pure vector "because it's the AI way" is the trap.
Reflection Question: A query is "what is the copay for plan HMO-203?" Explain why hybrid search outperforms pure vector here, and what the semantic ranker adds beyond hybrid.
