2.4. Reflection Checkpoint

Three engines, one geometric idea, three cost models — the data layer is where AI-200 concentrates its weight, and where the anatomy from 1.1.1 gets its memory.

Key Takeaways

• Cosmos DB bills work in RUs: point-read what you can, scope queries to partitions, exclude unqueried paths (especially embeddings) from indexing, and pay for strong consistency only when everyone must see everything immediately.
• The vector trade-off is universal: flat/exact for small and precise, ANN (diskANN, HNSW, IVFFlat) for scale — every engine prices recall against speed, and the query-time knobs (ef_search, probes) are the recall dial.
• pgvector's power is composability: distance ORDER BY plus SQL WHERE gives filtered, tenant-aware RAG in one query — but operators must match index opclasses, IVFFlat must be built after data load, and connections must be pooled (PgBouncer, port 6432).
• Redis is short-term memory: cache-aside with atomic SET ... EX, invalidation matched to staleness tolerance, and semantic caching to skip repeated model calls — always rebuildable, never the system of record.
• The change feed is Cosmos DB's event source — inserts and updates only, never deletes; soft-delete flags are the workaround, and processors must be idempotent.

Connecting Forward

Phase 3 moves from where data lives to where code runs. The workers that consume change feeds, generate embeddings, and serve RAG APIs all ship as container images — so we take the compute spectrum from 1.2.1 and go deep: building and versioning images in Azure Container Registry, hosting on App Service, and orchestrating with Container Apps (KEDA scaling included) and AKS.

Self-Check Questions

• A RAG query returns correct results but got 40x slower after the team "optimized" it. Trace three distinct causes from this phase and name the one-line fix for each.
• Your architecture stores embeddings in PostgreSQL, caches hot answers in Redis, and streams new documents through the Cosmos DB change feed. For each store, what happens — and what should happen — when a source document is deleted?