2.1.4. The Change Feed Processor
💡 First Principle: The change feed turns your database into an event source: an ordered, replayable log of every insert and update per partition. Instead of downstream systems polling for what's new ("anything yet? anything yet?"), they subscribe and react. Any pattern that needs "when data changes, do X" — including "when a document arrives, generate its embedding" — builds on this.
For AI solutions, the change feed is how vector stores stay current. Documents land in Cosmos DB; the change feed announces them; a processor generates embeddings and writes them back (or into another store). No batch jobs, no missed documents.
The change feed processor pattern has four named parts the exam expects you to place: the monitored container (where changes happen), the lease container (a second container storing checkpoints and coordinating which instance owns which partition range), the host name (a unique instance identifier), and the delegate (your handler function that receives batches of changed items). Scaling out is elegant: start another instance with the same lease container, and partition leases redistribute automatically — more instances, more parallelism, no code changes.
Delivery is at-least-once: after a crash, the processor resumes from the last checkpoint, so your delegate may see an item twice and must be idempotent — a principle that returns with Event Grid in 4.1.2. In the Python SDK you consume the feed with query_items_change_feed() (the pull model); the full processor abstraction lives in the .NET/Java SDKs, but the architecture — leases, delegates, checkpoints — is what the exam tests.
⚠️ Exam Trap: The change feed's default (latest version) mode does not include deletes. If a scenario requires reacting to removals, the answer is soft delete — set a deleted: true flag (which appears in the feed as an update), optionally with TTL to purge the item later. A separate all versions and deletes mode exists, but it requires continuous backup and a bounded retention window, and it isn't what the processor pipeline consumes — any answer claiming the default feed emits delete events is wrong.
Reflection Question: Why does the change feed processor need a separate lease container rather than tracking progress in memory?
