2.1. Streaming Data Ingestion

💡 First Principle: Streaming ingestion exists because some data loses its value with every passing second. Think of it like a stock ticker — yesterday's price is a historical fact, but the current price is what drives action. Streaming systems capture data the moment it's created and deliver it for processing before its value decays.

Without streaming ingestion, organizations that depend on real-time signals — fraud detection, IoT monitoring, live recommendation engines — are always reacting to the past. Imagine a credit card company that only checks for fraud in nightly batch runs: by morning, the damage is done. Streaming ingestion closes the gap between "data happened" and "we can act on it."

But streaming isn't free. It introduces complexity that batch systems don't have: ordering guarantees, exactly-once semantics, handling late-arriving data, scaling consumers to match producers, and managing backpressure when consumers fall behind. The exam tests whether you understand these trade-offs — and specifically, whether you can choose between Kinesis Data Streams, Kinesis Data Firehose, and Amazon MSK based on the scenario's requirements.

What's the key question to ask when an exam scenario describes incoming data? How fast does the consumer need to react? Sub-second latency demands Kinesis Data Streams or MSK. Near-real-time delivery (1–5 minutes) to S3 or Redshift suggests Firehose. Change data capture from a database points to DynamoDB Streams or DMS.