1.1. The Data Pipeline Mental Model

💡 First Principle: Every data engineering problem is fundamentally about moving data from where it's created to where it's useful — and transforming it along the way so that someone (or something) can make a decision from it.

Think of a data pipeline like a water treatment system. Raw water enters from various sources — rivers, rain, wells — and each source has different contaminants and flow rates. The treatment plant doesn't just move water; it cleans, filters, tests, and routes it to the right destination. Without this system, you either have no water where you need it, or you have contaminated water that's worse than useless.

Data engineering works the same way. Without pipelines, your organization's data sits trapped in the databases, APIs, and log files where it was created. Sales data stays in the CRM. Clickstream data stays in the web server logs. IoT sensor readings pile up and expire. What happens when the CFO needs a dashboard that combines all three? Without a pipeline, someone manually exports CSVs at 6 AM — and the data is already stale by the time anyone looks at it.

The exam tests whether you can design these systems on AWS — choosing the right services for ingestion, transformation, storage, analysis, and governance. Every question ultimately maps back to this mental model.