2.3.1. Event-Driven Ingestion with EventBridge

💡 First Principle: EventBridge is the central nervous system of event-driven architectures on AWS. It routes events from sources (S3, DynamoDB, custom applications) to targets (Lambda, Step Functions, Glue, SQS) based on rules that match event patterns — like a sophisticated mail sorting facility that reads the label and routes each package to the right destination.

EventBridge receives events from over 90 AWS services and supports custom events from your applications. For data engineering, the most common patterns are:

S3 → EventBridge → Processing. When a file arrives in S3, EventBridge receives the event and can filter on bucket name, key prefix, key suffix, and object size. A rule matches the event and triggers a target — Lambda for lightweight processing, Step Functions for multi-step workflows, or Glue to start an ETL job. This is more flexible than S3 Event Notifications directly to Lambda because EventBridge supports complex filtering and multiple targets from a single event.

Schedule-based triggers. EventBridge Scheduler (formerly CloudWatch Events schedules) supports cron expressions and rate expressions. Use it to trigger Lambda functions, Step Functions, or ECS tasks on a fixed schedule. For data pipelines, this replaces traditional cron jobs with a managed, serverless scheduler.

Cross-account and cross-region. EventBridge can route events between AWS accounts and regions, enabling centralized data pipeline orchestration for multi-account architectures.

⚠️ Exam Trap: EventBridge rules can have up to 5 targets per rule. If a question describes a single event needing to trigger 10 different processes, you need multiple rules or an intermediate target (SNS/SQS) that fans out. Also, EventBridge has an event size limit of 256 KB — large payloads must be stored in S3 with only a reference passed through EventBridge.

Reflection Question: New data files arrive in S3 at unpredictable intervals. Some are CSV (need Glue processing) and some are Parquet (ready to query directly). How would you use EventBridge rules to route these differently?