2.7.3. Serverless Deployment with Lambda and SAM

💡 First Principle: AWS SAM (Serverless Application Model) is CloudFormation optimized for serverless applications. It reduces the boilerplate for defining Lambda functions, API Gateway endpoints, DynamoDB tables, and Step Functions — resources that commonly appear together in event-driven data pipelines. SAM transforms a concise YAML definition into the full CloudFormation template.

Lambda is the glue (lowercase) of serverless data pipelines — it connects events to actions. An S3 upload triggers a Lambda function that validates the file, a Kinesis stream invokes Lambda for record-level processing, and an EventBridge rule triggers Lambda for scheduled tasks. SAM makes packaging and deploying these functions repeatable.

Key Lambda concepts for the exam:

Concurrency. Lambda scales by running multiple function instances in parallel. Reserved concurrency guarantees capacity for critical functions. Provisioned concurrency eliminates cold starts for latency-sensitive functions. Account-level concurrency limits (default 1,000) apply across all functions — exceeding this throttles invocations.

Lambda layers. Package shared dependencies (libraries, custom runtimes) separately from function code. Layers reduce deployment package size and enable sharing common code across functions.

Storage from Lambda. Functions have 10 GB of /tmp ephemeral storage. For larger needs, mount Amazon EFS file systems. Lambda can also read/write directly to S3. The exam tests understanding of when /tmp is sufficient vs. when EFS or S3 is needed.

SAM CLI provides local testing (sam local invoke), packaging (sam build), and deployment (sam deploy). It integrates with CodePipeline for CI/CD and supports gradual deployments (canary, linear) for Lambda functions.

⚠️ Exam Trap: Lambda's 15-minute execution timeout and 10 GB memory limit are hard constraints. For long-running data processing (large file transformation, complex joins), Lambda is the wrong choice even in a serverless architecture. Step Functions can coordinate multiple short Lambda invocations, or Glue/EMR Serverless handles the heavy processing.

Reflection Question: A serverless data pipeline uses Lambda to process individual records from Kinesis. During peak load, some invocations are throttled. What Lambda feature should you configure, and what's the architectural alternative?