5.2.1. CloudWatch, X-Ray, and Observability Tools
💡 First Principle: Observability has three pillars—metrics, logs, and traces—and AWS provides a dedicated service for each. Knowing which pillar a scenario requires tells you which service to recommend, and the exam frequently tests this mapping.
Metrics tell you what is happening: CPU utilization, memory usage, endpoint latency, invocation counts. Amazon CloudWatch collects and visualizes these metrics. You create CloudWatch Alarms that trigger when metrics cross thresholds—for example, alerting when endpoint latency exceeds 500ms or when GPU utilization drops below 10% (indicating over-provisioning).
Logs tell you why something happened: error messages, stack traces, request/response payloads. CloudWatch Logs stores log data from SageMaker AI training jobs, endpoints, and processing jobs. CloudWatch Logs Insights lets you query logs with a purpose-built query language—useful for troubleshooting why a specific inference request failed.
Traces tell you where time is spent across distributed systems: how long each service in a pipeline took, where bottlenecks exist. AWS X-Ray provides distributed tracing across AWS services. For ML pipelines that span Lambda → SageMaker AI → DynamoDB → S3, X-Ray reveals which component introduces latency. For agents, Amazon Bedrock AgentCore Observability records each step of an agent run as OpenTelemetry spans in CloudWatch (see 5.4.1).
| Tool | Pillar | Primary Use | Exam Signals |
|---|---|---|---|
| CloudWatch Metrics | Metrics | Monitor CPU, memory, latency, invocation count | "Monitor endpoint performance," "utilization metrics" |
| CloudWatch Alarms | Metrics | Alert on threshold breaches | "Alert when latency exceeds," "trigger scaling" |
| CloudWatch Dashboards | Metrics + Logs | One view of related metrics across services and Regions | "Single pane of glass," "operations dashboard" |
| CloudWatch Logs | Logs | Store and query application logs | "Troubleshoot errors," "debug failures" |
| CloudWatch Logs Insights | Logs | Interactive log querying | "Analyze log patterns," "find error causes" |
| Lambda Insights | Metrics + Logs | Lambda function monitoring | "Lambda cold starts," "function duration" |
| AWS X-Ray | Traces | Distributed request tracing | "End-to-end latency," "pipeline bottleneck" |
| AgentCore Observability | Metrics + Traces + Logs | Step-level agent tracing, sessions, tool calls | "Which agent step failed," "agent latency" |
| AWS CloudTrail | Audit logs | API call logging for compliance | "Who did what," "audit trail," "governance" |
| Amazon Quick (Quick Sight, formerly Amazon QuickSight) | Visualization | BI dashboards for business stakeholders | "Business dashboard," "visualize metrics for managers" |
Dashboards that answer a question. A useful operations dashboard is built around a decision, not around every metric that exists. For a real-time endpoint, put latency percentiles (ModelLatency p50/p99), error counts (Invocation5XXErrors), traffic (Invocations), and utilization (CPUUtilization or GPUUtilization) side by side, so a latency spike can be read against load and saturation in one glance. CloudWatch dashboards can combine metrics from several services and Regions and embed alarm status; Amazon Quick is the better fit when the audience is business users who need cost or KPI views rather than operators.
CloudTrail deserves special attention because it serves a different purpose than the other observability tools. While CloudWatch monitors operational health, CloudTrail monitors security and compliance by logging every API call made in your AWS account. The exam tests CloudTrail in Domain 4 security questions—"Who started that training job?", "When was the IAM policy changed?", "Was the S3 bucket accessed from an unusual location?"
⚠️ Exam Trap: CloudWatch's built-in endpoint metrics cover infrastructure (CPU, latency). Drift and model-quality monitoring covers data and model metrics (feature drift, prediction distribution). A question about "monitoring model accuracy in production" needs a baseline-driven model-quality check, not CloudWatch's default metrics (CloudWatch can alarm on accuracy only after you publish it as a custom metric). A question about "monitoring endpoint response time" needs CloudWatch, not a drift check.
Reflection Question: A SageMaker endpoint returns predictions in under 200ms during testing but experiences 2-second latency spikes in production during peak hours. Which observability tools would you use to diagnose this, and in what order?