3.2.1.6. Securely Storing & Managing Logs

3.2.1.6. Custom Metrics & Metric Filters (CloudWatch Agent)

Standard AWS metrics cover infrastructure health but not application behavior. Custom metrics fill the gap — tracking business-specific indicators like orders per minute, cart abandonment rate, or API latency percentiles.

Publishing custom metrics:

# CLI: Publish a custom metric
aws cloudwatch put-metric-data \
  --namespace "MyApp/Orders" \
  --metric-name "OrdersProcessed" \
  --value 42 \
  --unit Count \
  --dimensions Environment=Production,Service=OrderAPI

CloudWatch Agent collects OS-level metrics not available by default:

• Memory utilization, swap usage
• Disk usage (per-mount), disk I/O
• Network statistics (TCP connections, packets)
• Process-level metrics (CPU/memory per process)

Metric filters extract metrics from log data without code changes. Define a pattern that matches log entries, and CloudWatch creates a metric from matches:

# Create metric filter: count ERROR log entries
aws logs put-metric-filter \
  --log-group-name "/prod/app/api" \
  --filter-name "ErrorCount" \
  --filter-pattern '"ERROR"' \
  --metric-transformations \
    metricName=ApplicationErrors,metricNamespace=MyApp,metricValue=1

Metric filter patterns:

• Simple text match: "ERROR" matches any log with ERROR
• JSON pattern: { $.statusCode = 500 } matches structured JSON logs
• Space-delimited: [ip, user, timestamp, request, status_code = 5*, bytes]

Exam Trap: Metric filters only process log events after the filter is created — they don't backfill historical data. If you create a metric filter for error counting, you'll only see errors from that point forward. To analyze historical logs, use CloudWatch Logs Insights queries instead.