5.2.3. Secrets Management: Secrets Manager and Parameter Store

💡 First Principle: Hard-coded credentials in application code are the most common and most preventable security vulnerability. A developer checks in an access key to GitHub "just for testing" — the repo is public — the key is scraped by automated bots within minutes. Secrets management externalizes credentials from code entirely: the application retrieves secrets at runtime from a managed service, never touching source control.

Secrets Manager vs. Parameter Store — Decision Guide:

Use Secrets Manager when: Automatic rotation is required, especially for database credentials.
Use Parameter Store when: Storing non-sensitive configuration, environment variables, or when cost is a constraint.

Secrets Manager Rotation: When you enable rotation, Secrets Manager creates a Lambda function that:

1. Creates a new version of the secret with updated credentials
2. Sets the new credentials on the target service (e.g., RDS password)
3. Tests the new credentials work
4. Marks the old version as deprecated

AWS provides rotation Lambda templates for RDS MySQL, PostgreSQL, Oracle, MariaDB, SQL Server, and Redshift. Custom rotation Lambdas handle other systems.

Rotation Stages (AWSCURRENT, AWSPENDING, AWSPREVIOUS): Secrets Manager uses version stages to manage rotation without downtime:

During rotation, both AWSCURRENT (old) and AWSPENDING (new) are valid — applications that cached the old credentials continue working until rotation completes.

Accessing Secrets in Applications: Best practice is to retrieve secrets at application startup (or when needed) using the Secrets Manager SDK, not to pass them as environment variables (environment variables can be logged or exposed in console). For Lambda, use the Secrets Manager layer for automatic caching.

⚠️ Exam Trap: When Secrets Manager rotates a database password, the database password itself changes. If your application connection pool caches the old password and doesn't handle the SecretsManager.GetSecretValue API call correctly (with error handling for credential failures and re-fetching), the application will fail with authentication errors after rotation. Applications must be designed to re-fetch credentials on authentication failure.

Reflection Question: A web application running on EC2 connects to an RDS MySQL database. The current configuration has the database username and password hard-coded in a config file on the EC2 instance. Redesign this to: (1) eliminate the hard-coded credentials, (2) automatically rotate the password every 30 days, and (3) ensure the application handles rotation without downtime.