3.1.1.1. Disaster Recovery Strategies (RTO, RPO, Pilot Light, Warm Standby, Multi-Site Active/Active)

3.1.1.1. Disaster Recovery Strategies (RTO, RPO, Pilot Light, Warm Standby, Multi-Site Active/Active)

💡 First Principle: A structured disaster recovery strategy, driven by business-defined objectives for recovery time ("RTO") and data loss ("RPO"), is essential for minimizing the impact of a major disruption and ensuring business continuity.

Scenario: A gaming company's popular online game relies on a backend that experiences frequent updates and needs to be available globally with minimal latency. While extremely high availability is important, a full regional outage is a rare event, and the company can tolerate a few minutes of downtime for critical data.

Disaster Recovery ("DR") is a comprehensive plan to recover from large-scale outages, often involving a separate "AWS Region". Key metrics define the strategy:

• "Recovery Time Objective (RTO)": Max tolerable downtime after disaster.
• "Recovery Point Objective (RPO)": Max tolerable data loss after disaster.

"DR" Strategies (from highest "RTO"/"RPO"/lowest cost to lowest "RTO"/"RPO"/highest cost):

1. Backup and Restore:

   • Concept: Back up data to "S3" (potentially cross-"Region") and restore to a new environment upon disaster.
   • "RTO"/"RPO": Hours to days / Hours.
   • AWS Services: "AWS Backup", "S3", "EC2 AMIs", "RDS Snapshots".
   • Practical Relevance: Suitable for non-critical applications or data archives.

2. Pilot Light:

   • Concept: A minimal core infrastructure is kept running in the "DR Region", ready for quick scale-up. Data is replicated.
   • "RTO"/"RPO": Minutes to hours / Minutes.
   • AWS Services: "Cross-Region RDS Read Replicas", "S3 CRR", pre-built AMIs, "Auto Scaling Groups" scaled to zero or minimal instances.
   • Practical Relevance: Cost-effective for applications with some tolerance for downtime.

3. Warm Standby:

   • Concept: A scaled-down, fully functional production replica is continuously running and updated in the "DR Region".
   • "RTO"/"RPO": Minutes / Seconds to minutes.
   • AWS Services: Active-passive load balancers, small "Auto Scaling Groups", constantly synchronized databases (e.g., "RDS Cross-Region Read Replicas", "DynamoDB Global Tables").
   • Practical Relevance: For business-critical applications requiring rapid recovery.

4. Multi-Site Active/Active:

   • Concept: Application is fully deployed and actively serving traffic in multiple "Regions" simultaneously.
   • "RTO"/"RPO": Near Zero / Near Zero.
   • AWS Services: "DynamoDB Global Tables", "Aurora Global Database", "Route 53" (latency, geoproximity), "ALB", "CloudFront".
   • Practical Relevance: Highest cost, highest complexity. For mission-critical, global applications with no tolerance for downtime or data loss.

Visual: Disaster Recovery (DR) Strategy Spectrum

Choosing a Strategy, and the Tooling Behind It:

• Selection rule: Pick the cheapest strategy that meets BOTH the "RTO" and the "RPO": "Backup and Restore" for hours, "Pilot Light" for tens of minutes, "Warm Standby" for minutes, "Multi-Site Active/Active" for near zero. The "Pilot Light" vs "Warm Standby" line is whether compute is already running: Pilot Light runs only the replicated data layer, so servers must be launched and scaled after the disaster is declared (and anything slow to start, such as a cache warm-up, adds to the "RTO"); Warm Standby already runs a scaled-down working stack that can take traffic and only has to scale up.
• Replication lag is your RPO: Cross-Region replication is asynchronous, so the achievable "RPO" is the replication lag at the moment of failure; check it when validating a failover. Match the data store to the mechanism: "Aurora" -> "Aurora Global Database"; "RDS" for MySQL/MariaDB/PostgreSQL/Oracle -> cross-Region read replica ("Aurora Global Database" does not apply to RDS engines); "DynamoDB" -> "Global Tables"; "S3" -> "Cross-Region Replication" (new objects, asynchronous). A nightly copy job or periodic snapshot gives an hours-level RPO, and "CloudFront" caching is not replication.
• "AWS Backup": Centralized, policy-driven backup for "EBS", "RDS"/"Aurora", "EFS", "DynamoDB", "S3", "EC2", "FSx" "Storage Gateway" volumes and on-premises VMware VMs (through a Backup gateway). A backup plan sets frequency, retention and cross-Region/cross-account copy. Backups are periodic point-in-time recovery points, so the "RPO" is the schedule interval (an hourly plan, or an hourly "Amazon Data Lifecycle Manager" policy for "EBS" snapshots, gives a 1-hour RPO). "Backup Vault Lock" makes a vault WORM: in compliance mode the lock becomes immutable once the grace period (minimum 3 days) ends, and no user, including root, can delete recovery points early or shorten retention; governance mode can still be removed by users with permission. Immutable and Region-resilient together = Vault Lock compliance mode plus a cross-Region copy action. Versioning or custom copy scripts do not provide this. Stateful data goes in AWS Backup; the stateless tier is rebuilt from "CloudFormation" templates kept in version control.
• "AWS Elastic Disaster Recovery (DRS)": Continuous block-level replication of whole servers (physical, virtual, other clouds, or "EC2" in another "AZ"/"Region") into a low-cost staging area; "RPO" of seconds, "RTO" of minutes, point-in-time recovery, non-disruptive drills, and failback, which replicates the recovery instances back to the restored source site before cutover. Contrast: "AWS Backup" = periodic recovery points for data restore and retention; "DRS" = continuous server replication for fast full-system failover. "Application Migration Service (MGN)" is for one-time migration, not ongoing DR, and DRS replicates servers' disks, not managed "RDS"/"EFS" (use their native replication or AWS Backup).

⚠️ Common Pitfall: Implementing a "DR" strategy without regularly testing it. An untested "DR" plan is likely to fail during a real disaster due to configuration drift, outdated procedures, or unforeseen dependencies.

Key Trade-Offs:

• "RTO"/"RPO" vs. Cost: The lower your "RTO" and "RPO" (i.e., the faster you need to recover with less data loss), the more expensive and complex your "DR" strategy will be. A near-zero "RTO"/"RPO" (Multi-Site Active/Active) is the most expensive option.

Reflection Question: Given the gaming company's global nature, the need for rapid recovery, and tolerance for a few minutes of downtime for critical data, which disaster recovery strategy (from the list above) would you recommend, and what AWS services would be central to its implementation to balance "RTO"/"RPO" with cost?