Cloud Disaster Recovery: RTO, RPO and the Four DR Strategies

Cloud disaster recovery means being able to restore a workload in another location, usually another region and often another account, within a recovery time objective (RTO, how long you can be down) and a recovery point objective (RPO, how much data you can lose). AWS describes four strategies in order of rising cost and falling recovery time: backup and restore, pilot light, warm standby and multi-site active/active. Pick one per workload from the RTO and RPO the business agrees to pay for, keep backups out of reach of the credentials that can delete production, and test recovery on a schedule, because a plan you have never run is a guess.

Key takeaways

• RTO includes detection and decision time, not just the technical steps. Most real recoveries lose more time deciding than executing.
• Replication is not backup. Replication copies a bad deployment, a deletion or ransomware encryption to the recovery region within seconds.
• Match the strategy to each workload. Backup and restore is right for most systems; active/active is right for very few.
• Fail over with the data plane (health checks, routing controls), not with control-plane changes in a region that may be impaired.
• The recovery region needs everything: quotas, keys, secrets, images, certificates and a deploy path that does not depend on the failed region.
• Measure recovery in tests and record the real numbers. Those, not the design document, are your RTO and RPO.

RTO and RPO, defined so they are useful

RTO is the time from the start of the disruption to the moment the service is usable again. It includes noticing the problem, deciding to fail over, executing, and verifying. A runbook that executes in fifteen minutes behind an hour of debate has an RTO well over an hour.

RPO is the maximum window of data you can lose, measured backwards from the disaster. It is set by your replication or backup interval: nightly snapshots mean you can lose up to a day, continuous asynchronous replication means seconds to minutes, synchronous replication means zero, at a latency cost on every write.

Both are business decisions that engineering prices. The conversation goes better as a table of options with costs than as a request for "zero and zero". Tighter objectives cost more at every step, in idle infrastructure, replication charges and engineering time.

Which disasters are you planning for?

"Disaster" covers very different events, and each needs a different defense. Most outages that destroy data are not regional.

Write down which rows each workload's plan covers. A warm standby in another region does nothing for the bad migration that dropped a column an hour ago.

The four strategies

Backup and restore

Back up data on a schedule, copy the backups to another region and account, and keep infrastructure in code so the environment can be rebuilt. It is the right answer for most internal tools, back-office systems and anything where a few hours of downtime is acceptable.

AWS Backup handles scheduling, cross-region and cross-account copies and retention in one place:

resource "aws_backup_plan" "prod" {
  name = "prod-daily"
 
  rule {
    rule_name         = "daily"
    target_vault_name = aws_backup_vault.primary.name
    schedule          = "cron(0 3 * * ? *)"
    start_window      = 60
    completion_window = 360
 
    lifecycle {
      delete_after = 35
    }
 
    copy_action {
      destination_vault_arn = var.recovery_account_vault_arn
      lifecycle {
        delete_after = 35
      }
    }
  }
}
 
resource "aws_backup_selection" "prod" {
  name         = "tagged-daily"
  plan_id      = aws_backup_plan.prod.id
  iam_role_arn = aws_iam_role.backup.arn
 
  selection_tag {
    type  = "STRINGEQUALS"
    key   = "backup"
    value = "daily"
  }
}

In the recovery account, lock the destination vault so nobody, including an administrator with stolen credentials, can delete recovery points before their retention ends:

resource "aws_backup_vault_lock_configuration" "recovery" {
  backup_vault_name   = aws_backup_vault.recovery.name
  min_retention_days  = 7
  max_retention_days  = 400
  changeable_for_days = 3
}

With changeable_for_days set, the lock is created in compliance mode: you can still change or remove it during that grace period, and once it ends nobody can. Test with a short-lived vault first. Copies must be encrypted with KMS keys that exist in the destination region and that the production account cannot schedule for deletion.

The weakness of backup and restore is the restore. Rebuilding networks, restoring large databases and warming caches takes longer than people expect. Your infrastructure as code must be able to target the recovery region with a variable change, and you only know that if you try it.

Pilot light

Keep the data live in the recovery region and the rest dormant. Databases replicate continuously; networking, IAM and load balancers exist; compute is defined in code at zero or minimal capacity. In a disaster you promote the data, scale compute and switch traffic.

Pilot light is the sweet spot for many customer-facing systems: RPO in seconds to minutes, RTO in tens of minutes, at the cost of replication rather than a second environment.

Warm standby

Run a complete copy of the environment at reduced size, all the time. Every component exists and works, so failover is scaling up and moving traffic, not building. Because it is running, you can test it continuously, for example by sending synthetic traffic or a small weighted share of real traffic to it. That is its real advantage over pilot light: you find the broken piece before the disaster.

Multi-site active/active

Serve production from two or more regions at once. A regional failure becomes a routing change, not a recovery. The cost is not just double capacity. Writes in two regions mean conflicts, and you must choose how to resolve them: partition data by region so each record has one home, accept last-writer-wins semantics (as DynamoDB global tables do by default), or keep a single writer region and forward writes to it (as Aurora Global Database write forwarding does), which is closer to warm standby for writes.

Choose active/active only when the business case names the regional outage it is paying to survive, and the team can operate a distributed data model every day, not just during incidents.

Choosing per workload: a decision table

Start from what the business accepts, then pick the cheapest strategy that meets it.

Most estates end up mixed: active/active or warm standby for the one or two systems that take payments or serve the core product, pilot light for important internal systems, backup and restore for everything else.

Replicating data across regions

The data layer decides your RPO. On AWS:

For Aurora Global Database, the two operations are distinct. A planned switchover waits for replication to catch up and loses nothing; an unplanned failover promotes the secondary immediately and accepts the loss of anything not yet replicated:

aws rds switchover-global-cluster \
  --global-cluster-identifier orders-global \
  --target-db-cluster-identifier arn:aws:rds:us-west-2:123456789012:cluster:orders-usw2
 
aws rds failover-global-cluster \
  --global-cluster-identifier orders-global \
  --target-db-cluster-identifier arn:aws:rds:us-west-2:123456789012:cluster:orders-usw2 \
  --allow-data-loss

Replication protects against losing a region. It does not protect against bad writes. Keep point-in-time recovery and backups alongside every replicated store.

Moving traffic: fail over with the data plane

During a regional impairment, the services you would use to change things in that region (its APIs, its consoles) may be impaired too. Design failover so it relies on data-plane mechanisms that keep working: Route 53 health checks and failover records, or Amazon Application Recovery Controller (ARC) routing controls, which you flip through highly available cluster endpoints rather than by editing DNS records.

A failover record pair with a health check:

resource "aws_route53_health_check" "primary" {
  fqdn              = "api-primary.example.com"
  type              = "HTTPS"
  resource_path     = "/healthz"
  port              = 443
  failure_threshold = 3
  request_interval  = 10
}
 
resource "aws_route53_record" "api_primary" {
  zone_id         = var.zone_id
  name            = "api.example.com"
  type            = "A"
  set_identifier  = "primary"
  health_check_id = aws_route53_health_check.primary.id
 
  failover_routing_policy {
    type = "PRIMARY"
  }
 
  alias {
    name                   = aws_lb.primary.dns_name
    zone_id                = aws_lb.primary.zone_id
    evaluate_target_health = true
  }
}
 
resource "aws_route53_record" "api_secondary" {
  zone_id        = var.zone_id
  name           = "api.example.com"
  type           = "A"
  set_identifier = "secondary"
 
  failover_routing_policy {
    type = "SECONDARY"
  }
 
  alias {
    name                   = aws_lb.secondary.dns_name
    zone_id                = aws_lb.secondary.zone_id
    evaluate_target_health = true
  }
}

Two design choices matter more than the syntax. First, what the health check tests: a shallow check misses a broken database, a deep check fails over on every database blip. Check what the service needs to serve its main request, and nothing more. Second, whether failover is automatic. Automatic DNS failover is fine for stateless tiers. For anything with a database promotion, most teams are better served by a human decision followed by an automated, single-command execution, because an automatic promotion during a brief network partition can leave you with two writers.

The recovery runbook

Keep it in the repository next to the infrastructure code, and write it for a tired engineer at three in the morning:

1. Declare. Who can declare a disaster, and through which channel. Name people and deputies.
2. Decide. The criteria for failing over versus waiting, and a time box for the decision. Check the provider's health dashboard, but do not wait for it.
3. Freeze. Stop deployments and scheduled jobs that might write to the old primary.
4. Promote data. The exact commands, in order, with expected output.
5. Scale compute. Desired counts or capacity settings for the recovery region.
6. Move traffic. Flip the routing control or confirm the failover record took effect, and check from outside your network.
7. Verify. Synthetic checks, error rates, a business metric such as orders per minute.
8. Communicate. Status page and customer updates, with an owner. Our incident response playbook covers roles and communication.
9. Fail back. How and when you return, including re-establishing replication in the other direction. Failback is a second migration and needs its own plan.

Testing: the only evidence that counts

A restore test can be one command and a stopwatch:

aws rds restore-db-instance-to-point-in-time \
  --source-db-instance-identifier orders-prod \
  --target-db-instance-identifier orders-restore-test \
  --use-latest-restorable-time \
  --db-instance-class db.r6g.large \
  --no-publicly-accessible

Then run row counts and checksums against known values, record the elapsed time, and delete the instance. AWS Backup restore testing can run this kind of check on a schedule. AWS Fault Injection Service can simulate Availability Zone power interruptions and disrupt cross-region connectivity for game days, and AWS Resilience Hub can assess whether an application's design meets the RTO and RPO you set for it. All of these feed the Well-Architected reliability review, where "when did you last test recovery" is the question that matters.

What goes wrong in production

These are the failures that turn a regional incident into a long night.

Backups deletable by production credentials. The attacker or the faulty script that destroys production destroys the backups in the same account too. Copy to a separate account and lock the vault; our ransomware prevention guide covers the rest of that defense.

Replication faithfully copies the disaster. A migration drops a table and the replica drops it a second later. Without point-in-time recovery, the recovery region holds a perfect copy of the damage.

The recovery region is missing something. Service quotas never raised there, AMIs never copied, KMS keys that exist only in the primary region (so encrypted snapshots cannot be used), secrets not replicated, ACM certificates that are regional and were never issued in the recovery region. Find these in tests, not in incidents.

Recovery depends on the failed region. The CI/CD pipeline, the Terraform state bucket, the container registry or the identity provider lives only in the primary region. Check every tool in the runbook for where it runs.

Clients do not follow DNS. Long TTLs, connection pools that never re-resolve, mobile apps with hardcoded endpoints. Measure how long real traffic takes to move during a test.

The runbook has rotted. Commands reference resources renamed six months ago. Run the runbook regularly, and update it as part of every architecture change.

Nobody planned the way back. The failover worked, and now the primary region is healthy and stale. Without a failback plan, teams run in the recovery region for weeks and accumulate risk there.

Frequently Asked Questions

What is the difference between RTO and RPO?

The recovery time objective (RTO) is how long a workload may be unavailable after a disaster. The recovery point objective (RPO) is how much data, measured in time, you can afford to lose. RTO drives how much infrastructure you keep ready; RPO drives how often and how you replicate or back up data.

What are the four disaster recovery strategies in AWS?

AWS describes backup and restore, pilot light, warm standby and multi-site active/active. They run from cheapest with the longest recovery to most expensive with near-instant recovery. Most organizations use more than one, matched to each workload's RTO and RPO.

Is Multi-AZ enough for disaster recovery?

Multi-AZ protects against the failure of a single data center within a region, which is high availability, not disaster recovery. It does not protect against a regional impairment, a bad deployment, accidental deletion or ransomware, because those reach every zone. You still need point-in-time backups in a separate account and, where the RTO demands it, a second region.

How often should you test disaster recovery?

Test restores of critical data at least quarterly and run a full failover exercise for critical workloads at least once or twice a year. Tabletop walkthroughs are cheap and worth doing whenever the architecture or the team changes. Measure the actual recovery time and data loss each time and compare them with your targets.

How do you protect cloud backups from ransomware?

Keep backup copies in a separate account that production credentials cannot reach, and make them immutable with a feature such as AWS Backup Vault Lock in compliance mode. Encrypt them with keys the production account cannot delete. Then test restores from those copies, because an immutable backup you cannot restore is no protection.

What is the difference between high availability and disaster recovery?

High availability keeps a workload running through common component failures, such as an instance or an Availability Zone, usually automatically and without data loss. Disaster recovery restores the workload after a larger event that high availability cannot absorb, such as a regional outage, data corruption or a compromised account. You need both, and they are designed differently.

Need help with disaster recovery?

We design recovery from the RTO and RPO the business will pay for, build it in code, and run the tests that prove it. If we build it, we are on the pager for it, so the runbook is written by the people who will execute it.

• Cloud architecture and infrastructure
• Cybersecurity and compliance
• Talk to an engineer