Cloud & DevOps
A Practical Guide to Zero-Downtime Cloud Migrations
We recently moved a core banking platform across AWS regions with zero customer-facing downtime, processing live transactions the entire time. It wasn't luck — it was three disciplines applied consistently: traffic shadowing, incremental cutover, and rollback plans that we actually rehearsed before the real thing.
Why most migrations fail
Migrations rarely fail because of the infrastructure work itself — spinning up equivalent resources in a new region or provider is mostly mechanical. They fail because teams underestimate the drift between "the new environment as configured" and "the new environment as it behaves under real production load." Config that looks identical on paper produces different latency profiles, different connection pool behavior, and different failure modes once real traffic hits it.
Phase 1: Traffic shadowing
Before any customer traffic touches the new environment, we mirror a copy of live production traffic to it — without ever returning shadow responses to the user. This lets us compare response times, error rates, and payload correctness between old and new environments under real, unpredictable load, for days or weeks, with zero customer risk. Shadowing surfaces the issues that only show up under production traffic patterns: cache stampedes, N+1 queries that were invisible in staging, connection pool exhaustion under peak concurrency.
We don't move to phase 2 until shadow traffic error rates match production for at least 72 hours of sustained load, including peak traffic windows.
Phase 2: Incremental cutover
Once shadowing confirms parity, we cut real traffic over gradually — typically 1% → 5% → 25% → 50% → 100%, holding at each step long enough to catch issues under real load (usually 24–48 hours per step for consumer-facing systems, longer for financial systems). Traffic is routed using weighted DNS or a load balancer's weighted target groups, so the rollback at any stage is a config change, not a redeploy.
Critically, we monitor business metrics during cutover, not just infrastructure metrics. CPU and memory looking healthy doesn't tell you if checkout completion rates dropped 2% because of a subtle latency regression in a third-party payment integration.
Phase 3: Rollback plans, rehearsed in advance
Every migration plan includes a rollback procedure — and we run it in a dry run before the real migration, not just write it down. If your rollback plan has never actually been executed, it doesn't work; you just haven't found out yet. For the banking client, we rehearsed a full rollback from 50% traffic back to the old region three times in a staging exercise before the live migration, timing each step so we knew our maximum exposure window if something went wrong.
The migration-day checklist
- Shadow traffic run for at least 72 hours with error-rate parity confirmed
- Database replication lag monitored continuously and under an agreed threshold before cutover begins
- Rollback procedure rehearsed at least once in a non-production dry run
- Business metrics dashboards (not just infra metrics) watched live during each cutover step
- A named decision-maker with authority to trigger rollback, available for the full cutover window
The result
For the banking client, the entire migration — from first shadow traffic to 100% cutover — took five weeks. Customers never noticed. That's the actual goal: not a fast migration, but a boring one, where nothing interesting happens because every failure mode was already found and handled before it could reach production.