Pre-Migration Backup and Rollback Strategy: Planning Your Exit Before Switching

When migration issues arise, the extent of the damage is determined not by “how severe the problem is,” but by whether you have an exit strategy. With a rollback plan, the worst-case scenario is reverting to the starting point; without one, you are left struggling in a faulty new environment.

This article covers the backup content, rollback conditions, and drills required before migration, with a focus on the critical judgment call of “when to decide to roll back.”

1. What to Back Up

# ① Files
tar -czf /backup/www-$(date +%F).tar.gz -C /var/www html

# ② Database (with full parameters)
mysqldump -u root -p --default-character-set=utf8mb4 
  --single-transaction --routines --triggers --events dbname 
  | gzip > /backup/db-$(date +%F).sql.gz

# ③ Configurations
tar -czf /backup/conf-$(date +%F).tar.gz /etc/nginx /var/spool/cron

# ④ Certificates
tar -czf /backup/ssl-$(date +%F).tar.gz /etc/nginx/ssl/

# ⑤ Transfer to off-site location (Critical)
rsync -avz /backup/ root@BackupServer:/backup/

Two Requirements for Backups

1. Must be stored off-site (Local backups are lost if the machine fails).
2. Must verify availability (Ensure they can be extracted and imported).

# Verification (Don't just check if the file exists)
gzip -t /backup/db-$(date +%F).sql.gz && echo "Database backup is intact"
tar -tzf /backup/www-$(date +%F).tar.gz > /dev/null && echo "File backup is intact"

2. Define Rollback Conditions (Most Important)

This step is often skipped, yet it determines the quality of decision-making during the switch. Without predefined conditions, teams get stuck in indecision at the scene, debating whether to “fix it further” or “roll back.”

Example Rollback Conditions (Define before switching):

① Time Condition: Core functions not restored within 30 minutes after switch → Rollback
② Functional Condition: Any of the following features are unavailable → Rollback
   - Homepage accessible
   - User login functional
   - Order submission/submission process working
③ Data Condition: Data inconsistency or loss detected → Immediate Rollback
④ Impact Condition: Error rate exceeds a specific threshold → Rollback

The Timebox is the most practical rule: Agree that “if it’s not fixed within N minutes, we revert.” This prevents the most common failure mode—refusing to admit defeat and endlessly trying to fix minor issues, turning a small glitch into a major incident.

3. Rollback Methods

Most Effective Approach: Do Not Stop the Old Environment Yet

Correct Sequence:
① Keep old environment running (maintain availability)
② Deploy and verify new environment
③ Switch DNS resolution to new environment
④ Monitor for a period
⑤ Only stop old environment after confirmation (Recommended retention: at least 48 hours)

# To rollback after switch, change DNS resolution back to old IP
# Rollback speed depends on TTL — so lower TTL before switching
 dig example.com A | grep -A1 'ANSWER SECTION'

“Retaining the old environment for 48 hours” is the core of the rollback strategy. It turns rollback into changing a single DNS record, rather than rebuilding from scratch.

4. Drills (Determines If You Can Actually Roll Back)

# Perform a recovery drill before switching (on test machine or temp dir, never touch production)
# ① Restore database to test DB
gunzip < /backup/db-date.sql.gz | mysql -u root -p testdb

# ② Verify data integrity
mysql -e "USE testdb; SELECT COUNT(*) FROM main_table;"

# ③ Restore files to temp directory
tar -xzf /backup/www-date.tar.gz -C /tmp/restore-test/
ls /tmp/restore-test/html | head

The purpose of the drill is not just to “recover,” but to confirm “the recovered data is complete.” There have been cases where backup scripts failed silently, producing empty files that went unnoticed for months.

5. Scheduling the Migration Window

Migration Day Checklist:
□ Backups completed and transferred off-site
□ Backups verified as restorable
□ TLS/TTL lowered in advance
□ Rollback conditions defined and approved
□ Old environment is available
□ Verification checklist ready
□ Decision-maker for rollback is present

Decision Tree

Pre-Migration Preparation
├─ Backup
│  ├─ Files + Database + Configs + Certificates
│  ├─ Transfer off-site
│  └─ Verify restorability (not just checking file existence)
├─ Rollback Conditions (Defined before switch)
│  ├─ Timebox: Not restored within N minutes → Rollback
│  ├─ Feature Checklist: Core features down → Rollback
│  └─ Data Inconsistency → Immediate Rollback
├─ Rollback Method
│  └─ Keep old environment running (Rollback = Change one DNS record)
├─ Drill
│  └─ Restore once in test environment to confirm data integrity
└─ Post-Switch
   └─ Retain old environment for at least 48 hours

Scenario Simulation: Halfway Through Switch, Realize Something Is Wrong, But Cannot Roll Back

Phenomenon: During a migration switch, data inconsistencies are discovered halfway through.
Attempting to revert reveals that the original environment has already been decommissioned and no complete backup exists, forcing the team to struggle with fixes.

1. Analyze what is missing for reversion →
   Two things are missing: a fully usable backup and the unreleased original environment. Both should have been prepared before the switch.
2. Understand that “Backup Exists” ≠ “Can Recover” →
   The restoration process was not verified before the switch. Usability was only discovered when it was too late.
3. Replan: Treat Reversion as an Independent Plan →
   Practice restoration before switching, rather than treating backups as mere psychological comfort.
4. Adopt a “Coexistence” Window →
   Both environments run during the switch. If issues arise, reverting is just changing a pointer.
5. Identify Irreversible Operations →
   Actions like releasing resources or deleting data must be scheduled only after stability is confirmed.
6. Set a Clear Observation Period →
   Only allow cleanup of the old environment after the observation period ends and verification is passed.

# Pre-switch: Backup and truly practice restoring to a temp environment
zcat backup.sql.gz | mysql -u user -p temp_db_name
mysql -e "SELECT COUNT(*) FROM temp_db_name.core_table;"

# File backup and consistency check
tar czf web-$(date +%F-%H%M).tar.gz /website_dir
rsync -avnc /source_dir/ /backup_dir/ | head

# Maintain reversibility: Confirm no issues before proceeding (Irreversible actions last)
# Keep old environment serviceable during observation period; do not release early

Root Cause: Treating “Backup Done” as equivalent to “Reversion Available.”
The gap between them is a restoration drill. Prematurely releasing the original environment is essentially voluntarily abandoning the possibility of reversion.

Two Key Takeaways:

• Practice the reversion plan before switching; do not just create backups.
  Verified restoration is an exit strategy; unverified backups are merely hope.
• Irreversible actions (release, delete) must always follow stability confirmation.
  Allowing both environments to coexist for a period is the safest insurance for such operations.

8 Common Pitfalls

1. Backups only on local machine: Lost if the machine fails.
2. Not verifying backup restorability: Empty files still “exist.”
3. No defined rollback conditions: Leads to indecision at the scene.
4. Treating rollback as failure: Causes hesitation to revert when necessary.
5. Not lowering TTL before switch: Delays rollback significantly.
6. Stopping old environment immediately after switch: Loses the fastest exit route.
7. No recovery drill: Discover unavailability only when needed.
8. No decision-maker: No one authorized to approve rollback.

Real Customer Feedback (From Cloud Market Reviews and Consultations for Similar Services)

The following excerpts are taken from real buyer reviews and consultations for similar IT operations services on Alibaba Cloud, Tencent Cloud, and Huawei Cloud Cloud Markets (Service providers include our store and peers; provided solely as reference for real feedback, not fictional tickets from this site):

1****n Rating: 5.0 Points 2018-01-04 18:04

45***76 2021-04-14 5.0 On-time delivery. Excellent.

10***02 2024-07-31 5.0 Secure and reliable | Comprehensive features | Affordable price. Prompt handling, migration successful. 10***31 2022-06-22 5.0 Affordable price | Skilled technical support | Complete after-sales service. Good job, helped me migrate my years-old website to a new server.