Backup Strategies

Backup Strategies

Most backup strategies fail not because of tool choice but because of one of three problems: backups were never tested (discovered to be corrupt during recovery), backups were co-located with the data they protect (lost when the server failed), or the backup scope was incomplete (database was backed up but application config was not). A sound backup strategy addresses all three before any failure occurs.

Backup types

The 3-2-1 backup rule

3-2-1 Rule:
  3 copies of your data
  2 different storage media
  1 copy offsite (different location)

Example implementation:
  Copy 1: Production server (/var/lib/mysql) — live data
  Copy 2: Local NAS or external drive (/mnt/backup) — fast recovery
  Copy 3: Cloud storage (S3, Backblaze B2) — offsite, survives physical disaster

Without the offsite copy:
  Fire/flood/theft takes out both production AND local backup
  = total data loss despite "having backups"

What to back up

# Don't just backup databases — backup everything needed to reconstruct the server:
# /etc/              ← all configuration files
# /var/www/          ← web application files
# /var/lib/mysql/    ← MySQL data (or use mysqldump)
# /home/             ← user data and SSH keys
# /var/backups/      ← existing system backups
# /root/             ← root cron jobs and scripts
# /usr/local/        ← custom scripts and binaries

# What NOT to backup (wastes space, can be reinstalled):
# /proc/ /sys/ /dev/  ← virtual filesystems
# /tmp/               ← temporary files
# /var/cache/         ← package cache (apt install can restore)

# Create a backup exclusion list for rsync:
cat > /etc/backup-exclude.txt << 'EOF'
/proc
/sys
/dev
/tmp
/var/cache/apt
/var/tmp
EOF

Scheduling backups

# Example schedule for a production web server:
# /etc/cron.d/server-backups

# Daily: database backup at 1am
0 1 * * * root /usr/local/bin/db-backup.sh >> /var/log/backup.log 2>&1

# Daily: config and app files at 2am
0 2 * * * root rsync -az --exclude-from=/etc/backup-exclude.txt / /mnt/backup/daily/ >> /var/log/backup.log 2>&1

# Weekly: full backup to cloud storage (Sunday 3am)
0 3 * * 0 root /usr/local/bin/cloud-backup.sh >> /var/log/backup.log 2>&1

# Verify backup log daily:
0 8 * * * root tail -50 /var/log/backup.log | mail -s "Backup report $(hostname)" admin@company.com

Testing your backups

⚠️ WARNING: Untested backups are not backups. Schedule monthly restore tests to a separate VM or cloud instance. The worst time to discover a backup is corrupt is during an actual recovery emergency.

# Monthly restore test checklist:
# 1. Provision a test VM (or use cloud spot instance)
# 2. Copy latest backup to the test VM
# 3. Restore the database backup
# 4. Restore application files
# 5. Start the application and verify it works
# 6. Check row counts, spot-check data
# 7. Document the restore time (this is your actual RTO)
# 8. Destroy the test VM

# Automate the verify step — check backup file size is growing:
ls -lh /backup/ | tail -5    # Zero-byte files indicate a backup failure

Conclusion

The most important backup decisions: (1) apply the 3-2-1 rule so no single event destroys all copies; (2) back up everything needed for full reconstruction, not just the database; (3) test restores monthly — your actual RTO is how long the restore test takes, not what you estimate; (4) monitor backup logs daily for failures. A backup that silently fails for weeks before discovery is far worse than knowing immediately when a backup breaks.