Log Rotation

Log Rotation

Without rotation, log files grow without bound. A busy web server produces gigabytes of access logs per month; a production database can generate hundreds of megabytes per day of query logs. Unchecked, log growth fills the disk and crashes the server. Log rotation solves this by periodically compressing old log segments, keeping only a configured number of historical files, and optionally deleting files beyond a retention period. On Ubuntu, logrotate handles this for system and application logs.

Why log rotation is essential

Without rotation:
  /var/log/nginx/access.log: 50GB (server crashes when disk fills)

With rotation (daily, keep 14 days, compressed):
  /var/log/nginx/access.log            ← today (active, ~500MB)
  /var/log/nginx/access.log.1          ← yesterday (uncompressed for quick access)
  /var/log/nginx/access.log.2.gz       ← 2 days ago
  /var/log/nginx/access.log.3.gz       ← 3 days ago
  ...
  /var/log/nginx/access.log.14.gz      ← 14 days ago (oldest kept)

logrotate basics

# logrotate runs daily via cron or systemd timer
# Main config: /etc/logrotate.conf
# Drop-in configs: /etc/logrotate.d/

# Check when logrotate last ran:
cat /var/lib/logrotate/status | head -20

# View logrotate configuration for nginx:
cat /etc/logrotate.d/nginx

Default /etc/logrotate.d/nginx configuration

/var/log/nginx/*.log {
    daily           ← rotate daily
    missingok       ← don't error if log file is missing
    rotate 52       ← keep 52 rotations (52 weeks)
    compress        ← gzip old log files
    delaycompress   ← don't compress yesterday's log (keep .1 uncompressed)
    notifempty      ← don't rotate empty files
    sharedscripts   ← run postrotate once even if multiple files matched
    postrotate      ← tell nginx to reopen log files after rotation
        invoke-rc.d nginx rotate >/dev/null 2>&1
    endscript
}

Writing rotation configuration

# Create rotation config for a custom application
sudo nano /etc/logrotate.d/myapp

/etc/logrotate.d/myapp

/var/log/myapp/*.log {
    daily
    rotate 30           ← keep 30 days
    compress
    delaycompress
    missingok
    notifempty
    create 0640 myapp myapp    ← create new empty log file with these perms/owner
    postrotate
        systemctl reload myapp 2>/dev/null || true   ← signal app to reopen logs
    endscript
}

# Configuration options summary:
# weekly / monthly      — rotate weekly or monthly instead of daily
# size 100M             — rotate when file exceeds 100MB (regardless of time)
# maxsize 500M          — rotate if larger than 500MB even if time hasn't elapsed
# minsize 1M            — don't rotate if smaller than 1MB
# dateext               — add date to rotated filename instead of number
# maxage 90             — delete rotations older than 90 days

Testing and debugging

# Test your configuration without actually rotating:
sudo logrotate -d /etc/logrotate.d/myapp    # -d = debug/dry-run

# Force rotation immediately (useful for testing):
sudo logrotate -f /etc/logrotate.d/myapp    # -f = force

# Run all logrotate configs verbosely:
sudo logrotate -v /etc/logrotate.conf

logrotate -d output (dry run)

reading config file /etc/logrotate.d/myapp
Handling 1 logs from /etc/logrotate.d/myapp
rotating pattern: /var/log/myapp/*.log  after 1 days (30 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/myapp/app.log
  log needs rotating

systemd journal rotation

# The systemd journal has its own rotation controlled by journald.conf
# Configure in /etc/systemd/journald.conf:
# SystemMaxUse=1G        ← maximum total journal size
# SystemMaxFileSize=100M ← maximum size of individual journal file
# MaxRetentionSec=3month ← delete entries older than 3 months

# Manually clean up journal:
sudo journalctl --vacuum-size=500M   # Shrink journal to 500MB
sudo journalctl --vacuum-time=30d    # Remove entries older than 30 days

# Check current journal disk usage:
journalctl --disk-usage

Conclusion

Every custom application that writes log files needs a logrotate configuration in /etc/logrotate.d/. Without it, logs grow indefinitely. The most important settings: rotate N (how many historical files to keep), compress and delaycompress (save disk space), and a postrotate script to signal the application to reopen its log file descriptor. Always test new configs with logrotate -d before applying.