Running Background Processes

Running Background Processes

Running processes in the background is essential for any server task that takes longer than a few seconds: backups, database migrations, file conversion jobs, or any operation you need to initiate over SSH without keeping the connection open. Ubuntu provides several mechanisms, ranging from simple shell job control to proper systemd service units for production workloads.

When you need background execution

Ampersand and nohup

# Run in background (stops when you log out)
rsync -av /home/ /backup/ &

# Run immune to logout (output to nohup.out)
nohup rsync -av /home/ /backup/ &

# Run immune to logout with specific log file
nohup rsync -av /home/ /backup/ > /var/log/backup.log 2>&1 &

# Check the PID
echo $!    # PID of last background command

# Disown a running job (make it immune to SIGHUP)
rsync -av /home/ /backup/ &
disown    # Removes from job table, immune to HUP on logout

# Monitor the running process
tail -f /var/log/backup.log    # Follow the output
jobs                           # List all jobs in this shell

Using systemd for persistent background services

For applications that need to run continuously, restart on failure, and survive reboots, create a systemd service unit. This is the production-appropriate approach for anything more than a temporary one-time task.

# Create a systemd service unit
sudo nano /etc/systemd/system/myapp.service

/etc/systemd/system/myapp.service

[Unit]
Description=My Application
After=network.target

[Service]
Type=simple
User=myapp
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/bin/myapp --config /etc/myapp/config.yml
Restart=always
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

# Enable and start the service
sudo systemctl daemon-reload
sudo systemctl enable --now myapp

# Check status
sudo systemctl status myapp

# View logs
sudo journalctl -u myapp -f    # Follow logs

Scheduling one-time background tasks

# Run a command at a specific time
echo "rsync -av /home/ /backup/" | at 02:00
echo "rsync -av /home/ /backup/" | at "2am tomorrow"
echo "command" | at "14:30 Jun 15"

# List scheduled at jobs
atq

# Remove an at job
atrm 1    # Job number from atq output

# Run after a delay with systemd-run (no cron needed)
systemd-run --on-active=30m rsync -av /home/ /backup/
# Runs once after 30 minutes, then disappears

# Run at a specific time with systemd-run
systemd-run --on-calendar="02:00" rsync -av /home/ /backup/

Monitoring background processes

# Monitor a background process by PID
PID=12345

# Check if it's still running
kill -0 $PID 2>/dev/null && echo "Running" || echo "Finished"

# See its CPU and memory usage
ps -p $PID -o pid,%cpu,%mem,stat,etime,comm

# Watch its resource usage
watch -n 2 "ps -p $PID -o pid,%cpu,%mem,rss,stat"

# Follow output redirected to a log file
tail -f /var/log/background-job.log

# Wait for a background job to complete (in a script)
nohup long-command > /tmp/output.log 2>&1 &
JOB_PID=$!
wait $JOB_PID
echo "Job completed with exit code $?"

Conclusion

For production background services: always use systemd service units — they handle startup on boot, restart on failure, and centralized logging. For long-running SSH sessions: use tmux or screen to protect against disconnect. For one-off background tasks from the command line: nohup command > logfile 2>&1 & with the PID saved to monitor it. Avoid leaving background processes running unmonitored without a log file — you need to know when they finish and whether they succeeded.