Journalctl Complete Guide

Journalctl Complete Guide

The systemd journal (managed by journald) collects logs from the kernel, all system services, and applications. Unlike text log files, the journal is a structured binary format that stores extra metadata with each entry: priority level, systemd unit, PID, UID, executable path. This structure enables powerful filtering that text-based grep cannot easily replicate. journalctl is the command-line query tool for the journal.

Why journalctl over log files?

journalctl advantages over /var/log/ text files:

  grep /var/log/syslog                  journalctl
  ─────────────────────                 ──────────────────────────────
  Text search only                      Structured metadata filters
  Must know which file                  Single query across all sources
  No priority filtering                 -p err, -p warning etc.
  No boot-scoped queries                -b 0 (current), -b -1 (last)
  Can miss wrapped long lines           Full structured records always
  Needs root to read auth.log           Members of adm/systemd-journal group can read

Basic usage

journalctl                   # All journal entries, oldest first (in pager)
journalctl -r                # Reverse order — newest first
journalctl -n 50             # Last 50 lines
journalctl -f                # Follow mode — like tail -f
journalctl -e                # Jump to end of journal immediately

Filtering by time

# Time ranges
journalctl --since "2025-06-09 14:00:00"
journalctl --since "1 hour ago"
journalctl --since "10 minutes ago"
journalctl --since today
journalctl --since "2025-06-09" --until "2025-06-10"

# Boot-scoped queries (very useful for finding what caused a crash)
journalctl -b        # Current boot only
journalctl -b -1     # Previous boot (use after a crash to see what happened)
journalctl -b -2     # Boot before that

# List available boots
journalctl --list-boots

journalctl --list-boots output

-2 7f3b... Mon 2025-06-07 09:00:00 UTC—Mon 2025-06-07 17:34:01 UTC
-1 2c1a... Mon 2025-06-08 09:00:00 UTC—crash (power failure)
 0 9d4f... Tue 2025-06-09 09:00:15 UTC—Tue 2025-06-09 15:10:00 UTC

Filtering by unit and priority

# Filter by systemd unit (service name)
journalctl -u nginx                # All nginx logs
journalctl -u nginx -u postgresql  # Logs from both services
journalctl -u nginx -f             # Follow nginx logs in real time
journalctl -u ssh --since "1 hour ago"

# Filter by priority level (-p filters to that level AND above)
# Levels: emerg(0) alert(1) crit(2) err(3) warning(4) notice(5) info(6) debug(7)
journalctl -p err           # Errors and above (err, crit, alert, emerg)
journalctl -p warning       # Warnings and above
journalctl -p err --since "1 hour ago"  # Recent errors

# Filter by kernel messages only
journalctl -k               # Kernel messages (equivalent to dmesg)

Persistent journal storage

# By default, Ubuntu may store journal in memory only (lost on reboot)
# Check current storage location:
journalctl --disk-usage

# To enable persistent storage (survives reboots):
sudo mkdir -p /var/log/journal
sudo systemd-tmpfiles --create --prefix /var/log/journal
sudo systemctl restart systemd-journald

# Configure journal size limits (in /etc/systemd/journald.conf):
# SystemMaxUse=500M        ← Max journal disk space
# SystemKeepFree=1G        ← Keep at least 1G free on disk
# MaxRetentionSec=1month   ← Delete entries older than 1 month
sudo nano /etc/systemd/journald.conf
sudo systemctl restart systemd-journald

# Manually vacuum old journal entries:
sudo journalctl --vacuum-time=2weeks    # Delete entries older than 2 weeks
sudo journalctl --vacuum-size=200M      # Shrink to 200MB

Conclusion

The most useful journalctl patterns day-to-day: journalctl -u servicename -f to follow a service's logs, journalctl -b -1 -p err to see what errors occurred in the previous boot (critical after unexpected reboots), and journalctl --since "1 hour ago" -p warning to see recent problems. Enable persistent journal storage on any server you care about — without it you lose logs on every reboot.