sar Command in Linux: System Activity Reporter Guide

Quick take: sar is the System Activity Reporter — part of the sysstat toolkit. Use sar -u 1 5 for CPU, sar -r 1 5 for memory, sar -d 1 5 for disk I/O, and sar -n DEV 1 5 for network. Add -f /var/log/sysstat/saDD to read historical data from a specific day.

Introduction

Most performance monitoring tools answer the question "what is happening right now?" — top, htop, vmstat, iostat all show you a live snapshot. That is useful when you are watching an incident unfold. But some of the most important performance questions are about the past: the server was slow at 3am last Tuesday, what was the CPU doing? The database started responding slowly around 14:00, was there a disk I/O spike? The application began throwing errors overnight, did memory run out?

sar is the tool for those questions. It reads from data files that the sysstat data collector writes at regular intervals — every 10 minutes by default — building a historical record of system performance that you can query later. You look at it after the fact, narrow down when the problem started, and see exactly what the system was doing at that moment. No guessing, no correlating scattered log timestamps — just the numbers.

I keep sysstat enabled on every server I manage. It is a tiny overhead (negligible CPU and disk), and it has saved more post-incident investigations than I can count. When something happens at 2am and you are looking at it at 8am, sar is often the only source of ground truth about what the server was doing during those hours.

Installing sysstat

sar is part of the sysstat package. It is not always installed by default:

# Debian, Ubuntu
sudo apt install sysstat

# RHEL, CentOS, AlmaLinux, Rocky Linux
sudo dnf install sysstat

# Arch Linux
sudo pacman -S sysstat

After installing on Debian/Ubuntu, you must enable the data collector and start it:

# Edit the sysstat configuration to enable the collector
sudo nano /etc/default/sysstat
# Change: ENABLED="false"
# To:     ENABLED="true"

# Enable and start the sysstat service
sudo systemctl enable sysstat --now

# Verify it is running and see when the next collection is scheduled
systemctl status sysstat

On RHEL-based systems, the service is typically enabled automatically on install. The data collector (sadc) writes binary log files to /var/log/sysstat/ (or /var/log/sa/ on some systems), one file per day named sa01 through sa31. Data collection starts from the day you enable it — there is no retroactive data.

Syntax

The basic syntax of the sar command:

sar [OPTIONS] [INTERVAL [COUNT]]

INTERVAL is how many seconds between samples, and COUNT is how many samples to take. Running sar -u 1 10 takes 10 CPU measurements, one per second. Without INTERVAL and COUNT, sar prints the accumulated average from the current day's historical data file. Running sar -u with no interval shows today's CPU statistics hour by hour.

CPU Usage Statistics

sar -u shows CPU utilisation, broken down into the categories that matter for diagnosing what kind of load the CPU is under:

# CPU usage: 1 sample per second, 10 samples
sar -u 1 10

# CPU usage every 5 seconds, 12 samples (1 minute)
sar -u 5 12

# All CPUs individually on a multi-core system
sar -u ALL 1 5

# Per-core breakdown
sar -P ALL 1 5

Sample output of sar -u 1 3:

Linux 6.14.0 (webserver)   06/21/26   _x86_64_   (4 CPU)

14:32:01    CPU    %user  %nice  %system  %iowait  %steal  %idle
14:32:02    all    23.45   0.00     4.12     1.23    0.00   71.20
14:32:03    all    19.87   0.00     3.45     0.87    0.00   75.81
14:32:04    all    21.34   0.00     3.89     1.05    0.00   73.72
Average:    all    21.55   0.00     3.82     1.05    0.00   73.58

The columns to focus on: %user is CPU time spent running user-space processes — this is your application workload. %system is kernel time — high system time can mean excessive system calls, context switching, or interrupt handling. %iowait is time the CPU was idle waiting for I/O to complete — consistently above 20% means your storage is a bottleneck. %steal is time the hypervisor took from your virtual CPU — if you are on a cloud VM and steal is regularly above 5%, your host is overcommitted. %idle is time the CPU was idle.

A server with 21% user, 4% system, 1% iowait, and 74% idle is healthy and has room to grow. A server with 80% user and 0% idle is maxed out. A server with 5% user, 2% system, 40% iowait, and 53% idle has a storage bottleneck — the CPU is mostly sitting around waiting for disks to respond, even though it looks "available".

Memory Usage Statistics

sar -r shows memory statistics and sar -S shows swap usage:

# Memory usage: 1 sample per second for 10 samples
sar -r 1 10

# Swap usage
sar -S 1 5

# Combined memory + swap
sar -r -S 1 5

Sample output of sar -r 1 3:

14:32:01    kbmemfree  kbavail  kbmemused  %memused  kbbuffers  kbcached  kbcommit  %commit
14:32:02      1234567  3456789    5678901     62.11     345678   2345678   8901234     97.45
14:32:03      1198456  3412678    5715012     62.50     345890   2346001   8945678     97.93

The key columns: kbavail is the best measure of available memory — it includes reclaimable cache, not just truly free memory. If kbavail is near zero and swap is being used, the system is under memory pressure. %memused alone is misleading because Linux aggressively uses free memory for file caching, so 80% memused is normal. kbavail near zero is the real warning sign. %commit above 100% means the system has promised more memory to running processes than it physically has — relying on not all of it being used at once. Above 150% is risky on a server under variable load.

Disk I/O Statistics

sar -d shows disk I/O statistics per block device:

# Disk I/O per second for all devices
sar -d 1 5

# More readable output with device names
sar -d -p 1 5

# Specific device
sar -d -p 1 5 | grep sda

Sample output of sar -d -p 1 3:

14:32:01    DEV       tps    rkB/s    wkB/s    areq-sz    aqu-sz    await    svctm    %util
14:32:02    sda     45.00   2340.00   890.00     72.91      0.23     5.12     1.23    55.40
14:32:03    sdb      2.00     12.00     8.00     10.00      0.01     1.45     0.89     0.18

The columns to watch: tps is transfers per second (I/O operations). rkB/s and wkB/s are read and write throughput in kilobytes per second. await is the average time in milliseconds an I/O request takes from submission to completion — high await on an SSD (above 10ms) often indicates queueing. %util is the percentage of time the device was busy — 100% means the device is saturated and all requests are queuing.

A healthy SSD should show await under 2ms and %util rarely above 50% during normal operation. An HDD under significant load might show await of 10–30ms. If you see %util at 100% consistently with high await, the storage is the bottleneck — consider upgrading the disk, adding I/O caching, or distributing load across multiple volumes.

Network Interface Statistics

sar -n DEV shows network interface statistics — packets and bytes transmitted and received per second:

# Network statistics for all interfaces
sar -n DEV 1 5

# Network errors (dropped packets, collisions)
sar -n EDEV 1 5

# TCP connection statistics
sar -n TCP 1 5

# UDP statistics
sar -n UDP 1 5

# Socket statistics
sar -n SOCK 1 5

Sample output of sar -n DEV 1 2:

14:32:01   IFACE     rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s  txcmp/s  rxmcst/s  %ifutil
14:32:02   eth0      1245.00    987.00    890.45    234.12      0.00     0.00      0.00     8.91
14:32:03   lo          12.00     12.00      1.23      1.23      0.00     0.00      0.00     0.00

%ifutil shows what percentage of the interface's maximum bandwidth is in use. For a 1Gbps NIC, 8.91% means about 89Mbps of traffic. If you are seeing %ifutil consistently above 80%, the network interface is becoming a bottleneck. The EDEV report is important for detecting packet loss — if you see rxdrop/s or txdrop/s increasing, packets are being dropped, which causes TCP retransmits and degraded throughput.

Load Average and Run Queue

sar -q shows the run queue length and load average — how many processes are waiting to run versus actually running:

sar -q 1 5

14:32:01   runq-sz  plist-sz  ldavg-1  ldavg-5  ldavg-15  blocked
14:32:02         2       234     1.23     1.45      1.12        0
14:32:03         4       236     1.67     1.52      1.14        1

runq-sz is the number of processes in the run queue (ready to run but waiting for a CPU). On a 4-core server, a run queue consistently above 8–12 means the CPUs are oversubscribed. blocked is the number of processes blocked waiting for I/O — if this is consistently non-zero and correlating with high iowait in sar -u, your storage is causing the bottleneck.

Common Options and Flags

Reading Historical Data

The most powerful feature of sar is the ability to read historical data. The sysstat data collector stores daily files in /var/log/sysstat/:

# See today's full CPU history (all intervals since midnight)
sar -u

# Read yesterday's data
sar -u -f /var/log/sysstat/sa$(date -d yesterday +%d)

# Read data from a specific date (e.g., the 15th of the month)
sar -u -f /var/log/sysstat/sa15

# Read a specific time window from historical data
sar -u -s 14:00:00 -e 16:00:00 -f /var/log/sysstat/sa20

# Check yesterday's disk I/O between 2am and 4am
sar -d -p -s 02:00:00 -e 04:00:00 -f /var/log/sysstat/sa$(date -d yesterday +%d)

# Read memory history for today and pipe to grep for the peak hour
sar -r | grep -v "^$\|Average\|Linux\|CPU"

The -s and -e flags let you narrow the historical query to exactly the time window you are investigating. This is invaluable for post-incident analysis: the server was slow from 14:00 to 15:30, so you run sar -u -s 14:00:00 -e 15:30:00 -f /var/log/sysstat/saXX and immediately see whether it was CPU, memory, disk, or network that was stressed during that window.

Practical Examples

Real sar commands from post-incident analysis and capacity planning:

# Find the highest CPU usage point in the last 7 days
for i in $(seq 1 7); do
  date=$(date -d "$i days ago" +%d)
  echo "=== $(date -d "$i days ago" +%Y-%m-%d) ==="
  sar -u -f /var/log/sysstat/sa$date 2>/dev/null | \
    awk '$3~/[0-9]/ && $3+0 > 80 {print}'
done

# Check if memory was exhausted during a specific incident window
sar -r -s 02:00:00 -e 04:00:00 -f /var/log/sysstat/sa$(date -d yesterday +%d)

# Find I/O-heavy periods: disks above 80% utilisation
sar -d -p -f /var/log/sysstat/sa$(date +%d) | awk '$NF+0 > 80'

# Monitor network traffic every 30 seconds continuously
sar -n DEV 30 0

# Log performance to a file for later analysis
sar -u -r -d -n DEV 60 1440 > /tmp/perf-$(date +%Y%m%d).txt

# Calculate average CPU usage for today
sar -u | grep Average

sar vs top, vmstat, iostat

Each of these tools has a specific role in performance analysis and they complement rather than replace each other:

top / htop — real-time, per-process view. Shows which process is using the most CPU or memory right now. Cannot show historical data. Use when you need to identify which process is causing current high usage.

vmstat — lightweight real-time view of memory, CPU, and I/O summary. Good for a quick health check. Shows one line per interval, easy to watch for sustained issues. No historical data beyond what you observe in the terminal.

iostat — part of sysstat, focuses specifically on disk I/O per device. More detail than sar -d on current I/O, good for real-time disk debugging. Can also read sysstat data files for historical I/O analysis.

sar — the historical record. The main thing sar provides that no other tool in this list does is easy access to what happened in the past, at specific times, across all resource categories simultaneously. When investigating "why was the server slow at 3am", sar is the right tool. For debugging something happening right now, start with top or vmstat.

The tools work best together: use sar to identify the time window and the resource type involved, then reproduce or investigate further with the more focused real-time tools once you know what you are looking for.

sar in Monitoring and Automation

sar output is text-based and pipe-friendly, making it straightforward to use in monitoring scripts:

#!/bin/bash
# Alert if CPU idle drops below 20% (80%+ used)
IDLE=$(sar -u 1 3 | grep Average | awk '{print $NF}')
if (( $(echo "$IDLE < 20" | bc -l) )); then
  echo "ALERT: CPU idle is ${IDLE}% — high CPU load" | mail -s "CPU Alert $(hostname)" admin@company.com
fi

#!/bin/bash
# Generate daily performance summary
echo "=== Performance Summary: $(date +%Y-%m-%d) ===" > /tmp/daily-report.txt
echo "" >> /tmp/daily-report.txt
echo "--- CPU ---" >> /tmp/daily-report.txt
sar -u | tail -5 >> /tmp/daily-report.txt
echo "" >> /tmp/daily-report.txt
echo "--- Memory ---" >> /tmp/daily-report.txt
sar -r | tail -5 >> /tmp/daily-report.txt
echo "" >> /tmp/daily-report.txt
echo "--- Disk ---" >> /tmp/daily-report.txt
sar -d -p | tail -10 >> /tmp/daily-report.txt
mail -s "Daily Report $(hostname)" admin@company.com < /tmp/daily-report.txt

For structured data export, sar also supports JSON output on newer sysstat versions: sar -u 1 5 --json. This makes it easy to feed sar data into Elasticsearch, InfluxDB, or any other time-series system for longer-term retention and graphing alongside application metrics.

Common Mistakes

The most common mistake is installing sysstat but forgetting to enable the data collector. You install the package, try sar -u, and get "No data available for requested date". The fix is editing /etc/default/sysstat to set ENABLED=true and restarting the service. On RHEL-based systems this usually happens automatically, but on Debian/Ubuntu the manual enable step is required.

A second mistake is reading the %memused column in panic. Linux intentionally uses free memory as a disk cache, so %memused of 85–95% is completely normal on a healthy server. The column to watch is kbavail. If kbavail is consistently near zero, the system is under genuine memory pressure. If it is healthy (more than a few hundred megabytes), the memory usage numbers just reflect cache, not a problem.

Confusing %iowait with CPU utilisation is another frequent error. iowait means the CPU is idle but waiting for I/O — the CPU is not being used, but it cannot be used for other work because a process is blocked waiting for disk or network I/O. High iowait looks like "the server is slow and CPU is not high" — the CPU is idle but constrained by I/O. The fix is storage optimisation, not adding CPU cores.

Tips and Best Practices

• Install and enable sysstat on every server as part of your initial setup. The historical data it provides is invaluable the first time something goes wrong at 3am.
• Reduce the collection interval to 5 minutes (from the default 10) for more granular data on production servers — edit the cron job in /etc/cron.d/sysstat.
• When investigating a past incident, start with sar -u -r -d -n DEV -f on the same file to get all resource types at once, then narrow to the relevant one.
• Use kbavail, not %memused, as your memory health indicator.
• A %iowait above 20% sustained is a disk I/O problem, not a CPU problem — do not add CPU capacity to fix it.
• sysstat data files are rotated automatically. Check your retention with ls /var/log/sysstat/ — by default you get about 7 or 28 days depending on distribution, which is usually enough for post-incident analysis but may not be enough for capacity planning. Increase retention in /etc/sysstat/sysstat if needed.

Final Thoughts

sar is the performance monitoring tool that works when you are not watching. While top and vmstat require you to be at the terminal to see what is happening, sysstat's data collector quietly builds a complete record of every day's CPU, memory, disk, and network activity in the background. When an incident wakes you up at 6am, sar lets you look back and see exactly what was happening while you were asleep.

The investment in learning sar is small — a handful of flags, an understanding of which columns matter for each resource type — and the payoff is permanent. Every server you manage that has sysstat running is one you can investigate thoroughly after the fact. Install it, enable it, and the next time a performance problem is reported after the fact, you will already have the data.