awk Command in Linux: Text Processing Guide

Quick take: The awk command processes text field by field. Print a column with awk '{print $2}', set the separator with -F, and filter rows with patterns like awk '$3 > 100'. It excels at columnar data and reports.

Introduction

The awk command is a complete text-processing language built for column-based data. It splits each line into fields, lets you reference them as $1, $2, and so on, and runs your program against every line. From extracting a column to producing a formatted report with running totals, awk does in one line what would take a loop in most languages.

This guide introduces fields and records, filtering, the field separator, and the BEGIN/END blocks that make awk so capable.

Syntax

The basic syntax of the awk command is:

awk [OPTIONS] 'PATTERN { ACTION }' [FILE...]

Common Options and Parameters

The most useful options and parameters for the awk command:

Practical Examples

Real awk commands you can run today:

# Print the first column
awk '{print $1}' access.log
# Print columns 1 and 3 with a label
awk '{print $1, $3}' data.txt
# Use a custom separator (the passwd file)
awk -F ':' '{print $1}' /etc/passwd
# Filter rows where column 3 is over 100
awk '$3 > 100' sales.txt
# Print the line number with each line
awk '{print NR, $0}' file.txt
# Sum a column and print the total
awk '{sum += $2} END {print sum}' sales.txt
# Print lines with more than 4 fields
awk 'NF > 4' data.txt

Building Reports with awk

awk's BEGIN and END blocks turn raw data into formatted reports. BEGIN runs before any input — ideal for printing a header — and END runs after the last line, where you print totals you accumulated along the way.

# Sum a column with a header and total
awk 'BEGIN {print "User  Bytes"} {sum += $2; print $1, $2} END {print "Total:", sum}' usage.txt

# Average of a column
awk '{total += $3; n++} END {print "Average:", total/n}' scores.txt

# Count occurrences of each value (a frequency table)
awk '{count[$1]++} END {for (k in count) print k, count[k]}' access.log

That last pattern — using an associative array to tally values — is one of awk's most useful idioms, summarising a log in a single line.

Tips and Best Practices

• $0 is the whole line, $1 the first field — and $NF is always the last field, handy when the column count varies.
• Use BEGIN for headers and setup, and END for totals and summaries computed after the last line.
• Set the separator with -F for CSVs (-F ',') or system files like /etc/passwd (-F ':').

Final Thoughts

awk turns columnar text into structured data you can filter, reformat, and summarise in a single command. Learn fields ($1, $NF), the -F separator, simple conditions, and BEGIN/END blocks, and you will reach for awk constantly when grep and sed are not enough. It is the third pillar of Linux text processing.