awk: Usage, Options & Common CI Errors

awk processes text record by record, splitting each into fields you can address as $1, $2, …

awk shines at column extraction and quick aggregations in pipelines. Most CI bugs come from field-separator assumptions and from quoting the program against the shell.

What it does

awk reads input line by line, splits each line into fields (default on whitespace), and runs a program of pattern { action } rules. It is a full language with variables, arithmetic, and arrays.

Common usage

Terminal

awk '{print $2}' file.txt                  # second whitespace field
awk -F: '{print $1}' /etc/passwd           # colon separator
awk '/ERROR/ {c++} END {print c}' log.txt  # count matches
ps aux | awk '$3 > 50 {print $2}'          # PIDs over 50% CPU
awk -v t="${THRESH}" '$1 > t' data.txt

Options

Flag / item	What it does
-F <sep>	Set the input field separator
-v name=val	Pass a shell value into an awk variable
$1, $2, $NF	First, second, last field
NR / NF	Current record number / field count
BEGIN{} / END{}	Run before / after all input

Common errors in CI

awk: syntax error at source line 1 usually means the shell ate part of the program - wrap the program in single quotes and pass shell values with -v, never by string interpolation. Default whitespace splitting collapses runs of spaces, so a field index can shift when columns are blank; set -F explicitly. mawk (Debian default) lacks some gawk extensions like gensub() and length() on arrays - guard against version-specific functions.