sort -u: Sort and Deduplicate at Once
sort -u sorts the input and drops duplicate lines without needing a separate uniq step.
When you just need a sorted, deduplicated list, sort -u does both jobs and does not require the input to already be sorted.
What it does
sort -u sorts the input and outputs only the first of each run of equal lines, so the result is sorted and unique. Unlike piping to uniq, it does not require pre-sorted input because sort orders the data first.
Common usage
sort -u ips.txt
cat *.log | grep ERROR | sort -u
git diff --name-only | sort -u # unique changed filesOptions
| Flag | What it does |
|---|---|
| -u | Output only unique lines (first of each equal run) |
| -k <pos> | Define equality on a key; -u dedupes on that key only |
| -n | Combine with numeric ordering |
| -f | Fold case when comparing |
In CI
Reach for sort -u over sort | uniq when you only need a unique set; it is one process and never trips on unsorted input. Be aware that with -k, "duplicate" means equal on the key, so -u can drop lines that differ outside the key field.
Common errors in CI
sort -u -k1,1 keeps only one line per key value and discards the rest, which surprises people expecting whole-line dedupe; drop -k to compare full lines. Case folding (-f) and locale collation can treat lines as equal that look different; set LC_ALL=C for byte-exact comparison.