uniq -d and -u: Show Dupes or Uniques Only

uniq -d prints lines that appeared more than once; uniq -u prints lines that appeared exactly once.

To find duplicates in a list (or, conversely, the singletons), -d and -u filter to exactly those cases.

What it does

uniq -d outputs one copy of each line that has at least one adjacent duplicate, and uniq -u outputs only lines with no adjacent duplicate. -D prints every copy of the duplicated lines, not just one. As always, the comparison is over adjacent lines, so sort first.

Common usage

Terminal

sort names.txt | uniq -d         # values that repeat
sort names.txt | uniq -u         # values that appear once
sort names.txt | uniq -D         # all copies of repeated lines

Options

Flag	What it does
-d	Print one copy of each duplicated line
-u	Print only lines that are not repeated
-D	Print all copies of duplicated lines
-c	Add counts (with -d to count dupes)
-i	Case-insensitive comparison

In CI

sort file | uniq -d is a quick duplicate detector, for example flagging duplicate entries in a generated allowlist before it ships. Exit on non-empty output to fail the build when duplicates appear.

Common errors in CI

-d and -u are mutually exclusive in effect: -d shows repeats, -u shows singletons, so do not expect both at once. Forgetting to sort makes -u report nearly everything as unique because runs are broken up by intervening lines. -D is GNU; BSD/macOS uniq supports -d and -u but the -D long-run form differs.