shuf: Random Sampling and Shuffling in CI
shuf -n K prints K randomly selected lines from its input, without replacement by default.
To run a random subset of a huge test list on every push, shuf -n picks the sample. For repeatable CI you seed it with --random-source.
What it does
shuf outputs a random permutation of its input lines. -n K limits output to K lines (a sample). -e shuffles its arguments instead of file lines, and -i LO-HI shuffles a numeric range. Part of coreutils.
Common usage
# run a random sample of 20 test files
shuf -n 20 test-list.txt | xargs pytest
# shuffle explicit items
shuf -e alpha beta gamma
# a random number in a range
shuf -i 1-100 -n 1
# reproducible sample: seed from a fixed source
shuf -n 20 --random-source=<(yes 42) test-list.txtOptions
| Flag | What it does |
|---|---|
| -n <k> | Output at most k lines |
| -e | Treat each argument as an input line |
| -i LO-HI | Shuffle the integer range LO..HI |
| -r | Allow repeats (sample with replacement) |
| -o <file> | Write to a file (safe for in-place shuffles) |
| --random-source=<file> | Use file as the randomness source (reproducible) |
In CI
A flaky "random test order" bug is impossible to reproduce unless the shuffle is seeded. Pin it with --random-source=<(yes SEED) and print the seed to the log, so a failed run can be replayed exactly. Combine with -o to shuffle a file in place without a truncation race.
Common errors in CI
"shuf: command not found" on Alpine means coreutils is missing (BusyBox has no shuf; apk add coreutils). "shuf: no lines to repeat" appears with -r on empty input. Note -n without -r cannot output more lines than the input has; asking for more just returns all of them shuffled.