GNU parallel: Run Shell Jobs in Parallel

GNU parallel reads arguments (from ::: or stdin) and runs a command for each one concurrently, replacing {} with the argument and using all CPU cores by default.

parallel is the most flexible way to fan a command out over a list of inputs. The :::, {}, and --jobs trio covers most CI uses.

What it does

GNU parallel builds and runs command lines from input. Arguments come after ::: on the command line or from stdin; {} is replaced by each argument ({.} strips the extension, {/} the dirname). It runs as many jobs at once as you have cores unless --jobs says otherwise, and it preserves per-job output order with --keep-order.

Common usage

parallel echo ::: a b c                       # 3 jobs: echo a, echo b, echo c
parallel -j4 gzip ::: *.log                   # compress, 4 at a time
ls *.png | parallel convert {} {.}.webp       # from stdin, {} and {.}
parallel --halt now,fail=1 ./test.sh ::: $(seq 1 10)
parallel ::: ./build.sh ./lint.sh ./test.sh   # run 3 scripts at once

Options

In CI

parallel returns a non-zero exit code if any job failed, so a test fan-out gates correctly; add --halt now,fail=1 to stop early on the first failure. Pin --jobs to the runner core count to avoid oversubscription. Use --joblog to capture which input failed when the combined output is hard to read.

Common errors in CI

On first run parallel prints a citation notice that can pollute logs or block scripts; run parallel --citation once or add --will-cite. "parallel: command not found" (install via apt-get install -y parallel or brew install parallel); note some distros ship the unrelated moreutils parallel, so verify with parallel --version. Interleaved output is normal; use -k for ordered logs.

Related guides