zstd: Fast Compression for CI Build Caches

zstd compresses files to .zst with a wide range of levels and multi-threading, giving near-xz ratios at far higher speed, which is why CI caches favor it.

zstd is the modern default for build caches and artifacts: fast at low levels, small at high levels, and multi-threaded with -T0. It is what GitHub Actions cache uses under the hood.

What it does

zstd compresses input with the Zstandard algorithm, appends .zst, and keeps the source by default (unlike gzip). Levels run 1 to 19 normally, up to 22 with --ultra. -T0 uses all cores, --long enables a large window for better ratio on big inputs, and --adapt tunes the level to available I/O.

Common usage

Terminal

zstd -19 --long -T0 build.tar        # small, all cores, big window
zstd -3 cache.tar                    # fast level for hot caches
tar cf - node_modules/ | zstd -T0 -o deps.tar.zst
zstd -d deps.tar.zst                 # decompress
zstd --adapt big.log                 # adapt level to throughput

Options

Flag	What it does
-1 .. -19	Compression level (higher = smaller, slower)
--ultra -22	Unlock levels above 19
-T0 / -T<n>	Multi-threaded compression (0 = all cores)
--long[=<n>]	Enable long-distance matching for big inputs
--adapt	Adapt the level to available I/O speed
-d / --decompress	Decompress a .zst file
-o <file>	Explicit output path (needed when piping)

In CI

zstd is the best default for build caches: level 3 for speed on hot paths, -19 --long -T0 when the cache is large and reused often. Restoring is fast at any level. Install with apt-get install -y zstd or apk add zstd. Decompression ignores the level used to compress.

Common errors in CI

"zstd: /*stdin*\\: unsupported format" means the input is not a zstd stream (wrong tool or corrupt data). "zstd: command not found" means the zstd package is missing. "Decoding error (36) : Frame requires too much memory for decoding" appears when a file was written with --long and a large window; decompress with --long=31 to allow the window. "error : No output specified" when piping means you omitted -o.