sponge: Safely Write Back to the Same File
sponge reads all of stdin, then writes it to a file, so a pipeline can safely edit the same file in place.
The classic cmd file > file bug truncates the file before cmd reads it. sponge buffers the whole stream first, making in-place filtering safe.
What it does
sponge, from the moreutils package, reads its entire standard input into memory before opening and writing the output file. That ordering is the whole point: grep x file | sponge file works, whereas grep x file > file empties the file before grep can read it. -a appends instead of overwriting.
Common usage
# filter a file in place (safe)
grep -v DEBUG app.log | sponge app.log
# sort a file in place
sort data.txt | sponge data.txt
# expand tabs in place
expand -t 4 src.py | sponge src.py
# append the filtered result instead
grep ERROR app.log | sponge -a errors.logOptions
| Flag / form | What it does |
|---|---|
| sponge <file> | Soak all stdin, then overwrite file |
| -a | Append to file instead of overwriting |
| (no file) | Write soaked input to stdout |
In CI
sponge removes the need for a temp-file-and-mv dance in one-liners: jq '.version="1.2.3"' pkg.json | sponge pkg.json edits the file cleanly. It also preserves the output file's permissions in place, unlike a redirect that recreates the file, which matters for scripts and executables.
Common errors in CI
"sponge: command not found" is the usual issue: it is not coreutils but part of moreutils (apt-get install -y moreutils, apk add moreutils, brew install moreutils). Since sponge buffers everything in memory, a multi-gigabyte stream can exhaust RAM; for huge files use a temp file and mv instead. If a step still empties a file, confirm you piped into sponge and did not accidentally use > file.