rsync -P (--partial --progress) for Large Transfers
rsync -P is shorthand for --partial --progress: it keeps partially transferred files so a retry can resume instead of starting over.
When CI pushes large artifacts over a flaky link, --partial means a dropped connection does not waste the bytes already sent.
What it does
--partial tells rsync to keep a partially transferred file if the transfer is interrupted, so a subsequent run resumes from where it stopped rather than re-sending the whole file. --progress shows transfer progress. -P is the combination. For safer partials, --partial-dir=.rsync-partial stages incomplete files in a hidden directory.
Common usage
# Resume-friendly upload of large build artifacts
rsync -avP large-build/ user@host:/srv/releases/
# Stage partials in a sidecar dir to avoid corrupt-looking files
rsync -av --partial-dir=.rsync-partial large-build/ user@host:/srv/Options
| Flag | What it does |
|---|---|
| -P | Same as --partial --progress |
| --partial | Keep partial files for later resume |
| --partial-dir=DIR | Hold incomplete files in DIR until done |
| --append | Append data to shorter destination files |
| --append-verify | Append, then checksum the whole file |
In CI
On retry-prone deploys, wrap rsync in a retry loop and rely on --partial so each attempt makes forward progress. Avoid plain --append for files that may have changed in the middle; use --append-verify or just let rsync re-sync normally.
Common errors in CI
If a job is killed mid-transfer without --partial, the destination is left with a temporary file like ".index.html.XXXXXX" that rsync removes on the next clean run. With --partial you may briefly see a truncated file served; --partial-dir avoids exposing it under the live name.