rsync: Syncing a Cache Directory Between Jobs
rsync moves only changed files, which makes it a good fit for restoring and saving a CI cache directory incrementally.
When a built-in cache action is not enough, rsync to and from a cache host gives fine control over what gets stored and how stale entries are pruned.
What it does
rsync syncs a local cache directory (node_modules, ~/.cache, a build cache) to or from a persistent location, transferring only deltas. On restore you pull the cache down; after the build you push changes back up. Because rsync is incremental, repeated saves are cheap.
Common usage
# Restore: pull the cache down at job start
rsync -a --delete cache-host:/caches/build/ .cache/
# Save: push the updated cache back at job end
rsync -a --delete .cache/ cache-host:/caches/build/Cache sync choices
| Flag | Why for caches |
|---|---|
| -a | Preserve modes/times for correct incrementals |
| --delete | Prune entries removed from the working cache |
| --checksum | Sync by content if builds reset mtimes |
| --max-delete=N | Avoid wiping the cache on an empty run |
| --exclude | Skip lock files or volatile temp data |
In CI
If your tooling regenerates cache files with new mtimes but identical content, the default size+mtime check re-uploads them every run. Add --checksum so only real changes sync. Cap unbounded growth by periodically clearing the cache host or excluding paths you do not want persisted.
Common errors in CI
A first run with no remote cache yet can fail with "rsync: change_dir ... failed: No such file or directory (2)" then "rsync error: some files/attrs were not transferred (code 23)". Create the remote path first (ssh host mkdir -p) or ignore the restore failure on a cold cache, then push at the end.