Skip to content
Latchkey

ccache + distcc not caching or distributing in CI

ccache and distcc must be layered so ccache checks the cache first and only distributes real misses. The correct wiring is ccache in front with CCACHE_PREFIX=distcc; get it wrong and you lose caching, distribution, or both.

What this error means

Builds are slow because distcc runs every compile (no cache hits), or distcc never contacts hosts and everything compiles locally. ccache -s and distcc --show-hosts disagree with expectations.

ccache
# wrong: distcc wraps ccache, so distcc runs before the cache is consulted
export CXX="distcc ccache g++"   # caches nothing useful
# right: ccache in front, distcc as the prefix for misses
export CXX="ccache g++"
export CCACHE_PREFIX="distcc"

Common causes

The wrapper order is inverted

Putting distcc in front of ccache sends every compile to distcc before the cache is checked, so cacheable hits are lost.

CCACHE_PREFIX is not set to distcc

Without CCACHE_PREFIX=distcc, ccache runs the compiler locally on a miss, so distribution never happens.

How to fix it

Layer ccache in front with CCACHE_PREFIX

  1. Use ccache as the compiler launcher so the cache is consulted first.
  2. Set CCACHE_PREFIX=distcc so only real misses are distributed.
  3. Confirm hosts with distcc --show-hosts and hits with ccache -s.
Terminal
export CC="ccache gcc" CXX="ccache g++"
export CCACHE_PREFIX="distcc"
export DISTCC_HOSTS="localhost/4 10.0.0.5/8"

Verify both layers report activity

Check that ccache shows hits and distcc shows distributed jobs, so you know both layers engage.

Terminal
ccache -s | grep 'Cache hit rate'
distccmon-text 1

How to prevent it

  • Always put ccache in front and distribute misses via CCACHE_PREFIX=distcc.
  • Verify hosts and hit rate after wiring the two tools.
  • Keep DISTCC_HOSTS in sync with reachable build hosts.

Related guides

Tired of flaky CI? Latchkey auto-heals failed jobs and retries them for you. Start free →