Shared build cache across developers and CI

One build cache serves everybody: an artifact computed once - on a developer's laptop or on a CI runner - is reused by every later build that needs it, instead of being recomputed per machine. The cache is content-addressed, network-accessible, and populated by whoever gets there first. A developer who compiles a module on Monday morning has, without intending to, made that module free for the rest of the team and for every CI run that touches the same commit.

The shift from a per-machine cache to a shared one is smaller in configuration than it is in consequence. Local caching already removes the second build on one machine; sharing removes the second build anywhere. The most valuable case is the one that never happens with local caches: CI, running on an ephemeral container that has never seen this repository, hits a warm cache because a developer or an earlier run already computed what it needs. The run that used to be structurally cold becomes structurally warm.

This is a build system capability and it stops at the build system's boundary. How many runners exist, how work is queued and who waits behind whom belong to the CI/CD Pipeline; a shared cache makes each run cheaper, not better scheduled. The two are frequently confused because both present as "CI is slow", and the diagnosis matters: a shared cache fixes redundant computation, and no amount of it fixes contention.

What makes the cache trustworthy is not the storage but the build tool. A shared cache is only safe if a cache key genuinely captures everything a step consumed - inputs, tool versions, flags, platform - which is why build systems with declared, hermetic actions can share caches across machines and ad-hoc build scripts cannot. A shared cache built on unreliable keys does not merely miss; it serves the wrong artifact to someone else, which is a far worse failure than a slow build.

• CI stops paying the cold-start penalty - the run on a fresh container reuses what a developer already computed, which is the single largest source of avoidable CI time in most codebases
• Cost per build falls as team size rises - a shared cache turns more people building the same code from a multiplier on compute into a multiplier on hit rate
• Agent iteration gets its floor lowered - agents verify constantly, and verification that reuses most of its work is what makes a fleet's feedback loop tolerable
• Onboarding and branch switching stop being expensive - a new machine, a fresh clone or a jump to a long-lived branch pulls artifacts rather than recomputing them
• Cache hit rate becomes a usable health metric - once it exists, a falling hit rate is an early and specific warning that something has become non-deterministic, long before anyone reports a slow build

1. Confirm your build is reproducible before you share anything - Build the same commit twice on two different machines and compare outputs. Embedded timestamps, absolute paths, unpinned tool versions and undeclared inputs all produce artifacts that must not be shared. Fix these first: a shared cache amplifies non-determinism into cross-machine bugs.
2. Stand up the cache backend your build tool already speaks - Gradle has a build cache node and Develocity, Bazel and other REAPI-compatible tools have remote cache services including self-hostable options, Turborepo has remote caching, sccache backs onto object storage. Use the native integration rather than building your own layer around it.
3. Make CI a writer and developers readers, at least at first - The safest rollout has CI populating the cache and developers reading from it. It gives immediate local speedups, limits the blast radius of a bad entry to a controlled writer, and lets you watch hit rates before opening writes more widely.
4. Converge developer and CI build configuration - Sharing only works when the two produce interchangeable artifacts. Pin toolchain versions, align flags, and remove environment-dependent behaviour. Any difference between how a laptop and a runner build is a difference that silently halves your hit rate.
5. Instrument hit rate and treat it as a monitored signal - Track hit rate per build step, and alert when it drops. A step that suddenly stops hitting has usually gained an undeclared input - a generated file, an environment variable, a timestamp - and catching it the same week is the difference between a small fix and an archaeology project.
6. Set retention and access rules deliberately - Decide how long entries live, how large the store may grow, and who may write. Cache write access is effectively the ability to influence other people's builds, so it belongs behind authentication and should be scoped, not handed out by default.

Sharing a cache built on unreliable keys. If a build step reads a file it never declared, its cache key does not describe its behaviour, and a hit on another machine returns an artifact built from different inputs. The result is a wrong build that reproduces nowhere and is blamed on everything except the cache. Hermeticity is the precondition, not an optimisation to follow later.

Letting everyone write to the shared cache from day one. Open write access means any misconfigured machine can poison entries for the whole organisation, and finding the culprit afterwards is unpleasant. Start with a trusted writer, expand deliberately, and always require authentication.

Ignoring the network cost. A shared cache trades computation for transfer. For small artifacts this is overwhelmingly worthwhile; for multi-gigabyte outputs over a slow link it can be slower than rebuilding. Measure rather than assume, and consider a local mirror for offices or regions that are far from the store.

Configuring developers and CI differently and then blaming the cache. Different flags or toolchain versions produce different keys, so the two populations quietly maintain two disjoint caches and the headline benefit never materialises. Teams then conclude that shared caching does not work for them. Check key agreement explicitly before drawing conclusions.

Confusing this with a capacity fix. A shared cache reduces the work each run performs. If runs are waiting rather than working - queued behind a concurrency limit, a busy runner pool or a shared test fixture - the cache will not help, and the pipeline's orchestration is where the fix lives. Diagnose queue time and execution time separately.