Every build starts from scratch

Builds share nothing: every developer's machine and every CI run recomputes the same artifacts from source, over and over, because each cache dies with the process that produced it. A developer compiles the project. A colleague pulls the same commit and compiles it again. CI checks out the same commit on a fresh container and compiles it a third time. Three identical computations, no reuse, and the only artifact that survives is the one the pipeline was asked to publish.

Some of this is invisible because it looks like normal build time. A developer's local build tool may keep a cache in their home directory, so their second build is fast and they conclude the build is fine. CI has no such luxury: its runners are ephemeral by design, so every run is a first build. The gap between "fast on my machine" and "eleven minutes in CI" is usually not a difference in hardware. It is the difference between a warm local cache and no cache at all.

This is a property of the build system, not of the pipeline. Whether runs queue behind each other, how many runners exist and who gets them first are CI/CD Pipeline questions, and this guide leaves them there. The question here is narrower and more fundamental: given the same inputs, does your build recompute the same outputs? At this level the answer is yes, every time, and no amount of runner capacity changes that - it only lets you pay for the redundant computation in parallel.

The condition is stable until agents arrive. Redundant work is affordable when builds are infrequent; the cost is real but it is spread across a working day and everybody has learned to live with it. Agent fleets remove the slack. A team that goes from a few dozen builds a day to several hundred pays the full from-scratch cost several hundred times, and the first thing that breaks is not the budget but the feedback loop: agents iterate at the speed of their slowest verification, and a cold build is the slowest verification there is.

• The same computation is paid for many times a day - identical inputs producing identical outputs, recomputed per developer and per run, is pure waste that grows linearly with team size and build frequency
• CI is structurally slower than local development - ephemeral runners start cold every time, so CI experiences the worst case of a build that developers only ever see on a clean checkout
• Build time becomes the ceiling on agent iteration - an agent cannot converge faster than it can verify, and a from-scratch build sets that floor regardless of how capable the agent is
• Capacity spending papers over the problem - buying more runners makes redundant work concurrent rather than unnecessary, which is the most expensive way to not fix it
• It hides the real bottleneck - when everything is slow, nobody can tell whether the build, the tests or the pipeline is the constraint, so optimisation effort lands more or less at random

1. Measure cold and warm separately - Time a build on a clean checkout with no caches, then time an incremental build after a one-line change. The ratio is the size of the prize. Teams routinely discover their CI is paying a ten-to-one penalty that nobody had quantified because nobody had measured the two numbers side by side.
2. Turn on the build tool's own cache first - Gradle's build cache, Go's GOCACHE, Rust's target directory and sccache, Turborepo's local cache, and BuildKit's layer cache all exist and are usually off or default-scoped in CI. Local caching is a configuration change, not a project, and it is the prerequisite for anything shared later.
3. Find out why your build is not cacheable - Most builds that "cannot use caching" are non-deterministic: timestamps embedded in artifacts, absolute paths baked into outputs, code generation that emits a different ordering each run, or steps that read files they never declared. Build the same commit twice and diff the outputs. The diff is the list of things to fix.
4. Make CI check out and build the way developers do - If CI uses different flags, a different toolchain version or a different working directory layout, its outputs will never be interchangeable with a developer's, which forecloses sharing before you have started. Converge the configuration now, while there is no cache to invalidate.
5. Pick one module and prove the model - Rather than a build system migration, take the single most expensive compilation unit and make it cacheable end to end. A measured result on one module is what funds the rest; a plan for the whole repository is what gets deferred.
6. Write down the cost you are currently paying - Multiply the from-scratch build time by the number of builds per week, and do it again at the build volume you expect once agents are in normal use. This number is the argument for the next level, and it is much larger than most teams assume.

Concluding that the build is fast because it is fast locally. A warm local cache flatters the build and hides the cost everyone else is paying. The honest measurement is a clean checkout on a machine that has never built this project, which is what CI does on every single run.

Adding cache steps to the pipeline instead of fixing the build. Hand-written save and restore steps in a workflow file, with keys invented by hand, are a caching layer built by people with no visibility into the dependency graph. They produce stale artifacts and bugs that appear only in CI. Caching correctness belongs to the build tool, which knows what each step actually consumed.

Buying capacity to hide redundant computation. More and bigger runners reduce wall-clock time without reducing work, so the cost curve keeps rising with build volume and the underlying problem is never diagnosed. This is a particularly easy mistake to make when agents arrive, because the load increase looks like a capacity problem.

Assuming the build is not cacheable without checking. "Our build is too dynamic to cache" is almost always a statement about undeclared inputs and non-deterministic outputs rather than about the domain. Both are fixable, and the fix pays off again at every later level of build maturity.

Caching artifacts that should never be reused. Test reports, coverage output, logs and timestamped bundles are run-specific. Sweeping them into whatever gets preserved between runs produces results that describe a previous run, which is worse than no caching because it is wrong rather than merely slow.