Incremental builds: only changed fragments

Incremental builds are a build strategy where only the files, modules, or packages that have changed since the last build are recompiled, and only the tests that cover changed code are re-executed. Rather than rebuilding everything from scratch on every commit - the default behavior of most CI pipelines - incremental builds compute the minimal set of work required to validate a change.

The mechanism varies by build system. In Gradle, incremental compilation tracks which Java source files have changed and recompiles only those files and their transitive dependents. In TypeScript, tsc --incremental maintains a tsconfig.tsbuildinfo file that tracks which source files changed and recompiles accordingly. In Bazel, incremental builds are a core property: every build action declares its inputs, and Bazel only re-executes actions whose inputs have changed. In monorepo setups using tools like Turborepo (JavaScript), Nx (JavaScript/TypeScript), or Pants (Python/Java), the build graph is explicit and only the affected packages are built and tested.

For AI agents generating many small, focused commits, incremental builds are disproportionately valuable. An agent fixing a bug in a single module typically changes 2-3 files in a codebase of thousands. Without incremental builds, CI rebuilds and retests the entire codebase for that 3-file change. With incremental builds, CI rebuilds only the changed module and the modules that depend on it. On a large monorepo, this can reduce CI time from 8 minutes to 45 seconds for a typical agent-generated change. The impact compounds: agents that generate 20 small changes per hour get a 10x CI speedup compared to agents working against full-rebuild CI.

At L3 the unit of incrementality is increasingly the worktree rather than the branch. Meta's Muse Code, released 2026-08-05, spawns its subagents in isolated git worktrees, and a per-worktree pipeline is the natural counterpart: each concurrent agent gets its own checkout, its own build cache namespace and its own CI run, so parallel agents do not invalidate each other's incremental state.

The other thing that changed in August is that agent CI stopped being an internal system. At Black Hat USA on 2026-08-05, Novee Security showed a single public GitHub issue reaching CI secrets in three vendors' own repositories: Gemini CLI via CVE-2026-12537 (CVSS v4 10.0, OS command injection through a crafted .gemini/.env executing before sandbox init), Claude Code via CVE-2026-54316 chained with a command-validator quote-stripping bug, and Codex with no CVE but a fix that split workflow jobs and documented instruction files as an untrusted input surface. A Snowflake .NET connector issue disclosed 2026-08-17 leaked a Jira token the same way, by interpolating github.event.issue.title into a run: block. Fast incremental CI that hands an attacker your tokens is not an improvement.

The relationship between incremental builds and test impact analysis is complementary. Incremental builds reduce the compilation and build time by only recompiling changed code. Test impact analysis reduces the test execution time by only running tests affected by changed code. Mature CI pipelines at L3-L4 implement both: incremental compilation followed by targeted test selection.

• CI time scales with change size, not codebase size - a 3-file change in a 500-module monorepo should take seconds to validate, not minutes; incremental builds make CI time proportional to the change, not the total codebase
• Agents benefit disproportionately - agents generate many small, focused changes; incremental builds turn each of those small changes into a fast CI run, dramatically increasing the iteration rate
• Enables monorepo CI to stay fast as codebases grow - full-rebuild CI degrades as codebases grow; incremental build CI stays fast because it only rebuilds the changed sub-graph
• Reduces compute cost proportionally - fewer compilation and test execution actions mean lower CI compute costs; for teams with many agent-generated commits per day, the cost savings are substantial
• Makes the "change small things often" pattern sustainable - incremental builds reward the small, focused commit style that AI agents naturally produce, reinforcing the pattern rather than punishing it
• Per-worktree pipelines keep parallel agents from thrashing the cache - concurrent agents sharing one checkout invalidate each other's incremental state on every write; a worktree per agent keeps each cache namespace stable
• Agent CI is internet-facing whether you treat it that way or not - three vendors shipped fixes in August after a single public issue reached CI secrets, so the same pipeline that makes agents fast is also the one holding your tokens

1. Identify your build system's incremental build support - Gradle (Java/Kotlin): enable --build-cache and org.gradle.incremental=true. TypeScript: use tsc --incremental or ts-node --transpile-only for faster type checking. Rust: Cargo is incremental by default when CARGO_INCREMENTAL=1. Go: the build cache is incremental by default. JavaScript monorepos: consider Turborepo or Nx for package-level incremental builds.
2. Preserve the incremental build cache across CI runs - Incremental build state is stored on disk (.gradle/, .tsbuildinfo, target/ directory for Rust). Configure your CI cache to persist these directories between runs, keyed on the relevant source files. Without cache persistence, incremental builds restart from scratch every CI run and provide no benefit.
3. For monorepos: implement affected package detection - Turborepo's --filter=[HEAD^1], Nx's affected:test, and Pants' :: with change detection all determine which packages are affected by a commit and run builds and tests only for those packages. Setting up these tools is a one-time investment that immediately benefits all subsequent CI runs.
4. Validate incremental build correctness - Incremental builds can have correctness bugs: a change that should trigger a downstream rebuild doesn't. Run the full build weekly (or on merge to main) to catch any incremental build correctness issues. Compare the incremental build results to the full build results and investigate any discrepancies.
5. Configure your CI system to pass build state between jobs - If your CI pipeline splits into multiple jobs, each job needs access to the incremental build state. Pass build outputs between jobs using CI artifacts or a shared build cache. Don't split the build across jobs in a way that forces each job to recompile independently.
6. Give each concurrent agent its own worktree and cache namespace - git worktree add per agent session, with the build cache keyed by worktree, so ten parallel agents do not serialise on one checkout or repeatedly invalidate one shared incremental state.
7. Grep every workflow for ${{ github.event.* }} inside a run: block - this is the one-line audit that would have caught the Snowflake connector leak. Then split agent passes into separate jobs with separate checkouts, scope tokens per job rather than per workflow, and treat instruction files (AGENTS.md, .gemini/, .claude/) as attacker-controlled input.
8. Measure the before-and-after impact on CI time distribution - Track p50 and p95 CI time before and after enabling incremental builds. The p50 should drop significantly (typical changes benefit most). The p95 may not drop as much (changes to widely-depended-on modules still trigger large sub-graph builds). The distribution tells you where the remaining optimization opportunity is.

Incremental build false negatives. Not all build systems correctly detect all changes that should trigger a rebuild. If your build system uses file modification timestamps rather than content hashes to detect changes, fast file copies or git operations that don't change content can confuse it. Use content-hash based change detection (Bazel does this by default; Gradle supports it with --no-rebuild and correct task input declaration).

Cache invalidation bugs masking incremental build bugs. If your CI cache is misconfigured and always misses, incremental builds appear to work correctly because they always run from scratch - but you're not getting the speedup. Check whether your CI runs are actually restoring incremental build state by adding a log step that reports the size of the cache restore.

Applying incremental builds without correct dependency declarations. Build system tools infer dependencies between targets from explicit declarations (Bazel BUILD files, Turborepo turbo.json, Nx project.json). If a target has an implicit dependency that isn't declared, incremental builds won't rebuild it when its upstream dependency changes, producing incorrect results. Complete and accurate dependency declarations are required for correct incremental builds.

Over-relying on incremental builds and neglecting full builds. Incremental builds skip work by design. Over time, accumulated incremental build state can become stale or incorrect. Teams that never run full builds lose the safety net of a clean, from-scratch validation. Run full builds on merge to main and nightly on the main branch to catch anything that slips through incremental validation.

Treating the agent CI job as an internal system. The pipeline that runs an agent over a pull request usually has the widest token scope in the repository and reads text that anyone on the internet can write. The August disclosures all followed the same shape: untrusted text from an issue or an instruction file reaching a shell. Split the agent pass into its own job with its own checkout and its own minimally-scoped token, and never interpolate event data directly into a run: block.

Implementing incremental builds in one language without others. In polyglot codebases, implementing Gradle incremental builds for Java while leaving Python builds as full rebuilds creates a two-speed CI. The total CI time is dominated by the slowest full-rebuild component. Address all languages in the build or the improvement is partial.