Build tools

Turn source code into runnable artifacts, rebuilding only what changed since last time.

Live modeldata or requestresponsebusy / idle instance

What you’re watchingEach build step is looked up by a hash of its inputs: source files, flags and tool versions. Most steps hit the shared cache and come back at once; only steps whose inputs changed go on to run on a worker.

How it works

A build tool reads build files that say what to produce (a binary, a JAR, a container image, a JavaScript bundle), what each output is made from, and the command that makes it. From those it builds a dependency graph and runs the steps in order, side by side where they don’t depend on each other. The skill is in skipping work. Make, the original, rebuilds a target only if one of the files it depends on is newer. Newer tools such as Gradle, Bazel, Nx and Turborepo fingerprint each step’s inputs (source files, flags, tool versions) and reuse the earlier output when the fingerprint matches, from a local cache or a remote cache the whole team and CI share. That only works if a step’s output depends on nothing but its declared inputs, which is what a hermetic build promises: the same source and configuration always give the same output, whatever happens to be installed on the machine. Bazel, Buck2 and others can go further and send steps to a pool of remote workers through the Remote Execution API. Some languages ship a build tool with the compiler (Cargo for Rust, go build for Go), and in JavaScript the bundlers (webpack, esbuild, Vite) play the role, following imports from entry points to build one or more bundles. A build tool is not a CI service: CodeBuild, Cloud Build, Azure Pipelines or GitHub Actions decide when and where a build runs, and the build tool they call decides what gets built.

Think of it as

A kitchen that keeps prepped ingredients labelled with the recipe and date. When an order comes in, the cook only chops what isn’t already in the fridge, and a sister kitchen can borrow the prep too.

Reach for it when

Any project bigger than one file: the graph and incremental rebuilds save time on every change
A monorepo where CI should rebuild and retest only what a change affects (Bazel, Buck2, Nx, Turborepo, Gradle)
Builds slow enough that a shared remote cache, or remote execution, pays for its servers

Think twice when

Adopting Bazel or Buck2 for a small project: they ask you to declare every input and toolchain, which is real work
Trusting cached outputs from non-hermetic steps, such as ones that read the clock or download “latest”; they make stale or wrong builds

A real example

A monorepo holds 40 TypeScript packages, and a full CI build takes 25 minutes. With Turborepo and a remote cache, a pull request that changes one package rebuilds and retests that package and the three that depend on it; the other 36 restore their outputs from the cache, because their input hashes match a build that already ran. CI takes four minutes, and a developer pulling the branch gets the same cache hits locally.

What each provider calls it

AWS: AWS CodeBuild (a managed service that runs your build tool, not a build tool itself)
Google Cloud: Cloud Build (runs build steps in containers)
Azure: Azure Pipelines (builds, tests and deploys; runs your build tool)
Open source & others: Make, Maven, Gradle, Bazel, Buck2, Pants, Nx, Turborepo, Cargo, `go build`; bundlers webpack, esbuild, Vite; remote cache and execution servers Buildbarn, Buildfarm, bazel-remote

Families of build tools#

Family	Examples	Decides what to rebuild by	Sharing work
Classic	Make	File timestamps: a target older than a prerequisite is rebuilt	None built in
Task-based (JVM)	Ant, Maven, Gradle	Gradle compares each task’s declared inputs and outputs and marks it `UP-TO-DATE`	Gradle’s build cache, local or remote over HTTP
Artifact-based, hermetic	Bazel, Buck2, Pants	Hashes of each action’s declared inputs, command and environment	Remote cache and remote execution through the Remote Execution API
JavaScript monorepo task runners	Nx, Turborepo	A fingerprint of each task’s inputs; runs only affected projects	Remote cache shared by CI and developers
Built into the language	Cargo, `go build`	The toolchain tracks changed packages; Go keeps a build cache	Local by default
Bundlers	webpack, esbuild, Vite	A module graph followed from entry points	Built for JavaScript and CSS, not a general build system

Bazel’s documentation calls Ant, Maven and Gradle task-based: you describe tasks that run commands. Bazel’s build files instead declare the artifacts to produce and their dependencies, and the tool decides how to run them.

In code · text

# Makefile: target: what it depends on, then the command (indented with a tab)
app: main.o db.o
	cc -o app main.o db.o

main.o: main.c db.h
	cc -c main.c

db.o: db.c db.h
	cc -c db.c

# $ make          compiles main.o and db.o, then links app
# $ touch db.c
# $ make          recompiles only db.o, then relinks app

Often confused with

CI/CD pipelines Containers Version control Caching

Where it came from1976–2023

In the mid-1970s at Bell Labs, Steve Johnson stormed into Stuart Feldman’s office after losing a morning to debugging a program that was actually correct: the bug had been fixed, but the file hadn’t been recompiled. Feldman wrote Make that weekend; the base version in its change log is dated April 24, 1976. A Makefile records which files depend on which and how to rebuild each, and Make walks that dependency graph, rebuilding only targets older than what they depend on. Feldman’s paper describing it appeared in 1979.

Java brought new tools. Ant began as the tool that built the Tomcat server and became its own Apache project in 2000. Maven grew out of the Jakarta Turbine project, where each subproject had its own slightly different Ant file and JARs were checked into CVS; it offered one standard way to build and a clear definition of what a project consists of. Maven 2.0 came out in 2005. Gradle reached 1.0 in June 2012; it skips tasks whose inputs and outputs haven’t changed, and can share outputs through a build cache.

Very large codebases pushed further. Google, which had built its software with large generated Makefiles that grew slow and unreliable, wrote Blaze, a build tool over its monorepo whose promise was “{correct, fast}, choose two”. It open-sourced Blaze as Bazel on March 24, 2015. Facebook built Buck, and in April 2023 Meta open-sourced Buck2, a from-scratch rewrite in Rust that it said built twice as fast. Bazel’s Remote Execution API is now spoken by many clients and servers, and language toolchains took the lesson too: Go 1.10 added a build cache in 2018.

Why it’s built this way

A declared dependency graph: Programmers forgot which files depended on which, and recompiling everything to be safe was wasteful. Writing the dependencies down once lets the tool work out the minimum to rebuild every time.
Hashes of inputs instead of timestamps: A timestamp only says a file was touched, and it means nothing on another machine. Hashing the inputs of each step makes a result safe to reuse from someone else’s build, which is what makes a shared cache possible.
Hermetic builds: If a step can read whatever happens to be installed, the same source can build differently on two machines and a cached result can’t be trusted. Treating compilers and libraries as declared, versioned inputs makes outputs reproducible.
Declare artifacts, not tasks: When build files describe what to produce instead of which commands to run, the tool is free to parallelize, cache and distribute the work, at the cost of declaring everything up front.

Timeline

1976
Base version of Stuart Feldman’s Make, April 24, at Bell Labs
2000
Ant leaves the Tomcat code base to become Apache Ant
2005
Maven 2.0 is released (October)
2012
Gradle 1.0 (June 12)
2015
Google open-sources Blaze as Bazel (March 24)
2018
Go 1.10 adds a build cache to go build
2023
Meta open-sources Buck2, written in Rust (April 6)

Sources19

Improve this page on GitHub