apmake: A Reliable Parallel Build Manager

Derrick Coetzee, Anand Bhaskar, George Necula

Abstract

Build systems such as make support incremental and parallel building, but these features are unreliable in the presence of incomplete dependency information. We describe a system that automatically augments a build system to provide parallel and incremental building while guaranteeing the same final output as a clean, serial build. Each build task is run inside a transaction that isolates its effects from concurrently running build tasks, and the results of build tasks are cached for later reuse. By dynamically monitoring all filesystem accesses, all file-based dependencies between build tasks can be reliably identified. In experiments on three small builds on a quad-core machine, an initial build using our system took between 54% to 219% as long as a clean, serial build, while an incremental build (with no files changed) using our system took between 22% to 71% as long as a clean, serial build..