SCons — build your software better

A few years ago I worked on a large (60+ man-year) software project, involving C/C++ and Java. The software stack originated from demonstrator software, and its make files had become unmanageable over time. Even worse, they were also unreliable. After evaluating a lot of options we opted for a radical approach: rewrite the entire build system from scratch, based on SCons.

A few weeks ago an ex-colleague contacted me with some questions how we approached things back then.
Today, while attending a children's birthday party a discussion on make related issues popped up (can you imagine?). All of this reminded me of how elegant the SCons way of doing things is, which is what I would like to share with you today.

Consider a program that can be built in debug and release mode. Additionally, its sources are neatly organized in subdirectories. This is not an uncommon use case, but anybody who has ever written make files for such a use case knows it requires quite some discipline to keep things tidy (recursive makes files, passing down options, organizing build settings in top-level make files, ...).

This is how it is done in SCons. First, let's take an example directory layout:

The top level "make" file is called SConstruct, which is intentionally different from lower level "make" files named SConscript. You can invoke SCons using scons -u from any directory, and SCons will automatically search directories upwards from your current path for the top level SConstruct file.

All sources are neatly organized below the src directory.

There is a program called program that depends on a library named toolkit.

Import('env')
# toolkit.h is located in this directory, add it to the include path
env.Append(CPPPATH=['.'])
# Let's declare a library named toolkit, using toolkit.c as its only source
env.Library('toolkit', ['toolkit.c'])

With SCons you do not need to write cleanup rules. SCons derives a dependency tree from all the declarations inside the SConstruct and SConscript, across all subdirectories in your project. Given that tree it is perfectly clear what files to throw away when cleaning.

But there is more. Ever attempted to parallelize a build? Sure, there is make -j N, but try combining that with a recursive make structure, and you will find yourself jumping through hoops to maintain a proper build order. Not so with SCons. Given a complete and correct dependency tree of your complete project across all subdirectories, parallellizing your build is really just a matter of invoking scons -j N.

Finally, SCons files are in fact Python files, allowing you to use proper software development techniques even for your build system. Compare this to hacking macro's, or even generating make files when using Make).