Parallel

A Build System for Complex Projects: Part 5

By Gigi Sayfan, November 12, 2009

Testing and extending the ibs build system

Debugging Build Problems

Sometimes builds fail. There are many possible reasons. With ibs, there could be problems during the generation process or during the build itself. If the problem happens during the build system generation, then you can just run the build_system_generator.py script in the debugger and put a breakpoint in the problematic area.

One problem that happens a lot with other build systems is missing or misnamed files. This happens if a file is moved, renamed or just deleted but the corresponding build file is not updated. With ibs, it can't happen because the build files are generated automatically based on the existing files. But, a source file might reference a missing file. This will be discovered during the build itself.

Another common problem is link failure. That happens if an executable or dynamic library depend on a static library, which is not linked into it. There are two reasons for link failures:

The dependency is not specified in the build file

The static library failed to build

Failure #1 can't happen with ibs because it detects all dependencies automatically and adds them to the build files. Failure #2 is easy to detect because the static library will fail to build before the executable or dynamic library fail to link.

A more difficult failure that can't be detected automatically is dependency on dynamic library. You must come up with some system to track and manage dynamic library dependencies. You will usually get a clear error message that says that such and such dynamic library can't be loaded.

If you integrated testing into your build system then the most common failures will be test failures, which you just need to fix.

There could be other failures if your build system is even more sophisticated and perform other tasks like compiling documentation, packages your system for deployment, uploads to a staging area or a web server.

The key is always to prevent as much as you can and make sure that failures are easy to detect with good diagnostic messages that contain all the information needed to correct the problem.

Developing a Custom Build System

ibs, the build system I described in this series, focused on building C/C++ source files in a cross-platform way or rather generating build files. A serious industrial-strength build system does much more. If you want to develop your own build system, you need to consider these aspects. The absolute minimum must include checking out the sources from source control, building all the software artifacts on all platforms, running a test suite on all platforms, and reporting the results.

The automated test suite is a critical piece for a professional software organization and it get as fancy as you want with complete test environment that simulate your deployment environment, automated GUI tests and complete builds and tests of your source releases (that's right -- building the source is part of the test)

Then there is packaging. There are many ways today to distribute software. You may develop a web application, a native client, a smart phone app or a plugin to some other application like Firefox, Eclipse, or Visual Studio. Probably, you end up with multiple artifacts that need to be packaged and deployed. Your build system should take care of this aspect too.

Automatically generated documentation is also in the realm of the build system. In general, almost any repetitive task can be automated with some imagination.

It is important to start small and grow the build system incrementally. If you try to nail everything down before you let the developers make the firs checkin you won't get very far.

The best guideline is to address pain points as they show up. If your developers keep having problems with third-party dependencies then figure out a way to verify it before checkin. But if you release software every two years, there is probably no need for the automatically creating an installer for fancy GUI client.

Conclusion

In this five-part article series, I delved into the sometimes mysterious world of building software. I described the issues involved in building cross-platform C/C++ code and presented a unique build system called ibs (Invisible Build System) that addresses many of the issues. I explored the design and implementation in great detail. I even tried to be funny by showcasing the build system through the most bloated "Hello World" application I could conceive. I hope you liked this series and that some of you will find it useful and may even try to create your own build system. It's a lot of fun.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Video

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!