Related Articles

IncrediBuild - Fast C/C++ Compilation

IncrediBuild, integrated with Microsoft Visual C++, silently distributes compilation tasks across workstations in the network, cutting build times down by 90% and more.

Editorial Note

This article is in the Product Showcase section for our sponsors at CodeProject. These reviews are intended to provide you with information on products and services that we consider useful and of value to developers.

This is a continuation in our series of showcase reviews for our sponsors at CodeProject. These reviews are intended to provide you with information on products and services that we consider useful and of value to developers. They also allow you to discuss a product and interact with the manufacturer or provider on a one-to-one basis.

IncrediBuild Cuts C/C++ Build Time Down by 90%

How many times have you found yourself blankly staring at your screen monitor, waiting for MSVC to compile your latest code modifications? Are you familiar with that small pause of hesitation before making changes in a header file that you know will cause a near full rebuild of the project?

Slow compilation time has always been one of the C/C++ programming language's weakest spots. Whether it's a few hours wait for a full product build, or 10 minutes spent several times a day waiting for an incremental build to finish on a developer workstation - the effect is frustration, loss of productivity and wasted time.

Surprisingly, this has almost become a fact of life for most developers. Although the introduction of precompiled headers along with the improvement in P.C. hardware have improved the situation to some extent (thankfully, 5-10 hour builds are not the common case anymore), a medium to large scale software project typically wastes hundreds of precious developer hours yearly on compilation alone.

IncrediBuild, by Xoreax Software, introduces a unique and effective approach to this problem. Integrated with Microsoft Visual C++, it silently distributes compilation tasks across workstations in the network, cutting build times down by 90% and more.

One of the views in IncrediBuild's "Build Monitor" visually displays the distributed compilation progress. Each file is colored according to its compiler message severity.

Distributed Compilation

IncrediBuild is based on a peer-to-peer model. An "Agent" component is installed on different machines in the network, while a central component called the "Coordinator" is responsible for managing the compilation workload. When a build is initiated by an IncrediBuild Agent user, Agents in remote machines are assigned as compilation resources for that build. Source files are then compiled by those Agents, and the build output is returned to the building machine. Linking is finally performed on the machine that initiated the build.

The task of assigning different Agents to currently running builds is performed by the Coordinator. Based on knowledge of each Agent computer's hardware specifications, current CPU availability and other parameters, the Coordinator sees that at each point in time, a build will utilize the strongest, most available Agents. Moreover, when several builds are running at one time, the Coordinator's job is to distribute the available computing power evenly among the building Agents.

When functioning as compilation resources for remote builds, Agents perform compilation tasks in a low priority process, in order to maintain the computer's level of responsiveness. This is made even more effective by the Coordinator's ability to take into account each machine's availability, effectively assigning busier machines last (in fact, the second a machine reaches a certain minimal level of activity, it will immediately cease any remote compilation activity). Finally, at any given time, an IncrediBuild Agent user can temporarily disable the Agent on his/her machine, keeping the machine's resources out of other Agents' reach.

The IncrediBuild Coordinator application acts as a "control center" for the IncrediBuild environment. Any build activity taking place is displayed, along with other useful data.

Using IncrediBuild

IncrediBuild builds can be run from 3 distinct interfaces:

The Microsoft Visual C++ IDE - IncrediBuild adds an additional menu and toolbar to the IDE, allowing builds to be run directly from the development environment. Standard MSVC builds can still be run at any time, and IncrediBuild does not override any MSVC functionality.

The IncrediBuild command line interface - IncrediBuild features a strong command line interface that allows builds to be run from scripts, third party tools and console windows.

The IncrediBuild tray-icon menu - IncrediBuild's tray-icon menu allows for builds to be run without requiring the MS Visual C++ IDE to be open. Simply browse for a DSP/DSW file (or select one from the recent builds list) and start a build.

When running IncrediBuild builds, MSVC's standard output display is replaced by the IncrediBuild "Build Monitor". This unique display contains everything you need to monitor your build's progress and easily locate warning/error messages. A visual display of the distributed compilation process shows each compiled source file as a progress bar, colored according to the severity of compiler messages generated for that file. The build output can be displayed for the entire build as well as separately for each configuration (see below). To top it all off, extra features like a build progress bar and easy navigation to next/previous error message are there to make life even easier.

When used from within the MSVC IDE, a docked Build Monitor window displays the build results. Shown here is the Build Monitor's per-project configuration output view, along with build progress bars for each configuration.

A Word on Reliability

One of the most common concerns new IncrediBuild users have is: "How can I be sure my builds will come out right?". This is particularly an issue when distributing compilation over machines with different environments, operating systems, DLLs and COM objects installed.

IncrediBuild tackles this issue with the concept of a "Virtual Environment". When compiling for a remote Agent, IncrediBuild runs the compilation within a protected mirror image of the building machine's environment. This has several pluses. For a start, whenever the compiler references a source file, type library, registry entry, etc. - it is guaranteed to operate identically to a similar operation on the building machine. This also means that IncrediBuild requires no manual configuration whatsoever to work, since it will always use the building machine's environment. Another benefit is that IncrediBuild does not use the machine's file system to store remote source files, but rather uses its own source file caching mechanism, saving the trouble of worrying about different source file versions appearing in your folders.

Just to give an example that stresses how far the "Virtual environment" concept has been taken: Even the compiler version used for remote compilations is mirrored from the building machine. In fact, in order to use an IncrediBuild Agent on a machine used solely as a compilation resource (and not to initiate builds), there's no need for MS Visual C++ to be installed, since IncrediBuild will always disregard the locally installed version and use the mirrored version.

Closing Words

IncrediBuild offers the chance to instantly improve development productivity and the quality of code. Depending on the length of your project builds, the effect can be anything from a substantial improvement to a radical boost of a development team's effectiveness. The fully functional 30-day trial version available for download on the Xoreax Software web site offers you the chance to try it out for yourself (setup is a matter of minutes).

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

About the Author

Comments and Discussions

It would probably be worth while to port distcc to Windows/VC++. It is incredibly stable, and seems to be the inspiration for this tool. Plus its sister project ccache speeds things up even more. Check out Sourceforge.

The linking would be much improved by just doing it locally. No loss there, just no gain - but big deal.

That said, I find it to be a solid beta-quality project (regardless of its 2.x versioning.) Just too many problems... who wants to DEPEND on a product with a website's most prominant feature is "Known Issues"?

We are just in the process of trying out Incredibuild at our site, and after a rather painful week of dependency analysis it does now work. I really want to compare it against JetBuild (www.jetfrog.com) but no-one at the company seems to reply to email or answer the phones. Anyone out there have any experience of comparing both tools?

There is another tool like incredibuild, but it faster, cheaper and support new “features” like C# and VB. Take a look on www.jetfrog.com. Also it compiles C# projects under VS 2003 which was most valuable for me (just migrated ).

"7 You shall have no other gods before me. 8 You shall not make an images in order to bow down to them or serve them. 11 You shall not take the name of the LORD your God in vain. 12 Observe the sabbath day 16 Honor your father and your mother, that your days may be prolonged. 17 You shall not kill. 18 Neither shall you commit adultery. 19 Neither shall you steal. 20 Neither shall you bear false witness against your neighbor. 21 Neither shall you covet anything that's your neighbor's." Your God

Distributed build
Anyone could guess that bringing multiple processors to bear on a build would make it go much faster, but Xcode lets you act on the obvious solution. With the Rendezvous-enabled distributed build feature it’s easy to simply farm out your build by distributing compile workload across idle desktop machines or, better, deploy a dedicated Xserve build farm to do in minutes what would take hours on any single machine.

Predictive compile
Compiling is a sink hole for code turnaround. Xcode steers you around the problem by staying one step ahead of you and the compiler, advance-compiling changes while you are still making them in source code. By the time you’ve saved your edits and started to recompile your app, the fix is in and the compiling is mostly complete.

Zero links, no waiting While designing Xcode, Apple discovered the missing link for faster debugging — lose the links, or most of them. By linking only the object code needed to launch the application, instead of all the code, as in other development tools, Xcode dramatically reduces the debugging cycle. You spend less time waiting for builds and more time writing code.

How about IncrediRipoff? This is way too expensive for what it does. You can do the same with a few intelligently crafted batch files. Beside, where I work, we do nightly builds, who cares if it takes 5 minutes or 50 minutes.

Actually it's an amazing tool, and because it lets you utilize all of your machines efficiently, and the virtualization means that there's no effort involved (mapping paths, installing VC2005 on every machine .... etc) you can install the slaves anywhere. We get 5 - 10x performance improvements from it for a few hundred bucks per seat - tell me *any other way* I could spend that kind of money on my PC to make it 5x faster? And the users running it don't even notice that it's there ... it caches files to disk to avoid overloading the network, and you can even distribute other tasks (not just VC++ builds) with the XGE plugin. I reckon it's great value

It is very good that this small program can increase up to 90% compilation time. But there is a small problem. I'm new to IncrediBuild (just using trial version for a short time) and I can't run up its. It always stops during compiling StdAfx.cpp with following error:

error C2857: '#include' statement specified with the /Ycstdafx.h command-line option was not found in the source file

The project setting contains options for generating browse information and using precompiled header through .

IncrediBuild v1.2, Released April 15th, 2003, now supports distributed C/C++ builds in the MSVC.NET environment.

Beside support for MSVC 6.0 & 7.0, the new version also features additional improvements in build speed, new UI features and a renewal of the 30-day trial period (for users that evaluated an older version in the past).

Reading all responces, I'm confused:
Are you all guys just building, and/or you are programming too???
I'm looking for a tool that could decrease my programming time, not build time -- Do you know of any???... hehehehe
And not really "programming", -- I'm planning to charge not less number of hours then I do currently!!!...

This tool maybe great: But really I don't see alot difference between spending 30min vs 5 min on full rebuild -- just spend that 25 min difference smoking and thinking what's next...

At my company, there are about 20 of us that use Incredibuild. We have an application that has many files in the main app and then about 20 dll apps. Altogether, on a 2.5 ghz machine running XP, it took about 25 minutes to build everything. With Incredibuild, it takes about 5 minutes.

There are some problems though. For instance, edit and continue does not work when using Incredibuild. Incredibuild also links all dlls even if they have just been rebuilt. And if you have a 1 or 2 line change and hit the normal build button, all the dlls have to be linked, even if incredibuild just did it for you.

I know I know, code warrior sucks. But its defacto for PS2 development. And right now it takes a good 10 minutes to build and I do that daily about 20 times. I love coffee breaks but its getting kinda crazy.

I've tried IncrediBuild some days ago. First it was realy
nice. The compile-time was about 45 minutes after 2 hours before.
But some days later I run into problems, because after IncrediBuild
my programs didn't work. (I think because of wrong linking.)
So I had to rebuild all without IncrediBuild. That means, one big
error in 3 days. And I have to test all programs (the workspace
contains 64 projects (dll's and exe's)). That's to mutch and I can't
use it.

By the way: It seems, that VisualStudio and IncrediBuild
have a different dependency tree. After changing a header than
VisualStudio builds other files than IncrediBuild.