Speed up Visual Studio Builds

Recently I got involved in a big project where we had a single solution with approximately 100 projects.

Why 100 Projects in a Solution?

The reason for a 100 projects solution is that like in many modular systems these days, we have the following three tiers:

A few core / common projects every project will use.

A large amount of modules, independent of each other. This tier directly depends on tier 1.

A few end-projects which load the different modules. This tier indirectly depends on tier 2.

So, yes, we could create several solutions with each tier compiling only when needed and using DLL reference instead of project references, but the amount of changes in all the tiers was still large enough and I’ve already seen this kind of build process fail miserably. So this was no go.

Build time took 15 minutes for the whole solution. Since we enforced a gated check-in policy in the company, this was really a pain point for the developers.

After setting this up, I got an approximate build time of 10.2 minutes. Not bad for a few minutes of work! Also, got the following beautiful image out of my CPUs, where you can really see them at work:

Step 2: Beware of Copy Local = True

In our 100-projects solution, lots of projects reference each other, obviously. In addition to these references, we also reference several 3rd party components, practically from each module.

All the above caused that whenever we would compile the solution, over4.5 GB of files were written. The majority (95%) of the writes were DLLs which were copied to the output folder of each project.

To check out how many writes are done in your compilation, check out this post.

Anyway, 4.5 GB takes a long time to write, even on an SSD drive.

So the next step was to eliminate those writes. To do this, we changed almost all of the “Copy local” settings in all the referenced DLLs from the default True to False. This will prevent the referenced DLLs to be copied to the output folder of each project.

Note: Some file names were blacked to protect the client’s properties.

In addition, we also changed the output folder of all the projects to a single folder, so all generated DLLs are copied to the one and only place where we actually run them. Doing so dropped the writes while compiling to under 200 MB, a huge time saver. Specifically, build time dropped to 7.5 minutes!

Step 3: Use RAM Disk

A RAM disk is a logical disk which resides entirely on the RAM. It is extremely fast (faster than any SSD), but it is erased on every power-down, so only use it for temporary files.

Of course, you should have enough RAM to spare for this disk (the memory is pre-allocated for the disk use only), but on an 8GB PC, it’s usually not an issue.

There are several programs you can use to set up a RAM disk. I used DataRam RamDisk which supplies a free version with the ability to create a RAM disk up to 4 GB (1 or 2 GB should be sufficient for any build).

Configuration of the RAM disk is very easy:

After you download, install and format your new RAM disk, you can move your single output folder to it. If you want to keep your output folder in the same build drive, you can simply create a symbolic link between the current output folder and a folder on the RAM disk. This way makes the using of the RAM disk optional, only for users with sufficient memory.

To create a symbolic link between your build folder and your new RAM disk folder, use the following line:

mklink /D C:\Dev\MyCurrnetBuildFolder\Source\bin R:\bin

where R: is your RAM disk folder.

Note: You should change the path according to your build folder and your RAM drive settings.

Result: Adding the RAM disk reduced compilation time to less than 5 minutes! This is a 66% reduction of the original time!

Summary

In this post, we’ve seen how you can decrease build time. Of course, I can’t make any guarantees. Every project has its own characteristics and problems, but the steps provided can probably reduce the build time if you fit the profile of standard line-of-business applications.

An additional measure would be not loading (and not compiling) projects which are currently not needed. In a 100-projects solution developers are working most of the time on only a few projects, so instead of compiling 100 projects before each test run, just compile the few currently involved.

The free Visual Studio extension Solution Filters provides load filters for selective loading of project lists. After an initial compilation of all projects (which is required in order to fulfill all dependencies) only the projects which are really required for the current task can be loaded. Load filters can be defined for each aspect of the solution, combined with each other at load time and shared in teams.

Our solution has 60 projects. We have a few tiers, including 3 WCF services, a web site for hosting the XAPs, and a Silverlight Prism app with several modules.

I've been experimenting with ways to increase the performance of the build for everyone, as there are about 20 of us working on the project, and everyone's complaining over the build time.

I've followed most of the advice in this article, but I'm a little confused with a good strategy for where to output to, as well as which assemblies to stop copying locally. I've run through each project in the server tier so far, and my strategy was that if this project was not a "hosting" project (the wcf projects, the web project, a test project, etc) then it didn't need any assemblies copied local. Most things seem to work, I have one wcf project that complains over not loading an assembly, i'm assuming I just got too agressive.

As for the output folder, though, what are we trying to achieve? It seems that wcf services still want everything in the \bin folder of the project, which is on the c: drive. Do I set all projects that run in the same tier to the same output folder?

Well, the point in removing the local copies was to save time in compiling, since these DLLs are not needed in most output folders of the modules.However, these DLLs are required in the host application folder, so you can either leave the local copies of this project, which I think is what you have done.Or, simply change the output folder of the modules DLLs to the host build folder.

EDIT: I was having trouble getting the "My vote of 5" message to appear, which is what the "Testing something" above is about. I have submitted a bug report regarding the issue. By the way, I wasn't just testing... nice tips.