Optimizing Your CI and WebPack Builds

Recently, I was running into issues where the builds for a Node project were becoming painfully slow. Local development builds would take well over a minute or more. The CI builds on Jenkins would take at least eight and a half minutes, but would usually take longer. It was neccessary to find ways to optimize the scripts that triggered the builds and the webpack configuration.

NPM

On Jenkins, the npm install process ended up being half the issue. Now, when a build is triggered on a new server, or with a fresh clone of the repository, its expected that its going to take some time to download and install all your dependencies. However, in this case the script triggering the build was blowing away the installed packages on every build with rm -rf node_modules/.

Avoid A Full Install For Every CI Build

We elected to get rid of this in favor of running npm prune, which will remove any extraneous packages from your node_modules directory. A subsequent call of npm install will then only install new dependencies. Speaking of pruning, you should also check the package dependencies listed in your package.json file. Weed out any packages that are no longer in use. There's no sense in allowing them to lie around wasting space or taking up time and bandwidth when you run a fresh install.

Webpack

Moving on to the webpack configuration, the project I'm referencing here was sufficiently complex enough to contain roughly 14 entry points. The time spent just on the UglifyJS optimization alone was over 60 seconds.

Faced with a similar situation, you might follow a simple configuration example and end up with something that looks similar to the example below. It will work; it might even be fast at first. However, as the project grows, it will begin to slow you down.

Make use of the CommonsChunkPlugin

When a project has numerous entry points, it's quite likely that modules and dependencies are reused throughout the code. That's where the CommonsChunkPlugin is going to be invaluable. With the default configuration, the plugin will automatically package any dependencies that are shared amongst all of your bundles.

With a small modification, it's also possible to package common modules as well. By specifying the minimun number of chunks through minChunks, say a minimum of 3 for example, any module that is required by 3 or more of your bundles will be included into common.js.

This can save a significant portion of time, especially if you're optmizing your bundles with UglifyJS. By reducing the size of the entry point's bundled file, the amount of code that has to be optimized is also reduced. Furthermore, the common code is optimized just once, instead of multiple times (once for every bundle it was previously included with).

Build a Dynamically Linked Library (DLL)

If you have code that rarely changes, third party vendor code for example, and the CommonsChunkPlugin doesn't sufficiently speed things up, you can split the work for this out into a separate build. You build the DLL once, and then develop your app without having to worry about spending time rebuilding all of the static code in the process.

To accomplish this, you'll need to need to create another webpack config file to build the DLL, which will utilize the DLLPlugin to build the bundle.

Profile Your Builds

If you're still running into issues, webpack provides a very useful profiling tool through their analyze web app. To generate the necessary data file use the --json flag and provide a filename to pipe the data into:

$ webpack --profile --json > stats.json

The tool will show warnings and errors for diagnosing your build issues as well as provide suggestions and hints on how to optimize your code. Another neat feature is that it generates a force-directed graph of all your modules and breaks down the stats for all the code in the build.