The stuff we do, in more detail

Developers: frantic to free in 6 easy steps (part 3)

This is part of a series of posts chronicling my team’s adventure in making release day suck less.

Note: I would highly recommend giving each of these a shot in a way that fits your team. I would also recommend trying to implement each one at a time. If you spend time focused on each step, tweaking where there’s pain and adjusting to your team’s personality you’ll find everyone owning the end result. You’ll also show them you don’t know everything and need their help figuring some of these things out, which actually helps build a stronger team.

Here’s what we did, pretty much in the order in which we added them to our process:

Continuous integration

In my last post I mentioned that during our code review, the “build breaker” had to wear a distinct shirt. If you’re not familiar with continuous integration you’re probably not familiar with that term. In a nutshell, continuous integration is a process in which your new code is continuously being integrated into the rest of the code base. For our team, that means after every single check-in we have a server pull our code the source control, compile every last bit of it, and run our unit tests against it to be sure we didn’t introduce a change in behavior that will break something elsewhere in our system. Once you have a pretty decent suite of unit tests, you’ll begin to see the power of this concept.

To be fair, we actually have two levels of testing that occur in our system: unit tests and integration tests. Unit tests run after every check in and integration tests are only run once a night. Since integration tests test things like changes to database or web services our team has no control over, it verifies that those systems haven’t changed in any way that impacts our integration with them. They also tend to take a while to run since the code actually connects to and uses those systems. As a result, we run them once a day to be safe, but no more.

The great part about this step is that it takes almost no work to get running. Most source control servers have some sort of automation built in to enable teams to set this sort of thing up. In fact, most have an option to turn integration builds on or off and it will do the rest for you. We happen to use TFS, which falls into that category. Due to our branching structure, the build definitions need some babysitting after we branch for user testing and then when the code goes to prod, but once everything is set up, that’s all it take in terms in maintenance.

Over the past year or so we have uncovered several bugs in vendor code with which we have integrated. Their change was deemed to be “no impact” so no communication was sent, but it changed some return values and therefore broke several of our tests–which means it broke the way our code relied on theirs. We were able to compensate for that and deploy a fix prior to the bug being found in production simply because our integration tests were running.

The primary benefit of continuous integration in my mind is that you catch defects early. The earlier you find them, the easier they are to find and resolve. As you find them you write a unit test to expose the flaw and then fix it. The alternative is allowing your QA testers (or in our case, end users) to discover them, creating a lot of re-work and overhead in the QA process.

Since it costs almost nothing once you have committed to unit testing, not giving it a shot is just belligerent and proud. Come on man, be cool.