Monthly Archives: October 2013

I have been voraciously devouring every bit of information about the failures of healthcare.gov that I can find, because all indications are that the companies involved did not follow anything approaching industry standard methodologies. And then I stumbled upon this, from Slate:

As late as Sept. 26, there had been no tests to determine whether a consumer could complete the process from beginning to end: create an account, determine eligibility for federal subsidies and sign up for a health insurance plan, according to two sources familiar with the project.

It has been said by greater minds than me, but write tests into your code. In fact, write test code before you write product code. This serves many purposes. First, it helps you arrive at a solution by guiding you until your test passes. Second, it prevents regressions by alerting you when you later change code that causes a failure.

I manage the continuous integration process for dozens of applications, covering hundreds of thousands – if not millions – of lines of code. Each time a developer submits a change, the entire app is built from scratch in a clean environment, it is automatically tested to within an inch of its life, and then – if and only if it passes those tests – it is deployed so that human testers can continue to test it. The automated tests range from integration testing to unit testing to application testing to UI testing. Before a human ever sees the code, it has been tested to ensure that internal functions behave as designed, it has been tested to ensure that interdependent code behaves, and UI test suites have literally punched every button in the application to make sure that it does what it is supposed to do. Tests run to make sure that remote resources, such as internet servers that deliver information to the application, are working correctly and that the application can talk to them successfully. Tests run to make sure that the application provides meaningful feedback to the user when errors occur.

And all of this happens every single time a developer makes a change. During active, full-throttle development, this entire process may repeat dozens, if not hundreds, of times per day. If a build fails, the entire team gets an email detailing the failure and assigning blame to the person who caused the failure. The build becomes “red,” and it is that person’s responsibility to make it “green” again – either by removing their changes outright, or by fixing the problem properly.

None of this is exactly rocket science, and I did not invent any of this. It’s standard practice across the industry. And yet, somehow, QSSI and CGI managed to make it 3 years into the development process before attempting any end-to-end testing? Madness.

Apple asks, and answers, this question in every product they make. They’re usually right, but have their missteps, certainly. However, I would argue that their competitors don’t even acknowledge that this is a query, much less attempt to address it.