The Non-Tech Manager's Guide to (Legacy) Software Development

Why Do I Need Automated Tests?

Automated tests come in all shapes and sizes, but the main idea behind it is that you can run a suite of tests on your software by clicking a button. This suite of tests then provides certain input to pieces of your software and verifies that the output is what it should be. It serves as an easy health-check of your application.

One could say that writing software like this is a more scientific way of writing software. Instead of assuming the software is doing what it should, we set up a hypothesis and prove it right.

Now why are these kind of tests so important in modern software development? Both managers and developers sometimes feel they will finish the task faster if they don’t have to write tests. They are over-confident that the implementation will be right the first time and can simply be tested manually. This line of thought ignores some important aspects of software development.

Tests Don’t Slow Down Development

The speed of your software development isn’t determined by the amount of code that is typed. It might feel counter-intuitive, but even though writing tests amounts to more code being written, it can still make the development of a feature faster. Here’s why.

First of all, when developers are writing code, automated tests can give them very fast feedback about the code they’ve just written. Without automated tests, they often have to spin up the application and perform multiple steps before being able to verify their task. This approaches is costing you money.

Second, automated tests lead to fewer bugs. Without automated tests, developers might even skip the testing because they believe their code is correct. And because they don’t like the lengthy testing process I describe in the previous paragraph. This is a human flaw.

Finally, automated tests push developers towards a better design. The code will be structured more cleanly and allow for easier refactoring and updating later. It’s a virtuous cycle.

Testing Manually Is Flawed

One of the reasons so many manual processes are automated in the world is that humans make errors. If we can automated a human process and if we can do so without bugs, then we can rely on the process to always perform correctly.

Say you have your developers create a new feature. Someone then runs a series of manual tests and determines that all is well. Yet, a year later, suddenly this feature no longer works as expected. Sounds familiar? This is because often we stop testing features that were approved. But the developers won’t stop working in pieces of code that affect this feature. And so bugs get introduced and released.

Another point is that working in the code provides a different perspective. We can think about the features that we need, along with some edge-cases. But when developers write tests, new questions arise about special cases that nobody thought about previously. It forces them to think about the code more.

Testing Manually Is Expensive

Say you want to avoid the previous pitfalls and document all your manual tests so that they can be performed in the exact same way in the future. And maybe you even appoint a dedicated tester that manages and performs all these tests.

This could allow you to find bugs before they’re released. But it remains a ridiculously expensive operation. And it doesn’t address most of our issues.

Manual testing by a dedicated testing team means that you remove the responsibility of delivering bug-free code from the developers. You can say they should still provide quality code, but psychologically, they’re no longer responsible for testing. This leads to more bugs slipping through. Granted, your testing team will catch them, but this means the code then goes back to the developers, who have to drop whatever they were doing and fix the bug. It then goes back to the testing team, and this can go back and forth for quite a while. All this time you’re losing time and money.

A manual testing team also means developers have less incentive to write clean code that can easily be tested automatically. This leads to bad quality code and technical debt over time.

Finally, paying a human to test software is a ridiculous waste of money when a cheap computer can do the same. Most pieces of software testing can be automated. Over time, this gains increase exponentially.

Say a human tester needs one hour to run all the test scenario’s. At €80 per hour, this means it cost €80 for one test run. If the tester runs the tests once a week, this means €320 per month, or €3840 in a year.

Now take Bitbucket, an online platform to develop software. The standard plan starts at $10 a month. Let’s keep it simple and translate that to €10. For this €10 you get 500 build minutes. A build is where you can build your software product and run the tests. If we split it in half we get 250 minutes for building and 250 minutes for testing.

So our manual tester tested for 240 minutes and cost us €320. Bitbucket would cost use €10 for 250 minutes of testing. In one year the difference is even bigger as the manual tester cost us €3840 and Bitbucket only €120.

Manual Testing Still Has Value

I admit I made multiple assumptions in my calculations. But the general idea should be clear: automated testing is better and cheaper than manual testing.

However, manual testing still has value. It is very hard to automate the testing of user experience or aesthetics. The software might do what we want, but maybe working with it is very unpleasant.

Another fact is that developers don’t always think about certain cases. Someone with more business experience can help them think about certain scenario’s that need to be tested. These test cases often only come up after some manual testing.

But any manual test that can be automated, should be automated. To avoid having the test fail later but nobody knowing because nobody performed the test. To reduce the costs of testing. To catch bugs earlier (which also reduces cost).

Getting Started

So you want to start with automated tests but don’t know how or where? Reach out to me and I might be able to help. Depending on you situation you might need to educate developers or change the procedures you use to develop software. Their are various techniques to implementing automated tests and there are different kinds of tests.

There are different answers to these questions, and the answers may change over time. But the important thing is to start with automated tests if you haven’t already. While manual tests still can provide some value, automated tests allow you to move faster and cheaper.