Unit Testing – The Big Picture

Published April 22, 2017

This is not a crash course of what characters one needs to type in his code editor to produce unit tests. This is fuel the brain requires before attempting such actions.

The subject of Unit Testing is not as simple as one might think. Many of us, developers, go into unit testing based on pressure from clients, peers, colleagues, our heroes and so on. We quickly learn the value of it, and, once the tech setup is done, there is a tendency to forget the big picture, if it was ever learnt. This article will provide a short insight into what is and isn’t unit testing in PHP and in general, and unit test place in the quality assurance realm.

What is testing?

Before diving into unit tests, you should have a look at some deeper theory of what testing actually is, so you don’t make mistakes like one of the most popular PHP frameworks on their website showing integration tests and calling them unit tests. They’re not, Laravel, they’re really really not. I still love you though.

Software testing is defined as “investigation conducted to provide stakeholders with information about the quality of the product” as oppose to “Software testing is developers wasting project money doing nothing important and then asking for extra time and budget because “nothing” can be really expensive”. It also isn’t anything new. Have a look at my compiled brief history of testing:

If you are a millennial like me, you could be shocked that there were software test teams way, waaay before you were born. Take a moment. Relax. Deep breaths.

What the history tells us is the type of testing software required to be “good enough” for the stakeholders over time. Here are approximate testing orientation phases:

… – 1956 debugging

1957 – 1978 demonstration

1979 – 1982 destruction

1983 – 1987 evaluation

1988 – … prevention

Ergo, the unit testing is required to prevent discrepancies between the design and implementation.

What really is testing?

There are multiple ways to classify software testing, I will mention the most generally accepted ones to better understand the place of unit tests.

Tests can be categorized as: Static VS Dynamic, Boxed (White box, Black Box, Grey Box), and Levels and Types. Each of these 3 approaches classify tests by different criteria.

Static & Dynamic testing

Static testing is what one does without actually executing the code, such as proofreading, verification, code reviews (over the shoulder / pair programming), walkthroughs, inspections etc. Dynamic testing is when the actually code needs to be executed to receive a valid result, for instance, unit tests, integration tests, system tests, acceptance tests etc; It is testing with dynamic data, testing various inputs and outputs.

The Box Approach

According to the box approach, all software tests can be divided into 3 boxes:

White Box tests verify and validate internal structures, units, ignores end-user expected functionality. This could be API testing, fault injection, unit testing, integration testing. The contrasting Black Box testing is more interested in what the software does, not how it does is. This means that tester must have no knowledge on subject, no understanding on how things work under the hood. It’s all about that end user, his experience interacting with the visible surface of the product. Black box testing types include model based testing, use case testing, state transition tables, specification testing and so on. The third type falls sort of in between. Grey Box tests are designed with knowledge of software algorithms and data structures (white box), but executed at user level for experience (black box), such as regression testing or pattern testing.

Now, to screw with you, let me mention that unit testing can also be Black Box testing because you might have knowledge about the single unit you are testing, but not about rest of the system. I still think of it as white-box-only though, and wish you do the same.

Test Levels

Test levels vary, there are usually 4-6 tiers and all of the ones I’ve seen makes sense. The names also change, depending on companies culture you might know Integration tests as Functional tests and System tests as Automated tests and so on. So to keep things simple I will describe testing levels in these 5 tiers:

Unit testing

Integration testing

Component interface testing

System testing

Operational Acceptance testing

Unit testing verifies functionality of a specific section of code, mostly one function at a time. Integration testing verifies interfaces between components, verifies that units combined together form a working system as designed. This is an important point, because large portion of unit tests out there are actually integration tests, and developers assume they are unit. If it involves multiple units – it is the integration between them that is being tested, not the unit itself. Component interface testing verifies data passed between various units. For example, get data from unit one – verify – pass to unit 2 – verify outcome. System testing is end to end testing to verify all requirements are met. Operational Acceptance testing is conducted to verify operational readiness. It’s non-functional, it tests if services are healthy, if any sub-system does not damage environment and other services.

Testing types

Every type of testing can also have a type regardless of its level. There are more than 20 accepted software test types. The most widely spread are:

Regression testing

Acceptance testing

Smoke testing

UAT

Destructive testing

Performance testing

Continuous testing

Usability testing

Security testing

Types are relative to the purpose the test is executed for. The ones in bold are PHP unit tests. Ones in italic – sort of / sometimes / get off by back. If one really wanted to, he could put unit testing under each and every of those terms. Really, though, main type of unit tests are regression tests – to verify if all units of the system still execute validly after code changes have been introduced.

So now you know that unit tests are dynamic tests, white-box tests, unit level tests, they are regression tests, but can have multiple test types associated with them. But what are unit tests, really?

What is Unit Testing?

When detailed product requirements are verified and set, and code writing has started, the first line of defense against any discrepancies are unit tests. This is why companies who understand what they are doing are pushing developers for mandatory unit tests, or even TDD, because it will cost so much less to fix bugs here than in later stages.

And it makes sense. There are plenty of benefits of having unit tests. They:

Isolate each part of the program and validate its correctness

Help to find problems early

Force developers to think in inputs, outputs, and error conditions

Shape code in testable form, facilitate future refactoring

Simplify integration of working(!) units

Partially substitute technical documentation

Enforce interface separation from implementation

Prove that the code of the unit works as expected (at least mathematically)

Can be used as low level regression test suite

Demonstrate progress of unfinished system integration

Reduces cost of fixing bugs (even more with TDD)

Creates better application designs by defining unit responsibilities

If you can test it – you can join it to the rest of your system

Unit testing is FUN!

There are, however, a few limitations, that you probably imagined reading the list of wonders above. Unit testing:

Does not catch integration errors

Every Boolean statement requires at least 2 tests, and it grows quickly

Unit tests are as buggy as the code they test

Binding tests to couple of specific frameworks or libraries can limit the testing workflow

Most tests are written after development is finished… which is sad. Go TDD!

Possible that after small refactoring system is working as before, but tests – failing

Increases development costs

Human error – commenting out broken tests

Human error – adding workarounds in code just for unit tests to pass

The last one pisses me off more than anything. In (almost) every project I pick up there are lines right inside the source code of production application like “If this is unit test, load stubbed SQLite db, otherwise load other db” or “if this is unit test, don’t send email, otherwise do” etc. If the application design is bad, don’t pretend you can fix crappy software with good tests, it won’t make it better.

There have been plenty of discussions with my colleagues and client developers on what good unit test is. The bottom line – Good Unit test is:

Well named (and clearly enough to avoid debugging to know what failed)

To the ones smirking about “automated” – I don’t mean PHPUnit or JUnit integration in CI pipelines. I mean – if you change your code, save it, and do not know whether or not modified units still confirm to their tests – they’re not automated, and they should be. File watchers FTW.

What to unit-test?

In a healthy system unit tests should be written for:

Units – smallest isolated parts of system that perform single task (a function, a method, a class)

Public methods

Protected methods, but only in rare cases, and when nobody’s looking

Bugs & Bug fixes

The definition of what is a unit comes from developer, who wrote the code. In my case in PHP it is almost always a class method or a function, because that is the smallest part of the software that has meaning by itself. Few times I have also seen developers treating array of mini one-method classes as one unit, and it also makes sense if the minimum viable meaning requires multiple objects to exist.

So define whatever is a unit for you. Or test methods one by one and make life easier for the next guy that picks up the code.

If you don’t do unit testing, I suggest getting into it after your next bug comes along. Verify which method it comes from, write a failing test with proper args and result, fix the bug, re-run test. If unit test passes, you are sure that was the last time you needed to fix that bug (with your defined input scenarios).

This way the unit testing process becomes easier to grasp. Look at each method separately, it has an input and an output. Data Providers can help you define inputs and outputs for all scenarios that come in your mind, making sure whatever happens – you know what to expect.

What NOT to test

Defining what not to test is slightly trickier. Here I have tried to compile a list of items that should not be unit tested:

Functionality outside units scope(!)

Units integration with other units(!)

Non-isolated behavior (unmockable dependencies, real DB, network)

Private, Protected methods

Static methods

External libraries

Your framework

I am sure that you should not unit test any of the above, apart from static methods. I like to use the argument that being static basically means procedural, in many cases global. If static method calls another static method there is no way to override that dependency. Which means you are no longer testing in isolation. Which means this isn’t a unit test anymore. On the other hand, it is a piece of code able to live by itself, it has a meaning, and it should be tested to make sure whatever part of that sorry-ass system is calling it doesn’t break. So I think it is fine to test static methods if you can make sure that no other test can change the output of this test, and that language or framework allows doing that natively.

How to write a Unit test?

Write unit testable code, then test it

Write unit testable code, then test it

Write unit testable code, then test it

In case “then test it” is not enough, laracasts.com has very good videos on the topic for PHP, worth the time. Plenty of other sites for JS/Java/YouNameIt. There is no point for me to explain how I unit test code, because the tools for this methodology change quite rapidly, and, by the time you read this, I might have switched from PHPUnit to Kahlan. Or not. Who knows.

But to the first one – How to write unit testable code? – the answer is a bit easier and does not seem to change over time much:

SOLID

DRY

No new keyword in constructor

No loops in constructor (and switches, if clauses)

No static methods, parameters, classes

No setup() methods – Object must be fully initialized after construct

No Singletons (Global state) and other untestable anti-patterns

No God objects

No mixed concern classes

No hidden dependencies

Now – knowing what unit tests are, what they aren’t, what to test, and what not, and where is unit test place in software development life-cycle should make it easier to implement them. What’s left is to find a framework or library you like and stick with it. When in doubt – go with de-facto standard for framework/language.

In conclusion – unit tests are very important to both developers and businesses, they should be written, and there are best practices that help cover your units with tests easily, mostly by preparing units themselves. But none of that practical application makes sense without the testing theory I have provided above – one needs to differentiate unit tests from all other types of tests, and, once the difference is clear, it makes it so much more easier to write them.

I hope you enjoyed reading this as much as enjoyed writing it, and I hope that now you have better understanding of what exactly unit tests are, and why they are the most important type of software testing out there. Apart from integration tests.

I spent a little more time on the core concepts and included a few code as well as test examples, but didn’t go into that much detail regarding other test levels, or individual test types and techniques. I therefore think the two posts are good additions to each other, and I will definitely mention yours in the References & Further Reading section of my posts and talks/slides.