There has been a shift in service based architectures over the last few
years towards smaller, more focussed "micro" services. There are many
benefits with this approach such as the ability to independently
deploy, scale and maintain each component and
parallelize development across multiple teams. However,
once these additional network partitions have been introduced, the
testing strategies that applied for monolithic in process applications
need to be reconsidered.
Here, we plan to discuss a number of approaches for managing
the additional testing complexity of multiple independently
deployable components as well as how to have tests and the application
remain correct despite having multiple teams each acting as guardians for
different services.

Two computer security flaws were discovered in early 2014: Apple’s “goto
fail” bug and OpenSSL’s “Heartbleed” bug. Both had the potential for
widespread and severe security failures, the full extent of which we may
never know. Given their severity, it is important for the software
development profession to reflect on how they could have been detected so
we can improve our ability to prevent these kinds of defects in the
future. This article considers the role unit testing could play, showing
how unit tests, and more importantly a unit testing culture, could have
identified these particular bugs. It goes on to look at the costs and
benefits of such a culture and describes how such a culture was instilled
at Google.

The term 'Mock Objects' has become a popular one to
describe special case objects that mimic real objects for
testing. Most language environments now have frameworks that make it
easy to create mock objects. What's often not realized, however, is
that mock objects are but one form of special case test object, one
that enables a different style of testing. In this article I'll
explain how mock objects work, how they encourage testing based on
behavior verification, and how the community around them uses them
to develop a different style of testing.

We give a one-hour overview of Continuous Delivery.
Topics include the justification of Continuous Delivery, the
deployment pipeline, continuous integration, devops, and
deployment strategies. The highlight is Jez's personification of a
release candidate as a hero in a greek myth.

If you need to get the current date or time in your code, don't
access the system routines for that data directly. Put some form of
wrapper around it that allows you to override it by setting the "current date/time"
to a particular value. This is important to simplify testing.

Given-When-Then is a style of representing tests - or as its
advocates would say - specifying a system's behavior using SpecificationByExample. It's an
approach developed by Dan North and Chris Matts as part of
Behavior-Driven
Development (BDD). It appears as a structuring approach
for many testing frameworks such as Cucumber. You can also look at
it as a reformulation of the Four-Phase
Test pattern.

I often get questions that surround one of the design choices in
the JUnit testing framework - the decision to make a new object for
each test method run. Enough to warrant a quick bliki
entry. (However I feel almost compelled to point out that my writing
about JUnit does not mean that that I don't think that other
forms of testing are important. There are lots of useful testing
activities, and although JUnit and its cousins are valuable for many
of them it isn't the solution for everything. For more blogging on
testing I suggest you look at the blogs of Brett Pettichord, Brian
Marick, and James Bach. You should also not assume that my writing about
xUnit testing implies suggests the unimportance of refactoring,
use cases, or flossing.)

I spent some time recently with one of my favorite ever
ThoughtWorks projects. It's a project that started in 1998, using
then new J2EE technology. Over the years it's had a fascinating
history: starting with EJBs, ripping them out, going offshore to
Bangalore, coming back to Chicago. Many people have moved in and out
of the project and the project has varied in head-count between 6 and
60. Overall the project has had over 300 staff-years of effort on
it and weighs in at around 100 KLOC.

When you write tests against a web page, you need to refer to
elements within that web page in order to click links and determine
what's displayed. However, if you write tests that manipulate the
HTML elements directly your tests will be brittle to changes in the
UI. A page object wraps an HTML page, or fragment, with an application-specific
API, allowing you to manipulate page elements without digging around
in the HTML.

Self-Testing Code is the name I used in Refactoring to refer to the practice
of writing comprehensive automated tests in conjunction with the
functional software. When done well this allows you to invoke a
single command that executes the tests - and you are confident that
these tests will illuminate any bugs hiding in your code.

As I listen to our development teams talk about their work, one
common theme is their dislike of things held in statics. Typically
we see common services or components held in static variables with
static initializers. One of the big problems with statics (in most
languages) is you can't use polymorphism to substitute one
implementation with another. This bits us a lot because we are great
fans of testing - and to test well it's important to be able to replace
services with a Service Stub.

From time to time I hear people asking what value of test coverage (also
called code coverage) they should aim for, or stating their coverage
levels with pride. Such statements miss the point. Test coverage is
a useful tool for finding untested parts of a codebase. Test
coverage is of little use as a numeric statement of how good your
tests are.

Test-Driven Development (TDD) is a technique for building
software that guides software development by writing tests. It was
developed by Kent
Beck in the late 1990's as part of
Extreme Programming. In essence you follow three simple
steps repeatedly:

The test pyramid is a concept developed by Mike Cohn, described
in his book Succeeding with
Agile. Its essential point is that you should have many more
low-level unit tests than high level end-to-end tests running
through a GUI.

David Heinemeier
Hansson, the creator of Ruby on Rails, gave a keynote at RailsConf
where he declared that TDD is Dead. This caused a predictably large
amount of controversy in both the Rails and wider software
development community. It also led to some interesting conversations
between David, Kent, and myself. We decided that these conversations
were interesting enough that others might like to watch them too, so
recorded a series of video hangouts where we discuss the role of TDD
in software development.

An automated regression suite can play a vital role on a
software project, valuable both for reducing defects in production and
essential for evolutionary design. In talking with development teams
I've often heard about the problem of non-deterministic tests - tests
that sometimes pass and sometimes fail. Left uncontrolled,
non-deterministic tests can completely destroy the value of an
automated regression suite. In this article I outline how to deal with
non-deterministic tests. Initially quarantine helps to reduce their
damage to other tests, but you still have to fix them soon. Therefore I
discuss treatments for the common causes for non-determinism: lack of
isolation, asynchronous behavior, remote services, time, and resource
leaks.

There seems to be a common misconception in the JavaScript community that testing asynchronous code requires a different approach than testing ‘regular’ synchronous code. In this post I’ll explain why that’s not generally the case. I’ll highlight the difference between testing a unit of code which supports async behavior, as opposed code which is inherently asynchronous. I’ll also show how promise-based async code lends itself to clean and succinct unit tests which can be tested in a clear, readable way while still validating async behaviour.

Here's an interesting contrast I recently picked up. Two
enterprise application projects of a similar size (~100 KLOC), similar
environments (Java and .NET). One can do a full build and test in an
hour, the other takes 2-3 minutes.

I was working on some of my book example code the other day. I
made some changes, got everything working, ran tests, and committed
it to my personal repository. I then moved over to a different area
and made a couple of changes - and some unexpected tests broke in
the previous area. Now part of the point of running automated tests
is to find unexpected breaks, but this book code has completely
independent areas. This was odd.

An in-memory database is a database that runs entirely in main
memory, without touching a disk. Often they run as an embedded
database: created when a process starts, running embedded within that
process, and is destroyed when the process finishes.

A common problem with test-enhanced designs is how to create
Service Stubs in test mode while letting the real thing be there for
production (and for some tests). A couple of my colleagues have
shared their ideas.

One of the classic cases for using a TestDouble is
when you call a remote service. Remote services are usually slow and
often unreliable, so using a double is a good way to make your tests
faster and more stable.

As my career has turned into full-time authorship, I often worry
about distancing myself from the realities of day-to-day software
development. I've seen other well-known figures lose contact with
reality, and I fear the same fate. My greatest source of resistance
to this is ThoughtWorks, which acts as a regular dose of reality to
keep my feet on the ground.

ThoughtWorks also acts as a source of ideas from the field, and I
enjoy writing about useful things that my colleagues have discovered
and developed. Usually these are helpful ideas, that I hope that some
of my readers will be able to use. My topic today isn't such a
pleasant topic. It's a problem and one that we don't have an answer
for.

Gerard Meszaros is working on a book to capture patterns for using
the various Xunit frameworks. One of the awkward things
he's run into is the various names for stubs, mocks, fakes, dummies,
and other things that people use to stub out parts of a system for
testing. To deal with this he's come up with his own vocabulary
which I think is worth spreading further.

There's been a long-running, if low-key, argument between the
advocates of Design by Contract (DbC) and Test Driven Development
(TDD). I'm not going to delve into that right now, but I will pass
on an idea to merge the two that came up when I was talking with
Daniel Jackson.

I'm currently sitting in a session at XP day where Owen Rogers and Rob
Styles are talking about the differences between XP's unit and
acceptance tests. This triggered a thought in my mind - what should a
language for writing acceptance tests be?