What's a good way to leverage TDD to drive out thread-safe code? For example, say I have a factory method that utilizes lazy initialization to create only one instance of a class, and return it thereafter:

Now, I want to write a test in good TDD fashion that forces me to make this code thread-safe. Specifically, when two threads call this method at the same time, I don't want to create two instances and discard one. This is easily done, but how can I write a test that makes me do it?

I'm asking this in Java, but the answer should be more broadly applicable.

Thanks. It came up this morning while write just such a factory. It's come up before when discussing the double-checked locking mechanism, but that was before SO was around.
–
Don BransonFeb 11 '09 at 15:06

You could inject a "provider" (a really simple factory) that is responsible for just this line:

textLineEncoder = new TextLineEncoder();

Then your test would inject a really slow implementation of the provider. That way the two threads in the test could more easily collide. You could go as far as have the first thread wait on a Semaphore that would be released by the second thread. Then success of the test would ensure that the waiting thread times out. By giving the first thread a head-start you can make sure that it's waiting before the second one releases.

+1 I like this. A really slow TextLineEncoder constructor would virtually guarantee to bring out the concurrency issue here. An ITextLineEncoderProvider implementation could easily simulate that.
–
Wim CoenenFeb 25 '09 at 7:54

And as a bonus, you can gain flexibility and extensibility (can use different text line encoders) and more focused testing (can use TextLineEncoder mock).
–
Wim CoenenFeb 25 '09 at 8:58

1

Good idea... And it can even be implemented without using a slow factory for TextLineEncoder, by using a mocking tool which supports the mocking of constructors: in the first invocation to the constructor (first thread), it would simply block; then if the second thread comes in and executes the constructor a second time, the test would fail.
–
RogérioJul 24 '12 at 13:10

In the book Clean Code there are some tips on how to test concurrent code. One tip that has helped me to find concurrency bugs, is running concurrently more tests than the CPU has cores.

In my project, running the tests takes about 2 seconds on my quad core machine. When I want to test the concurrent parts (there are some tests for that), I hold down in IntelliJ IDEA the hotkey for running all tests, until I see in the status bar that 20, 50 or 100 test runs are in execution. I follow in Windows Task Manager the CPU and memory usage, to find out when all the test runs have finished executing (memory usage goes up by 1-2 GB when they all are running and then slowly goes back down).

Then I close one by one all the test run output dialogs, and check that there were no failures. Sometimes there are failed tests or tests which are in deadlock, and then I investigate them until I find the bug and have fixed it. That has helped me to find a couple of nasty concurrency bugs. The most important thing, when facing an exception/deadlock that should not have happened, is always assuming that the code is broken, and investigating the reason ruthlessly and fixing it. There are no cosmic rays which cause programs to crash randomly - bugs in code cause programs to crash.

There are also frameworks such as http://www.alphaworks.ibm.com/tech/contest which use bytecode manipulation to force the code to do more thread switching, thus increasing the probability of making concurrency bugs visible.

When I test drove an implementation that needed to be thread safe recently I came up with the solution I provided as an answer for this question. Hope that helps even though there are no tests there. Hope link is OK raher than duplicating teh answer...

Yes, a link is fine. I really like your answer - it's the first I've heard that has a feel of rightness about it. Perhaps because many of the others rely on non-deterministic behavior. Particularly cogent is your comment that "First of all your existing class has one responsibility and that is to provide some functionality." An object knowing about its thread-safeness is analogous to it knowing about that it's in a linked-list, or in a web container. It knows where it lives, so to speak, and for objects, that's not good. I'll have to try your approach, and see how it goes.
–
Don BransonNov 17 '09 at 16:33

Chapter 12 of Java Concurrency in Practice is called "Testing Concurrent Programs". It documents testing for safety and liveness, but says this is a hard subject. I am not sure this problem is solvable by the tools of that chapter.

Just off the top of my head could you compare the instances returned to see if they are indeed the same instance or if they are different? That's probably where I would start with C#, I would imagine you can do the same in java

Sounds like a useful start. It's easy to write a test of that sort, harder, however, to write it so that it fails due to lack of thread synchronization.
–
MorendilFeb 11 '09 at 14:39

I think it's a good start - but if a thread switch occurs between the construction and the return, they could create two instances, and still both return the same instance.
–
Don BransonFeb 11 '09 at 14:41

this is true. you would need to ensure that you have proper locks around the system under test
–
Sean ChambersFeb 11 '09 at 15:22