How to Test Multithreaded Code

Dec 14, 2015

Multithreaded code is hard to write and even harder to test.
Since much of my work is dedicated to making Ruby
threading easier for my users and customers, I thought some might be
interested in the patterns I've developed to make multithreaded code as simple
and testable as possible.

Separate Threading from Work

If you can't test a big block of code, break it into a set of smaller testable pieces.

Sidekiq::Processor is an object which is designed to run in its own
thread and doesn't have any public API aside from starting/stopping the
thread.

p =Sidekiq::Processor.new
p.start

Interally it has quite a bit of complexity - think of it like an iceberg.
In order to test those complex internals, I make its internal API public so that the test suite has full
access to the methods. The start method spins up a thread which calls a very simple run loop similar this:

defrunwhile!@done
job = fetch
process(job) if job
endend

I've kept the run method as simple as possible since we can't call it in
the test suite but we can call fetch and process in order to test them:

In this case, I've kept the thread management code as simple as
possible and pushed as much of the code complexity into separate methods
which can be called directly by the test suite and deterministically
verified.

How do I test the thread management code? Simple: in some cases I don't. 100% test
coverage is for fundamentalists. Keep the code simple, verify it
manually and then don't change it. Code complexity leads to churn which
leads to bugs. Since most of the complexity in Sidekiq::Processor is in the process and
fetch methods, they are most likely to change so we test those methods directly.

Use Callbacks

If you must test multithreaded code, you'll want to design testability
into the API. Ever seen or written a test littered with sleep calls?
We've all been there but you can test threaded code without sleep calls, I swear!
Generally the pattern is:

Start the other thread

Tell the other thread to process something

Wait for the result

Assert results

Most people don't know how to do (3) properly so they use sleep as a
hack. Here's a complete example of how to do it in Ruby:

require 'thread'# We want to test Upcaser by exercising its full API,# including the internal threading.classUpcaserRequest=Struct.new(:args, :block)
definitialize
@queue =Queue.new
enddefstart
@thread =Thread.new(&method(:run))
enddefprocess(*args, &block)
@queue <<Request.new(args, block)
trueenddefterminate
@queue <<nilendprivatedefrunloopdo
req = @queue.pop
breakunless req
# perform the actual work
result = req.args[0].upcase
# call the block with the result
req.block.call(result)
endendenddeftest_upcaser
m =Mutex.new
cv =ConditionVariable.new
a =Upcaser.new
# Step 1# tell Upcaser to start its internal thread
a.start
results =nil# the main thread will lock the mutex so it can pass data# to Upcaser and then wait for the results
m.synchronize do# Step 2# pass "something" to Upcaser for its internal thread to process# the internal thread must call the block with results when done
a.process("something") do|res|
results = res
m.synchronize do
cv.signal
endend# Step 3# the main thread will wait here for Upcaser's thread to finish.
cv.wait(m)
end# Step 4# assert whatever you want about the results
assert_equal "SOMETHING", results
# shut down Upcaser's internal thread
a.terminate
end

The "trick" is the callback block passed to the process method. That callback
will save the results and unlock the main thread once Upcaser's thread is finished processing. If your API
exposes a similar callback mechanism, it can be properly tested across threads.

I hope this helps people untangle some of their messy threading. Got any other
patterns for making threading easier to manage? Please link to them in
the comments.