The perils of writing request specs using concurrent-ruby under the JVM

26th August 2017

When I write an API, though I'm not a hard core TDD practitioner, I do like writing specs - especially requests specs that test the whole stack.

Adding them into an API is fast and yields quite good results compared to an app with an UI where you have to use chrome-cli or phantomjs just to get near of that level but at the cost of painfully slow execution time (yes, even if you optimise them to hell and back and get a runtime of five minutes - they're still slow in my book).

Anyhow, for some context - I've been writing Ruby APIs for quite some time but using the classic CRuby VM/interpreter - this time we switched to JRuby (thanks to Max ) since we needed to render extremely quick JSON responses (i.e. 202 status codes) - so everything from ActiveRecord CRUD operations to processing business logic is done using concurrent-ruby with Futures and ThreadPools (which in JRuby they use Java's native implementation - awesome stuff since one can play with STM and all that good stuff - if required).

Next stop: the core of the issue: running request specs with all this parallelism in the background breaks most of our specs because RSpec is not aware of anything which runs in parallel and the inherent non-deterministic nature of running code in parallel.

The initial fix for this was to add a helper that basically forces a shutdown of the ThreadPool which will block until it's done - this "ensured" that all the parallel tasks (like creating a record) would finish before we got to the expect part in the specs. All nice and rosy except it did not work as expected.

This was the initial implementation btw. :

defwait_for_thread_pool!(sleep_time=nil)thread_pool_executor=executor.instancepool=thread_pool_executor.executor# shutdown the pool and wait as long as it takespool.shutdownpool.wait_for_termination# we want a fresh thread pool for the next testthread_pool_executor.send(:initialize)end

This is quite self-explanatory - it issues a ThreadPool shutdown which waits for all the threads to finish their work - the problem? In 10% of the cases using RSpec random spec execution it failed.

For the time being - the next best fix was to monkey patch the library (well, to be more precise our adapter for it) in order to run everything sequentially - especially keeping in mind that concurrent-ruby is an externally tested library and the issue should be an odd interaction between RSpec, DatabaseCleaner and concurrent-ruby.

In any case until I can start digging into all those dependencies and see what the actual issue was this fixed the issue:

Regarding specs/coverage - the next step is to improve code-coverage in the unit-tests realm as to be sure all the small parts are working correctly.

Conclusion

JRuby + concurrent-ruby is a boon - there are some drawbacks as pointed above and many more which I'll detail in future posts but it really pays off when you really need quick responses. En plus, concurrent-ruby literally levels up JRuby to a place where it can compete with Elixir, well Erlang's concurrency model (yes it has Actors but they're in the edge branch for now). Its abstractions are top notch and they remove a lot of pain from dealing with Threads with some minor caveats.

Updates

A fellow redditor - i_know_sherman suggested concurrent-ruby's ImmediateExecutor which is a special executor that basically runs everything sequentially:

An executor service which runs all operations on the current thread, blocking as necessary. Operations are performed in the order they are received and no two operations can be performed simultaneously.

A small caveat here: it works correctly except when using Futures with the option dup_on_deref set to true.

Another suggestion from moomaka - this time for keeping parallelism on:

Problem may be that sometimes the tasks in the queue needs to queue other tasks on the same executor and those tasks are being rejected. May be able to fix this by setting the :fallback_policy of the executor to :caller_runs which will then run the rejected tasks immediately in the caller thread allowing everything to complete.