Using R for parallelizing OpenBUGS on a single Windows PC

It seems that most of the R-parallelizing business takes place on Linux clusters. And it makes sense. Why would you want to paralellize R on just a few processors (2 or 4) of a Windows laptop PC when the whole thing would be only 2-4x faster. This used to be evident from the selection of R libraries – the easy ones (foreach, snow) were for Linux only and I always found it difficult to paralellize R in Windows. One could use something called MPI but the documentation always seemed too complicated for my non-IT brain. And that was when I discovered that the embarrassingly parallel snow package has been adjusted for Windows.

There is a case when running embarrassingly parallel R makes utter sense on a single Windows machine: the parallelizing of MCMC chains in Bayesian modelling framework in OpenBUGS and WinBUGS. One usually runs several MCMC chains of iterations and these are nearly impossible to parallelize because of the within-chain dependence. But the chains themselves can be easily left running next to each other simultaneously as there is no between-chain dependence. Even a 2-4 fold increase of speed is desirable in this case, as it can facilitate debugging of the BUGS code and the overall workflow is more pleasant. The problem was is that, so far, OpenBUGS does not offer the option to paralellize the chains, although the developers state that it is on the way.

Here I present a simple way to parallelize MCMC chains in OpenBUGS using the snow and snowfall packages in R. I got the main idea from a similar blogpost by Awaypku. The trick is to create a separate working directory for each of the cores one wants to use. Then, several independent OpenBUGS sessions are invoked on several CPUs (using the sfLibrary function in snowfall package), their results are stored in the separate directories and, when the MCMC sampling is over, the resulting CODA chains are retrieved back to R. I am aware that this code may become obsolete when the OpenBUGS developers release the parallel version of OpenBUGS (God knows when). But meanwhile, my code may be handy.

I use an example of a simple Bayesian model in which I estimate mean and variance of a normally distributed random variable.

How about the real speed of this? When I run the 3 MCMC chains serially using the ordinary lapply() function, it takes 21.21s to run the 2000 iterations. Using the 3-chains option within OpenBUGS it takes 16.94s. And finally, my parallel code does the whole thing in 8.67s. It will probably show even better speed advantage on larger datasets and longer chains. Not bad.