Sunday, March 4, 2012

Running DEoptim in parallel has been on the development team's wishlist for awhile. It had not been a priority though, because none of us have personally needed it. An opportunity arose when Kris Boudt approached me about collaborating to add this functionality as part of a consultancy project for a financial services firm.

We were able to add and test the functionality within a week. The latest revision of DEoptim on R-Forge has the capability to evaluate the objective function on multiple cores using foreach. Very CPU-intensive problems will see speed increases in approximately linear time (less communication overhead).

There are a few things to keep in mind when using the parallel functionality. I quote from the meetup presentation:

Data communication between nodes can overwhelm gains from processing on multiple CPUs

Be careful with non-varying objects

Exclude them from formal function arguments

Copy them to nodes before optimization (clusterExport)

If mu and sigma were formal function arguments, they would be copied to each node for all 2037 function evaluations!

Please try it and give us feedback. R-Forge has been undergoing major updates, so please anonymously checkout the source and build it yourself if you're unable to download the pre-built source / binaries.