Hadoop Performance on vSphere 5.1

We've just published a third Hadoop performance paper, written by VMware performance expert Jeff Buell, which looks in detail at the relative performance of a bare-metal 32-node Hadoop cluster compared to a range of virtual clusters with up to 128 VMs. The executive summary is that while we saw a 13% performance degradation in a head-to-head comparison of a 32-node physical cluster against a 32-VM virtual cluster (one VM per host) running on the same hardware and running the same tests, virtualized performance can be increased significantly -- to the point where virtualized Hadoop actually runs a bit faster than physical -- by increasing the number of VMs per host. We've seen this effect before with Hadoop and with other resource-intensive HPC applications.

Read the full paper for detailed results and to learn about performance best practices for deploying Hadoop on vSphere.

With over 20 years of experience in High Performance Computing, Josh currently leads an effort within VMware's Office of the CTO to bring the full value of virtualization to HPC.
Previously, he was a Distinguished Engineer at Sun Microsystems with broad responsibilities for HPC direction and strategy ... More