ScaleMP cuts InfiniBand out of virtual SMP clusters

Fake SMPs for SMBs and clouds

ScaleMP, a maker of virtualization and aggregation software that allows a cluster of x64 servers to look like a big, bad, symmetric multiprocessing (SMP) shared-memory system to operating systems and selected classes of applications, is going downstream to target SMBs and upstream to chase cloud infrastructure providers.

While ScaleMP wants to make a lot of noise about its vSMP Foundation for Cloud edition, which starts shipping Wednesday, the more interesting of the two products the company is introducing today - and the one that might help it expand beyond its current 150 customers - is vSMP Foundation for SMB edition, which allows for customers to create an eight-socket, aggregated, and virtualized server from four two-socket servers (and here's the neat bit) without having to use an InfiniBand switch to lash the server nodes together.

Shai Fultheim, founder and CEO of ScaleMP, says that the SMB edition of the server-aggregation product makes use of something the company calls Direct Connect 2, or DC2 for short. (If there was ever a DC1, no one ever mentioned it.) With DC2, you slap InfiniBand adapters in four server nodes, load up the InfiniBand drivers in the vSMP hypervisor (which comes on a USB stick for each server node), and then cross-wire each server back-to-back directly to the other servers in the four-node cluster. The vSMP hypervisor creates what amounts to a virtual backplane to couple the four servers together into what looks like an eight-socket SMP machine as far as Linux is concerned.

The DC2 approach does two things. First, it allows a virtual eight-socket SMP machine to be created without a switch, which eliminates about 20 per cent of the overall cost of a vSMP setup, according to Fultheim. Because of the relatively low volume of sales and higher complexity, four-socket and eight-socket servers are considerably more expensive than buying two or four two-socket machines, which constitute the volume of rack and blade servers sold today in the x64 world. By eliminating the InfiniBand switch cost for an entry-level machine, making a virtual SMP will be a lot less expensive than a current eight-way Xeon or Opteron box.

The other big improvement with the vSMP for SMB edition and the DC2 interconnection scheme is that there is a lot more bandwidth available for supporting applications than is available using the switched version of the product, which scales across more nodes. The direct links have about three times the interconnection bandwidth, according to Fultheim, and that should translate into better performance on some workloads. For the sake of comparison, ScaleMP says that a four-node vSMP machine using Intel's quad-core Xeon 5570 processors delivers 35 per cent better floating point performance and 75 per cent more memory bandwidth compared to real eight-socket SMP servers using the latest six-core Istanbul chips from Advanced Micro Devices, and adds that the resulting four-server vSMP machine costs 25 per cent less.

However, vSMP doesn't support all workloads equally - it's really aimed at high performance computing and other kinds of workloads that have a lot of message passing but which benefit from a large shared-memory space. ScaleMP also only supports Linux operating systems atop the vSMP hypervisor, and while Fultheim says that the company will expand to support other operating systems in the coming year, he will not say that Windows Server 2008 is one of them. But it had better be.

It doesn't look like ScaleMP is all that interested in clustering other types of servers, either. "X64 is the majority of the market," says Fultheim. "We have had some discussions with other players about supporting other architectures, but we are not prepared to say more at this time."

The other new product coming out of ScaleMP with the vSMP Foundation 2.1 release is the cloud edition of the product, which is designed to allow clusters of hundreds of server nodes to be aggregated into virtual SMP machines ranging from 2 to 16 nodes, and do so on the fly from Moab, Rocks, and other management tools commonly used by supercomputer centers. With the vSMP Foundation Cloud Edition, there is still a 16 node limit on scalability for any single instance of an operating system, but a giant cluster can be rejiggered so the memory capacity and core-count profiles can be matched to the needs of specific workloads. Some workloads don't mind being on two-socket nodes with 32GB of memory; others need to have lots more cores or lots more memory in a node to work properly.

The important thing is this: you use vSMP to aggregate and carve up a cluster of cheap boxes and avoid having to use big, expensive boxes and then carving them up with virtual machine hypervisors such as VMware's ESX Server, Microsoft's Hyper-V, or Citrix Systems' XenServer. If the workload needs a specific amount of CPU and memory, you allocate it from vSMP Foundation. If that changes, you scale the resources up.

The cloud edition is slightly different from vSMP Foundation for SMP, which is for creating a static shared-memory virtual SMP that can span up to 128 cores and 4TB of main memory using Intel's Xeon 5500s, and vSMP Foundation for Clusters, which allows HPC applications tuned for the Message Passing Interface (MPI) protocol commonly used in parallel supercomputers to run on a clusters with between 4 and 64 nodes. The cluster variant still can have only one operating span across a maximum of 16 nodes in a virtual SMP configuration; it has a 512GB memory limit (4GB per core) and also runs on 2.4GHz or slower processors. The cloud edition is not deployed from the USB sticks (which are basically the license keys for the product), but rather are deployed from the network. The cluster and cloud editions of vSMP Foundation do not offer the DC2 links; that only comes with the SMB edition of the basic vSMP Foundation product.

Both the cloud and SMB editions of vSMP Foundation 2.1 are in beta testing now and will be generally available on December 1. The basic SMP products costs $3,500 for a license to cover the first two server nodes, and then $3,500 a pop for each additional server node added to the vSMP cluster. HPC shops using vSMP Cluster edition pay $12,250 for the first eight nodes and then $1,750 for each additional node under management. The Cloud edition runs a management server on an x64 box, and the license for the management code and the unlimited number of hypervisors it can manage costs $65,000. Each node in the cloud that will participate in the vSMP dance costs $650 a pop. ®