ScaleMP, IBM bond to fluff x64 systems

How come the li'l one is yet to be et?

Starting Tuesday, middleware-maker ScaleMP's Versatile SMP Foundation software is certified to run on IBM's System x3850 X5 servers — and that makes Big Blue the Cupertino, California company's best friend.

It is a wonder that ScaleMP, whose middleware glues multiple x64 systems together into fatter shared memory systems, has not been snapped up by one of the major server players that are all eager to get some sort of edge over their rivals. For whatever reason, ScaleMP remains on its own and is reliant on partnerships with those server makers who would naturally want to own the technology for itself — like its BFF, IBM

The System x3850 X5 server, as El Regexplained in March, is a four-socket box based on IBM's own eX5 chipset that can pack 32 cores and 64 memory DDR3 slots into a 4U chassis, for a total maximum memory of 1TB using 16GB sticks.

But lots of jobs want more main memory, and that's why instead of worrying so much about SMP scalability in its eX5 chipset, IBM created a funky feature called Max5 that allows for another 32 memory slots, or 512GB of main memory, to be added to the four-socket box.

In effect, the Max5 unit, which is a 1U rack-mounted box with memory modules and a cable for linking into the SMP electronics of the eX5 chipset, is an SMP server node that just so happens to not have any processors in it.

You would think that 1.5TB of memory for a four-socket node would be pretty good for a fat node in an HPC cluster. But more is never enough, at least not in the first 50 years of the systems business, and on some workloads — particularly those in financial services and life sciences where there is a lot of message traffic between the nodes in the cluster — ScaleMP is getting some traction peddling its vSMP clustering as a better way of lashing machines together and masking the network complexity from the applications running on the cluster.

With the vSMP Foundation 3.0 virtual symmetric multiprocessing software announced by ScaleMP back in May, vSMP is still restricted to using InfiniBand links between the nodes, is still only running on Linux, and is still not perfectly suited to all workloads. But the software can now scale to 128 nodes — up from 16 nodes — and the microcode at the heart of the vSMP clustering technology can address up to 64TB of memory across those nodes.

What this means is that companies that might have had to try to glue together eight-socket Xeon 7500 machines using 16GB memory sticks, can build up a big vSMP machine using cheaper 8GB sticks and cheaper four-socket server nodes with the Max5 features. To hit that 64TB limit, you'd need "only" 85 of IBM's System x3850 X5 servers, each with a Max5 memory extender and each using 8GB memory sticks and sporting 768GB of memory.

At the moment, supercomputer maker Appro International is probably one of the more enthusiastic sellers of vSMP, and is working on two flash-heavy supercomputers at the San Diego Supercomputer Center that make use of vSMP to create fat and virtual shared memory server nodes that are then interconnected by InfiniBand networks in a traditional way.

IBM, HP, Dell, and various downstream channel partners of Super Micro with HPC specialization are all resellers of ScaleMP's vSMP Foundation glue. You can see the latest hardware compatibility list for vSMP here. There are 36 certified machines on the list, including some whitebox machines from Intel. Here are some benchmark results that show how vSMP does in terms of scaling across server nodes — note that some of these stats are getting a bit long in the tooth and need to be updated.

Shai Fultheim, the company's founder and CEO, says that these tier-one players drive most of the company's business and account for most of its 175 installations.

Ironically, all of the marketing effort that Silicon Graphics does in talking up its "UltraViolet" Altix UV 1000 supercomputer nodes is helping make the case for ScaleMP's vSMP. The Altix UV 1000 machines create a global shared memory that maxes out at 16TB for 128 two-socket Xeon 7500 blade servers using the NUMAlink 5 hardware interconnect. For a lot of applications, a fat node with a single address space is better than a bunch of more loosely clustered servers. It is arguable that vSMP is not as tightly coupled as what the NUMAlink 5 interconnect offers, but vSMP works on whatever x64 servers you have lying around, as long as they have InfiniBand links.

IBM's lack of an x64-based system that scales beyond eight sockets and that can create something akin to a global shared memory space like the Altix UV 1000s explains Big Blue's interest in vSMP. The eX family of chips can, in theory, scale to 16 sockets and perhaps even to 32 or 64 sockets, but thus far IBM has preferred to make fat x64 machines based on Max5 technology and leave its largest SMP scalability for the Power 795 Unix box, which scales to 256 cores and 8TB.

In addition to announcing the certification on IBM's System x3850 X5 server, Fultheim said that ScaleMP has rejiggered its vSMP Foundation 3.0 prices to make them more fair across different server types.

A license and support contract for a two-socket server using four-core x64 processors, for example, now costs $1,750, down from $2,500. A two-socket node using six-core x64 chips costs the same, at $2,500 per server. If you are using eight-core chips in four-socket boxes, the price is still $7,500 per node, but if you drop down to the cheaper six-core x64 chips used in four-socket servers, then the license drops to $5,000 per box. While the pricing is not uniform per core, it is a lot closer with this pricing scheme.

Dell, HP, SGI, Appro, and IBM are all likely suitors for ScaleMP — if the company were for sale. Intel is an unlikely but interesting possibility — imagine if Intel pushed vSMP into the firmware. VMware could also use a server-aggregation technology to match its server-dicing hypervisor, ESX Server. And finally, there's Microsoft, which could kick Linux by acquiring ScaleMP while at the same time using the technology to bolster its own HPC position — if the vSMP server virtualization and aggregation technology is even compatible with Windows.

In late July, ScaleMP secured $2.4m in Series E funding to help grow its business, raising its total to $28.4m so far with investments from Sequoia Capital, Lightspeed Venture Partners, ABS Ventures, and TL Ventures. ScaleMP said that it had "record growth" in the quarter ended in June and that commercial companies, rather than just HPC labs, made up more than half of its sales.

ScaleMP is privately held and doesn't divulge its financials, so it is hard to say how much it might cost to acquire it based on its revenue stream and profits — if, indeed, it's even profitable at all. ®