HP reflogging DataDirect storage

Into Linux HPC clusters

HP has added a third and surprising scale-out file system product to its high-performance cluster (HPC) portfolio, DataDirect Networks S2A9900, the storage that's sold by IBM in its SONAS product.

HP already has its Ibrix and PolyServe-powered X9000 scale-out network-attached storage (NAS) in its cluster portfolio, along with its SFS G3, a product using MSA2000 storage and the Lustre file system, which is aimed at Linux cluster HPC users. In other words, this product line-up implies the X9000 isn't well-suited for Linux HPC cluster use.

Now HP has added the S2A9900 and its Lustre file system alongside these existing HP products. DataDirect Networks (DDN) will be handling the full delivery and support of the Lustre File System for HP as part of the agreement. DDN's statement said the S2A9900 is designed for customers "who are pushing the boundaries of Linux application scaling in traditional HPC applications such as research and engineering".

HP's Scott Misage, a research and development director for scalable computing and infrastructure at HP, stated: "Clients need to cost-effectively expand bandwidth and storage for data intensive research and engineering applications." OK, so they can choose the SFS G3. Why do they need this new product?

"The combination of DDN [DataDirect Networks] technology with HP Cluster Platforms provides clients the flexibility of a custom solution with the simplicity, reliability and value of a preconfigured, factory-built product for high-performance computing applications.” This doesn't really tell us why the S2A9900 is needed alongside the SFS G3.

The implication is that the S2A9900 is cheaper than the SFS G3, but no pricing information was revealed by either DataDirect or HP.

Dave Robinson, HP's Enterprise NAS technology lead, said: "Lustre is a file system which has been designed to a specific workload in the HPC space. This type of workload involves high concurrent access to very large files (eg seismic data analysis, fluid dymanics).

"For these specific workloads HP is offering an optimised architecture involving Lustre and MSA or DDN hardware. DDN is a dense, cheap and large scale array that is also designed specifically for the HPC market and is proving very popular there. In this case HP is offering best of breed for this niche market."

The X9000 is for the 90 per cent of the large file workloads HP is seeing, which involve are random access, multiple applications, and smaller file sizes. Robinson said: "For these 90 per cent of use cases the X9000, with its scale-out architecture and a segmented file system, is the far better [product]."

DDN's marketing VP, Jeff Denworth, said: "While you see HP supercomputers populated frequently on this [top 500 supercomputer] list, what is not common are instances of iBRIX or HP x9720 platforms within the top 100 ... this is generally the domain of the Lustre and GPFS file system. The scalability and performance and capability of these file systems are continually proven."

Denworth said DDN's S2A9900 has "a number of unique technology features which lend themselves to this class of computing and I/O, such as: zero-impact drive rebuilds to avoid the pitfalls of Amdahl's law in striped file environments, real time parity calculation and correction, custom storage algorithms optimized for large HPC-style file transfer, RAIDed storage enclosures, native InfiniBand support, and the list goes on..."

This looks more like a performance superiority positioning than one based on lower cost.

The S2A9900 box, a block storage product is popping up on the content-intensive, scale-out application radar screen. IBM has included it in its SONAS offering, IBM with its own multifarious storage offerings has done an OEM deal with DataDirect.

UK video processing and special effects company Smoke & Mirrors is using DDN's 6620 with IBM's GPFS file system in its media processing work. Quietly, it seems, Data Direct is pushing into the BlueArc, Isilon and Panasas scale-out NAS file system space for media-intensive and HPC application use. ®