IBM: Flash Storage Hits Tipping Point

Flash is now cheaper than most spinning disks -- and dramatically speeds up application and database performance, IBM says. What about Hana and Exadata?

Everybody agrees that spinning disk drives will give way to solid state alternatives, but there's much more debate about what share of storage will eventually move to server DRAM, what share will move to flash-based storage options and just how soon disk drives will disappear.

IBM on Thursday made the case that flash technology has reached an economic tipping point such that it's already cheaper than most spinning disks when you take into account power, cooling, floor space and software costs. IBM also argued that by adding all-flash storage arrays as an option within storage area networks -- a move that involves little more than plugging in racks -- organizations can realize dramatic improvements in both application and database performance without changes to software. Parts of these claims sound similar to claims being made by SAP and Oracle, but more on their very different approaches in a moment.

"If you slide in a flash array into an existing environment, you get a step-function change in economics and performance without a change in any system software, but that's just step one," said Ambuj Goyal, general manager of IBM System Storage & Networking. In step two you eliminate striping, caching and other storage tricks previously necessitated by slow disk access speeds to get a next level of performance improvement. And in a third step, you tweak applications to take advantage of flash-optimized databases to get even higher performance, Goyal said.

IBM made this case as part of an announcement of a new family of all-flash storage arrays, as well as a $1 billion investment in flash research and the establishment of 12 storage centers of competency around the globe that will help customers run proof-of-concept scenarios. The arrays, acquired last August with IBM's purchase of Texas Memory Systems (TMS), pack 6 terabytes to 24 terabytes of usable storage in a 1U, pizza-box-sized rack. The technology offers 74% lower power, cooling and floor space cost than spinning disks, according to IBM.

With their fast data-retrieval speeds, flash arrays are most effective in reducing I/O-bound applications. It's possible to reduce transaction times in line-of-business, ERP and analytic applications by up to 90%, according to IBM. The basic strategy is to move the hot data that becomes the biggest I/O bottleneck into flash while leaving less-active data on slower storage options.

Several customers on hand at IBM's announcement verified that the strategy works. Mobile network Sprint said its use of TMS flash arrays brought 45 times faster access speeds to customer records at its call centers without changing the Oracle database or CRM application in use. Thomson Reuters said flash arrays brought a 10X improvement in throughput and reductions in latency to its financial trading floor services. Grocery chain Kroger reported it, too, had seen a 10X improvement in database performance by using flash arrays.

Application and database giants SAP and Oracle have also had a lot to say about solid state memory performance gains in recent years. SAP is promising dramatic performance improvements by taking advantage of the DRAM in modern servers by way of its Hana in-memory database. Goyal acknowledged that DRAM is faster than flash, but he cautioned that it's also more expensive, power hungry and volatile, necessitating new high-availability and disaster-recovery procedures.

"A combination that takes advantage of memory and flash, like the DB2 BLU Acceleration announcement we made last week, will yield both economic and performance gains," he said.

As for Oracle Exadata, which relies heavily on flash, Goyal described it as a "closed, proprietary system" that necessitates the upheaval of migrating databases. By running Oracle Database in conjunction with IBM's flash arrays, Goyal said customers can improve performance, reduce the number of cores required -- cutting licensing costs -- or both.

IBM has plenty of competition in the all-flash array market, with rivals including EMC, NetApp, Hitachi, Dell and others either having or planning products in this space. Texas Memory Systems was one of the earliest to the market, shipping the technology starting in 2005. IBM said its technology stands apart in being enterprise hardened for greater durability and up to 10 times faster performance than the consumer-grade flash used by some manufacturers.

can we please get cost comparison between flash and non-flash in a typical production environment where they have one PBytes worth of data storage. Also if we get details about ROI,TCO statistics great.

Andrew - You're spot on about it being unlikely that CIOs will scale back software licenses freed up by greater efficiency. Steve Mills and Goyal talked up that prospect, but customers and even a few IBMers acknowledged that it's more likely to delay (perhaps long term) expansion of licenses that might have otherwise been required.

Doug - I think it makes sense to add flash as a storage tier as opposed to replacing all of the spinning media in one fell swoop.

For bulk storage, archives, disk to disk (and disk to disk to tape backup), rotating media makes a lot of sense. When you have an application that can actually utilize the performance increase available with flash, it's more than worth the investment.

However, I think Goyal may be dreaming a bit in thinking that using flash to run a database from may end up reducing the number of operating cores in an environment. If there's something I've learned over my career, quantities don't tend to roll back, especially when they're directly linked to processing power.

The idea is to put the 20% of data that causes 80% of the I/O bottlenecks into the flash storage layer. Customers at the IBM event -- Sprint, Thomson Reuters and Kroger -- described it as a new "tier 0" in their storage strategies, but none of them were talking about giving up conventional hard drives just yet. That they'll use for cold data, archival, etc. Most of the uses described were transactional, but Rob Bruce, a VP with government systems integrator Vion, talked about many analytic applications involving unnamed intelligence agencies "finding bad guys." These involve real-time analysis of network communications, etc., so it's not about data warehouse. Bruce says the analytics tools need to catch up to the real-time anlaysis needs.