Is Disk Storage in Trouble?

On the surface, the future of hard disk drives (HDDs) seems secure. They account for the bulk of all storage capacity shipped and have had a strong growth curve for years. It’s a $32 billion-per-year industry, and HDDs are found in just about every office and home.

But that rosy picture hides some underlying problems. At the high end, 15k SAS and Fibre Channel disk is being squeezed out of the picture by much faster flash for systems that require near-instant response.

“A greater amount of flash was shipped last year than 15k disk, and that trend is not going to stop,” said Chris Powers, vice president of the data center development unit at HP.

At the other end of the spectrum, lower-cost tape is challenging SATA and is gaining ground in the archiving market. As a result, HDD quarterly unit shipments have been declining since 2011. Even the PC and laptop HDD market, a mainstay of the industry, is under attack as users either switch to all-flash storage or ditch these devices for a tablet or smartphone. IDC puts quarterly PC shipments at their lowest level since 2009. This is reflected in the number of HDD manufacturers, which has sunk from 62 in 1980 to only three.

“HDD revenue is down 12 percent so far this year due to the rising deployment of flash,” said Fred Moore, an analyst with Horison Information Strategies.

Moore added that mobile devices now host 28 percent of all corporate data. The average HDD utilization rate is less than 50 percent and has been that way for a long time. He also sees the emergence of fewer and larger data centers, and a growing problem of storage and IO speeds steadily falling behind compute speeds. As a result, he sees more adoption of tiering with flash at the top, followed by enterprise disk, slower high-capacity disk and tape. In addition, he believes it is the beginning of the end for RAID on large enterprise arrays as the rebuild times are getting too long.

“RAID systems have so much capacity that it just takes too many days to recover them after a failure,” said Moore.

With flash being about 50 times faster than disk on reads and HDD performance gains flattening off in recent years and not likely to improve much, he thinks the HDD is going to lose more ground.

Tom Coughlin, an analyst at Coughlin Associates, has observed a similar picture. He advocates smaller faster disks working in conjunction with tape.

He noted that disk is pinning most of its hopes on the release of PMR (coming shortly) followed by HAMR in a couple of years.

“But that probably won’t be enough to have it keep pace with performance demands,” said Coughlin.

Moore threw in some damning statistics to highlight the hard error rate disparity between disk and tape. In terms of hours to reach their hard error rate, consumer SATA sits at only one hour, while enterprise-grade SATA is at six hours. Enterprise SAS is at 55 hours, and LTO tape and enterprise-grade SSD are at 473 hours.

“HDD performance has been growing at a rate of around 5 percent compared to capacity growing at 30 percent,” said Moore.

He doesn’t see much in the way of future HDD performance bumps as access density degrades response time due to more actuator arm contention.

“Access density will continue to lower response times as HDD capacity increases,” said Moore.

One area where disk has been growing in usage at the expense of tape is in backup.

“Tape was the medium for backup in the past but disk has taken a lot of that business,” said Powers.

This came about largely through the development of deduplication technology. What we have seen over the past decade, said Moore, is disk gaining backup business from tape due to deduplication, but at the same time, disk has been losing archiving business to tape in recent years due to economics, reliability and media life concerns. And he wonders if disk can continue to sustain its gains over tape in backup due purely to economics.

“Disk with deduplication is 2X to 4X more expensive for backup,” said Moore. “When archiving, disk is 15X more expensive.”

But Jon Toigo, a storage consultant with Toigo Partners, laid doubts on the longevity of the backup market.

“Backup software sucks and it needs to be gotten out of the way,” he said. “It would be easier to write the data using the Linear Tape File System (LTFS) directly to the archive medium.”