Saturday, July 30, 2016

900TB, 4U, $60k

The impasse of Peak Hard Drive that I identified two years ago -- that era when 4TB hard drives had been the tops for three years straight -- has been breached, thanks to advances in both hard drives (HDDs) and Solid State Disks (SSDs). 10TB 3.5" HDDs can be ordered from Amazon now for less than $600, and the density of SSDs is skyrocketing as manufacturers promised two years ago. At $10,000, the 2.5" 15TB SSD is indeed dense but at $670/TB, is twice as expensive per terabyte as enterprise 1TB SSDs.

Going forward, Toshiba is predicting 128TB SSDs by 2018 and 40TB HDDs by 2020. As shown in the chart above, SSDs would then firmly breach the long-term exponential growth line in density, while HDDs would merely continue their anemic density growth.

Rack chassis are getting more dense as well. SuperMicro now has a 4U chassis that can hold 90 3.5" HDDs.

Filled with the 10TB HDDs, that would be about $60k for 900TB storage in 4U of rack space. There is no similar solution I've found for 2.5" drives, so using the same chassis for the 15TB SSDs would be $900k for 1.4PB.

But of course that ignores the compute side of the equation. Such ultra-density in a rack would only be suitable for deep-freeze storage. For a Hadoop/Spark application to leverage data locality, a 1U chassis that accommodates 8 2.5" drives and dual processors from SuperMicro would be more appropriate. Populated with the 15TB SSDs, that would be 120TB/1U, or 960TB/8U. If each CPU socket were populated with 22-core Xeons, that would be a total of 352 cores in that 8U. Total cost would be about $750k for that 8U.