I'm thinking of building a ZFS server optimized for random access (PostgreSQL database). I figured that I have 2 options:

Use pools consisting of the Crucial M500 960 GB SATA III SSDs. Enterprise SAS SSDs are not price-effective, considering that they could cost 10x as much as their SATA III counterparts.

Use pools consisting of 15K SAS HDDs. (In contrast, the price differential between an enterprise SAS HDD and a regular SATA III HDD is much smaller, so buying a SAS HDD is still justifiable.)

My questions are:

SATA III SSD vs 15k SAS HDD: Most of the arguments against SATA III have to do with the duty cycle (not suitable for 24/7?), but given that the MTBFs of consumer SATA III SSDs (e.g. Samsung 840 Pro, Crucial M500) are comparable to those of SAS SSDs and 15k HDDs, is there anything wrong with using the M500s? Or am I better off with the 15k SAS HDDs despite the clear performance advantages of the SSDs for random access?

L2ARC and SSD ZFS pools: If I'm using SSDs-only for my ZFS pools, would I benefit from an L2ARC device?

1 Answer
1

My recommendations vary a bit based on the distribution and approach to ZFS...

SSD-only pools work well - However, you need to think very carefully about the controller, backplane and drive layout. Main considerations are: no expanders, use pure SAS HBAs, ZFS mirrors are king.

15k SAS disks in 2014 are Meh - If you need the random I/O performance of SSDs. You can used SAS disks as another tier or even nearline drives if you need the capacity.

L2ARC won't be necessary for your all-SSD pool - Optimize and maximize RAM. There are some considerations for Linux-based ZFS solutions, but in general, more RAM is better than gobs of L2ARC, especially if you don't know your I/O pattern and working set.

Miscellaneous - Use compression, monitor and scrub your pool, use good RAM, test, buy quality SSDs... Crucial isn't on my radar. There are other high quality SATA SSDs available that will have better performance characteristics.

@elleciel Depends on working set size and distro. My recommendation for Solaris-derivatives is different than Linux... For the former, I'd use L2ARC. For the latter... I've been doing some cool stuff with ZRAM and L2ARC. A lot of things need to be set in order for ZFS to behave on Linux, so I can only give a general answer now. Also, your SSD matters. For a high-quality low-latency SAS SSD, I would be okay going to the pool most of the time.
–
ewwhiteFeb 18 '14 at 23:58

1

@elleciel Depends on the chassis, the server backplane and the disks. If you were to say Supermicro was in use for JBODs, I'd say "stick with SAS". If you were to say an HP JBOD was in use, you could do SATA. In either case, you lose SAS multipath... so there are some design considerations there, too.
–
ewwhiteFeb 19 '14 at 0:06

1

A best case is a server with 1:1 controller lane to slot mappings... So how many disks are you planning? How much capacity do you need and what hardware are you planning to use?
–
ewwhiteFeb 19 '14 at 0:08