I work as a Network Administrator in a shop where we deploy 60+ ESXi VMs on a half dozen host servers. We heard about FreeNAS from my predecessor who implemented it where he is now employed. My boss asked me to evaluate FreeNAS as it is time to replace one of our existing SANs. I installed FreeNAS 9.3 on an old IBM server and took the four 3-hour FreeNAS classes from Linda Kateley. I then dove in head first with a new 4U SuperMicro SAN with dual Intel CPUs, 128GB EEC RAM, twenty 4TB SAS HDDs and a handful of consumer grade SSDs. The important lessons learned are:

Buy big RAM chips so you have available slots if you need more RAM.

eMLC SSDs cost more but they are definitely better than off the shelf SSDs.

There is a 4X & 5X FreeNAS rule that states you want 4 or 5 times as many gigabytes of SSD capacity for cache as you have gigabytes of RAM. I always go big so we deployed the 8X rule.

Download FreeNAS and install it on a test platform and then take Linda Kateley’s four paid and one free interactive online class to get started. The FreeNAS forum is wonderful as long as you research your questions before posting.

RAID cards are not needed but if you want an LSI RAID card make sure you flash it to IT (pass through mode). LSI support & the FreeNAS forum were great in guiding me to the correct utilities I needed for my LSI SAS 9211-8i Host Bus Adapter.

If you want the lights to work correctly on the front of a SuperMicro SAN use only SAS HDDs.

A very knowledgeable admin on the FreeNAS forum sent me a quote that is quite appropriate: “It’s like a learning cliff. We know.” That said, Linda’s class made it much more like a steep hill and very manageable.
Currently I have the fastest SAN I have ever had the pleasure of working with and all the hard work pays off when you see you have maxed out your state of the art fiber gigabit network.

I started using FreeNAS a couple of years ago at work when we started looking into backup storage solutions for our various (pricey) production arrays. I first experimented with 5-6 year old spare desktop computers of the Intel Core 2 CPU/DDR2 memory vintage and whatever hard drives that we had laying around. We were so impressed with FreeNAS’s versatility (CIFS, AFP, iSCSI) and performance that we decided to purchase enterprise class hardware for a custom built production backup storage solution using FreeNAS. It ended up costing about $10K for 40 TB initially and we have since added another 40 TB shelf to the same server as our storage needs have grown.

I have also been able to upgrade the hardware I use for my FreeNAS/VMware test lab and it is about this that I will give a more detailed account. I have two FreeNAS boxes, 1 to host VMware datastores over iSCSI and the other to receive backups for the first system and for CIFS and AFP (time machine) shares.

System 1

Motherboard – Asus P8Z68-V LE

CPU – Intel Core i5 2500K 3.30 GHz quad core

Memory – 32 GB G.SKILL Ripjaws X Series DDR3 1600 PC3 12800 (4 x 8GB)

Hard Drives – 6 x 600GB WD VelociRaptor

iSCSI NIC – 4-port gigabit Intel E1G44HT PCI-Express 2.0 I340-T4

OS drive – SanDisk Cruzer Fit 8 GB

Build – FreeNAS-9.3-STABLE-201412312006

Volume – RAIDZ2 2.2 TB available

System 2

Motherboard – Asus P7P55D-E LX

CPU – Intel Core i5 760 2.80 GHz quad core

Memory – 16 GB G.SKILL Ripjaws X Series DDR3 1600 PC3 12800 (2 x 8GB)

Hard Drives – 8 x 2 TB Hitachi Deskstar

OS drive – SanDisk Cruzer Fit 8 GB

Build – FreeNAS-9.3-STABLE-201412312006

Volume – RAIDZ2 10 TB available

Performance

I know that iXsystems doesn’t like to post performance numbers of their TrueNAS systems, but I’m going to post my numbers. I do it mainly as a point of comparison because I recently swapped out the 4-port gigabit NIC for a 10 Gbps Intel X540-T1. Here are the CrystalDiskMark numbers inside a Windows guest on an ESXi 5.1 iSCSI datastore:

(MB/s – IOPS)

4 x 1 Gbps

10 Gbps

Read Seq

103

538

Write Seq

100

864

Read 512K

94

576

Write 512K

92

760

Read 4K

10 – 2,352

14 – 3,340

Write 4K

14 – 2,683

30 –7,226

Read 4K QD32

111 – 27,184

282 – 68,823

Write 4K QD32

88 – 21.678

394 – 96,097

There’s a 1 Gbps bottleneck and it must be because the ESXi free license doesn’t have MPIO support. Going to 10 Gbps simplifies networking and has a huge performance increase; I highly recommend it.

Conclusion

FreeNAS is awesome!!! The more I use it, the more confident I am in its stability, performance, and ability to meet all my needs. I’m looking forward to FreeNAS 10 and you can be sure I’ll upgrade.

Last week, FreeNAS developers from around the globe arrived for 2 weeks of face-to-face discussions and coding sessions for FreeNAS 10. Yes, Virginia, there will be a FreeNAS 10, divided into a series of planned milestone releases (M2-M5) for those who would like to assist in testing and development, or merely just to follow along as we progress

Hackathon participants came from Australia, Brazil, the Czech Republic, Poland, the Ukraine, several US states, as well as those local to the Bay area. The non-locals stayed at a historic ranch in the Livermore hills with enough seclusion, Internet access, and food that time could be spent coding, doc’ing, and ramping up on tools, architecture, and design elements. In case you ever wondered what it takes to fuel a hackathon, Jordan has a good idea regarding how many shopping carts are needed:

Days were spent using a whiteboard, reviewing design mockups, and nailing down timelines, blockers, and assigning features. Over the next few weeks, we’ll have more blog posts detailing the various milestones, how you can check out the developers’ progress, where you can get more information about the new architecture, where you can get developer previews in order to test new features and help find bugs, and how you can setup a development environment and contribute to the development process.

Recent Comments

Yep, we have split FreeNAS 10 into multiple components parts largely to make it MUCH easier to hack on. You can now hack on the GUI straight from your Unix system or Mac, in fact, without even *having* a FreeNAS box

It seems like a lot of folks are unaware that you can still create a UFS filesystem (newfs is not going away at the command-line) and, for that matter, can create a ZFS pool on a single disk if you just want to be able to import that disk as a pool, copy data to it, and then export it again as part of your backup strategy?