Setting up a new drive on your Linux NAS

July 9, 2016

I received some new drives recently to expand my NAS. Here I document the process of setting up a new drive, mainly for my future reference, to get everything in one place so I don’t need to look at a load of different articles to figure out what to do.

Write down the details

When I first receive a drive the first thing I do is update a spreadsheet with the details of it. This consists of:

Manufacturer

Model

Capacity

Serial number (double check this is correct!)

Warranty expiry date

A unique identifier in the format , e.g. 001wd3

The machine and bay it is mounted in

Once you’ve inserted it into your machine you need to figure out the device identifier, which is something like sdX – where X is a letter (it may also be a different format, depending on the drive interface). This probably won’t map to physical bank or slot, so you need to figure out which drive is which. In my case I’m adding two new drives which are the same size and model. The lsblk command is the first step to figuring it out:

Drive Burn In

Before putting the drive in production you should perform a series of tests to ensure that it is working. This consists of running the passive SMART tests, and active badblocks test. If the drive fails this ask for a replacement from your retailer (don’t RMA it, as you’ll get a refurbished drive).

First run a couple of quick SMART tests, you’ll need to wait for the test to finish on a drive before running the next test, but you can test multiple drives at once. After running each command it’ll tell you how long the test will take (only a few minutes), and if you try to run multiple tests at once it’ll tell you that one is in progress.

$ smartctl -t short /dev/sdb
$ smartctl -t conveyance /dev/sdb

Next up is the badblocks test. The SMART tests are passive, so they don’t usually report anything until the drive has attempted to be used. The badblocks test will write a set pattern to the whole drive, then read it back and check it is correct. This will wipe the drive, so don’t do this if you want to keep the data on it!

$ badblocks -b 4096 -c 98304 -ws /dev/sdb

The -b 4096 parameter sets the block size of the drive (which is 4096 for modern hard disks), and the -c 98304 parameter says to read 98304 blocks at once which is faster – this doesn’t affect the integrity of the test but will use more memory. In my case each instance of badblocks (I was setting up two new drives at once) used around 2GB of RAM. Note that this test will take a while, so run it in screen or tmux. In my case it took 48 hours on a 3TB drive.

Once that is complete, run a long SMART test which will take another few hours:

$ smartctl -t long /dev/sdb

Once complete, you can then check the SMART parameter of the drive.

$ smartctl -a /dev/sdb

The important bits to check are:

The test log should show no errors (“Completed without error”)

The “Current_Pending_Sector” and “Offline_Uncorrectable” parameters should be zero

Depending on who you ask the SMART parameters can mean a million things (most not important), so I’ll leave this as an exercise for the reader to figure out.

Full Disk Encryption

If you are going to setup full disk encryption now is the time. In my case I won’t be storing anything I need to keep secure (e.g. business documents) on this, but I’d like to use it as if I ever need to sell or RMA a drive I won’t have to worry about the data being read (I don’t really want some random person getting hold of my family photos). Unfortunately the CPU on my NAS is too slow, so I am not using it.

You can run the following command to test the performance of decryption operations on your CPU:

The drives I have can each read at 150MB/s – 200MB/s, so with four in the NAS, it can potentially read at 800MB/s. It only has gigabit ethernet, so in reality I’ll only be doing reads at around 125MB/s. However, housekeeping operations like scrubs will be faster with higher read speeds.

In reality I’d like to get at least 200MB/s decryption speeds for full disk encryption to be worth it for me, but as you can see the CPU can barely achieve half that (while idle). At this speed it’ll just be too slow, as the CPU is old (6 years) and doesn’t have native encryption instructions. For comparison my laptop, which has a modern Intel Core i5 processor, achieves 350MiB/s encryption and 1900MiB/s decryption.

LVM

In this case 002sg3 is the identifier I am using for this drive, and pool0 is the identifier I’m using for the device pool. By using LVM it means if there is ever an issue with a drive, the filesystem tools will show your identifier instead of the system’s identifier – so you can easily figure out what drive is broken.

In this case we aren’t really using any features of LVM, but there is basically no performance impact, and it’s worth it for future expandability (and being able to easily identify a drive when it fails!). You could use it to, for example, split a drive amongst two pools, one in RAID1 and another in RAID6.

Filesystem

Next up create or add the drive to your filesystem pool. I’m using BTRFS, so to create a new pool it’s this:

$ mkfs.btrfs /dev/002sg3/pool0

And to add it to the existing pool (there is currently a single drive, I want to turn it into RAID1), it’s this: