Using Software RAID-1 with FreeBSD

Have you ever needed a software RAID solution for a low-end server install? Perhaps you've wanted your workstation to take advantage of the redundancy provided by a disk mirror without investing in a hardware RAID controller. Has a prior painful configuration experience turned you off software RAID altogether on Unix systems?

Since 5.3-Release, FreeBSD comes with gmirror(8), which allows you to easily configure a software RAID 1 solution. While tutorials on gmirror exist, I found them to require either manual calculations of partition sizes with bsdlabel or the use of a fix-it floppy on an existing system.

It made more sense to me to configure RAID during the install of the operating system. I also wanted a procedure that was easy to follow and didn't introduce human error in the form of a math miscalculation. After cobbling together the available documentation and experimenting my way through various configurations, I came across a procedure that has worked well for me on several different systems. I also received valuable feedback from Pawel Jakub Dawidek, the author of gmirror, who gave some insight into some of the not yet documented features of gmirror.

Some GEOM Background

Before demonstrating the configuration, it is useful to understand a bit about GEOM. GEOM is the modular disk framework introduced in FreeBSD 5.0. This modularity allows the creation of programs to manipulate disks. The best examples are the software RAID programs introduced with FreeBSD 5.3:

gstripe(8) provides a stripe set or RAID 0

gmirror(8) provides a mirror/duplex or RAID 1

graid3(8) provides a stripe with parity or RAID 3

The initial g indicates that each of these programs takes advantage of GEOM.

man 4 geom describes the terms it uses to refer to disks, some of which you'll see when setting up gmirror. These terms include the following:

provider--This GEOM entity appears in /dev. This article shows how to create a provider known as /dev/mirror/gm0, which represents the disk mirror/duplex.

consumer--This entity receives I/O requests. In the example of a mirror/duplex, it is the two physical drives. I use two IDE drives on separate cables; they are /dev/ad0 and /dev/ad2.

metadata--When referring to any RAID level, metadata includes the array members, their sizes and locations, descriptions of logical disks and partitions, and the current state of the disk array.

mirror/duplex--RAID 1 maintains the same data on two separate drives. In other words, it mirrors the data on one drive to another drive. If those two drives are attached to the same IDE cable, they are a mirror; if they are attached to separate cables, they are a duplex. Because a single cable introduces a single point of failure, most mirrors are actually duplexes.

Configuring the Mirror/Duplex During the Install

If you're going to use RAID 1, make your life easy and purchase two identical disks (of the same model and size). You can complicate things by insisting on different disks with different sizes, but in the end you just end up with a harder configuration that wastes the extra disk space on the larger disk. Cable the identical drives so that one is the primary master and the other is the secondary master. Before installing the operating system, double-check that your CMOS recognizes both disks.

Using your favorite installation method, start a FreeBSD install of any version (5.3 or higher). When you get to the Select Drives menu, it should show ad0 and ad2. Select ad0, as you will be installing the operating system on the primary master.

Within the fdisk utility, remove any existing partitions and then select "Use entire disk." When asked about the boot menu, choose "Standard MBR."

In the disklabel editor, set up the partitions on ad0 according to your requirements. If in doubt, choose a for automatic. Then choose your install sets and your install media, and let the operating system install as usual.

When finished, go through the postinstall configurations and set your time zone, create a user account, set the root password, and so on.

However, don't reboot when you end up back at the sysinstall main menu. Instead, press Alt-F4, which will take you to a command prompt. The first command I type is csh so I can get a shell with history (the default shell is Bourne).

Creating a mirror/duplex is as simple as typing:

# gmirror label -v -b round-robin gm0 /dev/ad0

where gmirror label creates the mirror; -v enables verbose mode; -b round-robin chooses a balance algorithm (at the moment, round-robin is the algorithm with the best performance); gm0 is the name of mirror/duplex (this name represents the first GEOM mirror); and /dev/ad0 represents the disk containing the data to mirror.

This is a security feature that indicates that the disk is currently mounted for writing and therefore is unavailable. However, you can get around this chicken-and-egg problem and temporarily force gmirror to bypass this measure in order to create the mirror/duplex by setting a sysctl MIB:

# sysctl kern.geom.debugflags=16
kern.geom.debugflags: 0 -> 16

Don't worry; this MIB will return to 0 when you reboot (which I'll have you do in just a few minutes). Try again:

It is, however, useful to tell the operating system to load it whenever you boot. This requires edits to two files. The first one is currently empty, so just echo over the required line:

# echo geom_mirror_load="YES" > /boot/loader.conf

However, /etc/fstab is not empty, so I recommend making a backup copy before editing it:

# cp /etc/fstab /etc/fstab.orig
# vi /etc/fstab

Change each ad to a gm, and insert a mirror after /dev. For example, /dev/ad0s1a becomes /dev/mirror/gm0s1a. Unless you've made extra partitions, you'll have ad0s1 devices ending in a, b, d, e, and f and will need to edit each of those lines.

When finished, triple-check your changes to both /etc/fstab and /boot/loader.conf. While it is fixable, it sucks not being able to boot into a new system because of a typo.

Note: some tutorials indicate you also need to add a swapoff option to /etc/rc.conf. This is no longer necessary, and neither is using shutdown -r now instead of reboot.

Once you're sure you don't have any typos, return to Alt-F1 and exit the installation menu after removing your installation media.