18.3.Â RAID1 - Mirroring

RAID1, or
mirroring, is the technique of writing
the same data to more than one disk drive. Mirrors are usually
used to guard against data loss due to drive failure. Each
drive in a mirror contains an identical copy of the data. When
an individual drive fails, the mirror continues to work,
providing data from the drives that are still functioning. The
computer keeps running, and the administrator has time to
replace the failed drive without user interruption.

Two common situations are illustrated in these examples.
The first creates a mirror out of two new drives and uses it as
a replacement for an existing single drive. The second example
creates a mirror on a single new drive, copies the old drive's
data to it, then inserts the old drive into the mirror. While
this procedure is slightly more complicated, it only requires
one new drive.

Traditionally, the two drives in a mirror are identical in
model and capacity, but gmirror(8) does not require that.
Mirrors created with dissimilar drives will have a capacity
equal to that of the smallest drive in the mirror. Extra space
on larger drives will be unused. Drives inserted into the
mirror later must have at least as much capacity as the smallest
drive already in the mirror.

Warning:

The mirroring procedures shown here are non-destructive,
but as with any major disk operation, make a full backup
first.

Warning:

While dump(8) is used in these procedures
to copy file systems, it does not work on file systems with
soft updates journaling. See tunefs(8) for information
on detecting and disabling soft updates journaling.

18.3.1.Â Metadata Issues

Many disk systems store metadata at the end of each disk.
Old metadata should be erased before reusing the disk for a
mirror. Most problems are caused by two particular types of
leftover metadata: GPT partition tables and
old metadata from a previous mirror.

GPT metadata can be erased with
gpart(8). This example erases both primary and backup
GPT partition tables from disk
ada8:

#gpart destroy -F ada8

A disk can be removed from an active mirror and the
metadata erased in one step using gmirror(8). Here, the
example disk ada8 is removed from the
active mirror gm4:

#gmirror remove gm4 ada8

If the mirror is not running, but old mirror metadata is
still on the disk, use gmirror clear to
remove it:

#gmirror clear ada8

gmirror(8) stores one block of metadata at the end of
the disk. Because GPT partition schemes
also store metadata at the end of the disk, mirroring entire
GPT disks with gmirror(8) is not
recommended. MBR partitioning is used here
because it only stores a partition table at the start of the
disk and does not conflict with the mirror metadata.

18.3.2.Â Creating a Mirror with Two New Disks

In this example, FreeBSD has already been installed on a
single disk, ada0. Two new disks,
ada1 and ada2, have
been connected to the system. A new mirror will be created on
these two disks and used to replace the old single
disk.

The geom_mirror.ko kernel module must
either be built into the kernel or loaded at boot- or
run-time. Manually load the kernel module now:

#gmirror load

Create the mirror with the two new drives:

#gmirror label -v gm0 /dev/ada1 /dev/ada2

gm0 is a user-chosen device name
assigned to the new mirror. After the mirror has been
started, this device name appears in
/dev/mirror/.

MBR and
bsdlabel partition tables can now
be created on the mirror with gpart(8). This example
uses a traditional file system layout, with partitions for
/, swap, /var,
/tmp, and /usr. A
single / and a swap partition
will also work.

Partitions on the mirror do not have to be the same size
as those on the existing disk, but they must be large enough
to hold all the data already present on
ada0.

If the geom_mirror.ko kernel module
has not been built into the kernel,
/mnt/boot/loader.conf is edited to load
the module at boot:

geom_mirror_load="YES"

Reboot the system to test the new mirror and verify that
all data has been copied. The BIOS will
see the mirror as two individual drives rather than a mirror.
Because the drives are identical, it does not matter which is
selected to boot.

See SectionÂ 18.3.4, “Troubleshooting” if there are
problems booting. Powering down and disconnecting the
original ada0 disk will allow it to be
kept as an offline backup.

In use, the mirror will behave just like the original
single drive.

18.3.3.Â Creating a Mirror with an Existing Drive

In this example, FreeBSD has already been installed on a
single disk, ada0. A new disk,
ada1, has been connected to the system.
A one-disk mirror will be created on the new disk, the
existing system copied onto it, and then the old disk will be
inserted into the mirror. This slightly complex procedure is
required because gmirror needs to put a
512-byte block of metadata at the end of each disk, and the
existing ada0 has usually had all of its
space already allocated.

Create a mirror on the new disk. To make certain that the
mirror capacity is not any larger than the original
ada0 drive, gnop(8) is used to
create a fake drive of the exact same size. This drive does
not store any data, but is used only to limit the size of the
mirror. When gmirror(8) creates the mirror, it will
restrict the capacity to the size of
gzero.nop, even if the new
ada1 drive has more space. Note that the
1000204821504 in the second line is
equal to ada0's media size as shown by
diskinfo above.

Since gzero.nop does not store any
data, the mirror does not see it as connected. The mirror is
told to “forget” unconnected components, removing
references to gzero.nop. The result is a
mirror device containing only a single disk,
ada1.

After creating gm0, view the
partition table on ada0. This output is
from a 1Â TB drive. If there is some unallocated space at
the end of the drive, the contents may be copied directly from
ada0 to the new mirror.

However, if the output shows that all of the space on the
disk is allocated, as in the following listing, there is no
space available for the 512-byte mirror metadata at the end of
the disk.

If no free space is shown at the end of the disk, the size
of both the slice and the last partition must be reduced by
one sector. Edit the two files, reducing the size of both the
slice and last partition by one. These are the last numbers
in each listing.

Restart the system, booting from
ada1. If everything is working, the
system will boot from mirror/gm0, which
now contains the same data as ada0 had
previously. See SectionÂ 18.3.4, “Troubleshooting” if
there are problems booting.

At this point, the mirror still consists of only the
single ada1 disk.

After booting from mirror/gm0
successfully, the final step is inserting
ada0 into the mirror.

Important:

When ada0 is inserted into the
mirror, its former contents will be overwritten by data from
the mirror. Make certain that
mirror/gm0 has the same contents as
ada0 before adding
ada0 to the mirror. If the contents
previously copied by dump(8) and restore(8) are
not identical to what was on ada0,
revert /etc/fstab to mount the file
systems on ada0, reboot, and start the
whole procedure again.

Forgetting to load the geom_mirror.ko
module in /boot/loader.conf can cause
this problem. To fix it, boot from a FreeBSD
installation media and choose Shell at the
first prompt. Then load the mirror module and mount the
mirror device:

#gmirror load#mount /dev/mirror/gm0s1a /mnt

Edit /mnt/boot/loader.conf, adding a
line to load the mirror module:

geom_mirror_load="YES"

Save the file and reboot.

Other problems that cause error 19
require more effort to fix. Although the system should boot
from ada0, another prompt to select a
shell will appear if /etc/fstab is
incorrect. Enter ufs:/dev/ada0s1a at the
boot loader prompt and press Enter. Undo the
edits in /etc/fstab then mount the file
systems from the original disk (ada0)
instead of the mirror. Reboot the system and try the
procedure again.

Enter full pathname of shell or RETURN for /bin/sh:
#cp /etc/fstab.orig /etc/fstab#reboot

18.3.5.Â Recovering from Disk Failure

The benefit of disk mirroring is that an individual disk
can fail without causing the mirror to lose any data. In the
above example, if ada0 fails, the mirror
will continue to work, providing data from the remaining
working drive, ada1.

To replace the failed drive, shut down the system and
physically replace the failed drive with a new drive of equal
or greater capacity. Manufacturers use somewhat arbitrary
values when rating drives in gigabytes, and the only way to
really be sure is to compare the total count of sectors shown
by diskinfo -v. A drive with larger
capacity than the mirror will work, although the extra space
on the new drive will not be used.

After the computer is powered back up, the mirror will be
running in a “degraded” mode with only one drive.
The mirror is told to forget drives that are not currently
connected:

#gmirror forget gm0

Any old metadata should be cleared from the replacement
disk using the instructions in
SectionÂ 18.3.1, “Metadata Issues”. Then the replacement
disk, ada4 for this example, is inserted
into the mirror:

#gmirror insert gm0 /dev/ada4

Resynchronization begins when the new drive is inserted
into the mirror. This process of copying mirror data to a new
drive can take a while. Performance of the mirror will be
greatly reduced during the copy, so inserting new drives is
best done when there is low demand on the computer.

Progress can be monitored with gmirror
status, which shows drives that are being
synchronized and the percentage of completion. During
resynchronization, the status will be
DEGRADED, changing to
COMPLETE when the process is
finished.