Using NetApp Filer Shelfs in Linux

By: Ben Rockwood (benr@cuddletech.com)Introduction:

Not everyone has all their NetApp Filers in service. Either you may have found
some NetApp Filers sitting in storage, or you may simply no longer need your Filers.
In any case, the Filer shelves are useful FC-AL arrays that shouldn't be considered
only usable when attached to a Filer head. In this article I will cover some tips
on how to use the shelves off your filers on any X86 Linux system, with some help
from QLogic and
Sistina LVM. I'll cover some procedures for
working with EVMS as well.

Getting Started:

Obviously, you'll need a Linux system with a free PCI slot, and the Filer shelfs.
You'll also need a Fibre Channel Arbitrated Loop card with copper interface, you can
either use the QLogic 2100 in the Filer head or you can pick up a QLA2100 or QLA2200
off eBay for about $100. Make sure that you also grab all the cables supplied with the
shelfs, if you are using an FC7/8 this also includes the copper terminator.

As for software, you'll need a Linux box, the distribution doesn't matter, but you
should have compilers and build tools installed. If your Linux system doesn't have
LVM (and/or Linux MD RAID) support you'll need to build a custom kernel. Building
a custom kernel is outside the scope of this article, however there is a large amount
of documentation with the kernel, and you can find help at
The Linux Documentation Project. It's recommended that
you install the following patches into your kernel:

LVM Support

Linux MD RAID Support (Optional)

EVMS (Optional)

I do not recommend that you use the QLogic 2100 driver supplied with the kernel. Instead,
you can build a module. I generally prefer that all support services (LVM, MD, EVMS) are
build into the kernel, and then modularize the FC drivers. Using FC drivers as a module
adds flexability in that if you stop, start, or add disks to your arrays you don't have to
reboot the system. The system probes and adds disks in the loop when the module is loaded.

While LVM and Linux MD is code already in the kernel base, you can
get the EMVS patches here. The kernel patches are
include in the tarball with the userland code. I recommending using the latest stable (2.4.20
at the writting of this) for your custom kernel. You can find information on how to install
the EMVS patches into the kernels codebase in the tarballs README. Once you've applied
the patches, you can finish building the userland code.

Once you have your new kernel compiled and booted, you can now add the QLogic module.
When the module is loaded, look at dmesg output to verify that the disks initialized and LIPed
and then you can see that they were added by looking at /proc/partitions or the /proc/scsi tree.

In order to use the NetApp disks with LVM you must destroy the first sector
of the disks. Understand what your doing when you do this. I've never tried
to put a disk that I've destroyed like this back into service on a NetApp Filer,
but I would imagine it'd be repaired by upgrading the firmware during a boot
of the Filer head.

Above you can see that we need to remove the partition table in order to
use it in conjunction with LVM. We can use the full SCSI device as above
or use the shortened Linux device (the /dev/sdX devices) as below.

This paper is mostly an assortment of jotted notes while I setup my enviroment,
but you can see here whats possible and the flexability that Linux provides.
Indeed you could use Solaris, AIX or any other UNIX platform you like, but
Linux provides the most power and flexability at the least cost.

Hopefully this paper can give you some ideas of how to turn your spare stock
of NetApp gear into useful storage for your office or personal needs.