In this sample chapter you'll obtain a taste of the complexities of data storage, and the variety of software and devices that comprise it. Also learn about the wide variety of connection protocols and configurations that go into assembling effective enterprise-class storage solutions.

Perhaps the most difficult part of writing this book was to decide how much
information to include about storage networks and storage techniques in general.
On the one hand, the Federated Management Architecture (FMA) and Jiro can be
applied to virtually any management solution. On the other hand, FMA was
originally built with a direct focus on storage, so many of the architectural
decisions can be justified if this is obvious from the start.

The basis for much of the content of this book is the concept of a storage
network. The partitioning of storage data, management, and operations from an
overall production network into a dedicated storage network is a relatively new
trend and a quickly evolving field of study. There are many different reasons to
separate storage traffic from a production network:

You avoid having users overwhelm a network and cut off storage traffic,
or vice versa.

You allow the storage network to be optimized for particular quality of
service (QoS) attributes that may differ from quality of service parameters
required in a production network.

You prevent confusion between storage management and network management,
two tasks that have largely different concerns and needs.

You allow the storage network to use a network protocol optimized for
storage access and data movement.

There are more reasons to maintain a split between a production network and a
storage network, but much more detail can be derived from one of the many books
on storage networks that are available. In addition to storage networks, FMA and
Jiro must be able to manage storage that is available on a production network in
two other forms.

Direct attach storage, which is attached directly to a host's
bus. A typical example of this is the hard drive in your personal computer or
server.

Network attach storage (NAS), which is a class of systems that
provide file services to host computers. A host system that uses NAS uses a file
system device driver to access data using file access protocols such as the
Network File System (NFS) or the Common Internet File System (CIFS). NAS systems
interpret these commands and perform the internal file and device I/O operations
necessary to execute them.1

To manage storage as a whole, a person first thinks of the hardware that is
required for storage management: routers, switches, disk devices, tape devices,
and more.2 What people sometimes forget is the
wide variety of software that goes into day-to-day storage management. Storage
management of any kind could not be achieved without the software. Software
components for managing storage include the following:

Device drivers: layers of code on hosts that translate operating system
requests to device requests.

As an enterprise or medium-sized business grows, more storage is required.
Furthermore, as businesses become distributed or embrace the Web, the amount of
time that storage must remain online increases. For many businesses, it is
essential for storage to remain online 24 3 7 3 365. The availability
requirement alone is a primary driver for storage networks. It is difficult to
replace a hard drive that is directly attached to a host without bringing the
host down during installation.

According to research by IDC, production storage between 1999 and 2004 is on
course to grow by 10,000 petabytesthat is, 10,000,000,000,000,000,000
bytes of information. Accompanying this increase in storage will be an increase
in storage management costs, and all this is coupled with a tight worker market.
This combination spells trouble for end users. Storage administrators and
companies with storage issues will attempt to solve problems in a variety of
ways:

Flexibility. The main goal of flexibility is to predict future storage
network requirements early to decrease the impact and maintenance when growth is
needed. An example is to outsource a large part of the storage networking needs
to a company that specializes in this storage network, such as a storage service
provider (SSP). The biggest single issue with an SSP is trust: Does your company
trust your data to be sent offsite to another company? There are other ways to
increase storage flexibility, including redesigning the existing storage network
in a modular and expandable fashion.

Time balancing. Who is impacted, and what is the company tolerance for
using and paying for time? For example, acknowledging that you cannot afford to
reengineer the storage network or hire additional resources means that you will
impact your employees and customers due to maintenance time as your storage
needs increase. Furthermore, the company will not be able to take advantage of
new storage opportunities that could create more efficient use of time. The
company could also choose to substantially increase the amount of time spent
dealing with storage networking. This approach acknowledges the value of the
employee and customer information, but if the company lacks the ability to be
flexible, time invested in the network will increase linearly (or exponentially)
with the amount of storage added.

Resources. Adding administrators to address storage networking needs
increases the total cost of ownership (TCO) but does not necessarily increase
the efficiency of the storage network. Resources can be acquired in the form of
onsite storage networking consultants who are dedicated to the maintenance of
your systems. To some degree, the issue of trust is relieved with this option,
although it does require higher capital expenditure.

No matter how a business chooses to address its ever-increasing storage
needsprobably through a combination of these approachesthere is
another variable that can aid in creating an effective storage management plan:
storage management software. The simple facts are that stored information is
increasing exponentially, and it is unlikely that the number of storage
management professionals will increase exponentially during the same time. The
only answer to this dilemma is to create effective storage management tools that
allow storage management experts, whether they are onsite or hired hands, to
more effectively manage increased storage without increasing the number of
experts or their training time.

A tool that proactively monitors your storage network and asks for help only
when necessary is sometimes referred to as the Holy
Grail3of storage management. In many cases,
this level of management can be achieved if you are willing to build storage
networks with products from a single vendor. By choosing a single vendor
solution, however, you are tied into its pricing and support mechanisms, forcing
you to trust a single vendor with your data and your budget.

The truth is that the storage industry suffers from commodity pricing. By
allowing businesses to choose a quality of service level and a corresponding
price point for the quality of service, the industry enables businesses to grow
their network without bounds and based on their own constraints of budgeting
vs.QoS. The problem today with heterogeneous storage networks is that each
vendor of a component within the storage network often uses its own management
techniques.

From the point of view of the storage manager, we are back to the first
problemincreasing the amount of storage increases the number of storage
management issues that must be dealt with. For example, by purchasing two
fibre4 channel switches from two different
companies, you require your storage management experts to understand two
management consoles.5

The Federated Management Architecture from Sun is meant to bring
heterogeneous environments back down to a single point of control. Furthermore,
the architecture dictates policy-based solutions that can grow unbounded with a
storage management network.

This chapter discusses the nuts and bolts of data centers, including
management techniques and protocols as well as the hardware and software
involved in a storage solution. After discussing storage and storage management,
we explain how FMA and Jiro fit in to the storage management picture.

The important thing to take from this chapter is not necessarily an
understanding of heterogeneous storage networks vs. homogeneous storage
networks, or one type of hardware vs. another type of hardware. The essential
information is simply that all these types of hardware and software exist. They
all must be managed, no matter who is doing the management for you. Your goal
should be to try to understand how a device ends up being managed by software,
and how software itself also requires management from a policy-based
solution.

2.1 Storage Hardware

Beyond host computer systems, there are two primary categories of hardware to
consider. In general, there are the physical devices that store data and the
network support that helps move the data to and from the correct locations. Both
categories contain many different kinds of devices. A few of the devices in each
category are profiled here.

Each type of device and configuration has trade-offs. For example, the
managed fibre channel switch profiled later seems like a perfect device for
network management. The drawbacks of a switch versus an average low-cost hub are
that switches involve propagation delay and tend to be expensive.

On the other hand, low-price hubs give no indication of trouble in a network,
can be difficult to manage, and share bandwidth between all attached devices
(switches can allocate all bandwidth to multiple zones). These limitations have
a direct impact on the ability of a storage administrator and storage management
software to detect problems in the storage network.

Again, you should devote thought to each storage network before spending the
company's budget. Even within a single data center, a wide variety of
hardware devices can be employed to fit the characteristics and QoS of a
particular department or area.

2.1.1 Disk Devices

If you are coming from a PC-centric background, when you think of storage,
you think of the drives that are attached to the bus in your system. This
isn't far from the truth of implementation for many large installations.
Host file servers often contain direct attach storage, which is physically
contained within a host. The host then shares these disks through a network file
protocol such as NFS or CIFS. To expand storage, the system administrator brings
down the host, adds a drive to the server tower, configures it, and shares
it.

In large data centers, storage is more partitioned than in the physical
containment model used in hosts. There are many reasons for this partitioning.
One is that mainframes have traditionally been very good at separating storage
from the systems. Another reason is simply that large data centers have
encountered problems with the old model and have already started to partition
into storage networks as their solution. Physical drives fit into rack-mounted
cabinets that are 19 inches wide and of variable height depending on the
contents of the rack-mounted equipment.

Redundant arrays of independent disks (RAID) hardware enables
high-performance data retrieval and high availability of data through the use of
multiple disks. Basically, to enable high performance, data is spread over
multiple disks to allow parallel reads and writes to the disks. By having more
disk arms moving, you relieve a major performance bottleneck: the disk arm. To
enable high availability, data is striped across disks, and then parity bits are
used to enable recovery of lost data. In the basic RAID levels, parity is used
to enable recovery of one lost disk in the disk array. So if four disks are
being used and one crashes, the crashed disk can be replaced and the data
retrieved from the parity bits.

RAID levels, 0 to 5, give different levels of redundancy or performance.
Advanced RAID techniques combine the RAID levels to try to give performance and
high availability. The basic RAID levels are

Level 0: striping

Level 1: mirrors

Level 3: dedicated parity disk

Level 4: parallel access with parity disk

Level 5: parallel access with distribution parity

Combining some of the RAID levels makes implementations more expensive (in
terms of hardware and possibly performance), but it creates benefits that
combine the best of both techniques. For example, RAID level 0 combined with
level 1 can give fast read and write access as well as good data redundancy.

RAID devices are put in the hardware section, but the location of the RAID
implementation varies widely. RAID can be implemented in three places:

Onboard a physical disk array

In a controller card residing in a server system

In software, such as a logical volume manager

Where you implement RAID capabilities affects both the cost and the
effectiveness of the implementation. For example, using software RAID
implementations may be inexpensive, but it creates a burden on the host that
implements the RAID capabilities. The software is burdened with manipulating the
distribution of data across physical devices. This robs memory and valuable
processor cycles from the file-serving processes. The result is that increased
traffic to the host increases the demands on the file-sharing software as well
as on the software RAID controller, a double hit to the server at a time when
you would prefer to lighten the load on the processor to aid in processing
requests. To relieve the host, RAID implementation can be moved to controller
cards or onto the disk arrays themselves. Typically, this locks the RAID
implementation into a single vendor, but it can create a very effective
implementation. The decision of where to implement RAID in a storage network is
an important one.

Just a bunch of disks, more widely known as JBOD devices, are low-cost
devices that contain . . . a bunch of disks. There are many different ways to
configure the disks. Typically, the JBOD is in a rack enclosure, and you
hot-swap drives in and out of the JBOD. Whereas literal RAID device has the RAID
capabilities onboard the device, if you want to use some or all of the disks
available in the JBOD for RAID configurations, it must be controlled by software
or an external RAID controller.

Network attach storage on the low end fits into the category of disk devices.
Devices fit into several price groups. On the high end of NAS price points, NAS
involves a rack-mounted system that attaches to an IP network. The high-end
device typically contains one or more disk drives that can be configured in
various RAID configurations. In the low-end price range, you will likely find
software-based RAID, limited management capabilities, and very limited backup
capabilities. Furthermore, on the low end, stand-alone devices are available
that can sit on desktops or even in the home. Onboard any NAS device is what
could be termed a specialized operating system that is optimized for file
serving. In this operating system many of the general functions of the kernel
and operating system are removed, such as any graphics capabilities, extraneous
port handling drivers (for USB or parallel devices), and other optimizations
that can be found for the specific device. The file system, volume management,
and security are all built into the operating system and services that are
hosted on the NAS device. Plug in the NAS, and you have instant space available
via CIFS or NFS attachable directories.

Higher-priced NAS devices contain a huge amount of functionality. They
contain everything from built-in tape libraries for archiving and backup to
custom file systems built for network sharing of data.

2.1.2 Tape Devices

There are essentially three types of tape storage enclosures that systems can
use:

Single tape drive. Targeted at user data backup, single tape drives often
exist on servers or single-user computers that contain important data.

Tape autoloader. This device loads tapes automatically and contains a
single read/write head. This is really a degenerate case of a tape library
(discussed next).

Tape library. Much larger than a tape autoloader, this device often
contains multiple read/write heads.

For management purposes, the physical devices are important, but much of the
data management will be accomplished through backup/archive manager or
hierarchical storage manager (HSM) software, both of them covered later in this
chapter.

2.1.3 Storage Networking Hardware

A variety of devices make up the category of what can be considered
storage networking hardware. Later in this chapter we talk more about
what it means to create a storage network, but the devices that fall into this
category are similar to traditional networking hardware. Hubs, routers, and
switches are combined to make up a network infrastructure. Each device has
different capabilities as far as network management is concerned, and each is
used in a different way.

Hubs. These devices provide a low-cost, easily installable way to expand
a storage network. Hubs have two major drawbacks. One is that they tend to be
less "manageable" than switches. The second is that bandwidth is
shared between all the devices on the hub. A switch has the ability to partition
devices and maintain full bandwidth to each partition of devices, even in a
degenerate configuration in which each attached device is in its own zone. In
this degenerate case, each attached device has full bandwidth. This
configuration is not possible with hubs.

Switches. Like hubs, switches allow network expansion. The difference is
that the switches have more management capabilities, more configuration options,
and typically have some ability to debug and maintain performance in fibre channel
network. The switch forms
the center point of what is known as a fabric. The switch can route data between
ports of any two devices that are connected to the fabric. You can also create
logical partitions of the fabric, known as zones, which give full throughput to
all logical partitions. Finally, a switch is often able to detect a misbehaving
component and eliminate it from the fabric without impacting the remaining
devices. The downside of switches is that they tend to be much more expensive
than hubs and can introduce a small amount of propagation delay. Expensive hubs
and inexpensive switches can overlap in capabilities. Furthermore, in the future
it is likely that low-end hubs will actually become low-end switches as
components used in switches hit lower and lower price points.

Routers. Used for routing network traffic, routers let you add a variety
of features to make them an integral part of a storage network. For example,
some routers can convert fibre channel protocol traffic to parallel SCSI
traffic, allowing you to attach legacy SCSI devices, such as tape libraries, to
a fibre channel network.

In some cases, switches and hubs can be used interchangeably. Switches are
more manageable than hubs but incur some propagation delay depending on their
zoning options. On the other hand, a switch will remove a misbehaving device
from a storage network automatically and will often signal the administrator in
multiple ways, perhaps through a nice red LED.

In addition to the devices that form a network infrastructure, controller
cards attach devices to the physical network. Sometimes these are termed host
bus adapters, or HBAs. If you have multiple HBAs installed on a host, one HBA
can fail while a storage network connection continues to be available. HBAs are
similar to a network interface card (NIC).

The hubs, switches, and routers discussed in this section come in two forms:
one for fibre channel networks and one for IP networks. A quickly advancing
standard known as SCSI over IP moves the most popular storage protocol, SCSI, to
an IP network. With the advent of SCSI over IP, similar management tools and
hardware can be used to manage both the client network and the storage network.
Increasing the capabilities of the management tools for these networks and
creating one set of hardware for a complete network (storage and production)
will lower the total cost of ownership for storage networks.