Querying ZFS Storage Pool Status

The zpool list command provides a number of ways
to request information regarding pool status. The information available generally
falls into three categories: basic usage information, I/O statistics, and
health status. All three types of storage pool information are covered in
this section.

Basic ZFS Storage Pool Information

You can use the zpool list command to display basic
information about pools.

Listing Information About All Storage Pools

With no arguments, the command displays all the fields for all pools
on the system. For example:

Listing Specific Storage Pool Statistics

Specific statistics can be requested by using the
o option.
This option allows for custom reports or a quick way to list pertinent information.
For example, to list only the name and size of each pool, you use the following
syntax:

# zpool list -o name,size
NAME SIZE
tank 80.0G
dozer 1.2T

The column names correspond to the properties that are listed in Listing Information About All Storage Pools.

Scripting ZFS Storage Pool Output

The default output for the zpool list command is
designed for readability, and is not easy to use as part of a shell script.
To aid programmatic uses of the command, the
H option can
be used to suppress the column headings and separate fields by tabs, rather
than by spaces. For example, to request a simple list of all pool names on
the system:

# zpool list -Ho name
tank
dozer

Here is another example:

# zpool list -H -o name,size
tank 80.0G
dozer 1.2T

ZFS Storage Pool I/O Statistics

To request I/O statistics for a pool or specific virtual devices, use
the zpool iostat command. Similar to the iostat command,
this command can display a static snapshot of all I/O activity so far, as
well as updated statistics for every specified interval. The following statistics
are reported:

USED CAPACITY

The amount of data currently stored in the pool or device.
This figure differs from the amount of space available to actual file systems
by a small amount due to internal implementation details.

For more information about the difference between pool space and dataset
space, see ZFS Space Accounting.

AVAILABLE CAPACITY

The amount of space available in the pool or device. As with
the used statistic, this is differs from the amount of
space available to datasets by a small margin.

READ OPERATIONS

The number of read I/O operations sent to the pool or device,
including metadata requests.

WRITE OPERATIONS

The number of write I/O operations sent to the pool or device.

READ BANDWIDTH

The bandwidth of all read operations (including metadata),
expressed as units per second.

WRITE BANDWIDTH

The bandwidth of all write operations, expressed as units
per second.

Listing Pool-Wide Statistics

With no options, the zpool iostat command displays
the accumulated statistics since boot for all pools on the system. For example:

Because these statistics are cumulative since boot, bandwidth might
appear low if the pool is relatively idle. You can request a more accurate
view of current bandwidth usage by specifying an interval. For example:

In this example, the command displays usage statistics only for the
pool tank every two seconds until the you type Ctrl-C.
Alternately, you can specify an additional count parameter,
which causes the command to terminate after the specified number of iterations.
For example, zpool iostat 2 3 would print a summary every
two seconds for three iterations, for a total of six seconds. If there is
a single pool, then the statistics is displayed on consecutive lines. If more
than one pool exists, then an additional dashed line delineates each iteration
to provide visual separation.

Listing Virtual Device Statistics

In addition to pool-wide I/O statistics, the zpool iostat command
can display statistics for specific virtual devices. This command can be used
to identify abnormally slow devices, or simply to observe the distribution
of I/O generated by ZFS. To request the complete virtual device layout as
well as all I/O statistics, use the zpool iostat -v command.
For example:

Note two important things when viewing I/O statistics on a virtual device
basis.

First, space usage is only available for top-level virtual
devices. The way in which space is allocated among mirror and RAID-Z virtual
devices is particular to the implementation and not easily expressed as a
single number.

Second, the numbers might not add up exactly as you would
expect them to. In particular, operations across RAID-Z and mirrored devices
will not be exactly equal. This difference is particularly noticeable immediately
after a pool is created, as a significant amount of I/O is done directly to
the disks as part of pool creation that is not accounted for at the mirror
level. Over time, these numbers should gradually equalize, although broken,
unresponsive, or offlined devices can affect this symmetry as well.

You can use the same set of options (interval and count) when examining
virtual device statistics.

Health Status of ZFS Storage Pools

ZFS provides an integrated method of examining pool and device health.
The health of a pool is determined from the state of all its devices. This
state information is displaying by using the zpool status command.
In addition, potential pool and device failures are reported by fmd and
are displayed on the system console and the /var/adm/messages file.
This section describes how to determine pool and device health. This chapter
does not document how to repair or recover from unhealthy pools. For more
information on troubleshooting and data recovery, see Chapter 9, ZFS Troubleshooting and Data Recovery.

Each device can fall into one of the following states:

ONLINE

The device is in normal working order. While some transient
errors might still occur, the device is otherwise in working order.

DEGRADED

The virtual device has experienced failure but is still able
to function. This state is most common when a mirror or RAID-Z device has
lost one or more constituent devices. The fault tolerance of the pool might
be compromised, as a subsequent fault in another device might be unrecoverable.

FAULTED

The virtual device is completely inaccessible. This status
typically indicates total failure of the device, such that ZFS is incapable
of sending or receiving data from it. If a top-level virtual device is in
this state, then the pool is completely inaccessible.

OFFLINE

The virtual device has been explicitly taken offline by the
administrator.

UNAVAILABLE

The device or virtual device cannot be opened. In some cases,
pools with UNAVAILABLE devices appear in DEGRADED mode.
If a top-level virtual device is unavailable, then nothing in the pool can
be accessed.

The health of a pool is determined from the health of all its top-level
virtual devices. If all virtual devices are ONLINE, then
the pool is also ONLINE. If any one of the virtual devices
is DEGRADED or UNAVAILABLE, then the
pool is also DEGRADED. If a top-level virtual device is FAULTED or OFFLINE, then the pool is also FAULTED. A pool in the faulted state is completely inaccessible. No data
can be recovered until the necessary devices are attached or repaired. A pool
in the degraded state continues to run, but you might not achieve the same
level of data replication or data throughput if the pool were online.

Basic Storage Pool Health Status

The simplest way to request a quick overview of pool health status is
to use the zpool status command:

# zpool status -x
all pools are healthy

Specific pools can be examined by specifying a pool name to the command.
Any pool that is not in the ONLINE state should be investigated
for potential problems, as described in the next section.

Detailed Health Status

You can request a more detailed health summary by using the
v option.
For example:

This output displays a complete description of why the pool is in its
current state, including a readable description of the problem and a link
to a knowledge article for more information. Each knowledge article provides
up-to-date information on the best way to recover from your current problem.
Using the detailed configuration information, you should be able to determine
which device is damaged and how to repair the pool.

In the above example, the faulted device should be replaced. After the
device is replaced, use the zpool online command to bring
the device back online. For example:

The READ and WRITE columns provides
a count of I/O errors seen on the device, while the CKSUM column
provides a count of uncorrectable checksum errors that occurred on the device.
Both of these error counts likely indicate potential device failure, and some
corrective action is needed. If non-zero errors are reported for a top-level
virtual device, portions of your data might have become inaccessible. The
errors count identifies any known data errors.

In the example output above, the offlined device is not causing data
errors.

For more information about diagnosing and repairing faulted pools and
data, see Chapter 9, ZFS Troubleshooting and Data Recovery.