ABOUT NICS

Lustre Architecture

The Lustre file system is made up of an underlying set of I/O servers
called Object Storage Servers (OSSs) and disks called
Object Storage Targets (OSTs). The file metadata is controlled by a
Metadata Server (MDS) and stored on a Metadata Target (MDT).
A single Lustre file system will contain one MDS and one MDT along with multiple
OSSs and OSTs. The functions of each of these components are described in the
following list:

Object Storage Servers (OSSs) manage a small set of OSTs by
controlling I/O access and handling network requests to them. OSSs contain some
metadata about the files stored on their OSTs. They typically serve between 2
and 8 OSTs, up to 16 TB in size each.

Object Storage Targets (OSTs) are block storage devices that store
user file data. An OST may be thought of as a virtual disk, though it often
consists of several physical disks, in a RAID configuration for instance. User
file data is stored in one or more objects, with each object stored on a
separate OST. The number of objects per file is user configurable and can be
tuned to optimize performance for a given workload.

The Metadata Server (MDS) is a single service node that assigns and
tracks all of the storage locations associated with each file in order to
direct file I/O requests to the correct set of OSTs and corresponding OSSs.
Once a file is opened, the MDS is not involved with I/O to the file. This is
different from many block-based clustered file systems where the MDS controls
block allocation, eliminating it as a source of contention for file I/O.

The Metadata Target (MDT) stores metadata (such as filenames,
directories, permissions and file layout) on storage attached to an MDS.
Storing the metadata on a MDT provides an efficient division of labor between
computing and storage resources. Each file on the MDT contains the layout of
the associated data file, including the OST number and object identifier and
points to one or more objects associated with the data file.

The following figure shows the interaction among Lustre components in a
basic cluster.
The route for data movement from application process memory to disk is shown by arrows.

When a compute node needs to create or access a file, it requests the
associated storage locations from the MDS and the associated MDT. I/O
operations then occur directly with the OSSs and OSTs associated with the file,
bypassing the MDS. For read operations, file data flows from the OSTs to memory.
Each OST and MDT maps to a distinct subset of the RAID devices. The total
storage capacity of a Lustre file system is the sum of the capacities provided by the OSTs.

The current Lustre configurations on NICS systems are listed in the
following table.