Amazon EBS and NVMe

EBS volumes are exposed as NVMe block devices on Nitro-based instances. The device names are
/dev/nvme0n1, /dev/nvme1n1, and so on. The
device names that you specify in a block device mapping are renamed using NVMe device
names (/dev/nvme[0-26]n1). The block device driver can assign NVMe device names in a
different order than you specified for the volumes in the block device
mapping.

Some of these instance types also support NVMe instance store volumes. For more
information, see NVMe SSD Volumes.

Identifying the EBS Device

EBS uses single-root I/O virtualization (SR-IOV) to provide volume attachments
on Nitro-based instances using the NVMe specification. These devices rely on
standard NVMe drivers on the operating system. These drivers typically discover
attached devices by scanning the PCI bus during instance boot, and create device
nodes based on the order in which the devices respond, not on how the devices
are specified in the block device mapping. In Linux, NVMe device names follow
the pattern /dev/nvme<x>n<y>, where <x>
is the enumeration order, and, for EBS, <y> is 1. Occasionally, devices
can respond to discovery in a different order in subsequent instance starts,
which causes the device name to change.

We recommend that you use stable identifiers for your EBS volumes within your
instance, such as one of the following:

For Nitro-based instances, the block device mappings that are specified
in the Amazon EC2 console when you are attaching an EBS volume or during
AttachVolume or RunInstances API calls are
captured in the vendor-specific data field of the NVMe controller
identification. With Amazon Linux AMIs later than version 2017.09.01, we provide
a udev rule that reads this data and creates a symbolic link
to the block-device mapping.

NVMe-attached EBS volumes have the EBS volume ID set as the serial number in
the device identification.

Amazon Linux also creates a symbolic link from the device name in the block device
mapping (for example, /dev/sdf), to the NVMe device name.

Other Linux AMIs

With a kernel version of 4.2 or later, you can run the nvme
id-ctrl command as follows to map an NVMe device to a volume ID.
First, install the NVMe command line package, nvme-cli, using
the package management tools for your Linux distribution.

The following example gets the volume ID and device name. The device name
is available through the NVMe controller vendor-specific extension (bytes 384:4095
of
the controller identification):

The lsblk command lists available devices and their
mount points (if applicable). This helps you determine the correct device name to
use.
In this example, /dev/nvme0n1p1 is mounted as the root device and
/dev/nvme1n1 is attached but not mounted.

Working with NVMe EBS Volumes

If you are using Linux kernel 4.2 or later, any change you make to the
volume size of an NVMe EBS volume is automatically reflected in the instance. For
older
Linux kernels, you might need to detach and attach the EBS volume or reboot the instance
for the size change to be reflected. With Linux kernel 3.19 or later, you can use
the
hdparm command as follows to force a rescan of the NVMe
device:

[ec2-user ~]$ sudo hdparm -z /dev/nvme1n1

Before you detach an NVMe EBS volume, you should sync and unmount it. When
you detach an NVMe EBS volume, the force option is implicitly enabled. Therefore,
the
instance does not have an opportunity to flush file system caches or metadata before
detaching the volume.

I/O Operation Timeout

EBS volumes attached to Nitro-based instances use the default NVMe
driver provided by the operating system. Most operating systems specify a timeout
for
I/O operations submitted to NVMe devices. The default timeout is 30 seconds and can
be
changed using the nvme_core.io_timeout boot parameter (or the
nvme.io_timeout boot parameter for Linux kernels before version 4.6).
For testing purposes, you can also dynamically update the timeout by writing to
/sys/module/nvme_core/parameters/io_timeout using your preferred text editor.
If I/O latency exceeds the value of this parameter, the Linux NVMe driver fails the
I/O
and return an error to the filesystem or application. Depending on the I/O operation,
your filesystem or application can retry the error. In some cases, your filesystem
may
be remounted as read-only.

For an experience similar to EBS volumes attached to Xen instances, we
recommend setting this timeout to the highest value possible. For current kernels,
the
maximum is 4294967295, while for earlier kernels the maximum is 255. The
nvme.io_timeout boot parameter is already set to the maximum value for
the following Linux distributions:

Amazon Linux AMI 2017.09.01 or later

Canonical 4.4.0-1041 or later

SLES 12 SP2 (4.4 kernel) or later

RHEL 7.5 (3.10.0-862 kernel) or later

You can verify the maximum value for your Linux distribution by writing a value higher
than the suggested maximum to /sys/module/nvme_core/parameters/io_timeout and
checking for the Numerical result out of range error when attempting to save
the file.

Javascript is disabled or is unavailable in your browser.

To use the AWS Documentation, Javascript must be enabled. Please refer to your browser's
Help pages for instructions.