QCIF NFS Mounting

Modified on: Tue, 8 Dec, 2015 at 8:41 AM

Mounting NFS file systems on NeCTAR instances

Introduction

This document is intended to be a starting point for a NeCTAR VM user who
wishes to gain access to files stored in an NFS file server. It does not
deal with how to set up and secure an NFS file server, or how to set up
identity management to support per-user access control.

Note that these instructions are intended to be generic. If you need to
acccess NFS servers implemented by a specific NeCTAR Node operator (for
example the QRIScloud RDSI collection servers), or virtual laboratory,
contact them for any service specific instructions.

Background

NFS (Network File System) is a protocol that allows one Linux or Unix system
to "mount" a file system that consists of collection of files and directories
that are stored on a different system.

The NFS protocol is designed to allow fast access to files over a local area
network. It supports the full range of Linux / Unix file system functionality,
and allows different systems to simultaneously access and update shared files
and directories.

An NFS file system is "exported" by a server to one or more client machines.
The clients gain access by "mounting" the NFS file system within the namespace
of the client's local file system.

Prerequisites

The following skills and privileges are required for the client-side setup.

Basic Linux system admin skills.

Root privilege / sudo access on the client NeCTAR VM.

In addition:

You need to know the IP address of the NFS server, and the "mount
path" that identifies the directory tree that you have been granted
access to. (The NFS server administrators will typically need to
enable access to your instance's IP address in the server's "exports"
file.)

The NFS server needs to be addressible by your client VM at the
IP protocol level. If the NFS server has a private IP address, then
your client typically needs a viable route to that network. (For
QRIScloud, this is achieved using the instance's second virtual NIC.)

Installing NFS client software

Before you attempt to use client-side NFS, you need to install some packages
using your system's package manager.

On a Debian, Ubuntu or similar system by running the following commands:

sudo apt-get update`
sudo apt-get install nfs-common autofs

On RedHat, CentOS or Scientific Linux, run the following:

sudo yum install nfs-utils autofs`

On (recent) Fedora, run the following:

sudo dnf install nfs-utils autofs

(The "autofs" package is needed if you want to configure the NFS client
to automount the remote file systems.)

Configuring the NFS mount

There are (at least) 3 ways to mount a file system on a remote NFS server

With a temporary mount, the mount does not survive an instance reboot.

With a permanent mount, the file system is mounted automatically on
instance startup.

With an automount, the file system is mounted when some program tries
to access files, and then unmounted after a period of inactivity.

The NFS mount string and options

The NFS mount string and mount options are the key pieces of information
that are needed when an NFS file system is mounted.

The <mount-string> specifies the remote NFS file system to be mounted
on your client VM. It consists of the NFS server's IP address, and
an exported NFS path, with a colon between them. For example:

10.255.120.220:/users/fred

The <mount-options> determine how the client-side kernel manages
NFS access. We recommend the following NFS mount options as a
starting point:

rw,hard,intr,nosuid,nodev,timeo=100,retrans=5,nolock

or the following if you need to use NFS version 3:

rw,nfsvers=3,hard,intr,nosuid,nodev,timeo=100,retrans=5,nolock

The above parameters have the following meanings:

"rw" means mount the file system with read-write access. An alternative
is "ro" for read-only access.

"hard" instructs the system to attempt to retry "for ever" to reconnect
if the client looses its NFS connection. This is strongly recommended.

"intr" is a backwards compatibility flag.

"nosuid" tells the kernel to ignore the "set user id" flag on executables.

"nodev" treats "device files" as uninterpretted.

the "timeo" value is the timeout (in 10ths of a second) for NFS
request retransmission if the client gets no response.

the "retrans" value is the number of simple retries before the client
attempts further recovery actions.

"nolock" disables file locking on the server. (If a client-side program
locks a file, the lock's coverage is limited to just this client.)

For more details on these and other NFS mount parameters, please refer to
"man 5 nfs" and other NFS documentation.

The mount point

When a Linux (or UNIX) system mounts a file system, it mounts it on top
of an existing directory. That directory is known as the mount point.
If you are doing a temporary NFS mount, or setting up a permanent mount,
the mount point directory needs to be created manually. (If you are
using the automount approach, the mount point directory can be created
automatically.)

The normal Linux convention is to either use "/mnt" or a subdirectory
of "/mnt" as the mount point. However NeCTAR VMs use "/mnt" as the mount
point for the (so-called) ephemeral file system.

You can use the "mkdir" command to create the mountpoint directory. It is
probably advisable to create it in direcftory that cannot be written by
unprivileged users.

Creating a temporary NFS mount

Before you get into the (relative) complexity of setting up a permanent
mount or an automount, it is a good idea to check that you can access
the NFS server by setting up a temporary mount. You can also use this
approach for once-off or ad-hoc access to NFS.

The steps are as follows:

Create a temporary mount point:

mkdir ~/tempMount

Run the mount command:

sudo mount -t nfs -o <mount-options> <mount-string> ~/tempMount

where the and are as described above.

Check that you can access data on the mounted file system:

$ ls -l ~/tempMount
<directory listing>

When you are done, unmount the file system and tidy up the mount point:

sudo umount ~/tempMount
rmdir ~/tempMount

Note that the "umount" will fail if any process (including as shell)
has files on the mounted file system open, or if it has a directory
in the file system as its current directory.

Configuring a fixed NFS mount

Fixed mounts are typically configured by adding entries to the "/etc/fstab"
file. This file is consulted to work out which file systems to mount on
system startup. It is also used when you run "mount -a" or "mount

".

A typical "/etc/fstab" file on a NeCTAR instance looks something like this:

Each line describes a file system, and has 6 fields separated by spaces
and tab characters.

Field 1 gives the device or other specification for the file
system to be mounted.

Field 2 gives the mount point on which the file system should be mounted.

Field 3 gives the file system type. In the example above, "ext4" is
a regular file system format, and the remainder have special meanings.

Field 4 gives any mount options relevant to the mount.

Field 5 is only relevant to the "dump(8)" program.

Field 6 determines the order in which (local) file systems are checked
by the "fsck(8)" program at boot time.

To configure a fixed NFS mount, you need to use a text editor to edit the
"/etc/fstab" file to add a line that looks like this:

<mount-string> <mount-point> nfs <mount-options> 0 0

where the <mount-string> and <mount-options> are as above, and the
<mount-point> is a directory that you have created using "mkdir"
as mentioned above.

Once you have added the entry, you should run "sudo mount "
or "sudo mount -a" to mount the NFS file system, and then check that you
can read files.

(Note: depending on how the NFS file system was exported, you
may find that local "root" account does not have special privileges on
the NFS file system; see the section below on "root squashing".)

Configuring an NFS automount

On a modern Linux distro, automounting is handled by a service called
"autofs". This service consults the "/etc/autofs.master" configuration
file to determine where the mount-points should be, and then launches
"automount" daemons to control the automatic mounting and unmounting.

The "autofs" / "automount" mechanisms are highly configurable, but we
recommend a simple configuration as a starting point. Here is a simple
recipe:

Install the "autofs" package (as above).

Edit the "/etc/autofs.master" file, and add the following line at
the end of the file:

/- file:/etc/auto.mynfsmounts

Create the "/etc/auto.mynfsmount" file, containing the following
line:

<mount-point> <mount-options> <mount-path>

where the <mount-point>, <mount-options> and <mount-path> are as
described above.

Configure the "autofs" service to start automatically on system
reboot.

sudo chkconfig --add autofs

or

sudo systemctl enable autofs

Client-side Security and Access Control

Possibly the most difficult aspect of using NFS in the NeCTAR context is
establishing who is allowed to access the files on the NFS server. The
first problem is ensuring that the client and server sides agree on user
identities.

On UNIX / Linux systems, the operating system and the file systems use
"uids" (user identifiers) to denote users. These uids are fundamentally
just numbers. The problem is ensuring that the uids are used consistently;
e.g. that 1001 means the same person, wherever it is used as a uid.

This is easy to do within a NeCTAR instance. The mapping between the
user accounts are created using "adduser", and the uid <-> account mapping
is represented by the "/etc/passwd" file.

If you are managing a cluster of NeCTAR instances, you create accounts on
one instance and push "/etc/passwd" (and the associated "/etc/shadow"
files) to the other instences. Alternatively, you can set up an LDAP
server (or similar) that your instances consult to get definitive
information about user identities.

Problems arise when you are trying to manage a group of instances, where
there isn't a shared source of user identities. In such cases, you need
to arrange that uids exposed by the NFS server are meaningful to the
NFS client(s).

Note: there is a similar mechanism for groups, with uids replaced by gids,
and "/etc/passwd" replaced by "/etc/groups".

Using root access to circumvent the problem

If a user has root access (e.g. via the "sudo" command) they can circumvent
normal file access control, and read or write other users' files. (This is
not entirely true if SELinux is used, but that is beyond the scope of this
documentation.) For example:

This kind of thing will allow you to access files on an NFS mounted file
system, provided that the file system has not been exported with
"root_squash" or "all_squash". (See below for an explanation.)

Creating access accounts and groups

One possible strategy is to figure out what uids are used on the NFS server,
and create client-side accounts that match them. This can work as a short
term solution; e.g. if users can use "sudo" to switch identities. However,
it is clunky, and if the uids conflict it can be problematic.

If you want to go down this route, then you need to identify the appropriate
subset of the uids and gids used on the NFS server, and then create
mirror accounts and groups on the NFS client instance using the "adduser"
or "useradd" command and the "groupadd" command respectively.

Use the "--uid" and "--gid" options to set the same uid and gid values
that are used on the NFS server.

If the uid and gid values are already in use on the client, you need
to consider whether the collision is going to allow one user to access
another user's files.

Mapping identities

The NFS version 4 allows uids to "name@domain" strings and back for use in
NFS requests. This can be used to deal with inconsistent uid <-> account
name mmapings. If you want / need to do this, you will need to coordinate
with the manager of the NFS server.

This deals with the case where the same user (account name) or group has
different uids and gids on the NFS server and client. However, it doesn't
deal with the case where the same account or group name is used for different
identities on the server and client. And it doesn't deal with the "root"
account

NFS security

NFS has some inherent security issues that the people responsible for
setting up the NFS server need to understand. If you are merely using the
NFS server implemented by someone else, then you have to rely on the server
administrator to address these issues. However, we mention them here
because the issues are relevant to you and the people whose data you may
be looking after.

NFS client and server host identities

By default, an NFS server relies on IP addresses as the sole means of
determining the identify of NFS clients. This is problematic if other machines
are able to spoof the IP address of a legitimate NFS client. One way to address
this (and other security concerns) is to use NFS with Kerberos.

There is also a concern that someone might spoof the IP address
of the NFS server, so that your NeCTAR instance mounts a "fake" file
system. This might sound like a strange thing to do, but consider the
case where you have put user home directories on the NFS server, and
the user's home directory holds a ".ssh/authorized_keys" file. If a
hacker can cause the client NFS to refer to a spoofed file, they can
open up a way to login to the NFS client instance.

Aside: unless there is a misconfiguration, it should be impossible for
NeCTAR OpenStack instances to spoof IP addresses of other instances.
However, it is not clear if someone outside of your data centre might
be able to spoof the IP address of an instance, and you probably would
be vulnerable to spoofing by a non-Openstack host in your data centre,
if such a host was ever compromised.

Root squashing

The default behavior of an NFS server is to "squash" the root account.
What this does is to cause the NFS server to treate any NFS requests
coming from the client using the root identity (user 0) as if they
were coming from the nobody user. This means that the root user
(or someone sudo'd to root) on the client does not have privileged
access to files on the mounted file system.

Generally speaking, root squashing is a sensible security measure,
especially if local root on the clients cannot always be trusted (see
below!). However, it does not prevent local root from assuming the
identity of some user, and accessing his / her files that way. In
fact, the only things that root squashing definitively stops are
operations like "chown" that inherently require root privilege.

Note that root squashing is controlled by the NFS export rules on the
NFS server side.

Deeper security issues

By default, NFS protocols send data, metadata and requests over the
network without any encryption. If it is possible for a 3rd party
to run network snooping (packet sniffing) software on any intervening
networks, they will be able see the files and metadata that is read or
written by your client. This is one reason why it is inadvisable to
run NFS over connections that go outside of your NeCTAR node's networking
infrastructure.

It is possible to address the problem above by using a virtual private
network (VPN). The problem is that the network traffic needs to be
encrypted, which impacts the performance of NFS running over the VPN.

A second issue is that the NFS file access control model assumes a level
of trust between people with root access to the client and server machines. If
that trust does not exist (or is ill-founded) then the end user (the notional
owner of the files) cannot rely on access controls being properly enforced.

The problem is that the group of people with root access can be larger
than you might expect. On the client side, it will include:

The person who launched the client-side instance, and who we can assume
has the private key for logging in on the service account.

Anyone who knows the root password (if one has been set) and is a member
of the tenant.

Anyone who can legitimately login to the service account, or to any other
account with sudo access.

Anyone who has NeCTAR tenant member access and can "rebuild" the instance.

Anyone who is able to hack into your instance and get root access, or
steal credentials for the tenant.

In short, if you are going to enable a NeCTAR instance to mount an NFS
file system, then the security of the NeCTAR instance (the NFS client) and
the trustworthiness of the administrators is paramount.

More information

There is more information on NFS in general and the topic of setting
up NFS mounts in the following places.