On September 11, 2008, the Stanford University
School of Earth Sciences file
server,
named
sesfs.stanford.edu,
opened to general use by all faculty, students, and staff in
Earth Sciences.

This high capacity and high performance network file server
provides generous data storage capacity for individual and
research group use. Its purpose is to provide a central
respository of protected shareable disk storage that can be accessed
from any computer on the network.

Who can use sesfs.stanford.edu?

All
regular faculty, staff, and students in any of the departments or
programs of the
School of Earth Sciences
may use this server.
Basically, if the
Stanford Directory
shows that you have an official affiliation with any part of the
School of Earth Sciences, you are eligible to be a "standard" user of
this server.

Standard users get a private home share of 10 gigabytes to
store personal files. They may also access common spaces, such as the
ftp
share for
anonymous ftp service,
the
scr1
scratch share, and any
research, teaching or administrative group file shares
to which they have been granted access by the faculty or staff "owner".
If a user is maintaining content on an Earth Sciences school, department,
or research group web site on pangea, the web managers will give him/her
access to the appropriate folder in the
WWW
file share.

Although you may be eligible, access is not automatic! The system
managers need to add your SUNet ID to the list of authorized users and
create your home share.
Departments and programs normally supply lists of new graduate students
each year who need access, but don't always notify the system managers
about new faculty, staff, and post-docs.
If you think you should have access, but cannot login, contact the
system managers.

Research, teaching, or administrative group share "owners"
can request that colleagues or
students in units of the university other than Earth Sciences be given
access to their group shares or to folders they maintain on the
common shares
(ftp
and
WWW).
Those individuals will have only
the access granted by the share owner, and will not get a home share on
sesfs.stanford.edu.

Similarly, colleagues at other institutions can be given guest access to
research or administrative group shares on
sesfs.stanford.edu.
However, the share owner must first
sponsor this colleague for a full-service SUNet ID.
Due to technical limitations in the way the new server authenticates
users,
guests will need a "full service" ID that must be charged to a university
financial account; the free "base service" is not sufficient.
Such guests will have only
the access granted by the share owner, and will not get a home share on
sesfs.stanford.edu.

Access termination

This file server follows the
official policy for termination of SUNet IDs,
which gives faculty and students a 120 day grace period
after they leave the university before their SUNet ID is de-activated.
The grace period is only 3 days for staff.
But faculty and staff emeriti have permanently active SUNet IDs.

When your SUNet ID is no longer active, you can no longer
connect to the
sesfs.stanford.edu
file server.
Your home share will be retained on disk for 30 days after your
SUNet ID becomes inactive. Then it will be archived to tape and erased
from disk, unless your SUNet ID is reactivated in the meanwhile
due to re-affiliation with the university or full-service
guest sponsorship.

Disk storage allocation and policies

Sesfs.stanford.edu
has total usable storage space of 23 terabytes (~23,600 gigabytes).
This can be incrementally expanded to at least 100 terabytes as funds
are available.
In addition to free space allocations
(see below),
groups in the School with needs for multiple terabytes of storage can
"cost-share" in the purchase of additional disk arrays and benefit from
data protection features
such as hardware redundancy, professional management, and data backup.

Disk space on
sesfs.stanford.edu
is organized into units called "file shares".
This file server is based on Unix but mimics a Windows file server.
With a few exceptions, all shares employ
the Windows NTFS file
permission model, to get inheritance and fine-grained access control.
All control is by lists of
specific SUNet IDs - use of active directory groups is not supported yet.

Four types of file shares can be created on
sesfs.stanford.edu:
home shares; common shares; group shares; and system shares
(for server internal use only).
Please see the table of share names
for complete details on the names, sizes, purposes, and access
policies of all non-system shares on this server.

Home shares

Every
standard user
is given a home share that is configured as a separate file share
in order to maintain privacy of user files.
No user can access another's home share in any way.
For special needs, you can request that the
system managers
give read or write access to a particular folder in your home share to
a specific list of SUNet IDs.
But you are encouraged to use group shares and the common
/scr1
share for sharing files with others.

Each home share is preconfigured with two special folders that
should not be removed by the user.

WWW
- this folder is your
personal web site.
Any files you place in this folder are accessible on the web using the URL
http://pangea.stanford.edu/~sunetid/filename
where
sunetid
is your SUNet ID name and
filename
is the name or path of the file to be served.
The tilde character
(~)
before your SUNet ID name is required.

GRIDLab
- this folder is used to automatically store files you make when
working on the computers in the School's
G.R.I.D. research lab.
Those computers automatically connect to your home share and
put the files you save to the
Desktop
or
Documents
folders into the
GRIDLab
folder of your home share. That way, your files are accessible from
any computer in the lab or outside the lab.

Files you store on this server, particularly those served
on the web in your home share
WWW
folder, are subject to the university's
Computer and Network Usage Policy.
Among other things, illegal copying or sharing of
copyrighted materials will not be tolerated.

All users are given an initial 10 gigabyte quota for their home share.
Earth Sciences faculty members may request a larger quota for
themselves or their students in order to store academically related
files, such as thesis project files. Send the request to the
system managers.
If the same files will be used by more than one person in the research
group, it is more appropriate for the faculty member to request
a group share (below).

How to apply for research, teaching, and administrative group space

Any faculty member or lab manager in Earth Sciences may request the
creation of a file share to be used for storage of common data,
results, analyses, etc., by members of his/her research group. These
research group shares may be
linked to the cluster computers
in the School's Center for Computational Earth and Environmental Science.

Earth Sciences faculty may also request creation of a file share to
store student work or other materials for a course they are
teaching.

Desired file share name. This name may only contain letters and numerals
with no punctuation or special characters. Examples:
sac,
ulfem
and
gesadmin.

Some idea of the purposes of the file share, particularly the
quantity, size and longevity of the files that will be stored.
Will there be many small, short-lived files? Or a growing
collection of large data sets that need to be archived? This information
will help in planning data protection.

Total amount of disk space desired during the first year.
Also, describe likely long-term growth needs.
Groups can request up to 200 gigabytes of storage for free.
If larger amounts are needed, the group will be asked to contribute
a one-time fee of $1,000 for each terabyte (1,024 gigabytes) needed.
This fee is used as a cost-sharing contribution to the purchase of
additional protected disk arrays. It covers only the marginal costs of the
new disks and their maintenance.

Names and SUNet IDs of the people who will be the regular users of the file
share. You can include collaborators from other departments not in
Earth Sciences, or guests whom you have
sponsored
for a full-service SUNet ID.
I will assume that all the people you list need write access
to the entire share, unless you specify restrictions.

Whether the share will be "open" or "closed". An open share can be
accessed in read-only mode by anyone with a valid full-service SUNet ID.
This is suitable for a
repository of data to be shared to all. A closed share can only be
accessed by the persons you list as regular users (question above).

All group shares are generally made accessible via direct connection to
sesfs.stanford.edu
to mount this share as a
network file share,
and via
sftp connections to
sestransfer.stanford.edu
to copy files back and forth.
Sftp access is often useful for
off-campus connections.
You may request that one of these connection methods be disabled if
there are special security requirements.

List any special permission settings needed, such as folders
that should be accessible only to certain people within your group.
You can also specify disk quotas (limits on how much disk space
particular people can use).
Additional restrictions can be added at any time in the future -
just send email.

Describe any other special needs, such as direct access to files in
this share from the pangea web server or the
cluster computers
in the Center for Computational Earth and Environmental Science.

How to connect to file shares on sesfs.stanford.edu

Sesfs.stanford.edu
can be directly accessed using built-in features of Windows and Macintosh PCs.
Linux workstations use the Samba software package (possibly an optional
installation) to access it.

The basic idea is that you "mount" one or more "network file shares"
defined on
sesfs.stanford.edu
so they appear to be directly connected to your workstation, using
the
Common
Internet
File
System
(CIFS) protocol. Then you
can copy files to or from the server or
directly open and edit files on the server. There is no need to
copy files to the local disk to work with them.
This makes it easy to use the same files on multiple computers.
The server enforces file locking to make sure simultaneous connections
from different computers cannot make conflicting file updates.

On Windows PCs, a mounted file share appears in
Windows Explorer
as another disk drive letter. On Mac OS X, it shows up on the desktop
as a separate disk volume. On Linux, you attach the share to a
empty local directory.

You can make the connection to a file share only when needed, or you
can make an automatic connection every time you startup or login to your
local system.

To get started, you need the name of the server -
sesfs.stanford.edu -
and the name of the file share(s) you want to use from the
table of share names.
Use your SUNet ID and password to login.

Alternate connection methods for off-campus use

Due to campus-wide firewall rules, "mounting" file shares from
sesfs.stanford.edu
via the CIFS protocol
only works for computers connected to the Stanford campus network
(including Stanford DSL).
If you want to access file shares from off-campus, you can install and use the
Stanford VPN client
on your computer. That client authenticates you (using your
SUNet ID) so your off-campus network connection will
be treated by the firewall as if you were on campus.

Alternatively, you can perform file transfers to/from your home share
and most other shares from any computer on the Internet
using an sftp client program, such as the Stanford site-licensed
SecureFX program for Windows PCs,
site-licensed
Fetch program for Mac OS X,
or the built-in
sftp command line program
on Linux and Mac OS X.
These programs allow you to copy files back and forth between your
computer and a server, but you cannot edit files directly
on the server this way.

The
sesfs.stanford.edu
file server does not directly support
sftp
connections, so we have set up another server for that purpose,
which has private "back-end" connections to the file server.
Connect your
sftp
client program to
sestransfer.stanford.edu
using your SUNet ID and password.

Once connected, you will be located in your home share.
You can navigate to most other shares using their directory locations
shown in the last column of the
table of share names.

Server specifications, including data protection features

Sesfs.stanford.edu
is a dual-node ExaStore clustered network file server from
Exanet, Inc., based on IBM servers and Xyratec disk arrays.

Each server node has two quad-core Xeon processors, four
gigabit ethernet interfaces, and 16 gigabytes of error-correcting
RAM.
Each node has redundant 4 gigabit/second fiber channel links
to the RAID-6 protected disk arrays. With this level of RAID
protection, each set of six disks can tolerate the simultaneous
failure of two disks without interruption in service or loss of data
while the information on the failed disks is automatically
rebuilt onto new disks.

A clustered file system
makes all files on the arrays accessible to both nodes.
Users connect to a single network name. The server nodes cooperate
to distribute the connection load between them. If one node
fails, users who were connected to that node can immediately reconnect
to the system and will transparently access the other node.

A virtual volume interface allows thousands of separate file shares
to be created and managed, each with its own access controls.
Free space can be re-allocated among the
shares with no disruption to users or data.

In addition to the service and data protection offered by the
redundant hardware features and cluster file system,
sesfs.stanford.edu
has two features designed to maintain copies of all files in case
of accidental disk erasure or corruption:
"snapshots" and a tape backup system.

A "snapshot" is a read-only image of all files on the share "frozen"
at a specific point in time. Any changes to files (including erasure)
made after the snapshot is taken do not affect the version stored in the
snapshot. Eventually, the snapshot is removed to recover disk space.
Prior versions of files can be recovered from the snapshots.
Snapshots are made automatically by the system at pre-set intervals.
They can be accessed directly by the user to
recover a file.

Snapshots are made several times a day for
home shares and small group shares with many files that
change frequently. Some are kept only a few hours, others for two weeks,
and one per week for two months.
Research group shares with large files that change infrequently
have fewer snapshots, taken at longer intervals.

Protection against total system failure or other disasters is provided
by a separate backup system with automated tape library. Backups are
made every night of all files on
sesfs.stanford.edu
that were created or changed during the previous day. Two copies of
each backup are kept - one in the tape library, for easy access, and
the other moved daily (Monday to Friday only) to an offsite location,
for security.