The following page gives a tour through a typical tara account. While it
is a standard Unix account, there are several special features to note,
including the location and intent of the different storage areas and the
availability of software. If you're having trouble with any of the
material, or believe that your account may be missing something,
contact user support.

Connecting to tara

The only node with a connection to the outside network is the user node.
Internally to the system, its full hostname is tara-fe1.rs.umbc.edu
(notice the "-fe1"), but from the outside its name is tara.rs.umbc.edu.
To log in to the system, you must use a secure shell like SSH from
Unix/Linux, PuTTY from Windows, or similar. You connect to the front end
node, which is the only node visible from the internet. For example, suppose
we're connecting to tara from the Linux machine "linux1.gl.umbc.edu".

where "username" is your UMBC username (that you use in myUMBC and
that is the base part of your UMBC e-mail address). You will be prompted
for your password when connecting; your password is your myUMBC password.

As another example, suppose we're SSHing to tara from a Windows machine
with PuTTY.
When setting up a connection, use "tara.rs.umbc.edu" as the hostname. Once you
connect, you will be prompted for your username and password, as mentioned above.

See resources for tara for a more complete list of
the available software, along with tutorials to help you get started.

Storage areas

The directory structure that DoIT will set up as part of your account creation is
designed to facilitate the work of
research groups consisting of several users and also reflects the fact that all
HPCF accounts must be sponsored by a faculty member at UMBC. This sponsor will be
referred to as PI (for principal investigator) in the following.
To give a concrete example, let us assume that you are a student with username "username", who works in the research
group of a PI with username "pi_name", who sponsors the account. Let's assume that the account for the PI has
already been created and discuss the example from the standpoint of a new account for the student "username" being
created. The following storage areas should be available in the specified locations (see
System Description for an overview).

Storage Area

Location

Description

User Home

/home/username/

This is where the user starts after logging in to tara.
Only accessible to the user by default. Default size is 100 MB,
storage is located on the management node. Backed up.

A storage area for files to be shared with the user's research group.
Ideal for working on papers or code together, for example, because it is
accessible with read and write permission to
all members of the research group and it is backed up regularly.

04/14/2010: Currently the Group Saved storage area is
being finalized by DoIT. It is not available to most users yet.

A central storage area for the user's own data,
accessible only to the user and with read permission to the PI,
but not accessible to other group members by default.
Ideal for storing output of parallel programs, for example.
Nightly snapshots of this data are kept for ten days, in case of accidental
deletion.

The same functionality and intent for use
as user workspace, except this area is accessible with read and write permission
to all members of the research group.
This area is like the group saved area, except it is larger
and not backed up.
Nightly snapshots of this data are kept for ten days, in case of accidental
deletion.

Scratch space

/scratch/NNNNN

Each compute node on the cluster has 100 GB of local /scratch storage.
This storage is convenient temporary space to use while your job is running, but
note that your files here persist only for the duration of the job. The space
in this area is shared among current users of the node. Use of this area is
encouraged over /tmp, which is also needed by critical system processes.
Note that a subdirectory NNNNN (e.g. 22704) is created for your job by the
scheduler at runtime.

For information on how to access scratch space from your job, see the
how to run page.

Tmp Space

/tmp/

Each machine on the cluster has its own local /tmp storage, as is
customary on Unix systems.
This scratch area is shared with other users, and is purged periodically by
the operating system, therefore is only suitable for temporary scratch storage.
Use of /scratch is encouraged over /tmp (see above)

AFS

/afs/umbc.edu/users/u/s/username/

Your AFS storage is conveniently available on the cluster, but can
only be accessed from the front end node. The "/u/s"
in the directory name should be replaced with the first two
letters of your username (for example user "straha1" would have directory
/afs/umbc.edu/users/s/t/straha1).

"Symlink" indicates that you will have a link created for you to the storage
area for your convenience. This link is placed in your home directory so that
you can easily navigate to your storage. For example, you may use the command
"cd ~/pi_name_user" to get to User Workspace. "Mount point"
indicates the actual location of the storage on tara's filesystem.

The amount of space available in the PI-specific areas depend
on the allocation given to your research group. Your AFS quota is determined
by DoIT. The quota for everyone's home directory is generally the same.

Some research groups have additional storage areas, or have storage organized
in a different way than shown above. For more information, contact your PI or
User Support. Also note that some users
may belong to multiple research groups, so may have access to the storage of
several PIs.

The tutorial below will walk you through your home directory, and the
specialized storage areas on tara.

A brief tour of your account

This section assumes that you already have an account, and you're a member of a
research group. If you need to apply for an account, see the
account request form. If you're not a member of a
research group, you won't have access to the various group spaces.

Home directory

First, log in to tara from your local machine by SSH:

me@mymachine:~> ssh username@tara.rs.umbc.edu
Password: (type your password)
WARNING: UNAUTHORIZED ACCESS to this computer is in violation of Criminal
Law Article section 8-606 and 7-302 of the Annotated Code of MD.
NOTICE: This system is for the use of authorized users only.
Individuals using this computer system without authority, or in
excess of their authority, are subject to having all of their
activities on this system monitored and recorded by system
personnel.
Last login: Sat Dec 5 01:39:23 2009 from hpc.rs.umbc.edu
UMBC High Performance Computing Facility http://www.umbc.edu/hpcf
--------------------------------------------------------------------------
If you have any questions or problems regarding this system, please send
mail to hpc-support@lists.umbc.edu.
Remember that the Division of Information Technology will never ask for
your password. Do NOT give out this information under any circumstances.
--------------------------------------------------------------------------
[username@tara-fe1 ~]$

The Bash shell as the default shell for tara users - this will be the shell
that you are assumed to be in for purposes of documentation and examples on
this webpage. Check your shell with the command "echo $SHELL" or by saying
"env" and searching for SHELL in the resulting lines of output.
At any given time, the directory that you are currently in is referred to
as your current working directory. Since you just logged in,
your home directory is your current working directory. The "~" symbol is
shorthand for your home directory. The program "pwd" tells you the full path
of the current working directory, so let's run pwd to see where your home
directory really is:

There is quite a bit of information on this line. If you're not sure of
what it means, this would be a good time to find a Linux/Unix reference.
One example available on the web is
The Linux Cookbook.
What we wanted to emphasize was the string of permissions. The string
"drwx------" indicates that only you have read, write, or execute access
to this directory. (For a directory, "execute" access means ability to browse
inside of it). Therefore your home is private.
The space in your home directory is limited though; you can see this by using
the "quota" command:

These numbers tell you how much space you're using and how much is available.
We limit two aspects of your storage: KB of disk space and the number of files
you can create. For each of those quantities, you have two limits: a soft limit
and a hard limit. When you reach your soft limit, you will have a little while
to reduce your usage. If you wait too long before reducing your usage, or if
you pass your hard limit, then you won't be able to make more files or enlarge
existing files until you delete enough to get under your soft limit. Your hard
and soft limits are also referred to as your quotas.

How far over my soft limit I've gone in terms of the number of files (currently 0 since I'm below my soft limit)

In your home directory, you are only allowed to create up to 10,000 files
that take up a total of 100,000 kB of storage space. That isn't much space
for high performance computing and so you should plan on using the special
storage areas that have been set up for you.

Group membership

Your account has membership in one or more Unix groups. On tara, groups are
usually (but not always) organized by research group and named after the PI.
The primary purpose of these groups is to facilitate sharing of files with
other users, through the Unix permissions system. To see your Unix groups,
try the following command:

In the example above, the user is a member of five groups - two of them
correspond to research groups.

Special storage areas

A typical account on tara will have access to several central storage
areas. These can be classified as "backed up" and "not backed up". They
can also be classified as "user" or "group" storage. See
above for the complete descriptions.
Let's look at these areas by running the following command.

We can see that these are symbolic links instead of normal directories.
Whenever you access "/home/username/pi_name_common" for example, you are
actually
redirected to "/umbc/research/pi_name/common". This has been set
up for you as a convenience, so that you can easily find your workspace.
Note that your links may point to different places than the ones shown here,
depending on your research group. If you are not a member of a research group
(e.g. a student in MATH 627), you may not have any of the group areas set up.
The "ls" listing has given us information about the symbolic links.

Group Workspace

The intention of Group Workspace is to store reasonably large volumes of
data, such as large datasets from computations, which can be accessed by
everyone in your group. By default, the permissions of Group Workspace
are set as follows to enable sharing among your group

The string "drwxrws---" indicates that the PI, who is the owner of the
group, has read, write, and execute permissions in this directory. In
addition, other members of the group also have read, write, and execute
permissions. The "s" indicates that all directories created under this
directory should inherit the same group permissions. (If this attribute
were set but execute permissions were not enabled for the group, this
would be displayed as a capital letter "S").

User Workspace

Where Group Workspace is intended as an area for collaboration, User
Workspace is intended for individual work. Again, it is intended to
store reasonably large volumes of data. Your PI and other group members
can see your work in this area, but cannot edit it.

The string "drwxr-sr-x", means that only you may make changes inside this
directory, but anyone in your group can list or read the contents. Other
users appear to also have this access, but they are restricted further
up the directory tree from accessing your PI's storage

Group Saved

Group Saved is another storage area intended for collaboration. This area
provides less space than Group Workspace, but more extensive backups. It
is appropriate for working on a paper with your research group, or for
storing your source code.

Checking disk usage vs. storage limits

There are two types of storage limits to be aware of: quotas and physical
limits of the filesystems where your space is hosted. The following
command will check the space on Group Saved, User Workspace, and Group
Workspace:

Of course your output will depend on which research group(s) you are a member.
The first column indicates where the data is physically stored. For example
the filesystem "rstor1-ib:/export/pi_name" means that the data is stored on
remote machine "rstor1-ib", in the directory "/export/pi_name".
In the last column, "/umbc/research/pi_name" indicates where this storage
is mounted on the local machine.

Notice that User Workspace and Group Workspace are sharing the same
storage, which will be the case for most users. Notice also that "/group_saved"
is not particular to our PI. In fact, all PI's with Group Saved storage
will share this space. The same thing is true for the "/home" directory,
in which the home directory for every user resides. In both cases, the
"quota" mechanism is used to partition the available space.

Earlier in the page, we demonstrated the plain "quota" command to
check your home directory. The quota for Group Saved can be seen
with the following command:

The "-g" option will display quotas associated with your group membership(s),
and the "-Q" option will suppress error messages (which are normal in this
case, but make the output harder to read). In this sample output, we can see
that our group has used 1183428 KB out of an available 10485760 KB. (Note that
this quota size is not typical of most groups).

More about permissions

Standard Unix permissions are used on tara to control which users have access
to your files. We've already seen some examples of this. It's important to emphasize
that this is the mechanism that determines the degree of sharing, and on the
other hand privacy, of your work on this system. In setting up your account, we've
taken a few steps to simplify things, assuming you use the storage areas for the
basic purposes they were designed. This should be sufficient for many users, but
you can also customize your use of the permissions system if you need additional
privacy, to share with additional users, etc.

Changing a file's permissions

For existing files and directories, you can modify permissions with the "chmod" command.
As a very basic example:

Umask

By default, your account will have a line in ~/.bashrc which sets your "umask"

umask 007

The umask is traditionally set to 022 on Unix systems, so this is a customization on tara.
The umask helps to determine the permissions for new files and directories you
create. Usually when you create a file, you don't specify what its permissions
will be. Instead some defaults are used, but they may be too liberal. For example,
suppose we created a file that got the following default permissions.

All users on the system could read this file if they had access to its directory.
The umask allows us to turn off specific permissions for newly created files.
Suppose we want all new files to have "rwx" turned off for anyone who isn't us (araim1)
or in our group (pi_nagaraj). A umask setting of "007" accomplishes this. To quickly
illustrate what this means, notice that 007 is three digit number in octal (base 8). We can
also represent it as a 9 digit binary number 000000111. We can also represent "rwxrwxr-x"
(from our file above) as a 9 digit binary number 111111101; dashes correspond to 0's and
letters correspond to 1's. The umask is applied the following way to set the new permissions
for our file

In other words, umask 007 ensures that outside users have no access to your new files.
See the Wikipedia entry for umask
for more explanation and examples.
On tara, the storage areas' permissions are already set up to enforce specific styles of
collaboration. We've selected 007 as the default umask to not prevent sharing with your
group, but to prevent sharing with outside users.
If you generally want to prevent your group from modifying your files (for example),
even in the shared storage areas, you may want to use a more restrictive umask.

If you have any need to change your umask, you can do so permanently by editing
~/.bashrc, or temporarily for the current SSH session by using the umask command directly.

Notice that typing "umask" with no arguments reports your current umask setting.

Configuring permissions of Group storage areas (PI only)

If you are a PI, you can add or remove the group write permissions (the w in
r-s/rws) by using the chmod command. You may want to do this if you intend
to place materials here for your group to read, but not for editing. To
add group write permissions and let all members of your group create
or delete files and directories in your Group Saved area:

pi_name@tara-fe1:~$ chmod g+w ~/pi_name_saved/

To remove group write permissions so that only you, the PI pi_name, can
create or delete files in your Group Saved directory:

pi_name@tara-fe1:~$ chmod g-w ~/pi_name_saved/

Of course, you may use the full path to ~/pi_name_common/ in
those commands: /group_saved/pi_name. The same commands can be applied
to the Group Workspace area to control write permissions there.

AFS Storage

Your AFS partition is the directory where your personal files are stored when
you use the DoIT computer labs or the gl.umbc.edu systems. You can access this
partition from tara. In order to access AFS, you need an AFS token.
You can see whether you currently have an AFS token

The "Tokens for afs@umbc.edu" line tells me that we currently have tokens that
let us access UMBC's AFS storage. The expiration date ("Expires Oct 25 00:16")
tells us when our tokens will expire. When your tokens expire, an empty list
will be returned

straha1@tara-fe1:~> tokens
Tokens held by the Cache Manager:
--End of list--

We can renew our tokens using the "kinit" and "aklog" commands as follows. Note
that kinit is asking for your MyUMBC password.

The mkdir command created the directory testdir. Since your current working
directory was ~ when you ran that command, testdir is inside your home
directory. Thus it is said to be a subdirectory of ~. The cd command changed
your working directory to ~/testdir and that is reflected by the new prompt:
username@tara-fe1:~/testdir$. Now lets create a file in testdir:

The echo command simply prints out its arguments ("HELLO WORLD"). The ">"
tells your shell to send the output of echo into the file testfile. Since
your current working directory is ~/testdir, testfile was created in testdir
and its full path is then ~/testdir/testfile. The program cat prints (aka
concatenates) out the contents of a file (where the argument to cat, testfile
or ~/testdir/testfile is the file to print out). As you can see, testfile does
indeed contain "HELLO WORLD". Now let's delete testdir and testfile. To use
the "rmdir" command and remove our directory, we must first ensure that it
is empty:

How to copy files to and from tara

Probably the most general way to transfer files between machines is by Secure
Copy (scp). Because some remote filesystems may be mounted to tara, it may
also be possible to transfer files using "local" file operations like cp, mv,
etc.

Method 1: Secure Copy (scp)

Th tara cluster only allows secure connection from the outside. Secure Copy is
the file copying program that is part of Secure Shell (SSH). To transfer files
in and out of the tara, you must use scp or similar secure software (such as
WinSCP or SSHFS). On Unix machines such as
Linux or MacOS X, you can execute scp from a terminal window.
Let's explain the use of scp by the following example: the user "username" has
a file hello.c in sub-directory math627/hw1 from his home directory on tara. To
copy the file to the current directory on another Unix/Linux system with scp,
use

scp username@tara.rs.umbc.edu:math627/hw1/hello.c .

Notice carefully the period "." at the end of the above sample command; it
signifies that you want the file copied to your current directory (without
changing the name of the file). You can copy data in the other direction
too (from your machine to tara). Let's say you have a file
/home/bobby-sue/myfile.m on your machine and you want to copy it to a
subdirectory matlab/ of your your tara home directory:

scp /home/bobby-sue/myfile.m username@tara.rs.umbc.edu:matlab/

The / after matlab ensures that scp will fail if the directory matlab
does not exist. If you had left out the / and matlab was not a directory
already, then scp would create a file matlab that contains all of
/home/bobby-sue/myfile.m's contents (which is clearly not what you want).
Coincidentally, that means you could also do this:

which would copy /home/bobby-sue/myfile.m to the file matlab/herfile.m
in your home directory on tara. Note that the source and destination
have different file names now.

As with SSH, you can leave out the username@, if your username is
the same on both machines. That is the case on the gl login servers
and the general lab Mac OS X and Linux machines. If you issue the command
from within UMBC, you can also abbreviate the machine name to tara.rs.
See the scp manual page for more information. You can access the scp
manual page (referred to as a "man page") on a unix machine by running
the command:

man scp

Method 2: AFS

Another way to copy data is to use the UMBC-wide AFS filesystem. The AFS
filesystem is where your UMBC GL data is stored. That includes your UMBC
email, your home directory on the gl.umbc.edu login servers and general
lab Linux and Mac OS X machines, your UMBC webpage (if you have one) and
your S: and some other drives on the general lab windows machines. Any
data you put in your AFS partition will be available on tara in the
directory /afs/umbc.edu/users/u/s/username/ where username should be
replaced with your username, and u and s should be replaced with the
first and second letters of your username, respectively. As an example,
suppose you're using a Mac OS X machine in a UMBC computer lab and
you've SSHed into tara in a terminal window. Then, in that window you
can type:

cp ~/myfile.m /afs/umbc.edu/users/u/s/username/home/

and your file myfile.m in your tara home directory will be copied to
myfile.m in your AFS home directory. Then, you can access that copy
of the file on the Mac you're using, via ~/myfile.m. Note that it's only
a copy of the file; ~/myfile.m on your Mac is not the same file as
~/myfile.m on tara. However, ~/myfile.m on your Mac is the same as
/afs/umbc.edu/users/u/s/username/home/myfile.m on both your Mac and tara.

Make sure you've noted the section on AFS tokens from earlier in
this page, if you plan on using this mount.