Globus File Transfer Services

For large data transfers between NREL’s high-performance computing (HPC) systems and
HPC Mass Storage, data centers, or even a laptop off-site, we recommend using Globus.

What Is Globus?

Globus provides services for research data management, including file transfer. It
enables you to quickly, securely and reliably move your data to and from locations you have access to.

Globus transfers files using GridFTP. GridFTP is a high-performance data transfer protocol, which is optimized for high-bandwidth
wide-area networks. It provides more reliable high-performance file transfer and
synchronization than scp or rsync. It automatically tunes parameters to maximize bandwidth
while providing automatic fault recovery and notification of completion or problems.

Get a Globus Account

Once you have a GlobusID, you will have access to the NREL endpoints via your NREL HPC user account credentials. Please submit a help ticket to HPC-Help@nrel.gov or contact us if you have any questions or issues.

If you need to move files to or from storage areas not associated with NREL HPC, please contact us to discuss options.

Authentication Methods

You will activate an NREL endpoint (nrel#globus) using your NREL HPC username and
password (not the globus.org username/password).

NREL Endpoints

These are the current NREL Globus Endpoints

nrel#globus - This endpoint will give you access to any files you have on Peregrine or Eagle:/scratch
and /projects.

Globus Personal Endpoints

You can set up a "Globus Connect Personal EndPoint", which turns your personal computer into an endpoint, by downloading and installing
the Globus Connect Personal application on your system. You may use 'Globus Connect
Personal' to transfer between your system and any NREL Globus Endpoint (e.g. nrel#globus)

Set Up a Personal EndPoint

Login to the Globus website. From the left globus menu, select Endpoints > Create new endpoint (top right).

Pick a name for your personal endpoint and select Generate Startup Key. Follow the
instructions on the web page to save your key.

Once Globus Connect Personal is installed on your system, set up the permissions for
reading or writing files to your local system. If you are using a Mac, click on the
"g" icon on the upper right portion of your screen to access the Globus Connect Personal
application. Select Preferences. To allow Globus to copy files to your local system,
make sure that the directory (folder) they will go in is Writable.

Transferring Files

Globus can be used either through the "Globus Online" service or via a CLI (command
line interface).

Globus Online is a hosted service that allows you to use a browser to transfer files
between trusted sites called "endpoints". To use it, the Globus software must be installed on the systems at both ends of
the data transfer. The following is an example for moving project data from Peregrine
(endpoint nrel#globus) to HPC Mass Storage (endpoint nrel#globus-mss.)

Click Login on the Globus website. If redirected to the Web App page, select "Use Globus ID to sign in" as the login
method and click continue. Use the Globus credentials you used to register your Globus.org
account.

Use File Manager (left menu) and search Collection for: nrel.

Select nrel#globus as the endpoint. In the box asking for authentication, enter your NREL HPC username and password. Do not use your globus.org username and password when authenticating with the nrel#globus
endpoint.

Select Transfer or Sync to... and in the new panel, search for another Globus endpoint (nrel) and select nrel#globus-mss. (To use your personal endpoint, first start the Globus Connect Personal application.
Then enter the endpoint name or your username in the Collection search bar and select
your endpoint.) After the first use, you should see your endpoints in the Recent tab.
You may also setup an endpoint/directory as a bookmark.

To transfer files

select the files you want to transfer from the nrel#globus endpoint. (Highlight as many dirs/files as you need.)

select the destination location (a folder or directory) from the dialog box on right
right.

click the large blue Start button at the bottom of the screen to begin to transfer
the files (be sure to verify the direction of transfer using the Start button from the source.)