Northwestern KB

Transferring Files to and from Quest

This page provides information on transferring files to and from Quest.

Quest supports several secure file transfer protocols and systems for transferring files to and from Quest. Use an SFTP client if you are unfamiliar with Linux systems and want to transfer files to or from a Windows or Mac computer.

Secure File Transfer Protocol (SFTP) Client

Both Mac and Windows FTP/SFTP clients are available from the
Cyberduck site. The
Cyberduck Help wiki provides basic usage instructions and feature lists, and should be reviewed before using it for the first time.

To connect to Quest, start Cyberduck and then :

Click Open Connection in the upper left of the Cyberduck window

At the top of the Open Connection window that appears, Select SFTP (SSH File Transfer Protocol) from the drop-down menu.

Enterquest.it.northwestern.edu for server specification

Enter your NetID
in the
Username: box and leave the Password: box empty to prevent your NetID password from being saved in a file on your personal computer. Public Key Authentication is not supported.

ClickConnect. You will see a Login failed window.

Enter your NetID password in the Password: field.

ClickLogin.​

Box

To move files from Box directly onto Quest:

Connect to Quest with X11 forwarding enabled. If you are using FastX to connect to Quest, or used the -X option with ssh, then you have X11 forwarding enabled.

Run Firefox by typing the following commands in the terminal window: module load firefox, followed by firefox to launch the Firefox browser.

In the Firefox browser, typenorthwestern.box.com and log in.

Select the file you wish to move onto Quest and clickDownload in the upper right corner. By default, the file will be placed into the Downloads directory in your home directory.

You can also use FTP to transfer files to and from Box. To do so, you must create an external password for Box (do NOT use your netid password for this). Find more information on the Box FTP page. From Quest, you can use the command line ftp program to connect to the Box FTP.

Secure Copy (SCP)

You can use Secure Copy to transfer individual files to or from Quest using the Linux or Mac OSX Terminal application on the computer you want to transfer files to/from. If you are using a Windows system, you will need PuTTY; Windows users substitute the command pscp for scp in the examples below.

These commands are executed from your computer, NOT from Quest (your Terminal prompt should reference your computer, not Quest).

To transfer files from Quest to your computer:
scp NetID@quest.it.northwestern.edu:/PathToSourceDir/file /PathToTargerDir/file
Example:
scp abc123@quest.it.northwestern.edu:/home/abc123/myfile.txt .
This would copy the file /home/abc123/myfile.txt from Quest to the current working directory on your computer with the same name (myfile.txt).

Wget

You can download datasets directly to Quest from external ftp and web servers using the wget command (see the Wget manual).

While connected to Quest, you could use commands like:
wget http://www.nber.org/nberces/bbg96_87.dta
wget ftp://ftp.bls.gov/pub/doc/ap.txt
To copy files into your current working directory on Quest.

With additional options wget can also download whole directories of files recursively and download from password-protected servers.

Globus

Globus is a Software-as-a-Service (SaaS) that provides a program interface for file transfer and sharing, as well as identity, profile, and group management. You can use Globus to transfer very large data sets to and from Quest.

A Globus transfer handles all the difficult aspects of data transfer by optimizing parameters to maximize bandwidth usage, managing security configurations, providing automatic fault recovery, and notifying users of completion and problems. Command line and web interfaces are available for use.

Learn more about
when to use globus to transfer files to and from Quest. Globus may also be used to transfer files to and from the Research Data Storage Services and FSMRESFILES. When transferring files to/from the Research Data Storage Service or FSMRESFILES, please avoid the use of symbolic links on Quest. This results in duplicate copies of your data on the Research Data Storage Service and FSMRESFILES. Please contact
quest-help@northwestern.edu if you would like to use this Globus.

﻿SMBCLIENT

File transfer between Quest and the
Research Data Storage Service is limited to using the non-audited storage zone, resfiles.northwestern.edu. Quest does not support analysis of regulated data (such as PHI and PII). This includes data placed in FSMRESFILES and other Research Data Storage Service audited storage zones designed for handling such data.

To transfer files, it is recommended to use Globus (see above) or smbclient via the command line. When transferring files to/from the Research Data Storage Service or FSMRESFILES, please avoid the use of symbolic links on Quest. This results in duplicate copies of your data on the Research Data Storage Service and FSMRESFILES.

To check for symbolic links in a directory, use the command:
find [path-to-directory] -type l -ls