Using Jupyter Notebooks on Bridges

This document explains how to run a Jupyter notebook on Bridges using a Jupyter client on your local machine. You can also run Jupyter on Bridges through the OnDemand web interface. See the OnDemand section of the Bridges User Guide for more information on using Jupyter through OnDemand.

Follow the steps below to run a Jupyter notebook on Bridges using a Jupyter client on your local machine.

To see what is included in a given module, use the module help command:

module help module-name [user@login018 ~]$ module help anaconda3/5.2.0 ----------- Module Specific Help for 'anaconda3/5.2.0' ------------ This modulefile prepares the environment for using anaconda3-5.2.0, which includes the conda package managerand a large collection of Python data science packages. The underlying Python in this Anaconda installation is 3.6.5.To use it, run the following commands: module load anaconda3/5.2.0 source activate $ANACONDA_HOME This is a master Anaconda installation, which can be a starting point for creating more specialized Anaconda environments.To do so, start by loading the module for this master Anaconda installation as before, and then cloning it, as follows: conda create --prefix --clone baseThis master Anaconda installation itself is write-protected to prevent any further additions or upgrades. To get further help with conda usage, you can try one of the following: conda -h conda -h[user@login018 ~]$

When you have identified the module that you want, load it by typing

module load module-name

​Note that the Anaconda module will define $JUPYTER_RUNTIME_DIR for you. This directory is used to store files like connection files which are useful only during the lifetime of a process. You can override this by explicitly setting $JUPYTER_RUNTIME_DIR after you have loaded the anaconda module, if you like.

3. Double check the versions of python and Jupyter

Do a sanity check: Use the --version flag with the jupyter and python commnds to be sure that you are using the versions of Jupyter and python that you expect.

4. Get a Bridges compute node allocated for your use

Get a Bridges compute node allocated for you by using the interact command. See the Running jobs section of the Bridges User Guide for more information on the interact command and the options available. You may have to wait for nodes to be allocated to you. This interact command asks for one node, and 4 cores on that node. You can vary the number of nodes and cores as needed. You will be allocated ~4.5 GB RAM for each core.

5. Find the hostname of the node you are running on

You will need the hostname when you are mapping a port on your local machine to the port on Bridges' compute node. Find the hostname of the node you are on from the prompt, or type the hostname command.

[user@r005 ~]$ hostname r005.pvt.bridges.psc.edu

^^^^^^​Note the first part of the name. Yours may be different. Note also you are not going to use the .pvt. bit. That would send traffic through the admin (pvt) network, but you will want that to go over "opa" (Intel's Omnipath) instead.

6. Start a Jupyter notebook and find the port number and token

Start a Jupyter notebook. From the output of that command, find the port that you are running on and your token. Pay attention to the port number you are given. You will need it to make the connection between the compute node and your local machine.

The port number Jupyter uses on the compute node can be different each time a notebook is started. Jupyter will attempt first to use port 8888, but if it is taken - by a different Jupyter user for example - it increases the port number by one and tries again, and repeats this until it finds a free port. The one it settles on is the one it will report.

In this case, the port number you have been given on the compute node is 8889.

7. Map a port on your local machine to the port Jupyter is using on the Bridges compute node

Use a second connection to Bridges to map the port 8888 in your local machine to the port you are using (8889 in this example) on the compute node. This is a bit of the chicken-and-the-egg situation: if you knew what node and what port you would end up on, you could have done the mapping on the first connection. But there is no way to know that a-priori, hence this.

On a separate shell, connect to Bridges with ssh and map the port Jupyter is using on the compute node to whatever port you like on the localhost. The format of this command is:

Here the localhost port is "8888". After the first ":" comes the long name of the compute node, a colon, and the port where Jupyter is running. Here, that string is "r005.opa.bridges.psc.edu:8889".

Note that even though the port on the compute node is 8889 in this case, we still use local port 8888 as usual.

8. Open a browser window to connect to the Jupyter server​

On your local machine, start up a browser and point it to http://localhost:8888. You will be prompted to enter a token to make the connection. Use the token given to when you started the Jupyter server on the Bridges compute node (step 6 above).

9. When you are done, close your interactive session on Bridges

Your allocation will be incurring SUs for the entire time that you are connected to your interactive session on the Bridges compute node. When you are finished, don't forget to return to the Bridges interactive session and end the Jupyter server with Control-C, and then end your interactive session by typing exit.