In both cases, ppn=12 means that a full node is reserved, and this would have no impact on other users. But if only 3 engines are needed, because the IPController counts as one process, the PBS script should reserve ppn=4.

Here is how to submit the job for each type of execution:

Interactive

Automatic

qsub -I -q sw script.sh

Wait for the interactive shell. Then, launch ipcluster:

ipcluster start --n=12 --profile=local0

qsub -q sw script.sh

Wait for the job to start. Use showq -u <username>.

The ipcluster command will be executed automatically.

The ipcluster needs some time to get ready. In the interactive mode, you will see the following message: "Engines appear to have started successfully". In the automatic mode, you would have to redirect (>) the output of ipcluster in a file and to monitor that file.

In this example, 2 full nodes will be reserved. The IPController will run on the first node reserved by Moab.

Here is how to submit the job for each type of execution:

Interactive

Automatic

qsub -I -q hb script.sh

Wait for the interactive shell. Then, launch ipcluster:

ipcluster start --n=24 --profile=mpi0

qsub -q hb script.sh

Wait for the job to start. Use showq -u <username>.

The ipcluster command will be executed automatically.

The ipcluster needs some time to get ready. In the interactive mode, you will see the following message: "Engines appear to have started successfully". In the automatic mode, you would have to redirect (>) the output of ipcluster in a file and to monitor that file.

[Single, Multiple] - Using IPython Parallel

Connect to Guillimin with a second terminal. On the login node, launch ipython with the appropriate profile:

In [4]: with c[:].sync_imports(): import socket
In [5]: %px print(socket.gethostname())

To exit IPython:

In [6]: exit

[Single, Multiple] - Stopping the job

Interactive

Automatic

Use Ctrl+C to kill ipcluster. Then, enter exit to stop the job.

Use canceljob to kill the remote processes and stop the job.

Conclusion

The interactive mode may do a cleaner and easier to monitor job than the automatic mode. On the other hand, the automatic mode does not require an opened terminal waiting for the job to start. The interactive mode is the recommended method.