If you load one of those (module load iomkl/2017b), you can see the other modules and versions of software that it loaded with the 'module list':

+

$ module list

+

Currently Loaded Modules:

+

1) icc/2017.4.196-GCC-6.4.0-2.28

+

2) binutils/2.28-GCCcore-6.4.0

+

3) ifort/2017.4.196-GCC-6.4.0-2.28

+

4) iccifort/2017.4.196-GCC-6.4.0-2.28

+

5) GCCcore/6.4.0

+

6) numactl/2.0.11-GCCcore-6.4.0

+

7) hwloc/1.11.7-GCCcore-6.4.0

+

8) OpenMPI/2.1.1-iccifort-2017.4.196-GCC-6.4.0-2.28

+

9) iompi/2017b

+

10) imkl/2017.3.196-iompi-2017b

+

11) iomkl/2017b

+

+

As you can see, toolchains can depend on each other. For instance, the iomkl toolchain, depends on iompi, which depends on iccifort, which depend on icc and ifort, which depend on GCCcore which depend on GCC. Hence it is very important that the correct versions of all related software are loaded.

+

+

With software we provide, the toolchain used to compile is always specified in the "version" of the software that you want to load.

== Most Commonly Used Software ==

== Most Commonly Used Software ==

=== [http://www.open-mpi.org/ OpenMPI] ===

=== [http://www.open-mpi.org/ OpenMPI] ===

−

Version 1.8.1

+

We provide lots of versions, you are most likely better off directly loading a toolchain or application to make sure you get the right version, but you can see the versions we have with 'module spider OpenMPI':

−

=== [http://www.scilab.org Scilab] ===

+

* OpenMPI/2.0.2-GCC-6.3.0-2.27

−

Version 5.4.0

+

* OpenMPI/2.0.2-iccifort-2017.1.132-GCC-6.3.0-2.27

+

* OpenMPI/2.1.1-GCC-6.4.0-2.28

+

* OpenMPI/2.1.1-GCC-7.2.0-2.29

+

* OpenMPI/2.1.1-gcccuda-2017b

+

* OpenMPI/2.1.1-iccifort-2017.4.196-GCC-6.4.0-2.28

+

* OpenMPI/2.1.1-iccifort-2018.0.128-GCC-7.2.0-2.29

=== [http://www.r-project.org/ R] ===

=== [http://www.r-project.org/ R] ===

−

Version 3.1.0

+

We currently provide (module -r spider '^R$'):

+

* R/3.4.0-foss-2017beocatb-X11-20170314

−

==== Modules ====

+

==== Packages ====

We provide a small number of R modules installed by default, these are generally modules that are needed by more than one person.

We provide a small number of R modules installed by default, these are generally modules that are needed by more than one person.

−

==== Installing your own modules ====

+

==== Installing your own R Packages ====

To install your own module, login to Beocat and start R interactively

To install your own module, login to Beocat and start R interactively

<syntaxhighlight lang="bash">

<syntaxhighlight lang="bash">

+

module load R

R

R

</syntaxhighlight>

</syntaxhighlight>

Line 32:

Line 83:

==== Running R Jobs ====

==== Running R Jobs ====

−

You cannot submit an R script directly. '<tt>qsub myscript.R</tt>' will result in an error. Instead, you need to make a bash [[AdvancedSGE#Running_from_a_qsub_Submit_Script|script]] that will call R appropriately. Here is a minimal example. We'll save this as submit-R.qsub

+

You cannot submit an R script directly. '<tt>sbatch myscript.R</tt>' will result in an error. Instead, you need to make a bash [[AdvancedSGE#Running_from_a_qsub_Submit_Script|script]] that will call R appropriately. Here is a minimal example. We'll save this as submit-R.sbatch

<syntaxhighlight lang="bash">

<syntaxhighlight lang="bash">

−

#!/bin/bash

+

#!/bin/bash

−

#$ -l mem=1G

+

#SBATCH --mem-per-cpu=1G

−

# Now we tell qsub how long we expect our work to take: 15 minutes (H:MM:SS)

+

# Now we tell qsub how long we expect our work to take: 15 minutes (D-H:MM:SS)

−

#$ -l h_rt=0:15:00

+

#SBATCH --time=0-0:15:00

−

+

−

# Now lets do some actual work. This starts R and loads the file myscript.R

+

# Now lets do some actual work. This starts R and loads the file myscript.R

We allow each user to select his or her Java version individually. If you do not select one, we default to IcedTea 1.7. This was changed from Oracle JDK 1.7 on May 29, 2015 due to a EOL notice from Oracle.

−

−

==== Selecting your Java version ====

−

First, lets list the available versions. This can be done with the command <code>eselect java-vm list</code>

−

<pre>

−

% eselect java-vm list

−

Available Java Virtual Machines:

−

[1] icedtea-bin-6

−

[2] icedtea-bin-7 system-vm

−

[3] oracle-jdk-bin-1.7

−

[4] oracle-jdk-bin-1.8

−

[5] sun-jdk-1.6

−

</pre>

−

If you'll note, icedtea-bin-7 (marked "system-vm") is the default for all users. If you have a custom version set, it will be marked with "user-vm". Now if you wanted to use icedtea-6, you could run the following:

−

<syntaxhighlight lang="bash">

−

eselect java-vm set user 1

−

</syntaxhighlight>

−

Now, we see the difference when running the above command

−

<pre>

−

% eselect java-vm list

−

Available Java Virtual Machines:

−

[1] icedtea-bin-6 user-vm

−

[2] icedtea-bin-7 system-vm

−

[3] oracle-jdk-bin-1.7

−

[4] oracle-jdk-bin-1.8

−

[5] sun-jdk-1.6

−

</pre>

−

To verify you are seeing the correct java, you can run <code>java -version</code>

If you need modules that we do not have installed, you should use [https://virtualenv.pypa.io/en/stable/userguide/ virtualenv] to setup a virtual python environment in your home directory. This will let you install python modules as you please.

−

* [http://docs.python.org/2.7/ CPython 2.7]

−

* [http://docs.python.org/3.3/ CPython 3.3]

−

* [http://pypy.org/ PyPy 2.3.1] (Python 2.7.3)

−

−

For the uninitiated PyPy provides [[wikipedia:Just-in-time_compilation|just-in-time compilation]] for python code. While it doesn't support all modules, code which does run under PyPy can see a significant performance increase.

−

−

If you just need python and its default modules, you can use python2 python3 or pypy as you would any other application.

−

−

If, however, you need modules that we do not have installed, you should use [http://www.doughellmann.com/projects/virtualenvwrapper/ virtualenvwrapper] to setup a virtual python environment in your home directory. This will let you install python modules as you please.

* Create a virtual environment. Here I will create a default virtual environment called 'test', a python2 virtual environment called 'testp2', a python3 virtual environment called 'testp3', and a pypy environment called testpypy. Note that <code>mkvirtualenv --help</code> has many more useful options.

+

* Create a virtual environment. Here I will create a default virtual environment called 'test'. Note that <code>virtualenv --help</code> has many more useful options.

<syntaxhighlight lang="bash">

<syntaxhighlight lang="bash">

−

mkvirtualenv -p $(which python2) testp2

+

virtualenv test

−

mkvirtualenv -p $(which python3) testp3

−

mkvirtualenv -p $(which pypy) testpypy

</syntaxhighlight>

</syntaxhighlight>

* Lets look at our virtual environments

* Lets look at our virtual environments

<pre>

<pre>

−

%workon

+

% ls ~/virtualenvs

−

testp2

+

test

−

testp3

−

testpypy

</pre>

</pre>

* Activate one of these

* Activate one of these

<pre>

<pre>

−

%workon testp2

+

%source ~/virtualenvs/test/bin/activate

</pre>

</pre>

+

(After running this command your virtual environment is activated. After you logoff and then logon again your virtual environment will not be loaded so you must rerun this command every time you logon.)

* You can now install the python modules you want. This can be done using <tt>pip</tt>.

* You can now install the python modules you want. This can be done using <tt>pip</tt>.

If you are wanting to use mpi with your python script and are using a virtual environment, you will need to send the correct environment variables to all of the mpi processes to make the virtual environment work.

−

<syntaxhighlight lang="bash">

−

#!/bin/bash

−

# sample mpi4py submit script

−

source /usr/bin/virtualenvwrapper.sh

−

workon testp2

−

# figure out the location of the python interpreter in the virtual environment

−

PYTHON_BINARY=$(which python)

−

# mpirun the python interpreter within the virtual environment

−

# if you don't use the interpreter within the virtual environment, i.e. just using 'python'

−

# the system python interpreter (without access to your other modules) will be used.

−

mpirun ${PYTHON_BINARY} ~/path/to/your/mpi-enabled/python/script.py

−

</syntaxhighlight>

−

==== A note on [http://www.scipy.org/ scipy] ====

−

SciPy requires numpy, unfortunately it doesn't properly define a dependency on numpy, so you just have to install it first.

−

<syntaxhighlight lang="bash">

−

source /usr/bin/virtualenvwrapper.sh

−

workon testp2

−

pip install numpy

−

# now scipy needs lapack and it doesn't detect the system one. lets fix it

−

export LAPACK=/usr/lib/libreflapack.so

−

export BLAS=/usr/lib/libopenblas_openmp.so

−

pip install scipy

</syntaxhighlight>

</syntaxhighlight>

=== [http://www.perl.org/ Perl] ===

=== [http://www.perl.org/ Perl] ===

The system-wide version of perl is tracking the stable releases of perl. Unfortunately there are some features that we do not include in the system distribution of perl, namely threads.

The system-wide version of perl is tracking the stable releases of perl. Unfortunately there are some features that we do not include in the system distribution of perl, namely threads.

+

+

If you need a newer version (or threads), just load one we provide in our modules (module spider Perl):

+

* Perl/5.26.0-foss-2017beocatb

+

* Perl/5.26.0-iompi-2017beocatb

+

==== Submitting a job with Perl ====

==== Submitting a job with Perl ====

−

Much like R (above), you cannot simply '<tt>qsub myProgram.pl</tt>', but you must create a [[AdvancedSGE#Running_from_a_qsub_Submit_Script|submit script]] which will call perl. Here is an example:

+

Much like R (above), you cannot simply '<tt>sbatch myProgram.pl</tt>', but you must create a [[AdvancedSlurm#Running_from_a_sbatch_Submit_Script|submit script]] which will call perl. Here is an example:

<syntaxhighlight lang="bash">

<syntaxhighlight lang="bash">

#!/bin/bash

#!/bin/bash

−

#$ -l mem=1G

+

#SBATCH --mem-per-cpu=1G

# Now we tell qsub how long we expect our work to take: 15 minutes (H:MM:SS)

# Now we tell qsub how long we expect our work to take: 15 minutes (H:MM:SS)

As you can see, toolchains can depend on each other. For instance, the iomkl toolchain, depends on iompi, which depends on iccifort, which depend on icc and ifort, which depend on GCCcore which depend on GCC. Hence it is very important that the correct versions of all related software are loaded.

With software we provide, the toolchain used to compile is always specified in the "version" of the software that you want to load.

Most Commonly Used Software

We provide lots of versions, you are most likely better off directly loading a toolchain or application to make sure you get the right version, but you can see the versions we have with 'module spider OpenMPI':

Packages

We provide a small number of R modules installed by default, these are generally modules that are needed by more than one person.

Installing your own R Packages

To install your own module, login to Beocat and start R interactively

module load R
R

Then install the package using

install.packages("PACKAGENAME")

Follow the prompts. Note that there is a CRAN mirror at KU - it will be listed as "USA (KS)".

After installing you can test before leaving interactive mode by issuing the command

library("PACKAGENAME")

Running R Jobs

You cannot submit an R script directly. 'sbatch myscript.R' will result in an error. Instead, you need to make a bash script that will call R appropriately. Here is a minimal example. We'll save this as submit-R.sbatch

#!/bin/bash#SBATCH --mem-per-cpu=1G# Now we tell qsub how long we expect our work to take: 15 minutes (D-H:MM:SS)#SBATCH --time=0-0:15:00# Now lets do some actual work. This starts R and loads the file myscript.R
module load R
R --no-save -q < myscript.R

Create a virtual environment. Here I will create a default virtual environment called 'test'. Note that virtualenv --help has many more useful options.

virtualenv test

Lets look at our virtual environments

% ls ~/virtualenvs
test

Activate one of these

%source ~/virtualenvs/test/bin/activate

(After running this command your virtual environment is activated. After you logoff and then logon again your virtual environment will not be loaded so you must rerun this command every time you logon.)

You can now install the python modules you want. This can be done using pip.

Using your virtual environment within a job

The system-wide version of perl is tracking the stable releases of perl. Unfortunately there are some features that we do not include in the system distribution of perl, namely threads.

If you need a newer version (or threads), just load one we provide in our modules (module spider Perl):

Perl/5.26.0-foss-2017beocatb

Perl/5.26.0-iompi-2017beocatb

Submitting a job with Perl

Much like R (above), you cannot simply 'sbatch myProgram.pl', but you must create a submit script which will call perl. Here is an example:

#!/bin/bash#SBATCH --mem-per-cpu=1G# Now we tell qsub how long we expect our work to take: 15 minutes (H:MM:SS)#SBATCH --time=0-0:15:00# Now lets do some actual work.
module load Perl
perl /path/to/myProgram.pl

Installing my own software

Installing and maintaining software for the many different users of Beocat would be very difficult, if not impossible. For this reason, we don't generally install user-run software on our cluster. Instead, we ask that you install it into your home directories.

In many cases, the software vendor or support site will incorrectly assume that you are installing the software system-wide or that you need 'sudo' access.

As a quick example of installing software in your home directory, we have a sample video on our Training Videos page. If you're still having problems or questions, please contact support as mentioned on our Main Page.