Recommended Compiler Optimizations

We recommend compiling at high optimization levels. For the above compilers, -O3 is recommended. We do not recommend -march=native as we have many different hardware generations; if you compile with -march=native on m8 the program may error with an illegal instruction if you run it on m7.

MPI

The MPI libraries and compiler wrappers are tied to particular compilers. Load the compiler you want first, then run one of these commands to see which MPI implementations are available:

module avail openmpimodule avail intel-mpi

For example, the output of module load gcc/7 ; module avail openmpi currently looks like this:

Intel Math Kernel Libraries

The Intel Math Kernel Libraries are a suite of libraries that provide highly optimized versions of common math routines. If your application needs to use a linear algebra package, such as LAPACK or BLAS, fast fourier transforms, or other common math routines, you may benefit by linking against the Intel MKL. The Intel MKL can be found in intel-mkl modules.

Generally, the Intel MKL should either be statically linked or have a RUNPATH set in the library/executables that need it. We set the LD_RUN_PATH environment variable, which many tools will use to automatically set the RPATH/RUNPATH. If that does not work you may need to add flags such as -Wl,--enable-new-dtags,-rpath,$MKLROOT/lib/intel64.

These percentages only report the utilization of the hardware. Current utilization is not necessarily
indicative of how long your jobs may wait to run.

Some jobs are preemptable and may be evicted the moment you submit new jobs. Others jobs
may be nearing completion and you might only have to wait a few minutes. Or maybe
the queue is so long and your priority so low that you'll wait for days.

The only way to know is to examine the queue or just submit your jobs. Don't assume that
100% utilization means "I might as well give up" because that is very likely incorrect.
Submit your jobs and hope for the best.