Stokes Supercomputer

Hardware

Stokes (stokes.ichec.ie is an SGI Altix ICE 8200EX cluster with 320 compute nodes. Each compute node has two Intel (Westmere) Xeon E5650 hex-core processors and 24GB of RAM. This results in a total of 3840 cores and 7680GB of RAM available for jobs.

The nodes are interconnected via two planes of ConnectX Infiniband (DDR) providing high bandwidth and low latency for both computational communications and storage access. Storage is provided via a Panasas ActiveStor 5200 cluster with 175TB (formatted) of capacity to the compute nodes via a PanFS filesystem.

In addition to the compute nodes a set of service and administrative nodes provide user login, batch scheduling, management, etc.

Software

A large number of scientific software packages, libraries, compilers and other tools are installed and maintained on Stokes by ICHEC. A full listing of these is available on the software page.

Scheduling Policy

The following queues are available on Stokes:

Job Size (cores)

Maximum Walltime

Maximum Available Cores

DevQ

12 - 96

30 minutes

192

ShortQ

24 - 516

24 hours

3264

ProdQ

24 - 516

84 hours

3264

LongQ

12

168 hours

192

Note:

Users should avoid specifying a queue in their PBS scripts. Specifying the walltime and processor requirements will result in the job being routed to the optimum queue.

Processing cores on Stokes are allocated in multiples of twelve (one node). This means that users can only request core counts of 12, 24, 36, 48...

Hard and soft throttling limits are in place to prevent a single user from flooding the entire system.

Queued jobs gain priority at different rates depending on many factors.

Documentation

Photos

Some photos of the Stokes system:

Acknowledgement

Stokes was funded under the PRTLI Cycle 4 funded project e-INIS. Direct contributions were received to Stokes from six institutions: UCD, NUIM, DCU, NUIG, UL and DIAS and each owns a percentage of system time.