Consulting

VASP

Description

VASP is a package for performing ab initio quantum-mechanical molecular dynamics (MD) using pseudopotentials and a plane wave basis set. The approach implemented in VASP is based on a finite-temperature local-density approximation (with the free energy as variational quantity) and an exact evaluation of the instantaneous electronic ground state at each MD step using efficient matrix diagonalization schemes and an efficient Pulay mixing.

Access to VASP

VASP is available only to NERSC users who already have an existing VASP license. If you have a VASP license, send your license information to the VASP support at Vienna at vasp.materialphysik@univie.ac.at (and copy vasp_licensing@nersc.gov) requesting that they confirm your VASP license to NERSC staff at vasp_licensing@nersc.gov in order to gain access to the VASP binaries at NERSC. Please note, VASP 5 requires a separate license. The VASP 4.6 users will NOT be automatically upgraded to VASP 5. Therefore, you need to confirm your VASP 5 license separately if you are a VASP 4 user and just upgraded your license to VASP 5.

Modules

NERSC uses modules to manage access to software. To access VASP at NERSC you must first verify your VASP license via email. When your request has been processed you will be able to access the vasp executables.

For instructions on how to obtain access to VASP at NERSC, please login to a NERSC machine (e.g., Hopper) and issue the command:

% module show vasp

To see the available vasp modules, type:

% module avail vasp

To access the default version of VASP, type:

% module load vasp

To access VASP 5, type:

% module load vasp/5.2

Using VASP on Hopper and Edison

We have installed many versions of VASP 5 and 4 on Hopper and Edison. The default version is VASP 5 on Hopper and Edison. To see the enviornment variables set by the vasp module, do

% module show vasp

or provided the full module name as follows:

% module show vasp/5.3.3

You can see where the VASP executables reside (the bin directory). There are several VASP executables available in the bin directory,

These are the gamma-point only (executable name:gvasp), the non-collinear (executable name: vasp_ncl), and the collinear (executable name: vasp) versions of the code, respectively. You need to choose an appropriate executable to run your job.

There are two ways of running VASP on Hopper and Edison: submitting a batch job, or running interactively in an interactive batch session.

To run VASP interactively on Hopper and Edison

When the batch session is returned to you, issue the following commands:

% cd $PBS_O_WORKDIR % module load vasp % aprun -n 32 vasp

Sample job script to run on unpacked nodes on Hopper

For large memory vasp jobs, users often need to run then on unpacked nodes, i.e., using only part of the cores available and leave some of them idle to allow more memory for the cores in use. Here is a same job script to run on unpacked nodes, eg., using only 8 cores out of 24, and please note, the -S1 option is important to get the optimal MPI task placement on a unpacked node. More information about how Hopper places MPI tasks onto its cores on the node can be found here.

#PBS -l mppwidth=768,walltime=00:30:00#PBS -q debug#PBS -j oe#PBS -V

cd $PBS_O_WORKDIR

module load vaspaprun -n384 -N12 -S3 vasp

Sample job script to run multi-threaded VASP on Hopper

VASP doesn't have OpenMP directives implemented in the source code, but we linked the code to the Cray multi-threaded scientific libraries to make use of the memory savings from using threads, and also the potential performance gains, although it is limitted, for users who have to run their jobs on the unpacked nodes to allow more memory for the cores in use. We suggest users to use low threads counts (2 or 3) to avoid the peformance penalty from NUMA effects. More performance results of the multi-threaded VASP can be found here. The sample job script is as follows:

Using VASP on Carver

You must first load the VASP module; this will set your LD_LIBRARY_PATH environment variable to include a library (Intel MKL) needed to run VASP. If you attempt to run VASP without loading its module you will get an error because the library was not loaded.

Using VASP Via the Web with NOVA

NERSC's Online Vasp Application (NOVA) is an experimental tool that enables users to submit VASP jobs to NERSC resources (currently, only Carver). NERSC users who are registered VASP users (i.e., in the vasp user group at NERSC) can use it to set up VASP jobs and monitor their progress from a web browser. NOVA jobs use the same directory and file structure as regular command-line VASP jobs, so it is possible to import existing VASP jobs into NOVA. You can also quickly modify an existing job and save it as a new NOVA job. NOVA makes it easy to select a pseudopotential, use automatic k-point generation, and set run parameters for VASP as well as the NERSC system running the job (currently Carver). It also allows the user to visualize convergence and monitor the status of multiple jobs. You can prepare a job partially, save it, and then come back another time to finish.

The best way to get acquanted with NOVA is to log on and import a sample job. Log on with your NIM user name and password, then click the Import Job button. In the dialog, give your job a name, choose a sample file set from the drop-down menu under Location of Job to be Imported, then select a place for NOVA to store your working copy of the sample files. We recommend selecting your scratch directory. Click Setup, and NOVA will create the directory and copy the necessary sample files into it. The sample files will all have POSCAR, POTCAR, KPOINTS, and INCAR files already. You'll just need to tweak what you like, then choose and save your computational settings.

When all the files are ready, click Run This Job to submit it to a queue. Once the job is submitted, NOVA will show you basic information about it, and once the OSZICAR file has some data, you should see a plot of the energy at each step. To stop the job or work on another one, navigate to the Job Control page. Here you can copy the job, delete it, move its files elsewhere, kill it, or get information about any job. You can also import another job or create a new one from scratch.

NOVA is still a work in progress, so feedback from users is much appreciated. If you give it a try, please let us know how it went and what we could do better. Comments can be sent to amgreiner@lbl.gov or deskinner@lbl.gov.

Trouble Shooting and Error Messages

In this section, we list some common errors users may run into when running VASP jobs on our machines, and our recommendations about how to resolve them.

If you use our VASP modules, and loaded a vasp module successfully, but still get an error on Hopper or Edison like this:

"aprun: file vasp not found"

or on Carver like this:

"mpirun was unable to launch the specified application as it could not find an executable:Executable: vasp"

it usually means that you don't have the permission to access the precompiled VASP binaries that NERSC provides. You need to confirm your license by following the instructions posted in the Description session of this page.

Building VASP

Some users have reported a build error on Hopper.

Build error: /usr/bin/ld: cannot find -lsci_mc12_mp

Cray has recently renamed the multi-threaded libsci libraries to help the performance of dynamic libraries applications on Hopper. So the compiler wrapper now links to the -lsci_pgi_mp (for pgi compiler) instead of -lsci_mc12_mp. You can simply remove the "-lsci_mc12_mp" in the makefile, or replace the -lsci_mc12_mp with the -lsci_pgi_mp, then the code should link without a problem (it will link to the -lsci_pgi_mp by default).