Context Navigation

Testing ORAC Code Updates

A Python 2.7 script has been produced to call the various ORAC executables and compare their outputs to those previously produced. It should be run before any commit to the repository to ensure there are no unforseen side effects. It can be found at http://proj.badc.rl.ac.uk/orac/browser#trunk/tools.

Regression test script

The primary script for testing changes to the ORAC code is regression.py. When called without arguments, it pre-processes six short segments of data, evaluates and merges liquid and ice phase cloud, and finally reports any changes in the files since the previous version of the ORAC code. These scripts are maintained by Adam Povey, who should be contacted if you have any issues.

Local defaults

You must create a local_defaults.py file in the script directory. An example is attached to this page. This defines where various auxilliary files are stored in your files system in order to minimise the number of arguments you normally pass the script.

Script arguments

The scripts has numerous optional arguments covering every option of the pre and post-processors and several other useful features. The most commonly used are:

Argument

Short version

Description

--help

-h

Print a full list of arguments and a description of their function.

--benchmark

-B

Produce a benchmark file for this current revision (i.e. don't compare the outputs).

--long

-L

Process the full data files, rather than just the short segments selected for regression testing.

The revision number used to label the files. This defaults to the svn revision number plus one, is written into the filenames and NCDF attributes, and is used to identify the previous data version when comparing.

--extra_lines []

Name of a file, the contents of which will be appended to the end of each main processing driver file. Used to set elements of Ctrl that aren't explicitly managed.

--chunking

Chunk the post-processing to reduce memory overhead.

--clobber []

-C []

Controls which files are overwritten. 3=All (the default), 2=Pre-processing, 1=Pre and main processing, 0=None.

--timing

Print how long each command took to run.

--sabotage

Methodically invalidate channels until every possible combination of missing channels are available; a debugging mode.

If you wish to test the aerosol processing, the following arguments may be useful:

Argument

Short version

Description

--all_phases

-A

Process aerosol and cloud phases (rather than just cloud); requires two views.

--out_dir []

-o []

Directory to store output files.

--l1_land_mask

Use the imager's land/sea mask, rather than that drawn from the USGS land-use database.

--use_oc

Use the Ocean Colour CCI data product rather than assuming a constant chlorophyll concentration.

Regression test input data

An AATSR orbit, an AVHRR orbit, and day and night MODIS granules have been selected as test cases for ORAC. These were selected to contain cloud-covered land and sea on 20 June 2008. These data and all of the files necessary to process them can be found at http://proj.badc.rl.ac.uk/orac/browser#data (which can be stored separately from the main trunk on your local system). For the daytime MODIS scene, both a spectrally-subsetted and full LAADS version of the data are available.

Due to issues with the server timeout, it is not possible to place particularly large files into the repository. As such, two of the test files have been split into smaller segments. To reconstitute the original files, use the following commands:

Regression test output data

The script will place the output files in the testoutput folder of the data directory set in the previous section. Each test is kept in a folder named after the test (e.g. 'DAYAATSRS' for the short daytime AATSR segment or 'AATSR' for the full orbit). The subsidiary output files are then further organised by:

The python code can also be used independently of the regression tests to call all or part of the ORAC software package. The scripts are:

orac.py) Calls the pre, main, and post-processors in turn to evaluate a level 1B file (specified by the positional argument target). Most of the regression test optional arguments are available, in addition to:

Argument

Short version

Description

--phases []

-p []

The particle phases to be evaluated. Options are WAT, ICE, and A70-A79.

orac_preproc.py) Run the pre-processor on a given level 1B imager file. Useful optional arguments:

Argument

Short version

Description

--in_dir []

-i []

The directory in which the input file can be found. (This can be ommitted if the target argument is the full path for the file.)

--channel_ids []

-c []

The channel numbers to be included in the output file.

--limit X0 X1 Y0 Y1

-l []

A subset of the satellite swath to be output. (Indexed from 1.)

orac_main.py) Run the main processor on a set of pre-processed files sharing the given root filename. Useful optional arguments:

Argument

Short version

Description

--approach

-a []

The retrieval approach to apply during retrieval. Options are CldWat, CldIce, AerOx, AerSw.

--land or --sea

Process only land or sea pixels, respectively.

--cloud_only or --aerosol_only

Process only cloudy or clear pixels, respectively.

orac_postproc.py) Run the postprocessor on a set of processed files sharing the given root filename. Useful optional arguments:

Argument

Short version

Description

--in_dir []

-i []

A list of directories in which the input files can be found.

--switch_phase

When processing only cloud phases, check if the cloud top temperature is appropriate when selecting type.