The CVS HEAD tag represents the most recently checked in source code and does not always
compile nor run. Users will generally want to download the most recent release with a specific
version number below. The documentation available at left is auto-generated daily and may be
missing or have formatting errors.

All platforms using GNU g++: The library build failure with GNU g++ 6.1 (and newer) has been corrected. The default option of ESMF_MOAB=internal can safely be used.

Linux with intel (Theia): The problem of applications built with optimization level -O2 or higher, and linked against ESMF using the linker flags provided by esmf.mk, failing after the October system upgrade has been mitigated. The linker flags provided by esmf.mk no longer explicitly contain the -lsvml option.

Note for Platforms Using MPICH 3:
The mpich 3 series split apart the Fortran and C bindings. This requires ESMF_CXXLINKLIBS=-lmpifort to be set for a successful ESMF build using mpich. If this doesn't resolve your build issue with mpich, please contact ESMF support at esmf_support@list.woc.noaa.gov.

The overestimation of the internal buffer requirement in ESMF_StateReconcile() was fixed. It is now possible to run beyond the 10,000 core limit that previously caused ESMF_StateReconcile() to run out of memory on typical configurations.

Note for Platforms Using MPICH 3:
The mpich 3 series split apart the Fortran and C bindings. This requires ESMF_CXXLINKLIBS=-lmpifort to be set for a successful ESMF build using mpich. If this doesn't resolve your build issue with mpich, please contact ESMF support at esmf_support@list.woc.noaa.gov.

This release is backward compatible with the last public release, ESMF 6.3.0rp1, for all the interfaces that are marked as backward compatible in the Reference Manual. There were API changes to some of the unmarked methods that may require modifications to user code that uses these methods. The entire list of API changes is summarized in a table showing interface changes since ESMF_6_3_0rp1, including the rationale and impact for each change.

Tables summarizing the ESMF regridding status have been updated to include the new capability to perform regridding to/from a set of disconnected points (i.e. an ESMF LocStream) and to perform bilinear regridding on 3D spherical grids. These tables include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.

The NUOPC Layer contained in this release is a refinement of the previously released reference API. One of the design goals was to provide a more object oriented interface that clearly reflects the NUOPC concepts of Driver, Model, Connector, Mediator. The following specific technical areas were also improved:

The NUOPC Driver now allows child components to be dynamically added without prior setting of the total number of components in the driver. This flexibility simplifies coding of highly configurable coupled systems.

The consistent use of labels to reference components and their initialize/run/finalize phases simplifies interaction with the NUOPC Layer and makes application code more readable and less error prone.

The introduction of namespaces provides more control over the automatic field pairing implemented by the NUOPC Connector.

The NUOPC Compliance Checker has become more context aware and as a consequence produces more usable output.

The transfer protocol of Geom objects (e.g. ESMF Grids and Meshes) between components during initialization has been extended to cover ESMF LocStreams (a geometric type that can be used to represent a set of disconnected points).

Added copy() for Grid, Mesh, LocStream, and Field to produce a shallow copy of the object.

Added ability to return Mesh element coordinates.

Added Anaconda packages for easy installation of ESMPy on Linux and Darwin. Specific package availability is recorded in a table of NESII Anaconda packages.

Added a new application ESMF_Regrid to support file-based regridding. This application regrids a data field defined in a grid file to another field defined in a different grid file. The regridded field will be written into the destination grid file. The grid files have to be in either UGRID or GRIDSPEC format and are limited to 2D grids.

ESMF LocStreams can now be used in regridding. This is useful for operations involving observational data, for example in data assimilation.

ESMF now offers the option to build the unstructured Mesh class on DOE's MOAB mesh library as an alternative to its native implementation. Meshes built on MOAB support a limited number of operations. They can be created, destroyed, queried for information, and used in the creation of an ESMF Field. The standard ESMF Field operations are expected to work on a Field build on a MOAB Mesh, however testing of these has been limited. In the case of ESMF regridding only the first order conservative regridding method is supported.

Added the ability for an element in an ESMF unstructured Mesh to consist of more than one polygon. This allows the representation of geographically disconnected areas (e.g. Hawaiian islands) as a single element. Meshes containing elements of this type will also work in the ESMF regrid weight generation methods and applications.

Added a new algorithm to compute bilinear interpolation weights in 3D. To use this algorithm set lineType=ESMF_LINETYPE_GREAT_CIRCLE when calling into the ESMF regrid weight generation methods and applications. Weights are then calculated along lines that follow the curve of a sphere instead of straight lines. This significantly reduces the error in 3D spherical grids with highly attenuated cells.

Expanded ESMF's first-order conservative interpolation algorithm to work on quadrilateral concave cells. Previously an error was returned under this condition.

Removed a restriction in the unstructured Mesh class so that now element coordinates can be set and retrieved for elements with more than four sides. Previously an error was returned under this condition.

Extended the ESMF_RegridWeightGen application and the ESMF_MeshCreate() method to support 3D unstructured grid definitions provided in the UGRID file format.

The ESMF_RegridWeightGen application now supports additional options:

Added --src_loc and --dst_loc options to allow non-conservative regridding on the cell center locations of an unstructured grid defined in the UGRID or the ESMF unstructured file format.

Added --ignore_degenerate to ignore degenerate cells in either the source or the destination grid. This can be useful when cells that collapse into a point or line aren't a concern (e.g. when they are located in a region not used in regridding).

Added --line_type or -l to allow users to choose between great circle or Cartesian paths for the line between two points on a sphere during the calculation of bilinear interpolation weights.

Added two new shortcut Grid create methods: ESMF_GridCreate1PeriDimUfrm() and ESMF_GridCreateNoPeriDimUfrm(). Each of these methods creates a Grid and fills the coordinate arrays with uniformly spaced coordinate values.

Added a new interface to create an ESMF LocStream from a grid file. Currently, SCRIP, UGRID and ESMF unstructured file formats are supported. For a grid in either UGRID or ESMF format, the center coordinates or the corner coordinates can be chosen to construct the LocStream. For a grid in SCRIP format, the center coordinates will be used to construct the LocStream.

Expanded the ESMF_MeshCreate() method to support creation of a dual Mesh using the center coordinates defined in a UGRID or ESMF unstructured grid file.

The ESMF I/O methods that depend on the Parallel I/O (PIO) library are now turned on by default on platforms where certain Fortran 2003 features are supported (compilers not supported are Absoft and Lahey). The affected methods are ESMF_FieldBundleRead(), ESMF_FieldBundleWrite(), ESMF_FieldRead(), ESMF_FieldWrite(), ESMF_ArrayBundleRead(), ESMF_ArrayBundleWrite(), and ESMF_ArrayRead(), ESMF_ArrayWrite(). On platforms that do not fulfill the above system requirements, the PIO dependent methods will return an error when invoked.

Extended FieldBundle, Field, ArrayBundle, and Array I/O to support multiple DEs per PET and undistributed dimensions.

Extended the Log class to allow closing of a Log object (including the default Log) and reopening it under different name.

All of the distributed data classes (FieldBundle, Field, ArrayBundle, Array, Grid, DistGrid, DELayout) now offer the "noGarbage" option in their Destroy() method. This allows the user to deallocate all memory associated with an object, even the reference held by the ESMF garbage collection. Use of this new option has shown to be necessary to prevent out-of-memory conditions in applications that use many temporary objects (e.g. create new temporary objects each time step).

The ESMF VM class is now accelerator aware. The user can query the VM for information about available accelerator hardware. This feature is implemented on top of four different lower level software stacks: OpenCL, OpenACC, Intel MIC, and OpenMP (version 4). The appropriate software stack must be chosen when building the ESMF library by setting the ESMF_ACC_SOFTWARE_STACK environment variable in order to activate this new feature. See the User's Guide for details.

Added functionality to retrieve names of all ESMF deep objects using ESMF_AttributeGet(). This is the next step in implementing full access to object information through the ESMF Attribute interface.

Added the attnestflag argument to ESMF_AttributeGet() to allow users to specify whether they want to search through the entire Attribute hierarchy when retrieving an Attribute.

The ESMF_StateReconcile() call overestimates the size requirement of an internal temporary buffer. The estimation error increases with core count and causes ESMF_StateReconcile() to run out of memory for large core counts (~10,000) on typical configurations.

FieldBundles don't currently enforce that every contained Field is built on the same Grid, Mesh, LocStream, or XGrid object, although the documentation says that this should be so.

When the ESMF regrid weight generation methods and applications are used with nearest destination to source interpolation method, the unmapped destination point detection does not work. Even if the option is set to return an error for unmapped destination points (the default) no error will be returned.

ESMF_FieldRegridStore() that takes XGrid as an argument does not support regridding between Fields that are constructed on Grids for which ESMF_GridMatch() locally returns a value of ESMF_GRIDMATCH_EXACT or greater. This may result in unexpected local errors.

The ESMF_Regrid application and the ESMF_FileRegrid() method abort with an error when the destination file is in NetCDF4 format and the destination variable does not exist.

The ESMF_Regrid application and the ESMF_FileRegrid() method abort with an error when the variables are of type other than double or float.

The ESMF_Regrid application and the ESMF_FileRegrid() method abort with an error when the variables have more than two additional dimensions over the coordinate dimension.

The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution will give incorrect results when used on a Grid with 3 or more dimensions and whose coordinate arrays are less than the full dimension of the Grid (i.e. it contains factorized coordinates).

The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreate1PeriDim()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.

ESMF_AttributeWrite() has only been verified to work for ESMF standard Attribute packages. Non standard Attribute packages may trigger a crash inside the ESMF_AttributeWrite() implementation.

ESMF does not link correctly with the Fortran libraries of the latest MPICH 3 series due to a separation of the libraries holding C and Fortran symbols.

Linux with gfortran (Pluto): Fails the ESMF_ArrayBundleIOUTest.F90 and the ESMF_ArrayIOUTest.F90 unit tests.

Linux with intel (Theia): After a system upgrade on Theia in October 2017, applications built with optimization level -O2 or higher, and linked against ESMF using the linker flags provided by esmf.mk, started failing. The symptoms were similar to memory corruption issues. It was determined that the problem could be triggered for a very simple Fortran program (not depending on ESMF), using ifort -O2 (or higher optimization), and explicitly linking against Intel's small vector math library via -lsvml. This finding was reported to Cray and Intel and a reproducer was made available to the vendor.

Windows/Cygwin with g95 (Nix): The command line tools ESMF_RegridWeightGen and ESMF_Regrid do not work.

Note for Platforms Using MPICH 3:
The mpich 3 series split apart the Fortran and C bindings. This requires ESMF_CXXLINKLIBS=-lmpifort to be set for a successful ESMF build using mpich. If this doesn't resolve your build issue with mpich, please contact ESMF support at esmf_support@list.woc.noaa.gov.

Tables summarizing the ESMF regridding status have been updated to include the new ESMF_NormType option. These tables include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.

An optional command line argument "--no_log" was added to the ESMF_RegridWeightGen application. This allows the user to turn off the ESMF log files that are written by default.

The ESMF Python interface (ESMPy) is now part of the ESMF source distribution. The build and installation process for both ESMF and ESMPy remain separate in this release.

When building ESMF with NetCDF support enabled, the NetCDF C++ interface is no longer required.

New build configurations were added to support the Apple Clang/LLVM C++ compiler on both Linux and Darwin systems.

Support was added to compile the ESMF library for the Intel MIC architecture under Linux. This allows ESMF applications to execute natively on the Intel Xeon Phi accelerator hardware.

Same as ESMF_6_3_0r with the following exceptions:

The ESMF_RegridWeightGen application now correctly handles cases of SCRIP unstructured grid files that contain corner coordinates with small round off errors. Including those where for some corners the round off error along one dimension is above the tolerance value while it is below the tolerance for the other dimension.

The issue with ESMF_RegridWeightGen() crashing when either of the optional arguments srcFileType or dstFileType is not given has been fixed. Now it is okay to not specify the arguments.

The ESMF_AttributeUpdate() method now works correctly for the following situations:

Making a small number of changes to a large Attribute package not longer leads to failures due to buffer issues.

A Grid shared by multiple Fields now correctly updates all Grid attributes on all PETs.

A State that is reconciled more than once to Components on exclusive petLists now correctly updates on all Components.

Any Attribute that is added during runtime and then modified will be correctly updated.

The debug print statement inside of the ESMF library that was triggered when calling ESMF_AttributeGet() has been removed. Calling ESMF_AttributeGet() many times will no longer cause issues with large volume of output.

The intent of the "pathName" argument in the ESMF_UtilIOGetCWD() method is now correctly specified as intent(out). This makes the method dependable.

The issue of building the unit tests and examples when build option ESMF_ARRAY_LITE is set to TRUE has been resolved. All test and example codes now build with this option.

Windows/Cygwin with g95: Fails the ESMF_UtilUTest.F90 and TestHarness unit tests when linked against the shared library version of ESMF.

Note for Platforms Using MPICH 3:
The mpich 3 series split apart the Fortran and C bindings. This requires ESMF_CXXLINKLIBS=-lmpifort to be set for a successful ESMF build using mpich. If this doesn't resolve your build issue with mpich, please contact ESMF support at esmf_support@list.woc.noaa.gov.

This release is backward compatible with ESMF 6.2.0 and the last public release, ESMF 5.2.0rp3, for all the interfaces that are marked as backward compatible in the Reference Manual. There were API changes to a few unmarked methods that may require minor modifications to user code that uses these methods. The entire list of API changes is summarized in a table showing interface changes since ESMF_6_2_0, including the rationale and impact for each change.

Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.

The NUOPC Layer contained in this release has been improved in the following specific technical areas:

The NUOPC Component Explorer was added. This is a run-time tool that can be used to analyze the compliance status of a Component.

Standard Component dependencies have been formalized and documented, making Components more interoperable on the build level.

The resolution of data dependencies during initialization was extended to cover components that run on different petLists.

The transfer of Grid and Mesh objects between Components is now supported during initialization.

The tools that allow Components to be made available as web-based services now provide the capability to use a job scheduler to manage the execution and resource allocation for the service.

The capability to create logically rectangular Grid objects from NetCDF files in SCRIP format was added to ESMPy, the ESMF Python interface. Documentation and downloads are available on the ESMPy web page.

Distributed objects Fields, Arrays, Grids, and Meshes created in one Component can now be transferred to another Component, even when the Components are defined on different petLists.

The "--check" option was added to the ESMF_RegridWeightGen application to calculate the interpolation and conservation error from regridding with the weights in the output file. The error is calculated using an analytical test field.

The ESMF_RegridWeightGen() interface was overloaded to allow regridding between grids with user specified element and optional nodal distribution. The grids must be supplied in the SCRIP format.

The ESMF_FieldRegridStore() call has been extended to work on Fields built on Meshes containing elements with more than four sides (e.g. pentagons and hexagons).

The ESMF_FieldRegridStore() call has been extended so that non-conservative interpolation methods (bilinear, patch, and nearest-neighbor) can be performed on Fields built on the Mesh element location (ESMF_MESHLOC_ELEMENT).

The ESMF_FieldRegridStore() call now allows the user to choose between great circle or Cartesian paths for the line between two points on a sphere during the calculation of bilinear interpolation weights.

The ESMF_FieldEmptySet() method now allows the Grid, Mesh, LocStream, or XGrid object to be replaced in a partially completed Field.

The ESMF_MeshCreate() interfaces have been extended to allow the user to specify a coordinate system. This information is used by the ESMF_FieldRegridStore() call to automatically determine the geometry of the source and destination Meshes.

The ESMF_MeshCreate() and ESMF_MeshAddElements() methods have been extended to allow the user to create a Mesh containing elements with more than four sides (e.g. pentagons and hexagons).

The ESMF_MeshCreate() and ESMF_MeshAddElements() methods have been extended to allow the user to optionally set element center coordinates. This information is used when doing non-conservative regridding on Fields built on the Mesh element location (ESMF_MESHLOC_ELEMENT).

The ESMF_MeshCreate() which allows the user to to create a copy of an existing Mesh, but with a different distribution, now allows the user to optionally omit the node distribution.

The ESMF_MeshCreate() method has been extended to allow generation of the nodal mask when reading from a UGRID file.

The ESMF_MeshCreate() method has been updated to support the new UGRID version 0.9.0 conventions.

The ESMF_MeshCreate() method has been extended to allow Meshes to be created from a grid file with user specified element and optional nodal distribution.

The ESMF_GridCreate() interface was overloaded to allow Grids to be created from a grid file with user specified distribution.

The ESMF_AttPack type was introduced, allowing a user to retrieve a handle to an ESMF Attribute package. This is a more compact way of working with Attribute packages.

The ESMF_UtilIOGetCWD() interface was added to allow retrieval of the absolute path and name of the current working directory.

A new parallel application ESMF_Scrip2Unstruct was added to convert a SCRIP format grid file to an unstructured ESMF format or a UGRID format grid file. This allows the user to execute the potentially expensive file format conversion operation once instead of every time the ESMF_RegridWeightGen application is called. The SCRIP file can be either unstructured or logically rectangular. The application has an option to generate a dual mesh, which is needed to support non-conservative regridding on cell centers.

The ESMF_StateReconcile() call overestimates the size requirement of an internal temporary buffer. The estimation error increases with core count and causes ESMF_StateReconcile() to run out of memory for large core counts (~10,000) on typical configurations.

FieldBundles don't currently enforce that every contained Field is built on the same Grid, Mesh, LocStream, or XGrid object, although the documentation says that this should be so.

When the ESMF regrid weight generation methods and applications are used with nearest destination to source interpolation method, the unmapped destination point detection does not work. Even if the option is set to return an error for unmapped destination points (the default) no error will be returned.

The ESMF_RegridWeightGen application has issues with some cases of SCRIP unstructured grid files that contain corner coordinates with small round off errors. Previously a small tolerance was introduced to determine if corner points of separate cells match to connect together neighboring cells. However, a problem can still occur in cases where for some corners the round off error along one dimension is above the tolerance value while it is below the tolerance for the other dimension. This can result in unexpected unmapped destination points in the regridding.

The ESMF_RegridWeightGen() method crashes when either or both of the optional arguments srcFileType or dstFileType are not given.

ESMF_FieldRegridStore() that takes XGrid as an argument does not support regridding between Fields that are constructed on Grids for which ESMF_GridMatch() locally returns a value of ESMF_GRIDMATCH_EXACT or greater. This may result in unexpected local errors.

ESMF_FieldRegridStore() can not be used with non-conservative interpolation methods (bilinear, patch, or nearest-neighbor) for Fields built on the element location of a Mesh (ESMF_MESHLOC_ELEMENT) that contains elements with more than four sides. The call will return with error if used this way.

A larger than expected difference in integrals (8.7E-10) is occurring when using the conservative regridding between a 1 deg lat lon grid and a 2.5 deg lat lon grid. This problem appears isolated to a subset of the cases when grid cell edges align, but not exactly due to round off. It is recommended that users test the integration of their conservative interpolation under these circumstances.

The ESMF_FieldRegridStore() call does not return an error when the regrid method is bilinear or patch and an input Field is built on a Grid or Mesh that contains a degenerate cell (i.e. one that collapses to a line or a point).

The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreate1PeriDim()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.

The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution will give incorrect results when used on a Grid with 3 or more dimensions and whose coordinate arrays are less than the full dimension of the Grid (i.e. it contains factorized coordinates).

The ESMF_MeshCreate() interface that allows the user to to create a copy of an existing Mesh, but with a different distribution will return an error if the element distribution is not specified.

The ESMF_AttributeGet() interface can return incorrect values when using the ESMF_AttGetCountFlag set to ESMF_ATTGETCOUNT_ATTRIBUTE.

The ESMF_AttributeUpdate() method does not work correctly for the following situations:

When a small number of changes is made to a large Attribute package a failure can occur with a "buffer too short" log message.

A Grid shared by multiple Fields may not have all Grid attributes updated on all PETs.

A State that is reconciled more than once to Components on exclusive petLists will not be updated on any but the first Component.

Any Attribute that is added during runtime and then modified will not be updated correctly.

ESMF_AttributeWrite() only works for ESMF standard Attribute packages.

When calling the ESMF_AttributeGet() method to query a character string attribute, a debugging print statement is triggered inside of the ESMF library. This can lead to a problematic volume of output when ESMF_AttributeGet() is called many times (e.g. from within a loop).

The intent of the "pathName" argument in the ESMF_UtilIOGetCWD() method is incorrectly specified as intent(in). This makes the method undependable.

The unit tests and examples fail to compile when build option ESMF_ARRAY_LITE is set to TRUE.

The ESMF_GridCreate() call, which allows the user to create a Grid from a SCRIP format file and a DistGrid, no longer creates the Grid using the wrong coordSys (Cartesian instead of spherical). A Grid created with this call will now work fine in methods (e.g. ESMF_FieldRegridStore()) which use the coordSys to determine the shape of the Grid.

The instability that led to undefined behavior (e.g. segmentation fault) for all the sparse matrix store calls (i.e., ESMF_FieldBundleSMMStore(), ESMF_FieldSMMStore(), ESMF_ArrayBundleSMMStore(), and ESMF_ArraySMMStore()) for very large problem sizes (~15 million factors in the sparse matrix) has been fixed.

The decomposition scheme of the original DistGrid (indicated by the ESMF_Decomp_Flag) is now correctly duplicated by the ESMF_DistGridCreate() methods that take a DistGrid input argument.

This release is backward compatible with ESMF 6.1.1 for all the interfaces that were previously marked as backward compatible. There were API changes to unmarked methods, and new methods were added. All of these changes are backward compatible with ESMF 6.1.1. The entire list of API changes is summarized in a table showing interface changes since ESMF_6_1_1, including the rationale and impact for each change.

Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.

This release contains a reference version of the NUOPC Layer software. The NUOPC Layer provides generic components that represent Drivers, Models, Mediators, and Connectors. These pre-fabricated, customizable components allow users to easily build interoperable ESMF applications. New web pages, a Reference Manual, and many examples are available to help users learn about the software.

Specific technical improvements to the NUOPC Layer in this release include:

The default entries in the Field Dictionary were reduced to those entries that are compliant with the Climate and Forecast (CF) Convention. It is the intention to grow the number of default entries over time, based on a clear relationship with CF.

Support for dynamic resolution of data-dependencies between components during initialization was added.

Support for compatibility checking of components that interact on multiple timescales was added.

The capability of retrieving coordinates from an unstructured Mesh object created from a NetCDF file in SCRIP or UGRID format was added to the ESMF Python interface. More information about the Python interface is available at the ESMP web page.

Support to write out weight files in the NetCDF4 format was added to the ESMF_RegridWeightGen application and the ESMF_RegridWeightGen() interface.

The ESMF_RegridWeightGen application and the ESMF_MeshCreate() interface have been extended to support 3D unstructured grids provided in the UGRID convention.

The ESMF_FieldRegridStore() call has been enhanced so that now regridding takes into account Mesh node masking. This will allow the user to mask out portions of an unstructured Mesh.

The user can now get the coordinate system that a Grid was created on from the ESMF_GridGet() interface.

A new ESMF_MeshCreate() interface was added. This interface allows a user to create a copy of an existing Mesh, but with a different distribution of the nodes and elements from that of the original Mesh.

The ESMF_MeshCreate() and ESMF_MeshAddNodes() methods have been extended to allow the user to optionally set node masks.

The ESMF_UtilIOMkDir() interface was added to allow creation of a directory in the file system. The complementary ESMF_UtilIORmDir() interface was also added that allows removal of an empty directory.

A new demonstration program, ESMF_Regrid, was added to the ESMF external_demos repository. This program demonstrates regridding between Fields built on Grids created from GRIDSPEC formatted files.

Same as ESMF_6_1_1 with the following exceptions:

The ESMF_RegridWeightGen application now correctly outputs the grid mask array in the output weight file when either input grid is supplied following the UGRID convention.

The ESMF_RegridWeightGen application and interface no longer fail if an input grid contains a polygon with more than 20 sides. The allocation is now done dynamically.

The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution now gives correct results if the Grid contains items (e.g. areas or mask values). The new Grid contains correctly redistributed versions of the original Grid items.

The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution now correctly works if the Grid contains stagger locations other than ESMF_STAGGERLOC_CENTER, ESMF_STAGGERLOC_EDGE1, and ESMF_STAGGERLOC_EDGE2. The new Grid will contain correctly redistributed versions of the original Grid stagger locations.

ESMF_AttributeUpdate() now works correctly even if the only changes have been Attribute removals.

The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution will give incorrect results when used on a Grid with 3 or more dimensions and whose coordinate arrays are less than the full dimension of the Grid (i.e. it contains factorized coordinates).

The code that allows the compliance checker to report memory usage on Linux is not thread-safe. The ESMF_ComplianceCheckerSTest.F90 system test fails or hangs on Linux systems when ESMF_TESTWITHTHREADS is set to ON.

This release is backward compatible with ESMF 6.1.0. There are no API changes.

The regridding capabilities in this release are identical to ESMF 6.1.0. Tables summarizing the ESMF regridding status are available. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.

Same as ESMF_6_1_0 with the following exceptions:

The bug in the NUOPC Layer that lead to an "Input argument out of range" error during Component initialization has been fixed. Full functionality of the NUOPC Layer has been restored.

This release is backward compatible with ESMF 5.3.0 for all the interfaces that were previously marked to remain backward compatible in future ESMF releases. However, there were API changes to unmarked methods that may require user code modifications. In FieldBundle, Field, ArrayBundle, and Array, changes were made to the Read/Write methods as a consequence of reworking the I/O capabilities. In Grid, an unmarked method was removed as obsolete. In XGrid, several unmarked methods were modified to provide a cleaner, more general interface. In Attribute the convention and purpose of a few Attribute packages were updated. In the NUOPC Layer, there were changes in the initialize phase and run sequence definitions. The entire list of API changes is summarized in a table showing interface changes since ESMF_5_3_0, including the rationale for each change.

Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.

The capability of creating an unstructured Mesh object from a NetCDF file in SCRIP or UGRID format was added to the ESMF Python interface. More information about the Python interface is available at the ESMP web page.

The NUOPC_Model component now supports nesting. Parent and nested domains are separate instances of the same model component, running as siblings under the same NUOPC_Driver component.

A Science Component class, named ESMF_SciComp, has been implemented. The ESMF_SciComp is intended to represent scientific domains, or realms, in an Earth Science Model. Its primary purpose is to provide a means for representing Component metadata within a hierarchy of Components by functioning as a container for Attributes as well as other Components.

The implementation of the ESMF_StateReconcile() method was redesigned to improve performance, and scalability. There were no changes to the user interface.

A new option was introduced to the container Get() methods to support the retrieval of container elements in the order in which they were added, instead of returning them in the default alphabetical order. All container classes (ESMF_State, ESMF_FieldBundle, and ESMF_ArrayBundle) support this new functionality.

Added two new arguments to the ESMF_RegridWeightGen application: --src_coordinates and --dst_coordinates to allow users to choose the coordinate variables if multiple sets of coordinates are defined in a GRIDSPEC file.

The ESMF_FieldRegridStore() method, and the offline regrid weight generation application (ESMF_RegridWeightGen), have been extended to support two new nearest neighbor regridding methods:

The nearest source to destination method allows the user to map each destination point to the closest source point.

The nearest destination to source method allows the user to map each source point to the closest destination point.

The ESMF_FieldRegridStore() call has been extended to allow the user to obtain a list of unmapped destination points. This list is available via an optional argument passed into the method.

An option was added to the ESMF_FieldRegridStore() call to allow users to ignore degenerate cells when doing first-order conservative regridding. Cells are considered degenerate if enough corners are closer than 1E-15 degrees, and practically collapse into a line or point. Previously these cells caused the method to return with an error.

A new method was added to the library that supports regrid weight generation from grid files. The new interface is called ESMF_RegridWeightGen().

The capability to support bit-for-bit reproducibility was added to the sparse matrix multiply methods. This capability is also available for the internal sparse matrix multiply part of the regrid method, although the regrid weight generation part is not yet guaranteed to be bit-for-bit. Three levels of bit-for-bit reproducibility are supported: strict reproducibility across different number of PETs, relaxed reproducibility when the number of PETs does not change, and no guarantee of bit-for-bit reproducibility when performance is most important.

The interfaces and the implementation of the ESMF I/O methods for ESMF_Field, ESMF_FieldBundle, ESMF_Array, and ESMF_ArrayBundle were redesigned to be more consistent and complete. None of these interfaces have been marked as backward compatible yet. The interface changes are detailed in the table showing interface changes since ESMF_5_3_0.

The ESMF_GridCreate() method that creates Grids from SCRIP or GRIDSPEC files has been parallelized to improve performance.

XGrid creation has been extended to support flexible ways of supplying Grids and Meshes on either side of the XGrid. Weights generated by regridding through XGrid can be renormalized by user supplied source or destination area. The user can retrieve the area and centroid information from the XGrid.

An Attribute package implementing the METAFOR Common Information Model (CIM) of version 1.5.1 was added to allow a CIM 1.5.1 compliant XML file to be written from the internal information stored in an ESMF Grid.

The ESMF_AttributeWrite() method was modified and is now a collective call. This means that all PETs that are part of the current VM must issue this call.

The ESMF_UtilSort() utility was added for sorting arrays of data. Currently supported data types include: character string, integer, long integer, real, and double precision.

Same as ESMF_5_3_0 with the following exceptions:

Added a small tolerance (1E-10 degrees) when determining whether two corner coordinates are identical in a SCRIP unstructured grid file. This allows the ESMF_RegridWeightGen application to correctly handle SCRIP unstructured grid files that have small round off errors in the corner coordinate arrays.

The instability that led to undefined behavior (e.g. segmentation fault) for all the sparse matrix store calls (i.e., ESMF_FieldBundleSMMStore(), ESMF_FieldSMMStore(), ESMF_ArrayBundleSMMStore(), and ESMF_ArraySMMStore()) for very large problem sizes (~15 million factors in the sparse matrix) has been fixed.

The decomposition scheme of the original DistGrid (indicated by the ESMF_Decomp_Flag) is now correctly duplicated by the ESMF_DistGridCreate() methods that take a DistGrid input argument.

A bug in the NUOPC Layer leads to an "Input argument out of range" error during Component initialization. This issue renders the NUOPC Layer practically nonfunctional in this ESMF release.

When the ESMF_FieldRegridStore() call or the ESMF_RegridWeightGen application is used with nearest destination to source method, the unmapped destination point detection does not work. Even if the option is set to return an error for unmapped destination points (the default) no error will be returned.

This release is backward compatible with ESMF 5.2.0rp2 for all the interfaces that were previously marked to remain backward compatible in future ESMF releases. However, there were API changes to unmarked methods in Mesh and Attachable Methods. These changes require user code modifications unless argument keyword syntax was used. The entire list of API changes is summarized in a table showing interface changes since ESMF_5_2_0rp2, including the rationale for each change.

Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.

A Python interface to ESMF, currently called ESMP, has been implemented. It is
available for download at the ESMP
web page. ESMP provides support for Grids, Meshes, Fields, and Field regridding.
The regridding capability includes bilinear, higher-order patch recovery and first-order
conservative regridding methods and the choice to ignore unmapped points. There is support
for applying masking or user areas to the Grid or Mesh, and retrieving areas or fractions
from a Field. There is also limited access to the ESMF VM and Log capability through ESMP.

A fault-tolerant extension to the ESMF_GridComp and ESMF_CplComp interfaces has
been implemented. This extension allows ESMF Components, running in separate executables,
to be linked via a socket based connection. A user controlled "timeout" mechanism allows
either side of this link to continue with normal execution even after the other side of the
connection has become unresponsive. The practical use cases of this new feature are large
ensemble configurations where individual ensemble members may crash during the course of a
simulation run. The fault-tolerant Component mechanism allows the ensemble driver, under
these circumstances, to complete the run in a controlled manner with the remaining ensemble
members. This mechanism was implemented by adding a "timeout" argument to ESMF_GridCompDestroy(),
ESMF_GridCompFinalize(), ESMF_GridCompInitialize(), ESMF_GridCompReadRestart(), ESMF_GridCompRun(),
ESMF_GridCompSetServices(), ESMF_GridCompWait(), and ESMF_GridCompWriteRestart(), as well
as the equivalent ESMF_CplComp methods. New methods ESMF_GridCompServiceLoop(), and
ESMF_CplCompServiceLoop() were added with the "timeout" argument.

The generic NUOPC_Driver component now supports petLists for all model and connector
components. This enables these components to be run concurrently if desired.

The generic NUOPC_Driver and NUOPC_Model components now support simple explicit,
semi-implicit, as well as complex implicit component run sequences.

The ESMF_StateReconcile() method can now be called repeatedly on the same State
during the course of execution. This can be used to re-reconcile the State after adding to
or removing objects from it.

Added an optional "index" argument to all of the ESMF_MethodAdd(), ESMF_MethodExecute(),
and ESMF_MethodRemove() calls to support multiple methods under the same "label". The
NUOPC layer utilizes this feature to implement phase-specific Component method specializations.

An ESMF_FieldCopy() method was added. This method copies data values from a source
Field to a destination Field. The underlying data arrays in the source and destination Fields
must have the same shape and distribution (i.e., their DistGrids must be identical).

The ESMF_FieldRegridStore() call has been extended to support first order conservative
regridding involving Fields built on 3D Grids or 3D Meshes. The 3D Meshes this capability will work on are ones composed of hexahedrons and tetrahedrons. (In contrast to this, bilinear interpolation is supported on 3D Meshes composed just of hexahedrons and higher-order patch interpolation isn't supported in 3D.)

The ESMF_FieldRegridStore() call has been enhanced so that now conservative
regridding takes into account Mesh cell masking. This will allow the user to mask out portions
of an unstructured Mesh.

The ESMF_FieldRegridStore() call has been extended so that now conservative regridding
is adjusted to use the cell areas set in the Grid or Mesh object upon which the source and
destination Fields are built. If these areas are set, the conservation will now hold for these
areas. If not set, the conservation will hold for the cell areas as calculated by ESMF.

The offline regrid weight generation application (ESMF_RegridWeightGen) now has
the capability to generate conservative weights for 3D unstructured Meshes specified in
the ESMF unstructured file format.

The ESMF_RegridWeightGen application allows users to construct a grid mask by using
the missing values of a data variable defined in the UGRID or the GRIDSPEC file.

A new command line option (--users_area) was added to the ESMF_RegridWeightGen
application. For conservative regridding this option causes the weight generation to use the
grid areas provided in the grid files instead of calculating the areas internally. If these
areas are read in, the conservation will now hold for these areas. If the option is not set,
the conservation will hold for the cell areas as calculated by ESMF.

An ESMF_ArrayCopy() method was added. This method takes two existing Array objects
and copies the data from one Array into the other, after verifying that both have the same
shape and distribution (i.e., the DistGrids must be identical).

The ESMF_GridCreate() method that supports creation of a Grid from a SCRIP format
NetCDF file was extended to support files using the CF GRIDSPEC convention. Options for user
area and masking were also added to the interface.

The ESMF_MeshCreate() and ESMF_MeshAddElements() methods have been extended to
allow the user to optionally set cell areas and cell masks.

The ESMF_MeshCreate() method that supports creation of a Mesh from a SCRIP format
or ESMF unstructured Mesh format NetCDF file was extended to support files using the CF
UGRID convention. Options for user area and masking were also added to the interface.

The ESMF_MeshCreate() method that supports creation of a Mesh from an ESMF
unstructured Mesh format NetCDF file was extended to support 3D unstructured Meshes.

The ESMF_XGridCreate() method was extended to support lists of Grids on either side
of the XGrid (exchange grid). The flux exchange calculated through the XGrid is first order conservative.
Masking and merging when creating an XGrid are not supported in this release.

The ability to access Grid object information through the Attribute class with string
valued input parameters was added to ESMF_AttributeGet(). This is an initial example
of how information can be drawn directly from data objects, and does not need to be entered by
the user.

Implemented the custom calendar kind, ESMF_CALKIND_CUSTOM, in Time Manager. This
can be used to define calendars for other planets, such as Mars, as well as custom Earth calendars.

Added support for Fortran logical data type in ESMF_VMGather() and ESMF_VMAllToAllV().

Added support for Fortran character array in ESMF_VMAllGatherV() and ESMF_VMAllToAllV().

Implemented ESMF_VMAllToAll() for real and integer data types.

Support for the Grid, Fields built on Grids and Field regridding was added to the C interface. There is support for applying masking or user areas to the Grid or Mesh, and retrieving areas or fractions from a Field.

By default, LAPACK dependencies are now satisfied using internal code. Linking against an external LAPACK library is no longer required to use the higher order patch interpolation.

The ESMF_StateReconcile() call overestimates the size requirement of an internal temporary buffer. The estimation error increases with core count and causes ESMF_StateReconcile() to run out of memory for large core counts (~10,000) on typical configurations.

FieldBundles don't currently enforce that every contained Field is built on the same Grid, Mesh, LocStream, or XGrid object, although the documentation says that this should be so.

ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids which contain a DE of width less than 2 elements.

ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids of arbitrary distribution.

ESMF_FieldRegridStore() that takes XGrid as an argument does not support regridding between Fields that are constructed on Grids for which ESMF_GridMatch() locally returns a value of ESMF_GRIDMATCH_EXACT or greater. This may result in unexpected local errors.

A larger than expected difference in integrals (8.7E-10) is occurring when using the conservative regridding between a 1 deg lat lon grid and a 2.5 deg lat lon grid. This problem appears isolated to a subset of the cases when grid cell edges align, but not exactly due to round off. It is recommended that users test the integration of their conservative interpolation under these circumstances.

The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreate1PeriDim()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.

The ESMF_RegridWeightGen application does not correctly handle SCRIP unstructured grid files that have small round off errors in the corner coordinate arrays. This problem leads to an abort with error about degenerated cells.

The ESMF_RegridWeightGen application does not correctly handle bilinear and patch regridding on SCRIP unstructured grid files containing concave cells. This problem leads to an abort with error about degenerated cells.

The ESMF_RegridWeightGen application does not output the grid mask array in the output weight file when either input grid is supplied following the UGRID convention. As a result, the output weight file produced under this condition does not work with the scrip_test utility program provided by the SCRIP package.

The ESMF regrid weight generation methods and applications can give wrong answers for the bilinear method on 3D Grids or 3D Meshes when a point in the destination falls very close to the edge of a source cell. In this case the destination point can sometimes be mapped to a wrong cell.

The ESMF regrid weight generation methods and applications take noticeably longer to generate regridding weights when going from a very high resolution grid to a coarse grid (e.g. a 2km global grid to a 7.5x1.875 degree global grid) than when going to another high resolution grid.

The ESMF_RegridWeightGen application and interface will fail with an error if an input grid contains a polygon with more than 20 sides.

All of the sparse matrix store calls (i.e., ESMF_FieldBundleSMMStore(), ESMF_FieldSMMStore(), ESMF_ArrayBundleSMMStore(), and ESMF_ArraySMMStore()) suffer from an instability that can lead to undefined behavior (e.g. segmentation fault) for very large problem sizes (~15 million factors in the sparse matrix).

The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution will give incorrect results if the Grid contains items (e.g. areas or mask values). The new Grid is created without items.

The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution will give incorrect results if the Grid contains stagger locations other than ESMF_STAGGERLOC_CENTER, ESMF_STAGGERLOC_EDGE1, and ESMF_STAGGERLOC_EDGE2. The new Grid will not contain any stagger locations other than the above.

The decomposition scheme of the original DistGrid (indicated by the ESMF_Decomp_Flag) is not duplicated by the ESMF_DistGridCreate() methods that take a DistGrid input argument. Instead, the duplicate DistGrid is always constructed using ESMF_DECOMP_BALANCED. This issue is most noticeable when building Grids on DistGrids with decomposition schemes other than the default ESMF_DECOMP_BALANCED. A Field built on such a Grid will incorrectly use a balanced decomposition.

ESMF_AttributeWrite() only works for ESMF standard Attribute packages.

ESMF_AttributeUpdate() will not work correctly if the only changes have been Attribute removals. In this situation the call will return success but no updating will have occurred. This situation can occur during the removal of Attribute packages as well as single Attributes.

Darwin with gfortran (Haumea): Fails the ESMF_CompTunnelUTest.F90 unit test with ESMF_COMM=mpich2 and ESMF_COMM=mvapich2.

Linux with Intel (Columbia): Fails the ESMF_FieldRegridUTest.F90 unit test with ESMF_COMM=mpi and mpiuni + ESMF_BOPT=O. It also fails the ESMF_FieldSphereRegridEx.F90 example with ESMF_COMM=mpi + ESMF_BOPT=O.

Linux with g95 (Pluto): Fails the ESMF_CompTunnelUTest.F90 unit test intermittently with ESMF_COMM=mvapich2 and mpich2. It also fails the ESMF_CompTunnelEx.F90 example intermittently with ESMF_COMM=mvapich2 + ESMF_BOPT=O.

Linux with gfortran (Pluto): Fails the ESMF_CompTunnelUTest.F90 unit test intermittently with ESMF_COMM=mpich2 and openmpi. It also fails the ESMF_CompTunnelEx.F90 example intermittently with ESMF_COMM=mpich2 + ESMF_BOPT=O.

This release is backward compatible with ESMF 5.2.0rp1. There are no API changes.

Extended ESMF_FieldRegridStore() and the offline regrid weight generation application (ESMF_RegridWeightGen) to support conservative regridding involving Fields built on Grids or Meshes which contain cells whose corners are in clockwise order.

This release is backward compatible with ESMF 5.2.0r. However, for four methods, it was found that issues with consistency were missed. A small number of argument deprecations were introduced in order to address this without breaking backward compatibility. The changes are summarized in a table showing interface changes since ESMF_5_2_0r, including the rationale for each change.

Tables summarizing the ESMF regridding status have been updated to reflect minor decreases in interpolation errors. These tables include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.

The Field and FieldComm API sections in the Fortran Reference Manual were consolidated into a single Field API section. Now all class API sections are structured this way.

Extended ESMF_FieldRegridStore() to support conservative regridding involving Fields built on Grids containing only a single cell.

Extended the offline regrid weight generation application (ESMF_RegridWeightGen) to support generation of conservative interpolation weights for grid files containing only a single cell.

The ESMF_RegridWeightGen application now supports conservative interpolation on Meshes which contain concave cells of five or more sides.

The "--64bit_offset" option was added to the ESMF_RegridWeightGen application. This option allows weight files to be generated in the NetCDF 64bit offset format. This option is available for ESMF built against NetCDF 3.6.0 or later.

The "--version" option was added to the ESMF_RegridWeightGen application. This option prints the ESMF license together with the exact ESMF version information.

XML output of Attributes in the
METAFOR Common Information Model (CIM) format can now
be done without the external library Xerces, by not setting the ESMF_XERCES environment variables
before building ESMF. See the "Third Party Libraries" section in the ESMF User's Guide for details.

Enabled the CIM "General Component Properties Description" Attribute package to hold one or more values for each Attribute name.

Added a new CIM "Scientific Properties Description" Attribute package which can hold any user-defined name/value sets.

Enhanced the CIM "General Component Properties Description" Attribute package to output the XML
attribute 'type="custom" ' as part of the CIM component property element, e.g. . This denotes it as a custom component property, to distinguish it from a scientific
property, which is written similarly, but without the 'type="custom" ' designation.

Concatenated the value set in Attribute "URL", part of the CIM Citation Attribute package, to the
value of Attribute "LongTitle" upon output to a CIM XML file. This is a temporary measure while waiting
for the CIM to have a separate spot for holding the URL value.

Created a new use test case, ESMF_AttributeCIM2ESG, to show how to set Attributes, and how
CIM XML output can be ingested into a gateway such as Earth System Grid (ESG).

Added automatically written comments to CIM XML generated by ESMF which document the sources of a ESMF CIM Attribute package/Attributes.

The convention specification for CIM Attribute packages is now just "CIM"; previously it was "CIM 1.5". This allows user code to remain unchanged as new ESMF/CIM versions are developed.

CIM XML output of Attributes validates against the CIM 1.7 release, as well as the previous CIM 1.5 release.

Same as ESMF_5_2_0r with the following exceptions:

The performance issue in ESMF_Finalize() that was most noticeable for applications with large numbers of DELayout objects (~1000 or more) has been fixed.

The API documentation for ESMF_StateAdd(), ESMF_StateAddReplace(), and ESMF_StateReplace() in the Fortran Reference Manual now correctly uses the "nestedStateList" keyword to match the actual implementation.

Calling ESMF_FieldEmptyComplete() on a Field object for which previously ESMF_FieldEmptySet() was called with a Grid no longer fails.

The ESMF_GridCreate1PeriDim() and ESMF_GridCreate2PeriDim() methods that create Grids with irregular distribution now correctly set up Grid connections. Consequently ESMF_FieldRegridStore() now correctly maps all destination points for these Grids.

The mistakes in section 29.3.5 of the Fortran Reference Manual, with the title "Create a Mesh from a SCRIP Grid file or an ESMF unstructured Grid file", have been corrected.

ESMF_FieldRegridStore() now works on Fields created on Grids with indexflag equal to ESMF_INDEX_DELOCAL. This is the default.

The ESMF_FieldRegridStore() no longer fails with unmapped destination points if the source Field is built on a Mesh or Grid which contains a quadrilateral cell in which particular pairs of nodes are identical so that the quadrilateral collapses to a triangle.

The ESMF_FieldRegridStore() no longer uses coordinates in the masked region. Errors in the coordinates in a masked region will not lead to ESMF_FieldRegridStore() returning with errors.

ESMF_FieldRegridStore() now returns with error for Fields built on a Mesh containing clockwise elements or a Grid with a left handed coordinate system.

The ESMF_RegridWeightGen application now works correctly when source or destination SCRIP file contains less than 2 cells in any dimension.

Running the ESMF_RegridWeightGen application on enough processors that the decomposition of the destination Mesh does not produce entries on all processors now works. This bug is also fixed when calling ESMF_FieldRegridStore().

The problem in the ESMF_RegridWeightGen application which lead to incorrect interpolation weights when using non-conservative interpolation on an unstructured SCRIP format file with the following conditions has been fixed. The problem occured if a corner node with coordinates (lon, lat) in the SCRIP file followed another corner with the same lon, but only strictly smaller lats, while at the same time another corner (lon2, lat2) had already been read where lat=lat2, but lon2 was the next larger longitude after lon that had been in the file so far.

The ESMF_RegridWeigthGen application now generates correct weights for the case where either the source or the destination grid is in the ESMF unstructured grid format and the node coordinate unit is "radians".

The interpolation weight file generated by the ESMF_RegridWeigthGen application now has correct dimension and values for the center coordinate variables (i.e., xc_a, yc_a or xc_b, yc_b) for the case where the source/destination grid file is in the ESMF unstructured file format and the regridding method is either bilinear or patch.

Using the ESMF_RegridWeightGen application on a SCRIP unstructured grid which has an element with one of the nodes on the North Pole (i.e. the latitude of the node is 90 degree north) now works.

The weight file produced by the ESMF_RegridWeightGen application no longer contains multiple entries for the same source and destination pair. (This problem did not affect the accuracy of the regridding, but it did increase the file size.)

The "src_grid_dims" and "dst_grid_dims" variables in the SCRIP weight file generated by ESMF_RegridWeightGen are now correctly set to the "elementCount" value of the source or destination input grid file, respectively.

Halo updates are now correct for Fields, FieldBundles, Arrays, and ArrayBundles with more than one DE per PET.

For the CIM "Inputs Description" Attribute package, values set for the CouplingSource and CouplingTarget Attributes now appear correctly in ESG.

The API documentation for the ESMF_ATTGETCOUNT_FLAG in the Fortran Reference Manual now correctly describes the purpose of this flag.

The previously ambiguous API documentation for ESMF_TimeSet() in the Fortran Reference Manual now documents which arguments are allowed for Gregorian and Julian calendars.

The default MPI tags used by the ESMF VM layer are now bounded by the largest supported MPI tag. This allows calls like ESMF_ArrayRedistStore() to execute on large processor counts (~16K and larger).

The inconsistency in the internal use of the ESMF Attribute API in the NUOPC addon layer that lead to failure of correct NUOPC user code has been fixed.

The ESMF install target now succeeds following a failed attempt to build the ESMF documentation for the ESMF source tarball.

The "ESMF_NETCDF_INCLUDE" environment variable now is correctly documented in the NETCDF section of the ESMF User's Guide.

The majority of the interfaces in established ESMF classes (e.g. Component, State, Field, Grid) will be backward compatible starting with this release. Interfaces that are backward compatible indicate this in the "STATUS:" section of their API description in the ESMF Reference Manual. Throughout the framework, interfaces were changed to improve consistency. Major system-wide changes are noted in other items below.
All changes since ESMF_5_1_0 are summarized in two tables including the rationale for and the impact of each change:

Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.

In order to be more consistent with other software packages, the access to the ESMF Fortran language binding has switched from "use ESMF_Mod" to "use ESMF".

Added "attachable methods" to ESMF Components. This allows users to attach other than standard Component methods to a Component.

Added tools that allow component developers to make their components available as web-based services. A client can access these components through provided Simple Object Access Protocol (SOAP) interfaces.

State "placeholder names" have been removed. This functionality had become redundant with the ability to create empty Fields and FieldBundles.

ESMF State, FieldBundle and ArrayBundle classes are now based on a unified and more efficient container class implementation. As a consequence the user experiences consistent ESMF container class interfaces and behavior.

Field partial creation now allows an intermediate state in which the underlying grid is specified. The new interfaces are: ESMF_FieldEmptyCreate(), ESMF_FieldEmptySet() and ESMF_FieldEmptyComplete().

An argument which allows the user to specify a coordinate system when creating a Grid has been added to the Grid create interfaces. This information is used by the ESMF_FieldRegridStore() call to automatically determine the geometry of the source and destination Grids.

Added new shortcut Grid interfaces broken up by the number of periodic dimensions. These allow the user to specify connection information when creating a Grid.

The connection information stored inside the Grid is now used by the ESMF_FieldRegridStore() call to automatically determine the topology of the source and destination Grids.

XGrids can now be created by providing Grids for both sides without needing to specify sparse matrix parameters.

Added the ability to get Attributes package information by name.

Added the ability to query for the presence of Attributes.

The Attribute packages that implement METAFOR's Common Information Model (CIM) now allow more complete ingestion into Earth System Grid (ESG). This includes the CIM 1.5 "Inputs Description" package.

The time spent in ESMF_Finalize() increases super-linearly with the number of DELayout objects. DELayout objects are created by the default method of creating Arrays and Fields.

The API documentation for ESMF_StateAdd(), ESMF_StateAddReplace(), and ESMF_StateReplace() in the Fortran Reference Manual incorrectly uses the "stateList" keyword. The actual implementation of these calls uses the "nestedStateList" keyword for this argument.

The ESMF_StateRemove() method fails to correctly unlink Attributes from the Attribute hierarchy.

FieldBundles don't currently enforce that every contained Field is built on the same Grid, although the documentation says that this should be so.

Calling ESMF_FieldEmptyComplete() on a Field object with a user defined Fortran array pointer for which previously ESMF_FieldEmptySet() was called with a Grid fails with an error, while it should return success.

The ESMF_GridCreate1PeriDim() and ESMF_GridCreate2PeriDim() methods that create Grids with irregular distribution fail to set up correct Grid connections. Consequently ESMF_FieldRegridStore() fails unexpectedly with unmapped destination points for these Grids.

The ESMF_GridCreate() call, which allows the user to create a Grid from a SCRIP format file and a DistGrid, creates the Grid using the wrong coordSys. Instead of using spherical coordinates (which are implied by the SCRIP format), the call creates the Grid with Cartesian coordinates. This can cause unexpected problems in methods (e.g. ESMF_FieldRegridStore() which use the coordSys to determine the shape of the Grid.

The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreate1PeriDim()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.

There are mistakes in section 29.3.5 of the Fortran Reference Manual, with the title "Create a Mesh from a SCRIP Grid file or an ESMF unstructured Grid file", that could be misleading. In the SCRIP file header example the "grid_imask" variable should be of type "int" instead of "double". In the ESMF unstructured grid file example the "units" attribute of the "nodeCoords" variable should be "degrees" instead of "degrees,degrees". Further, in the same example, variables "numElement" and "numNode" should be "elementCount" and "nodeCount", respectively.

ESMF_FieldRegridStore() doesn't currently work on Fields created on Meshes which contain tetrahedra.

ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids which contain a DE of width less than 2 elements.

ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids of arbitrary distribution.

ESMF_FieldRegridStore() only works on Fields created on Grids with indexflag equal to ESMF_INDEX_GLOBAL. This is not the default.

Masking is currently not supported for unstructured grids (Mesh) for any regridding method.

ESMF_FieldRegridStore() that takes XGrid as an argument does not support regridding between Fields that are constructed on Grids for which ESMF_GridMatch() locally returns a value of ESMF_GRIDMATCH_EXACT or greater. This may result in unexpected local errors.

A larger than expected difference in integrals (8.7E-10) is occurring when using the conservative regridding between a 1 deg lat lon grid and a 2.5 deg lat lon grid. This problem appears isolated to a subset of the cases when grid cell edges align, but not exactly due to round off. It is recommended that users test the integration of their conservative interpolation under these circumstances.

The ESMF_FieldRegridStore() call fails unexpectedly with unmapped destination points if the source Field is built on a Mesh or Grid which contains a quadrilateral cell in which particular pairs of nodes are identical so that the quadrilateral collapses to a triangle.

The ESMF_FieldRegridStore() mistakenly uses coordinates in the masked region. Errors in the coordinates in a masked region will lead to ESMF_FieldRegridStore() returning unexpectedly with errors.

ESMF_FieldRegridStore() will - without warning or error - produce wrong interpolation weights for Fields built on a Mesh containing clockwise elements or a Grid with a left handed coordinate system.

The ESMF_RegridWeightGen application crashes when source or destination SCRIP file contains less than 2 cells in any dimension.

Running the ESMF_RegridWeightGen application on enough processors that the decomposition of the destination Mesh does not produce entries on all processors leads to undefined behavior (segmentation fault). This bug will also occur when calling ESMF_FieldRegridStore() with the indices or weights arguments.

There is problem in the ESMF_RegridWeightGen application which can lead to incorrect interpolation weights when using non-conservative interpolation on an unstructured SCRIP format file. The problem occurs if a corner node with coordinates (lon, lat) in the SCRIP file follows another corner with the same lon, but only strictly smaller lats, while at the same time another corner (lon2, lat2) has already been read where lat=lat2, but lon2 is the next larger longitude after lon that has been in the file so far.

The ESMF_RegridWeigthGen application does not generate correct weights if either the source or the destination grid is in the ESMF unstructured grid format and the node coordinate unit is "radians".

The interpolation weight file generated by the ESMF_RegridWeigthGen application has wrong dimension and values for the center coordinate variables (i.e., xc_a, yc_a or xc_b, yc_b) if the source/destination grid file is in the ESMF unstructured file format and the regridding method is either bilinear or patch.

Using the ESMF_RegridWeightGen application on a SCRIP unstructured grid which has an element with one of the nodes on the North Pole (i.e. the latitude of the node is 90 degree north) results in wrong interpolation weights or other unexpected behavior, such as a segmentation fault.

The weight file produced by the ESMF_RegridWeightGen application may contain multiple entries for the same source and destination pair. This does not affect the accuracy of the regridding, but it does increase the file size.

The "src_grid_dims" and "dst_grid_dims" variables in the SCRIP weight file generated by ESMF_RegridWeightGen are incorrectly set to 1 if the source or destination input grid files are in the ESMF unstructured file format, respectively.

Halo updates are incorrect for Fields, FieldBundles, Arrays, and ArrayBundles with more than one DE per PET.

All of the sparse matrix store calls (i.e., ESMF_FieldBundleSMMStore(), ESMF_FieldSMMStore(), ESMF_ArrayBundleSMMStore(), and ESMF_ArraySMMStore()) will fail with an error for sparse matrices that contain multiple entries for the same source and destination pair when there are undistributed dimension in the source or destination array.

All of the sparse matrix store calls (i.e., ESMF_FieldBundleSMMStore(), ESMF_FieldSMMStore(), ESMF_ArrayBundleSMMStore(), and ESMF_ArraySMMStore()) suffer from an instability that can lead to undefined behavior (e.g. segmentation fault) for very large problem sizes (~15 million factors in the sparse matrix).

The decomposition scheme of the original DistGrid (indicated by the ESMF_Decomp_Flag) is not duplicated by the DistGridCreate() methods that take a DistGrid input argument. Instead, the duplicate DistGrid is always constructed using ESMF_DECOMP_BALANCED. This issue is most noticeable when building Grids on DistGrids with decomposition schemes other than the default ESMF_DECOMP_BALANCED. A Field built on such a Grid will incorrectly use a balanced decomposition.

ESMF_AttributeWrite() only works for ESMF standard Attribute packages.

For the CIM "Inputs Description" Attribute package, values set for the CouplingSource and CouplingTarget Attributes appear incorrectly in ESG.

ESMF_AttributeUpdate() will not work correctly if the only changes have been Attribute removals. In this situation the call will return success but no updating will have occurred. This situation can occur during the removal of Attribute packages as well as single Attributes.

The API documentation for the ESMF_ATTGETCOUNT_FLAG in the Fortran Reference Manual incorrectly uses the description for the ESMF_ATTTREE_FLAG.

The API documentation for ESMF_TimeSet() in the Fortran Reference Manual is ambiguous about allowing the updating of a Gregorian calendar object using Julian date arguments.

The default MPI tags used by the ESMF VM layer are unbounded. This leads to MPI errors on large processor counts (~16K), e.g. for ESMF_ArrayRedistStore().

An inconsistency in the internal use of the ESMF Attribute API in the NUOPC addon layer leads to unexpected failure of correct NUOPC user code.

The ESMF install target fails following a failed attempt to build the ESMF documentation for the ESMF source tarball.

The "ESMF_NETCDF_INCLUDE" environment variable is incorrectly documented as "ESMF_NETLIB_INCLUDE" in the NETCDF section of the ESMF User's Guide.

Cray with Intel-12.0.x: The ESMF library builds successfully, but applications fail to link.

Cray with PGI: Linker flags in the esmf.mk require that the same PGI version is used during application build as was used for the ESMF installation. According to Cray and PGI, this restriction is not necessary.

An intensive effort to make interfaces consistent in preparation for future backward compatibility resulted in significant interface changes for this release.
These are summarized in a table showing interface changes since ESMF_5_1_0, including the rationale for and the impact of each change.

Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.

Added ESMF_StateReplace() method, which replaces an existing State item with a new item of the same name.

Added ESMF_StateRemove() method, which removes an item from a State.

Masking is now supported for conservative interpolation, for logically rectangular grids, for both the offline regrid weight generation application and the ESMF_FieldRegridStore() method.

Support was added for regridding a regional Grid on a sphere. This allows regridding between two regional grids on a sphere, between a regional Grid on a sphere and spherical global Grid and between a spherical global Grid and a regional Grid on a sphere. This capability is available in both the offline regrid weight generation application and ESMF_FieldRegridStore().

Enabled a Grid constructed using DistGrid connections to work with ESMF_FieldRegridStore(). For example, this means that a user constructed tripole Grid can now participate in a regrid operation.

Optimized ESMF_FieldBundleRegridStore() to be faster when Fields are built on the same Grid. When computing the sparse matrices for the regridding, if the Grids in the source and destination Fields match the Grids in the previous set of source and destination Fields in their respective FieldBundle lists, then that same sparse matrix is used instead of recomputing a new one. In particular, this means that if all the Fields in the source FieldBundle are built on the same Grid and all the Fields in the destination FieldBundle are built on the same Grid, then the sparse matrix between those Grids only needs to be computed once. This can save a substantial amount of time for FieldBundles containing many Fields.

Support for options from ESMF_FieldRegridStore() were added to ESMF_FieldBundleRegridStore(). Masking, pole options and the ability to chose what happens when destination points can't be mapped were added to the ESMF_FieldBundleRegridStore() interface.

The Attribute values within the ESG/General and CF/Extended standard packages, nested within the standard CIM 1.0/Inputs Description package for Fields, now appear in the Component Properties section of the CIM XML output file. This allows input and output Fields to be described for components other than the top level component in CIM output files.

The ESMF Coupled Flow demonstration program has been updated to work with current interfaces. It is now part of the external demos module.

Added ESMF_UtilGetArgC() subroutine, which returns the number of command line arguments.

Added ESMF_UtilGetArg() subroutine, which returns a given command line argument.

Added ESMF_UtilGetArgIndex() subroutine, which searches for a specified command line argument (e.g., '-filename') and, if found, returns its index.

An online tutorial based on the Coupled Flow demonstration program was added to the ESMF website.

The prototype Exchange Grid implementation has been extended to work with multiple PETS.

Same as ESMF_5_1_0 with the following exceptions:

The argument attPackInstanceName, introduced in ESMF_5_1_0, in ESMF_AttributeGet(), ESMF_AttributeSet(), and ESMF_AttributeRemove() is now implemented, supporting multiple instances of Attribute packages as described in the release notes.

The coordinate unit attribute is now read correctly on all machines, so grid files with coordinates in radians will be interpreted correctly.

Masking is now supported for conservative regridding.

The Attribute Table in Section 33.2.6 of the pdf version of the Fortran Reference Manual is not complete, please refer to HTML version of the document for a complete table. This table was in Section 34.2.6 in release ESMF_5_1_0.

A larger than expected difference in integrals (8.7E-10) is occurring when using the conservative regridding between a 1 deg lat lon grid and a 2.5 deg lat lon grid. This problem appears isolated to a subset of the cases when grid cell edges align, but not exactly due to round off. It is recommended that users test the integration of their conservative interpolation under these circumstances.

The ESMF_FieldRegridStore() call or the ESMF_RegridWeigthGen application may erroneously report destination points as unmapped when running on multiple processors.

The matrix coming out of ESMF_FieldRegridStore() needs to be transposed before it can be used in ESMF_FieldSMMStore().

The weight file produced by the ESMF_RegridWeightGen application may contain multiple entries for the same source and destination pair. This does not affect the accuracy of the regridding, but it does increase the file size.

An integrated ESMF compliance checker has been implemented. It can be activated by setting the runtime environment variable ESMF_RUNTIME_COMPLIANCECHECK=ON. The compliance checker is currently geared toward the NUOPC requirements. The current version is described in more detail in the "NUOPC Layer: ESMF Compliance Checker - ESMF v5.1.0" document.

Tables summarizing the ESMF regridding status have been compiled. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.

ESMF_StateGet() has been enhanced to allow additional nested State capabilities. These include:

Items can now be searched by name using the itemSearch argument.

The nestedFlag argument controls whether the search traverses the entire State hierarchy.

State items can be accessed directly by using a path style naming convention.

The methods ESMF_StateSetNeeded(), ESMF_StateIsNeeded(), and ESMF_StateGetNeeded() that were removed in release ESMF_5_0_0 have been reinstated. This was necessary to reenable the ESMF COUPLED_FLOW demo.

The first-order conservative regridding algorithm has been reworked. This new version is based on the overlap of source cells with destination cells, and is less prone to interpolation artifacts and diffusion than the previous version. It is supported for regridding between any combination of Fields built on the center-stagger of 2D logically rectangular Grids and Fields built on the element location of 2D unstructured Meshes. Grids and Meshes can be either regional or mapped to the sphere (using the ESMF_REGRID_SCHEME_FULL3D option to ESMF_FieldRegridStore()). This functionality is available either offline (via the ESMF_RegridWeightGen application) or through the library (via ESMF_FieldRegridStore()).

The ESMF_FieldRegridStore(), ESMF_FieldRegrid() and ESMF_FieldRegridRelease() methods have been overloaded to support Fields created on XGrids.

The two offline weight generation applications ESMC_RegridWgtGenEx and ESMF_CubedSphereRegridEx have been merged into a single application named ESMF_RegridWeightGen. The application is documented under the "Applications" section of the ESMF Reference Manual. This application generates interpolation weights using either bilinear, finite element patch recovery or first order conservation methods. The input grids can be either 2D global logically rectangular or 2D global unstructured.

The new location argument in ESMF_FieldCreate() allows Fields to be created on Mesh elements or Mesh nodes. Conservative regridding is only supported between Fields created on Mesh elements.

Added ESMF_FieldCreate() method that supports creation of a Field on an XGrid.

ESMF_FieldCreate() and ESMF_ArrayCreate() have been overloaded to support typekind and rank arguments in addition to the ArraySpec versions.

Implemented halo support for arbitrarily decomposed FIeld and Array objects.

This release includes new parallel I/O capabilities introduced by using the Parallel I/O (PIO) library within ESMF. PIO development was led by John Dennis of NCAR and it has evolved under a NCAR/DOE laboratory partnership. ESMF data classes Field, FieldBundle, Array and ArrayBundle can now read and write NetCDF files in parallel using methods such as ESMF_FieldRead(), and ESMF_FieldWrite(). Please see the "IO Capabilties" section in the ESMF Reference Manual for how to enable PIO in the ESMF build.

A first implementation of the Exchange Grid (XGrid) class has been added. An XGrid object can be created from user supplied lists of Grids and sparse matrix parameters. The functionality of ths initial implementation is currently limited to single PET execution. This is not a limitation of the XGrid design but reflects the current implementation status.

Added standard ESMF-supplied Attribute packages for Components and Fields, whose XML output conforms to the Climate and Forecast Conventions (CF) and METAFOR's Common Information Model (CIM) schema. The output is ingestable into the Earth System Grid (ESG) web portal for display. For details and examples please see the Attribute section of the ESMF Reference Manual.

The new msgAllow argument in ESMF_LogSet() permits users to specify the type of Log messages allowed for output. Other Log messages are ignored.

The ESMF COUPLED_FLOW demo application has been restored but is currently limited to single PET execution. The documentation in the User's Guide has been updated.

Starting with this release a collection of external demonstration programs is available in form of a separate CVS module. These external demos have been written to serve as good examples to show how users would interact with an ESMF installation. A description of the available demos and download instructions are available on the ESMF website under Users->Code Examples->External Demos.

Added ESMF_ComplianceCheckerSTest.F90 system test which demonstrates the operation and output of the compliance checker within a multi-Component, multi-PET application.

Added ESMF_AttributeCIMSTest.F90 system test which demonstrates the use of the new standard ESMF-supplied, METAFOR CIM Attribute packages for Components and Fields, within a multi-Component, multi-PET application.

C interfaces:

ESMF now offers a Reference Manual for the ESMF C API. This document is a work in progress.

The available C interfaces are a subset of the Fortran API. This subset was constructed to fullfil the need of a specific coupling project.

Changes were made to improve interface consistency.

Same as ESMF_5_0_0 with the following exceptions:

The Field description in the reference manual now correctly states that the halo update operation is supported.

Conservative regridding has been reworked so that its accuracy is now within the expected range (15 decimal places).

The offline regrid weight generation application, ESMF_RegridWeightGen, only works for 2D global grids.

The offline regrid weight generation application, ESMF_RegridWeightGen, does not support masking, and thus any mask information in the grid files will be ignored.

On some machines (e.g. bluefire), the coordinate unit attribute is not read correctly by the ESMF_RegridWeightGen application, so grid files with coordinates in radians won't be interpreted correctly and will produce incorrect weights or errors.

Masking is currently not supported for conservative regridding.

Masking is currently not supported for unstructured grids (Mesh) for any regridding method.

ESMF_ArrayWrite() and ESMF_FieldWrite() will fail with an internal PIO error if a NetCDF output file already exists from a previous run.

XGrid does not support regridding between Fields that are constructed on Grids for which ESMF_GridMatch() locally returns a value of true. This may result in unexpected local errors.

Fields built on XGrids cannot be added to FieldBundles.

Only single-PET XGrids are supported. Creating and using multi-PET XGrids will result in undefined behavior.

ESMF_AttributeUpdate() will not work correctly if the only changes have been Attribute removals. In this situation the call will return success but no updating will have occurred. This situation can occur during the removal of Attribute packages as well as single Attributes.

The new argument attPackInstanceName in ESMF_AttributeGet(), ESMF_AttributeSet(), and ESMF_AttributeRemove() is not fully implemented. Consequently multiple instances of an Attribute package of the same type (same convention and purpose) within a nested Attribute package are not supported.

The Attribute Table in Section 34.2.6 of the pdf version of the Fortran Reference Manual is not complete, please refer to HTML version of the document for a complete table.

Tables summarizing the ESMF regridding status have been compiled. These include supported grids and capabilities in the offline and online regridding and numerical results of some specific test cases.

ESMF_StatePrint() has a new nestedFlag argument for printing nested States. The options argument now supports 'short' and 'long' printouts. (Default is short.)

ESMF_StateValidate() has a new nestedFlag argument for validating nested States.

ESMF_StateGet() has a new nestedFlag argument for inquiry requests, which allows returning lists of nested objects and their types.

The following methods were removed:
ESMF_StateSetNeeded(), ESMF_StateIsNeeded(), and ESMF_StateGetNeeded().
The reason for removal is that the semantics surrounding these routines
were not well-defined, and they were rarely used. These methods may be revisited as part of an ongoing investigation into broader interoperability issues.

Added regridding support for FieldBundle objects. First order conservative regridding has been added as a modification to the existing Field regridding methods in both the offline and online and regridding. This modification is based on a finite element L2 projection method. In this initial version quantities are conserved globally. Further work is required for satisfactory local conservation.

Improved the algorithm in ESMF_FieldRegridStore() to fix problems with interpolation on a sphere. The change fixes incidents of destination points that are lying close to the boundary between two source elements from being mapped into the wrong element. This led to errors in the interpolation weights.

The offline weight generation application for structured grids has been more accurately renamed to ESMC_RegridWgtGenEx. The bilinear interpolation option, as well as a conservative option was added. For details please see the "Structured Grid to Structured Grid" subsection under the "File Based Regrid Weight Applications" section in the "Infrastructure: Fields and Grids" part of the ESMF Reference Manual.

Added a new offline weight generation application called ESMF_CubedSphereRegridEx. This application supports regridding between a cubed sphere and a 2D logically rectangular grid mapped to a sphere. The user may use higher-order patch recovery interpolation or bilinear interpolation between the grids. For details please see the "Cubed Sphere to Structured Grid" subsection under the "File Based Regrid Weight Applications" section in the "Infrastructure: Fields and Grids" part of the ESMF Reference Manual.

Support the reuse of RouteHandles for Fields and Arrays where the size of the undistributed dimensions that come before the first distributed dimension is different from the size used during the precompute.

Improved the implementation of ESMF_GridMatch(). Previously the method just checked to see if the Grids had identical pointers, now it checks to see if the two Grids' internal data is the same (dimensions, coordinates, items, etc.). This means that two Grids constructed independently of each other are checked more reliably.

Added an option to ESMF_MeshGet() to allow users to retrieve coordinates for just the locally owned Mesh nodes. These coordinates will correspond exactly with the data elements stored in a Field created on the Mesh. This is useful for setting data in the Field or performing calculations involving coordinates on the data in a Field.

Added the ability to validate Attributes read from an XML file by specifying a corresponding XSD file in method ESMF_AttributeRead(), via new optional argument schemaFileName. For standard ESMF-supplied Component, Field, and Grid Attribute packages validation is performed automatically, without the need to specify schemaFileName. For examples see the Attribute section of the ESMF Reference Manual.

XML Attribute file writing is now done with the open source third party Xerces C++ library, replacing the previous internally hard-coded fprintf() technique. For reading and/or writing XML Attribute files, ESMF must be built against the Xerces library, which requires the setting of additional ESMF environment variables. See the "Third Party Libraries" section in the ESMF User's Guide for details.

A new system test, ESMF_FieldRegridMeshToMesh, was added to demonstrate regridding between Fields built on unstructured Meshes.

FieldBundles don't currently enforce that every contained Field is built on the same Grid, although the documentation says that this should be so.

The Field description in the reference manual incorrectly states that the halo update operation is not supported. Field halo is available in this release.

ESMF_FieldRegridStore() doesn't currently work on Fields created on Meshes which contain tetrahedra.

ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids which contain a DE of width less than 2 elements.

ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids of arbitrary distribution.

ESMF_FieldRegridStore() only works on Fields created on Grids with indexflag equal to ESMF_INDEX_GLOBAL. This is not the default.

Grid edge connections don't work in this release and will have no effect on the actions of ESMF_FieldRegridStore().

he ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShapeTile()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.

Testing on a constant field has shown that the integration weights that are generated for the conservative regridding are accurate to only 9 decimal places, rather than the expected 15 decimal places. This limits the precision of the conservative regridding method.

The conservative regridding may generate errors when the destination grid is of higher resolution than the source grid, specifically:

Values of destination points will not be set when the destination cell lies completely within a single source cell.

There can be regions of high interpolation error where shifting grid alignment causes different numbers of source points to contribute to the destination point.

These drawbacks are due to the fact that this conservative method does not have a constraint on local conservation.

The ESMF_CubedSphereRegridEx produces some suspicious interpolation weights (large, outside [0,1]).
These should be further investigated to see
if they are within the bounds of expected results for non-monotonic remapping methods or
represent true errors.

ESMF_AttributeWrite() only works for ESMF standard Attribute packages.

The performance of ESMF_StateReconcile() for large processor counts has been improved dramatically.

During ESMF_FieldRegridStore() pole elements are no longer created if the source Grid extends all the way to the pole. Previously degenerate zero-sized elements were erroneously created around the pole for this case.

Added ESMF_LocStreamCreate() that allows a user to create a LocStream from a background Grid or Mesh. This call creates a new LocStream that is a redistribution of an existing LocStream. The new LocStream is created so that an element with a given set of coordinates is located on the PET with the piece of background Grid (or Mesh) that contains those coordinates.

Added support for replicated dimensions in Field along all Grid dimensions. A gridToFieldMap argument containing all zeros in ESMF_FieldCreate() and ESMF_FieldSetCommit() indicates that no Grid dimensions are part of the Field. Instead the Field is replicated across all the DEs along all the Grid dimensions.

An "advice to installers" paragraph has been added to the Installation section of the User Guide. The paragraph recommends that a standard environment variable named ESMFMKFILE be set by the system to point to the "esmf.mk" file of the ESMF installation. Standardizing this aspect of the ESMF installation process promotes the design of portable ESMF application build systems.

The linker flags that are provided to the user through "esmf.mk" on the ORNL Cray XT systems are now portable between jaguar (XT4) and jaguarpf (XT5).

The "mvapich2" option was added to build environment variable ESMF_COMM. Please refer to the second table in the Supported Platforms section of the User Guide to see where it can be used.

A number of known bugs have been fixed, including issues with repeating sticky and non-sticky Alarms. Please see the known bugs list for more information.

The same as ESMF_4_0_0r with the following exceptions:

The problem that caused segmentation faults in ESMF_StateReconcile() has been fixed.

Repeating sticky Alarms whose ringInterval is less than or equal to the Clock's timeStep now ring properly.

Repeating sticky Alarms that are not turned off during a waiting period of a Clock run now ring properly after the waiting period is over, Alarms are cleared, and turning off of subsequent Alarms resumes.

Repeating non-sticky Alarms now ring properly for a Clock that goes into ESMF_MODE_REVERSE at the Clock's stopTime, while an Alarm is ringing.

Windows/Cygwin gfortran and g95 no longer require the setting of environment variables ESMF_F90LINKOPTS, ESMF_CXXLINKOPTS and ESMF_F90LINKLIBS.

The build issue with GCC 4.4.1 has been resolved.

The problem that caused segmentation faults on a user's Linux system with Intel compilers and OpenMPI has been identified and corrected in the ESMF library code.

Repeating sticky Alarms whose ringInterval is less than or equal to the Clock's timeStep now ring properly.

Repeating sticky Alarms that are not turned off during a waiting period of a Clock run now ring properly after the waiting period is over, Alarms are cleared, and turning off of subsequent Alarms resumes.

Repeating non-sticky Alarms now ring properly for a Clock that goes into ESMF_MODE_REVERSE at the Clock's stopTime, while an Alarm is ringing.

Component methods necessary to support user-level threading were added. These methods (such as ESMF_GridCompSetMaxPEs(); see the list of interface changes for others) allow a Component to set the characteristics of its VM before it is started up. One application of these additional methods is to share the Processing Elements (PEs), provided by the Persistent Execution Threads (PETs) of the parent Component, across a lesser number of child PETs. These PEs can then be used in user-level OpenMP threading during Component method execution.

Implemented the underpinnings of a Component-level automatic garbage collection scheme. An initial effect is that ESMF objects that were created within a Component will be destroyed when the Component is destroyed. Additional garbage collection features will be finished and documented in future releases.

Removed the (previously undocumented) restriction that prevented States, originating from partially overlapping Components, from being reconciled with ESMF_StateReconcile().

The internal buffer in ESMF_StateReconcile() is now dynamically
allocated. Applications with a large number of objects contained
in a State should no longer run up against internal buffer size
limitations.

Added ESMF_StateRead() and ESMF_StateWrite() methods to read and write undistributed Arrays serially from a NetCDF file. See example in the State section of the ESMF Reference Manual.

The Internal State implementation has been changed so that ESMF_GridCompGetInternalState() always returns the Internal State set by the last ESMF_GridCompSetInternalState() call. The previous behavior was that it would always return the Internal State set by the first ESMF_GridCompSetInternalState() call. The same is true with equivalent CplComp methods.

FieldBundles can now be created from Fields built on LocStreams or Meshes.

Fields built on LocStreams and Meshes now work with ESMF_StateReconcile() and can therefore be transferred between Components running on different sets of PETs.

The farray argument in ESMF_FieldGet() method has been renamed to farrayPtr to better reflect the nature of the returned parameter, and to be consistent with the ESMF_ArrayGet() method.

Regrid weight generation can be performed either through an offline application or through an online interface. The offline application is a separate executable which takes in two NetCDF grid files and outputs a NetCDF interpolation weight file. The online interface (ESMF_FieldRegridStore()) takes ESMF Field descriptions and generate weights. The weights can either be returned to the user through Fortran arrays, or can be applied with a parallel sparse matrix multiply in an ESMF_FieldRegrid() call.

The ESMF offline weight generation application supports:

Regridding between a pair of 2D logically rectangular Grids mapped to a sphere.

Multiple pole options: 1) Pole value is an average of source values around pole, 2) Pole value is an average of n-point stencil of source values, or 3) No pole.

Higher-order patch recovery interpolation.

For details please see the "File Based Regrid Weight Application" section in the "Infrastructure: Fields and Grids" part of the ESMF reference manual.

The ESMF online weight generation interface supports:

Regridding between any combination of 2D Meshes composed
of triangles and quadrilaterals, and 2D Grids composed of a single logically rectangular tile with no edge connections.

Regridding between any combination of 3D Meshes composed of hexahedrons, and 3D Grids composed of a single logically rectangular region.

Regridding between a pair of 2D logically rectangular Grids mapped to a sphere.

One pole option: Pole value is an average of source values around pole.

Source and destination point masking of Grids.

The ability to choose what happens to unmapped destination points: either 1) return an error if any are found, or 2) ignore and don't interpolate them.

Two interpolation options, 1) bilinear or 2) higher order.

For details please see the "Field Regrid" section in the "Infrastructure: Fields and Grids" part of the ESMF reference manual.

A more efficient tree-based search has been added to the ESMF_FieldRegridStore() method which improves its average running time by an order of magnitude.

A zeroflag and a checkflag have been added to the ESMF_FieldRegrid() call. The zeroflag allows the user to control how the destination Field should be initialized before regrid, and the checkflag allows the user to turn on more extensive error checking.

Optimized performance of ESMF_ArraySparseMatMulStore() for cases where the sparse matrix elements (i.e. factorList and factorIndexList arguments) are provided in parallel across the calling PETs.

Added an overloaded method for ESMF_GridCreate() to read the specification for a 2D regularly distributed rectilinear Grid from an XML file. See example in the Grid section of the ESMF Reference Manual.

The Grid class now contains a DistGrid per stagger location. This change allows the memory within a Field to exactly match the size of the exclusive region (the DE local portion) of the DistGrid associated with each stagger location. Previously the exclusive region of every stagger location needed to be sized to contain the exclusive region of all the stagger locations. This change affects some interfaces, such as ESMF_GridGet(), which now can return a DistGrid per stagger location.

Methods in the Mesh class now generate error messages through the standard ESMF LogErr system.

Mesh methods can now operate correctly in child Components which are running on fewer PETs than their parents.

The documentation for Meshes, and Fields created on Meshes, has been vastly improved, including multiple examples of Mesh creation and improved method descriptions. In addition, more extensive input checking has been added to Mesh to make its use easier.

Added bounds checking to the defaultvalueList argument in ESMF_AttributeGet().

Added a standard Attribute package to Array.

Added Attribute hierarchy linking to Array and ArrayBundle.

Time Manager interfaces have been made more robust to gracefully handle uninitialized objects and to eliminate memory leaks.

Two additional system tests were added to demonstrate the sparse matrix multiply operation for Fields based on LocStreams and Meshes.

A new system test, ESMF_ArrayRedistOpenMP, was added to demonstrate support for user-implemented OpenMP parallelism in ESMF Components.

Expanded the ESMF_CompFortranAndC system test to demonstrate the inter-language usage of Mesh and Field objects through States.

Reworked the VM documentation section in the Reference Manual to be more user oriented.

Removed the remaining dependency on explicit knowledge of the shape of the Fortran dope vector. This makes the library more portable across platforms, and more robust with compiler upgrades.

Implemented a clearer OpenMP support strategy. Two new environment variables were introduced that can be set to ON/OFF. ESMF_OPENMP, when set to ON during the library build, will compile all library sources using OpenMP compiler flags. In this case the OpenMP compiler flags automatically become part of the compiler flags exported by esmf.mk. If the ESMF library was compiled with ESMF_OPENMP set to OFF, a second environment variable, ESMF_TESTFORCEOPENMP, can be set to force the test and example codes to build with OpenMP compiler flags.

A new Windows option, intelcl, has been added. This allows building ESMF with the Intel Fortran compiler and the Microsoft cl (Visual C++) compiler, using the MinGW toolchain.

C interfaces:

Added to the C interface the ability to add and retrieve Field objects from a State.

Added C interfaces for the Field class. These enable a Field to be created on a Mesh, and allow the Mesh to be retrieved.

Segmentation faults have been reported for ESMF_StateReconcile().

ESMF_FieldRegridStore() doesn't currently work on Fields created on Meshes which contain tetrahedra.

ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids which contain a DE of width less than 2 elements.

ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids of arbitrary distribution.

ESMF_FieldRegridStore() only works on Fields created on Grids with indexflag equal to ESMF_INDEX_GLOBAL. This is not the default.

Grid edge connections don't work in this release and will have no effect on the actions of ESMF_FieldRegridStore().

The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShapeTile()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.

FieldBundles don't currently enforce that every contained Field is built on the same Grid, although the documentation says that this should be so.

Proxy Field objects, internally created during ESMF_StateReconcile(), return the wrong dimCount.

Repeating sticky Alarms whose ringInterval is less than or equal to the Clock's timeStep do not ring properly.

Repeating sticky Alarms that are not turned off during a waiting period of a Clock run do not ring properly after the waiting period is over, Alarms are cleared, or turning off of subsequent Alarms resumes.

Repeating non-sticky Alarms do not ring properly for a Clock that goes into ESMF_MODE_REVERSE at the Clock's stopTime, while an Alarm is ringing.

Added public C interfaces for select methods of the Mesh class. The prefix for these methods is ESMC. We do not yet have a Reference Manual that describes these interfaces.

Modifications were made to selected GridComp and CplComp methods that enable use of keyword syntax for procedure arguments. This was not permitted before because they did not have explicit interfaces. Using keyword syntax ensures that the compiler will check argument types. We strongly recommend that users take advantage of this feature. All of the ESMF calls that take dummy procedure arguments, or call into previously registered user procedures, now provide explicit interfaces: ESMF_GridCompSetVM(), ESMF_GridCompSetServices(), ESMF_GridCompSetEntryPoint(), ESMF_GridCompInitialize(), ESMF_GridCompRun(), ESMF_GridCompFinalize(), ESMF_GridCompReadRestart(), ESMF_GridCompWriteRestart(), ESMF_CplCompSetVM(), ESMF_CplCompSetServices(), ESMF_CplCompSetEntryPoint(), ESMF_CplCompInitialize(), ESMF_CplCompRun(), ESMF_CplCompFinalize(), ESMF_CplCompReadRestart(), ESMF_CplCompWriteRestart().

Those methods that call into a user supplied routine provide two separate (optional) return codes - the standard "rc" return code for the ESMF library, and the "userRc" return code set by the user routine. If keyword syntax is not used, the two return codes can get confused, with dire results. Specifically, using the old interface argument order, without keyword syntax, would lead to incorrect rc association. Users are strongly encouraged to make appropriate changes to their codes. See the API changes page for details.

ESMF_GridCompSetServices() and ESMF_CplCompSetServices() now support runtime loading of ESMF components via shared objects. This allows users to maintain separate build systems for individual components in the same application. The newly added ESMF_ArrayRedistSharedObj system test demonstrates this feature.

Changed the behavior of ESMF_GridCompSetServices() and ESMF_CplCompSetServices() to support the MAPL/GEOS-5 Component hierarchy paradigm. For most user code the SetServices change is expected to be completely transparent. All previous versions of ESMF executed the specified SetServices routine in the context of the parent VM.
Now, the specified public child routine is executed within the context of the child VM. This means that calls that would affect the child VM must be issued before the child SetServices routine is called.
Two new interfaces, ESMF_GridCompSetVM() and ESMF_CplCompSetVM(), have been added that allow child component code to be called and executed from within the context of the parent VM.
The benefit of the SetServices change is that the child SetServices routine may now contain calls that require execution from within the context of the child VM, e.g. creation of sub-components.

Added standard ReadRestart and WriteRestart methods to the ESMF_GridComp and ESMF_CplComp interfaces.

The ESMF_GridCompGet() and ESMF_CplCompGet() interfaces now offer access to the current method (for example, initialize, run, ...) and current phase of a Component.

Added "attachable methods" to ESMF States. User supplied methods can be attached, executed and removed. Currently attached user methods are ignored during ESMF_StateReconcile(), and attached methods cannot be copied or moved between States. For details see the new "Attachable Methods" section in the "Infrastructure: Utilities" part of the ESMF Reference Manual.

The InternalState documentation in the Reference Manual has been reworked to be more applicable to real world situations.

Added support in ESMF_FieldCreate() methods for Fields on arbitrarily distributed Grids. ESMF_FieldRegrid() does not yet work for arbitrarily distributed Grids. Overall, communications using Fields on arbitrarily distributed Grids have not been well tested.

The ESMF_FieldRegridStore() call now supports source and destination masking. The user can use this masking to tell the regrid to ignore source and/or destination points.

The ESMF_FieldRegridStore() call now has an unmappedDstAction flag. This allows the user to choose if they want unmapped destination points to cause an error or to be ignored.

A number of bug fixes and tweaks have improved the accuracy and stability of ESMF_FieldRegridStore(). This is in particular true of the higher-order interpolation weight generation.

Added a new public method ESMF_FieldGetBounds() to allow a user to query localDe specific data bounds.

Grids now have the capability to store other types of data besides coordinates. This data is referred to as Grid items. A set of new interfaces which mirror the ESMF_GridCoord interfaces has been added to Grid to support this functionality. The Grid currently supports four items and a set of new flags (ESMF_GRIDITEM_MASK, ESMF_GRIDITEM_AREA, ESMF_GRIDITEM_AREAM, ESMF_GRIDITEM_FRAC) has been added to be used in the interface to indicate the item being accessed. The ESMF_GRIDITEM_MASK item is used to store mask information for use by ESMF_FieldRegridStore().

Added support to create arbitrarily distributed Grids using ESMF_GridCreateShapeTile(), ESMF_GridCreate(), and ESMF_GridSetCommitShapeTile(). Other Grid interfaces have been modified as needed. Grid items do not work for arbitrarily distributed Grids.

Changed ESMF_GridGet() interface to get the information defined for the arbitrarily distributed Grid, including arbDim (the dimension in the DistGrid that represents the arbitrarily distributed dimension(s)), localCount (the number of PET-local arbitrarily distributed index locations), and localIndices (a list of PET-local arbitrarily distributed index locations). The names of these arguments are likely to change in future releases.

Added ability to read Attributes from an XML file via new method ESMF_AttributeRead(), which then attaches the Attributes to a Component, Field, or Grid.

XML Attribute file reading is done with the open source third party Xerces C++ library. ESMF must be built against the Xerces library, which requires the setting of additional ESMF environment variables. See the "Third Party Libraries" section in the ESMF User's Guide for details.

Implemented ability to specify time units as 8-byte floating point values in Time Manager methods.

Added ESMF_AttributeUpdate() routine to build consistent Attribute structure and values for objects defined across a VM. One use of this feature is to use it in conjunction with ESMF_AttributeCopy() during the CplComp run method in order to transfer Attributes between GridComps defined on mutually exclusive sets of PETs.

Overloaded ESMF_AttributeRemove() to remove entire Attribute packages, Attributes in an Attribute package, or just single Attributes.

Added the ability to nest Attribute packages inside of each other.

ESMF_StateReconcile() now has a flag to allow optional Attribute reconciliation. The default is currently that Attribute reconciliation is off, due to concerns about performance and robustness.

Added a set of overloaded ESMF_AttributeLinkRemove() interfaces. These detach an object's Attribute hierarchy from that of another object.

Modified the ESMF_Attribute system test to use ESMF_AttributeUpdate(), ESMF_AttributeCopy(), and ESMF_StateReconcile() on both States and Components where appropriate.

The range of Fortran unit numbers used within ESMF can be overridden via the ESMF_Initialize() call by using the new IOUnitLower and IOUnitUpper optional arguments. A new ESMF_IOUnitGet() method to find an unused unit number within the ESMF unit number range is provided as a service to application developers.

ESMF_IOUnitFlush() is provided as a common method to flush Fortran unit numbers. This is intended for use as a service to application developers.

Added ESMF_ConcurrentEnsemble system test demonstrating an ensemble in which different ensemble members, which are distinctly different components, run concurrently on distinct sets of PETs.

Added ESMF_InternalStateEnsemble system test demonstrating how different ensemble members representing the same Component can be written using a single Component object which indexes different elements of an ESMF internal State.

Activated TestHarness for Array and Field testing.

Added support for Windows MinGW/MSYS, using the Intel Fortran and C++ compiler and Microsoft MPI. This was developed and tested under Windows Compute Cluster Server (CCS) 2003.

An offline file-to-file regridding application has been developed as part of ESMF. This application takes two netcdf grid files and outputs interpolation weights generated using ESMF's higher-order interpolation. This application supports a number of options for handling the pole: no pole, treating the pole as the average of the values surrounding the pole, and an n-point stencil average of the points surrounding the pole. The format of the files makes this application a suitable replacement for SCRIP. For details see the new "File Based Regrid Weight Application" section in the "Infrastructure: Fields and Grids" part of the ESMF Reference Manual.

Furthermore, all the features listed in the release notes for ESMF_3_1_0rp2 apply to this release.

The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile()

The ESMF_StateReconcile() call fails for nested States.

ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.

Fields built on LocStreams and Meshes will not work with ESMF_StateReconcile() and therefore cannot be transfered between Components running concurrently.

ESMF_FieldRegrid() is only implemented for Fields created on Grids with indexflag equal to ESMF_INDEX_GLOBAL. This is not the default.

Proxy Field objects, internally created during ESMF_StateReconcile(), return the wrong dimCount.

The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShapeTile()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.

Grid connections don't work in this release and will have no effect on the actions of ESMF_FieldRegridStore().

Repeating sticky Alarms whose ringInterval is less than or equal to the Clock's timeStep do not ring properly.

Repeating sticky Alarms that are not turned off during a waiting period of a Clock run do not ring properly after the waiting period is over, Alarms are cleared, or turning off of subsequent Alarms resumes.

Repeating non-sticky Alarms do not ring properly for a Clock that goes into ESMF_MODE_REVERSE at the Clock's stopTime, while an Alarm is ringing.

ESMF_AttributeWrite() only works for ESMF standard Attribute packages.

In the Reference Manual, "Figure 29: The structure of the Attribute class" shows an attrCount variable which is no longer present, and does not show two vectors of Attribute pointers called packList and linkList.

In the API description of ESMF_AttributeLink() and ESMF_AttributeLinkRemove(), in the ESMF Reference Manual, the argument keyword should be "comp1" instead of "comp".

Added support for replicated dimensions in Field. A "0" entry in the gridToFieldMap argument in ESMF_FieldCreate() and ESMF_FieldSetCommit() indicates that a Grid dimension is not part of the Field. Instead the Field is replicated across all DEs along the corresponding Grid dimension.

Overloaded ESMF_FieldCreate() to accept a Fortran array pointer and preserve its properties (such as bounds and ability to deallocate).

Overloaded ESMF_FieldSetCommit() to accept a Fortran array pointer and preserve its properties (such as bounds and ability to deallocate).

The Grid now supports an ESMF_INDEX_USER option for the indexflag. This allows the user to explicitly set the lower bound of their local index space. This is necessary when creating a Field from a Fortran array pointer with non-standard indexing.

Overloaded ESMF_ArrayCreate() to accept a Fortran array pointer and preserve its properties (such as bounds and ability to deallocate).

Added public C interfaces for select methods of the following classes:

GridComp, CplComp

State

Clock, TimeInterval, Time, Calendar

Config

Array

ArraySpec

DistGrid

VM

The prefix for these methods is ESMC. We do not yet have a Reference Manual that describes these interfaces.

Restored most Field and FieldBundle communication methods except the halo update methods.

Added a method to perform a bilinear or higher order regridding between two Fields constructed on Grids. The Fields must be based on Grids with indexflag equal to ESMF_INDEX_GLOBAL (not the default) and must have data at cell centers (the default). This method will not work for Fields built on LocStreams or Meshes. Interpolation weights are computed in parallel.

Added a LocStream (Location Stream) class to represent and manipulate observational data streams. Fields can be created using a LocStream instead of a Grid.

Added ability to represent an unstructured Mesh in ESMF. Fields can be created using a Mesh instead of a Grid.

The DistGrid interface changes in this release reflect modifications that are necessary to support factorized sequence indices in the future. This feature is required for the ongoing arbitrary Grid development and is not fully supported yet.

Added Attributes to the CplComp and GridComp classes.

Added the ability to build Attribute hierarchies.

Added Attribute packages to CplComp, GridComp, and Field. Either standard or custom Attribute packages can be created.

Added a method to locally copy Attributes between States called ESMF_AttributeCopy().

Overloaded ESMF_AttributeGet() and ESMF_AttributeSet() to allow the manipulation of Attributes in Attribute packages.

Added an optional default value to ESMF_AttributeGet().

Added an interface to write out an Attribute hierarchy in either tab delimited or XML format called ESMF_AttributeWrite().

Added character lists as an option for Attribute type.

Standardized the use of count and itemCount in Attribute calls and made them optional in most cases.

Added support for the Modified Julian Day calendar, ESMF_CAL_MODJULIANDAY.

Alarms now ring properly for a clock in ESMF_MODE_REVERSE.

Argument lists for Fortran-callable entry points which contain logical data have been regularized to always use Fortran LOGICAL. Previously,
some routines used the internal ESMF_Logical derived type.

Parallel build of the ESMF library supported via GNU make -j option.

Added ESMF_CompFortranAndC system test that tests and demonstrates passing of State objects between Fortran and C components.

Added ESMF_ArrayRedist3D system test demonstrating a three dimensional redistribution operation for Arrays.

Added ESMF_SeqEnsemEx system test demonstrating an ensemble in which different ensemble members, which are distinctly different components, run on the same set of PETs.

The same as ESMF_3_1_0rp1 with the following exception:

The problem with the ESMC Component interfaces on XLF and PGI compilers has been fixed.

For higher-order interpolation, ESMF_FieldRegrid() can give wrong answers for multiple processors. We recommend only using this on a single processor.

In this release the accuracy of the higher-order interpolation has been degraded by a bug. For better results we recommend using ESMF v4.0.0. or later.

ESMF_FieldRegrid() has problems with the pole. When using ESMF_REGRID_SCHEME_FULL3D, we recommend not using any destination points which either lie in a cell next to the pole (for both bilinear and higher-order) or two cells away from the pole (for just higher-order).

Fields built on LocStreams and Meshes cannot be transferred between Components running concurrently.

ESMF_FieldRegrid() is only implemented for Fields with data located at cell centers.

ESMF_FieldRegrid() is only implemented for Fields created on Grids with indexflag equal to ESMF_INDEX_GLOBAL. This is not the default.

The Grid connections don't work in this release and will have no effect on the actions of ESMF_FieldRegrid().

New Attribute methods are unreliable across a range of platforms. They should be considered an early prototype and used with caution.

On IBM Bluegene the ESMF_FieldRegridEx.F90 and ESMF_FieldSphereRegridEx.F90 fail with ESMF_BOPT=O.

A problem in the Config C interface leads to failures in the ESMC_ConfigUTest.C on the XLF compiler.

ESMF_AttributeWrite() does not write out the values in the header of the .xml file generated when calling with ESMF_AttWriteFlag=ESMF_ATTWRITE_XML.

The Use Test Cases were not updated or tested for this release.

The ESMF_ArrayHarness2UTest is incorrectly reported as failing when executed through any of ESMF's batch mode mpirun scripts.

A new section in the User's Guide entitled "Setting up ESMF to run Test Suite Applications" was added that outlines how users can specify their own mpirun scripts. ESMF now consistently uses the setting of environment variable ESMF_MPIRUN to launch the bundled test suite and example applications.

A number of known bugs have been fixed. Please see the known bugs list.

The same as ESMF_3_1_0r with the following exceptions:

The problem with ESMF_ArrayRedist() and ESMF_ArraySMM() methods hanging in mpiuni mode has been fixed.

The problem with ESMF_ArrayRedist, ESMF_DirectCoupling and ESMF_RecursiveComponent system tests failing with a segmentation fault in ESMF_BOPT=O mode on x86_64 Darwin systems with Intel compilers has been fixed.

States containing FieldBundles can now be reconciled, so FieldBundles can be transferred between concurrently operating components.

Reconciliation errors for Fields created with user supplied arguments for gridToFieldMap, maxHaloWidths, and ungriddedBounds have been fixed.

Proxy Field objects, internally created during ESMF_StateReconcile(), are now properly destroyed during ESMF_StateDestroy().

The ESMF_AttributeGet() and ESMF_AttributeSet() Reference Manual entries that did not show a required "count" argument have been fixed.

Shortened the names of State add and get methods so that they do not refer to the item being retrieved. For example, ESMF_StateGetField() is now just ESMF_StateGet().

Fixed problem in the InternalState implementation that rendered the returned Fortran pointer to the user defined derived type incomplete and unsuitable for memory deallocation across a wide range of platforms.

Implemented ArrayBundle class with limited support for sparse matrix and redist communication methods.

State, Field, and FieldBundle Attribute methods were consolidated into a single set of Attribute methods that are overloaded for object type. For example, ESMF_AttributeGet() is overloaded to get Attributes from States, Fields, and Bundles. This reduces the number of documented calls in the interface significantly.

Field, Grid, and Array classes now interpret their width and bound arguments in a consistent way. The order of dimensions is defined by their index sequence in memory.

Restored the ESMF_FieldCreate() method that allows Field creation from an existing Grid and Array.

Restored the ESMF_FieldGet() that returns the native Fortran data pointer to Field data.

Implemented ESMF_FieldCreateEmpty(), which allocates the shell of a Field, and ESMF_FieldSetCommit(), which finishes a Field started by ESMF_FieldCreateEmpty().

Implemented a ESMF_FieldGet() method that provides precomputed bounds and counts from Grid and other Field creation parameters. This method helps users allocate native Fortran arrays to be used in ESMF_FieldCreate() and ESMF_FieldSetCommit() methods.

Switched Grid and Field localDe arguments to non-optional to conform to Array.

Grid undistributed dimensions are now always DistGrid dimensions distributed across 1 DE. This simplified interfaces and implementation. Field interfaces also reflect this change.

Grid now destroys internally created DELayouts, DistGrids, and Arrays.

Regular decompositions now behave more intuitively. Previously stagger edge padding could cause cells to be divided up oddly.

Added method to get GridStatus from a Grid.

Renamed ArraySparseMatMul methods ArraySMM to avoid method name size limits on some compilers.

Implemented more flexible zeroflag option in ESMF_ArraySMM() that supports zero out of only those destination elements that will be updated by the sparse matrix multiplication. This preserves special fill values if they exist.

Added support for Darwin on x86 and x86_64 systems with Intel compilers.

Added support for gfortran version 4.3.

Added explicit interface blocks that ensure conformance to ESMF interfaces in user-written routines called by ESMF_GridCompSetServices() or ESMF_CplCompSetServices() and registered via the ESMF_GridCompSetEntryPoint() or ESMF_CplCompSetEntryPoint() methods.

Implemented running only single processor system tests from the run_system_test_uni target.

Removed netcdf_stubs and hdf_stubs.

Checked for and fixed memory leaks.

Reviewed and updated introductory and general sections of the Reference Manual.

Stagger location and bounds diagrams were added to the Grid documentation.

The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile() .

FieldBundles contained inside States cannot be reconciled. This means that FieldBundles cannot be transferred between components running concurrently.

There are reconciliation errors for Fields created with user supplied arguments for gridToFieldMap, maxHaloWidths, and ungriddedBounds.

Proxy Field objects, internally created during ESMF_StateReconcile(), are not properly destroyed during ESMF_StateDestroy(). In applications that reconcile States across exclusive sets of PETs this will lead to a small memory leak for each contained Field object. This should not cause significant problems if ESMF_StateReconcile() is called once during initialization as recommended.

ESMF_ArrayRedist() and ESMF_ArraySMM() methods will hang in mpiuni mode. We expect this to be fixed in an upcomng patch release.

The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShapeTile()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.

Repeating sticky Alarms whose ringInterval is less than or equal to the Clock's timeStep do not ring properly.

Repeating sticky Alarms that are not turned off during a waiting period of a Clock run do not ring properly after the waiting period is over, Alarms are cleared, or turning off of subsequent Alarms resumes.

Repeating non-sticky Alarms do not ring properly for a Clock that goes into ESMF_MODE_REVERSE at the Clock's stopTime, while an Alarm is ringing.

The documentation for ESMF_AttributeGet() and ESMF_AttributeSet() calls for multi-valued list Attributes does not show a required integer input "count" argument after the "name" argument. This has been corrected in the Reference Manual for this release shown at left. However a user who builds the documentation themselves will still see this error.

ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.

On Cray X1 the ESMF_DistDir system test fails.

On Cray X1 the ESMF_String unit test fails.

On IBM BlueGene/L ESMF_VMSendVMRecvUTest.F90 fails in uni-PET mode.

On x86_64 Darwin systems with Intel compilers the ESMF_ArrayRedist, ESMF_DirectCoupling and ESMF_RecursiveComponent system tests fail with segmentation fault in ESMF_BOPT=O mode.

ESMC_Component examples fail with the XLF and PGI compilers as well as for the Linux/absoftintel combination.

All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. Consequently ESMF sets ESMF_OPTLEVEL to 0 (zero) by default when compiling ESMF with NAG and we advise NAG users to not use optimization on user code that links against the ESMF library.

Last time we were able to test with PGI 6.2.5 on Cray XT3/4 it failed to compile ESMF with an internal compiler error. The same version of PGI works fine on non-Cray machines. We were unable to test with this compiler version on Cray XT3/4 to determine the status of this issue with 3.1.0r.

Field, Bundle, State, and Component classes are now built on new Grid and Array classes. Field and Bundle method arguments have significantly changed so that they more closely resemble those of Grid and Array.

The following classes were removed:

IGrid (in the InternGrid directory, which was also removed)

FieldDataMap

BundleDataMap

The InternArray class is no longer visible at public interfaces. It will be removed in a future release along with InternDG.

Added system test FieldSparseMatMul. This creates Fields and gets Arrays from them in order to perform an ESMF_ArraySparseMatMul() operation.

The following were disabled:

Field and Bundle communication methods and corresponding unit tests and examples

Many Field and Bundle get and set methods and corresponding unit tests and examples

System tests *except* for CompCreate, DistDir, ArraySparseMatMul, FieldSparseMatMul, ArrayRedist, and ArrayScatterGather. Other system tests have not yet been modified to work with updated Field and Bundle classes, and will be restored in a future release.

Further work on the new Grid class:

Grids can be created or set with a regular distribution.

Coordinate arrays can be less than the Grid's rank (e.g. rectilinear coordinates)

Grids can be serialized and deserialized (e.g. as part of a state reconcile operation)

Grids can be queried for information about a stagger location including local DE exclusive and computational bounds, computationalEdgeWidths, and undistributed bounds.

Grids can be queried for element counts in addition to lower and upper bound information.

The Grids have been altered to live entirely within the Array exclusive region to allow for smoother interaction with Array methods.

Stagger locations have also been changed to allow:

Comparisons between staggerlocs

The output of a string describing the staggerloc

Implemented ESMF_ArrayGather()

Implemented ESMF_ArrayRedist()

Implemented ESMF_ArraySet()

Support the use of precomputed ArraySparseMatMul RouteHandles with different actual Arrays.

Optimized performance of ESMF_ArraySparseMatMulStore().

Optimized performance of ESMF_ArraySparseMatMul().

Implemented Array replication along specified DistGrid dimensions.

Generalized ArraySparseMatMul to support general Array - DistGrid dimension mapping.

Generalized ArraySparseMatMul to support tensor term mixing.

Implemented a more flexible Array computational bounds scheme that allows a computational region that is smaller than the exclusive region.

Changed Array total widths to be relative to the Array computational region.

Added computationalEdgeLWidth and computationalEdgeUWidth arguments to ESMF_ArrayCreate() interfaces to allow special treatment of the edge.

Increased robustness of g95 and gfortran configurations on Linux and Darwin.

Added support for Cray XT3/4 systems with CNL kernel.

The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShapeTile()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.

The only Use Test Case that currently builds is ESMF_SparseMatMul. Due to failures of the other UTCs it must be built and run separately, not from the collective UTC makefile.

The zeroflag=ESMF_TRUE option in the ESMF_ArraySparseMatMul() call is broken for Arrays that have total regions that are larger than the exclusive region. The symptom of this bug is that not all destination elements are reset to zero before adding the sparse matrix multiplication terms.

The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile() .

C/C++ Component examples fail with the XLF and PGI compilers as well as for the Linux/absoftintel combination.

The documentation for ESMF_GetAttribute() and ESMF_SetAttribute() calls for multi-valued list Attributes does not show a required integer input "count" argument after the "name" argument. If the "count" argument, which represents the number of Attribute values, is inserted the call will work.

ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.

On IRIX64 the exhaustive unit tests compiled against MPI will intermittently hang when run in uni-PET mode if the ESMF library was built with ESMF_PTHREADS=ON. This issue does not show up when running the MPI version in multi-PET mode or when using the MPIUNI version in uni-PET runs. The issue disappears when the ESMF library is compiled with ESMF_PTHREADS=OFF (which is the default for IRIX64).

ESMF fails to build on Cray XT3/XT4 with PGI 6.2.5 with an internal compiler error.

On x86_64 systems using the medium-memory model with PGI 6.2-3 or PathScale 2.4 ESMF_ArrayDataUTest.F90 will report two failures. These failures indicate a problem with the F90 dope vector and ESMF must be assumed unreliable under these conditions.

On x86_64 systems using the medium-memory model with PGI 6.2-3 the ESMF_ArrayLarrayEx.F90 fails.

On Cray X1 the ESMF_DistDir system test fails.

On IBM BlueGene/L ESMF_VMSendVMRecvUTest.F90 fails in uni-PET mode.

On Linux Lahey, a test in ESMF_ArrayDataUTest.F90 fails with ESMF_BOPT=g.

All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. Consequently ESMF sets ESMF_OPTLEVEL to 0 (zero) by default when compiling ESMF with NAG and we advise NAG users to not use optimization on user code that links against the ESMF library.

Added code to work around a bug in current Intel compiler versions on the NASA Altix system known as Columbia. This leads to failures in State and Grid interfaces when compiled in debug mode. It is unclear at this time if this is a system specific problem or not.

Renamed all references to Grids that existed in previous versions of the framework to IGrid and moved the class to an InternGrid directory. The IGrid class is deprecated and will be removed in a future release.

Created a new Grid class, which is still at an early stage of development. The new Grids can:

be 1D, 2D, 3D and higher dimensions

have multiple undistributed dimensions

hold curvilinear coordinates at multiple stagger locations.

The user can now:

create a new Grid either with shortcut methods or incrementally

get parameters from the new Grid

set and get Grid coordinates from an ESMF Array

get a pointer to the memory holding the coordinates in the Grid

get array bound information for the coordinate arrays

destroy a new Grid.

The ESMF_ArraySparseMatMulStore() and ESMF_ArraySparseMatMul() methods were rewritten and
improved performance and reduced memory usage are expected. The methods have the following new features:

support for distributed definition of factors

support for user-supplied arbitrary sequence indices via 1D DistGrid

support for ESMF_TYPEKIND_I4/I8/R4/R8

default initialization of destination Array to zero (can be overridden)

A collective ESMF_DistGridCreate() call was introduced that takes local lists of arbitrary sequence indices and returns a matching 1D DistGrid. This can be used in combination with the Array sparse matrix multiply to regrid unstructured grids.

The ESMF_DistGridGet() interface was massively updated to provide complete access to DistGrid information. The names of many of the arguments have changed.

The ESMF_ArrayCreate() calls now provide the "name" argument.

Many unimplemented methods will return the failure code ESMF_RC_NOT_IMPL. Validate methods are considered implemented if at least initialization checks are present.

Cygwin.g95.default and Cygwin.gfortran.default configurations were added.

Starting with this version of ESMF, support for PGI compilers has been limited to PGI versions >= 6.2. We will stop reporting ESMF known bugs for older PGI versions.

C/C++ Component examples fail with the XLF and PGI compilers as well as for the Linux/absoftintel combination.

The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile() .

ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.

When ESMF_InternArray objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.

Overall, we have found that regridding using ESMF_InternArrayRegrid(), FieldRegrid(), and BundleRegrid() is not robust. We encourage customers interested in ESMF regridding to move to the ESMF_ArraySparseMatMul() approach while regrid methods are reworked.

The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.

The conservative regrid method for both regional and global grids produces incorrect results for PGI in MPIUNI mode.

The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShape()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.

The documentation for ESMF_GetAttribute() and ESMF_SetAttribute() calls for multi-valued list Attributes does not show a required integer input "count" argument after the "name" argument. If the "count" argument, which represents the number of Attribute values, is inserted the call will work.

ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.

On IRIX64 the exhaustive unit tests compiled against MPI will intermittently hang when run in uni-PET mode if the ESMF library was built with ESMF_PTHREADS=ON. This issue does not show up when running the MPI version in multi-PET mode or when using the MPIUNI version in uni-PET runs. The issue disappears when the ESMF library is compiled with ESMF_PTHREADS=OFF (which is the default for IRIX64).

On Cray X1, the conservative regrid method does not work when the grid is defined as a uniform horizontal LatLon grid with periodic longitude coordinate and the source and destination grid are of the same size.

ESMF fails to build on Cray XT3/XT4 with PGI 6.2.5 with an internal compiler error.

On x86_64 systems using the medium-memory model with PGI 6.2-3 or PathScale 2.4 ESMF_ArrayDataUTest.F90 will report two failures. These failures indicate a problem with the F90 dope vector and ESMF must be assumed unreliable under these conditions.

On x86_64 systems using the medium-memory model with PGI 6.2-3 the ESMF_ArrayLarrayEx.F90 fails.

On Cray X1 the ESMF_DistDir system test fails.

On IBM BlueGene/L ESMF_VMSendVMRecvUTest.F90 fails in uni-PET mode.

On Linux Lahey, a test in ESMF_ArrayDataUTest.F90 fails with ESMF_BOPT=g.

All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. Consequently ESMF sets ESMF_OPTLEVEL to 0 (zero) by default when compiling ESMF with NAG and we advise NAG users to not use optimization on user code that links against the ESMF library.

The documentation of the ESMF_ArraySparseMatMulStore() API incorrectly states that the returned "routehandle" argument can be used to perform ESMF_ArraySparseMatMul() on any Array pair that is DistGrid-conforming to the Array pair used during the ESMF_ArraySparseMatMulStore(). This feature is _not_ yet supported, i.e. currently the "routehandle" can only be used to perform ESMF_ArraySparseMatMul() between the Array pair for which it was precomputed.

The ESMF_ArrayCreate(), ESMF_ArrayGet() and ESMF_ArrayScatter() interfaces were type/kind/rank overloaded.

ESMF_VMScatterV() and ESMF_VMGatherV() calls were added.

Methods across the framework had their return codes initialized to not implemented (ESMF_RC_NOT_IMPL). This is a step towards ensuring that unimplemented code returns appropriate errors when called.

LogErr calls were modified so that they need not set return codes prematurely to ESMF_SUCCESS. This is also a step towards ensuring that unimplemented code returns appropriate errors when called. This new LogErr behavior is not yet turned on, as there are still calls that rely on the former behavior of LogErr.

The declaration and usage of real and integer datatypes across the framework was standardized. The most significant change was the combination of type and kind into a single ESMF_TypeKind parameter. The new datatype conventions are included in the Developer's Guide.

A policy for the auto-promotion of user data via compile options was developed and is included in the Developer's Guide.

An ESMF_ConfigSetAttribute() call was implemented for 4-byte integers.

Conventions were established for a public ESMC optional argument API based on the variable-length argument list functionality provided by <stdarg.h>.

Added C interfaces for Config class (ESMC_Config). The ESMC_ConfigSetAttribute method is present, but not implemented.

An advanced stub for a new testing control system for running exhaustive tests on regrid and redist, called the test_harness, has been added. It runs and parses trivial bundle and field redistribution test configurations.

Fixed a bug that required Log files to be explicitly closed by the user before calling ESMF_Finalize.

Users may now open, write and close multiple Log files.

C/C++ Component examples fail with the XLF and PGI compilers as well as for the Linux/absoftintel combination.

The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile() .

ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.

When ESMF_InternArray objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.

Overall, we have found that regridding using ESMF_InternArrayRegrid(), FieldRegrid(), and BundleRegrid() is not robust. We encourage customers interested in ESMF regridding to move to the ESMF_ArraySparseMatMul() approach while regrid methods are reworked.

The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.

The conservative regrid method for both regional and global grids produces incorrect results for PGI in MPIUNI mode.

The documentation for ESMF_GetAttribute() and ESMF_SetAttribute() calls for multi-valued list Attributes does not show a required integer input "count" argument after the "name" argument. If the "count" argument, which represents the number of Attribute values, is inserted the call will work.

ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.

On IRIX64 the exhaustive unit tests compiled against MPI will intermittently hang when run in uni-PET mode if the ESMF library was built with ESMF_PTHREADS=ON. This issue does not show up when running the MPI version in multi-PET mode or when using the MPIUNI version in uni-PET runs. The issue disappears when the ESMF library is compiled with ESMF_PTHREADS=OFF (which is the default for IRIX64).

On Cray X1, the conservative regrid method does not work when the grid is defined as a uniform horizontal LatLon grid with periodic longitude coordinate and the source and destination grid are of the same size.

On Cray XT3/XT4, the ESMF_VMBarrierUTest is incorrectly reported as failing. The origin of this problem seems to be a bug in Cray's implementation of MPI_Wtime(). This problem will also affect the validity of ESMF_VMWtime().

ESMF fails to build on Cray XT3/XT4 with PGI 6.2.5 with an internal compiler error.

On x86_64 systems using the medium-memory model with PGI 6.2-3 or PathScale 2.4 ESMF_ArrayDataUTest.F90 will report two failures. These failures indicate a problem with the F90 dope vector and ESMF must be assumed unreliable under these conditions.

On Linux Lahey, a test in ESMF_ArrayDataUTest.F90 fails with ESMF_BOPT=g.

All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. Consequently ESMF sets ESMF_OPTLEVEL to 0 (zero) by default when compiling ESMF with NAG and we advise NAG users to not use optimization on user code that links against the ESMF library.

On the PGI 5.0-2 compiler with mpiuni, five regriding unit tests fail.

On the PGI 6.1-1 compiler with mpiuni, multiple unit tests, examples and 2 system tests fail.

Standardized the checking of initialization in ESMF. This introduces two new error return codes. The first, ESMF_RC_OBJ_NOT_CREATED, is returned if the user attempts to use a deep class variable without first creating it. The second ESMF_RC_OBJ_DELETED is returned if the user attempts to use a deep class variable after its deletion. Also, before this change, shallow class variables were not initialized on platforms with compilers that didn't support type component initialization. Now this initialization is handled by ESMF. Implementing this change necessitated the change of the intents of shallow class input parameters throughout ESMF from strictly in to inout.

Added an ESMF_LOG_NONE option to the "defaultLogType" argument of ESMF_Initialize() to turn off ESMF's default error logging.

Added "mpiCommunicator" argument to ESMF_Initialize() to allow ESMF to be initialized on a subset of MPI processes. The added VM example ESMF_VMUserMpiCommEx.F90 tests this new feature.

Added an ESMF_KEEPMPI option to the "terminationflag" argument of ESMF_Finalize() to allow ESMF finalization without MPI shutdown.

Removed deprecated GridCompCreate() and CplCompCreate() interfaces.

Optimized ESMF_BundleRedistStore(), ESMF_BundleRedist(), ESMF_FieldRedistStore(), and ESMF_FieldRedist() for arbitrarily distributed grids. A new algorithm was implemented i
n the ESMF Route code to initialize the arbitrary to arbitrary grid redistribution. The redist run routine has been optimized specifically for the Cray X1 machine.

Reworked the build system to be more robust and portable. Some of the previously used environment variables have become obsolete or were replaced by new variables. A full list of the environment variables as well as build system details can be found in the User's Guide. The main README contains enough information to get started quickly.

Introduced enhanced support for compiling and linking user applications against ESMF. See the User's Guide or main README for details.

Added a section to the User's Guide on how to port ESMF to other platforms.

Ported ESMF to the Cray XT3/XT4 and IBM BlueGene/L.

Added the following system tests to test the Bundle and Field redistribution methods for both arbitrarily distributed grids and block distributed grids:

ESMF_BundleRedistArb2ArbSTest.F90

ESMF_BundleRedistBlk2ArbSTest.F90

ESMF_BundleRedistBlk2BlkSTest.F90

ESMF_FieldRedistArb2ArbSTest.F90

ESMF_FieldRedistBlk2ArbSTest.F90

ESMF_FieldRedistBlk2BlkSTest.F90

Augmented the exhaustive regrid unit test ESMF_RegridToolUTest.F90 to include regrid scenarios with different resolution source grid and destination grid.

The problem with PGI 6.1.1 on Linux was resolved as a consequence of the reworked build system.

The following issues appear to be resolved due to compiler upgrades or indirectly as a result of other development:

Problem building with XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g.

Issues with configuration of MPICH with Intel 8.1.

The rank check error that incorrectly occurred during redistribution of Fields or Bundles that are transferred from block to arbitrary distribution (and vice versa) has been corrected.

The ESMF distribution available for download on the ESMF website has been pruned of many extraneous files, and is no longer bundled with documentation files. The distribution has been reduced in size from about 127MB to 12MB. Users can get documentation files by downloading from SourceForge.

This ESMF release is under the University of Illinois-NCSA Open Source License. Licenses are still a subject of discussion.

C/C++ Component examples fail with the XLF and PGI compilers as well as for the Linux/absoftintel combination.

The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a su
bsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile().

ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.

When ESMF_InternArray objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.

Overall, we have found that regridding using ESMF_InternArrayRegrid(), FieldRegrid(), and BundleRegrid() is not robust. We encourage customers interested in ESMF regridding to move to the ESMF_ArraySparseMatMul() approach while regrid methods are reworked.

On IRIX64 the exhaustive unit tests compiled against MPI will intermittently hang when run in uni-PET mode if the ESMF library was built with ESMF_PTHREADS=ON. This issue does not show up when running the MPI version in multi-PET mode or when using the MPIUNI version in uni-PET runs. The issue disappears when the ESMF library is compiled with ESMF_PTHREADS=OFF (which is the default for IRIX64).

The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.

The conservative regrid method for both regional and global grids produces incorrect results for PGI in MPIUNI mode.

The documentation for ESMF_GetAttribute() and ESMF_SetAttribute() calls for multi-valued list Attributes does not show a required integer input "count" argument after the "name" argument. If the "count" argument, which represents the number of Attribute values, is inserted the call will work.

ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.

On Cray X1, the conservative regrid method does not work when the grid is defined as a uniform horizontal LatLon grid with periodic longitude coordinate and the source and destination grid are of the same size.

On Cray X1, there are unit test failures across a wide range of classes, including Time classes, Field, State and Array, and we do not consider the release to be supported on this platform.

On Cray XT3/XT4, the ESMF_VMBarrierUTest is incorrectly reported as failing. The origin of this problem seems to be a bug in Cray's implementation of MPI_Wtime(). This problem will also affect the validity of ESMF_VMWtime().

On Cray XT3/XT4, the system test ESMF_ArraySparseMatMulSTest() and example ESMC_AppMainEx.C fail for ESMF_BOPT=O/g. System test ESMF_FieldHaloSTest() fails with ESMF_BOPT=O.

On IBM BlueGene/L, tests in ESMF_ArrayScatterUTest.F90 fail for ESMF_BOPT=g. Tests in ESMF_ClockUTest.F90 fail for ESMF_BOPT=O/g.

All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. We advise NAG users to set environment variable ESMF_OPTLEVEL to 0 (zero) when compiling ESMF and to not use optimization on user code that links against the ESMF library.

This is the first ESMF release under the University of Illinois-NCSA Open Source License.
Subsequent releases will be under this license.

Added an ESMF_LOG_NONE option to the "defaultLogType" argument of ESMF_Initialize() to turn
off ESMF's default error logging.

Added an ESMF_KEEPMPI option to the "terminationflag" argument of
ESMF_Finalize() to allow ESMF finalization without MPI shutdown.

Fixed the C/C++ interfaces for all but XLF and PGI compilers, and reactivated the C/C++
Component interface examples.

Optimized ESMF_BundleRedistStore(), ESMF_BundleRedist(), ESMF_FieldRedistStore(), and
ESMF_FieldRedist() for arbitrarily distributed grids. A new algorithm was implemented in the ESMF
Route code to initialize the arbitrary to arbitrary grid redistribution. The redist run routine
has been optimized specifically for the Cray X1 machine.
View performance report.

Added an ESMF_VMAllGatherV() method.

Reworked the build system to be more robust and portable. Some of the previously used environment variables have become obsolete or were replaced by new variables. A full list of the environmentvariables as well as build system details can be found in the User's Guide. The main README
contains enough information to get started quickly.

Introduced enhanced support for compiling and linking user applications against ESMF.
See the User's Guide or main README for details.

Added a section to the User's Guide on how to port ESMF to other platforms.

Ported ESMF to the Cray XT3, IBM BlueGene/L and Darwin xlfgcc.

Added the following system tests to test the Bundle and Field redistribution methods for both
arbitrarily distributed grids and block distributed grids:

ESMF_BundleRedistArb2ArbSTest.F90

ESMF_BundleRedistBlk2ArbSTest.F90

ESMF_BundleRedistBlk2BlkSTest.F90

ESMF_FieldRedistArb2ArbSTest.F90

ESMF_FieldRedistBlk2ArbSTest.F90

ESMF_FieldRedistBlk2BlkSTest.F90

Augmented the exhaustive regrid unit test ESMF_RegridToolUTest.F90 to include regrid
scenarios with different resolution source grid and destination grid.

Fixed unit test failures on the Linux PGI compiler:

ESMF_ArrayUTest.F90

ESMF_VMBarrierUTest.F90

On the Linux Lahey compiler, fixed failure of the unit test ESMF_VMSendNbVMRecvNbUTest.F90.

Added support for the LSF batch queuing system.

The following issues appear to be resolved due to compiler upgrades or indirectly as a result of other development:

Problem building with XL Fortran compiler version8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g.

Intermittent failure of the "ESMF_FlowWithCoupling" system test and the "ESMF_COUPLED_FLOW" demo on AIX.

On the Linux PGI 5.0-2 compiler, failure of the "ESMF_FlowWithCoupling" system test and the "ESMF_COUPLED_FLOW" demo to compile with -g, resulting in an internal compiler error.

Issues with configuration of MPICH with Intel 8.1.

C/C++ Component examples fail with the XLF and PGI compilers.

The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internalsize limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile().

ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.

When ESMF_Array objects are created with a non-zero halo width, that width is added to all
dimensions of the array, not only ones which correspond to the decomposed grid axes.

Overall, we have found that regridding using ArrayRegrid(), FieldRegrid(), and BundleRegrid() is not robust. We encourage customers interested in ESMF regridding to move to the ESMF_ArraySparseMatMul() approach in the ESMF 3 series while regrid methods are reworked.

The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.

The conservative regrid method for both regional and global grids produces incorrect
results for PGI in MPIUNI mode.

On Cray X1, the conservative regrid method does not work when the grid is defined as a uniform horizontal LatLon grid with periodic longitude coordinate and the source and destination
grid are of the same size.

On Cray XT3, the ESMF_VMBarrierUTest is incorrectly reported as failing. The origin of
this problem seems to be a bug in Cray's implementation of MPI_Wtime(). This problem will also affect the validity of ESMF_VMWtime().

'ESMF_LOG_SINGLE' is known to be broken. The symptoms are platform dependent and may appear non-deterministic.

All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. We advise NAG users to set environment variable ESMF_OPTLEVEL to 0 (zero) when compiling ESMF and to not use optimization on user code that links against the ESMF library.

The following classes were modified to implement an index space layer according to JST design
discussions:

Array

DELayout

DistGrid

Adjusted library sources, examples and tests to DELayout changes.

Added support for arbitary to arbitrary redistribution of Fields and Bundles.

Renamed previous Array class to InternArray. This class is expected to become an internal class in future releases.

Adjusted library sources, examples and tests to InternArray. Removed as many InternArray dependencies as possible.

Fixed VMBarrierUTest for PGI (Argonne Jazz cluster). This was a known bug in the previous release.

Fixed the C++ Component interface for most platforms. PGI and XLF still do not work.

Re-enable the C++ Component examples, which were disabled for the previous release.

Modified VMKernel and makefiles to allow compilation on NEC SX.

Added support for the LSF batch queueing system on the AIX platform. (For NCAR computer users, this supports 'bluevista'.)

The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile().

Redistribution of Fields or Bundles that are transferred from block to arbitrary distribution (and vice versa) fails with a message about mismatched ranks.

The C/C++ Component examples fail with the XLF and PGI compilers.

ESMF_BundleGather(), ESMF_FieldGather() and ESMF_InternArrayGather() are only expected to work on 2D grids with 2D data.

The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.

When ESMF_InternArray objects are created with a non-zero halo width, that width is added to
all dimensions of the array, not only ones which correspond to the
decomposed grid axes.

On AIX systems the ESMF fails to build with the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)

When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with: #define MPI2CPP_HAVE_BOOL 1 which may not be the default output from configure.

On the Linux PGI compiler unit test ESMF_InternArrayUTest.F90 fails with both ESMF_BOPT=O/g

Build fails on Longs Linux PGI 6.1.1 compiler with ESMF_BOPT=g.

On the Cray X1, several system tests, examples and unit tests fail.

All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. We advise NAG users to set environment variable ESMF_OPTLEVEL to 0 (zero) when compiling ESMF and to not use optimization on user code that links against the ESMF library.

The ESMF_FieldExclSTest system test has been modified to demonstrate a simplified run loop for the concurrent case. This simplification was made possible by the modified blocking behavior of Components that was introduced with ESMF release 2.2.1.

The top level README has been updated to include more information about linking applications and porting ESMF to other platforms.

Added an optional environment variable, ESMF_ARRAY_LITE, to restrict the number of array-level interfaces automatically generated. This is required on the NEC build since single byte integer arrays are not supported on that platform, but is available as a general feature to reduce the number of overloaded interfaces and code size.

Added additional environment variables, ESMF_CXX_LIBRARY_PATH and ESMF_F90_LIBRARY_PATH, to support explicit library search paths. If defined, these will be used at link time to locate the correct F90 and C++ libraries. Otherwise, the standard LD_LIBRARY_PATH will be used.

Changed the following in the ESMF_GridCompCreate() and ESMF_CplCompCreate() interfaces:

Added an interface with all optional arguments. In subsequent releases this may become the only supported interface.

We recommend that users call ESMF_xxxCompCreate() without specifying the parent VM. All tests, examples and demo codes have been modified to match the new convention.

If the parent VM is not specified the current VM is used as the parent VM (previously the global VM was used in this case).

Added support for high level MPMD. This makes xxxCompCreate() collective across all parent PETs and xxxCompSetServices() collective only across participating parent PETs.

Added a new blockingFlag for xxxCompInitialze(), xxxCompRun() and xxxCompFinalize() that eliminates some unnecessary synchronization points. This flag (ESMF_VASBLOCKING) is now the default.

The ESMF_StateReconcile() method no longer dumps core when the buffer size is exceeded. It returns an error message instead. (Please see the first item in the Known Bugs column.)

DELayout now issues warning if non 1-to-1 DELayouts or non logically rectangular DELayouts are created since most of the rest of the system does not handle these yet.

The Field and Bundle level redistribution communication calls have been rewritten. They now include 3+D data support and optimized communication calls for Bundles which contain Fields where the data arrays are congruent (same data type, same rank, etc).

Validate routines have been added for the Field and Bundle levels of the regrid, halo, and redistribution routines. If a hang or core dump appears to be associated with a call to the run routine for these operations, the route handle can be passed to a Validate routine which looks for inconsistencies and other problems.

Added system tests for MPMD (these are not automatically run by test scripts).

Added a unit test VMUserMpiInitUTest which checks that the user can explicitly call MPI_Init() before ESMF_Initialize().

Added unit tests for non-blocking VM communications.

Added support for the LSF batch queueing system on the AIX platform. (For NCAR computer users, this supports 'bluevista'.)

The files 'esmfuser.mk' and 'makefile.sample' in $ESMF_DIR/build replace the outdated 'link_rules.mk' files, showing how to link a user application with the ESMF library for any platform.

Added MPICH2 and IntelMPI support for Linux.intel.

The default linker front end on IRIX64 has been changed to CC, i.e. MIPSPro C++, for all ESMF applications. CC is now being used during link time regardless of the language in which the main program was written (Fortran90 or C++). In previous ESMF releases the f90 linker front end had been used to link applications on IRIX64 that provided the main program in Fortran. However, this led to problems during link time for applications that also contained C++ user code.

More documentation was added to the implementation notes part of the communication routines section.

Documented non-blocking features and limitations.

Added extensive new documentation on Grids and regridding.

Added reference documentation explaining that ESMF_CAL_NOCALENDAR does not track days, since the unit of a day is planet-specific.

The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is
no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile().

ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.

The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.

When ESMF_Array objects are created with a non-zero halo width, that width is added to
all dimensions of the array, not only ones which correspond to the decomposed grid axes.

The C/C++ Component interface is broken, so the C++ examples have been removed from the makefiles for this release.

On AIX systems there is a problem in the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)

On AIX the "ESMF_FlowWithCoupling" system test and the "ESMF_COUPLED_FLOW" demo fail intermittently.

On the Linux PGI 5.0-2 compiler the "ESMF_FlowWithCoupling" system test and the "ESMF_COUPLED_FLOW" demo fail to compile with -g, resulting in an internal compiler error. (PGI 5.2-4 works fine)

When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are
configured with: #define MPI2CPP_HAVE_BOOL 1 which may not be the default output from configure.

Main programs in C/C++ do not work with the PGI compiler.

On the Linux PGI compiler the followingunit tests fail:

ESMF_ArrayUTest.F90

ESMF_VMBarrierUTest.F90

On the Linux Lahey compiler the ESMF_VMSendNbVMRecvNbUTest.F90 unit test fails.

Support for the LSF batch queueing system on the AIX platform needs script not included in the release.

All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also whe
n user code that links with ESMF is compiled in optimized mode. We advise NAG users to set environment variable ESMF_OPTLEVEL to 0 (zero) when compiling ESMF a
nd to not use optimization on user code that links against the ESMF library.

Modified an ESMF_StateReconcile() unit test to catch the bug described above.

Increased the size of the buffer used by ESMF_StateReconcile(), which caused a core dump if the buffer size was exceeded (this needs a better solution, but the patch will work for
many users).

Removed all references to netcdf calls when ESMF_NO_IOCODE is set to true (due to conflicts between ESMF netcdf calls and user versions of the netcdf library)./li>

Fixed and documented an issue with MPICH that caused the working directory to be changed during the ESMF_Initialize() call. Now the API description of ESMF_Initialize() contains a paragraph discussing its dependency on MPI_Init() and the consequences thereof.

Added support for Intel 9.0 compilers on
IA32 and IA64.

Same as release ESMF_2_2_0r with the exception of the ESMF_StateReconcile() bug.

Calling the redistribution and halo functions from a parent, coupler, or higher level Component fails on ESMF_Fields created in a child Component if that child is running on only a subset of the caller Component's PETs. This is the case, for example, when child Components are running concurrently.

Fixed the issue in the first-order conservative regrid sweep algorithm that calculated correct results but was inefficient.

Renamed dePetList argument in ESMF_DELayoutCreate() to petList to be more consistent with rest of the framework. Added more helpful comments describing the function of the petList argument.

Eliminated a bug in the VM setup which caused the sequence of PETs in the petList argument to ESMF_XXXCompCreate() to be ignored. Now the PETs of the child Component map to the parent PETs as is expected according to the petList sequence.

The initial phases of new code needed to optimize ESMF_Bundle level communication code have been added. An ESMF_Bundle which contains identically structured ESMF_Fields will only compute a single Route table and apply it to all ESMF_Fields. Later releases will include more optimizations inside the communication code itself.

The Log HALT_ON_ERROR function is now implemented.

In the Time Manager, ESMF_Clocks and ESMF_Alarms can now be run in reverse via a new
optional ESMF_Direction type argument, which can have the values ESMF_MODE_REVERSE or ESMF_MODE_FORWARD in ESMF_ClockSet()/Get(). Forward mode is the default.

In the source tree, the Base directory was split into Base and Util, with utility routines which are not directly part of the Base class moved into the Util directory.

The default pointer size on Linux systems is now automatically determined using the output of the 'uname' command. 64-bit hardware defaults to ESMF_PREC 64, 32-bit hardware defaults to
ESMF_PREC 32.

The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 2, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile(). ESMF_StateReconcile() causes a core dump inthe buffer size is exceeded.

The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.

When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.

The following Time Manager calendar conversions do not work: Gregorian <--> NoLeap <--> 360 Day.

Main programs in C/C++ do not work with the PGI compiler.

The ESMF_Bundle halo and redist functions must be called with ESMF_Bundles which contain identical ESMF_Field data types, rank, shape, and relative locations. Any other type of
ESMF_Bundle will return with an ESMF_RC_NOT_IMPL error.

ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.

On the Cray X1:

The __LINE__ macro does not work in F90; "line 8" is always printed from methods such as ESMF_Test();

The ESMF_RowReduce system test, ESMF_LogErrEx.F90 and ESMFVMAllFullReduceEx.F90 examples and several unit tests fail.

On IRIX64:

The ESMF_LogErrUTest.F90 unit tests fail due to a difference in log file format as described in Sourceforge bug 1172026;

The ESMF_LogErrUTest.F90 crashes when attempting to read a log file in Fortran that was created in C as described in Sourceforge bug 1212893.

All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. We advise NAG users to set environment variable ESMF_OPTLEVEL to 0 (zero) when compiling ESMF and to not use optimization on user code that links against the ESMF library.

On AIX systems there is a problem in the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)

When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with: #define MPI2CPP_HAVE_BOOL 1 which may not be the default output from configure.

Halowidth removed from the ESMF_FieldCreate() calls where an already-created array is specified, which already contains halo information.

ESMF_VMAllGather() routine added.

Added option to create child Components which share the parent's VM.

Julian calendar added to Time Manager.

Added ESMF_CalendarIsLeapYear() and ESMF_TimeIsLeapYear()
methods.

Additional internal packing options added to support different memory/ communication overhead ratios on different hardware
platforms.

Extensive makefile and build system updates, including 64-bit PGI support, user-contributed g95 support, support for mpich on OSF (HP), updated make targets, and updated README's and User's Guide content.

It is now possible to compile ESMF withoutPthread dependency. If the environment variable ESMF_PTHREADS is set to OFF all Pthread dependent features of ESMF will be disabled. This can be used to simplify debugging and on systems that don't provide a functioning Pthread library. (This is different from the pthread-less default VM that was introduced in 2.1.0rp2!)

Compile-time options in system tests to run with threading enabled.

Output of unit and systems tests now go through the ESMF_Log facility, allowing multiprocessor test output to be cleanly collated.

Updated output scripts for results of unit and system tests.

More complete compiler and flag information printed for the output of 'gmake info'.

Site files need now only contain differences, not entire copies of the default makefiles.

Makefile target changes:

The previous default target was named 'all' but it only built libesmf. The current default target is named 'lib' and does the same thing. The new target 'all' builds the library and all the examples, tests, and demos.

The previous targets which were just 'tests' (e.g. build_tests) are now more specific: build_unit_tests, etc. There are new targets which are 'all_tests' to build everything.

Complete makefile target documentation may be found in the ESMF top level README file and in the User's Guide (subsection 6.2.3).

Added reference documentation about negative timesteps (as currently implemented, which is different from the upcoming reverse mode).

Expanded and updated the reference document bibliography, with citations.

The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 2, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile(). ESMF_StateReconcile() causes a core dump if the buffer size is exceeded.

Calling the redistribution and halo functions from a parent, coupler, or higher level Component fails on ESMF_Fields created in a child Component if that child is running on only a subset of the caller Component's PETs. This is the case, for example, when child Components are running concurrently. ESMF_FieldRegrid() does work correctly in this situation. Redistribution and halo functions
work correctly when called from a child Component on ESMF_Fields created in that child Component.

The Log function "Halt on Error" has not been implemented, so we expect an error from the unit test ESMF_LogErrHaltUTest.F90 on all platforms. Log message is: FAILED,
ESMF_LogErrHaltUTest.F90, line 96. The Halt on Error test failed.

The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors. There is also some logic that is not correctly handling the situation where the physical domains of two Grids are slightly offset. It does not result in errors or incorrect results but can be very inefficient.

ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.

When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.

The following Time Manager calendar conversions do not work: Gregorian <--> NoLeap
<--> 360 Day.

On several platforms examples with a main in C/C++ do not link.These include Linux/Lahey,
Linux/Absoft, Linux/PGI with mpiuni, and OSF1
(halem).

The operating system and all compilers on the Cray X1 at Ames were updated this week. There are problems running with the updated system; to run on the X1 follow the instructions for reverting back to the original environment, or contact us for help in running with the latest system.

The default compiler on the Compaq (halem), Fortran 551J, is compatible with this release. Fortran 551L core dumps when running system tests.

On AIX systems there is a problem in the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)

When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with:#define MPI2CPP_HAVE_BOOL 1 which may not be the default output from configure.

On the Cray X1, the __LINE__ macro does not work in F90; "line 8" is always printed from methods such as ESMF_Test().

The ESMF_VMAllFullReduceEx.F90 example fails on the Cray X1.

On the SGI IRIX64 with ESMF_PREC=32, all system tests pass but hang up with
ESMF_PREC=64.

The ESMF_LogErrUTest.F90 unit tests fail on IRIX64 due to a difference in log file format as described in Sourceforge bug 1172026.

Banding problem in bilinear regridding, at the bottom or left boundaries of some destination DEs (those bounding other DEs and not at domain edges in those directions), in regridding decompositions except R-to-C or C-to-R and under certain coordinate overlay circumstances.

Added unit tests to test more Regrid combinations and to verify that the above bug has been fixed.

Same as release ESMF_2_1_0rp2 with the exception of the bilinear regridding banding problem.

Pthread-less default VM, i.e. the VMs of newly created Components will by default run without the use of threads.

Extended the command line argument code to correctly set options for ps(1) of BSD
derived OSs, e.g. Darwin.

Fixed a problem with ESMF_StateReconcile() that would overwrite memory depending
on the DELayout type. Sometimes no symptoms were evident and other times it would core dump. When the problem occurred, system test ESMF_FieldExcl would fail intermittently on the Linux platforms.

The performance for the redistribution function with arbitrarily distributed grids has
been optimized.

The makefiles have been updated to support PGI 5.x, Absoft 9.x and NAG 5.x on Linux
clusters.

Modified the command line Perl scripts so that now the number of passes and failures
are printed as the last line of the script output instead of the first line.

Fixed bug in unit tests that falsely indicated that they were failing by relabeling tests that were mislabeled as non-exhaustive but were actually exhaustive. This problem occurred on all platforms. The following unit tests failed when run as non-exhaustive multi-processor:

ESMF_FRoute4UTest.F90

ESMF_FRoute8UTest.F90

ESMF_RegridUTest.F90

ESMF_TimeUTest.F90

The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 2, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile(). ESMF_StateReconcile() causes a core dump
if the buffer size is exceeded.

Calling the redistribution and halo functions from a parent, coupler, or higher level Component fails on ESMF_Fields created in a child Component if that child is running on only a subset of the
caller Component's PETs. This is the case, for example, when child Components are running concurrently. ESMF_FieldRegrid() does work correctly in this situation. Redistribution and halo functions work correctly when called from a child Component on ESMF_Fields created in that child Component.

Banding problem in bilinear regridding, at the bottom or left boundaries of some destination D Es (those bounding other DEs and not at domain edges in those directions), in regridding decompositions except R-to-C or C-to-R and under certain coordinate overlay circumstances.

ESMF_HaltType settings ESMF_LOG_HALTWARNING and ESMF_LOG_HALTERROR do not actually halt
the code.

The default compiler on the Compaq (halem), Fortran 551J, is compatible with this
release. Fortran 551L core dumps when running system tests.

ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.

When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.

On AIX systems there is a problem in the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O
also work with either compiler version.)

When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with:#define MPI2CPP_HAVE_BOOL 1which may not be the default output from configure.

Main programs in C/C++ do not work with the PGI compiler.

On the Cray X1, the __LINE__ macro does not work in F90; "line 8" is always printed from methods such as ESMF_Test().

The following Time Manager calendar conversions do not work: Gregorian <--> NoLeap <--> 360 Day.

On the Darwin XLF compiler:

Example ESMF_LogErrEx.F90 fails.

System test, ESMF_FieldRegridMulti uni fails with ESMF_BOPT=O and ESMF_COMM=mpiuni.

The ESMF_StateReconcile() call, generally used by
couplers dealing with concurrently running Components, has an internal
size limit (which will be addressed in a subsequent release). There is
no hard-coded limit to how many elements, i.e. States, Bundles, Fields,
Arrays can be added to the State. However, there is a limit, roughly
2, on the maximum number of sub-elements contained by any element in
the State passed into ESMF_StateReconcile(). ESMF_StateReconcile()
causes a core dump
if the buffer size is
exceeded.

Calling the redistribution and halo functions from a
parent, coupler, or higher level Component fails on ESMF_Fields created
in a child Component if that child is running on only a subset of the
caller Component's PETs. This is the case, for example, when
child
Components are running concurrently. ESMF_FieldRegrid() does work
correctly in this situation. Redistribution and halo functions
work
correctly when called from a child Component on ESMF_Fields created in
that child Component.

The
default
compiler on the Compaq (halem), Fortran 551J, is compatible with
this
release. Fortran 551L core dumps when running system tests.

ESMF_BundleGather(), ESMF_FieldGather()
and ESMF_ArrayGather() are only expected to work on 2D grids with 2D
data.

When
ESMF_Array
objects are created with a non-zero halo width, that width is added to
all
dimensions of the array, not only ones which correspond to the
decomposed
grid axes.

On
AIX
systems there is a problem in the XL Fortran compiler version 8.1.1.6,
when
compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler
version
8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or
ESMF_BOPT=O
also work with either compiler version.)

When
compiling
with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are
configured
with:
#define MPI2CPP_HAVE_BOOL 1
which may not be the default output from configure.

Main
programs
in C/C++ do not work with the PGI compiler.

The
ESMF_StateReconcile()
call does not work with the Lahey compiler. The ESMF_FieldExcl
system
test uses this call, so it will not work either.

On
the
Cray X1, the __LINE__ macro does not work in F90; "line 8" is always
printed
from methods such as ESMF_Test().

The
ESMF_VMGetMPICommunicatorEx
example fails on the Linux/intel 64-bit build.

The
ESMF_StateUTest
test fails on the Linux/PGI build.

The
ESMF_StateEx
example fails on the Linux/PGI build.

Several
of the examples fail on the Mac OS X/xlf build.

The
ESMF_StateUTest
fails 2 tests on the Linux/absoft build

The
following
Time Manager calendar conversions do not work: Gregorian <-->
NoLeap
<--> 360 Day

Concurrent
Component support has been added. Component initialize, run, and
finalize
routines can be called with a flag which causes control to return
immediately
to the calling Component which can then start other Components in
parallel.
Concurrency features are not yet operating correctly on Linux and Altix
platforms
(see Known Bugs).

Components
can be created and run on a subset of the parent's Persistent Execution
Threads,
PETs. An ESMF_StateReconcile()
call was added so that a parent Component can ensure that child
Components
who will be communicating each have all the State information that they
need,
even if Fields were not originally declared on both children.

Support
for arbitrarily distributed grids and their redistribution to and from
grids
with block decompositions has been added. This is currently very
slow.

The
issue
with regrid that occasionally caused banding has been fixed.

The
issue
with regrid that occasionally returned NaNs on spherical coordinate
systems
has been fixed.

The
issue
with halo for periodic boundaries on non-decomposed axes has been fixed.

Problems
with Route on PET counts that are not powers of 2 were fixed, and Route
now
works correctly for any PET count.

Some
Field
level communication routines now take a VM, not a DELayout.

Redist
and regrid use asynchronous communications internally.

Support
for concurrent Components has been activated in the VM.

Overhead
when entering/exiting VM contexts has been reduced.

Conflicts
between VM and MPICH's devices due to POSIX signal overlaps have been
resolved.

New
VM
now works reliably with old, non-posix compliant (pre-NPTL)
LinuxThreads,
as well as POSIX compliant threads.

Several
of the communication calls in the VM API have been overloaded to accept
data
of type 'ESMF_Logical' and 'Character'.

A new
method
ESMF_VMGetCurrent() has been added which can be used anywhere from
within
an ESMF application to obtain the current VM context without the need
to
provide any input information.

Component
wait can now be called even on not currently running Components, in
which
case it will return immediately.

Buffering
support has been added to LogErr. Writes to a Log now go through an
adjustable
buffer to improve performance.

Every
entry
in a Log now has the PET number prepended to the entry.

Multilog
support has been added to LogErr. When the multilog option is set
during
Log initialization, separate Log files are created for the output from
each
PET.

Units
of
fractional seconds (specified with integers) are now supported within
ESMF_Time
and ESMF_TimeInterval. The shortcut arguments ms, us, and ns
(millisecond,
microsecond, nanosecond) for Set()/Get() methods can be used as well as
the
more general arguments sN and sD (numerator, denominator).
Fractional
seconds in Times and Time Intervals may fully participate in Clock,
Alarm
and Calendar objects and methods. They also are fully functional
within
arithmetic (integer based) and comparison operations.

The
following
Time Manager bugs/enhancements have been fixed/done:

The
Alarm
limit per Clock is now dynamic in increments of 200.

ESMF_ClockPrint()/ESMF_TimePrint()
now correctly adjusts for leap years.

The
implicit
day boundary in ESMF_TimeGet() has changed to a user-supplied boundary
(fixed
bug 1099731).

An
Alarm
will now ring immediately if its ring time is set equal to the Clock
start
time.

Can
now
drive No-Leap and 360-Day calendar-based times with seconds only.

Direct
errors are now generated when dd < 1 or mm < 1 in ESMF_TimeSet().

Can
now
call ESMF_TimeSet() with only the dd argument; yy defaults to 0 and mm
defaults
to 1.

ESMF_ClockValidate()
now works correctly after the end of an ESMF_ClockAdvance() loop.

The ESMF_StateReconcile() call, generally used by
couplers dealing with concurrently running Components, has an internal
size limit (which will be addressed in a subsequent release). There is
no hard-coded limit to how many elements, i.e. States, Bundles, Fields,
Arrays can be added to the State. However, there is a limit, roughly
2, on the maximum number of sub-elements contained by any element in
the State passed into ESMF_StateReconcile(). ESMF_StateReconcile()
causes a core dump
if the buffer size is
exceeded.

Calling the redistribution and halo functions from a
parent, coupler, or higher level Component fails on ESMF_Fields created
in a child Component if that child is running on only a subset of the
caller Component's PETs. This is the case, for example, when
child
Components are running concurrently. ESMF_FieldRegrid() does work
correctly in this situation. Redistribution and halo functions
work
correctly when called from a child Component on ESMF_Fields created in
that child Component.

The default
compiler on the Compaq (halem), Fortran 551J, is compatible with
this
release. Fortran 551L core dumps when running system tests.

ESMF_BundleGather(), ESMF_FieldGather()
and ESMF_ArrayGather() are only expected to work on 2D grids with 2D
data.

When
ESMF_Array
objects are created with a non-zero halo width, that width is added to
all
dimensions of the array, not only ones which correspond to the
decomposed
grid axes.

On
AIX
systems there is a problem in the XL Fortran compiler version 8.1.1.6,
when
compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler
version
8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or
ESMF_BOPT=O
also work with either compiler version.)

When
compiling
with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are
configured
with:
#define MPI2CPP_HAVE_BOOL 1
which may not be the default output from configure.

Main
programs
in C/C++ do not work with the PGI compiler.

The
ESMF_StateReconcile()
call does not work on the Linux and Altix builds of ESMF. The
ESMF_FieldExcl
system test uses this call, so it will not work on these platforms
either.

On
the
Cray X1, the __LINE__ macro does not work in F90; "line 8" is always
printed
from methods such as ESMF_Test().

The
ESMF_VMGetMPICommunicatorEx
example fails on the Linux/intel 64-bit build.

The
maximum
number of alarms per clock has been increased to 200, from 100
previously.

Bug
996229
fixed; "Non-sticky alarms not functioning properly".

Build
and
test fixes:

ESMF_BUILD
fixed for building files in a different directory from the source.

Automated
nightly unit tests now run multiprocessor on some platforms.

Documents
now build without having to compile the library first.

More
documentation
added for Grid and Regrid functions.

The
default
compiler on the Compaq (halem), Fortran 551J, is compatible with
this
release. Fortran 551L core dumps when running system tests.

ESMF_BundleGather(), ESMF_FieldGather()
and ESMF_ArrayGather() are only expected to work on 2D grids with 2D
data.

When
ESMF_Array
objects are created with a non-zero halo width, that width is added to
all
dimensions of the array, not only ones which correspond to the
decomposed
grid axes.

This
release
returns an internal compile error with the PGI/Linux compiler and
ESMF_BOPT=g.
It builds successfully with ESMF_BOPT=O.

There
are
known problems running with -O on Mac OS X compiling with absoft and
running
with lam mpi. The workaround is to compile with ESMF_BOPT=g.

On
AIX
systems there is a problem in the XL Fortran compiler version 8.1.1.6,
when
compiling with the exact combination ESMF_PREC=64 and ESMF_BOPT=g.
(Compiler
version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32
or
ESMF_BOPT=O also work with either compiler version.)

When
compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH
libraries
are configured with:
#define MPI2CPP_HAVE_BOOL 1
which may not be the default output from configure.

Main
programs
in C/C++ do not work with the PGI compiler.

Routes
will only work on PET counts that are powers of 2.

Some
regrid
results may show "banding" when regridding between decompositions where
processor
domains overlap only slightly.

Halo
does
not work correctly for periodic boundaries on non-decomposed axes.