pandas is a fantastic library for analysis of low-dimensional labelled data -
if it can be sensibly described as “rows and columns”, pandas is probably the
right choice. However, sometimes we want to use higher dimensional arrays
(ndim > 2), or arrays for which the order of dimensions (e.g., columns vs
rows) shouldn’t really matter. For example, climate and weather data is often
natively expressed in 4 or more dimensions: time, x, y and z.

Pandas has historically supported N-dimensional panels, but deprecated them in
version 0.20 in favor of Xarray data structures. There are now built-in methods
on both sides to convert between pandas and Xarray, allowing for more focussed
development effort. Xarray objects have a much richer model of dimensionality -
if you were using Panels:

You need to create a new factory type for each dimensionality.

You can’t do math between NDPanels with different dimensionality.

Each dimension in a NDPanel has a name (e.g., ‘labels’, ‘items’,
‘major_axis’, etc.) but the dimension names refer to order, not their
meaning. You can’t specify an operation as to be applied along the “time”
axis.

You often have to manually convert collections of pandas arrays
(Series, DataFrames, etc) to have the same number of dimensions.
In contrast, this sort of data structure fits very naturally in an
xarray Dataset.

The main distinguishing feature of xarray’s DataArray over labeled arrays in
pandas is that dimensions can have names (e.g., “time”, “latitude”,
“longitude”). Names are much easier to keep track of than axis numbers, and
xarray uses dimension names for indexing, aggregation and broadcasting. Not only
can you write x.sel(time='2000-01-01') and x.mean(dim='time'), but
operations like x-x.mean(dim='time') always work, no matter the order
of the “time” dimension. You never need to reshape arrays (e.g., with
np.newaxis) to align them for arithmetic operations in xarray.

We are firm believers in the power of labeled data! In addition to dimensions
and coordinates, xarray supports arbitrary metadata in the form of global
(Dataset) and variable specific (DataArray) attributes (attrs).

Automatic interpretation of labels is powerful but also reduces flexibility.
With xarray, we draw a firm line between labels that the library understands
(dims and coords) and labels for users and user code (attrs). For
example, we do not automatically interpret and enforce units or CF
conventions. (An exception is serialization to and from netCDF files.)

An implication of this choice is that we do not propagate attrs through
most operations unless explicitly flagged (some methods have a keep_attrs
option, and there is a global flag for setting this to be always True or
False). Similarly, xarray does not check for conflicts between attrs when
combining arrays and datasets, unless explicitly requested with the option
compat='identical'. The guiding principle is that metadata should not be
allowed to get in the way.

netCDF4-python provides a lower level interface for working with
netCDF and OpenDAP datasets in Python. We use netCDF4-python internally in
xarray, and have contributed a number of improvements and fixes upstream. xarray
does not yet support all of netCDF4-python’s features, such as modifying files
on-disk.

Iris (supported by the UK Met office) provides similar tools for in-
memory manipulation of labeled arrays, aimed specifically at weather and
climate data needs. Indeed, the Iris Cube was direct
inspiration for xarray’s DataArray. xarray and Iris take very
different approaches to handling metadata: Iris strictly interprets
CF conventions. Iris particularly shines at mapping, thanks to its
integration with Cartopy.

We think the design decisions we have made for xarray (namely, basing it on
pandas) make it a faster and more flexible data analysis tool. That said, Iris
and CDAT have some great domain specific functionality, and xarray includes
methods for converting back and forth between xarray and these libraries. See
to_iris() and to_cdms2()
for more details.