Thursday, 1 October 2015

migration of all components of a previously distributed system to one location where all data are held and significant processing power is available

improvements to the way we derive sea surface temperature to achieve the stringent requirements of our users

Making improvements is an iterative process, where we introduce innovations progressively and validate they have positive impact. The team is currently assessing our most recent large-scale reprocessing, for which the new smooth-atmosphere algorithms have been used, along with other minor changes (e.g., to the prior assumptions used in optimal estimation).

The summary is that the stability of the AVHRR data streams is clearly improved, both with respect to the calibration anomalies on some instruments and sensitivity to desert dust outbreaks over the ocean.

Being experimental, we are not promoting use of this version of the data, but it can be shared with any interested researchers.

The next experiment reprocessing run will attempt to derive SSTs for the 1980s using our system, for the first time. It is due around the end of this year.

Tuesday, 14 July 2015

There are many effects that act as sources of errors in a climate data record. No measured value is perfect, whether taken in the laboratory or inferred from radiances measured in low Earth orbit.

The most obvious characteristic to establish about a source of error is the magnitude of its effect. The (standard) uncertainty is a measure of the typical size of errors, and is generally what is represented by "error bars" on a plot.

Less obvious is the need to know whether the error is correlated between different measured values. This becomes very important for climate data records: looking at climate change, highly correlated errors are the ones we need to worry about.

To illustrate this, consider a climate data record (CDR) for sea surface temperature (SST). Full resolution satellite data typically measures an instantaneous SST across a pixel of about 1 km. For a typical case, we might have three categories of effect, causing uncertainties of different size.

Noise, which is uncorrelated between different pixels.

Ambiguity in obtaining the SST from the measured radiances, which tends to be correlated "locally", where the state of the atmosphere is similar.

Systematic errors ("biases") including sensor calibration degradation over time, which tend to affect measured values in a highly correlated (non-random) way.

Data producers tend to put effort into correcting any "biases", but nonetheless, residual uncertainty remains after such corrections are applied.

For a single instantaneously measured SST from a well-designed instrument, the first two effects are the biggest, and may be comparable.

However, for climate change analyses, we may be more interested in how SST over one area and period of time compares with SST for that area at a different time. The larger the spatial scale and longer the period of time considered, the less important noise becomes (the SST errors tend to average down when data are aggregated) and the relatively more important calibration effects become.

This figure illustrates this effect for a reasonable set of assumptions for the case of SST. In a single 1 km SST retrieval, the random effects may dominate the uncertainty, followed by the locally systematic effects associated with retrieval ambiguity. However, as the scale of analysis of the SST data becomes larger in space and longer in time, first noise and then locally systematic effects become less important. If we are using the CDR across several years, more than one sensor is involved, and not only systematic calibration effects from a single sensor matter, but uncertainty in the (corrected) systematic differences between instruments in the sensor series also begin to matter.

Analyses of uncertainty budgets in SST products and for instrument design often focus on the regime at the left side of this diagram, where calibration effects (after bias correction) may be thought negligible. But CDRs also get used for applications at the right side of this diagram, where the systematic effects matter most.

When it comes to creating climate data records, all categories of effect causing uncertainty need to be considered and characterised, as far as practicable. All types of error source are relevant to the applications of at least some CDR users.

Friday, 26 June 2015

If you want some of our data in the simplest possible form, head to our new 'easy download' page. You can get the monthly SST time series for a particular latitude and longitude in a format readable into a spreadsheet.

For some applications, that is all someone needs, and deriving such data from the full archive could be too much work to be justified for a modest application. For this reason, Guy Griffiths has created this simplest-possible interface.

More difficult is to make sure the right people find this page when they hunt for simple SST data. You could help by linking there from any SST-relevant web pages you control: promote the page up the search-engine rankings!

Wednesday, 17 June 2015

German TV company ZDF will screen a documentary on Sunday, 21st of June, 7:30 pm Berlin time, which features an nice animation of the East Australian Current done by Guy Griffiths using the SST CCI analysis data. Here is the trailer (although the animation doesn't feature in it).UPDATE: Here is a clip showing the (brief) appearance of the data, in the context of a discussion about coral's sensitivity to water temperatures. The colours represent the SST relative to the annual average at a given latitude, and the warming off Australia is seasonal.

Friday, 5 June 2015

There has been much recently published about why, compared to the 1980s and 1990s, there has been a slowdown (or even a "pause" or "hiatus") in global warming. Burial of heat below the surface of the ocean has been fruitfully investigated. At the same time, commentary has pointed out

that the perception of a hiatus in part results from considering a time interval that starts with a year that was (back then) unusually warm (1998)

decade-to-decade variations in short-term (~10 year) climate warming rates are to be expected

Now Karl et al. in Science have declared the non-existence of a slowdown, after reassessing observational data.

They are right to make scrutiny of the observations part of the search to understand any (apparent) hiatus. Global warming is important for us in the long run, but is a subtle change to estimate over a single decade (~0.1°C global change). Part of the challenge is the variability of nature year to year, region to region. The other aspect is ensuring consistency across the changing mix of measurements that go into constructing a long-term dataset.

The major component of their re-appraisal is a revised estimate of the global sea surface temperature (SST) trend upwards to about 0.08°C between 1998 and 2012 (from about 0.01°C in IPCC). This revision comes from changing the assumptions about the relative biases between and the relative weight it is appropriate to give to drifting buoys and ships. This revision amounts to 1/200th °C/year.

When talking about a revision of temperatures that is so subtle, it is prudent to be cautious. Karl et al. recognise this in the title of their paper "Possible artifacts of data biases in the recent global surface warming hiatus" (my emphasis).

How can we gain more confidence about the recent global sea surface temperature changes? SST CCI's contribution is to develop a high stability, independent dataset of SST. We attempt this using satellite-derived SSTs, based on physics (not tuning to drifting buoys or other in situ observations). If we see compatible behaviour in independent datasets, that behaviour is unlikely to be driven by data artefacts.

Compared to an in situ data set, we in SST CCI have fewer instruments whose long-term calibration we need to worry about. On the other hand, the satellite measurement, being from space, is indirect, through the atmosphere; we go to considerable lengths to remove any residual atmospheric artefact in the SST time series.

Looking at our ATSR-basedobs4MIPS dataset, the global mean SST trend (not including sea ice areas) over 1998 to 2012 [1] is 0.085°C, which is 0.06°C/decade[2].

Karl et al.'s "new" value of SST change over this interval therefore fits pretty well with our independent [3] satellite data. These data featured in IPCC AR5. (They also agree well with the Hadley Centre in situ ensemble in the same figure.)

So, how new Karl et al.'s result is depends on what data you previously paid attention to.

In SST CCI we also have higher resolution data from blending in more types of satellite data, which, over 1998 to 2010 [4], has the same global mean SST trend of 0.06°C/decade. Patterns of warming tend to be more interesting that global mean numbers, and are related to the dynamics of heat uptake by the ocean. We need to understand how energy gets distributed around the climate system in order to understand decade-to-decade variations in global warming rates.

Notes:

1. The last ATSR failed in March 2012, so the period is not quite the full 1998 to 2012 interval.

2. The best uncertainty estimate that we can provide is that we have 95% confidence that the trend is between 0.04 to 0.085°C/decade. This is the trend uncertainty from the stability of the observations. We can only be confident that this trend uncertainty is valid in tropical regions, because the excellent global tropical moored buoy array is the only in situ reference we can compare with over that period where the thermometers are accurately pre- and post- calibrated. The fall-behind in maintenance of the array was very unhelpful, as it undermines this sort of long-term stability assessment for the whole satellite SST constellation, which has implications for knowledge of climate globally.

3. I think it is important that in situ and satellite observationalists try to preserve this independence by not letting too much understanding of each other's biases leak between the in-situ-only versus satellite-only datasets. But we also need to work together on getting the best possible reconstructions of SST using blends of in situ and satellite SST. Tricky when we are the same people!

4. The ongoing phase of SST CCI aims to extend this SST analysis to cover 1981 to 2016, over the next 2 years. But for now, it ends with 2010.

5. Thanks to Owen Embury and Simone Morak for help in putting this post together.

Friday, 15 May 2015

New sea surface temperature (SST) products were generated experimentally by SST CCI in March 2015. These included full resolution data from the Advanced Along Track Scanning Radiometers using a new retrieval method that reduces the noisiness of full resolution SSTs. Although ATSRs are generally relatively low noise sensors, the process of low-bias SST determination amplifies the sensor noise and creates images that look noisy. Noise filtering of the SST could address this cosmetically, but this would degrade the feature resolution and real data quality, and is not satisfactory. Instead, a new theoretical development by SST CCI has been tested: namely, multi-channel smoothing of the atmospheric influence on observed AATSR brightness temperatures. The conditions of the atmosphere vary more slowly with horizontal distance than the SST. The process of SST retrieval can be viewed as estimating the atmospheric influences on brightness temperatures and removing them to reveal the SST that underlies the atmosphere. The new method estimates atmospheric influences across a wider area around a given image pixel, but treats the SST estimation at full resolution. Compared to older methods (left panel of figure), the new SSTs are less noisy (middle panel) — NOT because the SST has been smoothed (it hasn’t), but because the new retrieval method amplifies noise less and gives a more faithful rendering of the true SST variation.

Left: full resolution (1 km) AATSR SST image from previous project. Clouds over the ocean are rendered on a grey scale, and the green feature is land. The rendering of temperature is such that the coolest temperatures are pale blue and warmer temperatures are deep blue. Middle: new result, using new technique to reduce noise amplification in the process of retrieval. Right: noise removed.

The noise removed by the new method compared to old is shown in the right panel. The noise should look completely random, but in fact there are some streaks where there is a tendency to blue or red, indicating non-random effects. This appears to be a result of imperfect co-registration of the forward and nadir views in the older imagery (we have improved the offset correction between the views in the latest experimental products as well).