Sunday, March 31, 2013

I liked this letter which appeared in Nature recently. Not just because I'd done something similar myself with the earlier Hansen forecast :-) In general, I think it's important to revisit historical statements to see how well they held up. Allen et al have gone back to a forecast they made about 10 years ago, and checked how well it matches up to reality. The answer is...

really well. On the left, is the original forecast with new data added, and the right is the same result re-expressed relative to a 1986-96 baseline. The forecast was originally expressed in terms of decadal means, so I don't think there is anything untoward in the smoothing. The solid line black in the left plot is the original HadCM2 output, with the dashed line and grey region representing the adjusted result after fitting to recent (at that time) obs using standard detection and attribution tchniques.

They also compared their forecast to a couple of alternative approaches:

This plot shows the HadCM2 forecast (black), CMIP5 models (blue) and another possible forecast of no forced trend, just a random walk (green). The red line is the observed temperature. They point out that their forecast performed better than the alternatives, in the sense that it assigned higher probability (density) to the observations.

So far, so good. However, I disagree with their statement that "the CMIP5 forecast also clearly outperforms the random walk,
primarily because it has better sharpness" (my emphasis). Actually, the CMIP5 forecast outperforms the random walk simply because it is clearly much closer to the data. The CMIP5 mean is about 0.41 in these units (all these numbers are just read off the graph, and may not be precise), the random walk is of course 0, and the observed anomaly is 0.27. The only ways a forecast based on the CMIP5 mean could have undererformed the random walk would have been if it was either so sharp that it excluded the obs (which in practice would mean a standard deviation of 0.06 or less, resulting in a 90% range of 0.31-0.51), or so diffuse that it assiged low probability across a huge range of values (ie, a standard deviation of 0.75 or greater, with associated 90% range of -0.8 to 1.6). The actual CMIP5 width here seems to be close to 0.1, well within those sharpness bounds.

I do think I know what the authors are trying to say, which is that if you are going to be at the 5th percentile of a distribution, it's better to be at the 5th percentile of a sharp forecast than a broad one. But changing the sharpness of the forecast based on CMIP5 would obviously mean the obs were no longer at the 5th percentile! In fact, despite not quite hitting the obs, the CMIP5 forecast is not that much worse than the tuned forecast (black curve), thanks to being quite sharp. And according to the authors' own estimation of how much uncertainty they had in their original forecast, they obviously got extraordinarily lucky to hit the data so precisely. With their forecast width, it would have been almost impossible to miss the 90% interval - this would have required a very large decadal jump in temperature. I don't think it is reasonable to say that one method is intrinsically better than the other, on the basis of a single verification point that both methods actually forecast correctly. If the obs had come in at say 0.4 - which they forecast with high probability - I hardly think they would have been saying that the CMIP5 ensemble of opportunity was a superior approach.

(For what it's worth, I think the method used in this forecast intrinsically has exaggerated uncertainty, but that's another story entirely.)

"Work by Julia Hargreaves of the Research Institute for Global Change in
Yokohama, which was published in 2012, suggests a 90% chance of the
actual change being in the range of 0.5-4.0°C, with a mean of 2.3°C"

which is referring to this, but the rest of the article seems mostly pretty good too. I do wonder about: "Nic Lewis, an independent climate scientist, got an even lower range in a
study accepted for publication: 1.0-3.0°C, with a mean of 1.6°C." I've not heard anything from him for some time, and I wasn't convinced last time. But the overall message of a lowering probability of a high sensitivity is hard to deny. Unless you are Reto Knutti, that is, in which case "my personal view is that the overall assessment hasn’t changed much". Of course he was only speaking personally there, and what matters in his role as IPCC lead author, is what the (credible) literature actually says.

(Yes, I know that technically it's not quite the same thing. Actually, a few years ago a colleague published something in a Hindawi journal, presumably a naive victim of a spam email. See also this article.)

Oh noes - on clicking through to the BEST paper, I see they have a second paper in the same journal! It's about the urban heat island, and according to the abstract, it concludes....drum-roll....that there isn't a significant contamination of the global temperature record. Phew. Somehow they failed to cite our magnum opus on that topic :-)

Rumour has it, their third paper addresses the defecatory habits of ursines in arboreal environments.

Let's face it: pay-to-view scientific publishing is dead, even though its zombie corpse is still staggering around, thrashing around aimlessly.

There's another article in Nature about open access (and at least this one isn't hidden behind a paywall). Actually, it's not that bad, and certainly doesn't seem to be agitating shamelessly in the way that some past articles appeared to be. Among the open access publishers they discuss, there is however one notable absentee: the entire EGU family of journals is conspicuous by its absence. Of course this is only one publishing house operating in a particular field of research, but it's a very big one, and within that area, certainly ACP, BG, CP and probably TC (I don't really know the latter) are important within their fields. It may be worth emphasising that as well as being economically viable (indeed comfortably profitable), the costs of the EGU journals are extremely low, often lower than the publication charges imposed by paywalled journals even before you consider what they are raking in through subscription fees. That makes the excuses of for-profit publishers hard to take seriously. What are they actually adding for their fees?

Although I recently pointed to some dodgy papers in EGU journals, I don't think they are any worse than the AGU or other publishers who use a paywall paradigm - rather, my concern is that I expect them to be better, given the open review and opportunity for additional unsolicited comments. But even Nature, with it's $30-40,000 of investment in every paper, manages to come up with its share of stuff that is known to be wrong before the ink is dry. One nice feature of the EGU system is that you can see the reviews, and in the cases I mentioned, it seems that the problem (if there is one) is that the eds are bending over backwards to be generous towards papers that have been roundly rubbished in review. It is important to maintain some sort of standards, if reviewers are going to be expected to donate their time and energy. It might be useful to see the second and subsequent rounds of reviews, and I'm not sure why this bit is kept secret.

Incidentally, something I have been agitating for over recent years has recently come to pass: there is now a "subscribe to comments" button on each discussion page! So if you spot an interesting manuscript under review, you can easily keep an eye on what the reviewers are saying. I hope this will lead to an increase in non-invited comments. There are also RSS feeds for both the discussion and final publication phases of the journals.

Of course, Nature aren't stupid, and while trying to defend their cash cow for as long as possible, are also increasingly buying in to the open access model. It's just a matter of time. If they can persuade either authors, or funding bodies, that they add up to $40,000 of value to every paper they publish, then maybe they get their money in other ways, and good luck to them. It's time to stop gouging readers who have already paid for the research with their taxes.

Thursday, March 28, 2013

I've been meaning to write about decadal prediction for some time, but kept on procrastinating, and the number of relevant papers has increased beyond the scope of a single post. So I'll do a number of short posts instead, until I run out of things to say, or interest, or readers...

First off the bat is this new paper from Geert Jan van Oldenborg et al. I remember meeting Geert Jan way back when I was first learning about data assimilation (with the help of google, I believe it was here) and he was scarily clever back then, but his primary focus has been on shorter-term prediction, so our paths only cross occasionally. Some readers may know of him through his Climate Explorer web portal thingy. In this paper he looked at the reliability of the trends of the CMIP5 ensemble over the last 60 years. We actually did something similar in a rather perfunctory way as part of this paper for CMIP3 (eg Fig 1) and also more recently here for CMIP5 (see Fig 2) but these papers were looking more at equilibrium climatologies and also single (perturbed parameter) versus the multi-model ensembles. When looking at the raw temperature trends from the models, this new paper gets a similar result to us, that the rank histogram is near enough flat to be called reliable, though there's a moderate tendency towards the models warming too much:

(The red line shows the histogram of the rank of the observed trend at each grid point, within the CMIP5 ensemble spread - ideally, it would be flat, and the slope up to the left means that there are relatively more obs in the low end of the model range than at the top end.)

But they then tried an additional step, to look at the regional trend relative to the global mean temperature for each model and obs. And in this case, the obs frequently lie outside the model (ie, the big bins at each end of the red histogram below):

The implication of this different result is that the
models have insufficient spatial variability in their regional trends (which is consistent with what others have shown too) -
broadly speaking, the models that matched the highest observed regional trends in the previous analysis, did so by
warming a lot globally, and those that matched the lowest observed trends,
warmed only a little everywhere. So, to the extent that the models did get the right results overall (in the first analysis), they did it by having a wide range in global responses which makes up for their unrealistically low spatial variability. The precipitation trends, however, are poor even without this extra normalisation step. There are a number of possible explanations for this behaviour, but the result is, as they say, "This implies that for near-term local climate forecasts the CMIP5 ensemble cannot simply be used as a reliable probabilistic forecast."

One thing I'd have liked to see is an investigation of the robustness of their result with respect to observational errors, which they don't seem to account for. There are some places where the observed trend seems to vary wildly between adjacent grid boxes, which seems physically unlikely. If not corrected for, obs errors will tend to increase the end bins of the histogram. It would be unreasonably optimistic to expect the ensemble to be perfectly reliable in all respects so I don't doubt the overall conclusion. But it would be interesting to see.

Wednesday, March 27, 2013

James hasn't quite got pizza construction sorted. The day before our symposium started, his latest attempt levered out my temporary dental bridge. So I got to scare the visitors throughout the symposium with my inability to say "F". I wonder if that's why they were all so well behaved. Anyway, on Friday it was back to Tsurumi dental hospital for some more construction.

The good news is that it may all get finished in mid-September.

...unlike the bridge construction near work, which, if the signs are to be believed wont be finished for another year or so! I can only assume that the final thing is to be more than just random bridging.

Monday, March 25, 2013

I've always been a fan of the EGU open review process, so it's a bit disconcerting to see a number of dodgy papers being published recently.

First we have the Makarieva et al stuff on ACP. This got strongly criticised in review and on a number of blogs. There were several long threads on Judith Curry's blog, and she said in her public comments that the manuscript required significant changes before it could be considered publishable. She was notably more reticent when Willard asked repeatedly whether she thought the published paper had fixed the problems. The editor implies pretty clearly that they think the theory has major problems but deserves a hearing. In some previous cases, heavily debunked and criticised papers have been rejected, partly with the explanation that the argument is already out in the open for all to see. (Update: the particular example I was thinking of, but had forgotten the details, was specifically the previous Makarieva manuscript on the same topic, which went to full editorial board appeal.) It is hard for me to see how the final manuscript addresses the major criticisms levied at it, though it is not my area of expertise.

ESD had a bit of a disaster with this paper. It is clear that the editor had her doubts, but the very limited reviews included a very positive (albeit partisan and nonsensical) one. I know from personal experience that finding reviewers is often a difficult task. The flip side is, that if several reviewers give generously of their time and effort to provide intelligent and useful reviews, they will be unimpressed by an editor who over-rides their input, as has apparently happened in these cases. There's a relevant manuscript under review here.

Another odd CP paper appeared not so long ago, despite a succession of hostile reviews. The Editor Eduardo Zorita has just broken cover and presented his view of what happened (perhaps provoked in part by a poorly-aimed jibe here a couple of days ago). Irrespective of the ins and outs of the tale, I'm a bit surprised by his claim to be satisfied at the way the process has turned out. As editor, he ultimately carries the can for the decision to publish, and the paper seems obviously silly to me, so I don't think it reflects very well on him that he approves of its publication. The paper presents an 18-parameter (yes really) fit to a highly smoothed time series, and concludes:

"It shows that the climate
dynamics is governed at present by periodic oscillations."

Saturday, March 23, 2013

If I was better at Japanese it wouldn't have been so bad - 5 minutes from every sub-leader in the ICA-RUS project... At least there was a giraffe...

Last month we saw a real giraffe in Yokohama. It was a little surreal... spotted between the skyscrapers through the picture windows as we were enjoying medicinal steak at TGI Friday's. Eventually we worked out that we were looking down on a circus. An elephant appeared at one point too. Anyway, only metal giraffes to be found today, although of course you never know what might be lurking inside the skyscrapers...

I might have mentioned before, that our research here is currently supported mostly by two 5-year projects. Of course, all projects require snappy acronyms (like JUMP). One of these projects is run by our friend Emori-san at NIES, who asked for our help in coming up with an appropriate English name. We eventually settled on ICA-RUS (Integrated Climate Assessment - Risks, Uncertainty and Society), which as well as referring to the main themes of the project also nods at the danger of overheating.

The other project is actually headed up by our lab, so of course no-one thought of asking our opinion. It is now called PRICC.

Tuesday, March 19, 2013

While jules was busy getting her hair pulled over there, I was having an exclusive interview with David Rose of the Daily Mail, some of which appeared in this article (apologies to anyone who suffers an allergic reaction to any of those words - firefox users may find this add-on useful).

In the interests of openness, here is the full transcript...

The reality is that there was no interview: he never even contacted me to check he had represented my views accurately, just as he didn't ask Ed Hawkins before apparently plagiarising his graph (and misrepresenting it into the bargain).

The "increasingly untenable" quote seems to have been pulled off the Revkin article (without attribution, naturally) which quoted me recently. Andy Revkin did exchange a few emails with me to ensure he had fairly represented my view, and I have absolutely no complaint with him on that score. The bit Rose adds about "the true figure likely to be about half of the IPCC prediction in its last report in 2007" is a complete fabrication of course, it's not something I can imagine having said, or being likely. I do think the IPCC range is a bit high, expecially the 17% probability of sensitivity greater than 4.5C. But their range, or best estimate, is certainly not something I would disagree with by a factor of 2. See here for some more extensive recent commentary from me.

Given all that, it's perhaps a bit pointless to comment on the other opinions quoted by Rose, as they may also be lies. However, Piers Forster appears to defend his (IMO reasonable) comment (though not the article as a whole). I'm a little more surprised by the comment attributed to Myles Allen - if he really thinks the higher estimates are "looking iffy" then it's hard to think of anyone who could still defend them. It wasn't long ago he was arguing that the IPCC projections were too optimistic.

Monday, March 18, 2013

It's official: I'm not-boring! How do I know?! HvS has interviewed me as part of his ongoing anthropological studies of "interesting" scientists. You can read the interview here, and the more nicely formatted PDF for you to download and keep is here. It is an amazing honour, although I know I'm as nothing on the "interestingness" scale compared to the likes of Yamagata-sensei, a previous interviewee.

Saturday, March 09, 2013

For 12 years I've admired this big old house from the back of the mountain tandem. But now James is doing his silly running, I have control of my brakes, so it is possible to stop and take photos!

But the thing that made me actually stop was the meeting of old and new Japan. These are the shocking next door neighbours to the old house, that appeared a few years ago. Bet each house has a Stepford wife inside.

Friday, March 08, 2013

It's sadly predictable, but still sad, that the spate of articles anticipating the 2-year anniversary of the March 2011 earthquake/tsunami can provide little more than a litany of bureacratic failure, despondency and stagnation.

Of course, rebuilding these economically nonviable retirement communities in the middle of nowhere was never a sensible solution, but it's a shame that no-one has managed to provide, assume, or assert sufficient leadership to actually drag the relevant parties towards sensible solutions. The emergency housing, which was initially designed for (and legally limited to) up to a year of use is now apparently going to be in operation for up to 10 years, or more prosaically, until sufficient numbers of its aging residents die out.

There are a couple of bright spots, where some fishermen are setting up local cooperatives. But I think they were struggling to find local labour even before the tsunami, with school-leavers heading off to seek their fortunes in the cities.

A P.S. from jules:
Here's part of the bridge over nothing... our legs are usually too tired from cycling or running to work to want climb these things unecessarily... so we'll still just cross the road "normally".