I won't be taking the usual line of consideration here (surprise!). Rather, let's go back to talking about peer review. As my wife said, it is not God review -- reviewers and editors are human, and therefore make mistakes. At the end of peer review, therefore, you don't have gospel, you have something that has a fairly good chance of being worth your time to read. To rephrase Wagner, papers that pass peer review should at least not contain fundamental errors of method or false claims. And it now seems likely to him that this paper (Spencer and Braswell) may well not pass that standard.

Richard Feynman's comment about fooling yourself is commonly quoted:We've learned from experience that the truth will come out.
Other experimenters will repeat your experiment and find out whether you
were wrong or right. Nature's phenomena will agree or they'll disagree
with your theory. And, although you may gain some temporary fame and
excitement, you will not gain a good reputation as a scientist if you
haven't tried to be very careful in this kind of work. And it's this
type of integrity, this kind of care not to fool yourself, that is
missing to a large extent in much of the research in cargo cult science.
"Cargo Cult Science", adapted from a commencement address given at Caltech (1974)

This underlies some parts, I think, of the failure in Spencer and Braswell's work, and the subsequent failure in the review and editorial process. Namely, it is assumed by reviewers and editors that authors have already done some work doubting themselves and checking to see how it is they might have fooled themselves -- and to take action against such possibilities. Further, the review process is based on the presumption that your purpose in publishing is to advance our understanding of science.
Wagner mentions one of those non-scientific aspects to the paper -- the public exaggeration of the paper's conclusions by the author himself. What the university press releases and certain media outlets do in exaggerating is pretty much outside the author's control. But Spencer's own exaggerations indicate (my thought) that his purpose was to make those grand claims, rather than to make an incremental improvement to our knowledge of how the universe works. As Wagner mentions, no single paper looking at a single data source, comparing with a single model, is capable of refuting all science on global warming.

A different non-scientific point, endemic to any of Spencer's work, is that he doesn't fundamentally view himself as engaging in science and an effort to understand the universe; he says "I view my job a little like a legislator, supported by the taxpayer, to
protect the interests of the taxpayer and to minimize the role of government." Note that -- his job. There are scientists who pursue political goals, whether to minimize government, or prevent what they think are bad political decisions -- but it's on their own time, as private citizens -- not as their job. The particularly active ones I know have always had a clear distinction between their job and their activities as a private citizen.

With Spencer, though, he views is job as being to achieve a political end. As such, science takes a back seat. This paper of his, then, is less a matter of trying to advance our understanding of the world, and more (as he views it as his job) a matter of trying to achieve a political result. Peer review was not developed for such things, and does not manage them well. When science is not the primary purpose of a paper, that presumption reviewers and editors make that the author has done his due effort at avoiding fooling himself is false.

Some comments have been made hither and thither about how the criticisms, like Bickmore's, of Spencer's work are not in the peer-reviewed literature. They then conclude that this means that Spencer is right and Bickmore (and others) are wrong, or the criticisms can, at least be ignored. If the authors have done their due efforts to not fool themselves, this may well be true. In such efforts the fundamental errors Wagner mentions would not be present.

Instead, as was the case for McLean et al., some errors were so elementary that they could be analyzed and discussed at blog-level. Granted I may often leave some of the middle-school kids behind, but Bickmore's analysis, for instance, should have been manageable by anybody with an undergraduate major in a math/science/engineering field.

There's a flip side to that. You seldom can publish things at that level in scientific journals. They're 'too obvious', or 'uninteresting', and the like. Consequently, if the errors in a paper are elementary enough, there will be no peer-reviewed literature to cite regarding the error. Authors are supposed to have done that checking themselves, and if they fail to, the reviewers are supposed to mention them. If both fail, well, now we do have the possibility of writing blog comments on the failings of a paper.

For comments and discussion here, I will encourage you to focus on the peer review process itself. I give links below for the science in the paper, and others for the story itself if you're more interested in those things.

Along the lines of what you've said, I think papers of irredeemably low quality just end up ignored. You read it, you realise the authors made some basic errors, you sigh, and you leave it aside. One might give it to a beginning grad student as an exercise in critical reading. Years later, you'll see that the paper has gone uncited, and was generally ignored. But what do you do when certain elements in society and the media seize upon the paper? In this case, a formal reply or comment seems to be called for. But it isn't necessarily easy to get a comment published, even if the comment has merit, and even if the journal will take a comment, it may have to be short. So it's hard to see what the best path is.

Are there useful analogues one can study from other fields, such as stratospheric ozone chemistry or evolutionary biology?

Anders: Spencer did elaborate that he didn't think he _was_ a legislator, just that he takes his mission to be something _like_ a legislator. I think your observation still holds. I want my scientists to be doing science on the job, not politics. (Since Ray is working on federal grants, and I pay taxes, he's one of 'my' scientists.)

The usual comment/reply mechanism is, like the review process itself, aimed at papers which are written on the aim of advancing our scientific knowledge. And they've done the responsible effort to avoid fooling themselves.

When an author considers his primary purpose to be political, rather than scientific, those presumptions fail.

The usual comment must have some real substance of its own. That's because the original paper should be substantive itself. One can have difficult issues in science, where it is hard to say what is the right way to go. The original paper may not be right, but after making serious effort at not fooling themselves, and the reviewers looking to ensure that this is the case, it enters the scientific discussion. Part of that discussion can be a reply, picking up one of those points that is hard to decide and making the case that it is important enough to cause reconsideration of the paper. I'm finishing formatting of a paper in which the back and forth between original author and commenter is useful in laying out a point in favor of my (brilliant of course) idea.

When, instead, the level of error includes things that undergraduates in the area would get downgraded for, now what? There's no real standing mechanism for what and how to deal with that. Historically, the desire to publish was a desire to share one's discoveries about the natural world -- not a matter of furthering your political agenda. So mechanisms are lacking. It looks like science blogs are developing as one of the means of response. But probably another decade (to make up a number) before it gets to be as normal and worked out a route as the review process itself (something which evolved in response to the expanding population of scientists).

For analogues, one which occurs to me is S. Fred Singer regarding the Ozone Hole. His comments on that were basically ignored, except in some parts of the media. But those parts of media did not have the power they do today. And there was a much simpler scientific smoking gun to be found. So much simpler that, as I recall it, Singer eventually did allow as how there really was an ozone hole and it really did work as was said much earlier by scientists. It only took a decade or so after the time the science was nailed. This is the same S. Fred Singer who has spent the last 20 or so years making similar claims against climate change.

Bob, Eli thinks you are dead wrong that university press releases are pretty much outside the author's control. In practice the author brings the "hot new" paper to the press office and works on it with them.

If the release gets picked up the author will be called by some of the pickers uppers for a quote or more, so even that is not always outside the author's control

Welcome

I'll be trying what seems to be an unusual approach in blogs -- writing to be inclusive of students in middle school and jr. high*, as well as teachers and parents (whether for their own information or to help their children). To that end, comments will have to pass a stricter standard than I'd apply for an all-comers site. It shouldn't be onerous, just keep to the topic and use clean language.

I expect it to be fun for all, however, as you really can get quite far in understanding the world, even climate, by understanding this sort of fundamental. If I get too much less fundamental, let me know where I went astray.

* Ok, I concede that not many middle school students will get everything. Even a fair number of adults will find some parts hard to follow. Still, some middle school kids will have fun. And almost everyone will follow a number of posts just fine.

Please see the comment policy for details. And the link policy for details about that. The latter is more open than you might expect.

About Me

In my day job I work on the oceanography, meteorology, climatology, glaciology end of my science interests, but I'm interested in everything, science or not. So I've also been on stage in a production of Comedy of Errors, run an ultramarathon, and been to Epidaurus, Greece, to see a production of Euripides' Iphigenia among the Taurians
Prior to starting the current job, I was a post-doc in oceanography in the UCAR ocean modelling program, and earned my doctorate from the Department of the Geophysical Sciences at the University of Chicago (1989). My undergraduate degree involved Applied Math, Engineering, Astrophysics, and Glaciology.
Of course I don't speak for my employer, whoever that may be.