Wednesday, May 15, 2013

About the SkS Study That Finds a 97% Consensus

Skeptical Science has a study that greatly extends Oreskes 2004 study on consensus in the scientific literature, and find

A new survey of over 12,000 peer-reviewed climate science papers by our citizen science team at Skeptical Science has found a 97% consensus in the peer-reviewed literature that humans are causing global warming.

That's all well and good (and not surprising), and you can examine their methodology if you want to.

I'm not very keen on these kinds of numbers -- they are made for lazy journalists who don't want to examine the complexity of the science, reporters who just want a number that quickly and easily supports their position.

You'd never hear a scientist use such a number, because they know there is a wide spectrum of opinions about the statement "humans are causing global warming." Some find some parts of the science more convincing than other parts. They may see parts that aren't known well at all, and parts that are very well known. They know that some papers are good and some not so good and they aren't all weighted equally, nor are authors. (About physics, Luis Alvarez said, "There is no democracy in physics. We can't say that some second-rate guy has as much right to opinion as Fermi.")

The simple statement doesn't address questions like how much warming? What kind of warming (where)? How much are humans causing? How are they causing it? How well is this knowledge known? How good is the data? What are the consequences?

People using these kinds of numbers aren't so much interested in these kinds of questions or these kinds of debates as they are in avoiding questions and ending debates. They're for activists, not for those who want to really understand what's going on.

15 comments:

They are for people who need to communicate with the voting public, which radically over-estimates the amount of scientific agreement that humans are causing climate change.

Read the Six Americans stuff. Even the most concerned underestimate the degree of consensus.

Effective communication with the broader public requires engaging with a broader audience than those sincerely and energetically applying themselves to the subject of climate change. Do I have to quote Adlai Stevenson to you?

"they are made for lazy journalists who don't want to examine the complexity of the science, reporters who just want a number that quickly and easily supports their position."

"People using these kinds of numbers aren't so much interested in these kinds of questions or these kinds of debates as they are in avoiding questions and ending debates. They're for activists, not for those who want to really understand what's going on."

Or, perhaps, a vast populace of non-journalists, who dramatically underestimate the level of agreement among relevant scientists, even on this basic issues?

You tend to hit more than you miss. But sometimes when you miss, you miss big, and embarrassingly. This is one of those times. Ooof.

This whole post sounds like a knee jerk reaction from someone who didn't bother reading the paper in its entirety or understand the role understanding the actual level of scientific agreement plays in the real world.

Of course most scientists don't talk about the consensus much, the consensus is the stuff that is, by definition, no longer worth discussing outside the classroom. Nor do they talk about the degree of consensus much either, unless it is in some field where public perceptions of scientists' views are wrong, like for vaccinations, evolution and climate change.

Expert consensus is a heuristic that we all use in evaluating truth claims in fields where we are not well informed. It's not laziness to accept the consensus, it's a necessary expediency for most of us, most of the time.

Speaking of 'lazy' - did you know that Google lists the Oreske's paper with 700 cites. Even in Science it has 30 cites listed. Not bad for something that doesn't address all the questions that the IPCC reports address. What other paper addresses all those questions?

If science was only written for energetic journalists, not much work would get done at all. And I have to say there must be very few energetic journalists because the vast majority science that's published never ever hits the mainstream media.

On the other hand, there must be an awful lot of lazy journalists around. The Cook et al paper has been picked up by all the major media outlets in Australia and many smaller ones. Plus the Guardian, ScienceDaily, Reuters, AAP and lots of other media outlets too. I'll be interested to see what happens in the USA tonight (my time).

Maybe you got out of the wrong side of the bed this morning. If it didn't interest you, why write about it?

My congrats to everyone who contributed to this. I say it was a job well done.

Obviously from a scientific perspective this survey isn't particularly interesting - our understanding of AGW is already essentially defined by the consenus existing in the literature so actually counting papers is not going to change anything. However, it does provide a counter to arguments made by skeptics who are deliberately exaggerating the uncertainty about whether it is actually warming or the extent to which we are responsible for it. Getting bogged down in arguments over issues which are not seriously in question does nothing to assist public understanding of what is actually happening to our climate, which is precisely why the skeptics do it. Surely settling these debates and moving the focus onto the real questions, ie what the impacts will be and what we can do about it, can only be a good thing.

I could have written this post, but after reading the comments, I realize that the consensus number is not just for lazy journalist and activists. It is also for normal journalists communicating with normal people who do not know all the details of climate science, nor want to know it all.

And the number is also interesting for social scientists, who would like to study how something like this "sceptic" movement is/was possible.

It will not change climate science. With David I would say that an expert does not care about consensus. Also an expert is non-expert on most topics and then consensus does provide some guidance, but this one number for an entire field is too general.

If it helps combating the lies of the "sceptics", it may help climate scientists a little. Help them to breath and to work on the important problems and not on fighting misinformation.

Ding et al 2011"The relationship is mediated by the four previously identified key beliefs about climate change, especially people’s certainty that global warming is occurring. In short, people who believe that scientists disagree on global warming tend to feel less certain that global warming is occurring, and show less support for climate policy. This suggests the potential importance of correcting the widely held public misperception about lack of scientific agreement on global warming."

There are 1,150,000 returns in Google Scholar for global cooling. I skimmed a few dozen abstracts and found no support for global warming in those paper's abstracts. So 100% of scientists studying global cooling say the Earth cooled. :-)

Great post. I agree that no scientist should ever be citing things like "consensus" because that is not how science is done. The fact that so many do speaks to the level of political activism that has crept in and is poisoning the well.

What people need to know are answers to all the questions you posed, which will be uncovered with continued fundamental research into understanding the climate system. That is where resources need to be put, not into advocacy disguised as science.

The only purpose of this exercise by John Cook was to come up with another way to prop up the 97% consensus figure. For the lazy, the exercise isn't needed, as they were unlikely to have bother checking into the basis of the original 97% number. But for those who serviously are concerned about human impacts on climate, relying on that number was becoming threadbare when discussioning the issues. It pretty quickly becomes embarrasing if you use it.

Enter John Cook. Instead of using a survey using limited and very broad questions and with a response basis of tens of people, he cast a wider net. What are the scientific papers on the subject saying? On the surface it sounds reasonable. And Cook seems to have learned from the Lewandowski fiasco and at least tried to be more transparent and employ a bit more rigor in his methods.

I'll leave aside commenting on any weaknesses or flaws in this paper, as I doubt I'm qualified enough. What I do find interesting is the touting of the 97% figure. Only a third of the 12,000 papers selected include causation. Of that third, 97% assign it to human activities. That still leaves two thirds of relevant papers which are silent on causation.

So, does Cook's paper achieve it's goal of butressing the 97% number? Probably. At least to those who are either lazy or are math challenged.

David has yet another dumb comment. Seriously David, what has got into you?

You say: "And let me also say that responses from a self-selected group are utterly meaningless, statistically speaking."

What the heck? Is this a reference to the SkS paper? Then note... your remark is true only if what we are doing is collecting data on the people responding. That is NOT what is happening here.

What is being surveyed are research papers. THOSE are NOT self-selected. The criteria are in the paper. There are a bunch of volunteers who helped classify the papers. The fact that they are volunteers does NOT make their results meaningless. The paper also describes how the accuracy of the work was tested.

These whole series of posts stands out as a real low point in your blog.

BTW. Other good examples of research on data collected by self selected volunteers include the "galaxy zoo" results, and some fascinating work on protein folding.