Lies, damned lies, and statistics, and how they relate to climate change research (or, mathematics as a language with many dialects)

Although other issues have become the focus of the manufactured tempests of both new and old media, that does not mean that there is no fallout from previous hyperbolic storms. The Dot Earth Blog at NYTimes.com has a post on the report from a committee of experts recommended by the Royal Society inquiring into the climate change research at the University of East Anglia, where emails and other files were released five months ago giving climate change skeptics what they thought was smoking gun proof of a conspiracy to manufacture data showing climate change is real and caused by human activity.

What won’t get the attention from the skeptics is that the researchers have now twice been exonerated of any willful misleading or scientific malpractice.

That is to be expected, sadly enough, especially nowadays in the hothouse of new, unedited media and pundits being regarded as more credible than real experts.

What is interesting to the more thoughtful is revisiting the origins of the controversy. It is fully worth reading some of the comments from the report summary (pdf of the report found here):

- It is regrettable that so few professional statisticians have been involved in this work because it is fundamentally statistical. Under such circumstances there must be an obligation on researchers to document the judgmental decisions they have made so that the work can in principle be replicated by others.

- After reading publications and interviewing the senior staff of CRU [Climatic Research Unit] in depth, we are satisfied that the CRU tree-ring work has been carried out with integrity, and that allegations of deliberate misrepresentation and unjustified selection of data are not valid.

- We believe that CRU did a public service of great value by carrying out much time-consuming meticulous work on temperature records at a time when it was unfashionable and attracted the interest of a rather small section of the scientific community. CRU has been among the leaders in international efforts to determining the overall uncertainty in the derived temperature records and where work is best focused to improve them.

- We saw no evidence of any deliberate scientific malpractice in any of the work of the Climatic Research Unit and had it been there we believe that it is likely that we would have detected it. Rather we found a small group of dedicated if slightly disorganized researchers who were ill-prepared for being the focus of public attention. As with many small research groups their internal procedures were rather informal.

As referenced in the title of this post, there is an old truism that was popularized by Mark Twain. The full quote is:

Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies and statistics.”

- Mark Twain’s Own Autobiography: The Chapters from the North American Review

Like words, statistics can be twisted to appear to show something that they do not show. In other words, often one should say the equivalent of “I don’t think that word means what you think it means” after yet another exclamation of “Inconceivable!” when statistics are presented, especially graphically. It’s amazing how the thousand word pictures can distort meanings even further.

In the end, though, it comes down to this, statistics is a branch of Mathematics, and Mathematics is a language that can be understood and mistreated just as any other language.

Bear with me here, and let me show you how math is really a language.

Newton’s Second Law is one of the first equations taught in any Physics class, and it is one of the simplest equations other than the Einstein equation that resulted in the atomic bomb. Newton’s Second Law is written mathematically as:

F=ma

Where F is force, m is mass, and a is the acceleration of that mass. What does this equation mean in words, though?

Well, it means that if you push on something (that’s the force), that thing with go faster in the direction you push (going faster is the acceleration), and how much faster it goes depends on how heavy it is (in other words, the mass). The heavier the thing is the harder it will be to make go faster.

That equation with four symbols expresses a relationship that to be described in English uses many more words than four, but the equation does completely describe the relationship in the physical law. Mathematics is a language, and a concise and powerful one at that.

Statistics is a specialized branch of Mathematics, and I think a good analogy is that it is a dialect of math just as there are dialects of spoken and written languages that may not be easily understood by those who speak a different dialect of what is nominally the same language. For example, Chinese has two main dialects, Mandarin and Cantonese (or Yue), and they are not necessarily mutually intelligible in all cases.

How does this relate to the East Anglia climate change research controversy? Refer to the bullet points above:

- It is regrettable that so few professional statisticians have been involved in this work because it is fundamentally statistical. Under such circumstances there must be an obligation on researchers to document the judgmental decisions they have made so that the work can in principle be replicated by others.

Many scientists believe they understand statistics better than they really do, just as many non-scientists believe they can interpret statistics and other data types better than they really can. So while there was no deliberate scientific malpractice nor a conspiracy to fabricate results, there was not quite the rigor that one would prefer for this type of data. This does not invalidate the results, it just means they need to be examined more closely, and future work needs to have more participation by experts in statistical analysis.

So, the climate change skeptics, while overly hyperbolic, had a small kernel of a point in their manufactured tempest, but those concerned by the possibility of climate change were and are justified in their worries.