My exposé of why academics’ private emails sometimes seem cranky has gotten a lot of attention. Joe Romm posted it at ClimateProgress, where it generated many comments, many expressing thanks for saying what needed to be said. George Monbiot posted a comment, pointing out that for a journalist, FoI laws are sacred: a hard won concession that allows them to fight the secrecy that normally surrounds the political establishment. So there’s clearly some mutual incomprehension between the two cultures, academic and journalistic. For journalists, FoI is a vital weapon to root out corruption in a world where few people can be trusted. For scientists, FoI is a blunt instrument, unneeded in a world where honesty and trust are the community norms, and data is freely shared as much as is practically possible.

George expands this theme on his blog, and I appear to have shifted his perspective on the CRU emails, although perhaps not as far as I might have hoped. His thesis is that scientists and journalists have each formed a closed culture, leading each to be suspicious (and worse) of the other. Well, I think this is not strictly accurate. I don’t think either culture is walled in. In fact, I’m beginning to think I overstated the case in my original post: scientists are certainly not “walled in like an anchorite”. Pretty much every scientist I know will happily talk at length to anyone who shows an interest in their work, and will nearly always share data and code with anyone who is engaged in honest scientific work. For our own research into software quality, we have obtained source code, datasets, software bug histories, and extensive access for interviews in every climate modeling centre we have approached.

Unfortunately, scientists tend to be way too focussed (obsessed?) for most people’s taste, so lay people don’t generally want to talk to them in the first place. But scientists will accept into the community anyone who’s willing to work at it (after all most of us spend a lot of time training students), as long as they show the necessary commitment to the scientific process and the pursuit of truth. Traditional investigative journalism used to share these values too, but this tradition now seems to be another endangered species. The experience when scientists talk to journalists is usually more about the journalist seeking a sensationalist angle to sell a story, rather than a quest for understanding. And a reliance on false balance rather than weighing up the evidence.

So there is a bit of a gulf between the two cultures, but its not insurmountable, and there are plenty of examples of good science reporting to show that people regularly do bridge this gulf.

No, the real story is not the relationship between science and the media at all. It’s the story of how the media has been completely taken in by a third group, a third culture, consisting of ideologically-driven, pathological liars, who will say almost anything in order to score political points, and will smear anyone they regard as an opponent. Stern calls climate change the greatest ever failure of the free markets. I think that looking back, we may come to regard the last six months as the greatest ever failure of mass media. Or alternatively, the most successful disinformation campaign ever waged.

At the centre of this story are people like Marc Morano and Jim Inhofe. They haven’t a clue what science is; to them it’s just one more political viewpoint to attack. They live in a world of paranoid fantasies, where some secret cabal is supposedly trying to set up a world government to take away their freedoms. Never mind that every credible scientific body on the planet is warning about the wealth of evidence we now have about the risk of dangerous climate change. Never mind that the IPCC puts together one of the most thorough (and balanced!) state-of-the-art surveys ever undertaken in any scientific field. Never mind that the newest research suggests that these assessments are, if anything, underestimating the risk. No, these people don’t like the message, and so set out to attack the messengers with a smear campaign based on hounding individual scientists for years and years until they snap, and then spreading stories in the media about what happens when the scientists tell them to piss off.

Throughout all this, in underfunded labs, and under a barrage of attacks, scientists have done their job admirably. They chase down the uncertainties, and report honestly and accurately what they know. They doggedly compile assessment reports year after year to present the mass of evidence to anyone who cares to listen. It simply beggars belief that journalists could, in 2010, still be writing opinion pieces arguing that the scientists need to do a better job, that they are poor communicators, that we need more openness and more data sharing. That these themes dominate the reporting is a testament to how effective the disinformation campaign has been. The problem is not in the science, or with scientists at all, nor with a culture gap between science and the media. The problem is with this third group, the disinformers, who have completely dominated the framing of the story, and how honest journalists have been completely taken in by this framing.

How did they do it? Well, one crucial element of their success is their use of FoI laws. By taking the journalists’ most prized weapon, and wielding it against climate scientists, they achieved a whole bunch of successes all at once. They got journalists on their side, because journalists have difficulty believing that FoI laws could be used for anything other than good old-fashioned citizen democracy. They got the public on their side by appearing to be the citizens fighting the establishment. They set up the false impression that scientists have stuff to hide, by ignoring the vast quantities of open data in climate science, and focussing on the few that were tied up with commercial licence agreements. And they effected a denial of service attack by flooding a few target scientists with huge numbers of FoI requests. Add to this the regular hate mail and death threats that climate scientists receive, and you have a recipe for personal meltdowns. And the media lapped up the story about personal meltdowns, picked it up and ran with it, and never once asked whose framing they were buying into.

And the result is that, faced with one of the greatest challenges humanity has ever faced, the media got the story completely backwards. Few journalists and few scientists seem to have any conception of how this misinformation campaign works, how nasty these people are, and how dirty they play. They have completely owned the story for the last few months, with their framing of “scientists making mistakes” and “scientists distorting their data”. They’ve successfully portrayed the scientists as being at fault, when it is the scientists who are the victims of one of the nastiest public bullying campaigns ever conducted. History will have to judge how it compares to other such episodes (McCarthyism would make a fascinating comparator). And the stakes are high: at risk is our ability to make sensible policy choices and international agreements based on good scientific evidence, to ensure that our children and grandchildren can flourish as we do.

We’re fucking this up bigtime, and it’s not the scientists who are at fault.

Share this:

Related

53 Comments

These tactics may be new to scientists, but I dispute your claim that, “Few journalists…seem to have any conception of how this misinformation campaign works, how nasty these people are, and how dirty they play.” Journalists know better than anyone that politics is a dirty fight. However, it’s their job to report the story, and right now, the denialists are giving journalists a more interesting story than the scientists.

“Few journalists and few scientists seem to have any conception of how this misinformation campaign works, how nasty these people are, and how dirty they play.”

It’s odd: watching, say, the Guardian – it’s headlines have recently fitted perfectly into a contrarian template. That is, they could be lifted as they stand and be broadcast through the denialosphere, giving a different message to the attached article. Most recently: “James Lovelock: ‘Fudging data is a sin against science'”

This is bizarre, it keeps on happening at broadsheets I used to trust, and I don’t understand it. One possible explanation, as you say, is that they really don’t understand the nature of the beast they’re dealing with. That headline – like many other recent Guardian ones – seem tailor-made for denialosphere broadcast. Don’t they know that??? But it must also be something about the shifting of the Overton Window; that shift isn’t just what can be talked about, but it changes the consensus on what phrases are considered marketable.

At the centre of this story are people like Marc Morano and Jim Inhofe. They haven’t a clue what science is; to them it’s just one more political viewpoint to attack.
[…]
They got the public on their side by appearing to be the citizens fighting the establishment.

You seem to be lumping the auditors in with the politicos. One group might be useful to the other at times (and at other times not), but they are fairly distinct. Claiming that they are all part of a unified front of denialism (with a man behind the curtain pulling the strings) is nearly as silly as the deniers who claim all scientists are part of an evil cabal (though the more serious bloggers in the ‘denialosphere’ have never made that claim, the arguments tend to be about subtle error cascades and institutional bias, though this probably has more to do with policy proscription end of things than the ‘physical science basis’).

Josh: I think it’s reasonable to lump the two together, because these self-styled “auditors” are playing the same game. They want people to think they are honest citizen-journalists, just trying to check up on the science. However, everything they do demonstrates they operate from an ideological basis, and have no understanding of how science works. They ignore the fact that scientists around the world already check on and correct one another’s work. They assume that there must be problems with the science because they can’t accept the results. They cherry-pick and distort the data. They refuse to give up on a point when they’re proven wrong. And they mix in political screeds in with their pretense at data analysis. Look at McIntyre’s associations with (and funding from) rightwing thinktanks. Look at Watts’ political rants. Read Ben Santer’s accounts of how they systematically distort the science year after year. They don’t audit science in any useful way – they inject obfuscation into the public discourse. I see no difference from Morano’s game.

What’s common is that they all have an ideologically-based belief that the climate science cannot be correct. And they mistakenly believe real scientists must also be ideologically-driven, because they are unable to understand what makes real scientists tick. But don’t take my word for it. Go attend some scientific meetings, and listen to the science being presented. If you’re familiar with how science works, the idea that we need “auditors” is quite simply ludicrous. It’s just another smokescreen.

I think this is the source of our disagreement; you are right, science doesn’t need auditors to achieve its goals (everyone’s an auditor, right?), but credible decision support does. All of the nit-picky, mundane, due-diligence kinds of things that don’t appeal to academic researchers (rightly seen as ‘fiddling around the edges’ for those interested in ground-breaking research, “we already answered that question back in ’98, next!”) are critical to successful decision support. If scientists are not interested in playing the decision support game, then they can continue to dismiss attention-to-detail as a shoe-clerk’s fascination unworthy of their high calling. Otherwise, the community needs to learn to welcome audits as a necessary and beneficial part of the decision support process (as you say, much of the data/code is available for the browsing, so it’s not as if there’s a great chasm to cross).

This is may be veering off-topic, but since you brought it up:

…they mix in political screeds in with their pretense at data analysis.
[…]
What’s common is that they all have an ideologically-based belief that the climate science cannot be correct. And they mistakenly believe real scientists must also be ideologically-driven

I find this a very odd thing for you to say; you are quite open about how your beliefs have shaped your research agenda. Not that it’s a bad thing; I find your honesty refreshing, open advocacy is much more palatable than the stealthy kind. On ‘pretense at data analysis’, I think many of the sites warrant a closer look (Lucia’s site and JeffId’s being two notable stand-outs), it’d be a shame to write them off just because you disagree with their politics. The math (when it’s there) is generally correct (and that write-up I linked above is actually demonstrating a novel improvement on the state of the practice for generating grid-ed temps from the station records).

BTW, this sort of thing:

…unable to understand what makes real scientists tick. But don’t take my word for it. Go attend some scientific meetings, and listen to the science being presented. If you’re familiar with how science works…

[Josh: Two things. 1) Nitpicking is what scientists do best. That’s why I suggest more people need to go attend scientific meetings & seminars, to see this in action. 2) I don’t accept your attempt to equate the ideology that drives the denialist machine with my belief that the science is correct. One is pure ideology, the other is reasoned from the evidence. But against the charge of elitism, I’m guilty as charged. What of it? – Steve]

Thanks for this. Another interesting post, though this time I think that you’re the one who has got the story backwards. Apart from anything else you’ve got the sequence of events wrong.

The FoI applications started to come in only after several years in which Phil Jones and CRU had frustrated requests for information. We didn’t have an FoI Act in this country until 2005 (and we still use chipped flints to cut up our dinner). In fact Jones made it clear that he was afraid of FoI because he wouldn’t be able to block such requests any more. Here are three examples:

1. “The two MMs have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone … We also have a data protection act, which I will hide behind. Tom Wigley has sent me a worried email when he heard about it – thought people could ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR should be relevant here, but I can see me getting into an argument with someone at UEA who’ll say we must adhere to it!”

To begin with, the FoI requests, like the previous applications for data, were few in number. It was only after they were blocked using bureaucratic obfuscation of the kind that could scarcely be better designed to make people furious that – in July 2009 – Steve McIntyre organised a mass application campaign.

As the former Information Commissioner Richard Thomas pointed out to parliament, this isn’t a great deal and did not in his view amount to a campaign of harassment. Though I think it’s fair to say that the McIntyre requests could be classified as vexatious. In either case, CRU was able to refuse the 58 requests McIntyre organised without breaking much sweat. Jones, as far as I can tell, didn’t have to deal with these requests himself: they appear to have been handled by CRU/UEA administrators, who just threw them all back in the water.

I don’t see evidence here of “a denial of service attack by flooding a few target scientists with huge numbers of FoI requests.”

Remember, if CRU hadn’t blocked the early requests for data, none of this would have happened.

George: Good points – I’ve conflated a number of things, and you’re right “flooding with FoI requests” is not strictly accurate. What I meant to say was that the FoI requests were just the most visible part of a much longer campaign, going back at least a decade. When Phil Jones writes “The two MMs have been after the CRU station data for years” it’s clear that we’re coming late into an ongoing battle, and Jones is already pretty pissed off with the whole thing. Taken out of context, the email reads like an attempt to cover up misdeeds. But in the context of many years harassment by people who have no interest in constructive criticism, a long track record in misrepresenting the science, and no obvious sign of ever backing off, well, it looks much more like letting off steam in private with trusted colleagues (particularly as there’s no evidence that he ever did destroy any data).

It’s clear from reading the accounts from Mann and Santer that they were under constant bombardment, and that they’d already come to understand that rebutting each criticism is pointless. Mann and Santer have been much more effective in getting their side of the story on record; Jones hasn’t communicated this so well, and I think he has been slow to understand the game that was being played, and a little foolish in what he said about it in the emails. But it’s wrong to say that if the CRU hadn’t blocked the early requests for data none of this would have happened. The original station data was tied up with restrictive licenses, because it’s commercially useful in the forecasting community. It’s not clear that Jones could ever have handed this over to M&M, and in the early days, it appears Jones offered constructive suggestions for how to get hold of the data directly from source (as the CRU never owned this data in the first place). But M&M saw this as a smokescreen, and went on the attack, and things just went downhill from there.

I remain convinced that the more important story is an examination of the motivation and sponsorship for M&M’s attacks, the impacts they have had on the scientists who have been the victims, and an account of how the science has stood up robustly to everything they’ve thrown at it. I don’t think endless examinations of the contents of Phil Jones’ emails serves any useful purpose.

And given that none of this changes the science in anyway whatsoever, I can only conclude that as a way of distracting everyone’s attention, it has been a remarkably successful propaganda exercise.

I haven’t yet read the Mashey report, which I will do now. But don’t the emails suggest that M&M might actually have been right to believe that the restrictive licences were, at least in part, a smokescreen?:

“If FOIA does ever get used by anyone, there is also IPR to consider as well. Data is covered by all the agreements we sign with people, so I will be hiding behind them.”

Here’s what Jones told another requester:

“Even if WMO agrees, I will still not pass on the data. We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it.”

And shouldn’t we be concerned that commercial considerations are getting in the way of the free flow of information? Is that not antithetical to the open society and the spirit of scientific inquiry? It’s something I’ve campaigned against in other areas: particularly the patenting of genetic material.

I react against it for two reasons: first that it’s an enclosure of a common (and in this case publicly-funded) resource, secondly that it’s the kind of convenient excuse everyone in authority hides behind these days, when there’s something they don’t want to tell us. (Sorry, I can’t tell you that – security you know, er, health and safety, er commercial confidentiality). Here in the UK the terms of the stealthy privatisation of almost the entire public sector (the Private Finance Initiative) were kept secret from the public on the grounds of commercial confidentiality. I campaigned against that excuse for years. Only by using the FOIA, and by harassing the bastards mercilessly, did we finally open up that can of worms, by which time it was all too late. I hope this gives you more of an idea of why my gut response is different from yours.

George,
Yes, the emails make it quite clear Jones doesn’t want to hand over his data, and is prepared to use any excuse not to. At face value, it certainly looks like he’s got something to hide. But what? His work on the temperature reconstructions is sound, as has been shown by other reconstructions. The allegations made by M&M about temperature biases in surface station selection were clearly shown to be wrong by several independent analyses, including work by Tamino:http://tamino.wordpress.com/2010/02/25/false-claims-proven-false/
by Wood and Steig:http://www.realclimate.org/index.php/archives/2009/12/are-the-cru-data-suspect-an-objective-assessment/
and many others. There’s nothing wrong with any of the data, nor any of the science from Jones’ team at the CRU. So why does Jones act in the emails like someone who wants to hide evidence?

The only reasonable explanation is that he wants to withhold the data because he’s quite simply fed up with the constant baseless attacks. Ben Santer had already warned him how much time and effort it takes having to correct misrepresentations of their work. Refusing to cooperate with time-wasters is completely rational. We can argue whether in the long run this was a good idea; we can argue over what strategy scientists might best adopt in this situation. But the fact is that Jones chose the path of withholding his data, motivated partly by anger and partly by the desire to protect his time.

And yes, there is a bigger context here – this science is fundamentally important for public policymaking, and must withstand a much higher degree of scrutiny than normal. Phil Jones might be guilty of not grasping this bigger issue, but then I don’t think many people have put much thought into how we do this. The climate modelers I’ve spoken to cannot defend their software practices, nor can they adequately demonstrate they are using best practices (comparable, say with the best in the commercial software world). However, when you look at what they do, it turns out to be phenomenally good. The problem isn’t that the code isn’t subjected to scrutiny; the problem is that it’s subjected to so much scrutiny in so many ways by so many people that nobody can give a coherent account of all of these, nor how they add up (I’ve made it my research goal to produce such an account for software development practices of climate scientists – I hope others can do the same for the data handling practices).

Bottom line: There are two hypotheses for why Jones talks this way in the emails: (1) because he has something nefarious to hide or (2) because he’s fed up with people wasting time and making unfounded allegations, and wants nothing to do with them. Hypothesis 1 depends on there being something wrong with his work, but the reconstructions show it’s sound. So we have to go with hypothesis 2.

BTW I agree completely with your comments about commercial considerations getting in the way of the free flow of information in the science community. Unfortunately, this is a result of lack of funding, particularly of meteorological research. Many weather services around the globe (the Met Office included) gain part of their funding from selling commercial forecasting services, and there is tough competition for some of the bigger customers. The dataset that M&M are after is drawn from meteorological services in many different countries, and many of them place different restrictions on it to protect their interests (several countries, including Canada, have refused to release their portions). Most of this, Jones and the CRU have no control over; when they obtained the data from these many different sources they had to sign agreements to respect confidentiality. Part of Jones frustration is the amount of effort it took to do this (hence his response that M&M go collect the data from these various sources themselves).

We’d all much prefer meteorological data data to be free and open – it would make our lives much easier. But we’re faced with constant pressure on universities and government research labs to prove their value by commercializing the outcomes of their research, and by selling services. A growing proportion of the research is no longer publically funded – it’s funded by industrial research grants and by income earned on “intellectual property”. We’re going to have to untangle the mess that free market ideology has made of our public research institutions to fix this problem. The real irony is that the those on the rightwing who have been most active in pushing this commercialization of research labs are now the ones screaming most loudly about freeing the data.

George:
I think a different cultural matter is involved, in addition to what Steve has mentioned. Namely, scientists are used to people trying to do science. The ‘auditors’ are not trying to do science. I’ll take it up at more length on my blog shortly.

In the mean time, there is a fair degree to which Jones is being tarred for following UK practice as set forward by their government. James Annan takes this up at more length and with better detail. My own observation, from the US, is that in the 90s, many national weather services, including both Canada and the UK, were pressed to recover costs — such as by charging for data they had collected. Surface station networks were particularly popular things for this. Never was sure why. Consequently, however, the primacy moved away from “how can we distribute these data” to “how can we make money from these data”.

Descendant from that were non-redistribution agreements. Very often, these included that it was ok to redistribute the data to other scientists. Hence part of the scene that played out in 2009, where Peter Webster, a scientist at Georgia Tech., was able to get certain data from Jones, while McIntyre, a non-scientist, could not.

Perhaps science would be better if there were no distinctions as to who ‘counted’ as scientists. I tend to think so. But legal agreements about who you can pass data to say that some determination has to be made, somewhere. Or else you either quit using data you can’t redistribute (so do science on only the subset that is open), or start devoting resources to figuring out how to distribute only what you’re allowed to and providing pointers to what you cannot.

Again, resources mean money. And the reason for the cost recovery was a belief that the weather services should be charging. Are they going to be able to charge even more? That doesn’t seem consistent with making all information available to everybody. Are their base budgets from their countries going to be increased to cover these new costs? I’ll hope so.

It’s perhaps a minor point, but if you want to obtain a data set for research purposes you’ll want to get it from the original source and *not* use someone else’s copy. I think Phil was entirely justified in reacting badly to McIntyre’s attempt to do otherwise, especially given McIntyre’s history of behaving badly when “auditing” temperature data from NASA GISS. All that’s been achieved from such activities (aside from a great deal of hooting all over the denialosphere and in the usual media outlets) is that GISS now delays posting their data until after they’ve thoroughly quality-controlled it. Rather ironically, the public sees *less* information.

So was it unreasonable of Phil to want to stand on a strict interpretation of the law in order to avoid the same sort of abuse? I believe so. George concludes above: “Remember, if CRU hadn’t blocked the early requests for data, none of this would have happened.” Perhaps not, but then other undesirable things would have been guaranteed to happen, and unlike GISS CRU isn’t staffed so as to be able to deal with such things easily.

As an common ‘man in the street’ I read with interest the latest email from George Monbiot – so you are both right! I believe in Climate Change, but I don’t believe all scientists (or journalists) are as straight as is being presented. In addition to contractual obligations we have vested interests – who possibly were at the root of this mess in the first place; but they have profits to protect.

Thanks for this Steve. I think you make a good case for what Jones’s motivation might have been (and I wish to God that he or the university had come forward at an early stage to explain it: the mishandling of this crisis is a whole other saga).

If true, it sounds like a tragedy straight out of a Hardy novel – Jude the Obscure or Tess of the Durbervilles perhaps – misplaced action, misunderstood intentions, letters under the doormat. I’m prepared to believe that Jones is a good man who, partly because of the constraints he was under, made a series of disastrous tactical errors. On this account he emerges as a tragic figure, in the classical sense.

But, as Hardy shows, ultimately it’s the actions, not the intentions that count. This is what we end up being judged by, and why it is so important both to act strategically and to act as if the world is watching. These days it probably is. It remains possible to praise the intention and condemn the action. Though he might have been acting with good intentions I still believe that what Jones did was wrong – and offensive to democratic values.

The points you make about commercial agreements are good ones. I would love to see scientists use this crisis to campaign for proper public funding of climate science and an end to the jumblesale of public assets.

Robert: thanks for your points on this too. Let’s mobilise against these stultifying legal agreements and cost recovery arrangements.

Climate contrarians (including the telegraph and, of course, fox news) attacked Copenhagen for its excessive carbon footprint. As with this issue about free access to data, this reveals a lot about the nature of our opponents: they don’t care about carbon output. In some cases, they argue that it’s a good thing. So why the sudden interest in the alleged carbon hypocrisy of international meetings? I’ve said elsewhere, it’s like the allies arriving at Dachau to find some people still alive, and condemning the nazis for not doing their job properly.

As far as I can tell, the only reasoning behind it is to attack climate science and policy in way possible, leaving principles to one side. I’m continually gobsmacked by the duplicity of those who attack climate science – and yet this is never the story. Monckton travels the world spreading his nonsense cuckoo science, and that’s never the story. (George Monbiot has, to his credit, pointed out the obvious scientific errors in Monckton’s sort of thinking.)

Someone else (lost the comment, sorry!) pointed out that, in the UK, people have been primed by the MP’s expenses affair and are ripe for any other tale of small-minded bureaucrats controlling their lives and taking their money. But it’s not the story. The story is that, perhaps due to this onslaught, Australia now has a climate denier in opposition, one day the republicans will presumably take power in the US, and they seem to have also completely gone over to dark side, and here in the UK, despite Cameron’s green gloss, the vast majority of tories don’t believe climate change is an issue or buy into the whole hoax theory. (The top ten Tory climate bloggers are ‘skeptics’.)

That’s terrifying: what is likely to happen to the science if we find ourselves in a world where several major governments have gone denialist? I’m imagining decades where the earth warms and institutes claiming its not co2 and anyway it’s good for us get all the funding. Some awful future of two ‘sciences’ swims before my eyes.

Perhaps not all climate contrarians are right wing, but for those who are, they’d argue the data belongs to them because they paid for it with taxes. Cop out. I want a policy document from any of the key denier thinktanks on how their future ideal science will be funded and structured, and who will own the output. The further to the right you go, the more information becomes just another commodity: how will that work in the world they propose?

Except I see no proposals: just unprincipled attacks, in any way possible, on people who over many decades have painstakingly helped to improve humanity’s understanding of how the Earth’s climate functions. That’s the fecking story.

Excellent commentary – both content and writing style. Don’t know how I’ve missed you before – added to feed reader!

Let’s hope you can influence Monbiot and all the other MSM pundits who have been so clearly fooled by the propagandists. It’s been baffling how badly they have been suckered in by the lies and innuendo – and disgusting that Monbiot still calls for Professor Jones termination even after he has been cleared of all charges.

The integrity of science is more important than whether the planet burns? That would be a tough call, but luckily it’s a false dilemma. – Steve

Dan said (among many other things):

the more information becomes just another commodity: how will that work in the world they propose?

What world are you living in? Information is already a commodity. It’s so cheap, in a lot of cases you can’t give it away. Perhaps you are worried about paying for the expert labor that it takes to create some of that information in the first place (in which case I’d encourage you to read up on sunk cost)?

Robert said:

Namely, scientists are used to people trying to do science. The ‘auditors’ are not trying to do science. I’ll take it up at more length on my blog shortly.

You are on to something there Robert; it will be interesting to hear what your take is on this; will you post a link on this thread when your post is up?

Great thread; we got Godwin-ed and hardly talked about software at all…

I’m reposting a comment by someone calling him or herself Gladiatrix, that has just turned up on the Guardian website. It’s pertinent to our discussion.

“I am going to write this post as an experienced public sector lawyer who has lived through the introduction of the FoI Act and given extensive advice to various clients on its effects.

The FoI Act was clearly expressed from the outset to apply to all public sector bodies of every kind. It was NEVER intended only to apply to HMG. The assertion that it was is factually incorrect. In other words, every body that is funded from the public purse or which carries out a public sector function is subject to the FoI Act and must comply with its provisions.

The FoI Act had a 5 year lead in period during which every public body had ample opportunity to put in place all and any necessary procedures, to appoint its Information Officer (a statutory office) and provide training to everyone concerned. The UEA had and has no excuse.

Commercial agreements and non-disclosure agreements are not exempt from disclosure under the FoI Act and never have been. They are covered by what is known as a qualified exemption only, i.e. they can be disclosed if the public interest requires disclosure. All exemptions under the FoI Act are required to be interpreted under what is known as the Public Interest Test. A public body may disclose a commercial agreement, and may indeed have a separate statutory obligation to do so and this will be upheld by the courts. See: Veolia ES Nottinghamshire Ltd v Nottinghamshire County Council and Shlomo Dowen and Audit Commission [2009] EWHC 2382 (Admin).

Under guidance issued by the Office of Government Commerce all public sector FoI contract clauses are expressed such that the public body may disclose commercial data and may consult with its contractor beforehand, but is under no obligation to do so. This is a direct read across from the FoI Act. All contractors try to have this clause changed to make consultation mandatory, only very inexperienced public sector officers would agree to this.

In short, UEA and its employees of all grades knew what the Act said, they knew what their legal obligations were, they knew what they could and could not disclose.

Professor Jones and his colleagues and his employers knowingly flouted the law of the land without any excuse whatsoever. What is worse is that they bullied a junior employee (who should have blown the whistle to the ICO and the Standards Board) into allowing them to flout the law.

For myself I would dearly love to know what the UEA’s legal advisers were doing while all this was going on, what they knew and when they knew it. This it seems to me is a question that both the Law Society and the Solicitors Regulation Authority could usefully ask.”

But why would you think the climatology data is commercial information?

It comes from another national government’s meteorology service under a nondisclosure requirement; it isn’t being used commercially by the receiving agency, which isn’t a business. Something similar _is_ being used commercially by the national government, which sells some data to businesses.

Is it in the (UK) national interest to violate another nation’s restriction on the UEA’s use of the data was given without charge, when the same source also sells similar data to businesses for commercial purposes?

Consequences matter. Is it in the (UK) national interest to lose access to such data in the future?

It was clearly in the UK interest to _request_ permission to release the data.
That was done.

Some thought would be required here, beyond one lawyer’s interpretation.

How many employees, total, does that climate service have? I thought it was 3?

> the public body may disclose commercial data
> and may consult with its contractor

The data is not commercial.
The meteorology service providing data to UEA would not be a “contractor.”
UEA is not a commercial business.
UEA never engaged a contractor to handle any data, so no contractor was ever commingling their commercial data with UEA’s data.

Nothing in these Regulations shall authorise a refusal to make available any environmental information contained in or otherwise held with other information which is withheld by virtue of these Regulations unless it is not reasonably capable of being separated from the other information for the purpose of making available that information.

The rejection notice to Steve McIntyre did not explicitly state that the CRU was not reasonably capable of separating out the exempted data from the non-exempted data, and therefore CRU should have supplied the non-exempt data to which the various claimed exemptions did NOT apply.

#
Gladiatrix
Posted Sep 30, 2009 at 11:55 AM

If the Met Office is using spurious reasons and/or ignoring the statutory public interest test contained in the FoIA and the EIRs then rather than sending what looks not far short of a chain letter the appeal process to the ICO should be used. This is what it is for. In addition, the Met Office should be reported to the relevant Secretary of State for misconduct with a request that disciplinary action be taken against the civil servants responsible.

#
Veronica
Posted Dec 1, 2009 at 3:34 PM
So find a friendly academic and get him/her to request it?

#
Per Edman
Posted Feb 9, 2010 at 3:15 AM

So there was indeed a campaign of vexatious FOI requests. Thanks for being blatant about it:

Cherrypicked quote:
“I assume that the reference number means that this is the 100th email Palmer has received! This will presumeably totally foul up his plans for a vacation.”

George: I’m still at a loss to understand how exactly he’s supposed to have broken the law. Gladiatrix’s comment talks a lot about the nature of the law, and the obligations it places on public institutions, and then goes straight to a guilty verdict. But, despite lots of sabre rattling in his emails, Phil Jones never deleted any information that was subject to FoI requests (indeed, it seems he never deleted anything), and his institution used the appropriate processes to handle all the FoI requests. The parliamentary enquiry found no instance in which anyone had broken FoI laws. So, unless the law prevents people moaning about the FoI act and threatening to withhold information, there’s nothing here. No law was broken.

Which brings me back to my original point. This is all a distraction. We shouldn’t be focussing on Phil Jones’ actions at all. We should be looking at how the denialist machine is attempting to undermine solid science through these attack campaigns. George – you wrote in your book about the similarity with the tactics used by the tobacco industry (and indeed how many of the same players are involved). Why aren’t we discussing how and why people like M&M are allowed to get away with a continual campaign of distraction and obfuscation, by trying to discredit scientists, and pick nits in a solid body of scientific research? What concerns me, is that you, along with many other journalists, are still stuck in M&M’s preferred framing (i.e. that they’re sniffing out corruption in the science and trying to free the data), rather than recognizing that they set the whole thing up as a smear campaign and that it was never about the data in the first place?

Gladiatrix on Sep 29th, 2009 at 12:42 pm
I have been continually surprised that no-one used the Freedom of Information Act against the Met Office; i.e. hand over the publicly funded data in the name of the public interest or be the subject of a complaint to the Information Commissioner which you will lose. The ICO would undoubtedly have applied the public interest test contained in the Act and ordered the Met Office to hand over the data. This could and should have been done years ago. There was no need for the data to be obtained in such a roundabout way when there was a well-established legal remedy already available.

….

grumbler on Sep 29th, 2009 at 3:11 pm:

Gladiatrix 12:42 “I have been continually surprised that no-one used the Freedom of Information Act against the Met Office; …..”

You can be assured that this has been tried time and time again and been resisted.

That comment from Gladiatrix is just depressing. Yes, Jones may not have acted properly under the FoI Act. Can we stone him to death now? And what about the junior employee who should have blown the whistle to the ICO and the Standards Board? Let’s find out who this was and destroy them too.

I would be happy for the lawyer attack-dogs to get stuck into, say, a scientist from MegaCorp who deleted information on how his company sold radioactive baby formula, but we’re talking about Phil Jones and climate data for heaven’s sake.

In your piece in the Guardian you say “Yes, some of the [FOI] requests appear to have been vexatious.”

Can you identify any requests that weren’t vexatious?

It seems to me that all of the requests fulfill four of the Information Commissioner’s five criteria for vexatiousness: obsessive, caused distress to staff, designed to cause disruption/annoyance, and lacked any serious purpose or value.

You: “some of those doing the badgering seemed to be motivated by something other than the unsullied spirit of scientific inquiry.”

Can you identify any of the badgerers who were trying to advance science?

You: “But there was a simple means of getting the hasslers off his back: release the sodding data.”

Sorry George, but surely you don’t really believe that this would have stopped them? The “auditors” wouldn’t have stopped until they had either (1) found a trivial error that they could blow out of all proportion, and/or (2) caused Jones to have a breakdown. This pattern has been well-established through previous attacks on, e.g., Michael Mann and Keith Briffa.

Your Thomas Hardy analogy might be a good one. It has been a while since I’ve read Tess of the d’Urbervilles, so I might have this wrong, but would you be joining the crowd cheering the outcome when Tess is hanged? “A good day for British justice” says George Monbiot, “she had it coming all along. If only she had been scrupulously honest and open in all her dealings from the beginning, this wouldn’t have happened. She has only herself to blame.”

George:
I’ll certainly cheer your efforts at getting EU data freely distributed. The similar data from my side of the Atlantic are already freely available.

jstults and George:
My note taking up the point I mentioned went up this morning as But is it science?.

All, regarding comments from people who claim to be lawyers:
My wife is a lawyer. (In addition to being a writer, a woman of many talents. I mention the writing in my above post.) So there have been some discussion between us about law and lawyers and what sorts of things can and can’t be said or done. The anonymous person proclaiming certain knowledge of what is and is not the case regarding the outcome of a case that has not been tried, involving facts not in her possession and so forth, is not writing like a lawyer. Granted there are differences between US and UK legal practice. But the fundamental approach, we inherited. And that fundamental approach makes for a lot less predictability about who is ‘certainly’ right or wrong in the eyes of the law. Makes lawyers almost as maddening to talk to as scientists, each with their qualifiers and ‘well it could be this way, but it could also be viewed in that way’.

Steve, surely incitement is an offence, regardless of whether anyone acted on that incitement? But my intention in reposting that comment was not to turn this into a legalistic discussion, but simply to remind us that commercial confidentiality, the issue we had been discussing, does not automatically exempt you from the FoI Act.

And sure, let’s also discuss the campaign of distraction and obfuscation: as you know, I have spent much of my career doing just that. But to retain the moral high ground we have to be sure that we’ve got our own house in order. That means demanding the highest standards of scientific openness, transparency and integrity. Blocking, “hiding behind”, urging deletions, regardless of the pressures at play, don’t meet those standards.

If we, as a society, want to hold scientists to high standards, we must ensure that the prevailing social context makes it possible for them to keep to those standards.

Scientists have learnt from experience that cooperation with denialists is harmful to science. Give them enough data and code and denialists will find something difficult to understand that they can misrepresent to the public as an error, and a credulous media will happily spread this ‘balanced’ misinformation. Then the scientist has to waste precious research time attempting to clean up the mess.

As a society we want scientists like Phil Jones to be confident that they can release data and code without it being widely misrepresented to the public. For this to happen, journalists like yourself have to be much less gullible.

You say that “to retain the moral high ground we have to be sure that we’ve got our own house in order”. This could also apply to journalists judging the actions of scientists.

But to retain the moral high ground we have to be sure that we’ve got our own house in order. That means demanding the highest standards of scientific openness, transparency and integrity. Blocking, “hiding behind”, urging deletions, regardless of the pressures at play, don’t meet those standards.

I don’t think anyone would disagree that scientific openness, transparency and integrity are paramount, but surely you have to consider the difference between “scientific” and “political” openness. They do not have the same purpose, and they are not guaranteed through the same mechanisms. I think I’m repeating what Steve has already said, but I’ll try anyway.

In this case, scientific openness has been served. Phil Jones has not stood in the way of honest and open scientific inquiry; his emails notwithstanding. His actions may not reflect normal standards of political openness and transparency, and there is room for criticism there, but let’s keep some perspective. The worst outcome that Jones could possibly have effected, in denying FoI requests, is the perception of dodgy practices.

This perception would be shattered if the public knew just a little more about the scientific process. The denialists frame the debate (as Steve says) using the language of political openness, which they simply call “scientific” openness. The debate over how scientists should handle FoI is a debate over perception, not reality. It’s essentially tabloid stuff.

Yes, nobody should abandon the scientific moral highground, but neither is anyone forced down from it due to actions that have nothing to do with science. I would like to see the media emphasising this, rather than encouraging people to draw conclusions on climate science from unrelated actions of its protagonists.

As a somewhat extreme software analogy, take filesystems (basically, the software infrastructure, inside the OS, that manages your hard drive). I happen to be using a filesystem designed by a convicted murderer (no joke), and doubtless many others are using it as well. I confess that I’m not an expert in such things (neither filesystems nor murder). However, clearly the technical merits of the filesystem do not hinge on the morals of its designer. If they did, the contents of my hard drive would have met with a grisly fate long ago.

If only comments’ thread were all like this one… One can dream alright !

My sole observation is regarding legalese. I never seen a blog discussion enlightening legal intricacies, and humbly submit that the crux of the matter is not legal anyway. To rephrase what I mean using George Monbiot’s (#15) words, _the_ saga is the mishandling of this crisis.

That CRU got caught unaware of what their auditors’ strategy might be explained by warfare innovation. But that CRU mishandled their public relations so badly is quite revealing as to the preparation of the entrance of the institutions into brave new webbed world. Nobody should expect the press to act properly if that means to forego of a good story.

So this is what it comes down to? Suggesting, in a moment of weakness, that emails should be deleted is now a resigning offense? And in the context of an information warfare, where the reason for the suggestion was to avoid further abuse by denialists, rather than to conceal anything? And even then, as nothing ever was deleted, and we’re only seeing carefully selected portions of the email conversation, presumably good judgment eventually prevailed? You’re setting impossible standards for hard pressed scientists.

And that, I think, is the crux of the problem. This argument that scientists should somehow be more virtuous is a huge fallacy. Openness is a great virtue to strive for. But it cannot ever be a standard by which we judge individual scientists. For a start, no scientific field has ever achieved the levels of openness that are being demanded here. The data is messy, the meta-data standards are not in place, the resources to curate this data are not in place. You’re falling for straight denialist logic here: that we can’t act on the science until every last bit of data is out in the public domain. While, in truth, climate science has developed a better culture of data sharing, replication, and results checking than almost any other scientific field. One datapoint to back this up: in no other field of computational science are there 25+ teams around the world building the same simulation models independently, and systematically comparing their results on thousands of different scenarios in order to understand the quality of those simulations.

We *should* demand from scientists that they do excellent science. But we should *not* expect them to also somehow be superhuman. This argument that scientists should never exhibit human weaknesses is not just fallacious, it’s dangerous. It promotes the idea that science depends on perfect people to carry it out, when in fact the opposite is the case. Science is a process that compensates for the human failings of the people who engage in it, by a continual process of questioning evidence, re-testing ideas, replicating results, collecting more data, and so on. Mistakes are made all the time. Individual scientists screw up. If they don’t make mistakes, they’re probably not doing worthwhile science. It’s vitally important that we get across to the public that this is how science works, and that the results of this process are more trustworthy than any other way of producing knowledge, precisely because the process is robust in the face of error.

In this particular case, calling for scientists to take the moral high ground, and to be more virtuous, is roughly the equivalent of suggesting that victims of sexual assault should act more virtuous. And if you think this analogy is over the top, you haven’t understood the nature of the attacks on scientists like Mann, Santer, Briffa, and Jones. Look at Jones now: he’s contemplated suicide, he’s on drugs just to help him get through the day, and more drugs to allow him to sleep at night. These bastards have destroyed a brilliant scientist. And you’re arguing that the response is to be more virtuous.

Pete has it spot on. Journalists such as yourself are a very active part of this story. If you want scientists to feel that they can be open, you need to help.

The CRU may have done bad things with regard to FOI. Maybe no matter what provocation they were faced with, they should have kept their own house whiter than white. But surely the context is important? After all, it does provide another possible explanation to the one that involves their being part of a deeply sinister conspiracy. It may not excuse their actions, but it can explain them.

You’re talking about the situation when an agency subcontracts work to a private company, acquires some of that company’s commercial information, and there’s a public interest reason to disclose it.

That’s red meat for journalists.

Say a health organization subcontracts research to a pharmaceutical company, and the results of the work are something the pharma doesn’t want made public. But you get wind of it and ask. The public interest leans toward disclosing even the private company’s part of the files.

George.
Please.
Point out a commercial business anywhere in the climate files?

You’re confusing records from other national governments.
They gave data from some records at no cost to the climate researchers.
They keep all their original data.
Got it? They take a subset out and do something with that copy.
Under some agreement. When they give it to scientists, they say ‘don’t pass on’

Those national weather agencies also pull data out of their records to sell.
They pull from the originals, which they keep.

Where, between those weather services and UEA, is there _any_ commercial use?

Nowhere.

You’re trying to defend use of a tool you value.
But you’re using the wrong end of the tool here.
And you’re beating on the wrong party with it.

[Thanks Hank, but as Willard suggests, let’s not get sucked into discussing the legal intricacies. – Steve]

@Robert Grumbine When we lawyers talk to clients, everything we say is full of qualifiers (both to fully disclose risks and to CYA in case you’re sued for malpractice). When we take a public position, we’re pushing a viewpoint and the qualifiers all go away (up to the line of the ethical duty not to lie).

That said, I have no idea if Gladiatrix is a who she/he/it says it/he/she is.

@steve
Thanks for mentioning my report, but see the later, and (I think) much higher-quality version called Crescendo to Climategate Cacophony. There will likelley be at least 2 updates over the next few months, as additional data has arrived to fill in some holes, but they should all get attached to that same URL.

In particular, the new Appendix A.6.2 shows the matrix of people versus anti-science activities and organizations. That is important because it helps establish context for a given event.

For example, if someone attacks a scientist for doing bad sciencee, and does it through extra-science methods, one gets nervous.

If they have years of recorded history doing such things, then one gets really nervous.

Further, if someone has spoken often at Heartland conferences, signed petitions that require abandonment of basic physics, cooperated closely with GMI/CEI/SEPP/Inhofe … a claim of a sincere desire to improve science might leave room for some skepticism.

In the real world, people normally assess the trustworthiness of people they know.

George monbiot @ #15
“But, as Hardy shows, ultimately it’s the actions, not the intentions that count. This is what we end up being judged by, and why it is so important both to act strategically and to act as if the world is watching.”

George monbiot @ #29

“Steve, surely incitement is an offence, regardless of whether anyone acted on that incitement?”

is it the actions or the incitement that count (or is the incitement an action?).
that is are you going to hold Prof. Jones accountable for his actions in dealing with FOIA requests or for his intentions?

George, “Gladiatrix” is wrong. Veolia ES Nottinghamshire Ltd v Nottinghamshire County Council and Shlomo Dowen and Audit Commission was a decision to do with the 1998 Audit Commission Act. It is not much to do with the FOI Act.

In fact, a law firm agrees with me:

“Section 15(3) of the 1998 Act lays down an exception to the section 15(1) right in circumstances where the documents sought to be inspected contain confidential personal information (“the personal confidentiality exception”). The 1998 Act contains no similar exception in respect of documents containing commercially confidential information, unlike the Freedom of Information Act and Environmental Information Regulations 2000 and the Environmental Information Regulations 2004.”

This took me about 5 minutes to find on google. Perhaps instead of complaining about supposed secretive scientists, George, you could stop acting like the rest of your colleagues in the media, and instead do some actual investigative journalism.

You’ve surely debated denialists long enough to know that you shouldn’t believe everything you read in the comments below your blog.

Agreed. Move on in what direction? That’s up to you and George, I think.

How about looking this way?

“If there exists a “conspiracy” around climate science, I think it is not of climate scientists, but of anti-science forces based on a long-active core group, amplified by a horde of helpers. It has been running for at least 20 years. Many people are named here whose email logs and funding requests would be quite instructive….”
(From John Mashey’s current text, linked above)

I’m very much with George here. All seem agreed that science demands full and frank disclosure, and that the commercial interests in secrecy should be discouraged.

It is now easy to post all data, codes etc on the web which removes any argument about difficulty, harassment etc.

There seems to these eyes the beginnings of a closed shop culture in science (as in many other areas) which gradually erodes destroys the rigour and turns it to dogma (it’s much easier to be frank and fearless if everyone who reads your papers agrees with at least the foundation of your belief).

In the case of climate change this is the only way we will get over the hump and regain credibility.