The CRUtape Letters™, an Alternative Explanation.

With the blogosphere all atwitter about the emails and data “stolen” from the Climatic Research Institute at the University of East Anglia, two theories have become dominant describing the origin of the incident.

CRU was hacked and the data stolen by skilled hackers, perhaps an individual or more insidiously some sophisticated group, such as Russian agents.

An insider leaked the information to the NSM (non-mainstream media)

Theory number one is the preferred explanation of the defenders of CRU. This allows them to portray CRU as victims of illegal acts. It allows them to scream bloody murder and call for an investigation of the crime. How can we take the fruits of hideous crime seriously? The end does not justify the means!

One of our favorite writers, Gavin Schmidt, has expanded on this theme with the report:

He [Gavin] said the breach at the University of East Anglia was discovered after hackers who had gained access to the correspondence sought Tuesday to hack into a different server supporting realclimate.org, a blog unrelated to NASA that he runs with several other scientists pressing the case that global warming is true.

The intruders sought to create a mock blog post there and to upload the full batch of files from Britain. That effort was thwarted, Dr. Schmidt said, and scientists immediately notified colleagues at the University of East Anglia’s Climatic Research Unit.

I believe the above statement by Gavin to be a big bunch of hooey. I believe the “hack” was a posting of the same blog comment which was posted at The Air Vent

which was also submitted here at WUWT, but never was visible publicly, because all comments are moderated and publicly invisible until approved by an administrator or moderator. Many of you have already seen it:

We feel that climate science is, in the current situation, too important to be kept under wraps.

We hereby release a random selection of correspondence, code, and documents.

Hopefully it will give some insight into the science and the people behind it.

1118949061.txt * we’d like to do some experiments with different proxy combinations

1120593115.txt * I am reviewing a couple of papers on extremes, so that I can refer to them in the chapter for AR4

I was the first at WUWT to see the comment above and immediately embargoed it. After discussions and many phone calls, we finally began to refer to the information after, and only after, we saw that it was available elsewhere, such as The Air Vent, and also after we knew that CRU was aware that it was circulating on the web.

Gavin’s elaborate description of the hacking attempt at RC is, in my humble opinion, nothing more than an attempt to add meat to the hacking theory in order to increase the vilification of the theoretical hackers. Gavin has demonstrated this kind of misdirection in the past in the Mystery Man incident where he attempted to obfuscate his own involvement in a data correction to station files held by the British Antarctic Survey. In this new spirit of transparency Gavin, why don’t you send Anthony the log files that demonstrate this attempted break in at realclimate.org?

After all, this is a criminal act of vandalism and of harassment of a group of scientists that are only going about their business doing science. It represents a whole new escalation in the war on climate scientists who are only trying to get at the truth. Think — this was a very concerted and sophisticated hacker attack. …Or at the next level, since the forces of darkness have moved to illegal operations, will we all have to get bodyguards to do climate science?

Sigh…and sigh again.

Theory number two is the preferred explanation of, for want of a better term, the Skeptics Camp. It is a romantic thought. Some CRU employee, fed up with the machinations, deceit, and corruption of science witnessed around him or her, took the noble action of becoming whistle-blower to the world, bravely thrusting the concealed behavior and data into the light for all to see. This theory is attractive for all the right reasons. Personal risk, ethics, selflessness etc.

I would like to offer a third possibility based on a bit of circumstantial evidence I noticed on the Web Saturday afternoon.

There’s an old adage, never assume malice when stupidity or incompetence will explain it.

A short time ago there was a previous leak of CRU data by an insider. In this case, Steve McIntyre acquired station data which he had been requesting for years, but someone inside CRU unofficially made the data available.

In this case, many commentators had various guesses as to the motivation or identity of the disgruntled mole even proposing that perhaps a disgruntled William Connelly was the perpetrator.

It would take a hacker massive amounts of work to parse through decades of emails and files but stealing or acquiring a single file is a distinct possibility and does not require massive conspiracy. The same constraints of time and effort would apply to any internal whistle blower. However, an ongoing process of internally collating this information for an FOI response is entirely consistent with what we find in the file.

In the past I have worked at organizations where the computer network grew organically in a disorganized fashion over time. Security policies often fail as users take advantage of shortcuts to simplify their day to day activities. One of these shortcuts is to share files using an FTP server. Casual shortcuts in these instances may lead to gaping security holes. This is not necessarily intentional, but a consequence of human nature to take a shortcut here and there. This casual internal sharing can also lead to unintentional sharing of files with the rest of the Internet as noted in the Phil Jones, CRU mole, example above. Often the FTP server for an organization may also be the organization’s external web server as the two functions are often combined on the same CPU or hardware box. When this occurs, if the organization does not lock down their network thoroughly, the security breaches which could happen by accident are far more likely to occur.

Since Friday November 20th a few users noticed this interesting notice on the CRU website.

This website is currently being served from the CRU Emergency Webserver.
Some pages may be out of date.
Normal service will be resumed as soon as possible.

Here is a screen grab for posterity.

So as part of the security crackdown at CRU they have taken down their external webserver? Network security professionals in the audience will be spitting up coffee all over their keyboards at this point.

So this is my theory is and this is only my theory:

A few people inside CRU possessed the archive of documents being held in reserve in case the FOI appeal decision was made in favor of Steve McIntyre. They shared it with others by putting it in an FTP directory which was on the same CPU as the external webserver, or even worse, was an on a shared drive somewhere to which the webserver had permissions to access. In other words, if you knew where to look, it was publicly available. Then, along comes our “hackers” who happened to find it, download it, and the rest is history unfolding before our eyes. So much for the cries of sophisticated hacking and victimization noted above.

If I had to bet money, I would guess that David Palmer, Information Policy & Compliance Manager, University of East Anglia, has an even chance of being the guilty party, but it would only be a guess.

To repeat the basic premise of this theory.

There’s an old adage, never assume malice when stupidity or incompetence will explain it.

474 thoughts on “The CRUtape Letters™, an Alternative Explanation.”

I had thought something similar. But I’m not a network security person (although I have worked with network security guys whose job was often to physically infiltrate systems, as well as the more mundane hacking). However, I think this has to be the best theory out there. The files are too specific to the FIA request to be anything but. Nice job!

I would buy into Theory No.3 on that basis. It is indeed plausible. It also means that ICT people at CRU were fully aware of the content and sensitivity of this material because of the editing process. So it was either by accident (or design) that a prepared FOI file ended up on an insecure FTP server.

Interesting thinking. Having been the unofficial server manager and maintainer on more than one occasion at various places of employment, I can attest to the “quick-and-dirty” method of sharing data and managing systems. Sometimes you kludge things together to get a task done, without thinking things through. Not excusing it, but this is reality.

I think your theory has merit. I’m really curious who the leaker is. Wheels within wheels.

Well my theory (no doubt already aired by others anyway) is that the file is labelled FOI, not because it was intended to be part of an FOI release of information but, on the contrary, were emails and texts meant for deletion so as to avoid being seen via FOI requests.

Curious. I wondered about the name of the file, FOI2009.zip, and this could explain that.

The Email selection is rather odd, it’s certainly not what the researchers would put together, what with very old Email from John Daly, but it might something culled by someone less coupled to the science but charged with collecting Emails.

I’m not convinced, but I think this is more likely than an outside breakin.

What have yet to see is any credible claim that the hacked documents are fake.

BTW: Would these documents be admissible in a court of law? I believe they would, given my understanding of the circumstances of how they were released. Would someone attempting to use these documents first have to prove they had no part in the hacking?

The careless/stupid scenario makes more sense than the alternatives. Whoever got into the FTP server probably found the zip file already prepared for exploitation. It makes little sense that someone would break into the server, download the entire volume, and then sort/organize hundreds of megabytes for later publication.

Like you, I believe that the CRU boys are the unwitting architects of their own outing.

I reckon that’s about right, NK. It’s surely no coincidence that all this stuff broke loose last week just as those chaps in Switzerland were starting up their – what’s it called? – – the Large HADCRUT Collider?

in all seriousness, your explanation is very plausable, it certainly passes Occam’s Razor. Congratulations. Let me add one observation to Chainpin’s question of why would this file “…contain such damning evidence… Something doesn’t add up.”
It’s possible this file was put together as a potential response to the FOI request, and circulated around CRU for review and a decision as to whether to release per the FOI. When the CRU types reviewed, they opted to stonewall, and no way let this file see the light of FOI day. Someone disagreed, and posted the file. Who and why? those are the pertinent questions. My naive hope is someone who was unhappy with the CRU crew’s attempt to stonewall the public. Although it was probably a CRU crewmember with some kind of grudge.

Hmm, not sure about the cock-up theory, but one thing is for sure. Those files were gathered together by someone in the know about these things. There’s just too much of it that applies to the sort of things we are discussing for it to have been done by some random guy on the internet. Being very good at getting into servers doesn’t automatically mean that person would know what is useful, and what is not, unless of course, they’ve been following WUWT and CA since their inception. I think the extra material that was put in along with the controversial material, plus the notice that it is a random selection of a possibly bigger set, was put there to put people off the scent.
We’ve already seen an email from Jones telling people to delete emails, it’s very unlikely that he would leave that email about for a possible FOI request, along with an admission that he had already smoothed things with some FOI officers so that information could be withheld. This has to be a major embarassment for those FOI officers too.
Notice too that there are no emails to loved ones asking if they need milk, or something, collected on the way home, or speculation about a rugby match (Didn’t I see Briffa with a Wallabies shirt on in one of the photos of him?). This has all the fingerprints of someone on the inside, in the know. Hopefully, time will tell.

“use of the registration symbol is improper where the mark has not been federally registered.” USPTO

Anyone has the right to add the TM symbol to put the public on notice that they consider the word a proprietary mark. However, it takes quite some time and expense to formally have a mark “registered”. There has hardly been time to file the paperwork, let alone have any trademark office respond. I highly doubt that it is “registered”. Thus recommend marking it as TM – a trademark, until you have official confirmation of formal registration.

Blame President Bush. He triggered a global economic crisis.
This is classic defense playing out. They act all violated and feign outrage to distract from the content of the e-mails and guilt in manipulating both data and weights on different years by tweeking the programs.

Clinton helped a troubled intern. Jones and Mann just are protecting us from the evil sceptics. I can’t aborrt a 2 year old toddler and appeal to privacy.

They can’t manipulate data and manipulate peer review and evoke some privacy protection.

I have another theory.
The files included in this leak were extracted from the main servers in preparation for the possible successful FOI request. This was done following an internal review of likely contentious, material.
The internal review would have come up with an “Eeek, we’re screwed if this lot gets out!” response. This then triggered the purge of the main servers in preparation for a complete (apparently) opening up to external scrutiny.

My belief is that this purged material was on a back-up/DR server, and the main servers have already had this material removed. They just had not got around to purging the back-ups yet.

As to who did it/how it got out – how about either the IT support function, or…the person tasked with responding to FOI requests, the one pressurised by Jones and the VC, et al?

You don’t need an FTP server if you have a website.
Someone I work with and know wants our full client address list pronto, but they are halfway across the globe.
They email me with the request.
I upload the file via FTP to my website, conveniently named mywebsite.com, and email my colleague to type http://mywebsite.com/addresslist.zip into their browser, and to email me when they have obtained and verified the data.
I receive their email and via my ftp program, I delete addresslist.zip, or if I am devious, I upload a file of garbage with the name addresslist.zip in its place.
Job done, and unless there are active spies surveying the files on my website,
not easily hacked in the time it takes.

I can not imagine they would give out these sensitive data/code/emails, even if forced by FOIA. I can imagine they would give out some filtered information, treering data without metadata and so on. I think name “FOIA2009″ was neat idea of the CRUmole.

In this case, never assume stupidity or incompetence, when malice will explain it.

Maybe it is my imagination but I recall someone making a comment about the irony that the data that McIntyre obtained earlier in the year was in a subdirectory named FOIA or something. The memory is vague but I do seem to recall someone making a comment about the irony of it.

I agree with your theory that the file contained emails and other files intended for destruction. Why would such incriminating information be collect in anticipation of having to turn it over to your most ardent skeptics?

After all, they have spent more than the last decade hiding their activity!

In general I prefer the cock up theory of history to the conspiracy one: which does not mean conspiracy does not go on as can be seen here.

So I think this supposition quite plausible if someone in admin put the files together ready to meet an FOI request if it passed. Presumably the collection is most files and only those relevant to an FOI would have been selected for release.

How it then got into the electronic ether and to whom who knows? The possibilities are endless, from finger trouble upwards.

This seems like a reasonable alternate to the insider feeling outrage and posting it. Having worked with network security, users always want access to things and rarely understand the security implications of were things are stored.

And as pointed out CRU has already demonstrated once that they can’t keep proper track of where they are storing things when they put data they didn’t want released on a public FTP server.

Theory 3 is solid and I would say the most likely scenario. I might rate them something like this.

10% chance it was a hacker.
30% chance it was an insider.
60% chance that it was an open FTP server.

That is just ball park figures but gives a feel for how likely I think each scenario is. I’ll note that the hacker hasn’t felt right as an explanation from the beginning.

Theory 3 is the most logical presented so far. Data placed on a public ftp server is not stolen, it’s picked up like a quarter lying on the sidewalk. I would bet that CRU now knows what happened, thus they shut down the server.

The comment by Pierrehumbert is telling. He’s essentially proving Jerry Pournelle’s “Iron Law of Bureaucracy” case.

This isn’t THEIR data; it’s my data. And yours. I paid for it, as did you. It belongs to us, and we may do with it as we please. It’s not up to Pierrehumbert to dole it out to those possessing the magic key or otherwise demonstrating worthiness to his satisfaction.

Of all of the things in this case, this one is the part that’s ultimately the most damaging, this assumption that the public pays for these people to run a fiefdom as they jolly well please.

The emails and the code commentary are interesting but don’t seem to demonstrate a concerted conspiracy. Sure there’s turf wars with others and blockades to control the mechanism of peer review. This is all part of how the big boys do it everywhere, not just here. The contents of the data release will not likely yield a smoking gun so much as provide some of the data that was supposed to have been released years back via FOIA.

In sum, the travesty playing out is that this is the data that will affect the lives of billions if certain political aims are achieved. Certainly something of this level of importance ought to have been gone through with a fine tooth comb and universally agreed upon accordingly. And they have been witholding the data as if it’s part of their personal playground.

I would not have included a bunch of emails talking about dodging FOIA if I were resigning myself to honouring the FOI requests.
How about the ignored subject lines DELETE AFTER READING? Why not just delete them?
My vote is Whistleblower. (translation for visitors from RC; 5th. column cyberterrorist)

Who and which organizations gave the “scientists” money for this “research?”

Any one of them has grounds to sue for fraud in a civil suit. Either the granting organizations or the donors ( individually or as a class) can do it.

The same goes for the donors who gave money to the granting organizations. If any board members can be shown to be in collusion with the researchers, then the board member or members become liable as well.

As for misuse of government money and property, that is a criminal matter.

If the CRU staff put it together as a response to the FOI request, why would they have included the HARRY_READ_ME file, since it’s neither an E-mail, data, or source code? The inclusion of that file makes it more likely that it was put together by an internal source.

While I was never in the network security branch at (nameless Government facility), I knew some of the network engineering people pretty well, and became aware of a number of permission screw-ups at our facility; i.e.:
While there is no way to know for sure at this point WRT the CRU saga, the ”third possibility” that Charles postulates is indeed quite plausible:

The multiple and complex layering of network permissions that can be applied by (likely) several people who have network admin capability, can quickly lead to a bunch of open ”back doors”; especially if both internally and externally accessible files reside on the same server (let alone the same disc spindle). Where I worked it was a firm rule that you did NOT mix any internal and external applications or data on the same platform, due to this ever-present risk and a couple unhappy experiences; where access permissions would become ”leaky”.

SIDEBAR: I believe the original of the ”old adage” Charles referred to is attributed to Napoleon Bonaparte; who is recorded to have said:
”Never automatically ascribe to deliberate malice, that which is easily explainable by ordinary, everyday incompetence.”

In any case: Whether climategate resulted from incompetence in network management, actions by an internal whistleblower, or a sophisticated external hack (all perhaps aided as mentioned by actions resulting from FOIA requests):
I’ll take it. When the history of the whole politically-correct AGW religion is finally written, this event just might make the book.

When I first saw this file, it looked to me to be a lot like the kind of file that gets put together for a lawyer. Typical an organization would get a discovery notice or a FOIA request and a lawyer will send out an email requesting all data, information and correspondance related to the issue at hand. A clerical person assembles all the responses into a file and the lawyer goes through it and issues an opinion. So labeling this kind of file with ‘FOI’ in the filename would make sense.

But this raises a troubling question, could independent officials at Met have looked at this file, with all the references to deleting data etc, and conclude that there was nothing afoul? It staggers the mind.

In some of the emails Jones’s smuggly talks about his success at corrupting the FOI people. It was very Obi Wan Kenobi-esk. “These are not the droids you are looking for”, Jones said in a hypnotic voice — kind of thing.

It would be interesting to see the organizational structure of CRU. Does Palmer report to Jones? Or is Palmer part of Met? Was Jones able to pull the wool over the eyes of the FOI people?

Maybe someone was smart enough to ‘get’ was going on, and when the decision was made, decided to go rogue. Or, maybe they just left the file in an insecure place.

Assumig this is the CRU’s own compilation of data intended for the FOIA, is this the “nicest” emails thay can cough up. Surly they must have deleted some files? Or do we assume full disclosure if they had lost the FOIA request.

Charles:
Excellent and thought provoking piece. As you put it, I am hoping for Theory II primarily because one way or the other another shoe will drop. If it is Theory III, then there is a good chance that they will simply stonewall and hide all incriminating stuff. It is a sad commentary on the state of our universities that such a tactic is likely to succeed. It would take but only one “deep throat” to end this painful episode.

Sounds very plausible, especially given the name of the file. However, if it was simply copied from the ftp site by someone, why do they remain anonymous? I would be proud to be associated with this historic download!

You’ve only given the verb ‘alternate’ – it is also a noun meaning one who substitutes for another (an alternate on a jury or sports team), and an adjective meaning ‘alternative’ (an alternate location).

Charles–
…. It’s possible this file was put together as a potential response to the FOI request, and circulated around CRU for review and a decision as to whether to release per the FOI.

After reading the post this was one of my thoughts. Whether in response to an FOI request or not, someone collected the most damaging material and started to circulate it in order to ensure a coordinated response if the material had to be released and possibly to provide a guide to what material should be purged to avoid disclosure. Receipents started to forward it to people they thought would be interested in a chain-reaction/chain-letter fashion that obscured the original intent of the compiler. Eventually it hit the inbox of someone who was ‘not reliable’ (in the CRU sense), maybe after someone on the fringe of the group skimmed the emails and saw a co-worker’s name.

I would agree with #3, due to the fact that code files were included. Wasn’t that part of the FOI request in the first place? In preparing for a ruling, it would make sense that a download was done and scrubbed of emails not related to the exact nature of the request, (meaning that for the download only, emails were removed, not for the original archived email). In some cases, blanks can be located if emails have a number or date stamp. It is a simple matter of creating a find function looking for these gaps.

But the first theory was so beloved by the media: This could only have been the work of a criminal mastermind of the highest order. Now all we’re left with is simple human error. Still, it smells right. The problem with the first theory is why would all these emails be bundled together in a single file? They would have been scattered around quite a bit I would have thought.

How did you figure it Charles? It was elementary my dear Hansen, or Jones, or Wigley.

Good scenario. It would take too much work for a simple hacker to have sifted through the tens of thousands of e-mails that have been written over the last few years.

Regarding whether these e-mails can be used in legal proceedings:

to answer John Galt:

first, ignore the likelihood that as bad as these look, there will be *no* legal proceedings rising from these e-mails. No one’s going to charge them with anything, anymore than they’re going to charge Gordon Browne for the way he’s running the UK these days. (many things that should happen, don’t.)

Also, I’m not completely clear as to UK law in this area.

but in the US – in a criminal case, the Prosecution can use anything it gets its hands on as long is it wasn’t directly implicated in the illegal action. If someone steals a gun from a murderer and gives it to the police, they’re allowed to use that gun as evidence at the murderer’s trial. If the Prosecution or the police stole the gun, then they can’t use it.

in a civil trial, the complainant simply has to ask for disclosure of any relevant documents. If the defense fails to turn over documents for which complainant has good evidence of their existence, then the defense can be sanctioned for failure to comply with discovery and probably will lose on a motion for summary judgment.

Writing in the Times, Lord Lawson, the former Conservative chancellor and long-time climate change sceptic, said: “The integrity of the scientific evidence on which not merely the British government, but other countries, too, through the Intergovernmental Panel on Climate Change, claim to base far-reaching and hugely expensive policy decisions, has been called into question. And the reputation of British science has been seriously tarnished. A high-level independent inquiry must be set up without delay.”

Bob Ward, director of policy and communications at the Grantham Research Institute on Climate Change and the Environment at the London School of Economics, said: “Once appropriate action has been taken over the hacking, there has to be some process to assess the substance of the email messages as well. The selective disclosure and dissemination of the messages has created the impression of impropriety, and the only way of clearing the air now would be through a rigorous investigation. I have sympathy for the climate researchers at the University of East Anglia and other institutions who have been the target of an aggressive campaign by so-called ‘sceptics’ over a number of years. But I fear that only a thorough investigation could now clear their names.”

Conversely those responsible for the data object:

A spokesman at the Met Office, which jointly produces global temperature datasets with the Climate Research Unit, said there was no need for an inquiry. “If you look at the emails, there isn’t any evidence that the data was falsified and there’s no evidence that climate change is a hoax. It’s a shame that some of the sceptics have had to take this rather shallow attempt to discredit robust science undertaken by some of the world’s most respected scientists. . . .

It is a matter of concern that data, including personal information about individuals, appears to have been illegally taken from the university and elements published selectively on a number of websites.

The volume of material published and its piecemeal nature makes it impossible to confirm what proportion is genuine. We took immediate action to remove the server in question from operation and have involved the police in what we consider to be a criminal investigation.

The material published relates to the work of our globally-respected Climatic Research Unit (CRU) and other scientists around the world. CRU’s published research is, and has always been, fully peer-reviewed by the relevant journals, and is one strand of research underpinning the strong consensus that human activity is affecting the world’s climate in ways that are potentially dangerous.

CRU is one of a number of independent centres working in this important area and reaching similar conclusions. It will continue to engage fully in reasoned debate on its findings with individuals and groups that are willing to have their research and theories subjected to scrutiny by the international scientific community. The selective publication of some stolen emails and other papers taken out of context is mischievous and cannot be considered a genuine attempt to engage with this issue in a responsible way.

The raw climate data which has been requested belongs to meteorological services around the globe and restrictions are in place which means that we are not in a position to release them. We are asking each service for their consent for their data to be published in future.

In addition to supporting the police in their enquiries, we will ourselves be conducting a review, with external support, into the circumstances surrounding the theft and publication of this information and any issues emerging from it. ”

The last sentence MAY indicate a REAL investigation in as they state “any issues emerging from it”.
I think it was a inside man.

PR GUY
[i]”But this raises a troubling question, could independent officials at Met have looked at this file, with all the references to deleting data etc, and conclude that there was nothing afoul? It staggers the mind.”[/i”

I’m wondering if this was the data excised from what was shown to the FOI officers at the University? Ie they pulled all of the questionable stuff into a side archive and tucked it out of the way. So when when the FOI officer checked things he didn’t see anything questionable.

Cheaters. All morning, Gavin Schmidt has been posting snarky comments into many posts on Real climate. He is a gubment worker and on the clock at NASA. He is cheating and moonlighting. this needs to be reported. We are his employer. It seems they forget who they work for. I fire workers for this and have the right to do so because our employee handbook calls for best work effort. He can sit there and run a gossip blog while the economy burns.

# charles the moderator
What Gavin claims is difficult to assess. The “hack” posting at The Air Vent may have had a special twist, as the subject there was about “An open letter reply to a letter written to government by 18 different scientific organizations concerning climate change legislation” that criticizes the associations about climate terminology:
“Dear President or Executive Director,
How could it happen that more than a dozen of the most prestigious scientific associations signed and submitted this letter on ‘climate change’ without having ensured that the used terminology is sufficiently defined. Good science can and is required to work with reasonable terms and explanations…… Etc etc” http://noconsensus.wordpress.com/2009/11/13/open-letter/ or: http://www.whatisclimate.com/

This posting went online together with three other subject on November 13, with the comments under: “Gone for Several Days”. Why did FOIA chose the OPEN LETTER and not one of the other posting. The posting came as Comment 10 on Tuesday, November 17, at 9:57 pm. That may raise several questions:
__When was CRU hacked ?
__Is a hack, as claimed, into a different server supporting realclimate.org on Tuesday realistic?
__Is it reasonable to assume that choosing The Air Vent and the OPEN LETTER was a cheer coincident, or was it intentionally? One option would speak more for hackers, the other more for an insider.
At least the author of the OPEN LETTER made not distinction but just thanked FOIA for his/her contribution, Comment21, at: http://noconsensus.wordpress.com/2009/11/13/open-letter/ .

BTW, a distinct lack of trolls on this thread. While the lack of trolls takes away some entertainment value, it leaves the comments to be all high quality thoughtful stuff. Back to the trolls, I think even they know this IS REALLY BAD for the Church of AGW.

Perhaps an hourly employee or intern was assigned the task of selecting material for the FOI request, and not familiar with the history, was making the file available for members of ‘the Team’ for editing. Such would explain why the information contains all the e-mails as well as code, data, and whatever else. The comments about John Daley, the cartoon, and so on seem to be unrelated to the issues and data sought by SM at CA and others.

The actions (or inactions) of the CRU appear to be consistent with preventing further inquiries into their e-mail, etc. , as opposed to stopping, reversing, or even explaining what just happened. In other words, this was small potatoes. The real meat is still out there to be had. CRU would do anything to prevent further FOI releases, even if it meant giving up grant money, or losing key people. When one resorts to defending the indefensibile, the game is lost (just a matter of time).

There are a couple of posters who are on the right track here, I believe. The file was prepared in response to FOI requests, but was the repository for the items to be “lost”, an insider then decided that such subterfuge should not go unpunished and then followed Dr. Hansen’s advice and embarked on a little civil disobedience.

RealClimate has gone quiet for an hour or so. Perhaps Gavin is simply exhausted or perhaps nobody wants to get further tainted or perhaps there is some “big” news about to break. Or they could have all gone downstairs for lunch and a discussion about yesterday’s shellacking of the Jets by the Patriots!

Some of your theory makes sense, but some does not. Why would Jones assemble a file containing emails that are at best embarrassing to him and his associates, and at worst indicative of criminal conduct because of FOI requests? If Jones recognized that he might have to ultimately respond to an FOI request for temperature data, then he would reasonably assemble that data in a single file, but not the damaging email messages and other files.

Whoever assembled the “random selection” of email messages, they had to have searched through quite a lot and eliminated all the personal stuff. “Hey, you want pizza tonight?”, or “Our grandson is sick, so I am staying ovenight to help.” is completely absent. They left only business related messages. Whoever this was, they clearly did not want to post other people’s personal lives on the internet. I think it odd for an outside ‘hacker” to show such sensibilities. Whoever this was, they focused on the real climate issues, not the chaff. (no pun intended :-))

Lucia has an interesting post that includes the above point.

So, it seems more likely to me that either:

1) Jones had assembled a series of damaging messages, raw data files, and documents in a single location so they could be quickly destroyed (for example, if an FOI request for raw data or emails was allowed). Then someone on the inside found the assembled files and thought they needed to be posted, or

2) Someone with network access (and maybe administrator authority) got hold of Jones’ password, poked around, copied a bunch of files that indicate unethical behavior, misconduct, and an overall bad attitude by Team members, and then posted the lot. Maybe the November 12 denial of the McIntyre’s FOI was the “last straw” for this person.

I lean heavily toward the later. Will we ever find out? I doubt it. The person who sent Steve McIntyre the raw temperature data does not appear to ever have been identified; I think it likely this recent posting was the same person.

I’ll bet the Team will be making more phone calls in place of email messages when the subject matter is (how shall we say) “sensitive” in nature.

Agreed. Again congrats to Charles, his thoughts on this generated several comments along this line. Unfortunately, as a practicing lawyer I have encountered the bureaucratic mindset and protocols for 27 years, and Charles’ reasoning fits how agency staffs try to stonewall legitimate FOI requests. This will play out, the public will learn more. But it will take continuous push back against these professional bureaucrats and grant mongers.

This all hacking story appears to me much ado about noting. A lot of (blog)wind for a flatulation in a bottle. Nowhere, at any place can be read that the global warming AGW theory is set up by a bunch of conspiring senile or malicious would be scientist.

I have the impression the the e-mails reveal that the AGW scientist just are human beings. Sorry this is rather a negative story in the case of Antropogenic global waring critics. If these critics must build a case on theft and robbary…then they are no dime better than those who try to lie about some 10th of a centigrade more or less.

“My belief is that this purged material was on a back-up/DR server, and the main servers have already had this material removed. ” Forgive my ignorance on such matters, but assume that your belief is true. Is it possible for an outsider both to download the file and then to delete it from the server? In other words, is it possible that briefly only the outsider had the file and that CRU had lost it completely?
Because, if so, CRU can’t really know whether the version that is now available to us all is complete and unaltered, can they?

A reasoned debate with Chris Horner of CEI utterly shaming the opponent into calling for a full investigation. Horner also alluded to “a total of 165Mb of data,” 65 released and another 100Mb to come.

More dynamite?? Does CRU really want another 100Mb of inside information released for the entire world to see? Have they no sense of self preservation? If I were the Chancellor of UEA I would be sitting down to hard nosed discussion of how to prevent the next release. That would in all probability require CRU turning State’s evidence. If they continue to stonewall the window of forgiveness will close and they’ll be just another culprit in the cover-up. The Nixon syndrome.

Dr. Phil et al are going to be sacrificed sooner or later. Question is, by sacrificing them early-on can Chancellor Gough stay the release of even more damaging revelations??

I’m heading to WalMart for an air popper – getting too fat on the hot oil corn.

If it was indeed an insider whistleblower, Jones, Briffa, Kelly, and some others could be crossed off of this CRU staff list, but who knows what lurks in the hearts of the others?

Hmmm, I’d put my money on Mr. Mike Salmon, just because he’s the IT guy. But, if it’s hard to get in contact with Salmon to inquire about this matter, it’s probably because he’s being water-boarded by Jones and Briffa.

Reply Andy Revkin
Dot Earth blogger, Reporter
November 23rd, 2009 1:19 pm
Thanks for reminding folks what this “Your Dot” feature is about. I’ll be posting a voice rebutting the “cyberterrorism” assertion as soon as I can get someone who’s a non-anonymous contributor to step forward with a reasoned, substantiated counterargument.

If you computer searched files with a specific set of keywords you could well have generated the silly cartoon. It is the Harry file that is tough to figure out – unless it has the names of key players or proxies embedded in the comments.

News flash. OJ Simpson was out looking for the killer of Nicole and bumped into the Climate folks looking for the hackers. Big oil must have funded the hackers.
The hackers story is a bunny trail many are giving up on.

Regarding the provenance of the files in question: Trenbreth(sp?) admitted that the files contained only what was ‘helpful to skeptics’ and that only part of what was taken was actually released. I posted this is in Tips and Notes, and do not feel like retyping my thoughts on that.

Scenario #3 seems the most likely of them, and actually is pretty consistent with Trenbreth(sp?)’s public statement on ABC.

It is a matter of concern that data, including personal information about individuals, appears to have been illegally taken from the university and elements published selectively on a number of websites.

The volume of material published and its piecemeal nature makes it impossible to confirm what proportion is genuine. We took immediate action to remove the server in question from operation and have involved the police in what we consider to be a criminal investigation.

The material published relates to the work of our globally-respected Climatic Research Unit (CRU) and other scientists around the world. CRU’s published research is, and has always been, fully peer-reviewed by the relevant journals, and is one strand of research underpinning the strong consensus that human activity is affecting the world’s climate in ways that are potentially dangerous.

CRU is one of a number of independent centres working in this important area and reaching similar conclusions. It will continue to engage fully in reasoned debate on its findings with individuals and groups that are willing to have their research and theories subjected to scrutiny by the international scientific community. The selective publication of some stolen emails and other papers taken out of context is mischievous and cannot be considered a genuine attempt to engage with this issue in a responsible way.

The raw climate data which has been requested belongs to meteorological services around the globe and restrictions are in place which means that we are not in a position to release them. We are asking each service for their consent for their data to be published in future.

In addition to supporting the police in their enquiries, we will ourselves be conducting a review, with external support, into the circumstances surrounding the theft and publication of this information and any issues emerging from it.

Given the high proportion of embarassing content in this selection of e-mails, it makes more sense to me that this is their archive of information extracted from their main mail record, which, having been cleaned, would then be made available for inspection under any FOI request.

I guess they might reason that any data provided under such a request, if supplied voluntarily, would be not be subject to the same intense scrutiny as would be the case if it was extracted under duress, thus a few omissions would not be spotted.

Since some of the e-mails actively encouraged FOIA avoidance, the attempt to hide them must be worth a try because their discovery would not increase the penaties any further.

“I’m thinking this collection is more along the lines of “Stuff to be deleted in the event of an unfavorable ruling on the FOI request”.”

Sorry – I am a skeptic and read through Bishop’s list of most damaging posts. There is not a whole lot of “damage” there. The emails clearly reveal a “them verses us” mentality and a somewhat disciplined effort to control the news flow. I see nothing that is obviously illegal or clearly dishonest.

I find the notion that HATCRU would engage in a systematic coverup of an approved FOI request not credible. So I go with option 2 or 3 as well.

Having worked in IT and as a network security professional for my entire career of 25 years thus far, this sounds like the most plausible theory. I was just discussing the difficulty of implementing effective security policy with a colleague the other day. By far the biggest hurdle for the IT manager is human nature. So many managers, directors, etc. in other departments will undermine policy of a weak CIO just to make things easier.

And we have a saying in the biz, security is the reciprocal of convenience.

Not only are these people in the Alarmist camp poor at science and programming, they are poor with computer and network security as well. Consistency.

Chris,
Nail on head.
I wonder how many hard drives are being destroyed, illegally, as we discuss this at GISS, etc.?
This small sampling that we have seen is the tip of the iceberg.
The climate charlatans have been doing this for *years*.

I’d also submit that possibly we have a hybrid of Theories 2 and 3. Once everything is neatly contained in a ZIP file, it wouldn’t take much for a low level, but scrupulous employee at CRU to move that file, or to disperse it to an external site. Many IT professionals I know tend to be very objective, and operate with a very high level of integrity. It’s part of the job. So no surprise that someone who became aware of behavior like that illustrated in the emails, would make sure that it was exposed and/or reported.

I can just imagine the politician’s approach when they realized what the screwed up temperature plots looked like.
“- Wow did you see that! it seems like the world is going to catch on fire within the next 50 years!”
“- No No Harry! Don’t bother reporting the bad data to your superiors. Leave it like that! That’s perfect”
Bush to Cheney to Obama to Gore (and all other corrupted bastards):
“- How can we take advantage of this bullshit data? Oh I know!!! lets establish an another tax disguised as Cap And Trade and impoverish even more our stupid population!”
– ” Great idea. Great work Tim and Harry!”

On Theory..
A little bird told me.. (and one I have been inclined to listen to) that it was a smash job.. a brute force attack.. not a clever hack.. but a resource laden effort.
this was not a leak in the deliberate sense.

now.. what that means is up to our esteemed public, but I would add, the nature of such capabilities, are in only a few places.

but whether tongue and cheek or no, the notion that espionage from the ghost of the Kremlin be considered
in theory… well….

It was an ínternal hacker close related with gouvernmental institutions or politicians. Nobody believes the science behind the global warming hoax anymore and the western world feels the immense growing economic power of India and China while at the same time we have to kill our economy and society due to requirements of the Kyoto protocol. India and China already stated not to fullfill these requirements, not yet and also not after Kopenhagen. So probably the Britisch play a nice and traditional game to give theirselves and others opportunities from escaping the rope we put around our neck. Also the chosen time is remarkable; very close to the start of Kopenhagen. It’s a version of the “legal” attack on Mantsjoerije by the Japanese at the beginning of WW2 in Asia (China).

Just read that press release posted above @Cold Lynx (10:14:21) — it’s a toss-up between nauseous and rushing to take a long, hot shower. I expect that they are trusting that people are either too lazy to have read what’s been uncovered, can’t hold a thought of their own in their own heads for more than five minutes, or are so into the AGW belief system that they’ll believe any pronouncement from ‘church’ officialdom.

“All their peer review is done by friends and people that push the same dogma.”

That’s kind of unavoidable, because in any field there aren’t that many “experts” to whom to send papers, so they will often be reviewed by a small cadre of scientists who know each other. That makes the system vulnerable to not only lapses in integrity, but also to the inertia of “group think.”

But the most serious flaw is that it can be so easily hijacked by a group of conspirators whose “integrity” is merely a carefully crafted perception as opposed to a reality.

“You’ve only given the verb ‘alternate’ – it is also a noun meaning one who substitutes for another (an alternate on a jury or sports team), and an adjective meaning ‘alternative’ (an alternate location).”

Modern dictionaries are more latitudinarian about their definitions than they used (and ought) to be. It’s possible that some or many of them now allow the adjectival “alternate” to mean “alternative.”

But why should we go along with them, just because they want to be descriptive of the language as the majority uses it? The cost is clarity. If I have an alternate pair of shoes, that means they are ones I wear by turns with my regular pair of shoes (presumably in order to give each pair a rest or allow them to get un-stinked on alternate days). An alternative pair of shoes would be one (probably in a much different style or form or color) that I wore to a different occasion, or with a different colored pair of pants, not one that I automatically us in a to-and-fro fashion. (There’s probably a better example, but I can’t think of it offhand.)

Here’s what R.H. Fiske’s Dictionary of Disagreeable English says on this matter. (The book is too much of a “stickler” for my taste, and is sometimes too heated, but it’s basically right IMO):

“The adjectival alternate does not mean, as alternative does, providing a choice between two or more things, nor does it mean, as alternative does, relating to an undertaking or institution that appeals to nontraditional interests. … If some people insist on maintaining the distinctions, it is because they prefer clarity to confusion ….”

If anyone is interested, I have looked at some important data that became public from CRU which seems to be to close the case global warming (man-made or otherwise). And it shows off that lovely 1940’s blip:

It is clear that the file was created internally at CRU. If it was and judging its content, the files were not gathered to be released in the eventual FOI request. Either it was put together to be destroyed or hidden for some sick reason. In any case, they then would also be guilty of hiding information.

“CRU was created by the Thatcher Government as an arm in its war against the coal miners and the oil sheiks. This was a case (unfortunately not isolated) in which the smart tactical manoeuvre became a grand strategic error, for it bequeathed a powerful tool to the new authoritarian left when they reins of power changed hands.

A quasi-scientific institute that is founded for political purposes is a misbegotten creature. It is conceived in cynicism and born to corruption. When the remit of such an institution is to manufacture evidence to support one particular hypothesis it is condemned not to produce just bad science but anti-science.

The basis of modern scientific method is the principle of falsification. We do not call upon it directly for every scientific investigation, just as we do not rush to the courts of law every time we sign a contract, but it is always there to provide the rigorous framework essential to progress. To pay someone to collect data that support one hypothesis is like, to adapt the classical analogy, paying someone to count white swans to “prove” the hypothesis that all swans are white. Furthermore, once that someone’s living depends upon that payment, he will be sorely tempted to cover up any evidence of black swans and, being human, he will try to salve his own conscience by creating a justification for ignoring inconvenient observations.”

http://www.climatesciencewatch.org/index.php/csw/details/phil-jones-and-ben-santer-comment-on-cei/
Prof. Phil Jones, Director of the Climatic Research Unit at the University of East Anglia in the UK and Ben Santer at Lawrence Livermore National Laboratory comment in response to a petition to EPA by the Competitive Enterprise Institute and Pat Michaels, which misleadingly seeks to obstruct EPA’s process in making an “endangerment” finding on greenhouse gases. This new CEI tactic is to call into question the integrity of the global temperature data record and, by implication, the integrity of leading climate scientists.

Almost all the data we have in the CRU archive is exactly the same as in the Global Historical Climatology Network (GHCN) archive used by the NOAA National Climatic Data Center [see here and here].

The original raw data are not “lost.” I could reconstruct what we had from U.S. Department of Energy reports we published in the mid-1980s. I would start with the GHCN data. I know that the effort would be a complete waste of time, though. I may get around to it some time. The documentation of what we’ve done is all in the literature.

WIKIPEDIA: The Climatic Research Unit e-mail hacking incident refers to a November 2009 incident involving the hacking and leaking of e-mails and documents on climate change research from the Climatic Research Unit of the University of East Anglia U.K.[1][2]http://en.wikipedia.org/wiki/Climategate

My question is, is it possible that they had several files on thier server(s)?
One file of FOI that there were willing to release, other file(s) of “hide this stuff till the cops leave” The Devil (to alliterate to Screwtape Letters) who sent the files out grabbed several file sets, then reorged (some of?) them into the current release.

There really is no other explanation of how one (1) hacker had gotten files and (archived?) emails, unless thier security really really sucked. (Global Warming started to deteriorate the electric cables?). I am no computer geek, but for how long do servers maintain a copy of deleted emails? What is the average lifespan of a server, and if one goes bad, do they move all the old data over?

I agree that this thesis has the highest probability of being correct. Of course, somebody would have had to take advantage of being able to poke around and discover it, but for some folks that is a casual hobby. Seeing a large blob of data available on the ftp server would almost guarantee that somebody would download it just out of curiosity.

The Brits have been notorious for their data screwups. NHS laptops left on trams full of unencrypted data and so on. We in the states have done some whoppers too. If all this stuff had been encrypted it would have taken either an insider to provide the key or a dedicated “black hat” to crack it. As it is, neither of these seems to have been the case.

“In addition to supporting the police in their enquiries, we will ourselves be conducting a review, with external support, into the circumstances surrounding the theft and publication of this information and any issues emerging from it. ”

This last statement emanating from the UEA head office, will be interesting to follow. CRU confirms that the “theft” was reported to them on Tuesday Nov. 17, yet they apparently only called in the police on Thursday or Friday. Why the delay?? Standard procedure for dealing with a theft is to immediately report the crime.

CRU took at least two whole days to report their “theft.” What were they doing in the interim?? The Norwich Police, Scotland Yard, MP Clarke, members of Parliament, U.S. Congress, Special Prosecutors and 2 billion people online will be very interested to hear their reasoning. Holy Chappaquidick!!

Dear Moderator slightly of post but it is response to the use of the word Tribalism at CA to describe the behaviour of the Not so Real Climate Scientists. They pass all tests for group think. Our Gavs a lovely boy but he just wont let me post see?

I dislike the word Tribalism it to me represents an unthinking loyalty I would think cohorts in cahoots or better still Groupthink.

I think the broken Hockey Stick boys pass all the following tests with flying colours and should not hide their light under a bushel?

1.Illusions of invulnerability creating excessive optimism and encouraging risk taking.
2.Rationalizing warnings that might challenge the group’s assumptions.
3.Unquestioned belief in the morality of the group, causing members to ignore the consequences of their actions.
4.Stereotyping those who are opposed to the group as weak, evil, biased, spiteful, disfigured, impotent, or stupid.
5.Direct pressure to conform placed on any member who questions the group, couched in terms of “disloyalty”.
6.Self censorship of ideas that deviate from the apparent group consensus.
7.Illusions of unanimity among group members, silence is viewed as agreement.
8.Mind guards — self-appointed members who shield the group from dissenting information.

CRU has an interesting problem. Their ineffective security has resulted in the release of personal and private information of several leading skeptics. Those skeptics now have a cause of action against CRU. Surely this must be the definition of karma.

I can buy the idea that the files were assembled into a convenient package in response to a FOI request. But the timing after that, and the knowledge of where to forward the these documents conveniently within hours of the FOI request denial, does not sound like run of the mill hackers.

Your insights as to the creation of the ZIP archive FOI2009.zip are absolutely correct. The file FOI2009.zip was created by the CRU itself. No self-respecting computer hacker would be that neat. If you open FOI2009.zip with a GUI tool such as StuffIt all of the date/time stamps and directory structures are absolutely consistent with a ZIP archive which was created in place from data organized for the possibility of external export.

After FOI2009.zip was created by the CRU, it found its way to the Internet. I had the privilege of working with John Daly on several investigations. In one case John documented just how incredible sloppy the CRU really was. From the grave read John’s indictment of CRU quality control

I would say that some files in the archive (for example the image marooned.jpg) don’t look like it was put into the archive for a FOI request. Or the other way round: if these are the files and mails they are willing to give out, its almost unthinkable what might still be hidden there.

But this raises a troubling question, could independent officials at Met have looked at this file, with all the references to deleting data etc, and conclude that there was nothing afoul? It staggers the mind.

Was not Jones informed by an IT person to not delete emails in response to FOI, but only if it is a standard practice? Pure speculation, but could he have been caught deleting material and therefore had put IT people in jeopardy of covering or covering up crimes?

I think someone in IT was given list of files and told to erase them after the ‘lost data affair’. Any IT employee would know that is an illegal request, so for his/her own protection ‘zipped’ whole lot on 12th November. Either same person or someone with a grudge came across unprotected file and released it.
Definitely an inside job.

Someone had posted this on another site I frequent and found it poignant:

Problem foreseen 48 years ago:

“Partly because of the huge costs involved, a government contract becomes virtually a substitute for intellectual curiosity…The prospect of domination of the nation’s scholars by Federal employment, project allocations, and the power of money is ever present – and is gravely to be regarded.”
~Dwight Eisenhower (from his Farewell Address, 17 January 1961, better known as his Military Industrial Complex Speech)

It is certainly unlikely a hacker had the time to find all those documents/emails and package them all together so neatly.

But all I really care about is their authenticity. Since CRU says these documents are taken out of context, will they now release everything so we can put them into context? Or have they already lost or destroyed the rest?

To follow on, Gavin’s defense has in many ways been remarkably slick. If the ‘hacked’ file had been carefully cherry picked (just enough to look authentic, but also open to plausible counter-arguments) the surely a counter defense would have been prepared in parallel? They may be without ethics but dumb they ain’t. Do we know who the chess player is on the team?

Oh, I admit my ignorance in these matters, but speculating about an IT insider angry at being put in a precarious position– could the CRU scientist realize that attacking an angry insider would not be a wise course of action because said person perhaps has evidence of file purge attempts?

I agree that this probably isn’t a “hacking” and someone probably had free access to a file they shouldn’t have and made some hay while the sun was shining. That seems to me to be the most likely scenario.

If the file was created for release as the result of an FOI request, One wonders if the file was edited for content BEFORE it was “published”. There are many gems in this file that demonstrate what was going on, but it looks to me like this is not every single email and attachment. I’m wondering if Dr. Phil and Co. removed the worst offending email and files and hoped that the remnants would satisfy the deniers among us. Or perhaps the more incriminating information was removed by the lady or gentleman that performed the operation. One wonders if s/he has more info that will be released. (sits back, reaches for the popcorn)

So Gavin now claims that “trick” means “a good way to deal with a problem.” That’s odd. Back in a November 9, 2006 post entitled “Cuckoo Science” criticizing Christopher Moncton, “trick” meant “absurdities that occasionally pass for serious ’science’ on the web and in the media” and “concepts are being mangled, logic is being thrown to the winds, and completetly unjustified conclusions are being drawn.”

Gavin said in November 2006 that:

Sometimes on Realclimate we discuss important scientific uncertainties, and sometimes we try and clarify some subtle point or context, but at other times, we have a little fun in pointing out some of the absurdities that occasionally pass for serious ’science’ on the web and in the media. These pieces look scientific to the layperson (they have equations! references to 19th Century physicists!), but like cuckoo eggs in a nest, they are only designed to look real enough to fool onlookers and crowd out the real science. A cursory glance from anyone knowledgeable is usually enough to see that concepts are being mangled, logic is being thrown to the winds, and completetly unjustified conclusions are being drawn – but the tricks being used are sometimes a little subtle.

OK, I’ll ask:
Was the reason that ‘incompetence’ was spelled incorrectly in BOTH of the quotes in the post a clever (perhaps too clever) device? If so, many of your readers and I suspect all of your detractors will not get it, or worse, use it to diminish your argument.

Reply: Just an ironic mistake, enhanced by a cut and paste. But I like your theory. Maybe I’ll leave it as is. ~ charles the contrite moderator

While I have no idea who provided the files and could care less , I really like the notion that this stuff was labeled FOI as a precursor to its being deleted – just in case . It would fit in perfectly with the apparent incompetence at CRU . Maybe the gang just couldn’t shoot straight .

“I was forwarded the chain of e-mails on the 12th October, which are comments from some of the worlds leading climate scientists written as a direct result of my article ‘whatever happened to global warming’. The e-mails released on the internet as a result of CRU being hacked into are identical to the ones I was forwarded and read at the time and so, as far as l can see, they are authentic.”

I belive there will be more files available soon.
Either of UEA “investigation” Or by this whistleblower.

I belive “our” Man will give UEA a chance to settle things. If not will more files be released.

Charles,
I have to disagree with you on this one. I’m more in agreement with this line of thought:
“chainpin (09:18:31) :

Interesting theory.

But why would the file contain such damning evidence?

If CRU had themselves prepared it for the FOI request, surely you don’t belive they would have handed over such files to SM, given what we now know?

Something doesn’t add up.”

I highly doubt that someone would have captured those specific emails as part of anticipating an FOI fullfillment request. I also doubt that someone was “browsing” through CRUs FTP server and just happened to see the zip file there so grabbed it, opened it, analyzed it, and then decided to release it with the accompanied statement.

There are just way too many things “off” with that scenario.

I put my money on the inside person who’s been witnessing this for some time, who knows the history, and who also found out the request was denied.
I think they had possibly been accumulating files for sometime, or maybe not, but knew where to look to get the more damning emails and data sources, and as an extra poke in the eye to The Team, named the zip file FOI.zip, as if to say “You’re all guilty, and the way the process is supposed to work, THIS is part of the information that SHOULD have been released, based on the request. I’ve just shortened the timeline.”

No hackers, no schizophrenic Dr. Jones…just plain and simple someone inside CRU wanting to do what they felt was the right thing.

It just keeps on giving:
This email:
939154709.txt (from Oct 1999) gives, at the end, a data series for temps from 1402 – 1994
– plot it & you will see no hockey-stick, and 1994 temps lower than the 1940s…
– I can only assume that the perpetrators of such a travesty were sacked!

Your CRUtape title is excellent since it points to the true source of all this, the Prince of Darkness himself Lord (Voldemort) Mandelson. UK Govt. finances are in a terrible state so rapid funding cuts are needed all round but nobody (except LVM) has the guts to drown the polar bear cubs.

Somewhat OT, but I have also formulated a new theory which is mine and belongs to me.

It’s all just a terrible misunderstanding. UEA-CRU has spent the past ten years trying to replicate Mann’s hockey stick, but have repeatedly missed an important cultural datum which could (at least in part) explain the pseudo-statistical gymnastics seen in the leaked documents…

How about we take up a collection and donate it to the CRU, then we can sue them for fraud for misusing our funds! As noted donors they would have to be answerable to us in a court of law, and it would be mighty interesting to have them open their books (financial and otherwise).

as this appears to be a compilation done for a by the FOI office, the immediate question arises, why the FOI officers involved did not stop or sanction the deletion of files and the outspoken intent to breach FOI regulations.

This is definitly a leak.
Paul Hudson of the BBC who wrote the article “Whatever happened to Global Warming” confirms in his blog that he received the E-Mails on 12 October 2009 already and confirms its authenticity (this is probably an eatlier version, since the neweset date is 12 November 2009). He will comment later on these.
His blog can be found at: http://www.bbc.co.uk/blogs/paulhudson/atom.xml

Notice also that just prior to this package being released that Phil had finished what he thought was a successful schmoozing job with the University FOIA compliance folks to squash the FOIA request. I wouldn’t be surprised if the IT guy tasked with compiling this file didn’t release it out of spite.

It would make sense, since the fact that Phil Jones asked everyone to delete their emails and then failed to delete his own is a puzzler… but it isn’t so puzzling if some data security person had already been tasked with retrieving the data from an archive of whose existence Phil was unaware.

The name of the file is consistent with the theory. Citing Phil Jones
“If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone.”
it wouldn’t be strange that he changed his mind, knowing that a FOIA would eventually be sent to McIntyre, to stage this up, so it could be a hacker’s fault.
Anyway, any good forensic investigator should know this by now…
Ecotretas

I, too, object to the term of “Church of AGW” on the grounds that it connotes an organized, hierarchical, catholic institution. I think of dogmatically religious AGW proponents more as independent imams, sermonizing before the ovine faithful, enforcing literalism, issuing their fatwas against heretics and blasphemers, and enjoying the advantages of having compliant media mutaween willing to apply beatings.

I’m still trying to figure out why someone on the inside would gather the data into a ZIP file if it wasn’t to get it “out” in a convenient package. If I were Prof Jones and wanted to avoid having inconvenient information come out, I’d be using “rm -f” and not “zip”. If I *were* boxing things up to respond to a FOIA request, why would I pick the 60 most damaging megabytes I could find and tie them up with a bow like that?

If I were a IT type gathering up stuff for a FOIA request, there’d be a lot more “kitchen sink” items in the archive and you’d probably need DVDs to store it all.

I’m still thinking “knowledgeable insider”, someone with both sufficient access and knowledge of the AGW debate and its particulars. Knowledge without access would probably leave all sorts of audit trails which would be made public by CRU by now. Access without knowledge wouldn’t result in such a compact form of the end product.

My view is that it would take someone a lot of time and effort to put together this leaked data… years of data had to be recovered from archive, visually reviewed and selected for release based upon their content… this is not a five minute job… it is a far more calculated action….

The “powers that be” that have invested so heavily in getting this “data” originated will have wanted an insurance policy so that they do not get caught in the “blow back” should a) the operation becomes exposed or compromised… or b) they wish to change their minds and stop the operation.

Therefore, this looks very much like someone has decided to use their “insurance policy” that has been collected over the years… the AGW cabal have been effectively “thrown under a bus”…. we are now “picking over the bones” of this rather messy road kill… and they will probably be allowed to retire quietly provided they keep their mouths shut… I have got the impression that the cabal is deeply shocked by this leak… stunned silence… they thought they had protection… they thought they were immune… just read the emails…

This story played on Channel 4 news tonight (still on air, so no clip available). They interviewed someone from UEA who put up a not very plausible defence. The questioner (Krishnan Guru-Murphy) asked him about “hiding the decline” several times and all the interviewee (I forgot his name already!) could do was give a character reference!

Google Trends reports that Google searches for global warming emails have leapt to a hotness of “Volcanic”. And the Timeline graph for all Google News sources covering the topic shows increasing coverage, with this morning’s number of sources at 50% more than the “Jennifer Lopez falls” coverage and about 50% of the number covering U.S. health care.

Might the amount of interest be related between two stories, as the emails story involves hiding information, and the Lopez fall story involves her falling on live TV being edited out of the two-hour-later rerun?

[ Alvin (10:10:23) :I just watched a FoxNews interview between Chris Horner at the Competitive Enterprise Institute and Howard Gould of the Clean Economy network. See how has more invested in this “event”. ]

In this interview, did anyone else hear Chris Horner say “there’s 62mb released and 100 to come”. Did he mix his words up or does he have privileged information?

I was forwarded the chain of e-mails on the 12th October, which are comments from some of the worlds leading climate scientists written as a direct result of my article ‘whatever happened to global warming’. The e-mails released on the internet as a result of CRU being hacked into are identical to the ones I was forwarded and read at the time and so, as far as l can see, they are authentic.

(80.Has there been any explanation given for charge that there was a request for emails to be deleted to avoid an FOI request? All I’ve heard is that no emails were deleted, but the request itself is completely unethical and most likely illegal. Everything else I’ve seen seems to due to poor word choice and/or lack of context. The FOI avoidance would be a big blow to CRU, even if it doesn’t affect climate science.

There was no ‘hacking’ and I don’t care what Gavin Schmidt says. Being in the computer industry for 18 years, I can safely say that hackers won’t waste their time with anything that doesn’t have some sort of return value; it takes months and sometimes even years to hack a server. Assuming that a hacker did break in, would they really know what they had in their possession?

Either someone deliberately leaked them or someone made a massive network security mistake. I tend to think the former.

The first thing an incompetent person will do, when dealing with digital data that has been lost or leaked, is to blame hackers.

I usually have nothing but disdain for Faux News, but the pretty talking head in the middle there actually asked some pertinent questions. Well done! Glad to see this is starting to get out into the networks. The commentator who said this is not a revelation, but an affirmation hit it squarely on the head.

The matter was out on national TV in the UK (ch4)tonight and professor Watson was asked what “hide the decline”meant. He answered that he couldn’t believe that Phil Jones, whose reputation was beyond reproach, and whose approach is transparent, wouldn’t manipulate data in such a manner, although he didn’t answer the questions of the interviewer as to what “hide the decline” meant

Charles’ theory makes sense for one simple reason: if this were a criminal act wouldn’t there be some evidence of police activity? Has WUWT, tAV or anyone associated with these sites been approached by law enforcement? Has there been a single email asking what did you know and when did you know it? Perhaps folks have been contacted and have been keeping it quiet, but it would be interesting to know just how seriously this is being pursued by the police.

On the other hand, if no crime were committed, I’m sure the folks at CRU are smart enough not to claim one had been commited to law enforcement. Filing a false police report is a serious crime in the US, I’m sure it is in the UK also.

In that same line of thinking, are police reports public record in the UK? Does anyone know exactly what, if anything, CRU reported to they police?

There has been a lot of talk about ‘erased’ files and how these may have been ‘lost’ forever.

It is possible to recover files that have been erased, even on disks reformatted after the deletion.

There are always stray magnetic ghosts remaining on the disks and there are forensic techniques available to enhance these ghosts.

It is also true there are techniques available to really scramble over the ghosts of files past, greatly complicating the forensic recovery of those files. However, it looks as if the CRU folks lack the technical skills or knowledge to even be aware of these techniques.

If one really wanted to do a thorough examination, a third party (perhaps law enforcement?) needs to physically secure the disk drives of CRU.

The oops theory makes sense but I can’t get past such a file being prepared with such devastating data. Is it possible that the person tasked with compiling the data in case the FOI was granted, eventually realized what they were finding and became concerned the FOI would be denied and this data “lost” and then decided to “oops” the file out into the wild? Perhaps they delegated that task to someone who was not predictable enough.

I am not in the least a lawyer (thankfully), but I have observed numerous criminal trials in my lifetime.

So if there is a “mob” operation, and a person from another “mob”, (Operation A and Person B, let’s call them) steals a whole lot of various materials and goodies from Operation A, including various documents and objects which would prove that Operation A was involved in criminal operations, and Person B gets nabbed by law enforcement, is what Person B stole acceptable in a court of law as evidence that Operation A was involved in criminal operations? Can what Person B stole be used as evidence against Operation A? Even if Person B never got convicted of any crime? (Might even call Person B a “confidential informer”, which is often used by law enforcement).

Of course the materials stolen by Person B could be used by law enforcement and the courts to convict Operation A, and the “players” of Operation A of any crimes that could be proved by the evidence stolen by Person B.

High crimes and felonies have been committed by these yahoos who have been masquerading as “scientists”, and it is long since past the time when they should have to answer for their criminal activities. Let the search warrants be issued, as there is probable cause, let the evidence be gathered by law enforcement officials not in the pay of those who would profit or have profited as a result these crimes, and let the trials begin.

I have to wonder, about this time, which of the “players” in this game are going to selected to be thrown on their swords, to save whoever is left who can be saved from disgrace and ignominy.

Were I one of the researchers who went along with this huge and costly to all fiasco for the purposes of getting government grants, I would really be worried about now.

Not sure where the rumour of another 100mb of information has come from? I think there may be a little confusion. 65mb was released in a zipped up format, unzipped, this came to 160mb of data. Are people sure this isn’t where the missing 100mb is?

De Rode Willem says:
November 23, 2009 at 10:34 am
If these critics must build a case on theft and robbary…then they are no dime better than those who try to lie about some 10th of a centigrade more or less.
—————————-

I would class this as theft as the files/info are in the public interest. Also, who says the files were stolen? Only those who have something to hide at this stage. In time, maybe we will know either way…

CRUTape, Screwtape is the allusion I presume.
For those not familiar with the writings of C S Lewis, Screwtape was a senior devil writing letters to a junior devil, and the tone of these wasn’t that much different from the CRU letters. A brilliant allusion.

Even more apt is N.I.C.E from Lewis’s That Hideous Strength. The National Institute of Co-ordinated Experiments was a devilish Scientific body in a small English University town which was able to overrule the laws of England, and whose purpose was to introduce a totalitarianism beyond imagination. A hero in the story (well eventually a hero) was a social scientist corrupted by the attraction of the “inner circle” where things “really happened”, and who assisted in manipulating the populace by writing media articles crafted to deceive.

I was reminded of a speech from the 1956 film “Forbidden Planet.” Referring to the remnants of an advanced civilization he’s found on Altair IV, Dr. Morbius says,

“Such portions then of the Krell science as I may from time to time deem suitable and safe I shall dispense to Earth. Other portions I shall withhold. And in this I shall be answerable exclusively to my own conscience and judgment.”

As it turned out, a little “peer review” might have saved his life and many others. :-)

Has anyone discussed the phrase “Random Selection”? Doesn’t this imply a larger section of Data may have been liberated and this 61 megs was just a sample?

I’ve been thinking the same thing, and why would the source call it “random” if there weren’t more of the exact same kind of stuff in its possession – or did the source think it just got lucky? If there is more to come, where is it such that its release can’t be stopped? We’ll just have to wait and see.

And I’m still a little flummoxed by the idea that Jones advises people to delete emails in the face of an FOI request, then doesn’t delete this very email itself. Does he think he’s invisible, but the others not? Well, by now I wouldn’t put that past any of these effete, “I speak it, therefore it is true”, Climate Scientists.

Someone who received the e-mails on the subject “BBC U-turn on climate” forwarded them to Paul Hudson. These particular e-mails pertained directly to him. There is no way he received the contents of the leaked folder on October 12 as there is material dated later. Even the discussion of his BBC story continues until Oct. 14. Mr. Hudson did not word what he posted very carefully. I think he simply means that he can confirm the content of a few e-mails contained in the archive.

This theory reminds me of panicked teenagers grabbing up all their drug/booze/sex stuff and throwing it out the window before their parents walk in. Only it wasn’t their parents they heard driving up, it was the police, walking around the house, doing an innocent neighborhood security check. Giggle.

So the FOI2009 files were extracted to be HIDDEN from FOI requests, and later retrieved from their safe hiding place. Sounds plausible. No “hacker” would know what to drag out unless it was precompiled for hiding.

What a terrible nightmare for the participants, if true. I actually said a prayer for those guys, in spite of what they are trying to do to every person in the world.

I doubt they ever dreamed all the power would end up in their hands and that they wold be called to such high accountability, eventually. Otherwise they would have had much tidier work and careful language from the start.

Now tell me, does it make an sense to have access to Briffa’s cushion face, but not to the disputed Yamal data?
If this is an Emergency Server running, I (personally) would have reduced the informations to the lowest limits.

But why can I see Briffa’s page, but not the Yamal data?
Let me guess:
The Yamal data is actually there, but stealthed with another file name.
But this would mean, this is no emergency server!

The link to the Yamal directory might be obviously a kind of tripwire.

“(80.Has there been any explanation given for charge that there was a request for emails to be deleted to avoid an FOI request? All I’ve heard is that no emails were deleted, but the request itself is completely unethical and most likely illegal. Everything else I’ve seen seems to due to poor word choice and/or lack of context. The FOI avoidance would be a big blow to CRU, even if it doesn’t affect climate science.

[Response: In my opinion that email was very ill-advised. – gavin])”

Looks like Gavin is running for cover. Could be he’s starting to have concerns about some of the things he’s reading. I spent the weekend reading the emails that Gavin was a part of, and there really wasn’t much there except for working on RC during NASA business hours. If more disclosures are on the way, now is the time for Gavin to break ranks if he want to survive this.

I think the idea that this was an accidental compilation is more likely. A lot of this stuff is not what would be prepared for a FOIA request if someone was looking to keep their job. But we do know that there was an email suggesting that people delete some of their emails that referred to particular topics. What I am thinking is that someone deleted their email, but did not empty the recycle bin. The files were out of sight, but not out of mind, so to speak.

It goes along with the incompetency thought. I’ve known many people who thought once they deleted the email from their inbox it was gone forever. You can guess their chagrin when I pulled emails back from the trash bin. If CRU uses a mail server program like Lotus Notes or some thing similar, it is like a three step process to permanently delete an email or group of emails from the database system it uses. Someone with basic access, such as a student at UEA, might have sufficient access to check the deleted mail folders from the database and pull out the files to make the zip file.

Except for the conspiracy aspects of the contents of FOIA2009.zip, exactly why was all this stuff secret in the first place ?

Likewise, if the part of the theory that says this was put together as part of the review to Steve McIntyre’s FOIA request, exactly what contained with FOIA2009.zip, aside from the embarrassing conspiracy perpetrated by CRU employees, were the CRU officials trying to protect by denying the request.

Finally, the legal department of CRU found nothing strange reading this material that obviously reveals many unethical, if not illegal, acts by CRU employees ?

Frankly, the best course for the CRU and the University of East Anglia is to announce that a ongoing probe had been started on Nov. 12, 2009 into the actions of various employees of the CRU, from material that came to light because of a FOIA request.

Probably not. Too easy for the defense to say the documents had been tampered with. BUT,I would think the originals could be subpoenaed and brought into evidence (in my humble non-legal opinion)

PR Guy (09:52:58) :

When I first saw this file, it looked to me to be a lot like the kind of file that gets put together for a lawyer. Typical an organization would get a discovery notice or a FOIA request and a lawyer will send out an email requesting all data, information and correspondance related to the issue at hand. A clerical person assembles all the responses into a file and the lawyer goes through it and issues an opinion. So labeling this kind of file with ‘FOI’ in the filename would make sense.

I like this idea. Some of the e-mails would probably never have been included in an FOI request. After all, the request for information was for data and code, not e-mails. I think some third party clerk at CRU was pulling data together for the FOI in case it was approved..and found more than he/she was looking for and included it. This insider either leaked the file, or like theory number 3…it was left intentionally or unintentionally on an unprotected server.

You are correct. The police would be investigating this “breach” and it would be all over the news. as of yet, I haven’t read, heard or seen anything on any of the network and cable news of any investigation.

This is nothing more than a ruse to shift blame and attention to a non existent entity. I would be watching for anyone being sacked from Hadley in the next few days or weeks.

It’s way past time for someone to come up with a legal request/response based on information in the file at this point.

Some government entity should be opening an investigation…oh …wait…the government NEEDS that boogey man! Almost forgot.

Seriously…this needs to be officially investigated by someone with the ability to subpoena those involved to get the rest of the information they’ve been hiding and threatening to delete.
As has been pointed out countless times, there are backup tapes of all this stuff out there somewhere, and someone needs to file suit to get access to them.

Also, even without any malfeasance…there is open admission that in fact they have no idea what’s in the databases. The data has been “adjusted” without being documented, and noone knows where/where/why/how, so based on that alone, everything they’re doing with models and series and tree rings is all a giant waste of time, as far as I can tell.

If the “pure” data no longer exists…sorry guys and gals…time to start over, and try and do a better job this time?

But I will in the meantime answer the question regarding the chain of e-mails which you have been commenting about on my blog, which can be seen here, and whether they are genuine or part of an elaborate hoax.

I was forwarded the chain of e-mails on the 12th October, which are comments from some of the worlds leading climate scientists written as a direct result of my article ‘whatever happened to global warming’. The e-mails released on the internet as a result of CRU being hacked into are identical to the ones I was forwarded and read at the time and so, as far as l can see, they are authentic.

The other theory that I’m starting to like more and more is that Jones directed some lower level IT guys to delete files from the server and backup tapes. We have an email where Jones is directing people to delete emails from their email accounts. He’s probably smart enough to know that there would be files on the servers and tapes as well. So he tries to get a low level IT guy to do the deed. But he picks the wrong guy, he picks a guy who has some integrity. Since he’s a low level guy, he feels trapped — he doesn’t know where to go or who to complain to and he worries (a lot) about retribution (gee, I don’t know why, they seem like such nice guys). So he leaks it.

The “leak” was a quite deliberate camouflage job, to hide the equally deliberate destruction of seriously indictable material that was perceived to be vulnerable to FOI exposure

The released material is embarassing, but falls far short of evidence that could support criminal prosecutions. The story is being suppressed in the main stream media, being presented as a spat between science camps.

For this to work, it is vital that the hacker is never found, or the hacking methodology.

If this is right, we will see claims that the hacking caused extensive deletion of files.

OT , but since I’m here , I’ll post it here . Obama was just on the tube – he’s still talking about green jobs and the climate “crisis” . It will take a lot to dissuade the current crowd in DC , I’m afraid . Crimony , if nothing else this should prove that the science is not now , nor has it ever been “settled” . Fools , the lot of them .

It appears that the leak was internal. On November 17th the zip file was sent to multiple recipients. Notice how the opening is worded, and how the selected emails are summarized to get the recipients interested:

We feel that climate science is, in the current situation, too important to be kept under wraps.

We hereby release a random selection of correspondence, code, and documents.

Hopefully it will give some insight into the science and the people behind it.

How is it that the emails are presented as text (,txt) files?
Whilst not an expert on email servers, the ones I have used (and the clients such as Microsoft Outlook) do not store in this format. All emails are stored in one large file (eg, ,pst), not individual text files. So, how did the released files get into this format? Could someone with full access to other’s email accounts (an insider?) have been looking for incriminating marerial and copied/pasted the offending emails into text files?

Also, the recipients and the original senders of the emails should have copies on their own mail servers/email clients/etc so it is useless to gather these together on any one server for deletion to escape detection.
Any investigation should look into whether there has been deletion of the same files from the many disparate email servers/accounts involved. If so, would this could prove collusion to ‘hide’ the actions of the people concerned?

Why is it we are always expected to extend kindness to those who never do so to us? To them, we are “climate criminals,” even though it is they who manufacture and manipulate data to falsify the record. Yet we are to believe they just made a stupid mistake resulting in “out of context” material to be “misinterpreted?”

No. I’m going with conspiracy on this one, just because the evidence is so overwhelming. Honest people don’t write the things they worte, even “in context.”

This is turning into a hyperpartisan tabloid feeding frenzy – and this will be many peoples’ first impression of WUWT.

What’s happening with El Nino? Record November snowfall in Whistler, BC where the Winter Olympics will be held in a few months. Floods in other parts of coastal BC. Heavy rains in northeast England. Fires in Australia…

Odd leetle story from one of my own clients (names omitted to protect the terminally innocent).

Requirement: to scan in source documents to an accounting system, using an early-generation mopier/scanner with a document feeder, to generate the images.

Unfortunately: said mopier cannot see anything else on the network except itself. So cannot do the preferred option, which is to write scans to a network resource.

Fortunately: the mopier can be set up as an FTP server, so it can write to itself, and be seen by the rest of the network.

Unfortunately: so can the rest of the world.

Fortunately: my scans happened just fine. I’m outta there and in the clear.

Unfortunately: over the following weekend, some enterprising types downloaded a mirror of their entire East European pron site onto the mopier, as revealed by the next Monday’s logs, which had their Internet pipe red-lined the whole time.

Fortunately, they realised what was up and shut down the FTP server on the mopier.

Unfortunately, they then could not easily delete the files downloaded, as the downloads included some tricky tricks to stop such maintenance.

Fortunately, the boys could read the files.

Unfortunately, there was a way round the hack, and the files were deleted.

“I have been a member of the SRI International Computer Science Laboratory since September 1971. I spent eight years at Harvard (1950-58, with my A.B. in Math in 1954, S.M. in Applied Math in 1955, and PhD in 1961 after returning from my two-year Fulbright in Germany (1958-60), where I also received the German Dr rerum naturarum in 1960. “

Re: My previous post
– that email/data from Tim Osbourn (939154709) is the one to which Phil Jones boasts about adding the real temp-data to (942777075), to create the nice hockey-stick effect….
– good work boys!

A ) On November 20, why did Phil Jones state “It was a hacker. We were aware of this about three or four days ago that someone had hacked into our system and taken and copied loads of data files and emails”?

B) If they knew the system was hacked 3 or 4 days before November 20, why did they wait until after FOI2009.zip hit the blogosphere before the system access/password lockdown?

Phil must have at least one other person to corroborate A, since he states “we were aware”. Item B could be explained by an upper management reaction to the press – they tend to freak out more about matters they don’t really understand, like network security. The IT department may have considered the leak plugged, but upper management has to be able to put some CYA language into their reports.

Still, very fishy. I do agree that it is unlikely this network was genuinely “hacked”. I think it’s more likely that someone either already had or was able to obtain access information. Of course you would use a proxy server for the access and download, but anyone can google how to do that. That may appear like a “hack”, but if you’re just typing in user names and passwords that you’re reading off of a piece of paper, not really.

Obama is going to announce a US carbon reduction plan in Denmark, ignoring all this new evidence that has come to light. Yet, he says we must investigate all options and come up with a plan first before making a rash decision in afganistan. How long has he been sitting on his hands making a decision on that one? two months+ all the while things get worse and more people die. But we must rush to action on climate change….

“Susan Watts looks into the University of East Anglia row. Thousands of emails and documents stolen and posted online suggest to some that researchers colluded to make the case for climate change. Can we trust the scientists?”

I know it’s a bit of a side issue but I have real problems with this business of a mid October leaking of the data (presumably shorter data and pre- Steve McIntyre’s last communication) to Hudson at the BBC. Why would anyone with any insight into these matters post to anyone at the BBC? It just does not seem credible. Most English people would not trust the BBC to tell you the time of day so why go there?

If we are to unravel just what has happened these past few day (? weeks), that will somehow have to be included. It just jars with the other quite logical line of argument developed here.

Viscount Monckton on Climategate: ‘They Are Criminals’
The man who challenged Al Gore to a debate is furious about the content of the leaked CRU emails — and says why you should be, too.
November 23, 2009 – by Christopher Monckton

Gavin’s elaborate description of the hacking attempt at RC is, in my humble opinion, nothing more than an attempt to add meat to the hacking theory in order to increase the vilification of the theoretical hackers.

Well, Dr Schmidt has provided some more [presuably in your world, fabricated] details, and also offered to share IP addresses with Steve M and Jeff Id.

At around 6.20am (EST) Nov 17th, somebody hacked into the RC server from an IP address associated with a computer somewhere in Turkey, disabled access from the legitimate users, and uploaded a file FOIA.zip to our server. They then created a draft post that would have been posted announcing the data to the world that was identical in content of the comment posted on The Air Vent later that day. They were intercepted before this could be posted on the blog

Charles’ theory makes sense for one simple reason: if this were a criminal act wouldn’t there be some evidence of police activity? Has WUWT, tAV or anyone associated with these sites been approached by law enforcement?

According to regional news (BBC East) yesterday, the Police have been called in. I don’t expect Inspector Knacker will find anything though but he’ll probably smash down a few front doors for dramatic effect.

From the Wall Street article posted earlier, this is exactly the kind of thing I was expecting. Other scientists who are actually believers in AGW shocked at the fact they didn’t imagine other scientists would be treated by a “Mafia” – These are the kind of level headed scientists in the Mainstream (of which I believe there are many) that will be crucial to the fall of the alarmists.

“Mojib Latif, a climate researcher at Germany’s Leibniz Institute of Marine Sciences, said he found it hard to believe that climate scientists were trying to squelch dissent. Mr. Latif, who believes in man-made global warming but who has co-authored a paper ascribing current cooling to temporary natural trends, said, “I simply can’t believe that there is a kind of mafia that is trying to inhibit critical papers from being published.””

Over 50% of people questioned don’t believe climate change will affect them
Only 1 in 5 (18%) think that climate change will take effect during their children’s lifetime.
74% of people would take immediate action to change their lifestyle now if they knew that climate change would affect their children’s lives
52% of people believe that their actions as an individual can help stop the effects of climate change
Only a quarter (26%) believe that climate change is already impacting on the UK
Respondents cited flooding as one of the most common effects of climate change that will happen in the UK (69%)

Two conclusions spring to mind. First, the existing efforts to brainwash the British public have failed. Apparently they believe their senses not the HADCRU guys and the AGW fanatics. Second, they do not have a clue as to why people do not believe them.

To solve this “problem” they are going to put their message on 900 bilboards across the country. A Jon Lovitz moment methinks: “Yeah! That’s the ticket!”

I think the intention in the BBC is to keep the discussion as discrete as possible and it speaks volumes. They have made comment for the first time about the CRU files contents hidden well away from climate discussions on a very lame blog called open secretshttp://www.bbc.co.uk/blogs/opensecrets/2009/11/hacked_climate_emails_and_foi.html#comments
Look forward to Newsnight though, I suspect it will be hard not to comment on Nigel Lawsons articlehttp://www.timesonline.co.uk/tol/comment/columnists/guest_contributors/article6927598.ece

This gets even more strange.
At RC Gavin wrote ( I dont know it is ok to quote his entire post but in these days ….)
“There seems to be some doubt about the timeline of events that led to the emails hack. For clarification and to save me going through this again, this is a summary of my knowledge of the topic. At around 6.20am (EST) Nov 17th, somebody hacked into the RC server from an IP address associated with a computer somewhere in Turkey, disabled access from the legitimate users, and uploaded a file FOIA.zip to our server. They then created a draft post that would have been posted announcing the data to the world that was identical in content of the comment posted on The Air Vent later that day. They were intercepted before this could be posted on the blog. This archive appears to be identical to the one posted on the Russian server except for the name change. Curiously, and unnoticed by anyone else so far, the first comment posted on this subject was not at the Air Vent, but actually at ClimateAudit (comment 49 on a thread related to stripbark trees, dated Nov 17 5.24am (Central Time I think)). The username of the commenter was linked to the FOIA.zip file at realclimate.org. Four downloads occurred from that link while the file was still there (it no longer is).

Thank you. I’m just one of the small-town observers, yet a denier, of this whole evil hoax. Now I can provide my antagonists with a reasoned explanation for why the science is NOT “settled”. The Canadian Broadcasting Corporation will interview Al Gore on Thursday morning, 26th Nov. I wonder if they have the “guts” to challenge him on this whole issue.

Instead of looking at the billions and billions of $$$$ “wasted”…how about this headline:

“World sleeps better…After massive investment of billions of $$$, researchers unable to find any warming, so no more worrying about it.
Today, researchers at CRU pronounced that any signs of AGW were
impossible to find after years of research, and that the ONLY way they
were able to show ANY man-made warming at all was to manually
alter data. Without the alterations, models continued to agree with
real temperature measurements, and indicate no man-made warming
at all.”

Then we might say “Well…it was a crapload of money, but at least we can sleep well tonight instead of worrying about that 20meter wall of water heading straight towards NY city.”

This may be OT, but wouldn’t it make a lot of sense for some of the leading analysts and scientists who understand the temperature record and the software coding that generates the usable output from the raw temperature data to get together and develop an organized approach to dissecting the released CRU data files and code? I’m thinking Lindzen, McIntyre, Spencer, EM Smith, Pielke, et. al. ought to be coordinating their efforts to determine if it is possible to see how much the HADCRUT (and by extension, the GISSTEMP) data-in-use is untrustworthy for the purposes of examining temperature trends of the magnitude involved.

What do others think? Does anyone have any idea what’s going on among these folks behind the scenes at this point? Or, is everyone waiting for a formal investigation of which to be a part?

I take great umbrage at the usage of the term stolen data being currently bandied around by various embarassed parties at CRU. I happen to be one of the owners of this and any other data stemming from any research that is funded by my taxes.

Furthermore, I take as dim a view of this AGW data shenanigans as I do of the MP’s expenses scam as it all falls into the same category of defrauding the taxpayer.

If it is in sincerity he’ll start that closer investigation now and hold judgement on the “science is settled” that he promotes in the article above his comment. I also think he has a few more apologies to issue.

Leak Source:
I think it was probably an inside-job in someway
– probably a disgruntled employee, or ex-employee, who couldn’t take the lies and the Spin, and the dodgey IT practices any more
– and the FOIA.zip file name is an ironic joke at the attempts of the CRU Crew to avoid the FOI requests!

My sneaking suspicion is that it was really Dr Briffa, who has held a secret grudge against Mann & Jones for the way they have hogged all the limelight, and glory, whilst insisting he has to add a hockey-stick to every damned climatic-series he produces!

can anyone shed light on why the only things redacted from the emails is the 2nd half of all the email addresses? Why would a hacker bother to do this? Is there some FOIA requirement that the second half of email addresses be redacted?

Yes, nice idea – get a climate dream team together from both sides and blogland. I reckon shut them all in a big room a la “Apollo 13″ movie and “failure is not an option” – within a week we’d have a pretty well thrashed out view of the real position of climate science. Not sure who should be Chair though… :)

Richard Black is still muzzled. The BBC thought police are active and no posts have been allowed on his thread since 48 hours – even ones that do NOT quote anything said in the emails (as mine does not and it is still awaiting moderation)

The real world is stranger than fiction!

Who would have thought that the UK’s BBC would be worse than any tightly controlled tinpot dictatorship.

How an organization as large as the BBC could remain so tightly under a tyrannical control is AMAZING – surely there are some BBC employees who KNOW THAT THE BBC’S BEHAVIOUR IS UNETHICAL?

BBC EMPOYEES WAKE UP – EXERCISE YOUR RIGHTS OF FREE SPEECH – YOU HAVE THE POWER TO DISOBEY YOUR MANAGEMENT DIRECTIVES – YOU CAN PUBLISH THE TRUTH.

“Gavin’s elaborate description of the hacking attempt at RC is, in my humble opinion, nothing more than an attempt to add meat to the hacking theory in order to increase the vilification of the theoretical hackers.”

“Well, Dr Schmidt has provided some more [presuably in your world, fabricated] details, and also offered to share IP addresses with Steve M and Jeff Id.”

Sorry – as they say, too little, too late…

Does GS do anything at GISS besides blog??! We taxpayers deserve something for the money we pump into that organization…

You are under a great amount of pressure from outside and inside your organization to respond to a FOIA request.
You know that there are damning emails and data files on servers and backup systems that you probably cannot delete.
You know that your “PET” FOIA officer is going to find some way to refuse the latest request.
You suspect this will not stop the requestor.
You MUST find a way to divert everyone from the real issues and the very important files.
You assemble the LEAST damning emails and files, while making sure that some are inflamatory.

Now, how to let these get out to the enemy without it being known as an intentional leak?

You “HACK” you own system (fake it), find a way to post the info anonomously and make sure that the right people get it.

Now, you have :
Made yourself the victim.
made anyone who looks at or comments on the files guilty of “something”.
Freed your side to criticize all as crooks who make any good points regarding the info in these files.
made almost everyone assume that the MOST damning stuff is already “out there”
pretty much made everyone assume that future FOIAS are going to be fruitless.
pretty much drawn a blueprint for anyone still pushing FOIA to limit their requests and expectations to the known files.
Given yourself more time to get rid of the really damning stuff.

I believe that there is much more to be had and certainly this explains the “randomly selected” nature of the files.
I like this version as I do not believe the “incompetent” theory. I am much more inclined to believe that if something is too good to be true, then it is not true.

I see a few holes in this theory but it is at least as good as the others I have heard.

In my previous post I suggested a culprit is a disgruntled IT employee. I’ve looked at some of the released codes, it is obvious that all sections are carefully selected. I do occasionally a bit of simple Fortran programming and have a great deal of trouble later on to find a particular section or decipher my own coding. This is done by someone well versed in programs content, including the offending sections, which an average IT technician may not be able to do efficiently. This was done by a scientist fed up by continuous interfering from the top with requests for data massaging, even someone who is among quoted emails (in order to deflect suspicion), someone either with a friendly Russian connection or with a web space earlier setup for the purpose. Since BBC received files on 12th and did not respond for a week, he/she activated plan B.

I think your analysis is correct – even studying the computer code and the hacks into those over time advertises incompetence – bureaucratic incompetence at its British best. No conspiracy, just people on the public payroll who don’t give too hoots about anything except appearing at work to justify their wages.

Most here would not realise that the UK is basically a socialist state in which the idea of individual responsibility has been purged. AGW seems, in this light, another instance of Lysenkoism, especially when Trenberth complains that there is something wrong with the data. Have these people ever considered the possibility that the theory might be wrong? But as they are trained in accepting the consensus, obviously the theory cannot be wrong, we all agree it has to be right.

This is pseudoscience, and this is the real problem which needs to be sorted out. And it is not surprising that the belief in AGW by the social democrats/liberals is so fervently held, the believe it to be true, but because of their cultural and philosophical conditioning in consensus, are intellectually unable to see the problem. This is the more frightening aspect of this climate change issue.

You might be wondering why an antisesquipedalian grammar n*zi like moi would not have leapt into the middle of the “alternate/alternative” controversy by now. Good question. Perhaps Dr Morbius ( Gary Hladik (12:26:47) : ) might help with the answer. He was the philologist on the Bellerophon expedition.
.

Another question and more germane to the thread is whether or not the underlying raw data sets have been altered by CRU, ala USHCN v2.

Here stateside, the USHCN v2 data set for Illinois has been heavily adjusted to show unambiguous warming. 27 of 34 station sets have their curves altered toward greater upward slope. Of the 7 others, 4 are ambiguous and only 3 are adjusted to reduce the slope.

Some of the stations have had their early reading lowered by 1 to as much as 2½ whole degrees. If the rest of the US states have as much adjustment, GISS won’t need to homogenize the station data to show “unprecedented warming.”

Theory number one is the preferred explanation of the defenders of CRU. This allows them to portray CRU as victims of illegal acts. It allows them to scream bloody murder and call for an investigation of the crime. How can we take the fruits of hideous crime seriously? The end does not justify the means!

One of our favorite writers, Gavin Schmidt, has expanded on this theme with the report:

He [Gavin] said the breach at the University of East Anglia was discovered after hackers who had gained access to the correspondence sought Tuesday to hack into a different server supporting realclimate.org, a blog unrelated to NASA that he runs with several other scientists pressing the case that global warming is true.

The intruders sought to create a mock blog post there and to upload the full batch of files from Britain. That effort was thwarted, Dr. Schmidt said, and scientists immediately notified colleagues at the University of East Anglia’s Climatic Research Unit.

I believe the above statement by Gavin to be a big bunch of hooey. I believe the “hack” was a posting of the same blog comment which was posted at The Air Vent

I take a different view since on wednesday morning RealClimate was down for some time and an unusual message was up from ‘webfaction.com’ saying ‘Site Not Configured’. Clearly something unusual was going on, consistent with Gavin’s account.

O yea, you i love this criminals hanging! Sorry, but the way they wanted to rule the world is realy one of the worst capitles sawn in history, almost like the nazis, again sorry, but i do not find any better explenation.

can anyone shed light on why the only things redacted from the emails is the 2nd half of all the email addresses? Why would a hacker bother to do this? Is there some FOIA requirement that the second half of email addresses be redacted?

This is being done by those who have enabled searching of the emails, or who are otherwise publicizing them. The full email addresses are in the original file that was “leaked.”

Mark (14:07:21) : “can anyone shed light on why the only things redacted from the emails is the 2nd half of all the email addresses? Why would a hacker bother to do this? Is there some FOIA requirement that the second half of email addresses be redacted?”

The full e-mail addresses are in the FOIA2009.zip file. Any redaction that you are seeing was done by the blogger or the person running the searchable server.

You “HACK” you own system (fake it), find a way to post the info anonomously and make sure that the right people get it.

Please…. this is starting to look a little silly. The idea that they’d “hack” their own system is, frankly, absurd! Let’s retain a little perspective, otherwise we just make ourselves look like a bunch of neurotic conspiracy theorists.

I’m way behind on the reading of this thread, so forgive me if this has already come up.

IMHO, the users deleted these items after they were were told of the pending FOI request. Whoever gathered them had the access and knowledge to gather up the items that had been “deleted” but before the database had been packed (or whatever term is used on a system such as theirs). That would have been a very efficient way to pick up incriminating materials with minimal effort.

I believe it is an inside job…by someone with the remnants of a conscience.

Fair enough he makes the point it isnt the nail in the coffin of AGW theory , but then he says

“There appears to be evidence here of attempts to prevent scientific data from being released, and even to destroy material that was subject to a freedom of information request.”

Also:

” Phil Jones, should now resign. ”

And in one reply to a poster called Sabraguy who asks:

” But now I suggest you review your file of correspondence and articles, and figure out who you need to apologize to.”

He replies:

“I apologise. I was too trusting of some of those who provided the evidence I championed. I would have been a better journalist if I had investigated their claims more closely.”

All in apparent sincerity, I am quite gobsmacked.”

I have been following agw as a sceptic for several years now and I can honestly say that George Monbiot saying this is without doubt the most stunning and amazing thing I have heard. It truly is a shattering blow to the warmists. It is extremely significant.
David

Unless they don’t keep any access records at all, they should be able to least determine the IP address of who downloaded the files.

OTOH, the UK has tough Defamation laws and I suspect that some of the people mentioned in the emails could sue even though there was no intent to publish the emails. Being called an idiot is a matter of opinion, but the journal editor that was sacked might be a different matter.

Just saw on the CA Mirror site Steve M’s posting about the November 13th denial of his FOI appeal, that he filed July 24th. That timing makes compelling some of our speculation that this file was put together by CRU crewmembers for review by Jones and the FOI officials. Occam’s Razor tells me that someone, probably on the FOI office, was unhappy with the FOI decision to stonewall Steve M. and not let this information see the light of day, despite the obvious right the public had to see the info. This was no outside hacker, this was someone who was involved to some degree in the FOI disclosure debate and took matters into their own hands. This is authentic documentation; the fact that all these code comments, data manipulation and these emails happened is a scandal, and the bigger scandal — as always– is the FOI cover-up. Unforgiveable.

Sabraguy:
But now I suggest you review your file of correspondence and articles, and figure out who you need to apologize to.
Monbiot
I apologise. I was too trusting of some of those who provided the evidence I championed. I would have been a better journalist if I had investigated their claims more closely.

In another comment
contrarian2:
But if the science is that “settled,” why refuse to disclose the data? If global warming so obvious and incontrovertible, why be in such a panic about FOI, why talk openly about re-defining “peer review”, why threaten to (or actually) delete data?
I agree. It is exactly for those reasons that Phil Jones should resign. There’s a word for his lack of openness and control of the data: unscientific.
___________________________
Something is changing don’t you think?

Newsnight, the primetime political program on BBC at 10:30pm, had just led on this issue. As you’d expect it didn’t really get into the meat of the issue but the “trick” and “hiding” lines were brought up and I’m sure plenty of previous unaware Britons are now Googling to find out what has been going on.

They had one guy from the UEA who seemed convinced that this was an “illegal hacking” incident, something Prof Singer didn’t respond to.

…. This was no outside hacker, this was someone who was involved to some degree in the FOI disclosure debate and took matters into their own hands. This is authentic documentation; the fact that all these code comments, data manipulation and these emails happened is a scandal, and the bigger scandal — as always– is the FOI cover-up. Unforgiveable.

Having got over my gobsmackedness a bit, I am beginning to wonder about the ramifications here. Monbiot has jumped before Andy Revkin and many other well know enviro jounalist names in the world. It could all be a rather self preserving position hes taking rather than genuine enlightenment. But hes going to get flak from all sides, and if Phil Jones does go because of the heightened pressure, then I think there would have to be some knock on consequences for the RealClimate crowd. Quite interesting times.

So they assemble not only loads of data, but also 1k+ emails just in case.. Did Steve McIntyre actually ask to see their emails as well. Or did they prepare them for him just beacuse they’re such a loving bunch? Doesn’t sound very likely to me. Stupid I buy. Stupid and over-zealous, I’m not convinced.
But then, the other possiblilites don’t seem that natural either. There’s no single person who sent or was a receiver on all mails in the file. Briffa seems to be the most popular guy, but he’s only in on around half of them, and he’s doesn’t come out as your everyday whistleblower:)
Could be it-department guy with loads of time to pick emails. But then it’s not like all of them are that interesting. Maybe they were choosen by a combination of sender/receiver + maybe some keyword(s)? That would be very easy, but you’d still come up with loads of crap unless these guys only ever talk about work.

50 years ago, I used to take hourly temps with a handheld sling psychrometer; insert a clean dry sock, verify that both mercury capsules agreed, wet the sock and spin until the wet bulb drops no more; a bit too adventurous and one might crack the tube by accidental impact; grab the backup sling and try again.

Whole degrees only.

Today a blind, dumb machine does this and more — it even amazingly calibrates itself.

“I have been following agw as a sceptic for several years now and I can honestly say that George Monbiot saying this is without doubt the most stunning and amazing thing I have heard. It truly is a shattering blow to the warmists. It is extremely significant.”

I agree totally. Monbiot has been one of the most vociferous of the warmers and this an amazing turnaround. If he can see the light, just about anyone can. Well done George.

Meanwhile up here in Canuckistan, the HQ of all our major media – CBC, CTV, Global the Globe & Mail, the Toronto Star . . . . all of them are within spitting distance of where Steve lives and there isn’t a peep, a whisper (Ok one skeptic Journalist – Lorne Gunter) of what’s going on.

Silence of the co-conspiratorial mass media. They do so love hysteria and panic.

UKIP (14:50:05) :
Newsnight, the primetime political program on BBC at 10:30pm, had just led on this issue. As you’d expect it didn’t really get into the meat of the issue but the “trick” and “hiding” lines were brought up and I’m sure plenty of previous unaware Britons are now Googling to find out what has been going on.

They had one guy from the UEA who seemed convinced that this was an “illegal hacking” incident, something Prof Singer didn’t respond to.

The UK Scientist was Bob Watson from UEA. The other person was American, didn’t catch his name.

Bob Watson wanted to talk about the emails and how they were being misinterpreted, the American chap wanted to bring up the fact that the scientific process was being corrupted and that many people had been denied the data to check the results.

Both called for a review (of the hacking and the consequences of the hacking).

Seventy years ago we declared war against Hitler. That involved subterfuge and killing people. But it was done with the agreed understanding of being the best option to prevent a greater evil.

I think that the action taken in releasing FOIA2009.zip is a far smaller example w.r.t harm and a somewhat smaller example w.r.t. damage prevention, but IMO it is in the same category of actions.

I find this fourth possibility interesting (taking both the following posts together):-

vukcevic (11:18:03) : …someone in IT was given list of files and told to erase them after the ‘lost data affair’. Any IT employee would know that is an illegal request, so for his/her own protection ‘zipped’ whole lot on 12th November.

Treachery Watch (11:20:34) : …could [CRU] realize that attacking an angry insider would not be a wise course of action because said person perhaps has evidence of file purge attempts?

Having said which, I also see a possible fifth scenario… but it is so strange that no-one would believe me, only that I’ve seen the phenomenon elsewhere, and that the tone of the whole release post, which is both civil, intelligent, relevant, and well-phrased, is very similar.

My theory (which I am putting forward for peer review!) is that the .zip file was months of ongoing work to hide e-mails, code and documents which could be seen as damaging to CRU.

If the FOI application was successful, there would have been a dump of millions of megabytes of totally boring, innocuous “I’m not coming in today because Jenny’s got the measles” type of e-mails, formal responses, and so on.

This file was not prepared for a FOI disclosure – this file was prepared as a record of excised documents!

This would enable CRU to immediately comply with a successful FOI application, and be publicly seen to be doing the “right” thing.

However, it was necessary to keep a record of the “other” documents, which were being excised from the official disclosure. This is what our heroic hacker grabbed.

…… another instance of Lysenkoism, especially when Trenberth complains that there is something wrong with the data. Have these people ever considered the possibility that the theory might be wrong?
——————
…. and blames it on the “observation system”, the same “observation system” that got them there in the first place. That’s not a first year grad student mistake – that’s a first month grad student mistake. Infantile data analysis, garbage science. To be followed, of course, by whatever happened next …..

I hope the man is as ashamed of himself as he should be but, given that he’s deluded himself into thinking he’s saving the planet, I doubt that he would even know what I’m talking about.

Do not know what Steve M.’s original FOI demanded, but it stands to reason it was a broad request for the Code, databases, and all related correspondence regarding same. I am not convinced about the FOI angle either, but something you said is further compelling evidence: the fact this large file had almost no idle chit-chat. Almost all docs are about the database and the FOI request; one item about John Daly, nothing about Euro 2008 Football championships, sale of Christiano Rinaldo, the Ashes cricket test, Madonna’s divorce etc etc. That’s why this file seems to be compiled for a specific purpose; it is that purpose about which we are speculating. Again, Occam’s Razor tells me the file was created in connection with Steve M’s FOI request. Is it an inventory of what may be relevant?, a hot documents list ? safe documents list? purging list? I hope we all find out soon.

Data from Fig. 1, Crowley 2000:
Decadally smoothed time series of Crowley-Lowery reconstruction
spliced into smoothed Jones et al instrumental record after 1860
(labeled CL2.Jns11), and a slight modification (labeled CL2)
of the original Crowley and Lowery reconstruction to 1965.”

all the data is there too, but it is clearly a splice of a proxie to an instrumental record, published in Science no less.

A certain UK military establishment (which I shall allow to remain nameless) used to maintain Unclassified and Secret versions of its web page for access to the day’s data. We discovered during a system study there a few years back that the secret files were still present on the unclassified server copy – they’d merely deleted the links from the front page! (And needless to say, the boss there was less than happy about this state of affairs once he found out, and They Don’t Do This Any More!)

As you say, incompetence always needs to be considered as an alternative to malice.

Dear Monbiot, thanks for your surprisingly unbiased opinion. Now I would like to make you a suggestion – Can you make now a Royal Flush about Climate Change Cheaters? Phill Jones can be the Ace of Spades for instance. Cheers

There were a few posts about the supposed hacker masking their identity using an HTTP Proxy. If you believe that HTTP Proxies are safe, I have a cheep bridge for you in Brooklyn. You should remove such thoughts from your mind; the student who hacked Sarah Palin’s Email account thought the same things and the proprietor of the Proxy server he used was very helpful in assisting the FBI with their investigation after they made the gentleman an offer he could not refuse. You have absolutely no idea who is running the HTTP Proxy and I know from first hand experience that in the past, two of the anonymous HTTP Proxies on the Internet were run by United States intelligence services. Even if the proprietor of the HTTP Proxy service tells you they don’t keep logs, don’t believe them. The logs are their get out of jail free card.

The most popular free HTTP Proxy software on the Internet is Squid. Squid leaks information like a sieve and is a gift from the gods to any security investigator. If you believe you are anonymous on the Internet you are fool and are in for a painful learning experience. In the United States most Internet users have absolutely no idea of for how long Internet Services Provides retain DHCP lease information. Hint: try multiple years for starters and the in case of one very large broadband provider the correct answer may well be forever.

There are ways to hide your identity on the Internet but unless you are very good don’t waste your time and never trust an HTTP Proxy. Being truly anonymous on the on the Internet is a fulltime job and frankly not worth the effort in most cases.

As the world and her mouse now know, a server used by the University of East Anglia’s Climatic Research Unit (CRU) has been hacked into*, and many files, including personal e-mail messages, published on a Russian web site on Thursday [see: here; and here;; and here; and here; and here; and here; and here; and here; among many other media outlets and blogs]. The story, and some of the key details, have travelled around the world’s blogosphere quicker than Puck in A Midsummer Night’s Dream (“I’ll put a girdle round about the earth in forty minutes”), leading to much febrile, and often ill-judged, hysteria from both sides of the more puerile end of the ‘global warming’ debate.

My own observations about this internet ‘event’ are as follows:

(a)First, hacking* into a University server to seek institutional and personal data and material is illegal, especially when it involves the details of many third-party people. It is theft, and the fraud police should be brought in immediately. Whatever one’s position on climate change, this action cannot be condoned, and remember that what is sauce for the goose can all too easily become sauce for the gander. The University is thus right to have taken immediate action to remove the server in question from operation, to undertake “a thorough internal investigation”, and to “have involved the police in this inquiry”;

(b)Secondly, the amount of data released is very large (160MbB), and, although the hack has been confirmed in general terms by the University, and by some of the e-mailers concerned, there is no way that we can yet know whether all the material posted is accurate, and that it has not been doctored to some degree. Caution is thus required in taking what is presented at face value;

(c)Thirdly, caution should further be exercised because e-mails, and other such data, are often difficult to interpret fairly out-of-context and historical contingency. Furthermore, professions employ in-house jargon which does not always mean precisely the same as it does in common parlance, and, in the academic world, such banter is part of the norm;

(d) Nevertheless, all this being said, it does appear, superficially at least, that some of the leaked material may prove to be a tad embarrassing for some of the folk involved. Following a quick perusal of what has currently been circulated widely, and, assuming, of course, that the alleged documents are both accurate and undoctored by the hacker(s), the following issues could well raise serious concerns: ill-judged comments about other scientists; the reluctance to share data; the deletion of selected information and e-mails; the manipulation of data to produce a certain ‘scientific’ outcome; and attempts to change peer-review teams and journal status. If any of these alleged concerns do prove to have a foundation in fact, then answers will most certainly be demanded and required. [Update: see the excellent work at the ‘Bishop Hill’ blog, which has started to summarise the more significant files and e-mails, using the e-mail reference number. I have to say that, as an academic, I find some of these deeply disturbing];

(e)Above all, however, this episode has demonstrated, yet again, that ‘global warming’ exhibits all the characteristics of a classic post-modern grand narrative, a narrative that increasingly seems to have little to do with science. This should not surprise us. The rise of the grand narrative, as many readers will know, has been littered with deeply worrying statements, of which the following are perhaps some of the more notorious: “Unless we announce disasters, no one will listen” (Sir John Houghton); “To capture the public imagination we have to offer up some scary scenarios, make simplified dramatic statements and little mention of any doubts one might have. Each of us has to strike the right balance between being effective and being honest” (Dr. Stephen Schneider); and, the alleged (there is some debate about the validity and accuracy of this quotation, I understand): “We have to get rid of the Medieval Warm Period” (attributed to Professor John Overpeck). Sadly, it is not impossible that the newly-hacked material may add to this litany.

In the end, I fear that, if and when the ‘global warming’ grand narrative collapses, as I judge it surely must, it could well seriously damage trust in science itself, and that would be a most dreadful tragedy for all of us.

________________

* There have also been suggestions that the material may have been released, or leaked, directly from within UEA.

There are claims from spokespeople defending the criminal acts in this case:

Several have suggested that these criminal actions were the work of whistleblowers. These spokespeople suggest in fact, that the hacked file was prepared under a FOI (Freedom of Information) directive. If that is the case, then it opens up some intriguing possibilities:

A whistleblower is someone in an organization, that realizes that it is hiding or concealing information from legal authorities and decides to go directly to the legal authorities. Or alternatively, the whistleblower realizes the organization is publicly saying one thing, and internally has information saying significantly different. The second case is damaging, if opponents or critics don’t have access to the information. Typically the whistleblower in this case would go to the press.

In this case, apparently the emails were contained in a file that was assembled for a FOI demand in Britain. The British courts apparently turned down the request, likely because the information wasn’t material to use as requested by the applicants, i.e. it isn’t useful for scientific purposes.

Surely the attorneys for the applicants saw this information, so agents for the applicants did have private access to the emails. But the British courts said no to public disclosure.

It is possible that in order to get around the British courts, the emails were “hacked” and used for cyber sabotage. The hackers clearly knew to look for this file, and where to find it, if, in fact the file was hacked at all. It may just have been leaked.

If so, any legal defense of these criminal acts disappears. And in no way, can these acts be considered whistleblowing. Only if the information was illegally withheld from the British legal authorities, could whistleblowing be considered justified.

It appears possible, that the applicants didn’t like the decision by the British courts, and decided to do an illegal end run.

It also interesting to note, that some of the spokespeople defending these criminal acts, received the hacked files either before or contemporaneously with the cyber sabotage attack on RealClimate.

“Now, how to let these get out to the enemy without it being known as an intentional leak?” – Harold Morris

It may or may not germane, but as I’ve noted at CA, Gavin has posted that “the first comment posted on this subject was” a username link in a thread on CA. It strikes me as curious in the extreme that somebody – even someone with a climate scientists disdain for uncertainty – would not qualify this comment in terms like “the earliest comment we’ve yet discovered”.

This certainty is doubly curious as presumably Gavin had no idea then that the link had also been posted here but moderated.

“The “leak” was a quite deliberate camouflage job, to hide the equally deliberate destruction of seriously indictable material that was perceived to be vulnerable to FOI exposure
……….
“If this is right, we will see claims that the hacking caused extensive deletion of files.”

Pretty devious–not very English-like. OTOH, complicated schemes like this abound in Sherlock Holmes’s adventures, and other British mysteries, so who knows.

“From the Wall Street article posted earlier, this is exactly the kind of thing I was expecting. Other scientists who are actually believers in AGW shocked at the fact they didn’t imagine other scientists would be treated by a “Mafia” – These are the kind of level headed scientists in the Mainstream (of which I believe there are many) that will be crucial to the fall of the alarmists.”

Absolutely. The overall “mafia” pattern and mindset, not any particular act, is what is the most shocking and will carry the most weight, That’s what should be hammered on–not on whether any particular revelatory tidbit proves this or that. We should be cautious about the latter, but aggressive about the former. And we should be cautious about our remedy, merely calling for an impartial expert examination of the matter, given what’s at stake. We mustn’t make claims that the warmists can characterize as overstated, giving them grounds to dismiss them.

“In June 2009, former Clinton Administration official Joe Romm defended a comment on his Climate Progress website warning skeptics would be strangled in their beds. “An entire generation will soon be ready to strangle you and your kind while you sleep in your beds,” stated the remarks, which Romm defended by calling them “not a threat, but a prediction.” ”

Kaboom (15:03:06) :
You said: “This file was not prepared for a FOI disclosure – this file was prepared as a record of excised documents!
This would enable CRU to immediately comply with a successful FOI application, and be publicly seen to be doing the “right” thing.
However, it was necessary to keep a record of the “other” documents, which were being excised from the official disclosure. This is what our heroic hacker grabbed.”

Isn’t this a variant of the FOI reason for creating the file? This is the list and compilation of ‘hot’ documents. The list led Jones and FOI compliance officers to deny the FOI and stonewall Steve M. Speculation to be sure, but it does stand to reason.

Mr. Watts’ theory seems like one of the best so far. However, I can’t help but think that Phil Jones had inadvertently revealed his password, perhaps at a conference using insecure wifi. Certainly the “sophisticated hack” explanation is childish – especially coming from people whose supposedly would put the bar rather high for what would be considered sophisticated.

If the proposed theory is true, why are there documents in the FOI attachment that are completely unrelated to any FOI requests e.g. grant proposals, paper reviews etc.? That smacks more of a data scrape than of a careful collation of FOI relevant materials.

“So, we can have a proper result, but only by including a load of garbage!”
From HARRY_READ_ME.txt……you’ll find it……

It’s time the community got it’s facts straight….with this kind of behaviour we’re no different than the damn bankers trying to skim their little percentages out of the worlds productive capacity…taxes pay subsidies, don’t they ?

Unquote

Took me a few days put after sweetening it a little it got published….and got censored without the last few lines…..have the pics to prove it….they know, frantically trying to control damage….this is gonna be sweet.

I is certainly possible the described sort of accident where data was pulled off of an open server is a more viable explanation than a hacking incident, but that still raises the question of why the data was bundled together in the first place.

I have seen institutional reactions to FOI requests first hand. In our case we got a FOI request and my supervisor was tasked with gathering the raw data for the response. He gathered a bunch of us together and told us in no uncertain terms, that screwing with a FOI request was not an option, that they would satisfy the request, and everyone was to be diligent in collecting anything and everything they knew of that “might” be relevant. Only after it was all pooled together would they cull out stuff that did not really fit the letter of the FOI request. This was in the 1980’s and our agency had boxes and boxes of paper records in a warehouse and a whole group of us went down to the warehouse and pulled all the record boxes and we each sat on the floor and went through the boxes we were assigned to screen page by page and pulled each and every page we found that had the slightest possible relationship with the request.

This file could fit that description as a first pass data mining operation to pull possibly relevant documents.

That said there is also much to be said for the idea that this file represents someones insurance policy.

Due to the high level of political intrigue in large government organizations a lot of folks both nefarious and ethical keep “little black book” files. The ethical folks do it to cover their butt if they are tasked with doing something that is deep in the gray area that they are not comfortable with. They make such protestations of the action that they can when it is proposed and then do it, but keep evidence that supports their view that something fishy was going on.

Some do it as an emergency “pay back file” for other folks in the agency that have done them or one of their friends wrong. That can then be used as an insurance policy if things go south at a later date, or if they are sufficiently aggrieved , as a “dish served cold” when the time is right.

Sort of like the events portrayed in the movie “Clear and Present Danger”, where both Jack Ryan and his nemesis are keeping track of each others activities.

I would find either possibility equally likely in a highly political organization. Many folks who work in those sort of highly political organizations quickly learn, that if a certain person does something to someone else, that they would have no compunction against doing it against you. The natural defense for that sort of potential ambush some time in the future, is to keep notes that will document behavior that are both undesirable and desirable. I know of cases where folks smelled a rat in a situation and made a point of sending a memo or email that documents that they were not comfortable with that decision or that they were complying with a given request under protest.

On superficial reading that document appears to make them just one of the guys, but if you look closely at the word choices, could also exist only to document that the person did not really support the action.

It might be interesting to data mine those emails to see if any specific person consistently includes such “I’m not sure this is the way to go” statements, or included tell tales that the email was sent to document some off the cuff conversation or phone call etc. that without that paper trail would be completely off the record.

Another FOOBL moment.
ABC Radio at 5.00am this morning had Tim “Hot Rocks” Flummery making excuses about context etc, and all the evidence for greenhouse happening. This was the very FIRST mention from the ABC that anything unsual had happened.
But then on A.M. this morning there was a fairly good discussion of it and interviews with former Chancellor(?) and Dr Watson. So the wheels are slowly creaking into motion.
If only someone smarter and with deeper pockets than me would write “You’re a liar” in the British press and challenge Jones to sue for libel.
We live in interesting times.
By the way most of the hot weather has passed, rain and a cool change have arrived in central Australia (Bedourie 18 degrees below average set a new record) .
Keep up the good work fellas!

I have a theory, only a theory though…. What if all the e-mails and data is really a ‘delete’ file directory.

You are an organization who might be forced to imminently disclose embarrassing information that you do not want revealed. Further, it is a large amount of data and correspondence between many people and sources going back over a numbers of years. Let us further suppose that you are ‘important’ people with a lot of work to do before an important conference. What would you do?

You would get some bright ‘on message’ underlings (the sort of people you would find on a University campus) to go through all the files and delete anything incriminating. But of course you could not allow them to do this as they went along; too much chance of deleting important non-incriminating stuff. So you get them to put it all in one directory so that you can vet it first before it is all destroyed. But what if one of the underlings had some integrity and made a copy before the big deletion. As it is mainly text data rather than pictures, it would all transfer very quickly to a small flash card. Just a theory though….

Oh, I appreciate the service Glen Beck does, whom he speaks to, and the courage he displays on a daily basis.
It takes guts to get on the air and challenge ‘the science is settled’ agenda.
You see, 99.9% of Americans are NOT scientists.
So, if you don’t like Glen Beck taking up cause on behalf of the 99.9% of us, please roll out your 2009 version of Carl Sagan.

I have a different theory, as far as the emails go. I have been searching through the emails using years as key words (i.e.”1996”, 1997”, 1998” etc.) so that I could view all the emails for a year in chronological order.

There are a lot of junk emails in there; housekeeping, meeting announcements, newsletters, travel arrangements and just your general blah, blah blah. These would not have been incorporated into a potential FOIA release file.

I think that these were a random selection of emails (as was claimed) that were selected using a multiple keyword search much like many of us have been doing at anelegantchaos.org (which has changed to eastangliaemails.com). It screened out “Honey, don’t forget to bring home milk”, but included office junk because they contained the keywords that were used, which makes sense.

Here is an example of the junk, a newsletter from “Earth Government” advocating a centralized one-world government. It was probably received as spam by someone at CRU:

This has nothing to do with the work CRU does (Well, not directly anyway…), but it does contain the word “climate”. Interestingly, this email has a “From” line, but no “To” line. WUWT?

I have figured out that the search string “briffa climate research data” will extract 938/1073 of the emails, but I cannot figure out what the last keyword(s) are. Maybe someone else can figure out a better set of keywords, but I’m sure this is how it was done. Nothing else makes sense, because of the office junk, but not personal junk that was included.

BTW, in the initial news stories, Phil Jones claimed that CRU was aware of the hack 3-4 days before the .zip file appeared online. Bull. The .zip file contains an email dated Nov 12, the same day the .zip file appeared at the Russian ftp site. Now, unless this was an ongoing hack over many days, that they couldn’t stop, I think ole Phil was just trying to make it appear as though CRU was always on top of the hack, and doesn’t mind telling a porky to give that impression. Jeez, that comes so easily to this group.

So who did it? I think it had to be a CRU insider who was aware of SteveM’s FOIA request (and that it was going to be denied), who got fed up and scraped the files, compiled the .zip file, named it appropriately and uploaded it to the ftp site all on the same day (Nov 12).

BTW2, I visited that ftp site and learned that it is all in Cyrillic (Russian) which I guess is no surprise, but I sure would hate to be a CRU worker known to be fluent in Russian right about now. (Does Briffa speak Russian?)

Thanks for posting that link. One interesting comment posted under the article:

“The American legal principle of withheld evidence is simple: If the party who controls the evidence refuses to provide it, it is permissible to infer that the evidence is contrary to the withholding party’s interests.”

It’s difficult to not infer that.

Jimmy Haigh’s link above is also very interesting. Michael Mann is not a Penn State alumni, although he’s employed there. The way it looks, he is tarnishing Penn’s reputation. Penn State alumni should make their feelings known.

Grumbler:
Re George Monbiot’s admission that he’s a dreadful journalist: I agree with your first comment, “wow” – I found myself repeating that over and over as I read your summary. I’m beginning to have hope that the CRUtape letters are going to have an effect on the MSM and the AGW debate sooner rather than later. Wow!

I do not buy this stuff as prepackaged for potential FOI release. The NOAA funding e-mail, for example, does not seem to have any bearing on any of the known FOI requests, but it is certainly something someone might want to sweep under the carpet. Furthermore, no individual FOI request was broad enough to encompass even a fraction of what is contained in the archive.

I will assume stupidty on theorizing William Connelly was the perp. he’s been working for years in Wikipedia to create his green image of global warming in it’s pages. why would he do anything to harm manmade global warming????????

Ninh
NOAA want to give us more money for the El Nino work with IGCN.
How much do we have left from the last budget? I reckon most has been spent but we need to show some left to cover the costs of the trip Roger didn’t make and also the fees/equipment/computer money we haven’t spent otherwise NOAA will be suspicious.
Politically this money may have to go through Simon’s institute but there overhead rate is high so maybe not!
Best wishes
Mick

RC is trying to spin this story in the most favorable light, focusing on the Soon 2003 paper to prove how inept the skeptic argument is when looking at the e-mails. I wrote a reply and posted on my blog. It’s too long to post here so here is the link.

I’m off to Washington now and will be off the grid for a couple of days.

The reason is obvious. They had it ready in case the FOI application by Steve McIntyre was successful….. It was unsecured and was revealed by a curious reader…. No hack. No mole. No whistle blower…. Just negligent security practices by CRU and co.

Lets hope it grows and grows and brings some sense to Copenhagen – or is that too much to hope for?

That is indeed too much to hope for. Government policy is like an oil tanker (that wasn’t a deliberate attempt at topical humour), it takes a long time to turn around.

First, a suitable exit strategy has to be found. This usually involves commissioning a whopping review that doesn’t report until the main players have retired. Secondly, the Minister for Climate Change has to change his title to Minister for Energy Security, thereby allowing the government to continue with exactly the same policies, but offer grants to a completely different set of scientists to do research and give advice on this completely different issue. Only then will the government be in a position to do precisely what it intended to do all along: raise taxes.

Robinson (15:44:45) :
Glen Beck, he did make me laugh. “The University of East Anjeeela”.
Why can’t you Americans speak English ;).

Actually, I remember a piece of an article a long time ago, don’t ask me for the citation, I’m just too busy now, that the English of Shakespeare’s time, the Elizabethan Age, was very similar to that currently spoken in the American South. Maybe it’s the Brits who have strayed from their magnificent linguistic roots….;>)

“In the end, I fear that, if and when the ‘global warming’ grand narrative collapses, as I judge it surely must, it could well seriously damage trust in science itself, and that would be a most dreadful tragedy for all of us.”

I think the anger at “science” is really anger at modern science’s bureaucratic overlay, at its arrogance in thinking that its peer review process provides it with a self-correcting mechanism, and that its “democratic” funding process at the NSF is a good guardian against science getting off the rails. A few simple reforms could fix these problems. Henry Bauer’s book Scientific Literacy and the Myth of the Scientific Method suggests some solutions, and I’ve suggested a few others in some of my comments. Here are extracts form one of Bauer’s papers:
=========

Science in the 21st Century: Knowledge Monopolies and Research Cartels

I spent a little time trolling through several sites that might be termed pro-CAGW, or maybe CRU apologista. Where comments allowed, those by skeptics seem to far outnumber those by CAGWers and other greenfolk. Occasionally there’ll be a comment by someone like Mandia detailing why this doesn’t matter (with only limited success, IMO), but generally people seem to be expressing mostly vindication and/or indignation.

It is still early in this saga, but it seems to me that The Team has circled the wagons, battened down the hatches, and are hoping this blows over. With the large number of apologists, journalists and bloggers on point for them, I think that they might just survive this. Hopefully the seriousness of the email contents filters through to one or two powerful MSM sites that can really turn up the heat on this issue.

Well it is up to the Republicans in the US to bring this to a head. Australia(my home), NZ and Europe(inc UK) will gloss over and ignore Climategate. I have no faith in Australian politicians and the European ones have lost all grip on reality. Whilst I am slightly left of centre in my political beliefs I urge all Republican politicians to expose this fraud and finally bring the UN to its knees.

Both of these are complete in the actual zip file version. I guess some glitches may have occurred when it was all put in this searchable form?
*******************************
Ah, thank you Stuart, good to know. I hope Hugh is reading this and fixes that. I did not download the .zip file, but I guess you did. So tell me, do any of the words “briffa climate research data” appear in either of those emails?

Has anyone else noticed that as CRU-gate heats up, the misanthropic alarmists seem to be shifting focus quickly to “peak oil?” This is only an anecdotal observation on my part, but if the good ship AGW is about to plunge to the bottom of the sea after striking the CRU iceberg, now might be a good time to man the peak oil lifeboats to keep the hysteria going.

I’ve always found it odd that the same folks who worry so much about AGW also seem to believe that carbon based fuels are about to enter a terminal decline. Either scenario results in billions of dead humans, which seems to suit them just fine. Jolly bunch, aren’t they?

I see the comments are raising all kinds of theories, but the most likely theory is the most simple theory. Guys, think like a criminal investigator.

First, it is highly likely that the court making the decision on the applicant’s FOI request DID SEE the email file. It is very unlikely that a court would make a decision like that without see the information that they were ruling on. If that is the case, there can’t be a whistleblower defense. The hackers committed a criminal act, even ignoring the later cyber sabotage attack on RealClimate.

Furthermore, it is highly likely that the legal team for the applicant saw the email file as part of the discovery process. So which is more likely, a CRU insider, or someone who talked to the applicant’s legal team?

Who had motive to release the information?

Who had the means of access to the information?

And who was on the scene, and knew what files to look for, and where to find them? And who had just lost a court case to release the information?

In fact, who might have had the information, without hacking into a computer system at all?

There are some pretty likely suspects for this crime, if you look at the questions investigators would normally ask.

What is really funny about this is that last week we had this hack/leak of liberal Goracle erodite lefty data at CRU and this week we have a PI in San Diego catching ACORN dumping (illegaly) over 20k documents in a dumpster. The documents contained information protected by privacy laws and is now “evidence”.

Sadly this is all coming too late for Australia… we are on the brink of being steamrolled into ETS legislation by KarbonKev (Rudd the Prime Minister). He is deliberately forcing the issue through before Copenhagen despite the CRU hack revelations.

These are sad times down under. And to think we survived so well through the GFC only to come unstuck on the CPRS … even the name of our legislation makes me squirm due to its completely ingenuous association of the words carbon and pollution.

The intent is to reduce Australia’s emissions by at least 5%. So that’s 5% of 1.5% (Australia’s share of global emissions) … so in global terms a whopping 0.075%…

Someone may have mentioned this before, but I’m wondering if they archived/zipped the emails, etc. and put them on an FTP site in preparation for deleting them from the main computers/email system and going “ooops, the dog at it!” again. If that were the case (who knows?) the irony of it leaking out from an unsecured FTP server would just be too delicious wouldn’t it?

Frank (15:55:43) – “I have a theory, only a theory though…. What if all the e-mails and data is really a ‘delete’ file directory.”

I am inclined to agree with Frank’s theory, this does appear to a very plausible explanation. The material in the file does seem to be a dirty laundry basket. It does not seem to be a random grab of files or files directly relating to the FOI requests. This theory also makes the leak more amusing as it would mean someone at CRU was paid to find all the dirt for skeptics.

If this is indeed the case, I am sure CRU will stick with the mysterious Russian hacker stories while quietly trying to identify the leak.

My top choices for the leak would be –
1. IT worker or programmer
2. Nathan Gillett
3. Keith Briffa
4. Kevin Trenberth

I’m inclined towards your thesis ChasMod, since the likelihood of an outside hacker getting loose in there for long enough to gather up all that incriminating (seemingly) fileage, and not include tons of ho-hum, seems quite remote; unless it was a former employee who knew what was there. Who better to gather the whole sorry mess together for the dog to eat, than somebody very familiar with the contents, and the revealing nature of it all.
‘Twould be funny indeed if Phil Jones put it all together to 86 in case Steve McIntyre got too persitent; and ended up leaking it himself via the same snafu that got Steve the Yamal tree ring off that mishapen Gorse bush.

It’s starting to look like the old Italian firing squad circle of deceit.

And it’s quite uproarious to find Andy Revkin in the middle along with Mann, and RealClimate.

I was really getting to like Andy; he was seeming like a reasonably fair chap.

You’ve got a lot of digging to do Andy, to extricate yourself from this tarpit.

According to Jesse, Walmart might have some openings for floor swabbing jobs.
The outside hack job is looking a lot less feasible; which is good, since I’m not a fan of that; but inside whistleblowers, always will have my support.

This thing is like a mongrel dog heading, in a terrified state, down the street, with a bunch of tin cans tied to its tale .

Dear All,
Thought you would be interested in this exchange, which John Holdren of Harvard has been
kind enough to pass along…
mike

Delivered-To: mem6u@xxxxxxxxx.xxx
X-Sender: jholdren@xxxxxxxxx.xxx
X-Mailer: QUALCOMM Windows Eudora Version 5.0.2
Date: Thu, 16 Oct 2003 13:53:08 -0400
To: “Michael Mann” , “Tom Wigley”
From: “John P. Holdren”
Subject: Correspondence on Harvard Crimson coverage of Soon / Baliunas
views on climate
Michael and Tom —
I’m forwarding for your entertainment an exchange that followed from my being quoted in
the Harvard Crimson to the effect that you and your colleagues are right and my
“Harvard” colleagues Soon and Baliunas are wrong about what the evidence shows
concerning surface temperatures over the past millennium. The cover note to faculty
and postdocs in a regular Wednesday breakfast discussion group on environmental science
and public policy in Harvard’s Department of Earth and Planetary Sciences is more or
less self-explanatory.
Best regards,
John

They purge the main server, but not the back-up. Then someone creates a program which subtracts all programs that remain on the main server from all the programs on the back-up, and what remains is the very programs they hoped to purge.

I have the strong feeling that these emails and files are the very ones they most hoped would NOT be viewed.

Mod: I’ve submitted this twice, but I haven’t seen the usual pop-up of “Your comment is awaiting moderation.” So I’ve rebooted my browser and I’m trying again. (If my previous submissions are in a queue somewhere, ignore them and use this.)
==========

Phillip Stott:

“In the end, I fear that, if and when the ‘global warming’ grand narrative collapses, as I judge it surely must, it could well seriously damage trust in science itself, and that would be a most dreadful tragedy for all of us.”

I think the anger at “science” is really anger at modern science’s bureaucratic overlay, at its arrogance in thinking that its peer review process provides it with a self-correcting mechanism, and that its “democratic” funding process at the NSF is a good guardian against science getting off the rails. A few simple reforms could fix these problems. Henry Bauer’s book Scientific Literacy and the Myth of the Scientific Method suggests some solutions, and I’ve suggested a few others in some of my comments. Here are extracts form one of Bauer’s papers (and a link to its full text):
=========

Science in the 21st Century: Knowledge Monopolies and Research Cartels

………….
Supposedly authoritative information about the most salient science-related matters has become dangerously misleading because of the power of bureaucracies that co-opt or control science.

Science as an Institution

Dysfunction and obsolescence begin to set in, unobtrusively but insidiously, from the very moment that an institution achieves pre-eminence. The leading illustration of this Parkinson’s Law (Parkinson, 1958) was the (British) Royal Navy. Having come to rule the seas, the Navy slowly succumbed to bureaucratic bloat. The ratio of administrators to operators rose inexorably, and the Navy’s purpose, defense of the realm, became subordinate to the bureaucracy’s aim of serving itself. The changes came so gradually that it was decades before their effect became obvious.

Science attained hegemony in Western culture toward the end of the 19th century (Barzun, 2000: 606–607; Knight, 1986). This very success immediately sowed seeds of dysfunction: it spawned scientism, the delusive belief that science and only science could find proper answers to any and all questions that human beings might ponder. Other dysfunctions arrived later: funding through bureaucracies, commercialization, conflicts of interest. But the changes came so gradually that it was the latter stages of the 20th century before it became undeniable that things had gone seriously amiss.

It remains to be appreciated that 21st-century science is a different kind of thing than the ‘‘modern science’’ of the 17th through 20th centuries; there has been a ‘‘radical, irreversible, structural’’ ‘‘world-wide transformation in the way that science is organized and performed’’ (Ziman, 1994). Around 1950, Derek Price (1963/1986) discovered that modern science had grown exponentially, and he predicted that the character of science would change during the latter part of the 20th century as further such growth became impossible. One aspect of that change is that the scientific ethos no longer corresponds to the traditional ‘‘Mertonian’’ norms of disinterested skepticism and public sharing; it has become subordinate to corporate values. Mertonian norms made science reliable; the new ones described by Ziman (1994) do not.

Symptoms

One symptom of change, identifiable perhaps only in hindsight, was science’s failure, from about the middle of the 20th century on, to satisfy public curiosity about mysterious phenomena that arouse wide interest: psychic phenomena, UFOs, Loch Ness Monsters, Bigfoot. By contrast, a century earlier, prominent scientists had not hesitated to look into such mysteries as mediumship, which had aroused great public interest.

My claim here is not that UFOs or mediumship are phenomena whose substance belongs in the corpus of science; I am merely suggesting that when the public wants to know ‘‘What’s going on when people report UFOs?’’, the public deserves an informed response. It used to be taken for granted that the purpose of science was to seek the truth about all aspects of the natural world. That traditional purpose had been served by the Mertonian norms: Science disinterestedly and with appropriate skepticism coupled with originality seeks universally valid knowledge as a public good.

These norms imply that science is done by independent, self-motivated individuals. However, from about the middle of the 20th century and in certain situations, some mainstream organizations of science were behaving not as voluntary associations of independent individuals but as bureaucracies. Popular dissatisfaction with some of the consequences stimulated ‘‘New Age’’ movements. ….

A more widely noticed symptom was the marked increase in fraud and cheating by scientists. In 1981, the U. S. Congress held hearings prompted by public disclosure of scientific misconduct at 4 prominent research institutions. Then, science journalists Broad and Wade (1982) published their sweeping indictment, Betrayers of the Truth: Fraud and Deceit in the Halls of Science. It has become almost routine to read in the NIH Guide of researchers who admitted to fraud and were then barred from certain activities for some specified number of years. In 1989, the National Institutes of Health (NIH) established an Office of Scientific Integrity. So prevalent was dishonesty that the new academic specialty of ‘‘research ethics’’ came into being. Professional scientific organizations drafted or revised codes of ethics. Various groups, including government agencies, attempted to make prescriptive for researchers what had traditionally been taken for granted, namely, something like the Mertonian norms.

This epidemic of cheating in the latter part of the 20th century meant, clearly enough, that an increasing number of scientists were seeking to serve their personal interests instead of the public good of universal knowledge.
………………………..
Throughout the history of modern science, the chief safeguard of reliability was communal critiquing (Ziman, 2000). Science begins as hunches. Those that work out become pieces of frontier science. If competent peers think it worthy of attention, an item gets published in the primary research literature. If other researchers find it useful and accurate, eventually the knowledge gets into review articles and monographs and finally into textbooks. The history of science demonstrates that, sooner or later, most frontier science turns out to need modifying or to have been misleading or even entirely wrong. Science employs a knowledge filter that slowly separates the wheat from the chaff (Bauer, 1992: chapter 3; see Figure 1).

This filter works in proportion to the honesty and disinterestedness of peer reviewers and researchers. In the early days of modern science, before knowledge became highly specialized and compartmentalized, knowledge-seekers could effectively critique one another’s claims across the board. Later and for a time, there were enough people working independently on a given topic that competent, disinterested critiques could often be obtained. Since about the middle of the 20th century, however, the costs of research and the need for teams of cooperating specialists have made it increasingly difficult to find reviewers who are both directly knowledgeable and also disinterested; truly informed people are effectively either colleagues or competitors. Correspondingly, reports from the big science bureaucracies do not have the benefit of independent review before being issued.
…………………..
Causes

Price (1963/1986) saw the exploding costs of research after WWII as a likely mechanism for bringing to an end the era of exponentially growing science. The mentioned symptoms may indeed be traced to the escalating costs of research and the continuing expansion of the number of would-be researchers without a proportionate increase in available funds. The stakes became very high. Researchers had to compete more and more vigorously, which tended to mean more unscrupulously. The temptation became greater to accept and solicit funds and patrons while ignoring tangible or moral attached strings.
……………..
Unrealistic expectations coupled with misunderstanding of how science works led to the unstated presumption that good science could be expanded and accelerated by recruiting more scientists. Instead, of course, the massive infusion of government funds since WWII had inevitably deleterious consequences. More researchers translate into less excellence and more mediocrity. Journeymen peer-reviewers tend to stifle rather than encourage creativity and genuine innovation. Centralized funding and centralized decision-making make science more bureaucratic and less an activity of independent, self-motivated truth-seekers. Science attracts careerists instead of curiosity-driven idealists. Universities and individuals are encouraged to view scientific research as a cash cow to bring in money as ‘‘indirect costs’’ for all sorts of purposes, instead of seeking needed funds for doing good science. The measure of scientific achievement becomes the amount of ‘‘research support’’ brought in, not the production of useful knowledge.
………………….
Knowledge Monopolies and Research Cartels

Skepticism toward research claims is absolutely necessary to safeguard reliability. In corporate settings, where results are expected to meet corporate goals, criticism may be brushed off as disloyalty, and skepticism is thereby suppressed. As Ziman (1994) pointed out, the Mertonian norms of ‘‘academic’’ science have been replaced by norms suited to a proprietary, patent- and profit-seeking environment in which researchers feel answerable not to a universally valid standard of trustworthy knowledge but to local managers. A similar effect, the suppression of skepticism, results from the funding of science and the dissemination of results by or through non-profit bureaucracies such as the NIH or agencies of the United Nations.

While the changes in the circumstances of scientific activity were quite gradual for 2 or 3 centuries, they have now cumulated into a change in kind. Corporate science, Big Science, is a different kind of thing than academic science, and society needs to deal with it differently. Large institutional bureaucracies now dominate the public face of science. Long-standing patrons—private foundations like Rockefeller and Ford, charitable organizations like the American Heart Association and the American Cancer Society—have been joined and dwarfed by government bureaucracies like the Centers for Disease Control and Prevention, the NIH, and the National Science Foundation, which, in turn, are being overshadowed by international bodies like the World Bank and various agencies of the United Nations—the World Health Organization, the Food and Agricultural Organization, UNAIDS, and more. Statements, press releases, and formal reports from these bodies often purport to convey scientific information, but in reality these releases are best viewed as propaganda designed to serve the corporate interests of the bureaucracies that issue them.
…………………….
The upshot is that policy makers and the public generally do not realize that there is doubt about, indeed evidence against, some theories almost universally viewed as true, about issues of enormous public import: global warming; healthy diet, heart-disease risk-factors, and appropriate medication; HIV/AIDS; gene therapy; stem cells; and more.

‘‘Everyone knows’’ that promiscuous burning of fossil fuels is warming up global climates. Everyone does not know that competent experts dispute this and that official predictions are based on tentative data fed into computer models whose validity could be known only many decades hence (Crichton, 2003).
……………………….
What ‘‘everyone knows’’ about the science related to major public issues, then, often fails to reflect the actual state of scientific knowledge. In effect, there exist knowledge monopolies composed of international and national bureaucracies. Since those same organizations play a large role in the funding of research as well as in the promulgation of findings, these monopolies are at the same time research cartels. Minority views are not published in widely read periodicals, and unorthodox work is not supported by the main funding organizations. Instead of disinterested peer review, mainstream insiders insist on their point of view in order to perpetuate their prestige and privileged positions. That is the case even on so academic a matter as the Big-Bang theory of the universe’s origin.
……………………….
It is not that knowledge monopolies are able to exercise absolute censorship. Contrary views are expressed, but one must know where to look for them; so one must already have some reason to make the effort. That constitutes a vicious circle. Moreover, the contrarian view will often seem a priori unreliable or politically partisan, as already noted. Altogether, people exposed chiefly to mainstream media will likely never suspect—will have no reason to suspect—that there could exist a credible case different from the officially accepted one.

The conventional wisdom about these matters is continually reinforced by publicly broadcast snippets that underscore the official dogma. What other reason might there be to publicize, for example, the guesstimate that global warming will cause an increase in asthma attacks (Daily Telegraph, 2004)? This is just another ‘‘fact’’ to convince us that we must curb the use of coal, gas, and oil.
…………………………..
Reform?

The ills of contemporary science—commercialization, fraud, untrustworthy public information—are plausibly symptoms of the crisis, foreseen by Derek Price (1963/1986), as the era of exponentially growing modern science comes to an end. Science in the 21st century will be a different animal from the so-called ‘‘modern science’’ of the 17th to 20th centuries. The question is not whether to reform the science we knew, but whether society can arrange the corporate, commercialized science of the future so that it can continue to expand the range of trustworthy knowledge. Ziman (1994: 276) points out that any research organization requires ‘‘generous measures’’ of

_ room for personal initiative and creativity;
_ time for ideas to grow to maturity;
_ openness to debate and criticism;
_ hospitality toward novelty;
_ respect for specialized expertise.

These describe a free intellectual market in which independent thinkers interact, and there may be a viable analogy with economic life. Economic free markets are supposed to be efficient and socially useful because the mutually competitive ventures of independent entrepreneurs are self-corrected by an ‘‘invisible hand’’ that regulates supply to demand; competition needs to be protected against monopolies that exploit rather than serve society. So, too, the scientific free market in which peer review acts as an invisible hand (Harnad, 2000) needs to be protected from knowledge monopolies and research cartels. Anti-trust actions are called for.

Where public funds are concerned, legislation might help. When government agencies support research or development ventures, they might be required to allocate, say, 10% of the total to competent people of past achievement who hold contrarian views.
………………….
It should also be legislated that scientific advisory panels and grant-reviewing arrangements include representatives of views that differ from the mainstream.
……………………….
Where legislation is being considered about public policy that involves scientific issues, a Science Court might be established to arbitrate between mainstream and variant views, something discussed in the 1960s but never acted upon.

Ombudsman offices might be established by journals, consortia of journals, private foundations, and government agencies to investigate charges of misleading claims, unwarranted publication, unsound interpretation, and the like. The existence of such offices could also provide assistance and protection for whistle-blowers.

Sorely needed is vigorously investigative science journalism, so that propaganda from the knowledge bureaucracies is not automatically passed on. To make this possible, the media need to know about and have access to the whole spectrum of scientific opinion on the given issue. The suggestions made above would all provide a measure of help along that line. A constant dilemma for reporters is that they need access to sources, and if they publish material that casts doubt on the official view, they risk losing access to official sources.

I better go and check with my investment counselor to make sure that none of my resources are invested in anything this guy is involved with.

Talk about a deer caught in the headlights; I’ve never heard a more pathetic pitch for something in my life. The FN Reporter seemed more on the ball with both the climate issues,and the seriousness of this breach; than this Howard chump.

Chris Horner had him surrounded on all sides; well then that is Chris Horner.

With hundreds of billions, even trillions at stake in this issue, would you expect Climategate to be what it appears to be?

There were 1003 emails, Word documents, .pdf files, Powerpoint presentations, and Fortran source code for the climate models. The latter may prove the smoking-est gun, in fact, the code is commented with some pretty damning stuff. The ‘trick’ email is just the tip of the iceburg, too. There’s quite a few other dirty ones in there. Check it out before you say too much.Honey Pot?
I’m speculating, though there is some evidence to suggest some plausibility to a hypothesis. The files were likely leaked, or set up as a honeypot.
In IT security, a honeypot is a network or data which appears attractive but is meant to entrap intruders. In spycraft (and politics) this concept can be nested. The leaked or hacked information is good information salted with fake information, to later be revealed as bogus. The proverbial turd in the punch bowl. It’s genius.Who is the victim>?
Honeypots are never set up by the victim. That would mean that the CRU is not the victim, but rather the perpetrator.
The banal content of most of the emails would appear to make the corpus genuine. A few bogus emails salted in could later be proven to be faked, putting the entire collection into question, discrediting the “hacker” and everyone else using this information.Cui Bono?
The corporate oligarchy is using AGW and nearly everthing green as a tool for economic and social control. They’re clever and will stop at nothing. It’s what they do, and they’re good at it, that why they’re them and you are you.

Then again, I may be wrong about my hypothesis. Maybe the CRU is just like most other groups of scientists; vain, greedy, and quick to supress conflicting views. Has history taught us nothing?

PS: One obvious reform would be to provide funding somehow so that all science journals could move online, enabling online critiques–including critiques before “official” publication. It would be harder for gatekeepers to censor one side and promote the other in such a context, harder to pack the deck of peer reviewers, harder to intimidate editors, etc.

@Paul K2 (16:58:37) :
I don’t believe a court rules on FOI requests, so a lot of what you’ve said isn’t relevant. It is only if a rejected FOI request is appealed is there a possibility of it going to court… and it’s only a possibility.

Also, I’m not confident that there was a cyberattack on RealClimate. The impression I got was that the person who made that original post tried to do so at a number of sites. Both WUWT and CA have, from memory, acknowledged that they received that post but declined to put it up. I strong suspect that the same happened at RC. If that’s the case, calling it an cyberattack is really stretching the truth.

However, assuming the people at RC have called in the police, I think we should leave it to the authorities to determine if there was an attack, and, if so, where it came from.

If any of the Team are reading this, there is this neat thing called the “Scientific Method” It has been around a while although I suspect Dr. Jones and the Team have never heard of it as they seem unfamiliar with its basic principles.

I suggest they give it a go. The most important part of the method is called “Testing the hypothesis”. What you do is use your theory to make a prediction about the real world and then go and have a look at the real world and see if your prediction is correct. If you make predictions that are wrong, then your theory is wrong and needs to be discarded.

For example, global warming theory says the temperature of the world is sensitive to changes in atmospheric CO2. A prediction might therefore be that there is always a close correlation between changes in CO2 and changes in temperature.

The theory also says there will be a hot spot in the troposphere. And it predicts there will be a positive feedback warming from water vapour. These can all be tested by looking at the real world. Not by looking at models though.

What puzzles me is that experiments to test this sort of thing are not very difficult to do. In fact I think some of them have been done. I’m sure clever people like Professor Lindzen or Dr. Christie could help them if they don’t know how to do this.

Why did the Team skip past the testing bit and move straight on to the predictions about what is going to happen 100 years from now. Wouldn’t it be a good idea to test the theory first?

Based on Steve (Paris) (11:19:49)who said “To follow on, Gavin’s defense has in many ways been remarkably slick. If the ‘hacked’ file had been carefully cherry picked (just enough to look authentic, but also open to plausible counter-arguments) the surely a counter defense would have been prepared in parallel? They may be without ethics but dumb they ain’t. Do we know who the chess player is on the team?

And on NK (10:17:16) who said; “TO ALL–BTW, a distinct lack of trolls on this thread. While the lack of trolls takes away some entertainment value, it leaves the comments to be all high quality thoughtful stuff. Back to the trolls, I think even they know this IS REALLY BAD for the Church of AGW.”

(I did wonder that the AGW faithful had little to say on this beond “nothing to see hear – move along”. Are they going along with something they know will come to nothing?)

Also on the statement from the Met Office; “A spokesman at the Met Office, which jointly produces global temperature datasets with the Climate Research Unit, said there was no need for an inquiry. “If you look at the emails, there isn’t any evidence that the data was falsified and there’s no evidence that climate change is a hoax…”

(Is the file really as damaging as some say?)

h/t to Mike G (13:00:49) and Harold Morris (14:12:30) who got there before me although Robinson (14:34:02) finds this nerve a little too raw.

So theory four goes like this.

1/ They want to get their detractors off their backs for a while. Maybe they are getting too close to the really damming stuff for comfort.

2/ They construct a file of “superficially” damaging information but which they know will do them no real damage. All the really damaging stuff has meanwhile been locked up good and tight or deleted (by a mysterious unknown hacker).

3/ They add in a whole lot of neutral stuff to fill it out and deliberately leak it by proxy (they are known to have sympathetic contacts in Turkey for example) into the public domain.

4/ Then they cry foul – “look at how the nasty skeptics treat us poor honest scientists” – and call for an investigation into the “hack” and (maybe if you must) into the leaked data, in order to get a waver on any investigation into all the data they didn’t release which has “got lost” anyway.

My view is that the timing re the refusal of Steve’s FOI request points to this being an archive of relevant material prepared for and probably transmitted to the legal counsel advising the University on its response.

Counsel would have made its recommendation to decline several days before the decision was advised to Steve. At the same time it would have told the University to archive the material as a protection against any subsequent legal challenge and accusations of destruction of material. The archive would have been updated to that point and probably transmitted to counsel for approval and safe-keeping.

The current traffic represents an opportunity to reach a wide audience with the main messages about natural climate variation. My instinct at this time might be to make every 2nd post a quick “remake” of a classic or a cut/paste of a recent abstract that directs attention towards natural variation classics – (short & sweet stuff). At a time like this it is easy to forget that a lot of the extra traffic may never have even heard of things as basic as PDO; it’s a teachable moment …and throwing up interesting wallpaper for the guests needn’t become a laborious endeavor, but if it’s not feasible, no worries – and whatever – and rest assured that the tireless volunteer hours are appreciated. Best Regards.

They know what they are doing. And what they are doing is very bad, and they do not care.

“This is what they did — these climate “scientists” on whose unsupported word the world’s classe politique proposes to set up an unelected global government this December in Copenhagen, with vast and unprecedented powers to control all formerly free markets, to tax wealthy nations and all of their financial transactions, to regulate the economic and environmental affairs of all nations, and to confiscate and extinguish all patent and intellectual property rights.”http://pajamasmedia.com/blog/viscount-monckton-on-global-warminggate-they-are-criminals-pjm-exclusive/

So as part of the security crackdown at CRU they have taken down their external webserver? Network security professionals in the audience will be spitting up coffee all over their keyboards at this point.

Well, it was Egg Nog and I managed to stop it on the sleeve… ;-) but I think you are on to something here…

That the web server was taken down is a “smoking gun” as to where they think the leak happened. I’d guess it is a “one box does all” set up. Web servers (inside and out), FTP servers (inside and out), maybe even EMAIL and DHCP (though perhaps only the outside visible half – but I’ve had sites that demanded both on the one box…) IFF they were really sloppy, they might have had a VPN server on the same box… (Bad JuJu… dedicated VPN only, please!)

The pattern of data in the file had been nagging at me. HOW, I wondered, had the “hacker” spent the months it would take to accumulate all this stuff, but NOT all the “Coffee at Tims”, “Don’t park in Bobs Space”, “Mary will be out Tuesday” etc.

Either they had to download Gigs of data and weed it all out… or they had to spend a very large time “exposed” on the inside.

But I had no good thesis to explain away the “problems” of massive volume, or large work, (or both) and long exposure times.

An insider, working to meet the requirements of an FOIA, filtering to just that stuff, and batching it up with a “release to FTP” script who fumbles a “move to archive” into a “move to FTP server” fits.
So this is my theory is and this is only my theory:

MODERATOR: I think you have one too many “IS” is..

[Reply: Yeah, I finished it off and did a bad job from memory only of making a too obscure reference to this, which I was hoping someone might notice. ~ charles the sometimes sloppy moderator]
If I had to bet money, I would guess that David Palmer, Information Policy & Compliance Manager, University of East Anglia, has an even chance of being the guilty party, but it would only be a guess.

In my experience, the IP and Compliance folks don’t make that kind of mistake. They create the file, but some other party does the “move to server / set up account for distribution / set access bits” part of the job. I would expect that to be the person… Perhaps a junior sys admin and maybe even a student working in the data center as an intern.

So David Palmer is very likely talking with Someone Right Now… And in about 2 weeks, you will see an advert for “Position Open” at UEA… And then you will know who…

And as a guess, I would say the “Webmaster / FTP Admin” is a combined position (person responsible for external servers) and managed to “ham hand” some permissions issue so that the “visible to the inside FOIA file” became “visible to both with pub access” instead of “Taken off server to archive”…

mv FOIA.txt inside:/ftp/server/dpalmer/FOIA.txt

or

mv FOIA.txt outside:/pub/server/dpalmer/FOIA.txt

It could really be that simple. Then “somebody” sees it, and runs with it.

It all fits. Very very nicely.

A little time with the ftp server logs would pretty much tell you everything you needed to know. The individual machine “command log” would confirm it. Whenever they come back up on the ‘regular’ machines, doing “host http://ftp.uea.ac.uk” (or whatever the real name is) and doing the same on the web server ought to show up the IPs, then attempting service connects to those IPs would confirm if they were the same box…

FWIW: I always had a battle with “upper management” over number of servers. My mantra was “One Service, One Server”. They (I think at Microsoft prompting) were always pushing to have multiple services loaded on one (typically Windows) box. The hardware costs are near nothing and the labor costs go way down with multiple boxes while reliability / availability go way up.

I wanted it to be very hard for a bug in, say, the email daemon to bring down the FTP server, or an FTP security hole to expose the email server. I also wanted the FTP server to still work even if the email server hung (or was getting maintenance).

It makes things much more secure and much easier to maintain if you have a “network switch” with a rack of gear plugged into it, a couple of routers, and each box has only the minimum software to do it’s job.

There were times a service died, and within minutes we were back up via a simple disk image swap into another generic PC box. And there were times that someone would try to hack the web server, but that did nothing to email flow. If I were doing it now, the only thing I would change is that I would build the server image onto a Bootable CD. Can’t hack a write only image very well… and if you need to replace a server, it’s just spit out CD, move one box over, boot.

Don’t know, really, why the Finance and Legal guys always want to push for the One Big Box solution. It leads to stuff like accidental file leaks… and the more services there are running on a box, the more likely it is that you can find one with an exploit to use…

With resepect to the hypothesis that the archives were put together for deletion:
Why wouldn’t you just delete them? And even if you weren’t going to just delete them, it would be insance to store such an archive on anything other than offline storage (a DVD having more than enough space to store the entire archive). The theory is far too complicated.

Who would have put this archive together? Answer: the “FOI people” who are mentioned in a number of the emails. There are most certainly university staff who have a legal responsibility to address FOIA requests with due dilligence.

The smoking gun for “FOI poeple” is that routing headers have been stripped off all the emails. There’s no reason why Phil Jones would have done this. No reason why a russian hacker would have done this. A very remote possibility that a whistelblower might have done this. But the simple explanation is that an FOIA officer would have stripped the headers, in preparation for releasing the emals, becuase leaving the headers in place would have disclosed sensitive information about university IT infrastructure.

who suggested that there was some damning material that you wouldn’t have needed to include for a FOI response. I agree, but suppose that what we have seen is merely the first draft. This could have been created using keyword searches based on Steve’s request, then put on the FTP server for everyone to have a go at deletions. But just then (next day) Steve’s request was turned down and the immediate need for deletions passed. So everyone forgot about and it stayed there as an unmodified first draft.

I’d also like to echo Paul Vaughan’s earlier comment: “The current traffic represents an opportunity to reach a wide audience with the main messages about natural climate variation.”

The intermittent availability of CA over the weekend is a tragic lost opportunity. The story that’s not getting told that needs to be told: what’s special about this data. As a long-time follower of CA, I know exactly what’s special about that data, and why the revelations in the archive are so directly on-point to McIntyre’s tireless efforts to understand how one of the most important graphs of the decade was constructed.

A summary article would be much appreciated.

One very good piece of advice from Phil Jones: keep it short. MSM lives eats and breathes sound-bytes.

Graeme W (19:08:06) :
@Paul K2 (16:58:37) :
Also, I’m not confident that there was a cyberattack on RealClimate. The impression I got was that the person who made that original post tried to do so at a number of sites. Both WUWT and CA have, from memory, acknowledged that they received that post but declined to put it up. I strong suspect that the same happened at RC. If that’s the case, calling it an cyberattack is really stretching the truth.

What happened at RC was that the file containing the emails etc. was mounted on RC by the ‘hacker’ (access to RC was blocked for a while) and a link to it was posted on CA.

Graeme W (19:08:06) :
@Paul K2 (16:58:37) :
Also, I’m not confident that there was a cyberattack on RealClimate. The impression I got was that the person who made that original post tried to do so at a number of sites. Both WUWT and CA have, from memory, acknowledged that they received that post but declined to put it up. I strong suspect that the same happened at RC. If that’s the case, calling it an cyberattack is really stretching the truth.

What happened at RC was that the file containing the emails etc. was mounted on RC by the ‘hacker’ (access to RC was blocked for a while) and a link to it was posted on CA.

Inquiring minds wonder how Phil. knows so much about what happened at RC.

Response to Graeme W (19:08:06) : You said there wasn’t any legal review unless the FOI request went to appeal. McIntyre’s request did go to appeal, which he lost just before the cyber attacks.

Here is my take on who the criminal investigation:
From normal criminal investigative methods, the cyber attacks were likely the work of a group of criminals, as it is unlikely these complex and orchestrated attacks were the work of only one person. The prime suspects should include people who had motive, means, and were familiar with the crime scene. In other words, a group who knew about the file of emails, where it was located, and how to find it, then had the motive and capability to pull off this series of attacks.

I would hope that the group associated with McIntyre’s appeal will be thoroughly investigated, since they normally would be considered prime suspects. The fact the attacks occurred immediately after this group lost their appeal would be consistent with an end run by this group around Britain’s laws.

Has anyone asked McIntryre whether he is sure his group wasn’t involved in these cyber attacks? If any of the people working with him were involved, it could pose a serious legal concern.

Orchestrated? You’ve been watching too many movies. There’s one file and it showed up on one ftp server and there were a few blog comments. There is the possibility that RC was hacked, but even this may not have required much sophistication. If I am wrong and this is Theory Number 2, an insider, he or she likely had access to RC user and password information.

SHOW US THE SERVER LOGS GAVIN! WHO LOGGED IN AT THE TIME OF THE “BREAK IN”?

Charles, I think orchestrated is the key word that describes these cyber attacks. The idea that this could be the work of one individual is pretty low. Clearly someone knew about the existence of the file, where it was located, how to retrieve it, and was cognizant and well informed on the status of McIntryre’s appeal of FOI denial. Then they constructed a cyber trail of servers to hide their identity, hacked the server, retrieved the file.

Then they knew how to hack the RealClimate system… I tried to visit RC during or immediately after the attack, and found the site down, so this wasn’t a simple attack. It would have been helpful to hacking RC if the hackers had previously exchanged files with RC. The same is true of the cyber attack in Britain. This points to someone, or a group, that had communicated with the sites, so this seem to implicate someone working on or against climate science.

After hacking the RC site, they had a fake post prepared, and knew how to post it on the site. In addition, key resource personnel who might be sympathetic to their political agenda received the file information before or simultaneously with the attack of RC.

All of this points to a very orchestrated attack that required a spectrum of skills and specific information not generally available to a single would be hacker who stumbled over an exposed internet server file. It is far more likely that a group planned and committed this series of criminal cyber attacks. It is unlikely, this was all pulled off by just one disgruntled CRU worker.

You show little knowledge of what is required here or what the circumstantial evidence shows. The fact that UEA CRU took their webserver offline shows a gaping security hole–one person, perhaps someone who just stumbled in through an open door could have downloaded the file.

I myself found and downloaded unpublished Catlin Expedition Videos exactly that way. No hacking required.

Have Gavin show us the server logs showing a true cyberattack and you may have something. His history of misdirection noted in the original post is on record. I am not asking for an IP address of a proxy server. I want the server logs.

I have prepared files like these in the past for passing on to my Government Department’s lawyer. They takes ages. Days and days, involving everybody who had any relevant material.

There is no way that an outside hacker could have assembled this by combing the files. They would just take everything and leave it up to us to sort it out. I would have sold it to a journalist myself. It has to be either inside job or screw up.

find the notion that HATCRU would engage in a systematic coverup of an approved FOI request not credible.

They will not have seen it that way. They will have seen it as protecting “their” information. Protecting their careers too, for that matter.

Many people, even when they know not to much around with this sort of stuff, can’t bring themselves to disclose fully. I know that I have been tempted to let particularly embarrassing material find its way to a shredder.

Perhaps they they suggested deleting it all. A leak from someone not complicit in the material but who stood to lose from an illegal activity (lawyer, IT guy) would make sense.

Given that the empirical evidence for cooling over the last decade is all but irrefutable I happen to think that the poor, formerly useful idiots at the CRU have been hung out to dry by their betters and erstwhile patrons in the Climate Racket Party as a form of damage control / plausible deniability.

The Communists used to do this all the time prior to a sea change in hitherto unalterable policy.

Look forward too massive amounts hand-wringing and angst among the Climate Racket Party’s Politburo over the coming months over how they were tricked and let astray down the wrong path by a few deviationists in the CRU.

So look out for the emergence of a new Global Cooling / Ice Age (ala the 1970s) scare over the next few years still under the rubric of climate change all in the name of reform.

1 Unauthorised access to computer material(1)
A person is guilty of an offence if—
(a) he causes a computer to perform any function with intent to secure access to any program or data held in any computer;
(b) the access he intends to secure is unauthorised; and
(c) he knows at the time when he causes the computer to perform the function that that is the case.
(2) The intent a person has to have to commit an offence under this section need not be directed at-
(a) any particular program or data;
(b) a program or data of any particular kind; or
(c) a program or data held in any particular computer.
(3) A person guilty of an offence under this section shall be liable on summary conviction to imprisonment for a term not exceeding six months or to a fine not exceeding level 5 on the standard scale or to both.
Jurisdiction

4 Territorial scope of offences under this Act
1) Except as provided below in this section, it is immaterial for the purposes of any offence under section 1 or 3 above-
(a) whether any act or other event proof of which is required for conviction of the offence occurred in the home country concerned; or
(b) whether the accused was in the home country concerned at the time of any such act or event.
(2) Subject to subsection (3) below, in the case of such an offence at least one significant link with domestic jurisdiction must exist in the circumstances of the case for the offence to be committed.
(3) There is no need for any such link to exist for the commission of an offence under section 1 above to be established in proof of an allegation to that effect in proceedings for an offence under section 2 above.
(4) Subject to section 8 below, where-
(a) any such link does in fact exist in the case of an offence under section 1 above; and
(b) commission of that offence is alleged in proceedings for an offence under section 2 above;
section 2 above shall apply as if anything the accused intended to do or facilitate in any place outside the home country concerned which would be an offence to which section 2 applies if it took place in the home country concerned were the offence in question.
(5) This section is without prejudice to any jurisdiction exercisable by a court in Scotland apart from this section.
17
Interpretation
(1) The following provisions of this section apply for the interpretation of this Act.
(2) A person secures access to any program or data held in a computer if by causing a computer to perform any function he—
(a) alters or erases the program or data;
(b) copies or moves it to any storage medium other than that in which it is held or to a different location in the storage medium in which it is held;
(c) uses it; or
(d) has it output from the computer in which it is held (whether by having it displayed or in any other manner);
and references to access to a program or data (and to an intent to secure such access) shall be read accordingly.
(5) Access of any kind by any person to any program or data held in a computer is unauthorised if-
(a) he is not himself entitled to control access of the kind in question to the program or data; and
(b) he does not have consent to access by him of the kind in question to the program or data from any person who is so entitled

It does not have to be locked away behind a security screen. If you were not explicitly given access then you are guilty!
Take care!

Inside job by someone in the know and who saw the import of the emails. Emails are examined all the time and there would a number who knew what had been said. If you faced with this content and realize consequences it would be very hard for anyone with a conscience to ignore it. Email users think they are private communications but they seen by many eyes. Even deleted by the originator they are kept by law usually and on backup tapes. The content of a FOI would be chosen carefully and vetted this is a dump of mail and attachments. The originators did not keep them personally for so many years they came from a backup or such. Who did it? I am damn sure if I did it you would not find out and if I had been in that position I would have.

edrowland a fair number I have seen do have headers and very sensitive information. Some have sufficient info to email all the major players in the AGW camp. There are over a thousand emails this is a dump not an FOI.

Paul Vaughan (20:19:17) : jeez (13:22:41) “It’s just weather”…..
The current traffic represents an opportunity to reach a wide audience with the main messages about natural climate variation. My instinct at this time might be to make every 2nd post a quick “remake” of a classic…

I’d second that. Heh, and plug my own primer (click on my name!) and, yes, a few others like James Peden.

Hard to believe they would have put such juicy stuff in a single file to turn over in case they last the FOIA contest. Rather, they would have put only the bare minimun to comply with the request or “inadvertently” lost anything this juicy in the compilation process. Forgive me if this has already been discussed. At work and don’t have time to read the whole thread…

But if this was the material prepared for FOI release, it would be unusual to add so many ‘smoking guns’ to the file. Why not choose some really innocuous emails instead? This theory also implies that even more explosive emails remain somewhere, which are being deleted as we speak.

I think the whistle-blower theory is best. Is there any common link between these emails – and common denominator in the addressees?

A monster thread, so this may well have already been said. But what if FOI2009.zip was indeed prepared in response to an FOI request, but instead of being the stuff they were planning to release, it is in fact the more embarrassing stuff they had excluded?

That makes more sense to me. Most of this is not appropriate as a response to an FOI request. Explicit budgets, most of the emails, etc.

The info was leaked intentionally by the CRU in order to control the story. If they knew the info would have to come out soon because of the FOIA request, releasing it now and pretending that hackers stole it changes the story from:

The name, John Holdren (Obama’s Science Czar) shows up in 6 of the e-mail streams. Five are merely cc:-s, but one to Michael Mann is from Holdren where he pokes fun at his “Harvard” colleagues Soon and Baliunas (1066337021).

Charles – I think you are being too simplistic, and ignoring the most likely scenario. The idea that a file created for FOI purposes, i.e. legal reasons, was left unprotected, and someone came along and just stumbled over it is pretty naive.

Other commentators seem to be reading too many romantic novels, if I have “been watching too many movies”. They have the idea that this is the work of a HadCRU insider. OK, perhaps a misguided worker would take the risk of releasing the file, but why would they orchestrate a cyber attack on RC? They would be taking too many risks and committing too many felony acts.

I like the name ClimateGate for these events. Like Watergate, we know a ‘burglary’ was committed. Now we need to find the ‘burglar(s)’, and their bosses. It is pretty easy to identify the prime suspects; clearly suspicion should fall on McIntyre’s group who lost the appeal for the FOI release.

I believe the FOI compliance authorities knew about the file of emails, but declined to release it since the emails are not useful for scientific purposes, the reason M’s group sought FOI release. It is likely HadCRU has already addressed some of the FOI concealment concerns.

The most likely scenario is M’s group also knew about the file, and few days after losing the appeal, orchestrated the release in violation of the FOI appeal decision. This is similar to the kidnapping of a child, a few days after a father loses a custody trial… the suspicion would immediately fall on the father.

Are you certain that people associated with McIntyre didn’t release the information?

But then, not knowing what the real story is allows us to come up with alternatives, which gets us disagreeing with ourselves, and they can hide behind the dust that’s kicked up in the process. Multiple conflicting hypotheses on our end give them the advantage of accusing us of not knowing what we are talking about. It’s a strategy they often use, so why not in this case?

An investigation is needed. But I doubt it will happen. There are too many potential investigators who would be hurt by it, and who will also be trying to bury it.

Are you certain that people associated with McIntyre didn’t release the information?

Uh, yeah, because I am the one that told him about the existence of the file, and my roommate spent hours on the phone with Steve reading the contents of the emails to him because I wouldn’t even forward a copy or forward the link to the Russian ftp server. As I noted in the original post, we only began to even refer the to file publicly after, and only after it began circulating on the Internet and CRU was in the process of notifying its personnel internally.

I didn’t think of it, just added my 2cents worth to vboring’s insight. I think it’s a good gambit, better than remaining passive. That kind of thing is done all the time, and in the ensuing confusion, the perp usually gets away with it, and sometimes even advances his agenda. I think it’s a very plausible scenario.

Paul Vaughan (12:36:57) :
bill, are you trying to hint that there is an alarmist plot to jail truth-seeking nonalarmists?

There is no problem with truthor truth seeking. However no one seems to understand the Computer Misuse Act as I linked to.

1. You cannot access data you have not been given rightss to access – it is illegal even if it was no security protected.
2. England is the “defamation capital of the world” It costs a mere £1700 to set the ball rolling. The accused are not assumed innocent till proven guilty – they have to prove they are innocent. So all those accusations of “fraud” and “criminal” need to have real provable backing!!

I do not know what the law is in this case, but I can imagine lawyers such as those employed by the RIAA holding that each email is a copyrighted entity subject to the provisions of the Digital Millennium Copyright Act. I assume the moderators here know this not to be a risk.

bill (17:09:41) “England is the “defamation capital of the world” It costs a mere £1700 to set the ball rolling.”

bill, I’m not sure if you realize it or if it was your intent, but it looks like you are suggesting a grand conspiracy to criminalize nonalarmism. I urge you to pause to broaden your perspective and realize how serious this is.

Paul Vaughan (21:19:39) :
bill, I’m not sure if you realize it or if it was your intent, but it looks like you are suggesting a grand conspiracy to criminalize nonalarmism. I urge you to pause to broaden your perspective and realize how serious this is

If your reputation is lowered by statements calling you a criminal/cheat/fraudster you have a right to defend your reputation. I’m sure you agree with this?

I was simply pointing out that defamatory comments need to be reigned in or else the hurt parties could make a claim for damages. If you have legally sound proof of your claims then there are no problems (although proving your innocence is not gong to be cheap).

I have also pointed out the computer misuse act – google it.

http://www.out-law.com/page-6207A man was convicted in London yesterday of hacking into a charity website, set up after the Indian Ocean tsunami disaster, in breach of the Computer Misuse Act.

Daniel James Cuthbert, a computer consultant formerly with ABN Amro bank, was given a £400 fine and ordered to pay £600 in costs at Horseferry Road Magistrates court yesterday, according to reports.

He fell foul of section one of the Computer Misuse Act, the UK’s main cybercrime legislation, on New Year’s Eve last year.

Cuthbert clicked on a banner ad to donate £30 to the Disaster Emergency Committee (DEC) appeal. However, when he did not get a confirmation or thank you in response to his donation, he feared that he had fallen for a phishing site, and decided to test the site to make sure. Unfortunately, in doing so he set off the DEC protection systems, and the police were called in.

According to SC Magazine, District Judge Mr Quentin Purdy found Cuthbert guilty with “some considerable regret”, but the wording of the Act made it clear that the security consultant was guilty. “Unauthorised access, however praiseworthy the motives, is an offence,” said the judge.

Cuthbert, 28, has lost his job with ABN Amro, and has since found it hard to find alternative employment, according to reports.

I return to the files timestamps. There are quite a lot of files with manipulated timestamps, most 2009-01-01-06:00:00, but also from 1980-01-01 and 2004-01-01, they all carry the “06:00:00″ part, that’s what shows up in my timezone CET. Why would someone set it to 06:00:00, wouldn’t it be easier to set it to 00:00:00, while your finger is ther at the “0”? Maybe he set it to 00:00:00 when he added those files, but he was doing it on a computer with timezone set to US Eastern Time?

If it was a hacker on the outside then not only did they know how to get onto the network but they new who the target was and what specific information was useful. They didn’t download the entire contents of the university network. That doesn’t seem a particularly likely scenario to me. If there was a hacker involved then there would likely be a trace of the IP addresses back to the destination of the “stolen” data.

I would say that in response to the FOI request the Uni lawyer sent out an email requesting compliance. The “team” at the CRU then discussed the implications of this. The “team” consisted of “dirty” individuals and “clean” individuals. The “dirty” individuals didn’t really know what to do and stonewalled while the “clean” individuals set about complying with the request. The “clean” individuals probably didn’t even realise there was something to be concerned about. The “dirty” individuals couldn’t immediately prevent the “clean” individuals from gathering the data without giving the game away. The “clean” individuals probably discovered eventually that the emails indicated “dirty tricks”. At the same time the “dirty” individuals had succeeded in blocking the original FOI request. Then an unholy row broke out and the question was asked “well now what the hell do we do?” and the decision was made within the CRU to stifle the information. Someone in the “clean” group was disgusted by this attitude and posted the gathered information anyway, then covered his tracks by claiming a hacker must have broken into the network. If the responsible individual is discovered, he or she will probably announce that he or she was only releasing data relevant to a legal FOI request – so my guess is that the individual responsible will not be turned over since that won’t be in the CRU’s interest anyway.

Something similar happened in the UK recently where a civil servant released details of MPs expenses to the newspapers because they had been subject to an FOI inquiry but he didn’t believe that the MPs would comply. He was quite open about what he had done and why – no hacking involved. A whistleblower seems more likely to me.

A good thread but IMHO we should be looking well beyond the techie angles in this case. I’d usually go with “cock-up over conspiracy” but there’s just too much riding on AGW across the world now. The ultimate goals may still be varied and illusory but the stakes are clearly gigantic.

“How will this all be resolved? Momentum does seem to be growing, from people on both sides of the argument, behind calls for a full independent enquiry that can once and for all get to the bottom of the many issues that have been raised. A recent survey showed that climate scepticism in this country is growing, and this episode may increase it further. Some would say that an enquiry is the only way to bring clarity to the science of global warming and climate change that has enormous implications for all of us.”

This is beginning to sound a smidge too convenient for my liking, as too is the recent posturing of Monbiot, Bob Watson, etc. The title Climate change debate: ‘calm, civilised, informative’ on Andrew Neal’s blog, while snappy and appropriate to the discussion that took place, is also a little too sickly sweet.

Bring on an enquiry, forget the troublesome historic temp record (which is complete tosh anyway just based on margins of error) and manage the legitimization of AGW onto a ‘superior’ authoritative basis. Never waste a good crisis.

OTOH, given the Machiavellian influences pushing hard from every angle here, maybe we’ve just witnessed a night of the long knives moment . Why would manufacturing a crisis orders of magnitude smaller than AGW itself be a problem? The careers of a few brownshirt scientists don’t matter in the scheme of things.

Time will tell whether an enquiry materialises but far more interesting will be how quickly and how open. After all, timing is everything in management. Watch carefully for both sides of a managed discourse supporting the same invisible goal that has nothing whatever to do with the physical climate.

Of course, there are problems with that, as Neo (15:20:35) points out. The BBC could be, ahem, “mistaken,” or something else.

Frankly, I don’t trust any of those who have historically been allies in pushing AGW against all reason, and that includes the BBC.

That’s not correct at all, what Hudson (BBC) said was that the emails which had been addressed to him, which he had received in October, were exactly the same as the versions of them in the stolen file. So to that extent the ‘hacked’ emails were accurate. That’s all.

It seems clear to me that this leak is based on McIntyre’s denial of FOI, because of the date of emails stopping the day before the denial. The name of the file also supports that. The most likely source would be a copy of the nightly backup, which would include emails up till the day before, but none for the day it was being done on.
This does not give us reason to conclude one way or another if it was an inside job or not. But it does tell us that it was being done by someone following the debate closely, which makes in insider more likely.
The message accompanying the file states a “random selection” of emails. Using the word “selection” seems to indicate that some sort of selection process has been exercised, i.e. this is not a complete archive. That doesn’t necessarily mean that there’s more to come though as the use of “random” may be tongue in cheek, i.e. not random at all, but clearly filtered. Indeed there almost seem to be too much damning information for it to be random.

* If it was legally downloaded from a public server, why has no one taken responsibility. I do not believe your theory. And I am certainly not convinced by the new ‘expert’. His analysis reeks of bias.

* The letter says “we”. That does not sound like an individual.

* The IP-adresses used on tAV, CA, & RC were from different parts of the world. That is pretty professional. IMO: not consistent with a leak.