Did Jones Delete Emails?

It turns out that Muir Russell didn’t bother asking, since that would have exposed Jones to potential liability.

Muir Russell under questioning from Stringer:

Q86 Graham Stringer: I find it a bit surprising, that you didn’t ask directly when a lot of the controversy had been about the request to delete e-mails. You didn’t personally ask Professor Jones—it was the 29th, not the 27th; I apologise for that—directly whether he had deleted those emails?

Sir Muir Russell: That would have been saying, “Did you commit a crime?”, and we would have had to go into a completely different area of the relationship and formal role for the inquiry.

But in a surprising new turn of events, it seems that VC Acton sort-of did what Muir Russell was supposed to do – ask Jones whether he had deleted emails. The Guardian reports Acton’s testimony as follows:

Prof Phil Jones told the University of East Anglia’s boss that he did not delete any of the emails that were released from the university last November, despite apparently saying he would in one of those emails.

In the narrowest sense, the very existence of the Climategate emails seems to show that, whatever Jones may or may not have attempted to do, he had not deleted the emails that survived on the back up server.

But, needless to say, you have to watch the pea under the thimble as there is more to the story than this, as I found out last spring.

Jones’ delete-all-emails request was directed particularly at the Wahl-Briffa exchange about IPCC in summer 2006. (In a related emails, Jones said that Briffa should deny the existence of such correspondence to the UEA administration – something that was never investigated as misconduct.)

Wahl’s insertions in the IPCC report – the unilateral changes in assessment that do not appear to have had any third party oversight other than Briffa’s – were made in attachments to his emails to Briffa.

Last spring, I sent an FOI request to the University of East Anglia for the attachments to the Wahl emails that would show precisely what Wahl had inserted. These, of course, are precisely the sort of thing that Muir Russell panel was obligated to examine but didn’t bother.

Contrary to claims by Jones and Acton that nothing had been deleted, the University refused the FOI request on the basis that the attachments had been deleted, that they no longer possessed the attachments to the emails – see previous review here.

In response to my request, they said:

We were unable to provide the following four documents as we had determined that these were no longer held by the University and cited Reg. 12(4)(a):

There is no single repository in which all information is held and in order to determine whether the University holds specific information searches are required in a number of locations. I have reviewed the criteria and searches that were undertaken to locate the requested documents and agree with the assessment that these documents are no longer held and agree that Reg. 12(4)(a) applies in this instance.

Acton tells the Sci Tech Committee that nothing has been deleted, but when asked for the documents that Jones specifically asked to be deleted, the university refuses the FOI request on the basis that they no longer have the documents.

Needless to say, Muir Russell didn’t bother trying to figure out what was going on.
UPDATE 4 pm Eastern:
Here is a rough transcript of part of the relevant exchange:

Stinger – Prof Acton, are you satisfied that these questions weren’t asked? That people in your university were sending out emails suggesting that emails be deleted and that it hasn’t been investigated.

Acton – It has been investigated. I’ve asked them and they’ve assured me that they’ve never knowingly deleted emails subject to [inaudible]

Stringer – Did you ask them under caution?

Acton – I have a rather different relationship rather different. It is part of my duty to address that kind of spirit and make sure that I drive it out and establish the fact. Can those emails be produced? Yes, they can. Did those might have deleted them say they deleted them? NO they say that they did not.

Stringer – and you’ve recorded those meeting with Prof Jones

MR – if you examine our website …

They continued:

Acton – My concern is that they are producible and that they’re there and …
Stringer – Are all the emails now available and can be read.

50 Comments

What kind of inquiry operates on the basis that it cannot consider the possibility of wrong doing!?

Anyway, back in the sane world:

Muir Russell and the University have incremental backup email files for three members of the CRU staff. These might well prove that ill advised email had been deleted and when such deletions occurred. (Through comparison of the backups from two dates).

As detailed here, this is difficult for the University and Muir Russell, because:

1) The University cannot investigate the backups properly for fear of what it might find
2) The University cannot delete the backups – they have used that tactic too many times

Here’s how I think how it worked — they first explained to Phil that they did not want to ask him, nor did they want to see any evidence that could incriminate him, carefully explaining that such evidence would be outside the scope of the inquiry. Then they published the results saying (paraphrasing), ‘we did not see any evidence that would incriminate Phil’. The conclusion is, therefore, factually accurate.

What I thought was a really nice flourish was when Acton asked Phil (paraphrasing) ‘did you delete any of the emails that were not deleted?’

Jones told Acton: “Prof Phil Jones told the University of East Anglia’s boss that he did not delete any of the emails that were released from the university last November”
but if he had already deleted them, they were not available to be released, they were already gone, and if released they clearly had not been deleted. It is like Alice in Wonderland conversation with the caterpillar (if memory serves).

It is impressive that Steve can show on the basis of the UEA’s reluctant communications with him that parts of emails have been deleted (as in this article), but Muir Russell’s inquiry can find ‘no evidence’ of such behavior!

Acton said that 1) Jones and Briffa denied deleting the emails and 2) the emails were available. He did not say whether the emails were available from backup or whether the originals were still in their mailboxes.

“Prof Phil Jones told the University of East Anglia’s boss that he did not delete any of the emails that were released from the university last November, despite apparently saying he would in one of those emails. ”

Of course he didn’t delete any of the emails that were released. He would have deleted ones that weren’t (couldn’t be) released.

The more I read these posts, the more it seems that Steve is playing the part of Sherlock Holmes in these cases and people like Muir Russell are acting like Inspector Clousseau. The analogy is not perfect,however, since it doesn’t take a Sherlock Holmes to figure out what is going on here. Nor is it likely that Muir Russell will ever inadvertantly stumble upon the truth and solve the case in the manner of Inspector Clousseau.

Am I just being naïve to think that any reasonable independently minded person will find doubt about the innocence of the UEA/CRU personnel and ‘the Team’ after even a casual review of: the UEA/CRU email release; the consequential discussions in the blogosphere; all the subsequent investigations and reports?

I don’t know why people go on about Muir Russell because wasn’t he just a management consultant who like every management consultant is paid to write the report the management wants … nothing more, nothing less.

As for the emails – I’m not surprised the climategate home guard at CRU have lost them.

Corporal Jones: “don’t panic, don’t panic” … was caught handing out sausages under the counter and claims he mistook them for coconuts because he lost his glasses!

To my mind, Steve is once again demonstrating the power of the Internet, information technology and a highly informed world citizenry. Steve points out the initial problem, technology provides the evidence and bloggers with expertise in the area of backup technology point out the absurdity of the problem. Muir Russell and Dr Jones don’t understand the technology upon which their assertions are based. But the informed citizenry understand very well indeed that backups on long term storage most probably do exist and that a proper forensic investigation would have discovered precisely which emails were deleted and when such deletions occurred. The various official inquiries that were made and public pronouncements coming from them only reinforce our opinion that white washing of the whole sordid affair began almost as soon as the emails were released into the public domain and at this point, sweeping it all under the rug seems to have been the intent all along and it continues even to this day.

I was just mulling over the subject and got to wondering: “how come this climate unit ended up in an undoubtedly third rate ‘university’ like East Anglia?”

Come on, most people didn’t even realise East Anglia had a university before climategate! And we all know that UK politics is dominated by the oxbridge elite – so why didn’t this (formally) prestige unit get half-inched by the old university cartel?

Was it Deny-ability when it all went wrong?

Something is very odd with the whole set up. This (formally) prestige unit was kept going in some out of the way place whilst the rest of “science” jumped on the global warming bandwagon … but not the key area of climate data. Was there no better university, it’s not as if this department had the brightest sparklers in the box, and almost anywhere could have supplied better staff.

So what was going on if it wasn’t keeping this liability of a unit at arms length?

Re: Mike Haseler (Oct 27 11:18), Originally, under Hubert Lamb, the CRU was just fine and did good work. Then alarmism set in under Maggie Thatcher’s aegis, though curiously, she herself came to realize that there was nothing to be alarmed about. But not before setting in motion this Sorcerer’s Apprentice. How? She cut funding grants to all research, whatever discipline, unless it was investigating “manmade global warming”. And she established the place in Exeter, the Hadley centre.

Why no spotlight there?

Thatcher’s motivators were her personal need to fight the miners and win and make a place in history for herself.

Having now seen the carefully-parsed questions that were put to Prof. Jones, the question it raises in my mind is:

> Were the questions provided to interview subjects at CRU ahead of time ?

I see a question like “did you delete the e-mails that were released ?” and I see a defense attorney’s question. Defense attorneys prepare their witnesses before deposition / trial. Were Jones and company prepared too well ? Carefully-parsed questions work to get obfuscating answers into the record – but only if the witness knows they’re coming, and so answers the carefully-constructed question he’s asked – rather than mistakenly answering the question one would expect to be asked.

You may be able to change the result of a midterm election and perhaps the next climate debate in Mexico. But attacking a man will not change the fact that the world is warming and it is more than likely caused by man. I hope you are correct in your stance for I would not be able to live with myself if I (and watts) have helped to wreck the lives of our children.

“You may be able to change the result of a midterm election and perhaps the next climate debate in Mexico.”

suggests me that you don’t have a strong grasp on the meaning of the word ‘irrelevant.’

I heartily recommend that you consider the consequences of being wrong. And please, save the drama-queenery with respect to the children. It takes an already absurd comment and embellishes it with breathtaking inanity.

Stinger – Prof Acton, are you satisfied that these questions weren’t asked? That people in your university were sending out emails suggesting that emails be deleted and that it hasn’t been investigated.

Acton – It has been investigated. I’ve asked them and they’ve assured me that they’ve never knowingly deleted emails subject to [inaudible]

Stringer – Did you ask them under caution?

Acton – I have a rather different relationship rather different. It is part of my duty to address that kind of spirit and make sure that I drive it out and establish the fact. Can those emails be produced? Yes, they can. Did those might have deleted them say they deleted them? NO they say that they did not.

Stringer – and you’ve recorded those meeting with Prof Jones

Acton – if you examine our website …

They continued:

Acton – My concern is that they are producible and that they’re there and …
Stringer – Are all the emails now available and can be read.

Stringer’s question about Acton asking Jones “under caution” was actually somewhat of a clever trap. Stringer had just finished asking Russell a series of questions on whether he had asked Jones about deleting e-mails and Russell had repeatedly dodged these questions by saying that since such questions might have involved a criminal matter, they would have to have been asked with Jones “under caution,” and he didn’t feel that was, as the English say, his brief. Then, less than a minute later, Acton blandly says, yeah, he asked Jones those questions without cautioning him; thereby taking the wind completely out of Russell’s sails.

I think document management/archive systems can now scan archives across multiple users and copies, deleting duplicate attachments and replacing them with pointers to a unique copy. This keeps archive space down but also means that supporting documents and context are reinstated when archived emails are recalled. Not sure how widespread this application is but I’d expect that it would be relevant to most med. to large institutions.

IF someone finds an multi-meg image on the web of a potato that looks like Elvis and sends it as an attachment to his friends in the university then within a few hours, the Email server could be holding hundreds of megabytes or gigabytes of Elvis the potato data not to mention the Viagra image spam data. Consider the large distribution lists that would be receiving PDF versions of papers with multiple climate images and one can understand how policies must be set up to manage Email storage so that a reasonable budget can be set.

I would not emphasize this issue until I had a copy of the university policy on Email storage. As others have noted, there are products in the market that allow for he orderly management of memory resources. These can range from elaborate mechanisms of transparently moving data to multiple levels of less costly but slower to access storage media to a simple policy of deleting Email attachments in archive. From the university’s response, it appears to me that they use one of the simpler less costly mechanisms

As best I recall, this has not yet been mentioned in the “Delete emails” posts, so here goes: If Jones had indeed a change of heart and decided after all NOT to delete any of the emails to which he alluded, then it makes sense (Is “making sense” relevant here?) that he would email back and say, “Hold it! What was I thinking, asking you to delete emails?? That is not right, and not respectful of you, to ask you to do it. So, if it’s not too late, DON’T DELETE. I repent. I myself am not going to delete after all.” PLUS (additional incrimination), where are the emails from Mann and Wahl to Jones saying that they have too much personal integrity to delete such emails, and suggesting or requesting that Jones not delete either. So all this email silence merely confirms what SM is stating: The material was deleted, apparently by all parties involved.

I am not an information technology (IT) person but I would consider myself an IT expert and know a lot about email systems. The email of all university employees are likely stored on a server in a different building from where the “researchers” are located. If the IT Department are doing their jobs properly they would be creating complete backups periodically and daily incremental backups periodically of their email database. These backups would usually be stored off site to prevent a fire etc.. from destroying them in an accident.

So what should be asked is “what are your backup policies for the university email system?”

If backups are stored somewhere, “do you have the backups of the “researchers” in question and the time period?”

Note, it’s almost immaterial what emails show up the “researchers” computers because email data for a university would be stored on a server and not the “researchers” computers. Even if the researchers delete the emails, the data for the emails still likely exists on the email database until it is overwritten. Certain types of software could undelete these emails if the data still exists in the email database. The researcher’s computers only act as a client and when they open their email a request goes out to the email server to transmit email data back to the “researchers” computers. This would not usually delete it from the server. Now the researchers could move the email off the server to a local email database on their own computers. I would guess the “researchers” did not do this due to lack of knowledge.

Once the backups are restored, you would be able to tell if “researchers” deleted any emails in question. The IT person at the university in charge of the email systems would be the person that could answer all these questions. The university probably even has a formal policy for email backups and retrieval written down somewhere.

A request should be made to the university immediately not to destroy any back tapes etc.. so they don’t have an excuse that the backups no longer exist.

It really comes down to how the email system was set up at the time. There was a time when mail was routed to whatever server was responsible for that given domain; for instance a server per distinct ‘school’ domain within the larger UEA domain could be quite possible. Now whether that particular server just did a hold and forward function with the client or actually ‘held’ the mailbox is the real question. The 1st form would only have basic envelope information in the logs (if they got archived), the second form ‘could’ have long lived backup’s but only if the IT support staff associated with it did that as a matter of course. You then need to depend upon actual end client machine back-ups and hopefully archive anything soon enough before it got wiped by the user.

Nowadays its highly likely everything goes through a central system and gets at least logged if not ‘held’ centrally. More for data privacy and security issues than anything else (i.e. stolen laptop problems).

So there are actually two bits of information per email you might be able to dig out: the original email itself as archived, and/or the envelope information associated with that email – which could be in many places indeed, some totally outside of UEA’s control or held on the senders mail gateway.. tick, tick..

Basically the more they try to hide this particular internet email elephant under the rug in the corner (along with all the other elephants, hippos and confused whales) – the more likely it will be revealed.

Surely the 12 month Norfolk Police investigation has determined the status of all the emails ?
Ask them if the investigation is complete or if they have a status report.
I assume they gained and maintained control over the UEA computer systems and all other computer systems used by UEA staff and relevant to the investigation.
Someone needs to ask the plods what is going on before the 1st anniversary of the disclosure of the Police & NDET investigation.

The police would be investigating the reported crime, and any other potential crimes they uncover during their investigation. So initially, whether it was a hack or leak. They are probably less interested in email deletion as FOI or EIA abuse is managed by the ICO who’ve already said there was prima facie evidence but they can’t prosecute due to the time limits. Legally, FOI issues are I think a bit of a dead end for the UK at least.

There is some information regarding the UEA’s archival policy in the Russell report, and shows it was a bit of a shambles wrt their FOI policy, eg-

“It should be noted that the University FoI team will argue that there original response was correct as they were unaware of the backup server where all these emails were aggregated”

From the interview notes between Norton and the ICO, and the UEA’s standard email archival policy in note between Norton and Iain Reeman. Problem for UEA’s IT people is typical for academia I guess in that CRU did their own thing and some archival was done seperately on a system built by Mike Salmon for them, leading to inconsistent policies for that box. The rest is becoming history and a case study as to why allowing ‘special needs’ can destroy otherwise sensible IS policies.

The police are attempting to determine if emails were stolen or if it was an inside job. They must have some idea as to the scope of the volume of emails sent or stolen. They may well know if all or some of the emails are out in the public realm. If it was hacked did the hacker delete any emails ?
We could think of many questions concerning the issue but the investigation is being ignored by the media.
The question being ‘What do the Norfolk Police and the NDET know and when will they tell the public what they know’
I believe we are entitled to an explanation as to the results of a 12 month investigation and if it will lead to charges. The silence is deafening.

However, backups are almost never kept indefinitely unless there is a legal reason to do so. A common technique is to maintain a cycle of backups to cover a particular period of time. For example, a backup system that would allow retrieving data as it was up to six months ago, say, but wouldn’t allow you to retrieve something that was lost/deleted more than six months ago.

I don’t find it unreasonable that they wouldn’t have email backups to the period in time in question. To ask what is their backup policy is still a good idea, though.

Great site, great thread. Steve McIntyre, you have my thanks and admiration for the terrific work you’ve done and continue to do on behalf of all of us. I am halfway through “The Hockey Stick Illusion” and it is excellent, even indispensable.

Regarding the emails and any deletion: I would second the comment of “JC” above (October 27, 2010 at 4:32 PM). Any thoughtful person trying to figure out what happened, and certainly any computer forensics experts involved in the inquiry (and why were they not?), would take the path suggested by JC. The backups should exist, both for operational reasons and for legal liability reasons (e.g. if work is done under contract, the contract may require that all results be preserved indefinitely or for some audit period). The first question should have been, as JC says, “What is the University’s policy on record retention and data security/backup? Who is the Chief Information Officer with accountability for protecting this asset base?” Because it is part of the institution’s asset base; probably more important, these days, than the bricks and mortar.

I worked for a large US company manufacturing safety critical “stuff”.

The mail policy is every thing.
Theirs was:
The emails are held on the system for 30days and not backed up.
Archiving was to local HDD – not backed up.
All emails archived or not were deleted (moved to deletion folder) after 30 days.
After a further period (30 days?) the deletion folder was cleared – leaving no record.

If you wanted to save them the only option was to convert to text and store as text file on local HDD (the attachments had to be saved separately).

This was a legal thing to safeguard the company against failures in the “stuff” and legal action – there were no uncontrolled copies of dodgy emails.

Loose the password to your email account and the emails were lost.
Loose your HDD and your backups were lost.

As someone pointed out a few 1MB Mr.Potato-head.jpgs gone viral on the email system would soon overload the servers. It is very unlikely that attachments are archived other than on the recipients PCs.

Also to remember is that FOIs that cost too much to service can be denied. Searching ISP mail servers, personal PCs, Memory sticks may hit the limit.

I work with a large company. Due to the nature of my job, the Email policy is that nothing is deleted. Not my Emails, not the attachments, not the massive amounts of SPAM. With my job, my Emails could be demanded in a court case and the lawyers tell me that failure to produce them could result in an “adverse inference” by the judge. For others in that company there is a strict quota on Email storage.

So the conclusion should be that the archving of Emails is a matter of policy and that can be quite different even in one organization

I understand the point that the attachments aren’t saved on the backup server. However, attachments do live on in personal PCs.

If the university says that the attachments to the Wahl-Briffa exchange no longer exist on university computers, the implication is that Briffa deleted the emails from his PC at work. Possibly they were in existence at one point among the emails that he had taken home, but Muir Russell didn’t bother investigating.

Yes, I agree with TAG’s comments. For legal reasons, many corporations and universities (in the U.S. at least) keep long term backups. They usually due this because emails are requested during lawsuits frequently. For example, if one worker is claiming sexual harassment due to the emails they received from another co-worker. If the corporation does not have a copy of the email then a Judge might perceive something sinister or blame the corporation or university for not having an adequate backup policy to protect workers. In the past, tape backups were the choice for most corporations or universities. Tapes are relatively cheap so there isn’t much downside to keep a long term back-up. Over the last couple of years, some companies and universities are starting to use a third-parties to backup their email via the internet. This allows the backups to be stored anywhere around the world and is a another safeguard against local catastrophes, or tapes going bad, etc..

What I don’t know is if the UK legal system almost demands backups like the U.S. system. It could be a totally different environment there.

Couple more thoughts. Concerning attachments. If I send an attachment say to 1000 people at a university, the email server software, if up-to-date, will not store 1000 copies of the attachment on it’s hard disk. Instead the software will be “smart enough” to only save one copy. Everyone’s email will essential have a link to that one file on the server. Also the attachment could be compressed by software. This would be an option for the person in charge of the server. So for example, a large Microsoft Word document would not take up as much space as you think. Pictures on the other hand are hard to compress so compression will not have much impact on those files. But attachments are not the problem they once were concerning storage. This all assumes their email server software is not from say the year 2000. If they have modern software, attachments would be less of a storage problem. Also, the cost of hard drives has come down dramatically over the years.

If the university is obligated to adhere to FOI laws, that almost obligates them to have long-term backups of their email. Their email backup policy should reflect this.

JC, that depends on the mail server. MS Exchange (likely what they have, based solely on market share) will start out the way you describe, but then if one of the recipients deletes their copy of the message, the attachment is actually expanded to each of the remaining recipients. That approach passes the initial RFP for the email purchase, but doesn’t require messy reference counting in the code. If they were using Sun JMS, then there is reference counting for the email (not the attachments), at least for a JMS storage location. Notes mail wouldn’t work the way you describe, unless they installed a third party attachment extraction product.

If they have an archiving system (I doubt it, usually non-regulated, and infrequently sued organizations don’t bother), then the archiving system might have a copy.

The key question as mentioned up thread is what is their backup policy, or more specifically what is their policy for recycling (email server) backup tapes? I would guess around 6 months, but just a WAG, based on the type of IT organization they likely have, or had at the time.

However, if Briffa had copies (especially if that were known in advance), and this was a legal discovery, in the manner of legal discovery in the US, he would have been asked (required is more like it) to make them available. It sure seems like a logical question for Russell to have put to Briffa though.

And ANY of this is important because? I know it’s fun to put on your double billed caps play detective while thinking that you’re really doing something and Lord knows everyone needs a hobby. But please don’t conflate the existence or lack thereof of these emails with anything of importance with respect to the behavior of Earth-ocean-atmosphere dynamics.

What? Deletion of emails is the topic. We were adding our thoughts on the technical details behind deleted emails. PA32R, if you want to opine about earth-ocean-atmosphere dynamics then do it in another topic or perhaps another blog. If he deleted these emails then it shows he has something to hide and perhaps shows somethings about his creditability as a real scientist.

If it’s trivial then find your interest elsewhere. Why even comment at all? If you followed the CRU chronology, then you would know that the interference of CRU with certain publications and the apparent collusion to hide such interference is a corruption of proper scientific behavior. There are plenty of people that think that honesty and credibility are related. Thus, making the deletion of emails quite important to many of us who fear AGW could be real and would like to know if it really is happening or not. Which means we would like to know if the science has been corrupted or not.

I don’t condone what happened at CRU, blowing it up into a question of the science being corrupted is foolish.

Steve: The confidence attached to proxy reconstructions is very suspect and CRU did much to prop up this field, which is only a small part of the overall “big picture” – a point that I’ve made on many occasions. I’ve also suggested that people worried about AGW should be the ones who should be most active in ensuring that inquiries did their job. Their failure to do so unfortunately fosters suspicion in the wider community. I’ve asked readers on many occasions not to extrapolate but the abysmal performance of the inquiries doesn’t help.

[…] But there is more relevent info on the record as well But in a surprising new turn of events, it seems that VC Acton sort-of did what Muir Russell was supposed to do – ask Jones whether he had deleted emails. The Guardian reports Acton’s testimony as follows: […]