So How'd They Do That?

Question
One of my follow-up FOI questions on Oct 31, 2008 about the gridded Briffa et al MXD data was the following:

I examined Gridbox 7(132.5E 72.5N) in more detail. It contains one series: omoloyla. The gridded series (#7) has values from 1400-1991, but the underlying omoloyla chronology at ITRDB only goes from 1496 to 1991 and the underlying measurement data for omoloyla at ITRDB only has values for the period 1496-1991. Please provide any manuals, computer code, documents or correspondence explaining how the values from 1400-1495 were obtained. If a different omoloyla data set was used for this study than the data set at ITRDB, would you please advise and provide the data actually used in this study. If there are similar discrepancies for other sites, would you please provide a listing of sites for which the version used differs from the ITRDB version.

The reason for asking for manuals, documents or correspondence is this: straightforward questions to the Team don’t get answers. If they don’t want to answer straightforward questions, I’ll do the next thing: go to FOI requests. But under FOI, I’m entitled to ask for documents, not for answers. So I ask for documents, such as manuals. Having refused to answer questions, the Team now takes offence at being asked for documents. The form of request is not onerous: no business accountant would blink at being asked such a question. However, Phil Jones wrote to the Santer 17 complaining as follows:

He now wants to know why some individual series were excluded from the chronologies and why some chronologies were excluded in subsequent analyses. This time they have asked for manuals, computer code and correspondence explaining the exclusions! It seems neverending. If they just did some paleo fieldwork with trees, corals, sediment cores they might understand why some samples are excluded.

As it happens, I have experience in mining exploration programs and I can assure Phil Jones that, contray to this experience enabling me to “understand why some samples are excluded”, it gives me exactly the opposite perspective. It makes it virtually impossible for me to think up valid explanations for “excluding” some samples. It’s illegal in the businesses that I know.

Anyhow, CRU answered as follows:

We have checked our files and no manuals, computer code, documents or correspondence are available. We can confirm, however, that we did not use a different Omoloyla data set and therefore there is no further data to provide.

Briffa has published over 8 articles on his MXD data and yet they have the nerve to say that “no manuals, computer code, documents or correspondence are available”.

Their website has values for Gridbox 7 from 1400-1495 even though there are no measurements from 1400-1495. But they have no idea how these numbers were calculated. And they don’t have any “manuals, computer code, documents or correspondence” explaining how they got these values. Did they just make the numbers up?

And the climate science community gets mad at me for criticizing this crap, rather than the people who deposited it.

Perhaps it’s worth contacting the journals which published these articles pointing out it has become apparant that not only is there no data for series for this period, but there is also no documented way by which it could have got there.

“The authors were supported by the UK Natural Environment Research Council (GR3/12107) and by the U.S. Department of Energy (DE-FG02-98ER62601). Additional support came from NOAA’s Office of Global Programs and the DOE’s Office of Energy Research”

Steve. Did your FOI request mention these programs by name and did your request go to all these agencies?

If they just did some paleo fieldwork with trees, corals, sediment cores they might understand why some samples are excluded.

Wouldn’t Jones’ conjecture here be rebutted by McIntyre’s Starbucks hypothesis with the recent empirical demonstration that it is possible to visit Starbuck’s and do “paleo fieldwork with trees” on the same day and still have time for a good lunch.

What are the repercussions of not complying with a Freedom of Information request? I can’t imagine that it’s a criminal matter. Is the problem just referred to higher bosses? Do I need to write my congressman?

I’m very cynic toward climate scientist. If they are so certain of their science they wouldn’t feel threaten by being audited. They would know that they can defend their statement, and nothing else than minor errors would be found.

Instead by refusing to answer or provide the requested materials, they only lead to the the conclusion that they know their work is tragically flawed, either intentionally or not.

Climate science is very sick, since such resistance behavior are valorized by the the scientific community.

Steve’s got them by the short hairs and they’re squealing… LOL What a bunch of losers. The more it’s revealed their methodology stinks the more is revealed they’re making all this up. It’s going to come back and bite them in the ass. Virtual reality has no chance against the real world.

We have checked our files and no manuals, computer code, documents or correspondence are available. We can confirm, however, that we did not use a different Omoloyla data set and therefore there is no further data to provide.

They must have learnt it from Nixon: when questioned, stonewall! What they forget is the results of such behavior.

When a federal agency refuses to comply with a valid FOI request, the requesting party must sue to compel disclosure. If successful, the court can award reasonable attorneys fees and costs of the suit. The court may also impose sanctions upon the non-complying agency.

This comment applies to this post and many others. What climatic data can be trusted to be “good”? What analytical techniques can be considered “good”? It seems that at every level, from the individuals on the “team” to the IPCC, lousy data is manipulated with equally lousy mathematical techniques to come up with questionable results that the world is supposed to take as the revealed truth. Why should anything involving global warming/climate change/chaos be believed? Why is it so difficult to get any cooperation from these people regarding data or methods?

A few things about FOIA requests that I’m sure Steve knows, but many of us may not:

“Finally, the U.S. FOIA process will now look more like that of Mexico and other countries that provide individuals with an alternative to litigation in the event of disputes with government agencies over information requests. The OPEN Government Act will create a new ombuds office at the National Archives and Records Administration (to be named the Office of Government Information Services) to mediate conflicts between agencies and requesters and review FOIA compliance at all federal agencies.”

Apparently, this special office got moved by the current adminstration to fall under the Justice Department, exactly what the authors of the bill creating it were trying to prevent.

Thanks again Steve, for taking the endless hours to demonstrate for the rest of us what is really going in in this broken science. Without your blog there would be skeptics but now there are informed skeptics.

— no files is pure methane producing bovine scatology. I don’t mind mistakes in the data but I hate being lied to.

I’ve made a prior request to DOE for Jones (temperature) data which they funded. DOE said that they did not have themselves any data and that Jones was not obliged under his contract to give them any data, so there was nothing they could do under the legislation.

If their purpose is to be uncooperative, then that’s probably true. IF they wanted to make Jones disgorge the data, I’m sure that they could do that too. But my view was that it was not worth pursuing.

If you only knew more about trees and coral and mud and core samples and stuff, you would truly know that you are in way over your heads. Paleoclimatology is really really hard, so quit asking how we do it. You will never figure it out. We know what answers we are after and we are going to get then, so we don’t need you poking around.

#22. Yeah. Rutherford et al 2005 was a joint venture between the CRU group and the Mann group. My guess is that the early data came from Mannian infilling by the Mann operation and it’s possible that the Briffa operation doesn’t know what Mann did. I don’t know at this time whether the calculation was done at CRU or in Mann-world. So this surmise might be completely wrong.

#15 and #22; careful planning pre-request and not taking anything short of complete compliance with a request is what is required. We should be able to get our hands on some data or start to get some court decisions that are unfavorable to those that fail to give up the information. And with that some press about those that hid the data. We are not talking about state’s secrets.

This a rather categorical denial. Could some group of structural engineers be able to claim the same for some high rise? The structure was peer reviewed by the local building officials. Could the structural guys be able to burn their calcs up approval of the local authorities? The local building officials (comparable here to journals) apparently are of no help with the archiving of calculations and methods.

This is where the wheels would/should fall off. I would think climate science would be the purest form of civil pursuit.

To hide, obfuscate verification and make a childish mess of the project archive speaks volumes and should resonate loudly among peers (peer review). I know that I couldn’t recommend a paper from a close colleague who couldn’t provide a set of calcs.

The point being that I could never submit any calcs if I didn’t provide a complete and professional copy to anyone who wanted them.

The idea of withholding data and methods from those who fund our research is still very foreign to me. I would think that if I work for the public, in a non classified position, I should bend over backwards for the public.

It has to stink, or at least as smell bad as a freshly dead skunk, that climate science, the purest of civil responsibilities, has been demeaned to this extent.

I am embarrassed for my country, and I apologize. I didn’t know it was this bad.

One could contend that this is only a UK thing. Unfortunately the record indicates otherwise.

Your next step would be to ask for an internal review by the Freedom of Information Officer – if this is denied (or not to your liking) then an appeal to the Information Commissioner will help get the information (although these can take years due to the backlog).

I would imagine that this would come under the stricter Environmental Information Regulations; however it is up to the Freedom of Information Officer to decide on which rules should be followed.

I should also note that you can make anonymous requests (or requests under a pseudonym) if you think that they may be being purposely obtuse due to who you are (the Freedom of Information Act simply asks for a name and an address (which can be an email address) not YOUR name and address! If you (or any of your readers) want to request information anonymously then this website makes things easy for you!

The idea of withholding data and methods from those who fund our research is still very foreign to me. I would think that if I work for the public, in a non classified position, I should bend over backwards for the public.

I still don’t get it. The data from 1400-1495 exists in some form or they would not have published it. Even if they made the data up from whole cloth, the values exist somewhere. What is the provenance of those values? They are saying they do not have any “manuals, computer code, documents or correspondence” but could provenance exist in some other form? I suppose “documents” would include computer files. I do not know what other form it could take. How can they say they have no provenance for the data with their face hanging out like that?

We have checked our files and no manuals, computer code, documents or correspondence are available.

Implicit is the inference that the respondent is telling the truth. However, the answer is vague – “our files” could refer to files belonging to anyone associated with the respondent, “available” could mean immediately available… not, for example, backed up to DVD and hidden in a cupboard in the toilet at home.

If there’s nothing to hide, why is there such a problem with sharing the information? Do people not understand that the truth will always come out in the end? They must be Visual Basic programmers…

Steady on JL :-), everyone who reads this blog knows that the team are in fact Fortran and latterly Python programmers so stop giving VB programmers a bad name by your misplaced incorrect association with the Team.

Sorry Kevin – wasn’t trying to besmirch the good name of VB programmers (that would include me, you see :). Merely referring to the old “Shooting Yourself in the Foot” with various programming languages; the thing is that in VB you can shoot yourself in the foot repeatedly, but you’re having so much fun you don’t care… 😉

Why not just ask for the files? You should be able to get the whole shooting match that way. So if there are no files, manuals,computer code, documents, correspondence or storage cds, someone is telling porkies.
In my dealings with FOI in Australia you really want to know how the public servants store their data.

We need someone who can tip Steve McIntyre off to the Asset ID of the external hard disk on which all the secret decoder files are loaded. S/he would perhaps be the brilliant statistician who should have been the Santer 18th or the “x” in MBHx. CA needs an Anonymous Tip email address…

Pretty soon we could start a new bestselling fiction genre. I anticipate titltes like: The Spy Who Came In From the Heat.

Under UK FOI law, you can contact the FOIA officer of the agency and that officer is required to help you frame the request for the information so it is complete and doesn’t violate exemptions. I believe an organization, foia.org.uk, may be able to help as well. They published guides back when the UK and Scotland passed their FOIAs. There is also an environmental information regulation that is even stricter regarding reports, studies and the likes if a link is demonstrated to emissions or environmental matters. Commercial confidentiality doesn’t count. Steve may have done all of this already, but I thought I’d just pass it along in case.

It is possible that their work is created by colliding data and methods with antidata and antimethods. All that would be left from the collisions would be the published papers with all the original data and methods destroyed in the process.

We have checked our files and no manuals, computer code, documents or correspondence are available. We can confirm, however, that we did not use a different Omoloyla data set and therefore there is no further data to provide.

Does that mean the requested data does not exist in their files, or that it merely is not “available” for the asking?

I would ask them that question directly in order to get the answer on the record.

Skeptics who express a mixture of amazement, fascination and curiosity about the program wonder,
first, about the relative complete lack of volatility in the reported monthly returns…experts ask why no one has been able to duplicate similar returns using the strategy

[To] those who express astonishment at the firm’s ability in those areas, Madoff points to long experience, excellent technology that provides superb and low-cost execution capabilities, good proprietary stock and options pricing models, well-established infrastructure, market making ability and market intelligence derived from the massive amount of order flow it handles each day. The strategy and trading, he says, are done mostly by signals from a proprietary “black box” system that allows for human intervention to take into account the “gut feel” of the firm’s professionals.

Still, when the many expert skeptics were asked by MAR/Hedge to respond to the explanations about the funds, the strategy and the consistently low volatility returns, most continued to express bewilderment and indicated they were still grappling to understand how such results have been achieved for so long. Madoff, who believes that he deserves “some credibility as a trader for 40 years,” says: “The strategy is the strategy and the returns are the returns.” He suggests that those who believe there is something more to it and are seeking an answer beyond that are wasting their time.

As for the specifics of how the firm manages risk and limits the market impact of moving so much
capital in and out of positions, Madoff responds first by saying, “I’m not interested in educating the world on our strategy, and I won’t get into the nuances of how we manage risk.” He reiterates the undisputed strengths and advantages the firm’s operations provide that make it possible.

High return, over a relatively short period of time, implies a high risk investment. A secret algorithm implies more risk and possible fraud.

I have heard that the investors were chided to invest by the mantra, not that were not making money on their investments, but that they were not making the highest returns.

The psychology of a consensus on this investment was a driver over those of skeptics’ warnings. While fraud is bad and needs too be punished, I somehow cannot bring forth much sympathy for the dumb asses with at least a million dollars to throw away that failed to do any due diligence.

Our Social Security and government health care systems have been referred to as Ponzi schemes, but here the evidence is there for all to see. But do people see or even want to see?

Ross:
The parallel WSJ article mentions the plaintive voice of one skeptic who no one would listen to. He proved to be correct, while smart guys running hedge funds got smashed. Most of the time, if it is too good to be true – it is too good to be true. Me, I have been largely in cash since the Dow went through 11000 in 2007. I sure felt foolish when it went to 14000 – but now I do not feel so bad. Between the lack of economic fundamentals and the plain excesses of many in the financial community, I am afraid that we are going to be in for a very long and difficult period.

[Indeed, says Madoff, the firm itself has received numerous buyout offers but has so far refused any
entreaties because he and the many members of his immediate and extended family who work there
continue to enjoy what they do and the independence it allows and have no desire to work for someone
else.]