The policy for the future mission will require the immediate dissemination of data.
“You get a larger community and you get a bigger workforce for free,” he says.
“It’s clear that the more people you get involved, the more support you get.”

How often have climate scientists lamented their inability to share their data, either because evil people might use them “improperly”, or because it is unfair that somebody take advantage of all their hard work.

Is this a problem of climate science alone? Perhaps, it is. How strange.

The nature of the mission is such that the data products can only be generated from an analysis of the full set of data; i.e. distribution of partial sets of data before the complete set is processed is not useful and will be avoided. In particular, all of the data acquired over the duration of the mission will be simultaneously used to calibrate and remove systematic effects. Thus, the time at which the 1-year proprietary period begins can be appropriately defined as the time at which the 9 all-sky channel maps (the main product described above) have reached a level of maturity such that they could be delivered to the community. The exact time when this level is reached will be determined by the PST, but will not be later than 1 year after the spacecraft power has been switched off.

It is worth noting here that the 1-year proprietary period will also be used to prepare the physical means of distribution of the scientific data products to the astronomical community, and the associated documentation (the ?Explanatory Supplement”); given the large amounts of data involved (in particular in the time series) this operation will require a non-negligible amount of time.

The observing program lasts 14 months. The Planck data will be made public two years after completion of the mission, in 2012. Prior to this, during the proprietary period, the data is analyzed and used by Planck Scientists and their collaborators

Larger telescopes, more sophisticated instrumentation and solid state detectors have increased the efficiency of the astronomical observations by several orders of magnitude. All the data collected by these new instruments are produced directly in digital form and stored on computer accessible media. Controlled calibration procedures are applied to the raw data that then fill Science Archives that are readily accessible via the WWW for scientific analysis and research. For most of these Archives, the data they contain are becoming publicly available after a short period of proprietary time.

Most observatories are currently distributing Principal Investigator data to their user communities after the proprietary period has expired. This enables the scientific utilisation of these data for completely different purposes than the ones of the original proposer, and thus optimises the scientific return of observation infrastructure. But the distribution method is quite variable, from distribution on demand to full on-line archives with sophisticated data retrieval tools.

Observatories and organizations such as ESO, ESA, PPARC, CNRS and Jodrell Bank have invested telescope time and financial resources in producing archives with the potential to be reutilized for a variety of scientific purposes. The AVO offers the opportunity to federate these resources and provide astronomers with access to an integrated multi-wavelength repository of data for data mining and enabling new science.

Astronomical archives are potential a valuable teaching tool for school and university students. ESO and ESA have received funding for education purposes through the European Week for Scientific and Technological Culture Program. Through these efforts ESO has realized the magnitude of the potential utilization of archives by teachers and students. By enabling uniform and interactive access to distributed data archives, the AVO will provide an environment in which the public can participate in exploring the digital cosmos and in which teachers can build new educational resources.

A basic issue to be addressed during the initial phases of the AVO design is the uniformity in the quality of the data. Different Archives and, within them, data from different telescopes and instruments, have adopted different quality control procedures and indicators. The issue is particularly acute for the data collected by individual PI (Principal Investigators) that become public after the expiration of the proprietary period. The first step in addressing this issue is a review of the scientific quality of the existing Archives and the drafting of the scientific requirements for the standard quality control procedures to be adopted by Archives that want to be part of the AVO. The study will proceed by defining a set of quality indicators that are suitable for use by automatic querying and browsing systems. In some cases, the production of higher level Archive products (metadata or specific survey products) will be suggested and implemented.

The WOCE Hydrographic Programme Office in Woods Hole was the first WOCE Data Assembly Centre (DAC) to be established (in 1990). Three years on we can look at how well we are doing. WHPO ‘data’ is digital data of either station locations and times or CTD and bottle data. These data with their attendant documentation come from PIs and at the end of the WHPO involvement are transferred to the WHP Special Analysis Centre (SAC) in Hamburg. After the proprietary period is over, data are made available to the WOCE community on an ftp server. The start of the path is the submission to WHPO of data sets and documentation by individual Chief Scientists. The WHPO usually re-formats the data and, in some cases, generates information files that are not otherwise available. Data are then sent out to a Data Quality Evaluator (DQE) for independent assessment of the quality. In the case of some repeat hydrography cruises, we hold a data set until we receive another one from the same group since it is as easy for a DQE to evaluate two data sets in the same region as one. Once the DQE reviews are complete, they are sent to the Chief Scientist, who is asked to revise his data set, (usually only by changing data flags or submitting additional information). The revised data set is re-submitted and, after further checking at the WHPO, is sent on to SAC and made publicly available once the proprietary period of 2 years elapses.

The WOCE data set is the most comprehensive data set ever collected from the global ocean.This site provides access to the final (3rd) Version of the data set collected during the World Ocean Circulation Experiment (1990-1997).

In accordance with NASA policy, all science data from the Hubble Space Telescope is archived with a one-year proprietary period by default. This period may be extended or shortened at the request of the principle investigator (PI) and on approval by the STScI Director’s Office. Calibration data (i.e., data obtained under calibration proposals), by default, carries no proprietary period; neither does engineering data, calibration files (derived from calibration observations), and observatory monitoring data.

In addition to the regular proprietary period, observations by General Observers (GOs) which are found to be duplicates of concurrent observations by a Guaranteed Time Observer (GTO) may be placed under restriction. Data under restriction (or “embargoed”) cannot be distributed to the GO until the restriction expires (usually, when the GTO data goes public).

How easy is it, to get a Proprietary Rights Modification? Not much:

Requests for proprietary rights modifications (extensions, reductions, and/or restrictions) should be sent to the archive hotseat (archive@stsci.edu). The hotseat staff will forward all requests to the appropriate authorizing officials. All requests must be approved by the Director’s Office before being enacted.

While individual images of my target will have some value, the scientific goals of this program are possible only when the entire dataset has been collected. I am now requesting that the proprietary data period be set to run from the completion of the last visit in the set. The timing on this request is important since your answer will greatly impact the upcoming budget submission.

Requests for non-standard proprietary data periods are unusual and are granted only in exceptional circumstances. Such a request should be addressed to Duccio Macchetto as the Associate Director for Science Policy.

on Monday a NASA advisory panel recommended that Kepler be allowed to censor 400 “objects of interest” — presumably good planet candidates — until February 2011, giving the mission team more time to firm up discoveries, rule out false positives and publish. If enacted, the new policy would represent a selective editing of data on the basis of its science content, rather than its quality — unprecedented for such NASA missions.

Many astrophysics programmes allow researchers a proprietary period with the data. For instance, guest observers on the Hubble Space Telescope get exclusive use of their data for a year before public release. But the tradition for NASA Discovery missions — small, principal-investigator-led missions like Kepler — is to make calibrated data available immediately. That policy has already been changed once for Kepler, last year, when the team was given more than a year to pursue confirmations and work out the kinks in its data processing.

Exoplanet astronomers outside the mission, however, are critical of what they say is an overly cautious approach. Scott Gaudi, an astronomer at Ohio State University in Columbus, says external astronomers might help the Kepler team, as it will be unable on its own to follow up and confirm all its candidate planets. “I think Kepler is being far too conservative, and far too closed about what’s going on,” he says, “and I think it’s to the detriment of science”.

[Malcolm] Fridlund, who is ESA’s study scientist for a planned follow-up planet-hunting mission called PLATO (Planetary Transits and Oscillations of Stars), wants to do things differently next time. The policy for the future mission will require the immediate dissemination of data. “You get a larger community and you get a bigger workforce for free,” he says. “It’s clear that the more people you get involved, the more support you get.”

London and Norwich, 26 Nov 2010 (MNN) – In an effort to reposition itself after the Climategate debacle, the University of East Anglian Silly Walks (aka UEA) is launching today an international competition aimed at exposing its true nature in a new spirit of transparency.

However, due to budgetary concerns and in order to lower CO2 emissions, the acronym will have to remain the same. This means the additional challenge for all budding brand experts is to come up with a more meaningful long-form for “UEA”.

Among the proposals received so far:

University of Eventful Archives

University of Expanded Animosity

University of Excised Adjustments

University of Extreme Airs

University of Enough Amusement

University of Effectively-measured-data–changing Attempts

University of Expert Attacks

University of Erased Authority

University of Expanded Alterations

University of Easy Annoyance

ps Sadly, all FOI requests to UEA for details about the above will never be fulfilled, as there was a North Korean student once at the University and any openness would be contrary to Juche.

Pearce was passionate in arguing that ‘Climategate’ was a very human tragedy, in respect of scientists feeling under siege and becoming fiercely defensive – which only spurred on the sceptics, who thought there must be something to hide. But he thought many CRU critics were not sceptics at all: “They are actually data libertarians, rather than climate sceptics, still less climate deniers. It turned into data wars.” Pearce’s conclusion was that at this turning point for climate science, more “candour” was needed from all.

Count me in as Data Libertarian!

“Scientists feeling under siege and becoming fiercely defensive – which only spurred on the sceptics, who thought there must be something to hide“? Just like Mr Bean at the airport then…

And yes, I would recommend medication for anybody still trying to smother FOI and/or in the business of hiding any data directly related to published scientific papers…

Steve obviously read the remarks from last night’s meeting and insisted on speaking from a lectern. This was a good move as it gave him more ‘authority’. And he was (mostly) crisper…making his points more directly. The others spoke while seated.

George Monbiot chaired the meeting and I think he did a fair job of it. He tried hard to be unbiased, and only once or twice strayed into partisan territory. And he managed to keep the speeches and questions mostly to time and to the point

Fred Pearce took a longer perspective than the others. He spoke well and described Climategate as a tragedy rather than a conspiracy…the tragedy being that the CRU guys had adopted siege mentality. Climategate has certainly widened his perspective.

Trevor Davies representing UEA/CRU was appallingly bad. He mouthed platitudes by the shedload, but was unfamiliar with the details of any of the subjects likely to be raised. And was several times embarrassed by doing so. Apart from the fact that he had a sharp suit. I can find nothing positive to say about him. Struck me as a devious smooth cove.

Bob Watson opening remark was that he hadn’t read the e-mails in question. This was a bad mistake – many in the audience were very familiar with them, and not happy to be lectured by somebody who wasn’t. IPCC was imperfect but the best that could be devised 95% of scientists agree…it is now just a risk management exercise. Errors corrected quickly…As good as having Ravendra, but no need for the extra slot at Heathrow for him to land his jet. Very much the Scientific Establishment figure.

Keenan was interested in research fraud and the lack of accountability in science as a whole. He accused Jones of committing fraud, even after being given a chance to withdraw the remark. Davies tried to defend Jones but had no details. Keenan showed a more street-savvy business approach than any of the other participants. I’d like to have heard him at greater length.

Overall conclusion: there was no conclusion. Everybody agreed that openness and transparency were good, that debate should be with all parties and that uncertainties should be made more clear.

But my own view is that the proof of the pudding is in the eating. This one still has legs and will run and run.

Recent public discussion of climate change and summaries and popularizations of the work of CRU and others often contain oversimplifications that omit serious discussion of uncertainties emphasized by the original authors. For example, CRU publications repeatedly emphasize the discrepancy between instrumental and tree-based proxy reconstructions of temperature during the late 20th century, but presentations of this work by the IPCC and others have sometimes neglected to highlight this issue. While we find this regrettable, we could find no such fault with the peer-reviewed papers we examined

The Intergovernmental Panel on Climate Change is the leading body for the assessment of climate change, established by the United Nations Environment Programme (UNEP) and the World Meteorological Organization (WMO) to provide the world with a clear scientific view on the current state of climate change and its potential environmental and socio-economic consequences.

Clearly, it has now been established beyond all doubt that the IPCC has been a failure regarding the provision of “a clear scientific view” on the “peer-reviewed papers” by CRU researchers. Those papers said one thing, the IPCC another.

With Climate Change too serious an issue to be left in oversimplifying hands, the Fifth Assessment Report is unlikely to be any good unless substantial organizational changes are implemented in the IPCC.

Professor Trevor Davies, Pro-Vice Chancellor for Research and Knowledge Transfer of the UEA, quoted yesterday by Willis Eschenbach in a comment to his “Freedom of information, my okole…“:

The University [of East Anglia, home of the CRU] takes its responsibilities under the Freedom of Information Act 2000, Environmental Information Regulations 2004, and the Data Protection Act 1998 very seriously and has, in all cases, handled and responded to requests in accordance with its obligations under each particular piece of legislation.

There are no accounting issues, no trading issues, no reserve issues, no previously unknown problem issues. I think I can honestly say that the company is probably in the strongest and best shape that it has probably ever been in.