Embargos and confidentiality

This post is not about climate science, but rather about the science/media interface.

As many of you may be aware, papers that are scheduled to appear in high-profile journals (Science, Nature and a few others) are often released a few days early to journalists under embargo in order to give them a chance to do a fuller story and talk to more people about it prior to filing. On balance, this works reasonably well, and the press stories that come out are on the whole better for it (there is a bigger issue with the need for a news peg for most science stories that this practice dominates, but that isn’t the focus of this post).

This is both very unusual and, frankly, appalling. Science and science publishing are not a branch of the PR industry but part of an open and self-critical process of inquiry. Authors and scientists who do not want their results discussed and/or criticised should not submit them for publication in the first place. As Koerth-Baker and Zimmer make clear, articles about new papers that are simply reworded official statements are not journalism at all. Indeed, the need for journalists to get an unaffiliated opinion from the author’s peers is an essential check on the occasional tendency of press releases to go over the top when pushing a particular study, and is something that should be happening more, not less.

This post is really just about putting on record that this is not a practice that ‘scientists’ are advocating for and we, like the journalists linked above, would strongly advise that if such conditions are imposed, journalists, journals and editors do not accede to them. That will help resolve the issue of whether ‘no publicity’ is indeed preferable.

Finally, while I am sort of on the topic of science journalism, can’t we please, please, please mandate the citation of the DOI tag in any online story about a new paper? Tracking down the actual paper being described is all-too-often far more difficult than it needs to be. (For example, the doi for the study referred to above took four links and a targeted google search to find. Why?).

Science journalism is going through a tough time at the moment, and practices that take the reporting of science further away from science as a process and more towards the repetition of ‘gee whiz’ factoids, should be resisted by anyone who cares about the public’s engagement with science.

UMmm,,,I can see why…it seems many “journalists” are little more than a public relations arm of a specific industry. Chemical or petroleum or coal industries will use media outlets in the ways they see fit to respond appropriately. It is why so few media companies are independent – it is just business.

Great post and you should know that his kind of stuff is showing up int he normal media world too: re The Arrival Of JK Rowling’s new novel ”The Casual Vacancy” And The Ruthless, Bullying Side Of Publishing:

The arrival of ”The Casual Vacancy”, her first book after the Harry Potter series, has been more remarkable for showing the ruthless, bullying side of publishing that has become all too common. And, given Rowling’s history of litigation, one can only imagine she has done little to discourage it. Katy Guest, a literary editor in the UK, was asked to sign a non-disclosure agreement before her book reviewer could be “hand-delivered” a copy of the book. Embargoes are normal, but within the legalese, Katy Guest found a clause stating that ”even the existence of the agreement could not be mentioned”. A sort of publishing superinjunction! So it’s not just science, it’s all over now. Caveat emptor

Worth noting that it was not the journal that had the embargo or the NDA but rather an interest group. I could not find an embargo policy for the journal on its web site. While the paper may be high profile, the journal is not. It’s impact factor is less than a tenth of that of Nature or Science.

Deep background, anonymous sources, NDAs and such are used to generate mystic in publicity. Reporters ought to be aware of that.

[Response: The authors had to be part of that decision, and that was a serious error of judgement. – gavin]

Thanks for the link, Gavin. Re DOI links–sometimes the problem is with journalists and editors who neglect to add them. But the journals need to get their act together, too. When I embed a DOI link for a Nature paper and then publish it when the embargo lifts, the DOI link typically doesn’t work until hours later. With PNAS, it’s even worse–the embargo lifts on Mondays, but the paper may not be accessible till later in the week. In my experience, PLOS is the best with this–not surprisingly, since they’re focused so much on open access and online-first publishing.

[Response: Carl, thanks. Two points. The DOI will work at some point, and since no-one is going back and editing stories (especially once they are released into the wild), including the correct (but possibly non-functional) DOI in the initial story is still justified. At worst, you are making clear whose problem it is that the paper is not accessible (i.e. not yours). Second, I see absolutely no justification for a mismatch between embargo-lifting dates and online access for a paper. PNAS (in particular) should just get their act together and assign go-live dates and embargos at the same time. It really isn’t that difficult. Pressure from all sides – journalists, scientists and the reading public – should help improve the situation. – gavin]

Working science journalist here. I too would be very, very tempted to walk away from any release given me under those conditions, though it might please me from an intellectual standpoint to find some creative way around it (my maternal grandfather was a scientist himself, and also spent his youth in a Yeshiva, so those in the tribe will appreciate my reasons there :-) ) But jokes aside, I really would have a lot of trouble covering this in a meaningful way, and I don’t recall signing an NDA or even being asked for one, ever. I have been asked not to publish stuff, and generally I respect embargoes unless there’s a terribly compelling reason not to. (Compelling = equals malfeasance like that described here). Frankly, were I writing this I might have put up a blurb that said “you know what, someone asked me to sign an NDA and that’s not cool.”

There are a lot of problems with the whole embargo system, but sometimes I wonder if it isn’t the best of a terrible set. I’d just as soon do without it, but I do understand why it exists, at least from the perspective of the journals and even the scientists involved.

By the way, one reason we don’t always include DOIs is that I, for one, simply link to the paper (without using the DOI notation) since that means absolutely nothing to 90 percent of the people reading it — it’s a random-looking series of numbers that makes people who aren’t familiar with it say “what the hell is that?” An embedded text link is much simpler to deal with.

Another reason is that most folks don’t click through. I know YOU do, or the kinds of people likely to read this blog, but again, for the vast, vast majority it’s simpler to just mention the journal’s name and if someone wants it that badly they will find it — I can’t imagine googling a few search terms, the name of the journal, and the name of the researcher will be insurmountable.

I am not saying you shouldn’t include the DOIs, but I feel like some folks don’t get just how utterly alien and incomprehensible they look to non-scientists.

[Response: Hi Jesse, I’m not trying to state that the DOI itself has to be in the main text – but that either a statement like “the paper” is hyperlinked to dx.doi.org or the doi is given at the end where only the people who know what it is will notice it. All of our doi links use a plugin that allows us to link it to any piece of text at all, and automatically provides the proper citation and link to the article with no extra work on our part. There are lots of solutions to these issues, and since DOI’s are now universal for articles published in the literature, it is time that media organisations found solutions that work for them. You are correct that the amount of link through is small – but it is often the case that the ability to check up on the actual paper lends credibility to the story even if the link is not followed. The cost of including the doi somewhere is so small that regardless of how few will directly benefit, the net affect will be positive, and once you include indirect impacts on public trust, the decision should be a no-brainer. – gavin]

In addition to asking other groups to weigh in on the latest claims, reporters should also check with the scientist to see if his institution’s press group has produced a press release that is faithful to the original work. In my experience, they fairly commonly have not…

I just could not resist:
“can’t we please, please, please mandate the citation of the DOI tag in any online story about a new paper? Tracking down the actual paper being described is all-too-often far more difficult than it needs to be.”
– sometimes it happens to RC as well, which makes tracking down the actual paper … :)

[Response: True, but if you have noticed we have been trying to regularly use the ‘kcite’ plugin (as above) to allow for direct links (via DOI) to the proper reference and journal. We aren’t yet at 100% compliance for new posts (but we are close). – gavin]

Steve Novella and the guys at The Skeptics Guide To The Universe also touched on this on their podcast this week. Journalists do sometimes fall down on the job, but some of the criticism I’ve seen seems a bit unfair, especially that they don’t check the new science release against older peer reviewed papers. Typically, the reporter has a tight deadline of only a couple of days, and will often be working on other stories concurrently. Because they cover so many different fields of science, it is difficult for them to have the type of command of a specific field of science. That makes it sometimes difficult to track down the best relevant information.

That said, yes, sometimes the reporting does seem worse than it should be.

Your post is on the handling of science news and how the outlets for the news ought to do a better job in that regard. My example, like yours, demonstrates exactly how it’s going wrong. If you’d like the communication problem to go away you need to behave more like scientists.

That means accepting critical review as you noted. It also means avoiding sensationalist media, the Huffington Post, if you’re serious about your work.

[Response: Oh please. You might be of the opinion that taking a general discussion about the science/media interface and making snide insinuations about the latest bee-in-the-blogosphere without either naming it nor actually making a point, is some kind of positive contribution to a discussion, but it isn’t. I am not in the least bit interested in you (or anyone else) playing games in the comments. Either make a serious point or take it elsewhere. – gavin]

I disengaged on the other point at your request. You’ll note that out of respect for your edit I didn’t reiterate the “bee-in-the-blogosphere” and the “snide insinuations.”

But I did reiterate the on point criticism of the use of sensationalist media outlets for announcing and discussing supposedly serious science. The general public’s respect for science resides in scientists’ lab coat image. Having one’s photo on the same page as breaking news on Kim Kardashian’s nip slip hardly sets the tone.

Your closing note in this post was, “Science journalism is going through a tough time at the moment, and practices that take the reporting of science further away from science as a process and more towards the repetition of ‘gee whiz’ factoids, should be resisted by anyone who cares about the public’s engagement with science.”

Watts has outdone himself with a post concurring that Science by Press Conference is a Bad Thing :

Over at RealClimate, Dr. Schmidt has written something that I’ve found myself wholly in agreement with – the sad state of science reporting and the use of embargoes that aren’t adhered to by some journalists.

Readers may recall that I’ve made similar complaints for sometime about the sloppy press releases that are often issued from university and organization science departments that don’t even include the title of the paper or a citation, forcing you to go looking for it, sometimes unsuccessfully.

alas, Watts post on the polar vortex, same page, same day , cites only a University of Utah press release to the exclusion of the ( cited) Nature Geophysics paper underlying it.

Dr. Russell Seitz @3:39PM is welcome to point out where the title of the paper on the polar vortex: “A stratospheric connection to Atlantic climate variability” is cited in the official press release from the University of Utah:

Those who restrict views from other scientists have only themselves to blame. A flaw in a soon to be published article can be found by a colleague and make the paper even better. I rather agree that science must be more open, that is what I do with my work.

So I wait for an Arctic sea ice model animation, set 30 days before estimated refreeze in mid September. Day by day displays, with or without temperature colouring. So we can all criticize and find out why sea ice models can’t mimic reality. We offer help, but no one shows. Has this something to do with privacy , property rights? A NDA? Here we are with the greatest sign of Earth climate in transformation seen from space and we wait for the press release!

Slightly OT, but I hope RC authors begin to look more at the bigger picture. Reporting climate science well is not common in most US media outlets. Reporters and publishers are under intense pressure from ownership and advertisers, often in the form of nudges to be sure to interview people like Watts and Lindzen for any story about global warming. There is an infrastructure for this, in the form of right wing think tanks, PR firms, and a few compliant scientists.
On Revkin’s blog, for example, “Ice” turns up about two dozen denier troll commenters.

Maybe climate scientists should be far more proactive, in the form of monitoring and detailed rebuttals. You know enough to realize how much is at stake, and that disinformation in our media is the single biggest factor in America’s laggard or nonexistent climate policies.

By the way one thing I also want to make clear — while I do make an effort to link to stories I find it silly that after a certain point there are always paywalls– another reason I don’t do it sometimes. You get an abstract (Wiley) and a request for $50 and I know I can’t be the only reader who just gets annoyed at that. For instance, I just did a story about a paper that came out a week ago. You can’t get to it without a hefty fee. And the journal (a Wiley product) charges even for stuff that old.

[Response: The paywalls are frustrating, but many people do have access via library services or university accounts, and with more info about the paper, it is sometimes possible to find an un-paywalled version or preprint. I don’t think this justifies not giving a proper citation though – if anything including the doi makes it clearer who is at fault in the restricting access (i.e. not the journalist!). – gavin]

Re paywalled articles it’s quite often the case that a Google scholar search will reveal full text copies squirreled away; search on the article title w/scholar.google, look to the right of the main results.

Try the general term “deep ocean warming” for an example of how much leakage there is.

I have often blessed authors or institutions (e.g. NASA) who go to the trouble of preparing really helpful press releases and background documents. A larg part of my gratitude goes to those who publish closely reasoned accounts or analyses of the papers concerned, bringing out significances, clarifying issues. Prominent among these in my bookmarks are Skeptical Science with their three levels, Real Climate, Tamino and several more.

There is little that is more boring or more esoteric to the layman than a carefully prepared science paper so the interpretive comment will always be required, paywalls or not.

But I can’t help thinking that this kind of “mushroom management” of the press by the researchers isn’t at all justified by fears of persecution from the Ag industry. Several of the RealClimate authors have first-hand experience in dealing with harassment and vexatious campaigns from those acting on behalf of moneyed interests, invested in blocking public discussion of climate science. The fact that RC is coming out against this blatant manipulation of the press is very heartening on the one hand while it also deflates the argument that the NDA was necessary to protect the researchers.

Indeed, most of the venues I’ve read about this from have instead seen a swarm of rabid accusations in the comment threads that those journalists standing up against the practice are really paid industry shills. If they’re critical of this group’s methods, findings, or attempts to ward off expert opinion in the press, then they must be bowing to the Ag industry! Perhaps it’s not the French team that needs to worry about smear campaigns.

Jesse @32:
in my experience, for significant papers >6-12 month old, the chance to find a paper behind pay wall free somewhere else on the internet is very high (>50%, and in the field of climate research I would even suggest >75%)
I usually use this:
at http://www.google.com :
filetype:pdf [doi of the article]
filetype:pdf [title of the article]

– at least one of these 4 searches usually provide a full article, if it exists
– it is quite important to try all 4 strategies, quite often just one or two of these possibilities succeed
– in case of scholar, don’t be put off, if some of the provided links does not work, quite often if 2 or 3 links at different urls do not work the third or fifth will

It has been suggested that advance publicity on the paper could have lead to the publication being cancelled.

Before you all dismiss this possibility with a wave of the hand, I suggest you consider the fact that this is the *first* paper to study toxicological effects of GMOs over the long term (two years); that the regulatory regimes that authorise GMOs for human consumption rely on three-month studies with tiny sample sizes, i.e. seem to be designed to be incapable of detecting adverse effects; and that there is a lot of money riding on regulatory regimes not being made more strict.

Research on the health effects of asbestos was successfully stonewalled for decades, by business interests. Tobacco, likewise. Millions of unnecessary deaths, in each case.

A press embargo on a ground-breaking study : nobody died.

[Response: A press embargo is not the issue – an embargo combined with an NDA to try and skew the media response is. Sure, nobody ‘died’ but science is under pressure as never before from all sorts of directions – industrial and from lobby groups etc. and behaviour that appears to undermine the proper functioning of criticism and discussion has a bigger long term impact than just the reception given to this one study. – gavin]

Gavin : I’m sure you understand the reasoning behind the NDA. It just so happens that the vast majority of the GMO experts who might have been consulted are either directly salaried by, or have their research funded by, “commercial interests”. The problem is not that their criticisms might minimize the impact of the paper — the problem is that their corporate masters might have prevented it from seeing the light of day.

The paper is probably flawed — the study only cost three million euros — but it’s likely that public research will now take up the subject, which has been shamefully ignored.

If it turns out that human consumption of GMOs increases incidence of cancer, and that corporate funding and manipulation of scientists was keeping this possibility under wraps, then I think the question of the questionable ethics of a NDA will be a very minor footnote.

[Response: I have no particular interest in the specifics of the paper at hand – many people far more qualified than me have now commented. But your argument is flawed in many respects. First, two ethical violations do not make a right, so whether an NDA was uniquely required for this paper has nothing to do with the alleged ethical lapses by others. Second, your claim that somehow other scientists would have prevented publication of a paper due to be published in a matter of days because they disagreed with it cannot be taken at face value. Where has such an event ever occurred? And even were such an attempt made, I can guarantee the publicity would have been far more favorable to the authors than this incident has actually engendered. Again, I stress I have no particular interest in this paper, or these authors, but this kind of thing has bigger implications for the science/media interface than just this one case, and that affects far more of us. – gavin]

Embargo and NDA — poor response.
Poor response to very effective suppression of previous research work. The scientists in this area have been badly scared by previous encounters. Look up Quist and Chapela, Syngenta, ‘Bivings Group’ — there’s a decade of this stuff.

“‘… There are some campaigns where it would be undesirable or even disastrous to let the audience know that your organization is directly involved … it is possible to make postings to these outlets that present your position as an uninvolved third party … Perhaps the greatest advantage of viral marketing is that your message is placed into a context where it is more likely to be considered seriously.’http://natureinstitute.org/pub/ic/ic9/maize.htm

The scientists, if they were the ones who came up with this embargo-and-NDA notion, or got convinced to do it by some political spinners, were rather naive — an ‘own goal’ for sure.

I found Monsanto’s response to Séralini et al moved the goalpost from toxicity (which Seralini found) to “he didn’t use the appropriate protocol to test for carcinogenicity” (which Seralini didn’t claim – only “tumors” which is not the same as “cancer” or “metastatic”).

If they had actually sent them an “in yo face” prepublication copy, they could have nipped this in the bud – before the “therefore Seralini is an incompetent actvist scientist” meme had time to spread around the PR denialshpere. Cut the lie off at the knees, giving truth time to put its boots on, e.g., “Some reviewers have raised the issue that this study did not follow OECD cancer test protocols. We only tested for toxicity, following the appropriate protocols. We did observe metastases and adenocarcinomas, and would like to know if Monsanto observed these in any unpublished experiments. This clearly warrants further study with the appropriate protocol. Perhaps these negative reviewers errors were induced by their vested interest in the results.” included in the press release. And perhaps a followup comment about how Monsanto didn’t honor the embargo for the same reasons, if they prematurely retaliate respond.

A similar issue to what is being discussed here is the discussion about “quote approval”, where journalists have been required to agree prior to being allowed to interview political figures that all quotes used in their subsequent stories will need to be approved before the story can be published. Many journalists who have been going along with the idea are having second thoughts now.

Jeremy Peters at the NYTimes wrote an article describing the problem several months ago which touched nerves:

Note what executive editor Jill Abramson put forward as one prime reason for the change: “the practice risks giving readers a mistaken impression that we are ceding too much control over a story to our sources”.

One might ask what’s the difference between an interview “on deep background”, which are still allowed, where no attributed quotes can be published, and an interview where the source speaks freely because they feel comfortable knowing their journalist/adversary can be stopped from using something later.

Is risking leaving an impression of doing something more important than doing something? And, what are they doing?

It seems the NYTimes “just say no” may not be the last word, just as Zimmer’s idea of “walking away” may not prove to be workable. It depends how things evolve.

[Response: Actually, I’m not sure that the journalists are taking the correct path here. One of the most common complaints from scientists about media reports is that their statements were either mis-quoted or used out-of-context. Quote checking is actually very beneficial to preventing that from happening and that greatly improves the process and actual stories. For an example, I was quoted once as invoking the “four dynamics of climate change” – a phrase that I have never uttered, and one that means nothing to anyone. Yet, the journalist obviously heard this and did not feel he could change it (even after I complained) – thus a completely nonsensical quote ends up in the story adding nothing to the discussion. Quote checking would have immediately flagged this as a misquote and something clearer could have been written. This isn’t to say that scientists should have veto power over quotes, but if the goal is to correctly reflect opinions, then quote checking is very useful. – gavin]

The aspect of “quote approval” that journalists are uncomfortable with now is the issue of who controls the news as practices change.

Many were going along with the practice of quote approval saying they were quite comfortable with it. The recent Michael Lewis story on Obama was subject to quote approval, and Lewis stated he was quite comfortable with the process. (Lewis discusses the issue in this NPR interview) But now a question has come up: will these kind of agreements lead to the wrong people controlling what is in the news? Which is why the NYTimes now has issued its just say no policy, and why they justified it by citing the need to be perceived to be in control.

Your issue with “the science/media interface” it seems to me, is the same, i.e. who controls news about science as practices change.

Authors of a paper likely to hit the news had an idea about how to control news coverage. They guessed that journalists, if restricted from showing the paper prior to its publication to other experts, would publish as soon as they could, before contacting those other experts. They guessed that journalists would rather do bad stories than let other journalists get the “scoop” on them while they waited to obtain expert comment, and it seems they were right.

You deplore the authors of the paper. Instead of advising journalists to be aware that this change seems to be aimed at increasing the chances that their stories will serve the narrow interests of certain groups publishing certain papers rather than the interest of the public to be well informed, i.e. the stories will be bad, and that instead they should contact the relevant experts after those experts have had a chance to read the paper in question, you say the journalists should turn down the chance to read embargoed papers if these certain conditions are attached.

You might be right, but one effect might be that a journalist who could otherwise spend some time working with the paper trying to understand it is advised to not get involved with it. Perhaps you think journalists can’t read papers?

I’d put a bit more emphasis on deploring the journalists. What’s wrong with suggesting to them not to publish until they’ve understood more about what they are writing about, on any subject, at any time?

It is regrettable that some science fields have very few publication portals, or excessive domination of outlets by paywalls from which there are exemptions bewtowed only upon known insiders and analysts who write much germane, specialized literature.

I still have quite an inventory of projects related to as-yet unfound original papers. Some of them even government appears to be interested in making enduringly obscure to find online; and a few, perhaps the authors themselves are happy to preserve as scarce, saving ongoing controversies.

“How far should journalists reach in trying to judge when one side is simply wrong on the facts? Media analyst Steven Corneliussen examines a New York Times article that tackled the contentious question….”

OK, I accept that the NDA stuff was a poor response. What would be the correct “media strategy” for a case such as this, given the precedents of successful suppression by vested interests?

The answer to this may depend on the objectives pursued by the media strategy.

My analysis is that the scientific merit of the paper and the political impact of its release need to be examined separately : the media strategy adopted neither strengthens nor weakens the science of the paper. (The fact that a scientist pursues a wider agenda is neither here nor there with respect to the science.)

It seems that the media strategy does not meet with the approval of scientists, independently of their evaluation of the science of the paper. But they are not the target audience.

Anecdotal evidence suggests that the general-public reaction in Europe retains the “GMO danger” meme. The back story of “improper media strategy” escapes the attention of most; or even meets the approval of those who are aware of what the researchers are up against.

My opinion is that the regulatory regime for the approval of GMOs in Europe, which is currently anti-scientific in the extreme, will probably evolve in a positive (more science-based) direction, as a result of this research and its media strategy.

Perhaps there is a media strategy that would have been effective in overcoming suppression, and which would meet the approval of the purists. I’d like to hear about that.

Finally, while I am sort of on the topic of science journalism, can’t we please, please, please mandate the citation of the DOI tag in any online story about a new paper? Tracking down the actual paper being described is all-too-often far more difficult than it needs to be.

I’m part of the moderation team of reddit/r/science, and we struggle with this daily. We have a direct link to the source as a requirement. Unfortunately, this does not always line up with quality of writing – some good stories are ruined by poor citing.

Finally, while I am sort of on the topic of science journalism, can’t we please, please, please mandate the citation of the DOI tag in any online story about a new paper? Tracking down the actual paper being described is all-too-often far more difficult than it needs to be.

I’m part of the moderation team of reddit/r/science, and we struggle with this daily. We have a direct link to the source as a requirement. Unfortunately, this does not always line up with quality of writing – some good articles are ruined by poor citing.