The daily business of science journalism includes getting independent comments on new studies, and (in my opinion) providing those comments is one of the most important ways in which scientists interact with the media.

But from talking to scientists on Twitter, I know that there’s a lot of nervousness about giving comments to journalists. And when I send papers out to people for comments, I often get replies that say, “Sure, but what do you want me to say?”

A straw poll, conducted this morning on Twitter, suggested that people would value a guide on doing this. So, here’s what I am looking for. (To clarify, this isn’t me asking about your own work. It’s me asking you to comment on someone else’s work.)

Let’s start with some assumptions.

These may not be true for all journalists, but they’ll be Standard Operating Procedure for the good ones.

A) I have read the paper that I sent you and understand it (or I’ve talked to the scientists in question and they’ve explained it to me). So you don’t need to explain its contents back to me.

B) I’m coming to you because you are an expert in this field and you know lots of stuff about it that I don’t know. I want you to use that expertise to help me put the research into context for my readers, and to help me point out any flaws and strengths.

C) I genuinely want to know what you make of the paper. I am not just trying to fill my story with a random cutaway quote to make it look like I did my job and asked around.

D) I’m not here to present people with the totality of your views, so what you say will almost certainly end up getting cut and distilled. BUT, I won’t do that in a way that misquotes or misrepresents you. If you say, “I’m fascinated by this approach but I think it has serious flaws”, I won’t cut that to “I’m fascinated”. I’m a journalist; I’m not making a movie poster.

E) All the tips below apply to situations where I email you a paper and ask for comments. If we’re chatting on the phone, it’s my job to guide you through all of this, but it will obviously take less time for both of us if you know what I’m after. And I’m talking about written interviews. Some of these will apply to TV and radio too, but those have very different constraints.

So, here’s what I would find useful:

1) Weaknesses. The most important things you can tell me about a study are its weaknesses. Are there inaccuracies in the paper? Statistical failings? Do you think the conclusions don’t hold water? The last thing I want to do is to credulously cover a weak study. But I don’t work in your field and my bullsh*t detector is probably less finely calibrated than yours. So I’m basically relying on you to help me not mislead my readers. Maybe your comments will persuade me to drop a story because it’s just that bad. Maybe your comments will help me to confront an editor and say: “We shouldn’t cover this story that you seem so insistent on. Look: all these scientists think it’s bunk.”

2) Strengths. But hey, it’s not all doom and gloom! If you’re excited, I want to hear that! Go, science, etc.! But also, tell me the reasons for that excitement. Did they get an unprecedentedly big data set? Some cool new method? An unusual model organism? Innovative technique?

3) Your reaction. When you read the paper, how did it make you feel? Were you excited? Impressed? Overwhelmed by a deep existential malaise?

4) The past. The paper will probably have a paragraph that crushes decades of earlier work. You will know all of that; I won’t have had time to read all those earlier papers. So tell me: How does this new discovery fit with what has come before? Is this based on a radical new approach? A long slog? Something that people in the field have been anticipating? Is it just reinventing the wheel?

5) The present. Have other people found similar things? Contradictory things? Is this one of many such studies, or something truly original? If this is, say, a new approach to fighting malaria, how does it compare to all the other approaches people are investigating?

6) The future. So, new discovery. Great. But what does it mean?Does it change what we knew about X? Does it open up new avenues for investigating Y? Will it lead to treatments or diagnostic tools for Z?

7) Detail. Opinions may differ on this, but I like detail and specifics. People sometimes send me quotes that are paragraphs long and “This is probably much more than you need”. That’s true, but I’d rather know all that stuff and have to condense it into something I can use, than to only have something boring, vanilla, and non-descriptive (see the list below).

8) Simple language, in some cases. Look, I know I’m asking a lot here, and it’s a bit much to expect you to lay out all the strength, weaknesses and context of a study for me and have to worry about jargon while you’re doing it. (Could you also rub my shoulders while you’re at it? Thanks ever so.) Just bear in mind that if something is riddled with jargon, I can paraphrase it but I can’t really quote it. That’s a little riskier for you, because maybe I might inadvertently misinterpret something you say. It’s also less good for me. I want to put your words in quote marks because it can really brighten up a piece.

Note that a lot of this boils down to you telling me something interesting that I couldn’t have predicted. That’s why, when people ask me, “Do you have any specific questions?” the answer is often, “No.” What you have to tell me—what springs into your head—is probably going to be far more interesting that anything I’m expecting you to tell me. Hence, any questions I have will be really broad like, “What does this mean?” or “Do you buy it?” or “How does this fit with other stuff?” or “Science me up, nerd.”

3) And on that note, the world’s most banal quote is:“This research is interesting but more work needs to be done”. It’s everywhere. It had invaded science stories like some linguistic cane toad. Of course, more research needs to be done. Otherwise, y’know, science would stop. But what research? What needs to be done? If you were doing that research, what experiments would you do? And if by “More work needs to be done” you really mean, “…because this impossibly flawed study tells us nowt”, then say that. Other banal quotes include, “We welcome any research that takes us further down the road towards [hand-wavy goal]” or “This adds to our understanding of [thingy]”.

4) Publication politics. “I don’t know why this paper was published in Nature/Science/FEBS Letters” and other such comments are(usually)not useful. My readers don’t really know, or care about, publication hierarchies. “This paper should never have been published” can be useful for indicating strength of opinion, but I’d always want to know specifics about why. Isolated outrage makes for fun quotes, but not informative quotes.

5) Citation politics. “The authors should have cited this paper instead of that paper.” Again, if an entire body of relevant work has been ignored, then let’s talk about that. But I’m not that bothered about whether reference 55 is the wrong reference 55.

And finally, a note on going off-the record.

Going off-the-record isn’t really a formal, enshrined, binding thing, but if you send me off-the-record comments, I won’t use them. However, my soul will ache when I see “This is off-the-record” followed by a long list of flaws and weaknesses and then “And now on-the-record” followed by something banal.

I get it. If you criticise a study, you risk angering colleagues who work in your field—the same people who you meet at conferences and review your papers. I’m not unsympathetic to that. But as I said, critical comments are probably the most useful variety that we get. You’re in a better position to criticise than I am. And it will probably carry more weight for a reader to see those words coming from the mouth of an expert in the field, than from some journalist. Critical comments do carry personal risk, but they also help us to fight credulous and uncritical science reporting.

Fellow journalists may totally disagree with any and all of this, in which case, have your say in the comments.

There are 35 Comments. Add Yours.

TR Gregory
May 22, 2013

Outstanding. Thanks, Ed!

jules
May 22, 2013

I think you might allow scientists to summarise the paper if they want to. Many scientists also do this when they review a paper for peer review. It’s also not actually necessary in that case, as the editor will also have read the paper! But it does at least indicate that the reviewer knows what it is about, and it may also indicate what they think the most important thing in the paper is. Scientists are human too – they may have quite different impresssions…and of course some may not get it at all…

[Hey, there’s no question of “allowing” here. People can do what they like. But my point is that it’s not useful to me. And sometimes, people have *only* sent a summary of the paper… – Ed]

Bill Sutley
May 22, 2013

Great list, Ed. The only area where I would quibble with your take is on point A. I think even a good science writer has to be open to the possibility they may occasionally misinterpret a researcher’s findings. Of course, getting their “take,” as in point C, may address this possibility. But, generally, I don’t mind hearing some affirmation of what I think are the paper’s major points.

One question concerning your methods. I’m inferring from your piece that you are often getting these comments or interviewing these sources via email, yet many journalism students are routinely coached to avoid that in favor of phone or face-to-face interviews. Any thoughts on that disagreement?

[Yes! I thought someone would say that. So, if I’m interviewing someone about their own work, I prefer to use the phone. It gets better results. If I’m getting outside comments about someone *else’s* work, I actually prefer email. I know there’s some debate about this, but I have my reasons. First, it fits better with my need for order. I can schedule my work more effectively. If I have 8 different news stories on the fly at any one time, and I want, say, 3 sources per piece, what I really don’t want to do is to call up 24 people until they pick up.

But mostly, I think that via email, I get more detailed comments that I can actually pore over, digest and understand. I get more stuff about methodological details and specifics about results and context over email. Also, I find that you pick up the phone to someone and they expect *questions*. Whereas, as I said above, I’m firmly of their opinion that their unguided reactions are more useful to me than answers to prepared questions. The quotes may be a little drier, but I think the content does a better service to me and to readers. My two cents, though. I’m sure others find the opposite. – Ed]

Dave Hone
May 22, 2013

I think this is all pretty fair and reasonable Ed. Speaking with my research hat on though, in my experience the problem is often that a) a lot of journalists (yes probably the less good ones, but there are plenty of them too rushed / not experts / not good enough) haven’t read or understood the paper properly, or only want banal quotes and b) some of those same people can and will twist even benign quotes into superlatives.

That means that I suspect researchers who have not done a lot of media stuff, or have not had this list of things pointed out to them up front, or have yet to learn that journalist A knows his stuff and plays fair and B doesn’t, want to go down the banal route. They think it’s what is wanted, or or is the best strategy in the face of likely misquoting / misunderstanding.

Not that I’m sure this is something easily dealt with on either side, but I do know that in the midst of a flurry of media stuff I did for one paper, I was profoundly misquoted by one major source and that lead to me being much more careful around media people I didn’t know after that to avoid it happening again. I suspect I’m far from the only one. It is, on the other hand, generally a joy to chat to people who really do know their stuff and want to do things right.

I guess my one suggestion would be starting off a conversation with a researcher (where neither journalist or research know each other well) with something like “OK so this paper boils down the X likely correlates with Y and implies Z, so what do you think it means for A….” and so shows you do get the paper and don’t need it explained, and points them in the direction you want.

[I sympathise with this and it would be impossible to write a one-size-fits-all guide to interacting with the media that didn’t itself become crushingly banal. This is why I stipulated that this is what works for *me* specifically, and why I wrote the assumptions up front. Which, one might note, are effectively a Guide for Journalist in Approaching Scientists for Comments… – Ed]

Willy van Strien
May 22, 2013

Excellent piece, and I definitely agree!

Wendy
May 22, 2013

While starting a reply with a summary of the paper may not be immediately useful to you, it does serve a purpose – to prove that you are both on the same page. If I start with my email with a paragraph explaining how the study aimed to help kittens climb rainbows and you (and any scientists you consulted) don’t recall any kittens being involved, that means either a) you misunderstood the material or b) I’m not as useful a source as you thought because I have no clue what the study’s actually about, either because I didn’t really read it or because I’m not quite in the same field you think I’m in.

Dave Hone
May 22, 2013

“…This is why I stipulated that this is what works for *me* specifically, and why I wrote the assumptions up front. Which, one might note, are effectively a Guide for Journalist in Approaching Scientists for Comments]”

I don’t disagree. But my thoguths my explain why some people are giving you what you don’t want, when they think it is, thanks to prior bad experiences.

Jan Moren
May 22, 2013

“1) A summary of what the paper showed”

This, I think, is something you’ll have to just put up with. It’s a common aspect of scientific communication, and I’m a little surprised you don’t seem aware of it. People are _not_ trying to tell you what the paper is saying. They are telling you _their understanding_ of what the paper is saying.

Why? Because that way you can see from the start if you and they (and the other people you’ve talked to) are on the same page. If the accounts completely differ, then somebody is getting everything wrong, and you all had better back up a step and make sure you know who and why before you go back to discussing specifics.

You see short forms of this happening even in QA sessions at conferences (“So, you have moltofied the bergiblustic, which gave you two ningimpbups. Now, my question is …”), and long forms during thesis defences and the like. I’d be surprised if you can make people stop doing that for your interviews.

Jonny O
May 22, 2013

” the world’s most banal quote is: ‘This research is interesting but more work needs to be done’. It’s everywhere. It had invaded science stories like some linguistic cane toad. ”

Unfortunately, that’s the way science works. Journalists may like delivering the final say on research or a project, but that quote is the means by which scientists admit that their findings are not the final say – and that future work may actually prove otherwise. At the same time, the quote may – or may not be – a subtle reference to work being done by other scientists that has ties to topic-at-hand, without citing anyone by name.

Jessica Wright
May 22, 2013

This is so great, also as a guide for how to pose questions to my outside sources.

I have one question about off the record, which I’ve been thinking about for a while. Often when there are problems with a study, scientists will refuse to comment. They don’t want to go on record with their concerns and they don’t want to be associated with the study. They may say it’s not worth covering, but won’t say why, which isn’t useful when communicating with an editor. As a journalist with a deadline, I’ll have to talk to the people who will talk to me, who tend to give more banal comments. I know there’s a bigger story there that I’m missing, but I’m sure how to get it.

I think the solution may be to gauge when scientists have those concerns and ask for them off the record, so I, at least, know what’s going on. But how to work that into the piece remains a challenge.

[Yeah, I’d rather know about serious concerns off the record than be completely ignorant of them. Even if I can’t quote someone directly, I may be able to point out flaws myself or at least weigh the structure/tone of the rest of the piece differently – Ed]

John Rennie
May 22, 2013

Jonny O., you’re right that more research always needs to be done. But that truth doesn’t invalidate Ed’s point that it’s still a banal quote. (In fact, it only underscores his point, because if more work always needs to be done, there’s no point in saying it.) Ed pointed out that a sharp journalist can wring a bit more information out of scientists who do want to make this point by asking them to be more specific about what they mean. Would a larger data set be more persuasive? Are the previous conflicting results that need to be overturned more persuasively? Do the results need independent confirmation? Are other labs doing similar work but, according to rumor, getting different results? These kinds of answers, broad as they are, can greatly enrich a story and take it past the banality.

Ben Cairns
May 22, 2013

You obviously don’t want to receive a referee’s report, but there’s a surprising amount of overlap here in what you want and what a journal editor wants when deciding whether to publish in the first place. It is encouraging, because most scientists should have most of the skills needed to form the opinions you are looking for. (Communicating those opinions clearly and concisely may be a separate issue…)

On the “short summary question”, it is often a convenient way to start writing a review (or response to a request for comment) so I’m not surprised you get them, unsolicited. Don’t worry too much that anyone is wasting their time crafting a careful summary solely for your edification.

[“You obviously don’t want to receive a referee’s report…” Actually if I could actually see referee reports for the papers I write about, I would *love* that. But yes, you’re right, I’m not asking people to do anything that’s fundamentally different to what they normally do. That’s part of the reason for writing this post. People get all worked up about the “media” and actually I’m just asking you to be a scientist, but at me, and on the record. – Ed]

Jenny Morber
May 22, 2013

I love this, Ed! So helpful to us journalists as well. I enjoyed, too, reading in the comments the reactions to your points. A wonderfully illuminating post on how (quality) sci comm gets done!

Nick Armenti
May 22, 2013

Ed, you provide important practical advice here for journalists, scientists and readers. As one especially interested in the applied aspects of science I am particularly attracted to the following in your piece:
6) The future. So, new discovery. Great. But what does it mean? ….Will it lead to treatments or diagnostic tools for Z?
This is not what most scientists, and especially most evolutionary scientists, are asking themselves when they investigate an issue and then publish their findings. As one evolutionary scientist informed me recently “my interest is in creating new knowledge. I don’ think about the applied aspects of my research”. I asked him to reconsider this position. In no way do I want to minimize the importance of generating new knowledge. But I ask “for what purpose”? I see the value of knowledge produced by the scientific method decided, ultimately, by its utility value. By ultimately I mean as that science-based knowledge become useful in enhancing the human condition in terms of both survival and reproductive fitness. I would encourage all scientists to consider the utility value of their work and to include in their publications, when allowed, a section on how their findings may be applied to solving problems in living. Yes, avoid jargon when possible as many readers will be turned away by it and will miss the opportunity to learn the fitness benefits of the message being offered. A brief focus on the practical, applied implications of scientific findings by the authors of published scientific articles, I believe, will enable tax payers to support tax funded research more effectively as they will be better informed citizens with regard to how their tax dollars are being used to support useful science.
Nick

[“I see the value of knowledge produced by the scientific method decided, ultimately, by its utility value.” Yeah, with respect, I utterly disagree with this. Many (most?) posts on this blog are about research with no practical applications whatsoever and those tend to be the ones that are most popular with readers. The point about implications is not necessarily about implications for our lives, but implications for our understanding. – Ed]

Shane McKee
May 22, 2013

Ed, this article is interesting, but it does highlight that more research needs to be done into how scientists and journalists interact. A cure is at least five years away.

Essentially, I feel a lot of your problems with how scientists feel about how they’re going to be portrayed would be alleviated if you could reassure them that the interview in its entirety would be available for all to read. Open data applied to journalism just makes sense.

Lisa Grossman
May 22, 2013

Ed, this is great! I’m surprised that you prefer doing your outside comment interviews by email, though. I understand that it’s easier to structure your time that way, but if the commenter misunderstands what you’re looking for, it wastes more time going back and forth (especially across time zones). What do you say in your initial email to get what you’re looking for across? Do you have a list of specific questions you send out?

[So, as I mention in the post, I think that asking a list of specific questions would be counter-productive. My usual wording just says that I’d be interested in their views on the new paper, whether it’s interesting/important, and how it fits with our understanding of [field]. That’s it. If they come back with “What specifically do you want to know about?”, I sometimes throw some of the more generic questions I listed above. There’s very rarely much back and forth. – Ed]

Dyani Lewis
May 22, 2013

Great piece! I will be keeping this article on hand.
In support of your comments-via-email approach, at least you get full thoughts. Too often I’ve had people trail off at the end of what would have been a superb quote. Or they have so many points they want to make that they can’t finish any thought without being distracted by the next, usually mid-sentence. Email forces people to consider what they want to say and make it somewhat comprehensible, and therefore more likely quotable.

Lou Jost
May 22, 2013

Ed, you mentioned that you would love to see the reviewer reports for the articles you write about. Have you tried asking a researcher for the reviewer comments on his or her paper? I bet most scientists would give them to you (and yes, they can be very informative). I can’t think of a reason why they shouldn’t be circulated, at least if they are anonymous.

I’ve shared reviews of my own papers with curious colleagues who asked me about them.

As both a scientist and a science writer, I’ve been told by a lot of my scientist colleagues that they have absolutely no idea what to expect from an interview. Granted, most of my friends are young scientists whose names aren’t yet out there as good candidates for commentary. Tips like this will be especially helpful for PIOs doing media training.

I think it’s also important to emphasize to scientists that they should prepare for an interview and not just expect to wing it. It’s important to have talking points and clear, thought-out opinions of the paper ahead of time, lest the interview get too chatty or lead to poorly-thought-out metaphors.

As a science writer, I find that your point #8 is one of the most important: coaching scientists to tone down the complexity of the language during an interview. I may (or may not) follow what you’re saying since I’ve read the paper, but I’m not going to be able to quote you if you don’t chill out with the 8-syllable words.

Ben Cairns
May 23, 2013

“Actually if I could actually see referee reports for the papers I write about, I would *love* that.”

Of course But, unlike another commenter, I assume that even if someone you asked to comment was a referee, they would be unlikely to tell you.

Do journals with open peer review (e.g. BMC journals) make the reviews available to you ahead of the embargo? I don’t see why they shouldn’t.

Nick Armenti
May 23, 2013

Ed, re- The point about implications is not necessarily about implications for our lives, but implications for our understanding.
Respectfully, I must disagree with the above. The course of science is a path from understanding to prediction to control. Understanding cause and effect is the 1st step leading to the prediction of events. Most scientists stop there and don’t go on to third step when that is required to show how the 1st two steps serve to solve human problems and improve the human condition – a consideration of how scientific findings that generate understanding is academic. I suggest that we move beyond mere understanding to the practical when the opprotunity arises. I agree that the most popular topics here are not related to applied science but I doubt that practical public policy makers and practitioners in the general public (as I) are your predominate readers. Of course, I may be wrong about this.
Nick

Claudiu Bandea
May 23, 2013

Ed,
While you are at it, it would be great if could expand on your piece, using some of the ideas in the comments, and produce ‘A Guide for Scientists on How to Conduct Research and Write Science Papers’ and ‘A Guide on How to Read and Evaluate Science Papers’. I think these guides are badly needed and their value priceless!

Ron Winslow
May 23, 2013

Excellent post, Ed, and equally applicable to “applied” science
– say, clinical trials, or to papers on the mating strategies of frogs. It’s also full of insight for both journalists and scientists. Would be a great topic for a session at AAAS or NASW.
Ron

Nick armenti
May 23, 2013

Ed, I’d be remiss if I didn’t say thank you for what your doing here. It’s a significant icontribution in that it addresses the clear need to make more effective the communication of scientific findings to a wide or wider audience.
Nick

[Cheers! – Ed]

akh
May 23, 2013

So how much do you pay for these opinions?

Dan Fagin
May 23, 2013

Well done, Ed. A few more key things I always ask expert sources: “Who else do you think I should speak to about this? What else should I read? Who knows even more than you do on this topic? Who’s doing the most interesting/important work?

Lou Jost
May 23, 2013

Ben Cairns wrote “unlike another commenter, I assume that even if someone you asked to comment was a referee, they would be unlikely to tell you.” As the “other commenter”, I just want to note that I said the researcher who wrote the article under discussion would be happy to provide the referee reports they received on that article. Referees themselves could not give their reports without losing anonymity.

F.J. Rodriguez-Macias
May 28, 2013

I haven’t been interviewed regarding articles in my field but I find this informative in case I ever get the chance.

May I suggest that instead of telling this to an editor “We shouldn’t cover this story that you seem so insistent on. Look: all these scientists think it’s bunk.”
You tell them instead:
“We should really cover this story you insisted, but not for the reasons you thought. We should cover it because all these scientists think it’s bunk and if it gets covered uncritically by other science reporters we have a chance to provide a much needed critical voice to counteract the hype”?

Critical coverage is sorely needed sometimes.
[Yes! I sometimes cover stories for this, but usually if something is wrong in an interesting or instructive way. Sometimes, it’s better to starve something of the oxygen of publicity rather than to fan the flames with a rebuttal. Killing a story is still one of the most important things we get to do sometimes. – E]

Agree with Ron Winslow that NASW or AHCJ could have a lively session. BUT, I propose it be Ed interviewing an actor who pretends to be a scientist doing all the “please don’ts” that Ed outlines. The audience gets to see him deftly use his strategy to win the decent quotes. Superpower.

Bobby Ramakant CNS
June 10, 2013

This is a very useful writing, especially for less experienced science and health writers. We learnt most of the points made above the hard way, unlearnt a lot too, the harder way… We will share this link with Health Writers’ Network we facilitate connecting health writers in Asia and Africa, warm wishes, and thanks again, b

Ian
February 14, 2014

Thanks for this informative article! I’m an early career scientist and have talked to reporters from time to time, and it makes me totally nervous. Plus, with all the education I’ve had about doing science (including plenty of education, in one form or another, on presenting my work in talks and papers), I haven’t had any education on talking with reporters. I’m most interested in the situation where I’m discussing my own work with reporters. So if you’re compiling a list of articles that people have asked you to write, perhaps you could add to it a companion to this piece on how you think scientists should talk to journalists about their own work. I’d be most interested to read it!

Michelle
February 2, 2015

This is a wonderful article. But as I’m sure you know, not all journalists are as scrupulous, careful, and interested in the real science as you are. Some journalists I’ve spoken with DO seem to be writing movie posters. Thank you for setting a better example! On another note, I am going to share with my students the advice about not using “the world’s most banal quote”. I see this in student writing / scientific writing all the time.

About Phenomena

Phenomena is a gathering of spirited science writers who take delight in the new, the strange, the beautiful and awe-inspiring details of our world. Phenomena is hosted by Jamie Shreeve, Executive Editor for Science at National Geographic magazine, who invites you to join the conversation. Follow on Twitter at @ngphenomena.

RSS

Posting Rules

Opinions expressed in blogs are those of the blogger and/or the blogger's organization, and not necessarily those of the National Geographic Society. Bloggers and commenters are required to observe National Geographic's community rules. Contact Info