Monday, 8 July 2013

The PNAS hotel

The Rat, 2013

Hi rats,

This post is about high impact factor journals, i.e., those
publications where you have to convince an entity called “editor” that your
article is “cool” and will “appeal to a wide audience” before having a chance
to get it reviewed by someone who's not an ignorant. Most of you won’t pass this first filter. That is, unless an individual whom I've never seen in any nonlocality workshop claims in Nature News that your result is “the most important general theorem relating to the foundations of quantum mechanics since Bell’s theorem”.

Hence I will focus my rant on this aspect, the editor stage,
because I find astounding that editors of professional scientific journals regularly
reject submissions on completely unscientific grounds. Whenever I get a good
original result and am able to summarize it in four pages, I know that I will
probably get it published in PRL. I wished I could say that whenever I get an exceptional result it will most likely be
published in Science, Nature or Nature Physics. If that were the case, I
wouldn’t be writing this post.

What’s wrong with high impact factor journals? Is it true
that under their human masks Nature editors are a reptilian species
sent to destroy Earthen civilization by featuring articles like this? Desperate
for answers, I went to the Bristol University Mensa and asked researchers on this matter. I
found that their opinions fall into three categories:

a) The Conformist: “Yes, sure, the system is far from
perfect, but that’s all there is. So stop whining and send those families to
the reeducation cam- sorry, wrong forum.”

b) The Outraged Conformist: “Those journals are a shame and
we should boycott them all. But first let me submit my new paper “If quantum
mechanics were more nonlocal, an avian compass would violate the second law of
thermodynamics in the Canary Islands””.

c) The Understanding: “You have to put yourself in their
shoes… these editors receive a lot of papers every week, they have to come up
with a system to release that load. And the simplest way is to reject all
papers written by junior scientists. Now I have to leave you, that man with the
scars over there says that he wants my “fucking” shirt. Poor soul, he must have
suffered so much. I will offer him my money, my house and my first-born baby”.

Between the lines, one can read some skepticism about the transparency of editorial decisions in high impact factor journals. Let us dig into
this:

In his book “Reason strangled”, chemist and
journalist Carlos Elías argues that Nature’s top-one priority is to maintain
its impact factor; that allows the journal to set the prize of the publication.
Problem is, Nature’s impact factor is already so high that a lot of effort must
go just into not letting it drop. Hence Nature editors have to make sure that
each article will be highly cited. Articles signed by famous scientists are
read more, and, consequently, have more citations. Likewise with prestigious
affiliations, fancy titles, articles already mentioned in the press. Elías
writes about Nature, but I guess that his conclusions apply to other high
impact factor journals as well.

It follows that the paper “Full algorithmic characterization
of LOCC quantum operations”, by John Unknown, from Mac&Cheese Community
College, will have a cold reception. On the other hand, the article “Quantum mechanics
is extra-spicy”, signed by Stephen Hawking, Edward Witten and David Bowie, and
featured on TV by Beakman and the Myth Busters, will make the same Nature
editors twist and shout. Don’t forget that we’re speaking of a journal that in
1996 published an article about the analgesic effects of myrrh. Because
it was a relevant result in the field? No! Because it was Christmas time!

The need to increase the journal’s impact factor explains
many things, but does not answer all my questions. How come that exactly the
same document is called “Supplementary Material” in Science, “Supplementary
Information” in Nature, “Supplemental Information” in PNAS and “Supplemental
Material” in PRL*? Do they want to drive us crazy? Why can’t PNAS
editors read a reference where the author’s initials are - sacrilege! - before the family name? Are they aware
of the amount of time that it takes to submit a paper to this journal, only to
see it rejected the next week? If I conduct the research, write a referee list
and prepare the paper in their damned
unique format, what do the so-called editors do? I mean, besides checking my affiliation and h-index and
replying “we receive more papers than we can publish, so we have to select
those that will be of the greatest interest to a wide audience”. Do they
understand how that sentence feels like when the next day they publish whatever crap with the words “spooky” and “quantum” in the abstract? Come to think of it, why do high
impact factor journals have an impact factor at all? Shouldn’t they be in the
same lot as other popular science magazines, like Scientific American, New Scientist and Physical
Review A**?

*I know, PRL is not a high impact factor journal. But it’s
where most good Physics papers end up after being rejected by the first three,
they should make the transition easier!

**OK, here I went too far.

Epilogue

When I was a young postdoc, within two months, two different
groups proposed two physical principles to limit quantum nonlocality. One group
included important figures in QI who had previously published in Nature. Even
unpublished, their work was soon echoed in the press (what? You never heard of
information causality? How long did you say you stayed in that coma?). The other group was
composed by a relatively unknown postdoc (me) and a second-year PhD student.

My collaborator and I knew that nothing short of building a
time machine would have allowed us to pass the editors of a high impact factor
journal. However, when we heard that the first group had managed to get their
paper accepted in Nature, we thought: “now we have them by the balls”.

On one hand, Nature editors could not claim that our work
wasn’t of general interest, because the same considerations would apply to
information causality. On the other hand, they couldn’t argue that our work
didn’t represent a significant advance, because it advanced the field as much
as information causality did. In sum, the two main arguments for rejection in
Nature didn’t apply. Cowabunga!

Of course, we were assuming that Nature editors can feel
human emotions, like shame. Our article never went to referees. Guess why?
Because “it was not of general interest and did not represent a significant advance”.
A prior submission to Nature Physics had had exactly the same response.

Outraged, I wrote to Nature’s Editor-in-Chief, explaining my
case*. My letter ended with some recommendations for the journal:

[…]

I therefore suggest
you to change the contents of the Nature webpage concerning how to get
published. It will not be so glamorous, but at least it will be honest. It
could start by:

1. It is completely
admissible to exaggerate one’s work to the point that no future research can
compete with your so-claimed results.

2. If you are not a
key figure in your field, stop reading. We are currently working on a webpage
that can only be accessed by scientific celebrities, but, meanwhile, we would
appreciate your cooperation if… wait a moment! Why am I wasting time talking to
a Nobody? Leave! Now!

3. Oh, it’s you again!
Didn’t I tell you to come back when you are famous? Go away, your anonymity
smell is making me dizzy… what? How can you become famous? Err… I don’t know…
By publishing in Nature?

4. Even if your work
is not good enough for Nature, you can still try with Nature Physics. There, an editor with a PhD in experimental ultraviolet LEDs will review your
theoretical paper on Foundational Physics and copy-paste his opinion about it.
In order to guarantee a polite response, we have removed the exclamation sign
from his keyboard.

[…]

And so on. That way,
people will not get the wrong impression that Nature is a scientific journal,
but the nerdy version of “Hello!”.

I think I have already
written enough, and I do not want to waste more of your precious time; you must
be very busy eating young researchers. But do not worry, this is not a “see you
soon”, it is more a “see you never”.

"That is, unless two individuals who haven’t read a paper on Foundations since the advent of pilot-wave theory claim in Nature News that your theorem is “the most important result in the foundations of quantum mechanics”."

First of all, I can confirm that Valentini has read papers on Foundations since the advent of pilot-wave theory. He just doesn't like any of them. You and I may not share his opinions, but his views are not obviously bunk. Secondly, he only claimed that it was the most important result since Bell's theorem, not in foundations as a whole. There is an argument to be made that this is indeed the case, and I would be willing to have that argument with you at some point. It is not as obviously a moronic statement as some of the other stuff you criticize on this blog.

The second individual you are referring to is I guess David Wallace, although if you read his papers it is fairly obvious that he has read a lot of modern stuff about quantum foundations, whether or not you agree with his opinions about it. He only claimed that it was the most important result he had seen in his 15 year career, which is even weaker than Valentini's claim.

I was interviewed for this article and I said it was the most important result in foundations in the last five years. I was worried that I might get ridiculed for being overly hyperbolic, but I guess there is some truth in the idea that you cannot be too hyperbolic if you want to get quoted. Still, I'd much rather be accurate than be quoted.

Secondly, and more importantly, your reference to this result is mistaken because the news article actually hurt this article's chances of appearing in Nature rather than improving them. You can read the full sordid story at http://blogs.discovermagazine.com/cosmicvariance/2012/08/27/guest-post-terry-rudolph-on-nature-versus-nurture/ In fact, if anything, this is yet another tale of the sort of editorial arbitrariness that you are criticising in your blog post.

Finally, I just wanted to point out that there is a strict editorial wall between the "news" section of Nature and the "scientific articles" section, so when you criticize things that appear in the news section you are really criticizing an entirely different magazine from the one that you submit articles to. There is an argument to be had about whether this editorial wall is really a good idea, especially when it seems that whether the news section reports on a preprint can affect the chances of the scientific article itself appearing in Nature. Nevertheless, you can't criticize the scientific editors at Nature on the basis of things that appear in the news section. Both sections have their problems, but you would do better to focus on the editors that are the actual target of your rant.

I agree with some of your points, and have modified the offending paragraph accordingly.

I am still pissed off by Valentini's statement, but not because I do not share his view. When you make claims like: "this is the best work in foundations since Bell's theorem", you should be absolutely sure that you have read all foundational work published since 1964. Otherwise, you may better invoke Wittgenstein's last proposition ("whereof one cannot speak, thereof one must be silent") and shut up.

That said, like you, I enjoyed a lot the PBR paper, and I find the reasons argued by the Nature editor to justify its rejection inadmissible. From Terry's account of the events, though, it is clear that the rejection had nothing to do with Nature News, but with the provocative title of their second article. If anything, the publicity generated by Nature News must have helped the PBR paper to get published in Nat. Phys.

As usual, bitingly sarcastic and enjoyable (I especially liked "Michal Navackowski" ;)). However, what do you *realistically* propose, assuming that a total boycott of such journals by respectable authors is not feasible?

You can start by signing the San Francisco Declaration on Research Assessment (DORA). The purpose of DORA is that research output is evaluated by its content, not by the journal where it gets published.

The moment funding agencies start adopting this policy, that will be the end of high impact factor journals.

I think that you are being too kind to PRL with "... it’s where most good Physics papers end up after being rejected by the first three". Accepted PRLs are often junk, and rejected ones can often enough be good.(e.g. elders of the field sometimes talk about the difficulty of getting foundational work into the prestige journals(including PRL) in the early years). So the problem of unfair and disgraceful editorial decisions is neither new norlimited to Nature or Science. It is sad that we still put up with the system however, and even help perpetuate it.

It would be great to have a mechanism to significantly penalize journals for rejecting good papers (can we give them a "negativeimpact factor" for great papers they rejected 20 years ago, and have turned out to be gems?), but something like thatwould be hard to implement.

I agree: many papers published in PRL are of very bad quality, and PRL sometimes rejects very good stuff. However, this rarely happens at the editor stage.

In my experience, if you truly believe that your paper is good and original, the chances that it gets rejected in PRL are low. Of course, you may bump into a referee that does not understand your point, but that's what replies and appeals are for.

>It would be great to have a mechanism to significantly >penalize journals for rejecting good papers

I think it would be enough to make the correspondence with the editor and referees public. If a rejected paper turned out to be a milestone in the field, it would be hilarious to read the reasons for rejection.

I think you could do something like the negative impact factor on a voluntary basis. I'm shooting from the hip here but... Set up a website and let people "register" their published papers on it in a data base. As part of the registration process they say what journals the paper was rejected from and provide the reviews/editorial letters if they want. If the paper has been published in a peer-reviewed journal they should be required to give the paper's coordinates. With appropriate data mining you could query the data base and ask it all sorts of interesting questions I think. I think it's a great idea. Someone ought to do it!

But I must admit I'd still like to have a paper in nature or science or whatever. I guess it would still make me feel kind of cool despite the above discussion, and most importantly I have the impression it can be useful for my future academic career (if I'll have one).

I think at the moment I am an outraged conformist, but I have the impression most of us are at the moment no?

Thanks so much Miguel for this blog. I always laugh like crazy with each new entry. The not funny part is that, even with my very limited experience in physics, I have personally experienced most of the problems that you humorously denounced. I think this is a tragedy. The meager resources that are put into science are not used efficiently. As you and some of the authors of the comments seem to think, the whole "high impact factor journal" measure of how good an article is may be at the core of it all. And with a bad measure of article quality comes then a bad measure of researchers quality, and the good positions in physics do not go to the best candidates... and so on.

Let us say that we were able to design a different evaluation system. How would you do it ?. Here is my two cents.

In something like the arxiv (for instance, in the arxiv), registered users are able to comment on your article publicly and the comments are signed with real names. You can then respond to the comment. Maybe at the end of the interchange, the initiator of the comment could somehow indicate that he/she is satisfied with the exchange (or changes in the article). All is public. Everyone can see all comments, replies, pending comments, comments on comments ... or whatever. Some metric can then be computed, taking also the number of citations into account.

Physicists do all the important work: Research and Peer review. I am not sure that anything else is needed.

Good point! Also I was thinking, since as you say all the work is done by us (writing the paper and refereeing the paper), why are the editors so goddamn busy that they have to resort to automated computer programs to reply to our mails (assuming what the rat says is true)? Am I being naive? What goes on all day in the offices of scientific journals? I admit my lack of knowledge of the real world (and of physics for that matter), so if any publishing person wants to explain more please do!

Could someone explain what is bad about only publishing in more specialized journals? When I think of physics, Elliott Lieb comes to mind and as far as I can see from his publication lists he does not have any publications in nature. So does that mean that Lieb is a bad physicist, ... that does not sound right...

hotel in hyderabad near airportThis article is superb one so I need much more articles of yours to read. So try to update and try to send me your articles to me. I am so exiting when to read your articles so please update your articles and i am reday to give 99% ratings to your articles. hotel in hyderabad near airport