Comments

Can I suggest an extension? Writing comments papers ought to be an important part of the post-publication review process, so that where a high-impact paper is found to be flawed, there is a peer-reviewed comment that explains the errors. It would be very helpful if PubPeer.com could also facilitate crowd-sourcing comments papers (which are time consuming to write). Having a forum where the comment could be written collaboratively in the open, where the author of the original paper could engage with the process, would avoid duplication of effort and for papers on contentious topics (such as climate change) demonstrate that the comment paper was scientifically motivated and that the original author had been given an opportunity to put forward their objections before the comment were submitted.

“First it requires participation by a large part of a given scientific community so that it reflects an average impression instead of an outlier’s impression. Second, it requires that the collective knowledge is centralized and easy to search in order find out what the community collectively thinks about an individual paper or a body of work.

Average impression and collective thinking – the paradigms for climate science?

I first ran into the PubPeer site a year ago. It looks like it is mainly for medical research. To register, you apparently have to have a paper with your name in the database, i.e. the qualifications to be a peer. Unfortunately, when I put my last name in the registration box, the only response is a paper by my MD sister-in-law. So, in effect, not many people can get in there to boot-strap the database away from a medical research focus.

The PeerPub people want comments so that is my 2 cents worth. I would have been using it a year ago, if I could figure out how to register as a peer.

Here’s a hint. Any one of us can be wrong. Einstein was wrong about the cosmological constant…twice. He was wrong about quantum theory. And yet somehow, science has marched on around him. Now how do you suppose that happened.

I have nothing but applause for post publication peer review. But pre publication peer-crowd review, as in Arxiv in the case of Physics, is also a great thing. Circulation of unpublished papers and a system for submitting comments about them allow authors to discover errors and shortcomings, to collect suggestions for improvement, and to have early warning about dissenting views. All do circulate their working papers with some colleagues and friends, but wider consultation with the scientific community would be much better, and would also facilitate the process of publication. Peer review is undoubtedly evolving in the Age of Information, and it is a wonderful thing to watch.

The biggest negative of this is that it’s taken 40 after the initial development of the Internet and 20 years after the introduction of the World Wide Web for this sort of forum to come into existence. For all of the hype in tech circles about ‘innovation’. (And I am in such circles) it’s these sorts of extremely obvious initiatives that need to be undertaken to advance humanity, not yet another way to combine micro blogging, personal photos and mobile devices. (not that there’s anything wrong with personal photos. But the disparity in resources put into such commercial activities versus those put into the basic ‘infrastructure of human understanding’ is orders of magnitude. And the triviality of many of the developments are disguised by the immense amounts of money involved and a non-critical group-think of buzwords, such as the above mentioned ‘innovation’.

I have a handful of published papers but none are in the database when I search for the DOI’s. This is because my papers were not published through PubMed but through Springer. My understanding is that PubPeer is predominately for medical papers with a vey limited number of exceptions. My field is Plant Pathology. It’s a shame PubPeer doesn’t have a wider usefulness because it’s a good idea.

Having been involved in a group reply, allow Eli to say that while it was fun, there were issues, mainly it was very hard to get everyone to speak with one voice, so group replies are going to be difficult.

Other than that, getting a better search mechanism as some have said (e.g. by field, etc.) would help a lot.

Related but on a side note, NPR recently did a big story on the new literary term of CLI FI, for climate fiction, which I coined in 2007 during my work with polar cities, which still goes on. I recommend you google “NPR + Judith Curry + CLI FI” to see how NPR introduced this new subgenre of sci fi to the world. COOL! or hot?

Thank you all for checking out PubPeer and for your suggestions. We really hope that it is used and that it helps the community.

Dikran Marsupial – Having a forum where the comment could be written collaboratively in the open, where the author of the original paper could engage with the process, …

We hope that this will be one of the main uses of PubPeer. Realtime review involving authors is exactly what we wanted.

We would also love to see it used in graduate seminar courses where literature is reviewed by graduate students. It could be an excellent way to engage students by showing them that their literature review is taken seriously by the community as well as the authors.

H.M. – But pre publication peer-crowd review, as in Arxiv in the case of Physics, is also a great thing.

We agree. We have implemented arXiv references into PubPeer as well. Anything in the arXiv can be commented on PubPeer as well. We will roll that out in the coming days.

John P. Reisman (OSS Foundation) – I tried a quick search. One might consider categorization for discipline or subject matter as well as other filtering aspects that are relevant?

We are trying to accomplish this via search. e.g. typing “climate” into the search bar will display all commented papers containing that category. This avoids the problem of requiring us to also subcategorize each comment (e.g. “climate” AND “trees”). The user can decide for themselves which categories to filter based on search terms.

Mike – I have a handful of published papers but none are in the database when I search for the DOI’s. This is because my papers were not published through PubMed but through Springer.

Before this post went up on realclimate we worked hard on making sure we have every article published with a DOI. However, we apparently didn’t make this obvious enough and will fix that soon. By pasting the DOI of any article ever published into the search bar you should be able to find your articles no matter what the discipline. After this first DOI discovery the article will then be searchable/sortable as described just above. Please let us know if this is not working for you.

WebHubTelescope – To register, you apparently have to have a paper with your name in the database, i.e. the qualifications to be a peer. Unfortunately, when I put my last name in the registration box, the only response is a paper by my MD sister-in-law. So, in effect, not many people can get in there to boot-strap the database away from a medical research focus.

This is a very good point. Thank you for pointing it out. We will immediately make it so that DOIs can also be used for account creation. This will make the search for your articles more specific and extend the account creation to ANY published article (outside the biomed disciplines). Check back soon.

I’m not sure how revolutionary this is. If a paper is really useful, it’s results and methods will disseminate pretty quickly into the community. This is just doing on line what previously got done by coffee urns and at conferences.

“I’m not sure how revolutionary this is. If a paper is really useful, it’s results and methods will disseminate pretty quickly into the community. This is just doing on line what previously got done by coffee urns and at conferences.”

This is kind of the point of PubPeer- it’s the internet version of a journal club. Now, I don’t think that it will fully replace the need for having conferences and face-to-face interactions; esp. as science gets more and more complex. What PubPeer will do is put the community reaction into the record, and make it easy for policy makers to see what peers think of a particular article. As one might imagine, there is some resistance to the idea of “losing control of the conversation”.

I do not see PubPeer as a loss of control, I see it as science correcting itself and returning power into the hands of the community from that of the “leadership”. Recall that the “leaders” in science are usually just the folks who are better at talking to non-experts than other folks. The leaders were never indented to control a field, they were intended to help explain it to everyone else.

Also, just to comment on people who have noted that PubPeer is mostly focused on biomedical papers at the moment: this is due to the fairly high amount of non-replicable results that the field has experienced over the past couple decades.

There is an extremely large amount of money involved in biomed research as well, making it very difficult for scientists- young and old- to run counter to the status quo. There are a very large number of biomed researchers worldwide; however biomed journals- the primary mechanism for “getting the word out” currently- are mostly run by the USA. In the USA, NIH funding is usually dispersed to those who publish in “high impact” journals. However, “high impact” does not necessarily correlate with “good science” any longer; it’s turned into a bit of a popularity contest over here. (I’m a chemist by training, and our funding system in the USA is a bit different than that of biomed. Esp. in regards to conflicts of interest.)

As PubPeer mentions, I am sure they will do their upmost to include other disciplines in the near future. Biomed is at a critical juncture right now. A platform like PubPeer is needed given the large amounts of money involved in research and the current global economic situation. (I am working on making the USA universities have transparent budgets for biomed grants. As an American, I have a strong desire to see exactly where my taxes are going.)

Just after finding out how difficult it is to criticise a high-impact article I tried to register. Two of my articles did show up. However, my institutional e-mail addres (knmi.nl) is not recognised, so hte service is not yet useful for me.

It is easy to confirm Ray’s view. Go to scholar.google.com and put in an author’s name. Try D.K. Hall; she’s a prolific and highly respected.
Pick any one of her papers and then follow the citations of that paper by others by date. It is amazing how quickly a paper is cited by others. Equally amazing is how long a really good paper persists as a benchmark.

Thanks for you comments and suggestions both here and on our contact page. We have fixed the issues below:

Geert Jan van Oldenborgh – However, my institutional e-mail addres (knmi.nl) is not recognised, so hte service is not yet useful for me.

Sorry about that. You should be able to register now as we expanded our list of recognized institutional addresses.

WebHubTelescope – To register, you apparently have to have a paper with your name in the database, i.e. the qualifications to be a peer. Unfortunately, when I put my last name in the registration box, the only response is a paper by my MD sister-in-law. So, in effect, not many people can get in there to boot-strap the database away from a medical research focus.

Any article with a DOI can now be used to create an account. Simply paste the DOI into the search bar during account creation.

Everything — everything — written that you intend to go out under the name PubPeer — everything that is meant for public reading (searching and indexing) — should go through a couple of rounds with your competent copy-editors. Do this before making things public.

Having done that in draft, thereafter, once you go public, never delete anything: use strikeout and underscore to show changes made in online material and explain the changes.
_________
Those using Word with spellcheck should, um, think about the risks and benefits.

Make the process as competent as the work you’re reviewing.
___________________

Yes, doing this is tedious, but less tedious than catching up with belated corrections.

can anyone here tell me the “direction” of all the major climate forcings right now? i’m talking about things like ENSO, the PDO, the AMO, solar radiation, and any other forcings you can think of?

i’m trying to figure out if this recent slowing-down of warming could be because all the forcings are negative (except, obviously co2). is it possible or likely that this slowing-down of warming would be an actual cooling period if not for co2?