How it works

Blogs provide a simple, light-weight system for publishing on the web. Traditionally, they’ve been used to transfer opinion and news. Blogging in the science has mostly been journalistic: about events which affect science, the personalities in science, or about the process of science. Largely, they haven’t been part of the process of science.

The technology doesn’t determine the use however. Blogs are good for publishing time-pointed, unchanging and archived material; pretty much the same as a scientific journal. For a knowledge blog, we apply a few social conventions on top of existing blog technology, and use this to write a journal, book or encyclopedia.

Authoring

Authoring doesn’t really change. The author writes the article using which ever technology they prefer; so long as they end product is some HTML, this is enough. When authors wish to refer to other articles, they just use normal citation practice, with a link to pubmed or similar resource if they choose.

If the authors want to see what the final copy of their article will look like, they upload it either to a knowledgeblog.org blog or to their own, keeping it “private” or “unpublished” with their blog.

Submission

Technically, this is very straight-forward; authors simply post their article, or if it has already been posted “unpublished”, they move it to published. At this time, the article will categorised as “under review”.

While technically simple, there is a major sociological change here; at the point that the article is submitted, it’s also published. It will be publicly visible, perma-linked and archived.

As an alternative, authors may wish to publish the post to their own blog. In this case, they post a short note, with the same title as their blog post, a link to it and, if they wish, a short summary.

Review Process

The review process is critically different from the normal process; it’s not blind. The authors know who the reviewers are, the reviewers know who the authors are, and everybody knows what the reviewers have said.

In most cases, we consider that the reviewers would be mostly found by the author, either from a defined list or freely. In some cases, editors might wish to add one reviewer of their own.

Once the reviewers have been selected, they also write their reviews as blogposts with a link at the top, through to the original article. The blog technology will ensure that this link to the article shows up as a comment.

If reviewers wish to publish the post to their own blog, the process is exactly the same, if their blog supports pingbacks. If not, trackbacks can also be used.

Once all the reviewers have posted their replies, the authors address any concerns, finally mailing the editors to tell them that the process is complete.

Editorial Process

The editorial process is much like normal, except with less paper work. Editors do not, in general, need to pick reviewers as this work is now done by the authors. The first interaction with a paper that the editor will generally have is after the reviewing has been completed, when they will be mailed by the authors. At this point, the editor reads the reviews, decides whether the paper is appropriate, and whether any points need addressing; these can be communicated via comments or the editor can blog the comments, using pingbacks in the same way as the reviews.

Once everything is complete, the editor simply re-categorises the to “reviewed”. Publication happened a long time ago, during submission. The readers now know, however, that the paper has been through a full peer review.

We don’t know whether it will or not, nor do I know any particular way of evaluating whether this has or has not happened. The knowledgeblog process supports different kinds of peer-review, not all of them being author-picked reviewers. But my feeling is that public peer-review increases quality; if the peer-review is poor than now the public will know this, which will reflect badly on the reviewer and the reviewee. In the end, this is likely to get reflected in the comments of the article — in the same way that you, for instance, have written this question.

My own experience of blind peer-review both as reviewer and reviewee is that it doesn’t introduce a huge amount of quality control anyway. The variance of scores that people give papers is pretty huge. And the helpfulness of reviews to the authoring process is similarly variable. My feeling is that this sort of quality control still reflects the limited page lengths of print journals. Quality should be judged post-hoc, after publication.

Non-blind peer-review would increase the quality in the sense that no longer could reviewers write outrageous comments and block publication of papers out of obscure interests. Most reviewers are honest scientists that are doing their best for the good of science; unfortunately the cloak of anonymity allows a small but damaging number of reviewers to take dubious positions that damage publication of good work. Doing the peer review in the open stops this nonsense.

Would it be OK if I and a few of my friends got together and just made up crazy results, and then reviewed each others’ work? Pick the right results, and they might echo repeatedly through blogs, both scientific and unscientific. Might even get noticed on cable news, discussed by pundits, and reported in mainstream news organizations. Heck, with the right strategy, we might even influence public policy. Cool!

Whether it would be okay or not, it is and does happen already. Within the science, obviously, this is a problem. I don’t think that the publication system can OR needs to stop this. To quote Feynman “reality must take precedence over public relations, for nature cannot be fooled.” In the end, it is experiment, and repetition that will sort out truth from fiction.

The responsibility of the publication system, I think, is this; it needs to establish and maintain a clear record. With knowledgeblog, identities are open, reviews are open, changes are open. If people try and game the system, they can. But the evidence that they have done so will be clear to see, and it will remain for the future. Personally, I think that this is an advance from the current system.