In my May 2009 Editor's Letter, "Conferences vs. Journals in
Computing Research" (p. 5), I addressed the publication culture
of our field: "As far as I know, we are the only scientific
community that considers conference publication as the primary
means of publishing our research results. In contrast, the
prevailing academic standard of 'publish' is 'publish in archival
journals.' Why are we the only discipline driving on the
conference side of the 'publication road?'"

In response to my editorial, Lance Fortnow wrote a Viewpoint
column (Aug. 2009, p. 33), entitled "Time for Computer Science to
Grow Up," in which he concluded: "Computer science has grown to
become a mature field where no major university can survive
without a strong CS department. It is time for computer science
to grow up and publish in a way that represents the major
discipline it has become."

The May 2009 editorial and the August 2009 column attracted a
lot of attention in the blogosphere. The reaction has been mostly
sympathetic to the point of view reflected in both pieces. For
example, Jeanette Wing asked in her blog: "How can we break the
cycle of deadline-driven research?", and Filippo Menczer, in a
Letter to the Editor published in the November 2009 issue, said:
"I propose the abolition of conference proceedings
altogether."

Not everyone, however, agreed with this point of view. For
example, in another Letter to the Editor from the November 2009
issue, Jano van Hemert said: "For CS to grow up, CS journals must
grow up first." Mr. van Hemert's issue with computing-research
journals is that they are known to have "slow turnaround, with
most taking at least a year to make a publish/reject decision and
some taking much longer before publishing." Such end-to-end
times, he argued, "are unheard of in other fields where journal
editors make decisions in weeks, sometimes days."

While I have not see concrete data comparing publishing
turnaround times for computing-research journals to those in
other technical fields, there is abundance of anecdotal data
supporting the claim that computing-research journals are indeed
quite slow. (The average time to editorial decision for
Communications is under two months; that takes a concerted
effort by the editorial board to ensure that the editorial
process does not stall.)

What is the reason for the unacceptably slow turnaround time
in computing-research journals? When considering this question,
one must factor the problem into two separate issues: time from
submission to editorial decision, and time from positive
editorial decision to publication.

First let us address the latter issue. All periodical journals
have editorial "pipelines." No publisher wants to face the threat
of an empty issue; it's akin to the dreaded dead air on
television! Successful journals that attract many submissions
often see their pipelines extend for up to two years. With the
advent of electronic publishing, this problem can be eliminated
or at least minimized. Communications uses its Virtual
Extension (VE) to ensure its pipeline does not get longer than
six months. VE articles undergo the same rigorous review process
as those in the print edition and are accepted for publication on
their merit. These articles are available in ACM's Digital
Library.

Let us now consider the editorial process in
computing-research journals. Why is it soooo slow? Consider who
is in charge of that process. It is not the publishers; it is the
editors and referees. In other words, it is us. The
process is slow because that is the way we run it. If we want it
changed, it is up to us to change it! I suspect that we cannot
separate our conference-focused publication culture from our
sluggish journal editorial process. Conferences have sharp
deadlines, journals do not. We simply do not take our roles as
editors and referees as seriously as we do as program committee
members because we do not take journals as seriously as other
fields. If we, as a community, decide that we need to shift from
conference-based publication to journal-based publication, we
definitely must address the slow editorial process, but we should
not complain about "them journals." We have found the enemy, and
it is us!

The 2010 Conference of the Computing Research Association
(July 1820, Snowbird, UT) will have a plenary panel on
"Peer Review in Computing Research." I look forward to that
discussion and hope it will help our community reach consensus on
this issue.

Footnotes

Permission to make digital or hard copies of all or part of
this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or
commercial advantage and that copies bear this notice and the
full citation on the first page. To copy otherwise, to republish,
to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee.

Comments

Abazer Ali

February 25, 2010 07:12

There is no doubt that computer science has become as significant Magazines, ranging from areas such as databases and web pages as well as computer networks. I agree with the statement "in order to grow up computer science has to grow up scientific journals.
Another point I want to acknowledge it, a very careful before publishing houses in the adoption of patrols in computer science.
Because I asked one of them days in the book fair, why he wrote the computer publishes are part of the lowest among the headings in the show?
He replied that this area is progressing rapidly enormous which we publish today may become undesirable in a year or less.
Accordingly, I believe that the process of transformation of conferences and publication to the journals require a lot of reflection and consideration in resolving the dilemmas faced by computer science before sagging.
With my sincere thanks and respect

Sameer Abufardeh

March 11, 2010 04:46

Yes, the editorial process in computing-research journals is very slow and somewhat frustrating. My own experience taught me that to prepare a paper for a journal I will need at least twice the time needed for a conference, twice the wait, etc. For a graduate student (MS, PhD) who is expected to publish some of his work before graduating, the long wait, the endless formatting requirements by journals, can be time consuming, overwhelming, and somewhat frustrating. Furthermore, I dont believe that there is any tangible quality gap between conference paper and journal paper. Conference papers are generally short and right to the point because of the limitations normally set forth on the # of pages allowed. - Kind Regards to all

CACM Administrator

June 06, 2014 11:58

The following letter was published in the Letters to the Editor in the July 2010 CACM (http://cacm.acm.org/magazines/2010/7/95049).
--CACM Administrator

In his Editor's Letter "Revisiting the Publication Culture in Computing Research" (Mar. 2010), Moshe Y. Vardi said computer science is "the only scientific community that considers conference publications as the primary means of publishing our research results," asking, "Why are we the only discipline driving on the conference side of the 'publication road?'"

As an old timer, I can say that in the early days, there was a belief (conceit might be a better word) that the field's pace of discovery was happening so quickly that only conferences, with subsequent prompt publication of proceedings, could communicate results in a timely manner. As a corollary, the traditional peer-reviewed published literature review fell behind, as it was relieved of temporal pressure through the published proceedings.

These days, the pace of discovery in the biological sciences, including molecular biology, genomics, and proteomics, far exceeds that of computer science. Yet the gold standard of publication in archival journals continues. It is the ultimate irony that computer science, along with various disciplines in the physical sciences, employs the tools developed by computer scientists to ensure timely dissemination of research results through the online editions of their publications. Science, Nature, Cell, and other leading journals routinely present their most important articles in online form first. If, perhaps, computer science would make greater use of its own tools, the shoemaker's children would no longer go barefoot, and published proceedings would fade into its proper historical niche.

Stuart Zimmerman
Houston, TX

CACM Administrator

June 19, 2014 12:02

The following letter was published in the Letters to the Editor in the May 2010 CACM (http://cacm.acm.org/magazines/2010/5/87250).
--CACM Administrator

Moshe Y. Vardi wrote in his Editor's Letter "Revisiting the Publication Culture in Computing Research" (Mar. 2010) about computer science being a field in which conferences are the primary venue for reporting research. Over the years, the premier conferences have evolved to where their acceptance rates are lower than the corresponding journals and their citation rates are higher. That is why, in the university, the other fields in the tenure councils accept the computer science argument that conference publications deserve the main weight.

When I first got involved in the ACM publication establishment (late 1960s) there was a terrible problem with backlogs in the two ACM research journals of the day: Communications and the Journal of the ACM. Editorial time submission to final decision was running 1218 months, and publication time decision to publication took at least as long. Many loud complaints agreed that 2436 months for the complete process was bad for such a fast-changing field. However, ACM was almost broke and could not afford more pages, and there was no online venue. Researchers began to look to the SIG conferences as a way to get their material into print sooner, much sooner. By the 1980s, some of these conferences were petitioning the Publication Board to designate them as "refereed" instead of "reviewed."

Studies by Bob Ashenhurst and John Rice documented the editorial delays. Even when editors were fastidious about pushing the review cycle along, the authors often took a long time to submit their revisions. Though individual reviews took a long time as well, the authors themselves contributed a lot of delay. The studies also found (with much less supporting data) that authors try different journals and conferences until they find a publisher, and that patient, persistent authors eventually publish over 90% of their papers. Rice calculated an average paper needs four referees, so authors "owe" their colleagues four reviews for every published paper.

A productive author publishing three journal papers a year would thus owe the field 12 reviews a year. Most researchers complain if they have to do half that number. Rice's conclusion, still valid today, was that as long as researchers do not want to do all those (timely) reviews, the editorial phase would not get significantly shorter and is simply the cost of an all-volunteer system.

In 1983 at the start of the Communications revitalization plan, we investigated how Science magazine gets its submissions reviewed so much quicker. We visited the editors and learned they phoned names from a reviewer database to ask for two-week turnaround. After locating three agreeable reviewers, the editors would FedEx them the manuscript.

We wanted to do likewise, but the ACM executive committee said it was way too expensive. At the time, ACM didn't have enough money to cover even the editors the Council had approved and could not mount a quick review process in Communications. Today's online tools make it much easier and less expensive to find reviewers, in which case the bottleneck is again the willingness of reviewers to quickly review and authors to quickly revise.

Peter J. Denning
ACM Past President, 19801982
Monterey, CA

CACM Administrator

June 19, 2014 12:04

The following letter was published in the Letters to the Editor in the May 2010 CACM (http://cacm.acm.org/magazines/2010/5/87250).
--CACM Administrator

A fundamental difference between the journal and conference publication systems, as discussed by Moshe Y. Vardi (Mar. 2010), is that conference leadership (chairs and program committees) changes much more frequently than (is practical) for journals. It is difficult (though perhaps not impossible) for a conference to consistently reflect the personal biases and hidden agendas of the same people over a long period of time. The same cannot be said of journals, which have much slower turnover of editors. Such stability allows journals to be, at least in principle, more coherent and less prone to fads, and perhaps less responsive to what the community views as important.