Blog Category: scientific publishing

The skilled adversarial reviewer can find reasons to reject any paper without even reading it. This is considered truly blind reviewing. [Cormode, G.]

Many conferences request that submitted papers be anonymized by removing the authors’ names, tweaking the references, removing mentions of the authors’ organization in the paper, etc. The goal of the double-blind review process is to reduce the bias (positive or negative) that reviewers might have based on their knowledge of who wrote the paper. SIGIR, for example, included the following on their submission page for the 2013 conference:

Anonymity. SIGIR reviewing is double-blind. Therefore, please anonymize your submission. This means that all submissions must contain no information identifying the author(s) or their organization(s): Do not put the author(s) names or affiliation(s) at the start of the paper, anonymize citations to and mentions of your own prior work that are directly related to your present work, and do not include funding or other acknowledgments. For example, if you are using your product that is well known in the domain and you think it will be easy for an expert to identify you or your company, we recommend that you use another name for your product (e.g., MyProduct_ABC, MyCompany_ABC). If your paper is accepted, then you will replace the original name in the final version for the proceedings.

Papers that do not follow the above Style, Language, Anonymity instructions will be rejected without review. [emphasis mine]

And, apparently, in some cases, they followed through on this policy. In my opinion, this is too harsh.

Today ACM announced a way for authors to pay for publishing open-access papers in the ACM DL. For a mere $1100 per conference paper ($1300 per journal article) for ACM members, authors can grant free access to their publications to anyone who wants it. I am all for open access to academic publications, but I have my doubts about this scheme.

Over the past six years of the HCIR series of meetings, we’ve accumulated a number of publications. We’ve had a series of reports about the meetings, papers published in the ACM Digital Library, and an up-coming Special Issue of IP&M. In the run-up to this year’s event (stay tuned!), I decided it might be useful to consolidate these publications in one place. Hence, we now have the HCIR Publications page.

Many people have asked me why I decided to write a book. A better questions is: “When you realized that writing the book was going to be orders of magnitude harder and take much longer than you thought it would, what made you decide to continue writing the book?”

My co-author, Wolfgang Polak, and I recently received a book review of the sort that is the dream of every author. A dream review is, of course, positive. But more importantly, it praises the aspects of the book that were most important to the author – the reasons the author kept going after other books on the subject came out and the author had a more reasonable (but still too optimistic) estimate of the vast amount of effort it would take to finish it. (The review appeared in Computing Reviews, but is behind a paywall. Excerpts appear on the book’s Amazon and MIT press web pages.)

The discussion on my previous post has raised some interesting and valid points regarding holding conferences in countries like China that block some (or all) internet traffic. Given that the conference has an audience that extends beyond the location of the conference, how can this audience be served in the presence of country-sponsored firewalls? Specifically, how can we get access to the Twitter stream and to other media being generated by the conference?

A number of ACM groups have recently made decisions to hold their conferences in China. The list of major conferences includes CSCW2011, SIGIR2011, Ubicomp 2011, and ICSE 2011, just to name a few. This seems like a strange trend. The purpose of academic conferences is to disseminate ideas in an open and public manner, and thus the argument has been made that taking these conferences to China will help expose China and Chinese researchers to these Western ideals. Yet what we see conference after conference are the restrictions that China imposes on electronic communication.

The field of information retrieval is inherently (some might say pathologically) data-driven. We need datasets to test algorithms, to compare systems, etc. This is all good. It’s particularly good to have data that are meaningful and relevant, because it makes it easier to motivate users and to generalize findings to data that people care about.

I expect that in the next few cycles of conference submissions, we will see a number of papers analyze the “cable” data leaked by Bradley Manning to Wikileaks. It’s a large enough dataset with topical relevance that is sure to attract all sorts of analyses, much like the Enron email dataset did in 2004.

For the upcoming rebuttals of CHI, it might be useful to understand what the reviewers really mean when writing their reviews. This year as I read with interest the reviews of my fellow reviewers, maybe due to my growing experience, or maybe because of the late hour reviewing, I started to see something new in the reviews: the hidden messages. Below is a collection of this years’ CHI, CSCW and past years’ CHI review’s opening remarks with possible interpretations.

Jeff Huang recently published a list of papers from several major conferences that won Best Paper awards. It’s a nice collection of papers, highlighted in a way that is difficult to obtain from the ACM Digital Library. (Why that should be the case is a different story.)

Clearly winning a Best Paper award is a significant achievement and authors of such papers should be proud of their work. But does this merit translate into impact? For example, do papers that win Best Paper awards get cited more frequently than other papers from the same conference?

For those of us with a passing (or greater) interest in algorithms, last week was particularly interesting: Vinay Deolalikar circulated a paper that attempted to prove P≠NP. This is one of the great unsolved problems in Computer Science, and its solution has some important implications for real-world problems such as keeping your money in your bank account.

I won’t attempt a summary of the proof, and will limit myself to social commentary.