A blog on novel user interfaces, mobile applications, pervasive and ubiquitous computing. I use the blog as a note pad ;-)

Monday, 12 November 2012

SIGCHI Rebuttals - Some suggestions to write them

ACM SIGCHI has in it's review process the opportunity for the authors to respond to the comments of the reviewers. I find this a good thing and to me it has two main functions:

The reviewers are usually more careful in what they write as they know they have to face a response for the authors

Authors can clarify points that they did not get across in the first place in the original submission.

We usually write for all submissions with an average score over 2.0 a rebuttal. For lower ranked submissions it may be OK if we think we have a chance to counter some of the arguments, which we believe are wrong or unfair.

For the rebuttal it is most critical to address the meta-review as good as possible. The primary will be in the PC meeting and if the rebuttal wins this person over the job is well done. The other reviews should be addressed, too.

For all the papers where we write a rebuttal I suggest the following steps(a table may be helpful):

read all reviews in detail

copy out all statements that have questions, criticism, suggestions for improvement from each review

for each of these statement make a short version (bullet points, short sentence) in your own words

sort the all the extracted statements by topic

combine all statements that address the same issue

order the combined statements according to priority (highest priority to primary reviewer)

for each combined statement decide if the criticism is justified, misunderstood, or unjustified

make a response for each combined statement

create a rebuttal that addresses as many points as possible, without being short (trade-off in the number of issue to address and detail one can give)

Point 8 is the core...
There are three basic options:

if justified: acknowledge that this is an issue and propose how to fix it

if misunderstood: explain again and propose you will improve the explanaition in the final version

if unjustified: explain that this point may be disputed and provide additional evidence why you think it should be as it is

The unjustified ones are the most tricky ones. We had cases where reviewers stated that
the method we used is not appropriate. Here a response could be to cite
other work that used this method in the same context. Similarly we had
reviewers arguing that the statistical tests we used cannot be used on
our data, here we also explained in more details the distribution of the
data and why the test is appropriate. Sometimes it may be better to
ignore cases where the criticism is unjustified - especially if it is
not from the primary.

Some additional points

be respectful to the reviewers – they put work in to review the papers

if the reviewers did not understand – we probably did not communicate well

do not promise unrealistic things in the rebuttal

try to answer direct questions with precise and direct answers

if you expect that one reviewer did not read the paper – do not directly write this – try to address the points (and perhaps add a hint it is in the paper, e.g. “ANSWER as we outline already in section X)