Computational Complexity and other fun stuff in math and computer science from Lance Fortnow and Bill Gasarch

Thursday, June 03, 2010

Conference Acceptance Rates

In the latest CACM, Jilin Chen and Joseph Konstan analyze data from the ACM DL and conclude that low-acceptance rate conference have papers with higher impact (more citations). This shouldn't be too surprising. If Joe believes a conference A bestows more prestige on their paper than conference B, Joe might submit his paper to A instead of B if it Joe's paper had a higher probability of being accepted into conference B. So in equilibrium conferences with higher prestige papers should have a lower acceptance rate.

Chen and Konstan suggest that conferences could lower than acceptance rate to get higher prestige. I have two problems with that advice. First of course my general annoyance of treating CS conferences as "journals that happen be be held at a hotel".

Chen and Konstan have the causation effect backwards. Prestigious conferences do have a lower acceptance rate with other factors kept the same. But you can't necessarily up the prestige by changing the acceptance rate. The one decision the PC can make is how many papers get accepted into the conference. But accept fewer papers and in the future less people will submit their papers since they no longer believe they have a decent chance of seeing their papers accepted. it's possible in the long run to increase the acceptance rate while lowering the number of papers accepted.

When STOC moved to double sessions and accepted more papers, did the acceptance rate dramatically go up? No, because we had more submissions to match. But why does STOC have a higher acceptance rate than similarly rated conferences in other subfields? Theoretical work by its nature is easier to self-judge and most people whose papers are well below the accept/reject border of quality don't bother to submit to STOC.

You have a prestigious conference by the people that come. You can't engineer prestige, it has to be nourished.

I think a large part of it is who is on the PC. If you read the list and you realize you've only vaguely heard of two of the people, you know it's not very prestigious. On the other hand, if you recognize many of the most important people in your field, you know that your submission has the potential for impact, or that at least you might get some worthwhile feedback.

I think an author would think about that before thinking about the acceptance rates.

I think that one could also legitimately question the use of citation count as a measure of a paper's impact. While having a high number of citations is probably a necessary condition for a paper to be deemed significant, it is not so obvious that it is a sufficient condition.

Impact factor = citations/paper in the past 2 years. Therefore it should more aptly be called the "immediate impact factor". It should not be confused with how good, important, or even interesting the results are in a paper, because no one has the insight/foresight into how each piece fits into where in the grand scheme of the research landscape. I frequently cite math papers from decades ago, for example.

Like the mathematician Steven Krantz says: "There are prestigious journals and there are excellent journals." Perhaps the same can be said about TCS conferences?