To rank or not to rank

Very shortly after establishing my website in 1999, I posted the first edition of my Journal Quality List, now in its 60th edition. Over the years I have also published a large number of articles dealing in one way or another with rankings. You would thus be forgiven for thinking that I am a big advocate of journal and university rankings. It might surprise you I am not. Just like many of you I would prefer to live in an academic world where rankings and public relations didn't dominate academic substance. However, I recognize that rankings are not going away and that even if we did away with formal rankings, most academics would still create rankings in their own minds.

So my main motivation has been to ensure that if rankings do persist, we at least make sure they are as inclusive as possible and understand that every ranking has its own biases. My first publication in this field thus presented an integrated ranking of the then 16 rankings included in the Journal Quality list. One of its motivations was to help the asessors in the Business & Management panel of the 2008 British Research Assessment Excercise in their assessment of over 10,000 publications by providing a ranking based on a variety of perspectives. Remember, this was before the days of the "one list to rule them all", aka the ABS journal ranking.

I then moved on to exploring new data sources that provide a fairer assessment of journal impact in the Social Sciences and Humanities, using Google Scholar as an alternative to Scopus and the Web of Science. For a more general introduction into the coverage of Google Scholar and its drawbacks, see here, here, here, here, here and here.

Subsequently, my attention was drawn towards university rankings. A short commentary on the - in my view - arbitrary decisions in one particular ranking of universities study grew into a more provocative piece, co-authored with the wonderful Nancy Adler. We highlight the problematic nature of academic ranking systems and question if such assessments are drawing scholarship away from its fundamental purpose. Ironically, this paper is now my top-ranked publication in terms of citations per year.

In recent years, I have looked at how problems in citation databases and rankings might reinforce existing academic hierarchies in terms of disciplines and universities. In these articles, I also discuss how these problems might disproportionally disadvantage academics with Asian names, women, and younger academics.

Anne-Wil Harzing is Professor of International Management at Middlesex University, London and visiting professor of International Management at Tilburg University. In addition to her academic duties, she also maintains the Journal Quality List and is the driving force behind the popular Publish or Perish software program.