Web Indicators for Research Evaluation: Altmetrics, Social Media, and Books and Non Standard Outputs. 3 Seminal Works from Thelwall and Kousha

Altmetrics

Web Indicators for Research Evaluation

Part 1: Citations and Links to Academic Articles from the Web

Abstract The extensive use of the web by many sectors of society has created the potential for new wider impact indicators. This article reviews research about Google Scholar and Google Patents, both of which can be used as sources of impact indicators for academic articles. It also briefly reviews methods to extract types of links and citations from the web as a whole, although the indicators that these generate are now probably too broad and too dominated by automatically generated websites, such as library and publisher catalogues, to be useful in practice. More valuable web-based indicators can be derived from specific types of web pages that cite academic research, such as online presentations, course syllabi, and science blogs. These provide evidence that is easier to understand and use and less likely to be affected by unwanted types of automatically generated content, although they are susceptible to gaming.

Part 2: Social Media Metrics

Abstract This literature review assesses indicators derived from social media sources, including both general and academic sites. Such indicators have been termed altmetrics, influmetrics, social media metrics, or a type of webometric, and have recently been commercialised by a number of companies and employed by some publishers and university administrators. The social media metrics analysed here derive mainly from Twitter, Facebook, Google+, F1000, Mendeley, ResearchGate, and Academia.edu. They have the apparent potential to deliver fast, free indicators of the wider societal impact of research, or of different types of academic impacts, complementing academic impact indicators from traditional citation indexes. Although it is unwise to employ them in formal evaluations with stakeholders, due to their susceptibility to gaming and lack of real evidence that they reflect wider research impacts, they are useful for formative evaluations and to investigate science itself. Mendeley reader counts are particularly promising.

Part 3: Books and Non Standard Outputs

Abstract This literature review describes web indicators for the impact of books, software, datasets, videos and other non-standard academic outputs. Although journal articles dominate academic research in the health and natural sciences, other types of outputs can make equally valuable contributions to scholarship and are more common in other fields. It is not always possible to get useful citation-based impact indicators for these due to their absence from, or incomplete coverage in, traditional citation indexes. In this context, the web is particularly valuable as a potential source of impact indicators for non-standard academic outputs. The main focus in this review is on books because of the much greater amount of relevant research for them and because they are regarded as particularly valuable in the arts and humanities and in some areas of the social sciences.

Mike Thelwall is the head of the Statistical Cybermetrics Research Group at the University of Wolverhampton, UK. He has developed a wide range of software for gathering and analysing web data, including hyperlink analysis, sentiment analysis and content analysis for Twitter, YouTube, MySpace, blogs and the web in general. http://orcid.org/0000-0001-6065-205X m.thelwall@wlv.ac.uk

Kayvan Kousha is a researcher in the Statistical Cybermetrics Research Group at the University of Wolverhampton, UK. His research includes web citation analysis and online scholarly impact assessment using webometrics and altmetrics methods. He has tested different web citation extraction methods from various types of web documents such as Google Books, Google Patents, academic course syllabi and online book reviews for research evaluation. http://orcid.org/0000-0003-4827-971X k.kousha@wlv.ac.uk