Citation Analysis & Impact Factors

Introduction

When thinking about citation analysis and impact factors, you should try to consider the perspective from your own field of study. Some possible questions include:

How do you measure success of an article or book in your discipline?

How do you measure the success of a scholar in your discipline?

Is there discussion of citation analysis/impact factor particular to your discipline?

Are impact factors used in promotion and tenure decisions regarding professors in your department? In evaluating job applicants?

Purpose of Citations

Attribution of ideas/research

for confirming/illustrating a point

for disputing/correcting/questioning

for use of methods, tools, design, definition or data

Provide proof that position is well-researched by providing

holistic view of research

literature review

primary and historical sources

Help disseminate useful knowledge

demonstrating other points of view

supplementary information

Give formal credit for research as a normative research practice

for purpose of association and community

to meet funding, legal, and governmental requirements

What is Citation Analysis

Citation Analysis is the examination of the frequency and patterns of citations to books, journal articles, individual journals, conference proceedings, etc. Citation analysis began in the sciences in the 1960s and spread to the social sciences and humanities in the 1970s. Citation analysis includes but is not limited to citation counts, statistical analysis (e.g. h-factor), and data visualization.

Citation Metrics are used to evaluate

the impact of an individual researcher

Citation metrics are commonly used as one measure in academic job applications as well as tenure and promotion applications

Must not be conflated with quality of research

the impact of an individual journal article, book, etc.

Can help with selecting key works in a field.

the impact of an individual journal

May be helpful in choosing a journal in which to publish

the impact of a department or university

Included in the calculation of widely-published university ranking lists.

Citation Metrics Shortcomings

Can be susceptible to self- or endogenous-citation practices. Examples

May not favour interdisciplinary work, new journals, or Canadian journals

Comparisons across disciplines are not valid. Disciplinary publishing and citation practices vary, so the top journal in once discipline may have an impact factor of 36 while the top journal in different discipline may have an impact factor of 8.

Scholarly Metrics

H-index from a plot of decreasing citations for numbered papers

Individual author/article citation counts Citation Tracking (CT) counts the number of times an individual article or a particular author has been cited by other scholars. Large numbers are associated with greater impact and influence. Article and author level citation counts are available on Web of Science, Google Scholar, PLoS, BioMed Central, plus numerous discipline-specific databases.

H-index A measure of author influence, an h-index is the number where the number of articles published by an author intersects on a graph with the number of citations for each article. For instance, an author with h-index of 10 has published 10 papers that have been cited at least 10 times each. The h-index is the first (and most well-known) of many author metrics. Available on Web of Science and Google Scholar if the scholar has created a user profile.

Journal Impact Factor (used by ISI Web of Science) A Journal Impact Factor (JIF) is measured by dividing the number of current citations a journal has in a given year, by the number of articles published in the two previous years. The JIF is used to indicate the relative importance of a journal within a given field and a higher JIF is seen as providing more “authority” or “weight” to a researchers’ work.

Eigenfactor A non-commercial alternative to ISI Web of Science's JIF. The Eigenfactor algorithm also takes into account the significance of the citations coming into a journal so that citations from important journals are weighted more highly.

Acceptance Rate Acceptance rate compares the number of articles selected for publication to the number of articles submitted. For example, the very prestigious journal Nature publishes fewer than 10% of the articles submitted. In general, a low acceptance rate is associated with high prestige. Not all journals publish their acceptance rates, but in many cases it is included in the "instructions to authors" section of the journal.

Concerns Use of the Journal Impact Factor as "the primary parameter with which to compare the scientific output of individuals and institutions" has been criticized by a number of scholars and organizations, most notably here.

Most scholarly metric tools include only a small fraction of citations that appear in books or book chapters: the work of Humanities and Social Sciences scholars is not accurately reflected.

Alternative Metrics

Download counts and View Counts These article level metrics count the number of times an article has been downloaded and the number of times that it has been viewed. Download counts can be used to demonstrate use of articles for purposes other than citation such as education, background research, presentation, or use in grey literature such as white papers or reports. Download and view counts are available for many, but certainly not all journals.

Authority scores (e.g. Klout) Authority scores attempt to measure how many people you reach through your own contacts, their amplified reach (e.g. how many people are followed by people who follow you on Twitter) and how often your content elicits a response (i.e. a comment or reply).

Tools

While many publishers and scholarly databases support searching for cited references, the feature is generally limited to articles within their own database. These academic and altmetric tools are commonly used for more thorough citation analysis.

"Captures hundreds of thousands of tweets, blog posts, news stories and other content that mention scholarly articles." Add the free Altmetric Bookmark to your bookmarks toolbar and see the metrics for any article with a DOI and items in digital repositories.

Readings

García-Pérez, M. A. (2010), Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: A case study for the computation of h indices in Psychology. Journal of the American Society for Information Science and Technology, 61: 2070–2085. doi: 10.1002/asi.21372

Kousha, K., Thelwall, M., & Rezaie, S. (2011). Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus. Journal of the American Society for Information Science and Technology, 62(11), 2147-2164. doi:10.1002/asi.21608

McDonald, John (2003). The Book is Dead: Citation Rates in the Humanities and Social Sciences. In Strauch, Katina (ed.) Charleston Conference Proceedings (21st, Charleston, South Carolina, November 1-3, 2001) 143-152. [Google Books Link, not all pages viewable]