Science Overlay Mapping as a Tool to Explore Interdisciplinary Research

Addressing complex societal challenges, such as climate change or the diabetes pandemic, requires bringing together different types of knowledge from disparate academic disciplines and societal stakeholders (Stirling, 2007). There is evidence to suggest that scientific teams and organizations with diverse types of expertise are more capable of solving complex problems than experts working individually in narrow areas of expertise (Page, 2007). However, the effective design and management of such teams and organizations requires an understanding of the cognitive diversity that a team mobilizes (i.e., the variety, balance and distance of the team members' bodies of knowledge). Awareness of cognitive diversity is important, for example, for addressing potential communication challenges, making decisions about the internal or external sourcing of instrumental expertise such as microscopes, and for preparing publication strategies.

While there are several approaches to map and measure cognitive diversity, one approach is to look at the disciplines represented in a team and the extent of their cross-disciplinary interactions. Loet Leydesdorff (University of Amsterdam), Alan Porter (Georgia Institute of Technology) and I have used a method called science overlay mapping, which can be useful to map interdisciplinary research (see Figure 1; Rafols et. al, 2010).

Science overlay mapping begins with a "global science basemap" representing all domains of science, such as one derived from bibliometric data available through publication indexing systems for scholarly literature, such as the Web of Science Categories. A global science basemap created using Web of Science data represents all domains of science through a two-dimensional network diagram in which nodes represent sub-disciplines, constituted by the Web of Science Categories. Links represent the degree of cognitive similarity (as elicited from citation patterns) among Web of Science Categories. The shorter and thicker the links, the more similar the categories. Hence, the relative position of nodes suggests the clusters that constitute a large body of knowledge (see interactive map).

These maps allow one to explore the degree of relatedness among sub-disciplines (closeness vs. distance), as indicated by citation patterns. Alternative approaches have yielded quite similar results for the ‘backbone’ structure of global science, as shown by overlay pioneers Boyack and Klavans (2009). Not surprisingly, one key feature of these global maps is a gap between the social and natural sciences – one that typically must be bridged in applied research that aims to advance progress toward solving societal problems.

Science overlay maps superimpose an activity of interest (e.g. publications or citations) of a given organization or team on top of the full science basemap. We provide a toolkit which allows users to download datasets from the Web of Science, to transform them with small executable programs, and then to upload the data into an overlay on the map of science using freeware visualisation software such as Pajek or VOSviewer (see freeware toolkit).

This approach is illustrated in Figure 2 using publication patterns for one institute on innovation studies that is considered highly interdisciplinary -- The Institute for the Study of Science Technology and Innovation, at the University of Edinburgh, ISSTI (top) -- and one business school that is narrowly focused on business and economics – the London Business School, LBS (bottom). The size of the nodes is proportional to the number of references of the organization in a given Web of Science Category. All links represent citations and the green links represent citations across diverse categories at a level five times larger than was expected based on the average citation patterns in the Web of Science. Using this approach, ISSTI appears to be highly interdisciplinary because its references are spread over many categories (position of nodes) and they connect disparate categories that usually do not reference each other. Associated measures such as Rao-Stirling diversity, Shannon entropy or average cognitive distance can be derived from the data shown in the maps (Rafols et al., 2012).

Science overlay maps have proved useful in comparing the research conducted by universities with the Research & Development work of large corporations (Rafols et al., 2010). They have also allowed us to show that the journal rankings used by the Association of Business Schools (ABS) are biased against interdisciplinary research, thus favoring traditional business schools, which have a narrow disciplinary focus (Rafols et al., 2012).

Figure 2: Overlay of number of citations on Web of Science Categories of the ISSTI (top) and the LBS (bottom) on the global map of science. The extent of citing between categories (as indicated by green links) by a given unit is shown only for observed values five times larger than expected. Diversity of citations, as reflected in the spread of nodes over the map, and citations across disparate sub-disciplines (i.e., the amount of cross-linking) are interpreted as signs of interdisciplinarity. Source: Rafols et al., 2012. See interactive version at www.interdisciplinaryscience.net/maps

While science overlap maps can serve as a valuable tool for examining type and degree of interdisciplinarity in organizations, academic departments and large research centers or networks, they have several important limitations. One limitation is that these maps are based on coarse categories (e.g., the Web of Science Categories). Given this coarseness, the assignment of publications to these categories is imprecise. When used for a large number of publications, this is not a problem, because the overall pattern obtained is statistically robust (Boyack and Klavans, 2009). But the patterns become unreliable when using a small number of publications (estimated to be less than 100), which is typically the case for small teams.

Another limitation is that the assessment of cognitive diversity in small teams is less accurately represented by sub-disciplines and more accurately represented by the specific research specialties brought in by the different team members or collaborating laboratories. Indeed, there are many forms of cognitive diversity that are not mapped into disciplines, nor easily captured by bibliometric methods. Examples include theoretical vs. experimental research, basic vs. applied research, and community-engaged and translational approaches, all of which encompass different methods and knowledge and cannot easily be distinguished through science maps.

A potential way to address the limitations of the science overlay mapping method is to improve the granularity of the categories used to classify the papers or references. However, higher granularity in a global science basemap leads to the creation of a network with many nodes and many similarity dimensions. As a result, the relative position of two specific nodes in a two dimensional surface (the map) may become more difficult to read and relatively less accurate because there are now many more dimensions that cannot be shown. We have observed some instances of these trade-offs between readability, category granularity and position accuracy in global journal maps (Leydesdorff and Rafols, see freeware toolkit; cf. with van Eck and Waltman, 2010, see map). Other researchers are improving granularity by using similarities in the topic of publications (e.g. via co-occurence of words or topic modelling), rather than similarities in citations. This also allows the maps to include other types of documents beyond peer-reviewed publications (see, for example, these maps based on NIH grants).

But there are conceptual grounds to believe that science maps will always be limited. In particular, they create artefacts by depicting multidimensional cognitive spaces in two dimensions. Therefore, rather than trying to create the most "accurate" maps, my preference is to use multiple science maps at different levels of aggregation. For example, the overlay global maps described above provide an initial approximation of the disciplinary areas of expertise within an organization or team. These maps are easy to make and read. However, since these maps are imprecise, when investigating cognitive diversity of teams, they should be complemented with bibliometric maps at lower levels of aggregation that tell us about specific research specialties. Combining maps at multiple levels of aggregation also helps to identify inconsistencies in the classifications used and dimensions not apparent at different levels of aggregation.

My view is that science maps should be used in a way similar to how we use geographical map when hiking: the map is practical if it helps understand potential paths and distances among points. But we are fully aware that the map misses many details. Science mapping offers a potentially useful tool to look at interdisciplinary efforts of teams, but one needs to remain aware of the limitations, particularly for small samples. Generating multiple maps at different levels of aggregation (e.g., by discipline, research specialty, or topic) is a way to provide multiple perspectives and richer understanding about cognitive diversity in teams.