Visualization of Disciplinary Profiles: Enhanced Science Overlay Maps

In order to understand the multidisciplinary profile of publication sets, disciplinary or sub-disciplinary categories can be assigned to the publications. These categories can then be used to represent the position of a publication set in the overall structure of science—i.e. to overlay a specific research activity onto the map of science (Rafols, Porter, & Leydesdorff, 2010).

One method to assign publications to a disciplinary category is to rely on the journal of the publication as an estimate of the scientific field. However, disciplines and fields of science develop above the level of individual journals. Scientometricians proposed the normalization of citations in terms of journal categories (ISI Subject Categories, now known as Web of Science Categories)—as proxies of scientific fields defined above the level of individual journals—in a series of publications during the 1980s (e.g. Schubert, Glänzel, & Braun, 1986; Schubert, Glänzel, & Braun, 1989; Vinkler, 1986).

Using these categories, Moed, de Bruin, & van Leeuwenet (1995) further developed the “crown indicator” at the Center for Science and Technology Studies (CWTS) in Leiden that was later improved as the “Mean Normalized Citation Score” (MNCS). This indicator remains based on the same subject categories, and it is currently the most widely used method to provide normalized comparisons across scientific areas.

The WCs tagged to the 11,000+ journals covered by the Science Citation Index (SCI) and the Social Sciences Citation Index (SSCI) are assigned by indexers on the basis of a number of criteria, including field experts’ judgment of relevance to a given field, the journal’s title, and its citation patterns (Bensman & Leydesdorff, 2009). As of 2015, there are 227 WCs covering SCI and SSCI. Pudovkin and Garfield (2002) described the methods used by the ISI (then provided by Thomson Reuters, and now Clarivate Analytics), and concluded that in many fields these categories are “sufficient;” but “in many areas of research these “classifications” are crude and do not permit the user to quickly learn which journals are most closely related” (p. 1113). Boyack, Börner, and Klavans (2007) estimated that the assignment of WCs is correct in approximately 50% of cases across the file. That said, the “correct” assignment based on detailed article content would usually be proximate.

On the basis of a comparison of this classification with algorithmically generated ones, Rafols and Leydesdorff (2009) (p. 1830) concluded that the WCs can be used for aggregate statistical purposes (i.e. above 100 or so publications, depending on the desired granularity); but are not well-suited for detailed analyses (e.g. to assess an individual’s research). The WCs sometimes cover similar sets of journals; for example, in the domain of biomedicine. In other cases, the categories added by an indexer cover areas that could be considered as separate sub-disciplines or subfields (Leydesdorff & Bornmann, 2016; van Eck et al., 2013). In the case of interdisciplinary publications, problems of imprecise or potentially erroneous classifications can be expected (Rafols & Meyer, 2010)

In scientometric evaluations, journals are sometimes attributed percentages proportional to the categories under which they are subsumed. These multiple categories have also been considered indicators of the interdisciplinarity of journals (Bordons, Bravo, & Barrigon, 2004; Katz & Hicks, 1995; Morillo, Bordons, & Gomez, 2001).

Notwithstanding these issues, WCs are a main basis for scientometric analyses. The use of these journal categories has become conventional among scientometricians (e.g. Rehn et al., 2014), including use to assess research portfolios. For example, InCites—a customized, Web-based research evaluation tool developed by Thomson Reuters—routinely provides normalizations of citation impact using WCs for the delineation of reference sets (e.g. Costas, van Leeuwen, & Bordons, 2010; Leydesdorff, Hammarfelt, & Salah, 2011). The Flemish ECOOM unit for evaluation in Leuven (SOOI) has developed a new classification system for journals (Glänzel & Schubert, 2003). Other authors have refined the journal lists within specific WCs to enable a more precise evaluation of a given discipline (van Leeuwen & Calero Medina, 2012). Another journal classification system in terms of fields and subfields has been made available by Elsevier’s Scopus in the meantime, but Wang and Waltman (2016) found it to be more problematic than WCs, in particular due to the high rate of multiple category assignments of a journal

The field/subfield classification of Scopus is available in the journal list from http://www.elsevier.com/online-tools/scopus/content-overview. WCs are available (under subscription) at http://images.webofknowledge.com/WOKRS56B5/help/WOS/hp_subject_category_terms_tasca.html.

WCs can also be considered “macro-journals” representing fields and subfields of science. Their sub-disciplinary level of detail fits well with a US National Academies recommendation for study of interdisciplinarity (2005). The current (2015 WoS data) matrix of 227 WCs citing one another can be decomposed using multi-variate (e.g. clustering) analysis. It can be analyzed as a network using, for example, community-finding algorithms. Initially (refer to Leydesdorff & Rafols, 2009; Rafols, Porter, & Leydesdorff, 2010), we used 2007 data to develop a global map of science. At that time, drawing a map using the approximately 10,000 journals in the database was technically not feasible due, among other things, to the cluttering of the labels on the screen. This problem was elegantly solved by VOSviewer (which became available in 2009), by allowing interactive zoom in/out functionality in the visualization (Klavans & Boyack, 2009; van Eck & Waltman, 2010)

Available at http://www.vosviewer.com.

Earlier maps were developed into an overlay-toolkit

http://www.leydesdorff.net/overlaytoolkit

We now make some choices differently from the ones we made some ten years ago. The wide use by a variety of stakeholders (including not only some researchers, but also scientometric students and practitioners) and requests for a current database, together with technical improvements in visualization during recent years, lead us to revise the overlay basemaps and toolkit based on the most recent version of the Journal Citation Reports (JCR), i.e. 2015.

We use the combined set of the JCRs 2015 for the Science Citation Index (SCI) (n of journals = 8,778) and the Social Sciences Citation Index (SSCI) (n = 3,212) leading to a total number of 11,365 journals; 625 journals are covered by both databases (Table 1). A JCR for the Arts & Humanities Citation Index is not available, but, in any event, the behavior of those journals’ citation practices differs considerably from that of SCI and SSCI journals (Leydesdorff, Hammarfelt, & Salah, 2011). We also note that Web of Science has expanded its coverage of other research resources, especially conference proceedings and books. Those are not included in the maps presented here.

Table 1

Numbers of journals and Web of Science categories in SCI and SSCI.

The set of WCs covering SCI and SSCI has expanded from 224 in 2010 to 227 in 2015. The three newly added WCs are: “Audiology & speech-language pathology,” “Green & sustainable science & technology,” and “Logic.” The former WC—“Biology, miscellaneous”—was no longer in use in 2010 and, therefore, not included in the analysis; it is also absent from the 2015 data and the current maps.

Using dedicated software, the matrix of 227 × 227 cells was generated on the basis of whole-number citation counting. As previously, we normalize this matrix using the cosine function. However, the default VOSviewer setting normalizes using Zitt, Bassecoulard, and Okubo’s so-called “probabilistic activity index” (PAI) (2000). PAI is equal to the ratio between observed and expected values in a contingency table based on a probability calculus (Equations (1) and (2)): PAI=pij/(pi×<!-- ? -->pj) $$ \it{PAI = p_{ij} \,/ \,(p_i \times p_j)} $$ (1)=nij×<!-- ? -->Σ<!-- S -->iΣ<!-- S -->jnij/Σ<!-- S -->inij×<!-- ? -->Σ<!-- S -->jnij. $$ \it{= n_{ij} \times \Sigma_i\, \,\Sigma_j n_{ij} \,\,/ \,\,\Sigma_i n_{ij} \times \Sigma_{j}n_{ij}.} $$ (2)

In the context of VOSviewer, this measure is renamed as the “association strength” (van Eck & Waltman, 2009).

Unlike the cosine, which is symmetrical, PAI can be used to normalize asymmetrically the vertical and horizontal dimension of a matrix. However, this possible advantage is not exploited in VOSviewer because the matrix is first made symmetric using the sums of lower and upper triangle values (cellij + cellji) in a new matrix. The cosine-normalized matrix remains worth investigating, because one is able to show the difference between the citation as the current activity (citing) versus the cited structures as archival representations (Wouters, 1998).

Taking these issues into consideration, we first develop the citing-side, cosine-normalized map using 2015 data and VOSviewer visualization with default parameter values. This map is a “descendant” of our previous maps; strong relationship can be seen in comparing Figure 1 (our 2015 basemap from VOSviewer) with A-1 in Appendix A (the 2010 basemap from Pajek). A routine for making overlays on the basis of the map (“wc15.exe”) is provided at http://www.leydesdorff.net/wc15/index.htm and described in Appendix B. If the file cosine.dbf is additionally downloaded from the website, the routine writes a value for the Rao-Stirling measure of diversity, which is a proxy of the disciplinary breadth of the publication subset (Stirling, 2007; Zhang, Rousseau, & Glänzel, 2016)

Rao-Stirling diversity is a measure that takes into account both the variety, balance, and the disparity of categories in a distribution. In the case of publication or patent portfolios the categories can be respectively, WCs or IPC classes. The indicator is defined as Equation (3) (Rao, 1982; Stirling, 2007): Δ<!-- ? -->=Σ<!-- S -->ijpipjdij, $$ {\it \Delta} = { \Sigma _{ij}\,\, p_i \,\,p_j \,\,d_{ij},} $$ (3)

where d_ij is a disparity measure between two categories i and j and p_i is the proportion of elements assigned to each category i. As the disparity measure, we use (1 − cosine).

As an additional resource, one can feed the citation matrix of 227 WCs (citing versus cited, but without prior normalization) into VOSviewer and develop a similar map (Appendix A, Figure A-2). Analogous routines as wc15.exe are provided by mtrx15.exe that produces a file “mtrx15.csv” as an input file for the mapping of a portfolio in VOSviewer using the non-normalized citation data (Figure A-2). In the case of a non-normalized matrix, a distance measure is not provided: the number of possible similarity criteria is large (see Klavans & Boyack (in press), and US National Academies (2005)) and the choice can be left to the user (using, for example, SPSS).

The routines also provide cluster and vector files for cos15.paj and matrix.paj made available on the website for Pajek, respectively (as previously). Pajek and Gephi contain a suite of tools for network analysis and visualization such as various decompositions, layouts, and visualization options. Using Pajek or Gephi, for example, one can also obtain the results of the Louvain algorithm (Blondel et al., 2008) for the decomposition in a format that can again be visualized in programs such as VOSviewer or Gephi.

Using VOSviewer, the user can change the number of clusters by changing the resolution parameter and running the clustering algorithm again. Using default values, both maps (i.e. cosine-normalized or not) show five clusters, but chi-square statistics reject the zero-hypothesis that the two classifications are similar (Cramer’s V = 0.707; p < 0.01). The corresponding five colors (blue, red, green, yellow, and pink) will also be used for the overlays, but the user can change this. Changing the granularity requires one to import the file with network data. More detailed instructions can be found in Appendix B and at http://www.leydesdorff.net/wc15/index.htm.

As mentioned in the previous section, the cosine-similarity matrix for the WCs provides both the basis for locating the WCs as nodes in science maps (Figure 1), and the basis to calculate measures of diversity. Footnotes 7 and 8 remind the users of Stirling’s measure and how it can be calculated using the 227-by-227 WC cosine-similarity matrix (see Rafols, Porter, & Leydesdorff (2010) for details).

Porter and colleagues introduced measures of interdisciplinarity and multidisciplinarity called “Integration scores” and “Specialization scores,” extended by Carley and Porter to “Diffusion scores” as well (Carley & Porter, 2012; Porter et al., 2007; Porter & Rafols, 2009). For a given set of publication from WoS, Specialization scores indicate the disciplinary diversity of the set based on the distribution of their WCs. Integration scores reflect the diversity of those publications’ cited references—again, using the cited WCs. Downloading the “cited references” of a given WoS search set allows one to pursue this metric. Conversely, Diffusion scores reflect the diversity of the disciplines citing a given set of papers, based on the citing journals’ WCs. This requires a citation search and data downloading from WoS.

These scores are different instances of the Rao-Stirling diversity measures (Footnote 7) (Stirling, 2007). As introduced earlier in this section, one can obtain the Specialization score (Rao-Stirling diversity for the WCs represented in the WoS search set) along with a science overlay map if desired, directly from the script provided at http://www.leydesdorff.net/wc15.

Integration or Diffusion scores need more detailed computation. Scripts have been prepared to run in VantagePoint software

Scripts available at http://www.vpinstitute.com/.

As what is introduced earlier in this paper, and enabled at the website (http://www.leydesdorff.net/wc15), one can perform a topical search at WoS and take the output as an “analyze.txt” file to enter directly at the site to generate the corresponding science overlay map in VOSviewer. And, as noted, one can vary the resulting overlay maps in several ways in VOSviewer to accentuate points of interest

Another way to compute the maps is to use VantagePoint (http://www.thevantagepoint.com) to process a search set downloaded from WoS. If one mainly wants a science overlay map of the full search set as is, it is easier to output the “analyze.txt” file from WoS for entry into http://www.leydesdorff.net/wc15. However, if you have cause to process the search set data further, VantagePoint provides helpful tools to facilitate data cleaning (e.g. to remove inappropriate items from the search set) or to analyze sub-data sets (e.g. to compare what selected organizations have published on, say, nanotechnology).

The website provides the option to generate either five-cluster science overlay maps or finer scaled (color-differentiated) 18-cluster overlay maps. Both cluster solutions were generated in VOSviewer, using its algorithm

Our previous clustering solutions were generated using factor analyses in SPSS, resulting in 4 “metadisciplines” (see Appendix Figure A-1) and 19 “macro-disciplines” for 2010 base data.