Published Online: Sep 30, 2018
Page range: 157 - 171
Received: Dec 18, 2017
Accepted: May 11, 2018
DOI: https://doi.org/10.2478/mgrsd-2018-0014
Keywords
© 2018 Katarzyna Słomska, published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.
Empirical studies in cartography allow users’ perceptions of maps and GIS to be explored. Consequently, cartographic and GIS products may be adapted to user needs. The first attempts were inspired by the book
The development of this research field has been described by e.g. Montello (2002; 2009) and Żyszkowska (2015). There have been two important shifts in researchers’ approaches. In the very beginning, cartographers focused on particular elements of a map and tested them in isolation. For instance, Flannery (1956), who studied the use of circles in proportional symbols maps. The first revision was influenced by, among others, the work of Petchenik (1977), who highlighted that a map constitutes a unity in itself and suggested that a holistic approach would be more suitable. This initiated a switch from testing specific elements of a map to analysing the perception of a map as a whole.
According to Montello (2009) another turning point for empirical research conducted by cartographers was initiated by technological development in the late 1990s. Easier access to new technologies, software and devices caused the interest of cartographers to turn to the cognition of maps displayed on screens and, with time, interactive maps. What is more, new types of geovisualisations came about. The new century brought new opportunities for the development of the field. New methods are used, but also the scope of geovisualisations and types of maps is wider. Nonetheless, in its every shape, empirical research was and still is a significant part of cartography. That is why analysis of its achievements is crucial for further development. The literature lacks analyses of the progress of cognitive cartography in the 21st century. What is more, it cannot be studied separately from the field of geovisualisation, as they influence each other.
Over time, the field of cognitive cartography has been revised by several researchers. Gilmartin (1992) reviewed the content of three journals:
Review papers attempt to identify research topics in cognitive studies in cartography by analysing the scope of research. Slocum et al. (2001) propose classification of research themes in geovisualisation in a cognitive context. The following topics are enumerated: geospatial virtual environments, dynamic representations, user interface design, individual and group differences, collaborative geovisualisation, and evaluating the effectiveness of geovisualisation methods. Vast descriptions of the evolution of the field with a division for particular topics, such as, e.g., education, ontology and the impact of empirical studies on cartographic practice have been described by Montello (2002) and Montello and Freundschuh (2005). Moreover, Montello (2009) focused on the achievements of empirical research in the context of GIS within the scope of navigation, geovisualisation, human factors in GIS, education and ontology. All these aforementioned papers suggested further research challenges to be addressed. However, the issues they focused on were different from one another; Montello (2002) recommended analysing the history of cognitive map-design research, whereas Slocum et al. (2001) endorsed the development of an interdisciplinary approach.
Roth (2013a) applied another approach to reviewing the achievements of experimental studies in cartography. The author juxtaposed 25 classifications of cartographic interactions published from 1987 to 2008. On the basis of this overview, a taxonomy of cartographic interaction primitives based on objective, operator and operand was proposed. The analysis outlined the most common tasks used in research in the context of user goal, data and the interface. Moreover, definitions and prioritised interaction primitives were provided. This work is a systematisation of previous knowledge and it constitutes a framework for future studies and makes them easier to compare and validate.
The same aim was shared by White (2017), who created a consistent database of 200 cartographical user-studies in order to gauge them. He focused on participants who took part in experiments in the contexts of recruitment, verification and evaluation. This overview can be recognised as a base of good and bad practices, so that common mistakes would not be perpetuated in later studies.
Issues such as evolution of the field, main research themes, the importance of participant selection and dimensions of interaction were analysed and described. Notably, the maps that are used as stimuli in empirical cartographical research never underwent revision.
The preparation of the empirical study in cognitive cartography is crucial to the quality of their results and conclusions drawn. Every aspect – such as the selection of stimuli, participants or methods – has to be conducted deliberately, as it will have an impact on the results.
Roth’s paper (2013a) constitutes a canvas for standardising experimental studies. A similar approach may be applied towards other elements of an empirical study in cartography. Despite the study areas in cartography being different, such as geovisualisation, cognitive cartography, communication theory or modelling approach, all cartographers have one common interest – a map.
The main aim of this paper is to present types of maps used as stimuli in empirical studies in cartography in the context of the user and researcher perspective. Such revisions could possibly bring such opportunities as the taxonomy of interaction primitives proposed by Roth (2013a). In many papers it is not stated clearly whether elements of the study were interactive and if so, which, nor whether the participants had previous knowledge of the data or the area depicted on the map, etc. In some cases, there is also no graphic example of a map included. Organised description of the stimuli would provide an opportunity for repetition and, consequently, verification of results.
Therefore, the material to be analysed must be delimited. As stated before, it was decided to focus on papers published since 2000. The query covers journals affiliated to the ICA:
The review juxtaposed 103 research papers presenting an experiment where a map was used as the stimulus. They constituted 8.6% of papers published in the mentioned journals since 2000 (Table 1), so the stake is three times smaller than in the late 1970s (Gilmartin 1992). What is interesting is that the
Number of analysed papers per journal (since 2000) Source: own elaborationTitle of a journal Published since Number of published papers since 2000 (nPp) Number of analysed papers since 2000 (nAp) The Cartographic Journal 1964 422 40 9.5% Cartographica: The International Journal for Geographic Information and Geovisualization 1965 273 17 6.2% Cartography and Geographic Information Science 1973 465 39 8.4% International Journal of Cartography 2015 36 7 19.4 %
In the analysed period the number of research papers concerning empirical studies in cartography has risen (Fig. 1). It is worth noticing that
Number of analysed papers per year Source: own elaborationFigure 1
The attributes of maps used for the identification of map types were chosen on the basis of the author’s description of stimuli. Four main features were chosen (Table 2, Fig. 2): map medium (paper, screen), reactiveness (interactive, non-interactive), method of cartographic presentation (quantitative, qualitative), users’ familiarity with the presented area and data (familiar, unfamiliar, fictional). In most experiments more than one type of map was used, but the aim was rarely to compare them. Such experiments are presented in figure 2 as transitional areas, marked by vertical banding in the middle of the figure, in order to point out differences in overlapping proportions between features. The same solution was applied to the graphs, which show proportions of different types through the analysed period (Fig. 3, Fig. 5, Fig. 6, Fig. 7). The green parts of bars in figures 3, 5, 6 and 7 and the vertical banding in figure 2 depict cases when both types were used in the same study.
Types of maps used in analysed papers Source: own elaborationFeature Medium Interactivity Method of cartographic presentation Familiarity with data or area 19 89 68 40 91 59 14 35 64 18,4% 86,4% 66,0% 38,9% 88,3% 57,3% 13,6% 34,0% 62,1%
Types of maps Source: own elaboration
Percent of map types used in analysed papers Source: own elaborationFigure 2
Percentage of paper and screen maps Source: own elaborationFigure 3
Relation between medium and level of reactiveness Source: own elaboration based on Kraak & Brown 2001Figure 4
Percentage of interactive and non-interactive maps Source: own elaborationFigure 5
Percentage of maps elaborated with qualitative and quantitative methods of cartographic presentation Source: own elaborationFigure 6
Percentage of maps of fictional, familiar and unfamiliar data or area Source: own elaborationFigure 7
Unfortunately, it is not possible to specify the number of maps used as stimuli in the analysed papers, as authors do not always provide this information.
The paper map may be considered the most traditional form. Although digital natives, who are proficient in technology, may associate the word “map” with a display on the screen of a laptop, smartphone or navigation device (Prensky 2001). There are more possibilities for presenting a map, but paper and screen are the most common media of transmission.
The stimuli of the first studies were only presented on paper. With the development of technology and research methods, display on a screen has become more common. Since 2000, about 86% of maps used in research have been displayed on a screen (Table 2, Fig. 2). They may be in different forms, from a raster of a scanned and calibrated paper map to a vector map outputted by a database (Ooms et al. 2012). It is worth noticing that the disproportion between use of paper and screen as a medium is large compared to differences among map types in other aspects. Screens were used as the medium for a map four times more often than paper (Table 2). This may be in response to user preferences. Users’ turn from paper to digital maps is noticeable, especially when it comes to the navigational functions (Axon et al. 2012; Speake & Axon 2013; Speake 2016). The disproportion may also be connected with the use of map applications prepared for the study and/or use of eye-tracking, which is often used as a supplementary method (Ooms et al. 2014; Opach et al. 2014). Applications facilitate control over the flow of the experiment, measurement of time needed for particular questions, and analysis of eye and mouse movements (e.g. open-source OGAMA, Vosskühler et al. 2008).
Traditional maps are disappearing from everyday life, and as a consequence are not often selected as stimuli in research. They were used in only 18% of analysed cases (Table 2.) Providing ecological validity (Carter et al. 2008) and obtaining comparable results were always a challenge for researchers. This obstacle may be easily overcome, as most study participants (who are mainly recruited from among students or faculty staff) use a computer or smartphone on a daily basis (Slocum et al. 2004). Problems may arise in the case of recruiting participants from an elderly population. Nonetheless, differences in level of proficiency in the use of computer and devices like smartphones must be taken into consideration (Andrienko et al. 2002; Opach & Rød 2014). What is interesting is that researchers may nowadays also include the possibility of inequalities in proficiency using paper maps, as young people and/or novice users consider it to be more challenging (Speake 2016). It may be thought-provoking that the percentage of paper maps used in experiments has risen (Fig. 3).
By level of reactiveness, maps can be divided into interactive and non-interactive (view only) (Kraak & Brown 2001). The reactiveness of a map is strongly connected with its medium, as all paper maps are non-interactive (Fig. 4). Digital maps can be interactive or non-interactive. The relationship between the discussed map features is presented in the following diagram (Fig. 4), where the horizontal dotted line represents the distinction between them and uninterrupted lines show relations between.
However, this type of view-only map, which is not editable, can also be displayed on a screen. Non-interactive maps constituted 66.0% of stimuli displayed on the screen (Table 2, Fig. 2). On the other side, interactive maps are inseparably related to electronic devices. Examples may be identified among geoportals, map applications, or GIS software.
Oxford Dictionaries defines reactive as “showing a response to a stimulus”, and interactive as “allowing a two-way flow of information between a computer and a computer-user; responding to a user’s input” (Oxford Dictionaries 2017). What is more, the map is given as an example (“a fully interactive map of the area”).
Responding to the map-readers’ input is also stressed in another definition, which states that an “interactive map reciprocates spatial information between map and map-reader” (Cartwright et al. 2007, p 156). According to Roth (2013b, p 64) cartographic interaction is “a dialogue between a human and a map mediated through a computing device”. In every definition, user and map are equal subjects.
Emphasis should be put on the word “map”, as its content should be interactive, not only the environment (e.g. the interface). Furthermore, both discussed types of maps can be dynamic (Kraak & Brown 2001). That is why, in this paper, dynamic display (animation) is not considered as a form of interaction, as was also the case in a study by Saint-Marc et al. (2017): neither are panning or zooming of a static display, unless it is a semantic zoom, which involves changes in the level of data generalisation (Tanaka & Ichikawa 1988).
The differences in usability of non-interactive and interactive maps is one of a prevailing research questions in cartography (Roth et al. 2017). The comparison may state a challenge, because of the differences between them (e.g. medium, graphic variables). An even greater obstacle may be presented by the difficulty in comparing experiment results, as an interactive map may enable automatic measurements of length and area, filtration and selection of data. These functions substitute skills obtained during map usage. It is one of the causes of the shift in user preferences (Axon, Speake & Crawford 2012).
The need for a comparison of non-interactive and interactive maps is crucial mainly due to the use of maps in education. According to Harvey and Kotting (2011), active learning with interactive display matches the needs of digital natives. Downsides may also be recognised, as software substitutes thinking because it processes information for users (Montello 2009). At this point, it is also worth tackling the issue of level of proficiency in using interactive maps among participants, as in the case of the map medium.
Non-interactive maps are more often used as a stimulus (66% of analysed studies) (Table 2, Fig. 2). There may be several causes for this. First, they allow studies of holistic map perception to be conducted, as users cannot choose layers to be displayed, change symbolisation or colours. When it comes to interactive maps, cartographers have to pre-define the limitations of the modifications (Kraak & Brown 2001; Kraak & Ormeling 2015). Therefore, the purpose is to check the usability of one of the aspects, or the interface. However, the disproportion is not as big as in the case of the map medium. Interactive maps were used in 39% of studies (Table 2.) It is interesting that the proportion is stable over time, since it would be expected that the share of interactive stimuli should rise. In five papers, interactive and non-interactive displays are used to simultaneously show different information (Behrens et al. 2015) (Fig. 2).
For the purpose of identification of map types, methods of cartographic presentation are considered in the context of data. In cartographic literature the distinction between qualitative, quantitative and ordinal data is used. “Qualitative” means records are distinguishable on the basis of nominal features (e.g. lake, river, pond). “Ordinal” is used when the features can be ranked (small, medium, big). “Quantitative” means that attributes are measurable (e.g. number of people, area, harvest in tonnes) (Robinson et al. 1995; Slocum 2010).
The character of the data has an impact on the choice of method of cartographic presentation. Qualitative data are used for: symbols (point, line, area) or chorochromatics. Quantitative and ordinal data are used for: diagrams, choropleths, isarithms, dots or quantitative signatures, which will subsequently be described as a group of quantitative methods.
Qualitative methods are much more often used in the analysed material (88%) (Table 2.) They are most commonly used as city plans, tourist maps or maps for navigation. Little cartographic experience is needed to use them efficiently and effectively. They are quicker to understand when the required response time is measured.
The use of quantitative methods was more infrequent (57%). They are used for depicting statistical data (Sun & Li 2010). Participants with little expertise in cartography may find these map harder to interpret, and as a result longer response times may be needed.
In 43% of analysed cases both types of methods were used (Fig. 2). The most common example of this solution is a topographic map with contours (Ooms, et al. 2014; Murakoshi & Higashi 2016; Ooms et al. 2016). Using it as a stimulus is coherent with ecological validity (Carter et al. 2008), as this type of map is commonly used, e.g. as the basis for tourist maps.
Based on the analysed papers, maps were divided into three types, depicting data that was: fictional, familiar to the participant, or unfamiliar. Each type may refer to area and statistics. The base-map was not taken into consideration if questions did not refer to it. As in the case of, for example, a map depicting statistical data attributed to administrative units to a resident of the country in question (Harrower 2007; Opach & Rød 2014).
It may seem that users’ previous experience and knowledge is not an attribute of the stimulus, but of the study participants. Nonetheless, the stimuli must be carefully selected so as not to disrupt the findings, as the results may vary depending on participants’ knowledge and experience.
This problem may be avoided by detailed selection of participants and establishment of a homogeneous group or preparation of a questionnaire and selection of data after the experiment. However, using a map of a distant or fictional territory as a stimulus may be less complicated than detailed verification of participants.
In over 60% of studies this most secure option was chosen and researchers decided to use a map of a distant region (Ooms et al. 2012) or statistical data (Fish et al. 2011) that was unlikely to be familiar to participants (Table 2). Use of a known area (34%) may be hazardous, as participants rarely have comparable knowledge about the depicted territory. This issue is commonly present when large scale maps are used (e.g. plan of a city, topographic map). In some of the analysed instances only a portion of the participants was familiar with the data. This happened even in the case when the goal was not to compare effectiveness based on knowledge (Hochmair 2009) (Fig. 2).
The case of maps of fictional areas is very interesting. It can be stated that they should be included to “unknown”. According to the definition of the International Cartographic Association, a map is a “symbolised representation of geographical reality” (A Strategic Plan for the ICA 2003-2011, p 17). By this definition, graphic representations of a fictional area cannot be called a “map”, but recognised as a substitute. Three types of fictional maps may be enumerated: explicitly fictional (Lloyd & Bunch 2005), maps pretending to present a real area (Fabrikant et al. 2008) or maps modified for the needs of a study (Gołębiowska 2015). There are several advantages and disadvantages of that solution. The main benefit is that participants do not know the depicted area. However, the objects on fictional maps often look artificial (e.g. trajectory of roads, borders or angular rivers).
The analysis was narrowed to papers published since 2000 in journals affiliated to the International Cartographic Association. A broader review would possibly show a wider scope of direction of experiments. What is more, if all types of geovisualisation were included the analysis would be extensive, as other types of aspects could be pointed out.
Unfortunately, the aspect of map complexity, which is a very important feature in empirical research, could not be assessed, as not all of the maps used in the analysed studies were included in the papers.
The elaboration of a consistent database of map types is a continuation of the work by Roth (2013a) and White (2017) and another step towards enhancing the reporting of empirical studies in cartography. Complying with this postulate would allow studies to be better validated and compared. In order to apply this proposal, explicit characteristic were defined. This allows this database to be further developed. Furthermore, the described database could be integrated with the database concerning participants by White (2017), which would allow profound insight to be gained into the empirical research conducted by cartographers since 2000.
By juxtaposing maps according to objective criteria and performing a meta-analysis of the gathered data, the extensive group of studies can be more precisely characterised. The conclusions may serve other researchers who would like to make informed decisions when preparing their studies.
In the scope of every feature, one of the types could be observed to dominate (Table 2, Fig. 2). The most common types are maps displayed on a screen (88%), non-interactive (66%), qualitative (88%) and unfamiliar (62%). Maps that fall into all these dominant categories simultaneously were the subject of 25% of all experimental studies.
It is interesting that maps depicting qualitative data constituted 87% of interactive maps and 89% of non-interactive maps, 87% of screen maps and 94% of paper maps, so the prevalence of this feature is balanced. In the scope of the methods of cartographic presentation, two types (qualitative and quantitative) occurred simultaneously most frequently – in 47 cases (Fig. 2, Fig. 5). When it comes to map medium and reactiveness, both types were used in the same study five times (Fig. 2, Fig. 4). However, the disproportion between the use of paper and screen maps and between interactive and non-interactive maps is more significant (Fig. 2, Fig. 3).
In summary, meticulous description of experimental studies is crucial for proper construction of forthcoming research. Revisions of conducted experiments may constitute a frame for reporting study results.