Although the perceptual association between verticality and pitch has been widely studied, the link between loudness and verticality is not fully understood yet. While loud and quiet sounds are assumed to be equally associated crossmodally with spatial elevation, there are perceptual differences between the two types of sounds that may suggest the contrary. For example, loud sounds tend to generate greater activity, both behaviourally and neurally, than quiet sounds. Here we investigated whether this difference percolates into the crossmodal correspondence between loudness and verticality. In an initial phase, participants learned one-to-one arbitrary associations between two tones differing in loudness (82dB vs. 56dB) and two coloured rectangles (blue vs. yellow). During the experimental phase, they were presented with the two-coloured stimuli (each one located above or below a central “departure” point) together with one of the two tones.
Participants had to indicate which of the two-coloured rectangles corresponded to the previously-associated tone by moving a mouse cursor from the departure point towards the target. The results revealed that participants were significantly faster responding to the loud tone when the visual target was located above (congruent condition) than when the target was below the departure point (incongruent condition). For quiet tones, no differences were found between the congruent (quiet-down) and the incongruent (quiet-up) conditions. Overall, this pattern of results suggests that possible differences in the neural activity generated by loud and quiet sounds influence the extent to which loudness and spatial elevation share representational content.
If the inline PDF is not rendering correctly, you can download the PDF file here.
Bach D. R. Schahinger H. Neuhoff J. G. Esposito F. Salle F. Di Lehmann C. … Seifritz E. (2008). Rising Sound Intensity: An Intrinsic Warning Cue Activating the Amygdala. Cerebral Cortex18(1) 145–150. https://doi.org/10.1093/cercor/bhm040
Bernstein I. H. & Edelstein B. A. (1971). Effects of some variations in auditory input upon visual choice reaction time. Journal of Experimental Psychology87(2) 241–247. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/5542226
Bien N. ten Oever S. Goebel R. & Sack A. T. (2012a). The sound of size: Crossmodal binding in pitch-size synesthesia: A combined TMS EEG and psychophysics study. NeuroImage59(1) 663–672. https://doi.org/10.1016/j.neuroimage.2011.06.095
Bien N. ten Oever S. Goebel R. & Sack A. T. (2012b). The sound of size: Crossmodal binding in pitch-size synesthesia: A combined TMS EEG and psychophysics study. NeuroImage59(1) 663–672. https://doi.org/10.1016/J.NEUROIMAGE.2011.06.095
Bond B. & Stevens S. S. (1969). Cross-modality matching of brightness to loudness by 5-year-olds. Perception & Psychophysics6(6) 337–339. https://doi.org/10.3758/BF03212787
Brochard R. Dufour A. & Després O. (2004). Effect of musical expertise on visuospatial abilities: Evidence from reaction times and mental imagery. Brain and Cognition54(2) 103–109. https://doi.org/10.1016/S0278-2626(03)00264-1
Bruzzi E. Talamini F. Priftis K. & Grassi M. (2017). A SMARC Effect for Loudness. I-Perception8(6) 204166951774217. https://doi.org/10.1177/2041669517742175
Bueti D. & Walsh V. (2009). The parietal cortex and the representation of time space number and other magnitudes. Philosophical Transactions of the Royal Society of London B: Biological Sciences364(1525) 1831–1840. https://doi.org/10.1098/rstb.2009.0028
Burro R. & Grassi M. (2001). Experiments on size and height of falling objects. Phenomenology of Sound Events IST Project No. IST-2000-25287 Report 1 31–39.
Cabrera D. & Tilley S. (2003a). Parameters for auditory display of height and size. In Proceedings of the 9th International Conference on Auditory Display (ICAD). Boston MA: Georgia Institute of Technology.
Cabrera D. & Tilley S. (2003b). Vertical Localization and Image Size Effects in Loudspeaker Reproduction. In Audio Engineering Society Conference: 24th International Conference: Multichannel Audio The New Reality. Banff Canada: Audio Engineering Society.
Carello C. Anderson K. L. & Kunkler-Peck A. J. (1998). Perception of Object Length by Sound. Psychological Science9(3) 211–214. https://doi.org/10.1111/1467-9280.00040
Critchley M. (1953). The parietal lobes. Oxford England: Williams and Wilkins.
Deroy O. Fernandez-Prieto I. Navarra J. & Spence C. (2018). Unraveling the Paradox of Spatial Pitch. In Spatial Biases in Perception and Cognition (pp. 77–93). Cambridge University Press. https://doi.org/10.1017/9781316651247.006
Dolscheid S. Hunnius S. Casasanto D. & Majid A. (2014). Prelinguistic Infants Are Sensitive to Space-Pitch Associations Found Across Cultures. Psychological Science25(6) 1256–1261. https://doi.org/10.1177/0956797614528521
Douglas K. M. & Bilkey D. K. (2007). Amusia is associated with deficits in spatial processing. Nature Neuroscience10(7) 915–921. https://doi.org/10.1038/nn1925
Dubus G. & Bresin R. (2013). A Systematic Review of Mapping Strategies for the Sonification of Physical Quantities. PLoS ONE8(12) e82491. https://doi.org/10.1371/journal.pone.0082491
Eitan Z. & Granot R. Y. (2006). How Music Moves. Music Perception: An Interdisciplinary Journal23(3). https://doi.org/10.1525/mp.2006.23.3.221
Eitan Z. Schupak A. & Marks L. E. (2008). Louder is Higher: Cross-Modal Interaction of Loudness Change and Vertical Motion in Speeded Classification. In Proceedings of the 10th international conference on music perception and cognition (ICMP10).
Evans K. K. & Treisman A. (2011). Natural cross-modal mappings between visual and auditory features. Journal of Vision10(1) 6–6. https://doi.org/10.1167/10.1.6
Fernández-Prieto I. & Navarra J. (2017). The higher the pitch the larger its crossmodal influence on visuospatial processing. Psychology of Music45(5) 713–724. https://doi.org/10.1177/0305735616684205
Fernández-Prieto I. Navarra J. & Pons F. (2015). How big is this sound? Crossmodal association between pitch and size in infants. Infant Behavior and Development38 77–81. https://doi.org/10.1016/j.infbeh.2014.12.008
Fernandez-Prieto I. Spence C. Pons F. & Navarra J. (2017). Does Language Influence the Vertical Representation of Auditory Pitch and Loudness? I-Perception8(3). https://doi.org/10.1177/2041669517716183
Ferri F. Tajadura-Jiménez A. Väljamäe A. Vastano R. & Costantini M. (2015). Emotion-inducing approaching sounds shape the boundaries of multisensory peripersonal space. Neuropsychologia70 468–475. https://doi.org/10.1016/J.NEUROPSYCHOLOGIA.2015.03.001
Foster N. E. V. Halpern A. R. & Zatorre R. J. (2013). Common parietal activation in musical mental transformations across pitch and time. NeuroImage75 27–35. https://doi.org/10.1016/j.neuroimage.2013.02.044
Foster N. E. V. & Zatorre R. J. (2010a). A role for the intraparietal sulcus in transforming musical pitch information. Cerebral Cortex (New York N.Y. : 1991)20(6) 1350– 1359. https://doi.org/10.1093/cercor/bhp199
Foster N. E. V. & Zatorre R. J. (2010b). Cortical structure predicts success in performing musical transformation judgments. NeuroImage53(1) 26–36. https://doi.org/10.1016/j.neuroimage.2010.06.042
Gallace A. & Spence C. (2006). Multisensory synesthetic interactions in the speeded classification of visual size. Perception & Psychophysics68(7) 1191–1203. https://doi.org/10.3758/bf03193720
Garner W. R. & Sutliff D. (1974). The effect of goodness onencoding time in visual pattern discrimination. Perception & Psychophysics16(3) 426–430. https://doi.org/10.3758/BF03198567
Ghazanfar A. A. Neuhoff J. G. & Logothetis N. K. (2002). Auditory looming perception in rhesus monkeys. Proceedings of the National Academy of Sciences of the United States of America99(24) 15755–15757. https://doi.org/10.1073/pnas.242469699
Hall D. A. & Moore D. R. (2003). Auditory Neuroscience: The Salience of Looming Sounds. Current Biology13(3) R91–R93. https://doi.org/10.1016/S0960-9822(03)00034-4
Jacobsen T. Horenkamp T. & Schröger E. (2003). Preattentive memory-based comparison of sound intensity. Audiology & Neuro-Otology8(6) 338–346. https://doi.org/73518
Lewkowicz D. J. & Minar N. J. (2014). Infants Are Not Sensitive to Synesthetic Cross-Modality Correspondences: A Comment on Walker et al. (2010). Psychological Science25(3) 832–834. https://doi.org/10.1177/0956797613516011
Lewkowicz D. J. & Turkewitz G. (1980). Cross-modal equivalence in early infancy: Auditory-visual intensity matching. Developmental Psychology16(6) 597–607. https://doi.org/10.1037/0012-16188.8.131.527
Lidji P. Kolinsky R. Lochy A. & Morais J. (2007). Spatial associations for musical stimuli: A piano in the head? Journal of Experimental Psychology: Human Perception and Performance33(5) 1189–1207. https://doi.org/10.1037/0096-15184.108.40.2069
Lipscomb S. D. & Kim E. M. (2004). Perceived match between visual parameters and auditory correlates: an experimental multimedia investigation. In S. Lipscomb R. Ashley R. Gjerdingen & P. Webster (Eds.) Proceedings of the 8th International Conference on Music Perception & Cognition (ICMPC8). Evanston IL USA: Adelaide Australia: Casual Productions.
Marks L. E. (1974). On Associations of Light and Sound: The Mediation of Brightness Pitch and Loudness. The American Journal of Psychology87(1/2) 173. https://doi.org/10.2307/1422011
Marks L. E. (1987). On cross-modal similarity: Auditory–visual interactions in speeded discrimination. Journal of Experimental Psychology: Human Perception and Performance13(3) 384–394. https://doi.org/10.1037/0096-15220.127.116.114
Melara R. D. & O’Brien T. P. (1987). Interaction between synesthetically corresponding dimensions. Journal of Experimental Psychology: General116(4) 323–336. https://doi.org/10.1037/0096-3418.104.22.1683
Näätänen R. Paavilainen P. Alho K. Reinikainen K. & Sams M. (1989). Do event-related potentials reveal the mechanism of the auditory sensory memory in the human brain? Neuroscience Letters98(2) 217–221. https://doi.org/10.1016/0304-3940(89)90513-2
Näätänen R. Paavilainen P. Rinne T. & Alho K. (2007). The mismatch negativity (MMN) in basic research of central auditory processing: A review. Clinical Neurophysiology118(12) 2544–2590. https://doi.org/10.1016/j.clinph.2007.04.026
Neuhoff J. G. (1998). Perceptual bias for rising tones. Nature395(6698) 123–124. https://doi.org/10.1038/25862
Occelli V. Spence C. & Zampini M. (2009). Compatibility effects between sound frequency and tactile elevation. NeuroReport20(8) 793–797. https://doi.org/10.1097/WNR.0b013e32832b8069
Parise C. V. & Spence C. (2009). ‘When Birds of a Feather Flock Together’: Synesthetic Correspondences Modulate Audiovisual Integration in Non-Synesthetes. PLoS ONE4(5) e5664. https://doi.org/10.1371/journal.pone.0005664
Parise C. V. (2016). Crossmodal Correspondences: Standing Issues and Experimental Guidelines. Multisensory Research29(1–3) 7–28. https://doi.org/10.1163/22134808-00002502
Parkinson C. Kohler P. J. Sievers B. & Wheatley T. (2012). Associations between Auditory Pitch and Visual Elevation Do Not Depend on Language: Evidence from a Remote Population. Http://Dx.Doi.Org/10.1068/P7225. https://doi.org/10.1068/P7225
Phillips J. G. & Triggs T. J. (2001). Characteristics of cursor trajectories controlled by the computer mouse. Ergonomics44(5) 527–536. https://doi.org/10.1080/00140130121560
Rinne T. Särkkä A. Degerman A. Schröger E. & Alho K. (2006). Two separate mechanisms underlie auditory change detection and involuntary control of attention. Brain Research1077(1) 135–143. https://doi.org/10.1016/j.brainres.2006.01.043
Root R. T. & Ross S. (1965). Further Validation of Subjective Scales for Loudness and Brightness by Means of Cross-Modality Matching. The American Journal of Psychology78(2) 285. https://doi.org/10.2307/1420502
Rusconi E. Kwan B. Giordano B. L. Umiltà C. & Butterworth B. (2006). Spatial representation of pitch height: the SMARC effect. Cognition99(2) 113–129. https://doi.org/10.1016/j.cognition.2005.01.004
Schröger E. (1996). The influence of stimulus intensity and inter-stimulus interval on the detection of pitch and loudness changes. Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section100(6) 517–526. https://doi.org/10.1016/S0168-5597(96)95576-8
Sluming V. Brooks J. Howard M. Downes J. J. & Roberts N. (2007). Broca’s Area Supports Enhanced Visuospatial Cognition in Orchestral Musicians. Journal of Neuroscience27(14) 3799–3806. https://doi.org/10.1523/JNEUROSCI.0147-07.2007
Smith L. B. & Sera M. D. (1992). A developmental analysis of the polar structure of dimensions. Cognitive Psychology24(1) 99–142. https://doi.org/10.1016/0010-0285(92)90004-L
Spence C. (2011). Crossmodal correspondences: A tutorial review. Attention Perception & Psychophysics73(4) 971–995. https://doi.org/10.3758/s13414-010-0073-7
Spence C. & Deroy O. (2013). How automatic are crossmodal correspondences? Consciousness and Cognition22(1) 245–260. https://doi.org/10.1016/j.concog.2012.12.006
Stevens J. & Marks L. (1965). Cross-modality matching of brightness and loudness. In Proceedings of the National Academy of Sciences (Vol. 54.2 pp. 407–411).
Tajadura-Jiménez A. Väljamäe A. Asutay E. & Västfjäll D. (2010). Embodied auditory perception: The emotional impact of approaching and receding sound sources. Emotion10(2) 216–229. https://doi.org/10.1037/a0018422
Tillmann B. Jolicœur P. Ishihara M. Gosselin N. Bertrand O. Rossetti Y. & Peretz I. (2010). The Amusic Brain: Lost in Music but Not in Space. PLoS ONE5(4) e10173. https://doi.org/10.1371/journal.pone.0010173
Walker P. Bremner J. G. Mason U. Spring J. Mattock K. Slater A. & Johnson S. P. (2010). Preverbal Infants’ Sensitivity to Synaesthetic Cross-Modality Correspondences. Psychological Science21(1) 21–25. https://doi.org/10.1177/0956797609354734
Walker P. Bremner J. G. Mason U. Spring J. Mattock K. Slater A. & Johnson S. P. (2014). Preverbal Infants Are Sensitive to Cross-Sensory Correspondences: Much Ado About the Null Results of Lewkowicz and Minar (2014). Psychological Science25(3) 835–836. https://doi.org/10.1177/0956797613520170
Walker R. (1987). The effects of culture environment age and musical training on choices of visual metaphors for sound. Perception & Psychophysics42(5) 491–502. https://doi.org/10.3758/BF03209757
Walsh V. (2003). A theory of magnitude: common cortical metrics of time space and quantity. Trends in Cognitive Sciences7(11) 483–488. https://doi.org/10.1016/j.tics.2003.09.002
Walsh V. Gallistell R. C. Gellman R. Brannon E. M. Roitman J. D. Rossetti Y. … Fischer M. H. (2003). A theory of magnitude: common cortical metrics of time space and quantity. Trends in Cognitive Sciences7(11) 483–488. https://doi.org/10.1016/J.TICS.2003.09.002
Wicker F. W. (1968). Mapping the Intersensory Regions of Perceptual Space. The American Journal of Psychology81(2) 178. https://doi.org/10.2307/1421262
Zahorik P. Brungart D. S. & Bronkhorst A. W. (2005). Auditory Distance Perception in Humans: A Summary of Past and Present Research. Acta Acustica United with Acustica91(3) 409–420.