A blended quantitative-ethnographic method for describing vocal sonification in dance coaching

Open access

Abstract

In this paper we present a micro-analytic description of the role vocalizing plays in a single case of professional dance instruction. We use a novel mix of qualitative and quantitative tools in order to investigate, and more thoroughly characterize, various forms of vocal co-organization. These forms involve a choreographer using vocalization to couple acoustic dynamics to the dynamics of their bodily movements, while demonstrating a dance routine, in order to enable watching dancers to coordinate the intrabodily dynamics of their own simultaneous performances. In addition to this descriptive project, the paper also suggests how such forms of coordination might emerge, by identifying those forms of voice-body coupling as potential instances of “instructional vocal sonification”. We offer a tentative theoretical model of how vocal sonification might operate when it is used in the teaching of movement skills, and in the choreographic teaching of dance in particular. While non-vocal sonification (both physical and computer-generated) is increasingly well-studied as a means of regulating coordinated inter-bodily movement, we know of no previous work that has systematically approached vocal sonification. We attempt to lay groundwork for future research by showing how our model of instructional vocal sonification might plausibly account for some of the effects of vocalization that we observe here. By doing so, the paper both provides a solid basis for hypothesis generation about a novel class of phenomena (i.e., vocal sonification), and contributes to bridging the methodological gap between isolated descriptions and statistical occurrences of a given type of event.

Arvaniti, A. (2009). Rhythm, timing, and the timing of rhythm. Phonetica, 66, 46-63.

Auvray, M., Lenay, C., & Stewart, J. (2009). Perceptual interactions in a minimalist virtual environment. New Ideas in Psychology, 27, 32-47.

Baber, C., Chemero, A., & Hall, J. (2017). What the jeweller’s hand tells the jeweller’s brain: Tool use, creativity, and embodied cognition. Philosophy & Technology.

Beckner, C., Ellis, N. C., Blythe, R., Holland, J., Bybee, J., Ke, J., . . . Schoenemann, T. (2009). Language is a complex adaptive system: Position paper. Language Learning, 59(suppl. 1), 1-26.

Bingham, G. P. (1988). Task-specific devices and the perceptual bottleneck. Human Movement Science, 7, 225-264.

Brinck, I. (2016, 8-10 Dec). Joint improvisation in the arts practices: Entrainment, engagement, and expert skill. Proceedings from “A Body of Knowledge - Embodied Cognition and the Arts”, Irvine, CA.

Buhrmann, T., Di Paolo, E., & Barandiaran, X. (2013). A dynamical systems account of sensorimotor contingencies. Frontiers in Psychology, 4, 285.

Burrell, S. (1995). Tune into “show and tell”. First Teacher, 16(1), 12-13.

Butler, E. A. (2011). Temporal interpersonal emotion systems: The “TIES” that form relationships. Personality and Social Psychology Review, 15(4), 367-393.

Cesari, P., Camponogara, I., Papetti, S., Rocchesso, D., & Fontana, F. (2014). Might as well jump: Sound affects muscle activation in skateboarders. PloS One, 9(3), e90156.

Chollet, D., Madani, M., & Micallef, J. P. (1992). Effects of two types of biomechanical bio-feedback on crawl performance. In D. MacLaren, T. Reilly, & A. Lees (Eds.), Biomechanics and medicine in swimming (swimming science) (pp. 48-53). London: E & F Spon.

Cowley S. J. (1998). Of timing, turn-taking, and conversations. Journal of Psycholinguistic Research, 27(5), 541-71.

Cowley, S. J. (2007). Cognitive dynamics and distributed language. Language Sciences, 29(5), 575-583.

Cowley, S. J. (2009). Distributed language and dynamics. Pragmatics & Cognition, 17(3), 495-507.

Cowley, S. J. (2011). Distributed language. Amsterdam: John Benjamins.

Cowley, S. J., & Harvey, M. (2016). The illusion of common ground. New Ideas in Psychology, 42, 56-63.

Cowley, S. J., & Madsen, J. K. (2014). Time and temporality: Linguistic distribution in human life-games. Cybernetics and Human Knowing, 20(3), 172-185.

Cuffari, E. C. (2014). Keep meaning in conversational coordination. Frontiers in Psychology, 5(1397).

Cummins, F. (2009). Rhythm as an affordance for the entrainment of movement. Phonetics, 66, 15-28.

Cummins, F. (2011). Periodic and aperiodic synchronization in skilled action. Frontiers in Human Neuroscience, 5(170), 1-9.

Cummins, F. (2012). Looking for rhythm in speech. Empirical Musicology Review, 7(1-2), 28-35.

Cummins, F., & Simko, J. (2009). Notes on phase coordination, and their application to rhythm and timing in speech. Journal of the Phonetic Society of Japan, 13(3), 1-12.

Dale, R., Fusaroli, R., Duran, N. D., & Richardson, D. C. (2014). The selforganization of human interaction. In B. H. Ross (Ed.), The Psychology of Learning and Motivation (1st ed., pp. 43-96). New York, NY: Elsevier Inc., Academic Press.

De Jaegher, H., & Di Paolo, E. (2007). Participatory sense-making: An enactive approach to social cognition. Phenomenology and the Cognitive Sciences, 6, 485-507.

Di Paolo, E., Rohde, M., & Iizuka, H. (2008). Sensitivity to social contingency or stability of interaction? Modelling the dynamics of perceptual crossing. New Ideas in Psychology, 26, 278-294.

Dubus, G., & Bresin, R. (2013). A systematic review of mapping strategies for the sonification of physical quantities. PLoS One, 8(12), e82491.

Duran, N. D., & Dale, R. (2014). Perspective-taking in dialogue as selforganization under social constraints. New Ideas in Psychology, 32, 131-146.

Dyer, J. F. (2017). Human movement sonification for motor skill learning. PhD thesis. Belfast: Queen’s University.

Effenberg, A. O. (2004). Using sonification to enhance perception and reproduction accuracy of human movement patterns. Proceedings from The International Workshop on Interactive Sonification, Bielefeld.

Effenberg, A. O. (2005). Movement sonification: Effects on perception and action. IEEE Multimedia, 53-59.

Effenberg, A. O. (2007). Movement sonification: Motion perception, behavioral effects, and functional data. Proceedings from 2nd International Workshop on Interactive Sonification, York.

Effenberg, A. O., Fehse, U., Schmitz, G., Krueger, B., & Mechling, H. (2016). Movement sonification: Effects on motor learning beyond rhythmic adjustments. Frontiers in Neuroscience, 10, 219.

Fagergren, E. (2012). ”WA-UM-EI” How a choreographer can use sonification to communicate with dancers during rehearsals. Unpublished thesis. Linköping: Linköping University. Available at http://www.diva-portal.org/smash/get/diva2:546262/FULLTEXT01.pdf.

Fusaroli, R., Perlman, M., Mislove, A., Paxton, A., Matlock, T., & Dale, R. (2016). Timescales of massive human entrainment. PLoS One, 10(4), e0122742.

Fusaroli, R., Rączaszek-Leonardi, J., & Tylén, K. (2014). Dialog as interpersonal synergy. New Ideas in Psychology, 32, 147-157.

Gangopadhyay, N. (2017). The future of social cognition: Paradigms, concepts, and experiments. Synthese, 194(3), 655-672.

Geertz, C. (1973). The interpretation of cultures. London: Fontana.

Gibson, J. J. (1979) The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum Associates.

Goffman, Erving. 1981. Forms of talk. Pennsylvania, PA: University of Pennsylvania Press, 124-159.

Golonka, S. (2015). Laws and conventions in language-related behaviors. Ecological Psychology, 27, 236-250.

Goodwin, C. (1994). Professional vision. American Anthropologist, 96(3), 606-633.

Goodwin, C. (2000). Action and embodiment within situated human interaction. Journal of Pragmatics, 32: 1489-1522.

Goodwin, C. (2002). Time in action. Current Anthropology, 43, 19-35.

Goodwin, C. (2003). The body in action. In J. Coupland and R. Gwyn (Eds.), Discourse, the body and identity (pp. 19-42). New York, NY: Palgrave/Macmillan.

Goodwin, C. (2007). Language, culture, social organization and the material world: Why a five field approach is necessary. Teaching Anthropology: SACC Notes, 13(2), 5-9 34 pages.

Goodwin, C. (2013). The co-operative, transformative organization of human action and knowledge. Pragmatics, 46, 8-23.

Hagendoorn, I. (2004). Some speculative hypotheses about the nature and perception of dance and choreography. Journal of Consciousness Studies, 11(3-4), 79-110.

Haken, H., Kelso, J. A. S., & Bunz, H. (1985). A theoretical model of phase transitions in human hand movements. Biological Cybernetics, 51, 347-356.

Harvey, M. (2015). Content in languaging: Why radical enactivism is incompatible with representational theories of language. Language Sciences, 48, 90-129.

Harvey, M. (2017). Posing-solving can be explained with representations, because it is a form of perception-action. Constructivist Foundations, 13(1), 710-712.

Harvey, M. (forthcoming). Towards an account of linguistic interaction as vocal co-organizing in soundscapes. PhD thesis. Odense: University of Southern Denmark.

Hodges, B. H. (2009). Ecological pragmatics: values, dialogical arrays, complexity and caring. Pragmatics & Cognition, 17(3), 628-652.

Hodges, B. H. (2014). Righting language: A view from ecological psychology. Language Sciences, 41, 93-103.

Hodges, B. H., & Baron, R. M. (2007). On making social psychology more ecological and ecological psychology more social. Ecological Psychology, 19, 79-84.

Hodges, B. H., Steffensen, S. V., & Martin, J. E. (2012). Caring, conversing, and realizing values: new directions in language studies. Language Sciences, 34, 499-506.

Hutchins, E. (1995). Cognition in the wild. Cambridge, MA: MIT Press.

Jarvilehto, T. (2009). The theory of the organism-environment system as a basis for experimental work in psychology. Ecological Psychology, 21(2), 112-120.

Jensen, T. W. (2014). Emotion in languaging: languaging as affective, adaptive, and flexible behaviour in social interaction. Frontiers in Psychology, 5(720), 1-14.

Jewitt C. (2009). The Routledge handbook of multimodal analysis. London: Routledge.

Kelso, J. A. S. (2009). Synergies: Atoms of brain and behavior. In D. Sternad (Ed.), Progress in motor control: A multidisciplinary perspective. New York, NY: Springer.

Kennel, C., Streese, L., Pizzera, A., Justen, C., Hohmann, T., & Raab, M. (2015). Auditory reafferences: the influence of real-time feedback. Frontiers in Psychology, 6(69).

Kimmel, M., & Preuschl, E. (2016). Dynamic coordination patterns in Tango Argentino: A cross-fertilization of subjective explication methods and motion capture. In J.-P. Laumond and N. Abe (Eds.), Dance notations and robot motion, Springer Tracts in Advanced Robotics 111. Switzerland: Springer International Publishing.

Kirby, R. (2009). Development of a real-time performance measurement and feedback system for alpine skiers. Sports Technology, 2, 43-52.

Kirsh, D. (2010). Thinking with the body. In S. Ohlsson & R. Catrambone (Eds.), Proceedings of The 32nd Annual Meeting of the Cognitive Science Society (pp. 2864-2869). Austin, TX: Cognitive Science Society.

Kirsh, D. (2012). Running it through the body. Proceedings of The 34th Annual Cognitive Science Society.

Kirsh, D. (2013). Embodied cognition and the magical future of interaction design. ACM Transactions on Computer-Human Interaction, 20(1), 1-30.

Kramer, G., Walker, B., Bonebright, T., Cook, P., Flowers, J. H., Miner, N., & Neuhoff, J. (2010). Sonification report. Department of Psychology, University of Nebraska at Lincoln.

Laroche, J., Berardi, A. M., & Brangier, E. (2014). Embodiment of intersubjective time: Relational dynamics as attractors in the temporal coordination of interpersonal behaviors and experiences. Frontiers in Psychology, 5(1180).

Latash, M. L. (2015). Bernstein’s “Desired Future” and the physics of human movement. In M. Nadin (Ed.), Anticipation: Learning from the past (pp. 287-299). Zurich: Springer International Publishing.

Linell, P. (2005). The written language bias in linguistics. New York, NY: Routledge.

Linell, P. (2009) Rethinking language, mind, and world dialogically: Interactional and contextual theories of human sense-making. Charlotte, NC: Information Age Publishing.

Linell, P. (2015). Dialogism and the distributed language approach: a rejoinder to Steffensen. Language Sciences, 50, 120-126.

Love, N. (2004). Cognition and the language myth. Language Sciences, 26, 524-544.

MacWhinney, B. (2005). The emergence of linguistic form in time. Connection Science, 17(3-4), 191-211.

Malafouris, L. (2014). How things shape the mind. Cambridge, MA: MIT Press.

Marmelat, V., & Delignières, D. (2012). Strong anticipation: Complexity matching in interpersonal coordination. Experimental Brain Research, 222, 137-148.

McBeath, M. K., Schaffer, D. M., & Kaiser, M. K. (1995). How baseball outfielders determine where to run to catch fly balls. Science, 268(5210), 569-573.

Mechner, F. (1995). Learning and practice in skilled performance. New York, NY: The Mechner Foundation.

Mechsner, F., Kerzel, D., Knoblich, G., & Prinz, W. (2001). Perceptual basis of bimanual coordination. Nature, 414, 69-73.

Mondada L (2008). Doing video for a sequential and multimodal analysis of social interaction: videotaping institutional telephone calls. FQS (Forum: Qualitative Sozialforschung), 9(3). Retrived from http://www.qualitative research.net/index.php/fqs/article/view/1161/2571.

Mondada, L. (2012). Video analysis and the temporality of inscriptions within social interaction: The case of architects at work. Qualitative Research, 12(3), 304-333.

Murgia, M., Sors, F., Vono, R., Muroni, A. F., Delitala, L., Di Corrado, D., & Agostini, T. (2012). Using auditory stimulation to enhance athletes’ strength: An experimental study in weightlifting. Review of Psychology, 19(1), 13-16.

Mylopoulos, M., & Farhat, W. (2014). “I can do better”: Exploring purposeful improvement in daily clinical work. Advances in Health Sciences Education. Theory and Practice, 20(2), 237-283.

Nolan, F., & Jeon, H.-S. (2014). Speech rhythm: A metaphor? Philosophical Transactions of the Royal Society, 369(1568), 1-11.

Norris S. (2004). Analyzing multimodal interaction: A methodological framework. London: Routledge.

Norris S. (2011). Identity in (inter)action: introducing multimodal (inter)action analysis. Frontiers in Human Psychology, 6, 69.

Pedersen, S. B. (2012). Interactivity in health care: Bodies, values and dynamics. Language Sciences, 34(5), 532-542.

Pedersen, S. B., & Steffensen, S. V. (2014). Temporal dynamics in medical visual systems. Cybernetics & Human Knowing, 21, 143-157.

Pedersen, S.B. (2015). The cognitive ecology of human errors in emergency medicine: An interactivity-based approach. Ph.D. thesis. Odense: University of Southern Denmark.

Phillips-Silver, J., & Trainor, L. J. (2005). Feeling the beat: Movement influences infant rhythm perception. Science, 308, 1430.

Port, R. F. (2010). Rich memory and distributed phonology. Language Sciences, 32, 43-55.

Port, R. F., Tajima, K., & Cummins, F. Speech and rhythmic behavior. In G. J. P. Savelsbergh, H. van der Maas, & P. C. L. van Geert (Eds.), The non-linear analyses of developmental processes. Amsterdam: Royal Dutch Academy of Arts and Sciences.

Rączaszek-Leonardi, J. (2010). Multiple time-scales of language dynamics: An example from psycholinguistics. Ecological Psychology, 22(4), 269-285.

Ramenzoni, V. C., Riley, M. A., Shockley, K., & Baker, A. A. (2012). Interpersonal and intrapersonal coordinative modes for joint and single task performance. Human Movement Science, 31, 1253-1267.

Ramezanzade, H., Abdoli, B., Farsi, A., & Sanjari, M. A. L. (2014). The effects of sonification modelling on perception and accuacy of performing jump shot basketball. International Journal of Sports Studies, 4(11), 1388-1392.

Richardson, D. C., Dale, R., & Kirkham, N. Z. (2007). The art of conversation is coordination: common ground and the coupling of eye movements during dialogue. Psychological Science, 18, 407-413.

Riley, M. A., Richardson, M. J., Shockley, K., & Ramenzoni, V. C. (2011). Interpersonal synergies. Frontiers in Psychology, 2, 38.

Rosenblum, L. D. (2008). Speech perception as a multimodal phenomenon. Current Directions in Psychological Science, 17(6), 405-409.

Rosenblum, L. D., Yakel, D. A., Baseer, N., Panchal, A., Nodarse, B. C., & Niehus, R. P. (2002). Visual speech information for face recognition. Perception & Psychophysics, 64(2), 220-229.

Schaffer, D. M., & Mcbeath, M. K. (2002). Baseball outfielders maintain a linear optical trajectory when tracking uncatchable fly balls. Journal of Experimental Psychology: Human Perception and Performance, 28(2), 335-348.

Schaffert, N., & Mattes, K. (2015). Effects of acoustic feedback training in elite-standard Para-Rowing. Journal of Sports Sciences, 33(4), 411-418.

Schaffert, N., Mattes, K., Barrass, S., & Effenberg, A. O. (2009). Exploring function and aesthetics in sonification for elite sports. Proceedings from Second International Conference on Music Communication, Sydney, Australia.

Shockley, K., Richardson, D. C., & Dale, R. (2009). Conversation and coordinative structures. Topics in Cognitive Science, 1, 305-319.

Sigrist, R., Rauter, G., Riener, R., & Wolf, P. (2013). Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review. Psychonomic Bulletin & Review, 20, 21-53.

Sloetjes, H., & Wittenburg, P. (2008). Annotation by category - ELAN and ISO DCR. In Proceedings of The 6th International Conference on Language Resources and Evaluation (LREC 2008).

Sors, F., Murgia, M., Santoro, I., & Agonstini, T. (2015). Audio-based interventions in sport. The Open Psychology Journal, 8(suppl. 3, M6), 212-219.

Steffensen, S. V. (2012). Care and conversing in dialogical systems. Language Sciences, 34, 513-531.

Steffensen, S. V., & Cowley, S.J. (2010). Signifying bodies and health: A nonlocal aftermath. In, S. J. Cowley, S. V. Steffensen, J. C. Major and A. Dinis (Eds.), Signifying bodies: Biosemiosis, interaction and health (pp. 331-356). Braga: Portuguese Catholic University Press.

Steffensen, S. V., & Pedersen, S. B. (2014). Temporal dynamics in human interaction. Cybernetics & Human Knowing, 21, 80-97.

Stepp, N., & Turvey, M. T. (2010). On strong anticipation. Cognitive Systems Research, 11, 148-164.

Stienstra, J., Overbeeke, K., & Wensveen, S. (2011). Embodying complexity through movement sonification: Case study on empowering the speed-skater. Proceedings of The 9th ACM Italian Chapter International Conference on Computer-Human Interaction.

Streeck, J. (2009). Gesturecraft: The manufacture of meaning. Amsterdam: Benjamins.

Streeck, J. (2010). New adventures in language and interaction. Amsterdam: Benjamins.

Streeck, J., Goodwin, C., & LeBaron, C. (2011). Embodied interaction: Language and body in the material world. Cambridge: Cambridge University Press.

Thibault, P. J. (2011). First-order languaging dynamics and second-order language: The distributed language view. Ecological Psychology, 23(3), 210-245.

Thibault, P. J., & King, M. E. (2016). Interactivity, values, and the microgenesis of learning in a tertiary setting. In C. C. Ng, R. Fox, & M. Nakano (Eds.), Reforming learning and teaching in asia-pacific universities (pp. 173-211). Springer Sciences and Business Media Singapore.

Tilsen, S. (2009). Multiscale dynamical interactions between speech rhythm and gesture. Cognitive Science, 33, 839-879.

Tilsen, S., & Arvaniti, A. (2013). Speech rhythm analysis with decomposition of the amplitude envelope: Characterizing rhythmic patterns within and across languages. Journal of the Acoustical Society of America, 134(1), 628-639.

Trasmundi, S. B. (2016) Distribueret kognition og distribueret sprog: analyse af kognitive events i en akutmedicinsk social praksis. Ny Kognitiv Sprogforskning, 50, 55-85.

Trasmundi, S. B., & Steffensen, S. V. (2016). Meaning emergence in the ecology of dialogical systems. Psychology of Language and Communication, 20(2).

Trasmundi, S.B. (forthcoming). The educational value of vocalisations and divergence in multimodal dance choreography. New Ideas in Psychology: Learning as Interactivity, Growth and Becoming: Explorations of Learners, Teachers and Teaching and Learning Environments.

Travis, C. (2008). Occasion-sensitivity. Oxford: Oxford University Press.

Turvey, M. T. (1990). Coordination. American Psychologist, 45(8), 938-953.

Turvey, M. T. (2007). Action and perception at the level of synergies. Human Movement Science, 26, 657-697.

Uryu, M., Steffensen, S. V., & Kramsch, C. (2013). The ecology of intercultural interaction: timescales, temporal ranges and identity dynamics. Language Sciences, 41, 41-59.

van den Herik, J. (2017). Linguistic know-how and the orders of language. Language Sciences, 61, 17-27.

Warren, W. H. (2006). The dynamics of perception and action. Psychological Review, 113, 358-389.

Wilson, A. D., & Golonka, S. (2013). Embodied cognition is not what you think it is. Frontiers in Psychology, 4, 58. doi:

Wilson, A. D., Weightman, A., Bingham, G., & Zhu, Q. (2016). Using task dynamics to quantify the affordances of throwing for long distance and accuracy. Journal of Experimental Psychology: Human Perception and Performance, 42(7), 965.

Journal Information


CiteScore 2017: 0.34

SCImago Journal Rank (SJR) 2017: 0.144
Source Normalized Impact per Paper (SNIP) 2017: 0.359

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 56 56 15
PDF Downloads 58 58 12