Behavioural changes and multimodal interaction within a performative space: an interdisciplinary investigation

• Grant number: 16/22619-0
• Support type: Scholarships in Brazil - Post-Doctorate
• Effective date (Start): February 01, 2019
• Status: Discontinued
• Field of knowledge: Linguistics, Literature and Arts - Arts - Music
• Principal researcher: Jônatas Manzolli
• Grantee: Elena Partesotti
• Home Institution: Núcleo Interdisciplinar de Comunicação Sonora (NICS). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil

Abstract

This project aims to provide a novel contribution and new insights in the field of motor learning through the use of musical and multimodal interactive technology. The proposed study will investigate how audiovisual-motor learning generates and strengthens new sensorimotor maps based on crossmodal encoding, in order to test the efficacy of the prototype e-mocomu developed at NICS in 2013, in healthy subjects. Hence, the subjects' behaviour within interactive and performative spaces and the brain's reaction/response to audiovisual and motor stimuli will be measured by conducting an appropriate pilot study using functional Magnetic Resonance Imaging (fMRI). The proposed program derives from my now finalized doctoral research: one grounded on the paradigm of embodied cognition (theorized by Johnson, 1980), the Sensorimotor Contingency Theory (O'Regan & Nöe, 2011), the Theory of Visual Perception (Gibson, 1979), music therapy and human computer interaction, and aimed at measuring how subjects engage with an interactive environment on the assumption that their actions are mediated by their motivation and perceptions. As a consequence, one of the paramount tasks of this project will be to work on the further development of the "Interactive Media Laboratory for Digital Immersion and Augmented Cognition" (ImCognita) that was developed within the umbrella of the research project "Criação e análise musicais a partir do estudo de processos cognitivos com suporte de mídia interativa e imersão digital", that received financial support by CNPq, process no. 470358/2014-9. To further develop the research tasks of the present proposal, we will collaborate with the FAPESP CEPID Brazilian Institute for Neuroscience and Neurotechnology (BRAINN), UNICAMP, process no. 2013/07559-3. Later on, this project paves to consider the inclusion of stroke patients to test the auditory-motor memorization that takes place during the rehabilitation process in people suffering from cerebral stroke. (AU)