Behavioural changes and multimodal interaction within a performative space: an interdisciplinary investigation

• Grant number: 16/22619-0
• Support Opportunities: Scholarships in Brazil - Post-Doctoral
• Start date: February 01, 2019
• End date: January 15, 2025
• Field of knowledge: Linguistics, Literature and Arts - Arts - Music
• Principal Investigator: Gabriela Castellano
• Grantee: Elena Partesotti
• Host Institution: Núcleo Interdisciplinar de Comunicação Sonora (NICS). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil
• Associated research grant: 13/07559-3 - BRAINN - The Brazilian Institute of Neuroscience and Neurotechnology, AP.CEPID

Abstract

This project aims to provide a novel contribution and new insights in the field of motor learning through the use of musical and multimodal interactive technology. The proposed study will investigate how audiovisual-motor learning generates and strengthens new sensorimotor maps based on crossmodal encoding, in order to test the efficacy of the prototype e-mocomu developed at NICS in 2013, in healthy subjects. Hence, the subjects' behaviour within interactive and performative spaces and the brain's reaction/response to audiovisual and motor stimuli will be measured by conducting an appropriate pilot study using functional Magnetic Resonance Imaging (fMRI). The proposed program derives from my now finalized doctoral research: one grounded on the paradigm of embodied cognition (theorized by Johnson, 1980), the Sensorimotor Contingency Theory (O'Regan & Nöe, 2011), the Theory of Visual Perception (Gibson, 1979), music therapy and human computer interaction, and aimed at measuring how subjects engage with an interactive environment on the assumption that their actions are mediated by their motivation and perceptions. As a consequence, one of the paramount tasks of this project will be to work on the further development of the "Interactive Media Laboratory for Digital Immersion and Augmented Cognition" (ImCognita) that was developed within the umbrella of the research project "Criação e análise musicais a partir do estudo de processos cognitivos com suporte de mídia interativa e imersão digital", that received financial support by CNPq, process no. 470358/2014-9. To further develop the research tasks of the present proposal, we will collaborate with the FAPESP CEPID Brazilian Institute for Neuroscience and Neurotechnology (BRAINN), UNICAMP, process no. 2013/07559-3. Later on, this project paves to consider the inclusion of stroke patients to test the auditory-motor memorization that takes place during the rehabilitation process in people suffering from cerebral stroke. (AU)

Scientific publications

(4)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

HEBLING, EDUARDO D.; PARTESOTTI, ELENA; SANTANA, CHARLES P.; FIGUEIREDO, ANDRE; DEZOTTI, CASSIO G.; BOTECHIA, TALES; DA SILVA, CESAR A. PEREIRA; DA SILVA, MICAEL A.; ROSSETTI, DANILO; DE OLIVEIRA, VICTOR A. W.; et al. MovieScape: Audiovisual Landscapes for Silent Movie. PROCEEDINGS OF THE 9TH INTERNATIONAL CONFERENCE ON DIGITAL AND INTERACTIVE ARTS (ARTECH 2019), v. N/A, p. 7-pg., 2019-01-01. (16/22619-0)

PARTESOTTI, ELENA; CASTELLANO, GABRIELA; MANZOLLI, JONATAS. Exploring the Theoretical Landscape of BehCreative: Artistic and Therapeutic Possibilities of an Extended Digital Musical Instrument. ARTSIT, INTERACTIVITY AND GAME CREATION, PT 1, ARTSIT 2023, v. 564, p. 13-pg., 2024-01-01. (13/07559-3, 16/22619-0)

PARTESOTTI, ELENA; CASTELLANO, GABRIELA; MANZOLLI, JONATAS. Preliminary Findings from BehCreative: Exploring the Potential of Extended Digital Music Instruments for Music Therapy and Rehabilitation. ARTSIT, INTERACTIVITY AND GAME CREATION, PT 1, ARTSIT 2023, v. 564, p. 14-pg., 2024-01-01. (13/07559-3, 16/22619-0)

PARTESOTTI, ELENA. Technology as a Means of Musical and Artistic Expression: A Comparative Study of Nono's Prometheus and Pink Floyd's Concert in Venice in the 1980s. ARTSIT, INTERACTIVITY AND GAME CREATION, PT 2, ARTSIT 2023, v. 565, p. 17-pg., 2024-01-01. (16/22619-0)