Dynamic Mapping Strategies Using Content-Based Classification: A Proposed Method for an Augmented Instrument

Author(s)

Rimoldi, Gabriel ; Manzolli, Jonatas ; Aramaki, M ; KronlandMartinet, R ; Ystad, S

• Document type: Journal article
• Source: BRIDGING PEOPLE AND SOUND; v. 10525, p. 10-pg., 2017-01-01.

Abstract

We discuss in this paper strategies of dynamic mapping applied to the design of augmented instruments. The proposed method is based on a feedback architecture that allows adjustment of mapping functions through pattern detection from the sonic response of an instrument. We applied this method to design Metaflute, an augmented instrument based on a hybrid system of gestural capture. We used a set of eight phase vocoder modules to process flute samples in real-time with performer movements detected by coupled sensors. A set of audio features was extracted from the sonic response of each synthesis module within a certain period and sent to a K-nearest neighbor algorithm (k-NN). Each synthesis module has its mapping functions modified based on patterns found by k-NN algorithm. This procedure was repeated iteratively so that the system adjusts itself in relation to its previous states. Preliminary results show a gradual differentiation of mapping functions for each synthesis module, allowing them to perform a different response in relation to data arising from the gestural interface. (AU)

• FAPESP's process: 14/13166-7 - Eco-improvisation: an approach to technological mediation in context of musical digital interfaces
• Grantee: Gabriel Rimoldi de Lima
• Support Opportunities: Scholarships in Brazil - Doctorate