Machine learning for signal processing applied to spatial audio

• Grant number: 14/14630-9
• Support Opportunities: Scholarships in Brazil - Doctorate
• Start date: December 01, 2014
• End date: July 31, 2018
• Field of knowledge: Engineering - Electrical Engineering
• Agreement: Coordination of Improvement of Higher Education Personnel (CAPES)
• Principal Investigator: Luiz César Martini
• Grantee: Felipe Leonel Grijalva Arévalo
• Host Institution: Faculdade de Engenharia Elétrica e de Computação (FEEC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil

Abstract

As auditory augmented reality applications become more important, there is increasing effort in spatial audio research. The spatial audio term refers to a set of techniques that model the anatomy of a person by means of digital filters. When filtering an audio source through these filters, the listener is able to perceive a sound as if it were played at a specific location in space. In the frequency domain, these filters are known as Head-Related Transfer Functions (HRTF). A significant problem in spatial audio is the fact that spectral features of HRTFs differ among individuals. If a subject uses another person's HRTFs, there is degradation in auditory perception. Thus, it is necessary to personalize HRTFs. The HRTFs of a subject can be measured experimentally. However, as this measurement is a complex, time consuming and not scalable task, various machine learning techniques have been applied to customize HRTFs. The problem of current techniques is that they do not take into account prior knowledge of the characteristics of HRTFs (e.g. symmetry). Thus, the overall goal of this proposal is to apply machine learning techniques in order to customize HRTFs by incorporating both spatial and frequency prior knowledge of HRTFs. With this aim, we will represent the HRTFs using nonlinear dimensionality reduction techniques (e.g. Isomap) in conjunction with filter bank techniques (e.g. wavelets), that take into account this prior knowledge. Later, as there are few HRTF measurements, due to the difficulty of obtaining them, we aim to merge multiple HRTF databases using transfer learning. Finally, using the merged HRTF database, we will utilize deep learning techniques to predict the HRTFs of an individual from their anatomical characteristics. (AU)

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

NETO, LAURINDO BRITTO; GRIJALVA, FELIPE; MARGARETH LIMA MAIKE, VANESSA REGINA; MARTINI, LUIZ CESAR; FLORENCIO, DINEI; CALANI BARANAUSKAS, MARIA CECILIA; ROCHA, ANDERSON; GOLDENSTEIN, SIOME. A Kinect-Based Wearable Face Recognition System to Aid Visually Impaired Users. IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, v. 47, n. 1, p. 52-64, FEB 2017. (12/50468-6, 15/19222-9, 13/21349-1, 14/14630-9)

GRIJALVA, FELIPE; MARTINI, LUIZ CESAR; FLORENCIO, DINEI; GOLDENSTEIN, SIOME. Interpolation of Head-Related Transfer Functions Using Manifold Learning. IEEE SIGNAL PROCESSING LETTERS, v. 24, n. 2, p. 221-225, FEB 2017. (12/50468-6, 13/21349-1, 14/14630-9)

GRIJALVA, FELIPE; MARTINI, LUIZ CESAR; MASIERO, BRUNO; GOLDENSTEIN, SIOME. A Recommender System for Improving Median Plane Sound Localization Performance Based on a Nonlinear Representation of HRTFs. IEEE ACCESS, v. 6, p. 24829-24836, 2018. (12/50468-6, 14/14630-9, 13/21349-1)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

ARÉVALO, Felipe Leonel Grijalva. Aprendizado de variedades para a síntese de áudio espacial. 2018. Doctoral Thesis - Universidade Estadual de Campinas (UNICAMP). Faculdade de Engenharia Elétrica e de Computação Campinas, SP.