Sound Events Localization and Detection Using Bio-Inspired Gammatone Filters and Temporal Convolutional Neural Networks

Author(s)

Rosero, Karen ; Grijalva, Felipe ; Masiero, Bruno

• Document type: Journal article
• Source: IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING; v. 31, p. 11-pg., 2023-01-01.

Abstract

The auditory brain circuits are biologically constructed to recand localize sounds by encoding a combination of cues that help individuals interpret sounds. The development of computational methods inspired by human capacities has established opportunities for improving machine hearing. Recent studies based on deep learning show that using convolutional recurrent neural networks (CRNNs) is a promising approach for sound event detection and localization in spatial sound. Nevertheless, depending on the sound environment, the performance of these systems is still far from reaching perfect metrics. Therefore, this work intends to boost the performance of state-of-the-art (SOTA) systems by using bio-inspired gammatone auditory filters and intensity vectors (IVs) for the acoustic feature extraction stage, along with the implementation of a temporal convolutional network (TCN) block into a CRNN model, to capture long term dependencies. Three data augmentation techniques are applied to increase the small number of samples in spatial audio datasets. The mentioned stages constitute our proposed Gammatone-based Sound Events Localization and Detection (G-SELD) system, which exceeded the SOTA results on four spatial audio datasets with different levels of acoustical complexity and with up to three sound sources overlapping in time. (AU)

• FAPESP's process: 17/08120-6 - 3D-Audio. Sampling, encoding and reproduction of spatial audio
• Grantee: Bruno Sanches Masiero
• Support Opportunities: Research Grants - Young Investigators Grants
• FAPESP's process: 19/22945-3 - 3D audio: spatial audio acquisition, coding, and reproduction
• Grantee: Karen Gissell Rosero Jácome
• Support Opportunities: Scholarships in Brazil - Master