...And Justice for All Tags: dealing with class imbalance in music auto-tagging using embeddings

• Grant number: 22/10969-8
• Support Opportunities: Scholarships abroad - Research Internship - Scientific Initiation
• Start date: November 05, 2022
• End date: February 04, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Diego Furtado Silva
• Grantee: Vitor Diniz de Oliveira Cunha
• Supervisor: Xavier Serra
• Host Institution: Centro de Ciências Exatas e de Tecnologia (CCET). Universidade Federal de São Carlos (UFSCAR). São Carlos , SP, Brazil
• Institution abroad: Universitat Pompeu Fabra (UPF), Spain
• Associated to the scholarship: 21/15221-9 - Study of auto-tagging techniques for domain-specific musical data and considering the long tail problem, BP.IC

Abstract

Music auto-tagging is a significant way to characterize recordings, with implications on music organization and retrieval. The state-of-the-art auto-taggers are based on deep learning. However, these techniques need a large volume of data to induce a suitable model. Besides, these models may underperform on least represented tags. While there are techniques to reduce this issue during the training phase, they may increase the data volume and make the training phase even more costly. On the other hand, the literature is plenty of algorithms to deal with the class imbalance in the domain of tabular data. In this work, we propose applying these techniques to music data transformed by pre-trained auto-tagging models and evaluate how they impact the performance of predicting the least represented tags. (AU)