On Accuracy and Time Processing Evaluation of Cover Song Identification Systems

Author(s)

Ferreira, Martha Dais ; Correa, Debora Cristina ; Grivet, Marcos Antonio ; dos Santos, Geovan Tavares ; de Mello, Rodrigo Fernandes ; Nonato, Luis Gustavo

• Document type: Journal article
• Source: JOURNAL OF NEW MUSIC RESEARCH; v. 45, n. 4, p. 333-342, DEC 2016.
• Web of Science Citations: 1

Abstract

Cover song identification (CSI) systems typically represent songs as chromagrams which are pairwise compared using different evaluation measurements. Chromagram comparison are usually computationally demanding, making most CSI systems unsuitable for real-world scenarios where millions of songs have to be processed. Evaluation mechanisms such as the ones proposed by Music Information Retrieval Evaluation eXchange (MIREX) handle MIR systems as black boxes, measuring their accuracy as a whole, thus disregarding the effectiveness of each particular component of the system. Moreover, computational times are usually not taken into account in the evaluation process, making difficult to assess the usefulness of the systems when facing large data-sets. In this manuscript, we present a systematic evaluation of the components of two state-of-the-art CSI systems that performed quite well according to MIREX. Our experiments indicate that the performance of those two systems strongly depends on the combination of the song descriptor and the comparison method used to quantify the similarity between songs. Nevertheless, the high computational cost involved in the comparison of song descriptors renders both approaches unfeasible to large data-sets, motivating the development of alternative methods. (AU)

• FAPESP's process: 11/22749-8 - Challenges in exploratory visualization of multidimensional data: paradigms, scalability and applications
• Grantee: Luis Gustavo Nonato
• Support Opportunities: Research Projects - Thematic Grants
• FAPESP's process: 14/13323-5 - An approach based on the stability of clustering algorithms to ensure concept drift detection on data streams
• Grantee: Rodrigo Fernandes de Mello
• Support Opportunities: Regular Research Grants
• FAPESP's process: 12/17961-0 - Musical Data Mining by Temporal Patterns
• Grantee: Débora Cristina Corrêa
• Support Opportunities: Scholarships in Brazil - Post-Doctoral