Investigation of Evaluation Methodologies for Multi-Label Classification Problems in Continuous Data Streams

• Grant number: 24/15875-7
• Support Opportunities: Scholarships in Brazil - Master
• Start date: October 01, 2024
• Status: Discontinued
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Ricardo Cerri
• Grantee: Rafaella de Oliveira
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Associated research grant: 22/02981-8 - Novelty detection in multi-label data streams classification, AP.PNGP.PI
• Associated scholarship(s): 25/25602-0 - A new Evaluation Measure for Novelty Detection in Multi-label Data Streams, BE.EP.MS

Abstract

A significant challenge in novelty detection for multi-label classification problems in continuous data streams is how to measure the performance of algorithms. Effectively evaluating novelty detection algorithms is crucial to ensure accuracy and reliability in results. Recent methodologies do not penalize the identification of many novelty patterns representing a class, do not weigh novelty patterns created before concept evolution, and do not consider multi-label associations. Based on this, the goal of this project is to develop an evaluation methodology for novelty detection in continuous multi-label data streams. The project development involves reviewing the literature for evaluation methods, analyzing similar methodologies and approaches, characterizing the strengths and weaknesses of existing methodologies, adapting methodologies proposed in other projects, and suggesting alternative methods. Experimental validation will be conducted using synthetic and real data generated by the MOA framework, and the evaluation method will highlight scenarios such as Anomaly Detection, Open Set Recognition, and Multi-Label Classification with a rejection option. It is expected that by the end of the research, it will be possible to refine and address the issues identified in previous studies based on the results obtained in this research. (AU)