Exploring multiobjective training in multiclass classification

Author(s)

Raimundo, Marcos M. ^[1] ; Drumond, Thalita F. ^[1] ; Marques, Alan Caio R. ^[1] ; Lyra, Christiano ^[1] ; Rocha, Anderson ^[2] ; Von Zuben, Fernando J. ^[1]

Affiliation

^[1] Univ Campinas UNICAMP, Sch Elect & Comp Engn, Av Albert Einstein 400, BR-13083852 Campinas, SP - Brazil

^[2] Univ Campinas UNICAMP, Inst Comp, Av Albert Einstein 1251, BR-13083852 Campinas, SP - Brazil

• Document type: Journal article
• Source: Neurocomputing; v. 435, p. 307-320, MAY 7 2021.
• Web of Science Citations: 0

Abstract

Multinomial logistic loss and L-2 regularization are often conflicting objectives as more robust regularization leads to restrained multinomial parameters. For many practical problems, leveraging the best of both worlds would be invaluable for better decision-making processes. This research proposes a novel framework to obtain representative and diverse L-2-regularized multinomial models, based on valuable tradeoffs between prediction error and model complexity. The framework relies upon the Non-Inferior Set Estimation (NISE) method - a deterministic multiobjective solver. NISE automatically implements hyperparameter tuning in a multiobjective context. Given the diverse set of efficient learning models, model selection and aggregation of the multiple models in an ensemble framework promote high performance in multiclass classification. Additionally, NISE uses the weighted sum method as scalarization, thus being able to deal with the learning formulation directly. Its deterministic nature and the convexity of the learning problem confer scalability to the proposal. The experiments show competitive performance in various setups, taking a broad set of multiclass classification methods as contenders. (C) 2021 Elsevier B.V. All rights reserved. (AU)

• FAPESP's process: 16/19080-2 - Optimization of epileptic seizure detectors through machine learning techniques
• Grantee: Fernando dos Santos Beserra
• Support Opportunities: Scholarships in Brazil - Master
• FAPESP's process: 14/11125-1 - Co-clustering in ensembles of collaborative filters
• Grantee: Thalita Firmo Drumond
• Support Opportunities: Scholarships in Brazil - Master
• FAPESP's process: 14/13533-0 - Multi-objective optimization in multi-task learning
• Grantee: Marcos Medeiros Raimundo
• Support Opportunities: Scholarships in Brazil - Doctorate
• FAPESP's process: 17/12646-3 - Déjà vu: feature-space-time coherence from heterogeneous data for media integrity analytics and interpretation of events
• Grantee: Anderson de Rezende Rocha
• Support Opportunities: Research Projects - Thematic Grants