Advanced search
Start date
Betweenand

Is my benchmark of datasets challenging enough?

Grant number: 22/10683-7
Support Opportunities:Scholarships in Brazil - Post-Doctoral
Start date: September 01, 2022
End date: March 26, 2025
Field of knowledge:Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
Principal Investigator:Ana Carolina Lorena
Grantee:João Luiz Junho Pereira
Host Institution: Divisão de Ciência da Computação (IEC). Instituto Tecnológico de Aeronáutica (ITA). São José dos Campos , SP, Brazil
Associated research grant:21/06870-3 - Beyond algorithm selection: meta-learning for data and algorithm analysis and understanding, AP.JP2
Associated scholarship(s):23/10419-0 - Multi-objective optimal selection of benchmarking datasets for unbiased and efficient machine learning algorithm evaluation, BE.EP.PD

Abstract

Whenever a new supervised Machine Learning (ML) algorithm or solu- tion is developed, it is imperative to evaluate the predictive performance it attains for diverse datasets. This is done in order to stress out the strengths and weak- nesses of the algorithms and evidence for which situations they are most useful. A common practice is to gather some datasets from public benchmark repositories for such an evaluation. But little or no specific criteria are used in the selection of these datasets, which is often ad-hoc. In this project we will investigate the importance of properly building a diverse and challenging benchmark of datasets in order to properly evaluate ML models and really understand their capabilities. (AU)

News published in Agência FAPESP Newsletter about the scholarship:
More itemsLess items
Articles published in other media outlets ( ):
More itemsLess items
VEICULO: TITULO (DATA)
VEICULO: TITULO (DATA)

Scientific publications (14)
(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)
NOGUEIRA, FELIPE SALES; PEREIRA, JOAO LUIZ JUNHO; CUNHA JR, SEBASTIAO SIMOES. Multi-objective sensor placement optimization and damage identification for an aircraft wing using Lichtenberg algorithm. ENGINEERING COMPUTATIONS, v. 41, n. 2, p. 30-pg., . (22/10683-7, 23/10419-0)
PEREIRA, JOAO LUIZ JUNHO; GUEDES, FELIPE CIOLINI; FRANCISCO, MATHEUS BRENDON; CHIARELLO, ANDRE GARCIA; GOMES, GUILHERME FERREIRA. Multi-objective design optimization of a high performance disk brake using lichtenberg algorithm. MECHANICS BASED DESIGN OF STRUCTURES AND MACHINES, v. N/A, p. 14-pg., . (22/10683-7)
PEREIRA, JOAO LUIZ JUNHO; FRANCISCO, MATHEUS BRENDON; DE ALMEIDA, FABRICIO ALVES; MA, BENEDICT JUN; CUNHA, SEBASTIAO SIMOES; GOMES, GUILHERME FERREIRA. Enhanced Lichtenberg algorithm: a discussion on improving meta-heuristics. SOFT COMPUTING, v. N/A, p. 29-pg., . (22/10683-7)
PEREIRA, JOAO LUIZ JUNHO; MA, BENEDICT JUN; FRANCISCO, MATHEUS BRENDON; RIBEIRO JR, RONNY FRANCIS; GOMES, GUILHERME FERREIRA. A comparison between chaos theory and Levy flights in sunflower optimization for feature selection. EXPERT SYSTEMS, v. 40, n. 8, p. 21-pg., . (22/10683-7)
OLIVEIRA, THIAGO LUIZ LARA; DE OLIVEIRA, MARLON MENDES; FRANCISCO, MATHEUS BRENDON; PEREIRA, MATHEUS COSTA; ANCELOTTI, ANTONIO CARLOS; DE PAIVA, ANDERSON PAULO; PEREIRA, JOAO LUIZ JUNHO. CFRP surface roughness prediction in machining by acoustic emission: A reliable machine learning study. INTERNATIONAL JOURNAL OF ADVANCED MANUFACTURING TECHNOLOGY, v. 140, n. 7-8, p. 15-pg., . (23/10419-0, 22/10683-7)
DE SOUZA MELLO, FELIPE MARTARELLA; PEREIRA, JOAO LUIZ JUNHO; GOMES, GUILHERME FERREIRA. Multi-objective sensor placement optimization in SHM systems with Kriging-based mode shape interpolation. Journal of Sound and Vibration, v. 568, p. 25-pg., . (22/10683-7)
SOROMOTIN, ANDREY, V; MARTYUSHEV, DMITRIY A.; PEREIRA, JOAO LUIZ JUNHO. On the application of machine learning algorithms in predicting the permeability of oil reservoirs. ARTIFICIAL INTELLIGENCE IN GEOSCIENCES, v. 6, n. 2, p. 23-pg., . (22/10683-7, 23/10419-0)
BENDINE, KOUIDER; PEREIRA, JOAO LUIZ JUNHO; GOMES, GUILHERME FERREIRA. Energy harvesting enhancement of nonuniform functionally graded piezoelectric beam using artificial neural networks and Lichtenberg algorithm. STRUCTURES, v. 57, p. 14-pg., . (22/10683-7)
GOMES, GUILHERME FERREIRA; RIBEIRO JUNIOR, RONNY FRANCIS RIBEIRO; PEREIRA, JOAO LUIZ JUNHO; FRANCISCO, MATHEUS BRENDON. An efficient deep learning model to predict the structural response of CFRP isogrid tubes. COMPOSITE STRUCTURES, v. 316, p. 17-pg., . (22/10683-7)
DE OLIVEIRA, LUCAS ANTONIO; GOMES, GUILHERME FERREIRA; PEREIRA, JOAO LUIZ JUNHO; FRANCISCO, MATHEUS BRENDON; DEMARBAIX, ANTHONIN; CUNHA JR, SEBASTIAO SIMOES. New Trends of Damage Detection and Identification Based on Vibrothermography in Composite Materials. JOURNAL OF NONDESTRUCTIVE EVALUATION, v. 42, n. 3, p. 16-pg., . (22/10683-7)
GOMES, GUILHERME FERREIRA; BENDINE, KOUIDER; PEREIRA, JOAO LUIZ JUNHO. Optimization and artificial intelligence: An in-depth analysis of multi-objective optimization, sampling methods, and regression algorithms applied to structural design. MECHANICS BASED DESIGN OF STRUCTURES AND MACHINES, v. N/A, p. 28-pg., . (22/10683-7, 23/10419-0)
PEREIRA, JOAO LUIZ JUNHO; SMITH-MILES, KATE; MUNOZ, MARIO ANDRES; LORENA, ANA CAROLINA. Optimal selection of benchmarking datasets for unbiased machine learning algorithm evaluation. DATA MINING AND KNOWLEDGE DISCOVERY, v. 38, n. 2, p. 40-pg., . (21/06870-3, 22/10683-7)
SOTOMONTE, CESAR; PEREIRA, JOAO; FRANCISCO, MATHEUS; DE SOUZA, TULIO; PINTO, GABRIEL; JULIO, ALISSON; BUNYA, BRUNO; CORONADO, CHRISTIAN; PALACIO, JOSE; GOMES, GUILHERME. ENERGY MINIMIZATION OF HYDROGEN PRODUCTION VIA BUTANOL STEAM REFORMING. ENVIRONMENTAL ENGINEERING AND MANAGEMENT JOURNAL, v. 22, n. 3, p. 201-pg., . (22/10683-7)
GONTIJO, R. G.; OSSEGE, F. E. L.; PEREIRA, J. L. J.. Evaluation of machine learning algorithms in the prediction of key therapeutic quantities in magnetic hyperthermia. COMPUTERS & MATHEMATICS WITH APPLICATIONS, v. 194, p. 17-pg., . (22/10683-7, 23/10419-0)