| Texto completo | |
| Autor(es): |
Pereira, Joao Luiz Junho
;
Smith-Miles, Kate
;
Munoz, Mario Andres
;
Lorena, Ana Carolina
Número total de Autores: 4
|
| Tipo de documento: | Artigo Científico |
| Fonte: | DATA MINING AND KNOWLEDGE DISCOVERY; v. 38, n. 2, p. 40-pg., 2023-10-20. |
| Resumo | |
Whenever a new supervised machine learning (ML) algorithm or solution is developed, it is imperative to evaluate the predictive performance it attains for diverse datasets. This is done in order to stress test the strengths and weaknesses of the novel algorithms and provide evidence for situations in which they are most useful. A common practice is to gather some datasets from public benchmark repositories for such an evaluation. But little or no specific criteria are used in the selection of these datasets, which is often ad-hoc. In this paper, the importance of gathering a diverse benchmark of datasets in order to properly evaluate ML models and really understand their capabilities is investigated. Leveraging from meta-learning studies evaluating the diversity of public repositories of datasets, this paper introduces an optimization method to choose varied classification and regression datasets from a pool of candidate datasets. The method is based on maximum coverage, circular packing, and the meta-heuristic Lichtenberg Algorithm for ensuring that diverse datasets able to challenge the ML algorithms more broadly are chosen. The selections were compared experimentally with a random selection of datasets and with clustering by k-medoids and proved to be more effective regarding the diversity of the chosen benchmarks and the ability to challenge the ML algorithms at different levels. (AU) | |
| Processo FAPESP: | 21/06870-3 - Além da seleção de algoritmos: meta-aprendizado para análise e entendimento de dados e algoritmos |
| Beneficiário: | Ana Carolina Lorena |
| Modalidade de apoio: | Auxílio à Pesquisa - Jovens Pesquisadores - Fase 2 |
| Processo FAPESP: | 22/10683-7 - Meu benchmark de conjuntos de dados é desafiador o suficiente? |
| Beneficiário: | João Luiz Junho Pereira |
| Modalidade de apoio: | Bolsas no Brasil - Pós-Doutorado |