A systematic comparison of supervised classifiers

• Grant number: 14/04930-5
• Support Opportunities: Regular Research Grants - Publications - Scientific article
• Start date: April 01, 2014
• End date: September 30, 2014
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Odemir Martinez Bruno
• Grantee: Odemir Martinez Bruno
• Host Institution: Instituto de Física de São Carlos (IFSC). Universidade de São Paulo (USP). São Carlos , SP, Brazil

Abstract

Pattern recognition techniques have been employed in a myriad of industrial, medical, commercial and academic applications. To tackle such a diversity of data, many techniques have been devised. However, despite the long tradition of pattern recognition research, there is no technique that yields the best classification in all scenarios. Therefore, the consideration of as many as possible techniques presents itself as an fundamental practice in applications aiming at high accuracy. Typical works comparing methods either emphasize the performance of a given algorithm in validation tests or systematically compare various algorithms, assuming that the practical use of these methods is done by experts. In many occasions, however, researchers have to deal with their practical classification tasks without an in-depth knowledge about the underlying mechanisms behind parameters. Actually, the adequate choice of classifiers and parameters alike in such practical circumstances constitutes a long-standing problem and is the subject of the current paper. We carried out a study on the performance of nine well-known classifiers implemented by the Weka framework and compared the dependence of the accuracy with their configuration parameter configurations. The analysis of performance with default parameters revealed that the k-nearest neighbors method exceeds by a large margin the other methods when high dimensional datasets are considered. When other configuration of parameters were allowed, we found that it is possible to improve the quality of SVM in more than 20\% even if parameters are set randomly. Taken together, the investigation conducted in this paper suggests that, apart from the SVM implementation, Weka's default configuration of parameters provides an performance close the one achieved with the optimal configuration. (AU)

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

AMANCIO, DIEGO RAPHAEL; COMIN, CESAR HENRIQUE; CASANOVA, DALCIMAR; TRAVIESO, GONZALO; BRUNO, ODEMIR MARTINEZ; RODRIGUES, FRANCISCO APARECIDO; COSTA, LUCIANO DA FONTOURA. A Systematic Comparison of Supervised Classifiers. PLoS One, v. 9, n. 4, APR 24 2014. (13/06717-4, 10/00927-9, 11/22639-8, 13/14984-2, 10/19440-2, 14/04930-5, 11/50761-2)