Effects of Random Sampling on SVM Hyper-parameter Tuning

Author(s)

Horvath, Tomas ; Mantovani, Rafael G. ; de Carvalho, Andre C. P. L. F. ; Madureira, AM ; Abraham, A ; Gamboa, D ; Novais, P

• Document type: Journal article
• Source: INTELLIGENT SYSTEMS DESIGN AND APPLICATIONS (ISDA 2016); v. 557, p. 11-pg., 2017-01-01.

Abstract

Hyper-parameter tuning is one of the crucial steps in the successful application of machine learning algorithms to real data. In general, the tuning process is modeled as an optimization problem for which several methods have been proposed. For complex algorithms, the evaluation of a hyper-parameter configuration is expensive and their runtime is speed up through data sampling. In this paper, the effect of sample sizes to the results of hyper-parameter tuning process is investigated. Hyper-parameters of Support Vector Machines are tuned on samples of different sizes generated from a dataset. Hausdorff distance is proposed for computing the differences between the results of hyper-parameter tuning on two samples of different size. 100 real-world datasets and two tuning methods (Random Search and Particle Swarm Optimization) are used in the experiments revealing some interesting relations between sample sizes and results of hyper-parameter tuning which open some promising directions for future investigation in this direction. (AU)

• FAPESP's process: 13/07375-0 - CeMEAI - Center for Mathematical Sciences Applied to Industry
• Grantee: Francisco Louzada Neto
• Support Opportunities: Research Grants - Research, Innovation and Dissemination Centers - RIDC
• FAPESP's process: 12/23114-9 - Use of meta-learning for parameter tuning for classification problems
• Grantee: Rafael Gomes Mantovani
• Support Opportunities: Scholarships in Brazil - Doctorate