Improving the kernel regularized least squares method for small-sample regression

Autor(es)

Braga, Igor ; Monard, Maria Carolina

• Tipo de documento: Artigo Científico
• Fonte: Neurocomputing; v. 163, p. 9-pg., 2015-09-02.

Resumo

The kernel regularized least squares (KRIS) method uses the kernel trick to perform non-linear regression estimation. Its performance depends on proper selection of both a kernel function and a regularization parameter. In practice, cross-validation along with the Gaussian RBF kernel have been widely used for carrying out model selection for KRLS. However, when training data is scarce, this combination often leads to poor regression estimation. In order to mitigate this issue, we follow two lines of investigation in this paper. First, we explore a new type of kernel function that is less susceptible to overfitting than the RBF kernel. Then, we consider alternative parameter selection methods that have been shown to perform well for other regression methods. Experiments conducted on real-world datasets show that an additive spline kernel greatly outperforms both the RBF and a previously proposed multiplicative spline kernel. We also find that the parameter selection procedure Finite Prediction Error (FPE) is a competitive alternative to cross-validation when using the additive splines kernel. (C) 2015 Elsevier B.V. All rights reserved. (AU)

• Processo FAPESP: 09/17773-7 - Inferência Seletiva em Aprendizado de Máquina: teoria, algoritmos e aplicações
• Beneficiário: Ígor Assis Braga
• Modalidade de apoio: Bolsas no Brasil - Doutorado