Improving the kernel regularized least squares method for small-sample regression

Author(s)

Braga, Igor ; Monard, Maria Carolina

• Document type: Journal article
• Source: Neurocomputing; v. 163, p. 9-pg., 2015-09-02.

Abstract

The kernel regularized least squares (KRIS) method uses the kernel trick to perform non-linear regression estimation. Its performance depends on proper selection of both a kernel function and a regularization parameter. In practice, cross-validation along with the Gaussian RBF kernel have been widely used for carrying out model selection for KRLS. However, when training data is scarce, this combination often leads to poor regression estimation. In order to mitigate this issue, we follow two lines of investigation in this paper. First, we explore a new type of kernel function that is less susceptible to overfitting than the RBF kernel. Then, we consider alternative parameter selection methods that have been shown to perform well for other regression methods. Experiments conducted on real-world datasets show that an additive spline kernel greatly outperforms both the RBF and a previously proposed multiplicative spline kernel. We also find that the parameter selection procedure Finite Prediction Error (FPE) is a competitive alternative to cross-validation when using the additive splines kernel. (C) 2015 Elsevier B.V. All rights reserved. (AU)

• FAPESP's process: 09/17773-7 - Selective Inference in Machine Learning: theory, algorithms and applications
• Grantee: Ígor Assis Braga
• Support Opportunities: Scholarships in Brazil - Doctorate