Evaluation of Error Metrics for Meta-learning Label Definition in the Forecasting Task

Author(s)

Santos, Moises R. ; Mundim, Leandro R. ; Carvalho, Andre C. P. L. F. ; DeLaCal, EA ; Flecha, JRV ; Quintian, H ; Corchado, E

• Document type: Journal article
• Source: HYBRID ARTIFICIAL INTELLIGENT SYSTEMS, HAIS 2020; v. 12344, p. 13-pg., 2020-01-01.

Abstract

Meta-learning has been successfully applied to time series forecasting. For such, it uses meta-datasets created by previous machine learning applications. Each row in a meta-dataset represents a time series dataset. Each row, apart from the last, is meta-feature describing aspects of the related dataset. The last column is a target value, a meta-label. Here, the meta-label is the forecasting model with the best predictive performance for a specific error metric. In the previous studies applying meta-learning to time series forecasting, error metrics have been arbitrarily chosen. We believe that the error metric used can affect the results obtained by meta-learning. This study presents an experimental analysis of the predictive performance obtained by using different error metrics for the definition of the meta-label value. The experiments performed used 100 time series collected from the ICMC time series prediction open access repository, which has time series from a large variety of application domains. A traditional meta-learning framework for time series forecasting was used in this work. According to the experimental results, the mean absolute error can be the best metric for meta-label definition. (AU)

• FAPESP's process: 19/10012-2 - Meta-learning for time-series forecasting
• Grantee: Moisés Rocha dos Santos
• Support Opportunities: Scholarships in Brazil - Doctorate