Algorithm selection and performance understanding for time series forecasting

Author(s)

Moisés Rocha dos Santos

• Document type: Doctoral Thesis
• Press: São Carlos.
• Institution: Universidade de São Paulo (USP). Instituto de Ciências Matemáticas e de Computação (ICMC/SB)
• Defense date: 2023-05-24

Examining board members

André Carlos Ponce de Leon Ferreira de Carvalho; Ricardo Bastos Cavalcante Prudêncio; Diego Furtado Silva; Bruno Feres de Souza

• Advisor: André Carlos Ponce de Leon Ferreira de Carvalho

Abstract

Time series forecasting is a strategic task in supporting decision-making. The wide availability of forecasting algorithms has generated a demand for algorithm selection methods and approaches that enable the understanding of predictive performance given a time series. This thesis focuses on developing new approaches for forecast combination selection and understanding the predictive performance of time series forecasting algorithms. The research started with a review of the state of the art in time series forecasting, focusing on algorithm selection and understanding of predictive performance. This study highlighted the limitations of existing approaches and identified gaps in the literature that this research could address. The main contributions of this thesis are fourfold: firstly, the development of a metalearning-based approach for selecting forecasting combinations with time series decomposition. A synthetic time series generation method based on dataset morphing. The empirical analysis of different performance measures for the choice of meta-label in the selection algorithms by metalearning. Finally, an analysis of applying Seasonal and Trend decomposition using Loess as a pre-processing step for machine learning algorithms in the time series forecasting task. The MetaFore approach selects combinations of machine learning algorithms for the trend and residual components in the time series forecasting task. The components are separated with the Seasonal and Trend decomposition using Loess, and the seasonality is forecasted with the seasonal naive method. MetaFore was evaluated in the monthly time series of the M4 competition and achieved better predictive and computational performance than LSTM neural networks in more than 70% of the datasets. The tsMorph method generates synthetic time series by gradually transforming a source time series into a target time series. TsMorph was applied to understand the predictive performance variation of Support Vector Regression and Long Short-Term Memory neural network prediction algorithms. The results showed that the tsMorph method generated time series with gradual predictive performance and meta-feature variation. Empirical analysis of different performance measures as meta-label in the selection of time series forecasting algorithms showed no statistical difference in the predictive performances of the meta and base level. The analysis of the application of Seasonal Trend with Loess decomposition as a pre-processing step in machine learning showed that when the residual component follows a normal distribution, the decomposition improves the predictive performance of the algorithms. In summary, this research contributed to developing new approaches for efficiently predicting and understanding algorithms performance. Studies on the selection of forecasting combinations and performance understanding can be easily included in time series forecasting processes and open perspectives for research and development in metalearning and autoML. (AU)

• FAPESP's process: 19/10012-2 - Meta-learning for time-series forecasting
• Grantee: Moisés Rocha dos Santos
• Support Opportunities: Scholarships in Brazil - Doctorate