Exploration of self-adaptive learning and forecasting for time series

Abstract

Time series forecasting poses several inherent challenges. Typically, the targets of interest are non-stationary random variables characterized by both short-term and long-term autocorrelations. Controlling for the variation of data distributions over time is a must, as so it is the exploitation of the different levels of data dependency. This thesis tackles these challenges by integrating methodologies from three seminal papers-Self-Adaptive Forecasting (SAF), adaRNN, and transformers-into a unified model, the AdaTransformer. By leveraging the adaptive capabilities of adaRNN and SAF and incorporating transformer architecture, the AdaTransformer captures complex temporal dependencies with improved accuracy. Additionally, the AdaTransformer introduces an enhanced Temporal Dependency Characterization (TDC) segmentation algorithm, building upon prior work to improve segmentation quality. Through comprehensive evaluation across diverse datasets encompassing air quality indices, electric power consumption, and financial markets represented by EUR/USD and TSLA datasets, our results highlight the AdaTransformer's consistent outperformance of the adaRNN model alone, as evidenced by lower Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). This research underscores the effectiveness of integrating transformer based models with adaptive techniques, offering a robust solution for accurate and reliable time series forecasting.