Wavelet entropy-based evaluation of intrinsic predictability of time series

CTF-former: A novel simplified multi-task learning strategy for simultaneous multivariate chaotic time series prediction

Multivariate chaotic time series prediction is a challenging task, especially when multiple variables are predicted simultaneously. For multiple related prediction tasks typically require multiple models, however, multiple models are difficult to keep synchronization, making immediate communication between predicted values challenging. Although multi-task learning can be applied to this problem, the principles of allocation and layout options between shared and specific representations are ambiguous. To address this issue, a novel simplified multi-task learning method was proposed for the precise implementation of simultaneous multiple chaotic time series prediction tasks. The scheme proposed consists of a cross-convolution operator designed to capture variable correlations and sequence correlations, and an attention module proposed to capture the information embedded in the sequence structure. In the attention module, a non-linear transformation was implemented with convolution, and its local receptive field and the global dependency of the attention mechanism achieve complementarity. In addition, an attention weight calculation was devised that takes into account not only the synergy of time and frequency domain features, but also the fusion of series and channel information. Notably the scheme proposed a purely simplified design principle of multi-task learning by reducing the specific network to single neuron. The precision of the proposed solution and its potential for engineering applications were verified with the Lorenz system and power consumption. The mean absolute error of the proposed method was reduced by an average of 82.9% in the Lorenz system and 19.83% in power consumption compared to the Gated Recurrent Unit.

Enhancing Predictability Assessment: An Overview and Analysis of Predictability Measures for Time Series and Network Links

Driven by the variety of available measures intended to estimate predictability of diverse objects such as time series and network links, this paper presents a comprehensive overview of the existing literature in this domain. Our overview delves into predictability from two distinct perspectives: the intrinsic predictability, which represents a data property independent of the chosen forecasting model and serves as the highest achievable forecasting quality level, and the realized predictability, which represents a chosen quality metric for a specific pair of data and model. The reviewed measures are used to assess predictability across different objects, starting from time series (univariate, multivariate, and categorical) to network links. Through experiments, we establish a noticeable relationship between measures of realized and intrinsic predictability in both generated and real-world time series data (with the correlation coefficient being statistically significant at a 5% significance level). The discovered correlation in this research holds significant value for tasks related to evaluating time series complexity and their potential to be accurately predicted.

A robust sparse identification method for nonlinear dynamic systems affected by non-stationary noise

In the field of science and engineering, identifying the nonlinear dynamics of systems from data is a significant yet challenging task. In practice, the collected data are often contaminated by noise, which often severely reduce the accuracy of the identification results. To address the issue of inaccurate identification induced by non-stationary noise in data, this paper proposes a method called weighted ℓ1-regularized and insensitive loss function-based sparse identification of dynamics. Specifically, the robust identification problem is formulated using a sparse identification mathematical model that takes into account the presence of non-stationary noise in a quantitative manner. Then, a novel weighted ℓ1-regularized and insensitive loss function is proposed to account for the nature of non-stationary noise. Compared to traditional loss functions like least squares and least absolute deviation, the proposed method can mitigate the adverse effects of non-stationary noise and better promote the sparsity of results, thereby enhancing the accuracy of identification. Third, to overcome the non-smooth nature of the objective function induced by the inclusion of loss and regularization terms, a smooth approximation of the non-smooth objective function is presented, and the alternating direction multiplier method is utilized to develop an efficient optimization algorithm. Finally, the robustness of the proposed method is verified by extensive experiments under different types of nonlinear dynamical systems. Compared to some state-of-the-art methods, the proposed method achieves better identification accuracy.

Two novel nonlinear multivariate grey models with kernel learning for small-sample time series prediction

For many applications, small-sample time series prediction based on grey forecasting models has become indispensable. Many algorithms have been developed recently to make them effective. Each of these methods has a specialized application depending on the properties of the time series that need to be inferred. In order to develop a generalized nonlinear multivariable grey model with higher compatibility and generalization performance, we realize the nonlinearization of traditional GM(1,N), and we call it NGM(1,N). The unidentified nonlinear function that maps the data into a better representational space is present in both the NGM(1,N) and its response function. The original optimization problem with linear equality constraints is established in terms of parameter estimation for the NGM(1,N), and two different approaches are taken to solve it. The former is the Lagrange multiplier method, which converts the optimization problem into a linear system to be solved; and the latter is the standard dualization method utilizing Lagrange multipliers, that uses a flexible estimation equation for the development coefficient. As the size of the training data increases, the estimation results of the potential development coefficient get richer and the final estimation results using the average value are more reliable. The kernel function expresses the dot product of two unidentified nonlinear functions during the solving process, greatly lowering the computational complexity of nonlinear functions. Three numerical examples show that the LDNGM(1,N) outperforms the other multivariate grey models compared in terms of generalization performance. The duality theory and framework with kernel learning are instructive for further research around multivariate grey models to follow.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s11071-023-08296-y.

Higuchi fractal dimension is a unique indicator of working memory content represented in spiking activity of visual neurons in extrastriate cortex

Working memory has been identified as a top-down modulation of the average spiking activity in different brain parts. However, such modification has not yet been reported in the middle temporal (MT) cortex. A recent study showed that the dimensionality of the spiking activity of MT neurons increases after deployment of spatial working memory. This study is devoted to analyzing the ability of nonlinear and classical features to capture the content of the working memory from the spiking activity of MT neurons. The results suggest that only the Higuchi fractal dimension can be considered as a unique indicator of working memory while the Margaos-Sun fractal dimension, Shannon entropy, corrected conditional entropy, and skewness are perhaps indicators of other cognitive factors such as vigilance, awareness, and arousal as well as working memory.

A Bayesian-based classification framework for financial time series trend prediction

Financial time series have been extensively studied within the past decades; however, the advent of machine learning and deep neural networks opened new horizons to apply supercomputing techniques to extract more insights from the underlying patterns of price data. This paper presents a tri-state labeling approach to classify the underlying patterns in price data into up, down and no-action classes. The introduction of a no-action state in our novel approach alleviates the burden of denoising the dataset as a preprocessing task. The performance of our labeling algorithm is experimented with using machine learning and deep learning models. The framework is augmented by applying the Bayesian optimization technique for the selection of the best tuning values of the hyperparameters. The price trend prediction module generates the required trading signals. The results show that the average annualized Sharpe ratio as the trading performance metric is about 2.823, indicating the framework produces excellent cumulative returns.

A complex network approach to study the extreme precipitation patterns in a river basin

The quantification of spatial propagation of extreme precipitation events is vital in water resources planning and disaster mitigation. However, quantifying these extreme events has always been challenging as many traditional methods are insufficient to capture the nonlinear interrelationships between extreme event time series. Therefore, it is crucial to develop suitable methods for analyzing the dynamics of extreme events over a river basin with a diverse climate and complicated topography. Over the last decade, complex network analysis emerged as a powerful tool to study the intricate spatiotemporal relationship between many variables in a compact way. In this study, we employ two nonlinear concepts of event synchronization and edit distance to investigate the extreme precipitation pattern in the Ganga river basin. We use the network degree to understand the spatial synchronization pattern of extreme rainfall and identify essential sites in the river basin with respect to potential prediction skills. The study also attempts to quantify the influence of precipitation seasonality and topography on extreme events. The findings of the study reveal that (1) the network degree is decreased in the southwest to northwest direction, (2) the timing of 50th percentile precipitation within a year influences the spatial distribution of degree, (3) the timing is inversely related to elevation, and (4) the lower elevation greatly influences connectivity of the sites. The study highlights that edit distance could be a promising alternative to analyze event-like data by incorporating event time and amplitude and constructing complex networks of climate extremes.

Categorical Nature of Major Factor Selection via Information Theoretic Measurements

Without assuming any functional or distributional structure, we select collections of major factors embedded within response-versus-covariate (Re-Co) dynamics via selection criteria [C1: confirmable] and [C2: irrepaceable], which are based on information theoretic measurements. The two criteria are constructed based on the computing paradigm called Categorical Exploratory Data Analysis (CEDA) and linked to Wiener–Granger causality. All the information theoretical measurements, including conditional mutual information and entropy, are evaluated through the contingency table platform, which primarily rests on the categorical nature within all involved features of any data types: quantitative or qualitative. Our selection task identifies one chief collection, together with several secondary collections of major factors of various orders underlying the targeted Re-Co dynamics. Each selected collection is checked with algorithmically computed reliability against the finite sample phenomenon, and so is each member’s major factor individually. The developments of our selection protocol are illustrated in detail through two experimental examples: a simple one and a complex one. We then apply this protocol on two data sets pertaining to two somewhat related but distinct pitching dynamics of two pitch types: slider and fastball. In particular, we refer to a specific Major League Baseball (MLB) pitcher and we consider data of multiple seasons.

Modifications in the Topological Structure of EEG Functional Connectivity Networks during Listening Tonal and Atonal Concert Music in Musicians and Non-Musicians

The present work aims to demonstrate the hypothesis that atonal music modifies the topological structure of electroencephalographic (EEG) connectivity networks in relation to tonal music. To this, EEG monopolar records were taken in musicians and non-musicians while listening to tonal, atonal, and pink noise sound excerpts. EEG functional connectivities (FC) among channels assessed by a phase synchronization index previously thresholded using surrogate data test were computed. Sound effects, on the topological structure of graph-based networks assembled with the EEG-FCs at different frequency-bands, were analyzed throughout graph metric and network-based statistic (NBS). Local and global efficiency normalized (vs. random-network) measurements (NLE|NGE) assessing network information exchanges were able to discriminate both music styles irrespective of groups and frequency-bands. During tonal audition, NLE and NGE values in the beta-band network get close to that of a small-world network, while during atonal and even more during noise its structure moved away from small-world. These effects were attributed to the different timbre characteristics (sounds spectral centroid and entropy) and different musical structure. Results from networks topographic maps for strength and NLE of the nodes, and for FC subnets obtained from the NBS, allowed discriminating the musical styles and verifying the different strength, NLE, and FC of musicians compared to non-musicians.

Quantifying Information without Entropy: Identifying Intermittent Disturbances in Dynamical Systems

A system’s response to disturbances in an internal or external driving signal can be characterized as performing an implicit computation, where the dynamics of the system are a manifestation of its new state holding some memory about those disturbances. Identifying small disturbances in the response signal requires detailed information about the dynamics of the inputs, which can be challenging. This paper presents a new method called the Information Impulse Function (IIF) for detecting and time-localizing small disturbances in system response data. The novelty of IIF is its ability to measure relative information content without using Boltzmann’s equation by modeling signal transmission as a series of dissipative steps. Since a detailed expression of the informational structure in the signal is achieved with IIF, it is ideal for detecting disturbances in the response signal, i.e., the system dynamics. Those findings are based on numerical studies of the topological structure of the dynamics of a nonlinear system due to perturbated driving signals. The IIF is compared to both the Permutation entropy and Shannon entropy to demonstrate its entropy-like relationship with system state and its degree of sensitivity to perturbations in a driving signal.

A Labeling Method for Financial Time Series Prediction Based on Trends

Time series prediction has been widely applied to the finance industry in applications such as stock market price and commodity price forecasting. Machine learning methods have been widely used in financial time series prediction in recent years. How to label financial time series data to determine the prediction accuracy of machine learning models and subsequently determine final investment returns is a hot topic. Existing labeling methods of financial time series mainly label data by comparing the current data with those of a short time period in the future. However, financial time series data are typically non-linear with obvious short-term randomness. Therefore, these labeling methods have not captured the continuous trend features of financial time series data, leading to a difference between their labeling results and real market trends. In this paper, a new labeling method called "continuous trend labeling" is proposed to address the above problem. In the feature preprocessing stage, this paper proposed a new method that can avoid the problem of look-ahead bias in traditional data standardization or normalization processes. Then, a detailed logical explanation was given, the definition of continuous trend labeling was proposed and also an automatic labeling algorithm was given to extract the continuous trend features of financial time series data. Experiments on the Shanghai Composite Index and Shenzhen Component Index and some stocks of China showed that our labeling method is a much better state-of-the-art labeling method in terms of classification accuracy and some other classification evaluation metrics. The results of the paper also proved that deep learning models such as LSTM and GRU are more suitable for dealing with the prediction of financial time series data.

Forecasting of extreme flood events using different satellite precipitation products and wavelet-based machine learning methods

An accurate and timely forecast of extreme events can mitigate negative impacts and enhance preparedness. Real-time forecasting of extreme flood events with longer lead times is difficult for regions with sparse rain gauges, and in such situations, satellite precipitation could be a better alternative. Machine learning methods have shown promising results for flood forecasting with minimum variables indicating the underlying nonlinear complex hydrologic system. Integration of machine learning methods in extreme event forecasting motivates us to develop reliable flood forecasting models that are simple, accurate, and applicable in data scare regions. In this study, we develop a forecasting method using the satellite precipitation product and wavelet-based machine learning models. We test the proposed approach in the flood-prone Vamsadhara river basin, India. The validation results show that the proposed method is promising and has the potential to forecast extreme flood events with longer lead times in comparison with the other benchmark models.