New perspectives for the prediction and statistical quantification of extreme events in high-dimensional dynamical systems

Time-series-analysis-based detection of critical transitions in real-world non-autonomous systems

Real-world non-autonomous systems are open, out-of-equilibrium systems that evolve in and are driven by temporally varying environments. Such systems can show multiple timescale and transient dynamics together with transitions to very different and, at times, even disastrous dynamical regimes. Since such critical transitions disrupt the systems' intended or desired functionality, it is crucial to understand the underlying mechanisms, to identify precursors of such transitions, and to reliably detect them in time series of suitable system observables to enable forecasts. This review critically assesses the various steps of investigation involved in time-series-analysis-based detection of critical transitions in real-world non-autonomous systems: from the data recording to evaluating the reliability of offline and online detections. It will highlight pros and cons to stimulate further developments, which would be necessary to advance understanding and forecasting nonlinear behavior such as critical transitions in complex systems.

Extreme rotational events in a forced-damped nonlinear pendulum

Since Galileo's time, the pendulum has evolved into one of the most exciting physical objects in mathematical modeling due to its vast range of applications for studying various oscillatory dynamics, including bifurcations and chaos, under various interests. This well-deserved focus aids in comprehending various oscillatory physical phenomena that can be reduced to the equations of the pendulum. The present article focuses on the rotational dynamics of the two-dimensional forced-damped pendulum under the influence of the ac and dc torque. Interestingly, we are able to detect a range of the pendulum's length for which the angular velocity exhibits a few intermittent extreme rotational events that deviate significantly from a certain well-defined threshold. The statistics of the return intervals between these extreme rotational events are supported by our data to be spread exponentially at a specific pendulum's length beyond which the external dc and ac torque are no longer sufficient for a full rotation around the pivot. The numerical results show a sudden increase in the size of the chaotic attractor due to interior crisis, which is the source of instability that is responsible for triggering large amplitude events in our system. We also notice the occurrence of phase slips with the appearance of extreme rotational events when the phase difference between the instantaneous phase of the system and the externally applied ac torque is observed.

Extreme transient dynamics

We study the extreme transient dynamics of four self-excited pendula coupled via the movable beam. A slight difference in the pendula lengths induces the appearance of traveling phase behavior, within which the oscillators synchronize, but the phases between the nodes change in time. We discuss various scenarios of traveling states (involving different pendula) and their properties, comparing them with classical synchronization patterns of phase-locking. The research investigates the problem of transient dynamics preceding the stabilization of the network on a final synchronous attractor, showing that the width of transient windows can become extremely long. The relation between the behavior of the system within the transient regime and its initial conditions is examined and described. Our results include both identical and non-identical pendula masses, showing that the distribution of the latter ones is related to the transients. The research performed in this paper underlines possible transient problems occurring during the analysis of the systems when the slow evolution of the dynamics can be misinterpreted as the final behavior.

Extreme events in a complex network: Interplay between degree distribution and repulsive interaction

The role of topological heterogeneity in the origin of extreme events in a network is investigated here. The dynamics of the oscillators associated with the nodes are assumed to be identical and influenced by mean-field repulsive interactions. An interplay of topological heterogeneity and the repulsive interaction between the dynamical units of the network triggers extreme events in the nodes when each node succumbs to such events for discretely different ranges of repulsive coupling. A high degree node is vulnerable to weaker repulsive interactions, while a low degree node is susceptible to stronger interactions. As a result, the formation of extreme events changes position with increasing strength of repulsive interaction from high to low degree nodes. Extreme events at any node are identified with the appearance of occasional large-amplitude events (amplitude of the temporal dynamics) that are larger than a threshold height and rare in occurrence, which we confirm by estimating the probability distribution of all events. Extreme events appear at any oscillator near the boundary of transition from rotation to libration at a critical value of the repulsive coupling strength. To explore the phenomenon, a paradigmatic second-order phase model is used to represent the dynamics of the oscillator associated with each node. We make an annealed network approximation to reduce our original model and, thereby, confirm the dual role of the repulsive interaction and the degree of a node in the origin of extreme events in any oscillator associated with a node.

Optimized ensemble deep learning framework for scalable forecasting of dynamics containing extreme events

The remarkable flexibility and adaptability of both deep learning models and ensemble methods have led to the proliferation for their application in understanding many physical phenomena. Traditionally, these two techniques have largely been treated as independent methodologies in practical applications. This study develops an optimized ensemble deep learning framework wherein these two machine learning techniques are jointly used to achieve synergistic improvements in model accuracy, stability, scalability, and reproducibility, prompting a new wave of applications in the forecasting of dynamics. Unpredictability is considered one of the key features of chaotic dynamics; therefore, forecasting such dynamics of nonlinear systems is a relevant issue in the scientific community. It becomes more challenging when the prediction of extreme events is the focus issue for us. In this circumstance, the proposed optimized ensemble deep learning (OEDL) model based on a best convex combination of feed-forward neural networks, reservoir computing, and long short-term memory can play a key role in advancing predictions of dynamics consisting of extreme events. The combined framework can generate the best out-of-sample performance than the individual deep learners and standard ensemble framework for both numerically simulated and real-world data sets. We exhibit the outstanding performance of the OEDL framework for forecasting extreme events generated from a Liénard-type system, prediction of COVID-19 cases in Brazil, dengue cases in San Juan, and sea surface temperature in the Niño 3.4 region.

Short- and long-term predictions of chaotic flows and extreme events: a physics-constrained reservoir computing approach

We propose a physics-constrained machine learning method—based on reservoir computing—to time-accurately predict extreme events and long-term velocity statistics in a model of chaotic flow. The method leverages the strengths of two different approaches: empirical modelling based on reservoir computing, which learns the chaotic dynamics from data only, and physical modelling based on conservation laws. This enables the reservoir computing framework to output physical predictions when training data are unavailable. We show that the combination of the two approaches is able to accurately reproduce the velocity statistics, and to predict the occurrence and amplitude of extreme events in a model of self-sustaining process in turbulence. In this flow, the extreme events are abrupt transitions from turbulent to quasi-laminar states, which are deterministic phenomena that cannot be traditionally predicted because of chaos. Furthermore, the physics-constrained machine learning method is shown to be robust with respect to noise. This work opens up new possibilities for synergistically enhancing data-driven methods with physical knowledge for the time-accurate prediction of chaotic flows.

Energy Markets Forecasting. From Inferential Statistics to Machine Learning: The German Case

In this work, we investigate a probabilistic method for electricity price forecasting, which overcomes traditional ones. We start considering statistical methods for point forecast, comparing their performance in terms of efficiency, accuracy, and reliability, and we then exploit Neural Networks approaches to derive a hybrid model for probabilistic type forecasting. We show that our solution reaches the highest standard both in terms of efficiency and precision by testing its output on German electricity prices data.

Understanding the origin of extreme events in El Niño southern oscillation

We investigate a low-dimensional slow-fast model to understand the dynamical origin of El Niño southern oscillation. A close inspection of the system dynamics using several bifurcation plots reveals that a sudden large expansion of the attractor occurs at a critical system parameter via a type of interior crisis. This interior crisis evolves through merging of a cascade of period-doubling and period-adding bifurcations that leads to the origin of occasional amplitude-modulated extremely large events. More categorically, a situation similar to homoclinic chaos arises near the critical point; however, atypical global instability evolves as a channellike structure in phase space of the system that modulates variability of amplitude and return time of the occasional large events and makes a difference from the homoclinic chaos. The slow-fast timescale of the low-dimensional model plays an important role on the onset of occasional extremely large events. Such extreme events are characterized by their heights when they exceed a threshold level measured by a mean-excess function. The probability density of events' height displays multimodal distribution with an upper-bounded tail. We identify the dependence structure of interevent intervals to understand the predictability of return time of such extreme events using autoregressive integrated moving average model and box-plot analysis.

Routes to extreme events in dynamical systems: Dynamical and statistical characteristics

Intermittent large amplitude events are seen in the temporal evolution of a state variable of many dynamical systems. Such intermittent large events suddenly start appearing in dynamical systems at a critical value of a system parameter and continues for a range of parameter values. Three important processes of instabilities, namely, interior crisis, Pomeau-Manneville intermittency, and the breakdown of quasiperiodic motion, are most common as observed in many systems that lead to such occasional and rare transitions to large amplitude spiking events. We characterize these occasional large events as extreme events if they are larger than a statistically defined significant height. We present two exemplary systems, a single system and a coupled system, to illustrate how the instabilities work to originate extreme events and they manifest as non-trivial dynamical events. We illustrate the dynamical and statistical properties of such events.

Predicting observed and hidden extreme events in complex nonlinear dynamical systems with partial observations and short training time series

Extreme events appear in many complex nonlinear dynamical systems. Predicting extreme events has important scientific significance and large societal impacts. In this paper, a new mathematical framework of building suitable nonlinear approximate models is developed, which aims at predicting both the observed and hidden extreme events in complex nonlinear dynamical systems for short-, medium-, and long-range forecasting using only short and partially observed training time series. Different from many ad hoc data-driven regression models, these new nonlinear models take into account physically motivated processes and physics constraints. They also allow efficient and accurate algorithms for parameter estimation, data assimilation, and prediction. Cheap stochastic parameterizations, judicious linear feedback control, and suitable noise inflation strategies are incorporated into the new nonlinear modeling framework, which provide accurate predictions of both the observed and hidden extreme events as well as the strongly non-Gaussian statistics in various highly intermittent nonlinear dyad and triad models, including the Lorenz 63 model. Then, a stochastic mode reduction strategy is applied to a 21-dimensional nonlinear paradigm model for topographic mean flow interaction. The resulting five-dimensional physics-constrained nonlinear approximate model is able to accurately predict extreme events and the regime switching between zonally blocked and unblocked flow patterns. Finally, incorporating judicious linear stochastic processes into a simple nonlinear approximate model succeeds in learning certain complicated nonlinear effects of a six-dimensional low-order Charney-DeVore model with strong chaotic and regime switching behavior. The simple nonlinear approximate model then allows accurate online state estimation and the short- and medium-range forecasting of extreme events.

Extreme events in a network of heterogeneous Josephson junctions

We report intermittent large spiking events in a heterogeneous network of forced Josephson junctions under the influence of repulsive interaction. The response of the individual junctions has been inspected instead of the collective response of the ensemble, which reveals the large spiking events in a subpopulation with characteristic features of extreme events (EE). The network splits into three clusters of junctions, one in coherent libration, one in incoherent rotational motion, and another subpopulation originating EE, which resembles a chimeralike pattern. EE migrates spatially from one to another subpopulation of junctions with the repulsive strength. The origin of EE in a subpopulation and chimera pattern is a generic effect of distributed damping parameter and repulsive interaction, which we verify with another network of the Liénard system. EE originates in the subpopulation via a local riddling of in-phase synchronization. The probability density function of event heights confirms the rare occurrence of large events and the return time of EE as expressed by interevent intervals in the subgroup follows a Poisson distribution. The mechanism of the origin of such a unique clustering is explained qualitatively.

Output-weighted optimal sampling for Bayesian regression and rare event statistics using few samples

For many important problems the quantity of interest is an unknown function of the parameters, which is a random vector with known statistics. Since the dependence of the output on this random vector is unknown, the challenge is to identify its statistics, using the minimum number of function evaluations. This problem can be seen in the context of active learning or optimal experimental design. We employ Bayesian regression to represent the derived model uncertainty due to finite and small number of input-output pairs. In this context we evaluate existing methods for optimal sample selection, such as model error minimization and mutual information maximization. We show that for the case of known output variance, the commonly employed criteria in the literature do not take into account the output values of the existing input-output pairs, while for the case of unknown output variance this dependence can be very weak. We introduce a criterion that takes into account the values of the output for the existing samples and adaptively selects inputs from regions of the parameter space which have an important contribution to the output. The new method allows for application to high-dimensional inputs, paving the way for optimal experimental design in high dimensions.

Intermittent large deviation of chaotic trajectory in Ikeda map: Signature of extreme events

We notice signatures of extreme eventslike behavior in a laser based Ikeda map. The trajectory of the system occasionally travels a large distance away from the bounded chaotic region, which appears as intermittent spiking events in the temporal dynamics. The large spiking events satisfy the conditions of extreme events as usually observed in dynamical systems. The probability density function of the large spiking events shows a long-tail distribution consistent with the characteristics of rare events. The interevent intervals obey a Poissonlike distribution. We locate the parameter regions of extreme events in phase diagrams. Furthermore, we study two Ikeda maps to explore how and when extreme events terminate via mutual interaction. A pure diffusion of information exchange is unable to terminate extreme events where synchronous occurrence of extreme events is only possible even for large interaction. On the other hand, a threshold-activated coupling can terminate extreme events above a critical value of mutual interaction.

Introduction to a topical issue 'nonlinear energy transfer in dynamical and acoustical Systems'

This topical issue is devoted to recent developments in the broader field of energy transfer across scales in nonlinear dynamical and acoustical systems. Nonlinear energy transfers are common in Nature, with perhaps the most famous example being energy cascading from large to small length scales in turbulent flows. Yet nonlinearity has been traditionally perceived either as an unavoidable nuisance or as an unwelcome design restriction in engineering systems. Nowadays, however, this trend is reversing, with nonlinear phenomena being intensely studied in diverse disciplines. Furthermore, strong nonlinearity is now intentionally used and explored in a variety of mechanical and physical settings, such as granular media, acoustic metamaterials, nonlinear energy sinks, essentially nonlinear and nonlocal lattices, vibro-impact oscillators, vibration and shock isolation systems, nanotechnology, biomimetic systems, microelectronics, energy harvesters and in other applications. This topical issue is an attempt to document in a single volume some of these recent research developments, in order to establish a common basis and provide motivation and incentive for further development. The aim is to discuss and compare theoretical and experimental approaches pursued by research groups in different areas, and describe the state of the art of nonlinear energy transfer phenomena in an as broad as possible range of applications of current interest.This article is part of the theme issue 'Nonlinear energy transfer in dynamical and acoustical systems'.