Dynamical indicators for the prediction of bursting phenomena in high-dimensional systems

Model-assisted deep learning of rare extreme events from partial observations

To predict rare extreme events using deep neural networks, one encounters the so-called small data problem because even long-term observations often contain few extreme events. Here, we investigate a model-assisted framework where the training data are obtained from numerical simulations, as opposed to observations, with adequate samples from extreme events. However, to ensure the trained networks are applicable in practice, the training is not performed on the full simulation data; instead, we only use a small subset of observable quantities, which can be measured in practice. We investigate the feasibility of this model-assisted framework on three different dynamical systems (Rössler attractor, FitzHugh-Nagumo model, and a turbulent fluid flow) and three different deep neural network architectures (feedforward, long short-term memory, and reservoir computing). In each case, we study the prediction accuracy, robustness to noise, reproducibility under repeated training, and sensitivity to the type of input data. In particular, we find long short-term memory networks to be most robust to noise and to yield relatively accurate predictions, while requiring minimal fine-tuning of the hyperparameters.

Optimized ensemble deep learning framework for scalable forecasting of dynamics containing extreme events

The remarkable flexibility and adaptability of both deep learning models and ensemble methods have led to the proliferation for their application in understanding many physical phenomena. Traditionally, these two techniques have largely been treated as independent methodologies in practical applications. This study develops an optimized ensemble deep learning framework wherein these two machine learning techniques are jointly used to achieve synergistic improvements in model accuracy, stability, scalability, and reproducibility, prompting a new wave of applications in the forecasting of dynamics. Unpredictability is considered one of the key features of chaotic dynamics; therefore, forecasting such dynamics of nonlinear systems is a relevant issue in the scientific community. It becomes more challenging when the prediction of extreme events is the focus issue for us. In this circumstance, the proposed optimized ensemble deep learning (OEDL) model based on a best convex combination of feed-forward neural networks, reservoir computing, and long short-term memory can play a key role in advancing predictions of dynamics consisting of extreme events. The combined framework can generate the best out-of-sample performance than the individual deep learners and standard ensemble framework for both numerically simulated and real-world data sets. We exhibit the outstanding performance of the OEDL framework for forecasting extreme events generated from a Liénard-type system, prediction of COVID-19 cases in Brazil, dengue cases in San Juan, and sea surface temperature in the Niño 3.4 region.

Cluster-based network modeling-From snapshots to complex dynamical systems

We propose a universal method for data-driven modeling of complex nonlinear dynamics from time-resolved snapshot data without prior knowledge. Complex nonlinear dynamics govern many fields of science and engineering. Data-driven dynamic modeling often assumes a low-dimensional subspace or manifold for the state. We liberate ourselves from this assumption by proposing cluster-based network modeling (CNM) bridging machine learning, network science, and statistical physics. CNM describes short- and long-term behavior and is fully automatable, as it does not rely on application-specific knowledge. CNM is demonstrated for the Lorenz attractor, ECG heartbeat signals, Kolmogorov flow, and a high-dimensional actuated turbulent boundary layer. Even the notoriously difficult modeling benchmark of rare events in the Kolmogorov flow is solved. This automatable universal data-driven representation of complex nonlinear dynamics complements and expands network connectivity science and promises new fast-track avenues to understand, estimate, predict, and control complex systems in all scientific fields.

Extreme events in a network of heterogeneous Josephson junctions

We report intermittent large spiking events in a heterogeneous network of forced Josephson junctions under the influence of repulsive interaction. The response of the individual junctions has been inspected instead of the collective response of the ensemble, which reveals the large spiking events in a subpopulation with characteristic features of extreme events (EE). The network splits into three clusters of junctions, one in coherent libration, one in incoherent rotational motion, and another subpopulation originating EE, which resembles a chimeralike pattern. EE migrates spatially from one to another subpopulation of junctions with the repulsive strength. The origin of EE in a subpopulation and chimera pattern is a generic effect of distributed damping parameter and repulsive interaction, which we verify with another network of the Liénard system. EE originates in the subpopulation via a local riddling of in-phase synchronization. The probability density function of event heights confirms the rare occurrence of large events and the return time of EE as expressed by interevent intervals in the subgroup follows a Poisson distribution. The mechanism of the origin of such a unique clustering is explained qualitatively.

Learning the tangent space of dynamical instabilities from data

For a large class of dynamical systems, the optimally time-dependent (OTD) modes, a set of deformable orthonormal tangent vectors that track directions of instabilities along any trajectory, are known to depend "pointwise" on the state of the system on the attractor but not on the history of the trajectory. We leverage the power of neural networks to learn this "pointwise" mapping from the phase space to OTD space directly from data. The result of the learning process is a cartography of directions associated with strongest instabilities in the phase space. Implications for data-driven prediction and control of dynamical instabilities are discussed.

Closed-loop adaptive control of extreme events in a turbulent flow

Extreme events that arise spontaneously in chaotic dynamical systems often have an adverse impact on the system or the surrounding environment. As such, their mitigation is highly desirable. Here, we introduce a control strategy for mitigating extreme events in a turbulent shear flow. The controller combines a probabilistic prediction of the extreme events with a deterministic actuator. The predictions are used to actuate the controller only when an extreme event is imminent. When actuated, the controller only acts on the degrees of freedom that are involved in the formation of the extreme events, exerting minimal interference with the flow dynamics. As a result, the attractors of the controlled and uncontrolled systems share the same chaotic core (containing the nonextreme events) and only differ in the tail of their distributions. We propose that such adaptive low-dimensional controllers should be used to mitigate extreme events in general chaotic dynamical systems, beyond the shear flow considered here.

Intermittent large deviation of chaotic trajectory in Ikeda map: Signature of extreme events

We notice signatures of extreme eventslike behavior in a laser based Ikeda map. The trajectory of the system occasionally travels a large distance away from the bounded chaotic region, which appears as intermittent spiking events in the temporal dynamics. The large spiking events satisfy the conditions of extreme events as usually observed in dynamical systems. The probability density function of the large spiking events shows a long-tail distribution consistent with the characteristics of rare events. The interevent intervals obey a Poissonlike distribution. We locate the parameter regions of extreme events in phase diagrams. Furthermore, we study two Ikeda maps to explore how and when extreme events terminate via mutual interaction. A pure diffusion of information exchange is unable to terminate extreme events where synchronous occurrence of extreme events is only possible even for large interaction. On the other hand, a threshold-activated coupling can terminate extreme events above a critical value of mutual interaction.

New perspectives for the prediction and statistical quantification of extreme events in high-dimensional dynamical systems

We discuss extreme events as random occurrences of strongly transient dynamics that lead to nonlinear energy transfers within a chaotic attractor. These transient events are the result of finite-time instabilities and therefore are inherently connected with both statistical and dynamical properties of the system. We consider two classes of problems related to extreme events and nonlinear energy transfers, namely (i) the derivation of precursors for the short-term prediction of extreme events, and (ii) the efficient sampling of random realizations for the fastest convergence of the probability density function in the tail region. We summarize recent methods on these problems that rely on the simultaneous consideration of the statistical and dynamical characteristics of the system. This is achieved by combining available data, in the form of second-order statistics, with dynamical equations that provide information for the transient events that lead to extreme responses. We present these methods through two high-dimensional, prototype systems that exhibit strongly chaotic dynamics and extreme responses due to transient instabilities, the Kolmogorov flow and unidirectional nonlinear water waves.This article is part of the theme issue 'Nonlinear energy transfer in dynamical and acoustical systems'.

Data-assisted reduced-order modeling of extreme events in complex dynamical systems

The prediction of extreme events, from avalanches and droughts to tsunamis and epidemics, depends on the formulation and analysis of relevant, complex dynamical systems. Such dynamical systems are characterized by high intrinsic dimensionality with extreme events having the form of rare transitions that are several standard deviations away from the mean. Such systems are not amenable to classical order-reduction methods through projection of the governing equations due to the large intrinsic dimensionality of the underlying attractor as well as the complexity of the transient events. Alternatively, data-driven techniques aim to quantify the dynamics of specific, critical modes by utilizing data-streams and by expanding the dimensionality of the reduced-order model using delayed coordinates. In turn, these methods have major limitations in regions of the phase space with sparse data, which is the case for extreme events. In this work, we develop a novel hybrid framework that complements an imperfect reduced order model, with data-streams that are integrated though a recurrent neural network (RNN) architecture. The reduced order model has the form of projected equations into a low-dimensional subspace that still contains important dynamical information about the system and it is expanded by a long short-term memory (LSTM) regularization. The LSTM-RNN is trained by analyzing the mismatch between the imperfect model and the data-streams, projected to the reduced-order space. The data-driven model assists the imperfect model in regions where data is available, while for locations where data is sparse the imperfect model still provides a baseline for the prediction of the system state. We assess the developed framework on two challenging prototype systems exhibiting extreme events. We show that the blended approach has improved performance compared with methods that use either data streams or the imperfect model alone. Notably the improvement is more significant in regions associated with extreme events, where data is sparse.

Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks

We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto-Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM-LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.

A variational approach to probing extreme events in turbulent dynamical systems

Extreme events are ubiquitous in a wide range of dynamical systems, including turbulent fluid flows, nonlinear waves, large-scale networks, and biological systems. We propose a variational framework for probing conditions that trigger intermittent extreme events in high-dimensional nonlinear dynamical systems. We seek the triggers as the probabilistically feasible solutions of an appropriately constrained optimization problem, where the function to be maximized is a system observable exhibiting intermittent extreme bursts. The constraints are imposed to ensure the physical admissibility of the optimal solutions, that is, significant probability for their occurrence under the natural flow of the dynamical system. We apply the method to a body-forced incompressible Navier-Stokes equation, known as the Kolmogorov flow. We find that the intermittent bursts of the energy dissipation are independent of the external forcing and are instead caused by the spontaneous transfer of energy from large scales to the mean flow via nonlinear triad interactions. The global maximizer of the corresponding variational problem identifies the responsible triad, hence providing a precursor for the occurrence of extreme dissipation events. Specifically, monitoring the energy transfers within this triad allows us to develop a data-driven short-term predictor for the intermittent bursts of energy dissipation. We assess the performance of this predictor through direct numerical simulations.

Reduced-order description of transient instabilities and computation of finite-time Lyapunov exponents

High-dimensional chaotic dynamical systems can exhibit strongly transient features. These are often associated with instabilities that have a finite-time duration. Because of the finite-time character of these transient events, their detection through infinite-time methods, e.g., long term averages, Lyapunov exponents or information about the statistical steady-state, is not possible. Here, we utilize a recently developed framework, the Optimally Time-Dependent (OTD) modes, to extract a time-dependent subspace that spans the modes associated with transient features associated with finite-time instabilities. As the main result, we prove that the OTD modes, under appropriate conditions, converge exponentially fast to the eigendirections of the Cauchy-Green tensor associated with the most intense finite-time instabilities. Based on this observation, we develop a reduced-order method for the computation of finite-time Lyapunov exponents (FTLE) and vectors. In high-dimensional systems, the computational cost of the reduced-order method is orders of magnitude lower than the full FTLE computation. We demonstrate the validity of the theoretical findings on two numerical examples.