A variational approach to probing extreme events in turbulent dynamical systems

Unambiguous Models and Machine Learning Strategies for Anomalous Extreme Events in Turbulent Dynamical System

Data-driven modeling methods are studied for turbulent dynamical systems with extreme events under an unambiguous model framework. New neural network architectures are proposed to effectively learn the key dynamical mechanisms including the multiscale coupling and strong instability, and gain robust skill for long-time prediction resistive to the accumulated model errors from the data-driven approximation. The machine learning model overcomes the inherent limitations in traditional long short-time memory networks by exploiting a conditional Gaussian structure informed of the essential physical dynamics. The model performance is demonstrated under a prototype model from idealized geophysical flow and passive tracers, which exhibits analytical solutions with representative statistical features. Many attractive properties are found in the trained model in recovering the hidden dynamics using a limited dataset and sparse observation time, showing uniformly high skill with persistent numerical stability in predicting both the trajectory and statistical solutions among different statistical regimes away from the training regime. The model framework is promising to be applied to a wider class of turbulent systems with complex structures.

Instanton-based importance sampling for extreme fluctuations in a shell model for turbulent energy cascade

Many out-of-equilibrium flows present non-Gaussian fluctuations in physically relevant observables, such as energy dissipation rate. This implies extreme fluctuations that, although rarely observed, have a significant phenomenology. Recently, path integral methods for importance sampling have emerged from formalism initially devised for quantum field theory and are being successfully applied to the Burgers equation and other fluid models. We proposed exploring the domain of application of these methods using a shell model, a dynamical system for turbulent energy cascade which can be numerically sampled for extreme events in an efficient manner and presents many interesting properties. We start from a validation of the instanton-based importance sampling methodology in the heat equation limit. We explored the limits of the method as nonlinearity grows stronger, finding good qualitative results for small values of the leading nonlinear coefficient. A worst agreement between numerical simulations of the whole systems and instanton results for estimation of the distribution's flatness is observed when increasing the nonlinear intensities.

Different routes to large-intensity pulses in Zeeman laser model

In this study, we report a rich variety of large-intensity pulses exhibited by a Zeeman laser model. The instabilities in the system occur via three different dynamical processes, such as quasiperiodic intermittency, Pomeau-Manneville intermittency, and the breakdown of quasiperiodic motion to chaos followed by an interior crisis. This Zeeman laser model is more capable of exploring the major possible types of instabilities when changing a specific system's parameter in a particular range. We exemplified distinct dynamical transitions of the Zeeman laser model. The statistical measures reveal the appearance of the low probability of large-intensity pulses above the qualifier threshold value. Moreover, they seem to follow an exponential decay that shows a Poisson-like distribution. The impact of noise and time delay effects have been analyzed near the transition point of the system.

Extreme transient dynamics

We study the extreme transient dynamics of four self-excited pendula coupled via the movable beam. A slight difference in the pendula lengths induces the appearance of traveling phase behavior, within which the oscillators synchronize, but the phases between the nodes change in time. We discuss various scenarios of traveling states (involving different pendula) and their properties, comparing them with classical synchronization patterns of phase-locking. The research investigates the problem of transient dynamics preceding the stabilization of the network on a final synchronous attractor, showing that the width of transient windows can become extremely long. The relation between the behavior of the system within the transient regime and its initial conditions is examined and described. Our results include both identical and non-identical pendula masses, showing that the distribution of the latter ones is related to the transients. The research performed in this paper underlines possible transient problems occurring during the analysis of the systems when the slow evolution of the dynamics can be misinterpreted as the final behavior.

Dissipation-range fluid turbulence and thermal noise

We revisit the issue of whether thermal fluctuations are relevant for incompressible fluid turbulence and estimate the scale at which they become important. As anticipated by Betchov in a prescient series of works more than six decades ago, this scale is about equal to the Kolmogorov length, even though that is several orders of magnitude above the mean free path. This result implies that the deterministic version of the incompressible Navier-Stokes equation is inadequate to describe the dissipation range of turbulence in molecular fluids. Within this range, the fluctuating hydrodynamics equation of Landau and Lifschitz is more appropriate. In particular, our analysis implies that both the exponentially decaying energy spectrum and the far-dissipation-range intermittency predicted by Kraichnan for deterministic Navier-Stokes will be generally replaced by Gaussian thermal equipartition at scales just below the Kolmogorov length. Stochastic shell model simulations at high Reynolds numbers verify our theoretical predictions and reveal furthermore that inertial-range intermittency can propagate deep into the dissipation range, leading to large fluctuations in the equipartition length scale. We explain the failure of previous scaling arguments for the validity of deterministic Navier-Stokes equations at any Reynolds number and we provide a mathematical interpretation and physical justification of the fluctuating Navier-Stokes equation as an "effective field theory" valid below some high-wave-number cutoff Λ, rather than as a continuum stochastic partial differential equation. At Reynolds number around a million, comparable to that in Earth's atmospheric boundary layer, the strongest turbulent excitations observed in our simulation penetrate down to a length scale of about eight microns, still two orders of magnitude greater than the mean free path of air. However, for longer observation times or for higher Reynolds numbers, more extreme turbulent events could lead to a local breakdown of fluctuating hydrodynamics.

Model-assisted deep learning of rare extreme events from partial observations

To predict rare extreme events using deep neural networks, one encounters the so-called small data problem because even long-term observations often contain few extreme events. Here, we investigate a model-assisted framework where the training data are obtained from numerical simulations, as opposed to observations, with adequate samples from extreme events. However, to ensure the trained networks are applicable in practice, the training is not performed on the full simulation data; instead, we only use a small subset of observable quantities, which can be measured in practice. We investigate the feasibility of this model-assisted framework on three different dynamical systems (Rössler attractor, FitzHugh-Nagumo model, and a turbulent fluid flow) and three different deep neural network architectures (feedforward, long short-term memory, and reservoir computing). In each case, we study the prediction accuracy, robustness to noise, reproducibility under repeated training, and sensitivity to the type of input data. In particular, we find long short-term memory networks to be most robust to noise and to yield relatively accurate predictions, while requiring minimal fine-tuning of the hyperparameters.

Investigating climate tipping points under various emission reduction and carbon capture scenarios with a stochastic climate model

We study the mitigation of climate tipping point transitions using an energy balance model. The evolution of the global mean surface temperature is coupled with the CO 2 concentration through the green-house effect. We model the CO 2 concentration with a stochastic delay differential equation (SDDE), accounting for various carbon emission and capture scenarios. The resulting coupled system of SDDEs exhibits a tipping point phenomena: if CO 2 concentration exceeds a critical threshold (around 478   ppm ), the temperature experiences an abrupt increase of about six degrees Celsius. We show that the CO 2 concentration exhibits a transient growth which may cause a climate tipping point, even if the concentration decays asymptotically. We derive a rigorous upper bound for the CO 2 evolution which quantifies its transient and asymptotic growths, and provides sufficient conditions for evading the climate tipping point. Combining this upper bound with Monte Carlo simulations of the stochastic climate model, we investigate the emission reduction and carbon capture scenarios that would avert the tipping point.

Instabilities in quasiperiodic motion lead to intermittent large-intensity events in Zeeman laser

We report intermittent large-intensity pulses that originate in Zeeman laser due to instabilities in quasiperiodic motion, one route follows torus-doubling to chaos and another goes via quasiperiodic intermittency in response to variation in system parameters. The quasiperiodic breakdown route to chaos via torus-doubling is well known; however, the laser model shows intermittent large-intensity pulses for parameter variation beyond the chaotic regime. During quasiperiodic intermittency, the temporal evolution of the laser shows intermittent chaotic bursting episodes intermediate to the quasiperiodic motion instead of periodic motion as usually seen during the Pomeau-Manneville intermittency. The intermittent bursting appears as occasional large-intensity events. In particular, this quasiperiodic intermittency has not been given much attention so far from the dynamical system perspective, in general. In both cases, the infrequent and recurrent large events show non-Gaussian probability distribution of event height extended beyond a significant threshold with a decaying probability confirming rare occurrence of large-intensity pulses.

Mitigation of rare events in multistable systems driven by correlated noise

We consider rare transitions induced by colored noise excitation in multistable systems. We show that undesirable transitions can be mitigated by a simple time-delay feedback control if the control parameters are judiciously chosen. We devise a parsimonious method for selecting the optimal control parameters, without requiring any Monte Carlo simulations of the system. This method relies on a new nonlinear Fokker-Planck equation whose stationary response distribution is approximated by a rapidly convergent iterative algorithm. In addition, our framework allows us to accurately predict, and subsequently suppress, the modal drift and tail inflation in the controlled stationary distribution. We demonstrate the efficacy of our method on two examples, including an optical laser model perturbed by multiplicative colored noise.

Cluster-based network modeling-From snapshots to complex dynamical systems

We propose a universal method for data-driven modeling of complex nonlinear dynamics from time-resolved snapshot data without prior knowledge. Complex nonlinear dynamics govern many fields of science and engineering. Data-driven dynamic modeling often assumes a low-dimensional subspace or manifold for the state. We liberate ourselves from this assumption by proposing cluster-based network modeling (CNM) bridging machine learning, network science, and statistical physics. CNM describes short- and long-term behavior and is fully automatable, as it does not rely on application-specific knowledge. CNM is demonstrated for the Lorenz attractor, ECG heartbeat signals, Kolmogorov flow, and a high-dimensional actuated turbulent boundary layer. Even the notoriously difficult modeling benchmark of rare events in the Kolmogorov flow is solved. This automatable universal data-driven representation of complex nonlinear dynamics complements and expands network connectivity science and promises new fast-track avenues to understand, estimate, predict, and control complex systems in all scientific fields.

Data-driven prediction of multistable systems from sparse measurements

We develop a data-driven method, based on semi-supervised classification, to predict the asymptotic state of multistable systems when only sparse spatial measurements of the system are feasible. Our method predicts the asymptotic behavior of an observed state by quantifying its proximity to the states in a precomputed library of data. To quantify this proximity, we introduce a sparsity-promoting metric-learning (SPML) optimization, which learns a metric directly from the precomputed data. The optimization problem is designed so that the resulting optimal metric satisfies two important properties: (i) it is compatible with the precomputed library and (ii) it is computable from sparse measurements. We prove that the proposed SPML optimization is convex, its minimizer is non-degenerate, and it is equivariant with respect to the scaling of the constraints. We demonstrate the application of this method on two multistable systems: a reaction-diffusion equation, arising in pattern formation, which has four asymptotically stable steady states, and a FitzHugh-Nagumo model with two asymptotically stable steady states. Classifications of the multistable reaction-diffusion equation based on SPML predict the asymptotic behavior of initial conditions based on two-point measurements with 95% accuracy when a moderate number of labeled data are used. For the FitzHugh-Nagumo, SPML predicts the asymptotic behavior of initial conditions from one-point measurements with 90% accuracy. The learned optimal metric also determines where the measurements need to be made to ensure accurate predictions.

The tipping times in an Arctic sea ice system under influence of extreme events

In light of the rapid recent retreat of Arctic sea ice, the extreme weather events triggering the variability in Arctic ice cover has drawn increasing attention. A non-Gaussian α-stable Lévy process is thought to be an appropriate model to describe such extreme events. The maximal likely trajectory, based on the nonlocal Fokker-Planck equation, is applied to a nonautonomous Arctic sea ice system under α-stable Lévy noise. Two types of tipping times, the early-warning tipping time and the disaster-happening tipping time, are used to predict the critical time for the maximal likely transition from a perennially ice-covered state to a seasonally ice-free one and from a seasonally ice-free state to a perennially ice-free one, respectively. We find that the increased intensity of extreme events results in shorter warning time for sea ice melting and that an enhanced greenhouse effect will intensify this influence, making the arrival of warning time significantly earlier. Meanwhile, for the enhanced greenhouse effect, we discover that increased intensity and frequency of extreme events will advance the disaster-happening tipping time, in which an ice-free state is maintained throughout the year in the Arctic Ocean. Finally, we identify values of the Lévy index α and the noise intensity ϵ in the αϵ-space that can trigger a transition between the Arctic sea ice state. These results provide an effective theoretical framework for studying Arctic sea ice variations under the influence of extreme events.

Routes to extreme events in dynamical systems: Dynamical and statistical characteristics

Intermittent large amplitude events are seen in the temporal evolution of a state variable of many dynamical systems. Such intermittent large events suddenly start appearing in dynamical systems at a critical value of a system parameter and continues for a range of parameter values. Three important processes of instabilities, namely, interior crisis, Pomeau-Manneville intermittency, and the breakdown of quasiperiodic motion, are most common as observed in many systems that lead to such occasional and rare transitions to large amplitude spiking events. We characterize these occasional large events as extreme events if they are larger than a statistically defined significant height. We present two exemplary systems, a single system and a coupled system, to illustrate how the instabilities work to originate extreme events and they manifest as non-trivial dynamical events. We illustrate the dynamical and statistical properties of such events.

Output-weighted optimal sampling for Bayesian regression and rare event statistics using few samples

For many important problems the quantity of interest is an unknown function of the parameters, which is a random vector with known statistics. Since the dependence of the output on this random vector is unknown, the challenge is to identify its statistics, using the minimum number of function evaluations. This problem can be seen in the context of active learning or optimal experimental design. We employ Bayesian regression to represent the derived model uncertainty due to finite and small number of input-output pairs. In this context we evaluate existing methods for optimal sample selection, such as model error minimization and mutual information maximization. We show that for the case of known output variance, the commonly employed criteria in the literature do not take into account the output values of the existing input-output pairs, while for the case of unknown output variance this dependence can be very weak. We introduce a criterion that takes into account the values of the output for the existing samples and adaptively selects inputs from regions of the parameter space which have an important contribution to the output. The new method allows for application to high-dimensional inputs, paving the way for optimal experimental design in high dimensions.

Mitigation of tipping point transitions by time-delay feedback control

In stochastic multistable systems driven by the gradient of a potential, transitions between equilibria are possible because of noise. We study the ability of linear delay feedback control to mitigate these transitions, ensuring that the system stays near a desirable equilibrium. For small delays, we show that the control term has two effects: (i) a stabilizing effect by deepening the potential well around the desirable equilibrium and (ii) a destabilizing effect by intensifying the noise by a factor of (1-τα)^-1/2, where τ and α denote the delay and the control gain, respectively. As a result, successful mitigation depends on the competition between these two factors. We also derive analytical results that elucidate the choice of the appropriate control gain and delay that ensure successful mitigations. These results eliminate the need for any Monte Carlo simulations of the stochastic differential equations and, therefore, significantly reduce the computational cost of determining the suitable control parameters. We demonstrate the application of our results on two examples.

Machine Learning Predictors of Extreme Events Occurring in Complex Dynamical Systems

The ability to characterize and predict extreme events is a vital topic in fields ranging from finance to ocean engineering. Typically, the most-extreme events are also the most-rare, and it is this property that makes data collection and direct simulation challenging. We consider the problem of deriving optimal predictors of extremes directly from data characterizing a complex system, by formulating the problem in the context of binary classification. Specifically, we assume that a training dataset consists of: (i) indicator time series specifying on whether or not an extreme event occurs; and (ii) observables time series, which are employed to formulate efficient predictors. We employ and assess standard binary classification criteria for the selection of optimal predictors, such as total and balanced error and area under the curve, in the context of extreme event prediction. For physical systems for which there is sufficient separation between the extreme and regular events, i.e., extremes are distinguishably larger compared with regular events, we prove the existence of optimal extreme event thresholds that lead to efficient predictors. Moreover, motivated by the special character of extreme events, i.e., the very low rate of occurrence, we formulate a new objective function for the selection of predictors. This objective is constructed from the same principles as receiver operating characteristic curves, and exhibits a geometric connection to the regime separation property. We demonstrate the application of the new selection criterion to the advance prediction of intermittent extreme events in two challenging complex systems: the Majda–McLaughlin–Tabak model, a 1D nonlinear, dispersive wave model, and the 2D Kolmogorov flow model, which exhibits extreme dissipation events.

Closed-loop adaptive control of extreme events in a turbulent flow

Extreme events that arise spontaneously in chaotic dynamical systems often have an adverse impact on the system or the surrounding environment. As such, their mitigation is highly desirable. Here, we introduce a control strategy for mitigating extreme events in a turbulent shear flow. The controller combines a probabilistic prediction of the extreme events with a deterministic actuator. The predictions are used to actuate the controller only when an extreme event is imminent. When actuated, the controller only acts on the degrees of freedom that are involved in the formation of the extreme events, exerting minimal interference with the flow dynamics. As a result, the attractors of the controlled and uncontrolled systems share the same chaotic core (containing the nonextreme events) and only differ in the tail of their distributions. We propose that such adaptive low-dimensional controllers should be used to mitigate extreme events in general chaotic dynamical systems, beyond the shear flow considered here.

New perspectives for the prediction and statistical quantification of extreme events in high-dimensional dynamical systems

We discuss extreme events as random occurrences of strongly transient dynamics that lead to nonlinear energy transfers within a chaotic attractor. These transient events are the result of finite-time instabilities and therefore are inherently connected with both statistical and dynamical properties of the system. We consider two classes of problems related to extreme events and nonlinear energy transfers, namely (i) the derivation of precursors for the short-term prediction of extreme events, and (ii) the efficient sampling of random realizations for the fastest convergence of the probability density function in the tail region. We summarize recent methods on these problems that rely on the simultaneous consideration of the statistical and dynamical characteristics of the system. This is achieved by combining available data, in the form of second-order statistics, with dynamical equations that provide information for the transient events that lead to extreme responses. We present these methods through two high-dimensional, prototype systems that exhibit strongly chaotic dynamics and extreme responses due to transient instabilities, the Kolmogorov flow and unidirectional nonlinear water waves.This article is part of the theme issue 'Nonlinear energy transfer in dynamical and acoustical systems'.

Data-assisted reduced-order modeling of extreme events in complex dynamical systems

The prediction of extreme events, from avalanches and droughts to tsunamis and epidemics, depends on the formulation and analysis of relevant, complex dynamical systems. Such dynamical systems are characterized by high intrinsic dimensionality with extreme events having the form of rare transitions that are several standard deviations away from the mean. Such systems are not amenable to classical order-reduction methods through projection of the governing equations due to the large intrinsic dimensionality of the underlying attractor as well as the complexity of the transient events. Alternatively, data-driven techniques aim to quantify the dynamics of specific, critical modes by utilizing data-streams and by expanding the dimensionality of the reduced-order model using delayed coordinates. In turn, these methods have major limitations in regions of the phase space with sparse data, which is the case for extreme events. In this work, we develop a novel hybrid framework that complements an imperfect reduced order model, with data-streams that are integrated though a recurrent neural network (RNN) architecture. The reduced order model has the form of projected equations into a low-dimensional subspace that still contains important dynamical information about the system and it is expanded by a long short-term memory (LSTM) regularization. The LSTM-RNN is trained by analyzing the mismatch between the imperfect model and the data-streams, projected to the reduced-order space. The data-driven model assists the imperfect model in regions where data is available, while for locations where data is sparse the imperfect model still provides a baseline for the prediction of the system state. We assess the developed framework on two challenging prototype systems exhibiting extreme events. We show that the blended approach has improved performance compared with methods that use either data streams or the imperfect model alone. Notably the improvement is more significant in regions associated with extreme events, where data is sparse.

Rogue waves and large deviations in deep sea

The appearance of rogue waves in deep sea is investigated by using the modified nonlinear Schrödinger (MNLS) equation in one spatial dimension with random initial conditions that are assumed to be normally distributed, with a spectrum approximating realistic conditions of a unidirectional sea state. It is shown that one can use the incomplete information contained in this spectrum as prior and supplement this information with the MNLS dynamics to reliably estimate the probability distribution of the sea surface elevation far in the tail at later times. Our results indicate that rogue waves occur when the system hits unlikely pockets of wave configurations that trigger large disturbances of the surface height. The rogue wave precursors in these pockets are wave patterns of regular height, but with a very specific shape that is identified explicitly, thereby allowing for early detection. The method proposed here combines Monte Carlo sampling with tools from large deviations theory that reduce the calculation of the most likely rogue wave precursors to an optimization problem that can be solved efficiently. This approach is transferable to other problems in which the system's governing equations contain random initial conditions and/or parameters.