An applied mathematics perspective on stochastic modelling for climate

Conditional Gaussian nonlinear system: A fast preconditioner and a cheap surrogate model for complex nonlinear systems

Developing suitable approximate models for analyzing and simulating complex nonlinear systems is practically important. This paper aims at exploring the skill of a rich class of nonlinear stochastic models, known as the conditional Gaussian nonlinear system (CGNS), as both a cheap surrogate model and a fast preconditioner for facilitating many computationally challenging tasks. The CGNS preserves the underlying physics to a large extent and can reproduce intermittency, extreme events, and other non-Gaussian features in many complex systems arising from practical applications. Three interrelated topics are studied. First, the closed analytic formulas of solving the conditional statistics provide an efficient and accurate data assimilation scheme. It is shown that the data assimilation skill of a suitable CGNS approximate forecast model outweighs that by applying an ensemble method even to the perfect model with strong nonlinearity, where the latter suffers from filter divergence. Second, the CGNS allows the development of a fast algorithm for simultaneously estimating the parameters and the unobserved variables with uncertainty quantification in the presence of only partial observations. Utilizing an appropriate CGNS as a preconditioner significantly reduces the computational cost in accurately estimating the parameters in the original complex system. Finally, the CGNS advances rapid and statistically accurate algorithms for computing the probability density function and sampling the trajectories of the unobserved state variables. These fast algorithms facilitate the development of an efficient and accurate data-driven method for predicting the linear response of the original system with respect to parameter perturbations based on a suitable CGNS preconditioner.

Four Theories of the Madden-Julian Oscillation

Studies of the Madden-Julian Oscillation (MJO) have progressed considerably during the past decades in observations, numerical modeling, and theoretical understanding. Many theoretical attempts have been made to identify the most essential processes responsible for the existence of the MJO. Criteria are proposed to separate a hypothesis from a theory (based on the first principles with quantitative and testable assumptions, able to predict quantitatively the fundamental scales and eastward propagation of the MJO). Four MJO theories are selected to be summarized and compared in this article: the skeleton theory, moisture-mode theory, gravity-wave theory, and trio-interaction theory of the MJO. These four MJO theories are distinct from each other in their key assumptions, parameterized processes, and, particularly, selection mechanisms for the zonal spatial scale, time scale, and eastward propagation of the MJO. The comparison of the four theories and more recent development in MJO dynamical approaches lead to a realization that theoretical thinking of the MJO is diverse and understanding of MJO dynamics needs to be further advanced.

Predicting observed and hidden extreme events in complex nonlinear dynamical systems with partial observations and short training time series

Extreme events appear in many complex nonlinear dynamical systems. Predicting extreme events has important scientific significance and large societal impacts. In this paper, a new mathematical framework of building suitable nonlinear approximate models is developed, which aims at predicting both the observed and hidden extreme events in complex nonlinear dynamical systems for short-, medium-, and long-range forecasting using only short and partially observed training time series. Different from many ad hoc data-driven regression models, these new nonlinear models take into account physically motivated processes and physics constraints. They also allow efficient and accurate algorithms for parameter estimation, data assimilation, and prediction. Cheap stochastic parameterizations, judicious linear feedback control, and suitable noise inflation strategies are incorporated into the new nonlinear modeling framework, which provide accurate predictions of both the observed and hidden extreme events as well as the strongly non-Gaussian statistics in various highly intermittent nonlinear dyad and triad models, including the Lorenz 63 model. Then, a stochastic mode reduction strategy is applied to a 21-dimensional nonlinear paradigm model for topographic mean flow interaction. The resulting five-dimensional physics-constrained nonlinear approximate model is able to accurately predict extreme events and the regime switching between zonally blocked and unblocked flow patterns. Finally, incorporating judicious linear stochastic processes into a simple nonlinear approximate model succeeds in learning certain complicated nonlinear effects of a six-dimensional low-order Charney-DeVore model with strong chaotic and regime switching behavior. The simple nonlinear approximate model then allows accurate online state estimation and the short- and medium-range forecasting of extreme events.

Model Error, Information Barriers, State Estimation and Prediction in Complex Multiscale Systems

Complex multiscale systems are ubiquitous in many areas. This research expository article discusses the development and applications of a recent information-theoretic framework as well as novel reduced-order nonlinear modeling strategies for understanding and predicting complex multiscale systems. The topics include the basic mathematical properties and qualitative features of complex multiscale systems, statistical prediction and uncertainty quantification, state estimation or data assimilation, and coping with the inevitable model errors in approximating such complex systems. Here, the information-theoretic framework is applied to rigorously quantify the model fidelity, model sensitivity and information barriers arising from different approximation strategies. It also succeeds in assessing the skill of filtering and predicting complex dynamical systems and overcomes the shortcomings in traditional path-wise measurements such as the failure in measuring extreme events. In addition, information theory is incorporated into a systematic data-driven nonlinear stochastic modeling framework that allows effective predictions of nonlinear intermittent time series. Finally, new efficient reduced-order nonlinear modeling strategies combined with information theory for model calibration provide skillful predictions of intermittent extreme events in spatially-extended complex dynamical systems. The contents here include the general mathematical theories, effective numerical procedures, instructive qualitative models, and concrete models from climate, atmosphere and ocean science.

Conditional Gaussian Systems for Multiscale Nonlinear Stochastic Systems: Prediction, State Estimation and Uncertainty Quantification

A conditional Gaussian framework for understanding and predicting complex multiscale nonlinear stochastic systems is developed. Despite the conditional Gaussianity, such systems are nevertheless highly nonlinear and are able to capture the non-Gaussian features of nature. The special structure of the system allows closed analytical formulae for solving the conditional statistics and is thus computationally efficient. A rich gallery of examples of conditional Gaussian systems are illustrated here, which includes data-driven physics-constrained nonlinear stochastic models, stochastically coupled reaction–diffusion models in neuroscience and ecology, and large-scale dynamical models in turbulence, fluids and geophysical flows. Making use of the conditional Gaussian structure, efficient statistically accurate algorithms involving a novel hybrid strategy for different subspaces, a judicious block decomposition and statistical symmetry are developed for solving the Fokker–Planck equation in large dimensions. The conditional Gaussian framework is also applied to develop extremely cheap multiscale data assimilation schemes, such as the stochastic superparameterization, which use particle filters to capture the non-Gaussian statistics on the large-scale part whose dimension is small whereas the statistics of the small-scale part are conditional Gaussian given the large-scale part. Other topics of the conditional Gaussian systems studied here include designing new parameter estimation schemes and understanding model errors.

A unified nonlinear stochastic time series analysis for climate science

Earth’s orbit and axial tilt imprint a strong seasonal cycle on climatological data. Climate variability is typically viewed in terms of fluctuations in the seasonal cycle induced by higher frequency processes. We can interpret this as a competition between the orbitally enforced monthly stability and the fluctuations/noise induced by weather. Here we introduce a new time-series method that determines these contributions from monthly-averaged data. We find that the spatio-temporal distribution of the monthly stability and the magnitude of the noise reveal key fingerprints of several important climate phenomena, including the evolution of the Arctic sea ice cover, the El Nio Southern Oscillation (ENSO), the Atlantic Nio and the Indian Dipole Mode. In analogy with the classical destabilising influence of the ice-albedo feedback on summertime sea ice, we find that during some time interval of the season a destabilising process operates in all of these climate phenomena. The interaction between the destabilisation and the accumulation of noise, which we term the memory effect, underlies phase locking to the seasonal cycle and the statistical nature of seasonal predictability.

Extremes in dynamic-stochastic systems

Extreme events capture the attention and imagination of the general public. Extreme events, especially meteorological and climatological extremes, cause significant economic damages and lead to a significant number of casualties each year. Thus, the prediction of extremes is of obvious importance. Here, I will survey the predictive skill and the predictability of extremes using dynamic-stochastic models. These dynamic-stochastic models combine deterministic nonlinear dynamics with a stochastic component, which consists potentially of both additive and multiplicative noise components. In these models, extremes are created by either the nonlinear dynamics, multiplicative noise, or additive heavy-tailed noises. These models naturally capture the observed clustering of extremes and can be used for the prediction of extremes.

Effective dynamics along given reaction coordinates, and reaction rate theory

In molecular dynamics and related fields one considers dynamical descriptions of complex systems in full (atomic) detail. In order to reduce the overwhelming complexity of realistic systems (high dimension, large timescale spread, limited computational resources) the projection of the full dynamics onto some reaction coordinates is examined in order to extract statistical information like free energies or reaction rates. In this context, the effective dynamics that is induced by the full dynamics on the reaction coordinate space has attracted considerable attention in the literature. In this article, we contribute to this discussion: we first show that if we start with an ergodic diffusion process whose invariant measure is unique then these properties are inherited by the effective dynamics. Then, we give equations for the effective dynamics, discuss whether the dominant timescales and reaction rates inferred from the effective dynamics are accurate approximations of such quantities for the full dynamics, and compare our findings to results from approaches like Mori–Zwanzig, averaging, or homogenization. Finally, by discussing the algorithmic realization of the effective dynamics, we demonstrate that recent algorithmic techniques like the “equation-free” approach and the “heterogeneous multiscale method” can be seen as special cases of our approach.

Blended particle filters for large-dimensional chaotic dynamical systems

A major challenge in contemporary data science is the development of statistically accurate particle filters to capture non-Gaussian features in large-dimensional chaotic dynamical systems. Blended particle filters that capture non-Gaussian features in an adaptively evolving low-dimensional subspace through particles interacting with evolving Gaussian statistics on the remaining portion of phase space are introduced here. These blended particle filters are constructed in this paper through a mathematical formalism involving conditional Gaussian mixtures combined with statistically nonlinear forecast models compatible with this structure developed recently with high skill for uncertainty quantification. Stringent test cases for filtering involving the 40-dimensional Lorenz 96 model with a 5-dimensional adaptive subspace for nonlinear blended filtering in various turbulent regimes with at least nine positive Lyapunov exponents are used here. These cases demonstrate the high skill of the blended particle filter algorithms in capturing both highly non-Gaussian dynamical features as well as crucial nonlinear statistics for accurate filtering in extreme filtering regimes with sparse infrequent high-quality observations. The formalism developed here is also useful for multiscale filtering of turbulent systems and a simple application is sketched below.

The role of additive and multiplicative noise in filtering complex dynamical systems

Covariance inflation is an ad hoc treatment that is widely used in practical real-time data assimilation algorithms to mitigate covariance underestimation owing to model errors, nonlinearity, or/and, in the context of ensemble filters, insufficient ensemble size. In this paper, we systematically derive an effective ‘statistical’ inflation for filtering multi-scale dynamical systems with moderate scale gap, , to the case of no scale gap with , in the presence of model errors through reduced dynamics from rigorous stochastic subgrid-scale parametrizations. We will demonstrate that for linear problems, an effective covariance inflation is achieved by a systematically derived additive noise in the forecast model, producing superior filtering skill. For nonlinear problems, we will study an analytically solvable stochastic test model, mimicking turbulent signals in regimes ranging from a turbulent energy transfer range to a dissipative range to a laminar regime. In this context, we will show that multiplicative noise naturally arises in addition to additive noise in a reduced stochastic forecast model. Subsequently, we will show that a ‘statistical’ inflation factor that involves mean correction in addition to covariance inflation is necessary to achieve accurate filtering in the presence of intermittent instability in both the turbulent energy transfer range and the dissipative range.

A new modelling framework for statistical cumulus dynamics

We propose a new modelling framework suitable for the description of atmospheric convective systems as a collection of distinct plumes. The literature contains many examples of models for collections of plumes in which strong simplifying assumptions are made, a diagnostic dependence of convection on the large-scale environment and the limit of many plumes often being imposed from the outset. Some recent studies have sought to remove one or the other of those assumptions. The proposed framework removes both, and is explicitly time dependent and stochastic in its basic character. The statistical dynamics of the plume collection are defined through simple probabilistic rules applied at the level of individual plumes, and van Kampen's system size expansion is then used to construct the macroscopic limit of the microscopic model. Through suitable choices of the microscopic rules, the model is shown to encompass previous studies in the appropriate limits, and to allow their natural extensions beyond those limits.

Predictability of extreme events in a nonlinear stochastic-dynamical model

The objective of this work is to evaluate the potential of reduced order models to reproduce the extreme event and predictability characteristics of higher dimensional dynamical systems. A nonlinear toy model is used which contains key features of comprehensive climate models. First, we demonstrate that the systematic stochastic mode reduction strategy leads to a reduced order model with the same extreme value characteristics as the full dynamical models for a wide range of time-scale separations. Second, we find that extreme events in this model follow a generalized Pareto distribution with a negative shape parameter; thus extreme events are bounded in this model. Third, we show that a precursor approach has good forecast skill for extreme events. We then find that the reduced stochastic models capture the predictive skill of extreme events of the full dynamical models well. Consistent with previous studies we also find that the larger the extreme events, the better predictable they are. Our results suggest that systematically derived reduced order models have the potential to be used for the modeling and statistical prediction of weather- and climate-related extreme events and, possibly, in other areas of science and engineering too.

High skill in low-frequency climate response through fluctuation dissipation theorems despite structural instability

Climate change science focuses on predicting the coarse-grained, planetary-scale, longtime changes in the climate system due to either changes in external forcing or internal variability, such as the impact of increased carbon dioxide. The predictions of climate change science are carried out through comprehensive, computational atmospheric, and oceanic simulation models, which necessarily parameterize physical features such as clouds, sea ice cover, etc. Recently, it has been suggested that there is irreducible imprecision in such climate models that manifests itself as structural instability in climate statistics and which can significantly hamper the skill of computer models for climate change. A systematic approach to deal with this irreducible imprecision is advocated through algorithms based on the Fluctuation Dissipation Theorem (FDT). There are important practical and computational advantages for climate change science when a skillful FDT algorithm is established. The FDT response operator can be utilized directly for multiple climate change scenarios, multiple changes in forcing, and other parameters, such as damping and inverse modelling directly without the need of running the complex climate model in each individual case. The high skill of FDT in predicting climate change, despite structural instability, is developed in an unambiguous fashion using mathematical theory as guidelines in three different test models: a generic class of analytical models mimicking the dynamical core of the computer climate models, reduced stochastic models for low-frequency variability, and models with a significant new type of irreducible imprecision involving many fast, unstable modes.

Normal forms for reduced stochastic climate models

The systematic development of reduced low-dimensional stochastic climate models from observations or comprehensive high-dimensional climate models is an important topic for atmospheric low-frequency variability, climate sensitivity, and improved extended range forecasting. Here techniques from applied mathematics are utilized to systematically derive normal forms for reduced stochastic climate models for low-frequency variables. The use of a few Empirical Orthogonal Functions (EOFs) (also known as Principal Component Analysis, Karhunen-Loéve and Proper Orthogonal Decomposition) depending on observational data to span the low-frequency subspace requires the assessment of dyad interactions besides the more familiar triads in the interaction between the low- and high-frequency subspaces of the dynamics. It is shown below that the dyad and multiplicative triad interactions combine with the climatological linear operator interactions to simultaneously produce both strong nonlinear dissipation and Correlated Additive and Multiplicative (CAM) stochastic noise. For a single low-frequency variable the dyad interactions and climatological linear operator alone produce a normal form with CAM noise from advection of the large scales by the small scales and simultaneously strong cubic damping. These normal forms should prove useful for developing systematic strategies for the estimation of stochastic models from climate data. As an illustrative example the one-dimensional normal form is applied below to low-frequency patterns such as the North Atlantic Oscillation (NAO) in a climate model. The results here also illustrate the short comings of a recent linear scalar CAM noise model proposed elsewhere for low-frequency variability.

Stochastic models for convective momentum transport

The improved parameterization of unresolved features of tropical convection is a central challenge in current computer models for long-range ensemble forecasting of weather and short-term climate change. Observations, theory, and detailed smaller-scale numerical simulations suggest that convective momentum transport (CMT) from the unresolved scales to the resolved scales is one of the major deficiencies in contemporary computer models. Here, a combination of mathematical and physical reasoning is utilized to build simple stochastic models that capture the significant intermittent upscale transports of CMT on the large scales due to organized unresolved convection from squall lines. Properties of the stochastic model for CMT are developed below in a test column model environment for the large-scale variables. The effects of CMT from the stochastic model on a large-scale convectively coupled wave in an idealized setting are presented below as a nontrivial test problem. Here, the upscale transports from stochastic effects are significant and even generate a large-scale mean flow which can interact with the convectively coupled wave.

Introduction. Stochastic physics and climate modelling

Finite computing resources limit the spatial resolution of state-of-the-art global climate simulations to hundreds of kilometres. In neither the atmosphere nor the ocean are small-scale processes such as convection, clouds and ocean eddies properly represented. Climate simulations are known to depend, sometimes quite strongly, on the resulting bulk-formula representation of unresolved processes. Stochastic physics schemes within weather and climate models have the potential to represent the dynamical effects of unresolved scales in ways which conventional bulk-formula representations are incapable of so doing. The application of stochastic physics to climate modelling is a rapidly advancing, important and innovative topic. The latest research findings are gathered together in the Theme Issue for which this paper serves as the introduction.