1
|
Lehnertz K. Time-series-analysis-based detection of critical transitions in real-world non-autonomous systems. CHAOS (WOODBURY, N.Y.) 2024; 34:072102. [PMID: 38985967 DOI: 10.1063/5.0214733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/19/2024] [Accepted: 06/21/2024] [Indexed: 07/12/2024]
Abstract
Real-world non-autonomous systems are open, out-of-equilibrium systems that evolve in and are driven by temporally varying environments. Such systems can show multiple timescale and transient dynamics together with transitions to very different and, at times, even disastrous dynamical regimes. Since such critical transitions disrupt the systems' intended or desired functionality, it is crucial to understand the underlying mechanisms, to identify precursors of such transitions, and to reliably detect them in time series of suitable system observables to enable forecasts. This review critically assesses the various steps of investigation involved in time-series-analysis-based detection of critical transitions in real-world non-autonomous systems: from the data recording to evaluating the reliability of offline and online detections. It will highlight pros and cons to stimulate further developments, which would be necessary to advance understanding and forecasting nonlinear behavior such as critical transitions in complex systems.
Collapse
|
2
|
Evangelou N, Cui T, Bello-Rivas JM, Makeev A, Kevrekidis IG. Tipping points of evolving epidemiological networks: Machine learning-assisted, data-driven effective modeling. CHAOS (WOODBURY, N.Y.) 2024; 34:063128. [PMID: 38865091 DOI: 10.1063/5.0187511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Accepted: 04/20/2024] [Indexed: 06/13/2024]
Abstract
We study the tipping point collective dynamics of an adaptive susceptible-infected-susceptible (SIS) epidemiological network in a data-driven, machine learning-assisted manner. We identify a parameter-dependent effective stochastic differential equation (eSDE) in terms of physically meaningful coarse mean-field variables through a deep-learning ResNet architecture inspired by numerical stochastic integrators. We construct an approximate effective bifurcation diagram based on the identified drift term of the eSDE and contrast it with the mean-field SIS model bifurcation diagram. We observe a subcritical Hopf bifurcation in the evolving network's effective SIS dynamics that causes the tipping point behavior; this takes the form of large amplitude collective oscillations that spontaneously-yet rarely-arise from the neighborhood of a (noisy) stationary state. We study the statistics of these rare events both through repeated brute force simulations and by using established mathematical/computational tools exploiting the right-hand side of the identified SDE. We demonstrate that such a collective SDE can also be identified (and the rare event computations also performed) in terms of data-driven coarse observables, obtained here via manifold learning techniques, in particular, Diffusion Maps. The workflow of our study is straightforwardly applicable to other complex dynamic problems exhibiting tipping point dynamics.
Collapse
Affiliation(s)
- Nikolaos Evangelou
- Department of Chemical and Biomolecular Engineering, Johns Hopkins University, Baltimore, Maryland 21218, USA
| | - Tianqi Cui
- Department of Chemical and Biomolecular Engineering, Johns Hopkins University, Baltimore, Maryland 21218, USA
| | - Juan M Bello-Rivas
- Department of Chemical and Biomolecular Engineering, Johns Hopkins University, Baltimore, Maryland 21218, USA
| | - Alexei Makeev
- Faculty of Computational Mathematics and Cybernetics, Moscow State University, 119991 Moscow, Russia
| | - Ioannis G Kevrekidis
- Department of Chemical and Biomolecular Engineering, Johns Hopkins University, Baltimore, Maryland 21218, USA
| |
Collapse
|
3
|
Hart JD. Attractor reconstruction with reservoir computers: The effect of the reservoir's conditional Lyapunov exponents on faithful attractor reconstruction. CHAOS (WOODBURY, N.Y.) 2024; 34:043123. [PMID: 38579149 DOI: 10.1063/5.0196257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 03/22/2024] [Indexed: 04/07/2024]
Abstract
Reservoir computing is a machine learning framework that has been shown to be able to replicate the chaotic attractor, including the fractal dimension and the entire Lyapunov spectrum, of the dynamical system on which it is trained. We quantitatively relate the generalized synchronization dynamics of a driven reservoir during the training stage to the performance of the trained reservoir computer at the attractor reconstruction task. We show that, in order to obtain successful attractor reconstruction and Lyapunov spectrum estimation, the maximal conditional Lyapunov exponent of the driven reservoir must be significantly more negative than the most negative Lyapunov exponent of the target system. We also find that the maximal conditional Lyapunov exponent of the reservoir depends strongly on the spectral radius of the reservoir adjacency matrix; therefore, for attractor reconstruction and Lyapunov spectrum estimation, small spectral radius reservoir computers perform better in general. Our arguments are supported by numerical examples on well-known chaotic systems.
Collapse
Affiliation(s)
- Joseph D Hart
- U.S. Naval Research Laboratory, Code 5675, Washington, DC 20375, USA
| |
Collapse
|
4
|
Köglmayr D, Räth C. Extrapolating tipping points and simulating non-stationary dynamics of complex systems using efficient machine learning. Sci Rep 2024; 14:507. [PMID: 38177246 PMCID: PMC10767041 DOI: 10.1038/s41598-023-50726-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Accepted: 12/23/2023] [Indexed: 01/06/2024] Open
Abstract
Model-free and data-driven prediction of tipping point transitions in nonlinear dynamical systems is a challenging and outstanding task in complex systems science. We propose a novel, fully data-driven machine learning algorithm based on next-generation reservoir computing to extrapolate the bifurcation behavior of nonlinear dynamical systems using stationary training data samples. We show that this method can extrapolate tipping point transitions. Furthermore, it is demonstrated that the trained next-generation reservoir computing architecture can be used to predict non-stationary dynamics with time-varying bifurcation parameters. In doing so, post-tipping point dynamics of unseen parameter regions can be simulated.
Collapse
Affiliation(s)
- Daniel Köglmayr
- German Aerospace Center (DLR), Institute for AI Safety and Security, 89081, Ulm, Germany.
| | - Christoph Räth
- German Aerospace Center (DLR), Institute for AI Safety and Security, 89081, Ulm, Germany
| |
Collapse
|
5
|
Durstewitz D, Koppe G, Thurm MI. Reconstructing computational system dynamics from neural data with recurrent neural networks. Nat Rev Neurosci 2023; 24:693-710. [PMID: 37794121 DOI: 10.1038/s41583-023-00740-7] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/18/2023] [Indexed: 10/06/2023]
Abstract
Computational models in neuroscience usually take the form of systems of differential equations. The behaviour of such systems is the subject of dynamical systems theory. Dynamical systems theory provides a powerful mathematical toolbox for analysing neurobiological processes and has been a mainstay of computational neuroscience for decades. Recently, recurrent neural networks (RNNs) have become a popular machine learning tool for studying the non-linear dynamics of neural and behavioural processes by emulating an underlying system of differential equations. RNNs have been routinely trained on similar behavioural tasks to those used for animal subjects to generate hypotheses about the underlying computational mechanisms. By contrast, RNNs can also be trained on the measured physiological and behavioural data, thereby directly inheriting their temporal and geometrical properties. In this way they become a formal surrogate for the experimentally probed system that can be further analysed, perturbed and simulated. This powerful approach is called dynamical system reconstruction. In this Perspective, we focus on recent trends in artificial intelligence and machine learning in this exciting and rapidly expanding field, which may be less well known in neuroscience. We discuss formal prerequisites, different model architectures and training approaches for RNN-based dynamical system reconstructions, ways to evaluate and validate model performance, how to interpret trained models in a neuroscience context, and current challenges.
Collapse
Affiliation(s)
- Daniel Durstewitz
- Dept. of Theoretical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany.
- Interdisciplinary Center for Scientific Computing, Heidelberg University, Heidelberg, Germany.
- Faculty of Physics and Astronomy, Heidelberg University, Heidelberg, Germany.
| | - Georgia Koppe
- Dept. of Theoretical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
- Dept. of Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
- Hector Institute for Artificial Intelligence in Psychiatry, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Max Ingo Thurm
- Dept. of Theoretical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| |
Collapse
|
6
|
Bury TM, Dylewsky D, Bauch CT, Anand M, Glass L, Shrier A, Bub G. Predicting discrete-time bifurcations with deep learning. Nat Commun 2023; 14:6331. [PMID: 37816722 PMCID: PMC10564974 DOI: 10.1038/s41467-023-42020-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Accepted: 09/27/2023] [Indexed: 10/12/2023] Open
Abstract
Many natural and man-made systems are prone to critical transitions-abrupt and potentially devastating changes in dynamics. Deep learning classifiers can provide an early warning signal for critical transitions by learning generic features of bifurcations from large simulated training data sets. So far, classifiers have only been trained to predict continuous-time bifurcations, ignoring rich dynamics unique to discrete-time bifurcations. Here, we train a deep learning classifier to provide an early warning signal for the five local discrete-time bifurcations of codimension-one. We test the classifier on simulation data from discrete-time models used in physiology, economics and ecology, as well as experimental data of spontaneously beating chick-heart aggregates that undergo a period-doubling bifurcation. The classifier shows higher sensitivity and specificity than commonly used early warning signals under a wide range of noise intensities and rates of approach to the bifurcation. It also predicts the correct bifurcation in most cases, with particularly high accuracy for the period-doubling, Neimark-Sacker and fold bifurcations. Deep learning as a tool for bifurcation prediction is still in its nascence and has the potential to transform the way we monitor systems for critical transitions.
Collapse
Affiliation(s)
- Thomas M Bury
- Department of Physiology, McGill University, 3655 Promenade Sir William Osler, Montreal, Canada.
| | - Daniel Dylewsky
- Department of Applied Mathematics, University of Waterloo, Waterloo, Canada
| | - Chris T Bauch
- Department of Applied Mathematics, University of Waterloo, Waterloo, Canada
| | - Madhur Anand
- School of Environmental Sciences, University of Guelph, Guelph, Canada
| | - Leon Glass
- Department of Physiology, McGill University, 3655 Promenade Sir William Osler, Montreal, Canada
| | - Alvin Shrier
- Department of Physiology, McGill University, 3655 Promenade Sir William Osler, Montreal, Canada
| | - Gil Bub
- Department of Physiology, McGill University, 3655 Promenade Sir William Osler, Montreal, Canada
| |
Collapse
|
7
|
Skardal PS, Restrepo JG. Detecting disturbances in network-coupled dynamical systems with machine learning. CHAOS (WOODBURY, N.Y.) 2023; 33:103137. [PMID: 37903406 DOI: 10.1063/5.0169237] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Accepted: 10/05/2023] [Indexed: 11/01/2023]
Abstract
Identifying disturbances in network-coupled dynamical systems without knowledge of the disturbances or underlying dynamics is a problem with a wide range of applications. For example, one might want to know which nodes in the network are being disturbed and identify the type of disturbance. Here, we present a model-free method based on machine learning to identify such unknown disturbances based only on prior observations of the system when forced by a known training function. We find that this method is able to identify the locations and properties of many different types of unknown disturbances using a variety of known forcing functions. We illustrate our results with both linear and nonlinear disturbances using food web and neuronal activity models. Finally, we discuss how to scale our method to large networks.
Collapse
Affiliation(s)
| | - Juan G Restrepo
- Department of Applied Mathematics, University of Colorado Boulder, Boulder, Colorado 80309, USA
| |
Collapse
|