1
|
Li X, Zhu Q, Zhao C, Duan X, Zhao B, Zhang X, Ma H, Sun J, Lin W. Higher-order Granger reservoir computing: simultaneously achieving scalable complex structures inference and accurate dynamics prediction. Nat Commun 2024; 15:2506. [PMID: 38509083 PMCID: PMC10954644 DOI: 10.1038/s41467-024-46852-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Accepted: 03/12/2024] [Indexed: 03/22/2024] Open
Abstract
Recently, machine learning methods, including reservoir computing (RC), have been tremendously successful in predicting complex dynamics in many fields. However, a present challenge lies in pushing for the limit of prediction accuracy while maintaining the low complexity of the model. Here, we design a data-driven, model-free framework named higher-order Granger reservoir computing (HoGRC), which owns two major missions: The first is to infer the higher-order structures incorporating the idea of Granger causality with the RC, and, simultaneously, the second is to realize multi-step prediction by feeding the time series and the inferred higher-order information into HoGRC. We demonstrate the efficacy and robustness of the HoGRC using several representative systems, including the classical chaotic systems, the network dynamical systems, and the UK power grid system. In the era of machine learning and complex systems, we anticipate a broad application of the HoGRC framework in structure inference and dynamics prediction.
Collapse
Affiliation(s)
- Xin Li
- Center for Applied Mathematics (NUDT), Changsha, 410073, Hunan, China
- Research Institute of Intelligent Complex Systems and MOE Frontiers Center for Brain Science, Fudan University, Shanghai, 200433, China
| | - Qunxi Zhu
- Research Institute of Intelligent Complex Systems and MOE Frontiers Center for Brain Science, Fudan University, Shanghai, 200433, China.
- School of Mathematical Sciences, SCMS, SCAM, and CCSB, Fudan University, Shanghai, 200433, China.
| | - Chengli Zhao
- Center for Applied Mathematics (NUDT), Changsha, 410073, Hunan, China.
| | - Xiaojun Duan
- Center for Applied Mathematics (NUDT), Changsha, 410073, Hunan, China
| | - Bolin Zhao
- Research Institute of Intelligent Complex Systems and MOE Frontiers Center for Brain Science, Fudan University, Shanghai, 200433, China
- School of Mathematical Sciences, SCMS, SCAM, and CCSB, Fudan University, Shanghai, 200433, China
| | - Xue Zhang
- Center for Applied Mathematics (NUDT), Changsha, 410073, Hunan, China
| | - Huanfei Ma
- School of Mathematical Sciences, Soochow University, Suzhou, 215006, China
| | - Jie Sun
- Research Institute of Intelligent Complex Systems and MOE Frontiers Center for Brain Science, Fudan University, Shanghai, 200433, China
- HUAWEI Technologies Co., Ltd., Hong Kong, China
| | - Wei Lin
- Research Institute of Intelligent Complex Systems and MOE Frontiers Center for Brain Science, Fudan University, Shanghai, 200433, China.
- School of Mathematical Sciences, SCMS, SCAM, and CCSB, Fudan University, Shanghai, 200433, China.
- Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China.
| |
Collapse
|
2
|
Zhong Z, Lin W, Qin BW. Modulating Biological Rhythms: A Noncomputational Strategy Harnessing Nonlinearity and Decoupling Frequency and Amplitude. PHYSICAL REVIEW LETTERS 2023; 131:138401. [PMID: 37832005 DOI: 10.1103/physrevlett.131.138401] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 07/17/2023] [Accepted: 08/30/2023] [Indexed: 10/15/2023]
Abstract
Understanding and achieving concurrent modulation of amplitude and frequency, particularly adjusting one quantity and simultaneously sustaining the other at an invariant level, are of paramount importance for complex biophysical systems, including the signal pathway where different frequency indicates different upstream signal yielding a certain downstream physiological function while different amplitude further determines different efficacy of a downstream output. However, such modulators with clearly described and universally valid mechanisms are still lacking. Here, we rigorously propose an easy-to-use control strategy containing only one or two steps, leveraging the nonlinearity in the modulated systems to decouple frequency and amplitude in a noncomputational manner. The strategy's efficacy is demonstrated using representative biochemical systems and, thus, it could be potentially applicable to modulating rhythms in experiments of biochemistry and synthetic biology.
Collapse
Affiliation(s)
- Zhaoyue Zhong
- School of Mathematical Sciences and Shanghai Center for Mathematical Sciences, Fudan University, 200433 Shanghai, China
| | - Wei Lin
- School of Mathematical Sciences and Shanghai Center for Mathematical Sciences, Fudan University, 200433 Shanghai, China
- Research Institute of Intelligent Complex Systems, Fudan University, 200433 Shanghai, China
- Shanghai Artificial Intelligence Laboratory, 200232 Shanghai, China
- State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institute of Brain Science, Fudan University, 200032 Shanghai, China
| | - Bo-Wei Qin
- Research Institute of Intelligent Complex Systems, Fudan University, 200433 Shanghai, China
- Shanghai Artificial Intelligence Laboratory, 200232 Shanghai, China
- State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institute of Brain Science, Fudan University, 200032 Shanghai, China
| |
Collapse
|
3
|
Zhu Q, Li X, Lin W. Leveraging neural differential equations and adaptive delayed feedback to detect unstable periodic orbits based on irregularly sampled time series. CHAOS (WOODBURY, N.Y.) 2023; 33:031101. [PMID: 37003829 DOI: 10.1063/5.0143839] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/26/2023] [Accepted: 02/14/2023] [Indexed: 06/19/2023]
Abstract
Detecting unstable periodic orbits (UPOs) based solely on time series is an essential data-driven problem, attracting a great deal of attention and arousing numerous efforts, in nonlinear sciences. Previous efforts and their developed algorithms, though falling into a category of model-free methodology, dealt with the time series mostly with a regular sampling rate. Here, we develop a data-driven and model-free framework for detecting UPOs in chaotic systems using the irregularly sampled time series. This framework articulates the neural differential equations (NDEs), a recently developed and powerful machine learning technique, with the adaptive delayed feedback (ADF) technique. Since the NDEs own the exceptional capability of accurate reconstruction of chaotic systems based on the observational time series with irregular sampling rates, UPOs detection in this scenario could be enhanced by an integration of the NDEs and the ADF technique. We demonstrate the effectiveness of the articulated framework on representative examples.
Collapse
Affiliation(s)
- Qunxi Zhu
- Research Institute of Intelligent Complex Systems, Fudan University, Shanghai 200433, China
| | - Xin Li
- College of Science, National University of Defense Technology, Changsha, Hunan 410073, China
| | - Wei Lin
- Research Institute of Intelligent Complex Systems, Fudan University, Shanghai 200433, China
| |
Collapse
|
4
|
Zhu Q, Ma H, Lin W. Detecting unstable periodic orbits based only on time series: When adaptive delayed feedback control meets reservoir computing. CHAOS (WOODBURY, N.Y.) 2019; 29:093125. [PMID: 31575157 DOI: 10.1063/1.5120867] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/22/2019] [Accepted: 08/28/2019] [Indexed: 06/10/2023]
Abstract
In this article, we focus on a topic of detecting unstable periodic orbits (UPOs) only based on the time series observed from the nonlinear dynamical system whose explicit model is completely unknown a priori. We articulate a data-driven and model-free method which connects a well-known machine learning technique, the reservoir computing, with a widely-used control strategy of nonlinear dynamical systems, the adaptive delayed feedback control. We demonstrate the advantages and effectiveness of the articulated method through detecting and controlling UPOs in representative examples and also show how those configurations of the reservoir computing in our method influence the accuracy of UPOs detection. Additionally and more interestingly, from the viewpoint of synchronization, we analytically and numerically illustrate the effectiveness of the reservoir computing in dynamical systems learning and prediction.
Collapse
Affiliation(s)
- Qunxi Zhu
- School of Mathematical Sciences, Fudan University, Shanghai 200433, China
| | - Huanfei Ma
- School of Mathematical Sciences, Soochow University, Suzhou 215006, China
| | - Wei Lin
- School of Mathematical Sciences, Fudan University, Shanghai 200433, China
| |
Collapse
|
5
|
Ma H, Leng S, Aihara K, Lin W, Chen L. Randomly distributed embedding making short-term high-dimensional data predictable. Proc Natl Acad Sci U S A 2018; 115:E9994-E10002. [PMID: 30297422 PMCID: PMC6205453 DOI: 10.1073/pnas.1802987115] [Citation(s) in RCA: 33] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/24/2023] Open
Abstract
Future state prediction for nonlinear dynamical systems is a challenging task, particularly when only a few time series samples for high-dimensional variables are available from real-world systems. In this work, we propose a model-free framework, named randomly distributed embedding (RDE), to achieve accurate future state prediction based on short-term high-dimensional data. Specifically, from the observed data of high-dimensional variables, the RDE framework randomly generates a sufficient number of low-dimensional "nondelay embeddings" and maps each of them to a "delay embedding," which is constructed from the data of a to be predicted target variable. Any of these mappings can perform as a low-dimensional weak predictor for future state prediction, and all of such mappings generate a distribution of predicted future states. This distribution actually patches all pieces of association information from various embeddings unbiasedly or biasedly into the whole dynamics of the target variable, which after operated by appropriate estimation strategies, creates a stronger predictor for achieving prediction in a more reliable and robust form. Through applying the RDE framework to data from both representative models and real-world systems, we reveal that a high-dimension feature is no longer an obstacle but a source of information crucial to accurate prediction for short-term data, even under noise deterioration.
Collapse
Affiliation(s)
- Huanfei Ma
- School of Mathematical Sciences, Soochow University, Suzhou 215006, China
| | - Siyang Leng
- Institute of Industrial Science, The University of Tokyo, Tokyo 153-8505, Japan
- School of Mathematical Sciences, Fudan University, Shanghai 200433, China
- Center for Computational Systems Biology, Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai 200433, China
| | - Kazuyuki Aihara
- Institute of Industrial Science, The University of Tokyo, Tokyo 153-8505, Japan;
- International Research Center for Neurointelligence, The University of Tokyo Institutes for Advanced Study, The University of Tokyo, Tokyo 113-0033, Japan
| | - Wei Lin
- School of Mathematical Sciences, Fudan University, Shanghai 200433, China;
- Center for Computational Systems Biology, Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai 200433, China
- Research Institute of Intelligent and Complex Systems, Fudan University, Shanghai 200433, China
- Key Laboratory of Mathematics for Nonlinear Sciences (Fudan University), Ministry of Education, Shanghai 200433, China
- Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University), Ministry of Education, Shanghai 200433, China
| | - Luonan Chen
- Key Laboratory of Systems Biology, Center for Excellence in Molecular Cell Science, Shanghai Institute of Biochemistry and Cell Biology, Chinese Academy of Sciences, Shanghai 200031, China;
- Center for Excellence in Animal Evolution and Genetics, Chinese Academy of Sciences, Kunming 650223, China
- School of Life Science and Technology, ShanghaiTech University, Shanghai 200031, China
- Shanghai Research Center for Brain Science and Brain-Inspired Intelligence, Shanghai 201210, China
| |
Collapse
|
6
|
Olyaei AA, Wu C, Kinsner W. Detecting unstable periodic orbits in chaotic time series using synchronization. Phys Rev E 2017; 96:012207. [PMID: 29347230 DOI: 10.1103/physreve.96.012207] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2017] [Indexed: 11/07/2022]
Abstract
An alternative approach of detecting unstable periodic orbits in chaotic time series is proposed using synchronization techniques. A master-slave synchronization scheme is developed, in which the chaotic system drives a system of harmonic oscillators through a proper coupling condition. The proposed scheme is designed so that the power of the coupling signal exhibits notches that drop to zero once the system approaches an unstable orbit yielding an explicit indication of the presence of a periodic motion. The results shows that the proposed approach is particularly suitable in practical situations, where the time series is short and noisy, or it is obtained from high-dimensional chaotic systems.
Collapse
Affiliation(s)
- Ali Azimi Olyaei
- Department of Mechanical and Manufacturing Engineering, University of Manitoba, Winnipeg, Manitoba, R3T 5V6 Canada
| | - Christine Wu
- Department of Mechanical and Manufacturing Engineering, University of Manitoba, Winnipeg, Manitoba, R3T 5V6 Canada
| | - Witold Kinsner
- Department of Electrical and Computer Engineering, University of Manitoba, Winnipeg, Manitoba, R3T 5V6 Canada
| |
Collapse
|
7
|
Ma H, Ho DWC, Lai YC, Lin W. Detection meeting control: Unstable steady states in high-dimensional nonlinear dynamical systems. PHYSICAL REVIEW. E, STATISTICAL, NONLINEAR, AND SOFT MATTER PHYSICS 2015; 92:042902. [PMID: 26565299 DOI: 10.1103/physreve.92.042902] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/18/2015] [Indexed: 06/05/2023]
Abstract
We articulate an adaptive and reference-free framework based on the principle of random switching to detect and control unstable steady states in high-dimensional nonlinear dynamical systems, without requiring any a priori information about the system or about the target steady state. Starting from an arbitrary initial condition, a proper control signal finds the nearest unstable steady state adaptively and drives the system to it in finite time, regardless of the type of the steady state. We develop a mathematical analysis based on fast-slow manifold separation and Markov chain theory to validate the framework. Numerical demonstration of the control and detection principle using both classic chaotic systems and models of biological and physical significance is provided.
Collapse
Affiliation(s)
- Huanfei Ma
- School of Mathematical Sciences, Soochow University, Suzhou 215006, China
- Center for Computational Systems Biology, Fudan University, Shanghai 200433, China
| | - Daniel W C Ho
- Department of Mathematics, City University of Hong Kong, Hongkong, China
| | - Ying-Cheng Lai
- School of Electrical, Computer, and Energy Engineering, Arizona State University, Tempe, Arizona 85287-5706, USA
| | - Wei Lin
- Center for Computational Systems Biology, Fudan University, Shanghai 200433, China
- School of Mathematical Sciences and SCMS, Fudan University, Shanghai 200433, China
| |
Collapse
|
8
|
Mishra N, Hasse M, Biswal B, Singh HP. Reliability of unstable periodic orbit based control strategies in biological systems. CHAOS (WOODBURY, N.Y.) 2015; 25:043104. [PMID: 25933652 DOI: 10.1063/1.4916899] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
Presence of recurrent and statistically significant unstable periodic orbits (UPOs) in time series obtained from biological systems is now routinely used as evidence for low dimensional chaos. Extracting accurate dynamical information from the detected UPO trajectories is vital for successful control strategies that either aim to stabilize the system near the fixed point or steer the system away from the periodic orbits. A hybrid UPO detection method from return maps that combines topological recurrence criterion, matrix fit algorithm, and stringent criterion for fixed point location gives accurate and statistically significant UPOs even in the presence of significant noise. Geometry of the return map, frequency of UPOs visiting the same trajectory, length of the data set, strength of the noise, and degree of nonstationarity affect the efficacy of the proposed method. Results suggest that establishing determinism from unambiguous UPO detection is often possible in short data sets with significant noise, but derived dynamical properties are rarely accurate and adequate for controlling the dynamics around these UPOs. A repeat chaos control experiment on epileptic hippocampal slices through more stringent control strategy and adaptive UPO tracking is reinterpreted in this context through simulation of similar control experiments on an analogous but stochastic computer model of epileptic brain slices. Reproduction of equivalent results suggests that far more stringent criteria are needed for linking apparent success of control in such experiments with possible determinism in the underlying dynamics.
Collapse
Affiliation(s)
- Nagender Mishra
- Department of Physics & Astrophysics, University of Delhi, Delhi 110007, India
| | - Maria Hasse
- Institut für Höchstleistungsrechnen, Universität Stuttgart, D-70569 Stuttgart, Germany
| | - B Biswal
- Cluster Innovation Center, University of Delhi, Delhi 110007, India
| | - Harinder P Singh
- Department of Physics & Astrophysics, University of Delhi, Delhi 110007, India
| |
Collapse
|
9
|
Ge T, Tian X, Kurths J, Feng J, Lin W. Achieving modulated oscillations by feedback control. PHYSICAL REVIEW. E, STATISTICAL, NONLINEAR, AND SOFT MATTER PHYSICS 2014; 90:022909. [PMID: 25215801 DOI: 10.1103/physreve.90.022909] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/22/2013] [Indexed: 06/03/2023]
Abstract
In this paper, we develop an approach to achieve either frequency or amplitude modulation of an oscillator merely through feedback control. We present and implement a unified theory of our approach for any finite-dimensional continuous dynamical system that exhibits oscillatory behavior. The approach is illustrated not only for the normal forms of dynamical systems but also for representative biological models, such as the isolated and coupled FitzHugh-Nagumo model. We demonstrate the potential usefulness of our approach to uncover the mechanisms of frequency and amplitude modulations experimentally observed in a wide range of real systems.
Collapse
Affiliation(s)
- Tian Ge
- School of Mathematical Sciences, Centre for Computational Systems Biology, Fudan University, Shanghai 200433, China, and Key Laboratory of Mathematics for Nonlinear Sciences (Fudan University), Ministry of Education, China and Department of Computer Science, University of Warwick, Coventry CV4 7AL, United Kingdom
| | - Xiaoying Tian
- School of Mathematical Sciences, Centre for Computational Systems Biology, Fudan University, Shanghai 200433, China, and Key Laboratory of Mathematics for Nonlinear Sciences (Fudan University), Ministry of Education, China
| | - Jürgen Kurths
- Potsdam Institute for Climate Impact Research, D-14412 Potsdam, Germany and Department of Physics, Humboldt University of Berlin, D-12489 Berlin, Germany and Department of Control Theory, Nizhny Novgorod State University, Gagarin Avenue 23, 606950 Nizhny Novgorod, Russia
| | - Jianfeng Feng
- School of Mathematical Sciences, Centre for Computational Systems Biology, Fudan University, Shanghai 200433, China, and Key Laboratory of Mathematics for Nonlinear Sciences (Fudan University), Ministry of Education, China and Department of Computer Science, University of Warwick, Coventry CV4 7AL, United Kingdom
| | - Wei Lin
- School of Mathematical Sciences, Centre for Computational Systems Biology, Fudan University, Shanghai 200433, China, and Key Laboratory of Mathematics for Nonlinear Sciences (Fudan University), Ministry of Education, China
| |
Collapse
|
10
|
Ji X, Wu Y, Sheng W, Lin W. Identification of interactions in fractional-order systems with high dimensions. CHAOS (WOODBURY, N.Y.) 2014; 24:023119. [PMID: 24985433 DOI: 10.1063/1.4876442] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
This article proposes an approach to identify fractional-order systems with sparse interaction structures and high dimensions when observation data are supposed to be experimentally available. This approach includes two steps: first, it is to estimate the value of the fractional order by taking into account the solution properties of fractional-order systems; second, it is to identify the interaction coefficients among the system variables by employing the compressed sensing technique. An error analysis is provided analytically for this approach and a further improved approach is also proposed. Moreover, the applicability of the proposed approach is fully illustrated by two examples: one is to estimate the mutual interactions in a complex dynamical network described by fractional-order systems, and the other is to identify a high fractional-order and homogeneous sequential differential equation, which is frequently used to describe viscoelastic phenomena. All the results demonstrate the feasibility of figuring out the system mechanisms behind the data experimentally observed in physical or biological systems with viscoelastic evolution characters.
Collapse
Affiliation(s)
- Xiaoxi Ji
- School of Mathematical Sciences and Centre for Computational Systems Biology, Fudan University, Shanghai 200433, China
| | - Yu Wu
- School of Mathematical Sciences and Centre for Computational Systems Biology, Fudan University, Shanghai 200433, China
| | - Wenbo Sheng
- School of Mathematical Sciences and Centre for Computational Systems Biology, Fudan University, Shanghai 200433, China
| | - Wei Lin
- School of Mathematical Sciences and Centre for Computational Systems Biology, Fudan University, Shanghai 200433, China
| |
Collapse
|