1
|
Maslennikov O, Perc M, Nekorkin V. Topological features of spike trains in recurrent spiking neural networks that are trained to generate spatiotemporal patterns. Front Comput Neurosci 2024; 18:1363514. [PMID: 38463243 PMCID: PMC10920356 DOI: 10.3389/fncom.2024.1363514] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2023] [Accepted: 02/06/2024] [Indexed: 03/12/2024] Open
Abstract
In this study, we focus on training recurrent spiking neural networks to generate spatiotemporal patterns in the form of closed two-dimensional trajectories. Spike trains in the trained networks are examined in terms of their dissimilarity using the Victor-Purpura distance. We apply algebraic topology methods to the matrices obtained by rank-ordering the entries of the distance matrices, specifically calculating the persistence barcodes and Betti curves. By comparing the features of different types of output patterns, we uncover the complex relations between low-dimensional target signals and the underlying multidimensional spike trains.
Collapse
Affiliation(s)
- Oleg Maslennikov
- Federal Research Center A.V. Gaponov-Grekhov Institute of Applied Physics of the Russian Academy of Sciences, Nizhny Novgorod, Russia
| | - Matjaž Perc
- Faculty of Natural Sciences and Mathematics, University of Maribor, Maribor, Slovenia
- Department of Medical Research, China Medical University Hospital, China Medical University, Taichung City, Taiwan
- Complexity Science Hub Vienna, Vienna, Austria
- Department of Physics, Kyung Hee University, Seoul, Republic of Korea
| | - Vladimir Nekorkin
- Federal Research Center A.V. Gaponov-Grekhov Institute of Applied Physics of the Russian Academy of Sciences, Nizhny Novgorod, Russia
| |
Collapse
|
2
|
Mahata A, Padhi R, Apte A. Variability of echo state network prediction horizon for partially observed dynamical systems. Phys Rev E 2023; 108:064209. [PMID: 38243433 DOI: 10.1103/physreve.108.064209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 11/10/2023] [Indexed: 01/21/2024]
Abstract
Study of dynamical systems using partial state observation is an important problem due to its applicability to many real-world systems. We address the problem by studying an echo state network (ESN) framework with partial state input with partial or full state output. Application to the Lorenz system and Chua's oscillator (both numerically simulated and experimental systems) demonstrate the effectiveness of our method. We show that the ESN, as an autonomous dynamical system, is capable of making short-term predictions up to a few Lyapunov times. However, the prediction horizon has high variability depending on the initial condition-an aspect that we explore in detail using the distribution of the prediction horizon. Further, using a variety of statistical metrics to compare the long-term dynamics of the ESN predictions with numerically simulated or experimental dynamics and observed similar results, we show that the ESN can effectively learn the system's dynamics even when trained with noisy numerical or experimental data sets. Thus, we demonstrate the potential of ESNs to serve as cheap surrogate models for simulating the dynamics of systems where complete observations are unavailable.
Collapse
Affiliation(s)
- Ajit Mahata
- Department of Data Science, Indian Institute of Science Education and Research, IISER Pune 411008, India
| | - Reetish Padhi
- Department of Data Science, Indian Institute of Science Education and Research, IISER Pune 411008, India
| | - Amit Apte
- Department of Data Science, Indian Institute of Science Education and Research, IISER Pune 411008, India
- International Centre for Theoretical Sciences (ICTS-TIFR), Bengaluru 560089, India
| |
Collapse
|
3
|
Maslennikov OV, Gao C, Nekorkin VI. Internal dynamics of recurrent neural networks trained to generate complex spatiotemporal patterns. CHAOS (WOODBURY, N.Y.) 2023; 33:093125. [PMID: 37722673 DOI: 10.1063/5.0166359] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Accepted: 08/28/2023] [Indexed: 09/20/2023]
Abstract
How complex patterns generated by neural systems are represented in individual neuronal activity is an essential problem in computational neuroscience as well as machine learning communities. Here, based on recurrent neural networks in the form of feedback reservoir computers, we show microscopic features resulting in generating spatiotemporal patterns including multicluster and chimera states. We show the effect of individual neural trajectories as well as whole-network activity distributions on exhibiting particular regimes. In addition, we address the question how trained output weights contribute to the autonomous multidimensional dynamics.
Collapse
Affiliation(s)
- Oleg V Maslennikov
- Federal Research Center A.V. Gaponov-Grekhov Institute of Applied Physics of the Russian Academy of Sciences, Nizhny Novgorod, Russia
| | - Chao Gao
- School of Artificial Intelligence, Optics and Electronics, Northwestern Polytechnical University, Xian, China
| | - Vladimir I Nekorkin
- Federal Research Center A.V. Gaponov-Grekhov Institute of Applied Physics of the Russian Academy of Sciences, Nizhny Novgorod, Russia
| |
Collapse
|
4
|
Pugavko MM, Maslennikov OV, Nekorkin VI. Multitask computation through dynamics in recurrent spiking neural networks. Sci Rep 2023; 13:3997. [PMID: 36899052 PMCID: PMC10006454 DOI: 10.1038/s41598-023-31110-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Accepted: 03/06/2023] [Indexed: 03/12/2023] Open
Abstract
In this work, inspired by cognitive neuroscience experiments, we propose recurrent spiking neural networks trained to perform multiple target tasks. These models are designed by considering neurocognitive activity as computational processes through dynamics. Trained by input-output examples, these spiking neural networks are reverse engineered to find the dynamic mechanisms that are fundamental to their performance. We show that considering multitasking and spiking within one system provides insightful ideas on the principles of neural computation.
Collapse
Affiliation(s)
- Mechislav M Pugavko
- Institute of Applied Physics of the Russian Academy of Sciences, Nizhny Novgorod, 603950, Russia
| | - Oleg V Maslennikov
- Institute of Applied Physics of the Russian Academy of Sciences, Nizhny Novgorod, 603950, Russia.
| | - Vladimir I Nekorkin
- Institute of Applied Physics of the Russian Academy of Sciences, Nizhny Novgorod, 603950, Russia
| |
Collapse
|
5
|
Roy M, Senapati A, Poria S, Mishra A, Hens C. Role of assortativity in predicting burst synchronization using echo state network. Phys Rev E 2022; 105:064205. [PMID: 35854538 DOI: 10.1103/physreve.105.064205] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2021] [Accepted: 05/11/2022] [Indexed: 06/15/2023]
Abstract
In this study, we use a reservoir computing based echo state network (ESN) to predict the collective burst synchronization of neurons. Specifically, we investigate the ability of ESN in predicting the burst synchronization of an ensemble of Rulkov neurons placed on a scale-free network. We have shown that a limited number of nodal dynamics used as input in the machine can capture the real trend of burst synchronization in this network. Further, we investigate the proper selection of nodal inputs of degree-degree (positive and negative) correlated networks. We show that for a disassortative network, selection of different input nodes based on degree has no significant role in the machine's prediction. However, in the case of assortative network, training the machine with the information (i.e., time series) of low degree nodes gives better results in predicting the burst synchronization. The results are found to be consistent with the investigation carried out with a continuous time Hindmarsh-Rose neuron model. Furthermore, the role of hyperparameters like spectral radius and leaking parameter of ESN on the prediction process has been examined. Finally, we explain the underlying mechanism responsible for observing these differences in the prediction in a degree correlated network.
Collapse
Affiliation(s)
- Mousumi Roy
- Department of Applied Mathematics, University of Calcutta, 92, A.P.C. Road, Kolkata 700009, India
| | - Abhishek Senapati
- Center for Advanced Systems Understanding (CASUS), 02826 Görlitz, Germany
| | - Swarup Poria
- Department of Applied Mathematics, University of Calcutta, 92, A.P.C. Road, Kolkata 700009, India
| | - Arindam Mishra
- Division of Dynamics, Lodz University of Technology, Stefanowskiego 1/15, 90924 Lodz, Poland
| | - Chittaranjan Hens
- Physics and Applied Mathematics Unit, Indian Statistical Institute, Kolkata 700108, India
| |
Collapse
|
6
|
Ray A, Chakraborty T, Ghosh D. Optimized ensemble deep learning framework for scalable forecasting of dynamics containing extreme events. CHAOS (WOODBURY, N.Y.) 2021; 31:111105. [PMID: 34881612 DOI: 10.1063/5.0074213] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Accepted: 11/02/2021] [Indexed: 06/13/2023]
Abstract
The remarkable flexibility and adaptability of both deep learning models and ensemble methods have led to the proliferation for their application in understanding many physical phenomena. Traditionally, these two techniques have largely been treated as independent methodologies in practical applications. This study develops an optimized ensemble deep learning framework wherein these two machine learning techniques are jointly used to achieve synergistic improvements in model accuracy, stability, scalability, and reproducibility, prompting a new wave of applications in the forecasting of dynamics. Unpredictability is considered one of the key features of chaotic dynamics; therefore, forecasting such dynamics of nonlinear systems is a relevant issue in the scientific community. It becomes more challenging when the prediction of extreme events is the focus issue for us. In this circumstance, the proposed optimized ensemble deep learning (OEDL) model based on a best convex combination of feed-forward neural networks, reservoir computing, and long short-term memory can play a key role in advancing predictions of dynamics consisting of extreme events. The combined framework can generate the best out-of-sample performance than the individual deep learners and standard ensemble framework for both numerically simulated and real-world data sets. We exhibit the outstanding performance of the OEDL framework for forecasting extreme events generated from a Liénard-type system, prediction of COVID-19 cases in Brazil, dengue cases in San Juan, and sea surface temperature in the Niño 3.4 region.
Collapse
Affiliation(s)
- Arnob Ray
- Physics and Applied Mathematics Unit, Indian Statistical Institute, Kolkata 700108, India
| | - Tanujit Chakraborty
- Department of Science and Engineering, Sorbonne University Abu Dhabi, Abu Dhabi, UAE
| | - Dibakar Ghosh
- Physics and Applied Mathematics Unit, Indian Statistical Institute, Kolkata 700108, India
| |
Collapse
|
7
|
Tang Y, Kurths J, Lin W, Ott E, Kocarev L. Introduction to Focus Issue: When machine learning meets complex systems: Networks, chaos, and nonlinear dynamics. CHAOS (WOODBURY, N.Y.) 2020; 30:063151. [PMID: 32611112 DOI: 10.1063/5.0016505] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/04/2020] [Accepted: 06/05/2020] [Indexed: 06/11/2023]
Affiliation(s)
- Yang Tang
- Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Jürgen Kurths
- Potsdam Institute for Climate Impact Research, Potsdam 14473, Germany
| | - Wei Lin
- Center for Computational Systems Biology of ISTBI and Research Institute of Intelligent Complex Systems, Fudan University, Shanghai 200433, China
| | - Edward Ott
- Department of Physics, University of Maryland, College Park, Maryland 20742, USA
| | - Ljupco Kocarev
- Macedonian Academy of Sciences and Arts, 1000 Skopje, Macedonia
| |
Collapse
|