Effects of connectivity structure of complex echo state network on its prediction performance for nonlinear time series

Exploration of a brain-inspired photon reservoir computing network based on quantum-dot spin-VCSELs

Based on small-world network theory, we have developed a brain-inspired photonic reservoir computing (RC) network system utilizing quantum dot spin-vertical-cavity surface-emitting lasers (QD spin-VCSELs) and formulated a comprehensive theoretical model for it. This innovative network system comprises input layers, a reservoir network layer, and output layers. The reservoir network layer features four distinct reservoir modules that are asymmetrically coupled. Each module is represented by a QD spin-VCSEL, characterized by optical feedback and optical injection. Within these modules, four chaotic polarization components, emitted from both the ground and excited states of the QD Spin-VCSEL, form four distinct reservoirs through a process of asymmetric coupling. Moreover, these components, when emitted by the ground and excited states of a driving QD spin-VCSEL within a specific parameter space, act as targets for prediction. Delving further, we investigated the correlation between various system parameters, such as the sampling period, the interval between virtual nodes, the strengths of optical injection and feedback, frequency detuning, and the predictive accuracy of each module's four photonic RCs concerning the four designated predictive targets. We also examined how these parameters influence the memory storage capabilities of the four photonics RCs within each module. Our findings indicate that when a module receives coupling injections from more than two other modules, and an RC within this module is also subject to coupling injections from over two other RCs, the system displays reduced predictive errors and enhanced memory storage capacities when the system parameters are fixed. Namely, the superior performance of the reservoir module in predictive accuracy and memory capacities follows from its complex interaction with multiple light injections and coupling injections, with its three various PCs benefiting from three, two, and one coupling injections respectively. Conversely, variations in optical injection and feedback strength, as well as frequency detuning, introduce only marginal fluctuations in the predictive errors across the four photonics RCs within each module and exert minimal impact on the memory storage capacity of individual photonics RCs within the modules. Our investigated results contribute to the development of photonic reservoir computing towards fast response biological neural networks.

A double-cycle echo state network topology for time series prediction

Echo state network (ESN) has gained wide acceptance in the field of time series prediction, relying on sufficiently complex reservoir connections to remember the historical features of the data and using these features to obtain the outputs by a simple linear readout. However, the randomness of its input and reservoir connections pose negative impacts on the prediction performance and performance stability of the models, the complexity of reservoir connections brings high time consumption during network computing, and the presence of randomness and complexity makes the hardware implementation of the ESN difficult. In response, we propose a double-cycle ESN (DCESN) based on the Li-ESN model, which has fixed weights to improve prediction performance and performance stability and simpler reservoir connections compared to the classical ESN to reduce the time consumption. The existence of both greatly reduces the difficulty of hardware implementation of the ESN and provides many conveniences for the future application of the ESN. Experimental results on many widely used time series datasets show that the DCESN has comparable or even better prediction performance than the ESN and good robustness against noise and parameter fluctuations.

Echo State Network-Based Content Prediction for Mobile Edge Caching Networks

With the rapid development of internet communication and the wide application of intelligent terminal, moving the cache to the edge of the network is an effective solution to shorten the delay of users accessing content. However, the existing cache work lacks the comprehensive consideration of users and content, resulting in low cache hit ratio and low accuracy of the whole system. In this paper, the authors propose a collaborative caching model that considers both user request content and content prediction, so as to improve the caching performance of the whole network. Firstly, the model uses the clustering algorithm based on Akike information criterion to cluster users. Then, combined with the clustering results, echo state network is used as the machine learning framework to predict the content. Finally, the cache contents are selected according to the prediction results and cached in the cache unit of the small base station. Simulation results show that compared with the existing cache algorithms, the proposed method has obvious improvement in cache hit ratio, accuracy, and recall rate.

On explaining the surprising success of reservoir computing forecaster of chaos? The universal machine learning dynamical system with contrast to VAR and DMD

Machine learning has become a widely popular and successful paradigm, especially in data-driven science and engineering. A major application problem is data-driven forecasting of future states from a complex dynamical system. Artificial neural networks have evolved as a clear leader among many machine learning approaches, and recurrent neural networks are considered to be particularly well suited for forecasting dynamical systems. In this setting, the echo-state networks or reservoir computers (RCs) have emerged for their simplicity and computational complexity advantages. Instead of a fully trained network, an RC trains only readout weights by a simple, efficient least squares method. What is perhaps quite surprising is that nonetheless, an RC succeeds in making high quality forecasts, competitively with more intensively trained methods, even if not the leader. There remains an unanswered question as to why and how an RC works at all despite randomly selected weights. To this end, this work analyzes a further simplified RC, where the internal activation function is an identity function. Our simplification is not presented for the sake of tuning or improving an RC, but rather for the sake of analysis of what we take to be the surprise being not that it does not work better, but that such random methods work at all. We explicitly connect the RC with linear activation and linear readout to well developed time-series literature on vector autoregressive (VAR) averages that includes theorems on representability through the Wold theorem, which already performs reasonably for short-term forecasts. In the case of a linear activation and now popular quadratic readout RC, we explicitly connect to a nonlinear VAR, which performs quite well. Furthermore, we associate this paradigm to the now widely popular dynamic mode decomposition; thus, these three are in a sense different faces of the same thing. We illustrate our observations in terms of popular benchmark examples including Mackey-Glass differential delay equations and the Lorenz63 system.

Evolving Local Plasticity Rules for Synergistic Learning in Echo State Networks

Existing synaptic plasticity rules for optimizing the connections between neurons within the reservoir of echo state networks (ESNs) remain to be global in that the same type of plasticity rule with the same parameters is applied to all neurons. However, this is biologically implausible and practically inflexible for learning the structures in the input signals, thereby limiting the learning performance of ESNs. In this paper, we propose to use local plasticity rules that allow different neurons to use different types of plasticity rules and different parameters, which are achieved by optimizing the parameters of the local plasticity rules using the evolution strategy (ES) with covariance matrix adaptation (CMA-ES). We show that evolving neural plasticity will result in a synergistic learning of different plasticity rules, which plays an important role in improving the learning performance. Meanwhile, we show that the local plasticity rules can effectively alleviate synaptic interferences in learning the structure in sensory inputs. The proposed local plasticity rules are compared with a number of the state-of-the-art ESN models and the canonical ESN using a global plasticity rule on a set of widely used prediction and classification benchmark problems to demonstrate its competitive learning performance.

A priori data-driven multi-clustered reservoir generation algorithm for echo state network

Echo state networks (ESNs) with multi-clustered reservoir topology perform better in reservoir computing and robustness than those with random reservoir topology. However, these ESNs have a complex reservoir topology, which leads to difficulties in reservoir generation. This study focuses on the reservoir generation problem when ESN is used in environments with sufficient priori data available. Accordingly, a priori data-driven multi-cluster reservoir generation algorithm is proposed. The priori data in the proposed algorithm are used to evaluate reservoirs by calculating the precision and standard deviation of ESNs. The reservoirs are produced using the clustering method; only the reservoir with a better evaluation performance takes the place of a previous one. The final reservoir is obtained when its evaluation score reaches the preset requirement. The prediction experiment results obtained using the Mackey-Glass chaotic time series show that the proposed reservoir generation algorithm provides ESNs with extra prediction precision and increases the structure complexity of the network. Further experiments also reveal the appropriate values of the number of clusters and time window size to obtain optimal performance. The information entropy of the reservoir reaches the maximum when ESN gains the greatest precision.