Reservoir Computing with an Ensemble of Time-Delay Reservoirs

Photonic next-generation reservoir computer based on distributed feedback in optical fiber

Reservoir computing (RC) is a machine learning paradigm that excels at dynamical systems analysis. Photonic RCs, which perform implicit computation through optical interactions, have attracted increasing attention due to their potential for low latency predictions. However, most existing photonic RCs rely on a nonlinear physical cavity to implement system memory, limiting control over the memory structure and requiring long warm-up times to eliminate transients. In this work, we resolve these issues by demonstrating a photonic next-generation reservoir computer (NG-RC) using a fiber optic platform. Our photonic NG-RC eliminates the need for a cavity by generating feature vectors directly from nonlinear combinations of the input data with varying delays. Our approach uses Rayleigh backscattering to produce output feature vectors by an unconventional nonlinearity resulting from coherent, interferometric mixing followed by a quadratic readout. Performing linear optimization on these feature vectors, our photonic NG-RC demonstrates state-of-the-art performance for the observer (cross-prediction) task applied to the Rössler, Lorenz, and Kuramoto-Sivashinsky systems. In contrast to digital NG-RC implementations, we show that it is possible to scale to high-dimensional systems while maintaining low latency and low power consumption.

Experimental Demonstration of Reservoir Computing with Self-Assembled Percolating Networks of Nanoparticles

The complex self-assembled network of neurons and synapses that comprises the biological brain enables natural information processing with remarkable efficiency. Percolating networks of nanoparticles (PNNs) are complex self-assembled nanoscale systems that have been shown to possess many promising brain-like attributes and which are therefore appealing systems for neuromorphic computation. Here experiments are performed that show that PNNs can be utilized as physical reservoirs within a nanoelectronic reservoir computing framework and demonstrate successful computation for several benchmark tasks (chaotic time series prediction, nonlinear transformation, and memory capacity). For each task, relevant literature results are compiled and it is shown that the performance of the PNNs compares favorably to that previously reported from nanoelectronic reservoirs. It is then demonstrated experimentally that PNNs can be used for spoken digit recognition with state-of-the-art accuracy. Finally, a parallel reservoir architecture is emulated, which increases the dimensionality and richness of the reservoir outputs and results in further improvements in performance across all tasks.

Exploiting Signal Propagation Delays to Match Task Memory Requirements in Reservoir Computing

Recurrent neural networks (RNNs) transmit information over time through recurrent connections. In contrast, biological neural networks use many other temporal processing mechanisms. One of these mechanisms is the inter-neuron delays caused by varying axon properties. Recently, this feature was implemented in echo state networks (ESNs), a type of RNN, by assigning spatial locations to neurons and introducing distance-dependent inter-neuron delays. These delays were shown to significantly improve ESN task performance. However, thus far, it is still unclear why distance-based delay networks (DDNs) perform better than ESNs. In this paper, we show that by optimizing inter-node delays, the memory capacity of the network matches the memory requirements of the task. As such, networks concentrate their memory capabilities to the points in the past which contain the most information for the task at hand. Moreover, we show that DDNs have a greater total linear memory capacity, with the same amount of non-linear processing power.

Adjustable short-term memory of SiOx:Ag-based memristor for reservoir computing

Temporal information processing is critical for a wide spectrum of applications, such as finance, biomedicine, and engineering. Reservoir computing (RC) can efficiently process temporal information with low training costs. Various memristors have been explored to demonstrate RC systems leveraging the short-term memory and nonlinear dynamic behaviours. However, the short-term memory is fixed after the device fabrication, limiting the applications to diverse temporal analysis tasks. In this work, we propose the approaches to modulating the short-term memory of Pt/SiOx:Ag/Pt memristor for the performance improvement of the RC systems. By controlling the read voltage, pulse amplitude and pulse width applied to the devices, the obtainable range of the characteristic time reaches three orders of magnitude from microseconds to around milliseconds. Based on the fabricated memristor, the classification of 4-bit pulse streams is demonstrated. Memristor-based RC systems with adjustable short-term memory are constructed for time-series prediction and pattern recognition tasks with different requirements for the characteristic times. The simulation results show that low normalized root mean square error of 0.003 (0.27) in Hénon map (Mackey-Glass time series) and excellent classification accuracy of 99.6% (91.7%) in spoken-digit recognition (MNIST image recognition) are achieved, which outperforms most memristor-based RC systems recently reported. Furthermore, the RC networks with diverse short-term memories are constructed to address more complicated tasks with low prediction errors. This work proves the high controllability of memristor-based RC systems to handle multiple temporal processing tasks.

A Neuromorphic Model With Delay-Based Reservoir for Continuous Ventricular Heartbeat Detection

There is a growing interest in neuromorphic hardware since it offers a more intuitive way to achieve bio-inspired algorithms. This paper presents a neuromorphic model for intelligently processing continuous electrocardiogram (ECG) signal. This model aims to develop a hardware-based signal processing model and avoid employing digitally intensive operations, such as signal segmentation and feature extraction, which are not desired in an analogue neuromorphic system. We apply delay-based reservoir computing as the information processing core, along with a novel training and labelling method. Different from the conventional ECG classification techniques, this computation model is a end-to-end dynamic system that mimics the real-time signal flow in neuromorphic hardware. The input is the raw ECG stream, while the amplitude of the output represents the risk factor of a ventricular ectopic heartbeat. The intrinsic memristive property of the reservoir empowers the system to retain the historical ECG information for high-dimensional mapping. This model was evaluated with the MIT-BIH database under the inter-patient paradigm and yields 81% sensitivity and 98% accuracy. Under this architecture, the minimum size of memory required in the inference process can be as low as 3.1 MegaByte(MB) because the majority of the computation takes place in the analogue domain. Such computational modelling boosts memory efficiency by simplifying the computing procedure and minimizing the required memory for future wearable devices.

Reservoir Computing with Delayed Input for Fast and Easy Optimisation

Reservoir computing is a machine learning method that solves tasks using the response of a dynamical system to a certain input. As the training scheme only involves optimising the weights of the responses of the dynamical system, this method is particularly suited for hardware implementation. Furthermore, the inherent memory of dynamical systems which are suitable for use as reservoirs mean that this method has the potential to perform well on time series prediction tasks, as well as other tasks with time dependence. However, reservoir computing still requires extensive task-dependent parameter optimisation in order to achieve good performance. We demonstrate that by including a time-delayed version of the input for various time series prediction tasks, good performance can be achieved with an unoptimised reservoir. Furthermore, we show that by including the appropriate time-delayed input, one unaltered reservoir can perform well on six different time series prediction tasks at a very low computational expense. Our approach is of particular relevance to hardware implemented reservoirs, as one does not necessarily have access to pertinent optimisation parameters in physical systems but the inclusion of an additional input is generally possible.

Understanding Violin Players’ Skill Level Based on Motion Capture: a Data-Driven Perspective

Learning to play and perform a music instrument is a complex cognitive task, requiring high conscious control and coordination of an impressive number of cognitive and sensorimotor skills. For professional violinists, there exists a physical connection with the instrument allowing the player to continuously manage the sound through sophisticated bowing techniques and fine hand movements. Hence, it is not surprising that great importance in violin training is given to right hand techniques, responsible for most of the sound produced. In this paper, our aim is to understand which motion features can be used to efficiently and effectively distinguish a professional performance from that of a student without exploiting sound-based features. We collected and made freely available a dataset consisting of motion capture recordings of different violinists with different skills performing different exercises covering different pedagogical and technical aspects. We then engineered peculiar features and trained a data-driven classifier to distinguish among two different levels of violinist experience, namely beginners and experts. In accordance with the hierarchy present in the dataset, we study two different scenarios: extrapolation with respect to different exercises and violinists. Furthermore, we study which features are the most predictive ones of the quality of a violinist to corroborate the significance of the results. The results, both in terms of accuracy and insight on the cognitive problem, support the proposal and support the use of the proposed technique as a support tool for students to monitor and enhance their home study and practice.

Delay-based reservoir computing: tackling performance degradation due to system response time

We analyze the degradation of the computational capacity of delay-based reservoir computers due to system response time. We demonstrate that this degradation is reduced when the delay time is greater than the data injection time. Performance improvement is demonstrated on several benchmarking tasks.

Machine learning based on reservoir computing with time-delayed optoelectronic and photonic systems

The concept of reservoir computing emerged from a specific machine learning paradigm characterized by a three-layered architecture (input, reservoir, and output), where only the output layer is trained and optimized for a particular task. In recent years, this approach has been successfully implemented using various hardware platforms based on optoelectronic and photonic systems with time-delayed feedback. In this review, we provide a survey of the latest advances in this field, with some perspectives related to the relationship between reservoir computing, nonlinear dynamics, and network theory.

Four-channels reservoir computing based on polarization dynamics in mutually coupled VCSELs system

A novel Four-channels reservoir computing (RC) based on polarization dynamics in mutually coupled vertical cavity surface emitting lasers (MDC-VCSELs) is proposed and demonstrated numerically. Here, the four channels are realized in two orthogonal polarization modes (x-polarization and y-polarization modes) of two VCSELs for the first time. A chaotic time series prediction task is employed to quantitatively evaluated the prediction performance of the proposed system. It is found that the Four-channels RC based on MDC-VCSELs can produce comparable prediction performance with One-channel RC, and it is possible to increase four times information processing rate by using the Four-channels RC. Besides, the effects of injection current, external injection strength, frequency detuning, coupling strength, as well as internal parameters on the prediction performance of the Four-channels RC based on MDC-VCSELs are carefully examined. Moreover, the influences of sampled period of input signal and the number of virtual nodes are also considered. The proposed Four-channels RC based on MDC-VCSELs is valuable for further enhancing the information processing rate of RC-based neuromorphic photonic systems.

A substrate-independent framework to characterize reservoir computers

The reservoir computing (RC) framework states that any nonlinear, input-driven dynamical system (the reservoir) exhibiting properties such as a fading memory and input separability can be trained to perform computational tasks. This broad inclusion of systems has led to many new physical substrates for RC. Properties essential for reservoirs to compute are tuned through reconfiguration of the substrate, such as change in virtual topology or physical morphology. As a result, each substrate possesses a unique 'quality'-obtained through reconfiguration-to realize different reservoirs for different tasks. Here we describe an experimental framework to characterize the quality of potentially any substrate for RC. Our framework reveals that a definition of quality is not only useful to compare substrates, but can help map the non-trivial relationship between properties and task performance. In the wider context, the framework offers a greater understanding as to what makes a dynamical system compute, helping improve the design of future substrates for RC.

Recent advances in physical reservoir computing: A review

Reservoir computing is a computational framework suited for temporal/sequential data processing. It is derived from several recurrent neural network models, including echo state networks and liquid state machines. A reservoir computing system consists of a reservoir for mapping inputs into a high-dimensional space and a readout for pattern analysis from the high-dimensional states in the reservoir. The reservoir is fixed and only the readout is trained with a simple method such as linear regression and classification. Thus, the major advantage of reservoir computing compared to other recurrent neural networks is fast learning, resulting in low training cost. Another advantage is that the reservoir without adaptive updating is amenable to hardware implementation using a variety of physical systems, substrates, and devices. In fact, such physical reservoir computing has attracted increasing attention in diverse fields of research. The purpose of this review is to provide an overview of recent advances in physical reservoir computing by classifying them according to the type of the reservoir. We discuss the current issues and perspectives related to physical reservoir computing, in order to further expand its practical applications and develop next-generation machine learning systems.