Delayed dynamical systems: networks, chimeras and reservoir computing

Photonic next-generation reservoir computer based on distributed feedback in optical fiber

Reservoir computing (RC) is a machine learning paradigm that excels at dynamical systems analysis. Photonic RCs, which perform implicit computation through optical interactions, have attracted increasing attention due to their potential for low latency predictions. However, most existing photonic RCs rely on a nonlinear physical cavity to implement system memory, limiting control over the memory structure and requiring long warm-up times to eliminate transients. In this work, we resolve these issues by demonstrating a photonic next-generation reservoir computer (NG-RC) using a fiber optic platform. Our photonic NG-RC eliminates the need for a cavity by generating feature vectors directly from nonlinear combinations of the input data with varying delays. Our approach uses Rayleigh backscattering to produce output feature vectors by an unconventional nonlinearity resulting from coherent, interferometric mixing followed by a quadratic readout. Performing linear optimization on these feature vectors, our photonic NG-RC demonstrates state-of-the-art performance for the observer (cross-prediction) task applied to the Rössler, Lorenz, and Kuramoto-Sivashinsky systems. In contrast to digital NG-RC implementations, we show that it is possible to scale to high-dimensional systems while maintaining low latency and low power consumption.

Master Memory Function for Delay-Based Reservoir Computers With Single-Variable Dynamics

We show that many delay-based reservoir computers considered in the literature can be characterized by a universal master memory function (MMF). Once computed for two independent parameters, this function provides linear memory capacity for any delay-based single-variable reservoir with small inputs. Moreover, we propose an analytical description of the MMF that enables its efficient and fast computation. Our approach can be applied not only to single-variable delay-based reservoirs governed by known dynamical rules, such as the Mackey-Glass or Stuart-Landau-like systems, but also to reservoirs whose dynamical model is not available.

Reservoir computing with noise

This paper investigates in detail the effects of measurement noise on the performance of reservoir computing. We focus on an application in which reservoir computers are used to learn the relationship between different state variables of a chaotic system. We recognize that noise can affect the training and testing phases differently. We find that the best performance of the reservoir is achieved when the strength of the noise that affects the input signal in the training phase equals the strength of the noise that affects the input signal in the testing phase. For all the cases we examined, we found that a good remedy to noise is to low-pass filter the input and the training/testing signals; this typically preserves the performance of the reservoir, while reducing the undesired effects of noise.

Deriving task specific performance from the information processing capacity of a reservoir computer

In the reservoir computing literature, the information processing capacity is frequently used to characterize the computing capabilities of a reservoir. However, it remains unclear how the information processing capacity connects to the performance on specific tasks. We demonstrate on a set of standard benchmark tasks that the total information processing capacity correlates poorly with task specific performance. Further, we derive an expression for the normalized mean square error of a task as a weighted function of the individual information processing capacities. Mathematically, the derivation requires the task to have the same input distribution as used to calculate the information processing capacities. We test our method on a range of tasks that violate this requirement and find good qualitative agreement between the predicted and the actual errors as long as the task input sequences do not have long autocorrelation times. Our method offers deeper insight into the principles governing reservoir computing performance. It also increases the utility of the evaluation of information processing capacities, which are typically defined on i.i.d. input, even if specific tasks deliver inputs stemming from different distributions. Moreover, it offers the possibility of reducing the experimental cost of optimizing physical reservoirs, such as those implemented in photonic systems.

Laminar chaos in systems with quasiperiodic delay

A type of chaos called laminar chaos was found in singularly perturbed dynamical systems with periodic time-varying delay [Phys. Rev. Lett. 120, 084102 (2018)]0031-900710.1103/PhysRevLett.120.084102. It is characterized by nearly constant laminar phases, which are periodically interrupted by irregular bursts, where the intensity levels of the laminar phases vary chaotically from phase to phase. In this paper, we demonstrate that laminar chaos can also be observed in systems with quasiperiodic delay, where we generalize the concept of conservative and dissipative delays to such systems. It turns out that the durations of the laminar phases vary quasiperiodically and follow the dynamics of a torus map in contrast to the periodic variation observed for periodic delay. Theoretical and numerical results indicate that introducing a quasiperiodic delay modulation into a time-delay system can lead to a giant reduction of the dimension of the chaotic attractors. By varying the mean delay and keeping other parameters fixed, we found that the Kaplan-Yorke dimension is modulated quasiperiodically over several orders of magnitudes, where the dynamics switches quasiperiodically between different types of high- and low-dimensional types of chaos.

Pseudolaminar chaos from on-off intermittency

In finite-dimensional, chaotic, Lorenz-like wave-particle dynamical systems one can find diffusive trajectories, which share their appearance with that of laminar chaotic diffusion [Phys. Rev. Lett. 128, 074101 (2022)0031-900710.1103/PhysRevLett.128.074101] known from delay systems with lag-time modulation. Applying, however, to such systems a test for laminar chaos, as proposed in [Phys. Rev. E 101, 032213 (2020)2470-004510.1103/PhysRevE.101.032213], these signals fail such a test, thus leading to the notion of pseudolaminar chaos. The latter can be interpreted as integrated periodically driven on-off intermittency. We demonstrate that, on a signal level, true laminar and pseudolaminar chaos are hardly distinguishable in systems with and without dynamical noise. However, very pronounced differences become apparent when correlations of signals and increments are considered. We compare and contrast these properties of pseudolaminar chaos with true laminar chaos.

Time shifts to reduce the size of reservoir computers

A reservoir computer is a type of dynamical system arranged to do computation. Typically, a reservoir computer is constructed by connecting a large number of nonlinear nodes in a network that includes recurrent connections. In order to achieve accurate results, the reservoir usually contains hundreds to thousands of nodes. This high dimensionality makes it difficult to analyze the reservoir computer using tools from the dynamical systems theory. Additionally, the need to create and connect large numbers of nonlinear nodes makes it difficult to design and build analog reservoir computers that can be faster and consume less power than digital reservoir computers. We demonstrate here that a reservoir computer may be divided into two parts: a small set of nonlinear nodes (the reservoir) and a separate set of time-shifted reservoir output signals. The time-shifted output signals serve to increase the rank and memory of the reservoir computer, and the set of nonlinear nodes may create an embedding of the input dynamical system. We use this time-shifting technique to obtain excellent performance from an opto-electronic delay-based reservoir computer with only a small number of virtual nodes. Because only a few nonlinear nodes are required, construction of a reservoir computer becomes much easier, and delay-based reservoir computers can operate at much higher speeds.

Chimera states for directed networks

We demonstrate that chimera behavior can be observed in ensembles of phase oscillators with unidirectional coupling. For a small network consisting of only three identical oscillators (cyclic triple), tiny chimera islands arise in the parameter space. They are surrounded by developed chaotic switching behavior caused by a collision of rotating waves propagating in opposite directions. For larger networks, as we show for a hundred oscillators (cyclic century), the islands merge into a single chimera continent, which incorporates the world of chimeras of different configurations. The phenomenon inherits from networks with intermediate ranges of the unidirectional coupling and it diminishes as the coupling range decreases.

Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops

Deep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron's dynamics. By adjusting the feedback-modulation within the loops, we adapt the network's connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

Chimera complexity

We show an amazing complexity of the chimeras in small networks of coupled phase oscillators with inertia. The network behavior is characterized by heteroclinic switching between multiple saddle chimera states and riddling basins of attractions, causing an extreme sensitivity to initial conditions and parameters. Additional uncertainty is induced by the presumable coexistence of stable phase-locked states or other stable chimeras as the switching trajectories can eventually tend to them. The system dynamics becomes hardly predictable, while its complexity represents a challenge in the network sciences.

Smallest Chimeras Under Repulsive Interactions

We present an exemplary system of three identical oscillators in a ring interacting repulsively to show up chimera patterns. The dynamics of individual oscillators is governed by the superconducting Josephson junction. Surprisingly, the repulsive interactions can only establish a symmetry of complete synchrony in the ring, which is broken with increasing repulsive interactions when the junctions pass through serials of asynchronous states (periodic and chaotic) but finally emerge into chimera states. The chimera pattern first appears in chaotic rotational motion of the three junctions when two junctions evolve coherently, while the third junction is incoherent. For larger repulsive coupling, the junctions evolve into another chimera pattern in a periodic state when two junctions remain coherent in rotational motion and one junction transits to incoherent librational motion. This chimera pattern is sensitive to initial conditions in the sense that the chimera state flips to another pattern when two junctions switch to coherent librational motion and the third junction remains in rotational motion, but incoherent. The chimera patterns are detected by using partial and global error functions of the junctions, while the librational and rotational motions are identified by a libration index. All the collective states, complete synchrony, desynchronization, and two chimera patterns are delineated in a parameter plane of the ring of junctions, where the boundaries of complete synchrony are demarcated by using the master stability function.

Colocalized Sensing and Intelligent Computing in Micro-Sensors

This work presents an approach to delay-based reservoir computing (RC) at the sensor level without input modulation. It employs a time-multiplexed bias to maintain transience while utilizing either an electrical signal or an environmental signal (such as acceleration) as an unmodulated input signal. The proposed approach enables RC carried out by sufficiently nonlinear sensory elements, as we demonstrate using a single electrostatically actuated microelectromechanical system (MEMS) device. The MEMS sensor can perform colocalized sensing and computing with fewer electronics than traditional RC elements at the RC input (such as analog-to-digital and digital-to-analog converters). The performance of the MEMS RC is evaluated experimentally using a simple classification task, in which the MEMS device differentiates between the profiles of two signal waveforms. The signal waveforms are chosen to be either electrical waveforms or acceleration waveforms. The classification accuracy of the presented MEMS RC scheme is found to be over 99%. Furthermore, the scheme is found to enable flexible virtual node probing rates, allowing for up to 4× slower probing rates, which relaxes the requirements on the system for reservoir signal sampling. Finally, our experiments show a noise-resistance capability for our MEMS RC scheme.

First-order synchronization transition in a large population of strongly coupled relaxation oscillators

Onset and loss of synchronization in coupled oscillators are of fundamental importance in understanding emergent behavior in natural and man-made systems, which range from neural networks to power grids. We report on experiments with hundreds of strongly coupled photochemical relaxation oscillators that exhibit a discontinuous synchronization transition with hysteresis, as opposed to the paradigmatic continuous transition expected from the widely used weak coupling theory. The resulting first-order transition is robust with respect to changes in network connectivity and natural frequency distribution. This allows us to identify the relaxation character of the oscillators as the essential parameter that determines the nature of the synchronization transition. We further support this hypothesis by revealing the mechanism of the transition, which cannot be accounted for by standard phase reduction techniques.

Identification of chimera using machine learning

Chimera state refers to the coexistence of coherent and non-coherent phases in identically coupled dynamical units found in various complex dynamical systems. Identification of chimera, on one hand, is essential due to its applicability in various areas including neuroscience and, on the other hand, is challenging due to its widely varied appearance in different systems and the peculiar nature of its profile. Therefore, a simple yet universal method for its identification remains an open problem. Here, we present a very distinctive approach using machine learning techniques to characterize different dynamical phases and identify the chimera state from given spatial profiles generated using various different models. The experimental results show that the performance of the classification algorithms varies for different dynamical models. The machine learning algorithms, namely, random forest, oblique random forest based on Tikhonov, axis-parallel split, and null space regularization achieved more than 96% accuracy for the Kuramoto model. For the logistic maps, random forest and Tikhonov regularization based oblique random forest showed more than 90% accuracy, and for the Hénon map model, random forest, null space, and axis-parallel split regularization based oblique random forest achieved more than 80% accuracy. The oblique random forest with null space regularization achieved consistent performance (more than 83% accuracy) across different dynamical models while the auto-encoder based random vector functional link neural network showed relatively lower performance. This work provides a direction for employing machine learning techniques to identify dynamical patterns arising in coupled non-linear units on large-scale and for characterizing complex spatiotemporal patterns in real-world systems for various applications.

Oscillating synchronization in delayed oscillators with time-varying time delay coupling: Experimental observation

The time-varying time-delayed (TVTD) systems attract the attention of research communities due to their rich complex dynamics and wide application potentiality. Particularly, coupled TVTD systems show several intriguing behaviors that cannot be observed in systems with a constant delay or no delay. In this context, a new synchronization scenario, namely, oscillating synchronization, was reported by Senthilkumar and Lakshmanan [Chaos 17, 013112 (2007)], which is exclusive to the time-varying time delay systems only. However, like most of the dynamical behavior of TVTD systems, its existence has not been established in an experiment. In this paper, we report the first experimental observation of oscillating synchronization in coupled nonlinear time-delayed oscillators induced by a time-varying time delay in the coupling path. We implement a simple yet effective electronic circuit to realize the time-varying time delay in an experiment. We show that depending upon the instantaneous variation of the time delay, the system shows a synchronization scenario oscillating among lag, complete, and anticipatory synchronization. This study may open up the feasibility of applying oscillating synchronization in engineering systems.

A Multiple-Input Multiple-Output Reservoir Computing System Subject to Optoelectronic Feedbacks and Mutual Coupling

In this paper, a multiple-input multiple-output reservoir computing (RC) system is proposed, which is composed of multiple nonlinear nodes (Mach–Zehnder modulators) and multiple mutual-coupling loops of optoelectronic delay lines. Each input signal is added into every mutual-coupling loop to implement the simultaneous recognition of multiple route signals, which results in the signal processing speed improving and the number of routes increasing. As an example, the four-route input and four-route output RC is simultaneously realized by numerical simulations. The results show that this type of RC system can successfully recognize the four-route optical packet headers with 3-bit, 8-bit, 16-bit, and 32-bit, and four-route independent digital speeches. When the white noise is added to the signals such that the signal-to-noise ratio (SNR) of the optical packet headers and the digital speeches are 35 dB and 20 dB respectively, the normalized root mean square errors (NRMSEs) of the signal recognition are all close to 0.1. The word error rates (WERs) of the optical packet header recognition are 0%. The WER of the digital speech recognition is 1.6%. The eight-route input and eight-route output RC is also numerically simulated. The recognition of the eight-route 3-bit optical packet headers is implemented. The parallel processing of multiple-route signals and the high recognition accuracy are implemented by this proposed system.

Machine learning based on reservoir computing with time-delayed optoelectronic and photonic systems

The concept of reservoir computing emerged from a specific machine learning paradigm characterized by a three-layered architecture (input, reservoir, and output), where only the output layer is trained and optimized for a particular task. In recent years, this approach has been successfully implemented using various hardware platforms based on optoelectronic and photonic systems with time-delayed feedback. In this review, we provide a survey of the latest advances in this field, with some perspectives related to the relationship between reservoir computing, nonlinear dynamics, and network theory.

Laminar Chaos in Experiments: Nonlinear Systems with Time-Varying Delays and Noise

A new type of dynamics called laminar chaos was recently discovered through a theoretical analysis of a scalar delay differential equation with time-varying delay. Laminar chaos is a low-dimensional dynamics characterized by laminar phases of nearly constant intensity with periodic durations and a chaotic variation of the intensity from one laminar phase to the next laminar phase. This is in stark contrast to the typically observed higher-dimensional turbulent chaos, which is characterized by strong fluctuations. In this Letter we provide the first experimental observation of laminar chaos by studying an optoelectronic feedback loop with time-varying delay. The noise inherent in the experiment requires the development of a nonlinear Langevin equation with variable delay. The results show that laminar chaos can be observed in higher-order systems, and that the phenomenon is robust to noise and a digital implementation of the variable time delay.

Nonlinear dynamics of delay systems: an overview

Time delays play an important role in many fields such as engineering, physics or biology. Delays occur due to finite velocities of signal propagation or processing delays leading to memory effects and, in general, infinite-dimensional systems. Time delay systems can be described by delay differential equations and often include non-negligible nonlinear effects. This overview article introduces the theme issue 'Nonlinear dynamics of delay systems', which contains new fundamental results in this interdisciplinary field as well as recent developments in applications. Fundamentally, new results were obtained especially for systems with time-varying delay and state-dependent delay and for delay system with noise, which do often appear in real systems in engineering and nature. The applications range from climate modelling over network dynamics and laser systems with feedback to human balancing and machine tool chatter. This article is part of the theme issue 'Nonlinear dynamics of delay systems'.