Reservoir time series analysis: Using the response of complex dynamical systems as a universal indicator of change

We present the idea of reservoir time series analysis (RTSA), a method by which the state space representation generated by a reservoir computing (RC) model can be used for time series analysis. We discuss the motivation for this with reference to the characteristics of RC and present three ad hoc methods for generating representative features from the reservoir state space. We then develop and implement a hypothesis test to assess the capacity of these features to distinguish signals from systems with varying parameters. In comparison to a number of benchmark approaches (statistical, Fourier, phase space, and recurrence analysis), we are able to show significant, generalized accuracy across the proposed RTSA features that surpasses the benchmark methods. Finally, we briefly present an application for bearing fault distinction to motivate the use of RTSA in application.

Parameter extraction with reservoir computing: Nonlinear time series analysis and application to industrial maintenance

We study the task of determining parameters of dynamical systems from their time series using variations of reservoir computing. Averages of reservoir activations yield a static set of random features that allows us to separate different parameter values. We study such random feature models in the time and frequency domain. For the Lorenz and Rössler systems throughout stable and chaotic regimes, we achieve accurate and robust parameter extraction. For vibration data of centrifugal pumps, we find a significant ability to recover the operating regime. While the time domain models achieve higher performance for the numerical systems, the frequency domain models are superior in the application context.

Reservoir computing with swarms

We study swarms as dynamical systems for reservoir computing (RC). By example of a modified Reynolds boids model, the specific symmetries and dynamical properties of a swarm are explored with respect to a nonlinear time-series prediction task. Specifically, we seek to extract meaningful information about a predator-like driving signal from the swarm's response to that signal. We find that the naïve implementation of a swarm for computation is very inefficient, as permutation symmetry of the individual agents reduces the computational capacity. To circumvent this, we distinguish between the computational substrate of the swarm and a separate observation layer, in which the swarm's response is measured for use in the task. We demonstrate the implementation of a radial basis-localized observation layer for this task. The behavior of the swarm is characterized by order parameters and measures of consistency and related to the performance of the swarm as a reservoir. The relationship between RC performance and swarm behavior demonstrates that optimal computational properties are obtained near a phase transition regime.

Laminar chaos in nonlinear electronic circuits with delay clock modulation

We study laminar chaos in an electronic experiment. A two-diode nonlinear circuit with delayed feedback shows chaotic dynamics similar to the Mackey-Glass or Ikeda delay systems. Clock modulation of a single delay line leads to a conservative variable delay, which with a second delay line is augmented to dissipative delays, leading to laminar chaotic regimes. We discuss the properties of this particular delay modulation and demonstrate experimental aspects of laminar chaos in terms of power spectra and return maps.

Learned emergence in selfish collective motion

To understand the collective motion of many individuals, we often rely on agent-based models with rules that may be computationally complex and involved. For biologically inspired systems in particular, this raises questions about whether the imposed rules are necessarily an accurate reflection of what is being followed. The basic premise of updating one's state according to some underlying motivation is well suited to the realm of reservoir computing; however, entire swarms of individuals are yet to be tasked with learning movement in this framework. This work focuses on the specific case of many selfish individuals simultaneously optimizing their domains in a manner conducive to reducing their personal risk of predation. Using an echo state network and data generated from the agent-based model, we show that, with an appropriate representation of input and output states, this selfish movement can be learned. This suggests that a more sophisticated neural network, such as a brain, could also learn this behavior and provides an avenue to further the search for realistic movement rules in systems of autonomous individuals.

The reservoir's perspective on generalized synchronization

We employ reservoir computing for a reconstruction task in coupled chaotic systems, across a range of dynamical relationships including generalized synchronization. For a drive-response setup, a temporal representation of the synchronized state is discussed as an alternative to the known instantaneous form. The reservoir has access to both representations through its fading memory property, each with advantages in different dynamical regimes. We also extract signatures of the maximal conditional Lyapunov exponent in the performance of variations of the reservoir topology. Moreover, the reservoir model reproduces different levels of consistency where there is no synchronization. In a bidirectional coupling setup, high reconstruction accuracy is achieved despite poor observability and independent of generalized synchronization.

Consistency in echo-state networks

Consistency is an extension to generalized synchronization which quantifies the degree of functional dependency of a driven nonlinear system to its input. We apply this concept to echo-state networks, which are an artificial-neural network version of reservoir computing. Through a replica test, we measure the consistency levels of the high-dimensional response, yielding a comprehensive portrait of the echo-state property.

Consistency properties of chaotic systems driven by time-delayed feedback

Consistency refers to the property of an externally driven dynamical system to respond in similar ways to similar inputs. In a delay system, the delayed feedback can be considered as an external drive to the undelayed subsystem. We analyze the degree of consistency in a generic chaotic system with delayed feedback by means of the auxiliary system approach. In this scheme an identical copy of the nonlinear node is driven by exactly the same signal as the original, allowing us to verify complete consistency via complete synchronization. In the past, the phenomenon of synchronization in delay-coupled chaotic systems has been widely studied using correlation functions. Here, we analytically derive relationships between characteristic signatures of the correlation functions in such systems and unequivocally relate them to the degree of consistency. The analytical framework is illustrated and supported by numerical calculations of the logistic map with delayed feedback for different replica configurations. We further apply the formalism to time series from an experiment based on a semiconductor laser with a double fiber-optical feedback loop. The experiment constitutes a high-quality replica scheme for studying consistency of the delay-driven laser and confirms the general theoretical results.

Synchronization of Heterogeneous Oscillators by Noninvasive Time-Delayed Cross Coupling

We demonstrate that nonidentical systems, in particular, nonlinear oscillators with different time scales, can be synchronized if a mutual coupling via time-delayed control signals is implemented. Each oscillator settles on an unstable state, say a fixed point or an unstable periodic orbit, with a coupling force which vanishes in the long time limit. We present the underlying theoretical considerations and numerical simulations, and, moreover, demonstrate the concept experimentally in nonlinear electronic oscillators.

Determining the sub-Lyapunov exponent of delay systems from time series

For delay systems the sign of the sub-Lyapunov exponent (sub-LE) determines key dynamical properties. This includes the properties of strong and weak chaos and of consistency. Here we present a robust algorithm based on reconstruction of the local linearized equations of motion, which allows for calculating the sub-LE from time series. The algorithm is inspired by a method introduced by Pyragas for a nondelayed drive-response scheme [K. Pyragas, Phys. Rev. E 56, 5183 (1997)]. In the presented extension to delay systems, the delayed feedback takes over the role of the drive, whereas the response of the low-dimensional node leads to the sub-Lyapunov exponent. Our method is based on a low-dimensional representation of the delay system. We introduce the basic algorithm for a discrete scalar map, extend the concept to scalar continuous delay systems, and give an outlook to the case of a full vector-state system, from which only a scalar observable is recorded.

The transition between strong and weak chaos in delay systems: Stochastic modeling approach

We investigate the scaling behavior of the maximal Lyapunov exponent in chaotic systems with time delay. In the large-delay limit, it is known that one can distinguish between strong and weak chaos depending on the delay scaling, analogously to strong and weak instabilities for steady states and periodic orbits. Here we show that the Lyapunov exponent of chaotic systems shows significant differences in its scaling behavior compared to constant or periodic dynamics due to fluctuations in the linearized equations of motion. We reproduce the chaotic scaling properties with a linear delay system with multiplicative noise. We further derive analytic limit cases for the stochastic model illustrating the mechanisms of the emerging scaling laws.

Consistency properties of a chaotic semiconductor laser driven by optical feedback

We experimentally study consistency properties of a semiconductor laser in response to a coherent optical drive originating from delayed feedback. The laser is connected to a short and a long optical fiber loop, switched such that only one is providing input to the laser at a time. This way, repeating the exact same optical drive twice, we find consistent or inconsistent responses depending on the pump parameter and we relate the kind of response to strong and weak chaos. Moreover, we are able to experimentally determine the sub-Lyapunov exponent, underlying the consistency properties.

Autocorrelation properties of chaotic delay dynamical systems: A study on semiconductor lasers

We present a detailed experimental characterization of the autocorrelation properties of a delayed feedback semiconductor laser for different dynamical regimes. We show that in many cases the autocorrelation function of laser intensity dynamics can be approximated by the analytically derived autocorrelation function obtained from a linear stochastic model with delay. We extract a set of dynamic parameters from the fit with the analytic solutions and discuss the limits of validity of our approximation. The linear model captures multiple fundamental properties of delay systems, such as the shift and asymmetric broadening of the different delay echoes. Thus, our analysis provides significant additional insight into the relevant physical and dynamical properties of delayed feedback lasers.

Stochastic switching in delay-coupled oscillators

A delay is known to induce multistability in periodic systems. Under influence of noise, coupled oscillators can switch between coexistent orbits with different frequencies and different oscillation patterns. For coupled phase oscillators we reduce the delay system to a nondelayed Langevin equation, which allows us to analytically compute the distribution of frequencies and their corresponding residence times. The number of stable periodic orbits scales with the roundtrip delay time and coupling strength, but the noisy system visits only a fraction of the orbits, which scales with the square root of the delay time and is independent of the coupling strength. In contrast, the residence time in the different orbits is mainly determined by the coupling strength and the number of oscillators, and only weakly dependent on the coupling delay. Finally we investigate the effect of a detuning between the oscillators. We demonstrate the generality of our results with delay-coupled FitzHugh-Nagumo oscillators.

Delayed feedback control of unstable steady states with high-frequency modulation of the delay

We analyze the stabilization of unstable steady states by delayed feedback control with a periodic time-varying delay in the regime of a high-frequency modulation of the delay. The average effect of the delayed feedback term in the control force is equivalent to a distributed delay in the interval of the modulation, and the obtained distribution depends on the type of the modulation. In our analysis we use a simple generic normal form of an unstable focus, and investigate the effects of phase-dependent coupling and the influence of the control loop latency on the controllability. In addition, we have explored the influence of the modulation of the delays in multiple delay feedback schemes consisting of two independent delay lines of Pyragas type. A main advantage of the variable delay is the considerably larger domain of stabilization in parameter space.

Strong and weak chaos in networks of semiconductor lasers with time-delayed couplings

Nonlinear networks with time-delayed couplings may show strong and weak chaos, depending on the scaling of their Lyapunov exponent with the delay time. We study strong and weak chaos for semiconductor lasers, either with time-delayed self-feedback or for small networks. We examine the dependence on the pump current and consider the question of whether strong and weak chaos can be identified from the shape of the intensity trace, the autocorrelations, and the external cavity modes. The concept of the sub-Lyapunov exponent λ(0) is generalized to the case of two time-scale-separated delays in the system. We give experimental evidence of strong and weak chaos in a network of lasers, which supports the sequence of weak to strong to weak chaos upon monotonically increasing the coupling strength. Finally, we discuss strong and weak chaos for networks with several distinct sub-Lyapunov exponents and comment on the dependence of the sub-Lyapunov exponent on the number of a laser's inputs in a network.

Experimental time-delayed feedback control with variable and distributed delays

We report on several improvements of the classical time-delayed feedback control method for stabilization of unstable periodic orbits or steady states. In an electronic circuit experiment, we were able to realize time-varying and distributed delays in the control force leading to successful control for large parameter sets, including large time delays. The presented techniques make advanced use of the natural torsion of the orbits, which is also necessary for the original control method to work.

Strong and weak chaos in nonlinear networks with time-delayed couplings

We study chaotic synchronization in networks with time-delayed coupling. We introduce the notion of strong and weak chaos, distinguished by the scaling properties of the maximum Lyapunov exponent within the synchronization manifold for large delay times, and relate this to the condition for stable or unstable chaotic synchronization, respectively. In simulations of laser models and experiments with electronic circuits, we identify transitions from weak to strong and back to weak chaos upon monotonically increasing the coupling strength.

Complete chaotic synchronization and exclusion of mutual Pyragas control in two delay-coupled Rössler-type oscillators

Two identical chaotic oscillators that are mutually coupled via time delayed signals show very complex patterns of completely synchronized dynamics including stationary states and periodic as well as chaotic oscillations. We have experimentally observed these synchronized states in delay-coupled electronic circuits and have analyzed their stability by numerical simulations and analytical calculations. We found that the conditions for longitudinal and transversal stability largely exclude each other and prevent, e.g., the synchronization of Pyragas-controlled orbits. Most striking is the observation of complete chaotic synchronization for large delay times, which should not be allowed in the given coupling scheme on the background of the actual paradigm.

Noise-free stochastic resonance at an interior crisis

We report on the observation of noise-free stochastic resonance in an externally driven diode resonator close to an interior crisis. At sufficiently high excitation amplitudes the diode resonator shows a strange attractor which after the collision with an unstable period-three orbit exhibits crisis-induced intermittency. In the intermittency regime the system jumps between the previously stable chaotic attractor and the phase space region which has been made accessible by the crisis. This random process can be used to amplify a weak periodic signal through the mechanism of stochastic resonance. In contrast to conventional stochastic resonance no external noise is needed. The chaotic intrinsic dynamics plays the role of the stochastic forcing. Our data obtained from the diode resonator are compared with numerical simulations of the logistic map where a similar crisis-induced intermittency is observed.