Introduction to Focus Issue: Time-delay dynamics

One-core neuron deep learning for time series prediction

The enormous computational requirements and unsustainable resource consumption associated with massive parameters of large language models and large vision models have given rise to challenging issues. Here, we propose an interpretable 'small model' framework characterized by only a single core-neuron, i.e. the one-core-neuron system (OCNS), to significantly reduce the number of parameters while maintaining performance comparable to the existing 'large models' in time-series forecasting. With multiple delay feedback designed in this single neuron, our OCNS is able to convert one input feature vector/state into one-dimensional time-series/sequence, which is theoretically ensured to fully represent the states of the observed dynamical system. Leveraging the spatiotemporal information transformation, the OCNS shows excellent and robust performance in forecasting tasks, in particular for short-term high-dimensional systems. The results collectively demonstrate that the proposed OCNS with a single core neuron offers insights into constructing deep learning frameworks with a small model, presenting substantial potential as a new way for achieving efficient deep learning.

Short delay limit of the delayed Duffing oscillator

The delayed Duffing equation, x^{″}+ɛx^{'}+x+x^{3}+cx(t-τ)=0, admits a Hopf bifurcation which becomes singular in the limit ɛ→0 and τ=O(ɛ)→0. To resolve this singularity, we develop an asymptotic theory where x(t-τ) is Taylor expanded in powers of τ. We derive a minimal system of ordinary differential equations that captures the Hopf bifurcation branch of the original delay differential equation. An unexpected result of our analysis is the necessity of expanding x(t-τ) up to third order rather than first order. Our work is motivated by laser stability problems exhibiting the same bifurcation problem as the delayed Duffing oscillator [Kovalev et al., Phys. Rev. E 103, 042206 (2021)2470-004510.1103/PhysRevE.103.042206]. Here we substantiate our theory based on the short delay limit by showing the overlap (matching) between our solution and two different asymptotic solutions derived for arbitrary fixed delays.

Square waves and Bykov T-points in a delay algebraic model for the Kerr-Gires-Tournois interferometer

We study theoretically the mechanisms of square wave formation of a vertically emitting micro-cavity operated in the Gires-Tournois regime that contains a Kerr medium and that is subjected to strong time-delayed optical feedback and detuned optical injection. We show that in the limit of large delay, square wave solutions of the time-delayed system can be treated as relative homoclinic solutions of an equation with an advanced argument. Based on this, we use concepts of classical homoclinic bifurcation theory to study different types of square wave solutions. In particular, we unveil the mechanisms behind the collapsed snaking scenario of square waves and explain the formation of complex-shaped multistable square wave solutions through a Bykov T-point. Finally, we relate the position of the T-point to the position of the Maxwell point in the original time-delayed system.

Transitional cluster dynamics in a model for delay-coupled chemical oscillators

Cluster synchronization is a fundamental phenomenon in systems of coupled oscillators. Here, we investigate clustering patterns that emerge in a unidirectional ring of four delay-coupled electrochemical oscillators. A voltage parameter in the experimental setup controls the onset of oscillations via a Hopf bifurcation. For a smaller voltage, the oscillators exhibit simple, so-called primary, clustering patterns, where all phase differences between each set of coupled oscillators are identical. However, upon increasing the voltage, secondary states, where phase differences differ, are detected, in addition to the primary states. Previous work on this system saw the development of a mathematical model that explained how the existence, stability, and common frequency of the experimentally observed cluster states could be accurately controlled by the delay time of the coupling. In this study, we revisit the mathematical model of the electrochemical oscillators in order to address open questions by means of bifurcation analysis. Our analysis reveals how the stable cluster states, corresponding to experimental observations, lose their stability via an assortment of bifurcation types. The analysis further reveals complex interconnectedness between branches of different cluster types. We find that each secondary state provides a continuous transition between certain primary states. These connections are explained by studying the phase space and parameter symmetries of the respective states. Furthermore, we show that it is only for a larger value of the voltage parameter that the branches of secondary states develop intervals of stability. For a smaller voltage, all the branches of secondary states are completely unstable and are, therefore, hidden to experimentalists.

Dynamical analysis of a damped harmonic forced duffing oscillator with time delay

This paper is concerned with a time-delayed controller of a damped nonlinear excited Duffing oscillator (DO). Since time-delayed techniques have recently been the focus of numerous studies, the topic of this investigation is quite contemporary. Therefore, time delays of position and velocity are utilized to reduce the nonlinear oscillation of the model under consideration. A much supplementary precise approximate solution is achieved using an advanced Homotopy perturbation method (HPM). The temporal variation of this solution is graphed for different amounts of the employed factors. The organization of the model is verified through a comparison between the plots of the estimated solution and the numerical one which is obtained utilizing the fourth order Runge-Kutta technique (RK4). The outcomes show that the improved HPM is appropriate for a variety of damped nonlinear oscillators since it minimizes the error of the solution while increasing the validation variety. Furthermore, it presents a potential model that deals with a diversity of nonlinear problems. The multiple scales homotopy technique is used to achieve an estimated formula for the suggested time-delayed structure. The controlling nonlinear algebraic equation for the amplitude oscillation at the steady state is gained. The effectiveness of the proposed controller, the time delays impact, controller gains, and feedback gains have been investigated. The realized outcomes show that the controller performance is influenced by the total of the product of the control and feedback gains, in addition to the time delays in the control loop. The analytical and numerical calculations reveal that for certain amounts of the control and feedback signal improvement, the suggested controller could completely reduce the system vibrations. The obtained outcomes are considered novel, in which the used methods are applied on the DO with time-delay. The increase of the time delay parameter leads to a stable case for the DO, which is in harmony with the influence of this parameter. This drawn curves show that the system reaches a stable fixed point which assert the presented discussion.

Laminar chaos in systems with quasiperiodic delay

A type of chaos called laminar chaos was found in singularly perturbed dynamical systems with periodic time-varying delay [Phys. Rev. Lett. 120, 084102 (2018)]0031-900710.1103/PhysRevLett.120.084102. It is characterized by nearly constant laminar phases, which are periodically interrupted by irregular bursts, where the intensity levels of the laminar phases vary chaotically from phase to phase. In this paper, we demonstrate that laminar chaos can also be observed in systems with quasiperiodic delay, where we generalize the concept of conservative and dissipative delays to such systems. It turns out that the durations of the laminar phases vary quasiperiodically and follow the dynamics of a torus map in contrast to the periodic variation observed for periodic delay. Theoretical and numerical results indicate that introducing a quasiperiodic delay modulation into a time-delay system can lead to a giant reduction of the dimension of the chaotic attractors. By varying the mean delay and keeping other parameters fixed, we found that the Kaplan-Yorke dimension is modulated quasiperiodically over several orders of magnitudes, where the dynamics switches quasiperiodically between different types of high- and low-dimensional types of chaos.

Generic generation of noise-driven chaos in stochastic time delay systems: Bridging the gap with high-end simulations

Nonlinear time delay systems produce inherently delay-induced periodic oscillations, which are, however, too idealistic compared to observations. We exhibit a unified stochastic framework to systematically rectify such oscillations into oscillatory patterns with enriched temporal variabilities through generic, nonlinear responses to stochastic perturbations. Two paradigms of noise-driven chaos in high dimension are identified, fundamentally different from chaos triggered by parameter-space noise. Noteworthy is a low-dimensional stretch-and-fold mechanism, leading to stochastic strange attractors exhibiting horseshoe-like structures mirroring turbulent transport of passive tracers. The other is high-dimensional , with noise acting along the critical eigendirection and transmitted to "deeper" stable modes through nonlinearity, leading to stochastic attractors exhibiting swarm-like behaviors with power-law and scale break properties. The theory is applied to cloud delay models to parameterize missing physics such as intermittent rain and Lagrangian turbulent effects. The stochastically rectified model reproduces with fidelity complex temporal variabilities of open-cell oscillations exhibited by high-end cloud simulations.

Conservative Solitons and Reversibility in Time Delayed Systems

Time delayed dynamical systems have proven to be a fertile framework for the study of physical phenomena. In natural sciences, their uses have been limited to the study of dissipative dynamics. In this Letter, we demonstrate the existence of nonlinear reversible conservative time delayed systems. We consider the example of a dispersive microcavity containing a Kerr medium coupled to a distant external mirror. At low energies and in the long delay limit, a multiscale analysis shows the equivalence with the nonlinear Schrödinger equation. We unveil some of the symmetries and conserved quantities, as well as bright temporal solitons. While elastic collisions occur for shallow wave packets, we observe the lack of integrability at higher energies. We recover the Lugiato-Lefever equation in the weakly dissipative regime.

Dynamics of coupled Kuramoto oscillators with distributed delays

This paper studies the effects of two different types of distributed-delay coupling in the system of two mutually coupled Kuramoto oscillators: one where the delay distribution is considered inside the coupling function and the other where the distribution enters outside the coupling function. In both cases, the existence and stability of phase-locked solutions is analyzed for uniform and gamma distribution kernels. The results show that while having the distribution inside the coupling function only changes parameter regions where phase-locked solutions exist, when the distribution is taken outside the coupling function, it affects both the existence, as well as stability properties of in- and anti-phase states. For both distribution types, various branches of phase-locked solutions are computed, and regions of their stability are identified for uniform, weak, and strong gamma distributions.

Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops

Deep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron's dynamics. By adjusting the feedback-modulation within the loops, we adapt the network's connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

Effect of time delay in a bistable synthetic gene network

The essence of logical stochastic resonance is the dynamic manipulation of potential wells. The effect of time delay on the depth of potential wells and the width of a bistable region can be inferred by logic operations in the bistable system with time delay. In a time-delayed synthetic gene network, time delay in the synthesis process can increase the depth of the potential wells, while that in the degradation process, it can reduce the depth of the potential wells, which will result in a decrease in the width of the bistable region (the reason for time delay to induce logic operations without external driving force) and the instability of the system (oscillation). These two opposite effects imply stretching and folding, leading to complex dynamical behaviors of the system, including period, chaos, bubble, chaotic bubble, forward and reverse period doubling bifurcation, intermittency, and coexisting attractors.

Spatiotemporal representation of long-delayed systems: An alternative approach

Dynamical systems with long-delay feedback can exhibit complicated temporal phenomena, which once reorganized in a two-dimensional space are reminiscent of spatiotemporal behavior. In this framework, a normal forms description has been developed to reproduce the dynamics, and the opportunity to treat the corresponding variables as true space and time has since been established. However, recently, an alternative approach has been proposed [F. Marino and G. Giacomelli, Phys. Rev. E 98, 060201(R) (2018)2470-004510.1103/PhysRevE.98.060201] with a different interpretation of the variables involved, which better takes into account their physical character and allows for an easier determination of the normal forms. In this paper, we extend such idea and apply it to a number of paradigmatic examples, paving the way to a rethinking of the concept of spatiotemporal representation of long-delayed systems.

Deep time-delay reservoir computing: Dynamics and memory capacity

The deep time-delay reservoir computing concept utilizes unidirectionally connected systems with time-delays for supervised learning. We present how the dynamical properties of a deep Ikeda-based reservoir are related to its memory capacity (MC) and how that can be used for optimization. In particular, we analyze bifurcations of the corresponding autonomous system and compute conditional Lyapunov exponents, which measure generalized synchronization between the input and the layer dynamics. We show how the MC is related to the systems' distance to bifurcations or magnitude of the conditional Lyapunov exponent. The interplay of different dynamical regimes leads to an adjustable distribution between the linear and nonlinear MC. Furthermore, numerical simulations show resonances between the clock cycle and delays of the layers in all degrees of MC. Contrary to MC losses in single-layer reservoirs, these resonances can boost separate degrees of MC and can be used, e.g., to design a system with maximum linear MC. Accordingly, we present two configurations that empower either high nonlinear MC or long time linear MC.

Evolutionary dynamics of the delayed replicator-mutator equation: Limit cycle and cooperation

Game theory deals with strategic interactions among players and evolutionary game dynamics tracks the fate of the players' populations under selection. In this paper, we consider the replicator equation for two-player-two-strategy games involving cooperators and defectors. We modify the equation to include the effect of mutation and also delay that corresponds either to the delayed information about the population state or in realizing the effect of interaction among players. By focusing on the four exhaustive classes of symmetrical games-the Stag Hunt game, the Snowdrift game, the Prisoners' Dilemma game, and the Harmony game-we analytically and numerically analyze the delayed replicator-mutator equation to find the explicit condition for the Hopf bifurcation bringing forth stable limit cycle. The existence of the asymptotically stable limit cycle imply the coexistence of the cooperators and the defectors; and in the games, where defection is a stable Nash strategy, a stable limit cycle does provide a mechanism for evolution of cooperation. We find that while mutation alone can never lead to oscillatory cooperation state in two-player-two-strategy games, the delay can change the scenario. On the other hand, there are situations when delay alone cannot lead to the Hopf bifurcation in the absence of mutation in the selection dynamics.

Performance boost of time-delay reservoir computing by non-resonant clock cycle

The time-delay-based reservoir computing setup has seen tremendous success in both experiment and simulation. It allows for the construction of large neuromorphic computing systems with only few components. However, until now the interplay of the different timescales has not been investigated thoroughly. In this manuscript, we investigate the effects of a mismatch between the time-delay and the clock cycle for a general model. Typically, these two time scales are considered to be equal. Here we show that the case of equal or resonant time-delay and clock cycle could be actively detrimental and leads to an increase of the approximation error of the reservoir. In particular, we can show that non-resonant ratios of these time scales have maximal memory capacities. We achieve this by translating the periodically driven delay-dynamical system into an equivalent network. Networks that originate from a system with resonant delay-times and clock cycles fail to utilize all of their degrees of freedom, which causes the degradation of their performance.

Delayed dynamical systems: networks, chimeras and reservoir computing

We present a systematic approach to reveal the correspondence between time delay dynamics and networks of coupled oscillators. After early demonstrations of the usefulness of spatio-temporal representations of time-delay system dynamics, extensive research on optoelectronic feedback loops has revealed their immense potential for realizing complex system dynamics such as chimeras in rings of coupled oscillators and applications to reservoir computing. Delayed dynamical systems have been enriched in recent years through the application of digital signal processing techniques. Very recently, we have showed that one can significantly extend the capabilities and implement networks with arbitrary topologies through the use of field programmable gate arrays. This architecture allows the design of appropriate filters and multiple time delays, and greatly extends the possibilities for exploring synchronization patterns in arbitrary network topologies. This has enabled us to explore complex dynamics on networks with nodes that can be perfectly identical, introduce parameter heterogeneities and multiple time delays, as well as change network topologies to control the formation and evolution of patterns of synchrony. This article is part of the theme issue 'Nonlinear dynamics of delay systems'.

Spiral wave initiation in excitable media

Spiral waves represent an important example of dissipative structures observed in many distributed systems in chemistry, biology and physics. By definition, excitable media occupy a stationary resting state in the absence of external perturbations. However, a perturbation exceeding a threshold results in the initiation of an excitation wave propagating through the medium. These waves, in contrast to acoustic and optical ones, disappear at the medium's boundary or after a mutual collision, and the medium returns to the resting state. Nevertheless, an initiation of a rotating spiral wave results in a self-sustained activity. Such activity unexpectedly appearing in cardiac or neuronal tissues usually destroys their dynamics which results in life-threatening diseases. In this context, an understanding of possible scenarios of spiral wave initiation is of great theoretical importance with many practical applications.This article is part of the theme issue 'Dissipative structures in matter out of equilibrium: from chemistry, photonics and biology (part 2)'.

Propagation delays determine neuronal activity and synaptic connectivity patterns emerging in plastic neuronal networks

In plastic neuronal networks, the synaptic strengths are adapted to the neuronal activity. Specifically, spike-timing-dependent plasticity (STDP) is a fundamental mechanism that modifies the synaptic strengths based on the relative timing of pre- and postsynaptic spikes, taking into account the spikes' temporal order. In many studies, propagation delays were neglected to avoid additional dynamic complexity or computational costs. So far, networks equipped with a classic STDP rule typically rule out bidirectional couplings (i.e., either loops or uncoupled states) and are, hence, not able to reproduce fundamental experimental findings. In this review paper, we consider additional features, e.g., extensions of the classic STDP rule or additional aspects like noise, in order to overcome the contradictions between theory and experiment. In addition, we review in detail recent studies showing that a classic STDP rule combined with realistic propagation patterns is able to capture relevant experimental findings. In two coupled oscillatory neurons with propagation delays, bidirectional synapses can be preserved and potentiated. This result also holds for large networks of type-II phase oscillators. In addition, not only the mean of the initial distribution of synaptic weights, but also its standard deviation crucially determines the emergent structural connectivity, i.e., the mean final synaptic weight, the number of two-neuron loops, and the symmetry of the final connectivity pattern. The latter is affected by the firing rates, where more symmetric synaptic configurations emerge at higher firing rates. Finally, we discuss these findings in the context of the computational neuroscience-based development of desynchronizing brain stimulation techniques.