Hebbian dreaming for small datasets

The dreaming Hopfield model constitutes a generalization of the Hebbian paradigm for neural networks, that is able to perform on-line learning when "awake" and also to account for off-line "sleeping" mechanisms. The latter have been shown to enhance storing in such a way that, in the long sleep-time limit, this model can reach the maximal storage capacity achievable by networks equipped with symmetric pairwise interactions. In this paper, we inspect the minimal amount of information that must be supplied to such a network to guarantee a successful generalization, and we test it both on random synthetic and on standard structured datasets (i.e., MNIST, Fashion-MNIST and Olivetti). By comparing these minimal thresholds of information with those required by the standard (i.e., always "awake") Hopfield model, we prove that the present network can save up to ∼90% of the dataset size, yet preserving the same performance of the standard counterpart. This suggests that sleep may play a pivotal role in explaining the gap between the large volumes of data required to train artificial neural networks and the relatively small volumes needed by their biological counterparts. Further, we prove that the model Cost function (typically used in statistical mechanics) admits a representation in terms of a standard Loss function (typically used in machine learning) and this allows us to analyze its emergent computational skills both theoretically and computationally: a quantitative picture of its capabilities as a function of its control parameters is achieved and consistency between the two approaches is highlighted. The resulting network is an associative memory for pattern recognition tasks that learns from examples on-line, generalizes correctly (in suitable regions of its control parameters) and optimizes its storage capacity by off-line sleeping: such a reduction of the training cost can be inspiring toward sustainable AI and in situations where data are relatively sparse.

Towards a universal mechanism for successful deep learning

Recently, the underlying mechanism for successful deep learning (DL) was presented based on a quantitative method that measures the quality of a single filter in each layer of a DL model, particularly VGG-16 trained on CIFAR-10. This method exemplifies that each filter identifies small clusters of possible output labels, with additional noise selected as labels outside the clusters. This feature is progressively sharpened with each layer, resulting in an enhanced signal-to-noise ratio (SNR), which leads to an increase in the accuracy of the DL network. In this study, this mechanism is verified for VGG-16 and EfficientNet-B0 trained on the CIFAR-100 and ImageNet datasets, and the main results are as follows. First, the accuracy and SNR progressively increase with the layers. Second, for a given deep architecture, the maximal error rate increases approximately linearly with the number of output labels. Third, similar trends were obtained for dataset labels in the range [3, 1000], thus supporting the universality of this mechanism. Understanding the performance of a single filter and its dominating features paves the way to highly dilute the deep architecture without affecting its overall accuracy, and this can be achieved by applying the filter's cluster connections (AFCC).

Enhancing the accuracies by performing pooling decisions adjacent to the output layer

Learning classification tasks of [Formula: see text] inputs typically consist of [Formula: see text]) max-pooling (MP) operators along the entire feedforward deep architecture. Here we show, using the CIFAR-10 database, that pooling decisions adjacent to the last convolutional layer significantly enhance accuracies. In particular, average accuracies of the advanced-VGG with [Formula: see text] layers (A-VGGm) architectures are 0.936, 0.940, 0.954, 0.955, and 0.955 for m = 6, 8, 14, 13, and 16, respectively. The results indicate A-VGG8's accuracy is superior to VGG16's, and that the accuracies of A-VGG13 and A-VGG16 are equal, and comparable to that of Wide-ResNet16. In addition, replacing the three fully connected (FC) layers with one FC layer, A-VGG6 and A-VGG14, or with several linear activation FC layers, yielded similar accuracies. These significantly enhanced accuracies stem from training the most influential input-output routes, in comparison to the inferior routes selected following multiple MP decisions along the deep architecture. In addition, accuracies are sensitive to the order of the non-commutative MP and average pooling operators adjacent to the output layer, varying the number and location of training routes. The results call for the reexamination of previously proposed deep architectures and their accuracies by utilizing the proposed pooling strategy adjacent to the output layer.

Efficient shallow learning as an alternative to deep learning

The realization of complex classification tasks requires training of deep learning (DL) architectures consisting of tens or even hundreds of convolutional and fully connected hidden layers, which is far from the reality of the human brain. According to the DL rationale, the first convolutional layer reveals localized patterns in the input and large-scale patterns in the following layers, until it reliably characterizes a class of inputs. Here, we demonstrate that with a fixed ratio between the depths of the first and second convolutional layers, the error rates of the generalized shallow LeNet architecture, consisting of only five layers, decay as a power law with the number of filters in the first convolutional layer. The extrapolation of this power law indicates that the generalized LeNet can achieve small error rates that were previously obtained for the CIFAR-10 database using DL architectures. A power law with a similar exponent also characterizes the generalized VGG-16 architecture. However, this results in a significantly increased number of operations required to achieve a given error rate with respect to LeNet. This power law phenomenon governs various generalized LeNet and VGG-16 architectures, hinting at its universal behavior and suggesting a quantitative hierarchical time-space complexity among machine learning architectures. Additionally, the conservation law along the convolutional layers, which is the square-root of their size times their depth, is found to asymptotically minimize error rates. The efficient shallow learning that is demonstrated in this study calls for further quantitative examination using various databases and architectures and its accelerated implementation using future dedicated hardware developments.

Brain inspired neuronal silencing mechanism to enable reliable sequence identification

Real-time sequence identification is a core use-case of artificial neural networks (ANNs), ranging from recognizing temporal events to identifying verification codes. Existing methods apply recurrent neural networks, which suffer from training difficulties; however, performing this function without feedback loops remains a challenge. Here, we present an experimental neuronal long-term plasticity mechanism for high-precision feedforward sequence identification networks (ID-nets) without feedback loops, wherein input objects have a given order and timing. This mechanism temporarily silences neurons following their recent spiking activity. Therefore, transitory objects act on different dynamically created feedforward sub-networks. ID-nets are demonstrated to reliably identify 10 handwritten digit sequences, and are generalized to deep convolutional ANNs with continuous activation nodes trained on image sequences. Counterintuitively, their classification performance, even with a limited number of training examples, is high for sequences but low for individual objects. ID-nets are also implemented for writer-dependent recognition, and suggested as a cryptographic tool for encrypted authentication. The presented mechanism opens new horizons for advanced ANN algorithms.

Efficient dendritic learning as an alternative to synaptic plasticity hypothesis

Synaptic plasticity is a long-lasting core hypothesis of brain learning that suggests local adaptation between two connecting neurons and forms the foundation of machine learning. The main complexity of synaptic plasticity is that synapses and dendrites connect neurons in series and existing experiments cannot pinpoint the significant imprinted adaptation location. We showed efficient backpropagation and Hebbian learning on dendritic trees, inspired by experimental-based evidence, for sub-dendritic adaptation and its nonlinear amplification. It has proven to achieve success rates approaching unity for handwritten digits recognition, indicating realization of deep learning even by a single dendrite or neuron. Additionally, dendritic amplification practically generates an exponential number of input crosses, higher-order interactions, with the number of inputs, which enhance success rates. However, direct implementation of a large number of the cross weights and their exhaustive manipulation independently is beyond existing and anticipated computational power. Hence, a new type of nonlinear adaptive dendritic hardware for imitating dendritic learning and estimating the computational capability of the brain must be built.

Long anisotropic absolute refractory periods with rapid rise times to reliable responsiveness

Refractoriness is a fundamental property of excitable elements, such as neurons, indicating the probability for re-excitation in a given time lag, and is typically linked to the neuronal hyperpolarization following an evoked spike. Here we measured the refractory periods (RPs) in neuronal cultures and observed that an average anisotropic absolute RP could exceed 10 ms and its tail is 20 ms, independent of a large stimulation frequency range. It is an order of magnitude longer than anticipated and comparable with the decaying membrane potential time scale. It is followed by a sharp rise-time (relative RP) of merely ∼1 md to complete responsiveness. Extracellular stimulations result in longer absolute RPs than solely intracellular ones, and a pair of extracellular stimulations from two different routes exhibits distinct absolute RPs, depending on their order. Our results indicate that a neuron is an accurate excitable element, where the diverse RPs cannot be attributed solely to the soma and imply fast mutual interactions between different stimulation routes and dendrites. Further elucidation of neuronal computational capabilities and their interplay with adaptation mechanisms is warranted.

Brain experiments imply adaptation mechanisms which outperform common AI learning algorithms

Attempting to imitate the brain's functionalities, researchers have bridged between neuroscience and artificial intelligence for decades; however, experimental neuroscience has not directly advanced the field of machine learning (ML). Here, using neuronal cultures, we demonstrate that increased training frequency accelerates the neuronal adaptation processes. This mechanism was implemented on artificial neural networks, where a local learning step-size increases for coherent consecutive learning steps, and tested on a simple dataset of handwritten digits, MNIST. Based on our on-line learning results with a few handwriting examples, success rates for brain-inspired algorithms substantially outperform the commonly used ML algorithms. We speculate this emerging bridge from slow brain function to ML will promote ultrafast decision making under limited examples, which is the reality in many aspects of human activity, robotic control, and network optimization.

Biological learning curves outperform existing ones in artificial intelligence algorithms

Recently, deep learning algorithms have outperformed human experts in various tasks across several domains; however, their characteristics are distant from current knowledge of neuroscience. The simulation results of biological learning algorithms presented herein outperform state-of-the-art optimal learning curves in supervised learning of feedforward networks. The biological learning algorithms comprise asynchronous input signals with decaying input summation, weights adaptation, and multiple outputs for an input signal. In particular, the generalization error for such biological perceptrons decreases rapidly with increasing number of examples, and it is independent of the size of the input. This is achieved using either synaptic learning, or solely through dendritic adaptation with a mechanism of swinging between reflecting boundaries, without learning steps. The proposed biological learning algorithms outperform the optimal scaling of the learning curve in a traditional perceptron. It also results in a considerable robustness to disparity between weights of two networks with very similar outputs in biological supervised learning scenarios. The simulation results indicate the potency of neurobiological mechanisms and open opportunities for developing a superior class of deep learning algorithms.

Mutual coupling and synchronization of optically coupled quantum-dot micropillar lasers at ultra-low light levels

Synchronization of coupled oscillators at the transition between classical physics and quantum physics has become an emerging research topic at the crossroads of nonlinear dynamics and nanophotonics. We study this unexplored field by using quantum dot microlasers as optical oscillators. Operating in the regime of cavity quantum electrodynamics (cQED) with an intracavity photon number on the order of 10 and output powers in the 100 nW range, these devices have high β-factors associated with enhanced spontaneous emission noise. We identify synchronization of mutually coupled microlasers via frequency locking associated with a sub-gigahertz locking range. A theoretical analysis of the coupling behavior reveals striking differences from optical synchronization in the classical domain with negligible spontaneous emission noise. Beyond that, additional self-feedback leads to zero-lag synchronization of coupled microlasers at ultra-low light levels. Our work has high potential to pave the way for future experiments in the quantum regime of synchronization.

Dendritic Learning as a Paradigm Shift in Brain Learning

Experimental and theoretical results reveal a new underlying mechanism for fast brain learning process, dendritic learning, as opposed to the misdirected research in neuroscience over decades, which is based solely on slow synaptic plasticity. The presented paradigm indicates that learning occurs in closer proximity to the neuron, the computational unit, dendritic strengths are self-oscillating, and weak synapses, which comprise the majority of our brain and previously were assumed to be insignificant, play a key role in plasticity. The new learning sites of the brain call for a reevaluation of current treatments for disordered brain functionality and for a better understanding of proper chemical drugs and biological mechanisms to maintain, control and enhance learning.

Less Might Be More: Conduction Failure as a Factor Possibly Limiting the Efficacy of Higher Frequencies in rTMS Protocols

Introduction: rTMS has been proven effective in the treatment of neuropsychiatric conditions, with class A (definite efficacy) evidence for treatment of depression and pain (Lefaucheur et al., 2014). The efficacy in stimulation protocols is, however, quite heterogeneous. Saturation of neuronal firing by HFrTMS without allowing time for recovery may lead to neuronal response failures (NRFs) that compromise the efficacy of stimulation with higher frequencies.

Objectives: To examine the efficacy of different rTMS temporal stimulation patterns focusing on a possible upper stimulation limit related to response failures. Protocol patterns were derived from published clinical studies on therapeutic rTMS for depression and pain. They were compared with conduction failures in cell cultures.

Methodology: From 57 papers using protocols rated class A for depression and pain (Lefaucheur et al., 2014) we extracted Inter-train interval (ITI), average frequency, total duration and total number of pulses and plotted them against the percent improvement on the outcome scale. Specifically, we compared 10 Hz trains with ITIs of 8 s (protocol A) and 26 s (protocol B) in vitro on cultured cortical neurons.

Results: In the in vitro experiments, protocol A with 8-s ITIs resulted in more frequent response failures, while practically no response failures occurred with protocol B (26-s intervals). The HFrTMS protocol analysis exhibited no significant effect of ITIs on protocol efficiency.

Discussion: In the neuronal culture, longer ITIs appeared to allow the neuronal response to recover. In the available human dataset on both depression and chronic pain, data concerning shorter ITIs is does not allow a significant conclusion.

Significance: NRF may interfere with the efficacy of rTMS stimulation protocols when the average stimulation frequency is too high, proposing ITIs as a variable in rTMS protocol efficacy. Clinical trials are necessary to examine effect of shorter ITIs on the clinical outcome in a controlled setting.

New Types of Experiments Reveal that a Neuron Functions as Multiple Independent Threshold Units

Neurons are the computational elements that compose the brain and their fundamental principles of activity are known for decades. According to the long-lasting computational scheme, each neuron sums the incoming electrical signals via its dendrites and when the membrane potential reaches a certain threshold the neuron typically generates a spike to its axon. Here we present three types of experiments, using neuronal cultures, indicating that each neuron functions as a collection of independent threshold units. The neuron is anisotropically activated following the origin of the arriving signals to the membrane, via its dendritic trees. The first type of experiments demonstrates that a single neuron’s spike waveform typically varies as a function of the stimulation location. The second type reveals that spatial summation is absent for extracellular stimulations from different directions. The third type indicates that spatial summation and subtraction are not achieved when combining intra- and extra- cellular stimulations, as well as for nonlocal time interference, where the precise timings of the stimulations are irrelevant. Results call to re-examine neuronal functionalities beyond the traditional framework, and the advanced computational capabilities and dynamical properties of such complex systems.

Oscillations in networks of networks stem from adaptive nodes with memory

We present an analytical framework that allows the quantitative study of statistical dynamic properties of networks with adaptive nodes that have memory and is used to examine the emergence of oscillations in networks with response failures. The frequency of the oscillations was quantitatively found to increase with the excitability of the nodes and with the average degree of the network and to decrease with delays between nodes. For networks of networks, diverse cluster oscillation modes were found as a function of the topology. Analytical results are in agreement with large-scale simulations and open the horizon for understanding network dynamics composed of finite memory nodes as well as their different phases of activity.

Simultaneous multi-patch-clamp and extracellular-array recordings: Single neuron reflects network activity

The increasing number of recording electrodes enhances the capability of capturing the network’s cooperative activity, however, using too many monitors might alter the properties of the measured neural network and induce noise. Using a technique that merges simultaneous multi-patch-clamp and multi-electrode array recordings of neural networks in-vitro, we show that the membrane potential of a single neuron is a reliable and super-sensitive probe for monitoring such cooperative activities and their detailed rhythms. Specifically, the membrane potential and the spiking activity of a single neuron are either highly correlated or highly anti-correlated with the time-dependent macroscopic activity of the entire network. This surprising observation also sheds light on the cooperative origin of neuronal burst in cultured networks. Our findings present an alternative flexible approach to the technique based on a massive tiling of networks by large-scale arrays of electrodes to monitor their activity.

Mimicking Collective Firing Patterns of Hundreds of Connected Neurons using a Single-Neuron Experiment

The experimental study of neural networks requires simultaneous measurements of a massive number of neurons, while monitoring properties of the connectivity, synaptic strengths and delays. Current technological barriers make such a mission unachievable. In addition, as a result of the enormous number of required measurements, the estimated network parameters would differ from the original ones. Here we present a versatile experimental technique, which enables the study of recurrent neural networks activity while being capable of dictating the network connectivity and synaptic strengths. This method is based on the observation that the response of neurons depends solely on their recent stimulations, a short-term memory. It allows a long-term scheme of stimulation and recording of a single neuron, to mimic simultaneous activity measurements of neurons in a recurrent network. Utilization of this technique demonstrates the spontaneous emergence of cooperative synchronous oscillations, in particular the coexistence of fast γ and slow δ oscillations, and opens the horizon for the experimental study of other cooperative phenomena within large-scale neural networks.

Broadband macroscopic cortical oscillations emerge from intrinsic neuronal response failures

Broadband spontaneous macroscopic neural oscillations are rhythmic cortical firing which were extensively examined during the last century, however, their possible origination is still controversial. In this work we show how macroscopic oscillations emerge in solely excitatory random networks and without topological constraints. We experimentally and theoretically show that these oscillations stem from the counterintuitive underlying mechanism-the intrinsic stochastic neuronal response failures (NRFs). These NRFs, which are characterized by short-term memory, lead to cooperation among neurons, resulting in sub- or several- Hertz macroscopic oscillations which coexist with high frequency gamma oscillations. A quantitative interplay between the statistical network properties and the emerging oscillations is supported by simulations of large networks based on single-neuron in-vitro experiments and a Langevin equation describing the network dynamics. Results call for the examination of these oscillations in the presence of inhibition and external drives.

Loss of lamin A function increases chromatin dynamics in the nuclear interior

Chromatin is organized in a highly ordered yet dynamic manner in the cell nucleus, but the principles governing this organization remain unclear. Similarly, it is unknown whether, and how, various proteins regulate chromatin motion and as a result influence nuclear organization. Here by studying the dynamics of different genomic regions in the nucleus of live cells, we show that the genome has highly constrained dynamics. Interestingly, depletion of lamin A strikingly alters genome dynamics, inducing a dramatic transition from slow anomalous diffusion to fast and normal diffusion. In contrast, depletion of LAP2α, a protein that interacts with lamin A and chromatin, has no such effect on genome dynamics. We speculate that chromosomal inter-chain interactions formed by lamin A throughout the nucleus contribute to chromatin dynamics, and suggest that the molecular regulation of chromatin diffusion by lamin A in the nuclear interior is critical for the maintenance of genome organization.

Neuronal response impedance mechanism implementing cooperative networks with low firing rates and μs precision

Realizations of low firing rates in neural networks usually require globally balanced distributions among excitatory and inhibitory links, while feasibility of temporal coding is limited by neuronal millisecond precision. We show that cooperation, governing global network features, emerges through nodal properties, as opposed to link distributions. Using in vitro and in vivo experiments we demonstrate microsecond precision of neuronal response timings under low stimulation frequencies, whereas moderate frequencies result in a chaotic neuronal phase characterized by degraded precision. Above a critical stimulation frequency, which varies among neurons, response failures were found to emerge stochastically such that the neuron functions as a low pass filter, saturating the average inter-spike-interval. This intrinsic neuronal response impedance mechanism leads to cooperation on a network level, such that firing rates are suppressed toward the lowest neuronal critical frequency simultaneously with neuronal microsecond precision. Our findings open up opportunities of controlling global features of network dynamics through few nodes with extreme properties.

Synchronization among neuronal pools without common inputs: in vivo study

Periodic synchronization of activity among neuronal pools has been related to substantial neural processes and information throughput in the neocortical network. However, the mechanisms of generating such periodic synchronization among distributed pools of neurons remain unclear. We hypothesize that to a large extent there is interplay between the topology of the neocortical networks and their reverberating modes of activity. The firing synchronization is governed by a nonlocal mechanism, the network delay loops, such that distant neuronal pools without common drives can be synchronized. This theoretical interplay between network topology and the synchronized mode is verified using an iterative procedure of a single intracellularly recorded neuron in vivo, imitating the dynamics of the entire network. The input is injected to the neuron via the recording electrode as current and computed from the filtered, evoked spikes of its pre-synaptic sources, previously emulated by the same neuron. In this manner we approximate subthreshold synaptic inputs from afferent neuronal pools to the neuron. Embedding the activity of these recurrent motifs in the intact brain allows us to measure the effects of connection probability, synaptic strength, and ongoing activity on the neuronal synchrony. Our in vivo experiments indicate that an initial stimulus given to a single pool is dynamically evolving into the formations of zero-lag and cluster synchronization. By applying results from theoretical models and in vitro experiments to in vivo activity in the intact brain, we support the notion that this mechanism may account for the binding activity across distributed brain areas.

A computational paradigm for dynamic logic-gates in neuronal activity

In 1943 McCulloch and Pitts suggested that the brain is composed of reliable logic-gates similar to the logic at the core of today's computers. This framework had a limited impact on neuroscience, since neurons exhibit far richer dynamics. Here we propose a new experimentally corroborated paradigm in which the truth tables of the brain's logic-gates are time dependent, i.e., dynamic logic-gates (DLGs). The truth tables of the DLGs depend on the history of their activity and the stimulation frequencies of their input neurons. Our experimental results are based on a procedure where conditioned stimulations were enforced on circuits of neurons embedded within a large-scale network of cortical cells in-vitro. We demonstrate that the underlying biological mechanism is the unavoidable increase of neuronal response latencies to ongoing stimulations, which imposes a non-uniform gradual stretching of network delays. The limited experimental results are confirmed and extended by simulations and theoretical arguments based on identical neurons with a fixed increase of the neuronal response latency per evoked spike. We anticipate our results to lead to better understanding of the suitability of this computational paradigm to account for the brain's functionalities and will require the development of new systematic mathematical methods beyond the methods developed for traditional Boolean algebra.

Error correction and fast detectors implemented by ultrafast neuronal plasticity

We experimentally show that the neuron functions as a precise time integrator, where the accumulated changes in neuronal response latencies, under complex and random stimulation patterns, are solely a function of a global quantity, the average time lag between stimulations. In contrast, momentary leaps in the neuronal response latency follow trends of consecutive stimulations, indicating ultrafast neuronal plasticity. On a circuit level, this ultrafast neuronal plasticity phenomenon implements error-correction mechanisms and fast detectors for misplaced stimulations. Additionally, at moderate (high) stimulation rates this phenomenon destabilizes (stabilizes) a periodic neuronal activity disrupted by misplaced stimulations.

Sudden synchrony leaps accompanied by frequency multiplications in neuronal activity

A classical view of neural coding relies on temporal firing synchrony among functional groups of neurons, however, the underlying mechanism remains an enigma. Here we experimentally demonstrate a mechanism where time-lags among neuronal spiking leap from several tens of milliseconds to nearly zero-lag synchrony. It also allows sudden leaps out of synchrony, hence forming short epochs of synchrony. Our results are based on an experimental procedure where conditioned stimulations were enforced on circuits of neurons embedded within a large-scale network of cortical cells in vitro and are corroborated by simulations of neuronal populations. The underlying biological mechanisms are the unavoidable increase of the neuronal response latency to ongoing stimulations and temporal or spatial summation required to generate evoked spikes. These sudden leaps in and out of synchrony may be accompanied by multiplications of the neuronal firing frequency, hence offering reliable information-bearing indicators which may bridge between the two principal neuronal coding paradigms.

Coupled lasers: phase versus chaos synchronization

The synchronization of chaotic lasers and the optical phase synchronization of light originating in multiple coupled lasers have both been extensively studied. However, the interplay between these two phenomena, especially at the network level, is unexplored. Here, we experimentally compare these phenomena by controlling the heterogeneity of the coupling delay times of two lasers. While chaotic lasers exhibit deterioration in synchronization as the time delay heterogeneity increases, phase synchronization is found to be independent of heterogeneity. The experimental results are found to be in agreement with numerical simulations for semiconductor lasers.

Fast physical random-number generation based on room-temperature chaotic oscillations in weakly coupled superlattices

An all-electronic physical random number generator at rates up to 80  Gbit/s is presented, based on weakly coupled GaAs/Ga0.55Al0.45As superlattices operated at room temperature. It is based on large-amplitude, chaotic current oscillations characterized by a bandwidth of several hundred MHz and do not require external feedback or conversion to an electronic signal prior to digitization. The method is robust and insensitive to external perturbations and its fully electronic implementation suggests scalability and minimal postprocessing in comparison to existing optical implementations.

Strong and weak chaos in networks of semiconductor lasers with time-delayed couplings

Nonlinear networks with time-delayed couplings may show strong and weak chaos, depending on the scaling of their Lyapunov exponent with the delay time. We study strong and weak chaos for semiconductor lasers, either with time-delayed self-feedback or for small networks. We examine the dependence on the pump current and consider the question of whether strong and weak chaos can be identified from the shape of the intensity trace, the autocorrelations, and the external cavity modes. The concept of the sub-Lyapunov exponent λ(0) is generalized to the case of two time-scale-separated delays in the system. We give experimental evidence of strong and weak chaos in a network of lasers, which supports the sequence of weak to strong to weak chaos upon monotonically increasing the coupling strength. Finally, we discuss strong and weak chaos for networks with several distinct sub-Lyapunov exponents and comment on the dependence of the sub-Lyapunov exponent on the number of a laser's inputs in a network.

Discontinuous attractor dimension at the synchronization transition of time-delayed chaotic systems

The attractor dimension at the transition to complete synchronization in a network of chaotic units with time-delayed couplings is investigated. In particular, we determine the Kaplan-Yorke dimension from the spectrum of Lyapunov exponents for iterated maps and for two coupled semiconductor lasers. We argue that the Kaplan-Yorke dimension must be discontinuous at the transition and compare it to the correlation dimension. For a system of Bernoulli maps, we indeed find a jump in the correlation dimension. The magnitude of the discontinuity in the Kaplan-Yorke dimension is calculated for networks of Bernoulli units as a function of the network size. Furthermore, the scaling of the Kaplan-Yorke dimension as well as of the Kolmogorov entropy with system size and time delay is investigated.

Chaos pass filter: linear response of synchronized chaotic systems

The linear response of synchronized time-delayed chaotic systems to small external perturbations, i.e., the phenomenon of chaos pass filter, is investigated for iterated maps. The distribution of distances, i.e., the deviations between two synchronized chaotic units due to external perturbations on the transferred signal, is used as a measure of the linear response. It is calculated numerically and, for some special cases, analytically. Depending on the model parameters this distribution has power law tails in the region of synchronization leading to diverging moments of distances. This is a consequence of multiplicative and additive noise in the corresponding linear equations due to chaos and external perturbations. The linear response can also be quantified by the bit error rate of a transmitted binary message which perturbs the synchronized system. The bit error rate is given by an integral over the distribution of distances and is calculated analytically and numerically. It displays a complex nonmonotonic behavior in the region of synchronization. For special cases the distribution of distances has a fractal structure leading to a devil's staircase for the bit error rate as a function of coupling strength. The response to small harmonic perturbations shows resonances related to coupling and feedback delay times. A bidirectionally coupled chain of three units can completely filter out the perturbation. Thus the second moment and the bit error rate become zero.

Synchronization by elastic neuronal latencies

Psychological and physiological considerations entail that formation and functionality of neuronal cell assemblies depend upon synchronized repeated activation such as zero-lag synchronization. Several mechanisms for the emergence of this phenomenon have been suggested, including the global network quantity, the greatest common divisor of neuronal circuit delay loops. However, they require strict biological prerequisites such as precisely matched delays and connectivity, and synchronization is represented as a stationary mode of activity instead of a transient phenomenon. Here we show that the unavoidable increase in neuronal response latency to ongoing stimulation serves as a nonuniform gradual stretching of neuronal circuit delay loops. This apparent nuisance is revealed to be an essential mechanism in various types of neuronal time controllers, where synchronization emerges as a transient phenomenon and without predefined precisely matched synaptic delays. These findings are described in an experimental procedure where conditioned stimulations were enforced on a circuit of neurons embedded within a large-scale network of cortical cells in vitro, and are corroborated and extended by simulations of circuits composed of Hodgkin-Huxley neurons with time-dependent latencies. These findings announce a cortical time scale for time controllers based on tens of microseconds stretching of neuronal circuit delay loops per spike. They call for a reexamination of the role of the temporal periodic mode in brain functionality using advanced in vitro and in vivo experiments.

Random symmetry breaking and freezing in chaotic networks

Parameter space of a driven damped oscillator in a double well potential presents either a chaotic trajectory with sign oscillating amplitude or a nonchaotic trajectory with a fixed sign amplitude. A network of such delay coupled damped oscillators is shown to present chaotic dynamics while the sign amplitude of each damped oscillator is randomly frozen. This phenomenon of random broken global symmetry of the network simultaneous with random freezing of each degree of freedom is accompanied by the existence of exponentially many randomly frozen chaotic attractors with the size of the network. Results are exemplified by a network of modified Duffing oscillators with infinite range pseudoinverse delayed interactions.

Phase transition in crowd synchrony of delay-coupled multilayer laser networks

An analogy between crowd synchrony and multi-layer neural network architectures is proposed. It indicates that many non-identical dynamical elements (oscillators) communicating indirectly via a few mediators (hubs) can synchronize when the number of delayed couplings to the hubs or the strength of the couplings is large enough. This phenomenon is modeled using a system of semiconductor lasers optically delay-coupled in either a fully connected or a diluted manner to a fixed number of non-identical central hub lasers. A universal phase transition to crowd synchrony with hysteresis is observed, where the time to achieve synchronization diverges near the critical coupling independent of the number of hubs.

Controlling synchronization in large laser networks

Synchronization in large laser networks with both homogeneous and heterogeneous coupling delay times is examined. The number of synchronized clusters of lasers is established to equal the greatest common divisor of network loops. We experimentally demonstrate up to 16 multicluster phase synchronization scenarios within unidirectional coupled laser networks, whereby synchronization in heterogeneous networks is deduced by mapping to an equivalent homogeneous network. The synchronization in large laser networks is controlled by means of tunable coupling and self-coupling.

Sensitivity of global network dynamics to local parameters versus motif structure in a cortexlike neuronal model

In the field of network dynamics it has been suggested that statistical information of motifs, small subnetworks, can help in understanding global activity of the entire network. We present a counterexample where the relation between the stable synchronized activity modes and network connectivity was studied using the Hodgkin-Huxley brain dynamics model. Simulations indicate that small motifs of three nodes exhibit different synchronization modes depending on their local parameters such as delays, synaptic strength, and external drives. Thus the activity of a complex network composed of interconnected motifs cannot be extracted from the activity mode of each individual motif and is governed by local parameters. Finally, we exemplify how local dynamics ultimately enriches the ability of a network to generate diverse modes with a given motif structure.

Synchronization in small networks of time-delay coupled chaotic diode lasers

Topologies of two, three and four time-delay-coupled chaotic semiconductor lasers are experimentally and theoretically found to show new types of synchronization. Generalized zero-lag synchronization is observed for two lasers separated by long distances even when their self-feedback delays are not equal. Generalized sub-lattice synchronization is observed for quadrilateral geometries while the equilateral triangle is zero-lag synchronized. Generalized zero-lag synchronization, without the limitation of precisely matched delays, opens possibilities for advanced multi-user communication protocols.

Strong and weak chaos in nonlinear networks with time-delayed couplings

We study chaotic synchronization in networks with time-delayed coupling. We introduce the notion of strong and weak chaos, distinguished by the scaling properties of the maximum Lyapunov exponent within the synchronization manifold for large delay times, and relate this to the condition for stable or unstable chaotic synchronization, respectively. In simulations of laser models and experiments with electronic circuits, we identify transitions from weak to strong and back to weak chaos upon monotonically increasing the coupling strength.

Coexistence of exponentially many chaotic spin-glass attractors

A chaotic network of size N with delayed interactions which resembles a pseudoinverse associative memory neural network is investigated. For a load α = P/N < 1, where P stands for the number of stored patterns, the chaotic network functions as an associative memory of 2P attractors with macroscopic basin of attractions which decrease with α. At finite α, a chaotic spin-glass phase exists, where the number of distinct chaotic attractors scales exponentially with N. Each attractor is characterized by a coexistence of chaotic behavior and freezing of each one of the N chaotic units or freezing with respect to the P patterns. Results are supported by large scale simulations of networks composed of Bernoulli map units and Mackey-Glass time delay differential equations.

Synchronized cluster formation in coupled laser networks

We experimentally investigate the phase dynamics of laser networks with homogenous time-delayed mutual coupling and establish the fundamental rules that govern their state of synchronization. We identified a specific substructure that imposes its synchronization state on the entire network and show that for any coupling configuration the network forms at most two synchronized clusters. Our results indicate that the synchronization state of the network is a nonlocal phenomenon and cannot be deduced by decomposing the network into smaller substructures, each with its individual synchronization state.

Synchronization of chaotic networks with time-delayed couplings: an analytic study

Networks of nonlinear units with time-delayed couplings can synchronize to a common chaotic trajectory. Although the delay time may be very large, the units can synchronize completely without time shift. For networks of coupled Bernoulli maps, analytic results are derived for the stability of the chaotic synchronization manifold. For a single delay time, chaos synchronization is related to the spectral gap of the coupling matrix. For networks with multiple delay times, analytic results are obtained from the theory of polynomials. Finally, the analytic results are compared with networks of iterated tent maps and Lang-Kobayashi equations, which imitate the behavior of networks of semiconductor lasers.

Optimal observation time window for forecasting the next earthquake

We report that the accuracy of predicting the occurrence time of the next earthquake is significantly enhanced by observing the latest rate of earthquake occurrences. The observation period that minimizes the temporal uncertainty of the next occurrence is on the order of 10 hours. This result is independent of the threshold magnitude and is consistent across different geographic areas. This time scale is much shorter than the months or years that have previously been considered characteristic of seismic activities.

Statistical-mechanics approach to wide-band digital communication

The emerging popular scheme of fourth generation wireless communication, orthogonal frequency-division multiplexing, is mapped onto a variant of a random field Ising Hamiltonian and results in an efficient physical intercarrier interference (ICI) cancellation decoding scheme. This scheme is based on Monte Carlo (MC) dynamics at zero temperature as well as at the Nishimori temperature and demonstrates improved bit error rate (BER) and robust convergence time compared to the state of the art ICI cancellation decoding scheme. An optimal BER performance is achieved with MC dynamics at the Nishimori temperature but with a substantial computational cost overhead. The suggested ICI cancellation scheme also supports the transmission of biased signals.

Synchronization of random bit generators based on coupled chaotic lasers and application to cryptography

Random bit generators (RBGs) constitute an important tool in cryptography, stochastic simulations and secure communications. The later in particular has some difficult requirements: high generation rate of unpredictable bit strings and secure key-exchange protocols over public channels. Deterministic algorithms generate pseudo-random number sequences at high rates, however, their unpredictability is limited by the very nature of their deterministic origin. Recently, physical RBGs based on chaotic semiconductor lasers were shown to exceed Gbit/s rates. Whether secure synchronization of two high rate physical RBGs is possible remains an open question. Here we propose a method, whereby two fast RBGs based on mutually coupled chaotic lasers, are synchronized. Using information theoretic analysis we demonstrate security against a powerful computational eavesdropper, capable of noiseless amplification, where all parameters are publicly known. The method is also extended to secure synchronization of a small network of three RBGs.

Zero lag synchronization of chaotic systems with time delayed couplings

Zero-lag synchronization (ZLS) between chaotic units, which do not have self-feedback or a relay unit connecting them, is experimentally demonstrated for two mutually coupled chaotic semiconductor lasers. The mechanism is based on two mutual coupling delay times with certain allowed integer ratios, whereas for a single mutual delay time ZLS cannot be achieved. This mechanism is also found numerically for mutually coupled chaotic maps where its stability is analyzed using the Schur-Cohn theorem for the roots of polynomials. The symmetry of the polynomials allows only specific integer ratios for ZLS. In addition, we present a general argument for ZLS when several mutual coupling delay times are present.

Zero-lag synchronization and multiple time delays in two coupled chaotic systems

Zero-lag synchronization (ZLS) between two chaotic systems coupled by a portion of their signal is achieved for restricted ratios between the delays of the self-feedback and the mutual coupling. We extend this scenario to the case of a set of multiple self-feedbacks {Ndi} and a set of multiple mutual couplings {Ncj}. We demonstrate both analytically and numerically that ZLS can be achieved when SigmaliNdi+igmamjNcj=0, where li,mj(epsilon)Z. Results which were mainly derived for Bernoulli maps and exemplified with simulations of the Lang-Kobayashi differential equations, indicate that ZLS can be achieved for a continuous range of mutual coupling delay. This phenomenon has an important implication in the possible use of ZLS in communication networks.

On chaos synchronization and secure communication

Chaos synchronization, in particular isochronal synchronization of two chaotic trajectories to each other, may be used to build a means of secure communication over a public channel. In this paper, we give an overview of coupling schemes of Bernoulli units deduced from chaotic laser systems, different ways to transmit information by chaos synchronization and the advantage of bidirectional over unidirectional coupling with respect to secure communication. We present the protocol for using dynamical private commutative filters for tap-proof transmission of information that maps the task of a passive attacker to the class of non-deterministic polynomial time-complete problems.

Pulses of chaos synchronization in coupled map chains with delayed transmission

Pulses of synchronization in chaotic coupled map lattices are discussed in the context of transmission of information. Synchronization and desynchronization propagate along the chain with different velocities which are calculated analytically from the spectrum of convective Lyapunov exponents. Since the front of synchronization travels slower than the front of desynchronization, the maximal possible chain length for which information can be transmitted by modulating the first unit of the chain is bounded.

Ultrahigh-speed random number generation based on a chaotic semiconductor laser

The fluctuating intensity of a chaotic semiconductor laser is used for generating random sequences at rates up to 12.5 Gbits/s. The conversion of the fluctuating intensity to a random bit sequence can be implemented in either software or hardware and the overall rate of generation is much faster than any previously reported random number generator based on a physical mechanism. The generator's simplicity, robustness, and insensitivity to control parameters should enable its application to tasks of secure communication and calculation procedures requiring ultrahigh-speed generation of random bit sequences.

Synchronization of networks of chaotic units with time-delayed couplings

A network of chaotic units is investigated where the units are coupled by signals with a transmission delay. Any arbitrary finite network is considered where the chaotic trajectories of the uncoupled units are a solution of the dynamic equations of the network. It is shown that chaotic trajectories cannot be synchronized if the transmission delay is larger than the time scales of the individual units. For several models the master stability function is calculated which determines the maximal delay time for which synchronization is possible.