Reconstructing networks of pulse-coupled oscillators from spike trains

Inferring connectivity of an oscillatory network via the phase dynamics reconstruction

We review an approach for reconstructing oscillatory networks' undirected and directed connectivity from data. The technique relies on inferring the phase dynamics model. The central assumption is that we observe the outputs of all network nodes. We distinguish between two cases. In the first one, the observed signals represent smooth oscillations, while in the second one, the data are pulse-like and can be viewed as point processes. For the first case, we discuss estimating the true phase from a scalar signal, exploiting the protophase-to-phase transformation. With the phases at hand, pairwise and triplet synchronization indices can characterize the undirected connectivity. Next, we demonstrate how to infer the general form of the coupling functions for two or three oscillators and how to use these functions to quantify the directional links. We proceed with a different treatment of networks with more than three nodes. We discuss the difference between the structural and effective phase connectivity that emerges due to high-order terms in the coupling functions. For the second case of point-process data, we use the instants of spikes to infer the phase dynamics model in the Winfree form directly. This way, we obtain the network's coupling matrix in the first approximation in the coupling strength.

Reconstruction of sparse recurrent connectivity and inputs from the nonlinear dynamics of neuronal networks

Reconstructing the recurrent structural connectivity of neuronal networks is a challenge crucial to address in characterizing neuronal computations. While directly measuring the detailed connectivity structure is generally prohibitive for large networks, we develop a novel framework for reverse-engineering large-scale recurrent network connectivity matrices from neuronal dynamics by utilizing the widespread sparsity of neuronal connections. We derive a linear input-output mapping that underlies the irregular dynamics of a model network composed of both excitatory and inhibitory integrate-and-fire neurons with pulse coupling, thereby relating network inputs to evoked neuronal activity. Using this embedded mapping and experimentally feasible measurements of the firing rate as well as voltage dynamics in response to a relatively small ensemble of random input stimuli, we efficiently reconstruct the recurrent network connectivity via compressive sensing techniques. Through analogous analysis, we then recover high dimensional natural stimuli from evoked neuronal network dynamics over a short time horizon. This work provides a generalizable methodology for rapidly recovering sparse neuronal network data and underlines the natural role of sparsity in facilitating the efficient encoding of network data in neuronal dynamics.

Optimal phase-selective entrainment of heterogeneous oscillator ensembles

We develop a framework to design optimal entrainment signals that entrain an ensemble of heterogeneous nonlinear oscillators, described by phase models, at desired phases. We explicitly take into account heterogeneity in both oscillation frequency and the type of oscillators characterized by different Phase Response Curves. The central idea is to leverage the Fourier series representation of periodic functions to decode a phase-selective entrainment task into a quadratic program. We demonstrate our approach using a variety of phase models, where we entrain the oscillators into distinct phase patterns. Also, we show how the generalizability gained from our formulation enables us to meet a wide range of design objectives and constraints, such as minimum-power, fast entrainment, and charge-balanced controls.

Statistical inference links data and theory in network science

The number of network science applications across many different fields has been rapidly increasing. Surprisingly, the development of theory and domain-specific applications often occur in isolation, risking an effective disconnect between theoretical and methodological advances and the way network science is employed in practice. Here we address this risk constructively, discussing good practices to guarantee more successful applications and reproducible results. We endorse designing statistically grounded methodologies to address challenges in network science. This approach allows one to explain observational data in terms of generative models, naturally deal with intrinsic uncertainties, and strengthen the link between theory and applications.

Theoretical models and structures recovered from measured data serve for analysis of complex networks. The authors discuss here existing gaps between theoretical methods and real-world applied networks, and potential ways to improve the interplay between theory and applications.

Optimizing charge-balanced pulse stimulation for desynchronization

Collective synchronization in a large population of self-sustained units appears both in natural and engineered systems. Sometimes this effect is in demand, while in some cases, it is undesirable, which calls for control techniques. In this paper, we focus on pulsatile control, with the goal to either increase or decrease the level of synchrony. We quantify this level by the entropy of the phase distribution. Motivated by possible applications in neuroscience, we consider pulses of a realistic shape. Exploiting the noisy Kuramoto-Winfree model, we search for the optimal pulse profile and the optimal stimulation phase. For this purpose, we derive an expression for the change of the phase distribution entropy due to the stimulus. We relate this change to the properties of individual units characterized by generally different natural frequencies and phase response curves and the population's state. We verify the general result by analyzing a two-frequency population model and demonstrating a good agreement of the theory and numerical simulations.

Inferring the connectivity of coupled chaotic oscillators using Kalman filtering

Inferring the interactions between coupled oscillators is a significant open problem in complexity science, with multiple interdisciplinary applications. While the Kalman filter (KF) technique is a well-known tool, widely used for data assimilation and parameter estimation, to the best of our knowledge, it has not yet been used for inferring the connectivity of coupled chaotic oscillators. Here we demonstrate that KF allows reconstructing the interaction topology and the coupling strength of a network of mutually coupled Rössler-like chaotic oscillators. We show that the connectivity can be inferred by considering only the observed dynamics of a single variable of the three that define the phase space of each oscillator. We also show that both the coupling strength and the network architecture can be inferred even when the oscillators are close to synchronization. Simulation results are provided to show the effectiveness and applicability of the proposed method.

Estimating Transfer Entropy in Continuous Time Between Neural Spike Trains or Other Event-Based Data

Transfer entropy (TE) is a widely used measure of directed information flows in a number of domains including neuroscience. Many real-world time series for which we are interested in information flows come in the form of (near) instantaneous events occurring over time. Examples include the spiking of biological neurons, trades on stock markets and posts to social media, amongst myriad other systems involving events in continuous time throughout the natural and social sciences. However, there exist severe limitations to the current approach to TE estimation on such event-based data via discretising the time series into time bins: it is not consistent, has high bias, converges slowly and cannot simultaneously capture relationships that occur with very fine time precision as well as those that occur over long time intervals. Building on recent work which derived a theoretical framework for TE in continuous time, we present an estimation framework for TE on event-based data and develop a k-nearest-neighbours estimator within this framework. This estimator is provably consistent, has favourable bias properties and converges orders of magnitude more quickly than the current state-of-the-art in discrete-time estimation on synthetic examples. We demonstrate failures of the traditionally-used source-time-shift method for null surrogate generation. In order to overcome these failures, we develop a local permutation scheme for generating surrogate time series conforming to the appropriate null hypothesis in order to test for the statistical significance of the TE and, as such, test for the conditional independence between the history of one point process and the updates of another. Our approach is shown to be capable of correctly rejecting or accepting the null hypothesis of conditional independence even in the presence of strong pairwise time-directed correlations. This capacity to accurately test for conditional independence is further demonstrated on models of a spiking neural circuit inspired by the pyloric circuit of the crustacean stomatogastric ganglion, succeeding where previous related estimators have failed.

Impact of local network characteristics on network reconstruction

When a network is inferred from data, two types of errors can occur: false positive and false negative conclusions about the presence of links. We focus on the influence of local network characteristics on the probability α of false positive conclusions, and on the probability β of false negative conclusions, in the case of networks of coupled oscillators. We demonstrate that false conclusion probabilities are influenced by local connectivity measures such as the shortest path length and the detour degree, which can also be estimated from the inferred network when the true underlying network is not known a priori. These measures can then be used for quantification of the confidence level of link conclusions, and for improving the network reconstruction via advanced concepts of link weights thresholding.

Exploring node interaction relationship in complex networks by using high-frequency signal injection

Many practical systems can be described by complex networks. These networks produce, day and night, rich data which can be used to extract information from the systems. Often, output data of some nodes in the networks can be successfully measured and collected while the structures of networks producing these data are unknown. Thus, revealing network structures by analyzing available data, referred to as network reconstruction, turns to be an important task in many realistic problems. Limitation of measurable data is a very common challenge in network reconstruction. Here we consider an extreme case, i.e., we can only measure and process the data of a pair of nodes in a large network, and the task is to explore the relationship between these two nodes while all other nodes in the network are hidden. A driving-response approach is proposed to do so. By loading a high-frequency signal to a node (defined as node A), we can measure data of the partner node (node B), and work out the connection structure, such as the distance from node A to node B and the effective intensity of interaction from A to B, with the data of node B only. A systematical smoothing technique is suggested for treating noise problem. The approach has practical significance.

Interaction mechanisms quantified from dynamical features of frog choruses

We employ a mathematical model (a phase oscillator model) to describe the deterministic and stochastic features of frog choruses in which male frogs attempt to avoid call overlaps. The mathematical model with a general interaction term is identified using a Bayesian approach, and it qualitatively reproduces the stationary and dynamical features of the empirical data. In addition, we quantify the magnitude of attention paid among the male frogs from the identified model, and then analyse the relationship between attention and behavioural parameters using a statistical approach. Our analysis demonstrates a negative correlation between attention and inter-frog distance, and also suggests a behavioural strategy in which male frogs selectively attend to a less attractive male frog (i.e. a male producing calls at longer intervals) in order to more effectively advertise their superior relative attractiveness to females.

A practical method for estimating coupling functions in complex dynamical systems

A foremost challenge in modern network science is the inverse problem of reconstruction (inference) of coupling equations and network topology from the measurements of the network dynamics. Of particular interest are the methods that can operate on real (empirical) data without interfering with the system. One such earlier attempt (Tokuda et al. 2007 Phys. Rev. Lett. 99, 064101. (doi:10.1103/PhysRevLett.99.064101)) was a method suited for general limit-cycle oscillators, yielding both oscillators' natural frequencies and coupling functions between them (phase equations) from empirically measured time series. The present paper reviews the above method in a way comprehensive to domain-scientists other than physics. It also presents applications of the method to (i) detection of the network connectivity, (ii) inference of the phase sensitivity function, (iii) approximation of the interaction among phase-coherent chaotic oscillators, and (iv) experimental data from a forced Van der Pol electric circuit. This reaffirms the range of applicability of the method for reconstructing coupling functions and makes it accessible to a much wider scientific community. This article is part of the theme issue 'Coupling functions: dynamical interaction mechanisms in the physical, biological and social sciences'.

Model reconstruction from temporal data for coupled oscillator networks

In a complex system, the interactions between individual agents often lead to emergent collective behavior such as spontaneous synchronization, swarming, and pattern formation. Beyond the intrinsic properties of the agents, the topology of the network of interactions can have a dramatic influence over the dynamics. In many studies, researchers start with a specific model for both the intrinsic dynamics of each agent and the interaction network and attempt to learn about the dynamics of the model. Here, we consider the inverse problem: given data from a system, can one learn about the model and the underlying network? We investigate arbitrary networks of coupled phase oscillators that can exhibit both synchronous and asynchronous dynamics. We demonstrate that, given sufficient observational data on the transient evolution of each oscillator, machine learning can reconstruct the interaction network and identify the intrinsic dynamics.

Universality in the firing of minicolumnar-type neural networks

An open question in biological neural networks is whether changes in firing modalities are mainly an individual network property or whether networks follow a joint pathway. For the early developmental period, our study focusing on a simple network class of excitatory and inhibitory neurons suggests the following answer: Networks with considerable variation of topology and dynamical parameters follow a universal firing paradigm that evolves as the overall connectivity strength and firing level increase, as seen in the process of network maturation. A simple macroscopic model reproduces the main features of the paradigm as a result of the competition between the fundamental dynamical system notions of synchronization vs chaos and explains why in simulations the paradigm is robust regarding differences in network topology and largely independent from the neuron model used. The presented findings reflect the first dozen days of dissociated neuronal in vitro cultures (upon following the developmental period bears similarly universal features but is characterized by the processes of neuronal facilitation and depression that do not require to be considered for the first developmental period).

Reconstructing dynamical networks via feature ranking

Empirical data on real complex systems are becoming increasingly available. Parallel to this is the need for new methods of reconstructing (inferring) the structure of networks from time-resolved observations of their node-dynamics. The methods based on physical insights often rely on strong assumptions about the properties and dynamics of the scrutinized network. Here, we use the insights from machine learning to design a new method of network reconstruction that essentially makes no such assumptions. Specifically, we interpret the available trajectories (data) as "features" and use two independent feature ranking approaches-Random Forest and RReliefF-to rank the importance of each node for predicting the value of each other node, which yields the reconstructed adjacency matrix. We show that our method is fairly robust to coupling strength, system size, trajectory length, and noise. We also find that the reconstruction quality strongly depends on the dynamical regime.

Reconstruction of dynamic networks with time-delayed interactions in the presence of fast-varying noises

Most complex social, biological and technological systems can be described by dynamic networks. Reconstructing network structures from measurable data is a fundamental problem in almost all interdisciplinary fields. Network nodes interact with each other and those interactions often have diversely distributed time delays. Accurate reconstruction of any targeted interaction to a node requires measured data of all its neighboring nodes together with information on the time delays of interactions from these neighbors. When networks are large, these data are often not available and time-delay factors are deeply hidden. Here we show that fast-varying noise can be of great help in solving these challenging problems. By computing suitable correlations, we can infer the intensity and time delay of any targeted interaction with the data of two related nodes (driving and driven nodes) only while all other nodes in the network are hidden. This method is analytically derived and fully justified by extensive numerical simulations.

Inferring directed networks using a rank-based connectivity measure

Inferring the topology of a network using the knowledge of the signals of each of the interacting units is key to understanding real-world systems. One way to address this problem is using data-driven methods like cross-correlation or mutual information. However, these measures lack the ability to distinguish the direction of coupling. Here, we use a rank-based nonlinear interdependence measure originally developed for pairs of signals. This measure not only allows one to measure the strength but also the direction of the coupling. Our results for a system of coupled Lorenz dynamics show that we are able to consistently infer the underlying network for a subrange of the coupling strength and link density. Furthermore, we report that the addition of dynamical noise can benefit the reconstruction. Finally, we show an application to multichannel electroencephalographic recordings from an epilepsy patient.

Inferring the phase response curve from observation of a continuously perturbed oscillator

Phase response curves are important for analysis and modeling of oscillatory dynamics in various applications, particularly in neuroscience. Standard experimental technique for determining them requires isolation of the system and application of a specifically designed input. However, isolation is not always feasible and we are compelled to observe the system in its natural environment under free-running conditions. To that end we propose an approach relying only on passive observations of the system and its input. We illustrate it with simulation results of an oscillator driven by a stochastic force.

Inferring Network Connectivity from Event Timing Patterns

Reconstructing network connectivity from the collective dynamics of a system typically requires access to its complete continuous-time evolution, although these are often experimentally inaccessible. Here we propose a theory for revealing physical connectivity of networked systems only from the event time series their intrinsic collective dynamics generate. Representing the patterns of event timings in an event space spanned by interevent and cross-event intervals, we reveal which other units directly influence the interevent times of any given unit. For illustration, we linearize an event-space mapping constructed from the spiking patterns in model neural circuits to reveal the presence or absence of synapses between any pair of neurons, as well as whether the coupling acts in an inhibiting or activating (excitatory) manner. The proposed model-independent reconstruction theory is scalable to larger networks and may thus play an important role in the reconstruction of networks from biology to social science and engineering.

Analytical approach to network inference: Investigating degree distribution

When the network is reconstructed, two types of errors can occur: false positive and false negative errors about the presence or absence of links. In this paper, the influence of these two errors on the vertex degree distribution is analytically analyzed. Moreover, an analytic formula of the density of the biased vertex degree distribution is found. In the inverse problem, we find a reliable procedure to reconstruct analytically the density of the vertex degree distribution of any network based on the inferred network and estimates for the false positive and false negative errors based on, e.g., simulation studies.

Functional network stability and average minimal distance - A framework to rapidly assess dynamics of functional network representations

BACKGROUND

Recent advances in neurophysiological recording techniques have increased both the spatial and temporal resolution of data. New methodologies are required that can handle large data sets in an efficient manner as well as to make quantifiable, and realistic, predictions about the global modality of the brain from under-sampled recordings.

NEW METHOD

To rectify both problems, we first propose an analytical modification to an existing functional connectivity algorithm, Average Minimal Distance (AMD), to rapidly capture functional network connectivity. We then complement this algorithm by introducing Functional Network Stability (FuNS), a metric that can be used to quickly assess the global network dynamic changes over time, without being constrained by the activities of a specific set of neurons.

RESULTS

We systematically test the performance of AMD and FuNS (1) on artificial spiking data with different statistical characteristics, (2) from spiking data generated using a neural network model, and (3) using in vivo data recorded from mouse hippocampus during fear learning. Our results show that AMD and FuNS are able to monitor the change in network dynamics during memory consolidation.

COMPARISON WITH OTHER METHODS

AMD outperforms traditional bootstrapping and cross-correlation (CC) methods in both significance and computation time. Simultaneously, FuNS provides a reliable way to establish a link between local structural network changes, global dynamics of network-wide representations activity, and behavior.

CONCLUSIONS

The AMD-FuNS framework should be universally useful in linking long time-scale, global network dynamics and cognitive behavior.

Robustness and versatility of a nonlinear interdependence method for directional coupling detection from spike trains

The detection of directional couplings between dynamics based on measured spike trains is a crucial problem in the understanding of many different systems. In particular, in neuroscience it is important to assess the connectivity between neurons. One of the approaches that can estimate directional coupling from the analysis of point processes is the nonlinear interdependence measure L. Although its efficacy has already been demonstrated, it still needs to be tested under more challenging and realistic conditions prior to an application to real data. Thus, in this paper we use the Hindmarsh-Rose model system to test the method in the presence of noise and for different spiking regimes. We also examine the influence of different parameters and spike train distances. Our results show that the measure L is versatile and robust to various types of noise, and thus suitable for application to experimental data.