Reinforcement learning for suppression of collective activity in oscillatory ensembles

Data-driven control of oscillator networks with population-level measurement

Controlling complex networks of nonlinear limit-cycle oscillators is an important problem pertinent to various applications in engineering and natural sciences. While in recent years the control of oscillator populations with comprehensive biophysical models or simplified models, e.g., phase models, has seen notable advances, learning appropriate controls directly from data without prior model assumptions or pre-existing data remains a challenging and less developed area of research. In this paper, we address this problem by leveraging the network's current dynamics to iteratively learn an appropriate control online without constructing a global model of the system. We illustrate through a range of numerical simulations that the proposed technique can effectively regulate synchrony in various oscillator networks after a small number of trials using only one input and one noisy population-level output measurement. We provide a theoretical analysis of our approach, illustrate its robustness to system variations, and compare its performance with existing model-based and data-driven approaches.

Deep reinforcement learning with significant multiplications inference

We propose a sparse computation method for optimizing the inference of neural networks in reinforcement learning (RL) tasks. Motivated by the processing abilities of the brain, this method combines simple neural network pruning with a delta-network algorithm to account for the input data correlations. The former mimics neuroplasticity by eliminating inefficient connections; the latter makes it possible to update neuron states only when their changes exceed a certain threshold. This combination significantly reduces the number of multiplications during the neural network inference for fast neuromorphic computing. We tested the approach in popular deep RL tasks, yielding up to a 100-fold reduction in the number of required multiplications without substantial performance loss (sometimes, the performance even improved).

Desynchronizing oscillators coupled in multi-cluster networks through adaptively controlling partial networks

This article introduces an adaptive control scheme with a feedback delay, specifically designed for controlling partial networks, to achieve desynchronization in a coupled network with two or multiple clusters. The proposed scheme's effectiveness is validated through several representative examples of coupled neuronal networks with two interconnected clusters. The efficacy of this scheme is attributed to the rigorous and numerical analyses on the corresponding transcendental characteristic equation, which includes time delay and other network parameters. In addition to investigating the impact of time delay and inter-connectivity on the stability of an incoherent state, we also rigorously find that controlling only one cluster cannot realize the desynchronization in the coupled oscillators within three or more clusters. All these, we believe, can deepen the understanding of the deep brain stimulation techniques presently used in the clinical treatment of neurodegenerative diseases and suggest future avenues for enhancing these clinical techniques through adaptive feedback settings.

Discovering Themes in Deep Brain Stimulation Research Using Explainable Artificial Intelligence

Deep brain stimulation is a treatment that controls symptoms by changing brain activity. The complexity of how to best treat brain dysfunction with deep brain stimulation has spawned research into artificial intelligence approaches. Machine learning is a subset of artificial intelligence that uses computers to learn patterns in data and has many healthcare applications, such as an aid in diagnosis, personalized medicine, and clinical decision support. Yet, how machine learning models make decisions is often opaque. The spirit of explainable artificial intelligence is to use machine learning models that produce interpretable solutions. Here, we use topic modeling to synthesize recent literature on explainable artificial intelligence approaches to extracting domain knowledge from machine learning models relevant to deep brain stimulation. The results show that patient classification (i.e., diagnostic models, precision medicine) is the most common problem in deep brain stimulation studies that employ explainable artificial intelligence. Other topics concern attempts to optimize stimulation strategies and the importance of explainable methods. Overall, this review supports the potential for artificial intelligence to revolutionize deep brain stimulation by personalizing stimulation protocols and adapting stimulation in real time.

Dynamics of phase oscillator networks with synaptic weight and structural plasticity

We study the dynamics of Kuramoto oscillator networks with two distinct adaptation processes, one varying the coupling strengths and the other altering the network structure. Such systems model certain networks of oscillatory neurons where the neuronal dynamics, synaptic weights, and network structure interact with and shape each other. We model synaptic weight adaptation with spike-timing-dependent plasticity (STDP) that runs on a longer time scale than neuronal spiking. Structural changes that include addition and elimination of contacts occur at yet a longer time scale than the weight adaptations. First, we study the steady-state dynamics of Kuramoto networks that are bistable and can settle in synchronized or desynchronized states. To compare the impact of adding structural plasticity, we contrast the network with only STDP to one with a combination of STDP and structural plasticity. We show that the inclusion of structural plasticity optimizes the synchronized state of a network by allowing for synchronization with fewer links than a network with STDP alone. With non-identical units in the network, the addition of structural plasticity leads to the emergence of correlations between the oscillators' natural frequencies and node degrees. In the desynchronized regime, the structural plasticity decreases the number of contacts, leading to a sparse network. In this way, adding structural plasticity strengthens both synchronized and desynchronized states of a network. Second, we use desynchronizing coordinated reset stimulation and synchronizing periodic stimulation to induce desynchronized and synchronized states, respectively. Our findings indicate that a network with a combination of STDP and structural plasticity may require stronger and longer stimulation to switch between the states than a network with STDP only.

Optimizing charge-balanced pulse stimulation for desynchronization

Collective synchronization in a large population of self-sustained units appears both in natural and engineered systems. Sometimes this effect is in demand, while in some cases, it is undesirable, which calls for control techniques. In this paper, we focus on pulsatile control, with the goal to either increase or decrease the level of synchrony. We quantify this level by the entropy of the phase distribution. Motivated by possible applications in neuroscience, we consider pulses of a realistic shape. Exploiting the noisy Kuramoto-Winfree model, we search for the optimal pulse profile and the optimal stimulation phase. For this purpose, we derive an expression for the change of the phase distribution entropy due to the stimulus. We relate this change to the properties of individual units characterized by generally different natural frequencies and phase response curves and the population's state. We verify the general result by analyzing a two-frequency population model and demonstrating a good agreement of the theory and numerical simulations.

Eliminating synchronization of coupled neurons adaptively by using feedback coupling with heterogeneous delays

In this paper, we present an adaptive scheme involving heterogeneous delay interactions to suppress synchronization in a large population of oscillators. We analytically investigate the incoherent state stability regions for several specific kinds of distributions for delays. Interestingly, we find that, among the distributions that we discuss, the exponential distribution may offer great convenience to the performance of our adaptive scheme because this distribution renders an unbounded stability region. Moreover, we demonstrate our scheme in the realization of synchronization elimination in some representative, realistic neuronal networks, which makes it possible to deepen the understanding and even refine the existing techniques of deep brain stimulation in the treatment of some synchronization-induced mental disorders.

Controlling collective synchrony in oscillatory ensembles by precisely timed pulses

We present an efficient technique for control of synchrony in a globally coupled ensemble by pulsatile action. We assume that we can observe the collective oscillation and can stimulate all elements of the ensemble simultaneously. We pay special attention to the minimization of intervention into the system. The key idea is to stimulate only at the most sensitive phase. To find this phase, we implement an adaptive feedback control. Estimating the instantaneous phase of the collective mode on the fly, we achieve efficient suppression using a few pulses per oscillatory cycle. We discuss the possible relevance of the results for neuroscience, namely, for the development of advanced algorithms for deep brain stimulation, a medical technique used to treat Parkinson's disease.