Energy-efficient information transfer at thalamocortical synapses

Signatures of Bayesian inference emerge from energy-efficient synapses

Biological synaptic transmission is unreliable, and this unreliability likely degrades neural circuit performance. While there are biophysical mechanisms that can increase reliability, for instance by increasing vesicle release probability, these mechanisms cost energy. We examined four such mechanisms along with the associated scaling of the energetic costs. We then embedded these energetic costs for reliability in artificial neural networks (ANNs) with trainable stochastic synapses, and trained these networks on standard image classification tasks. The resulting networks revealed a tradeoff between circuit performance and the energetic cost of synaptic reliability. Additionally, the optimised networks exhibited two testable predictions consistent with pre-existing experimental data. Specifically, synapses with lower variability tended to have (1) higher input firing rates and (2) lower learning rates. Surprisingly, these predictions also arise when synapse statistics are inferred through Bayesian inference. Indeed, we were able to find a formal, theoretical link between the performance-reliability cost tradeoff and Bayesian inference. This connection suggests two incompatible possibilities: evolution may have chanced upon a scheme for implementing Bayesian inference by optimising energy efficiency, or alternatively, energy-efficient synapses may display signatures of Bayesian inference without actually using Bayes to reason about uncertainty.

Information encoded in volumes and areas of dendritic spines is nearly maximal across mammalian brains

Many experiments suggest that long-term information associated with neuronal memory resides collectively in dendritic spines. However, spines can have a limited size due to metabolic and neuroanatomical constraints, which should effectively limit the amount of encoded information in excitatory synapses. This study investigates how much information can be stored in the population of sizes of dendritic spines, and whether it is optimal in any sense. It is shown here, using empirical data for several mammalian brains across different regions and physiological conditions, that dendritic spines nearly maximize entropy contained in their volumes and surface areas for a given mean size in cortical and hippocampal regions. Although both short- and heavy-tailed fitting distributions approach [Formula: see text] of maximal entropy in the majority of cases, the best maximization is obtained primarily for short-tailed gamma distribution. We find that most empirical ratios of standard deviation to mean for spine volumes and areas are in the range [Formula: see text], which is close to the theoretical optimal ratios coming from entropy maximization for gamma and lognormal distributions. On average, the highest entropy is contained in spine length ([Formula: see text] bits per spine), and the lowest in spine volume and area ([Formula: see text] bits), although the latter two are closer to optimality. In contrast, we find that entropy density (entropy per spine size) is always suboptimal. Our results suggest that spine sizes are almost as random as possible given the constraint on their size, and moreover the general principle of entropy maximization is applicable and potentially useful to information and memory storing in the population of cortical and hippocampal excitatory synapses, and to predicting their morphological properties.

Paying the brain's energy bill

How have animals managed to maintain metabolically expensive brains given the volatile and fleeting availability of calories in the natural world? Here we review studies in support of three strategies that involve: 1) a reallocation of energy from peripheral tissues and functions to cover the costs of the brain, 2) an implementation of energy-efficient neural coding, enabling the brain to operate at reduced energy costs, and 3) efficient use of costly neural resources during food scarcity. Collectively, these studies reveal a heterogeneous set of energy-saving mechanisms that make energy-costly brains fit for survival.

Entropy, Amnesia, and Abnormal Déjà Experiences

Previous research has contrasted fleeting erroneous experiences of familiarity with equally convincing, and often more stubborn erroneous experiences of remembering. While a subset of the former category may present as nonpathological “déjà vu,” the latter, termed “déjà vécu” can categorize a delusion-like confabulatory phenomenon first described in elderly dementia patients. Leading explanations for this experience include the dual process view, in which erroneous familiarity and erroneous recollection are elicited by inappropriate activation of the parahippocampal cortex and the hippocampus, respectively, and the more popular encoding-as-retrieval explanation in which normal memory encoding processes are falsely flagged and interpreted as memory retrieval. This paper presents a novel understanding of this recollective confabulation that builds on the encoding-as-retrieval hypothesis but more adequately accounts for the co-occurrence of persistent déjà vécu with both perceptual novelty and memory impairment, the latter of which occurs not only in progressive dementia but also in transient epileptic amnesia (TEA) and psychosis. It makes use of the growing interdisciplinary understanding of the fluidity of time and posits that the functioning of memory and the perception of novelty, long known to influence the subjective experience of time, may have a more fundamental effect on the flow of time.

Associations between executive function impairment and biochemical abnormalities in depressed adolescents with non-suicidal self-injury

BACKGROUND

H protons magnetic resonance spectroscopy (¹H-MRS) has been used to detect the biochemical metabolism changes and the mechanism of executive dysfunction in major depressive disorder (MDD). While, finding information associated with non-suicidal self-injury (NSSI) among adolescents with MDD is challenging. The present study aimed to examine the executive function and biochemical metabolism alterations, as well as to elucidate their associations in depressed adolescents with NSSI.

METHODS

A total of 86 adolescents with MDD (40 with NSSI, and 46 without NSSI) and 28 healthy controls were recruited in the current study. The executive function was assessed by Digital symbol test (DST), Wisconsin Card Sorting Test (WCST), Trail Making Test, part B (TMT-B), and Verbal fluency (VF). Bilateral metabolite levels of the prefrontal cortex (PFC), anterior cingulated cortex (ACC), lenticular nucleus (LN) of basal ganglia and thalamus were obtained by ¹H-MRS at 3.0 T, and then the ratios of N-acetyl aspartate (NAA) and choline-containing compounds (Cho) to creatine (Cr) were determined, respectively. Finally, association analysis was conducted to investigate their relationships.

RESULTS

The depressed adolescents with NSSI showed significantly lower VF scores than those without NSSI and healthy controls. We also found significantly higher NAA/Cr ratios in the right thalamus, while significantly lower Cho/Cr ratios in the right thalamus of NSSI group than the MDD without NSSI group and healthy controls. And NSSI group also showed lower NAA/Cr ratio in the right LN than the MDD without NSSI group. For MDD with NSSI, the NAA/Cr ratios of the left thalamus were positively correlated with the time of TMTB and the Cho/Cr ratios of the left ACC were positively correlated with the VF scores.

CONCLUSIONS

Depressed adolescents with NSSI may have executive dysfunction and NAA and Cho metabolism abnormalities in the thalamus. And the NAA/Cr ratios of the right LN could distinguish NSSI from depressed adolescents. Further, the executive dysfunction may be associated with the abnormal NAA metabolism in the left thalamus and ACC.

Emergence and fragmentation of the alpha-band driven by neuronal network dynamics

Rhythmic neuronal network activity underlies brain oscillations. To investigate how connected neuronal networks contribute to the emergence of the α-band and to the regulation of Up and Down states, we study a model based on synaptic short-term depression-facilitation with afterhyperpolarization (AHP). We found that the α-band is generated by the network behavior near the attractor of the Up-state. Coupling inhibitory and excitatory networks by reciprocal connections leads to the emergence of a stable α-band during the Up states, as reflected in the spectrogram. To better characterize the emergence and stability of thalamocortical oscillations containing α and δ rhythms during anesthesia, we model the interaction of two excitatory networks with one inhibitory network, showing that this minimal topology underlies the generation of a persistent α-band in the neuronal voltage characterized by dominant Up over Down states. Finally, we show that the emergence of the α-band appears when external inputs are suppressed, while fragmentation occurs at small synaptic noise or with increasing inhibitory inputs. To conclude, α-oscillations could result from the synaptic dynamics of interacting excitatory neuronal networks with and without AHP, a principle that could apply to other rhythms.

Brain oscillations, recorded from electroencephalograms characterize behaviors such as sleep, wakefulness, brain evoked responses, coma or anesthesia. The underlying rhythms for these oscillations are associated at a neuronal population level to fluctuations of the membrane potential between Up (depolarized) and Down (hyperpolarized) states. During anesthesia with propofol, a dominant α-band (8–12Hz) can emerge or disappear, but the underlying mechanism remains unclear. Using modeling, we report that the α-band appears during Up states in neuronal populations driven by short-term synaptic plasticity and synaptic noise. Moreover, we show that three connected neuronal networks representing the thalamocortical loop reproduce the dynamics of the α-band, which emerges following the arrest of excitatory stimulations, but that can disappear by increasing inhibitory inputs. To conclude, short-term plasticity in well connected neuronal networks can explain the emergence and fragmentation of the α-band.

Presynaptic stochasticity improves energy efficiency and helps alleviate the stability-plasticity dilemma

When an action potential arrives at a synapse there is a large probability that no neurotransmitter is released. Surprisingly, simple computational models suggest that these synaptic failures enable information processing at lower metabolic costs. However, these models only consider information transmission at single synapses ignoring the remainder of the neural network as well as its overall computational goal. Here, we investigate how synaptic failures affect the energy efficiency of models of entire neural networks that solve a goal-driven task. We find that presynaptic stochasticity and plasticity improve energy efficiency and show that the network allocates most energy to a sparse subset of important synapses. We demonstrate that stabilising these synapses helps to alleviate the stability-plasticity dilemma, thus connecting a presynaptic notion of importance to a computational role in lifelong learning. Overall, our findings present a set of hypotheses for how presynaptic plasticity and stochasticity contribute to sparsity, energy efficiency and improved trade-offs in the stability-plasticity dilemma.

Optimising the energetic cost of the glutamatergic synapse

As for electronic computation, neural information processing is energetically expensive. This is because information is coded in the brain as membrane voltage changes, which are generated largely by passive ion movements down electrochemical gradients, and these ion movements later need to be reversed by active ATP-dependent ion pumping. This article will review how much of the energetic cost of the brain reflects the activity of glutamatergic synapses, consider the relative amount of energy used pre- and postsynaptically, outline how evolution has energetically optimised synapse function by adjusting the presynaptic release probability and the postsynaptic number of glutamate receptors, and speculate on how energy use by synapses may be sensed and adjusted.

Artificial neurovascular network (ANVN) to study the accuracy vs. efficiency trade-off in an energy dependent neural network

Artificial feedforward neural networks perform a wide variety of classification and function approximation tasks with high accuracy. Unlike their artificial counterparts, biological neural networks require a supply of adequate energy delivered to single neurons by a network of cerebral microvessels. Since energy is a limited resource, a natural question is whether the cerebrovascular network is capable of ensuring maximum performance of the neural network while consuming minimum energy? Should the cerebrovascular network also be trained, along with the neural network, to achieve such an optimum? In order to answer the above questions in a simplified modeling setting, we constructed an Artificial Neurovascular Network (ANVN) comprising a multilayered perceptron (MLP) connected to a vascular tree structure. The root node of the vascular tree structure is connected to an energy source, and the terminal nodes of the vascular tree supply energy to the hidden neurons of the MLP. The energy delivered by the terminal vascular nodes to the hidden neurons determines the biases of the hidden neurons. The "weights" on the branches of the vascular tree depict the energy distribution from the parent node to the child nodes. The vascular weights are updated by a kind of "backpropagation" of the energy demand error generated by the hidden neurons. We observed that higher performance was achieved at lower energy levels when the vascular network was also trained along with the neural network. This indicates that the vascular network needs to be trained to ensure efficient neural performance. We observed that below a certain network size, the energetic dynamics of the network in the per capita energy consumption vs. classification accuracy space approaches a fixed-point attractor for various initial conditions. Once the number of hidden neurons increases beyond a threshold, the fixed point appears to vanish, giving place to a line of attractors. The model also showed that when there is a limited resource, the energy consumption of neurons is strongly correlated to their individual contribution to the network's performance.

An Information-Theoretic Framework to Measure the Dynamic Interaction Between Neural Spike Trains

OBJECTIVE

While understanding the interaction patterns among simultaneous recordings of spike trains from multiple neuronal units is a key topic in neuroscience, existing methods either do not consider the inherent point-process nature of spike trains or are based on parametric assumptions. This work presents an information-theoretic framework for the model-free, continuous-time estimation of both undirected (symmetric) and directed (Granger-causal) interactions between spike trains.

METHODS

The framework computes the mutual information rate (MIR) and the transfer entropy rate (TER) for two point processes X and Y, showing that the MIR between X and Y can be decomposed as the sum of the TER along the directions X → Y and Y → X. We present theoretical expressions and introduce strategies to estimate efficiently the two measures through nearest neighbor statistics.

RESULTS

Using simulations of independent and coupled point processes, we show the accuracy of MIR and TER to assess interactions even for weakly coupled and short realizations, and demonstrate the superiority of continuous-time estimation over the standard discrete-time approach. We also apply the MIR and TER to real-world data, specifically, recordings from in-vitro preparations of spontaneously-growing cultures of cortical neurons. Using this dataset, we demonstrate the ability of MIR and TER to describe how the functional networks between recording units emerge over the course of the maturation of the neuronal cultures.

CONCLUSION AND SIGNIFICANCE

the proposed framework provides principled measures to assess undirected and directed spike train interactions with more efficiency and flexibility than previous discrete-time or parametric approaches, opening new perspectives for the analysis of point-process data in neuroscience and many other fields.

Estimating Transfer Entropy in Continuous Time Between Neural Spike Trains or Other Event-Based Data

Transfer entropy (TE) is a widely used measure of directed information flows in a number of domains including neuroscience. Many real-world time series for which we are interested in information flows come in the form of (near) instantaneous events occurring over time. Examples include the spiking of biological neurons, trades on stock markets and posts to social media, amongst myriad other systems involving events in continuous time throughout the natural and social sciences. However, there exist severe limitations to the current approach to TE estimation on such event-based data via discretising the time series into time bins: it is not consistent, has high bias, converges slowly and cannot simultaneously capture relationships that occur with very fine time precision as well as those that occur over long time intervals. Building on recent work which derived a theoretical framework for TE in continuous time, we present an estimation framework for TE on event-based data and develop a k-nearest-neighbours estimator within this framework. This estimator is provably consistent, has favourable bias properties and converges orders of magnitude more quickly than the current state-of-the-art in discrete-time estimation on synthetic examples. We demonstrate failures of the traditionally-used source-time-shift method for null surrogate generation. In order to overcome these failures, we develop a local permutation scheme for generating surrogate time series conforming to the appropriate null hypothesis in order to test for the statistical significance of the TE and, as such, test for the conditional independence between the history of one point process and the updates of another. Our approach is shown to be capable of correctly rejecting or accepting the null hypothesis of conditional independence even in the presence of strong pairwise time-directed correlations. This capacity to accurately test for conditional independence is further demonstrated on models of a spiking neural circuit inspired by the pyloric circuit of the crustacean stomatogastric ganglion, succeeding where previous related estimators have failed.

Information Processing in the Brain as Optimal Entropy Transport: A Theoretical Approach

We consider brain activity from an information theoretic perspective. We analyze the information processing in the brain, considering the optimality of Shannon entropy transport using the Monge–Kantorovich framework. It is proposed that some of these processes satisfy an optimal transport of informational entropy condition. This optimality condition allows us to derive an equation of the Monge–Ampère type for the information flow that accounts for the branching structure of neurons via the linearization of this equation. Based on this fact, we discuss a version of Murray’s law in this context.

Local Design Principles at Hippocampal Synapses Revealed by an Energy-Information Trade-Off

Synapses across different brain regions display distinct structure-function relationships. We investigated the interplay of fundamental design constraints that shape the transmission properties of the excitatory CA3-CA1 pyramidal cell connection, a prototypic synapse for studying the mechanisms of learning in the mammalian hippocampus. This small synapse is characterized by probabilistic release of transmitter, which is markedly facilitated in response to naturally occurring trains of action potentials. Based on a physiologically motivated computational model of the rat CA3 presynaptic terminal, we show how unreliability and short-term dynamics of vesicular release work together to regulate the trade-off of information transfer versus energy use. We propose that individual CA3-CA1 synapses are designed to operate near the maximum possible capacity of information transmission in an efficient manner. Experimental measurements reveal a wide range of vesicular release probabilities at hippocampal synapses, which may be a necessary consequence of long-term plasticity and homeostatic mechanisms that manifest as presynaptic modifications of the release probability. We show that the timescales and magnitude of short-term plasticity (STP) render synaptic information transfer nearly independent of differences in release probability. Thus, individual synapses transmit optimally while maintaining a heterogeneous distribution of presynaptic strengths indicative of synaptically-encoded memory representations. Our results support the view that organizing principles that are evident on higher scales of neural organization percolate down to the design of an individual synapse.

The effect of inhibition on rate code efficiency indicators

In this paper we investigate the rate coding capabilities of neurons whose input signal are alterations of the base state of balanced inhibitory and excitatory synaptic currents. We consider different regimes of excitation-inhibition relationship and an established conductance-based leaky integrator model with adaptive threshold and parameter sets recreating biologically relevant spiking regimes. We find that given mean post-synaptic firing rate, counter-intuitively, increased ratio of inhibition to excitation generally leads to higher signal to noise ratio (SNR). On the other hand, the inhibitory input significantly reduces the dynamic coding range of the neuron. We quantify the joint effect of SNR and dynamic coding range by computing the metabolic efficiency-the maximal amount of information per one ATP molecule expended (in bits/ATP). Moreover, by calculating the metabolic efficiency we are able to predict the shapes of the post-synaptic firing rate histograms that may be tested on experimental data. Likewise, optimal stimulus input distributions are predicted, however, we show that the optimum can essentially be reached with a broad range of input distributions. Finally, we examine which parameters of the used neuronal model are the most important for the metabolically efficient information transfer.