Rapid Neural Coding in the Retina with Relative Spike Latencies

High-performance deep spiking neural networks with 0.3 spikes per neuron

Communication by rare, binary spikes is a key factor for the energy efficiency of biological brains. However, it is harder to train biologically-inspired spiking neural networks than artificial neural networks. This is puzzling given that theoretical results provide exact mapping algorithms from artificial to spiking neural networks with time-to-first-spike coding. In this paper we analyze in theory and simulation the learning dynamics of time-to-first-spike-networks and identify a specific instance of the vanishing-or-exploding gradient problem. While two choices of spiking neural network mappings solve this problem at initialization, only the one with a constant slope of the neuron membrane potential at threshold guarantees the equivalence of the training trajectory between spiking and artificial neural networks with rectified linear units. For specific image classification architectures comprising feed-forward dense or convolutional layers, we demonstrate that deep spiking neural network models can be effectively trained from scratch on MNIST and Fashion-MNIST datasets, or fine-tuned on large-scale datasets, such as CIFAR10, CIFAR100 and PLACES365, to achieve the exact same performance as that of artificial neural networks, surpassing previous spiking neural networks. Our approach accomplishes high-performance classification with less than 0.3 spikes per neuron, lending itself for an energy-efficient implementation. We also show that fine-tuning spiking neural networks with our robust gradient descent algorithm enables their optimization for hardware implementations with low latency and resilience to noise and quantization.

Decoding dynamic visual scenes across the brain hierarchy

Understanding the computational mechanisms that underlie the encoding and decoding of environmental stimuli is a crucial investigation in neuroscience. Central to this pursuit is the exploration of how the brain represents visual information across its hierarchical architecture. A prominent challenge resides in discerning the neural underpinnings of the processing of dynamic natural visual scenes. Although considerable research efforts have been made to characterize individual components of the visual pathway, a systematic understanding of the distinctive neural coding associated with visual stimuli, as they traverse this hierarchical landscape, remains elusive. In this study, we leverage the comprehensive Allen Visual Coding-Neuropixels dataset and utilize the capabilities of deep learning neural network models to study neural coding in response to dynamic natural visual scenes across an expansive array of brain regions. Our study reveals that our decoding model adeptly deciphers visual scenes from neural spiking patterns exhibited within each distinct brain area. A compelling observation arises from the comparative analysis of decoding performances, which manifests as a notable encoding proficiency within the visual cortex and subcortical nuclei, in contrast to a relatively reduced encoding activity within hippocampal neurons. Strikingly, our results unveil a robust correlation between our decoding metrics and well-established anatomical and functional hierarchy indexes. These findings corroborate existing knowledge in visual coding related to artificial visual stimuli and illuminate the functional role of these deeper brain regions using dynamic stimuli. Consequently, our results suggest a novel perspective on the utility of decoding neural network models as a metric for quantifying the encoding quality of dynamic natural visual scenes represented by neural responses, thereby advancing our comprehension of visual coding within the complex hierarchy of the brain.

Gap junctions fine-tune ganglion cell signals to equalize response kinetics within a given electrically coupled array

Retinal ganglion cells (RGCs) summate inputs and forward a spike train code to the brain in the form of either maintained spiking (sustained) or a quickly decaying brief spike burst (transient). We report diverse response transience values across the RGC population and, contrary to the conventional transient/sustained scheme, responses with intermediary characteristics are the most abundant. Pharmacological tests showed that besides GABAergic inhibition, gap junction (GJ)-mediated excitation also plays a pivotal role in shaping response transience and thus visual coding. More precisely GJs connecting RGCs to nearby amacrine and RGCs play a defining role in the process. These GJs equalize kinetic features, including the response transience of transient OFF alpha (tOFFα) RGCs across a coupled array. We propose that GJs in other coupled neuron ensembles in the brain are also critical in the harmonization of response kinetics to enhance the population code and suit a corresponding task.

The mechanism of human color vision and potential implanted devices for artificial color vision

Vision plays a major role in perceiving external stimuli and information in our daily lives. The neural mechanism of color vision is complicated, involving the co-ordinated functions of a variety of cells, such as retinal cells and lateral geniculate nucleus cells, as well as multiple levels of the visual cortex. In this work, we reviewed the history of experimental and theoretical studies on this issue, from the fundamental functions of the individual cells of the visual system to the coding in the transmission of neural signals and sophisticated brain processes at different levels. We discuss various hypotheses, models, and theories related to the color vision mechanism and present some suggestions for developing novel implanted devices that may help restore color vision in visually impaired people or introduce artificial color vision to those who need it.

Sampling bias corrections for accurate neural measures of redundant, unique, and synergistic information

Shannon Information theory has long been a tool of choice to measure empirically how populations of neurons in the brain encode information about cognitive variables. Recently, Partial Information Decomposition (PID) has emerged as principled way to break down this information into components identifying not only the unique information carried by each neuron, but also whether relationships between neurons generate synergistic or redundant information. While it has been long recognized that Shannon information measures on neural activity suffer from a (mostly upward) limited sampling estimation bias, this issue has largely been ignored in the burgeoning field of PID analysis of neural activity. We used simulations to investigate the limited sampling bias of PID computed from discrete probabilities (suited to describe neural spiking activity). We found that PID suffers from a large bias that is uneven across components, with synergy by far the most biased. Using approximate analytical expansions, we found that the bias of synergy increases quadratically with the number of discrete responses of each neuron, whereas the bias of unique and redundant information increase only linearly or sub-linearly. Based on the understanding of the PID bias properties, we developed simple yet effective procedures that correct for the bias effectively, and that improve greatly the PID estimation with respect to current state-of-the-art procedures. We apply these PID bias correction procedures to datasets of 53117 pairs neurons in auditory cortex, posterior parietal cortex and hippocampus of mice performing cognitive tasks, deriving precise estimates and bounds of how synergy and redundancy vary across these brain regions.

An artificial visual neuron with multiplexed rate and time-to-first-spike coding

Human visual neurons rely on event-driven, energy-efficient spikes for communication, while silicon image sensors do not. The energy-budget mismatch between biological systems and machine vision technology has inspired the development of artificial visual neurons for use in spiking neural network (SNN). However, the lack of multiplexed data coding schemes reduces the ability of artificial visual neurons in SNN to emulate the visual perception ability of biological systems. Here, we present an artificial visual spiking neuron that enables rate and temporal fusion (RTF) coding of external visual information. The artificial neuron can code visual information at different spiking frequencies (rate coding) and enables precise and energy-efficient time-to-first-spike (TTFS) coding. This multiplexed sensory coding scheme could improve the computing capability and efficacy of artificial visual neurons. A hardware-based SNN with the RTF coding scheme exhibits good consistency with real-world ground truth data and achieves highly accurate steering and speed predictions for self-driving vehicles in complex conditions. The multiplexed RTF coding scheme demonstrates the feasibility of developing highly efficient spike-based neuromorphic hardware.

Representational formats of human memory traces

Neural representations are internal brain states that constitute the brain's model of the external world or some of its features. In the presence of sensory input, a representation may reflect various properties of this input. When perceptual information is no longer available, the brain can still activate representations of previously experienced episodes due to the formation of memory traces. In this review, we aim at characterizing the nature of neural memory representations and how they can be assessed with cognitive neuroscience methods, mainly focusing on neuroimaging. We discuss how multivariate analysis techniques such as representational similarity analysis (RSA) and deep neural networks (DNNs) can be leveraged to gain insights into the structure of neural representations and their different representational formats. We provide several examples of recent studies which demonstrate that we are able to not only measure memory representations using RSA but are also able to investigate their multiple formats using DNNs. We demonstrate that in addition to slow generalization during consolidation, memory representations are subject to semantization already during short-term memory, by revealing a shift from visual to semantic format. In addition to perceptual and conceptual formats, we describe the impact of affective evaluations as an additional dimension of episodic memories. Overall, these studies illustrate how the analysis of neural representations may help us gain a deeper understanding of the nature of human memory.

An actor-model framework for visual sensory encoding

A fundamental challenge in neuroengineering is determining a proper artificial input to a sensory system that yields the desired perception. In neuroprosthetics, this process is known as artificial sensory encoding, and it holds a crucial role in prosthetic devices restoring sensory perception in individuals with disabilities. For example, in visual prostheses, one key aspect of artificial image encoding is to downsample images captured by a camera to a size matching the number of inputs and resolution of the prosthesis. Here, we show that downsampling an image using the inherent computation of the retinal network yields better performance compared to learning-free downsampling methods. We have validated a learning-based approach (actor-model framework) that exploits the signal transformation from photoreceptors to retinal ganglion cells measured in explanted mouse retinas. The actor-model framework generates downsampled images eliciting a neuronal response in-silico and ex-vivo with higher neuronal reliability than the one produced by a learning-free approach. During the learning process, the actor network learns to optimize contrast and the kernel's weights. This methodological approach might guide future artificial image encoding strategies for visual prostheses. Ultimately, this framework could be applicable for encoding strategies in other sensory prostheses such as cochlear or limb.

Brain-computer interface paradigms and neural coding

Brain signal patterns generated in the central nervous system of brain-computer interface (BCI) users are closely related to BCI paradigms and neural coding. In BCI systems, BCI paradigms and neural coding are critical elements for BCI research. However, so far there have been few references that clearly and systematically elaborated on the definition and design principles of the BCI paradigm as well as the definition and modeling principles of BCI neural coding. Therefore, these contents are expounded and the existing main BCI paradigms and neural coding are introduced in the review. Finally, the challenges and future research directions of BCI paradigm and neural coding were discussed, including user-centered design and evaluation for BCI paradigms and neural coding, revolutionizing the traditional BCI paradigms, breaking through the existing techniques for collecting brain signals and combining BCI technology with advanced AI technology to improve brain signal decoding performance. It is expected that the review will inspire innovative research and development of the BCI paradigm and neural coding.

Sparse-firing regularization methods for spiking neural networks with time-to-first-spike coding

The training of multilayer spiking neural networks (SNNs) using the error backpropagation algorithm has made significant progress in recent years. Among the various training schemes, the error backpropagation method that directly uses the firing time of neurons has attracted considerable attention because it can realize ideal temporal coding. This method uses time-to-first-spike (TTFS) coding, in which each neuron fires at most once, and this restriction on the number of firings enables information to be processed at a very low firing frequency. This low firing frequency increases the energy efficiency of information processing in SNNs. However, only an upper limit has been provided for TTFS-coded SNNs, and the information-processing capability of SNNs at lower firing frequencies has not been fully investigated. In this paper, we propose two spike-timing-based sparse-firing (SSR) regularization methods to further reduce the firing frequency of TTFS-coded SNNs. Both methods are characterized by the fact that they only require information about the firing timing and associated weights. The effects of these regularization methods were investigated on the MNIST, Fashion-MNIST, and CIFAR-10 datasets using multilayer perceptron networks and convolutional neural network structures.

Functional Dissection of Ipsilateral and Contralateral Neural Activity Propagation Using Voltage-Sensitive Dye Imaging in Mouse Prefrontal Cortex

Prefrontal cortex (PFC) intrahemispheric activity and the interhemispheric connection have a significant impact on neuropsychiatric disorder pathology. This study aimed to generate a functional map of FC intrahemispheric and interhemispheric connections. Functional dissection of mouse PFCs was performed using the voltage-sensitive dye (VSD) imaging method with high speed (1 ms/frame), high resolution (256 × 256 pixels), and a large field of view (∼10 mm). Acute serial 350 μm slices were prepared from the bregma covering the PFC and numbered 1-5 based on their distance from the bregma (i.e., 1.70, 1.34, 0.98, 0.62, and 0.26 mm) with reference to the Mouse Brain Atlas (Paxinos and Franklin, 2008). The neural response to electrical stimulation was measured at nine sites and then averaged, and a functional map of the propagation patterns was created. Intracortical propagation was observed in slices 3-5, encompassing the anterior cingulate cortex (ACC) and corpus callosum (CC). The activity reached area 33 of the ACC. Direct white matter stimulation activated area 33 in both hemispheres. Similar findings were obtained via DiI staining of the CC. Imaging analysis revealed directional biases in neural signals traveling within the ACC, whereby the signal transmission speed and probability varied based on the signal direction. Specifically, the spread of neural signals from cg2 to cg1 was stronger than that from cingulate cortex area 1(cg1) to cingulate cortex area 2(cg2), which has implications for interhemispheric functional connections. These findings highlight the importance of understanding the PFC functional anatomy in evaluating neuromodulators like serotonin and dopamine, as well as other factors related to neuropsychiatric diseases.

Awake responses suggest inefficient dense coding in the mouse retina

The structure and function of the vertebrate retina have been extensively studied across species with an isolated, ex vivo preparation. Retinal function in vivo, however, remains elusive, especially in awake animals. Here, we performed single-unit extracellular recordings in the optic tract of head-fixed mice to compare the output of awake, anesthetized, and ex vivo retinas. While the visual response properties were overall similar across conditions, we found that awake retinal output had in general (1) faster kinetics with less variability in the response latencies; (2) a larger dynamic range; and (3) higher firing activity, by ~20 Hz on average, for both baseline and visually evoked responses. Our modeling analyses further showed that such awake response patterns convey comparable total information but less efficiently, and allow for a linear population decoder to perform significantly better than the anesthetized or ex vivo responses. These results highlight distinct retinal behavior in awake states, in particular suggesting that the retina employs dense coding in vivo, rather than sparse efficient coding as has been often assumed from ex vivo studies.

An exact mapping from ReLU networks to spiking neural networks

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an exact mapping from a network with Rectified Linear Units (ReLUs) to an SNN that fires exactly one spike per neuron. For our constructive proof, we assume that an arbitrary multi-layer ReLU network with or without convolutional layers, batch normalization and max pooling layers was trained to high performance on some training set. Furthermore, we assume that we have access to a representative example of input data used during training and to the exact parameters (weights and biases) of the trained ReLU network. The mapping from deep ReLU networks to SNNs causes zero percent drop in accuracy on CIFAR10, CIFAR100 and the ImageNet-like data sets Places365 and PASS. More generally our work shows that an arbitrary deep ReLU network can be replaced by an energy-efficient single-spike neural network without any loss of performance.

SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence

Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency by introducing neural dynamics and spike properties. As the emerging spiking deep learning paradigm attracts increasing interest, traditional programming frameworks cannot meet the demands of the automatic differentiation, parallel computation acceleration, and high integration of processing neuromorphic datasets and deployment. In this work, we present the SpikingJelly framework to address the aforementioned dilemma. We contribute a full-stack toolkit for preprocessing neuromorphic datasets, building deep SNNs, optimizing their parameters, and deploying SNNs on neuromorphic chips. Compared to existing methods, the training of deep SNNs can be accelerated 11×, and the superior extensibility and flexibility of SpikingJelly enable users to accelerate custom models at low costs through multilevel inheritance and semiautomatic code generation. SpikingJelly paves the way for synthesizing truly energy-efficient SNN-based machine intelligence systems, which will enrich the ecology of neuromorphic computing.

First-spike coding promotes accurate and efficient spiking neural networks for discrete events with rich temporal structures

Spiking neural networks (SNNs) are well-suited to process asynchronous event-based data. Most of the existing SNNs use rate-coding schemes that focus on firing rate (FR), and so they generally ignore the spike timing in events. On the contrary, methods based on temporal coding, particularly time-to-first-spike (TTFS) coding, can be accurate and efficient but they are difficult to train. Currently, there is limited research on applying TTFS coding to real events, since traditional TTFS-based methods impose one-spike constraint, which is not realistic for event-based data. In this study, we present a novel decision-making strategy based on first-spike (FS) coding that encodes FS timings of the output neurons to investigate the role of the first-spike timing in classifying real-world event sequences with complex temporal structures. To achieve FS coding, we propose a novel surrogate gradient learning method for discrete spike trains. In the forward pass, output spikes are encoded into discrete times to generate FS times. In the backpropagation, we develop an error assignment method that propagates error from FS times to spikes through a Gaussian window, and then supervised learning for spikes is implemented through a surrogate gradient approach. Additional strategies are introduced to facilitate the training of FS timings, such as adding empty sequences and employing different parameters for different layers. We make a comprehensive comparison between FS and FR coding in the experiments. Our results show that FS coding achieves comparable accuracy to FR coding while leading to superior energy efficiency and distinct neuronal dynamics on data sequences with very rich temporal structures. Additionally, a longer time delay in the first spike leads to higher accuracy, indicating important information is encoded in the timing of the first spike.

Learning heterogeneous delays in a layer of spiking neurons for fast motion detection

The precise timing of spikes emitted by neurons plays a crucial role in shaping the response of efferent biological neurons. This temporal dimension of neural activity holds significant importance in understanding information processing in neurobiology, especially for the performance of neuromorphic hardware, such as event-based cameras. Nonetheless, many artificial neural models disregard this critical temporal dimension of neural activity. In this study, we present a model designed to efficiently detect temporal spiking motifs using a layer of spiking neurons equipped with heterogeneous synaptic delays. Our model capitalizes on the diverse synaptic delays present on the dendritic tree, enabling specific arrangements of temporally precise synaptic inputs to synchronize upon reaching the basal dendritic tree. We formalize this process as a time-invariant logistic regression, which can be trained using labeled data. To demonstrate its practical efficacy, we apply the model to naturalistic videos transformed into event streams, simulating the output of the biological retina or event-based cameras. To evaluate the robustness of the model in detecting visual motion, we conduct experiments by selectively pruning weights and demonstrate that the model remains efficient even under significantly reduced workloads. In conclusion, by providing a comprehensive, event-driven computational building block, the incorporation of heterogeneous delays has the potential to greatly improve the performance of future spiking neural network algorithms, particularly in the context of neuromorphic chips.

Two Sides of the Same Coin: Efficient and Predictive Neural Coding

Some visual properties are consistent across a wide range of environments, while other properties are more labile. The efficient coding hypothesis states that many of these regularities in the environment can be discarded from neural representations, thus allocating more of the brain's dynamic range to properties that are likely to vary. This paradigm is less clear about how the visual system prioritizes different pieces of information that vary across visual environments. One solution is to prioritize information that can be used to predict future events, particularly those that guide behavior. The relationship between the efficient coding and future prediction paradigms is an area of active investigation. In this review, we argue that these paradigms are complementary and often act on distinct components of the visual input. We also discuss how normative approaches to efficient coding and future prediction can be integrated.

Interpreting the retinal neural code for natural scenes: From computations to neurons

Understanding the circuit mechanisms of the visual code for natural scenes is a central goal of sensory neuroscience. We show that a three-layer network model predicts retinal natural scene responses with an accuracy nearing experimental limits. The model's internal structure is interpretable, as interneurons recorded separately and not modeled directly are highly correlated with model interneurons. Models fitted only to natural scenes reproduce a diverse set of phenomena related to motion encoding, adaptation, and predictive coding, establishing their ethological relevance to natural visual computation. A new approach decomposes the computations of model ganglion cells into the contributions of model interneurons, allowing automatic generation of new hypotheses for how interneurons with different spatiotemporal responses are combined to generate retinal computations, including predictive phenomena currently lacking an explanation. Our results demonstrate a unified and general approach to study the circuit mechanisms of ethological retinal computations under natural visual scenes.

Backbone spiking sequence as a basis for preplay, replay, and default states in human cortex

Sequences of spiking activity have been heavily implicated as potential substrates of memory formation and retrieval across many species. A parallel line of recent evidence also asserts that sequential activity may arise from and be constrained by pre-existing network structure. Here we reconcile these two lines of research in the human brain by measuring single unit spiking sequences in the temporal lobe cortex as participants perform an episodic memory task. We find the presence of an average backbone spiking sequence identified during pre-task rest that is stable over time and different cognitive states. We further demonstrate that these backbone sequences are composed of both rigid and flexible sequence elements, and that flexible elements within these sequences serve to promote memory specificity when forming and retrieving new memories. These results support the hypothesis that pre-existing network dynamics serve as a scaffold for ongoing neural activity in the human cortex.

Dynamic Recruitment of the Feedforward and Recurrent Mechanism for Black-White Asymmetry in the Primary Visual Cortex

Black and white information is asymmetrically distributed in natural scenes, evokes asymmetric neuronal responses, and causes asymmetric perceptions. Recognizing the universality and essentiality of black-white asymmetry in visual information processing, the neural substrates for black-white asymmetry remain unclear. To disentangle the role of the feedforward and recurrent mechanisms in the generation of cortical black-white asymmetry, we recorded the V1 laminar responses and LGN responses of anesthetized cats of both sexes. In a cortical column, we found that black-white asymmetry starts at the input layer and becomes more pronounced in the output layer. We also found distinct dynamics of black-white asymmetry between the output layer and the input layer. Specifically, black responses dominate in all layers after stimulus onset. After stimulus offset, black and white responses are balanced in the input layer, but black responses still dominate in the output layer. Compared with that in the input layer, the rebound response in the output layer is significantly suppressed. The relative suppression strength evoked by white stimuli is notably stronger and depends on the location within the ON-OFF cortical map. A model with delayed and polarity-selective cortical suppression explains black-white asymmetry in the output layer, within which prominent recurrent connections are identified by Granger causality analysis. In addition to black-white asymmetry in response strength, the interlaminar differences in spatial receptive field varied dynamically. Our findings suggest that the feedforward and recurrent mechanisms are dynamically recruited for the generation of black-white asymmetry in V1.SIGNIFICANCE STATEMENT Black-white asymmetry is universal and essential in visual information processing, yet the neural substrates for cortical black-white asymmetry remain unknown. Leveraging V1 laminar recordings, we provided the first laminar pattern of black-white asymmetry in cat V1 and found distinct dynamics of black-white asymmetry between the output layer and the input layer. Comparing black-white asymmetry across three visual hierarchies, the LGN, V1 input layer, and V1 output layer, we demonstrated that the feedforward and recurrent mechanisms are dynamically recruited for the generation of cortical black-white asymmetry. Our findings not only enhance our understanding of laminar processing within a cortical column but also elucidate how feedforward connections and recurrent connections interact to shape neuronal response properties.

Rate versus synchrony codes for cerebellar control of motor behavior

Information transmission between neural populations could occur through either coordinated changes in firing rates or the precise transmission of spike timing. We investigate the code for information transmission from a part of the cerebellar cortex that is crucial for the accurate execution of a quantifiable motor behavior. Simultaneous recordings from Purkinje cell pairs in the cerebellum of rhesus macaques reveal how these cells coordinate their activity to drive smooth pursuit eye movements. Purkinje cells show millisecond-scale coordination of spikes (synchrony), but the level of synchrony is small and insufficient to impact the firing of downstream vestibular nucleus neurons. Analysis of previous metrics that purported to reveal Purkinje cell synchrony demonstrates that these metrics conflate changes in firing rate and neuron-neuron covariance. We conclude that the output of the cerebellar cortex uses primarily a rate rather than a synchrony code to drive the activity of downstream neurons and thus control motor behavior.

Exploring retinal ganglion cells encoding to multi-modal stimulation using 3D microelectrodes arrays

Microelectrode arrays (MEA) are extensively utilized in encoding studies of retinal ganglion cells (RGCs) due to their capacity for simultaneous recording of neural activity across multiple channels. However, conventional planar MEAs face limitations in studying RGCs due to poor coupling between electrodes and RGCs, resulting in low signal-to-noise ratio (SNR) and limited recording sensitivity. To overcome these challenges, we employed photolithography, electroplating, and other processes to fabricate a 3D MEA based on the planar MEA platform. The 3D MEA exhibited several improvements compared to planar MEA, including lower impedance (8.73 ± 1.66 kΩ) and phase delay (-15.11° ± 1.27°), as well as higher charge storage capacity (CSC = 10.16 ± 0.81 mC/cm2), cathodic charge storage capacity (CSCc = 7.10 ± 0.55 mC/cm2), and SNR (SNR = 8.91 ± 0.57). Leveraging the advanced 3D MEA, we investigated the encoding characteristics of RGCs under multi-modal stimulation. Optical, electrical, and chemical stimulation were applied as sensory inputs, and distinct response patterns and response times of RGCs were detected, as well as variations in rate encoding and temporal encoding. Specifically, electrical stimulation elicited more effective RGC firing, while optical stimulation enhanced RGC synchrony. These findings hold promise for advancing the field of neural encoding.

Robust encoding of natural stimuli by neuronal response sequences in monkey visual cortex

Parallel multisite recordings in the visual cortex of trained monkeys revealed that the responses of spatially distributed neurons to natural scenes are ordered in sequences. The rank order of these sequences is stimulus-specific and maintained even if the absolute timing of the responses is modified by manipulating stimulus parameters. The stimulus specificity of these sequences was highest when they were evoked by natural stimuli and deteriorated for stimulus versions in which certain statistical regularities were removed. This suggests that the response sequences result from a matching operation between sensory evidence and priors stored in the cortical network. Decoders trained on sequence order performed as well as decoders trained on rate vectors but the former could decode stimulus identity from considerably shorter response intervals than the latter. A simulated recurrent network reproduced similarly structured stimulus-specific response sequences, particularly once it was familiarized with the stimuli through non-supervised Hebbian learning. We propose that recurrent processing transforms signals from stationary visual scenes into sequential responses whose rank order is the result of a Bayesian matching operation. If this temporal code were used by the visual system it would allow for ultrafast processing of visual scenes.

Homeostatic synaptic plasticity rescues neural coding reliability

To survive, animals must recognize reoccurring stimuli. This necessitates a reliable stimulus representation by the neural code. While synaptic transmission underlies the propagation of neural codes, it is unclear how synaptic plasticity can maintain coding reliability. By studying the olfactory system of Drosophila melanogaster, we aimed to obtain a deeper mechanistic understanding of how synaptic function shapes neural coding in the live, behaving animal. We show that the properties of the active zone (AZ), the presynaptic site of neurotransmitter release, are critical for generating a reliable neural code. Reducing neurotransmitter release probability of olfactory sensory neurons disrupts both neural coding and behavioral reliability. Strikingly, a target-specific homeostatic increase of AZ numbers rescues these defects within a day. These findings demonstrate an important role for synaptic plasticity in maintaining neural coding reliability and are of pathophysiological interest by uncovering an elegant mechanism through which the neural circuitry can counterbalance perturbations.

SpikeSEE: An energy-efficient dynamic scenes processing framework for retinal prostheses

Intelligent and low-power retinal prostheses are highly demanded in this era, where wearable and implantable devices are used for numerous healthcare applications. In this paper, we propose an energy-efficient dynamic scenes processing framework (SpikeSEE) that combines a spike representation encoding technique and a bio-inspired spiking recurrent neural network (SRNN) model to achieve intelligent processing and extreme low-power computation for retinal prostheses. The spike representation encoding technique could interpret dynamic scenes with sparse spike trains, decreasing the data volume. The SRNN model, inspired by the human retina's special structure and spike processing method, is adopted to predict the response of ganglion cells to dynamic scenes. Experimental results show that the Pearson correlation coefficient of the proposed SRNN model achieves 0.93, which outperforms the state-of-the-art processing framework for retinal prostheses. Thanks to the spike representation and SRNN processing, the model can extract visual features in a multiplication-free fashion. The framework achieves 8 times power reduction compared with the convolutional recurrent neural network (CRNN) processing-based framework. Our proposed SpikeSEE predicts the response of ganglion cells more accurately with lower energy consumption, which alleviates the precision and power issues of retinal prostheses and provides a potential solution for wearable or implantable prostheses.

The MAPK/ERK channel capacity exceeds 6 bit/hour

Living cells utilize signaling pathways to sense, transduce, and process information. As the extracellular stimulation often has rich temporal characteristics which may govern dynamic cellular responses, it is important to quantify the rate of information flow through the signaling pathways. In this study, we used an epithelial cell line expressing a light-activatable FGF receptor and an ERK activity reporter to assess the ability of the MAPK/ERK pathway to transduce signal encoded in a sequence of pulses. By stimulating the cells with random light pulse trains, we demonstrated that the MAPK/ERK channel capacity is at least 6 bits per hour. The input reconstruction algorithm detects the light pulses with 1-min accuracy 5 min after their occurrence. The high information transmission rate may enable the pathway to coordinate multiple processes including cell movement and respond to rapidly varying stimuli such as chemoattracting gradients created by other cells.

Narrowband gamma oscillations propagate and synchronize throughout the mouse thalamocortical visual system

Oscillations of neural activity permeate sensory systems. In the visual system, broadband gamma oscillations (30-80 Hz) are thought to act as a communication mechanism underlying perception. However, these oscillations show widely varying frequency and phase, providing constraints for coordinating spike timing across areas. Here, we examined Allen Brain Observatory data and performed causal experiments to show that narrowband gamma (NBG) oscillations (50-70 Hz) propagate and synchronize throughout the awake mouse visual system. Lateral geniculate nucleus (LGN) neurons fired precisely relative to NBG phase in primary visual cortex (V1) and multiple higher visual areas (HVAs). NBG neurons across areas showed a higher likelihood of functional connectivity and stronger visual responses; remarkably, NBG neurons in LGN, preferring bright (ON) versus dark (OFF), fired at distinct NBG phases aligned across the cortical hierarchy. NBG oscillations may thus serve to coordinate spike timing across brain areas and facilitate communication of distinct visual features during perception.

Models Developed for Spiking Neural Networks

Emergence of deep neural networks (DNNs) has raised enormous attention towards artificial neural networks (ANNs) once again. They have become the state-of-the-art models and have won different machine learning challenges. Although these networks are inspired by the brain, they lack biological plausibility, and they have structural differences compared to the brain. Spiking neural networks (SNNs) have been around for a long time, and they have been investigated to understand the dynamics of the brain. However, their application in real-world and complicated machine learning tasks were limited. Recently, they have shown great potential in solving such tasks. Due to their energy efficiency and temporal dynamics there are many promises in their future development. In this work, we reviewed the structures and performances of SNNs on image classification tasks. The comparisons illustrate that these networks show great capabilities for more complicated problems. Furthermore, the simple learning rules developed for SNNs, such as STDP and R-STDP, can be a potential alternative to replace the backpropagation algorithm used in DNNs.•Different building blocks of spiking neural networks are explained in this work.•Developed models for SNNs are introduced based on their characteristics and building blocks.

Rate versus synchrony codes for cerebellar control of motor behavior

Control of movement requires the coordination of multiple brain areas, each containing populations of neurons that receive inputs, process these inputs via recurrent dynamics, and then relay the processed information to downstream populations. Information transmission between neural populations could occur through either coordinated changes in firing rates or the precise transmission of spike timing. We investigate the nature of the code for transmission of signals to downstream areas from a part of the cerebellar cortex that is crucial for the accurate execution of a quantifiable motor behavior. Simultaneous recordings from Purkinje cell pairs in the cerebellar flocculus of rhesus macaques revealed how these cells coordinate their activity to drive smooth pursuit eye movements. Purkinje cells show millisecond-scale coordination of spikes (synchrony), but the level of synchrony is small and likely insufficient to impact the firing of downstream neurons in the vestibular nucleus. Further, analysis of previous metrics for assaying Purkinje cell synchrony demonstrates that these metrics conflate changes in firing rate and neuron-neuron covariance. We conclude that the output of the cerebellar cortex uses primarily a rate code rather than synchrony code to drive activity of downstream neurons and thus control motor behavior.

IMPACT STATEMENT

Information transmission in the brain can occur via changes in firing rate or via the precise timing of spikes. Simultaneous recordings from pairs of Purkinje cells in the floccular complex reveals that information transmission out of the cerebellar cortex relies almost exclusively on changes in firing rates rather than millisecond-scale coordination of spike timing across the Purkinje cell population.

Artificial intelligence techniques for retinal prostheses: a comprehensive review and future direction

Objective. Retinal prostheses are promising devices to restore vision for patients with severe age-related macular degeneration or retinitis pigmentosa disease. The visual processing mechanism embodied in retinal prostheses play an important role in the restoration effect. Its performance depends on our understanding of the retina's working mechanism and the evolvement of computer vision models. Recently, remarkable progress has been made in the field of processing algorithm for retinal prostheses where the new discovery of the retina's working principle and state-of-the-arts computer vision models are combined together.Approach. We investigated the related research on artificial intelligence techniques for retinal prostheses. The processing algorithm in these studies could be attributed to three types: computer vision-related methods, biophysical models, and deep learning models.Main results. In this review, we first illustrate the structure and function of the normal and degenerated retina, then demonstrate the vision rehabilitation mechanism of three representative retinal prostheses. It is necessary to summarize the computational frameworks abstracted from the normal retina. In addition, the development and feature of three types of different processing algorithms are summarized. Finally, we analyze the bottleneck in existing algorithms and propose our prospect about the future directions to improve the restoration effect.Significance. This review systematically summarizes existing processing models for predicting the response of the retina to external stimuli. What's more, the suggestions for future direction may inspire researchers in this field to design better algorithms for retinal prostheses.

Precise Spiking Motifs in Neurobiological and Neuromorphic Data

Why do neurons communicate through spikes? By definition, spikes are all-or-none neural events which occur at continuous times. In other words, spikes are on one side binary, existing or not without further details, and on the other, can occur at any asynchronous time, without the need for a centralized clock. This stands in stark contrast to the analog representation of values and the discretized timing classically used in digital processing and at the base of modern-day neural networks. As neural systems almost systematically use this so-called event-based representation in the living world, a better understanding of this phenomenon remains a fundamental challenge in neurobiology in order to better interpret the profusion of recorded data. With the growing need for intelligent embedded systems, it also emerges as a new computing paradigm to enable the efficient operation of a new class of sensors and event-based computers, called neuromorphic, which could enable significant gains in computation time and energy consumption-a major societal issue in the era of the digital economy and global warming. In this review paper, we provide evidence from biology, theory and engineering that the precise timing of spikes plays a crucial role in our understanding of the efficiency of neural networks.

Long-range functional loops in the mouse olfactory system and their roles in computing odor identity

Elucidating the neural circuits supporting odor identification remains an open challenge. Here, we analyze the contribution of the two output cell types of the mouse olfactory bulb (mitral and tufted cells) to decode odor identity and concentration and its dependence on top-down feedback from their respective major cortical targets: piriform cortex versus anterior olfactory nucleus. We find that tufted cells substantially outperform mitral cells in decoding both odor identity and intensity. Cortical feedback selectively regulates the activity of its dominant bulb projection cell type and implements different computations. Piriform feedback specifically restructures mitral responses, whereas feedback from the anterior olfactory nucleus preferentially controls the gain of tufted representations without altering their odor tuning. Our results identify distinct functional loops involving the mitral and tufted cells and their cortical targets. We suggest that in addition to the canonical mitral-to-piriform pathway, tufted cells and their target regions are ideally positioned to compute odor identity.

Fractional Spiking Neuron: Fractional Leaky Integrate-and-Fire Circuit Described with Dendritic Fractal Model

As dendrites are essential parts of neurons, they are crucial factors for neuronal activities to follow multiple timescale dynamics, which ultimately affect information processing and cognition. However, in the common SNN (Spiking Neural Networks), the hardware-based LIF (Leaky Integrate-and-Fire) circuit only simulates the single timescale dynamic of soma without relating dendritic morphologies, which may limit the capability of simulating neurons to process information. This study proposes the dendritic fractal model mainly for quantifying dendritic morphological effects containing branch and length. To realize this model, We design multiple analog fractional-order circuits (AFCs) which match their extended structures and parameters with the dendritic features. Then introducing AFC into FLIF (Fractional Leaky Integrate-and-Fire) neuron circuits can demonstrate the same multiple timescale dynamics of spiking patterns as biological neurons, including spiking adaptation, inter-spike variability with power-law distribution, first-spike latency, and intrinsic memory. By contrast, it further enhances the degree of mimicry of neuron models and provides a more accurate model for understanding neural computation and cognition mechanisms.

Coexisting neuronal coding strategies in the barrel cortex

During tactile sensation by rodents, whisker movements across surfaces generate complex whisker motions, including discrete, transient stick-slip events, which carry information about surface properties. The characteristics of these events and how the brain encodes this tactile information remain enigmatic. We found that cortical neurons show a mixture of synchronized and nontemporally correlated spikes in their tactile responses. Synchronous spikes convey the magnitude of stick-slip events by numerous aspects of temporal coding. These spikes show preferential selectivity for kinetic and kinematic whisker motion. By contrast, asynchronous spikes in each neuron convey the magnitude of stick-slip events by their discharge rates, response probability, and interspike intervals. We further show that the differentiation between these two types of activity is highly dependent on the magnitude of stick-slip events and stimulus and response history. These results suggest that cortical neurons transmit multiple components of tactile information through numerous coding strategies.

Multiregion neuronal activity: the forest and the trees

The past decade has witnessed remarkable advances in the simultaneous measurement of neuronal activity across many brain regions, enabling fundamentally new explorations of the brain-spanning cellular dynamics that underlie sensation, cognition and action. These recently developed multiregion recording techniques have provided many experimental opportunities, but thoughtful consideration of methodological trade-offs is necessary, especially regarding field of view, temporal acquisition rate and ability to guarantee cellular resolution. When applied in concert with modern optogenetic and computational tools, multiregion recording has already made possible fundamental biological discoveries - in part via the unprecedented ability to perform unbiased neural activity screens for principles of brain function, spanning dozens of brain areas and from local to global scales.

Neural synchrony in cortical networks: mechanisms and implications for neural information processing and coding

Synchronization of neuronal discharges on the millisecond scale has long been recognized as a prevalent and functionally important attribute of neural activity. In this article, I review classical concepts and corresponding evidence of the mechanisms that govern the synchronization of distributed discharges in cortical networks and relate those mechanisms to their possible roles in coding and cognitive functions. To accommodate the need for a selective, directed synchronization of cells, I propose that synchronous firing of distributed neurons is a natural consequence of spike-timing-dependent plasticity (STDP) that associates cells repetitively receiving temporally coherent input: the “synchrony through synaptic plasticity” hypothesis. Neurons that are excited by a repeated sequence of synaptic inputs may learn to selectively respond to the onset of this sequence through synaptic plasticity. Multiple neurons receiving coherent input could thus actively synchronize their firing by learning to selectively respond at corresponding temporal positions. The hypothesis makes several predictions: first, the position of the cells in the network, as well as the source of their input signals, would be irrelevant as long as their input signals arrive simultaneously; second, repeating discharge patterns should get compressed until all or some part of the signals are synchronized; and third, this compression should be accompanied by a sparsening of signals. In this way, selective groups of cells could emerge that would respond to some recurring event with synchronous firing. Such a learned response pattern could further be modulated by synchronous network oscillations that provide a dynamic, flexible context for the synaptic integration of distributed signals. I conclude by suggesting experimental approaches to further test this new hypothesis.

Implications of Neural Plasticity in Retinal Prosthesis

Retinal degenerative diseases such as retinitis pigmentosa cause a progressive loss of photoreceptors that eventually prevents the affected person from perceiving visual sensations. The absence of a visual input produces a neural rewiring cascade that propagates along the visual system. This remodeling occurs first within the retina. Then, subsequent neuroplastic changes take place at higher visual centers in the brain, produced by either the abnormal neural encoding of the visual inputs delivered by the diseased retina or as the result of an adaptation to visual deprivation. While retinal implants can activate the surviving retinal neurons by delivering electric current, the unselective activation patterns of the different neural populations that exist in the retinal layers differ substantially from those in physiologic vision. Therefore, artificially induced neural patterns are being delivered to a brain that has already undergone important neural reconnections. Whether or not the modulation of this neural rewiring can improve the performance for retinal prostheses remains a critical question whose answer may be the enabler of improved functional artificial vision and more personalized neurorehabilitation strategies.

Temporal Coding in Spiking Neural Networks With Alpha Synaptic Function: Learning With Backpropagation

The timing of individual neuronal spikes is essential for biological brains to make fast responses to sensory stimuli. However, conventional artificial neural networks lack the intrinsic temporal coding ability present in biological networks. We propose a spiking neural network model that encodes information in the relative timing of individual spikes. In classification tasks, the output of the network is indicated by the first neuron to spike in the output layer. This temporal coding scheme allows the supervised training of the network with backpropagation, using locally exact derivatives of the postsynaptic spike times with respect to presynaptic spike times. The network operates using a biologically plausible synaptic transfer function. In addition, we use trainable pulses that provide bias, add flexibility during training, and exploit the decayed part of the synaptic function. We show that such networks can be successfully trained on multiple data sets encoded in time, including MNIST. Our model outperforms comparable spiking models on MNIST and achieves similar quality to fully connected conventional networks with the same architecture. The spiking network spontaneously discovers two operating modes, mirroring the accuracy-speed tradeoff observed in human decision-making: a highly accurate but slow regime, and a fast but slightly lower accuracy regime. These results demonstrate the computational power of spiking networks with biological characteristics that encode information in the timing of individual neurons. By studying temporal coding in spiking networks, we aim to create building blocks toward energy-efficient, state-based biologically inspired neural architectures. We provide open-source code for the model.

Optimal noise level for coding with tightly balanced networks of spiking neurons in the presence of transmission delays

Neural circuits consist of many noisy, slow components, with individual neurons subject to ion channel noise, axonal propagation delays, and unreliable and slow synaptic transmission. This raises a fundamental question: how can reliable computation emerge from such unreliable components? A classic strategy is to simply average over a population of N weakly-coupled neurons to achieve errors that scale as [Formula: see text]. But more interestingly, recent work has introduced networks of leaky integrate-and-fire (LIF) neurons that achieve coding errors that scale superclassically as 1/N by combining the principles of predictive coding and fast and tight inhibitory-excitatory balance. However, spike transmission delays preclude such fast inhibition, and computational studies have observed that such delays can cause pathological synchronization that in turn destroys superclassical coding performance. Intriguingly, it has also been observed in simulations that noise can actually improve coding performance, and that there exists some optimal level of noise that minimizes coding error. However, we lack a quantitative theory that describes this fascinating interplay between delays, noise and neural coding performance in spiking networks. In this work, we elucidate the mechanisms underpinning this beneficial role of noise by deriving analytical expressions for coding error as a function of spike propagation delay and noise levels in predictive coding tight-balance networks of LIF neurons. Furthermore, we compute the minimal coding error and the associated optimal noise level, finding that they grow as power-laws with the delay. Our analysis reveals quantitatively how optimal levels of noise can rescue neural coding performance in spiking neural networks with delays by preventing the build up of pathological synchrony without overwhelming the overall spiking dynamics. This analysis can serve as a foundation for the further study of precise computation in the presence of noise and delays in efficient spiking neural circuits.

Advances in computer-generated holography for targeted neuronal modulation

Genetically encoded calcium indicators and optogenetics have revolutionized neuroscience by enabling the detection and modulation of neural activity with single-cell precision using light. To fully leverage the immense potential of these techniques, advanced optical instruments that can place a light on custom ensembles of neurons with a high level of spatial and temporal precision are required. Modern light sculpting techniques that have the capacity to shape a beam of light are preferred because they can precisely target multiple neurons simultaneously and modulate the activity of large ensembles of individual neurons at rates that match natural neuronal dynamics. The most versatile approach, computer-generated holography (CGH), relies on a computer-controlled light modulator placed in the path of a coherent laser beam to synthesize custom three-dimensional (3D) illumination patterns and illuminate neural ensembles on demand. Here, we review recent progress in the development and implementation of fast and spatiotemporally precise CGH techniques that sculpt light in 3D to optically interrogate neural circuit functions.

Analyzing time-to-first-spike coding schemes: A theoretical approach

Spiking neural networks (SNNs) using time-to-first-spike (TTFS) codes, in which neurons fire at most once, are appealing for rapid and low power processing. In this theoretical paper, we focus on information coding and decoding in those networks, and introduce a new unifying mathematical framework that allows the comparison of various coding schemes. In an early proposal, called rank-order coding (ROC), neurons are maximally activated when inputs arrive in the order of their synaptic weights, thanks to a shunting inhibition mechanism that progressively desensitizes the neurons as spikes arrive. In another proposal, called NoM coding, only the first N spikes of M input neurons are propagated, and these “first spike patterns” can be readout by downstream neurons with homogeneous weights and no desensitization: as a result, the exact order between the first spikes does not matter. This paper also introduces a third option—“Ranked-NoM” (R-NoM), which combines features from both ROC and NoM coding schemes: only the first N input spikes are propagated, but their order is readout by downstream neurons thanks to inhomogeneous weights and linear desensitization. The unifying mathematical framework allows the three codes to be compared in terms of discriminability, which measures to what extent a neuron responds more strongly to its preferred input spike pattern than to random patterns. This discriminability turns out to be much higher for R-NoM than for the other codes, especially in the early phase of the responses. We also argue that R-NoM is much more hardware-friendly than the original ROC proposal, although NoM remains the easiest to implement in hardware because it only requires binary synapses.

Context-dependent selectivity to natural images in the retina

Retina ganglion cells extract specific features from natural scenes and send this information to the brain. In particular, they respond to local light increase (ON responses), and/or decrease (OFF). However, it is unclear if this ON-OFF selectivity, characterized with synthetic stimuli, is maintained under natural scene stimulation. Here we recorded ganglion cell responses to natural images slightly perturbed by random noise patterns to determine their selectivity during natural stimulation. The ON-OFF selectivity strongly depended on the specific image. A single ganglion cell can signal luminance increase for one image, and luminance decrease for another. Modeling and experiments showed that this resulted from the non-linear combination of different retinal pathways. Despite the versatility of the ON-OFF selectivity, a systematic analysis demonstrated that contrast was reliably encoded in these responses. Our perturbative approach uncovered the selectivity of retinal ganglion cells to more complex features than initially thought.

Ganglion cells classically respond to either light increase (ON) or decrease (OFF). Here, the authors show that during natural scene stimulation, a single ganglion cell can switch between ON and OFF depending on the visual context.

A surrogate gradient spiking baseline for speech command recognition

Artificial neural networks (ANNs) are the basis of recent advances in artificial intelligence (AI); they typically use real valued neuron responses. By contrast, biological neurons are known to operate using spike trains. In principle, spiking neural networks (SNNs) may have a greater representational capability than ANNs, especially for time series such as speech; however their adoption has been held back by both a lack of stable training algorithms and a lack of compatible baselines. We begin with a fairly thorough review of literature around the conjunction of ANNs and SNNs. Focusing on surrogate gradient approaches, we proceed to define a simple but relevant evaluation based on recent speech command tasks. After evaluating a representative selection of architectures, we show that a combination of adaptation, recurrence and surrogate gradients can yield light spiking architectures that are not only able to compete with ANN solutions, but also retain a high degree of compatibility with them in modern deep learning frameworks. We conclude tangibly that SNNs are appropriate for future research in AI, in particular for speech processing applications, and more speculatively that they may also assist in inference about biological function.

Phase advancing is a common property of multiple neuron classes in the mouse retina

Behavioral interactions with moving objects are challenged by response latencies within the sensory and motor nervous systems. In vision, the combined latency from phototransduction and synaptic transmission from the retina to central visual areas amounts to 50-100 ms, depending on stimulus conditions. Time required for generating appropriate motor output adds to this latency and further compounds the behavioral delay. Neuronal adaptations that help counter sensory latency within the retina have been demonstrated in some species, but how general these specializations are, and where in the circuitry they originate, remains unclear. To address this, we studied the timing of object motion-evoked responses at multiple signaling stages within the mouse retina using two-photon fluorescence calcium and glutamate imaging, targeted whole-cell electrophysiology, and computational modeling. We found that both ON and OFF-type ganglion cells, as well as the bipolar cells that innervate them, temporally advance the position encoding of a moving object and so help counter the inherent signaling delay in the retina. Model simulations show that this predictive capability is a direct consequence of the spatial extent of the cells' linear visual receptive field, with no apparent specialized circuits that help predict beyond it.Significance StatementSignal transduction and synaptic transmission within sensory signaling pathways costs time. Not a lot of time, just tens to a few hundred milliseconds depending on the sensory system, but enough to challenge fast behavioral interactions under dynamic stimulus conditions, like catching a moving fly. To counter neuronal delays, nervous systems of many species use anticipatory mechanisms. One such mechanism in the mammalian visual system helps predict the future position of a moving target through a process called phase advancing. Here we ask for functionally diverse neuron populations in the mouse retina how common is phase advancing and demonstrate that it is common and generated at multiple signaling stages.

SmaRT2P: a software for generating and processing smart line recording trajectories for population two-photon calcium imaging

Two-photon fluorescence calcium imaging allows recording the activity of large neural populations with subcellular spatial resolution, but it is typically characterized by low signal-to-noise ratio (SNR) and poor accuracy in detecting single or few action potentials when large number of neurons are imaged. We recently showed that implementing a smart line scanning approach using trajectories that optimally sample the regions of interest increases both the SNR fluorescence signals and the accuracy of single spike detection in population imaging in vivo. However, smart line scanning requires highly specialised software to design recording trajectories, interface with acquisition hardware, and efficiently process acquired data. Furthermore, smart line scanning needs optimized strategies to cope with movement artefacts and neuropil contamination. Here, we develop and validate SmaRT2P, an open-source, user-friendly and easy-to-interface Matlab-based software environment to perform optimized smart line scanning in two-photon calcium imaging experiments. SmaRT2P is designed to interface with popular acquisition software (e.g., ScanImage) and implements novel strategies to detect motion artefacts, estimate neuropil contamination, and minimize their impact on functional signals extracted from neuronal population imaging. SmaRT2P is structured in a modular way to allow flexibility in the processing pipeline, requiring minimal user intervention in parameter setting. The use of SmaRT2P for smart line scanning has the potential to facilitate the functional investigation of large neuronal populations with increased SNR and accuracy in detecting the discharge of single and few action potentials.

All-optical interrogation of neural circuits in behaving mice

Recent advances combining two-photon calcium imaging and two-photon optogenetics with computer-generated holography now allow us to read and write the activity of large populations of neurons in vivo at cellular resolution and with high temporal resolution. Such 'all-optical' techniques enable experimenters to probe the effects of functionally defined neurons on neural circuit function and behavioral output with new levels of precision. This greatly increases flexibility, resolution, targeting specificity and throughput compared with alternative approaches based on electrophysiology and/or one-photon optogenetics and can interrogate larger and more densely labeled populations of neurons than current voltage imaging-based implementations. This protocol describes the experimental workflow for all-optical interrogation experiments in awake, behaving head-fixed mice. We describe modular procedures for the setup and calibration of an all-optical system (~3 h), the preparation of an indicator and opsin-expressing and task-performing animal (~3-6 weeks), the characterization of functional and photostimulation responses (~2 h per field of view) and the design and implementation of an all-optical experiment (achievable within the timescale of a normal behavioral experiment; ~3-5 h per field of view). We discuss optimizations for efficiently selecting and targeting neuronal ensembles for photostimulation sequences, as well as generating photostimulation response maps from the imaging data that can be used to examine the impact of photostimulation on the local circuit. We demonstrate the utility of this strategy in three brain areas by using different experimental setups. This approach can in principle be adapted to any brain area to probe functional connectivity in neural circuits and investigate the relationship between neural circuit activity and behavior.

Error-based or target-based? A unified framework for learning in recurrent spiking networks

The field of recurrent neural networks is over-populated by a variety of proposed learning rules and protocols. The scope of this work is to define a generalized framework, to move a step forward towards the unification of this fragmented scenario. In the field of supervised learning, two opposite approaches stand out, error-based and target-based. This duality gave rise to a scientific debate on which learning framework is the most likely to be implemented in biological networks of neurons. Moreover, the existence of spikes raises the question of whether the coding of information is rate-based or spike-based. To face these questions, we proposed a learning model with two main parameters, the rank of the feedback learning matrix R and the tolerance to spike timing τ_⋆. We demonstrate that a low (high) rank R accounts for an error-based (target-based) learning rule, while high (low) tolerance to spike timing promotes rate-based (spike-based) coding. We show that in a store and recall task, high-ranks allow for lower MSE values, while low-ranks enable a faster convergence. Our framework naturally lends itself to Behavioral Cloning and allows for efficiently solving relevant closed-loop tasks, investigating what parameters (R,τ⋆) are optimal to solve a specific task. We found that a high R is essential for tasks that require retaining memory for a long time (Button and Food). On the other hand, this is not relevant for a motor task (the 2D Bipedal Walker). In this case, we find that precise spike-based coding enables optimal performances. Finally, we show that our theoretical formulation allows for defining protocols to estimate the rank of the feedback error in biological networks. We release a PyTorch implementation of our model supporting GPU parallelization.

Learning in biological or artificial networks means changing the laws governing the network dynamics in order to better behave in a specific situation. However, there exists no consensus on what rules regulate learning in biological systems. To face these questions, we propose a novel theoretical formulation for learning with two main parameters, the number of learning constraints (R) and the tolerance to spike timing (τ_⋆). We demonstrate that a low (high) rank R accounts for an error-based (target-based) learning rule, while high (low) tolerance to spike timing τ_⋆ promotes rate-based (spike-based) coding.

Our approach naturally lends itself to Imitation Learning (and Behavioral Cloning in particular) and we apply it to solve relevant closed-loop tasks such as the button-and-food task, and the 2D Bipedal Walker. The button-and-food is a navigation task that requires retaining a long-term memory, and benefits from a high R. On the other hand, the 2D Bipedal Walker is a motor task and benefits from a low τ_⋆.

Finally, we show that our theoretical formulation suggests protocols to deduce the structure of learning feedback in biological networks.

The superior colliculus/lateral posterior thalamic nuclei in mice rapidly transmit fear visual information through the theta frequency band

Animals perceive threat information mainly from vision, and the subcortical visual pathway plays a critical role in the rapid processing of fear visual information. The superior colliculus (SC) and lateral posterior (LP) nuclei of the thalamus are key components of the subcortical visual pathway; however, how animals encode and transmit fear visual information is unclear. To evaluate the response characteristics of neurons in SC and LP thalamic nuclei under fear visual stimuli, extracellular action potentials (spikes) and local field potential signals were recorded under looming and dimming visual stimuli. The results showed that both SC and LP thalamic nuclei were strongly responsive to looming visual stimuli but not sensitive to dimming visual stimuli. Under the looming visual stimulus, the theta (θ) frequency bands of both nuclei showed obvious oscillations, which markedly enhanced the synchronization between neurons. The functional network characteristics also indicated that the network connection density and information transmission efficiency were higher under fear visual stimuli. These findings suggest that both SC and LP thalamic nuclei can effectively identify threatening fear visual information and rapidly transmit it between nuclei through the θ frequency band. This discovery can provide a basis for subsequent coding and decoding studies in the subcortical visual pathways.

Pre- and postsynaptically expressed spike-timing-dependent plasticity contribute differentially to neuronal learning

A plethora of experimental studies have shown that long-term synaptic plasticity can be expressed pre- or postsynaptically depending on a range of factors such as developmental stage, synapse type, and activity patterns. The functional consequences of this diversity are not clear, although it is understood that whereas postsynaptic expression of plasticity predominantly affects synaptic response amplitude, presynaptic expression alters both synaptic response amplitude and short-term dynamics. In most models of neuronal learning, long-term synaptic plasticity is implemented as changes in connective weights. The consideration of long-term plasticity as a fixed change in amplitude corresponds more closely to post- than to presynaptic expression, which means theoretical outcomes based on this choice of implementation may have a postsynaptic bias. To explore the functional implications of the diversity of expression of long-term synaptic plasticity, we adapted a model of long-term plasticity, more specifically spike-timing-dependent plasticity (STDP), such that it was expressed either independently pre- or postsynaptically, or in a mixture of both ways. We compared pair-based standard STDP models and a biologically tuned triplet STDP model, and investigated the outcomes in a minimal setting, using two different learning schemes: in the first, inputs were triggered at different latencies, and in the second a subset of inputs were temporally correlated. We found that presynaptic changes adjusted the speed of learning, while postsynaptic expression was more efficient at regulating spike timing and frequency. When combining both expression loci, postsynaptic changes amplified the response range, while presynaptic plasticity allowed control over postsynaptic firing rates, potentially providing a form of activity homeostasis. Our findings highlight how the seemingly innocuous choice of implementing synaptic plasticity by single weight modification may unwittingly introduce a postsynaptic bias in modelling outcomes. We conclude that pre- and postsynaptically expressed plasticity are not interchangeable, but enable complimentary functions.

Differences between functional properties of pre- or postsynaptically expressed long-term plasticity have not yet been explored in much detail. In this paper, we used minimalist models of STDP with different expression loci, in search of fundamental functional consequences. Biologically, presynaptic expression acts mostly on neurotransmitter release, thereby altering short-term synaptic dynamics, whereas postsynaptic expression affects mainly synaptic gain. We compared models where plasticity was expressed only presynaptically or postsynaptically, or in both ways. We found that postsynaptic plasticity had a bigger impact over response times, while both pre- and postsynaptic plasticity were similarly capable of detecting correlated inputs. A model with biologically tuned expression of plasticity achieved the same outcome over a range of frequencies. Also, postsynaptic spiking frequency was not directly affected by presynaptic plasticity of short-term plasticity alone, however in combination with a postsynaptic component, it helped restrain positive feedback, contributing to activity homeostasis. In conclusion, expression locus may determine affinity for distinct coding schemes while also contributing to keep activity within bounds. Our findings highlight the importance of carefully implementing expression of plasticity in biological modelling, since the locus of expression may affect functional outcomes in simulations.