A Probabilistic Framework for Decoding Behavior From in vivo Calcium Imaging Data

End-to-end deep learning approach to mouse behavior classification from cortex-wide calcium imaging

Deep learning is a powerful tool for neural decoding, broadly applied to systems neuroscience and clinical studies. Interpretable and transparent models that can explain neural decoding for intended behaviors are crucial to identifying essential features of deep learning decoders in brain activity. In this study, we examine the performance of deep learning to classify mouse behavioral states from mesoscopic cortex-wide calcium imaging data. Our convolutional neural network (CNN)-based end-to-end decoder combined with recurrent neural network (RNN) classifies the behavioral states with high accuracy and robustness to individual differences on temporal scales of sub-seconds. Using the CNN-RNN decoder, we identify that the forelimb and hindlimb areas in the somatosensory cortex significantly contribute to behavioral classification. Our findings imply that the end-to-end approach has the potential to be an interpretable deep learning method with unbiased visualization of critical brain regions.

Lateral entorhinal cortex subpopulations represent experiential epochs surrounding reward

During goal-directed navigation, 'what' information, describing the experiences occurring in periods surrounding a reward, can be combined with spatial 'where' information to guide behavior and form episodic memories. This integrative process likely occurs in the hippocampus, which receives spatial information from the medial entorhinal cortex; however, the source of the 'what' information is largely unknown. Here, we show that mouse lateral entorhinal cortex (LEC) represents key experiential epochs during reward-based navigation tasks. We discover separate populations of neurons that signal goal approach and goal departure and a third population signaling reward consumption. When reward location is moved, these populations immediately shift their respective representations of each experiential epoch relative to reward, while optogenetic inhibition of LEC disrupts learning the new reward location. Therefore, the LEC contains a stable code of experiential epochs surrounding and including reward consumption, providing reward-centric information to contextualize the spatial information carried by the medial entorhinal cortex.

Correlated signatures of social behavior in cerebellum and anterior cingulate cortex

The cerebellum has been implicated in the regulation of social behavior. Its influence is thought to arise from communication, via the thalamus, to forebrain regions integral in the expression of social interactions, including the anterior cingulate cortex (ACC). However, the signals encoded or the nature of the communication between the cerebellum and these brain regions is poorly understood. Here, we describe an approach that overcomes technical challenges in exploring the coordination of distant brain regions at high temporal and spatial resolution during social behavior. We developed the E-Scope, an electrophysiology-integrated miniature microscope, to synchronously measure extracellular electrical activity in the cerebellum along with calcium imaging of the ACC. This single coaxial cable device combined these data streams to provide a powerful tool to monitor the activity of distant brain regions in freely behaving animals. During social behavior, we recorded the spike timing of multiple single units in cerebellar right Crus I (RCrus I) Purkinje cells (PCs) or dentate nucleus (DN) neurons while synchronously imaging calcium transients in contralateral ACC neurons. We found that during social interactions a significant subpopulation of cerebellar PCs were robustly inhibited, while most modulated neurons in the DN were activated, and their activity was correlated with positively modulated ACC neurons. These distinctions largely disappeared when only non-social epochs were analyzed suggesting that cerebellar-cortical interactions were behaviorally specific. Our work provides new insights into the complexity of cerebellar activation and co-modulation of the ACC during social behavior and a valuable open-source tool for simultaneous, multimodal recordings in freely behaving mice.

Low-Frequency Stimulation of Trpv1-Lineage Peripheral Afferents Potentiates the Excitability of Spino-Periaqueductal Gray Projection Neurons

High-threshold dorsal root ganglion (HT DRG) neurons fire at low frequencies during inflammatory injury, and low-frequency stimulation (LFS) of HT DRG neurons selectively potentiates excitatory synapses onto spinal neurons projecting to the periaqueductal gray (spino-PAG). Here, in male and female mice, we have identified an underlying peripheral sensory population driving this plasticity and its effects on the output of spino-PAG neurons. We provide the first evidence that Trpv1-lineage sensory neurons predominantly induce burst firing, a unique mode of neuronal activity, in lamina I spino-PAG projection neurons. We modeled inflammatory injury by optogenetically stimulating Trpv1+ primary afferents at 2 Hz for 2 min (LFS), as peripheral inflammation induces 1-2 Hz firing in high-threshold C fibers. LFS of Trpv1+ afferents enhanced the synaptically evoked and intrinsic excitability of spino-PAG projection neurons, eliciting a stable increase in the number of action potentials (APs) within a Trpv1+ fiber-induced burst, while decreasing the intrinsic AP threshold and increasing the membrane resistance. Further experiments revealed that this plasticity required Trpv1+ afferent input, postsynaptic G protein-coupled signaling, and NMDA receptor activation. Intriguingly, an inflammatory injury and heat exposure in vivo also increased APs per burst, in vitro These results suggest that inflammatory injury-mediated plasticity is driven though Trpv1+ DRG neurons and amplifies the spino-PAG pathway. Spinal inputs to the PAG could play an integral role in its modulation of heat sensation during peripheral inflammation, warranting further exploration of the organization and function of these neural pathways.

Full field-of-view virtual reality goggles for mice

Visual virtual reality (VR) systems for head-fixed mice offer advantages over real-world studies for investigating the neural circuitry underlying behavior. However, current VR approaches do not fully cover the visual field of view of mice, do not stereoscopically illuminate the binocular zone, and leave the lab frame visible. To overcome these limitations, we developed iMRSIV (Miniature Rodent Stereo Illumination VR)-VR goggles for mice. Our system is compact, separately illuminates each eye for stereo vision, and provides each eye with an ∼180° field of view, thus excluding the lab frame while accommodating saccades. Mice using iMRSIV while navigating engaged in virtual behaviors more quickly than in a current monitor-based system and displayed freezing and fleeing reactions to overhead looming stimulation. Using iMRSIV with two-photon functional imaging, we found large populations of hippocampal place cells during virtual navigation, global remapping during environment changes, and unique responses of place cell ensembles to overhead looming stimulation.

Correlated signatures of social behavior in cerebellum and anterior cingulate cortex

The cerebellum has been implicated in the regulation of social behavior. Its influence is thought to arise from communication, via the thalamus, to forebrain regions integral in the expression of social interactions, including the anterior cingulate cortex (ACC). However, the signals encoded or the nature of the communication between the cerebellum and these brain regions is poorly understood. Here, we describe an approach that overcomes technical challenges in exploring the coordination of distant brain regions at high temporal and spatial resolution during social behavior. We developed the E-Scope, an electrophysiology-integrated miniature microscope, to synchronously measure extracellular electrical activity in the cerebellum along with calcium imaging of the ACC. This single coaxial cable device combined these data streams to provide a powerful tool to monitor the activity of distant brain regions in freely behaving animals. During social behavior, we recorded the spike timing of multiple single units in cerebellar right Crus I (RCrus I) Purkinje cells (PCs) or dentate nucleus (DN) neurons while synchronously imaging calcium transients in contralateral ACC neurons. We found that during social interactions a significant subpopulation of cerebellar PCs were robustly inhibited, while most modulated neurons in the DN were activated, and their activity was correlated with positively modulated ACC neurons. These distinctions largely disappeared when only non-social epochs were analyzed suggesting that cerebellar-cortical interactions were behaviorally specific. Our work provides new insights into the complexity of cerebellar activation and co-modulation of the ACC during social behavior and a valuable open-source tool for simultaneous, multimodal recordings in freely behaving mice.

Lateral entorhinal cortex subpopulations represent experiential epochs surrounding reward

During goal-directed navigation, "what" information, which describes the experiences occurring in periods surrounding a reward, can be combined with spatial "where" information to guide behavior and form episodic memories1,2. This integrative process is thought to occur in the hippocampus3, which receives spatial information from the medial entorhinal cortex (MEC)4; however, the source of the "what" information and how it is represented is largely unknown. Here, by establishing a novel imaging method, we show that the lateral entorhinal cortex (LEC) of mice represents key experiential epochs during a reward-based navigation task. We discover a population of neurons that signals goal approach and a separate population of neurons that signals goal departure. A third population of neurons signals reward consumption. When reward location is moved, these populations immediately shift their respective representations of each experiential epoch relative to reward, while optogenetic inhibition of LEC disrupts learning of the new reward location. Together, these results indicate the LEC provides a stable code of experiential epochs surrounding and including reward consumption, providing reward-centric information to contextualize the spatial information carried by the MEC. Such parallel representations are well-suited for generating episodic memories of rewarding experiences and guiding flexible and efficient goal-directed navigation5-7.

An adaptive behavioral control motif mediated by cortical axo-axonic inhibition

Genetically defined subgroups of inhibitory interneurons are thought to play distinct roles in learning, but heterogeneity within these subgroups has limited our understanding of the scope and nature of their specific contributions. Here we reveal that the chandelier cell (ChC), an interneuron type that specializes in inhibiting the axon-initial segment (AIS) of pyramidal neurons, establishes cortical microcircuits for organizing neural coding through selective axo-axonic synaptic plasticity. We found that organized motor control is mediated by enhanced population coding of direction-tuned premotor neurons, with tuning refined through suppression of irrelevant neuronal activity. ChCs contribute to learning-dependent refinements by providing selective inhibitory control over individual pyramidal neurons rather than global suppression. Quantitative analysis of structural plasticity across axo-axonic synapses revealed that ChCs redistributed inhibitory weights to individual pyramidal neurons during learning. These results demonstrate an adaptive logic of the inhibitory circuit motif responsible for organizing distributed neural representations. Thus, ChCs permit efficient cortical computation in a targeted cell-specific manner.

Active experience, not time, determines within-day representational drift in dorsal CA1

Memories of past events can be recalled long after the event, indicating stability. But new experiences are also integrated into existing memories, indicating plasticity. In the hippocampus, spatial representations are known to remain stable but have also been shown to drift over long periods of time. We hypothesized that experience, more than the passage of time, is the driving force behind representational drift. We compared the within-day stability of place cells' representations in dorsal CA1 of the hippocampus of mice traversing two similar, familiar tracks for different durations. We found that the more time the animals spent actively traversing the environment, the greater the representational drift, regardless of the total elapsed time between visits. Our results suggest that spatial representation is a dynamic process, related to the ongoing experiences within a specific context, and is related to memory update rather than to passive forgetting.

Coregistration of heading to visual cues in retrosplenial cortex

Spatial cognition depends on an accurate representation of orientation within an environment. Head direction cells in distributed brain regions receive a range of sensory inputs, but visual input is particularly important for aligning their responses to environmental landmarks. To investigate how population-level heading responses are aligned to visual input, we recorded from retrosplenial cortex (RSC) of head-fixed mice in a moving environment using two-photon calcium imaging. We show that RSC neurons are tuned to the animal's relative orientation in the environment, even in the absence of head movement. Next, we found that RSC receives functionally distinct projections from visual and thalamic areas and contains several functional classes of neurons. While some functional classes mirror RSC inputs, a newly discovered class coregisters visual and thalamic signals. Finally, decoding analyses reveal unique contributions to heading from each class. Our results suggest an RSC circuit for anchoring heading representations to environmental visual landmarks.

FPGA-Based In-Vivo Calcium Image Decoding for Closed-Loop Feedback Applications

Miniaturized calcium imaging is an emerging neural recording technique that has been widely used for monitoring neural activity on a large scale at a specific brain region of rats or mice. Most existing calcium-image analysis pipelines operate offline. This results in long processing latency, making it difficult to realize closed-loop feedback stimulation for brain research. In recent work, we have proposed an FPGA-based real-time calcium image processing pipeline for closed-loop feedback applications. It can perform real-time calcium image motion correction, enhancement, fast trace extraction, and real-time decoding from extracted traces. Here, we extend this work by proposing a variety of neural network based methods for real-time decoding and evaluate the tradeoff among these decoding methods and accelerator designs. We introduce the implementation of the neural network based decoders on the FPGA, and show their speedup against the implementation on the ARM processor. Our FPGA implementation enables the real-time calcium image decoding with sub-ms processing latency for closed-loop feedback applications.

An adaptive behavioral control motif mediated by cortical axo-axonic inhibition

Neural circuits are reorganized with specificity during learning. Genetically-defined subgroups of inhibitory interneurons are thought to play distinct roles in learning, but heterogeneity within these subgroups has limited our understanding of the scope and nature of their specific contributions to learning. Here we reveal that the chandelier cell (ChC), an interneuron type that specializes in inhibiting the axon-initial segment (AIS) of pyramidal neurons, establishes cortical microcircuits for organizing neural coding through selective axo-axonic synaptic plasticity. We find that organized motor control is mediated by enhanced population coding of direction-tuned premotor neurons, whose tuning is refined through suppression of irrelevant neuronal activity. ChCs are required for learning-dependent refinements via providing selective inhibitory control over pyramidal neurons rather than global suppression. Quantitative analysis on structural plasticity of axo-axonic synapses revealed that ChCs redistributed inhibitory weights to individual pyramidal neurons during learning. These results demonstrate an adaptive logic of the inhibitory circuit motif responsible for organizing distributed neural representations. Thus, ChCs permit efficient cortical computation in a target cell specific manner, which highlights the significance of interneuron diversity.

A frontosensory circuit for visual context processing is synchronous in the theta/alpha band

Visual processing is strongly influenced by context. Stimuli that deviate from contextual regularities elicit augmented responses in primary visual cortex (V1). These heightened responses, known as "deviance detection," require both inhibition local to V1 and top-down modulation from higher areas of cortex. Here we investigated the spatiotemporal mechanisms by which these circuit elements interact to support deviance detection. Local field potential recordings in mice in anterior cingulate area (ACa) and V1 during a visual oddball paradigm showed that interregional synchrony peaks in the theta/alpha band (6-12 Hz). Two-photon imaging in V1 revealed that mainly pyramidal neurons exhibited deviance detection, while vasointestinal peptide-positive interneurons (VIPs) increased activity and somatostatin-positive interneurons (SSTs) decreased activity (adapted) to redundant stimuli (prior to deviants). Optogenetic drive of ACa-V1 inputs at 6-12 Hz activated V1-VIPs but inhibited V1-SSTs, mirroring the dynamics present during the oddball paradigm. Chemogenetic inhibition of VIP interneurons disrupted ACa-V1 synchrony and deviance detection responses in V1. These results outline spatiotemporal and interneuron-specific mechanisms of top-down modulation that support visual context processing.

Persistent representation of the environment in the hippocampus

In the hippocampus, environmental changes elicit rearrangement of active neuronal ensembles or remapping of place cells. However, it remains elusive how the brain ensures a consistent representation of a certain environment itself despite salient events occurring there. Here, we longitudinally tracked calcium dynamics of dorsal hippocampal CA1 neurons in mice subjected to contextual fear conditioning and extinction training. Overall population activities were significantly changed by fear conditioning and were responsive to footshocks and freezing. However, a small subset of neurons, termed environment cells, were consistently active in a specific environment irrespective of experiences. A decoder modeling study showed that these cells, but not place cells, were able to predict the environment to which the mouse was exposed. Environment cells might underlie the constancy of cognition for distinct environments across time and events. Additionally, our study highlights the functional heterogeneity of cells in the hippocampus.

Optogenetic frequency scrambling of hippocampal theta oscillations dissociates working memory retrieval from hippocampal spatiotemporal codes

The precise temporal coordination of activity in the brain is thought to be fundamental for memory function. Inhibitory neurons in the medial septum provide a prominent source of innervation to the hippocampus and play a major role in controlling hippocampal theta (~8 Hz) oscillations. While pharmacological inhibition of medial septal neurons is known to disrupt memory, the exact role of septal inhibitory neurons in regulating hippocampal representations and memory is not fully understood. Here, we dissociate the role of theta rhythms in spatiotemporal coding and memory using an all-optical interrogation and recording approach. We find that optogenetic frequency scrambling stimulations abolish theta oscillations and modulate a portion of neurons in the hippocampus. Such stimulation decreased episodic and working memory retrieval while leaving hippocampal spatiotemporal codes intact. Our study suggests that theta rhythms play an essential role in memory but may not be necessary for hippocampal spatiotemporal codes.

Calcium activity is a degraded estimate of spikes

Recording action potentials extracellularly during behavior has led to fundamental discoveries regarding neural function-hippocampal neurons respond to locations in space,1 motor cortex neurons encode movement direction,2 and dopamine neurons signal reward prediction errors3-observations undergirding current theories of cognition,4 movement,5 and learning.6 Recently it has become possible to measure calcium flux, an internal cellular signal related to spiking. The ability to image calcium flux in anatomically7,8 or genetically9 identified neurons can extend our knowledge of neural circuit function by allowing activity to be monitored in specific cell types or projections, or in the same neurons across many days. However, while initial studies were grounded in prior unit recording work, it has become fashionable to assume that calcium is identical to spiking, even though the spike-to-fluorescence transformation is nonlinear, noisy, and unpredictable under real-world conditions.10 It remains an open question whether calcium provides a high-fidelity representation of single-unit activity in awake, behaving subjects. Here, we have addressed this question by recording both signals in the lateral orbitofrontal cortex (OFC) of rats during olfactory discrimination learning. Activity in the OFC during olfactory learning has been well-studied in humans,11,12,13,14 nonhuman primates,15,16 and rats,17,18,19,20,21 where it has been shown to signal information about both the sensory properties of odor cues and the rewards they predict. Our single-unit results replicated prior findings, whereas the calcium signal provided only a degraded estimate of the information available in the single-unit spiking, reflecting primarily reward value.

Pattern decorrelation in the mouse medial prefrontal cortex enables social preference and requires MeCP2

Sociability is crucial for survival, whereas social avoidance is a feature of disorders such as Rett syndrome, which is caused by loss-of-function mutations in MECP2. To understand how a preference for social interactions is encoded, we used in vivo calcium imaging to compare medial prefrontal cortex (mPFC) activity in female wild-type and Mecp2-heterozygous mice during three-chamber tests. We found that mPFC pyramidal neurons in Mecp2-deficient mice are hypo-responsive to both social and nonsocial stimuli. Hypothesizing that this limited dynamic range restricts the circuit's ability to disambiguate coactivity patterns for different stimuli, we suppressed the mPFC in wild-type mice and found that this eliminated both pattern decorrelation and social preference. Conversely, stimulating the mPFC in MeCP2-deficient mice restored social preference, but only if it was sufficient to restore pattern decorrelation. A loss of social preference could thus indicate impaired pattern decorrelation rather than true social avoidance.

Bulk Dye Loading for In Vivo Calcium Imaging of Visual Responses in Populations of Xenopus Tectal Neurons

Bulk loading of neurons with fluorescent calcium indicators in transparent albino Xenopus tadpoles offers a rapid and easy method for tracking sensory-evoked activity in large numbers of neurons within an awake developing brain circuit. In vivo two-photon time-lapse imaging of an image plane through the optic tectum allows defining receptive field properties from visual-evoked responses for studies of single-neuron and network-level encoding and plasticity. Here, we describe loading the Xenopus tadpole optic tectum with the membrane-permeable AM ester of Oregon Green 488 BAPTA-1 (OGB-1 AM) for in vivo imaging experiments.

Conjunctive spatial and self-motion codes are topographically organized in the GABAergic cells of the lateral septum

The hippocampal spatial code’s relevance for downstream neuronal populations—particularly its major subcortical output the lateral septum (LS)—is still poorly understood. Here, using calcium imaging combined with unbiased analytical methods, we functionally characterized and compared the spatial tuning of LS GABAergic cells to those of dorsal CA3 and CA1 cells. We identified a significant number of LS cells that are modulated by place, speed, acceleration, and direction, as well as conjunctions of these properties, directly comparable to hippocampal CA1 and CA3 spatially modulated cells. Interestingly, Bayesian decoding of position based on LS spatial cells reflected the animal’s location as accurately as decoding using the activity of hippocampal pyramidal cells. A portion of LS cells showed stable spatial codes over the course of multiple days, potentially reflecting long-term episodic memory. The distributions of cells exhibiting these properties formed gradients along the anterior–posterior and dorsal–ventral axes of the LS, directly reflecting the topographical organization of hippocampal inputs to the LS. Finally, we show using transsynaptic tracing that LS neurons receiving CA3 and CA1 excitatory input send projections to the hypothalamus and medial septum, regions that are not targeted directly by principal cells of the dorsal hippocampus. Together, our findings demonstrate that the LS accurately and robustly represents spatial, directional as well as self-motion information and is uniquely positioned to relay this information from the hippocampus to its downstream regions, thus occupying a key position within a distributed spatial memory network.

Calcium imaging of neurons in freely behaving mice reveals how the lateral septum, the main output of the hippocampal place cells, effectively represents information about not only location, but also head direction and self-movement, and may be pivotal in sending this information to downstream brain regions.

Choice of method of place cell classification determines the population of cells identified

Place cells, spatially responsive hippocampal cells, provide the neural substrate supporting navigation and spatial memory. Historically most studies of these neurons have used electrophysiological recordings from implanted electrodes but optical methods, measuring intracellular calcium, are becoming increasingly common. Several methods have been proposed as a means to identify place cells based on their calcium activity but there is no common standard and it is unclear how reliable different approaches are. Here we tested four methods that have previously been applied to two-photon hippocampal imaging or electrophysiological data, using both model datasets and real imaging data. These methods use different parameters to identify place cells, including the peak activity in the place field, compared to other locations (the Peak method); the stability of cells' activity over repeated traversals of an environment (Stability method); a combination of these parameters with the size of the place field (Combination method); and the spatial information held by the cells (Information method). The methods performed differently from each other on both model and real data. In real datasets, vastly different numbers of place cells were identified using the four methods, with little overlap between the populations identified as place cells. Therefore, choice of place cell detection method dramatically affects the number and properties of identified cells. Ultimately, we recommend the Peak method be used in future studies to identify place cell populations, as this method is robust to moderate variations in place field within a session, and makes no inherent assumptions about the spatial information in place fields, unless there is an explicit theoretical reason for detecting cells with more narrowly defined properties.

Calcium Imaging and the Curse of Negativity

The imaging of neuronal activity using calcium indicators has become a staple of modern neuroscience. However, without ground truths, there is a real risk of missing a significant portion of the real responses. Here, we show that a common assumption, the non-negativity of the neuronal responses as detected by calcium indicators, biases all levels of the frequently used analytical methods for these data. From the extraction of meaningful fluorescence changes to spike inference and the analysis of inferred spikes, each step risks missing real responses because of the assumption of non-negativity. We first show that negative deviations from baseline can exist in calcium imaging of neuronal activity. Then, we use simulated data to test three popular algorithms for image analysis, CaImAn, suite2p, and CellSort, finding that suite2p may be the best suited to large datasets. We also tested the spike inference algorithms included in CaImAn, suite2p, and Cellsort, as well as the dedicated inference algorithms MLspike and CASCADE, and found each to have limitations in dealing with inhibited neurons. Among these spike inference algorithms, FOOPSI, from CaImAn, performed the best on inhibited neurons, but even this algorithm inferred spurious spikes upon the return of the fluorescence signal to baseline. As such, new approaches will be needed before spikes can be sensitively and accurately inferred from calcium data in inhibited neurons. We further suggest avoiding data analysis approaches that, by assuming non-negativity, ignore inhibited responses. Instead, we suggest a first exploratory step, using k-means or PCA for example, to detect whether meaningful negative deviations are present. Taking these steps will ensure that inhibition, as well as excitation, is detected in calcium imaging datasets.