Modulation of temporal precision in thalamic population responses to natural visual stimuli

Computational assessment of visual coding across mouse brain areas and behavioural states

Introduction

Our brain is bombarded by a diverse range of visual stimuli, which are converted into corresponding neuronal responses and processed throughout the visual system. The neural activity patterns that result from these external stimuli vary depending on the object or scene being observed, but they also change as a result of internal or behavioural states. This raises the question of to what extent it is possible to predict the presented visual stimuli from neural activity across behavioural states, and how this varies in different brain regions.

Methods

To address this question, we assessed the computational capacity of decoders to extract visual information in awake behaving mice, by analysing publicly available standardised datasets from the Allen Brain Institute. We evaluated how natural movie frames can be distinguished based on the activity of units recorded in distinct brain regions and under different behavioural states. This analysis revealed the spectrum of visual information present in different brain regions in response to binary and multiclass classification tasks.

Results

Visual cortical areas showed highest classification accuracies, followed by thalamic and midbrain regions, with hippocampal regions showing close to chance accuracy. In addition, we found that behavioural variability led to a decrease in decoding accuracy, whereby large behavioural changes between train and test sessions reduced the classification performance of the decoders. A generalised linear model analysis suggested that this deterioration in classification might be due to an independent modulation of neural activity by stimulus and behaviour. Finally, we reconstructed the natural movie frames from optimal linear classifiers, and observed a strong similarity between reconstructed and actual movie frames. However, the similarity was significantly higher when the decoders were trained and tested on sessions with similar behavioural states.

Conclusion

Our analysis provides a systematic assessment of visual coding in the mouse brain, and sheds light on the spectrum of visual information present across brain areas and behavioural states.

Topology of synaptic connectivity constrains neuronal stimulus representation, predicting two complementary coding strategies

In motor-related brain regions, movement intention has been successfully decoded from in-vivo spike train by isolating a lower-dimension manifold that the high-dimensional spiking activity is constrained to. The mechanism enforcing this constraint remains unclear, although it has been hypothesized to be implemented by the connectivity of the sampled neurons. We test this idea and explore the interactions between local synaptic connectivity and its ability to encode information in a lower dimensional manifold through simulations of a detailed microcircuit model with realistic sources of noise. We confirm that even in isolation such a model can encode the identity of different stimuli in a lower-dimensional space. We then demonstrate that the reliability of the encoding depends on the connectivity between the sampled neurons by specifically sampling populations whose connectivity maximizes certain topological metrics. Finally, we developed an alternative method for determining stimulus identity from the activity of neurons by combining their spike trains with their recurrent connectivity. We found that this method performs better for sampled groups of neurons that perform worse under the classical approach, predicting the possibility of two separate encoding strategies in a single microcircuit.

Efficient and robust coding in heterogeneous recurrent networks

Cortical networks show a large heterogeneity of neuronal properties. However, traditional coding models have focused on homogeneous populations of excitatory and inhibitory neurons. Here, we analytically derive a class of recurrent networks of spiking neurons that close to optimally track a continuously varying input online, based on two assumptions: 1) every spike is decoded linearly and 2) the network aims to reduce the mean-squared error between the input and the estimate. From this we derive a class of predictive coding networks, that unifies encoding and decoding and in which we can investigate the difference between homogeneous networks and heterogeneous networks, in which each neurons represents different features and has different spike-generating properties. We find that in this framework, 'type 1' and 'type 2' neurons arise naturally and networks consisting of a heterogeneous population of different neuron types are both more efficient and more robust against correlated noise. We make two experimental predictions: 1) we predict that integrators show strong correlations with other integrators and resonators are correlated with resonators, whereas the correlations are much weaker between neurons with different coding properties and 2) that 'type 2' neurons are more coherent with the overall network activity than 'type 1' neurons.

Feedforward Thalamocortical Connectivity Preserves Stimulus Timing Information in Sensory Pathways

Reliable timing of cortical spikes in response to visual events is crucial in representing visual inputs to the brain. Spikes in the primary visual cortex (V1) need to occur at the same time within a repeated visual stimulus. Two classical mechanisms are employed by the cortex to enhance reliable timing. First, cortical neurons respond reliably to a restricted set of stimuli through their preference for certain patterns of membrane potential due to their intrinsic properties. Second, intracortical networking of excitatory and inhibitory neurons induces lateral inhibition that, through the timing and strength of IPSCs and EPSCs, produces sparse and reliably timed cortical neuron spike trains to be transmitted downstream. Here, we describe a third mechanism that, through preferential thalamocortical synaptic connectivity, enhances the trial-to-trial timing precision of cortical spikes in the presence of spike train variability within each trial that is introduced between LGN neurons in the retino-thalamic pathway. Applying experimentally recorded LGN spike trains from the anesthetized cat to a detailed model of a spiny stellate V1 neuron, we found that output spike timing precision improved with increasing numbers of convergent LGN inputs. The improvement was consistent with the predicted proportionality of [Formula: see text] for n LGN source neurons. We also found connectivity configurations that maximize reliability and that generate V1 cell output spike trains quantitatively similar to the experimental recordings. Our findings suggest a general principle, namely intra-trial variability among converging inputs, that increases stimulus response precision and is widely applicable to synaptically connected spiking neurons.SIGNIFICANCE STATEMENT The early visual pathway of the cat is favorable for studying the effects of trial-to-trial variability of synaptic inputs and intra-trial variability of thalamocortical connectivity on information transmission into the visual cortex. We have used a detailed model to show that there are preferred combinations of the number of thalamic afferents and the number of synapses per afferent that maximize the output reliability and spike-timing precision of cortical neurons. This provides additional insights into how synchrony in thalamic spike trains can reduce trial-to-trial variability to produce highly reliable reporting of sensory events to the cortex. The same principles may apply to other converging pathways where temporally jittered spike trains can reliably drive the downstream neuron and improve temporal precision.

Push-Pull Receptive Field Organization and Synaptic Depression: Mechanisms for Reliably Encoding Naturalistic Stimuli in V1

Neurons in the primary visual cortex are known for responding vigorously but with high variability to classical stimuli such as drifting bars or gratings. By contrast, natural scenes are encoded more efficiently by sparse and temporal precise spiking responses. We used a conductance-based model of the visual system in higher mammals to investigate how two specific features of the thalamo-cortical pathway, namely push-pull receptive field organization and fast synaptic depression, can contribute to this contextual reshaping of V1 responses. By comparing cortical dynamics evoked respectively by natural vs. artificial stimuli in a comprehensive parametric space analysis, we demonstrate that the reliability and sparseness of the spiking responses during natural vision is not a mere consequence of the increased bandwidth in the sensory input spectrum. Rather, it results from the combined impacts of fast synaptic depression and push-pull inhibition, the later acting for natural scenes as a form of “effective” feed-forward inhibition as demonstrated in other sensory systems. Thus, the combination of feedforward-like inhibition with fast thalamo-cortical synaptic depression by simple cells receiving a direct structured input from thalamus composes a generic computational mechanism for generating a sparse and reliable encoding of natural sensory events.

The role of thalamic population synchrony in the emergence of cortical feature selectivity

In a wide range of studies, the emergence of orientation selectivity in primary visual cortex has been attributed to a complex interaction between feed-forward thalamic input and inhibitory mechanisms at the level of cortex. Although it is well known that layer 4 cortical neurons are highly sensitive to the timing of thalamic inputs, the role of the stimulus-driven timing of thalamic inputs in cortical orientation selectivity is not well understood. Here we show that the synchronization of thalamic firing contributes directly to the orientation tuned responses of primary visual cortex in a way that optimizes the stimulus information per cortical spike. From the recorded responses of geniculate X-cells in the anesthetized cat, we synthesized thalamic sub-populations that would likely serve as the synaptic input to a common layer 4 cortical neuron based on anatomical constraints. We used this synchronized input as the driving input to an integrate-and-fire model of cortical responses and demonstrated that the tuning properties match closely to those measured in primary visual cortex. By modulating the overall level of synchronization at the preferred orientation, we show that efficiency of information transmission in the cortex is maximized for levels of synchronization which match those reported in thalamic recordings in response to naturalistic stimuli, a property which is relatively invariant to the orientation tuning width. These findings indicate evidence for a more prominent role of the feed-forward thalamic input in cortical feature selectivity based on thalamic synchronization.

While the visual system is selective for a wide range of different inputs, orientation selectivity has been considered the preeminent property of the mammalian visual cortex. Existing models of this selectivity rely on varying relative importance of feedforward thalamic input and intracortical influence. Recently, we have shown that pairwise timing relationships between single thalamic neurons can be predictive of a high degree of orientation selectivity. Here we have constructed a computational model that predicts cortical orientation tuning from thalamic populations. We show that this arrangement, relying on precise timing differences between thalamic responses, accurately predicts tuning properties as well as demonstrates that certain timing relationships are optimal for transmitting information about the stimulus to cortex.

Animation of natural scene by virtual eye-movements evokes high precision and low noise in V1 neurons

Synaptic noise is thought to be a limiting factor for computational efficiency in the brain. In visual cortex (V1), ongoing activity is present in vivo, and spiking responses to simple stimuli are highly unreliable across trials. Stimulus statistics used to plot receptive fields, however, are quite different from those experienced during natural visuomotor exploration. We recorded V1 neurons intracellularly in the anaesthetized and paralyzed cat and compared their spiking and synaptic responses to full field natural images animated by simulated eye-movements to those evoked by simpler (grating) or higher dimensionality statistics (dense noise). In most cells, natural scene animation was the only condition where high temporal precision (in the 10–20 ms range) was maintained during sparse and reliable activity. At the subthreshold level, irregular but highly reproducible membrane potential dynamics were observed, even during long (several 100 ms) “spike-less” periods. We showed that both the spatial structure of natural scenes and the temporal dynamics of eye-movements increase the signal-to-noise ratio by a non-linear amplification of the signal combined with a reduction of the subthreshold contextual noise. These data support the view that the sparsening and the time precision of the neural code in V1 may depend primarily on three factors: (1) broadband input spectrum: the bandwidth must be rich enough for recruiting optimally the diversity of spatial and time constants during recurrent processing; (2) tight temporal interplay of excitation and inhibition: conductance measurements demonstrate that natural scene statistics narrow selectively the duration of the spiking opportunity window during which the balance between excitation and inhibition changes transiently and reversibly; (3) signal energy in the lower frequency band: a minimal level of power is needed below 10 Hz to reach consistently the spiking threshold, a situation rarely reached with visual dense noise.

Visual orientation and directional selectivity through thalamic synchrony

Thalamic neurons respond to visual scenes by generating synchronous spike trains on the timescale of 10-20 ms that are very effective at driving cortical targets. Here we demonstrate that this synchronous activity contains unexpectedly rich information about fundamental properties of visual stimuli. We report that the occurrence of synchronous firing of cat thalamic cells with highly overlapping receptive fields is strongly sensitive to the orientation and the direction of motion of the visual stimulus. We show that this stimulus selectivity is robust, remaining relatively unchanged under different contrasts and temporal frequencies (stimulus velocities). A computational analysis based on an integrate-and-fire model of the direct thalamic input to a layer 4 cortical cell reveals a strong correlation between the degree of thalamic synchrony and the nonlinear relationship between cortical membrane potential and the resultant firing rate. Together, these findings suggest a novel population code in the synchronous firing of neurons in the early visual pathway that could serve as the substrate for establishing cortical representations of the visual scene.

Is a 4-bit synaptic weight resolution enough? - constraints on enabling spike-timing dependent plasticity in neuromorphic hardware

Large-scale neuromorphic hardware systems typically bear the trade-off between detail level and required chip resources. Especially when implementing spike-timing dependent plasticity, reduction in resources leads to limitations as compared to floating point precision. By design, a natural modification that saves resources would be reducing synaptic weight resolution. In this study, we give an estimate for the impact of synaptic weight discretization on different levels, ranging from random walks of individual weights to computer simulations of spiking neural networks. The FACETS wafer-scale hardware system offers a 4-bit resolution of synaptic weights, which is shown to be sufficient within the scope of our network benchmark. Our findings indicate that increasing the resolution may not even be useful in light of further restrictions of customized mixed-signal synapses. In addition, variations due to production imperfections are investigated and shown to be uncritical in the context of the presented study. Our results represent a general framework for setting up and configuring hardware-constrained synapses. We suggest how weight discretization could be considered for other backends dedicated to large-scale simulations. Thus, our proposition of a good hardware verification practice may rise synergy effects between hardware developers and neuroscientists.

Natural movies evoke spike trains with low spike time variability in cat primary visual cortex

Neuronal responses in primary visual cortex have been found to be highly variable. This has led to the widespread notion that neuronal responses have to be averaged over large numbers of neurons to obtain suitably invariant responses that can be used to reliably encode or represent external stimuli. However, it is possible that the high variability of neuronal responses may result from the use of simple, artificial stimuli and that the visual cortex may respond differently to dynamic, naturalistic images. To investigate this question, we recorded the responses of primary visual cortical neurons in the anesthetized cat under stimulation with time-varying natural movies. We found that cortical neurons on the whole exhibited a high degree of spike count variability, but a surprisingly low degree of spike time variability. The spike count variability was further reduced when all but the first spike in a burst were removed. We also found that responses exhibiting low spike time variability exhibited low spike count variability, suggesting that rate coding and temporal coding might be more compatible than previously thought. In addition, we found the spike time variability to be significantly lower when stimulated by natural movies as compared with stimulation using drifting gratings. Our results indicate that response variability in primary visual cortex is stimulus dependent and significantly lower than previous measurements have indicated.

The episodic nature of spike trains in the early visual pathway

An understanding of the neural code in a given visual area is often confounded by the immense complexity of visual stimuli combined with the number of possible meaningful patterns that comprise the response spike train. In the lateral geniculate nucleus (LGN), visual stimulation generates spike trains comprised of short spiking episodes ("events") separated by relatively long intervals of silence, which establishes a basis for in-depth analysis of the neural code. By studying this event structure in both artificial and natural visual stimulus contexts and at different contrasts, we are able to describe the dependence of event structure on stimulus class and discern which aspects generalize. We find that the event structure on coarse time scales is robust across stimulus and contrast and can be explained by receptive field processing. However, the relationship between the stimulus and fine-time-scale features of events is less straightforward, partially due to a significant amount of trial-to-trial variability. A new measure called "label information" identifies structural elements of events that can contain ≤30% more information in the context of natural movies compared with what is available from the overall event timing. The first interspike interval of an event most robustly conveys additional information about the stimulus and is somewhat more informative than the event spike count and much more informative than the presence of bursts. Nearly every event is preserved across contrast despite changes in their fine-time-scale features, suggesting that--at least on a coarse level--the stimulus selectivity of LGN neurons is contrast invariant. Event-based analysis thus casts previously studied elements of LGN coding such as contrast adaptation and receptive field processing in a new light and leads to broad conclusions about the composition of the LGN neuronal code.