Integration of visual motion and locomotion in mouse visual cortex

Wide-field calcium imaging of cortical activation and functional connectivity in externally- and internally-driven locomotion

The role of the cerebral cortex in self-initiated versus sensory-driven movements is central to understanding volitional action. Whether the differences in these two movement classes are due to specific cortical areas versus more cortex-wide engagement is debated. Using wide-field Ca2+ imaging, we compared neural dynamics during spontaneous and motorized treadmill locomotion, determining the similarities and differences in cortex-wide activation and functional connectivity (FC). During motorized locomotion, the cortex exhibits greater activation globally prior to and during locomotion starting compared to spontaneous and less during steady-state walking, during stopping, and after termination. Both conditions are characterized by FC increases in anterior secondary motor cortex (M2) nodes and decreases in all other regions. There are also cortex-wide differences; most notably, M2 decreases in FC with all other nodes during motorized stopping and after termination. Therefore, both internally- and externally-generated movements widely engage the cortex, with differences represented in cortex-wide activation and FC patterns.

Modulation of cortical representations of sensory and contextual information underlies aversive associative learning

Cortical neurons encode both sensory and contextual information, yet it remains unclear how experiences modulate these cortical representations. Here, we demonstrate that trace eyeblink conditioning (TEC), an aversive associative-learning paradigm linking conditioned (CS) with unconditioned stimuli (US), finely tunes cortical coding at both population and single-neuron levels. Initially, we show that the primary somatosensory cortex (S1) is necessary for TEC acquisition, as evidenced by local muscimol administration. At the population level, TEC enhances activity in a small subset (∼20%) of CS- or US-responsive primary neurons (rPNs) while diminishing activity in non-rPNs, including locomotion-tuned or unresponsive PNs. Crucially, TEC learning modulates the encoding of sensory versus contextual information in single rPNs: CS-responsive neurons become less responsive, while US-responsive neurons gain responses to CS. Moreover, we find that the cholinergic pathway, via nicotinic receptors, underlies TEC-induced modulations. These findings suggest that experiences dynamically tune cortical representations through cholinergic pathways.

Decoding dynamic visual scenes across the brain hierarchy

Understanding the computational mechanisms that underlie the encoding and decoding of environmental stimuli is a crucial investigation in neuroscience. Central to this pursuit is the exploration of how the brain represents visual information across its hierarchical architecture. A prominent challenge resides in discerning the neural underpinnings of the processing of dynamic natural visual scenes. Although considerable research efforts have been made to characterize individual components of the visual pathway, a systematic understanding of the distinctive neural coding associated with visual stimuli, as they traverse this hierarchical landscape, remains elusive. In this study, we leverage the comprehensive Allen Visual Coding-Neuropixels dataset and utilize the capabilities of deep learning neural network models to study neural coding in response to dynamic natural visual scenes across an expansive array of brain regions. Our study reveals that our decoding model adeptly deciphers visual scenes from neural spiking patterns exhibited within each distinct brain area. A compelling observation arises from the comparative analysis of decoding performances, which manifests as a notable encoding proficiency within the visual cortex and subcortical nuclei, in contrast to a relatively reduced encoding activity within hippocampal neurons. Strikingly, our results unveil a robust correlation between our decoding metrics and well-established anatomical and functional hierarchy indexes. These findings corroborate existing knowledge in visual coding related to artificial visual stimuli and illuminate the functional role of these deeper brain regions using dynamic stimuli. Consequently, our results suggest a novel perspective on the utility of decoding neural network models as a metric for quantifying the encoding quality of dynamic natural visual scenes represented by neural responses, thereby advancing our comprehension of visual coding within the complex hierarchy of the brain.

Flexible neural population dynamics govern the speed and stability of sensory encoding in mouse visual cortex

Time courses of neural responses underlie real-time sensory processing and perception. How these temporal dynamics change may be fundamental to how sensory systems adapt to different perceptual demands. By simultaneously recording from hundreds of neurons in mouse primary visual cortex, we examined neural population responses to visual stimuli at sub-second timescales, during different behavioural states. We discovered that during active behavioural states characterised by locomotion, single-neurons shift from transient to sustained response modes, facilitating rapid emergence of visual stimulus tuning. Differences in single-neuron response dynamics were associated with changes in temporal dynamics of neural correlations, including faster stabilisation of stimulus-evoked changes in the structure of correlations during locomotion. Using Factor Analysis, we examined temporal dynamics of latent population responses and discovered that trajectories of population activity make more direct transitions between baseline and stimulus-encoding neural states during locomotion. This could be partly explained by dampening of oscillatory dynamics present during stationary behavioural states. Functionally, changes in temporal response dynamics collectively enabled faster, more stable and more efficient encoding of new visual information during locomotion. These findings reveal a principle of how sensory systems adapt to perceptual demands, where flexible neural population dynamics govern the speed and stability of sensory encoding.

Context-dependence of deterministic and nondeterministic contributions to closed-loop steering control

In natural circumstances, sensory systems operate in a closed loop with motor output, whereby actions shape subsequent sensory experiences. A prime example of this is the sensorimotor processing required to align one's direction of travel, or heading, with one's goal, a behavior we refer to as steering. In steering, motor outputs work to eliminate errors between the direction of heading and the goal, modifying subsequent errors in the process. The closed-loop nature of the behavior makes it challenging to determine how deterministic and nondeterministic processes contribute to behavior. We overcome this by applying a nonparametric, linear kernel-based analysis to behavioral data of monkeys steering through a virtual environment in two experimental contexts. In a given context, the results were consistent with previous work that described the transformation as a second-order linear system. Classically, the parameters of such second-order models are associated with physical properties of the limb such as viscosity and stiffness that are commonly assumed to be approximately constant. By contrast, we found that the fit kernels differed strongly across tasks in these and other parameters, suggesting context-dependent changes in neural and biomechanical processes. We additionally fit residuals to a simple noise model and found that the form of the noise was highly conserved across both contexts and animals. Strikingly, the fitted noise also closely matched that found previously in a human steering task. Altogether, this work presents a kernel-based analysis that characterizes the context-dependence of deterministic and non-deterministic components of a closed-loop sensorimotor task.

Environmental context sculpts spatial and temporal visual processing in thalamus

Behavioral state modulates neural activity throughout the visual system; this is largely due to changes in arousal that alter internal brain state. However, behaviors are constrained by the external environmental context, so it remains unclear if this context itself dictates the regime of visual processing, apart from ongoing changes in arousal. Here, we addressed this question in awake head-fixed mice while they passively viewed visual stimuli in two different environmental contexts: either a cylindrical tube, or a circular running wheel. We targeted high-density silicon probe recordings to the dorsal lateral geniculate nucleus (dLGN) and simultaneously measured several electrophysiological and behavioral correlates of arousal changes, and thus controlled for them across contexts. We found surprising differences in spatial and temporal processing in dLGN across contexts, even in identical states of alertness and stillness. The wheel context (versus tube) showed elevated baseline activity, faster visual responses, and smaller but less selective spatial receptive fields. Further, arousal caused similar changes to visual responsiveness across all conditions, but the environmental context mainly changed the overall set-point for this relationship. Together, our results reveal an unexpected influence of the physical environmental context on fundamental aspects of visual processing in the early visual system.

Cholinergic input to mouse visual cortex signals a movement state and acutely enhances layer 5 responsiveness

Acetylcholine is released in visual cortex by axonal projections from the basal forebrain. The signals conveyed by these projections and their computational significance are still unclear. Using two-photon calcium imaging in behaving mice, we show that basal forebrain cholinergic axons in the mouse visual cortex provide a binary locomotion state signal. In these axons, we found no evidence of responses to visual stimuli or visuomotor prediction errors. While optogenetic activation of cholinergic axons in visual cortex in isolation did not drive local neuronal activity, when paired with visuomotor stimuli, it resulted in layer-specific increases of neuronal activity. Responses in layer 5 neurons to both top-down and bottom-up inputs were increased in amplitude and decreased in latency, whereas those in layer 2/3 neurons remained unchanged. Using opto- and chemogenetic manipulations of cholinergic activity, we found acetylcholine to underlie the locomotion-associated decorrelation of activity between neurons in both layer 2/3 and layer 5. Our results suggest that acetylcholine augments the responsiveness of layer 5 neurons to inputs from outside of the local network, possibly enabling faster switching between internal representations during locomotion.

Repeated exposure to multiple concurrent stressors alters visual processing in the adult posterior parietal cortex

Chronic stress is well known to erode cognitive functions. Yet, our understanding of how repeated stress exposure impacts one of the fundamental bases of cognition: sensory processing, remains limited. The posterior parietal cortex (PPC) is a high order visual region, known for its role in visually guided decision making, multimodal integration, attention, and working memory. Here, we used functional measures to determine how repeated exposure to multiple concurrent stressors (RMS) affects sensory processing in the PPC in adult male mice. A longitudinal experimental design, repeatedly surveying the same population of neurons using in vivo two-photon imaging, revealed that RMS disrupts the balanced turnover of visually responsive cells in layer 2/3 of the PPC. Across the population, RMS-induced changes in visual responsiveness followed a bimodal distribution suggesting idiosyncratic stress effects. In cells that maintained their responsiveness across recording sessions, we found that stress reduced visual response magnitudes and feature selectivity. While we did not observe stress-induced elimination of excitatory synapses, noise correlation statistics indicated that RMS altered visual input to the neuronal population. The impact of RMS was restricted to visually evoked responses and was not evident in neuronal activity associated with locomotion onset. Together, our results indicate that despite no apparent synaptic reorganization, stress exposure in adulthood can disrupt sensory processing in the PPC, with the effects showing remarkable individual variation.

Common and distinct neural mechanisms of attention

Despite a constant deluge of sensory stimulation, only a fraction of it is used to guide behavior. This selective processing is generally referred to as attention, and much research has focused on the neural mechanisms controlling it. Recently, research has broadened to include more ways by which different species selectively process sensory information, whether due to the sensory input itself or to different behavioral and brain states. This work has produced a complex and disjointed body of evidence across different species and forms of attention. However, it has also provided opportunities to better understand the breadth of attentional mechanisms. Here, we summarize the evidence that suggests that different forms of selective processing are supported by mechanisms both common and distinct.

Feedforward and disinhibitory circuits differentially control activity of cortical somatostatin interneurons during behavioral state transitions

Interneurons (INs), specifically those in disinhibitory circuits like somatostatin (SST) and vasoactive intestinal peptide (VIP)-INs, are strongly modulated by the behavioral context. Yet, the mechanisms by which these INs are recruited during active states and whether their activity is consistent across sensory cortices remain unclear. We now report that in mice, locomotor activity strongly recruits SST-INs in the primary somatosensory (S1) but not the visual (V1) cortex. This diverse engagement of SST-INs cannot be explained by differences in VIP-IN function but is absent in the presence of visual input, suggesting the involvement of feedforward sensory pathways. Accordingly, inactivating the somatosensory thalamus, but not decreasing VIP-IN activity, significantly reduces the modulation of SST-INs by locomotion. Model simulations suggest that the differences in SST-INs across behavioral states can be explained by varying ratios of VIP- and thalamus-driven activity. By integrating feedforward activity with neuromodulation, SST-INs are anticipated to be crucial for adapting sensory processing to behavioral states.

The meso-connectomes of mouse, marmoset, and macaque: network organization and the emergence of higher cognition

The recent publications of the inter-areal connectomes for mouse, marmoset, and macaque cortex have allowed deeper comparisons across rodent vs. primate cortical organization. In general, these show that the mouse has very widespread, "all-to-all" inter-areal connectivity (i.e. a "highly dense" connectome in a graph theoretical framework), while primates have a more modular organization. In this review, we highlight the relevance of these differences to function, including the example of primary visual cortex (V1) which, in the mouse, is interconnected with all other areas, therefore including other primary sensory and frontal areas. We argue that this dense inter-areal connectivity benefits multimodal associations, at the cost of reduced functional segregation. Conversely, primates have expanded cortices with a modular connectivity structure, where V1 is almost exclusively interconnected with other visual cortices, themselves organized in relatively segregated streams, and hierarchically higher cortical areas such as prefrontal cortex provide top-down regulation for specifying precise information for working memory storage and manipulation. Increased complexity in cytoarchitecture, connectivity, dendritic spine density, and receptor expression additionally reveal a sharper hierarchical organization in primate cortex. Together, we argue that these primate specializations permit separable deconstruction and selective reconstruction of representations, which is essential to higher cognition.

Cell-type-specific optogenetic fMRI on basal forebrain reveals functional network basis of behavioral preference

The basal forebrain (BF) is a complex structure that plays key roles in regulating various brain functions. However, it remains unclear how cholinergic and non-cholinergic BF neurons modulate large-scale functional networks and their relevance in intrinsic and extrinsic behaviors. With an optimized awake mouse optogenetic fMRI approach, we revealed that optogenetic stimulation of four BF neuron types evoked distinct cell-type-specific whole-brain BOLD activations, which could be attributed to BF-originated low-dimensional structural networks. Additionally, optogenetic activation of VGLUT2, ChAT, and PV neurons in the BF modulated the preference for locomotion, exploration, and grooming, respectively. Furthermore, we uncovered the functional network basis of the above BF-modulated behavioral preference through a decoding model linking the BF-modulated BOLD activation, low-dimensional structural networks, and behavioral preference. To summarize, we decoded the functional network basis of differential behavioral preferences with cell-type-specific optogenetic fMRI on the BF and provided an avenue for investigating mouse behaviors from a whole-brain view.

Disruption of Cholinergic Retinal Waves Alters Visual Cortex Development and Function

Retinal waves represent an early form of patterned spontaneous neural activity in the visual system. These waves originate in the retina before eye-opening and propagate throughout the visual system, influencing the assembly and maturation of subcortical visual brain regions. However, because it is technically challenging to ablate retina-derived cortical waves without inducing compensatory activity, the role these waves play in the development of the visual cortex remains unclear. To address this question, we used targeted conditional genetics to disrupt cholinergic retinal waves and their propagation to select regions of primary visual cortex, which largely prevented compensatory patterned activity. We find that loss of cholinergic retinal waves without compensation impaired the molecular and synaptic maturation of excitatory neurons located in the input layers of visual cortex, as well as layer 1 interneurons. These perinatal molecular and synaptic deficits also relate to functional changes observed at later ages. We find that the loss of perinatal cholinergic retinal waves causes abnormal visual cortex retinotopy, mirroring changes in the retinotopic organization of gene expression, and additionally impairs the processing of visual information. We further show that retinal waves are necessary for higher order processing of sensory information by impacting the state-dependent activity of layer 1 interneurons, a neuronal type that shapes neocortical state-modulation, as well as for state-dependent gain modulation of visual responses of excitatory neurons. Together, these results demonstrate that a brief targeted perinatal disruption of patterned spontaneous activity alters early cortical gene expression as well as synaptic and physiological development, and compromises both fundamental and, notably, higher-order functions of visual cortex after eye-opening.

Stimulus encoding by specific inactivation of cortical neurons

Neuronal ensembles are groups of neurons with correlated activity associated with sensory, motor, and behavioral functions. To explore how ensembles encode information, we investigated responses of visual cortical neurons in awake mice using volumetric two-photon calcium imaging during visual stimulation. We identified neuronal ensembles employing an unsupervised model-free algorithm and, besides neurons activated by the visual stimulus (termed "onsemble"), we also find neurons that are specifically inactivated (termed "offsemble"). Offsemble neurons showed faster calcium decay during stimuli, suggesting selective inhibition. In response to visual stimuli, each ensemble (onsemble+offsemble) exhibited small trial-to-trial variability, high orientation selectivity, and superior predictive accuracy for visual stimulus orientation, surpassing the sum of individual neuron activity. Thus, the combined selective activation and inactivation of cortical neurons enhances visual encoding as an emergent and distributed neural code.

Antipsychotic drugs selectively decorrelate long-range interactions in deep cortical layers

Psychosis is characterized by a diminished ability of the brain to distinguish externally driven activity patterns from self-generated activity patterns. Antipsychotic drugs are a class of small molecules with relatively broad binding affinity for a variety of neuromodulator receptors that, in humans, can prevent or ameliorate psychosis. How these drugs influence the function of cortical circuits, and in particular their ability to distinguish between externally and self-generated activity patterns, is still largely unclear. To have experimental control over self-generated sensory feedback, we used a virtual reality environment in which the coupling between movement and visual feedback can be altered. We then used widefield calcium imaging to determine the cell type-specific functional effects of antipsychotic drugs in mouse dorsal cortex under different conditions of visuomotor coupling. By comparing cell type-specific activation patterns between locomotion onsets that were experimentally coupled to self-generated visual feedback and locomotion onsets that were not coupled, we show that deep cortical layers were differentially activated in these two conditions. We then show that the antipsychotic drug clozapine disrupted visuomotor integration at locomotion onsets also primarily in deep cortical layers. Given that one of the key components of visuomotor integration in cortex is long-range cortico-cortical connections, we tested whether the effect of clozapine was detectable in the correlation structure of activity patterns across dorsal cortex. We found that clozapine as well as two other antipsychotic drugs, aripiprazole and haloperidol, resulted in a strong reduction in correlations of layer 5 activity between cortical areas and impaired the spread of visuomotor prediction errors generated in visual cortex. Our results are consistent with the interpretation that a major functional effect of antipsychotic drugs is a selective alteration of long-range layer 5-mediated communication.

Movement-Related Modulation in Mouse Auditory Cortex Is Widespread Yet Locally Diverse

Neurons in the mouse auditory cortex are strongly influenced by behavior, including both suppression and enhancement of sound-evoked responses during movement. The mouse auditory cortex comprises multiple fields with different roles in sound processing and distinct connectivity to movement-related centers of the brain. Here, we asked whether movement-related modulation in male mice might differ across auditory cortical fields, thereby contributing to the heterogeneity of movement-related modulation at the single-cell level. We used wide-field calcium imaging to identify distinct cortical fields and cellular-resolution two-photon calcium imaging to visualize the activity of layer 2/3 excitatory neurons within each field. We measured each neuron's responses to three sound categories (pure tones, chirps, and amplitude-modulated white noise) as mice rested and ran on a non-motorized treadmill. We found that individual neurons in each cortical field typically respond to just one sound category. Some neurons are only active during rest and others during locomotion, and those that are responsive across conditions retain their sound-category tuning. The effects of locomotion on sound-evoked responses vary at the single-cell level, with both suppression and enhancement of neural responses, and the net modulatory effect of locomotion is largely conserved across cortical fields. Movement-related modulation in auditory cortex also reflects more complex behavioral patterns, including instantaneous running speed and nonlocomotor movements such as grooming and postural adjustments, with similar patterns seen across all auditory cortical fields. Our findings underscore the complexity of movement-related modulation throughout the mouse auditory cortex and indicate that movement-related modulation is a widespread phenomenon.

Walking modulates visual detection performance according to stride cycle phase

Walking is among our most frequent and natural of voluntary behaviours, yet the consequences of locomotion upon perceptual and cognitive function remain largely unknown. Recent work has highlighted that although walking feels smooth and continuous, critical phases exist within each step for the successful coordination of perceptual and motor function. Here, we test whether these phasic demands impact upon visual perception, by assessing performance in a visual detection task during natural unencumbered walking. We finely sample visual performance over the stride cycle as participants walk along a smooth linear path at a comfortable speed in a wireless virtual reality environment. At the group-level, accuracy, reaction times, and response likelihood show strong oscillations, modulating at approximately 2 cycles per stride (~2 Hz) with a marked phase of optimal performance aligned with the swing phase of each step. At the participant level, Bayesian inference of population prevalence reveals highly prevalent oscillations in visual detection performance that cluster in two idiosyncratic frequency ranges (2 or 4 cycles per stride), with a strong phase alignment across participants.

Modeling circuit mechanisms of opposing cortical responses to visual flow perturbations

In an ever-changing visual world, animals' survival depends on their ability to perceive and respond to rapidly changing motion cues. The primary visual cortex (V1) is at the forefront of this sensory processing, orchestrating neural responses to perturbations in visual flow. However, the underlying neural mechanisms that lead to distinct cortical responses to such perturbations remain enigmatic. In this study, our objective was to uncover the neural dynamics that govern V1 neurons' responses to visual flow perturbations using a biologically realistic computational model. By subjecting the model to sudden changes in visual input, we observed opposing cortical responses in excitatory layer 2/3 (L2/3) neurons, namely, depolarizing and hyperpolarizing responses. We found that this segregation was primarily driven by the competition between external visual input and recurrent inhibition, particularly within L2/3 and L4. This division was not observed in excitatory L5/6 neurons, suggesting a more prominent role for inhibitory mechanisms in the visual processing of the upper cortical layers. Our findings share similarities with recent experimental studies focusing on the opposing influence of top-down and bottom-up inputs in the mouse primary visual cortex during visual flow perturbations.

Decision-making dynamics are predicted by arousal and uninstructed movements

During sensory-guided behavior, an animal's decision-making dynamics unfold through sequences of distinct performance states, even while stimulus-reward contingencies remain static. Little is known about the factors that underlie these changes in task performance. We hypothesize that these decision-making dynamics can be predicted by externally observable measures, such as uninstructed movements and changes in arousal. Here, using computational modeling of visual and auditory task performance data from mice, we uncovered lawful relationships between transitions in strategic task performance states and an animal's arousal and uninstructed movements. Using hidden Markov models applied to behavioral choices during sensory discrimination tasks, we find that animals fluctuate between minutes-long optimal, sub-optimal, and disengaged performance states. Optimal state epochs are predicted by intermediate levels, and reduced variability, of pupil diameter and movement. Our results demonstrate that externally observable uninstructed behaviors can predict optimal performance states and suggest that mice regulate their arousal during optimal performance.

Unpacking the navigation toolbox: insights from comparative cognition

The study of navigation is informed by ethological data from many species, laboratory investigation at behavioural and neurobiological levels, and computational modelling. However, the data are often species-specific, making it challenging to develop general models of how biology supports behaviour. Wiener et al. outlined a framework for organizing the results across taxa, called the 'navigation toolbox' (Wiener et al. In Animal thinking: contemporary issues in comparative cognition (eds R Menzel, J Fischer), pp. 51-76). This framework proposes that spatial cognition is a hierarchical process in which sensory inputs at the lowest level are successively combined into ever-more complex representations, culminating in a metric or quasi-metric internal model of the world (cognitive map). Some animals, notably humans, also use symbolic representations to produce an external representation, such as a verbal description, signpost or map that allows communication of spatial information or instructions between individuals. Recently, new discoveries have extended our understanding of how spatial representations are constructed, highlighting that the hierarchical relationships are bidirectional, with higher levels feeding back to influence lower levels. In the light of these new developments, we revisit the navigation toolbox, elaborate it and incorporate new findings. The toolbox provides a common framework within which the results from different taxa can be described and compared, yielding a more detailed, mechanistic and generalized understanding of navigation.

Sleep restores an optimal computational regime in cortical networks

Sleep is assumed to subserve homeostatic processes in the brain; however, the set point around which sleep tunes circuit computations is unknown. Slow-wave activity (SWA) is commonly used to reflect the homeostatic aspect of sleep; although it can indicate sleep pressure, it does not explain why animals need sleep. This study aimed to assess whether criticality may be the computational set point of sleep. By recording cortical neuron activity continuously for 10-14 d in freely behaving rats, we show that normal waking experience progressively disrupts criticality and that sleep functions to restore critical dynamics. Criticality is perturbed in a context-dependent manner, and waking experience is causal in driving these effects. The degree of deviation from criticality predicts future sleep/wake behavior more accurately than SWA, behavioral history or other neural measures. Our results demonstrate that perturbation and recovery of criticality is a network homeostatic mechanism consistent with the core, restorative function of sleep.

Time-varying generalized linear models: characterizing and decoding neuronal dynamics in higher visual areas

To create a behaviorally relevant representation of the visual world, neurons in higher visual areas exhibit dynamic response changes to account for the time-varying interactions between external (e.g., visual input) and internal (e.g., reward value) factors. The resulting high-dimensional representational space poses challenges for precisely quantifying individual factors' contributions to the representation and readout of sensory information during a behavior. The widely used point process generalized linear model (GLM) approach provides a powerful framework for a quantitative description of neuronal processing as a function of various sensory and non-sensory inputs (encoding) as well as linking particular response components to particular behaviors (decoding), at the level of single trials and individual neurons. However, most existing variations of GLMs assume the neural systems to be time-invariant, making them inadequate for modeling nonstationary characteristics of neuronal sensitivity in higher visual areas. In this review, we summarize some of the existing GLM variations, with a focus on time-varying extensions. We highlight their applications to understanding neural representations in higher visual areas and decoding transient neuronal sensitivity as well as linking physiology to behavior through manipulation of model components. This time-varying class of statistical models provide valuable insights into the neural basis of various visual behaviors in higher visual areas and hold significant potential for uncovering the fundamental computational principles that govern neuronal processing underlying various behaviors in different regions of the brain.

Computational components of visual predictive coding circuitry

If a full visual percept can be said to be a 'hypothesis', so too can a neural 'prediction' - although the latter addresses one particular component of image content (such as 3-dimensional organisation, the interplay between lighting and surface colour, the future trajectory of moving objects, and so on). And, because processing is hierarchical, predictions generated at one level are conveyed in a backward direction to a lower level, seeking to predict, in fact, the neural activity at that prior stage of processing, and learning from errors signalled in the opposite direction. This is the essence of 'predictive coding', at once an algorithm for information processing and a theoretical basis for the nature of operations performed by the cerebral cortex. Neural models for the implementation of predictive coding invoke specific functional classes of neuron for generating, transmitting and receiving predictions, and for producing reciprocal error signals. Also a third general class, 'precision' neurons, tasked with regulating the magnitude of error signals contingent upon the confidence placed upon the prediction, i.e., the reliability and behavioural utility of the sensory data that it predicts. So, what is the ultimate source of a 'prediction'? The answer is multifactorial: knowledge of the current environmental context and the immediate past, allied to memory and lifetime experience of the way of the world, doubtless fine-tuned by evolutionary history too. There are, in consequence, numerous potential avenues for experimenters seeking to manipulate subjects' expectation, and examine the neural signals elicited by surprising, and less surprising visual stimuli. This review focuses upon the predictive physiology of mouse and monkey visual cortex, summarising and commenting on evidence to date, and placing it in the context of the broader field. It is concluded that predictive coding has a firm grounding in basic neuroscience and that, unsurprisingly, there remains much to learn.

Brain-wide neural recordings in mice navigating physical spaces enabled by a cranial exoskeleton

Complex behaviors are mediated by neural computations occurring throughout the brain. In recent years, tremendous progress has been made in developing technologies that can record neural activity at cellular resolution at multiple spatial and temporal scales. However, these technologies are primarily designed for studying the mammalian brain during head fixation - wherein the behavior of the animal is highly constrained. Miniaturized devices for studying neural activity in freely behaving animals are largely confined to recording from small brain regions owing to performance limitations. We present a cranial exoskeleton that assists mice in maneuvering neural recording headstages that are orders of magnitude larger and heavier than the mice, while they navigate physical behavioral environments. Force sensors embedded within the headstage are used to detect the mouse's milli-Newton scale cranial forces which then control the x, y, and yaw motion of the exoskeleton via an admittance controller. We discovered optimal controller tuning parameters that enable mice to locomote at physiologically realistic velocities and accelerations while maintaining natural walking gait. Mice maneuvering headstages weighing up to 1.5 kg can make turns, navigate 2D arenas, and perform a navigational decision-making task with the same performance as when freely behaving. We designed an imaging headstage and an electrophysiology headstage for the cranial exoskeleton to record brain-wide neural activity in mice navigating 2D arenas. The imaging headstage enabled recordings of Ca2+ activity of 1000s of neurons distributed across the dorsal cortex. The electrophysiology headstage supported independent control of up to 4 silicon probes, enabling simultaneous recordings from 100s of neurons across multiple brain regions and multiple days. Cranial exoskeletons provide flexible platforms for largescale neural recording during the exploration of physical spaces, a critical new paradigm for unraveling the brain-wide neural mechanisms that control complex behavior.

Activity in primate visual cortex is minimally driven by spontaneous movements

Organisms process sensory information in the context of their own moving bodies, an idea referred to as embodiment. This idea is important for developmental neuroscience, robotics and systems neuroscience. The mechanisms supporting embodiment are unknown, but a manifestation could be the observation in mice of brain-wide neuromodulation, including in the primary visual cortex, driven by task-irrelevant spontaneous body movements. We tested this hypothesis in macaque monkeys (Macaca mulatta), a primate model for human vision, by simultaneously recording visual cortex activity and facial and body movements. We also sought a direct comparison using an analogous approach to those used in mouse studies. Here we found that activity in the primate visual cortex (V1, V2 and V3/V3A) was associated with the animals' own movements, but this modulation was largely explained by the impact of the movements on the retinal image, that is, by changes in visual input. These results indicate that visual cortex in primates is minimally driven by spontaneous movements and may reflect species-specific sensorimotor strategies.

Intrinsic forebrain arousal dynamics governs sensory stimulus encoding

The neural encoding of sensory stimuli is subject to the brain's internal circuit dynamics. Recent work has demonstrated that the resting brain exhibits widespread, coordinated activity that plays out over multisecond timescales in the form of quasi-periodic spiking cascades. Here we demonstrate that these intrinsic dynamics persist during the presentation of visual stimuli and markedly influence the efficacy of feature encoding in the visual cortex. During periods of passive viewing, the sensory encoding of visual stimuli was determined by quasi-periodic cascade cycle evolving over several seconds. During this cycle, high efficiency encoding occurred during peak arousal states, alternating in time with hippocampal ripples, which were most frequent in low arousal states. However, during bouts of active locomotion, these arousal dynamics were abolished: the brain remained in a state in which visual coding efficiency remained high and ripples were absent. We hypothesize that the brain's observed dynamics during awake, passive viewing reflect an adaptive cycle of alternating exteroceptive sensory sampling and internal mnemonic function.

How 'visual' is the visual cortex? The interactions between the visual cortex and other sensory, motivational and motor systems as enabling factors for visual perception

The definition of the visual cortex is primarily based on the evidence that lesions of this area impair visual perception. However, this does not exclude that the visual cortex may process more information than of retinal origin alone, or that other brain structures contribute to vision. Indeed, research across the past decades has shown that non-visual information, such as neural activity related to reward expectation and value, locomotion, working memory and other sensory modalities, can modulate primary visual cortical responses to retinal inputs. Nevertheless, the function of this non-visual information is poorly understood. Here we review recent evidence, coming primarily from studies in rodents, arguing that non-visual and motor effects in visual cortex play a role in visual processing itself, for instance disentangling direct auditory effects on visual cortex from effects of sound-evoked orofacial movement. These findings are placed in a broader framework casting vision in terms of predictive processing under control of frontal, reward- and motor-related systems. In contrast to the prevalent notion that vision is exclusively constructed by the visual cortical system, we propose that visual percepts are generated by a larger network-the extended visual system-spanning other sensory cortices, supramodal areas and frontal systems. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Learning induced neuronal identity switch in the superficial layers of the primary somatosensory cortex

During learning, multi-dimensional inputs are integrated within the sensory cortices. However, the strategies by which the sensory cortex employs to achieve learning remains poorly understood. We studied the sensory cortical neuronal coding of trace eyeblink conditioning (TEC) in head-fixed, freely running mice, where whisker deflection was used as a conditioned stimulus (CS) and an air puff to the cornea delivered after an interval was used as unconditioned stimulus (US). After training, mice learned the task with a set of stereotypical behavioral changes, most prominent ones include prolonged closure of eyelids, and increased reverse running between CS and US onset. The local blockade of the primary somatosensory cortex (S1) activities with muscimol abolished the behavior learning suggesting that S1 is required for the TEC. In naive animals, based on the response properties to the CS and US, identities of the small proportion (~20%) of responsive primary neurons (PNs) were divided into two subtypes: CR (i.e. CS-responsive) and UR neurons (i.e. US-responsive). After animals learned the task, identity of CR and UR neurons changed: while the CR neurons are less responsive to CS, UR neurons gain responsiveness to CS, a new phenomenon we defined as 'learning induced neuronal identity switch (LINIS)'. To explore the potential mechanisms underlying LINIS, we found that systemic and local (i.e. in S1) administration of the nicotinic receptor antagonist during TEC training blocked the LINIS, and concomitantly disrupted the behavior learning. Additionally, we monitored responses of two types of cortical interneurons (INs) and observed that the responses of the somatostatin-expressing (SST), but not parvalbumin-expressing (PV) INs are negatively correlated with the learning performance, suggesting that SST-INs contribute to the LINIS. Thus, we conclude that L2/3 PNs in S1 encode perceptual learning by LINIS like mechanisms, and cholinergic pathways and cortical SST interneurons are involved in the formation of LINIS.

Auditory cortex ensembles jointly encode sound and locomotion speed to support sound perception during movement

The ability to process and act upon incoming sounds during locomotion is critical for survival and adaptive behavior. Despite the established role that the auditory cortex (AC) plays in behavior- and context-dependent sound processing, previous studies have found that auditory cortical activity is on average suppressed during locomotion as compared to immobility. While suppression of auditory cortical responses to self-generated sounds results from corollary discharge, which weakens responses to predictable sounds, the functional role of weaker responses to unpredictable external sounds during locomotion remains unclear. In particular, whether suppression of external sound-evoked responses during locomotion reflects reduced involvement of the AC in sound processing or whether it results from masking by an alternative neural computation in this state remains unresolved. Here, we tested the hypothesis that rather than simple inhibition, reduced sound-evoked responses during locomotion reflect a tradeoff with the emergence of explicit and reliable coding of locomotion velocity. To test this hypothesis, we first used neural inactivation in behaving mice and found that the AC plays a critical role in sound-guided behavior during locomotion. To investigate the nature of this processing, we used two-photon calcium imaging of local excitatory auditory cortical neural populations in awake mice. We found that locomotion had diverse influences on activity of different neurons, with a net suppression of baseline-subtracted sound-evoked responses and neural stimulus detection, consistent with previous studies. Importantly, we found that the net inhibitory effect of locomotion on baseline-subtracted sound-evoked responses was strongly shaped by elevated ongoing activity that compressed the response dynamic range, and that rather than reflecting enhanced "noise," this ongoing activity reliably encoded the animal's locomotion speed. Decoding analyses revealed that locomotion speed and sound are robustly co-encoded by auditory cortical ensemble activity. Finally, we found consistent patterns of joint coding of sound and locomotion speed in electrophysiologically recorded activity in freely moving rats. Together, our data suggest that rather than being suppressed by locomotion, auditory cortical ensembles explicitly encode it alongside sound information to support sound perception during locomotion.

Behavioral decomposition reveals rich encoding structure employed across neocortex in rats

The cortical population code is pervaded by activity patterns evoked by movement, but it remains largely unknown how such signals relate to natural behavior or how they might support processing in sensory cortices where they have been observed. To address this we compared high-density neural recordings across four cortical regions (visual, auditory, somatosensory, motor) in relation to sensory modulation, posture, movement, and ethograms of freely foraging male rats. Momentary actions, such as rearing or turning, were represented ubiquitously and could be decoded from all sampled structures. However, more elementary and continuous features, such as pose and movement, followed region-specific organization, with neurons in visual and auditory cortices preferentially encoding mutually distinct head-orienting features in world-referenced coordinates, and somatosensory and motor cortices principally encoding the trunk and head in egocentric coordinates. The tuning properties of synaptically coupled cells also exhibited connection patterns suggestive of area-specific uses of pose and movement signals, particularly in visual and auditory regions. Together, our results indicate that ongoing behavior is encoded at multiple levels throughout the dorsal cortex, and that low-level features are differentially utilized by different regions to serve locally relevant computations.

Brain-wide neural recordings in mice navigating physical spaces enabled by a cranial exoskeleton

Complex behaviors are mediated by neural computations occurring throughout the brain. In recent years, tremendous progress has been made in developing technologies that can record neural activity at cellular resolution at multiple spatial and temporal scales. However, these technologies are primarily designed for studying the mammalian brain during head fixation - wherein the behavior of the animal is highly constrained. Miniaturized devices for studying neural activity in freely behaving animals are largely confined to recording from small brain regions owing to performance limitations. We present a cranial exoskeleton that assists mice in maneuvering neural recording headstages that are orders of magnitude larger and heavier than the mice, while they navigate physical behavioral environments. Force sensors embedded within the headstage are used to detect the mouse's milli-Newton scale cranial forces which then control the x, y, and yaw motion of the exoskeleton via an admittance controller. We discovered optimal controller tuning parameters that enable mice to locomote at physiologically realistic velocities and accelerations while maintaining natural walking gait. Mice maneuvering headstages weighing up to 1.5 kg can make turns, navigate 2D arenas, and perform a navigational decision-making task with the same performance as when freely behaving. We designed an imaging headstage and an electrophysiology headstage for the cranial exoskeleton to record brain-wide neural activity in mice navigating 2D arenas. The imaging headstage enabled recordings of Ca2+ activity of 1000s of neurons distributed across the dorsal cortex. The electrophysiology headstage supported independent control of up to 4 silicon probes, enabling simultaneous recordings from 100s of neurons across multiple brain regions and multiple days. Cranial exoskeletons provide flexible platforms for largescale neural recording during the exploration of physical spaces, a critical new paradigm for unraveling the brain-wide neural mechanisms that control complex behavior.

Distinct brain-wide presynaptic networks underlie the functional identity of individual cortical neurons

Neuronal connections provide the scaffolding for neuronal function. Revealing the connectivity of functionally identified individual neurons is necessary to understand how activity patterns emerge and support behavior. Yet, the brain-wide presynaptic wiring rules that lay the foundation for the functional selectivity of individual neurons remain largely unexplored. Cortical neurons, even in primary sensory cortex, are heterogeneous in their selectivity, not only to sensory stimuli but also to multiple aspects of behavior. Here, to investigate presynaptic connectivity rules underlying the selectivity of pyramidal neurons to behavioral state ^1-12 in primary somatosensory cortex (S1), we used two-photon calcium imaging, neuropharmacology, single-cell based monosynaptic input tracing, and optogenetics. We show that behavioral state-dependent neuronal activity patterns are stable over time. These are not determined by neuromodulatory inputs but are instead driven by glutamatergic inputs. Analysis of brain-wide presynaptic networks of individual neurons with distinct behavioral state-dependent activity profiles revealed characteristic patterns of anatomical input. While both behavioral state-related and unrelated neurons had a similar pattern of local inputs within S1, their long-range glutamatergic inputs differed. Individual cortical neurons, irrespective of their functional properties, received converging inputs from the main S1-projecting areas. Yet, neurons that tracked behavioral state received a smaller proportion of motor cortical inputs and a larger proportion of thalamic inputs. Optogenetic suppression of thalamic inputs reduced behavioral state-dependent activity in S1, but this activity was not externally driven. Our results revealed distinct long-range glutamatergic inputs as a substrate for preconfigured network dynamics associated with behavioral state.

Stimulus-specific prediction error neurons in mouse auditory cortex

Comparing expectation with experience is an important neural computation performed throughout the brain and is a hallmark of predictive processing. Experiments that alter the sensory outcome of an animal's behavior reveal enhanced neural responses to unexpected self-generated stimuli, indicating that populations of neurons in sensory cortex may reflect prediction errors - mismatches between expectation and experience. However, enhanced neural responses to self-generated stimuli could also arise through non-predictive mechanisms, such as the movement-based facilitation of a neuron's inherent sound responses. If sensory prediction error neurons exist in sensory cortex, it is unknown whether they manifest as general error responses, or respond with specificity to errors in distinct stimulus dimensions. To answer these questions, we trained mice to expect the outcome of a simple sound-generating behavior and recorded auditory cortex activity as mice heard either the expected sound or sounds that deviated from expectation in one of multiple distinct dimensions. Our data reveal that the auditory cortex learns to suppress responses to self-generated sounds along multiple acoustic dimensions simultaneously. We identify a distinct population of auditory cortex neurons that are not responsive to passive sounds or to the expected sound but that explicitly encode prediction errors. These prediction error neurons are abundant only in animals with a learned motor-sensory expectation, and encode one or two specific violations rather than a generic error signal.

Wide-field calcium imaging of cortical activation and functional connectivity in externally- and internally-driven locomotion

The neural dynamics underlying self-initiated versus sensory driven movements is central to understanding volitional action. Upstream motor cortices are associated with the generation of internally-driven movements over externally-driven. Here we directly compare cortical dynamics during internally- versus externally-driven locomotion using wide-field Ca2+ imaging. We find that secondary motor cortex (M2) plays a larger role in internally-driven spontaneous locomotion transitions, with increased M2 functional connectivity during starting and stopping than in the externally-driven, motorized treadmill locomotion. This is not the case in steady-state walk. In addition, motorized treadmill and spontaneous locomotion are characterized by markedly different patterns of cortical activation and functional connectivity at the different behavior periods. Furthermore, the patterns of fluorescence activation and connectivity are uncorrelated. These experiments reveal widespread and striking differences in the cortical control of internally- and externally-driven locomotion, with M2 playing a major role in the preparation and execution of the self-initiated state.

Decision-making dynamics are predicted by arousal and uninstructed movements

During sensory-guided behavior, an animal's decision-making dynamics unfold through sequences of distinct performance states, even while stimulus-reward contingencies remain static. Little is known about the factors that underlie these changes in task performance. We hypothesize that these decision-making dynamics can be predicted by externally observable measures, such as uninstructed movements and changes in arousal. Here, combining behavioral experiments in mice with computational modeling, we uncovered lawful relationships between transitions in strategic task performance states and an animal's arousal and uninstructed movements. Using hidden Markov models applied to behavioral choices during sensory discrimination tasks, we found that animals fluctuate between minutes-long optimal, sub-optimal and disengaged performance states. Optimal state epochs were predicted by intermediate levels, and reduced variability, of pupil diameter, along with reduced variability in face movements and locomotion. Our results demonstrate that externally observable uninstructed behaviors can predict optimal performance states, and suggest mice regulate their arousal during optimal performance.

The role of consciously timed movements in shaping and improving auditory timing

Our subjective sense of time is intertwined with a plethora of perceptual, cognitive and motor functions, and likewise, the brain is equipped to expertly filter, weight and combine these signals for seamless interactions with a dynamic world. Until relatively recently, the literature on time perception has excluded the influence of simultaneous motor activity, yet it has been found that motor circuits in the brain are at the core of most timing functions. Several studies have now identified that concurrent movements exert robust effects on perceptual timing estimates, but critically have not assessed how humans consciously judge the duration of their own movements. This creates a gap in our understanding of the mechanisms driving movement-related effects on sensory timing. We sought to address this gap by administering a sensorimotor timing task in which we explicitly compared the timing of isolated auditory tones and arm movements, or both simultaneously. We contextualized our findings within a Bayesian cue combination framework, in which separate sources of temporal information are weighted by their reliability and integrated into a unitary time estimate that is more precise than either unisensory estimate. Our results revealed differences in accuracy between auditory, movement and combined trials, and (crucially) that combined trials were the most accurately timed. Under the Bayesian framework, we found that participants' combined estimates were more precise than isolated estimates, yet were sub-optimal when compared with the model's prediction, on average. These findings elucidate previously unknown qualities of conscious motor timing and propose computational mechanisms that can describe how movements combine with perceptual signals to create unified, multimodal experiences of time.

Walking humans and running mice: perception and neural encoding of optic flow during self-motion

Locomotion produces full-field optic flow that often dominates the visual motion inputs to an observer. The perception of optic flow is in turn important for animals to guide their heading and interact with moving objects. Understanding how locomotion influences optic flow processing and perception is therefore essential to understand how animals successfully interact with their environment. Here, we review research investigating how perception and neural encoding of optic flow are altered during self-motion, focusing on locomotion. Self-motion has been found to influence estimation and sensitivity for optic flow speed and direction. Nonvisual self-motion signals also increase compensation for self-driven optic flow when parsing the visual motion of moving objects. The integration of visual and nonvisual self-motion signals largely follows principles of Bayesian inference and can improve the precision and accuracy of self-motion perception. The calibration of visual and nonvisual self-motion signals is dynamic, reflecting the changing visuomotor contingencies across different environmental contexts. Throughout this review, we consider experimental research using humans, non-human primates and mice. We highlight experimental challenges and opportunities afforded by each of these species and draw parallels between experimental findings. These findings reveal a profound influence of locomotion on optic flow processing and perception across species. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Contextual Fear Learning and Extinction in the Primary Visual Cortex of Mice

Fear memory contextualization is critical for selecting adaptive behavior to survive. Contextual fear conditioning (CFC) is a classical model for elucidating related underlying neuronal circuits. The primary visual cortex (V1) is the primary cortical region for contextual visual inputs, but its role in CFC is poorly understood. Here, our experiments demonstrated that bilateral inactivation of V1 in mice impaired CFC retrieval, and both CFC learning and extinction increased the turnover rate of axonal boutons in V1. The frequency of neuronal Ca2+ activity decreased after CFC learning, while CFC extinction reversed the decrease and raised it to the naïve level. Contrary to control mice, the frequency of neuronal Ca2+ activity increased after CFC learning in microglia-depleted mice and was maintained after CFC extinction, indicating that microglial depletion alters CFC learning and the frequency response pattern of extinction-induced Ca2+ activity. These findings reveal a critical role of microglia in neocortical information processing in V1, and suggest potential approaches for cellular-based manipulation of acquired fear memory.

Neuronal structures controlling locomotor behavior during active and inactive motor states

Animal behaviors can be divided into two states according to their motor activity: the active motor state, which involves significant body movements, and the inactive motor state, which refers to when the animal is stationary. The timing and duration of these states are determined by the activity of the neuronal circuits involved in motor control. Among these motor circuits, those that generate locomotion are some of the most studied neuronal networks and are widely distributed from the spinal cord to the cerebral cortex. In this review, we discuss recent discoveries, mainly in rodents using state-of-the-art experimental approaches, of the neuronal mechanisms underlying the initiation and termination of locomotion in the brainstem, basal ganglia, and prefrontal cortex. These findings is discussed with reference to studies on the neuronal mechanism of motor control during sleep and the modulation of cortical states in these structures. Accumulating evidence has unraveled the complex yet highly structured network that controls the transition between motor states.

Independent response modulation of visual cortical neurons by attentional and behavioral states

Sensory processing is influenced by cognitive and behavioral states, but how these states interact to modulate responses of individual neurons is unknown. We trained mice in a visual discrimination task wherein they attended to different locations within a hemifield while running or sitting still, enabling us to examine how visual responses are modulated by spatial attention and running behavior. We found that spatial attention improved discrimination performance and strengthened visual responses of excitatory neurons in the primary visual cortex whose receptive fields overlapped with the attended location. Although individual neurons were modulated by both spatial attention and running, the magnitudes of these influences were not correlated. While running-dependent modulation was stable across days, attentional modulation was dynamic, influencing individual neurons to different degrees after repeated changes in attentional states. Thus, despite similar effects on neural responses, spatial attention and running act independently with different dynamics, implying separable mechanisms for their implementation.

Optic flow in the natural habitats of zebrafish supports spatial biases in visual self-motion estimation

Animals benefit from knowing if and how they are moving. Across the animal kingdom, sensory information in the form of optic flow over the visual field is used to estimate self-motion. However, different species exhibit strong spatial biases in how they use optic flow. Here, we show computationally that noisy natural environments favor visual systems that extract spatially biased samples of optic flow when estimating self-motion. The performance associated with these biases, however, depends on interactions between the environment and the animal's brain and behavior. Using the larval zebrafish as a model, we recorded natural optic flow associated with swimming trajectories in the animal's habitat with an omnidirectional camera mounted on a mechanical arm. An analysis of these flow fields suggests that lateral regions of the lower visual field are most informative about swimming speed. This pattern is consistent with the recent findings that zebrafish optomotor responses are preferentially driven by optic flow in the lateral lower visual field, which we extend with behavioral results from a high-resolution spherical arena. Spatial biases in optic-flow sampling are likely pervasive because they are an effective strategy for determining self-motion in noisy natural environments.

Optic flow in the natural habitats of zebrafish supports spatial biases in visual self-motion estimation

Animals benefit from knowing if and how they are moving. Across the animal kingdom, sensory information in the form of optic flow over the visual field is used to estimate self-motion. However, different species exhibit strong spatial biases in how they use optic flow. Here, we show computationally that noisy natural environments favor visual systems that extract spatially biased samples of optic flow when estimating self-motion. The performance associated with these biases, however, depends on interactions between the environment and the animal's brain and behavior. Using the larval zebrafish as a model, we recorded natural optic flow associated with swimming trajectories in the animal's habitat with an omnidirectional camera mounted on a mechanical arm. An analysis of these flow fields suggests that lateral regions of the lower visual field are most informative about swimming speed. This pattern is consistent with the recent findings that zebrafish optomotor responses are preferentially driven by optic flow in the lateral lower visual field, which we extend with behavioral results from a high-resolution spherical arena. Spatial biases in optic-flow sampling are likely pervasive because they are an effective strategy for determining self-motion in noisy natural environments.

Optic flow in the natural habitats of zebrafish supports spatial biases in visual self-motion estimation

Animals benefit from knowing if and how they are moving. Across the animal kingdom, sensory information in the form of optic flow over the visual field is used to estimate self-motion. However, different species exhibit strong spatial biases in how they use optic flow. Here, we show computationally that noisy natural environments favor visual systems that extract spatially biased samples of optic flow when estimating self-motion. The performance associated with these biases, however, depends on interactions between the environment and the animal's brain and behavior. Using the larval zebrafish as a model, we recorded natural optic flow associated with swimming trajectories in the animal's habitat with an omnidirectional camera mounted on a mechanical arm. An analysis of these flow fields suggests that lateral regions of the lower visual field are most informative about swimming speed. This pattern is consistent with the recent findings that zebrafish optomotor responses are preferentially driven by optic flow in the lateral lower visual field, which we extend with behavioral results from a high-resolution spherical arena. Spatial biases in optic-flow sampling are likely pervasive because they are an effective strategy for determining self-motion in noisy natural environments.

Reshaping sensory representations by task-specific brain states: Toward cortical circuit mechanisms

Perception is internally constructed by integrating brain states with external sensory inputs, a process depending on the topdown modulation of sensory representations. A wealth of earlier studies described task-dependent modulations of sensory cortex corroborating perceptual and behavioral phenomena. But only with recent technological advancements, the underlying circuit-level mechanisms began to be unveiled. We review recent progress along this line of research. It begins to be appreciated that topdown signals can encode various types of task-related information, ranging from task engagement, and category knowledge to decision execution; these signals are transferred via feedback pathways originating from distinct association cortices and interact with sensory cortical circuits. These plausible mechanisms support a broad range of computations from predictive coding to inference making, ultimately form dynamic percepts and endow behavioral flexibility.

Naturalistic neuroscience and virtual reality

Virtual reality (VR) is one of the techniques that became particularly popular in neuroscience over the past few decades. VR experiments feature a closed-loop between sensory stimulation and behavior. Participants interact with the stimuli and not just passively perceive them. Several senses can be stimulated at once, large-scale environments can be simulated as well as social interactions. All of this makes VR experiences more natural than those in traditional lab paradigms. Compared to the situation in field research, a VR simulation is highly controllable and reproducible, as required of a laboratory technique used in the search for neural correlates of perception and behavior. VR is therefore considered a middle ground between ecological validity and experimental control. In this review, I explore the potential of VR in eliciting naturalistic perception and behavior in humans and non-human animals. In this context, I give an overview of recent virtual reality approaches used in neuroscientific research.

Wide-Field Calcium Imaging of Neuronal Network Dynamics In Vivo

A central tenet of neuroscience is that sensory, motor, and cognitive behaviors are generated by the communications and interactions among neurons, distributed within and across anatomically and functionally distinct brain regions. Therefore, to decipher how the brain plans, learns, and executes behaviors requires characterizing neuronal activity at multiple spatial and temporal scales. This includes simultaneously recording neuronal dynamics at the mesoscale level to understand the interactions among brain regions during different behavioral and brain states. Wide-field Ca2+ imaging, which uses single photon excitation and improved genetically encoded Ca2+ indicators, allows for simultaneous recordings of large brain areas and is proving to be a powerful tool to study neuronal activity at the mesoscopic scale in behaving animals. This review details the techniques used for wide-field Ca2+ imaging and the various approaches employed for the analyses of the rich neuronal-behavioral data sets obtained. Also discussed is how wide-field Ca2+ imaging is providing novel insights into both normal and altered neural processing in disease. Finally, we examine the limitations of the approach and new developments in wide-field Ca2+ imaging that are bringing new capabilities to this important technique for investigating large-scale neuronal dynamics.

Midbrain dopamine neurons signal phasic and ramping reward prediction error during goal-directed navigation

Goal-directed navigation requires learning to accurately estimate location and select optimal actions in each location. Midbrain dopamine neurons are involved in reward value learning and have been linked to reward location learning. They are therefore ideally placed to provide teaching signals for goal-directed navigation. By imaging dopamine neural activity as mice learned to actively navigate a closed-loop virtual reality corridor to obtain reward, we observe phasic and pre-reward ramping dopamine activity, which are modulated by learning stage and task engagement. A Q-learning model incorporating position inference recapitulates our results, displaying prediction errors resembling phasic and ramping dopamine neural activity. The model predicts that ramping is followed by improved task performance, which we confirm in our experimental data, indicating that the dopamine ramp may have a teaching effect. Our results suggest that midbrain dopamine neurons encode phasic and ramping reward prediction error signals to improve goal-directed navigation.

Look-up and look-down neurons in the mouse visual thalamus during freely moving exploration

Visual information reaches cortex via the thalamic dorsal lateral geniculate nucleus (dLGN). dLGN activity is modulated by global sleep/wake states and arousal, indicating that it is not simply a passive relay station. However, its potential for more specific visuomotor integration is largely unexplored. We addressed this question by developing robust 3D video reconstruction of mouse head and body during spontaneous exploration paired with simultaneous neuronal recordings from dLGN. Unbiased evaluation of a wide range of postures and movements revealed a widespread coupling between neuronal activity and few behavioral parameters. In particular, postures associated with the animal looking up/down correlated with activity in >50% neurons, and the extent of this effect was comparable with that induced by full-body movements (typically locomotion). By contrast, thalamic activity was minimally correlated with other postures or movements (e.g., left/right head and body torsions). Importantly, up/down postures and full-body movements were largely independent and jointly coupled to neuronal activity. Thus, although most units were excited during full-body movements, some expressed highest firing when the animal was looking up ("look-up" neurons), whereas others expressed highest firing when the animal was looking down ("look-down" neurons). These results were observed in the dark, thus representing a genuine behavioral modulation, and were amplified in a lit arena. Our results demonstrate that the primary visual thalamus, beyond global modulations by sleep/awake states, is potentially involved in specific visuomotor integration and reveal two distinct couplings between up/down postures and neuronal activity.

Neural coding: Looking up and down the visual thalamus

Integrating sensory and postural information is essential for perception and behavior. A new study shows that information about whether mice are looking up or down is combined with visual information in the primary visual thalamus, an early sensory stage of visual processing.

A Standardized Nonvisual Behavioral Event Is Broadcasted Homogeneously across Cortical Visual Areas without Modulating Visual Responses

Multiple recent studies have shown that motor activity greatly impacts the activity of primary sensory areas like V1. Yet, the role of this motor related activity in sensory processing is still unclear. Here, we dissect how these behavior signals are broadcast to different layers and areas of the visual cortex. To do so, we leveraged a standardized and spontaneous behavioral fidget event in passively viewing mice. Importantly, this behavior event had no relevance to any ongoing task allowing us to compare its neuronal correlates with visually relevant behaviors (e.g., running). A large two-photon Ca²⁺ imaging database of neuronal responses uncovered four neural response types during fidgets that were consistent in their proportion and response patterns across all visual areas and layers of the visual cortex. Indeed, the layer and area identity could not be decoded above chance level based only on neuronal recordings. In contrast to running behavior, fidget evoked neural responses that were independent to visual processing. The broad availability of visually orthogonal standardized behavior signals could be a key component in how the cortex selects, learns and binds local sensory information with motor outputs. Contrary to behaviorally relevant motor outputs, irrelevant motor signals could project to separate local neural subspaces.