Opposing Influence of Top-down and Bottom-up Input on Excitatory Layer 2/3 Neurons in Mouse Primary Visual Cortex

Computational model of layer 2/3 in mouse primary visual cortex explains observed visuomotor mismatch response

Activity in layer 2/3 of the mouse primary visual cortex has been shown to depend both on visual input and the mouse's locomotion. Moreover, this activity is altered by a mismatch between the observed visual flow and the predicted visual flow from locomotion. Here, I present a simple computational model that explains previously reported recordings from layer 2/3 neurons in mice. In my model, layer 2/3 encodes the velocity difference between the estimate from visual flow and the prediction from locomotion using a neural population code. Moreover, I describe a hypothesized mechanism for how the brain may carry out computations of variables encoded in population codes. This mechanism may point to a general principle for computing any mathematical function in the brain.

Daily oscillations of neuronal membrane capacitance

Capacitance of biological membranes is determined by the properties of the lipid portion of the membrane as well as the morphological features of a cell. In neurons, membrane capacitance is a determining factor of synaptic integration, action potential propagation speed, and firing frequency due to its direct effect on the membrane time constant. Besides slow changes associated with increased morphological complexity during postnatal maturation, neuronal membrane capacitance is considered a stable, non-regulated, and constant magnitude. Here we report that, in two excitatory neuronal cell types, pyramidal cells of the mouse primary visual cortex and granule cells of the hippocampus, the membrane capacitance significantly changes between the start and the end of a daily light-dark cycle. The changes are large, nearly 2-fold in magnitude in pyramidal cells, but are not observed in cortical parvalbumin-expressing inhibitory interneurons. Consistent with daily capacitance fluctuations, the time window for synaptic integration also changes in pyramidal cells.

Confidence and second-order errors in cortical circuits

Minimization of cortical prediction errors has been considered a key computational goal of the cerebral cortex underlying perception, action, and learning. However, it is still unclear how the cortex should form and use information about uncertainty in this process. Here, we formally derive neural dynamics that minimize prediction errors under the assumption that cortical areas must not only predict the activity in other areas and sensory streams but also jointly project their confidence (inverse expected uncertainty) in their predictions. In the resulting neuronal dynamics, the integration of bottom-up and top-down cortical streams is dynamically modulated based on confidence in accordance with the Bayesian principle. Moreover, the theory predicts the existence of cortical second-order errors, comparing confidence and actual performance. These errors are propagated through the cortical hierarchy alongside classical prediction errors and are used to learn the weights of synapses responsible for formulating confidence. We propose a detailed mapping of the theory to cortical circuitry, discuss entailed functional interpretations, and provide potential directions for experimental work.

Genetic mechanisms for impaired synaptic plasticity in schizophrenia revealed by computational modeling

Schizophrenia phenotypes are suggestive of impaired cortical plasticity in the disease, but the mechanisms of these deficits are unknown. Genomic association studies have implicated a large number of genes that regulate neuromodulation and plasticity, indicating that the plasticity deficits have a genetic origin. Here, we used biochemically detailed computational modeling of postsynaptic plasticity to investigate how schizophrenia-associated genes regulate long-term potentiation (LTP) and depression (LTD). We combined our model with data from postmortem RNA expression studies (CommonMind gene-expression datasets) to assess the consequences of altered expression of plasticity-regulating genes for the amplitude of LTP and LTD. Our results show that the expression alterations observed post mortem, especially those in the anterior cingulate cortex, lead to impaired protein kinase A (PKA)-pathway-mediated LTP in synapses containing GluR1 receptors. We validated these findings using a genotyped electroencephalogram (EEG) dataset where polygenic risk scores for synaptic and ion channel-encoding genes as well as modulation of visual evoked potentials were determined for 286 healthy controls. Our results provide a possible genetic mechanism for plasticity impairments in schizophrenia, which can lead to improved understanding and, ultimately, treatment of the disorder.

Desegregation of neuronal predictive processing

Neural circuits construct internal 'world-models' to guide behavior. The predictive processing framework posits that neural activity signaling sensory predictions and concurrently computing prediction-errors is a signature of those internal models. Here, to understand how the brain generates predictions for complex sensorimotor signals, we investigate the emergence of high-dimensional, multi-modal predictive representations in recurrent networks. We find that robust predictive processing arises in a network with loose excitatory/inhibitory balance. Contrary to previous proposals of functionally specialized cell-types, the network exhibits desegregation of stimulus and prediction-error representations. We confirmed these model predictions by experimentally probing predictive-coding circuits using a rich stimulus-set to violate learned expectations. When constrained by data, our model further reveals and makes concrete testable experimental predictions for the distinct functional roles of excitatory and inhibitory neurons, and of neurons in different layers along a laminar hierarchy, in computing multi-modal predictions. These results together imply that in natural conditions, neural representations of internal models are highly distributed, yet structured to allow flexible readout of behaviorally-relevant information. The generality of our model advances the understanding of computation of internal models across species, by incorporating different types of predictive computations into a unified framework.

Predictive Processing: A Circuit Approach to Psychosis

Predictive processing is a computational framework that aims to explain how the brain processes sensory information by making predictions about the environment and minimizing prediction errors. It can also be used to explain some of the key symptoms of psychotic disorders such as schizophrenia. In recent years, substantial advances have been made in our understanding of the neuronal circuitry that underlies predictive processing in cortex. In this review, we summarize these findings and how they might relate to psychosis and to observed cell type-specific effects of antipsychotic drugs. We argue that quantifying the effects of antipsychotic drugs on specific neuronal circuit elements is a promising approach to understanding not only the mechanism of action of antipsychotic drugs but also psychosis. Finally, we outline some of the key experiments that should be done. The aims of this review are to provide an overview of the current circuit-based approaches to psychosis and to encourage further research in this direction.

Cholinergic input to mouse visual cortex signals a movement state and acutely enhances layer 5 responsiveness

Acetylcholine is released in visual cortex by axonal projections from the basal forebrain. The signals conveyed by these projections and their computational significance are still unclear. Using two-photon calcium imaging in behaving mice, we show that basal forebrain cholinergic axons in the mouse visual cortex provide a binary locomotion state signal. In these axons, we found no evidence of responses to visual stimuli or visuomotor prediction errors. While optogenetic activation of cholinergic axons in visual cortex in isolation did not drive local neuronal activity, when paired with visuomotor stimuli, it resulted in layer-specific increases of neuronal activity. Responses in layer 5 neurons to both top-down and bottom-up inputs were increased in amplitude and decreased in latency, whereas those in layer 2/3 neurons remained unchanged. Using opto- and chemogenetic manipulations of cholinergic activity, we found acetylcholine to underlie the locomotion-associated decorrelation of activity between neurons in both layer 2/3 and layer 5. Our results suggest that acetylcholine augments the responsiveness of layer 5 neurons to inputs from outside of the local network, possibly enabling faster switching between internal representations during locomotion.

A sensory-motor theory of the neocortex

Recent neurophysiological and neuroanatomical studies suggest a close interaction between sensory and motor processes across the neocortex. Here, I propose that the neocortex implements active predictive coding (APC): each cortical area estimates both latent sensory states and actions (including potentially abstract actions internal to the cortex), and the cortex as a whole predicts the consequences of actions at multiple hierarchical levels. Feedback from higher areas modulates the dynamics of state and action networks in lower areas. I show how the same APC architecture can explain (1) how we recognize an object and its parts using eye movements, (2) why perception seems stable despite eye movements, (3) how we learn compositional representations, for example, part-whole hierarchies, (4) how complex actions can be planned using simpler actions, and (5) how we form episodic memories of sensory-motor experiences and learn abstract concepts such as a family tree. I postulate a mapping of the APC model to the laminar architecture of the cortex and suggest possible roles for cortico-cortical and cortico-subcortical pathways.

Variable recruitment of distal tuft dendrites shapes new hippocampal place fields

Hippocampal pyramidal neurons support episodic memory by integrating complementary information streams into new 'place fields'. Distal tuft dendrites are widely thought to initiate place field formation by locally generating prolonged, globally-spreading Ca 2+ spikes known as plateau potentials. However, the hitherto experimental inaccessibility of distal tuft dendrites in the hippocampus has rendered their in vivo function entirely unknown. Here we gained direct optical access to this elusive dendritic compartment. We report that distal tuft dendrites do not serve as the point of origin for place field-forming plateau potentials. Instead, the timing and extent of peri-formation distal tuft recruitment is variable and closely predicts multiple properties of resultant place fields. Therefore, distal tuft dendrites play a more powerful role in hippocampal feature selectivity than simply initiating place field formation. Moreover, place field formation is not accompanied by global Ca 2+ influx as previously thought. In addition to shaping new somatic place fields, distal tuft dendrites possess their own local place fields. Tuft place fields are back-shifted relative to that of their soma and appear to maintain somatic place fields via post-formation plateau potentials. Through direct in vivo observation, we provide a revised dendritic basis for hippocampal feature selectivity during navigational learning.

Event segmentation in ADHD: neglect of social information and deviant theta activity point to a mechanism underlying ADHD

Background

Attention-deficit/hyperactivity disorder (ADHD) is one of the most frequently diagnosed psychiatric conditions in children and adolescents. Although the symptoms appear to be well described, no coherent conceptual mechanistic framework integrates their occurrence and variance and the associated problems that people with ADHD face.

Aims

The current study proposes that altered event segmentation processes provide a novel mechanistic framework for understanding deficits in ADHD.

Methods

Adolescents with ADHD and neurotypically developing (NT) peers watched a short movie and were then asked to indicate the boundaries between meaningful segments of the movie. Concomitantly recorded electroencephalography (EEG) data were analysed for differences in frequency band activity and effective connectivity between brain areas.

Results

Compared with their NT peers, the ADHD group showed less dependence of their segmentation behaviour on social information, indicating that they did not consider social information to the same extent as their unaffected peers. This divergence was accompanied by differences in EEG theta band activity and a different effective connectivity network architecture at the source level. Specifically, NT adolescents primarily showed error signalling in and between the left and right fusiform gyri related to social information processing, which was not the case in the ADHD group. For the ADHD group, the inferior frontal cortex associated with attentional sampling served as a hub instead, indicating problems in the deployment of attentional control.

Conclusions

This study shows that adolescents with ADHD perceive events differently from their NT peers, in association with a different brain network architecture that reflects less adaptation to the situation and problems in attentional sampling of environmental information. The results call for a novel conceptual view of ADHD, based on event segmentation theory.

Organization of corticocortical and thalamocortical top-down inputs in the primary visual cortex

Unified visual perception requires integration of bottom-up and top-down inputs in the primary visual cortex (V1), yet the organization of top-down inputs in V1 remains unclear. Here, we used optogenetics-assisted circuit mapping to identify how multiple top-down inputs from higher-order cortical and thalamic areas engage V1 excitatory and inhibitory neurons. Top-down inputs overlap in superficial layers yet segregate in deep layers. Inputs from the medial secondary visual cortex (V2M) and anterior cingulate cortex (ACA) converge on L6 Pyrs, whereas ventrolateral orbitofrontal cortex (ORBvl) and lateral posterior thalamic nucleus (LP) inputs are processed in parallel in Pyr-type-specific subnetworks (Pyr←ORBvl and Pyr←LP) and drive mutual inhibition between them via local interneurons. Our study deepens understanding of the top-down modulation mechanisms of visual processing and establishes that V2M and ACA inputs in L6 employ integrated processing distinct from the parallel processing of LP and ORBvl inputs in L5.

Predictive processing: Layer-specific prediction error signals in human cortex

A new study leveraging advances in high-field fMRI provides evidence that superficial cortical layers in humans play a crucial role in signaling prediction errors, a finding that is consistent with the predictive processing framework.

Predictions and errors are distinctly represented across V1 layers

Popular accounts of mind and brain propose that the brain continuously forms predictions about future sensory inputs and combines predictions with inputs to determine what we perceive.1,2,3,4,5,6 Under "predictive processing" schemes, such integration is supported by the hierarchical organization of the cortex, whereby feedback connections communicate predictions from higher-level deep layers to agranular (superficial and deep) lower-level layers.7,8,9,10 Predictions are compared with input to compute the "prediction error," which is transmitted up the hierarchy from superficial layers of lower cortical regions to the middle layers of higher areas, to update higher-level predictions until errors are reconciled.11,12,13,14,15 In the primary visual cortex (V1), predictions have thereby been proposed to influence representations in deep layers while error signals may be computed in superficial layers. Despite the framework's popularity, there is little evidence for these functional distinctions because, to our knowledge, unexpected sensory events have not previously been presented in human laminar paradigms to contrast against expected events. To this end, this 7T fMRI study contrasted V1 responses to expected (75% likely) and unexpected (25%) Gabor orientations. Multivariate decoding analyses revealed an interaction between expectation and layer, such that expected events could be decoded with comparable accuracy across layers, while unexpected events could only be decoded in superficial laminae. Although these results are in line with these accounts that have been popular for decades, such distinctions have not previously been demonstrated in humans. We discuss how both prediction and error processes may operate together to shape our unitary perceptual experiences.

Predictive coding with spiking neurons and feedforward gist signaling

Predictive coding (PC) is an influential theory in neuroscience, which suggests the existence of a cortical architecture that is constantly generating and updating predictive representations of sensory inputs. Owing to its hierarchical and generative nature, PC has inspired many computational models of perception in the literature. However, the biological plausibility of existing models has not been sufficiently explored due to their use of artificial neurons that approximate neural activity with firing rates in the continuous time domain and propagate signals synchronously. Therefore, we developed a spiking neural network for predictive coding (SNN-PC), in which neurons communicate using event-driven and asynchronous spikes. Adopting the hierarchical structure and Hebbian learning algorithms from previous PC neural network models, SNN-PC introduces two novel features: (1) a fast feedforward sweep from the input to higher areas, which generates a spatially reduced and abstract representation of input (i.e., a neural code for the gist of a scene) and provides a neurobiological alternative to an arbitrary choice of priors; and (2) a separation of positive and negative error-computing neurons, which counters the biological implausibility of a bi-directional error neuron with a very high baseline firing rate. After training with the MNIST handwritten digit dataset, SNN-PC developed hierarchical internal representations and was able to reconstruct samples it had not seen during training. SNN-PC suggests biologically plausible mechanisms by which the brain may perform perceptual inference and learning in an unsupervised manner. In addition, it may be used in neuromorphic applications that can utilize its energy-efficient, event-driven, local learning, and parallel information processing nature.

Antipsychotic drugs selectively decorrelate long-range interactions in deep cortical layers

Psychosis is characterized by a diminished ability of the brain to distinguish externally driven activity patterns from self-generated activity patterns. Antipsychotic drugs are a class of small molecules with relatively broad binding affinity for a variety of neuromodulator receptors that, in humans, can prevent or ameliorate psychosis. How these drugs influence the function of cortical circuits, and in particular their ability to distinguish between externally and self-generated activity patterns, is still largely unclear. To have experimental control over self-generated sensory feedback, we used a virtual reality environment in which the coupling between movement and visual feedback can be altered. We then used widefield calcium imaging to determine the cell type-specific functional effects of antipsychotic drugs in mouse dorsal cortex under different conditions of visuomotor coupling. By comparing cell type-specific activation patterns between locomotion onsets that were experimentally coupled to self-generated visual feedback and locomotion onsets that were not coupled, we show that deep cortical layers were differentially activated in these two conditions. We then show that the antipsychotic drug clozapine disrupted visuomotor integration at locomotion onsets also primarily in deep cortical layers. Given that one of the key components of visuomotor integration in cortex is long-range cortico-cortical connections, we tested whether the effect of clozapine was detectable in the correlation structure of activity patterns across dorsal cortex. We found that clozapine as well as two other antipsychotic drugs, aripiprazole and haloperidol, resulted in a strong reduction in correlations of layer 5 activity between cortical areas and impaired the spread of visuomotor prediction errors generated in visual cortex. Our results are consistent with the interpretation that a major functional effect of antipsychotic drugs is a selective alteration of long-range layer 5-mediated communication.

Local prediction-learning in high-dimensional spaces enables neural networks to plan

Planning and problem solving are cornerstones of higher brain function. But we do not know how the brain does that. We show that learning of a suitable cognitive map of the problem space suffices. Furthermore, this can be reduced to learning to predict the next observation through local synaptic plasticity. Importantly, the resulting cognitive map encodes relations between actions and observations, and its emergent high-dimensional geometry provides a sense of direction for reaching distant goals. This quasi-Euclidean sense of direction provides a simple heuristic for online planning that works almost as well as the best offline planning algorithms from AI. If the problem space is a physical space, this method automatically extracts structural regularities from the sequence of observations that it receives so that it can generalize to unseen parts. This speeds up learning of navigation in 2D mazes and the locomotion with complex actuator systems, such as legged bodies. The cognitive map learner that we propose does not require a teacher, similar to self-attention networks (Transformers). But in contrast to Transformers, it does not require backpropagation of errors or very large datasets for learning. Hence it provides a blue-print for future energy-efficient neuromorphic hardware that acquires advanced cognitive capabilities through autonomous on-chip learning.

Modeling circuit mechanisms of opposing cortical responses to visual flow perturbations

In an ever-changing visual world, animals' survival depends on their ability to perceive and respond to rapidly changing motion cues. The primary visual cortex (V1) is at the forefront of this sensory processing, orchestrating neural responses to perturbations in visual flow. However, the underlying neural mechanisms that lead to distinct cortical responses to such perturbations remain enigmatic. In this study, our objective was to uncover the neural dynamics that govern V1 neurons' responses to visual flow perturbations using a biologically realistic computational model. By subjecting the model to sudden changes in visual input, we observed opposing cortical responses in excitatory layer 2/3 (L2/3) neurons, namely, depolarizing and hyperpolarizing responses. We found that this segregation was primarily driven by the competition between external visual input and recurrent inhibition, particularly within L2/3 and L4. This division was not observed in excitatory L5/6 neurons, suggesting a more prominent role for inhibitory mechanisms in the visual processing of the upper cortical layers. Our findings share similarities with recent experimental studies focusing on the opposing influence of top-down and bottom-up inputs in the mouse primary visual cortex during visual flow perturbations.

Learned response dynamics reflect stimulus timing and encode temporal expectation violations in superficial layers of mouse V1

The ability to recognize ordered event sequences is a fundamental component of sensory cognition and underlies the capacity to generate temporally specific expectations of future events based on previous experience. Various lines of evidence suggest that the primary visual cortex participates in some form of predictive processing, but many details remain ambiguous. Here we use two-photon calcium imaging in layer 2/3 (L2/3) of the mouse primary visual cortex (V1) to study changes to neural activity under a multi-day sequence learning paradigm with respect to prediction error responses, stimulus encoding, and time. We find increased neural activity at the time an expected, but omitted, stimulus would have occurred but no significant prediction error responses following an unexpected stimulus substitution. Sequence representations became sparser and less correlated with training, although these changes had no effect on decoding accuracy of stimulus identity or timing. Additionally, we find that experience modifies the temporal structure of stimulus responses to produce a bias towards predictive stimulus-locked activity. Finally, we find significant temporal structure during intersequence rest periods that was largely unchanged by training.

Fast adaptation to rule switching using neuronal surprise

In humans and animals, surprise is a physiological reaction to an unexpected event, but how surprise can be linked to plausible models of neuronal activity is an open problem. We propose a self-supervised spiking neural network model where a surprise signal is extracted from an increase in neural activity after an imbalance of excitation and inhibition. The surprise signal modulates synaptic plasticity via a three-factor learning rule which increases plasticity at moments of surprise. The surprise signal remains small when transitions between sensory events follow a previously learned rule but increases immediately after rule switching. In a spiking network with several modules, previously learned rules are protected against overwriting, as long as the number of modules is larger than the total number of rules-making a step towards solving the stability-plasticity dilemma in neuroscience. Our model relates the subjective notion of surprise to specific predictions on the circuit level.

Synthetic surprise as the foundation of the psychedelic experience

Psychedelic agents, such as LSD and psilocybin, induce marked alterations in consciousness via activation of the 5-HT2A receptor (5-HT2ARs). We hypothesize that psychedelics enforce a state of synthetic surprise through the biased activation of the 5-HTRs system. This idea is informed by recent insights into the role of 5-HT in signaling surprise. The effects on consciousness, explained by the cognitive penetrability of perception, can be described within the predictive coding framework where surprise corresponds to prediction error, the mismatch between predictions and actual sensory input. Crucially, the precision afforded to the prediction error determines its effect on priors, enabling a dynamic interaction between top-down expectations and incoming sensory data. By integrating recent findings on predictive coding circuitry and 5-HT2ARs transcriptomic data, we propose a biological implementation with emphasis on the role of inhibitory interneurons. Implications arise for the clinical use of psychedelics, which may rely primarily on their inherent capacity to induce surprise in order to disrupt maladaptive patterns.

The locus coeruleus as a global model failure system

Predictive processing models posit that brains constantly attempt to predict their sensory inputs. Prediction errors signal when these predictions are incorrect and are thought to be instructive signals that drive corrective plasticity. Recent findings support the idea that the locus coeruleus (LC) - a brain-wide neuromodulatory system - signals several types of prediction error. I discuss how these findings support models proposing that the LC signals global model failures: instances where predictions about the world are strongly violated. Focusing on the cortex, I explore the utility of this signal in learning rate control, how the LC circuit may compute the signal, and how this view may aid our understanding of neurodivergence.

Dynamic predictive coding: A model of hierarchical sequence learning and prediction in the neocortex

We introduce dynamic predictive coding, a hierarchical model of spatiotemporal prediction and sequence learning in the neocortex. The model assumes that higher cortical levels modulate the temporal dynamics of lower levels, correcting their predictions of dynamics using prediction errors. As a result, lower levels form representations that encode sequences at shorter timescales (e.g., a single step) while higher levels form representations that encode sequences at longer timescales (e.g., an entire sequence). We tested this model using a two-level neural network, where the top-down modulation creates low-dimensional combinations of a set of learned temporal dynamics to explain input sequences. When trained on natural videos, the lower-level model neurons developed space-time receptive fields similar to those of simple cells in the primary visual cortex while the higher-level responses spanned longer timescales, mimicking temporal response hierarchies in the cortex. Additionally, the network's hierarchical sequence representation exhibited both predictive and postdictive effects resembling those observed in visual motion processing in humans (e.g., in the flash-lag illusion). When coupled with an associative memory emulating the role of the hippocampus, the model allowed episodic memories to be stored and retrieved, supporting cue-triggered recall of an input sequence similar to activity recall in the visual cortex. When extended to three hierarchical levels, the model learned progressively more abstract temporal representations along the hierarchy. Taken together, our results suggest that cortical processing and learning of sequences can be interpreted as dynamic predictive coding based on a hierarchical spatiotemporal generative model of the visual world.

Responses to Pattern-Violating Visual Stimuli Evolve Differently Over Days in Somata and Distal Apical Dendrites

Scientists have long conjectured that the neocortex learns patterns in sensory data to generate top-down predictions of upcoming stimuli. In line with this conjecture, different responses to pattern-matching vs pattern-violating visual stimuli have been observed in both spiking and somatic calcium imaging data. However, it remains unknown whether these pattern-violation signals are different between the distal apical dendrites, which are heavily targeted by top-down signals, and the somata, where bottom-up information is primarily integrated. Furthermore, it is unknown how responses to pattern-violating stimuli evolve over time as an animal gains more experience with them. Here, we address these unanswered questions by analyzing responses of individual somata and dendritic branches of layer 2/3 and layer 5 pyramidal neurons tracked over multiple days in primary visual cortex of awake, behaving female and male mice. We use sequences of Gabor patches with patterns in their orientations to create pattern-matching and pattern-violating stimuli, and two-photon calcium imaging to record neuronal responses. Many neurons in both layers show large differences between their responses to pattern-matching and pattern-violating stimuli. Interestingly, these responses evolve in opposite directions in the somata and distal apical dendrites, with somata becoming less sensitive to pattern-violating stimuli and distal apical dendrites more sensitive. These differences between the somata and distal apical dendrites may be important for hierarchical computation of sensory predictions and learning, since these two compartments tend to receive bottom-up and top-down information, respectively.

Experience shapes chandelier cell function and structure in the visual cortex

Detailed characterization of interneuron types in primary visual cortex (V1) has greatly contributed to understanding visual perception, yet the role of chandelier cells (ChCs) in visual processing remains poorly characterized. Using viral tracing we found that V1 ChCs predominantly receive monosynaptic input from local layer 5 pyramidal cells and higher-order cortical regions. Two-photon calcium imaging and convolutional neural network modeling revealed that ChCs are visually responsive but weakly selective for stimulus content. In mice running in a virtual tunnel, ChCs respond strongly to events known to elicit arousal, including locomotion and visuomotor mismatch. Repeated exposure of the mice to the virtual tunnel was accompanied by reduced visual responses of ChCs and structural plasticity of ChC boutons and axon initial segment length. Finally, ChCs only weakly inhibited pyramidal cells. These findings suggest that ChCs provide an arousal-related signal to layer 2/3 pyramidal cells that may modulate their activity and/or gate plasticity of their axon initial segments during behaviorally relevant events.

Computational components of visual predictive coding circuitry

If a full visual percept can be said to be a 'hypothesis', so too can a neural 'prediction' - although the latter addresses one particular component of image content (such as 3-dimensional organisation, the interplay between lighting and surface colour, the future trajectory of moving objects, and so on). And, because processing is hierarchical, predictions generated at one level are conveyed in a backward direction to a lower level, seeking to predict, in fact, the neural activity at that prior stage of processing, and learning from errors signalled in the opposite direction. This is the essence of 'predictive coding', at once an algorithm for information processing and a theoretical basis for the nature of operations performed by the cerebral cortex. Neural models for the implementation of predictive coding invoke specific functional classes of neuron for generating, transmitting and receiving predictions, and for producing reciprocal error signals. Also a third general class, 'precision' neurons, tasked with regulating the magnitude of error signals contingent upon the confidence placed upon the prediction, i.e., the reliability and behavioural utility of the sensory data that it predicts. So, what is the ultimate source of a 'prediction'? The answer is multifactorial: knowledge of the current environmental context and the immediate past, allied to memory and lifetime experience of the way of the world, doubtless fine-tuned by evolutionary history too. There are, in consequence, numerous potential avenues for experimenters seeking to manipulate subjects' expectation, and examine the neural signals elicited by surprising, and less surprising visual stimuli. This review focuses upon the predictive physiology of mouse and monkey visual cortex, summarising and commenting on evidence to date, and placing it in the context of the broader field. It is concluded that predictive coding has a firm grounding in basic neuroscience and that, unsurprisingly, there remains much to learn.

How deep is the brain? The shallow brain hypothesis

Deep learning and predictive coding architectures commonly assume that inference in neural networks is hierarchical. However, largely neglected in deep learning and predictive coding architectures is the neurobiological evidence that all hierarchical cortical areas, higher or lower, project to and receive signals directly from subcortical areas. Given these neuroanatomical facts, today's dominance of cortico-centric, hierarchical architectures in deep learning and predictive coding networks is highly questionable; such architectures are likely to be missing essential computational principles the brain uses. In this Perspective, we present the shallow brain hypothesis: hierarchical cortical processing is integrated with a massively parallel process to which subcortical areas substantially contribute. This shallow architecture exploits the computational capacity of cortical microcircuits and thalamo-cortical loops that are not included in typical hierarchical deep learning and predictive coding networks. We argue that the shallow brain architecture provides several critical benefits over deep hierarchical structures and a more complete depiction of how mammalian brains achieve fast and flexible computational capabilities.

Stimulus-Specific Prediction Error Neurons in Mouse Auditory Cortex

Comparing expectation with experience is an important neural computation performed throughout the brain and is a hallmark of predictive processing. Experiments that alter the sensory outcome of an animal's behavior reveal enhanced neural responses to unexpected self-generated stimuli, indicating that populations of neurons in sensory cortex may reflect prediction errors (PEs), mismatches between expectation and experience. However, enhanced neural responses to self-generated stimuli could also arise through nonpredictive mechanisms, such as the movement-based facilitation of a neuron's inherent sound responses. If sensory prediction error neurons exist in sensory cortex, it is unknown whether they manifest as general error responses, or respond with specificity to errors in distinct stimulus dimensions. To answer these questions, we trained mice of either sex to expect the outcome of a simple sound-generating behavior and recorded auditory cortex activity as mice heard either the expected sound or sounds that deviated from expectation in one of multiple distinct dimensions. Our data reveal that the auditory cortex learns to suppress responses to self-generated sounds along multiple acoustic dimensions simultaneously. We identify a distinct population of auditory cortex neurons that are not responsive to passive sounds or to the expected sound but that encode prediction errors. These prediction error neurons are abundant only in animals with a learned motor-sensory expectation, and encode one or two specific violations rather than a generic error signal. Together, these findings reveal that cortical predictions about self-generated sounds have specificity in multiple simultaneous dimensions and that cortical prediction error neurons encode specific violations from expectation.SIGNIFICANCE STATEMENT Audette et. al record neural activity in the auditory cortex while mice perform a sound-generating forelimb movement and measure neural responses to sounds that violate an animal's expectation in different ways. They find that predictions about self-generated sounds are highly specific across multiple stimulus dimensions and that a population of typically nonsound-responsive neurons respond to sounds that violate an animal's expectation in a specific way. These results identify specific prediction error (PE) signals in the mouse auditory cortex and suggest that errors may be calculated early in sensory processing.

Neural mechanisms for the localization of unexpected external motion

To localize objects during active sensing, animals must differentiate stimuli caused by volitional movement from real-world object motion. To determine a neural basis for this ability, we examined the mouse superior colliculus (SC), which contains multiple egocentric maps of sensorimotor space. By placing mice in a whisker-guided virtual reality, we discovered a rapidly adapting tactile response that transiently emerged during externally generated gains in whisker contact. Responses to self-generated touch that matched self-generated history were significantly attenuated, revealing that transient response magnitude is controlled by sensorimotor predictions. The magnitude of the transient response gradually decreased with repetitions in external motion, revealing a slow habituation based on external history. The direction of external motion was accurately encoded in the firing rates of transiently responsive neurons. These data reveal that whisker-specific adaptation and sensorimotor predictions in SC neurons enhance the localization of unexpected, externally generated changes in tactile space.

Top-down input modulates visual context processing through an interneuron-specific circuit

Visual stimuli that deviate from the current context elicit augmented responses in the primary visual cortex (V1). These heightened responses, known as "deviance detection," require local inhibition in the V1 and top-down input from the anterior cingulate area (ACa). Here, we investigated the mechanisms by which the ACa and V1 interact to support deviance detection. Local field potential recordings in mice during an oddball paradigm showed that ACa-V1 synchrony peaks in the theta/alpha band (≈10 Hz). Two-photon imaging in the V1 revealed that mainly pyramidal neurons exhibited deviance detection, while contextually redundant stimuli increased vasoactive intestinal peptide (VIP)-positive interneuron (VIP) activity and decreased somatostatin-positive interneuron (SST) activity. Optogenetic drive of ACa-V1 inputs at 10 Hz activated V1-VIPs but inhibited V1-SSTs, mirroring the dynamics present during the oddball paradigm. Chemogenetic inhibition of V1-VIPs disrupted Aca-V1 synchrony and deviance detection in the V1. These results outline temporal and interneuron-specific mechanisms of top-down modulation that support visual context processing.

Local minimization of prediction errors drives learning of invariant object representations in a generative network model of visual perception

The ventral visual processing hierarchy of the cortex needs to fulfill at least two key functions: perceived objects must be mapped to high-level representations invariantly of the precise viewing conditions, and a generative model must be learned that allows, for instance, to fill in occluded information guided by visual experience. Here, we show how a multilayered predictive coding network can learn to recognize objects from the bottom up and to generate specific representations via a top-down pathway through a single learning rule: the local minimization of prediction errors. Trained on sequences of continuously transformed objects, neurons in the highest network area become tuned to object identity invariant of precise position, comparable to inferotemporal neurons in macaques. Drawing on this, the dynamic properties of invariant object representations reproduce experimentally observed hierarchies of timescales from low to high levels of the ventral processing stream. The predicted faster decorrelation of error-neuron activity compared to representation neurons is of relevance for the experimental search for neural correlates of prediction errors. Lastly, the generative capacity of the network is confirmed by reconstructing specific object images, robust to partial occlusion of the inputs. By learning invariance from temporal continuity within a generative model, the approach generalizes the predictive coding framework to dynamic inputs in a more biologically plausible way than self-supervised networks with non-local error-backpropagation. This was achieved simply by shifting the training paradigm to dynamic inputs, with little change in architecture and learning rule from static input-reconstructing Hebbian predictive coding networks.

Robust multisensory deviance detection in the mouse parietal associative area

Context modulates how information is processed in the mammalian brain. For example, brain responses are amplified to contextually unusual stimuli. This phenomenon, known as "deviance detection,"1,2 is well documented in early, primary sensory cortex, where large responses are generated to simple stimuli that deviate from their context in low-order properties, such as line orientation, size, or pitch.2,3,4,5 However, the extent to which neural deviance detection manifests (1) in broader cortical networks and (2) to simple versus complex stimuli, which deviate only in their higher-order, multisensory properties, is not known. Consistent with a predictive processing framework,6,7 we hypothesized that deviance detection manifests in a hierarchical manner across cortical networks,8,9 emerging later and further downstream when stimulus deviance is complex. To test this, we examined brain responses of awake mice to simple unisensory deviants (e.g., visual line gratings, deviating from context in their orientation alone) versus complex multisensory deviants (i.e., audiovisual pairs, deviating from context only in their audiovisual pairing but not visual or auditory content alone). We find that mouse parietal associative area-a higher cortical region-displays robust multisensory deviance detection. In contrast, primary visual cortex exhibits strong unisensory visual deviance detection but weaker multisensory deviance detection. These results suggest that deviance detection signals in the cortex may be conceptualized as "prediction errors," which are primarily fed forward-or downstream-in cortical networks.6,7.

Molecularly targetable cell types in mouse visual cortex have distinguishable prediction error responses

Predictive processing postulates the existence of prediction error neurons in cortex. Neurons with both negative and positive prediction error response properties have been identified in layer 2/3 of visual cortex, but whether they correspond to transcriptionally defined subpopulations is unclear. Here we used the activity-dependent, photoconvertible marker CaMPARI2 to tag neurons in layer 2/3 of mouse visual cortex during stimuli and behaviors designed to evoke prediction errors. We performed single-cell RNA-sequencing on these populations and found that previously annotated Adamts2 and Rrad layer 2/3 transcriptional cell types were enriched when photolabeling during stimuli that drive negative or positive prediction error responses, respectively. Finally, we validated these results functionally by designing artificial promoters for use in AAV vectors to express genetically encoded calcium indicators. Thus, transcriptionally distinct cell types in layer 2/3 that can be targeted using AAV vectors exhibit distinguishable negative and positive prediction error responses.

Signal processing in the vagus nerve: Hypotheses based on new genetic and anatomical evidence

Each organism must regulate its internal state in a metabolically efficient way as it interacts in space and time with an ever-changing and only partly predictable world. Success in this endeavor is largely determined by the ongoing communication between brain and body, and the vagus nerve is a crucial structure in that dialogue. In this review, we introduce the novel hypothesis that the afferent vagus nerve is engaged in signal processing rather than just signal relay. New genetic and structural evidence of vagal afferent fiber anatomy motivates two hypotheses: (1) that sensory signals informing on the physiological state of the body compute both spatial and temporal viscerosensory features as they ascend the vagus nerve, following patterns found in other sensory architectures, such as the visual and olfactory systems; and (2) that ascending and descending signals modulate one another, calling into question the strict segregation of sensory and motor signals, respectively. Finally, we discuss several implications of our two hypotheses for understanding the role of viscerosensory signal processing in predictive energy regulation (i.e., allostasis) as well as the role of metabolic signals in memory and in disorders of prediction (e.g., mood disorders).

Functional and spatial rewiring principles jointly regulate context-sensitive computation

Adaptive rewiring provides a basic principle of self-organizing connectivity in evolving neural network topology. By selectively adding connections to regions with intense signal flow and deleting underutilized connections, adaptive rewiring generates optimized brain-like, i.e. modular, small-world, and rich club connectivity structures. Besides topology, neural self-organization also follows spatial optimization principles, such as minimizing the neural wiring distance and topographic alignment of neural pathways. We simulated the interplay of these spatial principles and adaptive rewiring in evolving neural networks with weighted and directed connections. The neural traffic flow within the network is represented by the equivalent of diffusion dynamics for directed edges: consensus and advection. We observe a constructive synergy between adaptive and spatial rewiring, which contributes to network connectedness. In particular, wiring distance minimization facilitates adaptive rewiring in creating convergent-divergent units. These units support the flow of neural information and enable context-sensitive information processing in the sensory cortex and elsewhere. Convergent-divergent units consist of convergent hub nodes, which collect inputs from pools of nodes and project these signals via a densely interconnected set of intermediate nodes onto divergent hub nodes, which broadcast their output back to the network. Convergent-divergent units vary in the degree to which their intermediate nodes are isolated from the rest of the network. This degree, and hence the context-sensitivity of the network's processing style, is parametrically determined in the evolving network model by the relative prominence of spatial versus adaptive rewiring.

Spatiotemporal dynamics across visual cortical laminae support a predictive coding framework for interpreting mismatch responses

Context modulates neocortical processing of sensory data. Unexpected visual stimuli elicit large responses in primary visual cortex (V1)-a phenomenon known as deviance detection (DD) at the neural level, or "mismatch negativity" (MMN) when measured with EEG. It remains unclear how visual DD/MMN signals emerge across cortical layers, in temporal relation to the onset of deviant stimuli, and with respect to brain oscillations. Here we employed a visual "oddball" sequence-a classic paradigm for studying aberrant DD/MMN in neuropsychiatric populations-and recorded local field potentials in V1 of awake mice with 16-channel multielectrode arrays. Multiunit activity and current source density profiles showed that although basic adaptation to redundant stimuli was present early (50 ms) in layer 4 responses, DD emerged later (150-230 ms) in supragranular layers (L2/3). This DD signal coincided with increased delta/theta (2-7 Hz) and high-gamma (70-80 Hz) oscillations in L2/3 and decreased beta oscillations (26-36 Hz) in L1. These results clarify the neocortical dynamics elicited during an oddball paradigm at a microcircuit level. They are consistent with a predictive coding framework, which posits that predictive suppression is present in cortical feed-back circuits, which synapse in L1, whereas "prediction errors" engage cortical feed-forward processing streams, which emanate from L2/3.

Genetic mechanisms for impaired synaptic plasticity in schizophrenia revealed by computational modelling

Schizophrenia phenotypes are suggestive of impaired cortical plasticity in the disease, but the mechanisms of these deficits are unknown. Genomic association studies have implicated a large number of genes that regulate neuromodulation and plasticity, indicating that the plasticity deficits have a genetic origin. Here, we used biochemically detailed computational modelling of post-synaptic plasticity to investigate how schizophrenia-associated genes regulate long-term potentiation (LTP) and depression (LTD). We combined our model with data from post-mortem mRNA expression studies (CommonMind gene-expression datasets) to assess the consequences of altered expression of plasticity-regulating genes for the amplitude of LTP and LTD. Our results show that the expression alterations observed post mortem, especially those in anterior cingulate cortex, lead to impaired PKA-pathway-mediated LTP in synapses containing GluR1 receptors. We validated these findings using a genotyped EEG dataset where polygenic risk scores for synaptic and ion channel-encoding genes as well as modulation of visual evoked potentials (VEP) were determined for 286 healthy controls. Our results provide a possible genetic mechanism for plasticity impairments in schizophrenia, which can lead to improved understanding and, ultimately, treatment of the disorder.

The locus coeruleus broadcasts prediction errors across the cortex to promote sensorimotor plasticity

Prediction errors are differences between expected and actual sensory input and are thought to be key computational signals that drive learning related plasticity. One way that prediction errors could drive learning is by activating neuromodulatory systems to gate plasticity. The catecholaminergic locus coeruleus (LC) is a major neuromodulatory system involved in neuronal plasticity in the cortex. Using two-photon calcium imaging in mice exploring a virtual environment, we found that the activity of LC axons in the cortex correlated with the magnitude of unsigned visuomotor prediction errors. LC response profiles were similar in both motor and visual cortical areas, indicating that LC axons broadcast prediction errors throughout the dorsal cortex. While imaging calcium activity in layer 2/3 of the primary visual cortex, we found that optogenetic stimulation of LC axons facilitated learning of a stimulus-specific suppression of visual responses during locomotion. This plasticity - induced by minutes of LC stimulation - recapitulated the effect of visuomotor learning on a scale that is normally observed during visuomotor development across days. We conclude that prediction errors drive LC activity, and that LC activity facilitates sensorimotor plasticity in the cortex, consistent with a role in modulating learning rates.

Dual-stream cortical pathways mediate sensory prediction

Predictions are constantly generated from diverse sources to optimize cognitive functions in the ever-changing environment. However, the neural origin and generation process of top-down induced prediction remain elusive. We hypothesized that motor-based and memory-based predictions are mediated by distinct descending networks from motor and memory systems to the sensory cortices. Using functional magnetic resonance imaging (fMRI) and a dual imagery paradigm, we found that motor and memory upstream systems activated the auditory cortex in a content-specific manner. Moreover, the inferior and posterior parts of the parietal lobe differentially relayed predictive signals in motor-to-sensory and memory-to-sensory networks. Dynamic causal modeling of directed connectivity revealed selective enabling and modulation of connections that mediate top-down sensory prediction and ground the distinctive neurocognitive basis of predictive processing.

Stimulus-specific prediction error neurons in mouse auditory cortex

Comparing expectation with experience is an important neural computation performed throughout the brain and is a hallmark of predictive processing. Experiments that alter the sensory outcome of an animal's behavior reveal enhanced neural responses to unexpected self-generated stimuli, indicating that populations of neurons in sensory cortex may reflect prediction errors - mismatches between expectation and experience. However, enhanced neural responses to self-generated stimuli could also arise through non-predictive mechanisms, such as the movement-based facilitation of a neuron's inherent sound responses. If sensory prediction error neurons exist in sensory cortex, it is unknown whether they manifest as general error responses, or respond with specificity to errors in distinct stimulus dimensions. To answer these questions, we trained mice to expect the outcome of a simple sound-generating behavior and recorded auditory cortex activity as mice heard either the expected sound or sounds that deviated from expectation in one of multiple distinct dimensions. Our data reveal that the auditory cortex learns to suppress responses to self-generated sounds along multiple acoustic dimensions simultaneously. We identify a distinct population of auditory cortex neurons that are not responsive to passive sounds or to the expected sound but that explicitly encode prediction errors. These prediction error neurons are abundant only in animals with a learned motor-sensory expectation, and encode one or two specific violations rather than a generic error signal.

Multimodal mismatch responses in associative but not primary visual cortex support hierarchical predictive coding in cortical networks

A key function of the mammalian neocortex is to process sensory data in the context of current and past stimuli. Primary sensory cortices, such as V1, respond weakly to stimuli that typical in their context but strongly to novel stimuli, an effect known as "deviance detection". How deviance detection occurs in associative cortical regions that are downstream of V1 is not well-understood. Here we investigated parietal associative area (PTLp) responses to auditory, visual, and audio-visual mismatches with two-photon calcium imaging and local field potential recordings. We employed basic unisensory auditory and visual oddball paradigms as well as a novel multisensory oddball paradigm, involving typical parings (VaAc or VbAd) presented at p=.88 with rare "deviant" pairings (e.g. VaAd or VbAc) presented at p=.12. We found that PTLp displayed robust deviance detection responses to auditory-visual mismatches, both in individual neurons and in population theta and gamma-band oscillations. In contrast, V1 neurons displayed deviance detection only to visual deviants in a unisensory context, but not to auditory or auditory-visual mismatches. Taken together, these results accord with a predictive processing framework for cortical responses, wherein modality specific prediction errors (i.e. deviance detection responses) are computed in functionally specified cortical areas and feed-forward to update higher brain regions.

How the conception of control influences our understanding of actions

Wilful movement requires neural control. Commonly, neural computations are thought to generate motor commands that bring the musculoskeletal system - that is, the plant - from its current physical state into a desired physical state. The current state can be estimated from past motor commands and from sensory information. Modelling movement on the basis of this concept of plant control strives to explain behaviour by identifying the computational principles for control signals that can reproduce the observed features of movements. From an alternative perspective, movements emerge in a dynamically coupled agent-environment system from the pursuit of subjective perceptual goals. Modelling movement on the basis of this concept of perceptual control aims to identify the controlled percepts and their coupling rules that can give rise to the observed characteristics of behaviour. In this Perspective, we discuss a broad spectrum of approaches to modelling human motor control and their notions of control signals, internal models, handling of sensory feedback delays and learning. We focus on the influence that the plant control and the perceptual control perspective may have on decisions when modelling empirical data, which may in turn shape our understanding of actions.

Locomotion-induced gain of visual responses cannot explain visuomotor mismatch responses in layer 2/3 of primary visual cortex

The aim of this work is to provide a comment on a recent paper by Muzzu and Saleem (2021), which claims that visuomotor mismatch responses in mouse visual cortex can be explained by a locomotion-induced gain of visual halt responses. Our primary concern is that without directly comparing these responses with mismatch responses, the claim that one response can explain the other appears difficult to uphold, more so because previous work finds that a uniform locomotion-induced gain cannot explain mismatch responses. To support these arguments, we analyze layer 2/3 calcium imaging datasets and show that coupling between visual flow and locomotion greatly enhances mismatch responses in an experience-dependent manner compared with halts in non-coupled visual flow. This is consistent with mismatch responses representing visuomotor prediction errors. Thus, we conclude that while feature selectivity might contribute to mismatch responses in mouse visual cortex, it cannot explain these responses.

Bayesian surprise shapes neural responses in somatosensory cortical circuits

Numerous psychophysical studies show that Bayesian inference governs sensory decision-making; however, the specific neural circuitry underlying this probabilistic mechanism remains unclear. We record extracellular neural activity along the somatosensory pathway of mice while delivering sensory stimulation paradigms designed to isolate the response to the surprise generated by Bayesian inference. Our results demonstrate that laminar cortical circuits in early sensory areas encode Bayesian surprise. Systematic sensitivity to surprise is not identified in the somatosensory thalamus, rather emerging in the primary (S1) and secondary (S2) somatosensory cortices. Multiunit spiking activity and evoked potentials in layer 6 of these regions exhibit the highest sensitivity to surprise. Gamma power in S1 layer 2/3 exhibits an NMDAR-dependent scaling with surprise, as does alpha power in layers 2/3 and 6 of S2. These results show a precise spatiotemporal neural representation of Bayesian surprise and suggest that Bayesian inference is a fundamental component of cortical processing.

A frontosensory circuit for visual context processing is synchronous in the theta/alpha band

Visual processing is strongly influenced by context. Stimuli that deviate from contextual regularities elicit augmented responses in primary visual cortex (V1). These heightened responses, known as "deviance detection," require both inhibition local to V1 and top-down modulation from higher areas of cortex. Here we investigated the spatiotemporal mechanisms by which these circuit elements interact to support deviance detection. Local field potential recordings in mice in anterior cingulate area (ACa) and V1 during a visual oddball paradigm showed that interregional synchrony peaks in the theta/alpha band (6-12 Hz). Two-photon imaging in V1 revealed that mainly pyramidal neurons exhibited deviance detection, while vasointestinal peptide-positive interneurons (VIPs) increased activity and somatostatin-positive interneurons (SSTs) decreased activity (adapted) to redundant stimuli (prior to deviants). Optogenetic drive of ACa-V1 inputs at 6-12 Hz activated V1-VIPs but inhibited V1-SSTs, mirroring the dynamics present during the oddball paradigm. Chemogenetic inhibition of VIP interneurons disrupted ACa-V1 synchrony and deviance detection responses in V1. These results outline spatiotemporal and interneuron-specific mechanisms of top-down modulation that support visual context processing.

Cerebro-cerebellar networks facilitate learning through feedback decoupling

Behavioural feedback is critical for learning in the cerebral cortex. However, such feedback is often not readily available. How the cerebral cortex learns efficiently despite the sparse nature of feedback remains unclear. Inspired by recent deep learning algorithms, we introduce a systems-level computational model of cerebro-cerebellar interactions. In this model a cerebral recurrent network receives feedback predictions from a cerebellar network, thereby decoupling learning in cerebral networks from future feedback. When trained in a simple sensorimotor task the model shows faster learning and reduced dysmetria-like behaviours, in line with the widely observed functional impact of the cerebellum. Next, we demonstrate that these results generalise to more complex motor and cognitive tasks. Finally, the model makes several experimentally testable predictions regarding cerebro-cerebellar task-specific representations over learning, task-specific benefits of cerebellar predictions and the differential impact of cerebellar and inferior olive lesions. Overall, our work offers a theoretical framework of cerebro-cerebellar networks as feedback decoupling machines.

Where is the error? Hierarchical predictive coding through dendritic error computation

Top-down feedback in cortex is critical for guiding sensory processing, which has prominently been formalized in the theory of hierarchical predictive coding (hPC). However, experimental evidence for error units, which are central to the theory, is inconclusive and it remains unclear how hPC can be implemented with spiking neurons. To address this, we connect hPC to existing work on efficient coding in balanced networks with lateral inhibition and predictive computation at apical dendrites. Together, this work points to an efficient implementation of hPC with spiking neurons, where prediction errors are computed not in separate units, but locally in dendritic compartments. We then discuss the correspondence of this model to experimentally observed connectivity patterns, plasticity, and dynamics in cortex.

Probing top-down information in neocortical layer 1

Accurate perception of the environment is a constructive process that requires integration of external bottom-up sensory signals with internally generated top-down information. Decades of work have elucidated how sensory neocortex processes physical stimulus features. By contrast, examining how top-down information is encoded and integrated with bottom-up signals has been challenging using traditional neuroscience methods. Recent technological advances in functional imaging of brain-wide afferents in behaving mice have enabled the direct measurement of top-down information. Here, we review the emerging literature on encoding of these internally generated signals by different projection systems enriched in neocortical layer 1 during defined brain functions, including memory, attention, and predictive coding. Moreover, we identify gaps in current knowledge and highlight future directions for this rapidly advancing field.

Precise movement-based predictions in the mouse auditory cortex

Many of the sensations experienced by an organism are caused by their own actions, and accurately anticipating both the sensory features and timing of self-generated stimuli is crucial to a variety of behaviors. In the auditory cortex, neural responses to self-generated sounds exhibit frequency-specific suppression, suggesting that movement-based predictions may be implemented early in sensory processing. However, it remains unknown whether this modulation results from a behaviorally specific and temporally precise prediction, nor is it known whether corresponding expectation signals are present locally in the auditory cortex. To address these questions, we trained mice to expect the precise acoustic outcome of a forelimb movement using a closed-loop sound-generating lever. Dense neuronal recordings in the auditory cortex revealed suppression of responses to self-generated sounds that was specific to the expected acoustic features, to a precise position within the movement, and to the movement that was coupled to sound during training. Prediction-based suppression was concentrated in L2/3 and L5, where deviations from expectation also recruited a population of prediction-error neurons that was otherwise unresponsive. Recording in the absence of sound revealed abundant movement signals in deep layers that were biased toward neurons tuned to the expected sound, as well as expectation signals that were present throughout the cortex and peaked at the time of expected auditory feedback. Together, these findings identify distinct populations of auditory cortical neurons with movement, expectation, and error signals consistent with a learned internal model linking an action to its specific acoustic outcome.

Allostasis as a core feature of hierarchical gradients in the human brain

This paper integrates emerging evidence from two broad streams of scientific literature into one common framework: (a) hierarchical gradients of functional connectivity that reflect the brain's large-scale structural architecture (e.g., a lamination gradient in the cerebral cortex); and (b) approaches to predictive processing and one of its specific instantiations called allostasis (i.e., the predictive regulation of energetic resources in the service of coordinating the body's internal systems). This synthesis begins to sketch a coherent, neurobiologically inspired framework suggesting that predictive energy regulation is at the core of human brain function, and by extension, psychological and behavioral phenomena, providing a shared vocabulary for theory building and knowledge accumulation.

Distinguishing externally from saccade-induced motion in visual cortex

Distinguishing sensory stimuli caused by changes in the environment from those caused by an animal’s own actions is a hallmark of sensory processing¹. Saccades are rapid eye movements that shift the image on the retina. How visual systems differentiate motion of the image induced by saccades from actual motion in the environment is not fully understood². Here we discovered that in mouse primary visual cortex (V1) the two types of motion evoke distinct activity patterns. This is because, during saccades, V1 combines the visual input with a strong non-visual input arriving from the thalamic pulvinar nucleus. The non-visual input triggers responses that are specific to the direction of the saccade and the visual input triggers responses that are specific to the direction of the shift of the stimulus on the retina, yet the preferred directions of these two responses are uncorrelated. Thus, the pulvinar input ensures differential V1 responses to external and self-generated motion. Integration of external sensory information with information about body movement may be a general mechanism for sensory cortices to distinguish between self-generated and external stimuli.

Distinct activity patterns in the primary visual cortex distinguish movement in the environment from motion caused by eye movements.