Decision-related activity in sensory neurons: correlations among neurons and with behavior

A neural mechanism for optic flow parsing in macaque visual cortex

For the brain to compute object motion in the world during self-motion, it must discount the global patterns of image motion (optic flow) caused by self-motion. Optic flow parsing is a proposed visual mechanism for computing object motion in the world, and studies in both humans and monkeys have demonstrated perceptual biases consistent with the operation of a flow-parsing mechanism. However, the neural basis of flow parsing remains unknown. We demonstrate, at both the individual unit and population levels, that neural activity in macaque middle temporal (MT) area is biased by peripheral optic flow in a manner that can at least partially account for perceptual biases induced by flow parsing. These effects cannot be explained by conventional surround suppression mechanisms or choice-related activity and have substantial neural latency. Together, our findings establish the first neural basis for the computation of scene-relative object motion based on flow parsing.

Neuronal and Behavioral Responses to Naturalistic Texture Images in Macaque Monkeys

The visual world is richly adorned with texture, which can serve to delineate important elements of natural scenes. In anesthetized macaque monkeys, selectivity for the statistical features of natural texture is weak in V1, but substantial in V2, suggesting that neuronal activity in V2 might directly support texture perception. To test this, we investigated the relation between single cell activity in macaque V1 and V2 and simultaneously measured behavioral judgments of texture. We generated stimuli along a continuum between naturalistic texture and phase-randomized noise and trained two macaque monkeys to judge whether a sample texture more closely resembled one or the other extreme. Analysis of responses revealed that individual V1 and V2 neurons carried much less information about texture naturalness than behavioral reports. However, the sensitivity of V2 neurons, especially those preferring naturalistic textures, was significantly closer to that of behavior compared with V1. The firing of both V1 and V2 neurons predicted perceptual choices in response to repeated presentations of the same ambiguous stimulus in one monkey, despite low individual neural sensitivity. However, neither population predicted choice in the second monkey. We conclude that neural responses supporting texture perception likely continue to develop downstream of V2. Further, combined with neural data recorded while the same two monkeys performed an orientation discrimination task, our results demonstrate that choice-correlated neural activity in early sensory cortex is unstable across observers and tasks, untethered from neuronal sensitivity, and therefore unlikely to directly reflect the formation of perceptual decisions.

Dynamic ensemble balance in direct- and indirect-pathway striatal projection neurons underlying decision-related action selection

The posterior dorsal striatum (pDS) plays an essential role in sensory-guided decision-making. However, it remains unclear how the antagonizing direct- and indirect-pathway striatal projection neurons (dSPNs and iSPNs) work in concert to support action selection. Here, we employed deep-brain two-photon imaging to investigate pathway-specific single-neuron and population representations during an auditory-guided decision-making task. We found that the majority of pDS projection neurons predominantly encode choice information. Both dSPNs and iSPNs comprise divergent subpopulations of comparable sizes representing competing choices, rendering a multi-ensemble balance between the two pathways. Intriguingly, such ensemble balance displays a dynamic shift during the decision period: dSPNs show a significantly stronger preference for the contraversive choice than iSPNs. This dynamic shift is further manifested in the inter-neuronal coactivity and population trajectory divergence. Our results support a balance-shift model as a neuronal population mechanism coordinating the direct and indirect striatal pathways for eliciting selected actions during decision-making.

A latent pool of neurons silenced by sensory-evoked inhibition can be recruited to enhance perception

To investigate which activity patterns in sensory cortex are relevant for perceptual decision-making, we combined two-photon calcium imaging and targeted two-photon optogenetics to interrogate barrel cortex activity during perceptual discrimination. We trained mice to discriminate bilateral whisker deflections and report decisions by licking left or right. Two-photon calcium imaging revealed sparse coding of contralateral and ipsilateral whisker input in layer 2/3, with most neurons remaining silent during the task. Activating pyramidal neurons using two-photon holographic photostimulation evoked a perceptual bias that scaled with the number of neurons photostimulated. This effect was dominated by optogenetic activation of non-coding neurons, which did not show sensory or motor-related activity during task performance. Photostimulation also revealed potent recruitment of cortical inhibition during sensory processing, which strongly and preferentially suppressed non-coding neurons. Our results suggest that a pool of non-coding neurons, selectively suppressed by network inhibition during sensory processing, can be recruited to enhance perception.

Coexistence of state, choice, and sensory integration coding in barrel cortex LII/III

During perceptually guided decisions, correlates of choice are found as upstream as in the primary sensory areas. However, how well these choice signals align with early sensory representations, a prerequisite for their interpretation as feedforward substrates of perception, remains an open question. We designed a two alternative forced choice task (2AFC) in which male mice compared stimulation frequencies applied to two adjacent vibrissae. The optogenetic silencing of individual columns in the primary somatosensory cortex (wS1) resulted in predicted shifts of psychometric functions, demonstrating that perception depends on focal, early sensory representations. Functional imaging of layer II/III single neurons revealed mixed coding of stimuli, choices and engagement in the task. Neurons with multi-whisker suppression display improved sensory discrimination and had their activity increased during engagement in the task, enhancing selectively representation of the signals relevant to solving the task. From trial to trial, representation of stimuli and choice varied substantially, but mostly orthogonally to each other, suggesting that perceptual variability does not originate from wS1 fluctuations but rather from downstream areas. Together, our results highlight the role of primary sensory areas in forming a reliable sensory substrate that could be used for flexible downstream decision processes.

Differential representation of sensory information and behavioral choice across layers of the mouse auditory cortex

The activity of neurons in sensory areas sometimes covaries with upcoming choices in decision-making tasks. However, the prevalence, causal origin, and functional role of choice-related activity remain controversial. Understanding the circuit-logic of decision signals in sensory areas will require understanding their laminar specificity, but simultaneous recordings of neural activity across the cortical layers in forced-choice discrimination tasks have not yet been performed. Here, we describe neural activity from such recordings in the auditory cortex of mice during a frequency discrimination task with delayed report, which, as we show, requires the auditory cortex. Stimulus-related information was widely distributed across layers but disappeared very quickly after stimulus offset. Choice selectivity emerged toward the end of the delay period-suggesting a top-down origin-but only in the deep layers. Early stimulus-selective and late choice-selective deep neural ensembles were correlated, suggesting that the choice-selective signal fed back to the auditory cortex is not just action specific but develops as a consequence of the sensory-motor contingency imposed by the task.

Tactile processing in mouse cortex depends on action context

The brain receives constant tactile input, but only a subset guides ongoing behavior. Actions associated with tactile stimuli thus endow them with behavioral relevance. It remains unclear how the relevance of tactile stimuli affects processing in the somatosensory (S1) cortex. We developed a cross-modal selection task in which head-fixed mice switched between responding to tactile stimuli in the presence of visual distractors or to visual stimuli in the presence of tactile distractors using licking movements to the left or right side in different blocks of trials. S1 spiking encoded tactile stimuli, licking actions, and direction of licking in response to tactile but not visual stimuli. Bidirectional optogenetic manipulations showed that sensory-motor activity in S1 guided behavior when touch but not vision was relevant. Our results show that S1 activity and its impact on behavior depend on the actions associated with a tactile stimulus.

Neuronal and behavioral responses to naturalistic texture images in macaque monkeys

The visual world is richly adorned with texture, which can serve to delineate important elements of natural scenes. In anesthetized macaque monkeys, selectivity for the statistical features of natural texture is weak in V1, but substantial in V2, suggesting that neuronal activity in V2 might directly support texture perception. To test this, we investigated the relation between single cell activity in macaque V1 and V2 and simultaneously measured behavioral judgments of texture. We generated stimuli along a continuum between naturalistic texture and phase-randomized noise and trained two macaque monkeys to judge whether a sample texture more closely resembled one or the other extreme. Analysis of responses revealed that individual V1 and V2 neurons carried much less information about texture naturalness than behavioral reports. However, the sensitivity of V2 neurons, especially those preferring naturalistic textures, was significantly closer to that of behavior compared with V1. The firing of both V1 and V2 neurons predicted perceptual choices in response to repeated presentations of the same ambiguous stimulus in one monkey, despite low individual neural sensitivity. However, neither population predicted choice in the second monkey. We conclude that neural responses supporting texture perception likely continue to develop downstream of V2. Further, combined with neural data recorded while the same two monkeys performed an orientation discrimination task, our results demonstrate that choice-correlated neural activity in early sensory cortex is unstable across observers and tasks, untethered from neuronal sensitivity, and thus unlikely to reflect a critical aspect of the formation of perceptual decisions.

Significance statement

As visual signals propagate along the cortical hierarchy, they encode increasingly complex aspects of the sensory environment and likely have a more direct relationship with perceptual experience. We replicate and extend previous results from anesthetized monkeys differentiating the selectivity of neurons along the first step in cortical vision from area V1 to V2. However, our results further complicate efforts to establish neural signatures that reveal the relationship between perception and the neuronal activity of sensory populations. We find that choice-correlated activity in V1 and V2 is unstable across different observers and tasks, and also untethered from neuronal sensitivity and other features of nonsensory response modulation.

Employment of time-varying sensory evidence to test the mechanisms underlying flexible decision-making

To make flexible decisions in dynamic environments, the brain must integrate behaviorally relevant information while simultaneously discarding irrelevant information. This study aimed to investigate the mechanisms responsible for discarding irrelevant information during context-dependent decision-making. We trained two macaque monkeys to switch between direction and depth discrimination tasks in successive trials. During decision-making, the strength of the motion or depth signal changes transiently at various times, introducing a brief pulse. We assessed the effects of pulse on behavioral choices. Consistent with previous findings, early relevant pulses, such as motion pulses during direction discrimination, had a significantly larger effect compared to late pulses. Critically, the effects of irrelevant pulses, such as motion pulses during depth discrimination, exhibited an initial minimal effect, followed by an increase and subsequent decrease as a function of pulse timing. Gating mechanisms alone, aimed at discarding irrelevant information, did not account for the observed time course of pulse effects. Instead, the observed increase in the effects of irrelevant pulses with time suggested the involvement of a leaky integration mechanism. The results suggested that the brain controls the amount of disposal in accumulating sensory evidence during flexible decision-making.

Category representation in primary visual cortex after visual perceptual learning

The visual perceptual learning (VPL) leads to long-term enhancement of visual task performance. The subjects are often trained to link different visual stimuli to several options, such as the widely used two-alternative forced choice (2AFC) task, which involves an implicit categorical decision. The enhancement of performance has been related to the specific changes of neural activities, but few studies investigate the effects of categorical responding on the changes of neural activities. Here we investigated whether the neural activities would exhibit the categorical characteristics if the subjects are requested to respond visual stimuli in a categorical manner during VPL. We analyzed the neural activities of two monkeys in a contour detection VPL. We found that the neural activities in primary visual cortex (V1) converge to one pattern if the contour can be detected by monkey and another pattern if the contour cannot be detected, exhibiting a kind of category learning that the neural representations of detectable contour become less selective for number of bars forming contour and diverge from the representations of undetectable contour.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09926-8.

Online Estimating Pairwise Neuronal Functional Connectivity in Brain-Machine Interface

Neurons respond to external stimuli and form functional networks through pairwise interactions. A neural encoding model can describe a single neuron's behavior, and brain-machine interfaces (BMIs) provide a platform to investigate how neurons adapt, functionally connect, and encode movement. Movement modulation and pairwise functional connectivity are modeled as high-dimensional tuning states, estimated from neural spike train observations. However, accurate estimation of this neural state vector can be challenging as pairwise neural interactions are highly dimensional, change in different temporal scales from movement, and could be non-stationary. We propose an Adam-based gradient descent method to online estimate high-dimensional pairwise neuronal functional connectivity and single neuronal tuning adaptation simultaneously. By minimizing negative log-likelihood based on point process observation, the proposed method adaptively adjusts the learning rate for each dimension of the neural state vectors by employing momentum and regularizer. We test the method on real recordings of two rats performing the brain control mode of a two-lever discrimination task. Our results show that our method outperforms existing methods, especially when the state is sparse. Our method is more stable and faster for an online scenario regardless of the parameter initializations. Our method provides a promising tool to track and build the time-variant functional neural connectivity, which dynamically forms the functional network and results in better brain control.

Behavioral read-out from population value signals in primate orbitofrontal cortex

The primate orbitofrontal cortex (OFC) has long been recognized for its role in value-based decisions; however, the exact mechanism linking value representations in the OFC to decision outcomes has remained elusive. Here, to address this question, we show, in non-human primates, that trial-wise variability in choices can be explained by variability in value signals decoded from many simultaneously recorded OFC neurons. Mechanistically, this relationship is consistent with the projection of activity within a low-dimensional value-encoding subspace onto a potentially higher-dimensional, behaviorally potent output subspace. Identifying this neural-behavioral link answers longstanding questions about the role of the OFC in economic decision-making and suggests population-level read-out mechanisms for the OFC similar to those recently identified in sensory and motor cortex.

Stable sound decoding despite modulated sound representation in the auditory cortex

The activity of neurons in the auditory cortex is driven by both sounds and non-sensory context. To investigate the neuronal correlates of non-sensory context, we trained head-fixed mice to perform a two-alternative-choice auditory task in which either reward or stimulus expectation (prior) was manipulated in blocks. Using two-photon calcium imaging to record populations of single neurons in the auditory cortex, we found that both stimulus and reward expectation modulated the activity of these neurons. A linear decoder trained on this population activity could decode stimuli as well or better than predicted by the animal's performance. Interestingly, the optimal decoder was stable even in the face of variable sensory representations. Neither the context nor the mouse's choice could be reliably decoded from the recorded neural activity. Our findings suggest that, in spite of modulation of auditory cortical activity by task priors, the auditory cortex does not represent sufficient information about these priors to exploit them optimally. Thus, the combination of rapidly changing sensory information with more slowly varying task information required for decisions in this task might be represented in brain regions other than the auditory cortex.

Visuo-vestibular heading perception: a model system to study multi-sensory decision making

Integrating noisy signals across time as well as sensory modalities, a process named multi-sensory decision making (MSDM), is an essential strategy for making more accurate and sensitive decisions in complex environments. Although this field is just emerging, recent extraordinary works from different perspectives, including computational theory, psychophysical behaviour and neurophysiology, begin to shed new light onto MSDM. In the current review, we focus on MSDM by using a model system of visuo-vestibular heading. Combining well-controlled behavioural paradigms on virtual-reality systems, single-unit recordings, causal manipulations and computational theory based on spiking activity, recent progress reveals that vestibular signals contain complex temporal dynamics in many brain regions, including unisensory, multi-sensory and sensory-motor association areas. This challenges the brain for cue integration across time and sensory modality such as optic flow which mainly contains a motion velocity signal. In addition, new evidence from the higher-level decision-related areas, mostly in the posterior and frontal/prefrontal regions, helps revise our conventional thought on how signals from different sensory modalities may be processed, converged, and moment-by-moment accumulated through neural circuits for forming a unified, optimal perceptual decision. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Stable sound decoding despite modulated sound representation in the auditory cortex

The activity of neurons in the auditory cortex is driven by both sounds and non-sensory context. To investigate the neuronal correlates of non-sensory context, we trained head-fixed mice to perform a two-alternative choice auditory task in which either reward or stimulus expectation (prior) was manipulated in blocks. Using two-photon calcium imaging to record populations of single neurons in auditory cortex, we found that both stimulus and reward expectation modulated the activity of these neurons. A linear decoder trained on this population activity could decode stimuli as well or better than predicted by the animal's performance. Interestingly, the optimal decoder was stable even in the face of variable sensory representations. Neither the context nor the mouse's choice could be reliably decoded from the recorded neural activity. Our findings suggest that in spite of modulation of auditory cortical activity by task priors, auditory cortex does not represent sufficient information about these priors to exploit them optimally and that decisions in this task require that rapidly changing sensory information be combined with more slowly varying task information extracted and represented in brain regions other than auditory cortex.

Functional network properties of the auditory cortex

The auditory system transforms auditory stimuli from the external environment into perceptual auditory objects. Recent studies have focused on the contribution of the auditory cortex to this transformation. Other studies have yielded important insights into the contributions of neural activity in the auditory cortex to cognition and decision-making. However, despite this important work, the relationship between auditory-cortex activity and behavior/perception has not been fully elucidated. Two of the more important gaps in our understanding are (1) the specific and differential contributions of different fields of the auditory cortex to auditory perception and behavior and (2) the way networks of auditory neurons impact and facilitate auditory information processing. Here, we focus on recent work from non-human-primate models of hearing and review work related to these gaps and put forth challenges to further our understanding of how single-unit activity and network activity in different cortical fields contribution to behavior and perception.

Contributions of early and mid-level visual cortex to high-level object categorization

The complexity of visual features for which neurons are tuned increases from early to late stages of the ventral visual stream. Thus, the standard hypothesis is that high-level functions like object categorization are primarily mediated by higher visual areas because they require more complex image formats that are not evident in early visual processing stages. However, human observers can categorize images as objects or animals or as big or small even when the images preserve only some low- and mid-level features but are rendered unidentifiable ('texforms', Long et al., 2018). This observation suggests that even the early visual cortex, in which neurons respond to simple stimulus features, may already encode signals about these more abstract high-level categorical distinctions. We tested this hypothesis by recording from populations of neurons in early and mid-level visual cortical areas while rhesus monkeys viewed texforms and their unaltered source stimuli (simultaneous recordings from areas V1 and V4 in one animal and separate recordings from V1 and V4 in two others). Using recordings from a few dozen neurons, we could decode the real-world size and animacy of both unaltered images and texforms. Furthermore, this neural decoding accuracy across stimuli was related to the ability of human observers to categorize texforms by real-world size and animacy. Our results demonstrate that neuronal populations early in the visual hierarchy contain signals useful for higher-level object perception and suggest that the responses of early visual areas to simple stimulus features display preliminary untangling of higher-level distinctions.

Correlated variability in primate superior colliculus depends on functional class

Correlated variability in neuronal activity (spike count correlations, r_SC) can constrain how information is read out from populations of neurons. Traditionally, r_SC is reported as a single value summarizing a brain area. However, single values, like summary statistics, stand to obscure underlying features of the constituent elements. We predict that in brain areas containing distinct neuronal subpopulations, different subpopulations will exhibit distinct levels of r_SC that are not captured by the population r_SC. We tested this idea in macaque superior colliculus (SC), a structure containing several functional classes (i.e., subpopulations) of neurons. We found that during saccade tasks, different functional classes exhibited differing degrees of r_SC. "Delay class" neurons displayed the highest r_SC, especially during saccades that relied on working memory. Such dependence of r_SC on functional class and cognitive demand underscores the importance of taking functional subpopulations into account when attempting to model or infer population coding principles.

Weak evidence for neural correlates of task-switching in macaque V1

A central goal of systems neuroscience is to understand how populations of sensory neurons encode and relay information to the rest of the brain. Three key quantities of interest are 1) how mean neural activity depends on the stimulus (sensitivity), 2) how neural activity (co)varies around the mean (noise correlations), and 3) how predictive these variations are of the subject's behavior (choice probability). Previous empirical work suggests that both choice probability and noise correlations are affected by task training, with decision-related information fed back to sensory areas and aligned to neural sensitivity on a task-by-task basis. We used Utah arrays to record activity from populations of primary visual cortex (V1) neurons from two macaque monkeys that were trained to switch between two coarse orientation-discrimination tasks. Surprisingly, we find no evidence for significant trial-by-trial changes in noise covariance between tasks, nor do we find a consistent relationship between neural sensitivity and choice probability, despite recording from well-tuned task-sensitive neurons, many of which were histologically confirmed to be in supragranular V1, and despite behavioral evidence that the monkeys switched their strategy between tasks. Thus our data at best provide weak support for the hypothesis that trial-by-trial task-switching induces changes to noise correlations and choice probabilities in V1. However, our data agree with a recent finding of a single "choice axis" across tasks. They also raise the intriguing possibility that choice-related signals in early sensory areas are less indicative of task learning per se and instead reflect perceptual learning that occurs in highly overtrained subjects.NEW & NOTEWORTHY Converging evidence suggests that decision processes affect sensory neural activity, and this has informed numerous theories of neural processing. We set out to replicate and extend previous results on decision-related information and noise correlations in V1 of macaque monkeys. However, in our data, we find little evidence for a number of expected effects. Our null results therefore call attention to differences in task training, stimulus design, recording, and analysis techniques between our and prior studies.

Illusory object recognition is either perceptual or cognitive in origin depending on decision confidence

We occasionally misinterpret ambiguous sensory input or report a stimulus when none is presented. It is unknown whether such errors have a sensory origin and reflect true perceptual illusions, or whether they have a more cognitive origin (e.g., are due to guessing), or both. When participants performed an error-prone and challenging face/house discrimination task, multivariate electroencephalography (EEG) analyses revealed that during decision errors (e.g., mistaking a face for a house), sensory stages of visual information processing initially represent the presented stimulus category. Crucially however, when participants were confident in their erroneous decision, so when the illusion was strongest, this neural representation flipped later in time and reflected the incorrectly reported percept. This flip in neural pattern was absent for decisions that were made with low confidence. This work demonstrates that decision confidence arbitrates between perceptual decision errors, which reflect true illusions of perception, and cognitive decision errors, which do not.

Hierarchical differences in the encoding of sound and choice in the subcortical auditory system

Detection of sounds is a fundamental function of the auditory system. Although studies of auditory cortex have gained substantial insight into detection performance using behaving animals, previous subcortical studies have mostly taken place under anesthesia, in passively listening animals, or have not measured performance at threshold. These limitations preclude direct comparisons between neuronal responses and behavior. To address this, we simultaneously measured auditory detection performance and single-unit activity in the inferior colliculus (IC) and cochlear nucleus (CN) in macaques. The spontaneous activity and response variability of CN neurons were higher than those observed for IC neurons. Signal detection theoretic methods revealed that the magnitude of responses of IC neurons provided more reliable estimates of psychometric threshold and slope compared with the responses of single CN neurons. However, pooling small populations of CN neurons provided reliable estimates of psychometric threshold and slope, suggesting sufficient information in CN population activity. Trial-by-trial correlations between spike count and behavioral response emerged 50-75 ms after sound onset for most IC neurons, but for few neurons in the CN. These results highlight hierarchical differences between neurometric-psychometric correlations in CN and IC and have important implications for how subcortical information could be decoded.NEW & NOTEWORTHY The cerebral cortex is widely recognized to play a role in sensory processing and decision-making. Accounts of the neural basis of auditory perception and its dysfunction are based on this idea. However, significantly less attention has been paid to midbrain and brainstem structures in this regard. Here, we find that subcortical auditory neurons represent stimulus information sufficient for detection and predict behavioral choice on a trial-by-trial basis.

Behaviorally relevant decision coding in primary somatosensory cortex neurons

Primary sensory cortex is thought to process incoming sensory information, while decision variables important for driving behavior are assumed to arise downstream in the processing hierarchy. Here, we used population two-photon calcium imaging and targeted two-photon optogenetic stimulation of neurons in layer 2/3 of mouse primary somatosensory cortex (S1) during a texture discrimination task to test for the presence of decision signals and probe their behavioral relevance. Small but distinct populations of neurons carried information about the stimulus irrespective of the behavioral outcome (stimulus neurons), or about the choice irrespective of the presented stimulus (decision neurons). Decision neurons show categorical coding that develops during learning, and lack a conclusive decision signal in Miss trials. All-optical photostimulation of decision neurons during behavior improves behavioral performance, establishing a causal role in driving behavior. The fact that stimulus and decision neurons are intermingled challenges the idea of S1 as a purely sensory area, and causal perturbation suggests a direct involvement of S1 decision neurons in the decision-making process.

A conflict between spatial selection and evidence accumulation in area LIP

The lateral intraparietal area (LIP) contains spatially selective neurons that help guide eye movements and, according to numerous studies, do so by accumulating sensory evidence in favor of one choice (e.g., look left) or another (look right). To examine this functional link, we trained two monkeys on an urgent motion discrimination task, a task with which the evolution of both the recorded neuronal activity and the subject's choice can be tracked millisecond by millisecond. We found that while choice accuracy increased steeply with increasing sensory evidence, at the same time, the LIP selection signal became progressively weaker, as if it hindered performance. This effect was consistent with the transient deployment of spatial attention to disparate locations away from the relevant sensory cue. The results demonstrate that spatial selection in LIP is dissociable from, and may even conflict with, evidence accumulation during informed saccadic choices.

Perceptual Weighting of V1 Spikes Revealed by Optogenetic White Noise Stimulation

During visually guided behaviors, mere hundreds of milliseconds can elapse between a sensory input and its associated behavioral response. How spikes occurring at different times are integrated to drive perception and action remains poorly understood. We delivered random trains of optogenetic stimulation (white noise) to excite inhibitory interneurons in V1 of mice of both sexes while they performed a visual detection task. We then performed a reverse correlation analysis on the optogenetic stimuli to generate a neuronal-behavioral kernel, an unbiased, temporally precise estimate of how suppression of V1 spiking at different moments around the onset of a visual stimulus affects detection of that stimulus. Electrophysiological recordings enabled us to capture the effects of optogenetic stimuli on V1 responsivity and revealed that the earliest stimulus-evoked spikes are preferentially weighted for guiding behavior. These data demonstrate that white noise optogenetic stimulation is a powerful tool for understanding how patterns of spiking in neuronal populations are decoded in generating perception and action.SIGNIFICANCE STATEMENT During visually guided actions, continuous chains of neurons connect our retinas to our motoneurons. To unravel circuit contributions to behavior, it is crucial to establish the relative functional position(s) that different neural structures occupy in processing and relaying the signals that support rapid, precise responses. To address this question, we randomly inhibited activity in mouse V1 throughout the stimulus-response cycle while the animals did many repetitions of a visual task. The period that led to impaired performance corresponded to the earliest stimulus-driven response in V1, with no effect of inhibition immediately before or during late stages of the stimulus-driven response. This approach offers experimenters a powerful method for uncovering the temporal weighting of spikes from stimulus to response.

Equivalent noise characterization of human lightness constancy

A goal of visual perception is to provide stable representations of task-relevant scene properties (e.g. object reflectance) despite variation in task-irrelevant scene properties (e.g. illumination and reflectance of other nearby objects). To study such stability in the context of the perceptual representation of lightness, we introduce a threshold-based psychophysical paradigm. We measure how thresholds for discriminating the achromatic reflectance of a target object (task-relevant property) in rendered naturalistic scenes are impacted by variation in the reflectance functions of background objects (task-irrelevant property), using a two-alternative forced-choice paradigm in which the reflectance of the background objects is randomized across the two intervals of each trial. We control the amount of background reflectance variation by manipulating a statistical model of naturally occurring surface reflectances. For low background object reflectance variation, discrimination thresholds were nearly constant, indicating that observers' internal noise determines threshold in this regime. As background object reflectance variation increases, its effects start to dominate performance. A model based on signal detection theory allows us to express the effects of task-irrelevant variation in terms of the equivalent noise, that is relative to the intrinsic precision of the task-relevant perceptual representation. The results indicate that although naturally occurring background object reflectance variation does intrude on the perceptual representation of target object lightness, the effect is modest - within a factor of two of the equivalent noise level set by internal noise.

Beyond dimension reduction: Stable electric fields emerge from and allow representational drift

It is known that the exact neurons maintaining a given memory (the neural ensemble) change from trial to trial. This raises the question of how the brain achieves stability in the face of this representational drift. Here, we demonstrate that this stability emerges at the level of the electric fields that arise from neural activity. We show that electric fields carry information about working memory content. The electric fields, in turn, can act as "guard rails" that funnel higher dimensional variable neural activity along stable lower dimensional routes. We obtained the latent space associated with each memory. We then confirmed the stability of the electric field by mapping the latent space to different cortical patches (that comprise a neural ensemble) and reconstructing information flow between patches. Stable electric fields can allow latent states to be transferred between brain areas, in accord with modern engram theory.

Task-induced neural covariability as a signature of approximate Bayesian learning and inference

Perception is often characterized computationally as an inference process in which uncertain or ambiguous sensory inputs are combined with prior expectations. Although behavioral studies have shown that observers can change their prior expectations in the context of a task, robust neural signatures of task-specific priors have been elusive. Here, we analytically derive such signatures under the general assumption that the responses of sensory neurons encode posterior beliefs that combine sensory inputs with task-specific expectations. Specifically, we derive predictions for the task-dependence of correlated neural variability and decision-related signals in sensory neurons. The qualitative aspects of our results are parameter-free and specific to the statistics of each task. The predictions for correlated variability also differ from predictions of classic feedforward models of sensory processing and are therefore a strong test of theories of hierarchical Bayesian inference in the brain. Importantly, we find that Bayesian learning predicts an increase in so-called “differential correlations” as the observer’s internal model learns the stimulus distribution, and the observer’s behavioral performance improves. This stands in contrast to classic feedforward encoding/decoding models of sensory processing, since such correlations are fundamentally information-limiting. We find support for our predictions in data from existing neurophysiological studies across a variety of tasks and brain areas. Finally, we show in simulation how measurements of sensory neural responses can reveal information about a subject’s internal beliefs about the task. Taken together, our results reinterpret task-dependent sources of neural covariability as signatures of Bayesian inference and provide new insights into their cause and their function.

Perceptual decision-making has classically been studied in the context of feedforward encoding/ decoding models. Here, we derive predictions for the responses of sensory neurons under the assumption that the brain performs hierarchical Bayesian inference, including feedback signals that communicate task-specific prior expectations. Interestingly, those predictions stand in contrast to some of the conclusions drawn in the classic framework. In particular, we find that Bayesian learning predicts the increase of a type of correlated variability called “differential correlations” over the course of learning. Differential correlations limit information, and hence are seen as harmful in feedforward models. Since our results are also specific to the statistics of a given task, and since they hold under a wide class of theories about how Bayesian probabilities may be represented by neural responses, they constitute a strong test of the Bayesian Brain hypothesis. Our results can explain the task-dependence of correlated variability in prior studies and suggest a reason why these kinds of correlations are surprisingly common in empirical data. Interpreted in a probabilistic framework, correlated variability provides a window into an observer’s task-related beliefs.

Neuronal modulation in the mouse superior colliculus during covert visual selective attention

Covert visual attention is accomplished by a cascade of mechanisms distributed across multiple brain regions. Visual cortex is associated with enhanced representations of relevant stimulus features, whereas the contributions of subcortical circuits are less well understood but have been associated with selection of relevant spatial locations and suppression of distracting stimuli. As a step toward understanding these subcortical circuits, here we identified how neuronal activity in the intermediate layers of the superior colliculus (SC) of head-fixed mice is modulated during covert visual attention. We found that spatial cues modulated both firing rate and spike-count correlations. Crucially, the cue-related modulation in firing rate was due to enhancement of activity at the cued spatial location rather than suppression at the uncued location, indicating that SC neurons in our task were modulated by an excitatory or disinhibitory circuit mechanism focused on the relevant location, rather than broad inhibition of irrelevant locations. This modulation improved the neuronal discriminability of visual-change-evoked activity, but only when assessed for neuronal activity between the contralateral and ipsilateral SC. Together, our findings indicate that neurons in the mouse SC can contribute to covert visual selective attention by biasing processing in favor of locations expected to contain task-relevant information.

Neural activity in the mouse claustrum in a cross-modal sensory selection task

The claustrum, a subcortical nucleus forming extensive connections with the neocortex, has been implicated in sensory selection. Sensory-evoked claustrum activity is thought to modulate the neocortex's context-dependent response to sensory input. Recording from claustrum neurons while mice performed a tactile-visual sensory-selection task, we found that neurons in the anterior claustrum, including putative optotagged claustrocortical neurons projecting to the primary somatosensory cortex (S1), were rarely modulated by sensory input. Rather, they exhibited different types of direction-tuned motor responses. Furthermore, we found that claustrum neurons encoded upcoming movement during intertrial intervals and that pairs of claustrum neurons exhibiting synchronous firing were enriched for pairs preferring contralateral lick directions, suggesting that the activity of specific ensembles of similarly tuned claustrum neurons may modulate cortical activity. Chemogenetic inhibition of claustrocortical neurons decreased lick responses to inappropriate sensory stimuli. Altogether, our data indicate that the claustrum is integrated into higher-order premotor circuits recently implicated in decision-making.

Attention improves information flow between neuronal populations without changing the communication subspace

Visual attention allows observers to change the influence of different parts of a visual scene on their behavior, suggesting that information can be flexibly shared between visual cortex and neurons involved in decision making. We investigated the neural substrate of flexible information routing by analyzing the activity of populations of visual neurons in the medial temporal area (MT) and oculo-motor neurons in the superior colliculus (SC) while rhesus monkeys switched spatial attention. We demonstrated that attention increases the efficacy of visuomotor communication: trial-to-trial variability in SC population activity could be better predicted by the activity of the MT population (and vice versa) when attention was directed toward their joint receptive fields. Surprisingly, this improvement in prediction was not explained by changes in the dimensionality of the shared subspace or in the magnitude of local or shared pairwise noise correlations. These results lay a foundation for future theoretical and experimental studies into how visual attention can flexibly change information flow between sensory and decision neurons.

Joint representation of working memory and uncertainty in human cortex

Neural representations of visual working memory (VWM) are noisy, and thus, decisions based on VWM are inevitably subject to uncertainty. However, the mechanisms by which the brain simultaneously represents the content and uncertainty of memory remain largely unknown. Here, inspired by the theory of probabilistic population codes, we test the hypothesis that the human brain represents an item maintained in VWM as a probability distribution over stimulus feature space, thereby capturing both its content and uncertainty. We used a neural generative model to decode probability distributions over memorized locations from fMRI activation patterns. We found that the mean of the probability distribution decoded from retinotopic cortical areas predicted memory reports on a trial-by-trial basis. Moreover, in several of the same mid-dorsal stream areas, the spread of the distribution predicted subjective trial-by-trial uncertainty judgments. These results provide evidence that VWM content and uncertainty are jointly represented by probabilistic neural codes.

Modulation of Spike Count Correlations Between Macaque Primary Visual Cortex Neurons by Difficulty of Attentional Task

Studies have shown that spatial attention remarkably affects the trial-to-trial response variability shared between neurons. Difficulty in the attentional task adjusts how much concentration we maintain on what is currently important and what is filtered as irrelevant sensory information. However, how task difficulty mediates the interactions between neurons with separated receptive fields (RFs) that are attended to or attended away is still not clear. We examined spike count correlations between single-unit activities recorded simultaneously in the primary visual cortex (V1) while monkeys performed a spatial attention task with two levels of difficulty. Moreover, the RFs of the two neurons recorded were non-overlapping to allow us to study fluctuations in the correlated responses between competing visual inputs when the focus of attention was allocated to the RF of one neuron. While increasing difficulty in the spatial attention task, spike count correlations were either decreased to become negative between neuronal pairs, implying competition among them, with one neuron (or none) exhibiting attentional enhancement of firing rate, or increased to become positive, suggesting inter-neuronal cooperation, with one of the pair showing attentional suppression of spiking responses. Besides, the modulation of spike count correlations by task difficulty was independent of the attended locations. These findings provide evidence that task difficulty affects the functional interactions between different neuronal pools in V1 when selective attention resolves the spatial competition.

A confirmation bias in perceptual decision-making due to hierarchical approximate inference

Making good decisions requires updating beliefs according to new evidence. This is a dynamical process that is prone to biases: in some cases, beliefs become entrenched and resistant to new evidence (leading to primacy effects), while in other cases, beliefs fade over time and rely primarily on later evidence (leading to recency effects). How and why either type of bias dominates in a given context is an important open question. Here, we study this question in classic perceptual decision-making tasks, where, puzzlingly, previous empirical studies differ in the kinds of biases they observe, ranging from primacy to recency, despite seemingly equivalent tasks. We present a new model, based on hierarchical approximate inference and derived from normative principles, that not only explains both primacy and recency effects in existing studies, but also predicts how the type of bias should depend on the statistics of stimuli in a given task. We verify this prediction in a novel visual discrimination task with human observers, finding that each observer's temporal bias changed as the result of changing the key stimulus statistics identified by our model. The key dynamic that leads to a primacy bias in our model is an overweighting of new sensory information that agrees with the observer's existing belief-a type of 'confirmation bias'. By fitting an extended drift-diffusion model to our data we rule out an alternative explanation for primacy effects due to bounded integration. Taken together, our results resolve a major discrepancy among existing perceptual decision-making studies, and suggest that a key source of bias in human decision-making is approximate hierarchical inference.

Reliable Sensory Processing in Mouse Visual Cortex through Cooperative Interactions between Somatostatin and Parvalbumin Interneurons

Intrinsic neuronal variability significantly limits information encoding in the primary visual cortex (V1). However, under certain conditions, neurons can respond reliably with highly precise responses to the same visual stimuli from trial to trial. This suggests that there exists intrinsic neural circuit mechanisms that dynamically modulate the intertrial variability of visual cortical neurons. Here, we sought to elucidate the role of different inhibitory interneurons (INs) in reliable coding in mouse V1. To study the interactions between somatostatin-expressing interneurons (SST-INs) and parvalbumin-expressing interneurons (PV-INs), we used a dual-color calcium imaging technique that allowed us to simultaneously monitor these two neural ensembles while awake mice, of both sexes, passively viewed natural movies. SST neurons were more active during epochs of reliable pyramidal neuron firing, whereas PV neurons were more active during epochs of unreliable firing. SST neuron activity lagged that of PV neurons, consistent with a feedback inhibitory SST→PV circuit. To dissect the role of this circuit in pyramidal neuron activity, we used temporally limited optogenetic activation and inactivation of SST and PV interneurons during periods of reliable and unreliable pyramidal cell firing. Transient firing of SST neurons increased pyramidal neuron reliability by actively suppressing PV neurons, a proposal that was supported by a rate-based model of V1 neurons. These results identify a cooperative functional role for the SST→PV circuit in modulating the reliability of pyramidal neuron activity.SIGNIFICANCE STATEMENT Cortical neurons often respond to identical sensory stimuli with large variability. However, under certain conditions, the same neurons can also respond highly reliably. The circuit mechanisms that contribute to this modulation remain unknown. Here, we used novel dual-wavelength calcium imaging and temporally selective optical perturbation to identify an inhibitory neural circuit in visual cortex that can modulate the reliability of pyramidal neurons to naturalistic visual stimuli. Our results, supported by computational models, suggest that somatostatin interneurons increase pyramidal neuron reliability by suppressing parvalbumin interneurons via the inhibitory SST→PV circuit. These findings reveal a novel role of the SST→PV circuit in modulating the fidelity of neural coding critical for visual perception.

Bridging neuronal correlations and dimensionality reduction

Two commonly used approaches to study interactions among neurons are spike count correlation, which describes pairs of neurons, and dimensionality reduction, applied to a population of neurons. Although both approaches have been used to study trial-to-trial neuronal variability correlated among neurons, they are often used in isolation and have not been directly related. We first established concrete mathematical and empirical relationships between pairwise correlation and metrics of population-wide covariability based on dimensionality reduction. Applying these insights to macaque V4 population recordings, we found that the previously reported decrease in mean pairwise correlation associated with attention stemmed from three distinct changes in population-wide covariability. Overall, our work builds the intuition and formalism to bridge between pairwise correlation and population-wide covariability and presents a cautionary tale about the inferences one can make about population activity by using a single statistic, whether it be mean pairwise correlation or dimensionality.

Decision-related feedback in visual cortex lacks spatial selectivity

Feedback in the brain is thought to convey contextual information that underlies our flexibility to perform different tasks. Empirical and computational work on the visual system suggests this is achieved by targeting task-relevant neuronal subpopulations. We combine two tasks, each resulting in selective modulation by feedback, to test whether the feedback reflected the combination of both selectivities. We used visual feature-discrimination specified at one of two possible locations and uncoupled the decision formation from motor plans to report it, while recording in macaque mid-level visual areas. Here we show that although the behavior is spatially selective, using only task-relevant information, modulation by decision-related feedback is spatially unselective. Population responses reveal similar stimulus-choice alignments irrespective of stimulus relevance. The results suggest a common mechanism across tasks, independent of the spatial selectivity these tasks demand. This may reflect biological constraints and facilitate generalization across tasks. Our findings also support a previously hypothesized link between feature-based attention and decision-related activity.

Feedback modulates visual neurons, thought to help achieve flexible task performance. Here, the authors show decision-related feedback is not only relayed to task-relevant neurons, suggesting a broader mechanism and supporting a previously hypothesized link to feature-based attention.

Correlated structure of neuronal firing in macaque visual cortex limits information for binocular depth discrimination

Variability in cortical neural activity potentially limits sensory discriminations. Theoretical work shows that information required to discriminate two similar stimuli is limited by the correlation structure of cortical variability. We investigated these information-limiting correlations by recording simultaneously from visual cortical areas primary visual cortex (V1) and extrastriate area V4 in macaque monkeys performing a binocular, stereo depth discrimination task. Within both areas, noise correlations on a rapid temporal scale (20–30 ms) were stronger for neuron pairs with similar selectivity for binocular depth, meaning that these correlations potentially limit information for making the discrimination. Between-area correlations (V1 to V4) were different, being weaker for neuron pairs with similar tuning and having a slower temporal scale (100+ ms). Fluctuations in these information-limiting correlations just prior to the detection event were associated with changes in behavioral accuracy. Although these correlations limit the recovery of information about sensory targets, their impact may be curtailed by integrative processing of signals across multiple brain areas.

NEW & NOTEWORTHY Correlated noise reduces the stimulus information in visual cortical neurons during experimental performance of binocular depth discriminations. The temporal scale of these correlations is important. Rapid (20–30 ms) correlations reduce information within and between areas V1 and V4, whereas slow (>100 ms) correlations between areas do not. Separate cortical areas appear to act together to maintain signal fidelity. Rapid correlations reduce the neuronal signal difference between stimuli and adversely affect perceptual discrimination.

Stimulus-dependent relationships between behavioral choice and sensory neural responses

Understanding perceptual decision-making requires linking sensory neural responses to behavioral choices. In two-choice tasks, activity-choice covariations are commonly quantified with a single measure of choice probability (CP), without characterizing their changes across stimulus levels. We provide theoretical conditions for stimulus dependencies of activity-choice covariations. Assuming a general decision-threshold model, which comprises both feedforward and feedback processing and allows for a stimulus-modulated neural population covariance, we analytically predict a very general and previously unreported stimulus dependence of CPs. We develop new tools, including refined analyses of CPs and generalized linear models with stimulus-choice interactions, which accurately assess the stimulus- or choice-driven signals of each neuron, characterizing stimulus-dependent patterns of choice-related signals. With these tools, we analyze CPs of macaque MT neurons during a motion discrimination task. Our analysis provides preliminary empirical evidence for the promise of studying stimulus dependencies of choice-related signals, encouraging further assessment in wider data sets.

Decision Signals in the Local Field Potentials of Early and Mid-Level Macaque Visual Cortex

The neural basis of perceptual decision making has typically been studied using measurements of single neuron activity, though decisions are likely based on the activity of large neuronal ensembles. Local field potentials (LFPs) may, in some cases, serve as a useful proxy for population activity and thus be useful for understanding the neural basis of perceptual decision making. However, little is known about whether LFPs in sensory areas include decision-related signals. We therefore analyzed LFPs recorded using two 48-electrode arrays implanted in primary visual cortex (V1) and area V4 of macaque monkeys trained to perform a fine orientation discrimination task. We found significant choice information in low (0-30 Hz) and higher (70-500 Hz) frequency components of the LFP, but little information in gamma frequencies (30-70 Hz). Choice information was more robust in V4 than V1 and stronger in LFPs than in simultaneously measured spiking activity. LFP-based choice information included a global component, common across electrodes within an area. Our findings reveal the presence of robust choice-related signals in the LFPs recorded in V1 and V4 and suggest that LFPs may be a useful complement to spike-based analyses of decision making.

Visual Neuroscience: What to Do with All of These Cortical Visual Areas?

Visual information is represented across multiple areas in the mouse visual cortex. A new study has revealed that some higher visual areas are important for seeing even simple visual features, whereas other areas have more complex effects on visual decisions.

Amplitude modulations of cortical sensory responses in pulsatile evidence accumulation

How does the brain internally represent a sequence of sensory information that jointly drives a decision-making behavior? Studies of perceptual decision-making have often assumed that sensory cortices provide noisy but otherwise veridical sensory inputs to downstream processes that accumulate and drive decisions. However, sensory processing in even the earliest sensory cortices can be systematically modified by various external and internal contexts. We recorded from neuronal populations across posterior cortex as mice performed a navigational decision-making task based on accumulating randomly timed pulses of visual evidence. Even in V1, only a small fraction of active neurons had sensory-like responses time-locked to each pulse. Here, we focus on how these 'cue-locked' neurons exhibited a variety of amplitude modulations from sensory to cognitive, notably by choice and accumulated evidence. These task-related modulations affected a large fraction of cue-locked neurons across posterior cortex, suggesting that future models of behavior should account for such influences.

Pairwise Synchrony and Correlations Depend on the Structure of the Population Code in Visual Cortex

In visual areas of primates, neurons activate in parallel while the animal is engaged in a behavioral task. In this study, we examine the structure of the population code while the animal performs delayed match-to-sample tasks on complex natural images. The macaque monkeys visualized two consecutive stimuli that were either the same or different, while being recorded with laminar arrays across the cortical depth in cortical areas V1 and V4. We decode correct choice behavior from neural populations of simultaneously recorded units. Utilizing decoding weights, we divide neurons into most informative and less informative and show that most informative neurons in V4, but not in V1, are more strongly synchronized, coupled, and correlated than less informative neurons. Because neurons are divided into two coding pools according to their coding preference, in V4, but not in V1, spiking synchrony, coupling, and correlations within the coding pool are stronger than across coding pools.

Coding Perceptual Decisions: From Single Units to Emergent Signaling Properties in Cortical Circuits

Spiking activity in single neurons of the primate visual cortex has been tightly linked to perceptual decisions. Any mechanism that reads out these perceptual signals to support behavior must respect the underlying neuroanatomy that shapes the functional properties of sensory neurons. Spatial distribution and timing of inputs to the next processing levels are critical, as conjoint activity of precursor neurons increases the spiking rate of downstream neurons and ultimately drives behavior. I set out how correlated activity might coalesce into a micropool of task-sensitive neurons signaling a particular percept to determine perceptual decision signals locally and for flexible interarea transmission depending on the task context. As data from more and more neurons and their complex interactions are analyzed, the space of computational mechanisms must be constrained based on what is plausible within neurobiological limits. This review outlines experiments to test the new perspectives offered by these extended methods.

The Correlation of Neuronal Signals with Behavior at Different Levels of Visual Cortex and Their Relative Reliability for Behavioral Decisions

Behavior can be guided by neuronal activity in visual, auditory, or somatosensory cerebral cortex, depending on task requirements. In contrast to this flexible access of cortical signals, several observations suggest that behaviors depend more on neurons in later areas of visual cortex than those in earlier areas, although neurons in earlier areas would provide more reliable signals for many tasks. We recorded from neurons in different levels of visual cortex of 2 male rhesus monkeys while the animals did a visual discrimination task and examined trial-to-trial correlations between neuronal and behavioral responses. These correlations became stronger in primary visual cortex as neuronal signals in that area became more reliable relative to the other areas. The results suggest that the mechanisms that read signals from cortex might access any cortical area depending on the relative value of those signals for the task at hand.SIGNIFICANCE STATEMENT Information is encoded by the action potentials of neurons in various cortical areas in a hierarchical manner such that increasingly complex stimulus features are encoded in successive stages. The brain must extract information from the response of appropriate neurons to drive optimal behavior. A widely held view of this decoding process is that the brain relies on the output of later cortical areas to make decisions, although neurons in earlier areas can provide more reliable signals. We examined correlations between perceptual decisions and the responses of neurons in different levels of monkey visual cortex. The results suggest that the brain may access signals in any cortical area depending on the relative value of those signals for the task at hand.

Cortical State Fluctuations across Layers of V1 during Visual Spatial Perception

Many factors modulate the state of cortical activity, but the importance of cortical state variability for sensory perception remains debated. We trained mice to detect spatially localized visual stimuli and simultaneously measured local field potentials and excitatory and inhibitory neuron populations across layers of primary visual cortex (V1). Cortical states with low spontaneous firing and correlations in excitatory neurons, and suppression of 3- to 7-Hz oscillations in layer 4, accurately predicted single-trial visual detection. Our results show that cortical states exert strong effects at the initial stage of cortical processing in V1 and can play a prominent role for visual spatial behavior in mice.

The cognitive penetrability of perception: A blocked debate and a tentative solution

Despite the extensive body of psychological findings suggesting that cognition influences perception, the debate between defenders and detractors of the cognitive penetrability of perception persists. While detractors demand more strictness in psychological experiments, proponents consider that empirical studies show that cognitive penetrability occurs. These considerations have led some theorists to propose that the debate has reached a dead end. The issue about where perception ends and cognition begins is, I argue, one of the reasons why the debate is cornered. Another reason is the inability of psychological studies to present uncontroversial interpretations of the results obtained. To dive into other kinds of empirical sources is, therefore, required to clarify the debate. In this paper, I explain where the debate is blocked, and suggest that neuroscientific evidence together with the predictive coding account, might decant the discussion on the side of the penetrability thesis.

Brainstem Modulation of Large-Scale Intrinsic Cortical Activity Correlations

Brain activity fluctuates continuously, even in the absence of changes in sensory input or motor output. These intrinsic activity fluctuations are correlated across brain regions and are spatially organized in macroscale networks. Variations in the strength, topography, and topology of correlated activity occur over time, and unfold upon a backbone of long-range anatomical connections. Subcortical neuromodulatory systems send widespread ascending projections to the cortex, and are thus ideally situated to shape the temporal and spatial structure of intrinsic correlations. These systems are also the targets of the pharmacological treatment of major neurological and psychiatric disorders, such as Parkinson's disease, depression, and schizophrenia. Here, we review recent work that has investigated how neuromodulatory systems shape correlations of intrinsic fluctuations of large-scale cortical activity. We discuss studies in the human, monkey, and rodent brain, with a focus on non-invasive recordings of human brain activity. We provide a structured but selective overview of this work and distil a number of emerging principles. Future efforts to chart the effect of specific neuromodulators and, in particular, specific receptors, on intrinsic correlations may help identify shared or antagonistic principles between different neuromodulatory systems. Such principles can inform models of healthy brain function and may provide an important reference for understanding altered cortical dynamics that are evident in neurological and psychiatric disorders, potentially paving the way for mechanistically inspired biomarkers and individualized treatments of these disorders.

Choice (-history) correlations in sensory cortex: cause or consequence?

One challenge in neuroscience, as in other areas of science, is to make inferences about the underlying causal structure from correlational data. Here, we discuss this challenge in the context of choice correlations in sensory neurons, that is, trial-by-trial correlations, unexplained by the stimulus, between the activity of sensory neurons and an animal's perceptual choice. Do these choice-correlations reflect feedforward, feedback signalling, both, or neither? We highlight recent results of correlational and causal examinations of choice and choice-history signals in sensory, and in part sensorimotor, cortex and address formal statistical frameworks to infer causal interactions from data.

Data-Driven Approaches to Understanding Visual Neuron Activity

With modern neurophysiological methods able to record neural activity throughout the visual pathway in the context of arbitrarily complex visual stimulation, our understanding of visual system function is becoming limited by the available models of visual neurons that can be directly related to such data. Different forms of statistical models are now being used to probe the cellular and circuit mechanisms shaping neural activity, understand how neural selectivity to complex visual features is computed, and derive the ways in which neurons contribute to systems-level visual processing. However, models that are able to more accurately reproduce observed neural activity often defy simple interpretations. As a result, rather than being used solely to connect with existing theories of visual processing, statistical modeling will increasingly drive the evolution of more sophisticated theories.