1
|
Gardères PM, Le Gal S, Rousseau C, Mamane A, Ganea DA, Haiss F. Coexistence of state, choice, and sensory integration coding in barrel cortex LII/III. Nat Commun 2024; 15:4782. [PMID: 38839747 PMCID: PMC11153558 DOI: 10.1038/s41467-024-49129-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2023] [Accepted: 05/23/2024] [Indexed: 06/07/2024] Open
Abstract
During perceptually guided decisions, correlates of choice are found as upstream as in the primary sensory areas. However, how well these choice signals align with early sensory representations, a prerequisite for their interpretation as feedforward substrates of perception, remains an open question. We designed a two alternative forced choice task (2AFC) in which male mice compared stimulation frequencies applied to two adjacent vibrissae. The optogenetic silencing of individual columns in the primary somatosensory cortex (wS1) resulted in predicted shifts of psychometric functions, demonstrating that perception depends on focal, early sensory representations. Functional imaging of layer II/III single neurons revealed mixed coding of stimuli, choices and engagement in the task. Neurons with multi-whisker suppression display improved sensory discrimination and had their activity increased during engagement in the task, enhancing selectively representation of the signals relevant to solving the task. From trial to trial, representation of stimuli and choice varied substantially, but mostly orthogonally to each other, suggesting that perceptual variability does not originate from wS1 fluctuations but rather from downstream areas. Together, our results highlight the role of primary sensory areas in forming a reliable sensory substrate that could be used for flexible downstream decision processes.
Collapse
Affiliation(s)
- Pierre-Marie Gardères
- Institut Pasteur, Université Paris Cité, Unit of Neural Circuits Dynamics and Decision Making, F-75015, Paris, France.
- IZKF Aachen, Medical School, RWTH Aachen University, 52074, Aachen, Germany.
| | - Sébastien Le Gal
- Institut Pasteur, Université Paris Cité, Unit of Neural Circuits Dynamics and Decision Making, F-75015, Paris, France
| | - Charly Rousseau
- Institut Pasteur, Université Paris Cité, Unit of Neural Circuits Dynamics and Decision Making, F-75015, Paris, France
| | - Alexandre Mamane
- Institut Pasteur, Université Paris Cité, Unit of Neural Circuits Dynamics and Decision Making, F-75015, Paris, France
| | - Dan Alin Ganea
- IZKF Aachen, Medical School, RWTH Aachen University, 52074, Aachen, Germany
- University of Basel, Department of Biomedicine, 4001, Basel, Switzerland
| | - Florent Haiss
- Institut Pasteur, Université Paris Cité, Unit of Neural Circuits Dynamics and Decision Making, F-75015, Paris, France.
| |
Collapse
|
2
|
Kramer LE, Chen YC, Long B, Konkle T, Cohen MR. Contributions of early and mid-level visual cortex to high-level object categorization. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.31.541514. [PMID: 37398251 PMCID: PMC10312552 DOI: 10.1101/2023.05.31.541514] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/04/2023]
Abstract
The complexity of visual features for which neurons are tuned increases from early to late stages of the ventral visual stream. Thus, the standard hypothesis is that high-level functions like object categorization are primarily mediated by higher visual areas because they require more complex image formats that are not evident in early visual processing stages. However, human observers can categorize images as objects or animals or as big or small even when the images preserve only some low- and mid-level features but are rendered unidentifiable ('texforms', Long et al., 2018). This observation suggests that even the early visual cortex, in which neurons respond to simple stimulus features, may already encode signals about these more abstract high-level categorical distinctions. We tested this hypothesis by recording from populations of neurons in early and mid-level visual cortical areas while rhesus monkeys viewed texforms and their unaltered source stimuli (simultaneous recordings from areas V1 and V4 in one animal and separate recordings from V1 and V4 in two others). Using recordings from a few dozen neurons, we could decode the real-world size and animacy of both unaltered images and texforms. Furthermore, this neural decoding accuracy across stimuli was related to the ability of human observers to categorize texforms by real-world size and animacy. Our results demonstrate that neuronal populations early in the visual hierarchy contain signals useful for higher-level object perception and suggest that the responses of early visual areas to simple stimulus features display preliminary untangling of higher-level distinctions.
Collapse
Affiliation(s)
| | | | - Bria Long
- University of California, Los Angeles
| | | | | |
Collapse
|
3
|
Lakshminarasimhan KJ, Avila E, Pitkow X, Angelaki DE. Dynamical latent state computation in the male macaque posterior parietal cortex. Nat Commun 2023; 14:1832. [PMID: 37005470 PMCID: PMC10067966 DOI: 10.1038/s41467-023-37400-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2022] [Accepted: 03/15/2023] [Indexed: 04/04/2023] Open
Abstract
Success in many real-world tasks depends on our ability to dynamically track hidden states of the world. We hypothesized that neural populations estimate these states by processing sensory history through recurrent interactions which reflect the internal model of the world. To test this, we recorded brain activity in posterior parietal cortex (PPC) of monkeys navigating by optic flow to a hidden target location within a virtual environment, without explicit position cues. In addition to sequential neural dynamics and strong interneuronal interactions, we found that the hidden state - monkey's displacement from the goal - was encoded in single neurons, and could be dynamically decoded from population activity. The decoded estimates predicted navigation performance on individual trials. Task manipulations that perturbed the world model induced substantial changes in neural interactions, and modified the neural representation of the hidden state, while representations of sensory and motor variables remained stable. The findings were recapitulated by a task-optimized recurrent neural network model, suggesting that task demands shape the neural interactions in PPC, leading them to embody a world model that consolidates information and tracks task-relevant hidden states.
Collapse
Affiliation(s)
| | - Eric Avila
- Center for Neural Science, New York University, New York City, NY, USA
| | - Xaq Pitkow
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA
- Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, USA
- Electrical & Computer Engineering, Rice University, Houston, TX, USA
| | - Dora E Angelaki
- Center for Neural Science, New York University, New York City, NY, USA
- Department of Mechanical and Aerospace Engineering, New York University, New York City, NY, USA
| |
Collapse
|
4
|
Orlandi JG, Abdolrahmani M, Aoki R, Lyamzin DR, Benucci A. Distributed context-dependent choice information in mouse posterior cortex. Nat Commun 2023; 14:192. [PMID: 36635318 PMCID: PMC9837177 DOI: 10.1038/s41467-023-35824-6] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Accepted: 01/03/2023] [Indexed: 01/14/2023] Open
Abstract
Choice information appears in multi-area brain networks mixed with sensory, motor, and cognitive variables. In the posterior cortex-traditionally implicated in decision computations-the presence, strength, and area specificity of choice signals are highly variable, limiting a cohesive understanding of their computational significance. Examining the mesoscale activity in the mouse posterior cortex during a visual task, we found that choice signals defined a decision variable in a low-dimensional embedding space with a prominent contribution along the ventral visual stream. Their subspace was near-orthogonal to concurrently represented sensory and motor-related activations, with modulations by task difficulty and by the animals' attention state. A recurrent neural network trained with animals' choices revealed an equivalent decision variable whose context-dependent dynamics agreed with that of the neural data. Our results demonstrated an independent, multi-area decision variable in the posterior cortex, controlled by task features and cognitive demands, possibly linked to contextual inference computations in dynamic animal-environment interactions.
Collapse
Affiliation(s)
- Javier G Orlandi
- RIKEN Center for Brain Science, 2-1 Hirosawa, Wako-shi, Saitama, 351-0198, Japan.,Department of Physics and Astronomy, University of Calgary, Calgary, Alberta, T2N 1N4, Canada
| | | | - Ryo Aoki
- RIKEN Center for Brain Science, 2-1 Hirosawa, Wako-shi, Saitama, 351-0198, Japan
| | - Dmitry R Lyamzin
- RIKEN Center for Brain Science, 2-1 Hirosawa, Wako-shi, Saitama, 351-0198, Japan
| | - Andrea Benucci
- RIKEN Center for Brain Science, 2-1 Hirosawa, Wako-shi, Saitama, 351-0198, Japan. .,University of Tokyo, Graduate School of Information Science and Technology, Department of Mathematical Informatics, 1-1-1 Yayoi, Bunkyo City, Tokyo, 113-0032, Japan.
| |
Collapse
|
5
|
Niemeyer JE, Akers-Campbell S, Gregoire A, Paradiso MA. Perceptual enhancement and suppression correlate with V1 neural activity during active sensing. Curr Biol 2022; 32:2654-2667.e4. [PMID: 35584697 PMCID: PMC9233080 DOI: 10.1016/j.cub.2022.04.067] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2021] [Revised: 03/03/2022] [Accepted: 04/22/2022] [Indexed: 11/26/2022]
Abstract
Perception in multiple sensory modalities is an active process that involves exploratory behaviors. In humans and other primates, vision results from sensory sampling guided by saccadic eye movements. Saccades are known to modulate visual perception, and a corollary discharge signal associated with saccades appears to establish a sense of visual stability. Neural recordings have shown that saccades also modulate activity widely across the brain. To investigate the neural basis of saccadic effects on perception, simultaneous recordings from multiple neurons in area V1 were made as animals performed a contrast detection task. Perceptual and neural measures were compared when the animal made real saccades that brought a stimulus into V1 receptive fields and when simulated saccades were made (identical retinal stimulation but no eye movement). When real saccades were made and low spatial frequency stimuli were presented, we observed a reduction in both perceptual sensitivity and neural activity compared with simulated saccades; conversely, with higher spatial frequency stimuli, saccades increased visual sensitivity and neural activity. The performance of neural decoders, which used the activity of the population of simultaneously recorded neurons, showed saccade effects on sensitivity that mirrored the frequency-dependent perceptual changes, suggesting that the V1 population activity could support the perceptual effects. A minority of V1 neurons had significant choice probabilities, and the saccades decreased both average choice probability and pairwise noise correlations. Taken together, the findings suggest that a signal related to saccadic eye movements alters V1 spiking to increase the independence of spiking neurons and bias the system toward processing higher spatial frequencies, presumably to enhance object recognition. The effects of saccades on visual perception and noise correlations appear to parallel effects observed in other sensory modalities, suggesting a general principle of active sensory processing.
Collapse
Affiliation(s)
- James E Niemeyer
- Department of Neurological Surgery, Weill Cornell Medicine, New York, NY, USA; Department of Neuroscience, Brown University, Providence, RI, USA
| | | | - Aaron Gregoire
- School of Engineering, Brown University, Providence, RI, USA
| | | |
Collapse
|
6
|
Lange RD, Haefner RM. Task-induced neural covariability as a signature of approximate Bayesian learning and inference. PLoS Comput Biol 2022; 18:e1009557. [PMID: 35259152 PMCID: PMC8963539 DOI: 10.1371/journal.pcbi.1009557] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Revised: 03/29/2022] [Accepted: 10/12/2021] [Indexed: 11/30/2022] Open
Abstract
Perception is often characterized computationally as an inference process in which uncertain or ambiguous sensory inputs are combined with prior expectations. Although behavioral studies have shown that observers can change their prior expectations in the context of a task, robust neural signatures of task-specific priors have been elusive. Here, we analytically derive such signatures under the general assumption that the responses of sensory neurons encode posterior beliefs that combine sensory inputs with task-specific expectations. Specifically, we derive predictions for the task-dependence of correlated neural variability and decision-related signals in sensory neurons. The qualitative aspects of our results are parameter-free and specific to the statistics of each task. The predictions for correlated variability also differ from predictions of classic feedforward models of sensory processing and are therefore a strong test of theories of hierarchical Bayesian inference in the brain. Importantly, we find that Bayesian learning predicts an increase in so-called “differential correlations” as the observer’s internal model learns the stimulus distribution, and the observer’s behavioral performance improves. This stands in contrast to classic feedforward encoding/decoding models of sensory processing, since such correlations are fundamentally information-limiting. We find support for our predictions in data from existing neurophysiological studies across a variety of tasks and brain areas. Finally, we show in simulation how measurements of sensory neural responses can reveal information about a subject’s internal beliefs about the task. Taken together, our results reinterpret task-dependent sources of neural covariability as signatures of Bayesian inference and provide new insights into their cause and their function. Perceptual decision-making has classically been studied in the context of feedforward encoding/ decoding models. Here, we derive predictions for the responses of sensory neurons under the assumption that the brain performs hierarchical Bayesian inference, including feedback signals that communicate task-specific prior expectations. Interestingly, those predictions stand in contrast to some of the conclusions drawn in the classic framework. In particular, we find that Bayesian learning predicts the increase of a type of correlated variability called “differential correlations” over the course of learning. Differential correlations limit information, and hence are seen as harmful in feedforward models. Since our results are also specific to the statistics of a given task, and since they hold under a wide class of theories about how Bayesian probabilities may be represented by neural responses, they constitute a strong test of the Bayesian Brain hypothesis. Our results can explain the task-dependence of correlated variability in prior studies and suggest a reason why these kinds of correlations are surprisingly common in empirical data. Interpreted in a probabilistic framework, correlated variability provides a window into an observer’s task-related beliefs.
Collapse
Affiliation(s)
- Richard D. Lange
- Brain and Cognitive Sciences, University of Rochester, Rochester, New York, United States of America
- Center for Visual Science, University of Rochester, Rochester, New York, United States of America
- * E-mail: (RDL); (RMH)
| | - Ralf M. Haefner
- Brain and Cognitive Sciences, University of Rochester, Rochester, New York, United States of America
- Center for Visual Science, University of Rochester, Rochester, New York, United States of America
- * E-mail: (RDL); (RMH)
| |
Collapse
|
7
|
Ramirez AD, Aksay ERF. Ramp-to-threshold dynamics in a hindbrain population controls the timing of spontaneous saccades. Nat Commun 2021; 12:4145. [PMID: 34230474 PMCID: PMC8260785 DOI: 10.1038/s41467-021-24336-w] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2019] [Accepted: 06/11/2021] [Indexed: 02/06/2023] Open
Abstract
Organisms have the capacity to make decisions based solely on internal drives. However, it is unclear how neural circuits form decisions in the absence of sensory stimuli. Here we provide a comprehensive map of the activity patterns underlying the generation of saccades made in the absence of visual stimuli. We perform calcium imaging in the larval zebrafish to discover a range of responses surrounding spontaneous saccades, from cells that display tonic discharge only during fixations to neurons whose activity rises in advance of saccades by multiple seconds. When we lesion cells in these populations we find that ablation of neurons with pre-saccadic rise delays saccade initiation. We analyze spontaneous saccade initiation using a ramp-to-threshold model and are able to predict the times of upcoming saccades using pre-saccadic activity. These findings suggest that ramping of neuronal activity to a bound is a critical component of self-initiated saccadic movements.
Collapse
Affiliation(s)
- Alexandro D Ramirez
- Institute for Computational Biomedicine and the Department of Physiology and Biophysics, Weill Cornell Medicine, New York, NY, USA.
| | - Emre R F Aksay
- Institute for Computational Biomedicine and the Department of Physiology and Biophysics, Weill Cornell Medicine, New York, NY, USA
| |
Collapse
|
8
|
Essig J, Hunt JB, Felsen G. Inhibitory neurons in the superior colliculus mediate selection of spatially-directed movements. Commun Biol 2021; 4:719. [PMID: 34117346 PMCID: PMC8196039 DOI: 10.1038/s42003-021-02248-1] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Accepted: 05/18/2021] [Indexed: 02/05/2023] Open
Abstract
Decision making is a cognitive process that mediates behaviors critical for survival. Choosing spatial targets is an experimentally-tractable form of decision making that depends on the midbrain superior colliculus (SC). While physiological and computational studies have uncovered the functional topographic organization of the SC, the role of specific SC cell types in spatial choice is unknown. Here, we leveraged behavior, optogenetics, neural recordings and modeling to directly examine the contribution of GABAergic SC neurons to the selection of opposing spatial targets. Although GABAergic SC neurons comprise a heterogeneous population with local and long-range projections, our results demonstrate that GABAergic SC neurons do not locally suppress premotor output, suggesting that functional long-range inhibition instead plays a dominant role in spatial choice. An attractor model requiring only intrinsic SC circuitry was sufficient to account for our experimental observations. Overall, our study elucidates the role of GABAergic SC neurons in spatial choice.
Collapse
Affiliation(s)
- Jaclyn Essig
- Department of Physiology and Biophysics, and Neuroscience Program University of Colorado School of Medicine, Aurora, CO, USA
| | - Joshua B Hunt
- Department of Physiology and Biophysics, and Neuroscience Program University of Colorado School of Medicine, Aurora, CO, USA
| | - Gidon Felsen
- Department of Physiology and Biophysics, and Neuroscience Program University of Colorado School of Medicine, Aurora, CO, USA.
| |
Collapse
|
9
|
Wild B, Treue S. Primate extrastriate cortical area MST: a gateway between sensation and cognition. J Neurophysiol 2021; 125:1851-1882. [PMID: 33656951 DOI: 10.1152/jn.00384.2020] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023] Open
Abstract
Primate visual cortex consists of dozens of distinct brain areas, each providing a highly specialized component to the sophisticated task of encoding the incoming sensory information and creating a representation of our visual environment that underlies our perception and action. One such area is the medial superior temporal cortex (MST), a motion-sensitive, direction-selective part of the primate visual cortex. It receives most of its input from the middle temporal (MT) area, but MST cells have larger receptive fields and respond to more complex motion patterns. The finding that MST cells are tuned for optic flow patterns has led to the suggestion that the area plays an important role in the perception of self-motion. This hypothesis has received further support from studies showing that some MST cells also respond selectively to vestibular cues. Furthermore, the area is part of a network that controls the planning and execution of smooth pursuit eye movements and its activity is modulated by cognitive factors, such as attention and working memory. This review of more than 90 studies focuses on providing clarity of the heterogeneous findings on MST in the macaque cortex and its putative homolog in the human cortex. From this analysis of the unique anatomical and functional position in the hierarchy of areas and processing steps in primate visual cortex, MST emerges as a gateway between perception, cognition, and action planning. Given this pivotal role, this area represents an ideal model system for the transition from sensation to cognition.
Collapse
Affiliation(s)
- Benedict Wild
- Cognitive Neuroscience Laboratory, German Primate Center, Leibniz Institute for Primate Research, Goettingen, Germany.,Goettingen Graduate Center for Neurosciences, Biophysics, and Molecular Biosciences (GGNB), University of Goettingen, Goettingen, Germany
| | - Stefan Treue
- Cognitive Neuroscience Laboratory, German Primate Center, Leibniz Institute for Primate Research, Goettingen, Germany.,Faculty of Biology and Psychology, University of Goettingen, Goettingen, Germany.,Leibniz-ScienceCampus Primate Cognition, Goettingen, Germany.,Bernstein Center for Computational Neuroscience, Goettingen, Germany
| |
Collapse
|
10
|
Cortical Localization of the Sensory-Motor Transformation in a Whisker Detection Task in Mice. eNeuro 2021; 8:ENEURO.0004-21.2021. [PMID: 33495240 PMCID: PMC7901152 DOI: 10.1523/eneuro.0004-21.2021] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2021] [Accepted: 01/08/2021] [Indexed: 11/21/2022] Open
Abstract
Responding to a stimulus requires transforming an internal sensory representation into an internal motor representation. Where and how this sensory-motor transformation occurs is a matter of vigorous debate. Here, we trained male and female mice in a whisker detection go/no-go task in which they learned to respond (lick) following a transient whisker deflection. Using single unit recordings, we quantified sensory-related, motor-related, and choice-related activities in whisker primary somatosensory cortex (S1), whisker region of primary motor cortex (wMC), and anterior lateral motor cortex (ALM), three regions that have been proposed to be critical for the sensory-motor transformation in whisker detection. We observed strong sensory encoding in S1 and wMC, with enhanced encoding in wMC, and a lack of sensory encoding in ALM. We observed strong motor encoding in all three regions, yet largest in wMC and ALM. We observed the earliest choice probability in wMC, despite earliest sensory responses in S1. Based on the criteria of having both strong sensory and motor representations and early choice probability, we identify whisker motor cortex as the cortical region most directly related to the sensory-motor transformation. Our data support a model of sensory encoding originating in S1, sensory amplification and sensory-motor transformation occurring within wMC, and motor signals emerging in ALM after the sensory-motor transformation.
Collapse
|
11
|
Chuang YS, Su YS, Goh JOS. Neural responses reveal associations between personal values and value-based decisions. Soc Cogn Affect Neurosci 2020; 15:1299-1309. [PMID: 33150949 PMCID: PMC7745144 DOI: 10.1093/scan/nsaa150] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2019] [Revised: 10/06/2020] [Accepted: 10/28/2020] [Indexed: 11/26/2022] Open
Abstract
Personal values are thought to modulate value-based decisions, but the neural mechanisms underlying this influence remain unclear. Using a Lottery Choice Task functional brain imaging experiment, we examined the associations between personal value for hedonism and security (based on the Schwartz Value Survey) and subjective neurocognitive processing of reward and loss probability and magnitude objectively coded in stimuli. Hedonistic individuals accepted more losing stakes and showed increased right dorsolateral prefrontal and striatal and left parietal responses with increasing probability of losing. Individuals prioritizing security rejected more stakes and showed reduced right inferior frontal and amygdala responses with increasing stake magnitude, but increased precuneus responses for high-magnitude high-winning probability. With higher hedonism, task-related functional connectivity with the whole brain was higher in right insula and lower in bilateral habenula. For those with higher security ratings, whole-brain functional connectivity was higher in bilateral insula, supplementary motor areas, right superior frontal gyrus, dorsal anterior cingulate cortex, and lower in right middle occipital gyrus. These findings highlight distinct neural engagement across brain systems involved in reward and affective processing, and cognitive control that subserves how individual differences in personal value for gaining rewards or maintaining status quo modulate value-based decisions
Collapse
Affiliation(s)
- Yun-Shiuan Chuang
- Department of Psychology, National Taiwan University, Taipei 10617, Taiwan.,Graduate Institute of Brain and Mind Sciences, College of Medicine, National Taiwan University, Taipei 10051, Taiwan.,Neurobiology and Cognitive Science Center, National Taiwan University, Taipei 10617, Taiwan
| | - Yu-Shiang Su
- Graduate Institute of Brain and Mind Sciences, College of Medicine, National Taiwan University, Taipei 10051, Taiwan.,Taiwan International Graduate Program in Interdisciplinary Neuroscience, National Taiwan University and Academia Sinica, Taipei 10617, Taiwan
| | - Joshua O S Goh
- Department of Psychology, National Taiwan University, Taipei 10617, Taiwan.,Graduate Institute of Brain and Mind Sciences, College of Medicine, National Taiwan University, Taipei 10051, Taiwan.,Neurobiology and Cognitive Science Center, National Taiwan University, Taipei 10617, Taiwan.,Center for Artificial Intelligence and Advanced Robotics, National Taiwan University, Taipei 10617, Taiwan
| |
Collapse
|
12
|
Zhao Y, Yates JL, Levi AJ, Huk AC, Park IM. Stimulus-choice (mis)alignment in primate area MT. PLoS Comput Biol 2020; 16:e1007614. [PMID: 32421716 PMCID: PMC7259805 DOI: 10.1371/journal.pcbi.1007614] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2019] [Revised: 05/29/2020] [Accepted: 04/05/2020] [Indexed: 12/12/2022] Open
Abstract
For stimuli near perceptual threshold, the trial-by-trial activity of single neurons in many sensory areas is correlated with the animal's perceptual report. This phenomenon has often been attributed to feedforward readout of the neural activity by the downstream decision-making circuits. The interpretation of choice-correlated activity is quite ambiguous, but its meaning can be better understood in the light of population-wide correlations among sensory neurons. Using a statistical nonlinear dimensionality reduction technique on single-trial ensemble recordings from the middle temporal (MT) area during perceptual-decision-making, we extracted low-dimensional latent factors that captured the population-wide fluctuations. We dissected the particular contributions of sensory-driven versus choice-correlated activity in the low-dimensional population code. We found that the latent factors strongly encoded the direction of the stimulus in single dimension with a temporal signature similar to that of single MT neurons. If the downstream circuit were optimally utilizing this information, choice-correlated signals should be aligned with this stimulus encoding dimension. Surprisingly, we found that a large component of the choice information resides in the subspace orthogonal to the stimulus representation inconsistent with the optimal readout view. This misaligned choice information allows the feedforward sensory information to coexist with the decision-making process. The time course of these signals suggest that this misaligned contribution likely is feedback from the downstream areas. We hypothesize that this non-corrupting choice-correlated feedback might be related to learning or reinforcing sensory-motor relations in the sensory population.
Collapse
Affiliation(s)
- Yuan Zhao
- Department of Neurobiology and Behavior, Stony Brook University, Stony Brook, New York, United States of America
| | - Jacob L. Yates
- Brain and Cognitive Science, University of Rochester, Rochester, New York, United States of America
| | - Aaron J. Levi
- Center for Perceptual Systems, Departments of Neuroscience & Psychology, The University of Texas at Austin, Austin, Texas, United States of America
| | - Alexander C. Huk
- Center for Perceptual Systems, Departments of Neuroscience & Psychology, The University of Texas at Austin, Austin, Texas, United States of America
| | - Il Memming Park
- Department of Neurobiology and Behavior, Stony Brook University, Stony Brook, New York, United States of America
| |
Collapse
|
13
|
Choice (-history) correlations in sensory cortex: cause or consequence? Curr Opin Neurobiol 2019; 58:148-154. [PMID: 31581052 DOI: 10.1016/j.conb.2019.09.005] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2019] [Revised: 08/04/2019] [Accepted: 09/06/2019] [Indexed: 01/27/2023]
Abstract
One challenge in neuroscience, as in other areas of science, is to make inferences about the underlying causal structure from correlational data. Here, we discuss this challenge in the context of choice correlations in sensory neurons, that is, trial-by-trial correlations, unexplained by the stimulus, between the activity of sensory neurons and an animal's perceptual choice. Do these choice-correlations reflect feedforward, feedback signalling, both, or neither? We highlight recent results of correlational and causal examinations of choice and choice-history signals in sensory, and in part sensorimotor, cortex and address formal statistical frameworks to infer causal interactions from data.
Collapse
|
14
|
Crapse TB, Lau H, Basso MA. A Role for the Superior Colliculus in Decision Criteria. Neuron 2019; 97:181-194.e6. [PMID: 29301100 DOI: 10.1016/j.neuron.2017.12.006] [Citation(s) in RCA: 59] [Impact Index Per Article: 11.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2017] [Revised: 09/27/2017] [Accepted: 12/01/2017] [Indexed: 10/18/2022]
Abstract
Simple decisions arise from the evaluation of sensory evidence. But decisions are determined by more than just evidence. Individuals establish internal decision criteria that influence how they respond. Where or how decision criteria are established in the brain remains poorly understood. Here, we show that neuronal activity in the superior colliculus (SC) predicts changes in decision criteria. Using a novel "Yes-No" task that isolates changes in decision criterion from changes in decision sensitivity, and computing neuronal measures of sensitivity and criterion, we find that SC neuronal activity correlates with the decision criterion regardless of the location of the choice report. We also show that electrical manipulation of activity within the SC produces changes in decisions consistent with changes in decision criteria and are largely independent of the choice report location. Our correlational and causal results together provide strong evidence that SC activity signals the position of a decision criterion. VIDEO ABSTRACT.
Collapse
Affiliation(s)
- Trinity B Crapse
- Fuster Laboratory of Cognitive Neuroscience, UCLA, Los Angeles, CA 90095, USA; Departments of Psychiatry and Biobehavioral Sciences and Neurobiology, UCLA, Los Angeles, CA 90095, USA; Semel Institute of Neuroscience and Human Behavior , UCLA, Los Angeles, CA 90095, USA; Brain Research Institute , UCLA, Los Angeles, CA 90095, USA; David Geffen School of Medicine, UCLA, Los Angeles, CA 90095, USA
| | - Hakwan Lau
- Department of Psychology, UCLA, Los Angeles, CA 90095, USA
| | - Michele A Basso
- Fuster Laboratory of Cognitive Neuroscience, UCLA, Los Angeles, CA 90095, USA; Departments of Psychiatry and Biobehavioral Sciences and Neurobiology, UCLA, Los Angeles, CA 90095, USA; Semel Institute of Neuroscience and Human Behavior , UCLA, Los Angeles, CA 90095, USA; Brain Research Institute , UCLA, Los Angeles, CA 90095, USA; David Geffen School of Medicine, UCLA, Los Angeles, CA 90095, USA.
| |
Collapse
|
15
|
Predicting Perceptual Decisions Using Visual Cortical Population Responses and Choice History. J Neurosci 2019; 39:6714-6727. [PMID: 31235648 DOI: 10.1523/jneurosci.0035-19.2019] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2019] [Revised: 06/12/2019] [Accepted: 06/18/2019] [Indexed: 01/06/2023] Open
Abstract
Our understanding of the neural basis of perceptual decision making has been built in part on relating co-fluctuations of single neuron responses to perceptual decisions on a trial-by-trial basis. The strength of this relationship is often compared across neurons or brain areas, recorded in different sessions, animals, or variants of a task. We sought to extend our understanding of perceptual decision making in three ways. First, we measured neuronal activity simultaneously in early [primary visual cortex (V1)] and midlevel (V4) visual cortex while macaque monkeys performed a fine orientation discrimination perceptual task. This allowed a direct comparison of choice signals in these two areas, including their dynamics. Second, we asked how our ability to predict animals' decisions would be improved by considering small simultaneously-recorded neuronal populations rather than individual units. Finally, we asked whether predictions would be improved by taking into account the animals' choice and reward histories, which can strongly influence decision making. We found that responses of individual V4 neurons were weakly predictive of decisions, but only in a brief epoch between stimulus offset and the indication of choice. In V1, few neurons showed significant decision-related activity. Analysis of neuronal population responses revealed robust choice-related information in V4 and substantially weaker signals in V1. Including choice- and reward-history information improved performance further, particularly when the recorded populations contained little decision-related information. Our work shows the power of using neuronal populations and decision history when relating neuronal responses to the perceptual decisions they are thought to underlie.SIGNIFICANCE STATEMENT Decades of research has provided a rich description of how visual information is represented in the visual cortex. Yet how cortical responses relate to visual perception remains poorly understood. Here we relate fluctuations in small neuronal population responses, recorded simultaneously in primary visual cortex (V1) and area V4 of monkeys, to perceptual reports in an orientation discrimination task. Choice-related signals were robust in V4, particularly late in the behavioral trial, but not in V1. Models that include both neuronal responses and choice-history information were able to predict a substantial portion of decisions. Our work shows the power of integrating information across neurons and including decision history in relating neuronal responses to perceptual decisions.
Collapse
|
16
|
Lakshminarasimhan KJ, Pouget A, DeAngelis GC, Angelaki DE, Pitkow X. Inferring decoding strategies for multiple correlated neural populations. PLoS Comput Biol 2018; 14:e1006371. [PMID: 30248091 PMCID: PMC6188888 DOI: 10.1371/journal.pcbi.1006371] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2017] [Revised: 10/15/2018] [Accepted: 07/17/2018] [Indexed: 12/29/2022] Open
Abstract
Studies of neuron-behaviour correlation and causal manipulation have long been used separately to understand the neural basis of perception. Yet these approaches sometimes lead to drastically conflicting conclusions about the functional role of brain areas. Theories that focus only on choice-related neuronal activity cannot reconcile those findings without additional experiments involving large-scale recordings to measure interneuronal correlations. By expanding current theories of neural coding and incorporating results from inactivation experiments, we demonstrate here that it is possible to infer decoding weights of different brain areas at a coarse scale without precise knowledge of the correlation structure. We apply this technique to neural data collected from two different cortical areas in macaque monkeys trained to perform a heading discrimination task. We identify two opposing decoding schemes, each consistent with data depending on the nature of correlated noise. Our theory makes specific testable predictions to distinguish these scenarios experimentally without requiring measurement of the underlying noise correlations. The neocortex is structurally organized into distinct brain areas. The role of specific brain areas in sensory perception is typically studied using two kinds of laboratory experiments: those that measure correlations between neural activity and reported percepts, and those that inactivate a brain region and measure the resulting changes in percepts. The two types of experiments have generally been interpreted in isolation, in part because no theory has been able combine their outcomes. Here, we describe a mathematical framework that synthesizes both kinds of results, giving us a new way to assess how different brain areas contribute to perception. When we apply our framework to experiments on behaving monkeys, we discover two models that can explain the perplexing finding that one brain area can predict an animal’s reported percepts, even though the percepts are not affected when that brain area is inactivated. The two models ascribe dramatically different efficiencies to brain computation. We show that these two models could be distinguished by a proposed experiment that measures correlations while inactivating different brain areas.
Collapse
Affiliation(s)
| | - Alexandre Pouget
- Department of Basic Neuroscience, University of Geneva, Geneva, Switzerland
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY, United States of America
| | - Gregory C. DeAngelis
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY, United States of America
| | - Dora E. Angelaki
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, United States of America
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, United States of America
- Department of Mechanical and Aerospace Engineering, New York University, New York, United States of America
- Center for Neural Science, New York University, New York, United States of America
| | - Xaq Pitkow
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, United States of America
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, United States of America
- Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, United States of America
- * E-mail:
| |
Collapse
|
17
|
Yu X, Gu Y. Probing Sensory Readout via Combined Choice-Correlation Measures and Microstimulation Perturbation. Neuron 2018; 100:715-727.e5. [PMID: 30244884 DOI: 10.1016/j.neuron.2018.08.034] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2017] [Revised: 01/19/2018] [Accepted: 08/22/2018] [Indexed: 12/18/2022]
Abstract
It is controversial whether covariation between neuronal activity and perceptual choice (i.e., choice correlation) reflects the functional readout of sensory signals. Here, we combined choice-correlation measures and electrical microstimulation on a site-to-site basis in the medial superior temporal area (MST), middle temporal area (MT), and ventral intraparietal area (VIP) when macaques discriminated between motion directions in both fine and coarse tasks. Microstimulation generated comparable effects between tasks but heterogeneous effects across and within brain regions. Within the MST and MT, microstimulation significantly biased an animal's choice toward the sensory preference instead of choice-related signals of the stimulated units. This was particularly evident for sites with conflict preference of sensory and choice-related signals. In the VIP, microstimulation failed to produce significant effects in either task despite strong choice correlations presented in this area. Our results suggest that sensory readout may not be inferred from choice-related signals during perceptual decision-making tasks.
Collapse
Affiliation(s)
- Xuefei Yu
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China; University of Chinese Academy of Sciences, Beijing 100049, China
| | - Yong Gu
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China.
| |
Collapse
|
18
|
Hegdé J. Neural Mechanisms of High-Level Vision. Compr Physiol 2018; 8:903-953. [PMID: 29978891 DOI: 10.1002/cphy.c160035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
The last three decades have seen major strides in our understanding of neural mechanisms of high-level vision, or visual cognition of the world around us. Vision has also served as a model system for the study of brain function. Several broad insights, as yet incomplete, have recently emerged. First, visual perception is best understood not as an end unto itself, but as a sensory process that subserves the animal's behavioral goal at hand. Visual perception is likely to be simply a side effect that reflects the readout of visual information processing that leads to behavior. Second, the brain is essentially a probabilistic computational system that produces behaviors by collectively evaluating, not necessarily consciously or always optimally, the available information about the outside world received from the senses, the behavioral goals, prior knowledge about the world, and possible risks and benefits of a given behavior. Vision plays a prominent role in the overall functioning of the brain providing the lion's share of information about the outside world. Third, the visual system does not function in isolation, but rather interacts actively and reciprocally with other brain systems, including other sensory faculties. Finally, various regions of the visual system process information not in a strict hierarchical manner, but as parts of various dynamic brain-wide networks, collectively referred to as the "connectome." Thus, a full understanding of vision will ultimately entail understanding, in granular, quantitative detail, various aspects of dynamic brain networks that use visual sensory information to produce behavior under real-world conditions. © 2017 American Physiological Society. Compr Physiol 8:903-953, 2018.
Collapse
Affiliation(s)
- Jay Hegdé
- Brain and Behavior Discovery Institute, Augusta University, Augusta, Georgia, USA.,James and Jean Culver Vision Discovery Institute, Augusta University, Augusta, Georgia, USA.,Department of Ophthalmology, Medical College of Georgia, Augusta University, Augusta, Georgia, USA.,The Graduate School, Augusta University, Augusta, Georgia, USA
| |
Collapse
|
19
|
Bondy AG, Haefner RM, Cumming BG. Feedback determines the structure of correlated variability in primary visual cortex. Nat Neurosci 2018; 21:598-606. [PMID: 29483663 PMCID: PMC5876152 DOI: 10.1038/s41593-018-0089-1] [Citation(s) in RCA: 77] [Impact Index Per Article: 12.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2017] [Accepted: 01/22/2018] [Indexed: 02/07/2023]
Abstract
The variable responses of sensory neurons tend to be weakly correlated (spike-count correlation, rsc). This is widely thought to reflect noise in shared afferents, in which case rsc can limit the reliability of sensory coding. However, it could also be due to feedback from higher-order brain regions. Currently, the relative contributions of these sources are unknown. We addressed this by recording from populations of V1 neurons in macaques performing different discrimination tasks involving the same visual input. We found that the structure of rsc (the way rsc varied with neuronal stimulus preference) changed systematically with task instruction. Therefore, even at the earliest stage in the cortical visual hierarchy, rsc structure during task performance primarily reflects feedback dynamics. Consequently, previous proposals for how rsc constrains sensory processing need not apply. Furthermore, we show that correlations between the activity of single neurons and choice depend on feedback engaged by the task.
Collapse
Affiliation(s)
- Adrian G Bondy
- Laboratory of Sensorimotor Research, National Eye Institute, NIH, Bethesda, MD, USA. .,Brown-NIH Neuroscience Graduate Partnership Program, Providence, RI, USA.
| | - Ralf M Haefner
- Brain & Cognitive Sciences, Center for Visual Science, University of Rochester, Rochester, NY, USA
| | - Bruce G Cumming
- Laboratory of Sensorimotor Research, National Eye Institute, NIH, Bethesda, MD, USA
| |
Collapse
|
20
|
Decision-Related Activity in Macaque V2 for Fine Disparity Discrimination Is Not Compatible with Optimal Linear Readout. J Neurosci 2017; 37:715-725. [PMID: 28100751 DOI: 10.1523/jneurosci.2445-16.2016] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2016] [Revised: 11/20/2016] [Accepted: 11/29/2016] [Indexed: 11/21/2022] Open
Abstract
Fine judgments of stereoscopic depth rely mainly on relative judgments of depth (relative binocular disparity) between objects, rather than judgments of the distance to where the eyes are fixating (absolute disparity). In macaques, visual area V2 is the earliest site in the visual processing hierarchy for which neurons selective for relative disparity have been observed (Thomas et al., 2002). Here, we found that, in macaques trained to perform a fine disparity discrimination task, disparity-selective neurons in V2 were highly selective for the task, and their activity correlated with the animals' perceptual decisions (unexplained by the stimulus). This may partially explain similar correlations reported in downstream areas. Although compatible with a perceptual role of these neurons for the task, the interpretation of such decision-related activity is complicated by the effects of interneuronal "noise" correlations between sensory neurons. Recent work has developed simple predictions to differentiate decoding schemes (Pitkow et al., 2015) without needing measures of noise correlations, and found that data from early sensory areas were compatible with optimal linear readout of populations with information-limiting correlations. In contrast, our data here deviated significantly from these predictions. We additionally tested this prediction for previously reported results of decision-related activity in V2 for a related task, coarse disparity discrimination (Nienborg and Cumming, 2006), thought to rely on absolute disparity. Although these data followed the predicted pattern, they violated the prediction quantitatively. This suggests that optimal linear decoding of sensory signals is not generally a good predictor of behavior in simple perceptual tasks. SIGNIFICANCE STATEMENT Activity in sensory neurons that correlates with an animal's decision is widely believed to provide insights into how the brain uses information from sensory neurons. Recent theoretical work developed simple predictions to differentiate decoding schemes, and found support for optimal linear readout of early sensory populations with information-limiting correlations. Here, we observed decision-related activity for neurons in visual area V2 of macaques performing fine disparity discrimination, as yet the earliest site for this task. These findings, and previously reported results from V2 in a different task, deviated from the predictions for optimal linear readout of a population with information-limiting correlations. Our results suggest that optimal linear decoding of early sensory information is not a general decoding strategy used by the brain.
Collapse
|
21
|
Niemeyer JE, Paradiso MA. Contrast sensitivity, V1 neural activity, and natural vision. J Neurophysiol 2017; 117:492-508. [PMID: 27832603 PMCID: PMC5288473 DOI: 10.1152/jn.00635.2016] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2016] [Accepted: 10/30/2016] [Indexed: 11/22/2022] Open
Abstract
Contrast sensitivity is fundamental to natural visual processing and an important tool for characterizing both visual function and clinical disorders. We simultaneously measured contrast sensitivity and neural contrast response functions and compared measurements in common laboratory conditions with naturalistic conditions. In typical experiments, a subject holds fixation and a stimulus is flashed on, whereas in natural vision, saccades bring stimuli into view. Motivated by our previous V1 findings, we tested the hypothesis that perceptual contrast sensitivity is lower in natural vision and that this effect is associated with corresponding changes in V1 activity. We found that contrast sensitivity and V1 activity are correlated and that the relationship is similar in laboratory and naturalistic paradigms. However, in the more natural situation, contrast sensitivity is reduced up to 25% compared with that in a standard fixation paradigm, particularly at lower spatial frequencies, and this effect correlates with significant reductions in V1 responses. Our data suggest that these reductions in natural vision result from fast adaptation on one fixation that lowers the response on a subsequent fixation. This is the first demonstration of rapid, natural-image adaptation that carries across saccades, a process that appears to constantly influence visual sensitivity in natural vision. NEW & NOTEWORTHY Visual sensitivity and activity in brain area V1 were studied in a paradigm that included saccadic eye movements and natural visual input. V1 responses and contrast sensitivity were significantly reduced compared with results in common laboratory paradigms. The parallel neural and perceptual effects of eye movements and stimulus complexity appear to be due to a form of rapid adaptation that carries across saccades.
Collapse
Affiliation(s)
- James E Niemeyer
- Department of Neuroscience, Brown University, Providence, Rhode Island
| | | |
Collapse
|
22
|
Urai AE, Murphy PR. Commentary: Sensory integration dynamics in a hierarchical network explains choice probabilities in cortical area MT. Front Syst Neurosci 2016; 10:37. [PMID: 27199686 PMCID: PMC4853373 DOI: 10.3389/fnsys.2016.00037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2016] [Accepted: 04/14/2016] [Indexed: 11/16/2022] Open
Affiliation(s)
- Anne E. Urai
- Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-EppendorfHamburg, Germany
- Department of Psychology, University of AmsterdamAmsterdam, Netherlands
- *Correspondence: Anne E. Urai
| | - Peter R. Murphy
- Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-EppendorfHamburg, Germany
- Peter R. Murphy
| |
Collapse
|