1
|
Boundy-Singer ZM, Ziemba CM, Hénaff OJ, Goris RLT. How does V1 population activity inform perceptual certainty? J Vis 2024; 24:12. [PMID: 38884544 PMCID: PMC11185272 DOI: 10.1167/jov.24.6.12] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2024] [Accepted: 05/06/2024] [Indexed: 06/18/2024] Open
Abstract
Neural population activity in sensory cortex informs our perceptual interpretation of the environment. Oftentimes, this population activity will support multiple alternative interpretations. The larger the spread of probability over different alternatives, the more uncertain the selected perceptual interpretation. We test the hypothesis that the reliability of perceptual interpretations can be revealed through simple transformations of sensory population activity. We recorded V1 population activity in fixating macaques while presenting oriented stimuli under different levels of nuisance variability and signal strength. We developed a decoding procedure to infer from V1 activity the most likely stimulus orientation as well as the certainty of this estimate. Our analysis shows that response magnitude, response dispersion, and variability in response gain all offer useful proxies for orientation certainty. Of these three metrics, the last one has the strongest association with the decoder's uncertainty estimates. These results clarify that the nature of neural population activity in sensory cortex provides downstream circuits with multiple options to assess the reliability of perceptual interpretations.
Collapse
Affiliation(s)
- Zoe M Boundy-Singer
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA
| | - Corey M Ziemba
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA
| | | | - Robbe L T Goris
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA
| |
Collapse
|
2
|
Charlton JA, Goris RLT. Abstract deliberation by visuomotor neurons in prefrontal cortex. Nat Neurosci 2024; 27:1167-1175. [PMID: 38684894 PMCID: PMC11156582 DOI: 10.1038/s41593-024-01635-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 03/29/2024] [Indexed: 05/02/2024]
Abstract
During visually guided behavior, the prefrontal cortex plays a pivotal role in mapping sensory inputs onto appropriate motor plans. When the sensory input is ambiguous, this involves deliberation. It is not known whether the deliberation is implemented as a competition between possible stimulus interpretations or between possible motor plans. Here we study neural population activity in the prefrontal cortex of macaque monkeys trained to flexibly report perceptual judgments of ambiguous visual stimuli. We find that the population activity initially represents the formation of a perceptual choice before transitioning into the representation of the motor plan. Stimulus strength and prior expectations both bear on the formation of the perceptual choice, but not on the formation of the action plan. These results suggest that prefrontal circuits involved in action selection are also used for the deliberation of abstract propositions divorced from a specific motor plan, thus providing a crucial mechanism for abstract reasoning.
Collapse
Affiliation(s)
- Julie A Charlton
- Center for Perceptual Systems, The University of Texas at Austin, Austin, TX, USA
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Robbe L T Goris
- Center for Perceptual Systems, The University of Texas at Austin, Austin, TX, USA.
| |
Collapse
|
3
|
Goris RLT, Coen-Cagli R, Miller KD, Priebe NJ, Lengyel M. Response sub-additivity and variability quenching in visual cortex. Nat Rev Neurosci 2024; 25:237-252. [PMID: 38374462 DOI: 10.1038/s41583-024-00795-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/24/2024] [Indexed: 02/21/2024]
Abstract
Sub-additivity and variability are ubiquitous response motifs in the primary visual cortex (V1). Response sub-additivity enables the construction of useful interpretations of the visual environment, whereas response variability indicates the factors that limit the precision with which the brain can do this. There is increasing evidence that experimental manipulations that elicit response sub-additivity often also quench response variability. Here, we provide an overview of these phenomena and suggest that they may have common origins. We discuss empirical findings and recent model-based insights into the functional operations, computational objectives and circuit mechanisms underlying V1 activity. These different modelling approaches all predict that response sub-additivity and variability quenching often co-occur. The phenomenology of these two response motifs, as well as many of the insights obtained about them in V1, generalize to other cortical areas. Thus, the connection between response sub-additivity and variability quenching may be a canonical motif across the cortex.
Collapse
Affiliation(s)
- Robbe L T Goris
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA.
| | - Ruben Coen-Cagli
- Department of Systems and Computational Biology, Albert Einstein College of Medicine, Bronx, NY, USA
- Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, USA
- Department of Ophthalmology and Visual Sciences, Albert Einstein College of Medicine, Bronx, NY, USA
| | - Kenneth D Miller
- Center for Theoretical Neuroscience, Columbia University, New York, NY, USA
- Kavli Institute for Brain Science, Columbia University, New York, NY, USA
- Dept. of Neuroscience, College of Physicians and Surgeons, Columbia University, New York, NY, USA
- Morton B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
- Swartz Program in Theoretical Neuroscience, Columbia University, New York, NY, USA
| | - Nicholas J Priebe
- Center for Learning and Memory, University of Texas at Austin, Austin, TX, USA
| | - Máté Lengyel
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK
- Center for Cognitive Computation, Department of Cognitive Science, Central European University, Budapest, Hungary
| |
Collapse
|
4
|
Primary visual cortex straightens natural video trajectories. Nat Commun 2021; 12:5982. [PMID: 34645787 PMCID: PMC8514453 DOI: 10.1038/s41467-021-25939-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2020] [Accepted: 09/08/2021] [Indexed: 11/08/2022] Open
Abstract
Many sensory-driven behaviors rely on predictions about future states of the environment. Visual input typically evolves along complex temporal trajectories that are difficult to extrapolate. We test the hypothesis that spatial processing mechanisms in the early visual system facilitate prediction by constructing neural representations that follow straighter temporal trajectories. We recorded V1 population activity in anesthetized macaques while presenting static frames taken from brief video clips, and developed a procedure to measure the curvature of the associated neural population trajectory. We found that V1 populations straighten naturally occurring image sequences, but entangle artificial sequences that contain unnatural temporal transformations. We show that these effects arise in part from computational mechanisms that underlie the stimulus selectivity of V1 cells. Together, our findings reveal that the early visual system uses a set of specialized computations to build representations that can support prediction in the natural environment. Many behaviours depend on predictions about the environment. Here the authors find neural populations in primary visual cortex to straighten the temporal trajectories of natural video clips, facilitating the extrapolation of past observations.
Collapse
|
5
|
Bao X, Salloum A, Gordon SG, Lomber SG. The limited capacity of visual temporal integration in cats. J Vis 2020; 20:28. [PMID: 32852533 PMCID: PMC7453054 DOI: 10.1167/jov.20.8.28] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
It has been long known that prolonging stimulus duration may increase the perceived brightness of a visual stimulus. The interaction between intensity and duration generally follows a rule, such as that described in Bloch's law. This visual temporal integration relationship has been identified in human subjects and in non-human primates. However, although auditory temporal integration has been extensively studied in the cat, visual temporal integration has not. Therefore, the goal of this study was to examine visual temporal integration in the cat. We trained five cats to respond when a brief luminance change was detected in a fixation dot. After training, we measured the success rate of detecting the luminance change with varying durations at threshold, subthreshold, and suprathreshold luminance levels. Psychometric functions showed that prolonging stimulus duration improved task performance, more noticeably for stimuli below 100 ms than beyond. Most psychometric functions were better fit to an exponential model than to a linear model. The gradually saturated performance observed here, as in previous studies, can be explained by the “leaky integrator” hypothesis, that is, temporal integration is only valid below a critical duration. Overall, we developed a task whereby visual temporal integration was successfully demonstrated in the cat. The effect of stimulus duration on detection success rate displayed a pattern generally consistent with previous human and non-human primate findings on visual temporal integration.
Collapse
Affiliation(s)
- Xiaohan Bao
- Integrated Program in Neuroscience, McGill University, Montreal, Quebec, Canada
| | - Anas Salloum
- Undergraduate Program in Physiology and Pharmacology, University of Western Ontario, London, Ontario, Canada
| | - Stephen G Gordon
- Department of Physiology, Faculty of Medicine, McGill University, Montreal, Quebec, Canada
| | - Stephen G Lomber
- Department of Physiology, Faculty of Medicine, McGill University, Montreal, Quebec, Canada
| |
Collapse
|
6
|
Hénaff OJ, Boundy-Singer ZM, Meding K, Ziemba CM, Goris RLT. Representation of visual uncertainty through neural gain variability. Nat Commun 2020; 11:2513. [PMID: 32427825 PMCID: PMC7237668 DOI: 10.1038/s41467-020-15533-0] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2019] [Accepted: 03/14/2020] [Indexed: 01/25/2023] Open
Abstract
Uncertainty is intrinsic to perception. Neural circuits which process sensory information must therefore also represent the reliability of this information. How they do so is a topic of debate. We propose a model of visual cortex in which average neural response strength encodes stimulus features, while cross-neuron variability in response gain encodes the uncertainty of these features. To test this model, we studied spiking activity of neurons in macaque V1 and V2 elicited by repeated presentations of stimuli whose uncertainty was manipulated in distinct ways. We show that gain variability of individual neurons is tuned to stimulus uncertainty, that this tuning is specific to the features encoded by these neurons and largely invariant to the source of uncertainty. We demonstrate that this behavior naturally arises from known gain-control mechanisms, and illustrate how downstream circuits can jointly decode stimulus features and their uncertainty from sensory population activity.
Collapse
Affiliation(s)
- Olivier J Hénaff
- Center for Neural Science, New York University, New York, NY, USA.,DeepMind, London, UK
| | - Zoe M Boundy-Singer
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA
| | - Kristof Meding
- Neural Information Processing Group, University of Tübingen, Tübingen, Germany
| | - Corey M Ziemba
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA
| | - Robbe L T Goris
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA.
| |
Collapse
|
7
|
Abstract
To model the responses of neurons in the early visual system, at least three basic components are required: a receptive field, a normalization term, and a specification of encoding noise. Here, we examine how the receptive field, the normalization factor, and the encoding noise affect the drive to model-neuron responses when stimulated with natural images. We show that when these components are modeled appropriately, the response drives elicited by natural stimuli are Gaussian-distributed and scale invariant, and very nearly maximize the sensitivity (d') for natural-image discrimination. We discuss the statistical models of natural stimuli that can account for these response statistics, and we show how some commonly used modeling practices may distort these results. Finally, we show that normalization can equalize important properties of neural response across different stimulus types. Specifically, narrowband (stimulus- and feature-specific) normalization causes model neurons to yield Gaussian response-drive statistics when stimulated with natural stimuli, 1/f noise stimuli, and white-noise stimuli. The current work makes recommendations for best practices and lays a foundation, grounded in the response statistics to natural stimuli, upon which to build principled models of more complex visual tasks.
Collapse
Affiliation(s)
- Arvind Iyer
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA
| | - Johannes Burge
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA.,Neuroscience Graduate Group, University of Pennsylvania, Philadelphia, PA, USA.,Bioengineering Graduate Group, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
8
|
Yates JL, Katz LN, Levi AJ, Pillow JW, Huk AC. A simple linear readout of MT supports motion direction-discrimination performance. J Neurophysiol 2019; 123:682-694. [PMID: 31852399 DOI: 10.1152/jn.00117.2019] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
Motion discrimination is a well-established model system for investigating how sensory signals are used to form perceptual decisions. Classic studies relating single-neuron activity in the middle temporal area (MT) to perceptual decisions have suggested that a simple linear readout could underlie motion discrimination behavior. A theoretically optimal readout, in contrast, would take into account the correlations between neurons and the sensitivity of individual neurons at each time point. However, it remains unknown how sophisticated the readout needs to be to support actual motion-discrimination behavior or to approach optimal performance. In this study, we evaluated the performance of various neurally plausible decoders, trained to discriminate motion direction from small ensembles of simultaneously recorded MT neurons. We found that decoding the stimulus without knowledge of the interneuronal correlations was sufficient to match an optimal (correlation aware) decoder. Additionally, a decoder could match the psychophysical performance of the animals with flat integration of up to half the stimulus and inherited temporal dynamics from the time-varying MT responses. These results demonstrate that simple, linear decoders operating on small ensembles of neurons can match both psychophysical performance and optimal sensitivity without taking correlations into account and that such simple read-out mechanisms can exhibit complex temporal properties inherited from the sensory dynamics themselves.NEW & NOTEWORTHY Motion perception depends on the ability to decode the activity of neurons in the middle temporal area. Theoretically optimal decoding requires knowledge of the sensitivity of neurons and interneuronal correlations. We report that a simple correlation-blind decoder performs as well as the optimal decoder for coarse motion discrimination. Additionally, the decoder could match the psychophysical performance with moderate temporal integration and dynamics inherited from sensory responses.
Collapse
Affiliation(s)
- Jacob L Yates
- Brain and Cognitive Science, University of Rochester, Rochester, New York.,Center for Perceptual Systems, University of Texas at Austin, Austin, Texas.,Department of Neuroscience, University of Texas at Austin, Austin, Texas
| | - Leor N Katz
- Center for Perceptual Systems, University of Texas at Austin, Austin, Texas.,Department of Neuroscience, University of Texas at Austin, Austin, Texas.,Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland
| | - Aaron J Levi
- Center for Perceptual Systems, University of Texas at Austin, Austin, Texas.,Department of Neuroscience, University of Texas at Austin, Austin, Texas.,Department of Psychology, University of Texas at Austin, Austin, Texas
| | - Jonathan W Pillow
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey.,Department of Psychology, Princeton University, Princeton, New Jersey
| | - Alexander C Huk
- Center for Perceptual Systems, University of Texas at Austin, Austin, Texas.,Department of Neuroscience, University of Texas at Austin, Austin, Texas.,Department of Psychology, University of Texas at Austin, Austin, Texas
| |
Collapse
|
9
|
Whiteway MR, Butts DA. The quest for interpretable models of neural population activity. Curr Opin Neurobiol 2019; 58:86-93. [PMID: 31426024 DOI: 10.1016/j.conb.2019.07.004] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2018] [Accepted: 07/14/2019] [Indexed: 11/24/2022]
Abstract
Many aspects of brain function arise from the coordinated activity of large populations of neurons. Recent developments in neural recording technologies are providing unprecedented access to the activity of such populations during increasingly complex experimental contexts; however, extracting scientific insights from such recordings requires the concurrent development of analytical tools that relate this population activity to system-level function. This is a primary motivation for latent variable models, which seek to provide a low-dimensional description of population activity that can be related to experimentally controlled variables, as well as uncontrolled variables such as internal states (e.g. attention and arousal) and elements of behavior. While deriving an understanding of function from traditional latent variable methods relies on low-dimensional visualizations, new approaches are targeting more interpretable descriptions of the components underlying system-level function.
Collapse
Affiliation(s)
- Matthew R Whiteway
- Zuckerman Mind Brain Behavior Institute, Jerome L Greene Science Center, Columbia University, 3227 Broadway, 5th Floor, Quad D, New York, NY 10027, USA
| | - Daniel A Butts
- Department of Biology and Program in Neuroscience and Cognitive Science, University of Maryland, 1210 Biology-Psychology Bldg. #144, College Park, MD 20742, USA.
| |
Collapse
|