1
|
Denagamage S, Morton MP, Hudson NV, Nandy AS. Widespread receptive field remapping in early primate visual cortex. Cell Rep 2024; 43:114557. [PMID: 39058592 DOI: 10.1016/j.celrep.2024.114557] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Revised: 04/24/2024] [Accepted: 07/13/2024] [Indexed: 07/28/2024] Open
Abstract
Predictive remapping of receptive fields (RFs) is thought to be one of the critical mechanisms for enforcing perceptual stability during eye movements. While RF remapping has been observed in several cortical areas, its role in early visual cortex and its consequences on the tuning properties of neurons have been poorly understood. Here, we track remapping RFs in hundreds of neurons from visual area V2 while subjects perform a cued saccade task. We find that remapping is widespread in area V2 across neurons from all recorded cortical layers and cell types. Furthermore, our results suggest that remapping RFs not only maintain but also transiently enhance their feature selectivity due to untuned suppression. Taken together, these findings shed light on the dynamics and prevalence of remapping in the early visual cortex, forcing us to revise current models of perceptual stability during saccadic eye movements.
Collapse
Affiliation(s)
- Sachira Denagamage
- Department of Neuroscience, Yale University, New Haven, CT 06510, USA; Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06510, USA.
| | - Mitchell P Morton
- Department of Neuroscience, Yale University, New Haven, CT 06510, USA; Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06510, USA
| | - Nyomi V Hudson
- Department of Neuroscience, Yale University, New Haven, CT 06510, USA
| | - Anirvan S Nandy
- Department of Neuroscience, Yale University, New Haven, CT 06510, USA; Interdepartmental Neuroscience Program, Yale University, New Haven, CT 06510, USA; Kavli Institute for Neuroscience, Yale University, New Haven, CT 06510, USA; Wu Tsai Institute, Yale University, New Haven, CT 06510, USA; Department of Psychology, Yale University, New Haven, CT 06510, USA.
| |
Collapse
|
2
|
Liu X, Melcher D, Carrasco M, Hanning NM. Pre-saccadic Preview Shapes Post-Saccadic Processing More Where Perception is Poor. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.05.18.541028. [PMID: 37292871 PMCID: PMC10245755 DOI: 10.1101/2023.05.18.541028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
The pre-saccadic preview of a peripheral target enhances the efficiency of its post-saccadic processing, termed the extrafoveal preview effect. Peripheral visual performance -and thus the quality of the preview- varies around the visual field, even at iso-eccentric locations: it is better along the horizontal than vertical meridian and along the lower than upper vertical meridian. To investigate whether these polar angle asymmetries influence the preview effect, we asked human participants (to preview four tilted gratings at the cardinals, until a central cue indicated to which one to saccade. During the saccade, the target orientation either remained or slightly changed (valid/invalid preview). After saccade landing, participants discriminated the orientation of the (briefly presented) second grating. Stimulus contrast was titrated with adaptive staircases to assess visual performance. Expectedly, valid previews increased participants' post-saccadic contrast sensitivity. This preview benefit, however, was inversely related to polar angle perceptual asymmetries; largest at the upper, and smallest at the horizontal meridian. This finding reveals that the visual system compensates for peripheral asymmetries when integrating information across saccades, by selectively assigning higher weights to the less-well perceived preview information. Our study supports the recent line of evidence showing that perceptual dynamics around saccades vary with eye movement direction.
Collapse
|
3
|
Xiao W, Sharma S, Kreiman G, Livingstone MS. Feature-selective responses in macaque visual cortex follow eye movements during natural vision. Nat Neurosci 2024; 27:1157-1166. [PMID: 38684892 PMCID: PMC11156562 DOI: 10.1038/s41593-024-01631-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Accepted: 03/26/2024] [Indexed: 05/02/2024]
Abstract
In natural vision, primates actively move their eyes several times per second via saccades. It remains unclear whether, during this active looking, visual neurons exhibit classical retinotopic properties, anticipate gaze shifts or mirror the stable quality of perception, especially in complex natural scenes. Here, we let 13 monkeys freely view thousands of natural images across 4.6 million fixations, recorded 883 h of neuronal responses in six areas spanning primary visual to anterior inferior temporal cortex and analyzed spatial, temporal and featural selectivity in these responses. Face neurons tracked their receptive field contents, indicated by category-selective responses. Self-consistency analysis showed that general feature-selective responses also followed eye movements and remained gaze-dependent over seconds of viewing the same image. Computational models of feature-selective responses located retinotopic receptive fields during free viewing. We found limited evidence for feature-selective predictive remapping and no viewing-history integration. Thus, ventral visual neurons represent the world in a predominantly eye-centered reference frame during natural vision.
Collapse
Affiliation(s)
- Will Xiao
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA.
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA.
| | - Saloni Sharma
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Gabriel Kreiman
- Department of Ophthalmology, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA
| | | |
Collapse
|
4
|
Wang X, Zhang C, Yang L, Jin M, Goldberg ME, Zhang M, Qian N. Perisaccadic and attentional remapping of receptive fields in lateral intraparietal area and frontal eye fields. Cell Rep 2024; 43:113820. [PMID: 38386553 PMCID: PMC11011051 DOI: 10.1016/j.celrep.2024.113820] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 12/15/2023] [Accepted: 02/01/2024] [Indexed: 02/24/2024] Open
Abstract
The nature and function of perisaccadic receptive field (RF) remapping have been controversial. We use a delayed saccade task to reduce previous confounds and examine the remapping time course in the lateral intraparietal area and frontal eye fields. In the delay period, the RF shift direction turns from the initial fixation to the saccade target. In the perisaccadic period, RFs first shift toward the target (convergent remapping), but around the time of saccade onset/offset, the shifts become predominantly toward the post-saccadic RF locations (forward remapping). Thus, unlike forward remapping that depends on the corollary discharge (CD) of the saccade command, convergent remapping appears to follow attention from the initial fixation to the target. We model the data with attention-modulated and CD-gated connections and show that both sets of connections emerge automatically in neural networks trained to update stimulus retinal locations across saccades. Our work thus unifies previous findings into a mechanism for transsaccadic visual stability.
Collapse
Affiliation(s)
- Xiao Wang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Department of Neuroscience and Zuckerman Institute, Columbia University, New York, NY, USA
| | - Cong Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
| | - Lin Yang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Min Jin
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Michael E Goldberg
- Department of Neuroscience and Zuckerman Institute, Columbia University, New York, NY, USA; Departments of Neurology, Psychiatry, and Ophthalmology, Columbia University, New York, NY, USA
| | - Mingsha Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.
| | - Ning Qian
- Department of Neuroscience and Zuckerman Institute, Columbia University, New York, NY, USA; Department of Physiology & Cellular Biophysics, Columbia University, New York, NY, USA.
| |
Collapse
|
5
|
Wang X, Zhang C, Yang L, Jin M, Goldberg ME, Zhang M, Qian N. Perisaccadic and Attentional Remapping of Receptive Fields in Lateral Intraparietal Area and Frontal Eye Fields. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.09.23.558993. [PMID: 37790528 PMCID: PMC10542176 DOI: 10.1101/2023.09.23.558993] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/05/2023]
Abstract
The nature and function of perisaccadic receptive-field (RF) remapping have been controversial. We used a delayed saccade task to reduce previous confounds and examined the remapping time course in areas LIP and FEF. In the delay period, the RF shift direction turned from the initial fixation to the saccade target. In the perisaccadic period, RFs first shifted toward the target (convergent remapping) but around the time of saccade onset/offset, the shifts became predominantly toward the post-saccadic RF locations (forward remapping). Thus, unlike forward remapping that depends on the corollary discharge (CD) of the saccade command, convergent remapping appeared to follow attention from the initial fixation to the target. We modelled the data with attention-modulated and CD-gated connections, and showed that both sets of connections emerged automatically in neural networks trained to update stimulus retinal locations across saccades. Our work thus unifies previous findings into a mechanism for transsaccadic visual stability.
Collapse
Affiliation(s)
- Xiao Wang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Department of Neuroscience and Zuckerman Institute, Columbia University, New York, NY, USA
| | - Cong Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Lin Yang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Min Jin
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Michael E. Goldberg
- Department of Neuroscience and Zuckerman Institute, Columbia University, New York, NY, USA
- Departments of Neurology, Psychiatry, and Ophthalmology, Columbia University, New York, NY, USA
| | - Mingsha Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Ning Qian
- Department of Neuroscience and Zuckerman Institute, Columbia University, New York, NY, USA
- Department of Physiology & Cellular Biophysics, Columbia University, New York, NY, USA
| |
Collapse
|
6
|
Li HH, Curtis CE. Neural population dynamics of human working memory. Curr Biol 2023; 33:3775-3784.e4. [PMID: 37595590 PMCID: PMC10528783 DOI: 10.1016/j.cub.2023.07.067] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 06/20/2023] [Accepted: 07/31/2023] [Indexed: 08/20/2023]
Abstract
The activity of neurons in macaque prefrontal cortex (PFC) persists during working memory (WM) delays, providing a mechanism for memory.1,2,3,4,5,6,7,8,9,10,11 Although theory,11,12 including formal network models,13,14 assumes that WM codes are stable over time, PFC neurons exhibit dynamics inconsistent with these assumptions.15,16,17,18,19 Recently, multivariate reanalyses revealed the coexistence of both stable and dynamic WM codes in macaque PFC.20,21,22,23 Human EEG studies also suggest that WM might contain dynamics.24,25 Nonetheless, how WM dynamics vary across the cortical hierarchy and which factors drive dynamics remain unknown. To elucidate WM dynamics in humans, we decoded WM content from fMRI responses across multiple cortical visual field maps.26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48 We found coexisting stable and dynamic neural representations of WM during a memory-guided saccade task. Geometric analyses of neural subspaces revealed that early visual cortex exhibited stronger dynamics than high-level visual and frontoparietal cortex. Leveraging models of population receptive fields, we visualized and made the neural dynamics interpretable. We found that during WM delays, V1 population initially encoded a narrowly tuned bump of activation centered on the peripheral memory target. Remarkably, this bump then spread inward toward foveal locations, forming a vector along the trajectory of the forthcoming memory-guided saccade. In other words, the neural code transformed into an abstraction of the stimulus more proximal to memory-guided behavior. Therefore, theories of WM must consider both sensory features and their task-relevant abstractions because changes in the format of memoranda naturally drive neural dynamics.
Collapse
Affiliation(s)
- Hsin-Hung Li
- Department of Psychology, New York University, New York, NY 10003, USA; Center for Neural Science, New York University, New York, NY 10003, USA
| | - Clayton E Curtis
- Department of Psychology, New York University, New York, NY 10003, USA; Center for Neural Science, New York University, New York, NY 10003, USA.
| |
Collapse
|
7
|
Qian N, Goldberg ME, Zhang M. Tuning curves vs. population responses, and perceptual consequences of receptive-field remapping. Front Comput Neurosci 2023; 16:1060757. [PMID: 36714528 PMCID: PMC9880053 DOI: 10.3389/fncom.2022.1060757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Accepted: 12/21/2022] [Indexed: 01/15/2023] Open
Abstract
Sensory processing is often studied by examining how a given neuron responds to a parameterized set of stimuli (tuning curve) or how a given stimulus evokes responses from a parameterized set of neurons (population response). Although tuning curves and the corresponding population responses contain the same information, they can have different properties. These differences are known to be important because the perception of a stimulus should be decoded from its population response, not from any single tuning curve. The differences are less studied in the spatial domain where a cell's spatial tuning curve is simply its receptive field (RF) profile. Here, we focus on evaluating the common belief that perrisaccadic forward and convergent RF shifts lead to forward (translational) and convergent (compressive) perceptual mislocalization, respectively, and investigate the effects of three related factors: decoders' awareness of RF shifts, changes of cells' covering density near attentional locus (the saccade target), and attentional response modulation. We find that RF shifts alone produce either no shift or an opposite shift of the population responses depending on whether or not decoders are aware of the RF shifts. Thus, forward RF shifts do not predict forward mislocalization. However, convergent RF shifts change cells' covering density for aware decoders (but not for unaware decoders) which may predict convergent mislocalization. Finally, attentional modulation adds a convergent component to population responses for stimuli near the target. We simulate the combined effects of these factors and discuss the results with extant mislocalization data. We speculate that perisaccadic mislocalization might be the flash-lag effect unrelated to perisaccadic RF remapping but to resolve the issue, one has to address the question of whether or not perceptual decoders are aware of RF shifts.
Collapse
Affiliation(s)
- Ning Qian
- Department of Neuroscience and Zuckerman Institute, Columbia University, New York, NY, United States
- Department of Physiology and Cellular Biophysics, Columbia University, New York, NY, United States
| | - Michael E. Goldberg
- Department of Neuroscience and Zuckerman Institute, Columbia University, New York, NY, United States
- Departments of Neurology, Psychiatry, and Ophthalmology, Columbia University, New York, NY, United States
| | - Mingsha Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| |
Collapse
|
8
|
Zhang B, Wang F, Zhang Q, Naya Y. Distinct networks coupled with parietal cortex for spatial representations inside and outside the visual field. Neuroimage 2022; 252:119041. [PMID: 35231630 DOI: 10.1016/j.neuroimage.2022.119041] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Revised: 02/22/2022] [Accepted: 02/24/2022] [Indexed: 11/19/2022] Open
Abstract
Our mental representation of egocentric space is influenced by the disproportionate sensory perception of the body. Previous studies have focused on the neural architecture for egocentric representations within the visual field. However, the space representation underlying the body is still unclear. To address this problem, we applied both functional Magnitude Resonance Imaging (fMRI) and Magnetoencephalography (MEG) to a spatial-memory paradigm by using a virtual environment in which human participants remembered a target location left, right, or back relative to their own body. Both experiments showed larger involvement of the frontoparietal network in representing a retrieved target on the left/right side than on the back. Conversely, the medial temporal lobe (MTL)-parietal network was more involved in retrieving a target behind the participants. The MEG data showed an earlier activation of the MTL-parietal network than that of the frontoparietal network during retrieval of a target location. These findings suggest that the parietal cortex may represent the entire space around the self-body by coordinating two distinct brain networks.
Collapse
Affiliation(s)
- Bo Zhang
- School of Psychological and Cognitive Sciences, Peking University, No. 52, Haidian Road, Haidian District, Beijing 100805, China; Beijing Academy of Artificial Intelligence, Beijing, 100084, China; Tsinghua Laboratory of Brain and Intelligence, 160 Chengfu Rd., SanCaiTang Building, Haidian District, Beijing, 100084, China
| | - Fan Wang
- State Key Laboratory of Brain and Cognitive Science, Institute of Biophysics, Chinese Academy of Sciences, 15 Datun Road, Beijing 100101, China; CAS Center for Excellence in Brain Science and Intelligence Technology, Institute of Biophysics, Chinese Academy of Sciences, 15 Datun Road, Beijing 100101, China; University of Chinese Academy of Sciences, 19A Yuquan Road, Beijing 100049, China
| | - Qi Zhang
- School of Psychological and Cognitive Sciences, Peking University, No. 52, Haidian Road, Haidian District, Beijing 100805, China; School of Educational Science, Minnan Normal University, No. 36, Xianqianzhi Street, Zhangzhou 363000, China
| | - Yuji Naya
- School of Psychological and Cognitive Sciences, Peking University, No. 52, Haidian Road, Haidian District, Beijing 100805, China; IDG/McGovern Institute for Brain Research at Peking University, No. 52, Haidian Road, Haidian District, Beijing 100805, China; Center for Life Sciences, Peking University, No. 52, Haidian Road, Haidian District, Beijing 100805, China; Beijing Key Laboratory of Behavior and Mental Health, Peking University, No. 52, Haidian Road, Haidian District, Beijing 100805, China.
| |
Collapse
|
9
|
Han JK, Oh J, Yun GJ, Yoo D, Kim MS, Yu JM, Choi SY, Choi YK. Cointegration of single-transistor neurons and synapses by nanoscale CMOS fabrication for highly scalable neuromorphic hardware. SCIENCE ADVANCES 2021; 7:7/32/eabg8836. [PMID: 34348898 PMCID: PMC8336957 DOI: 10.1126/sciadv.abg8836] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/03/2021] [Accepted: 06/17/2021] [Indexed: 05/03/2023]
Abstract
Cointegration of multistate single-transistor neurons and synapses was demonstrated for highly scalable neuromorphic hardware, using nanoscale complementary metal-oxide semiconductor (CMOS) fabrication. The neurons and synapses were integrated on the same plane with the same process because they have the same structure of a metal-oxide semiconductor field-effect transistor with different functions such as homotype. By virtue of 100% CMOS compatibility, it was also realized to cointegrate the neurons and synapses with additional CMOS circuits. Such cointegration can enhance packing density, reduce chip cost, and simplify fabrication procedures. The multistate single-transistor neuron that can control neuronal inhibition and the firing threshold voltage was achieved for an energy-efficient and reliable neural network. Spatiotemporal neuronal functionalities are demonstrated with fabricated single-transistor neurons and synapses. Image processing for letter pattern recognition and face image recognition is performed using experimental-based neuromorphic simulation.
Collapse
Affiliation(s)
- Joon-Kyu Han
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| | - Jungyeop Oh
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| | - Gyeong-Jun Yun
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| | - Dongeun Yoo
- National Nanofab Center (NNFC), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| | - Myung-Su Kim
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| | - Ji-Man Yu
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| | - Sung-Yool Choi
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| | - Yang-Kyu Choi
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea.
| |
Collapse
|
10
|
Schwenk JCB, Klingenhoefer S, Werner BO, Dowiasch S, Bremmer F. Perisaccadic encoding of temporal information in macaque area V4. J Neurophysiol 2021; 125:785-795. [PMID: 33502931 DOI: 10.1152/jn.00387.2020] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
The accurate processing of temporal information is of critical importance in everyday life. Yet, psychophysical studies in humans have shown that the perception of time is distorted around saccadic eye movements. The neural correlates of this misperception are still poorly understood. Behavioral and neural evidence suggest that it is tightly linked to other known perisaccadic modulations of visual perception. To further our understanding of how temporal processing is affected by saccades, we studied the representations of brief visual time intervals during fixation and saccades in area V4 of two awake macaques. We presented random sequences of vertical bar stimuli and extracted neural responses to double-pulse stimulation at varying interstimulus intervals. Our results show that temporal information about very brief intervals of as brief as 20 ms is reliably represented in the multiunit activity in area V4. Response latencies were not systematically modulated by the saccade. However, a general increase in perisaccadic activity altered the ratio of response amplitudes within stimulus pairs compared with fixation. In line with previous studies showing that the perception of brief time intervals is partly based on response levels, this may be seen as a possible correlate of the perisaccadic misperception of time.NEW & NOTEWORTHY We investigated for the first time how temporal information on very brief timescales is represented in area V4 around the time of saccadic eye movements. Overall, the responses showed an unexpectedly precise representation of time intervals. Our finding of a perisaccadic modulation of relative response amplitudes introduces a new possible correlate of saccade-related perceptual distortions of time.
Collapse
Affiliation(s)
- Jakob C B Schwenk
- Department of Neurophysics, Philipps-Universität Marburg, Marburg, Germany.,Center for Mind, Brain and Behavior (CMBB), Philipps-Universität Marburg and Justus-Liebig-University Giessen, Germany
| | | | - Björn-Olaf Werner
- Department of Neurophysics, Philipps-Universität Marburg, Marburg, Germany
| | - Stefan Dowiasch
- Department of Neurophysics, Philipps-Universität Marburg, Marburg, Germany.,Center for Mind, Brain and Behavior (CMBB), Philipps-Universität Marburg and Justus-Liebig-University Giessen, Germany
| | - Frank Bremmer
- Department of Neurophysics, Philipps-Universität Marburg, Marburg, Germany
| |
Collapse
|
11
|
Neupane S, Guitton D, Pack CC. Perisaccadic remapping: What? How? Why? Rev Neurosci 2020; 31:505-520. [PMID: 32242834 DOI: 10.1515/revneuro-2019-0097] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2019] [Accepted: 12/31/2019] [Indexed: 11/15/2022]
Abstract
About 25 years ago, the discovery of receptive field (RF) remapping in the parietal cortex of nonhuman primates revealed that visual RFs, widely assumed to have a fixed retinotopic organization, can change position before every saccade. Measuring such changes can be deceptively difficult. As a result, studies that followed have generated a fascinating but somewhat confusing picture of the phenomenon. In this review, we describe how observations of RF remapping depend on the spatial and temporal sampling of visual RFs and saccade directions. Further, we summarize some of the theories of how remapping might occur in neural circuitry. Finally, based on neurophysiological and psychophysical observations, we discuss the ways in which remapping information might facilitate computations in downstream brain areas.
Collapse
Affiliation(s)
- Sujaya Neupane
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Daniel Guitton
- Department of Neurology and Neurosurgery, McGill University, Montreal, Quebec H3A2B4, Canada
| | - Christopher C Pack
- Department of Neurology and Neurosurgery, McGill University, Montreal, Quebec H3A2B4, Canada
| |
Collapse
|
12
|
Spiking neurons with spatiotemporal dynamics and gain modulation for monolithically integrated memristive neural networks. Nat Commun 2020; 11:3399. [PMID: 32636385 PMCID: PMC7341810 DOI: 10.1038/s41467-020-17215-3] [Citation(s) in RCA: 56] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2020] [Accepted: 06/15/2020] [Indexed: 11/18/2022] Open
Abstract
As a key building block of biological cortex, neurons are powerful information processing units and can achieve highly complex nonlinear computations even in individual cells. Hardware implementation of artificial neurons with similar capability is of great significance for the construction of intelligent, neuromorphic systems. Here, we demonstrate an artificial neuron based on NbOx volatile memristor that not only realizes traditional all-or-nothing, threshold-driven spiking and spatiotemporal integration, but also enables dynamic logic including XOR function that is not linearly separable and multiplicative gain modulation among different dendritic inputs, therefore surpassing neuronal functions described by a simple point neuron model. A monolithically integrated 4 × 4 fully memristive neural network consisting of volatile NbOx memristor based neurons and nonvolatile TaOx memristor based synapses in a single crossbar array is experimentally demonstrated, showing capability in pattern recognition through online learning using a simplified δ-rule and coincidence detection, which paves the way for bio-inspired intelligent systems. Designing energy efficient and scalable artificial networks for neuromorphic computing remains a challenge. Here, the authors demonstrate online learning in a monolithically integrated 4 × 4 fully memristive neural network consisting of volatile NbOx memristor neurons and nonvolatile TaOx memristor synapses.
Collapse
|
13
|
Li S, Zhu H, Tian X. Corollary Discharge Versus Efference Copy: Distinct Neural Signals in Speech Preparation Differentially Modulate Auditory Responses. Cereb Cortex 2020; 30:5806-5820. [PMID: 32542347 DOI: 10.1093/cercor/bhaa154] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2020] [Revised: 05/15/2020] [Accepted: 05/16/2020] [Indexed: 11/14/2022] Open
Abstract
Actions influence sensory processing in a complex way to shape behavior. For example, during actions, a copy of motor signals-termed "corollary discharge" (CD) or "efference copy" (EC)-can be transmitted to sensory regions and modulate perception. However, the sole inhibitory function of the motor copies is challenged by mixed empirical observations as well as multifaceted computational demands for behaviors. We hypothesized that the content in the motor signals available at distinct stages of actions determined the nature of signals (CD vs. EC) and constrained their modulatory functions on perceptual processing. We tested this hypothesis using speech in which we could precisely control and quantify the course of action. In three electroencephalography (EEG) experiments using a novel delayed articulation paradigm, we found that preparation without linguistic contents suppressed auditory responses to all speech sounds, whereas preparing to speak a syllable selectively enhanced the auditory responses to the prepared syllable. A computational model demonstrated that a bifurcation of motor signals could be a potential algorithm and neural implementation to achieve the distinct functions in the motor-to-sensory transformation. These results suggest that distinct motor signals are generated in the motor-to-sensory transformation and integrated with sensory input to modulate perception.
Collapse
Affiliation(s)
- Siqi Li
- Shanghai Key Laboratory of Brain Functional Genomics (Ministry of Education), School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China.,NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai 200062, China
| | - Hao Zhu
- NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai 200062, China.,Division of Arts and Sciences, New York University Shanghai, Shanghai 200122, China
| | - Xing Tian
- NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai 200062, China.,Division of Arts and Sciences, New York University Shanghai, Shanghai 200122, China
| |
Collapse
|
14
|
Murdison TS, Blohm G, Bremmer F. Saccade-induced changes in ocular torsion reveal predictive orientation perception. J Vis 2020; 19:10. [PMID: 31533148 DOI: 10.1167/19.11.10] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Natural orienting of gaze often results in a retinal image that is rotated relative to space due to ocular torsion. However, we perceive neither this rotation nor a moving world despite visual rotational motion on the retina. This perceptual stability is often attributed to the phenomenon known as predictive remapping, but the current remapping literature ignores this torsional component. In addition, studies often simply measure remapping across either space or features (e.g., orientation) but in natural circumstances, both components are bound together for stable perception. One natural circumstance in which the perceptual system must account for the current and future eye orientation to correctly interpret the orientation of external stimuli occurs during movements to or from oblique eye orientations (i.e., eye orientations with both a horizontal and vertical angular component relative to the primary position). Here we took advantage of oblique eye orientation-induced ocular torsion to examine perisaccadic orientation perception. First, we found that orientation perception was largely predicted by the rotated retinal image. Second, we observed a presaccadic remapping of orientation perception consistent with maintaining a stable (but spatially inaccurate) retinocentric perception throughout the saccade. These findings strongly suggest that our seamless perceptual stability relies on retinocentric signals that are predictively remapped in all three ocular dimensions with each saccade.
Collapse
Affiliation(s)
- T Scott Murdison
- Centre for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada.,Canadian Action and Perception Network (CAPnet), Toronto, Ontario, Canada.,Association for Canadian Neuroinformatics and Computational Neuroscience (CNCN), Kingston, Ontario, Canada
| | - Gunnar Blohm
- Centre for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada.,Canadian Action and Perception Network (CAPnet), Toronto, Ontario, Canada.,Association for Canadian Neuroinformatics and Computational Neuroscience (CNCN), Kingston, Ontario, Canada
| | - Frank Bremmer
- Department of Neurophysics, Philipps-Universität Marburg, Germany
| |
Collapse
|
15
|
Ten Brink AF, Fabius JH, Weaver NA, Nijboer TC, Van der Stigchel S. Trans-saccadic memory after right parietal brain damage. Cortex 2019; 120:284-297. [DOI: 10.1016/j.cortex.2019.06.006] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2019] [Revised: 04/24/2019] [Accepted: 06/12/2019] [Indexed: 10/26/2022]
|
16
|
Zhou Y, Liu Y, Wu S, Zhang M. Neuronal Representation of the Saccadic Timing Signals in Macaque Lateral Intraparietal Area. Cereb Cortex 2019; 28:2887-2900. [PMID: 28968649 DOI: 10.1093/cercor/bhx166] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2016] [Accepted: 06/15/2017] [Indexed: 11/13/2022] Open
Abstract
Primates frequently make saccades direct fovea on interesting objects to receive acute visual information. However, saccade displaces the images on retina and disrupts the visual constancy. One possible mechanism to retain visual constancy is by integrating the presaccadic and postsaccadic visual information right at the time of saccade, which makes the timing of saccade crucial. So far, the saccadic timing signals have been found only in the subcortical regions, for example, the cerebellum and superior colliculus, but not in the neocortex. Here we report 2 types of saccadic timing signals in macaque lateral intraparietal area (LIP). First, many presaccadic response neurons started to decline activity either right around the start (saccade-on-decay) or the end (saccade-off-decay) of saccades. Notably, the time difference between saccade-off-decay and saccade-on-decay was highly correlated with the mean duration of saccades but not with the individual ones, and both saccade-off-decay and saccade-on-decay were better aligned with saccade end than saccade start-reflecting prediction. Second, the peak activity plateau of a group of postsaccadic response neurons was highly correlated with the actual duration of saccade-reflecting reality. While the predicted timing signals might facilitate the integration of visual information across saccades in LIP, the actual duration signals might calibrate the prediction errors.
Collapse
Affiliation(s)
- Yang Zhou
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China.,Institute of Neuroscience, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, and University of Chinese Academy of Sciences, Shanghai, China.,Department of Neurobiology, The University of Chicago, Chicago, IL, USA
| | - Yining Liu
- The First Affiliated Hospital of Zhengzhou University, Henan, China
| | - Si Wu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
| | - Mingsha Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
| |
Collapse
|
17
|
Abstract
Our vision depends upon shifting our high-resolution fovea to objects of interest in the visual field. Each saccade displaces the image on the retina, which should produce a chaotic scene with jerks occurring several times per second. It does not. This review examines how an internal signal in the primate brain (a corollary discharge) contributes to visual continuity across saccades. The article begins with a review of evidence for a corollary discharge in the monkey and evidence from inactivation experiments that it contributes to perception. The next section examines a specific neuronal mechanism for visual continuity, based on corollary discharge that is referred to as visual remapping. Both the basic characteristics of this anticipatory remapping and the factors that control it are enumerated. The last section considers hypotheses relating remapping to the perceived visual continuity across saccades, including remapping's contribution to perceived visual stability across saccades.
Collapse
Affiliation(s)
- Robert H Wurtz
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland 20892-4435, USA;
| |
Collapse
|
18
|
No direction specific costs in trans-saccadic memory. Neuropsychologia 2019; 125:23-29. [DOI: 10.1016/j.neuropsychologia.2019.01.014] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2018] [Revised: 01/04/2019] [Accepted: 01/24/2019] [Indexed: 11/21/2022]
|
19
|
Abstract
Humans move their eyes several times per second, yet we perceive the outside world as continuous despite the sudden disruptions created by each eye movement. To date, the mechanism that the brain employs to achieve visual continuity across eye movements remains unclear. While it has been proposed that the oculomotor system quickly updates and informs the visual system about the upcoming eye movement, behavioral studies investigating the time course of this updating suggest the involvement of a slow mechanism, estimated to take more than 500 ms to operate effectively. This is a surprisingly slow estimate, because both the visual system and the oculomotor system process information faster. If spatiotopic updating is indeed this slow, it cannot contribute to perceptual continuity, because it is outside the temporal regime of typical oculomotor behavior. Here, we argue that the behavioral paradigms that have been used previously are suboptimal to measure the speed of spatiotopic updating. In this study, we used a fast gaze-contingent paradigm, using high phi as a continuous stimulus across eye movements. We observed fast spatiotopic updating within 150 ms after stimulus onset. The results suggest the involvement of a fast updating mechanism that predictively influences visual perception after an eye movement. The temporal characteristics of this mechanism are compatible with the rate at which saccadic eye movements are typically observed in natural viewing.
Collapse
|
20
|
Szinte M, Jonikaitis D, Rangelov D, Deubel H. Pre-saccadic remapping relies on dynamics of spatial attention. eLife 2018; 7:37598. [PMID: 30596475 PMCID: PMC6328271 DOI: 10.7554/elife.37598] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2018] [Accepted: 12/30/2018] [Indexed: 01/01/2023] Open
Abstract
Each saccade shifts the projections of the visual scene on the retina. It has been proposed that the receptive fields of neurons in oculomotor areas are predictively remapped to account for these shifts. While remapping of the whole visual scene seems prohibitively complex, selection by attention may limit these processes to a subset of attended locations. Because attentional selection consumes time, remapping of attended locations should evolve in time, too. In our study, we cued a spatial location by presenting an attention-capturing cue at different times before a saccade and constructed maps of attentional allocation across the visual field. We observed no remapping of attention when the cue appeared shortly before saccade. In contrast, when the cue appeared sufficiently early before saccade, attentional resources were reallocated precisely to the remapped location. Our results show that pre-saccadic remapping takes time to develop suggesting that it relies on the spatial and temporal dynamics of spatial attention.
Collapse
Affiliation(s)
- Martin Szinte
- Department of Cognitive Psychology, Vrije Universiteit, Amsterdam, The Netherlands
| | - Donatas Jonikaitis
- Department of Neurobiology, Howard Hughes Medical Institute, Stanford University School of Medicine, Stanford, United States
| | - Dragan Rangelov
- Queensland Brain Institute, The University of Queensland, Brisbane, Australia
| | - Heiner Deubel
- Allgemeine und Experimentelle Psychologie, Ludwig-Maximilians-Universität München, Munich, Germany
| |
Collapse
|
21
|
Abstract
The perceptual consequences of eye movements are manifold: Each large saccade is accompanied by a drop of sensitivity to luminance-contrast, low-frequency stimuli, impacting both conscious vision and involuntary responses, including pupillary constrictions. They also produce transient distortions of space, time, and number, which cannot be attributed to the mere motion on the retinae. All these are signs that the visual system evokes active processes to predict and counteract the consequences of saccades. We propose that a key mechanism is the reorganization of spatiotemporal visual fields, which transiently increases the temporal and spatial uncertainty of visual representations just before and during saccades. On one hand, this accounts for the spatiotemporal distortions of visual perception; on the other hand, it implements a mechanism for fusing pre- and postsaccadic stimuli. This, together with the active suppression of motion signals, ensures the stability and continuity of our visual experience.
Collapse
Affiliation(s)
- Paola Binda
- Department of Translational Research on New Technologies in Medicine and Surgery, University of Pisa, 56123 Pisa, Italy;,
- CNR Institute of Neuroscience, 56123 Pisa, Italy
| | - Maria Concetta Morrone
- Department of Translational Research on New Technologies in Medicine and Surgery, University of Pisa, 56123 Pisa, Italy;,
- IRCCS Fondazione Stella-Maris, Calambrone, 56128 Pisa, Italy
| |
Collapse
|
22
|
Chen X, Zirnsak M, Moore T. Dissonant Representations of Visual Space in Prefrontal Cortex during Eye Movements. Cell Rep 2018; 22:2039-2052. [PMID: 29466732 PMCID: PMC5850980 DOI: 10.1016/j.celrep.2018.01.078] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2017] [Revised: 10/30/2017] [Accepted: 01/25/2018] [Indexed: 11/25/2022] Open
Abstract
We used local field potentials (LFPs) and spikes to investigate representations of visual space in prefrontal cortex and the dynamics of those representations during eye movements. Spatial information contained in LFPs of the frontal eye field (FEF) was differentially distributed across frequencies, with a majority of that information being carried in alpha and high-gamma bands and minimal signal in the low-gamma band. During fixation, spatial information from alpha and high-gamma bands and spiking activity was robust across cortical layers. Receptive fields (RFs) derived from alpha and high-gamma bands were retinocentrically organized, and they were spatially correlated both with each other and with spiking RFs. However, alpha and high-gamma RFs probed before eye movements were dissociated. Whereas high-gamma and spiking RFs immediately converged toward the movement goal, alpha RFs remained largely unchanged during the initial probe response, but they converged later. These observations reveal possible mechanisms of dynamic spatial representations that underlie visual perception during eye movements.
Collapse
Affiliation(s)
- Xiaomo Chen
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA 94305, USA; Howard Hughes Medical Institute, USA.
| | - Marc Zirnsak
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA 94305, USA; Howard Hughes Medical Institute, USA
| | - Tirin Moore
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA 94305, USA; Howard Hughes Medical Institute, USA
| |
Collapse
|
23
|
Wang X, Wu Y, Zhang M, Wu S. Learning Peri-saccadic Remapping of Receptive Field from Experience in Lateral Intraparietal Area. Front Comput Neurosci 2017; 11:110. [PMID: 29249953 PMCID: PMC5715402 DOI: 10.3389/fncom.2017.00110] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2017] [Accepted: 11/14/2017] [Indexed: 11/13/2022] Open
Abstract
Our eyes move constantly at a frequency of 3-5 times per second. These movements, called saccades, induce the sweeping of visual images on the retina, yet we perceive the world as stable. It has been suggested that the brain achieves this visual stability via predictive remapping of neuronal receptive field (RF). A recent experimental study disclosed details of this remapping process in the lateral intraparietal area (LIP), that is, about the time of the saccade, the neuronal RF expands along the saccadic trajectory temporally, covering the current RF (CRF), the future RF (FRF), and the region the eye will sweep through during the saccade. A cortical wave (CW) model was also proposed, which attributes the RF remapping as a consequence of neural activity propagating in the cortex, triggered jointly by a visual stimulus and the corollary discharge (CD) signal responsible for the saccade. In this study, we investigate how this CW model is learned naturally from visual experiences at the development of the brain. We build a two-layer network, with one layer consisting of LIP neurons and the other superior colliculus (SC) neurons. Initially, neuronal connections are random and non-selective. A saccade will cause a static visual image to sweep through the retina passively, creating the effect of the visual stimulus moving in the opposite direction of the saccade. According to the spiking-time-dependent-plasticity rule, the connection path in the opposite direction of the saccade between LIP neurons and the connection path from SC to LIP are enhanced. Over many such visual experiences, the CW model is developed, which generates the peri-saccadic RF remapping in LIP as observed in the experiment.
Collapse
Affiliation(s)
- Xiao Wang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Yan Wu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Mingsha Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| | - Si Wu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| |
Collapse
|
24
|
Corollary Discharge and Oculomotor Proprioception: Two Mechanisms for Spatially Accurate Perception and Action. J Indian Inst Sci 2017. [DOI: 10.1007/s41745-017-0050-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
25
|
Halassa MM, Kastner S. Thalamic functions in distributed cognitive control. Nat Neurosci 2017; 20:1669-1679. [DOI: 10.1038/s41593-017-0020-1] [Citation(s) in RCA: 262] [Impact Index Per Article: 37.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2016] [Accepted: 08/27/2017] [Indexed: 01/08/2023]
|
26
|
Bremmer F, Churan J, Lappe M. Heading representations in primates are compressed by saccades. Nat Commun 2017; 8:920. [PMID: 29030557 PMCID: PMC5640607 DOI: 10.1038/s41467-017-01021-5] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2016] [Accepted: 08/13/2017] [Indexed: 01/06/2023] Open
Abstract
Perceptual illusions help to understand how sensory signals are decoded in the brain. Here we report that the opposite approach is also applicable, i.e., results from decoding neural activity from monkey extrastriate visual cortex correctly predict a hitherto unknown perceptual illusion in humans. We record neural activity from monkey medial superior temporal (MST) and ventral intraparietal (VIP) area during presentation of self-motion stimuli and concurrent reflexive eye movements. A heading-decoder performs veridically during slow eye movements. During fast eye movements (saccades), however, the decoder erroneously reports compression of heading toward straight ahead. Functional equivalents of macaque areas MST and VIP have been identified in humans, implying a perceptual correlate (illusion) of this perisaccadic decoding error. Indeed, a behavioral experiment in humans shows that perceived heading is perisaccadically compressed toward the direction of gaze. Response properties of primate areas MST and VIP are consistent with being the substrate of the newly described visual illusion.Macaque higher visual areas MST and VIP encode heading direction based on self-motion stimuli. Here the authors show that, while making saccades, the heading direction decoded from the neural responses is compressed toward straight-ahead, and independently demonstrate a perceptual illusion in humans based on this perisaccadic decoding error.
Collapse
Affiliation(s)
- Frank Bremmer
- Department of Neurophysics & Marburg Center for Mind, Brain and Behavior - MCMBB, Philipps-Universität Marburg, Karl-von-Frisch Straße 8a, 35043, Marburg, Germany.
| | - Jan Churan
- Department of Neurophysics & Marburg Center for Mind, Brain and Behavior - MCMBB, Philipps-Universität Marburg, Karl-von-Frisch Straße 8a, 35043, Marburg, Germany
| | - Markus Lappe
- Department of Psychology & Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Muenster, Fliednerstraße 21, 48149, Münster, Germany
| |
Collapse
|
27
|
Coherent alpha oscillations link current and future receptive fields during saccades. Proc Natl Acad Sci U S A 2017; 114:E5979-E5985. [PMID: 28673993 DOI: 10.1073/pnas.1701672114] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Oscillations are ubiquitous in the brain, and they can powerfully influence neural coding. In particular, when oscillations at distinct sites are coherent, they provide a means of gating the flow of neural signals between different cortical regions. Coherent oscillations also occur within individual brain regions, although the purpose of this coherence is not well understood. Here, we report that within a single brain region, coherent alpha oscillations link stimulus representations as they change in space and time. Specifically, in primate cortical area V4, alpha coherence links sites that encode the retinal location of a visual stimulus before and after a saccade. These coherence changes exhibit properties similar to those of receptive field remapping, a phenomenon in which individual neurons change their receptive fields according to the metrics of each saccade. In particular, alpha coherence, like remapping, is highly dependent on the saccade vector and the spatial arrangement of current and future receptive fields. Moreover, although visual stimulation plays a modulatory role, it is neither necessary nor sufficient to elicit alpha coherence. Indeed, a similar pattern of coherence is observed even when saccades are made in darkness. Together, these results show that the pattern of alpha coherence across the retinotopic map in V4 matches many of the properties of receptive field remapping. Thus, oscillatory coherence might play a role in constructing the stable representation of visual space that is an essential aspect of conscious perception.
Collapse
|
28
|
Nakajima M, Halassa MM. Thalamic control of functional cortical connectivity. Curr Opin Neurobiol 2017; 44:127-131. [PMID: 28486176 DOI: 10.1016/j.conb.2017.04.001] [Citation(s) in RCA: 113] [Impact Index Per Article: 16.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2016] [Revised: 03/18/2017] [Accepted: 04/03/2017] [Indexed: 11/28/2022]
Abstract
The thalamus is an evolutionarily conserved structure with extensive reciprocal connections to cortical regions. While its role in transmitting sensory signals is well-studied, its broader engagement in cognition is unclear. In this review, we discuss evidence that the thalamus regulates functional connectivity within and between cortical regions, determining how a cognitive process is implemented across distributed cortical microcircuits. Within this framework, thalamic circuits do not necessarily determine the categorical content of a cognitive process (e.g., sensory details in feature-based attention), but rather provide a route by which task-relevant cortical representations are sustained and coordinated. Additionally, thalamic control of cortical connectivity bridges general arousal to the specific processing of categorical content, providing an intermediate level of cognitive and circuit description that will facilitate mapping neural computations onto thought and behavior.
Collapse
Affiliation(s)
- Miho Nakajima
- NYU Neuroscience Institute, Department of Neuroscience and Physiology, NYU Langone Medical Center, New York, NY 10016, United States
| | - Michael M Halassa
- NYU Neuroscience Institute, Department of Neuroscience and Physiology, NYU Langone Medical Center, New York, NY 10016, United States; Center for Neural Science, New York University, New York, NY 10003, United States.
| |
Collapse
|
29
|
Sun LD, Goldberg ME. Corollary Discharge and Oculomotor Proprioception: Cortical Mechanisms for Spatially Accurate Vision. Annu Rev Vis Sci 2016; 2:61-84. [PMID: 28532350 PMCID: PMC5691365 DOI: 10.1146/annurev-vision-082114-035407] [Citation(s) in RCA: 37] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
A classic problem in psychology is understanding how the brain creates a stable and accurate representation of space for perception and action despite a constantly moving eye. Two mechanisms have been proposed to solve this problem: Herman von Helmholtz's idea that the brain uses a corollary discharge of the motor command that moves the eye to adjust the visual representation, and Sir Charles Sherrington's idea that the brain measures eye position to calculate a spatial representation. Here, we discuss the cognitive, neuropsychological, and physiological mechanisms that support each of these ideas. We propose that both are correct: A rapid corollary discharge signal remaps the visual representation before an impending saccade, computing accurate movement vectors; and an oculomotor proprioceptive signal enables the brain to construct a more accurate craniotopic representation of space that develops slowly after the saccade.
Collapse
Affiliation(s)
- Linus D Sun
- Mahoney-Keck Center for Brain and Behavior Research, Department of Neuroscience, Columbia University College of Physicians and Surgeons, New York, NY 10032;
- Department of Neuroscience, Columbia University College of Physicians and Surgeons, New York, NY 10032
- Department of Ophthalmology, Columbia University College of Physicians and Surgeons, New York, NY 10032
- Division of Neurobiology and Behavior, New York State Psychiatric Institute, New York, NY 10032
| | - Michael E Goldberg
- Mahoney-Keck Center for Brain and Behavior Research, Department of Neuroscience, Columbia University College of Physicians and Surgeons, New York, NY 10032;
- Department of Neuroscience, Columbia University College of Physicians and Surgeons, New York, NY 10032
- Department of Neurology, Columbia University College of Physicians and Surgeons, New York, NY 10032
- Department of Psychiatry, Columbia University College of Physicians and Surgeons, New York, NY 10032
- Department of Ophthalmology, Columbia University College of Physicians and Surgeons, New York, NY 10032
- Kavli Institute for Neuroscience, Columbia University, New York, NY 10032
- Division of Neurobiology and Behavior, New York State Psychiatric Institute, New York, NY 10032
| |
Collapse
|
30
|
Rao HM, Mayo JP, Sommer MA. Circuits for presaccadic visual remapping. J Neurophysiol 2016; 116:2624-2636. [PMID: 27655962 DOI: 10.1152/jn.00182.2016] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2016] [Accepted: 09/14/2016] [Indexed: 01/08/2023] Open
Abstract
Saccadic eye movements rapidly displace the image of the world that is projected onto the retinas. In anticipation of each saccade, many neurons in the visual system shift their receptive fields. This presaccadic change in visual sensitivity, known as remapping, was first documented in the parietal cortex and has been studied in many other brain regions. Remapping requires information about upcoming saccades via corollary discharge. Analyses of neurons in a corollary discharge pathway that targets the frontal eye field (FEF) suggest that remapping may be assembled in the FEF's local microcircuitry. Complementary data from reversible inactivation, neural recording, and modeling studies provide evidence that remapping contributes to transsaccadic continuity of action and perception. Multiple forms of remapping have been reported in the FEF and other brain areas, however, and questions remain about the reasons for these differences. In this review of recent progress, we identify three hypotheses that may help to guide further investigations into the structure and function of circuits for remapping.
Collapse
Affiliation(s)
- Hrishikesh M Rao
- Department of Biomedical Engineering, Pratt School of Engineering, Duke University, Durham, North Carolina;
| | - J Patrick Mayo
- Department of Neurobiology, Duke School of Medicine, Duke University, Durham, North Carolina; and
| | - Marc A Sommer
- Department of Biomedical Engineering, Pratt School of Engineering, Duke University, Durham, North Carolina.,Department of Neurobiology, Duke School of Medicine, Duke University, Durham, North Carolina; and.,Center for Cognitive Neuroscience, Duke University, Durham, North Carolina
| |
Collapse
|
31
|
Rao HM, San Juan J, Shen FY, Villa JE, Rafie KS, Sommer MA. Neural Network Evidence for the Coupling of Presaccadic Visual Remapping to Predictive Eye Position Updating. Front Comput Neurosci 2016; 10:52. [PMID: 27313528 PMCID: PMC4889583 DOI: 10.3389/fncom.2016.00052] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2016] [Accepted: 05/18/2016] [Indexed: 11/13/2022] Open
Abstract
As we look around a scene, we perceive it as continuous and stable even though each saccadic eye movement changes the visual input to the retinas. How the brain achieves this perceptual stabilization is unknown, but a major hypothesis is that it relies on presaccadic remapping, a process in which neurons shift their visual sensitivity to a new location in the scene just before each saccade. This hypothesis is difficult to test in vivo because complete, selective inactivation of remapping is currently intractable. We tested it in silico with a hierarchical, sheet-based neural network model of the visual and oculomotor system. The model generated saccadic commands to move a video camera abruptly. Visual input from the camera and internal copies of the saccadic movement commands, or corollary discharge, converged at a map-level simulation of the frontal eye field (FEF), a primate brain area known to receive such inputs. FEF output was combined with eye position signals to yield a suitable coordinate frame for guiding arm movements of a robot. Our operational definition of perceptual stability was "useful stability," quantified as continuously accurate pointing to a visual object despite camera saccades. During training, the emergence of useful stability was correlated tightly with the emergence of presaccadic remapping in the FEF. Remapping depended on corollary discharge but its timing was synchronized to the updating of eye position. When coupled to predictive eye position signals, remapping served to stabilize the target representation for continuously accurate pointing. Graded inactivations of pathways in the model replicated, and helped to interpret, previous in vivo experiments. The results support the hypothesis that visual stability requires presaccadic remapping, provide explanations for the function and timing of remapping, and offer testable hypotheses for in vivo studies. We conclude that remapping allows for seamless coordinate frame transformations and quick actions despite visual afferent lags. With visual remapping in place for behavior, it may be exploited for perceptual continuity.
Collapse
Affiliation(s)
- Hrishikesh M Rao
- Department of Biomedical Engineering, Pratt School of Engineering, Duke University Durham, NC, USA
| | - Juan San Juan
- Department of Biomedical Engineering, Pratt School of Engineering, Duke University Durham, NC, USA
| | - Fred Y Shen
- Department of Biomedical Engineering, Pratt School of Engineering, Duke University Durham, NC, USA
| | - Jennifer E Villa
- Department of Biomedical Engineering, Pratt School of Engineering, Duke University Durham, NC, USA
| | - Kimia S Rafie
- Department of Biomedical Engineering, Pratt School of Engineering, Duke University Durham, NC, USA
| | - Marc A Sommer
- Department of Biomedical Engineering, Pratt School of Engineering, Duke UniversityDurham, NC, USA; Department of Neurobiology, Duke School of Medicine, Duke UniversityDurham, NC, USA; Center for Cognitive Neuroscience, Duke UniversityDurham, NC, USA
| |
Collapse
|