1
|
Wen P, Landy MS, Rokers B. Identifying cortical areas that underlie the transformation from 2D retinal to 3D head-centric motion signals. Neuroimage 2023; 270:119909. [PMID: 36801370 PMCID: PMC10061442 DOI: 10.1016/j.neuroimage.2023.119909] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2022] [Revised: 01/26/2023] [Accepted: 01/28/2023] [Indexed: 02/18/2023] Open
Abstract
Accurate motion perception requires that the visual system integrate the 2D retinal motion signals received by the two eyes into a single representation of 3D motion. However, most experimental paradigms present the same stimulus to the two eyes, signaling motion limited to a 2D fronto-parallel plane. Such paradigms are unable to dissociate the representation of 3D head-centric motion signals (i.e., 3D object motion relative to the observer) from the associated 2D retinal motion signals. Here, we used stereoscopic displays to present separate motion signals to the two eyes and examined their representation in visual cortex using fMRI. Specifically, we presented random-dot motion stimuli that specified various 3D head-centric motion directions. We also presented control stimuli, which matched the motion energy of the retinal signals, but were inconsistent with any 3D motion direction. We decoded motion direction from BOLD activity using a probabilistic decoding algorithm. We found that 3D motion direction signals can be reliably decoded in three major clusters in the human visual system. Critically, in early visual cortex (V1-V3), we found no significant difference in decoding performance between stimuli specifying 3D motion directions and the control stimuli, suggesting that these areas represent the 2D retinal motion signals, rather than 3D head-centric motion itself. In voxels in and surrounding hMT and IPS0 however, decoding performance was consistently superior for stimuli that specified 3D motion directions compared to control stimuli. Our results reveal the parts of the visual processing hierarchy that are critical for the transformation of retinal into 3D head-centric motion signals and suggest a role for IPS0 in their representation, in addition to its sensitivity to 3D object structure and static depth.
Collapse
Affiliation(s)
- Puti Wen
- Psychology, New York University Abu Dhabi, United Arab Emirates.
| | - Michael S Landy
- Department of Psychology and Center for Neural Science, New York University, United States
| | - Bas Rokers
- Psychology, New York University Abu Dhabi, United Arab Emirates; Department of Psychology and Center for Neural Science, New York University, United States
| |
Collapse
|
2
|
Duan Y, Thatte J, Yaklovleva A, Norcia AM. Disparity in Context: Understanding how monocular image content interacts with disparity processing in human visual cortex. Neuroimage 2021; 237:118139. [PMID: 33964460 PMCID: PMC10786599 DOI: 10.1016/j.neuroimage.2021.118139] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2020] [Revised: 04/16/2021] [Accepted: 04/19/2021] [Indexed: 11/24/2022] Open
Abstract
Horizontal disparities between the two eyes' retinal images are the primary cue for depth. Commonly used random ot tereograms (RDS) intentionally camouflage the disparity cue, breaking the correlations between monocular image structure and the depth map that are present in natural images. Because of the nonlinear nature of visual processing, it is unlikely that simple computational rules derived from RDS will be sufficient to explain binocular vision in natural environments. In order to understand the interplay between natural scene structure and disparity encoding, we used a depth-image-based-rendering technique and a library of natural 3D stereo pairs to synthesize two novel stereogram types in which monocular scene content was manipulated independent of scene depth information. The half-images of the novel stereograms comprised either random-dots or scrambled natural scenes, each with the same depth maps as the corresponding natural scene stereograms. Using these stereograms in a simultaneous Event-Related Potential and behavioral discrimination task, we identified multiple disparity-contingent encoding stages between 100 ~ 500 msec. The first disparity sensitive evoked potential was observed at ~100 msec after an earlier evoked potential (between ~50-100 msec) that was sensitive to the structure of the monocular half-images but blind to disparity. Starting at ~150 msec, disparity responses were stereogram-specific and predictive of perceptual depth. Complex features associated with natural scene content are thus at least partially coded prior to disparity information, but these features and possibly others associated with natural scene content interact with disparity information only after an intermediate, 2D scene-independent disparity processing stage.
Collapse
Affiliation(s)
- Yiran Duan
- Wu Tsai Neurosciences Institute, 290 Jane Stanford Way, Stanford, CA 94305
| | - Jayant Thatte
- Department of Electrical Engineering, David Packard Building, Stanford University, 350 Jane Stanford Way, Stanford, CA 94305
| | | | - Anthony M Norcia
- Wu Tsai Neurosciences Institute, 290 Jane Stanford Way, Stanford, CA 94305.
| |
Collapse
|
3
|
Russ BE, Petkov CI, Kwok SC, Zhu Q, Belin P, Vanduffel W, Hamed SB. Common functional localizers to enhance NHP & cross-species neuroscience imaging research. Neuroimage 2021; 237:118203. [PMID: 34048898 PMCID: PMC8529529 DOI: 10.1016/j.neuroimage.2021.118203] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2020] [Revised: 05/15/2021] [Accepted: 05/24/2021] [Indexed: 11/25/2022] Open
Abstract
Functional localizers are invaluable as they can help define regions of interest, provide cross-study comparisons, and most importantly, allow for the aggregation and meta-analyses of data across studies and laboratories. To achieve these goals within the non-human primate (NHP) imaging community, there is a pressing need for the use of standardized and validated localizers that can be readily implemented across different groups. The goal of this paper is to provide an overview of the value of localizer protocols to imaging research and we describe a number of commonly used or novel localizers within NHPs, and keys to implement them across studies. As has been shown with the aggregation of resting-state imaging data in the original PRIME-DE submissions, we believe that the field is ready to apply the same initiative for task-based functional localizers in NHP imaging. By coming together to collect large datasets across research group, implementing the same functional localizers, and sharing the localizers and data via PRIME-DE, it is now possible to fully test their robustness, selectivity and specificity. To do this, we reviewed a number of common localizers and we created a repository of well-established localizer that are easily accessible and implemented through the PRIME-RE platform.
Collapse
Affiliation(s)
- Brian E Russ
- Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, United States; Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York City, NY, United States; Department of Psychiatry, New York University at Langone, New York City, NY, United States.
| | - Christopher I Petkov
- Biosciences Institute, Newcastle University Medical School, Newcastle upon Tyne, United Kingdom
| | - Sze Chai Kwok
- Shanghai Key Laboratory of Brain Functional Genomics, Key Laboratory of Brain Functional Genomics Ministry of Education, Shanghai Key Laboratory of Magnetic Resonance, Affiliated Mental Health Center (ECNU), School of Psychology and Cognitive Science, East China Normal University, Shanghai, China; Division of Natural and Applied Sciences, Duke Kunshan University, Kunshan, Jiangsu, China; NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai, China
| | - Qi Zhu
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin Center, 91191 Gif/Yvette, France; Laboratory for Neuro-and Psychophysiology, Department of Neurosciences, KU Leuven Medical School, Leuven, 3000, Belgium
| | - Pascal Belin
- Institut de Neurosciences de La Timone, Aix-Marseille Université et CNRS, Marseille, 13005, France
| | - Wim Vanduffel
- Laboratory for Neuro-and Psychophysiology, Department of Neurosciences, KU Leuven Medical School, Leuven, 3000, Belgium; Leuven Brain Institute, KU Leuven, Leuven, 3000, Belgium; Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA 02129, United States; Department of Radiology, Harvard Medical School, Boston, MA 02144, United States.
| | - Suliann Ben Hamed
- Institut des Sciences Cognitives Marc Jeannerod, UMR 5229, Université de Lyon - CNRS, France.
| |
Collapse
|
4
|
Li Z. Unique Neural Activity Patterns Among Lower Order Cortices and Shared Patterns Among Higher Order Cortices During Processing of Similar Shapes With Different Stimulus Types. Iperception 2021; 12:20416695211018222. [PMID: 34104383 PMCID: PMC8161881 DOI: 10.1177/20416695211018222] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2020] [Accepted: 04/28/2021] [Indexed: 11/16/2022] Open
Abstract
We investigated the neural mechanism of the processing of three-dimensional (3D) shapes defined by disparity and perspective. We measured blood oxygenation level-dependent signals as participants viewed and classified 3D images of convex-concave shapes. According to the cue (disparity or perspective) and element type (random dots or black and white dotted lines), three types of stimuli were used: random dot stereogram, black and white dotted lines with perspective, and black and white dotted lines with binocular disparity. The blood oxygenation level-dependent images were then classified by multivoxel pattern analysis. To identify areas selective to shape, we assessed convex-concave classification accuracy with classifiers trained and tested using signals evoked by the same stimulus type (same cue and element type). To identify cortical regions with similar neural activity patterns regardless of stimulus type, we assessed the convex-concave classification accuracy of transfer classification in which classifiers were trained and tested using different stimulus types (different cues or element types). Classification accuracy using the same stimulus type was high in the early visual areas and subregions of the intraparietal sulcus (IPS), whereas transfer classification accuracy was high in the dorsal subregions of the IPS. These results indicate that the early visual areas process the specific features of stimuli, whereas the IPS regions perform more generalized processing of 3D shapes, independent of a specific stimulus type.
Collapse
Affiliation(s)
- Zhen Li
- Department of Psychology, The University of Hong Kong, Hong Kong, China; Graduate School of Engineering, Kochi University of Technology, Kochi, Japan
| |
Collapse
|
5
|
Pelekanos V, Mok RM, Joly O, Ainsworth M, Kyriazis D, Kelly MG, Bell AH, Kriegeskorte N. Rapid event-related, BOLD fMRI, non-human primates (NHP): choose two out of three. Sci Rep 2020; 10:7485. [PMID: 32366956 PMCID: PMC7198564 DOI: 10.1038/s41598-020-64376-8] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2019] [Accepted: 04/15/2020] [Indexed: 12/03/2022] Open
Abstract
Human functional magnetic resonance imaging (fMRI) typically employs the blood-oxygen-level-dependent (BOLD) contrast mechanism. In non-human primates (NHP), contrast enhancement is possible using monocrystalline iron-oxide nanoparticles (MION) contrast agent, which has a more temporally extended response function. However, using BOLD fMRI in NHP is desirable for interspecies comparison, and the BOLD signal’s faster response function promises to be beneficial for rapid event-related (rER) designs. Here, we used rER BOLD fMRI in macaque monkeys while viewing real-world images, and found visual responses and category selectivity consistent with previous studies. However, activity estimates were very noisy, suggesting that the lower contrast-to-noise ratio of BOLD, suboptimal behavioural performance, and motion artefacts, in combination, render rER BOLD fMRI challenging in NHP. Previous studies have shown that rER fMRI is possible in macaques with MION, despite MION’s prolonged response function. To understand this, we conducted simulations of the BOLD and MION response during rER, and found that no matter how fast the design, the greater amplitude of the MION response outweighs the contrast loss caused by greater temporal smoothing. We conclude that although any two of the three elements (rER, BOLD, NHP) have been shown to work well, the combination of all three is particularly challenging.
Collapse
Affiliation(s)
- Vassilis Pelekanos
- Medical Research Council, Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK. .,Department of Experimental Psychology, University of Oxford, Oxford, UK. .,School of Medicine, University of Nottingham, Nottingham, UK.
| | - Robert M Mok
- Medical Research Council, Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK.,Department of Experimental Psychology, University College London, London, UK
| | - Olivier Joly
- Medical Research Council, Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK.,Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Matthew Ainsworth
- Medical Research Council, Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK.,Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Diana Kyriazis
- Medical Research Council, Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK.,Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Maria G Kelly
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Andrew H Bell
- Medical Research Council, Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK.,Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Nikolaus Kriegeskorte
- Medical Research Council, Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK.,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, USA
| |
Collapse
|