1
|
Xiao W, Sharma S, Kreiman G, Livingstone MS. Feature-selective responses in macaque visual cortex follow eye movements during natural vision. Nat Neurosci 2024; 27:1157-1166. [PMID: 38684892 PMCID: PMC11156562 DOI: 10.1038/s41593-024-01631-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Accepted: 03/26/2024] [Indexed: 05/02/2024]
Abstract
In natural vision, primates actively move their eyes several times per second via saccades. It remains unclear whether, during this active looking, visual neurons exhibit classical retinotopic properties, anticipate gaze shifts or mirror the stable quality of perception, especially in complex natural scenes. Here, we let 13 monkeys freely view thousands of natural images across 4.6 million fixations, recorded 883 h of neuronal responses in six areas spanning primary visual to anterior inferior temporal cortex and analyzed spatial, temporal and featural selectivity in these responses. Face neurons tracked their receptive field contents, indicated by category-selective responses. Self-consistency analysis showed that general feature-selective responses also followed eye movements and remained gaze-dependent over seconds of viewing the same image. Computational models of feature-selective responses located retinotopic receptive fields during free viewing. We found limited evidence for feature-selective predictive remapping and no viewing-history integration. Thus, ventral visual neurons represent the world in a predominantly eye-centered reference frame during natural vision.
Collapse
Affiliation(s)
- Will Xiao
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA.
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA.
| | - Saloni Sharma
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Gabriel Kreiman
- Department of Ophthalmology, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA
| | | |
Collapse
|
2
|
Jiahui G, Feilong M, Visconti di Oleggio Castello M, Nastase SA, Haxby JV, Gobbini MI. Modeling naturalistic face processing in humans with deep convolutional neural networks. Proc Natl Acad Sci U S A 2023; 120:e2304085120. [PMID: 37847731 PMCID: PMC10614847 DOI: 10.1073/pnas.2304085120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 09/11/2023] [Indexed: 10/19/2023] Open
Abstract
Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The ways in which the internal face representations in DCNNs relate to human cognitive representations and brain activity are not well understood. Nearly all previous studies focused on static face image processing with rapid display times and ignored the processing of naturalistic, dynamic information. To address this gap, we developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces). We used this naturalistic dataset to compare representational geometries estimated from DCNNs, behavioral responses, and brain responses. We found that DCNN representational geometries were consistent across architectures, cognitive representational geometries were consistent across raters in a behavioral arrangement task, and neural representational geometries in face areas were consistent across brains. Representational geometries in late, fully connected DCNN layers, which are optimized for individuation, were much more weakly correlated with cognitive and neural geometries than were geometries in late-intermediate layers. The late-intermediate face-DCNN layers successfully matched cognitive representational geometries, as measured with a behavioral arrangement task that primarily reflected categorical attributes, and correlated with neural representational geometries in known face-selective topographies. Our study suggests that current DCNNs successfully capture neural cognitive processes for categorical attributes of faces but less accurately capture individuation and dynamic features.
Collapse
Affiliation(s)
- Guo Jiahui
- Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH03755
| | - Ma Feilong
- Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH03755
| | | | - Samuel A. Nastase
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ08544
| | - James V. Haxby
- Center for Cognitive Neuroscience, Dartmouth College, Hanover, NH03755
| | - M. Ida Gobbini
- Department of Medical and Surgical Sciences, University of Bologna, Bologna40138, Italy
- Istituti di Ricovero e Cura a Carattere Scientifico, Istituto delle Scienze Neurologiche di Bologna, Bologna40139, Italia
| |
Collapse
|
3
|
Yamane Y, Ito J, Joana C, Fujita I, Tamura H, Maldonado PE, Doya K, Grün S. Neuronal Population Activity in Macaque Visual Cortices Dynamically Changes through Repeated Fixations in Active Free Viewing. eNeuro 2023; 10:ENEURO.0086-23.2023. [PMID: 37798110 PMCID: PMC10591287 DOI: 10.1523/eneuro.0086-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Revised: 09/20/2023] [Accepted: 10/02/2023] [Indexed: 10/07/2023] Open
Abstract
During free viewing, we move our eyes and fixate on objects to recognize the visual scene of our surroundings. To investigate the neural representation of objects in this process, we studied individual and population neuronal activity in three different visual regions of the brains of macaque monkeys (Macaca fuscata): the primary and secondary visual cortices (V1, V2) and the inferotemporal cortex (IT). We designed a task where the animal freely selected objects in a stimulus image to fixate on while we examined the relationship between spiking activity, the order of fixations, and the fixated objects. We found that activity changed across repeated fixations on the same object in all three recorded areas, with observed reductions in firing rates. Furthermore, the responses of individual neurons became sparser and more selective with individual objects. The population activity for individual objects also became distinct. These results suggest that visual neurons respond dynamically to repeated input stimuli through a smaller number of spikes, thereby allowing for discrimination between individual objects with smaller energy.
Collapse
Affiliation(s)
- Yukako Yamane
- Graduate School of Frontier Biosciences, Osaka University, Osaka 565-0871, Japan
- Okinawa Institute of Science and Technology Graduate University, Okinawa 904-0495, Japan
| | - Junji Ito
- Institute of Neuroscience and Medicine (INM-6 and INM-10) and Institute for Advanced Simulation (IAS-6), Jülich Research Centre, Jülich 52425, Germany
| | - Cristian Joana
- Institute of Neuroscience and Medicine (INM-6 and INM-10) and Institute for Advanced Simulation (IAS-6), Jülich Research Centre, Jülich 52425, Germany
- CAS Key Laboratory of Theoretical Physics, Institute of Theoretical Physics, Chinese Academy of Sciences, Beijing 100190, China
| | - Ichiro Fujita
- Graduate School of Frontier Biosciences, Osaka University, Osaka 565-0871, Japan
- Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology, Osaka University, Osaka 565-0871, Japan
| | - Hiroshi Tamura
- Graduate School of Frontier Biosciences, Osaka University, Osaka 565-0871, Japan
- Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology, Osaka University, Osaka 565-0871, Japan
| | - Pedro E Maldonado
- Department of Neuroscience and Instituto de Neurosciencia Biomedica (BNI), Facultad de Medicina, Universidad de Chile, Santiago 8380453, Chile
| | - Kenji Doya
- Okinawa Institute of Science and Technology Graduate University, Okinawa 904-0495, Japan
| | - Sonja Grün
- Institute of Neuroscience and Medicine (INM-6 and INM-10) and Institute for Advanced Simulation (IAS-6), Jülich Research Centre, Jülich 52425, Germany
- Theoretical Systems Neurobiology, Rheinisch Westfaelische Technische Hochschule (RWTH) Aachen University, Aachen 52056, Germany
| |
Collapse
|
4
|
Yates JL, Coop SH, Sarch GH, Wu RJ, Butts DA, Rucci M, Mitchell JF. Detailed characterization of neural selectivity in free viewing primates. Nat Commun 2023; 14:3656. [PMID: 37339973 PMCID: PMC10282080 DOI: 10.1038/s41467-023-38564-9] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Accepted: 05/08/2023] [Indexed: 06/22/2023] Open
Abstract
Fixation constraints in visual tasks are ubiquitous in visual and cognitive neuroscience. Despite its widespread use, fixation requires trained subjects, is limited by the accuracy of fixational eye movements, and ignores the role of eye movements in shaping visual input. To overcome these limitations, we developed a suite of hardware and software tools to study vision during natural behavior in untrained subjects. We measured visual receptive fields and tuning properties from multiple cortical areas of marmoset monkeys who freely viewed full-field noise stimuli. The resulting receptive fields and tuning curves from primary visual cortex (V1) and area MT match reported selectivity from the literature which was measured using conventional approaches. We then combined free viewing with high-resolution eye tracking to make the first detailed 2D spatiotemporal measurements of foveal receptive fields in V1. These findings demonstrate the power of free viewing to characterize neural responses in untrained animals while simultaneously studying the dynamics of natural behavior.
Collapse
Affiliation(s)
- Jacob L Yates
- Brain and Cognitive Sciences, University of Rochester, Rochester, NY, USA.
- Center for Visual Science, University of Rochester, Rochester, NY, USA.
- Department of Biology and Program in Neuroscience and Cognitive Science, University of Maryland, College Park, MD, USA.
- Herbert Wertheim School of Optometry and Vision Science, UC Berkeley, Berkeley, CA, USA.
| | - Shanna H Coop
- Brain and Cognitive Sciences, University of Rochester, Rochester, NY, USA
- Center for Visual Science, University of Rochester, Rochester, NY, USA
- Neurobiology, Stanford University, Stanford, CA, USA
| | - Gabriel H Sarch
- Brain and Cognitive Sciences, University of Rochester, Rochester, NY, USA
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Ruei-Jr Wu
- Center for Visual Science, University of Rochester, Rochester, NY, USA
- Institute of Optics, University of Rochester, Rochester, NY, USA
| | - Daniel A Butts
- Department of Biology and Program in Neuroscience and Cognitive Science, University of Maryland, College Park, MD, USA
| | - Michele Rucci
- Brain and Cognitive Sciences, University of Rochester, Rochester, NY, USA
- Center for Visual Science, University of Rochester, Rochester, NY, USA
| | - Jude F Mitchell
- Brain and Cognitive Sciences, University of Rochester, Rochester, NY, USA
- Center for Visual Science, University of Rochester, Rochester, NY, USA
| |
Collapse
|
5
|
Kay K, Bonnen K, Denison RN, Arcaro MJ, Barack DL. Tasks and their role in visual neuroscience. Neuron 2023; 111:1697-1713. [PMID: 37040765 DOI: 10.1016/j.neuron.2023.03.022] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2023] [Revised: 03/13/2023] [Accepted: 03/15/2023] [Indexed: 04/13/2023]
Abstract
Vision is widely used as a model system to gain insights into how sensory inputs are processed and interpreted by the brain. Historically, careful quantification and control of visual stimuli have served as the backbone of visual neuroscience. There has been less emphasis, however, on how an observer's task influences the processing of sensory inputs. Motivated by diverse observations of task-dependent activity in the visual system, we propose a framework for thinking about tasks, their role in sensory processing, and how we might formally incorporate tasks into our models of vision.
Collapse
Affiliation(s)
- Kendrick Kay
- Center for Magnetic Resonance Research, Department of Radiology, University of Minnesota, Minneapolis, MN 55455, USA.
| | - Kathryn Bonnen
- School of Optometry, Indiana University, Bloomington, IN 47405, USA
| | - Rachel N Denison
- Department of Psychological and Brain Sciences, Boston University, Boston, MA 02215, USA
| | - Mike J Arcaro
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19146, USA
| | - David L Barack
- Departments of Neuroscience and Philosophy, University of Pennsylvania, Philadelphia, PA 19146, USA
| |
Collapse
|
6
|
Samara A, Eilbott J, Margulies DS, Xu T, Vanderwal T. Cortical gradients during naturalistic processing are hierarchical and modality-specific. Neuroimage 2023; 271:120023. [PMID: 36921679 DOI: 10.1016/j.neuroimage.2023.120023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2022] [Revised: 02/21/2023] [Accepted: 03/10/2023] [Indexed: 03/14/2023] Open
Abstract
Understanding cortical topographic organization and how it supports complex perceptual and cognitive processes is a fundamental question in neuroscience. Previous work has characterized functional gradients that demonstrate large-scale principles of cortical organization. How these gradients are modulated by rich ecological stimuli remains unknown. Here, we utilize naturalistic stimuli via movie-fMRI to assess macroscale functional organization. We identify principal movie gradients that delineate separate hierarchies anchored in sensorimotor, visual, and auditory/language areas. At the opposite/heteromodal end of these perception-to-cognition axes, we find a more central role for the frontoparietal network along with the default network. Even across different movie stimuli, movie gradients demonstrated good reliability, suggesting that these hierarchies reflect a brain state common across different naturalistic conditions. The relative position of brain areas within movie gradients showed stronger and more numerous correlations with cognitive behavioral scores compared to resting state gradients. Together, these findings provide an ecologically valid representation of the principles underlying cortical organization while the brain is active and engaged in multimodal, dynamic perceptual and cognitive processing.
Collapse
Affiliation(s)
- Ahmad Samara
- Department of Psychiatry, University of British Columbia, 2255 Wesbrook Mall, Vancouver, BC V6T 2A1, Canada
| | - Jeffrey Eilbott
- BC Children's Hospital Research Institute, Vancouver, BC V5Z 4H4, Canada
| | - Daniel S Margulies
- CNRS, Integrative Neuroscience and Cognition Center (UMR 8002), Université de Paris, Paris 75006, France
| | - Ting Xu
- Center for the Developing Brain, The Child Mind Institute, New York, NY 10022, USA
| | - Tamara Vanderwal
- Department of Psychiatry, University of British Columbia, 2255 Wesbrook Mall, Vancouver, BC V6T 2A1, Canada; BC Children's Hospital Research Institute, Vancouver, BC V5Z 4H4, Canada; Yale Child Study Center, Yale University, New Haven, CT 06519, USA.
| |
Collapse
|
7
|
Russ BE, Koyano KW, Day-Cooney J, Perwez N, Leopold DA. Temporal continuity shapes visual responses of macaque face patch neurons. Neuron 2023; 111:903-914.e3. [PMID: 36630962 PMCID: PMC10023462 DOI: 10.1016/j.neuron.2022.12.021] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 09/09/2022] [Accepted: 12/13/2022] [Indexed: 01/12/2023]
Abstract
Macaque inferior temporal cortex neurons respond selectively to complex visual images, with recent work showing that they are also entrained reliably by the evolving content of natural movies. To what extent does temporal continuity itself shape the responses of high-level visual neurons? We addressed this question by measuring how cells in face-selective regions of the macaque visual cortex were affected by the manipulation of a movie's temporal structure. Sampling a 5-min movie at 1 s intervals, we measured neural responses to randomized, brief stimuli of different lengths, ranging from 800 ms dynamic movie snippets to 100 ms static frames. We found that the disruption of temporal continuity strongly altered neural response profiles, particularly in the early response period after stimulus onset. The results suggest that models of visual system function based on discrete and randomized visual presentations may not translate well to the brain's natural modes of operation.
Collapse
Affiliation(s)
- Brian E Russ
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20814, USA; Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY 10962, USA; Nash Family Department of Neuroscience and Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Psychiatry, New York University at Langone, New York City, NY 10016, USA.
| | - Kenji W Koyano
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20814, USA
| | - Julian Day-Cooney
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20814, USA
| | - Neda Perwez
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20814, USA
| | - David A Leopold
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20814, USA; Neurophysiology Imaging Facility, National Institute of Mental Health, National Institute of Neurological Disorders and Stroke, National Eye Institute, Bethesda, MD 20814, USA
| |
Collapse
|
8
|
Encoding of 3D physical dimensions by face-selective cortical neurons. Proc Natl Acad Sci U S A 2023; 120:e2214996120. [PMID: 36802419 PMCID: PMC9992780 DOI: 10.1073/pnas.2214996120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/23/2023] Open
Abstract
Neurons throughout the primate inferior temporal (IT) cortex respond selectively to visual images of faces and other complex objects. The response magnitude of neurons to a given image often depends on the size at which the image is presented, usually on a flat display at a fixed distance. While such size sensitivity might simply reflect the angular subtense of retinal image stimulation in degrees, one unexplored possibility is that it tracks the real-world geometry of physical objects, such as their size and distance to the observer in centimeters. This distinction bears fundamentally on the nature of object representation in IT and on the scope of visual operations supported by the ventral visual pathway. To address this question, we assessed the response dependency of neurons in the macaque anterior fundus (AF) face patch to the angular versus physical size of faces. We employed a macaque avatar to stereoscopically render three-dimensional (3D) photorealistic faces at multiple sizes and distances, including a subset of size/distance combinations designed to cast the same size retinal image projection. We found that most AF neurons were modulated principally by the 3D physical size of the face rather than its two-dimensional (2D) angular size on the retina. Further, most neurons responded strongest to extremely large and small faces, rather than to those of normal size. Together, these findings reveal a graded encoding of physical size among face patch neurons, providing evidence that category-selective regions of the primate ventral visual pathway participate in a geometric analysis of real-world objects.
Collapse
|
9
|
Active visual search in naturalistic environments reflects individual differences in classic visual search performance. Sci Rep 2023; 13:631. [PMID: 36635491 PMCID: PMC9837148 DOI: 10.1038/s41598-023-27896-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Accepted: 01/10/2023] [Indexed: 01/13/2023] Open
Abstract
Visual search is a ubiquitous activity in real-world environments. Yet, traditionally, visual search is investigated in tightly controlled paradigms, where head-restricted participants locate a minimalistic target in a cluttered array that is presented on a computer screen. Do traditional visual search tasks predict performance in naturalistic settings, where participants actively explore complex, real-world scenes? Here, we leverage advances in virtual reality technology to test the degree to which classic and naturalistic search are limited by a common factor, set size, and the degree to which individual differences in classic search behavior predict naturalistic search behavior in a large sample of individuals (N = 75). In a naturalistic search task, participants looked for an object within their environment via a combination of head-turns and eye-movements using a head-mounted display. Then, in a classic search task, participants searched for a target within a simple array of colored letters using only eye-movements. In each task, we found that participants' search performance was impacted by increases in set size-the number of items in the visual display. Critically, we observed that participants' efficiency in classic search tasks-the degree to which set size slowed performance-indeed predicted efficiency in real-world scenes. These results demonstrate that classic, computer-based visual search tasks are excellent models of active, real-world search behavior.
Collapse
|
10
|
Parker PRL, Abe ETT, Leonard ESP, Martins DM, Niell CM. Joint coding of visual input and eye/head position in V1 of freely moving mice. Neuron 2022; 110:3897-3906.e5. [PMID: 36137549 PMCID: PMC9742335 DOI: 10.1016/j.neuron.2022.08.029] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Revised: 07/16/2022] [Accepted: 08/30/2022] [Indexed: 12/15/2022]
Abstract
Visual input during natural behavior is highly dependent on movements of the eyes and head, but how information about eye and head position is integrated with visual processing during free movement is unknown, as visual physiology is generally performed under head fixation. To address this, we performed single-unit electrophysiology in V1 of freely moving mice while simultaneously measuring the mouse's eye position, head orientation, and the visual scene from the mouse's perspective. From these measures, we mapped spatiotemporal receptive fields during free movement based on the gaze-corrected visual input. Furthermore, we found a significant fraction of neurons tuned for eye and head position, and these signals were integrated with visual responses through a multiplicative mechanism in the majority of modulated neurons. These results provide new insight into coding in the mouse V1 and, more generally, provide a paradigm for investigating visual physiology under natural conditions, including active sensing and ethological behavior.
Collapse
Affiliation(s)
- Philip R L Parker
- Institute of Neuroscience and Department of Biology, University of Oregon, Eugene, OR, USA
| | - Elliott T T Abe
- Institute of Neuroscience and Department of Biology, University of Oregon, Eugene, OR, USA
| | - Emmalyn S P Leonard
- Institute of Neuroscience and Department of Biology, University of Oregon, Eugene, OR, USA
| | - Dylan M Martins
- Institute of Neuroscience and Department of Biology, University of Oregon, Eugene, OR, USA
| | - Cristopher M Niell
- Institute of Neuroscience and Department of Biology, University of Oregon, Eugene, OR, USA.
| |
Collapse
|
11
|
Parker PRL, Abe ETT, Beatie NT, Leonard ESP, Martins DM, Sharp SL, Wyrick DG, Mazzucato L, Niell CM. Distance estimation from monocular cues in an ethological visuomotor task. eLife 2022; 11:e74708. [PMID: 36125119 PMCID: PMC9489205 DOI: 10.7554/elife.74708] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Accepted: 08/29/2022] [Indexed: 12/02/2022] Open
Abstract
In natural contexts, sensory processing and motor output are closely coupled, which is reflected in the fact that many brain areas contain both sensory and movement signals. However, standard reductionist paradigms decouple sensory decisions from their natural motor consequences, and head-fixation prevents the natural sensory consequences of self-motion. In particular, movement through the environment provides a number of depth cues beyond stereo vision that are poorly understood. To study the integration of visual processing and motor output in a naturalistic task, we investigated distance estimation in freely moving mice. We found that mice use vision to accurately jump across a variable gap, thus directly coupling a visual computation to its corresponding ethological motor output. Monocular eyelid suture did not affect gap jumping success, thus mice can use cues that do not depend on binocular disparity and stereo vision. Under monocular conditions, mice altered their head positioning and performed more vertical head movements, consistent with a shift from using stereopsis to other monocular cues, such as motion or position parallax. Finally, optogenetic suppression of primary visual cortex impaired task performance under both binocular and monocular conditions when optical fiber placement was localized to binocular or monocular zone V1, respectively. Together, these results show that mice can use monocular cues, relying on visual cortex, to accurately judge distance. Furthermore, this behavioral paradigm provides a foundation for studying how neural circuits convert sensory information into ethological motor output.
Collapse
Affiliation(s)
- Philip RL Parker
- Institute of Neuroscience, University of OregonEugeneUnited States
| | - Elliott TT Abe
- Institute of Neuroscience, University of OregonEugeneUnited States
| | - Natalie T Beatie
- Institute of Neuroscience, University of OregonEugeneUnited States
| | | | - Dylan M Martins
- Institute of Neuroscience, University of OregonEugeneUnited States
| | - Shelby L Sharp
- Institute of Neuroscience, University of OregonEugeneUnited States
| | - David G Wyrick
- Institute of Neuroscience, University of OregonEugeneUnited States
| | - Luca Mazzucato
- Institute of Neuroscience, University of OregonEugeneUnited States
- Department of Mathematics, University of OregonEugeneUnited States
| | - Cristopher M Niell
- Institute of Neuroscience, University of OregonEugeneUnited States
- Department of Biology, University of OregonEugeneUnited States
| |
Collapse
|
12
|
Taubert J, Wardle SG, Tardiff CT, Patterson A, Yu D, Baker CI. Clutter Substantially Reduces Selectivity for Peripheral Faces in the Macaque Brain. J Neurosci 2022; 42:6739-6750. [PMID: 35868861 PMCID: PMC9436017 DOI: 10.1523/jneurosci.0232-22.2022] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Revised: 04/29/2022] [Accepted: 06/06/2022] [Indexed: 11/21/2022] Open
Abstract
According to a prominent view in neuroscience, visual stimuli are coded by discrete cortical networks that respond preferentially to specific categories, such as faces or objects. However, it remains unclear how these category-selective networks respond when viewing conditions are cluttered, i.e., when there is more than one stimulus in the visual field. Here, we asked three questions: (1) Does clutter reduce the response and selectivity for faces as a function of retinal location? (2) Is the preferential response to faces uniform across the visual field? And (3) Does the ventral visual pathway encode information about the location of cluttered faces? We used fMRI to measure the response of the face-selective network in awake, fixating macaques (two female, five male). Across a series of four experiments, we manipulated the presence and absence of clutter, as well as the location of the faces relative to the fovea. We found that clutter reduces the response to peripheral faces. When presented in isolation, without clutter, the selectivity for faces is fairly uniform across the visual field, but, when clutter is present, there is a marked decrease in the selectivity for peripheral faces. We also found no evidence of a contralateral visual field bias when faces were presented in clutter. Nonetheless, multivariate analyses revealed that the location of cluttered faces could be decoded from the multivoxel response of the face-selective network. Collectively, these findings demonstrate that clutter blunts the selectivity of the face-selective network to peripheral faces, although information about their retinal location is retained.SIGNIFICANCE STATEMENT Numerous studies that have measured brain activity in macaques have found visual regions that respond preferentially to faces. Although these regions are thought to be essential for social behavior, their responses have typically been measured while faces were presented in isolation, a situation atypical of the real world. How do these regions respond when faces are presented with other stimuli? We report that, when clutter is present, the preferential response to foveated faces is spared but preferential response to peripheral faces is reduced. Our results indicate that the presence of clutter changes the response of the face-selective network.
Collapse
Affiliation(s)
- Jessica Taubert
- Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, Maryland 20814
- School of Psychology, The University of Queensland, Brisbane, Queensland 4072, Australia
| | - Susan G Wardle
- Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, Maryland 20814
| | - Clarissa T Tardiff
- Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, Maryland 20814
| | - Amanda Patterson
- Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, Maryland 20814
| | - David Yu
- Neurophysiology Imaging Facility, National Institutes of Health, Bethesda, Maryland 20814
| | - Chris I Baker
- Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, Maryland 20814
| |
Collapse
|
13
|
Ngo V, Gorman JC, De la Fuente MF, Souto A, Schiel N, Miller CT. Active vision during prey capture in wild marmoset monkeys. Curr Biol 2022; 32:3423-3428.e3. [PMID: 35750054 PMCID: PMC10203885 DOI: 10.1016/j.cub.2022.06.028] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Revised: 06/08/2022] [Accepted: 06/09/2022] [Indexed: 10/17/2022]
Abstract
A foundational pressure in the evolution of all animals is the ability to travel through the world, inherently coupling the sensory and motor systems. While this relationship has been explored in several species,1-4 it has been largely overlooked in primates, which have typically relied on paradigms in which head-restrained subjects view stimuli on screens.5 Natural visual behaviors, by contrast, are typified by locomotion through the environment guided by active sensing as animals explore and interact with the world,4,6 a relationship well illustrated by prey capture.7-12 Here, we characterized prey capture in wild marmoset monkeys as they negotiated their dynamic, arboreal habitat to illustrate the inherent role of vision as an active process in natural nonhuman primate behavior. Not only do marmosets share the core properties of vision that typify the primate Order,13-18 but they are prolific hunters that prey on a diverse set of prey animals.19-22 Marmosets pursued prey using vision in several different contexts, but executed precise visually guided motor control that predominantly involved grasping with hands for successful capture of prey. Applying markerless tracking for the first time in wild primates yielded novel findings that precisely quantified how marmosets track insects prior to initiating an attack and the rapid visually guided corrections of the hands during capture. These findings offer the first detailed insight into the active nature of vision to guide multiple facets of a natural goal-directed behavior in wild primates and can inform future laboratory studies of natural primate visual behaviors and the supporting neural processes.
Collapse
Affiliation(s)
- Victoria Ngo
- Cortical Systems and Behavior Laboratory, University of California, San Diego, La Jolla, CA 92039, USA
| | - Julia C Gorman
- Cortical Systems and Behavior Laboratory, University of California, San Diego, La Jolla, CA 92039, USA; Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA 92039, USA
| | - María Fernanda De la Fuente
- Programa de Pós-graduação em Etnobiologia e Conservação da Natureza, Universidade Estadual da Paraíba, Campina Grande, Paraíba 58429-500, Brazil; Laboratório de Etologia Teórica e Aplicada, Departamento de Biologia, Universidade Federal Rural de Pernambuco, Recife, Pernambuco 52171-900, Brazil
| | - Antonio Souto
- Laboratório de Etologia, Departamento de Zoologia, Universidade Federal de Pernambuco, Recife, Pernambuco 50670-901, Brazil
| | - Nicola Schiel
- Laboratório de Etologia Teórica e Aplicada, Departamento de Biologia, Universidade Federal Rural de Pernambuco, Recife, Pernambuco 52171-900, Brazil
| | - Cory T Miller
- Cortical Systems and Behavior Laboratory, University of California, San Diego, La Jolla, CA 92039, USA; Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA 92039, USA.
| |
Collapse
|
14
|
Kamps FS, Richardson H, Murty NAR, Kanwisher N, Saxe R. Using child-friendly movie stimuli to study the development of face, place, and object regions from age 3 to 12 years. Hum Brain Mapp 2022; 43:2782-2800. [PMID: 35274789 PMCID: PMC9120553 DOI: 10.1002/hbm.25815] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2021] [Revised: 02/11/2022] [Accepted: 02/13/2022] [Indexed: 01/21/2023] Open
Abstract
Scanning young children while they watch short, engaging, commercially-produced movies has emerged as a promising approach for increasing data retention and quality. Movie stimuli also evoke a richer variety of cognitive processes than traditional experiments, allowing the study of multiple aspects of brain development simultaneously. However, because these stimuli are uncontrolled, it is unclear how effectively distinct profiles of brain activity can be distinguished from the resulting data. Here we develop an approach for identifying multiple distinct subject-specific Regions of Interest (ssROIs) using fMRI data collected during movie-viewing. We focused on the test case of higher-level visual regions selective for faces, scenes, and objects. Adults (N = 13) were scanned while viewing a 5.6-min child-friendly movie, as well as a traditional localizer experiment with blocks of faces, scenes, and objects. We found that just 2.7 min of movie data could identify subject-specific face, scene, and object regions. While successful, movie-defined ssROIS still showed weaker domain selectivity than traditional ssROIs. Having validated our approach in adults, we then used the same methods on movie data collected from 3 to 12-year-old children (N = 122). Movie response timecourses in 3-year-old children's face, scene, and object regions were already significantly and specifically predicted by timecourses from the corresponding regions in adults. We also found evidence of continued developmental change, particularly in the face-selective posterior superior temporal sulcus. Taken together, our results reveal both early maturity and functional change in face, scene, and object regions, and more broadly highlight the promise of short, child-friendly movies for developmental cognitive neuroscience.
Collapse
Affiliation(s)
- Frederik S. Kamps
- Department of Brain and Cognitive SciencesMassachusetts Institute of TechnologyCambridgeMassachusettsUSA
| | - Hilary Richardson
- School of Philosophy, Psychology and Language SciencesUniversity of EdinburghEdinburghUK
| | - N. Apurva Ratan Murty
- Department of Brain and Cognitive SciencesMassachusetts Institute of TechnologyCambridgeMassachusettsUSA
| | - Nancy Kanwisher
- Department of Brain and Cognitive SciencesMassachusetts Institute of TechnologyCambridgeMassachusettsUSA
| | - Rebecca Saxe
- Department of Brain and Cognitive SciencesMassachusetts Institute of TechnologyCambridgeMassachusettsUSA
| |
Collapse
|
15
|
Miller CT, Gire D, Hoke K, Huk AC, Kelley D, Leopold DA, Smear MC, Theunissen F, Yartsev M, Niell CM. Natural behavior is the language of the brain. Curr Biol 2022; 32:R482-R493. [PMID: 35609550 PMCID: PMC10082559 DOI: 10.1016/j.cub.2022.03.031] [Citation(s) in RCA: 47] [Impact Index Per Article: 23.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
The breadth and complexity of natural behaviors inspires awe. Understanding how our perceptions, actions, and internal thoughts arise from evolved circuits in the brain has motivated neuroscientists for generations. Researchers have traditionally approached this question by focusing on stereotyped behaviors, either natural or trained, in a limited number of model species. This approach has allowed for the isolation and systematic study of specific brain operations, which has greatly advanced our understanding of the circuits involved. At the same time, the emphasis on experimental reductionism has left most aspects of the natural behaviors that have shaped the evolution of the brain largely unexplored. However, emerging technologies and analytical tools make it possible to comprehensively link natural behaviors to neural activity across a broad range of ethological contexts and timescales, heralding new modes of neuroscience focused on natural behaviors. Here we describe a three-part roadmap that aims to leverage the wealth of behaviors in their naturally occurring distributions, linking their variance with that of underlying neural processes to understand how the brain is able to successfully navigate the everyday challenges of animals' social and ecological landscapes. To achieve this aim, experimenters must harness one challenge faced by all neurobiological systems, namely variability, in order to gain new insights into the language of the brain.
Collapse
Affiliation(s)
- Cory T Miller
- Cortical Systems and Behavior Laboratory, University of California San Diego, 9500 Gilman Drive, La Jolla, CA 92039, USA.
| | - David Gire
- Department of Psychology, University of Washington, Guthrie Hall, Seattle, WA 98105, USA
| | - Kim Hoke
- Department of Biology, Colorado State University, 1878 Campus Delivery, Fort Collins, CO 80523, USA
| | - Alexander C Huk
- Center for Perceptual Systems, Departments of Neuroscience and Psychology, University of Texas at Austin, 116 Inner Campus Drive, Austin, TX 78712, USA
| | - Darcy Kelley
- Department of Biological Sciences, Columbia University, 1212 Amsterdam Avenue, New York, NY 10027, USA
| | - David A Leopold
- Section of Cognitive Neurophysiology and Imaging, National Institute of Mental Health, 49 Convent Drive, Bethesda, MD 20892, USA
| | - Matthew C Smear
- Department of Psychology and Institute of Neuroscience, University of Oregon, 1227 University Street, Eugene, OR 97403, USA
| | - Frederic Theunissen
- Department of Psychology, University of California Berkeley, 2121 Berkeley Way, Berkeley, CA 94720, USA
| | - Michael Yartsev
- Department of Bioengineering, University of California Berkeley, 306 Stanley Hall, Berkeley, CA 94720, USA
| | - Cristopher M Niell
- Department of Biology and Institute of Neuroscience, University of Oregon, 222 Huestis Hall, Eugene, OR 97403, USA.
| |
Collapse
|
16
|
Park SH, Koyano KW, Russ BE, Waidmann EN, McMahon DBT, Leopold DA. Parallel functional subnetworks embedded in the macaque face patch system. SCIENCE ADVANCES 2022; 8:eabm2054. [PMID: 35263138 PMCID: PMC8906740 DOI: 10.1126/sciadv.abm2054] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/01/2021] [Accepted: 01/19/2022] [Indexed: 06/14/2023]
Abstract
During normal vision, our eyes provide the brain with a continuous stream of useful information about the world. How visually specialized areas of the cortex, such as face-selective patches, operate under natural modes of behavior is poorly understood. Here we report that, during the free viewing of movies, cohorts of face-selective neurons in the macaque cortex fractionate into distributed and parallel subnetworks that carry distinct information. We classified neurons into functional groups on the basis of their movie-driven coupling with functional magnetic resonance imaging time courses across the brain. Neurons from each group were distributed across multiple face patches but intermixed locally with other groups at each recording site. These findings challenge prevailing views about functional segregation in the cortex and underscore the importance of naturalistic paradigms for cognitive neuroscience.
Collapse
Affiliation(s)
- Soo Hyun Park
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD, USA
| | - Kenji W. Koyano
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD, USA
| | - Brian E. Russ
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD, USA
| | - Elena N. Waidmann
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD, USA
| | - David B. T. McMahon
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD, USA
| | - David A. Leopold
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD, USA
- Neurophysiology Imaging Facility, National Institute of Mental Health, National Institute of Neurological Disorders and Stroke, National Eye Institute, Bethesda, MD, USA
| |
Collapse
|
17
|
Kaneko T, Komatsu M, Yamamori T, Ichinohe N, Okano H. Cortical neural dynamics unveil the rhythm of natural visual behavior in marmosets. Commun Biol 2022; 5:108. [PMID: 35115680 PMCID: PMC8814246 DOI: 10.1038/s42003-022-03052-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Accepted: 01/13/2022] [Indexed: 01/13/2023] Open
Abstract
Numerous studies have shown that the visual system consists of functionally distinct ventral and dorsal streams; however, its exact spatial-temporal dynamics during natural visual behavior remain to be investigated. Here, we report cerebral neural dynamics during active visual exploration recorded by an electrocorticographic array covering the entire lateral surface of the marmoset cortex. We found that the dorsal stream was activated before the primary visual cortex with saccades and followed by the alteration of suppression and activation signals along the ventral stream. Similarly, the signal that propagated from the dorsal to ventral visual areas was accompanied by a travelling wave of low frequency oscillations. Such signal dynamics occurred at an average of 220 ms after saccades, which corresponded to the timing when whole-brain activation returned to background levels. We also demonstrated that saccades could occur at any point of signal flow, indicating the parallel computation of motor commands. Overall, this study reveals the neural dynamics of active vision, which are efficiently linked to the natural rhythms of visual exploration.
Collapse
Affiliation(s)
- Takaaki Kaneko
- Laboratory for Marmoset Neural Architecture, RIKEN Center for Brain Science, Saitama, Japan. .,Systems Neuroscience Section, Primate Research Institute, Kyoto University, Aichi, Japan.
| | - Misako Komatsu
- Laboratory for Molecular Analysis of Higher Brain Function, Center for Brain Science, RIKEN, Saitama, Japan
| | - Tetsuo Yamamori
- Laboratory for Molecular Analysis of Higher Brain Function, Center for Brain Science, RIKEN, Saitama, Japan
| | - Noritaka Ichinohe
- Department of Ultrastructural Research, National Institute of Neuroscience, National Center of Neurology and Psychiatry, Tokyo, Japan
| | - Hideyuki Okano
- Laboratory for Marmoset Neural Architecture, RIKEN Center for Brain Science, Saitama, Japan. .,Department of Physiology, Keio University School of Medicine, Tokyo, Japan.
| |
Collapse
|
18
|
Finn ES, Glerean E, Hasson U, Vanderwal T. Naturalistic imaging: The use of ecologically valid conditions to study brain function. Neuroimage 2021; 247:118776. [PMID: 34864153 DOI: 10.1016/j.neuroimage.2021.118776] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
|
19
|
One object, two networks? Assessing the relationship between the face and body-selective regions in the primate visual system. Brain Struct Funct 2021; 227:1423-1438. [PMID: 34792643 DOI: 10.1007/s00429-021-02420-7] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2021] [Accepted: 10/22/2021] [Indexed: 10/19/2022]
Abstract
Faces and bodies are often treated as distinct categories that are processed separately by face- and body-selective brain regions in the primate visual system. These regions occupy distinct regions of visual cortex and are often thought to constitute independent functional networks. Yet faces and bodies are part of the same object and their presence inevitably covary in naturalistic settings. Here, we re-evaluate both the evidence supporting the independent processing of faces and bodies and the organizational principles that have been invoked to explain this distinction. We outline four hypotheses ranging from completely separate networks to a single network supporting the perception of whole people or animals. The current evidence, especially in humans, is compatible with all of these hypotheses, making it presently unclear how the representation of faces and bodies is organized in the cortex.
Collapse
|
20
|
Zhang Y, Kim JH, Brang D, Liu Z. Naturalistic Stimuli: A Paradigm for Multi-Scale Functional Characterization of the Human Brain. CURRENT OPINION IN BIOMEDICAL ENGINEERING 2021; 19:100298. [PMID: 34423178 PMCID: PMC8376216 DOI: 10.1016/j.cobme.2021.100298] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
Movies, audio stories, and virtual reality are increasingly used as stimuli for functional brain imaging. Such naturalistic paradigms are in sharp contrast to the tradition of experimental reductionism in neuroscience research. Being complex, dynamic, and diverse, naturalistic stimuli set up a more ecologically relevant condition and induce highly reproducible brain responses across a wide range of spatiotemporal scales. Here, we review recent technical advances and scientific findings on imaging the brain under naturalistic stimuli. Then we elaborate on the premise of using naturalistic paradigms for multi-scale, multi-modal, and high-throughput functional characterization of the human brain. We further highlight the growing potential of using deep learning models to infer neural information processing from brain responses to naturalistic stimuli. Lastly, we advocate large-scale collaborations to combine brain imaging and recording data across experiments, subjects, and labs that use the same set of naturalistic stimuli.
Collapse
Affiliation(s)
- Yizhen Zhang
- Department of Electrical Engineering and Computer Science, University of Michigan
| | - Jung-Hoon Kim
- Department of Biomedical Engineering, University of Michigan
- Weldon School of Biomedical Engineering, Purdue University
| | - David Brang
- Department of Psychology, University of Michigan
| | - Zhongming Liu
- Department of Electrical Engineering and Computer Science, University of Michigan
- Department of Biomedical Engineering, University of Michigan
| |
Collapse
|
21
|
Russ BE, Petkov CI, Kwok SC, Zhu Q, Belin P, Vanduffel W, Hamed SB. Common functional localizers to enhance NHP & cross-species neuroscience imaging research. Neuroimage 2021; 237:118203. [PMID: 34048898 PMCID: PMC8529529 DOI: 10.1016/j.neuroimage.2021.118203] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2020] [Revised: 05/15/2021] [Accepted: 05/24/2021] [Indexed: 11/25/2022] Open
Abstract
Functional localizers are invaluable as they can help define regions of interest, provide cross-study comparisons, and most importantly, allow for the aggregation and meta-analyses of data across studies and laboratories. To achieve these goals within the non-human primate (NHP) imaging community, there is a pressing need for the use of standardized and validated localizers that can be readily implemented across different groups. The goal of this paper is to provide an overview of the value of localizer protocols to imaging research and we describe a number of commonly used or novel localizers within NHPs, and keys to implement them across studies. As has been shown with the aggregation of resting-state imaging data in the original PRIME-DE submissions, we believe that the field is ready to apply the same initiative for task-based functional localizers in NHP imaging. By coming together to collect large datasets across research group, implementing the same functional localizers, and sharing the localizers and data via PRIME-DE, it is now possible to fully test their robustness, selectivity and specificity. To do this, we reviewed a number of common localizers and we created a repository of well-established localizer that are easily accessible and implemented through the PRIME-RE platform.
Collapse
Affiliation(s)
- Brian E Russ
- Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, United States; Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York City, NY, United States; Department of Psychiatry, New York University at Langone, New York City, NY, United States.
| | - Christopher I Petkov
- Biosciences Institute, Newcastle University Medical School, Newcastle upon Tyne, United Kingdom
| | - Sze Chai Kwok
- Shanghai Key Laboratory of Brain Functional Genomics, Key Laboratory of Brain Functional Genomics Ministry of Education, Shanghai Key Laboratory of Magnetic Resonance, Affiliated Mental Health Center (ECNU), School of Psychology and Cognitive Science, East China Normal University, Shanghai, China; Division of Natural and Applied Sciences, Duke Kunshan University, Kunshan, Jiangsu, China; NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai, China
| | - Qi Zhu
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin Center, 91191 Gif/Yvette, France; Laboratory for Neuro-and Psychophysiology, Department of Neurosciences, KU Leuven Medical School, Leuven, 3000, Belgium
| | - Pascal Belin
- Institut de Neurosciences de La Timone, Aix-Marseille Université et CNRS, Marseille, 13005, France
| | - Wim Vanduffel
- Laboratory for Neuro-and Psychophysiology, Department of Neurosciences, KU Leuven Medical School, Leuven, 3000, Belgium; Leuven Brain Institute, KU Leuven, Leuven, 3000, Belgium; Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA 02129, United States; Department of Radiology, Harvard Medical School, Boston, MA 02144, United States.
| | - Suliann Ben Hamed
- Institut des Sciences Cognitives Marc Jeannerod, UMR 5229, Université de Lyon - CNRS, France.
| |
Collapse
|
22
|
Niell CM, Scanziani M. How Cortical Circuits Implement Cortical Computations: Mouse Visual Cortex as a Model. Annu Rev Neurosci 2021; 44:517-546. [PMID: 33914591 PMCID: PMC9925090 DOI: 10.1146/annurev-neuro-102320-085825] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
The mouse, as a model organism to study the brain, gives us unprecedented experimental access to the mammalian cerebral cortex. By determining the cortex's cellular composition, revealing the interaction between its different components, and systematically perturbing these components, we are obtaining mechanistic insight into some of the most basic properties of cortical function. In this review, we describe recent advances in our understanding of how circuits of cortical neurons implement computations, as revealed by the study of mouse primary visual cortex. Further, we discuss how studying the mouse has broadened our understanding of the range of computations performed by visual cortex. Finally, we address how future approaches will fulfill the promise of the mouse in elucidating fundamental operations of cortex.
Collapse
Affiliation(s)
- Cristopher M. Niell
- Department of Biology and Institute of Neuroscience, University of Oregon, Eugene, Oregon 97403, USA
| | - Massimo Scanziani
- Department of Physiology and Howard Hughes Medical Institute, University of California San Francisco, San Francisco, California 94158, USA;
| |
Collapse
|
23
|
Huber-Huber C, Buonocore A, Melcher D. The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception. J Vis 2021; 21:12. [PMID: 34283203 PMCID: PMC8300052 DOI: 10.1167/jov.21.7.12] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2020] [Accepted: 05/18/2021] [Indexed: 01/02/2023] Open
Abstract
A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.
Collapse
Affiliation(s)
- Christoph Huber-Huber
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, The Netherlands
- CIMeC, University of Trento, Italy
| | - Antimo Buonocore
- Werner Reichardt Centre for Integrative Neuroscience, Tübingen University, Tübingen, BW, Germany
- Hertie Institute for Clinical Brain Research, Tübingen University, Tübingen, BW, Germany
| | - David Melcher
- CIMeC, University of Trento, Italy
- Division of Science, New York University Abu Dhabi, UAE
| |
Collapse
|
24
|
The future of intelligence: The central meaning-making unit of intelligence in the mind, the brain, and artificial intelligence. INTELLIGENCE 2021. [DOI: 10.1016/j.intell.2021.101562] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
|
25
|
Noel JP, Caziot B, Bruni S, Fitzgerald NE, Avila E, Angelaki DE. Supporting generalization in non-human primate behavior by tapping into structural knowledge: Examples from sensorimotor mappings, inference, and decision-making. Prog Neurobiol 2021; 201:101996. [PMID: 33454361 PMCID: PMC8096669 DOI: 10.1016/j.pneurobio.2021.101996] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2020] [Revised: 12/15/2020] [Accepted: 01/12/2021] [Indexed: 02/05/2023]
Abstract
The complex behaviors we ultimately wish to understand are far from those currently used in systems neuroscience laboratories. A salient difference are the closed loops between action and perception prominently present in natural but not laboratory behaviors. The framework of reinforcement learning and control naturally wades across action and perception, and thus is poised to inform the neurosciences of tomorrow, not only from a data analyses and modeling framework, but also in guiding experimental design. We argue that this theoretical framework emphasizes active sensing, dynamical planning, and the leveraging of structural regularities as key operations for intelligent behavior within uncertain, time-varying environments. Similarly, we argue that we may study natural task strategies and their neural circuits without over-training animals when the tasks we use tap into our animal's structural knowledge. As proof-of-principle, we teach animals to navigate through a virtual environment - i.e., explore a well-defined and repetitive structure governed by the laws of physics - using a joystick. Once these animals have learned to 'drive', without further training they naturally (i) show zero- or one-shot learning of novel sensorimotor contingencies, (ii) infer the evolving path of dynamically changing latent variables, and (iii) make decisions consistent with maximizing reward rate. Such task designs allow for the study of flexible and generalizable, yet controlled, behaviors. In turn, they allow for the exploitation of pillars of intelligence - flexibility, prediction, and generalization -, properties whose neural underpinning have remained elusive.
Collapse
Affiliation(s)
- Jean-Paul Noel
- Center for Neural Science, New York University, New York, USA
| | - Baptiste Caziot
- Center for Neural Science, New York University, New York, USA
| | - Stefania Bruni
- Center for Neural Science, New York University, New York, USA
| | | | - Eric Avila
- Center for Neural Science, New York University, New York, USA
| | - Dora E Angelaki
- Center for Neural Science, New York University, New York, USA; Tandon School of Engineering, New York University, New York, USA.
| |
Collapse
|
26
|
Nastase SA, Goldstein A, Hasson U. Keep it real: rethinking the primacy of experimental control in cognitive neuroscience. Neuroimage 2020; 222:117254. [PMID: 32800992 PMCID: PMC7789034 DOI: 10.1016/j.neuroimage.2020.117254] [Citation(s) in RCA: 99] [Impact Index Per Article: 24.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2019] [Revised: 07/08/2020] [Accepted: 08/04/2020] [Indexed: 01/17/2023] Open
Abstract
Naturalistic experimental paradigms in neuroimaging arose from a pressure to test the validity of models we derive from highly-controlled experiments in real-world contexts. In many cases, however, such efforts led to the realization that models developed under particular experimental manipulations failed to capture much variance outside the context of that manipulation. The critique of non-naturalistic experiments is not a recent development; it echoes a persistent and subversive thread in the history of modern psychology. The brain has evolved to guide behavior in a multidimensional world with many interacting variables. The assumption that artificially decoupling and manipulating these variables will lead to a satisfactory understanding of the brain may be untenable. We develop an argument for the primacy of naturalistic paradigms, and point to recent developments in machine learning as an example of the transformative power of relinquishing control. Naturalistic paradigms should not be deployed as an afterthought if we hope to build models of brain and behavior that extend beyond the laboratory into the real world.
Collapse
Affiliation(s)
- Samuel A Nastase
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
| | - Ariel Goldstein
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Uri Hasson
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA; Department of Psychology, Princeton University, Princeton, NJ, USA
| |
Collapse
|