51
|
Ramp-shaped neural tuning supports graded population-level representation of the object-to-scene continuum. Sci Rep 2022; 12:18081. [PMID: 36302932 PMCID: PMC9613906 DOI: 10.1038/s41598-022-21768-2] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2022] [Accepted: 09/30/2022] [Indexed: 01/24/2023] Open
Abstract
We can easily perceive the spatial scale depicted in a picture, regardless of whether it is a small space (e.g., a close-up view of a chair) or a much larger space (e.g., an entire class room). How does the human visual system encode this continuous dimension? Here, we investigated the underlying neural coding of depicted spatial scale, by examining the voxel tuning and topographic organization of brain responses. We created naturalistic yet carefully-controlled stimuli by constructing virtual indoor environments, and rendered a series of snapshots to smoothly sample between a close-up view of the central object and far-scale view of the full environment (object-to-scene continuum). Human brain responses were measured to each position using functional magnetic resonance imaging. We did not find evidence for a smooth topographic mapping for the object-to-scene continuum on the cortex. Instead, we observed large swaths of cortex with opposing ramp-shaped profiles, with highest responses to one end of the object-to-scene continuum or the other, and a small region showing a weak tuning to intermediate scale views. However, when we considered the population code of the entire ventral occipito-temporal cortex, we found smooth and linear representation of the object-to-scene continuum. Our results together suggest that depicted spatial scale information is encoded parametrically in large-scale population codes across the entire ventral occipito-temporal cortex.
Collapse
|
52
|
Bleau M, Paré S, Chebat DR, Kupers R, Nemargut JP, Ptito M. Neural substrates of spatial processing and navigation in blindness: An activation likelihood estimation meta-analysis. Front Neurosci 2022; 16:1010354. [PMID: 36340755 PMCID: PMC9630591 DOI: 10.3389/fnins.2022.1010354] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Accepted: 09/30/2022] [Indexed: 12/02/2022] Open
Abstract
Even though vision is considered the best suited sensory modality to acquire spatial information, blind individuals can form spatial representations to navigate and orient themselves efficiently in space. Consequently, many studies support the amodality hypothesis of spatial representations since sensory modalities other than vision contribute to the formation of spatial representations, independently of visual experience and imagery. However, given the high variability in abilities and deficits observed in blind populations, a clear consensus about the neural representations of space has yet to be established. To this end, we performed a meta-analysis of the literature on the neural correlates of spatial processing and navigation via sensory modalities other than vision, like touch and audition, in individuals with early and late onset blindness. An activation likelihood estimation (ALE) analysis of the neuroimaging literature revealed that early blind individuals and sighted controls activate the same neural networks in the processing of non-visual spatial information and navigation, including the posterior parietal cortex, frontal eye fields, insula, and the hippocampal complex. Furthermore, blind individuals also recruit primary and associative occipital areas involved in visuo-spatial processing via cross-modal plasticity mechanisms. The scarcity of studies involving late blind individuals did not allow us to establish a clear consensus about the neural substrates of spatial representations in this specific population. In conclusion, the results of our analysis on neuroimaging studies involving early blind individuals support the amodality hypothesis of spatial representations.
Collapse
Affiliation(s)
- Maxime Bleau
- École d’Optométrie, Université de Montréal, Montreal, QC, Canada
| | - Samuel Paré
- École d’Optométrie, Université de Montréal, Montreal, QC, Canada
| | - Daniel-Robert Chebat
- Visual and Cognitive Neuroscience Laboratory (VCN Lab), Department of Psychology, Faculty of Social Sciences and Humanities, Ariel University, Ariel, Israel
- Navigation and Accessibility Research Center of Ariel University (NARCA), Ariel University, Ariel, Israel
| | - Ron Kupers
- École d’Optométrie, Université de Montréal, Montreal, QC, Canada
- Institute of Neuroscience, Faculty of Medicine, Université de Louvain, Brussels, Belgium
- Department of Neuroscience, University of Copenhagen, Copenhagen, Denmark
| | | | - Maurice Ptito
- École d’Optométrie, Université de Montréal, Montreal, QC, Canada
- Department of Neuroscience, University of Copenhagen, Copenhagen, Denmark
- Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, QC, Canada
- *Correspondence: Maurice Ptito,
| |
Collapse
|
53
|
Rolls ET. The hippocampus, ventromedial prefrontal cortex, and episodic and semantic memory. Prog Neurobiol 2022; 217:102334. [PMID: 35870682 DOI: 10.1016/j.pneurobio.2022.102334] [Citation(s) in RCA: 39] [Impact Index Per Article: 19.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Revised: 06/07/2022] [Accepted: 07/19/2022] [Indexed: 11/24/2022]
Abstract
The human ventromedial prefrontal cortex (vmPFC)/anterior cingulate cortex is implicated in reward and emotion, but also in memory. It is shown how the human orbitofrontal cortex connecting with the vmPFC and anterior cingulate cortex provide a route to the hippocampus for reward and emotional value to be incorporated into episodic memory, enabling memory of where a reward was seen. It is proposed that this value component results in primarily episodic memories with some value component to be repeatedly recalled from the hippocampus so that they are more likely to become incorporated into neocortical semantic and autobiographical memories. The same orbitofrontal and anterior cingulate regions also connect in humans to the septal and basal forebrain cholinergic nuclei, thereby helping to consolidate memory, and helping to account for why damage to the vMPFC impairs memory. The human hippocampus and vmPFC thus contribute in complementary ways to forming episodic and semantic memories.
Collapse
Affiliation(s)
- Edmund T Rolls
- Oxford Centre for Computational Neuroscience, Oxford, UK; University of Warwick, Department of Computer Science, Coventry, UK.
| |
Collapse
|
54
|
Rolls ET, Deco G, Huang CC, Feng J. Multiple cortical visual streams in humans. Cereb Cortex 2022; 33:3319-3349. [PMID: 35834308 DOI: 10.1093/cercor/bhac276] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Revised: 06/16/2022] [Accepted: 06/17/2022] [Indexed: 11/14/2022] Open
Abstract
The effective connectivity between 55 visual cortical regions and 360 cortical regions was measured in 171 HCP participants using the HCP-MMP atlas, and complemented with functional connectivity and diffusion tractography. A Ventrolateral Visual "What" Stream for object and face recognition projects hierarchically to the inferior temporal visual cortex, which projects to the orbitofrontal cortex for reward value and emotion, and to the hippocampal memory system. A Ventromedial Visual "Where" Stream for scene representations connects to the parahippocampal gyrus and hippocampus. An Inferior STS (superior temporal sulcus) cortex Semantic Stream receives from the Ventrolateral Visual Stream, from visual inferior parietal PGi, and from the ventromedial-prefrontal reward system and connects to language systems. A Dorsal Visual Stream connects via V2 and V3A to MT+ Complex regions (including MT and MST), which connect to intraparietal regions (including LIP, VIP and MIP) involved in visual motion and actions in space. It performs coordinate transforms for idiothetic update of Ventromedial Stream scene representations. A Superior STS cortex Semantic Stream receives visual inputs from the Inferior STS Visual Stream, PGi, and STV, and auditory inputs from A5, is activated by face expression, motion and vocalization, and is important in social behaviour, and connects to language systems.
Collapse
Affiliation(s)
- Edmund T Rolls
- Oxford Centre for Computational Neuroscience, Oxford, United Kingdom.,Department of Computer Science, University of Warwick, Coventry CV4 7AL, United Kingdom.,Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai 200403, China
| | - Gustavo Deco
- Computational Neuroscience Group, Department of Information and Communication Technologies, Center for Brain and Cognition, Universitat Pompeu Fabra, Roc Boronat 138, Barcelona 08018, Spain.,Brain and Cognition, Pompeu Fabra University, Barcelona 08018, Spain.,Institució Catalana de la Recerca i Estudis Avançats (ICREA), Universitat Pompeu Fabra, Passeig Lluís Companys 23, Barcelona 08010, Spain
| | - Chu-Chung Huang
- Shanghai Key Laboratory of Brain Functional Genomics (Ministry of Education), Institute of Brain and Education Innovation, School of Psychology and Cognitive Science, East China Normal University, Shanghai 200602, China.,Shanghai Center for Brain Science and Brain-Inspired Technology, Shanghai 200602, China
| | - Jianfeng Feng
- Department of Computer Science, University of Warwick, Coventry CV4 7AL, United Kingdom.,Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai 200403, China
| |
Collapse
|
55
|
Cohen SS, Tottenham N, Baldassano C. Developmental changes in story-evoked responses in the neocortex and hippocampus. eLife 2022; 11:69430. [PMID: 35787304 PMCID: PMC9328767 DOI: 10.7554/elife.69430] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2021] [Accepted: 06/17/2022] [Indexed: 11/13/2022] Open
Abstract
How does the representation of naturalistic life events change with age? Here, we analyzed fMRI data from 414 children and adolescents (5–19 years) as they watched a narrative movie. In addition to changes in the degree of inter-subject correlation (ISC) with age in sensory and medial parietal regions, we used a novel measure (between-group ISC) to reveal age-related shifts in the responses across the majority of the neocortex. Over the course of development, brain responses became more discretized into stable and coherent events and shifted earlier in time to anticipate upcoming perceived event transitions, measured behaviorally in an age-matched sample. However, hippocampal responses to event boundaries actually decreased with age, suggesting a shifting division of labor between episodic encoding processes and schematic event representations between the ages of 5 and 19.
Collapse
Affiliation(s)
- Samantha S Cohen
- Department of Psychology, Columbia University, New York, United States
| | - Nim Tottenham
- Department of Psychology, Columbia University, New York, United States
| | | |
Collapse
|
56
|
Tang K, Chin M, Chun M, Xu Y. The contribution of object identity and configuration to scene representation in convolutional neural networks. PLoS One 2022; 17:e0270667. [PMID: 35763531 PMCID: PMC9239439 DOI: 10.1371/journal.pone.0270667] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2022] [Accepted: 06/14/2022] [Indexed: 11/23/2022] Open
Abstract
Scene perception involves extracting the identities of the objects comprising a scene in conjunction with their configuration (the spatial layout of the objects in the scene). How object identity and configuration information is weighted during scene processing and how this weighting evolves over the course of scene processing however, is not fully understood. Recent developments in convolutional neural networks (CNNs) have demonstrated their aptitude at scene processing tasks and identified correlations between processing in CNNs and in the human brain. Here we examined four CNN architectures (Alexnet, Resnet18, Resnet50, Densenet161) and their sensitivity to changes in object and configuration information over the course of scene processing. Despite differences among the four CNN architectures, across all CNNs, we observed a common pattern in the CNN's response to object identity and configuration changes. Each CNN demonstrated greater sensitivity to configuration changes in early stages of processing and stronger sensitivity to object identity changes in later stages. This pattern persists regardless of the spatial structure present in the image background, the accuracy of the CNN in classifying the scene, and even the task used to train the CNN. Importantly, CNNs' sensitivity to a configuration change is not the same as their sensitivity to any type of position change, such as that induced by a uniform translation of the objects without a configuration change. These results provide one of the first documentations of how object identity and configuration information are weighted in CNNs during scene processing.
Collapse
Affiliation(s)
- Kevin Tang
- Department of Psychology, Yale University, New Haven, CT, United States of America
| | - Matthew Chin
- Department of Psychology, Yale University, New Haven, CT, United States of America
| | - Marvin Chun
- Department of Psychology, Yale University, New Haven, CT, United States of America
| | - Yaoda Xu
- Department of Psychology, Yale University, New Haven, CT, United States of America
- * E-mail:
| |
Collapse
|
57
|
Song Y, Wu H, Chen S, Ge H, Yan Z, Xue C, Qi W, Yuan Q, Liang X, Lin X, Chen J. Differential Abnormality in Functional Connectivity Density in Preclinical and Early-Stage Alzheimer's Disease. Front Aging Neurosci 2022; 14:879836. [PMID: 35693335 PMCID: PMC9177137 DOI: 10.3389/fnagi.2022.879836] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2022] [Accepted: 04/27/2022] [Indexed: 12/23/2022] Open
Abstract
Background Both subjective cognitive decline (SCD) and amnestic mild cognitive impairment (aMCI) have a high risk of progression to Alzheimer's disease (AD). While most of the available evidence described changes in functional connectivity (FC) in SCD and aMCI, there was no confirmation of changes in functional connectivity density (FCD) that have not been confirmed. Therefore, the purpose of this study was to investigate the specific alterations in resting-state FCD in SCD and aMCI and further assess the extent to which these changes can distinguish the preclinical and early-stage AD. Methods A total of 57 patients with SCD, 59 patients with aMCI, and 78 healthy controls (HC) were included. The global FCD, local FCD, and long-range FCD were calculated for each voxel to identify brain regions with significant FCD alterations. The brain regions with abnormal FCD were then used as regions of interest for FC analysis. In addition, we calculated correlations between neuroimaging alterations and cognitive function and performed receiver-operating characteristic analyses to assess the diagnostic effect of the FCD and FC alterations on SCD and aMCI. Results FCD mapping revealed significantly increased global FCD in the left parahippocampal gyrus (PHG.L) and increased long-range FCD in the left hippocampus for patients with SCD when compared to HCs. However, when compared to SCD, patients with aMCI showed significantly decreased global FCD and long-range FCD in the PHG.L. The follow-up FC analysis further revealed significant variations between the PHG.L and the occipital lobe in patients with SCD and aMCI. In addition, patients with SCD also presented significant changes in FC between the left hippocampus, the left cerebellum anterior lobe, and the inferior temporal gyrus. Moreover, changes in abnormal indicators in the SCD and aMCI groups were significantly associated with cognitive function. Finally, combining FCD and FC abnormalities allowed for a more precise differentiation of the clinical stages. Conclusion To our knowledge, this study is the first to investigate specific alterations in FCD and FC for both patients with SCD and aMCI and confirms differential abnormalities that can serve as potential imaging markers for preclinical and early-stage Alzheimer's disease (AD). Also, it adds a new dimension of understanding to the diagnosis of SCD and aMCI as well as the evaluation of disease progression.
Collapse
Affiliation(s)
- Yu Song
- Department of Neurology, The Affiliated Brain Hospital of Nanjing Medical University, Nanjing, China
| | - Huimin Wu
- Department of Neurology, The Affiliated Brain Hospital of Nanjing Medical University, Nanjing, China
| | - Shanshan Chen
- Department of Neurology, The Affiliated Brain Hospital of Nanjing Medical University, Nanjing, China
| | - Honglin Ge
- Institute of Neuropsychiatry, The Affiliated Brain Hospital of Nanjing Medical University, Nanjing, China
- Institute of Brain Functional Imaging, Nanjing Medical University, Nanjing, China
| | - Zheng Yan
- Institute of Neuropsychiatry, The Affiliated Brain Hospital of Nanjing Medical University, Nanjing, China
- Institute of Brain Functional Imaging, Nanjing Medical University, Nanjing, China
| | - Chen Xue
- Department of Radiology, The Affiliated Brain Hospital of Nanjing Medical University, Nanjing, China
| | - Wenzhang Qi
- Department of Radiology, The Affiliated Brain Hospital of Nanjing Medical University, Nanjing, China
| | - Qianqian Yuan
- Department of Radiology, The Affiliated Brain Hospital of Nanjing Medical University, Nanjing, China
| | - Xuhong Liang
- Department of Radiology, The Affiliated Brain Hospital of Nanjing Medical University, Nanjing, China
| | - Xingjian Lin
- Department of Neurology, The Affiliated Brain Hospital of Nanjing Medical University, Nanjing, China
- *Correspondence: Xingjian Lin
| | - Jiu Chen
- Institute of Neuropsychiatry, The Affiliated Brain Hospital of Nanjing Medical University, Nanjing, China
- Institute of Brain Functional Imaging, Nanjing Medical University, Nanjing, China
- Jiu Chen
| |
Collapse
|
58
|
Arcaro MJ, Livingstone MS, Kay KN, Weiner KS. The retrocalcarine sulcus maps different retinotopic representations in macaques and humans. Brain Struct Funct 2022; 227:1227-1245. [PMID: 34921348 PMCID: PMC9046316 DOI: 10.1007/s00429-021-02427-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2021] [Accepted: 11/09/2021] [Indexed: 11/30/2022]
Abstract
Primate cerebral cortex is highly convoluted with much of the cortical surface buried in sulcal folds. The origins of cortical folding and its functional relevance have been a major focus of systems and cognitive neuroscience, especially when considering stereotyped patterns of cortical folding that are shared across individuals within a primate species and across multiple species. However, foundational questions regarding organizing principles shared across species remain unanswered. Taking a cross-species comparative approach with a careful consideration of historical observations, we investigate cortical folding relative to primary visual cortex (area V1). We identify two macroanatomical structures-the retrocalcarine and external calcarine sulci-in 24 humans and 6 macaque monkeys. We show that within species, these sulci are identifiable in all individuals, fall on a similar part of the V1 retinotopic map, and thus, serve as anatomical landmarks predictive of functional organization. Yet, across species, the underlying eccentricity representations corresponding to these macroanatomical structures differ strikingly across humans and macaques. Thus, the correspondence between retinotopic representation and cortical folding for an evolutionarily old structure like V1 is species-specific and suggests potential differences in developmental and experiential constraints across primates.
Collapse
Affiliation(s)
- Michael J Arcaro
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, 19146, USA
| | | | - Kendrick N Kay
- Center for Magnetic Resonance Research (CMRR), Department of Radiology, University of Minnesota, Minneapolis, MN, 55455, USA
| | - Kevin S Weiner
- Department of Psychology, University of California, Berkeley, Berkeley, CA, 94720, USA.
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, 94720, USA.
| |
Collapse
|
59
|
Abstract
Humans are exquisitely sensitive to the spatial arrangement of visual features in objects and scenes, but not in visual textures. Category-selective regions in the visual cortex are widely believed to underlie object perception, suggesting such regions should distinguish natural images of objects from synthesized images containing similar visual features in scrambled arrangements. Contrarily, we demonstrate that representations in category-selective cortex do not discriminate natural images from feature-matched scrambles but can discriminate images of different categories, suggesting a texture-like encoding. We find similar insensitivity to feature arrangement in Imagenet-trained deep convolutional neural networks. This suggests the need to reconceptualize the role of category-selective cortex as representing a basis set of complex texture-like features, useful for a myriad of behaviors. The human visual ability to recognize objects and scenes is widely thought to rely on representations in category-selective regions of the visual cortex. These representations could support object vision by specifically representing objects, or, more simply, by representing complex visual features regardless of the particular spatial arrangement needed to constitute real-world objects, that is, by representing visual textures. To discriminate between these hypotheses, we leveraged an image synthesis approach that, unlike previous methods, provides independent control over the complexity and spatial arrangement of visual features. We found that human observers could easily detect a natural object among synthetic images with similar complex features that were spatially scrambled. However, observer models built from BOLD responses from category-selective regions, as well as a model of macaque inferotemporal cortex and Imagenet-trained deep convolutional neural networks, were all unable to identify the real object. This inability was not due to a lack of signal to noise, as all observer models could predict human performance in image categorization tasks. How then might these texture-like representations in category-selective regions support object perception? An image-specific readout from category-selective cortex yielded a representation that was more selective for natural feature arrangement, showing that the information necessary for natural object discrimination is available. Thus, our results suggest that the role of the human category-selective visual cortex is not to explicitly encode objects but rather to provide a basis set of texture-like features that can be infinitely reconfigured to flexibly learn and identify new object categories.
Collapse
|
60
|
Häusler CO, Eickhoff SB, Hanke M. Processing of visual and non-visual naturalistic spatial information in the "parahippocampal place area". Sci Data 2022; 9:147. [PMID: 35365659 PMCID: PMC8975992 DOI: 10.1038/s41597-022-01250-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Accepted: 02/14/2022] [Indexed: 11/09/2022] Open
Abstract
The "parahippocampal place area" (PPA) in the human ventral visual stream exhibits increased hemodynamic activity correlated with the perception of landscape photos compared to faces or objects. Here, we investigate the perception of scene-related, spatial information embedded in two naturalistic stimuli. The same 14 participants were watching a Hollywood movie and listening to its audio-description as part of the open-data resource studyforrest.org. We model hemodynamic activity based on annotations of selected stimulus features, and compare results to a block-design visual localizer. On a group level, increased activation correlating with visual spatial information occurring in the movie is overlapping with a traditionally localized PPA. Activation correlating with semantic spatial information occurring in the audio-description is more restricted to the anterior PPA. On an individual level, we find significant bilateral activity in the PPA of nine individuals and unilateral activity in one individual. Results suggest that activation in the PPA generalizes to spatial information embedded in a movie and an auditory narrative, and may call for considering a functional subdivision of the PPA.
Collapse
Affiliation(s)
- Christian O Häusler
- Psychoinformatics Lab, Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich, Germany. .,Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University, Düsseldorf, Germany.
| | - Simon B Eickhoff
- Psychoinformatics Lab, Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.,Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University, Düsseldorf, Germany
| | - Michael Hanke
- Psychoinformatics Lab, Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.,Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University, Düsseldorf, Germany
| |
Collapse
|
61
|
Tullo MG, Almgren H, Van de Steen F, Sulpizio V, Marinazzo D, Galati G. Individual differences in mental imagery modulate effective connectivity of scene-selective regions during resting state. Brain Struct Funct 2022; 227:1831-1842. [PMID: 35312868 PMCID: PMC9098601 DOI: 10.1007/s00429-022-02475-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Accepted: 02/23/2022] [Indexed: 11/28/2022]
Abstract
Successful navigation relies on the ability to identify, perceive, and correctly process the spatial structure of a scene. It is well known that visual mental imagery plays a crucial role in navigation. Indeed, cortical regions encoding navigationally relevant information are also active during mental imagery of navigational scenes. However, it remains unknown whether their intrinsic activity and connectivity reflect the individuals' ability to imagine a scene. Here, we primarily investigated the intrinsic causal interactions among scene-selective brain regions such as Parahipoccampal Place Area (PPA), Retrosplenial Complex, and Occipital Place Area (OPA) using Dynamic Causal Modelling for resting-state functional magnetic resonance data. Second, we tested whether resting-state effective connectivity parameters among scene-selective regions could reflect individual differences in mental imagery in our sample, as assessed by the self-reported Vividness of Visual Imagery Questionnaire. We found an inhibitory influence of occipito-medial on temporal regions, and an excitatory influence of more anterior on more medial and posterior brain regions. Moreover, we found that a key role in imagery is played by the connection strength from OPA to PPA, especially in the left hemisphere, since the influence of the signal between these scene-selective regions positively correlated with good mental imagery ability. Our investigation contributes to the understanding of the complexity of the causal interaction among brain regions involved in navigation and provides new insight in understanding how an essential ability, such as mental imagery, can be explained by the intrinsic fluctuation of brain signal.
Collapse
Affiliation(s)
- Maria Giulia Tullo
- Department of Translational and Precision Medicine, "Sapienza" University of Rome, Via Benevento, 6, 00161, Roma, RM, Italy. .,Brain Imaging Laboratory, Department of Psychology, "Sapienza" University of Rome, Rome, Italy. .,PhD Program in Behavioral Neuroscience, "Sapienza" University of Rome, Rome, Italy.
| | - Hannes Almgren
- Department of Data Analysis, Faculty of Psychology and Educational Sciences, Ghent University, Ghent, Belgium.,Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.,Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Frederik Van de Steen
- Department of Data Analysis, Faculty of Psychology and Educational Sciences, Ghent University, Ghent, Belgium.,AIMS, Center For Neurosciences, Vrije Universiteit Brussel, Brussel, Belgium
| | - Valentina Sulpizio
- Brain Imaging Laboratory, Department of Psychology, "Sapienza" University of Rome, Rome, Italy.,Cognitive and Motor Rehabilitation and Neuroimaging Unit, IRCCS Santa Lucia Foundation, Rome, Italy
| | - Daniele Marinazzo
- Department of Data Analysis, Faculty of Psychology and Educational Sciences, Ghent University, Ghent, Belgium
| | - Gaspare Galati
- Department of Translational and Precision Medicine, "Sapienza" University of Rome, Via Benevento, 6, 00161, Roma, RM, Italy.,Brain Imaging Laboratory, Department of Psychology, "Sapienza" University of Rome, Rome, Italy
| |
Collapse
|
62
|
Ferko KM, Blumenthal A, Martin CB, Proklova D, Minos AN, Saksida LM, Bussey TJ, Khan AR, Köhler S. Activity in perirhinal and entorhinal cortex predicts perceived visual similarities among category exemplars with highest precision. eLife 2022; 11:66884. [PMID: 35311645 PMCID: PMC9020819 DOI: 10.7554/elife.66884] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Accepted: 03/17/2022] [Indexed: 01/22/2023] Open
Abstract
Vision neuroscience has made great strides in understanding the hierarchical organization of object representations along the ventral visual stream (VVS). How VVS representations capture fine-grained visual similarities between objects that observers subjectively perceive has received limited examination so far. In the current study, we addressed this question by focussing on perceived visual similarities among subordinate exemplars of real-world categories. We hypothesized that these perceived similarities are reflected with highest fidelity in neural activity patterns downstream from inferotemporal regions, namely in perirhinal (PrC) and anterolateral entorhinal cortex (alErC) in the medial temporal lobe. To address this issue with functional magnetic resonance imaging (fMRI), we administered a modified 1-back task that required discrimination between category exemplars as well as categorization. Further, we obtained observer-specific ratings of perceived visual similarities, which predicted behavioural discrimination performance during scanning. As anticipated, we found that activity patterns in PrC and alErC predicted the structure of perceived visual similarity relationships among category exemplars, including its observer-specific component, with higher precision than any other VVS region. Our findings provide new evidence that subjective aspects of object perception that rely on fine-grained visual differentiation are reflected with highest fidelity in the medial temporal lobe.
Collapse
Affiliation(s)
- Kayla M Ferko
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Robarts Research Institute Schulich School of Medicine and Dentistry, University of Western Ontario, London, Canada
| | - Anna Blumenthal
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Cervo Brain Research Center, University of Laval, Quebec, Canada
| | - Chris B Martin
- Department of Psychology, Florida State University, Tallahassee, United States
| | - Daria Proklova
- Brain and Mind Institute, University of Western Ontario, London, Canada
| | - Alexander N Minos
- Brain and Mind Institute, University of Western Ontario, London, Canada
| | - Lisa M Saksida
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Robarts Research Institute Schulich School of Medicine and Dentistry, University of Western Ontario, London, Canada.,Department of Physiology and Pharmacology, University of Western Ontario, London, Canada
| | - Timothy J Bussey
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Robarts Research Institute Schulich School of Medicine and Dentistry, University of Western Ontario, London, Canada.,Department of Physiology and Pharmacology, University of Western Ontario, London, Canada
| | - Ali R Khan
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Robarts Research Institute Schulich School of Medicine and Dentistry, University of Western Ontario, London, Canada.,School of Biomedical Engineering, University of Western Ontario, London, Canada.,Department of Medical Biophysics, University of Western Ontario, London, Canada
| | - Stefan Köhler
- Brain and Mind Institute, University of Western Ontario, London, Canada.,Department of Psychology, University of Western Ontario, London, Canada
| |
Collapse
|
63
|
Li W, Joseph Raj AN, Tjahjadi T, Zhuang Z. Fusion of ANNs as decoder of retinal spike trains for scene reconstruction. APPL INTELL 2022. [DOI: 10.1007/s10489-022-03402-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
64
|
Three cortical scene systems and their development. Trends Cogn Sci 2022; 26:117-127. [PMID: 34857468 PMCID: PMC8770598 DOI: 10.1016/j.tics.2021.11.002] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2021] [Revised: 10/14/2021] [Accepted: 11/06/2021] [Indexed: 02/03/2023]
Abstract
Since the discovery of three scene-selective regions in the human brain, a central assumption has been that all three regions directly support navigation. We propose instead that cortical scene processing regions support three distinct computational goals (and one not for navigation at all): (i) The parahippocampal place area supports scene categorization, which involves recognizing the kind of place we are in; (ii) the occipital place area supports visually guided navigation, which involves finding our way through the immediately visible environment, avoiding boundaries and obstacles; and (iii) the retrosplenial complex supports map-based navigation, which involves finding our way from a specific place to some distant, out-of-sight place. We further hypothesize that these systems develop along different timelines, with both navigation systems developing slower than the scene categorization system.
Collapse
|
65
|
Woolnough O, Kadipasaoglu CM, Conner CR, Forseth KJ, Rollo PS, Rollo MJ, Baboyan VG, Tandon N. Dataset of human intracranial recordings during famous landmark identification. Sci Data 2022; 9:28. [PMID: 35102154 PMCID: PMC8803828 DOI: 10.1038/s41597-022-01125-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2021] [Accepted: 11/03/2021] [Indexed: 11/18/2022] Open
Abstract
For most people, recalling information about familiar items in a visual scene is an effortless task, but it is one that depends on coordinated interactions of multiple, distributed neural components. We leveraged the high spatiotemporal resolution of direct intracranial recordings to better delineate the network dynamics underpinning visual scene recognition. We present a dataset of recordings from a large cohort of humans while they identified images of famous landmarks (50 individuals, 52 recording sessions, 6,775 electrodes, 6,541 trials). This dataset contains local field potential recordings derived from subdural and penetrating electrodes covering broad areas of cortex across both hemispheres. We provide this pre-processed data with behavioural metrics (correct/incorrect, response times) and electrode localisation in a population-normalised cortical surface space. This rich dataset will allow further investigation into the spatiotemporal progression of multiple neural processes underlying visual processing, scene recognition and cued memory recall.
Collapse
Affiliation(s)
- Oscar Woolnough
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX, 77030, United States of America
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX, 77030, United States of America
| | - Cihan M Kadipasaoglu
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX, 77030, United States of America
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX, 77030, United States of America
| | - Christopher R Conner
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX, 77030, United States of America
- Memorial Hermann Hospital, Texas Medical Center, Houston, TX, 77030, United States of America
| | - Kiefer J Forseth
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX, 77030, United States of America
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX, 77030, United States of America
| | - Patrick S Rollo
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX, 77030, United States of America
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX, 77030, United States of America
| | - Matthew J Rollo
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX, 77030, United States of America
| | - Vatche G Baboyan
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX, 77030, United States of America
| | - Nitin Tandon
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX, 77030, United States of America.
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX, 77030, United States of America.
- Memorial Hermann Hospital, Texas Medical Center, Houston, TX, 77030, United States of America.
| |
Collapse
|
66
|
Smith DM, Yang YY, Subramanian DL, Miller AMP, Bulkin DA, Law LM. The limbic memory circuit and the neural basis of contextual memory. Neurobiol Learn Mem 2022; 187:107557. [PMID: 34808337 PMCID: PMC8755583 DOI: 10.1016/j.nlm.2021.107557] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2021] [Revised: 11/12/2021] [Accepted: 11/15/2021] [Indexed: 01/03/2023]
Abstract
The hippocampus, retrosplenial cortex and anterior thalamus are key components of a neural circuit known to be involved in a variety of memory functions, including spatial, contextual and episodic memory. In this review, we focus on the role of this circuit in contextual memory processes. The background environment, or context, is a powerful cue for memory retrieval, and neural representations of the context provide a mechanism for efficiently retrieving relevant memories while avoiding interference from memories that belong to other contexts. Data from experimental lesions and neural manipulation techniques indicate that each of these regions is critical for contextual memory. Neurophysiological evidence from the hippocampus and retrosplenial cortex suggest that contextual information is represented within this circuit by population-level neural firing patterns that reliably differentiate each context a subject encounters. These findings indicate that encoding contextual information to support context-dependent memory retrieval is a key function of this circuit.
Collapse
Affiliation(s)
- David M Smith
- Department of Psychology, Cornell University, Ithaca, NY, United States.
| | - Yan Yu Yang
- Department of Psychology, Cornell University, Ithaca, NY, United States
| | | | - Adam M P Miller
- Department of Psychology, Cornell University, Ithaca, NY, United States
| | - David A Bulkin
- Department of Psychology, Cornell University, Ithaca, NY, United States
| | - L Matthew Law
- Department of Psychology, Cornell University, Ithaca, NY, United States
| |
Collapse
|
67
|
Li X, Zhu Q, Vanduffel W. Submillimeter fMRI reveals an extensive, fine-grained and functionally-relevant scene-processing network in monkeys. Prog Neurobiol 2022; 211:102230. [DOI: 10.1016/j.pneurobio.2022.102230] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Revised: 12/20/2021] [Accepted: 01/26/2022] [Indexed: 11/27/2022]
|
68
|
Polti I, Nau M, Kaplan R, van Wassenhove V, Doeller CF. Rapid encoding of task regularities in the human hippocampus guides sensorimotor timing. eLife 2022; 11:79027. [PMID: 36317500 PMCID: PMC9625083 DOI: 10.7554/elife.79027] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Accepted: 10/02/2022] [Indexed: 11/17/2022] Open
Abstract
The brain encodes the statistical regularities of the environment in a task-specific yet flexible and generalizable format. Here, we seek to understand this process by bridging two parallel lines of research, one centered on sensorimotor timing, and the other on cognitive mapping in the hippocampal system. By combining functional magnetic resonance imaging (fMRI) with a fast-paced time-to-contact (TTC) estimation task, we found that the hippocampus signaled behavioral feedback received in each trial as well as performance improvements across trials along with reward-processing regions. Critically, it signaled performance improvements independent from the tested intervals, and its activity accounted for the trial-wise regression-to-the-mean biases in TTC estimation. This is in line with the idea that the hippocampus supports the rapid encoding of temporal context even on short time scales in a behavior-dependent manner. Our results emphasize the central role of the hippocampus in statistical learning and position it at the core of a brain-wide network updating sensorimotor representations in real time for flexible behavior.
Collapse
Affiliation(s)
- Ignacio Polti
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Jebsen Centre for Alzheimer’s Disease, Norwegian University of Science and TechnologyTrondheimNorway,Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
| | - Matthias Nau
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Jebsen Centre for Alzheimer’s Disease, Norwegian University of Science and TechnologyTrondheimNorway,Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
| | - Raphael Kaplan
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Jebsen Centre for Alzheimer’s Disease, Norwegian University of Science and TechnologyTrondheimNorway,Department of Basic Psychology, Clinical Psychology, and Psychobiology, Universitat Jaume ICastellón de la PlanaSpain
| | - Virginie van Wassenhove
- CEA DRF/Joliot, NeuroSpin; INSERM, Cognitive Neuroimaging Unit; CNRS, Université Paris-SaclayGif-Sur-YvetteFrance
| | - Christian F Doeller
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Jebsen Centre for Alzheimer’s Disease, Norwegian University of Science and TechnologyTrondheimNorway,Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany,Wilhelm Wundt Institute of Psychology, Leipzig UniversityLeipzigGermany
| |
Collapse
|
69
|
Ma Q, Rolls ET, Huang CC, Cheng W, Feng J. Extensive cortical functional connectivity of the human hippocampal memory system. Cortex 2021; 147:83-101. [PMID: 35026557 DOI: 10.1016/j.cortex.2021.11.014] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2021] [Revised: 10/12/2021] [Accepted: 11/22/2021] [Indexed: 01/09/2023]
Abstract
The cortical connections of the human hippocampal memory system are fundamental to understanding its operation in health and disease, especially in the context of the great development of the human cortex. The functional connectivity of the human hippocampal system was analyzed in 172 participants imaged at 7T in the Human Connectome Project. The human hippocampus has high functional connectivity not only with the entorhinal cortex, but also with areas that are more distant in the ventral 'what' stream including the perirhinal cortex and temporal cortical visual areas. Parahippocampal gyrus TF in humans has connectivity with this ventral 'what' subsystem. Correspondingly for the dorsal stream, the hippocampus has high functional connectivity not only with the presubiculum, but also with areas more distant, the medial parahippocampal cortex TH which includes the parahippocampal place or scene area, the posterior cingulate including retrosplenial cortex, and the parietal cortex. Further, there is considerable cross connectivity between the ventral and dorsal streams with the hippocampus. The findings are supported by anatomical connections, which together provide an unprecedented and quantitative overview of the extensive cortical connectivity of the human hippocampal system that goes beyond hierarchically organised and segregated pathways connecting the hippocampus and neocortex, and leads to new concepts on the operation of the hippocampal memory system in humans.
Collapse
Affiliation(s)
- Qing Ma
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, China
| | - Edmund T Rolls
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, China; Department of Computer Science, University of Warwick, Coventry, UK; Oxford Centre for Computational Neuroscience, Oxford, UK.
| | - Chu-Chung Huang
- Shanghai Key Laboratory of Brain Functional Genomics (Ministry of Education), School of Psychology and Cognitive Science, East China Normal University, Shanghai, China
| | - Wei Cheng
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, China; Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence, Fudan University, Ministry of Education, Shanghai 200433, China; Fudan ISTBI-ZJNU Algorithm Centre for Brain-inspired Intelligence, Zhejiang Normal University, Jinhua, China.
| | - Jianfeng Feng
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, China; Department of Computer Science, University of Warwick, Coventry, UK; Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence, Fudan University, Ministry of Education, Shanghai 200433, China; Fudan ISTBI-ZJNU Algorithm Centre for Brain-inspired Intelligence, Zhejiang Normal University, Jinhua, China.
| |
Collapse
|
70
|
Groen IIA, Dekker TM, Knapen T, Silson EH. Visuospatial coding as ubiquitous scaffolding for human cognition. Trends Cogn Sci 2021; 26:81-96. [PMID: 34799253 DOI: 10.1016/j.tics.2021.10.011] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2021] [Revised: 10/19/2021] [Accepted: 10/19/2021] [Indexed: 01/28/2023]
Abstract
For more than 100 years we have known that the visual field is mapped onto the surface of visual cortex, imposing an inherently spatial reference frame on visual information processing. Recent studies highlight visuospatial coding not only throughout visual cortex, but also brain areas not typically considered visual. Such widespread access to visuospatial coding raises important questions about its role in wider cognitive functioning. Here, we synthesise these recent developments and propose that visuospatial coding scaffolds human cognition by providing a reference frame through which neural computations interface with environmental statistics and task demands via perception-action loops.
Collapse
Affiliation(s)
- Iris I A Groen
- Institute for Informatics, University of Amsterdam, Amsterdam, The Netherlands
| | - Tessa M Dekker
- Institute of Ophthalmology, University College London, London, UK
| | - Tomas Knapen
- Behavioral and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands; Spinoza Centre for NeuroImaging, Royal Dutch Academy of Sciences, Amsterdam, The Netherlands
| | - Edward H Silson
- Department of Psychology, School of Philosophy, Psychology & Language Sciences, University of Edinburgh, Edinburgh, UK.
| |
Collapse
|
71
|
One object, two networks? Assessing the relationship between the face and body-selective regions in the primate visual system. Brain Struct Funct 2021; 227:1423-1438. [PMID: 34792643 DOI: 10.1007/s00429-021-02420-7] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2021] [Accepted: 10/22/2021] [Indexed: 10/19/2022]
Abstract
Faces and bodies are often treated as distinct categories that are processed separately by face- and body-selective brain regions in the primate visual system. These regions occupy distinct regions of visual cortex and are often thought to constitute independent functional networks. Yet faces and bodies are part of the same object and their presence inevitably covary in naturalistic settings. Here, we re-evaluate both the evidence supporting the independent processing of faces and bodies and the organizational principles that have been invoked to explain this distinction. We outline four hypotheses ranging from completely separate networks to a single network supporting the perception of whole people or animals. The current evidence, especially in humans, is compatible with all of these hypotheses, making it presently unclear how the representation of faces and bodies is organized in the cortex.
Collapse
|
72
|
Lauer T, Schmidt F, Võ MLH. The role of contextual materials in object recognition. Sci Rep 2021; 11:21988. [PMID: 34753999 PMCID: PMC8578445 DOI: 10.1038/s41598-021-01406-z] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Accepted: 10/22/2021] [Indexed: 01/01/2023] Open
Abstract
While scene context is known to facilitate object recognition, little is known about which contextual "ingredients" are at the heart of this phenomenon. Here, we address the question of whether the materials that frequently occur in scenes (e.g., tiles in a bathroom) associated with specific objects (e.g., a perfume) are relevant for the processing of that object. To this end, we presented photographs of consistent and inconsistent objects (e.g., perfume vs. pinecone) superimposed on scenes (e.g., a bathroom) and close-ups of materials (e.g., tiles). In Experiment 1, consistent objects on scenes were named more accurately than inconsistent ones, while there was only a marginal consistency effect for objects on materials. Also, we did not find any consistency effect for scrambled materials that served as color control condition. In Experiment 2, we recorded event-related potentials and found N300/N400 responses-markers of semantic violations-for objects on inconsistent relative to consistent scenes. Critically, objects on materials triggered N300/N400 responses of similar magnitudes. Our findings show that contextual materials indeed affect object processing-even in the absence of spatial scene structure and object content-suggesting that material is one of the contextual "ingredients" driving scene context effects.
Collapse
Affiliation(s)
- Tim Lauer
- Scene Grammar Lab, Department of Psychology, Goethe University Frankfurt, Theodor-W.-Adorno-Platz 6, PEG 5.G144, 60323, Frankfurt am Main, Germany.
| | - Filipp Schmidt
- Department of Experimental Psychology, Justus Liebig University Giessen, 35394, Giessen, Germany
- Center for Mind, Brain and Behavior (CMBB), University of Marburg and Justus Liebig University, Giessen, Germany
| | - Melissa L-H Võ
- Scene Grammar Lab, Department of Psychology, Goethe University Frankfurt, Theodor-W.-Adorno-Platz 6, PEG 5.G144, 60323, Frankfurt am Main, Germany
| |
Collapse
|
73
|
Peer M, Epstein RA. The human brain uses spatial schemas to represent segmented environments. Curr Biol 2021; 31:4677-4688.e8. [PMID: 34473949 PMCID: PMC8578397 DOI: 10.1016/j.cub.2021.08.012] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2021] [Revised: 06/25/2021] [Accepted: 08/03/2021] [Indexed: 11/25/2022]
Abstract
Humans and animals use cognitive maps to represent the spatial structure of the environment. Although these maps are typically conceptualized as extending in an equipotential manner across known space, psychological evidence suggests that people mentally segment complex environments into subspaces. To understand the neurocognitive mechanisms behind this operation, we familiarized participants with a virtual courtyard that was divided into two halves by a river; we then used behavioral testing and fMRI to understand how spatial locations were encoded within this environment. Participants' spatial judgments and multivoxel activation patterns were affected by the division of the courtyard, indicating that the presence of a boundary can induce mental segmentation even when all parts of the environment are co-visible. In the hippocampus and occipital place area (OPA), the segmented organization of the environment manifested in schematic spatial codes that represented geometrically equivalent locations in the two subspaces as similar. In the retrosplenial complex (RSC), responses were more consistent with an integrated spatial map. These results demonstrate that people use both local spatial schemas and integrated spatial maps to represent segmented environment. We hypothesize that schematization may serve as a general mechanism for organizing complex knowledge structures in terms of their component elements.
Collapse
Affiliation(s)
- Michael Peer
- Department of Psychology, University of Pennsylvania, 3710 Hamilton Walk, Philadelphia, PA 19104, USA.
| | - Russell A Epstein
- Department of Psychology, University of Pennsylvania, 3710 Hamilton Walk, Philadelphia, PA 19104, USA
| |
Collapse
|
74
|
Wurm MF, Caramazza A. Two 'what' pathways for action and object recognition. Trends Cogn Sci 2021; 26:103-116. [PMID: 34702661 DOI: 10.1016/j.tics.2021.10.003] [Citation(s) in RCA: 38] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Revised: 09/03/2021] [Accepted: 10/01/2021] [Indexed: 10/20/2022]
Abstract
The ventral visual stream is conceived as a pathway for object recognition. However, we also recognize the actions an object can be involved in. Here, we show that action recognition critically depends on a pathway in lateral occipitotemporal cortex, partially overlapping and topographically aligned with object representations that are precursors for action recognition. By contrast, object features that are more relevant for object recognition, such as color and texture, are typically found in ventral occipitotemporal cortex. We argue that occipitotemporal cortex contains similarly organized lateral and ventral 'what' pathways for action and object recognition, respectively. This account explains a number of observed phenomena, such as the duplication of object domains and the specific representational profiles in lateral and ventral cortex.
Collapse
Affiliation(s)
- Moritz F Wurm
- Center for Mind/Brain Sciences - CIMeC, University of Trento, Corso Bettini 31, 38068 Rovereto, Italy.
| | - Alfonso Caramazza
- Center for Mind/Brain Sciences - CIMeC, University of Trento, Corso Bettini 31, 38068 Rovereto, Italy; Department of Psychology, Harvard University, 33 Kirkland St, Cambridge, MA 02138, USA
| |
Collapse
|
75
|
Abstract
During natural vision, our brains are constantly exposed to complex, but regularly structured environments. Real-world scenes are defined by typical part-whole relationships, where the meaning of the whole scene emerges from configurations of localized information present in individual parts of the scene. Such typical part-whole relationships suggest that information from individual scene parts is not processed independently, but that there are mutual influences between the parts and the whole during scene analysis. Here, we review recent research that used a straightforward, but effective approach to study such mutual influences: By dissecting scenes into multiple arbitrary pieces, these studies provide new insights into how the processing of whole scenes is shaped by their constituent parts and, conversely, how the processing of individual parts is determined by their role within the whole scene. We highlight three facets of this research: First, we discuss studies demonstrating that the spatial configuration of multiple scene parts has a profound impact on the neural processing of the whole scene. Second, we review work showing that cortical responses to individual scene parts are shaped by the context in which these parts typically appear within the environment. Third, we discuss studies demonstrating that missing scene parts are interpolated from the surrounding scene context. Bridging these findings, we argue that efficient scene processing relies on an active use of the scene's part-whole structure, where the visual brain matches scene inputs with internal models of what the world should look like.
Collapse
Affiliation(s)
- Daniel Kaiser
- Justus-Liebig-Universität Gießen, Germany.,Philipps-Universität Marburg, Germany.,University of York, United Kingdom
| | - Radoslaw M Cichy
- Freie Universität Berlin, Germany.,Humboldt-Universität zu Berlin, Germany.,Bernstein Centre for Computational Neuroscience Berlin, Germany
| |
Collapse
|
76
|
Chaisilprungraung T, Park S. "Scene" from inside: The representation of Observer's space in high-level visual cortex. Neuropsychologia 2021; 161:108010. [PMID: 34454940 DOI: 10.1016/j.neuropsychologia.2021.108010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2021] [Revised: 07/30/2021] [Accepted: 08/23/2021] [Indexed: 10/20/2022]
Abstract
Human observers are remarkably adept at perceiving and interacting with visual stimuli around them. Compared to visual stimuli like objects or faces, scenes are unique in that they provide enclosures for observers. An observer looks at a scene by being physically inside the scene. The current research explored this unique observer-scene relationship by studying the neural representation of scenes' spatial boundaries. Previous studies hypothesized that scenes' boundaries were processed in sets of high-level visual cortices. Notably, the parahippocampal place area (PPA), exhibited neural sensitivity to scenes that had closed vs. open spatial boundaries (Kravitz et al., 2011; Park et al., 2011). We asked whether this sensitivity reflected the openness of landscape (e.g., forest vs. beach), or the openness of the environment immediately surrounding the observer (i.e., whether a scene was viewed from inside vs. outside a room). Across two human fMRI experiments, we found that the PPA, as well as another well-known navigation-processing area, the occipital place area (OPA), processed scenes' boundaries according to the observer's space rather than the landscape. Moreover, we found that the PPA's activation pattern was susceptible to manipulations involving mid-level perceptual properties of scenes (e.g., rectilinear pattern of window frames), while the OPA's response was not. Our results have important implications for research in visual scene processing and suggest an important role of an observer's location in representing the spatial boundary, beyond the low-level visual input of a landscape.
Collapse
Affiliation(s)
| | - Soojin Park
- Department of Psychology, Yonsei University, Seoul, South Korea.
| |
Collapse
|
77
|
Whitwell RL, Striemer CL, Cant JS, Enns JT. The Ties that Bind: Agnosia, Neglect and Selective Attention to Visual Scale. Curr Neurol Neurosci Rep 2021; 21:54. [PMID: 34586544 DOI: 10.1007/s11910-021-01139-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/19/2021] [Indexed: 10/20/2022]
Abstract
PURPOSE OF REVIEW Historical and contemporary treatments of visual agnosia and neglect regard these disorders as largely unrelated. It is thought that damage to different neural processes leads directly to one or the other condition, yet apperceptive variants of agnosia and object-centered variants of neglect share remarkably similar deficits in the quality of conscious experience. Here we argue for a closer association between "apperceptive" variants of visual agnosia and "object-centered" variants of visual neglect. We introduce a theoretical framework for understanding these conditions based on "scale attention", which refers to selecting boundary and surface information at different levels of the structural hierarchy in the visual array. RECENT FINDINGS We review work on visual agnosia, the cortical structures and cortico-cortical pathways that underlie visual perception, visuospatial neglect and object-centered neglect, and attention to scale. We highlight direct and indirect pathways involved in these disorders and in attention to scale. The direct pathway involves the posterior vertical segments of the superior longitudinal fasciculus that are positioned to link the established dorsal and ventral attentional centers in the parietal cortex with structures in the inferior occipitotemporal cortex associated with visual apperceptive agnosia. The connections in the right hemisphere appear to be more important for visual conscious experience, whereas those in the left hemisphere appear to be more strongly associated with the planning and execution of visually guided grasps directed at multi-part objects such as tools. In the latter case, semantic and functional information must drive the selection of the appropriate hand posture and grasp points on the object. This view is supported by studies of grasping in patients with agnosia and in patients with neglect that show that the selection of grasp points when picking up a tool involves both scale attention and semantic contributions from inferotemporal cortex. The indirect pathways, which include the inferior fronto-occipital and horizontal components of the superior longitudinal fasciculi, involve the frontal lobe, working memory and the "multiple demands" network, which can shape the content of visual awareness through the maintenance of goal- and task-based abstractions and their influence on scale attention. Recent studies of human cortico-cortical pathways necessitate revisions to long-standing theoretical views on visual perception, visually guided action and their integrations. We highlight findings from a broad sample of seemingly disparate areas of research to support the proposal that attention to scale is necessary for typical conscious visual experience and for goal-directed actions that depend on functional and semantic information. Furthermore, we suggest that vertical pathways between the parietal and occipitotemporal cortex, along with indirect pathways that involve the premotor and prefrontal cortex, facilitate the operations of scale attention.
Collapse
Affiliation(s)
- Robert L Whitwell
- Department of Psychology, University of British Columbia, Vancouver, Canada.
| | | | - Jonathan S Cant
- Department of Psychology, University of Toronto Scarborough, Toronto, Canada
| | - James T Enns
- Department of Psychology, University of British Columbia, Vancouver, Canada
| |
Collapse
|
78
|
Ratan Murty NA, Bashivan P, Abate A, DiCarlo JJ, Kanwisher N. Computational models of category-selective brain regions enable high-throughput tests of selectivity. Nat Commun 2021; 12:5540. [PMID: 34545079 PMCID: PMC8452636 DOI: 10.1038/s41467-021-25409-6] [Citation(s) in RCA: 36] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Accepted: 08/04/2021] [Indexed: 02/08/2023] Open
Abstract
Cortical regions apparently selective to faces, places, and bodies have provided important evidence for domain-specific theories of human cognition, development, and evolution. But claims of category selectivity are not quantitatively precise and remain vulnerable to empirical refutation. Here we develop artificial neural network-based encoding models that accurately predict the response to novel images in the fusiform face area, parahippocampal place area, and extrastriate body area, outperforming descriptive models and experts. We use these models to subject claims of category selectivity to strong tests, by screening for and synthesizing images predicted to produce high responses. We find that these high-response-predicted images are all unambiguous members of the hypothesized preferred category for each region. These results provide accurate, image-computable encoding models of each category-selective region, strengthen evidence for domain specificity in the brain, and point the way for future research characterizing the functional organization of the brain with unprecedented computational precision.
Collapse
Affiliation(s)
- N Apurva Ratan Murty
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA.
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA.
- The Center for Brains, Minds and Machines, Massachusetts Institute of Technology, Cambridge, MA, USA.
| | - Pouya Bashivan
- Department of Physiology, McGill University, Montréal, QC, Canada
| | - Alex Abate
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - James J DiCarlo
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
- The Center for Brains, Minds and Machines, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Nancy Kanwisher
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
- The Center for Brains, Minds and Machines, Massachusetts Institute of Technology, Cambridge, MA, USA
| |
Collapse
|
79
|
Huang CC, Rolls ET, Hsu CCH, Feng J, Lin CP. Extensive Cortical Connectivity of the Human Hippocampal Memory System: Beyond the "What" and "Where" Dual Stream Model. Cereb Cortex 2021; 31:4652-4669. [PMID: 34013342 PMCID: PMC8866812 DOI: 10.1093/cercor/bhab113] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2021] [Revised: 04/09/2021] [Accepted: 04/10/2021] [Indexed: 10/06/2023] Open
Abstract
The human hippocampus is involved in forming new memories: damage impairs memory. The dual stream model suggests that object "what" representations from ventral stream temporal cortex project to the hippocampus via the perirhinal and then lateral entorhinal cortex, and spatial "where" representations from the dorsal parietal stream via the parahippocampal gyrus and then medial entorhinal cortex. The hippocampus can then associate these inputs to form episodic memories of what happened where. Diffusion tractography was used to reveal the direct connections of hippocampal system areas in humans. This provides evidence that the human hippocampus has extensive direct cortical connections, with connections that bypass the entorhinal cortex to connect with the perirhinal and parahippocampal cortex, with the temporal pole, with the posterior and retrosplenial cingulate cortex, and even with early sensory cortical areas. The connections are less hierarchical and segregated than in the dual stream model. This provides a foundation for a conceptualization for how the hippocampal memory system connects with the cerebral cortex and operates in humans. One implication is that prehippocampal cortical areas such as the parahippocampal TF and TH subregions and perirhinal cortices may implement specialized computations that can benefit from inputs from the dorsal and ventral streams.
Collapse
Affiliation(s)
- Chu-Chung Huang
- Key Laboratory of Brain Functional Genomics (MOE & STCSM), Affiliated Mental Health Center (ECNU), School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
- Shanghai Changning Mental Health Center, Shanghai 200335, China
| | - Edmund T Rolls
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai 200433, China
- Department of Computer Science, University of Warwick, Coventry CV4 7AL, UK
- Oxford Centre for Computational Neuroscience, Oxford, UK
| | - Chih-Chin Heather Hsu
- Center for Geriatrics and Gerontology, Taipei Veterans General Hospital, Taipei 11217, Taiwan
- Institute of Neuroscience, National Yang-Ming Chiao Tung University, Taipei 11217, Taiwan
| | - Jianfeng Feng
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai 200433, China
- Department of Computer Science, University of Warwick, Coventry CV4 7AL, UK
| | - Ching-Po Lin
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai 200433, China
- Center for Geriatrics and Gerontology, Taipei Veterans General Hospital, Taipei 11217, Taiwan
- Brain Research Center, National Yang-Ming Chiao Tung University, Taipei 11217, Taiwan
| |
Collapse
|
80
|
Jung Y, Walther DB. Neural Representations in the Prefrontal Cortex Are Task Dependent for Scene Attributes But Not for Scene Categories. J Neurosci 2021; 41:7234-7245. [PMID: 34103357 PMCID: PMC8387120 DOI: 10.1523/jneurosci.2816-20.2021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2020] [Revised: 05/28/2021] [Accepted: 06/03/2021] [Indexed: 11/21/2022] Open
Abstract
Natural scenes deliver rich sensory information about the world. Decades of research has shown that the scene-selective network in the visual cortex represents various aspects of scenes. However, less is known about how such complex scene information is processed beyond the visual cortex, such as in the prefrontal cortex. It is also unknown how task context impacts the process of scene perception, modulating which scene content is represented in the brain. In this study, we investigate these questions using scene images from four natural scene categories, which also depict two types of scene attributes, temperature (warm or cold), and sound level (noisy or quiet). A group of healthy human subjects from both sexes participated in the present study using fMRI. In the study, participants viewed scene images under two different task conditions: temperature judgment and sound-level judgment. We analyzed how these scene attributes and categories are represented across the brain under these task conditions. Our findings show that scene attributes (temperature and sound level) are only represented in the brain when they are task relevant. However, scene categories are represented in the brain, in both the parahippocampal place area and the prefrontal cortex, regardless of task context. These findings suggest that the prefrontal cortex selectively represents scene content according to task demands, but this task selectivity depends on the types of scene content: task modulates neural representations of scene attributes but not of scene categories.SIGNIFICANCE STATEMENT Research has shown that visual scene information is processed in scene-selective regions in the occipital and temporal cortices. Here, we ask how scene content is processed and represented beyond the visual brain, in the prefrontal cortex (PFC). We show that both scene categories and scene attributes are represented in PFC, with interesting differences in task dependency: scene attributes are only represented in PFC when they are task relevant, but scene categories are represented in PFC regardless of the task context. Together, our work shows that scene information is processed beyond the visual cortex, and scene representation in PFC reflects how adaptively our minds extract relevant information from a scene.
Collapse
Affiliation(s)
- Yaelan Jung
- Department of Psychology, University of Toronto, Toronto, Ontario M5S 3G3, Canada
| | - Dirk B Walther
- Department of Psychology, University of Toronto, Toronto, Ontario M5S 3G3, Canada
| |
Collapse
|
81
|
Çelik E, Keles U, Kiremitçi İ, Gallant JL, Çukur T. Cortical networks of dynamic scene category representation in the human brain. Cortex 2021; 143:127-147. [PMID: 34411847 DOI: 10.1016/j.cortex.2021.07.008] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2020] [Revised: 06/28/2021] [Accepted: 07/14/2021] [Indexed: 10/20/2022]
Abstract
Humans have an impressive ability to rapidly process global information in natural scenes to infer their category. Yet, it remains unclear whether and how scene categories observed dynamically in the natural world are represented in cerebral cortex beyond few canonical scene-selective areas. To address this question, here we examined the representation of dynamic visual scenes by recording whole-brain blood oxygenation level-dependent (BOLD) responses while subjects viewed natural movies. We fit voxelwise encoding models to estimate tuning for scene categories that reflect statistical ensembles of objects and actions in the natural world. We find that this scene-category model explains a significant portion of the response variance broadly across cerebral cortex. Cluster analysis of scene-category tuning profiles across cortex reveals nine spatially-segregated networks of brain regions consistently across subjects. These networks show heterogeneous tuning for a diverse set of dynamic scene categories related to navigation, human activity, social interaction, civilization, natural environment, non-human animals, motion-energy, and texture, suggesting that the organization of scene category representation is quite complex.
Collapse
Affiliation(s)
- Emin Çelik
- Neuroscience Program, Sabuncu Brain Research Center, Bilkent University, Ankara, Turkey; National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara, Turkey.
| | - Umit Keles
- National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara, Turkey; Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - İbrahim Kiremitçi
- Neuroscience Program, Sabuncu Brain Research Center, Bilkent University, Ankara, Turkey; National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara, Turkey
| | - Jack L Gallant
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA; Department of Bioengineering, University of California, Berkeley, Berkeley, CA, USA; Department of Psychology, University of California, Berkeley, CA, USA
| | - Tolga Çukur
- Neuroscience Program, Sabuncu Brain Research Center, Bilkent University, Ankara, Turkey; National Magnetic Resonance Research Center (UMRAM), Bilkent University, Ankara, Turkey; Department of Electrical and Electronics Engineering, Bilkent University, Ankara, Turkey
| |
Collapse
|
82
|
Ruiz-Rizzo AL, Pruitt PJ, Finke K, Müller HJ, Damoiseaux JS. Lower-Resolution Retrieval of Scenes in Older Adults With Subjective Cognitive Decline. Arch Clin Neuropsychol 2021; 37:408-422. [PMID: 34342647 PMCID: PMC8865194 DOI: 10.1093/arclin/acab061] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Revised: 06/09/2021] [Accepted: 07/05/2021] [Indexed: 11/21/2022] Open
Abstract
Objective Scenes with more perceptual detail can help detect subtle memory deficits more than scenes with less detail. Here, we investigated whether older adults with subjective cognitive decline (SCD) show less brain activation and more memory deficits to scenes with more (vs. scenes with less) perceptual detail compared to controls (CON). Method In 37 healthy older adults (SCD: 16), we measured blood oxygenation level-dependent-functional magnetic resonance imaging during encoding and behavioral performance during retrieval. Results During encoding, higher activation to scenes with more (vs. less) perceptual detail in the parahippocampal place area predicted better memory performance in SCD and CON. During retrieval, superior performance for new scenes with more (vs. less) perceptual detail was significantly more pronounced in CON than inSCD. Conclusions Together, these results suggest a present, but attenuated benefit from perceptual detail for memory retrieval in SCD. Memory complaints in SCD might, thus, refer to a decreased availability of perceptual detail of previously encoded stimuli.
Collapse
Affiliation(s)
- Adriana L Ruiz-Rizzo
- Department of Psychology, General and Experimental Psychology Unit, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Patrick J Pruitt
- Institute of Gerontology, Wayne State University, Detroit, MI, USA
| | - Kathrin Finke
- Department of Psychology, General and Experimental Psychology Unit, Ludwig-Maximilians-Universität München, Munich, Germany.,Hans-Berger Department of Neurology, Jena University Hospital, Jena, Germany
| | - Hermann J Müller
- Department of Psychology, General and Experimental Psychology Unit, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Jessica S Damoiseaux
- Institute of Gerontology, Wayne State University, Detroit, MI, USA.,Department of Psychology, Wayne State University, Detroit, MI, USA
| |
Collapse
|
83
|
Yates TS, Ellis CT, Turk-Browne NB. The promise of awake behaving infant fMRI as a deep measure of cognition. Curr Opin Behav Sci 2021. [DOI: 10.1016/j.cobeha.2020.11.007] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
|
84
|
Dwivedi K, Bonner MF, Cichy RM, Roig G. Unveiling functions of the visual cortex using task-specific deep neural networks. PLoS Comput Biol 2021; 17:e1009267. [PMID: 34388161 PMCID: PMC8407579 DOI: 10.1371/journal.pcbi.1009267] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Revised: 08/31/2021] [Accepted: 07/11/2021] [Indexed: 11/20/2022] Open
Abstract
The human visual cortex enables visual perception through a cascade of hierarchical computations in cortical regions with distinct functionalities. Here, we introduce an AI-driven approach to discover the functional mapping of the visual cortex. We related human brain responses to scene images measured with functional MRI (fMRI) systematically to a diverse set of deep neural networks (DNNs) optimized to perform different scene perception tasks. We found a structured mapping between DNN tasks and brain regions along the ventral and dorsal visual streams. Low-level visual tasks mapped onto early brain regions, 3-dimensional scene perception tasks mapped onto the dorsal stream, and semantic tasks mapped onto the ventral stream. This mapping was of high fidelity, with more than 60% of the explainable variance in nine key regions being explained. Together, our results provide a novel functional mapping of the human visual cortex and demonstrate the power of the computational approach.
Collapse
Affiliation(s)
- Kshitij Dwivedi
- Department of Education and Psychology, Freie Universität Berlin, Germany
- Department of Computer Science, Goethe University, Frankfurt am Main, Germany
| | - Michael F. Bonner
- Department of Cognitive Science, Johns Hopkins University, Baltimore, Maryland, United States of America
| | | | - Gemma Roig
- Department of Computer Science, Goethe University, Frankfurt am Main, Germany
| |
Collapse
|
85
|
Rolls ET. Learning Invariant Object and Spatial View Representations in the Brain Using Slow Unsupervised Learning. Front Comput Neurosci 2021; 15:686239. [PMID: 34366818 PMCID: PMC8335547 DOI: 10.3389/fncom.2021.686239] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2021] [Accepted: 06/29/2021] [Indexed: 11/13/2022] Open
Abstract
First, neurophysiological evidence for the learning of invariant representations in the inferior temporal visual cortex is described. This includes object and face representations with invariance for position, size, lighting, view and morphological transforms in the temporal lobe visual cortex; global object motion in the cortex in the superior temporal sulcus; and spatial view representations in the hippocampus that are invariant with respect to eye position, head direction, and place. Second, computational mechanisms that enable the brain to learn these invariant representations are proposed. For the ventral visual system, one key adaptation is the use of information available in the statistics of the environment in slow unsupervised learning to learn transform-invariant representations of objects. This contrasts with deep supervised learning in artificial neural networks, which uses training with thousands of exemplars forced into different categories by neuronal teachers. Similar slow learning principles apply to the learning of global object motion in the dorsal visual system leading to the cortex in the superior temporal sulcus. The learning rule that has been explored in VisNet is an associative rule with a short-term memory trace. The feed-forward architecture has four stages, with convergence from stage to stage. This type of slow learning is implemented in the brain in hierarchically organized competitive neuronal networks with convergence from stage to stage, with only 4-5 stages in the hierarchy. Slow learning is also shown to help the learning of coordinate transforms using gain modulation in the dorsal visual system extending into the parietal cortex and retrosplenial cortex. Representations are learned that are in allocentric spatial view coordinates of locations in the world and that are independent of eye position, head direction, and the place where the individual is located. This enables hippocampal spatial view cells to use idiothetic, self-motion, signals for navigation when the view details are obscured for short periods.
Collapse
Affiliation(s)
- Edmund T Rolls
- Oxford Centre for Computational Neuroscience, Oxford, United Kingdom.,Department of Computer Science, University of Warwick, Coventry, United Kingdom
| |
Collapse
|
86
|
Tharmaratnam V, Patel M, Lowe MX, Cant JS. Shared cognitive mechanisms involved in the processing of scene texture and scene shape. J Vis 2021; 21:11. [PMID: 34269793 PMCID: PMC8297417 DOI: 10.1167/jov.21.7.11] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/04/2022] Open
Abstract
Recent research has demonstrated that the parahippocampal place area represents both the shape and texture features of scenes, with the importance of each feature varying according to perceived scene category. Namely, shape features are predominately more diagnostic to the processing of artificial human–made scenes, while shape and texture are equally diagnostic in natural scene processing. However, to date little is known regarding the degree of interactivity or independence observed in the processing of these scene features. Furthermore, manipulating the scope of visual attention (i.e., globally vs. locally) when processing ensembles of multiple objects—stimuli that share a functional neuroanatomical link with scenes—has been shown to affect their cognitive visual representation. It remains unknown whether manipulating the scope of attention impacts scene processing in a similar manner. Using the well-established Garner speeded-classification behavioral paradigm, we investigated the influence of both feature diagnosticity and the scope of visual attention on potential interactivity or independence in the shape and texture processing of artificial human–made scenes. The results revealed asymmetric interference between scene shape and texture processing, with the more diagnostic feature (i.e., shape) interfering with the less diagnostic feature (i.e., texture), but not vice versa. Furthermore, this interference was attenuated and enhanced with more local and global visual processing strategies, respectively. These findings suggest that the scene shape and texture processing are mediated by shared cognitive mechanisms and that, although these representations are governed primarily via feature diagnosticity, they can nevertheless be influenced by the scope of visual attention.
Collapse
Affiliation(s)
| | | | - Matthew X Lowe
- Graduate Program in Psychology, University of Toronto, Toronto, ON, Canada.,
| | - Jonathan S Cant
- Graduate Program in Psychology, University of Toronto, Toronto, ON, Canada.,Department of Psychology, University of Toronto Scarborough, Toronto, ON, Canada.,
| |
Collapse
|
87
|
Katz JS, Forloines MR, Strassberg LR, Bondy B. Observational drawing in the brain: A longitudinal exploratory fMRI study. Neuropsychologia 2021; 160:107960. [PMID: 34274380 DOI: 10.1016/j.neuropsychologia.2021.107960] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2021] [Revised: 06/29/2021] [Accepted: 07/12/2021] [Indexed: 10/20/2022]
Abstract
Observational drawing involves acquiring a number of basic drawing techniques and concepts. There is limited knowledge on how observational drawing skills are represented by brain responses. Here, we investigate the behavioral and functional changes behind students learning to draw in a longitudinal study on 45 participants by testing art students (n = 26) at the beginning and end of a 16-week observational drawing course compared to a matched group of non-art students (n = 19). Four novel tasks were used that involve making decisions about light sources, tonal value, line variation and linear perspective using task-based 7 T-functional Magnetic Resonance Imaging (fMRI). While exploratory in nature, we expected to find improvement on each task over time and functional changes in the prefrontal cortex and cerebellum for the art students. Art students' performance significantly improved on the light sources, line variation, and linear perspective tasks and functional changes were found for the line variation, linear perspective, and tonal value tasks. Using whole brain analyses diffuse functional changes were discovered including prefrontal cortex areas and cerebellum. Brain areas involved in cognitive processing, including attention, decision making, motor control, top-down control, visual information processing, and working memory all functionally changed with experience. These findings demonstrate some of the first functional changes in the brain due to training in the arts and have implications for pedagogy and mental health.
Collapse
Affiliation(s)
- Jeffrey S Katz
- Department of Psychological Sciences, Auburn University, Auburn, AL, USA; AU MRI Research Center, Department of Electrical & Computer Engineering, Auburn University, Auburn, AL, USA; Alabama Advanced Imaging Consortium, Birmingham, AL, USA; Center for Neuroscience, Auburn University, Auburn, AL, USA.
| | - Martha R Forloines
- Alzheimer's Disease Research Center, Department of Neurology, University of California, Davis, Sacramento, CA, USA
| | - Lily R Strassberg
- Department of Psychological Sciences, Auburn University, Auburn, AL, USA
| | - Barbara Bondy
- Department of Art and Art History, Auburn University, Auburn, AL, USA
| |
Collapse
|
88
|
Persichetti AS, Denning JM, Gotts SJ, Martin A. A Data-Driven Functional Mapping of the Anterior Temporal Lobes. J Neurosci 2021; 41:6038-6049. [PMID: 34083253 PMCID: PMC8276737 DOI: 10.1523/jneurosci.0456-21.2021] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2021] [Revised: 05/03/2021] [Accepted: 05/12/2021] [Indexed: 12/14/2022] Open
Abstract
Although the anterior temporal lobe (ATL) comprises several anatomic and functional subdivisions, it is often reduced to a homogeneous theoretical entity, such as a domain-general convergence zone, or "hub," for semantic information. Methodological limitations are largely to blame for the imprecise mapping of function to structure in the ATL. There are two major obstacles to using fMRI to identify the precise functional organization of the ATL: the difficult choice of stimuli and tasks to activate, and dissociate, specific regions within the ATL; and poor signal quality because of magnetic field distortions near the sinuses. To circumvent these difficulties, we developed a data-driven parcellation routine using resting-state fMRI data (24 females, 64 males) acquired using a sequence that was optimized to enhance signal in the ATL. Focusing on patterns of functional connectivity between each ATL voxel and the rest of the brain, we found that the ATL comprises at least 34 distinct functional parcels that are arranged into bands along the lateral and ventral cortical surfaces, extending from the posterior temporal lobes into the temporal poles. In addition, the anterior region of the fusiform gyrus, most often cited as the location of the semantic hub, was found to be part of a domain-specific network associated with face and social processing, rather than a domain-general semantic hub. These findings offer a fine-grained functional map of the ATL and offer an initial step toward using more precise language to describe the locations of functional responses in this heterogeneous region of human cortex.SIGNIFICANCE STATEMENT The functional role of the anterior aspects of the temporal lobes (ATL) is a contentious issue. While it is likely that different regions within the ATL subserve unique cognitive functions, most studies revert to vaguely referring to particular functional regions as "the ATL," and, thus, the mapping of function to anatomy remains unclear. We used resting-state fMRI connectivity patterns between the ATL and the rest of the brain to reveal that the ATL comprises at least 34 distinct functional parcels that are organized into a three-level functional hierarchy. These results provide a detailed functional map of the anterior temporal lobes that can guide future research on how distinct regions within the ATL support diverse cognitive functions.
Collapse
Affiliation(s)
- Andrew S Persichetti
- Section on Cognitive Neuropsychology, Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892
| | - Joseph M Denning
- Section on Cognitive Neuropsychology, Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892
| | - Stephen J Gotts
- Section on Cognitive Neuropsychology, Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892
| | - Alex Martin
- Section on Cognitive Neuropsychology, Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892
| |
Collapse
|
89
|
Yang J, Huber L, Yu Y, Bandettini PA. Linking cortical circuit models to human cognition with laminar fMRI. Neurosci Biobehav Rev 2021; 128:467-478. [PMID: 34245758 DOI: 10.1016/j.neubiorev.2021.07.005] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2020] [Revised: 06/29/2021] [Accepted: 07/05/2021] [Indexed: 10/20/2022]
Abstract
Laboratory animal research has provided significant knowledge into the function of cortical circuits at the laminar level, which has yet to be fully leveraged towards insights about human brain function on a similar spatiotemporal scale. The use of functional magnetic resonance imaging (fMRI) in conjunction with neural models provides new opportunities to gain important insights from current knowledge. During the last five years, human studies have demonstrated the value of high-resolution fMRI to study laminar-specific activity in the human brain. This is mostly performed at ultra-high-field strengths (≥ 7 T) and is known as laminar fMRI. Advancements in laminar fMRI are beginning to open new possibilities for studying questions in basic cognitive neuroscience. In this paper, we first review recent methodological advances in laminar fMRI and describe recent human laminar fMRI studies. Then, we discuss how the use of laminar fMRI can help bridge the gap between cortical circuit models and human cognition.
Collapse
Affiliation(s)
- Jiajia Yang
- Graduate School of Interdisciplinary Science and Engineering in Health Systems, Okayama University, Okayama, Japan; Section on Functional Imaging Methods, National Institute of Mental Health, Bethesda, MD, USA.
| | - Laurentius Huber
- MR-Methods Group, Faculty of Psychology and Neuroscience, University of Maastricht, Maastricht, the Netherlands
| | - Yinghua Yu
- Graduate School of Interdisciplinary Science and Engineering in Health Systems, Okayama University, Okayama, Japan; Section on Functional Imaging Methods, National Institute of Mental Health, Bethesda, MD, USA
| | - Peter A Bandettini
- Section on Functional Imaging Methods, National Institute of Mental Health, Bethesda, MD, USA; Functional MRI Core Facility, National Institute of Mental Health, Bethesda, MD, USA
| |
Collapse
|
90
|
Bonner MF, Epstein RA. Object representations in the human brain reflect the co-occurrence statistics of vision and language. Nat Commun 2021; 12:4081. [PMID: 34215754 PMCID: PMC8253839 DOI: 10.1038/s41467-021-24368-2] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2020] [Accepted: 06/09/2021] [Indexed: 11/17/2022] Open
Abstract
A central regularity of visual perception is the co-occurrence of objects in the natural environment. Here we use machine learning and fMRI to test the hypothesis that object co-occurrence statistics are encoded in the human visual system and elicited by the perception of individual objects. We identified low-dimensional representations that capture the latent statistical structure of object co-occurrence in real-world scenes, and we mapped these statistical representations onto voxel-wise fMRI responses during object viewing. We found that cortical responses to single objects were predicted by the statistical ensembles in which they typically occur, and that this link between objects and their visual contexts was made most strongly in parahippocampal cortex, overlapping with the anterior portion of scene-selective parahippocampal place area. In contrast, a language-based statistical model of the co-occurrence of object names in written text predicted responses in neighboring regions of object-selective visual cortex. Together, these findings show that the sensory coding of objects in the human brain reflects the latent statistics of object context in visual and linguistic experience.
Collapse
Affiliation(s)
- Michael F Bonner
- Department of Cognitive Science, Johns Hopkins University, Baltimore, MD, USA.
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA.
| | - Russell A Epstein
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
91
|
Li J, Zhang R, Liu S, Liang Q, Zheng S, He X, Huang R. Human spatial navigation: Neural representations of spatial scales and reference frames obtained from an ALE meta-analysis. Neuroimage 2021; 238:118264. [PMID: 34129948 DOI: 10.1016/j.neuroimage.2021.118264] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Revised: 05/26/2021] [Accepted: 05/27/2021] [Indexed: 11/16/2022] Open
Abstract
Humans use different spatial reference frames (allocentric or egocentric) to navigate successfully toward their destination in different spatial scale spaces (environmental or vista). However, it remains unclear how the brain represents different spatial scales and different spatial reference frames. Thus, we conducted an activation likelihood estimation (ALE) meta-analysis of 47 fMRI articles involving human spatial navigation. We found that both the environmental and vista spaces activated the parahippocampal place area (PPA), retrosplenial complex (RSC), and occipital place area in the right hemisphere. The environmental space showed stronger activation than the vista space in the occipital and frontal regions. No brain region exhibited stronger activation for the vista than the environmental space. The allocentric and egocentric reference frames activated the bilateral PPA and right RSC. The allocentric frame showed more stronger activations than the egocentric frame in the right culmen, left middle frontal gyrus, and precuneus. No brain region displayed stronger activation for the egocentric than the allocentric navigation. Our findings suggest that navigation in different spatial scale spaces can evoke specific and common brain regions, and that the brain regions representing spatial reference frames are not absolutely separated.
Collapse
Affiliation(s)
- Jinhui Li
- Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; School of Psychology, Center for Studies of Psychological Application, and Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, Guangdong, 510631, China
| | - Ruibin Zhang
- Department of Psychology, School of Public Health, Southern Medical University (Guangdong Provincial Key Laboratory of Tropical Disease Research), Guangzhou, China; Department of Psychiatry, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Siqi Liu
- Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; School of Psychology, Center for Studies of Psychological Application, and Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, Guangdong, 510631, China
| | - Qunjun Liang
- Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; School of Psychology, Center for Studies of Psychological Application, and Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, Guangdong, 510631, China
| | - Senning Zheng
- Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; School of Psychology, Center for Studies of Psychological Application, and Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, Guangdong, 510631, China
| | - Xianyou He
- Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; School of Psychology, Center for Studies of Psychological Application, and Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, Guangdong, 510631, China
| | - Ruiwang Huang
- Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; School of Psychology, Center for Studies of Psychological Application, and Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, Guangdong, 510631, China.
| |
Collapse
|
92
|
Steel A, Billings MM, Silson EH, Robertson CE. A network linking scene perception and spatial memory systems in posterior cerebral cortex. Nat Commun 2021; 12:2632. [PMID: 33976141 PMCID: PMC8113503 DOI: 10.1038/s41467-021-22848-z] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2020] [Accepted: 04/05/2021] [Indexed: 02/03/2023] Open
Abstract
The neural systems supporting scene-perception and spatial-memory systems of the human brain are well-described. But how do these neural systems interact? Here, using fine-grained individual-subject fMRI, we report three cortical areas of the human brain, each lying immediately anterior to a region of the scene perception network in posterior cerebral cortex, that selectively activate when recalling familiar real-world locations. Despite their close proximity to the scene-perception areas, network analyses show that these regions constitute a distinct functional network that interfaces with spatial memory systems during naturalistic scene understanding. These "place-memory areas" offer a new framework for understanding how the brain implements memory-guided visual behaviors, including navigation.
Collapse
Affiliation(s)
- Adam Steel
- grid.254880.30000 0001 2179 2404Department of Psychology and Brain Sciences, Dartmouth College, Hanover, NH USA
| | - Madeleine M. Billings
- grid.254880.30000 0001 2179 2404Department of Psychology and Brain Sciences, Dartmouth College, Hanover, NH USA
| | - Edward H. Silson
- grid.4305.20000 0004 1936 7988Psychology, School of Philosophy, Psychology, and Language Sciences, University of Edinburgh, Edinburgh, EH8 9JZ UK
| | - Caroline E. Robertson
- grid.254880.30000 0001 2179 2404Department of Psychology and Brain Sciences, Dartmouth College, Hanover, NH USA
| |
Collapse
|
93
|
The parahippocampal place area and hippocampus encode the spatial significance of landmark objects. Neuroimage 2021; 236:118081. [PMID: 33882351 DOI: 10.1016/j.neuroimage.2021.118081] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2021] [Revised: 03/13/2021] [Accepted: 04/12/2021] [Indexed: 11/23/2022] Open
Abstract
Landmark objects are points of reference that can anchor one's internal cognitive map to the external world while navigating. They are especially useful in indoor environments where other cues such as spatial geometries are often similar across locations. We used functional magnetic resonance imaging (fMRI) and multivariate pattern analysis (MVPA) to understand how the spatial significance of landmark objects is represented in the human brain. Participants learned the spatial layout of a virtual building with arbitrary objects as unique landmarks in each room during a navigation task. They were scanned while viewing the objects before and after learning. MVPA revealed that the neural representation of landmark objects in the right parahippocampal place area (rPPA) and the hippocampus transformed systematically according to their locations. Specifically, objects in different rooms became more distinguishable than objects in the same room. These results demonstrate that rPPA and the hippocampus encode the spatial significance of landmark objects in indoor spaces.
Collapse
|
94
|
Rolls ET. Neurons including hippocampal spatial view cells, and navigation in primates including humans. Hippocampus 2021; 31:593-611. [PMID: 33760309 DOI: 10.1002/hipo.23324] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Revised: 03/01/2021] [Accepted: 03/13/2021] [Indexed: 01/11/2023]
Abstract
A new theory is proposed of mechanisms of navigation in primates including humans in which spatial view cells found in the primate hippocampus and parahippocampal gyrus are used to guide the individual from landmark to landmark. The navigation involves approach to each landmark in turn (taxis), using spatial view cells to identify the next landmark in the sequence, and does not require a topological map. Two other cell types found in primates, whole body motion cells, and head direction cells, can be utilized in the spatial view cell navigational mechanism, but are not essential. If the landmarks become obscured, then the spatial view representations can be updated by self-motion (idiothetic) path integration using spatial coordinate transform mechanisms in the primate dorsal visual system to transform from egocentric to allocentric spatial view coordinates. A continuous attractor network or time cells or working memory is used in this approach to navigation to encode and recall the spatial view sequences involved. I also propose how navigation can be performed using a further type of neuron found in primates, allocentric-bearing-to-a-landmark neurons, in which changes of direction are made when a landmark reaches a particular allocentric bearing. This is useful if a landmark cannot be approached. The theories are made explicit in models of navigation, which are then illustrated by computer simulations. These types of navigation are contrasted with triangulation, which requires a topological map. It is proposed that the first strategy utilizing spatial view cells is used frequently in humans, and is relatively simple because primates have spatial view neurons that respond allocentrically to locations in spatial scenes. An advantage of this approach to navigation is that hippocampal spatial view neurons are also useful for episodic memory, and for imagery.
Collapse
Affiliation(s)
- Edmund T Rolls
- Oxford Centre for Computational Neuroscience, Oxford, UK.,Department of Computer Science, University of Warwick, Coventry, UK
| |
Collapse
|
95
|
Interdependent self-construal predicts increased gray matter volume of scene processing regions in the brain. Biol Psychol 2021; 161:108050. [PMID: 33592270 DOI: 10.1016/j.biopsycho.2021.108050] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2020] [Revised: 02/08/2021] [Accepted: 02/09/2021] [Indexed: 12/21/2022]
Abstract
Interdependent self-construal (SC) is thought to lead to a more holistic cognitive style that emphasizes the processing of the background scene of a focal object. At present, little is known about whether the structural properties of the brain might underlie this functional relationship. Here, we examined the gray matter (GM) volume of three cortical regions involved in scene processing -- a cornerstone of contextual processing. Study 1 tested 78 European American non-student adults and found that interdependent (vs. independent) SC predicts higher GM volume in the parahippocampal place area (PPA), one of the three target regions. Testing both European American and East Asian college students (total N = 126), Study 2 replicated this association. Moreover, the GM volume of all the three target regions was greater for East Asians than for European Americans. Our findings suggest that there is a structural neural underpinning for the cultural variation in cognitive style.
Collapse
|
96
|
Abstract
Rapid visual perception is often viewed as a bottom-up process. Category-preferred neural regions are often characterized as automatic, default processing mechanisms for visual inputs of their categorical preference. To explore the sensitivity of such regions to top-down information, we examined three scene-preferring brain regions, the occipital place area (OPA), the parahippocampal place area (PPA), and the retrosplenial complex (RSC) and tested whether the processing of outdoor scenes is influenced by the functional contexts in which they are seen. Context was manipulated by presenting real-world landscape images as if being viewed through a window or within a picture frame-manipulations that do not affect scene content but do affect one's functional knowledge regarding the scene. This manipulation influences neural scene processing (as measured by fMRI): The OPA and the PPA exhibited greater neural activity when participants viewed images as if through a window as compared with within a picture frame, whereas the RSC did not show this difference. In a separate behavioral experiment, functional context affected scene memory in predictable directions (boundary extension). Our interpretation is that the window context denotes three dimensionality, therefore rendering the perceptual experience of viewing landscapes as more realistic. Conversely, the frame context denotes a 2-D image. As such, more spatially biased scene representations in the OPA and the PPA are influenced by differences in top-down, perceptual expectations generated from context. In contrast, more semantically biased scene representations in the RSC are likely to be less affected by top-down signals that carry information about the physical layout of a scene.
Collapse
|
97
|
Srokova S, Hill PF, Elward RL, Rugg MD. Effects of age on goal-dependent modulation of episodic memory retrieval. Neurobiol Aging 2021; 102:73-88. [PMID: 33765433 DOI: 10.1016/j.neurobiolaging.2021.02.004] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Revised: 02/02/2021] [Accepted: 02/03/2021] [Indexed: 10/22/2022]
Abstract
Retrieval gating refers to the ability to modulate the retrieval of features of a single memory episode according to behavioral goals. Recent findings demonstrate that younger adults engage retrieval gating by attenuating the representation of task-irrelevant features of an episode. Here, we examine whether retrieval gating varies with age. Younger and older adults incidentally encoded words superimposed over scenes or scrambled backgrounds that were displayed in one of three spatial locations. Participants subsequently underwent fMRI as they completed two memory tasks: the background task, which tested memory for the word's background, and the location task, testing memory for the word's location. Employing univariate and multivariate approaches, we demonstrated that younger, but not older adults, exhibited attenuated reinstatement of scene information when it was goal-irrelevant (during the location task). Additionally, in younger adults only, the strength of scene reinstatement in the parahippocampal place area during the background task was related to item and source memory performance. Together, these findings point to an age-related decline in the ability to engage retrieval gating.
Collapse
Affiliation(s)
- Sabina Srokova
- Center for Vital Longevity, University of Texas at Dallas, Dallas, TX, USA; School of Behavioral and Brain Sciences, University of Texas at Dallas, Richardson, TX, USA.
| | - Paul F Hill
- Center for Vital Longevity, University of Texas at Dallas, Dallas, TX, USA; School of Behavioral and Brain Sciences, University of Texas at Dallas, Richardson, TX, USA
| | - Rachael L Elward
- School of Applied Sciences, Division of Psychology, London South Bank University, London, UK
| | - Michael D Rugg
- Center for Vital Longevity, University of Texas at Dallas, Dallas, TX, USA; School of Behavioral and Brain Sciences, University of Texas at Dallas, Richardson, TX, USA; School of Psychology, University of East Anglia, Norwich, UK; Department of Psychiatry, University of Texas Southwestern Medical Center, Dallas, TX, USA
| |
Collapse
|
98
|
Cox CR, Rogers TT. Finding Distributed Needles in Neural Haystacks. J Neurosci 2021; 41:1019-1032. [PMID: 33334868 PMCID: PMC7880292 DOI: 10.1523/jneurosci.0904-20.2020] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2020] [Revised: 12/02/2020] [Accepted: 12/04/2020] [Indexed: 11/21/2022] Open
Abstract
The human cortex encodes information in complex networks that can be anatomically dispersed and variable in their microstructure across individuals. Using simulations with neural network models, we show that contemporary statistical methods for functional brain imaging-including univariate contrast, searchlight multivariate pattern classification, and whole-brain decoding with L1 or L2 regularization-each have critical and complementary blind spots under these conditions. We then introduce the sparse-overlapping-sets (SOS) LASSO-a whole-brain multivariate approach that exploits structured sparsity to find network-distributed information-and show in simulation that it captures the advantages of other approaches while avoiding their limitations. When applied to fMRI data to find neural responses that discriminate visually presented faces from other visual stimuli, each method yields a different result, but existing approaches all support the canonical view that face perception engages localized areas in posterior occipital and temporal regions. In contrast, SOS LASSO uncovers a network spanning all four lobes of the brain. The result cannot reflect spurious selection of out-of-system areas because decoding accuracy remains exceedingly high even when canonical face and place systems are removed from the dataset. When used to discriminate visual scenes from other stimuli, the same approach reveals a localized signal consistent with other methods-illustrating that SOS LASSO can detect both widely distributed and localized representational structure. Thus, structured sparsity can provide an unbiased method for testing claims of functional localization. For faces and possibly other domains, such decoding may reveal representations more widely distributed than previously suspected.SIGNIFICANCE STATEMENT Brain systems represent information as patterns of activation over neural populations connected in networks that can be widely distributed anatomically, variable across individuals, and intermingled with other networks. We show that four widespread statistical approaches to functional brain imaging have critical blind spots in this scenario and use simulations with neural network models to illustrate why. We then introduce a new approach designed specifically to find radically distributed representations in neural networks. In simulation and in fMRI data collected in the well studied domain of face perception, the new approach discovers extensive signal missed by the other methods-suggesting that prior functional imaging work may have significantly underestimated the degree to which neurocognitive representations are distributed and variable across individuals.
Collapse
Affiliation(s)
- Christopher R Cox
- Department of Psychology, Louisiana State University, Baton Rouge, Louisiana 70803
| | - Timothy T Rogers
- Department of Psychology, University of Wisconsin, Madison, Wisconsin 53706
| |
Collapse
|
99
|
The brain dynamics of architectural affordances during transition. Sci Rep 2021; 11:2796. [PMID: 33531612 PMCID: PMC7854617 DOI: 10.1038/s41598-021-82504-w] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2020] [Accepted: 01/20/2021] [Indexed: 01/30/2023] Open
Abstract
Action is a medium of collecting sensory information about the environment, which in turn is shaped by architectural affordances. Affordances characterize the fit between the physical structure of the body and capacities for movement and interaction with the environment, thus relying on sensorimotor processes associated with exploring the surroundings. Central to sensorimotor brain dynamics, the attentional mechanisms directing the gating function of sensory signals share neuronal resources with motor-related processes necessary to inferring the external causes of sensory signals. Such a predictive coding approach suggests that sensorimotor dynamics are sensitive to architectural affordances that support or suppress specific kinds of actions for an individual. However, how architectural affordances relate to the attentional mechanisms underlying the gating function for sensory signals remains unknown. Here we demonstrate that event-related desynchronization of alpha-band oscillations in parieto-occipital and medio-temporal regions covary with the architectural affordances. Source-level time-frequency analysis of data recorded in a motor-priming Mobile Brain/Body Imaging experiment revealed strong event-related desynchronization of the alpha band to originate from the posterior cingulate complex, the parahippocampal region as well as the occipital cortex. Our results firstly contribute to the understanding of how the brain resolves architectural affordances relevant to behaviour. Second, our results indicate that the alpha-band originating from the occipital cortex and parahippocampal region covaries with the architectural affordances before participants interact with the environment, whereas during the interaction, the posterior cingulate cortex and motor areas dynamically reflect the affordable behaviour. We conclude that the sensorimotor dynamics reflect behaviour-relevant features in the designed environment.
Collapse
|
100
|
Myelin development in visual scene-network tracts beyond late childhood: A multimethod neuroimaging study. Cortex 2021; 137:18-34. [PMID: 33588130 DOI: 10.1016/j.cortex.2020.12.016] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2020] [Revised: 07/30/2020] [Accepted: 12/14/2020] [Indexed: 12/15/2022]
Abstract
The visual scene-network-comprising the parahippocampal place area (PPA), retrosplenial cortex (RSC), and occipital place area (OPA)-shows a prolonged functional development. Structural development of white matter that underlies the scene-network has not been investigated despite its potential influence on scene-network function. The key factor for white matter maturation is myelination. However, research on myelination using the gold standard method of post-mortem histology is scarce. In vivo alternatives diffusion-weighted imaging (DWI) and myelin water imaging (MWI) so far report broad-scale findings that prohibit inferences concerning the scene-network. Here, we combine MWI, DWI tractography, and fMRI to investigate myelination in scene-network tracts in middle childhood, late childhood, and adulthood. We report increasing myelin from middle childhood to adulthood in right PPA-OPA, and trends towards increases in the left and right RSC-OPA tracts. Investigating tracts to regions highly connected with the scene-network, such as early visual cortex and the hippocampus, did not yield any significant age group differences. Our findings indicate that structural development coincides with functional development in the scene-network, possibly enabling structure-function interactions.
Collapse
|