Category Selectivity for Face and Scene Recognition in Human Medial Parietal Cortex

Dissociable Contributions of the Medial Parietal Cortex to Recognition Memory

Human neuroimaging studies of episodic memory retrieval routinely observe the engagement of specific cortical regions beyond the medial temporal lobe. Of these, medial parietal cortex (MPC) is of particular interest given its distinct functional characteristics during different retrieval tasks. Specifically, while recognition and autobiographical recall tasks are both used to probe episodic retrieval, these paradigms consistently drive distinct spatial patterns of response within MPC. However, other studies have emphasized alternate MPC functional dissociations in terms of brain network connectivity profiles or stimulus category selectivity. As the unique contributions of MPC to episodic memory remain unclear, adjudicating between these different accounts can provide better consensus regarding MPC function. Therefore, we used a precision-neuroimaging dataset (7T functional magnetic resonance imaging) to examine how MPC regions are differentially engaged during recognition memory and how these task-related dissociations may also reflect distinct connectivity and stimulus category functional profiles. We observed interleaved, though spatially distinct, subregions of MPC where responses were sensitive to either recognition decisions or the semantic representation of stimuli. In addition, this dissociation was further accentuated by functional subregions displaying distinct profiles of connectivity with the hippocampus during task and rest. Finally, we show that recent observations of dissociable person and place selectivity within the MPC reflect category-specific responses from within identified semantic regions that are sensitive to mnemonic demands. Together, by examining precision functional mapping within individuals, these data suggest that previously distinct observations of functional dissociation within MPC conform to a common principle of organization throughout hippocampal-neocortical memory systems.

Ventral temporal and posteromedial sulcal morphology in autism spectrum disorder

Two parallel research tracks link the morphology of small and shallow indentations, or sulci, of the cerebral cortex with functional features of the cortex and human cognition, respectively. The first track identified a relationship between the mid-fusiform sulcus (MFS) in ventral temporal cortex (VTC) and cognition in individuals with Autism Spectrum Disorder (ASD). The second track identified a new sulcus, the inframarginal sulcus (IFRMS), that serves as a tripartite landmark within the posteromedial cortex (PMC). As VTC and PMC are structurally and functionally different in ASD, here, we integrated these two tracks and tested if there are morphological differences in VTC and PMC sulci in a sample of young (5-17 years old) male participants (50 participants with ASD and 50 neurotypical controls). Our approach replicates and extends recent findings in four ways. First, regarding replication, the standard deviation (STD) of MFS cortical thickness (CT) was increased in ASD. Second, MFS length was shorter in ASD. Third, the CT STD effect extended to other VTC and to PMC sulci. Fourth, additional morphological features of VTC sulci (depth, surface area, gray matter volume) and PMC sulci (mean CT) were decreased in ASD, including putative tertiary sulci, which emerge last in gestation and continue to develop after birth. To our knowledge, this study is the most extensive comparison of the sulcal landscape (including putative tertiary sulci) in multiple cortical expanses between individuals with ASD and NTs based on manually defined sulci at the level of individual hemispheres, providing novel targets for future studies of neurodevelopmental disorders more broadly.

Scene construction and autobiographical memory retrieval in autism spectrum disorder

Individuals with autism spectrum disorder (ASD) frequently exhibit difficulties in retrieving autobiographical memories (AMs) of specific events from their life. Such memory deficits are frequently attributed to underlying disruptions in self-referential or social cognition processes. This makes intuitive sense as these are hallmarks of ASD. However, an emerging literature suggests that parallel deficits also exist in ASD individuals' ability to reconstruct the rich spatial contexts in which events occur. This is a capacity known as scene construction, and in typically developing individuals is considered a core process in retrieving AMs. In this review, we discuss evidence of difficulties with scene construction in ASD, drawing upon experiments that involve AM retrieval, other forms of mental time travel, and spatial navigation. We also highlight aspects of extant data that cannot be accounted for using purely social explanations of memory deficits in ASD. We conclude by identifying key questions raised by our framework and suggest how they might be addressed in future research.

The spatiotemporal dynamics of semantic integration in the human brain

Language depends critically on the integration of lexical information across multiple words to derive semantic concepts. Limitations of spatiotemporal resolution have previously rendered it difficult to isolate processes involved in semantic integration. We utilized intracranial recordings in epilepsy patients (n = 58) who read written word definitions. Descriptions were either referential or non-referential to a common object. Semantically referential sentences enabled high frequency broadband gamma activation (70-150 Hz) of the inferior frontal sulcus (IFS), medial parietal cortex, orbitofrontal cortex (OFC) and medial temporal lobe in the left, language-dominant hemisphere. IFS, OFC and posterior middle temporal gyrus activity was modulated by the semantic coherence of non-referential sentences, exposing semantic effects that were independent of task-based referential status. Components of this network, alongside posterior superior temporal sulcus, were engaged for referential sentences that did not clearly reduce the lexical search space by the final word. These results indicate the existence of complementary cortical mosaics for semantic integration in posterior temporal and inferior frontal cortex.

Adding Meaning to Memories: How Parietal Cortex Combines Semantic Content with Episodic Experience

Neuroimaging studies of human memory have consistently found that univariate responses in parietal cortex track episodic experience with stimuli (whether stimuli are 'old' or 'new'). More recently, pattern-based fMRI studies have shown that parietal cortex also carries information about the semantic content of remembered experiences. However, it is not well understood how memory-based and content-based signals are integrated within parietal cortex. Here, in humans (males and females), we used voxel-wise encoding models and a recognition memory task to predict the fMRI activity patterns evoked by complex natural scene images based on (1) the episodic history and (2) the semantic content of each image. Models were generated and compared across distinct subregions of parietal cortex and for occipitotemporal cortex. We show that parietal and occipitotemporal regions each encode memory and content information, but they differ in how they combine this information. Among parietal subregions, angular gyrus was characterized by robust and overlapping effects of memory and content. Moreover, subject-specific semantic tuning functions revealed that successful recognition shifted the amplitude of tuning functions in angular gyrus but did not change the selectivity of tuning. In other words, effects of memory and content were additive in angular gyrus. This pattern of data contrasted with occipitotemporal cortex where memory and content effects were interactive: memory effects were preferentially expressed by voxels tuned to the content of a remembered image. Collectively, these findings provide unique insight into how parietal cortex combines information about episodic memory and semantic content.SIGNIFICANCE STATEMENT Neuroimaging studies of human memory have identified multiple brain regions that not only carry information about "whether" a visual stimulus is successfully recognized but also "what" the content of that stimulus includes. However, a fundamental and open question concerns how the brain integrates these two types of information (memory and content). Here, using a powerful combination of fMRI analysis methods, we show that parietal cortex, particularly the angular gyrus, robustly combines memory- and content-related information, but these two forms of information are represented via additive, independent signals. In contrast, memory effects in high-level visual cortex critically depend on (and interact with) content representations. Together, these findings reveal multiple and distinct ways in which the brain combines memory- and content-related information.

Brain Network Classification for Accurate Detection of Alzheimer's Disease via Manifold Harmonic Discriminant Analysis

Mounting evidence shows that Alzheimer's disease (AD) manifests the dysfunction of the brain network much earlier before the onset of clinical symptoms, making its early diagnosis possible. Current brain network analyses treat high-dimensional network data as a regular matrix or vector, which destroys the essential network topology, thereby seriously affecting diagnosis accuracy. In this context, harmonic waves provide a solid theoretical background for exploring brain network topology. However, the harmonic waves are originally intended to discover neurological disease propagation patterns in the brain, which makes it difficult to accommodate brain disease diagnosis with high heterogeneity. To address this challenge, this article proposes a network manifold harmonic discriminant analysis (MHDA) method for accurately detecting AD. Each brain network is regarded as an instance drawn on a Stiefel manifold. Every instance is represented by a set of orthonormal eigenvectors (i.e., harmonic waves) derived from its Laplacian matrix, which fully respects the topological structure of the brain network. An MHDA method within the Stiefel space is proposed to identify the group-dependent common harmonic waves, which can be used as group-specific references for downstream analyses. Extensive experiments are conducted to demonstrate the effectiveness of the proposed method in stratifying cognitively normal (CN) controls, mild cognitive impairment (MCI), and AD.

Uncovering the fast, directional signal flow through the human temporal pole during semantic processing

The temporal pole (TP) plays a central role in semantic memory, yet its neural machinery is unknown. Intracerebral recordings in patients discriminating visually the gender or actions of an actor, yielded gender discrimination responses in the ventrolateral (VL) and tip (T) regions of right TP. Granger causality revealed task-specific signals travelling first forward from VL to T, under control of orbitofrontal cortex (OFC) and neighboring prefrontal cortex, and then, strongly, backwards from T to VL. Many other cortical regions provided inputs to or received outputs from both TP regions, often with longer delays, with ventral temporal afferents to VL signaling the actor's physical appearance. The TP response timing reflected more that of the connections to VL, controlled by OFC, than that of the input leads themselves. Thus, visual evidence for gender categories, collected by VL, activates category labels in T, and consequently, category features in VL, indicating a two-stage representation of semantic categories in TP.

ROSE: A Neurocomputational Architecture for Syntax

A comprehensive model of natural language processing in the brain must accommodate four components: representations, operations, structures and encoding. It further requires a principled account of how these different components mechanistically, and causally, relate to each another. While previous models have isolated regions of interest for structure-building and lexical access, and have utilized specific neural recording measures to expose possible signatures of syntax, many gaps remain with respect to bridging distinct scales of analysis that map onto these four components. By expanding existing accounts of how neural oscillations can index various linguistic processes, this article proposes a neurocomputational architecture for syntax, termed the ROSE model (Representation, Operation, Structure, Encoding). Under ROSE, the basic data structures of syntax are atomic features, types of mental representations (R), and are coded at the single-unit and ensemble level. Elementary computations (O) that transform these units into manipulable objects accessible to subsequent structure-building levels are coded via high frequency broadband γ activity. Low frequency synchronization and cross-frequency coupling code for recursive categorial inferences (S). Distinct forms of low frequency coupling and phase-amplitude coupling (δ-θ coupling via pSTS-IFG; θ-γ coupling via IFG to conceptual hubs in lateral and ventral temporal cortex) then encode these structures onto distinct workspaces (E). Causally connecting R to O is spike-phase/LFP coupling; connecting O to S is phase-amplitude coupling; connecting S to E is a system of frontotemporal traveling oscillations; connecting E back to lower levels is low-frequency phase resetting of spike-LFP coupling. This compositional neural code has important implications for algorithmic accounts, since it makes concrete predictions for the appropriate level of study for psycholinguistic parsing models. ROSE is reliant on neurophysiologically plausible mechanisms, is supported at all four levels by a range of recent empirical research, and provides an anatomically precise and falsifiable grounding for the basic property of natural language syntax: hierarchical, recursive structure-building.

Sustained neural representations of personally familiar people and places during cued recall

The recall and visualization of people and places from memory is an everyday occurrence, yet the neural mechanisms underpinning this phenomenon are not well understood. In particular, the temporal characteristics of the internal representations generated by active recall are unclear. Here, we used magnetoencephalography (MEG) and multivariate pattern analysis to measure the evolving neural representation of familiar places and people across the whole brain when human participants engage in active recall. To isolate self-generated imagined representations, we used a retro-cue paradigm in which participants were first presented with two possible labels before being cued to recall either the first or second item. We collected personalized labels for specific locations and people familiar to each participant. Importantly, no visual stimuli were presented during the recall period, and the retro-cue paradigm allowed the dissociation of responses associated with the labels from those corresponding to the self-generated representations. First, we found that following the retro-cue it took on average ∼1000 ms for distinct neural representations of freely recalled people or places to develop. Second, we found distinct representations of personally familiar concepts throughout the 4 s recall period. Finally, we found that these representations were highly stable and generalizable across time. These results suggest that self-generated visualizations and recall of familiar places and people are subserved by a stable neural mechanism that operates relatively slowly when under conscious control.

Proper and common names in the semantic system

Proper names are an important part of language and communication. They are thought to have a special status due to their neuropsychological and psycholinguistic profile. To what extent proper names rely on the same semantic system as common names is not clear. In an fMRI study, we presented the same group of participants with both proper and common names to compare the associated activations. Both person and place names, as well as personally familiar and famous names were used, and compared with words representing concrete and abstract concepts. A whole-brain analysis was followed by a detailed analysis of subdivisions of four regions of interest known to play a central role in the semantic system: angular gyrus, anterior temporal lobe, posterior cingulate complex, and medial temporal lobe. We found that most subdivisions within these regions bilaterally were activated by both proper names and common names. The bilateral perirhinal and right entorhinal cortex showed a response specific to proper names, suggesting an item-specific role in retrieving person and place related information. While activation to person and place names overlapped greatly, place names were differentiated by activating areas associated with spatial memory and navigation. Person names showed greater right hemisphere involvement compared to places, suggesting a wider range of associations. Personally familiar names showed stronger activation bilaterally compared to famous names, indicating representations that are enhanced by autobiographic and episodic details. Both proper and common names are processed in the wider semantic system that contains associative, episodic, and spatial components. Processing of proper names is characterized by a somewhat stronger involvement these components, rather than by a fundamentally different system.

Functional specialization of parallel distributed networks revealed by analysis of trial-to-trial variation in processing demands

Multiple large-scale networks populate human association cortex. Here, we explored the functional properties of these networks by exploiting trial-to-trial variation in component-processing demands. In two behavioral studies (n = 136 and n = 238), participants quantified strategies used to solve individual task trials that spanned remembering, imagining future scenarios, and various control trials. These trials were also all scanned in an independent sample of functional MRI participants (n = 10), each with sufficient data to precisely define within-individual networks. Stable latent factors varied across trials and correlated with trial-level functional responses selectively across networks. One network linked to parahippocampal cortex, labeled Default Network A (DN-A), tracked scene construction, including for control trials that possessed minimal episodic memory demands. To the degree, a trial encouraged participants to construct a mental scene with imagery and awareness about spatial locations of objects or places, the response in DN-A increased. The juxtaposed Default Network B (DN-B) showed no such response but varied in relation to social processing demands. Another adjacent network, labeled Frontoparietal Network B (FPN-B), robustly correlated with trial difficulty. These results support that DN-A and DN-B are specialized networks differentially supporting information processing within spatial and social domains. Both networks are dissociable from a closely juxtaposed domain-general control network that tracks cognitive effort.NEW & NOTEWORTHY Tasks shown to differentially recruit parallel association networks are multifaceted, leaving open questions about network processes. Here, examining trial-to-trial network response properties in relation to trial traits reveals new insights into network functions. In particular, processes linked to scene construction selectively recruit a distributed network with links to parahippocampal and retrosplenial cortices, including during trials designed not to rely on the personal past. Adjacent networks show distinct patterns, providing novel evidence of functional specialization.

Multidimensional memory topography in the medial parietal cortex identified from neuroimaging of thousands of daily memory videos

Our memories form a tapestry of events, people, and places, woven across the decades of our lives. However, research has often been limited in assessing the nature of episodic memory by using artificial stimuli and short time scales. The explosion of social media enables new ways to examine the neural representations of naturalistic episodic memories, for features like the memory's age, location, memory strength, and emotions. We recruited 23 users of a video diary app ("1 s Everyday"), who had recorded 9266 daily memory videos spanning up to 7 years. During a 3 T fMRI scan, participants viewed 300 of their memory videos intermixed with 300 from another individual. We find that memory features are tightly interrelated, highlighting the need to test them in conjunction, and discover a multidimensional topography in medial parietal cortex, with subregions sensitive to a memory's age, strength, and the familiarity of the people and places involved.

Top-down modulation and cortical-AMG/HPC interaction in familiar face processing

Humans can accurately recognize familiar faces in only a few hundred milliseconds, but the underlying neural mechanism remains unclear. Here, we recorded intracranial electrophysiological signals from ventral temporal cortex (VTC), superior/middle temporal cortex (STC/MTC), medial parietal cortex (MPC), and amygdala/hippocampus (AMG/HPC) in 20 epilepsy patients while they viewed faces of famous people and strangers as well as common objects. In posterior VTC and MPC, familiarity-sensitive responses emerged significantly later than initial face-selective responses, suggesting that familiarity enhances face representations after they are first being extracted. Moreover, viewing famous faces increased the coupling between cortical areas and AMG/HPC in multiple frequency bands. These findings advance our understanding of the neural basis of familiar face perception by identifying the top-down modulation in local face-selective response and interactions between cortical face areas and AMG/HPC.

Multiple-stage impairments of unfamiliar face learning in developmental prosopagnosia: Evidence from fMRI repetition suppression and multi-voxel pattern stability

Individuals with developmental prosopagnosia (DP) are characterized by severe face recognition deficits, yet it remains unknown how they are hindered in the process of unfamiliar face learning. Here we tracked the changes of neural activation during unfamiliar face repetition in DP with fMRI to reveal their neural deficits in learning unfamiliar faces. At the perceptual level, we found that the bilateral fusiform face area (FFA) in individuals with DP showed attenuated repetition suppression for faces, suggesting an inefficient perceptual analysis for learned faces. At the mnemonic level, individuals with DP showed decreased multi-voxel pattern stability for repeated faces in bilateral medial temporal lobe (MTL), suggesting an unstable mnemonic representation for learned faces. In addition, resting-state functional connectivity between the FFA and MTL was also disrupted in individuals with DP. Finally, the MTL's unstable mnemonic representation was associated with the impaired face recognition performance in DP. In sum, our study provides evidence that individuals with DP showed multi-stage neural deficits in unfamiliar face learning and sheds new light on how unfamiliar faces are learned in normal population.

Event-Related Phase Synchronization Propagates Rapidly across Human Ventral Visual Cortex

Visual inputs to early visual cortex integrate with semantic, linguistic and memory inputs in higher visual cortex, in a manner that is rapid and accurate, and enables complex computations such as face recognition and word reading. This implies the existence of fundamental organizational principles that enable such efficiency. To elaborate on this, we performed intracranial recordings in 82 individuals while they performed tasks of varying visual and cognitive complexity. We discovered that visual inputs induce highly organized posterior-to-anterior propagating patterns of phase modulation across the ventral occipitotemporal cortex. At individual electrodes there was a stereotyped temporal pattern of phase progression following both stimulus onset and offset, consistent across trials and tasks. The phase of low frequency activity in anterior regions was predicted by the prior phase in posterior cortical regions. This spatiotemporal propagation of phase likely serves as a feed-forward organizational influence enabling the integration of information across the ventral visual stream. This phase modulation manifests as the early components of the event related potential; one of the most commonly used measures in human electrophysiology. These findings illuminate fundamental organizational principles of the higher order visual system that enable the rapid recognition and characterization of a variety of inputs.

Intraoperative localization and preservation of reading in ventral occipitotemporal cortex

OBJECTIVE

Resective surgery in language-dominant ventral occipitotemporal cortex (vOTC) carries the risk of causing impairment to reading. Because it is not on the lateral surface, it is not easily accessible for intraoperative mapping, and extensive stimulation mapping can be time-consuming. Here the authors assess the feasibility of using task-based electrocorticography (ECoG) recordings intraoperatively to help guide stimulation mapping of reading in vOTC.

METHODS

In 11 patients undergoing extraoperative, intracranial seizure mapping, the authors recorded induced broadband gamma activation (70-150 Hz) during a visual category localizer. In 2 additional patients, whose pathologies necessitated resections in language-dominant vOTC, task-based functional mapping was performed intraoperatively using subdural ECoG alongside direct cortical stimulation.

RESULTS

Word-responsive cortex localized using ECoG showed a high sensitivity (72%) to stimulation-induced reading deficits, and the confluence of ECoG and stimulation-positive sites appears to demarcate the visual word form area. Intraoperative task-based ECoG mapping was possible in < 3 minutes, providing a high signal quality, and initial intraoperative data analysis took < 3 minutes, allowing for rapid assessment of broad areas of cortex. Cortical areas critical for reading were mapped and successfully preserved, while also enabling pathological tissue to be completely removed.

CONCLUSIONS

Eloquent cortex in ventral visual cortex can be rapidly mapped intraoperatively using ECoG. This method acts to guide high-probability targets for stimulation with limited patient participation and can be used to avoid iatrogenic dyslexia following surgery.

Dataset of human intracranial recordings during famous landmark identification

For most people, recalling information about familiar items in a visual scene is an effortless task, but it is one that depends on coordinated interactions of multiple, distributed neural components. We leveraged the high spatiotemporal resolution of direct intracranial recordings to better delineate the network dynamics underpinning visual scene recognition. We present a dataset of recordings from a large cohort of humans while they identified images of famous landmarks (50 individuals, 52 recording sessions, 6,775 electrodes, 6,541 trials). This dataset contains local field potential recordings derived from subdural and penetrating electrodes covering broad areas of cortex across both hemispheres. We provide this pre-processed data with behavioural metrics (correct/incorrect, response times) and electrode localisation in a population-normalised cortical surface space. This rich dataset will allow further investigation into the spatiotemporal progression of multiple neural processes underlying visual processing, scene recognition and cued memory recall.

Spatial Integration in Normal Face Processing and Its Breakdown in Congenital Prosopagnosia

Congenital prosopagnosia (CP), a life-long impairment in face processing that occurs in the absence of any apparent brain damage, provides a unique model in which to explore the psychological and neural bases of normal face processing. The goal of this review is to offer a theoretical and conceptual framework that may account for the underlying cognitive and neural deficits in CP. This framework may also provide a novel perspective in which to reconcile some conflicting results that permits the expansion of the research in this field in new directions. The crux of this framework lies in linking the known behavioral and neural underpinnings of face processing and their impairments in CP to a model incorporating grid cell-like activity in the entorhinal cortex. Moreover, it stresses the involvement of active, spatial scanning of the environment with eye movements and implicates their critical role in face encoding and recognition. To begin with, we describe the main behavioral and neural characteristics of CP, and then lay down the building blocks of our proposed model, referring to the existing literature supporting this new framework. We then propose testable predictions and conclude with open questions for future research stemming from this model. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

A network linking scene perception and spatial memory systems in posterior cerebral cortex

The neural systems supporting scene-perception and spatial-memory systems of the human brain are well-described. But how do these neural systems interact? Here, using fine-grained individual-subject fMRI, we report three cortical areas of the human brain, each lying immediately anterior to a region of the scene perception network in posterior cerebral cortex, that selectively activate when recalling familiar real-world locations. Despite their close proximity to the scene-perception areas, network analyses show that these regions constitute a distinct functional network that interfaces with spatial memory systems during naturalistic scene understanding. These "place-memory areas" offer a new framework for understanding how the brain implements memory-guided visual behaviors, including navigation.