An algorithmic method for functionally defining regions of interest in the ventral visual pathway

The brain hierarchically represents the past and future during multistep anticipation

Memory for temporal structure enables both planning of future events and retrospection of past events. We investigated how the brain flexibly represents extended temporal sequences into the past and future during anticipation. Participants learned sequences of environments in immersive virtual reality. Pairs of sequences had the same environments in a different order, enabling context-specific learning. During fMRI, participants anticipated upcoming environments multiple steps into the future in a given sequence. Temporal structure was represented in the hippocampus and across higher-order visual regions (1) bidirectionally, with graded representations into the past and future and (2) hierarchically, with further events into the past and future represented in successively more anterior brain regions. In hippocampus, these bidirectional representations were context-specific, and suppression of far-away environments predicted response time costs in anticipation. Together, this work sheds light on how we flexibly represent sequential structure to enable planning over multiple timescales.

The brain hierarchically represents the past and future during multistep anticipation

Memory for temporal structure enables both planning of future events and retrospection of past events. We investigated how the brain flexibly represents extended temporal sequences into the past and future during anticipation. Participants learned sequences of environments in immersive virtual reality. Pairs of sequences had the same environments in a different order, enabling context-specific learning. During fMRI, participants anticipated upcoming environments multiple steps into the future in a given sequence. Temporal structure was represented in the hippocampus and across higher-order visual regions (1) bidirectionally, with graded representations into the past and future and (2) hierarchically, with further events into the past and future represented in successively more anterior brain regions. In hippocampus, these bidirectional representations were context-specific, and suppression of far-away environments predicted response time costs in anticipation. Together, this work sheds light on how we flexibly represent sequential structure to enable planning over multiple timescales.

Modeling short visual events through the BOLD moments video fMRI dataset and metadata

Studying the neural basis of human dynamic visual perception requires extensive experimental data to evaluate the large swathes of functionally diverse brain neural networks driven by perceiving visual events. Here, we introduce the BOLD Moments Dataset (BMD), a repository of whole-brain fMRI responses to over 1000 short (3 s) naturalistic video clips of visual events across ten human subjects. We use the videos' extensive metadata to show how the brain represents word- and sentence-level descriptions of visual events and identify correlates of video memorability scores extending into the parietal cortex. Furthermore, we reveal a match in hierarchical processing between cortical regions of interest and video-computable deep neural networks, and we showcase that BMD successfully captures temporal dynamics of visual events at second resolution. With its rich metadata, BMD offers new perspectives and accelerates research on the human brain basis of visual event perception.

A familiar face and person processing area in the human temporal pole

How does the brain process the faces of familiar people? Neuropsychological studies have argued for an area of the temporal pole (TP) linking faces with person identities, but magnetic susceptibility artifacts in this region have hampered its study with fMRI. Using data acquisition and analysis methods optimized to overcome this artifact, we identify a familiar face response in TP, reliably observed in individual brains. This area responds strongly to visual images of familiar faces over unfamiliar faces, objects, and scenes. However, TP did not just respond to images of faces, but also to a variety of high-level social cognitive tasks, including semantic, episodic, and theory of mind tasks. The response profile of TP contrasted with a nearby region of the perirhinal cortex that responded specifically to faces, but not to social cognition tasks. TP was functionally connected with a distributed network in the association cortex associated with social cognition, while PR was functionally connected with face-preferring areas of the ventral visual cortex. This work identifies a missing link in the human face processing system that specifically processes familiar faces, and is well placed to integrate visual information about faces with higher-order conceptual information about other people. The results suggest that separate streams for person and face processing reach anterior temporal areas positioned at the top of the cortical hierarchy.

A scene with an invisible wall - navigational experience shapes visual scene representation

Human navigation heavily relies on visual information. Although many previous studies have investigated how navigational information is inferred from visual features of scenes, little is understood about the impact of navigational experience on visual scene representation. In this study, we examined how navigational experience influences both the behavioral and neural responses to a visual scene. During training, participants navigated in the virtual reality (VR) environments which we manipulated navigational experience while holding the visual properties of scenes constant. Half of the environments allowed free navigation (navigable), while the other half featured an 'invisible wall' preventing the participants to continue forward even though the scene was visually navigable (non-navigable). During testing, participants viewed scene images from the VR environment while completing either a behavioral perceptual identification task (Experimentl) or an fMRI scan (Experiment2). Behaviorally, we found that participants judged a scene pair to be significantly more visually different if their prior navigational experience varied, even after accounting for visual similarities between the scene pairs. Neurally, multi-voxel pattern of the parahippocampal place area (PPA) distinguished visual scenes based on prior navigational experience alone. These results suggest that the human visual scene cortex represents information about navigability obtained through prior experience, beyond those computable from the visual properties of the scene. Taken together, these results suggest that scene representation is modulated by prior navigational experience to help us construct a functionally meaningful visual environment.

Cortical Face-Selective Responses Emerge Early in Human Infancy

In human adults, multiple cortical regions respond robustly to faces, including the occipital face area (OFA) and fusiform face area (FFA), implicated in face perception, and the superior temporal sulcus (STS) and medial prefrontal cortex (MPFC), implicated in higher-level social functions. When in development, does face selectivity arise in each of these regions? Here, we combined two awake infant functional magnetic resonance imaging (fMRI) datasets to create a sample size twice the size of previous reports (n = 65 infants; 2.6-9.6 months). Infants watched movies of faces, bodies, objects, and scenes, while fMRI data were collected. Despite variable amounts of data from each infant, individual subject whole-brain activation maps revealed responses to faces compared to nonface visual categories in the approximate location of OFA, FFA, STS, and MPFC. To determine the strength and nature of face selectivity in these regions, we used cross-validated functional region of interest analyses. Across this larger sample size, face responses in OFA, FFA, STS, and MPFC were significantly greater than responses to bodies, objects, and scenes. Even the youngest infants (2-5 months) showed significantly face-selective responses in FFA, STS, and MPFC, but not OFA. These results demonstrate that face selectivity is present in multiple cortical regions within months of birth, providing powerful constraints on theories of cortical development.

Linguistic inputs must be syntactically parsable to fully engage the language network

Human language comprehension is remarkably robust to ill-formed inputs (e.g., word transpositions). This robustness has led some to argue that syntactic parsing is largely an illusion, and that incremental comprehension is more heuristic, shallow, and semantics-based than is often assumed. However, the available data are also consistent with the possibility that humans always perform rule-like symbolic parsing and simply deploy error correction mechanisms to reconstruct ill-formed inputs when needed. We put these hypotheses to a new stringent test by examining brain responses to a) stimuli that should pose a challenge for syntactic reconstruction but allow for complex meanings to be built within local contexts through associative/shallow processing (sentences presented in a backward word order), and b) grammatically well-formed but semantically implausible sentences that should impede semantics-based heuristic processing. Using a novel behavioral syntactic reconstruction paradigm, we demonstrate that backward-presented sentences indeed impede the recovery of grammatical structure during incremental comprehension. Critically, these backward-presented stimuli elicit a relatively low response in the language areas, as measured with fMRI. In contrast, semantically implausible but grammatically well-formed sentences elicit a response in the language areas similar in magnitude to naturalistic (plausible) sentences. In other words, the ability to build syntactic structures during incremental language processing is both necessary and sufficient to fully engage the language network. Taken together, these results provide strongest to date support for a generalized reliance of human language comprehension on syntactic parsing.

How the intrinsic functional connectivity patterns of the semantic network support semantic processing

Semantic processing, a core of language comprehension, involves the activation of brain regions dispersed extensively across the frontal, temporal, and parietal cortices that compose the semantic network. To comprehend the functional structure of this semantic network and how it prepares for semantic processing, we investigated its intrinsic functional connectivity (FC) and the relation between this pattern and semantic processing ability in a large sample from the Human Connectome Project (HCP) dataset. We first defined a well-studied brain network for semantic processing, and then we characterized the within-network connectivity (WNC) and the between-network connectivity (BNC) within this network using a voxel-based global brain connectivity (GBC) method based on resting-state functional magnetic resonance imaging (fMRI). The results showed that 97.73% of the voxels in the semantic network displayed considerably greater WNC than BNC, demonstrating that the semantic network is a fairly encapsulated network. Moreover, multiple connector hubs in the semantic network were identified after applying the criterion of WNC > 1 SD above the mean WNC of the semantic network. More importantly, three of these connector hubs (i.e., the left anterior temporal lobe, angular gyrus, and orbital part of the inferior frontal gyrus) were reliably associated with semantic processing ability. Our findings suggest that the three identified regions use WNC as the central mechanism for supporting semantic processing and that task-independent spontaneous connectivity in the semantic network is essential for semantic processing.

Neuronal tuning and population representations of shape and category in human visual cortex

Object recognition and categorization are essential cognitive processes which engage considerable neural resources in the human ventral visual stream. However, the tuning properties of human ventral stream neurons for object shape and category are virtually unknown. We performed large-scale recordings of spiking activity in human Lateral Occipital Complex in response to stimuli in which the shape dimension was dissociated from the category dimension. Consistent with studies in nonhuman primates, the neuronal representations were primarily shape-based, although we also observed category-like encoding for images of animals. Surprisingly, linear decoders could reliably classify stimulus category even in data sets that were entirely shape-based. In addition, many recording sites showed an interaction between shape and category tuning. These results represent a detailed study on shape and category coding at the neuronal level in the human ventral visual stream, furnishing essential evidence that reconciles human imaging and macaque single-cell studies.

Perception and Memory Reinstatement Engage Overlapping Face-Selective Regions within Human Ventral Temporal Cortex

Humans have the remarkable ability to vividly retrieve sensory details of past events. According to the theory of sensory reinstatement, during remembering, brain regions specialized for processing specific sensory stimuli are reactivated to support content-specific retrieval. Recently, several studies have emphasized transformations in the spatial organization of these reinstated activity patterns. Specifically, studies of scene stimuli suggest a clear anterior shift in the location of retrieval activations compared with the activity observed during perception. However, it is not clear that such transformations occur universally, with inconsistent evidence for other important stimulus categories, particularly faces. One challenge in addressing this question is the careful delineation of face-selective cortices, which are interdigitated with other selective regions, in configurations that spatially differ across individuals. Therefore, we conducted a multisession neuroimaging study to first carefully map individual participants' (nine males and seven females) face-selective regions within ventral temporal cortex (VTC), followed by a second session to examine the activity patterns within these regions during face memory encoding and retrieval. While face-selective regions were expectedly engaged during face perception at encoding, memory retrieval engagement exhibited a more selective and constricted reinstatement pattern within these regions, but did not show any consistent direction of spatial transformation (e.g., anteriorization). We also report on unique human intracranial recordings from VTC under the same experimental conditions. These findings highlight the importance of considering the complex configuration of category-selective cortex in elucidating principles shaping the neural transformations that occur from perception to memory.

Predicting high-level visual areas in the absence of task fMRI

The ventral visual stream is organized into units, or functional regions of interest (fROIs), specialized for processing high-level visual categories. Task-based fMRI scans ("localizers") are typically used to identify each individual's nuanced set of fROIs. The unique landscape of an individual's functional activation may rely in large part on their specialized connectivity patterns; recent studies corroborate this by showing that connectivity can predict individual differences in neural responses. We focus on the ventral visual stream and ask: how well can an individual's resting state functional connectivity localize their fROIs for face, body, scene, and object perception? And are the neural processors for any particular visual category better predicted by connectivity than others, suggesting a tighter mechanistic relationship between connectivity and function? We found, among 18 fROIs predicted from connectivity for each subject, all but one were selective for their preferred visual category. Defining an individual's fROIs based on their connectivity patterns yielded regions that were more selective than regions identified from previous studies or atlases in nearly all cases. Overall, we found that in the absence of a domain-specific localizer task, a 10-min resting state scan can be reliably used for defining these fROIs.

Neural Representations of Observed Interpersonal Synchrony/Asynchrony in the Social Perception Network

The visual perception of individuals is thought to be mediated by a network of regions in the occipitotemporal cortex that supports specialized processing of faces, bodies, and actions. In comparison, we know relatively little about the neural mechanisms that support the perception of multiple individuals and the interactions between them. The present study sought to elucidate the visual processing of social interactions by identifying which regions of the social perception network represent interpersonal synchrony. In an fMRI study with 32 human participants (26 female, 6 male), we used multivoxel pattern analysis to investigate whether activity in face-selective, body-selective, and interaction-sensitive regions across the social perception network supports the decoding of synchronous versus asynchronous head-nodding and head-shaking. Several regions were found to support significant decoding of synchrony/asynchrony, including extrastriate body area (EBA), face-selective and interaction-sensitive mid/posterior right superior temporal sulcus, and occipital face area. We also saw robust cross-classification across actions in the EBA, suggestive of movement-invariant representations of synchrony/asynchrony. Exploratory whole-brain analyses also identified a region of the right fusiform cortex that responded more strongly to synchronous than to asynchronous motion. Critically, perceiving interpersonal synchrony/asynchrony requires the simultaneous extraction and integration of dynamic information from more than one person. Hence, the representation of synchrony/asynchrony cannot be attributed to augmented or additive processing of individual actors. Our findings therefore provide important new evidence that social interactions recruit dedicated visual processing within the social perception network that extends beyond that engaged by the faces and bodies of the constituent individuals.

Representation of navigational affordances and ego-motion in the occipital place area

Humans effortlessly use vision to plan and guide navigation through the local environment, or "scene". A network of three cortical regions responds selectively to visual scene information, including the occipital (OPA), parahippocampal (PPA), and medial place areas (MPA) - but how this network supports visually-guided navigation is unclear. Recent evidence suggests that one region in particular, the OPA, supports visual representations for navigation, while PPA and MPA support other aspects of scene processing. However, most previous studies tested only static scene images, which lack the dynamic experience of navigating through scenes. We used dynamic movie stimuli to test whether OPA, PPA, and MPA represent two critical kinds of navigationally-relevant information: navigational affordances (e.g., can I walk to the left, right, or both?) and ego-motion (e.g., am I walking forward or backward? turning left or right?). We found that OPA is sensitive to both affordances and ego-motion, as well as the conflict between these cues - e.g., turning toward versus away from an open doorway. These effects were significantly weaker or absent in PPA and MPA. Responses in OPA were also dissociable from those in early visual cortex, consistent with the idea that OPA responses are not merely explained by lower-level visual features. OPA responses to affordances and ego-motion were stronger in the contralateral than ipsilateral visual field, suggesting that OPA encodes navigationally relevant information within an egocentric reference frame. Taken together, these results support the hypothesis that OPA contains visual representations that are useful for planning and guiding navigation through scenes.

FMRI correlates of autobiographical memory: Comparing silent retrieval with narrated retrieval

FMRI studies of autobiographical memory (AM) retrieval typically ask subjects to retrieve memories silently to avoid speech-related motion artifacts. Recently, some fMRI studies have started to use overt (spoken) retrieval to probe moment-to-moment retrieved content. However, the extent to which the overt retrieval method alters fMRI activations during retrieval is unknown. Here we examined this question by eliciting unrehearsed AMs during fMRI scanning either overtly or silently, in the same subjects, in different runs. Differences between retrieval modality (silent vs. narrated) included greater activation for silent retrieval in the anterior hippocampus, left angular gyrus, PCC, and superior PFC, and greater activation for narrated retrieval in speech production regions, posterior hippocampus, and the DLPFC. To probe temporal dynamics, we divided each retrieval period into an initial search phase and a later elaboration phase. The activations during the search and elaboration phases were broadly similar regardless of modality, and these activations were in line with previous fMRI studies of AM temporal dynamics employing silent retrieval. For both retrieval modalities, search activated the hippocampus, mPFC, ACC, and PCC, and elaboration activated the left DLPFC and middle temporal gyri. To examine content-specific reactivation during retrieval, the timecourse of narrated memory content was transcribed and modeled. We observed dynamic activation associated with object content in the lateral occipital complex, and activation associated with scene content in the retrosplenial cortex. The current findings show that both silent and narrated AMs activate a broadly similar memory network, with some key differences, and add to current knowledge regarding the content-specific dynamics of AM retrieval. However, these observed differences between retrieval modality suggest that studies using overt retrieval should carefully consider this method's possible effects on cognitive and neural processing.

Limited but specific engagement of the mature language network during linguistic statistical learning

Statistical learning (SL) is the ability to detect and learn regularities from input and is foundational to language acquisition. Despite the dominant role of SL as a theoretical construct for language development, there is a lack of direct evidence supporting the shared neural substrates underlying language processing and SL. It is also not clear whether the similarities, if any, are related to linguistic processing, or statistical regularities in general. The current study tests whether the brain regions involved in natural language processing are similarly recruited during auditory, linguistic SL. Twenty-two adults performed an auditory linguistic SL task, an auditory nonlinguistic SL task, and a passive story listening task as their neural activation was monitored. Within the language network, the left posterior temporal gyrus showed sensitivity to embedded speech regularities during auditory, linguistic SL, but not auditory, nonlinguistic SL. Using a multivoxel pattern similarity analysis, we uncovered similarities between the neural representation of auditory, linguistic SL, and language processing within the left posterior temporal gyrus. No other brain regions showed similarities between linguistic SL and language comprehension, suggesting that a shared neurocomputational process for auditory SL and natural language processing within the left posterior temporal gyrus is specific to linguistic stimuli.

"Walking selectivity" in the occipital place area in 8-year-olds, not 5-year-olds

A recent neuroimaging study in adults found that the occipital place area (OPA)-a cortical region involved in "visually guided navigation" (i.e. moving about the immediately visible environment, avoiding boundaries, and obstacles)-represents visual information about walking, not crawling, suggesting that OPA is late developing, emerging only when children are walking, not beforehand. But when precisely does this "walking selectivity" in OPA emerge-when children first begin to walk in early childhood, or perhaps counterintuitively, much later in childhood, around 8 years of age, when children are adult-like walking? To directly test these two hypotheses, using functional magnetic resonance imaging (fMRI) in two groups of children, 5- and 8-year-olds, we measured the responses in OPA to first-person perspective videos through scenes from a "walking" perspective, as well as three control perspectives ("crawling," "flying," and "scrambled"). We found that the OPA in 8-year-olds-like adults-exhibited walking selectivity (i.e. responding significantly more to the walking videos than to any of the others, and no significant differences across the crawling, flying, and scrambled videos), while the OPA in 5-year-olds exhibited no walking selectively. These findings reveal that OPA undergoes protracted development, with walking selectivity only emerging around 8 years of age.

Integrative processing in artificial and biological vision predicts the perceived beauty of natural images

Previous research shows that the beauty of natural images is already determined during perceptual analysis. However, it is unclear which perceptual computations give rise to the perception of beauty. Here, we tested whether perceived beauty is predicted by spatial integration across an image, a perceptual computation that reduces processing demands by aggregating image parts into more efficient representations of the whole. We quantified integrative processing in an artificial deep neural network model, where the degree of integration was determined by the amount of deviation between activations for the whole image and its constituent parts. This quantification of integration predicted beauty ratings for natural images across four studies with different stimuli and designs. In a complementary functional magnetic resonance imaging study, we show that integrative processing in human visual cortex similarly predicts perceived beauty. Together, our results establish integration as a computational principle that facilitates perceptual analysis and thereby mediates the perception of beauty.

Parafoveal vision reveals qualitative differences between fusiform face area and parahippocampal place area

The center-periphery visual field axis guides early visual system organization with enhanced resources devoted to central vision leading to reduced peripheral performance relative to that of central vision (i.e., behavioral eccentricity effect) for many visual functions. The center-periphery organization extends to high-order visual cortex where, for example, the well-studied face-sensitive fusiform face area (FFA) shows sensitivity to central vision and the place-sensitive parahippocampal place area (PPA) shows sensitivity to peripheral vision. As we have recently found that face perception is more sensitive to eccentricity than place perception, here we examined whether these behavioral findings reflect differences in FFA's and PPA's sensitivities to eccentricity. We assumed FFA would show higher sensitivity to eccentricity than PPA would, but that both regions' modulation by eccentricity would be invariant to the viewed category. We parametrically investigated (fMRI, n = 32) how FFA's and PPA's activations are modulated by eccentricity (≤8°) and category (upright/inverted faces/houses) while keeping stimulus size constant. As expected, FFA showed an overall higher sensitivity to eccentricity than PPA. However, both regions' activation modulations by eccentricity were dependent on the viewed category. In FFA, a reduction of activation with growing eccentricity ("BOLD eccentricity effect") was found (with different amplitudes) for all categories. In PPA however, qualitatively different BOLD eccentricity effect modulations were found (e.g., at 8° mild BOLD eccentricity effect for houses but a reverse BOLD eccentricity effect for faces and no modulation for inverted faces). Our results emphasize that peripheral vision investigations are critical to further our understanding of visual processing.

Disentangled deep generative models reveal coding principles of the human face processing network

Despite decades of research, much is still unknown about the computations carried out in the human face processing network. Recently, deep networks have been proposed as a computational account of human visual processing, but while they provide a good match to neural data throughout visual cortex, they lack interpretability. We introduce a method for interpreting brain activity using a new class of deep generative models, disentangled representation learning models, which learn a low-dimensional latent space that "disentangles" different semantically meaningful dimensions of faces, such as rotation, lighting, or hairstyle, in an unsupervised manner by enforcing statistical independence between dimensions. We find that the majority of our model's learned latent dimensions are interpretable by human raters. Further, these latent dimensions serve as a good encoding model for human fMRI data. We next investigate the representation of different latent dimensions across face-selective voxels. We find that low- and high-level face features are represented in posterior and anterior face-selective regions, respectively, corroborating prior models of human face recognition. Interestingly, though, we find identity-relevant and irrelevant face features across the face processing network. Finally, we provide new insight into the few "entangled" (uninterpretable) dimensions in our model by showing that they match responses in the ventral stream and carry information about facial identity. Disentangled face encoding models provide an exciting alternative to standard "black box" deep learning approaches for modeling and interpreting human brain data.

A retinotopic code structures the interaction between perception and memory systems

Conventional views of brain organization suggest that regions at the top of the cortical hierarchy processes internally oriented information using an abstract amodal neural code. Despite this, recent reports have described the presence of retinotopic coding at the cortical apex, including the default mode network. What is the functional role of retinotopic coding atop the cortical hierarchy? Here we report that retinotopic coding structures interactions between internally oriented (mnemonic) and externally oriented (perceptual) brain areas. Using functional magnetic resonance imaging, we observed robust inverted (negative) retinotopic coding in category-selective memory areas at the cortical apex, which is functionally linked to the classic (positive) retinotopic coding in category-selective perceptual areas in high-level visual cortex. These functionally linked retinotopic populations in mnemonic and perceptual areas exhibit spatially specific opponent responses during both bottom-up perception and top-down recall, suggesting that these areas are interlocked in a mutually inhibitory dynamic. These results show that retinotopic coding structures interactions between perceptual and mnemonic neural systems, providing a scaffold for their dynamic interaction.

Multi-modal Representation of the Size of Space in the Human Brain

To estimate the size of an indoor space, we must analyze the visual boundaries that limit the spatial extent and acoustic cues from reflected interior surfaces. We used fMRI to examine how the brain processes the geometric size of indoor scenes when various types of sensory cues are presented individually or together. Specifically, we asked whether the size of space is represented in a modality-specific way or in an integrative way that combines multimodal cues. In a block-design study, images or sounds that depict small- and large-sized indoor spaces were presented. Visual stimuli were real-world pictures of empty spaces that were small or large. Auditory stimuli were sounds convolved with different reverberations. By using a multivoxel pattern classifier, we asked whether the two sizes of space can be classified in visual, auditory, and visual-auditory combined conditions. We identified both sensory-specific and multimodal representations of the size of space. To further investigate the nature of the multimodal region, we specifically examined whether it contained multimodal information in a coexistent or integrated form. We found that angular gyrus and the right medial frontal gyrus had modality-integrated representation, displaying sensitivity to the match in the spatial size information conveyed through image and sound. Background functional connectivity analysis further demonstrated that the connection between sensory-specific regions and modality-integrated regions increases in the multimodal condition compared with single modality conditions. Our results suggest that spatial size perception relies on both sensory-specific and multimodal representations, as well as their interplay during multimodal perception.

A neural circuit for spatial orientation derived from brain lesions

There is disagreement regarding the major components of the brain network supporting spatial cognition. To address this issue, we applied a lesion mapping approach to the clinical phenomenon of topographical disorientation. Topographical disorientation is the inability to maintain accurate knowledge about the physical environment and use it for navigation. A review of published topographical disorientation cases identified 65 different lesion sites. Our lesion mapping analysis yielded a topographical disorientation brain map encompassing the classic regions of the navigation network: medial parietal, medial temporal, and temporo-parietal cortices. We also identified a ventromedial region of the prefrontal cortex, which has been absent from prior descriptions of this network. Moreover, we revealed that the regions mapped are correlated with the Default Mode Network sub-network C. Taken together, this study provides causal evidence for the distribution of the spatial cognitive system, demarking the major components and identifying novel regions.

Brain connectivity and academic skills in English learners

English learners (ELs) are a rapidly growing population in schools in the United States with limited experience and proficiency in English. To better understand the path for EL's academic success in school, it is important to understand how EL's brain systems are used for academic learning in English. We studied, in a cohort of Hispanic middle-schoolers (n = 45, 22F) with limited English proficiency and a wide range of reading and math abilities, brain network properties related to academic abilities. We applied a method for localizing brain regions of interest (ROIs) that are group-constrained, yet individually specific, to test how resting state functional connectivity between regions that are important for academic learning (reading, math, and cognitive control regions) are related to academic abilities. ROIs were selected from task localizers probing reading and math skills in the same participants. We found that connectivity across all ROIs, as well as connectivity of just the cognitive control ROIs, were positively related to measures of reading skills but not math skills. This work suggests that cognitive control brain systems have a central role for reading in ELs. Our results also indicate that an individualized approach for localizing brain function may clarify brain-behavior relationships.

Efficient functional localization of language regions in the brain

Important recent advances in the cognitive neuroscience of language have been made using functional localizers to demarcate language-selective regions in individual brains. Although single-subject localizers offer insights that are unavailable in classic group analyses, they require additional scan time that imposes costs on investigators and participants. In particular, the unique practical challenges of scanning children and other special populations has led to less adoption of localizers for neuroimaging research with these theoretically and clinically important groups. Here, we examined how measurements of the spatial extent and functional response profiles of language regions are affected by the duration of an auditory language localizer. We compared how parametrically smaller amounts of data collected from one scanning session affected (i) consistency of group-level whole-brain parcellations, (ii) functional selectivity of subject-level activation in individually defined functional regions of interest (fROIs), (iii) sensitivity and specificity of subject-level whole-brain and fROI activation, and (iv) test-retest reliability of subject-level whole-brain and fROI activation. For many of these metrics, the localizer duration could be reduced by 50-75% while preserving the stability and reliability of both the spatial extent and functional response profiles of language areas. These results indicate that, for most measures relevant to cognitive neuroimaging studies, the brain's language network can be localized just as effectively with 3.5 min of scan time as it can with 12 min. Minimizing the time required to reliably localize the brain's language network allows more effective localizer use in situations where each minute of scan time is particularly precious.

Hierarchical organization of social action features along the lateral visual pathway

Recent theoretical work has argued that in addition to the classical ventral (what) and dorsal (where/how) visual streams, there is a third visual stream on the lateral surface of the brain specialized for processing social information. Like visual representations in the ventral and dorsal streams, representations in the lateral stream are thought to be hierarchically organized. However, no prior studies have comprehensively investigated the organization of naturalistic, social visual content in the lateral stream. To address this question, we curated a naturalistic stimulus set of 250 3-s videos of two people engaged in everyday actions. Each clip was richly annotated for its low-level visual features, mid-level scene and object properties, visual social primitives (including the distance between people and the extent to which they were facing), and high-level information about social interactions and affective content. Using a condition-rich fMRI experiment and a within-subject encoding model approach, we found that low-level visual features are represented in early visual cortex (EVC) and middle temporal (MT) area, mid-level visual social features in extrastriate body area (EBA) and lateral occipital complex (LOC), and high-level social interaction information along the superior temporal sulcus (STS). Communicative interactions, in particular, explained unique variance in regions of the STS after accounting for variance explained by all other labeled features. Taken together, these results provide support for representation of increasingly abstract social visual content-consistent with hierarchical organization-along the lateral visual stream and suggest that recognizing communicative actions may be a key computational goal of the lateral visual pathway.

Differential functional reorganization of ventral and dorsal visual pathways following childhood hemispherectomy

Hemispherectomy is a surgical procedure in which an entire hemisphere of a patient's brain is resected or functionally disconnected to manage seizures in individuals with drug-resistant epilepsy. Despite the extensive loss of both ventral and dorsal visual pathways in one hemisphere, pediatric patients who have undergone hemispherectomy show a remarkably high degree of perceptual function across many domains. In the current study, we sought to understand the extent to which functions of the ventral and dorsal visual pathways reorganize to the contralateral hemisphere following childhood hemispherectomy. To this end, we collected fMRI data from an equal number of left and right hemispherectomy patients who completed tasks that typically elicit lateralized responses from the ventral or the dorsal pathway, namely, word (left ventral), face (right ventral), tool (left dorsal), and global form (right dorsal) perception. Overall, there was greater evidence of functional reorganization in the ventral pathway than in the dorsal pathway. Importantly, because ventral and dorsal reorganization was tested within the very same patients, these results cannot be explained by idiosyncratic factors such as disease etiology, age at the time of surgery, or age at testing. These findings suggest that because the dorsal pathway may mature earlier, it may have a shorter developmental window of plasticity than the ventral pathway and, hence, be less malleable after perturbation.

Alpha-frequency feedback to early visual cortex orchestrates coherent naturalistic vision

During naturalistic vision, the brain generates coherent percepts by integrating sensory inputs scattered across the visual field. Here, we asked whether this integration process is mediated by rhythmic cortical feedback. In electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) experiments, we experimentally manipulated integrative processing by changing the spatiotemporal coherence of naturalistic videos presented across visual hemifields. Our EEG data revealed that information about incoherent videos is coded in feedforward-related gamma activity while information about coherent videos is coded in feedback-related alpha activity, indicating that integration is indeed mediated by rhythmic activity. Our fMRI data identified scene-selective cortex and human middle temporal complex (hMT) as likely sources of this feedback. Analytically combining our EEG and fMRI data further revealed that feedback-related representations in the alpha band shape the earliest stages of visual processing in cortex. Together, our findings indicate that the construction of coherent visual experiences relies on cortical feedback rhythms that fully traverse the visual hierarchy.

A personalized cortical atlas for functional regions of interest

Advances in functional MRI (fMRI) allow mapping an individual's brain function in vivo. Task fMRI can localize domain-specific regions of cognitive processing or functional regions of interest (fROIs) within an individual. Moreover, data from resting state (no task) fMRI can be used to define an individual's connectome, which can characterize that individual's functional organization via connectivity-based parcellations. However, can connectivity-based parcellations alone predict an individual's fROIs? Here, we describe an approach to compute individualized rs-fROIs (i.e., regions that correspond to given fROI constructed using only resting state data) for motor control, working memory, high-level vision, and language comprehension. The rs-fROIs were computed and validated using a large sample of young adults (n = 1,018) with resting state and task fMRI from the Human Connectome Project. First, resting state parcellations were defined across a sequence of resolutions from broadscale to fine-grained networks in a training group of 500 individuals. Second, 21 rs-fROIs were defined from the training group by identifying the rs network that most closely matched task-defined fROIs across all individuals. Third, the selectivity of rs-fROIs was investigated in a training set of the remaining 518 individuals. All computed rs-fROIs were indeed selective for their preferred category. Critically, the rs-fROIs had higher selectivity than probabilistic atlas parcels for nearly all fROIs. In conclusion, we present a potential approach to define selective fROIs on an individual-level circumventing the need for multiple task-based localizers.NEW & NOTEWORTHY We compute individualized resting state parcels that identify an individual's own functional regions of interest (fROIs) for high-level vision, language comprehension, motor control, and working memory, using only their functional connectome. This approach demonstrates a rapid and powerful alternative for finding a large set of fROIs in an individual, using only their unique connectivity pattern, which does not require the costly acquisition of multiple fMRI localizer tasks.

Cognitive inflexibility is linked to abnormal frontoparietal-related activation and connectivity in obsessive-compulsive disorder

Although it was acknowledged that patients with obsessive-compulsive disorder (OCD) would exhibit cognitive inflexibility, the underlying neural mechanism has not been fully clarified. Therefore, this study aimed to investigate the neural substrates involved in cognitive inflexibility among individuals with OCD. A total of 42 patients with OCD and 48 healthy controls (HCs) completed clinical assessment and functional magnetic resonance imaging (fMRI) data collection during cued task switching. Behavioral performances and fMRI activation were compared between the OCD group and the HC group. Psychophysiological interactions (PPIs) analyses were applied to explore functional connectivity related to task switching. Pearson correlation was used to investigate the relationships among behavioral performance, fMRI activity, and obsessive-compulsive symptoms in OCD. The OCD group had a greater switch cost than HCs (χ2  = 5.89, p < .05). A significant difference in reaction time was found during switch (χ2  = 17.72, p < .001) and repeat (χ2  = 16.60, p = .018) between the two groups, while there was no significant difference in group accuracy. Comparison of group differences showed that the OCD group had increased activation in the right superior parietal cortex (rSPL) during task switching, and exhibited increased connectivity of frontoparietal network/default mode network (FPN-DMN; i.e., middle frontal gyrus [MFG]/inferior parietal cortex-precuneus, MFG-middle/posterior cingulate gyrus) and within the FPN (inferior parietal cortex-postcentral gyrus). In the OCD group, the compulsion score was positively correlated with accuracy during switch (r = .405, p = .008, FDRq <.05), and negatively correlated with activation of rSPL (r = -.328, p = .034, FDRq >.05). Patients with OCD had impaired cognitive flexibility and cautious response strategy. The neural mechanism of cognitive inflexibility in OCD may involve increased activation in the rSPL, as well as hyperconnectivity within the FPN and between the FPN and DMN.

Driving and suppressing the human language network using large language models

Transformer models such as GPT generate human-like language and are highly predictive of human brain responses to language. Here, using fMRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of brain response associated with each sentence. Then, we use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also noninvasively control neural activity in higher-level cortical areas, like the language network.

A retinotopic code structures the interaction between perception and memory systems

Conventional views of brain organization suggest that the cortical apex processes internally-oriented information using an abstract, amodal neural code. Yet, recent reports have described the presence of retinotopic coding at the cortical apex, including the default mode network. What is the functional role of retinotopic coding atop the cortical hierarchy? Here, we report that retinotopic coding structures interactions between internally-oriented (mnemonic) and externally-oriented (perceptual) brain areas. Using fMRI, we observed robust, inverted (negative) retinotopic coding in category-selective memory areas at the cortical apex, which is functionally linked to the classic (positive) retinotopic coding in category-selective perceptual areas in high-level visual cortex. Specifically, these functionally-linked retinotopic populations in mnemonic and perceptual areas exhibit spatially-specific opponent responses during both bottom-up perception and top-down recall, suggesting that these areas are interlocked in a mutually-inhibitory dynamic. Together, these results show that retinotopic coding structures interactions between perceptual and mnemonic neural systems, thereby scaffolding their dynamic interaction.

The language network is not engaged in object categorization

The relationship between language and thought is the subject of long-standing debate. One claim states that language facilitates categorization of objects based on a certain feature (e.g. color) through the use of category labels that reduce interference from other, irrelevant features. Therefore, language impairment is expected to affect categorization of items grouped by a single feature (low-dimensional categories, e.g. "Yellow Things") more than categorization of items that share many features (high-dimensional categories, e.g. "Animals"). To test this account, we conducted two behavioral studies with individuals with aphasia and an fMRI experiment with healthy adults. The aphasia studies showed that selective low-dimensional categorization impairment was present in some, but not all, individuals with severe anomia and was not characteristic of aphasia in general. fMRI results revealed little activity in language-responsive brain regions during both low- and high-dimensional categorization; instead, categorization recruited the domain-general multiple-demand network (involved in wide-ranging cognitive tasks). Combined, results demonstrate that the language system is not implicated in object categorization. Instead, selective low-dimensional categorization impairment might be caused by damage to brain regions responsible for cognitive control. Our work adds to the growing evidence of the dissociation between the language system and many cognitive tasks in adults.

TMS over the pre-SMA enhances semantic cognition via remote network effects on task-based activity and connectivity

BACKGROUND

The continuous decline of executive abilities with age is mirrored by increased neural activity of domain-general networks during task processing. So far, it remains unclear how much domain-general networks contribute to domain-specific processes such as language when cognitive demands increase. The current neuroimaging study explored the potential of intermittent theta-burst stimulation (iTBS) over a domain-general hub to enhance executive and semantic processing in healthy middle-aged to older adults.

METHODS

We implemented a cross-over within-subject study design with three task-based neuroimaging sessions per participant. Using an individualized stimulation approach, each participant received once effective and once sham iTBS over the pre-supplementary motor area (pre-SMA), a region of domain-general control. Subsequently, task-specific stimulation effects were assessed in functional MRI using a semantic and a non-verbal executive task with varying cognitive demand.

RESULTS

Effective stimulation increased activity only during semantic processing in visual and dorsal attention networks. Further, iTBS induced increased seed-based connectivity in task-specific networks for semantic and executive conditions with high cognitive load but overall reduced whole-brain coupling between domain-general networks. Notably, stimulation-induced changes in activity and connectivity related differently to behavior: While stronger activity of the parietal dorsal attention network was linked to poorer semantic performance, its enhanced coupling with the pre-SMA was associated with more efficient semantic processing.

CONCLUSIONS

iTBS modulates networks in a task-dependent manner and generates effects at regions remote to the stimulation site. These neural changes are linked to more efficient semantic processing, which underlines the general potential of network stimulation approaches in cognitive aging.

Individual-specific memory reinstatement patterns within human face-selective cortex

Humans have the remarkable ability to vividly retrieve sensory details of past events. According to the theory of sensory reinstatement, during remembering, brain regions involved in the sensory processing of prior events are reactivated to support this perception of the past. Recently, several studies have emphasized potential transformations in the spatial organization of reinstated activity patterns. In particular, studies of scene stimuli suggest a clear anterior shift in the location of retrieval activations compared with those during perception. However, it is not clear that such transformations occur universally, with evidence lacking for other important stimulus categories, particularly faces. Critical to addressing these questions, and to studies of reinstatement more broadly, is the growing importance of considering meaningful variations in the organization of sensory systems across individuals. Therefore, we conducted a multi-session neuroimaging study to first carefully map individual participants face-selective regions within ventral temporal cortex (VTC), followed by a second session to examine the correspondence of activity patterns during face memory encoding and retrieval. Our results showed distinct configurations of face-selective regions within the VTC across individuals. While a significant degree of overlap was observed between face perception and memory encoding, memory retrieval engagement exhibited a more selective and constricted reinstatement pattern within these regions. Importantly, these activity patterns were consistently tied to individual-specific neural substrates, but did not show any consistent direction of spatial transformation (e.g., anteriorization). To provide further insight to these findings, we also report on unique human intracranial recordings from VTC under the same experimental conditions. Our findings highlight the importance of considering individual variations in functional neuroanatomy in the context of assessing the nature of cortical reinstatement. Consideration of such factors will be important for establishing general principles shaping the neural transformations that occur from perception to memory.

Differential functional reorganization of ventral and dorsal visual pathways following childhood hemispherectomy

Hemispherectomy is a surgical procedure in which an entire hemisphere of a patient's brain is resected or functionally disconnected to manage seizures in individuals with drug-resistant epilepsy. Despite the extensive loss of input from both ventral and dorsal visual pathways of one hemisphere, pediatric patients who have undergone hemispherectomy show a remarkably high degree of perceptual function across many domains. In the current study, we sought to understand the extent to which functions of the ventral and dorsal visual pathways reorganize to the contralateral hemisphere following childhood hemispherectomy. To this end, we collected fMRI data from an equal number of left and right hemispherectomy patients who completed tasks that typically elicit lateralized responses from the ventral or the dorsal pathway, namely, word (left ventral), face (right ventral), tool (left dorsal), and global form (right dorsal) perception. Overall, there was greater evidence of functional reorganization in the ventral pathway than in the dorsal pathway. Importantly, because ventral and dorsal reorganization was tested in the very same patients, these results cannot be explained by idiosyncratic factors such as disease etiology, age at the time of surgery, or age at testing. These findings suggest that because the dorsal pathway may mature earlier, it may have a shorter developmental window of plasticity than the ventral pathway and, hence, be less malleable.

Scene Perception and Visuospatial Memory Converge at the Anterior Edge of Visually Responsive Cortex

To fluidly engage with the world, our brains must simultaneously represent both the scene in front of us and our memory of the immediate surrounding environment (i.e., local visuospatial context). How does the brain's functional architecture enable sensory and mnemonic representations to closely interface while also avoiding sensory-mnemonic interference? Here, we asked this question using first-person, head-mounted virtual reality and fMRI. Using virtual reality, human participants of both sexes learned a set of immersive, real-world visuospatial environments in which we systematically manipulated the extent of visuospatial context associated with a scene image in memory across three learning conditions, spanning from a single FOV to a city street. We used individualized, within-subject fMRI to determine which brain areas support memory of the visuospatial context associated with a scene during recall (Experiment 1) and recognition (Experiment 2). Across the whole brain, activity in three patches of cortex was modulated by the amount of known visuospatial context, each located immediately anterior to one of the three scene perception areas of high-level visual cortex. Individual subject analyses revealed that these anterior patches corresponded to three functionally defined place memory areas, which selectively respond when visually recalling personally familiar places. In addition to showing activity levels that were modulated by the amount of visuospatial context, multivariate analyses showed that these anterior areas represented the identity of the specific environment being recalled. Together, these results suggest a convergence zone for scene perception and memory of the local visuospatial context at the anterior edge of high-level visual cortex.SIGNIFICANCE STATEMENT As we move through the world, the visual scene around us is integrated with our memory of the wider visuospatial context. Here, we sought to understand how the functional architecture of the brain enables coexisting representations of the current visual scene and memory of the surrounding environment. Using a combination of immersive virtual reality and fMRI, we show that memory of visuospatial context outside the current FOV is represented in a distinct set of brain areas immediately anterior and adjacent to the perceptually oriented scene-selective areas of high-level visual cortex. This functional architecture would allow efficient interaction between immediately adjacent mnemonic and perceptual areas while also minimizing interference between mnemonic and perceptual representations.

Auditory dyadic interactions through the "eye" of the social brain: How visual is the posterior STS interaction region?

Human interactions contain potent social cues that meet not only the eye but also the ear. Although research has identified a region in the posterior superior temporal sulcus as being particularly sensitive to visually presented social interactions (SI-pSTS), its response to auditory interactions has not been tested. Here, we used fMRI to explore brain response to auditory interactions, with a focus on temporal regions known to be important in auditory processing and social interaction perception. In Experiment 1, monolingual participants listened to two-speaker conversations (intact or sentence-scrambled) and one-speaker narrations in both a known and an unknown language. Speaker number and conversational coherence were explored in separately localised regions-of-interest (ROI). In Experiment 2, bilingual participants were scanned to explore the role of language comprehension. Combining univariate and multivariate analyses, we found initial evidence for a heteromodal response to social interactions in SI-pSTS. Specifically, right SI-pSTS preferred auditory interactions over control stimuli and represented information about both speaker number and interactive coherence. Bilateral temporal voice areas (TVA) showed a similar, but less specific, profile. Exploratory analyses identified another auditory-interaction sensitive area in anterior STS. Indeed, direct comparison suggests modality specific tuning, with SI-pSTS preferring visual information while aSTS prefers auditory information. Altogether, these results suggest that right SI-pSTS is a heteromodal region that represents information about social interactions in both visual and auditory domains. Future work is needed to clarify the roles of TVA and aSTS in auditory interaction perception and further probe right SI-pSTS interaction-selectivity using non-semantic prosodic cues.

Multi-modal representation of the size of space in the human brain

To estimate the size of an indoor space, we must analyze the visual boundaries that limit the spatial extent and acoustic cues from reflected interior surfaces. We used fMRI to examine how the brain processes geometric size of indoor scenes when various types of sensory cues are presented individually or together. Specifically, we asked whether the size of space is represented in a modality-specific way or in an integrative way that combines multimodal cues. In a block-design study, images or sounds that depict small and large sized indoor spaces were presented. Visual stimuli were real-world pictures of empty spaces that were small or large. Auditory stimuli were sounds convolved with different reverberation. By using a multi-voxel pattern classifier, we asked whether the two sizes of space can be classified in visual, auditory, and visual-auditory combined conditions. We identified both sensory specific and multimodal representations of the size of space. To further investigate the nature of the multimodal region, we specifically examined whether it contained multimodal information in a coexistent or integrated form. We found that AG and the right IFG pars opercularis had modality-integrated representation, displaying sensitivity to the match in the spatial size information conveyed through image and sound. Background functional connectivity analysis further demonstrated that the connection between sensory specific regions and modality-integrated regions increase in the multimodal condition compared to single modality conditions. Our results suggest that the spatial size perception relies on both sensory specific and multimodal representations, as well as their interplay during multimodal perception.

Functionally analogous body- and animacy-responsive areas are present in the dog (Canis familiaris) and human occipito-temporal lobe

Comparing the neural correlates of socio-cognitive skills across species provides insights into the evolution of the social brain and has revealed face- and body-sensitive regions in the primate temporal lobe. Although from a different lineage, dogs share convergent visuo-cognitive skills with humans and a temporal lobe which evolved independently in carnivorans. We investigated the neural correlates of face and body perception in dogs (N = 15) and humans (N = 40) using functional MRI. Combining univariate and multivariate analysis approaches, we found functionally analogous occipito-temporal regions involved in the perception of animate entities and bodies in both species and face-sensitive regions in humans. Though unpredicted, we also observed neural representations of faces compared to inanimate objects, and dog compared to human bodies in dog olfactory regions. These findings shed light on the evolutionary foundations of human and dog social cognition and the predominant role of the temporal lobe.

Humans predict the forest, not the trees: statistical learning of spatiotemporal structure in visual scenes

The human brain is capable of using statistical regularities to predict future inputs. In the real world, such inputs typically comprise a collection of objects (e.g. a forest constitutes numerous trees). The present study aimed to investigate whether perceptual anticipation relies on lower-level or higher-level information. Specifically, we examined whether the human brain anticipates each object in a scene individually or anticipates the scene as a whole. To explore this issue, we first trained participants to associate co-occurring objects within fixed spatial arrangements. Meanwhile, participants implicitly learned temporal regularities between these displays. We then tested how spatial and temporal violations of the structure modulated behavior and neural activity in the visual system using fMRI. We found that participants only showed a behavioral advantage of temporal regularities when the displays conformed to their previously learned spatial structure, demonstrating that humans form configuration-specific temporal expectations instead of predicting individual objects. Similarly, we found suppression of neural responses for temporally expected compared with temporally unexpected objects in lateral occipital cortex only when the objects were embedded within expected configurations. Overall, our findings indicate that humans form expectations about object configurations, demonstrating the prioritization of higher-level over lower-level information in temporal expectation.

No evidence of theory of mind reasoning in the human language network

Language comprehension and the ability to infer others' thoughts (theory of mind [ToM]) are interrelated during development and language use. However, neural evidence that bears on the relationship between language and ToM mechanisms is mixed. Although robust dissociations have been reported in brain disorders, brain activations for contrasts that target language and ToM bear similarities, and some have reported overlap. We take another look at the language-ToM relationship by evaluating the response of the language network, as measured with fMRI, to verbal and nonverbal ToM across 151 participants. Individual-participant analyses reveal that all core language regions respond more strongly when participants read vignettes about false beliefs compared to the control vignettes. However, we show that these differences are largely due to linguistic confounds, and no such effects appear in a nonverbal ToM task. These results argue against cognitive and neural overlap between language processing and ToM. In exploratory analyses, we find responses to social processing in the "periphery" of the language network-right-hemisphere homotopes of core language areas and areas in bilateral angular gyri-but these responses are not selectively ToM-related and may reflect general visual semantic processing.

Non-literal language processing is jointly supported by the language and theory of mind networks: Evidence from a novel meta-analytic fMRI approach

Going beyond the literal meaning of language is key to communicative success. However, the mechanisms that support non-literal inferences remain debated. Using a novel meta-analytic approach, we evaluate the contribution of linguistic, social-cognitive, and executive mechanisms to non-literal interpretation. We identified 74 fMRI experiments (n = 1,430 participants) from 2001 to 2021 that contrasted non-literal language comprehension with a literal control condition, spanning ten phenomena (e.g., metaphor, irony, indirect speech). Applying the activation likelihood estimation approach to the 825 activation peaks yielded six left-lateralized clusters. We then evaluated the locations of both the individual-study peaks and the clusters against probabilistic functional atlases (cf. anatomical locations, as is typically done) for three candidate brain networks-the language-selective network (Fedorenko, Behr, & Kanwisher, 2011), which supports language processing, the Theory of Mind (ToM) network (Saxe & Kanwisher, 2003), which supports social inferences, and the domain-general Multiple-Demand (MD) network (Duncan, 2010), which supports executive control. These atlases were created by overlaying individual activation maps of participants who performed robust and extensively validated 'localizer' tasks that selectively target each network in question (n = 806 for language; n = 198 for ToM; n = 691 for MD). We found that both the individual-study peaks and the ALE clusters fell primarily within the language network and the ToM network. These results suggest that non-literal processing is supported by both i) mechanisms that process literal linguistic meaning, and ii) mechanisms that support general social inference. They thus undermine a strong divide between literal and non-literal aspects of language and challenge the claim that non-literal processing requires additional executive resources.

Category Trumps Shape as an Organizational Principle of Object Space in the Human Occipitotemporal Cortex

The organizational principles of the object space represented in the human ventral visual cortex are debated. Here we contrast two prominent proposals that, in addition to an organization in terms of animacy, propose either a representation related to aspect ratio (stubby-spiky) or to the distinction between faces and bodies. We designed a critical test that dissociates the latter two categories from aspect ratio and investigated responses from human fMRI (of either sex) and deep neural networks (BigBiGAN). Representational similarity and decoding analyses showed that the object space in the occipitotemporal cortex and BigBiGAN was partially explained by animacy but not by aspect ratio. Data-driven approaches showed clusters for face and body stimuli and animate-inanimate separation in the representational space of occipitotemporal cortex and BigBiGAN, but no arrangement related to aspect ratio. In sum, the findings go in favor of a model in terms of an animacy representation combined with strong selectivity for faces and bodies.SIGNIFICANCE STATEMENT We contrasted animacy, aspect ratio, and face-body as principal dimensions characterizing object space in the occipitotemporal cortex. This is difficult to test, as typically faces and bodies differ in aspect ratio (faces are mostly stubby and bodies are mostly spiky). To dissociate the face-body distinction from the difference in aspect ratio, we created a new stimulus set in which faces and bodies have a similar and very wide distribution of values along the shape dimension of the aspect ratio. Brain imaging (fMRI) with this new stimulus set showed that, in addition to animacy, the object space is mainly organized by the face-body distinction and selectivity for aspect ratio is minor (despite its wide distribution).

Precision fMRI reveals that the language-selective network supports both phrase-structure building and lexical access during language production

A fronto-temporal brain network has long been implicated in language comprehension. However, this network's role in language production remains debated. In particular, it remains unclear whether all or only some language regions contribute to production, and which aspects of production these regions support. Across 3 functional magnetic resonance imaging experiments that rely on robust individual-subject analyses, we characterize the language network's response to high-level production demands. We report 3 novel results. First, sentence production, spoken or typed, elicits a strong response throughout the language network. Second, the language network responds to both phrase-structure building and lexical access demands, although the response to phrase-structure building is stronger and more spatially extensive, present in every language region. Finally, contra some proposals, we find no evidence of brain regions-within or outside the language network-that selectively support phrase-structure building in production relative to comprehension. Instead, all language regions respond more strongly during production than comprehension, suggesting that production incurs a greater cost for the language network. Together, these results align with the idea that language comprehension and production draw on the same knowledge representations, which are stored in a distributed manner within the language-selective network and are used to both interpret and generate linguistic utterances.

THINGS-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior

Understanding object representations requires a broad, comprehensive sampling of the objects in our visual world with dense measurements of brain activity and behavior. Here, we present THINGS-data, a multimodal collection of large-scale neuroimaging and behavioral datasets in humans, comprising densely sampled functional MRI and magnetoencephalographic recordings, as well as 4.70 million similarity judgments in response to thousands of photographic images for up to 1,854 object concepts. THINGS-data is unique in its breadth of richly annotated objects, allowing for testing countless hypotheses at scale while assessing the reproducibility of previous findings. Beyond the unique insights promised by each individual dataset, the multimodality of THINGS-data allows combining datasets for a much broader view into object processing than previously possible. Our analyses demonstrate the high quality of the datasets and provide five examples of hypothesis-driven and data-driven applications. THINGS-data constitutes the core public release of the THINGS initiative (https://things-initiative.org) for bridging the gap between disciplines and the advancement of cognitive neuroscience.

Intact reading ability despite lacking a canonical visual word form area in an individual born without the left superior temporal lobe

The visual word form area (VWFA), a region canonically located within left ventral temporal cortex (VTC), is specialized for orthography in literate adults presumbly due to its connectivity with frontotemporal language regions. But is a typical, left-lateralized language network critical for the VWFA's emergence? We investigated this question in an individual (EG) born without the left superior temporal lobe but who has normal reading ability. EG showed canonical typical face-selectivity bilateraly but no wordselectivity either in right VWFA or in the spared left VWFA. Moreover, in contrast with the idea that the VWFA is simply part of the language network, no part of EG's VTC showed selectivity to higher-level linguistic processing. Interestingly, EG's VWFA showed reliable multivariate patterns that distinguished words from other categories. These results suggest that a typical left-hemisphere language network is necessary for acanonical VWFA, and that orthographic processing can otherwise be supported by a distributed neural code.

Temporal asymmetries and interactions between dorsal and ventral visual pathways during object recognition

Despite their anatomical and functional distinctions, there is growing evidence that the dorsal and ventral visual pathways interact to support object recognition. However, the exact nature of these interactions remains poorly understood. Is the presence of identity-relevant object information in the dorsal pathway simply a byproduct of ventral input? Or, might the dorsal pathway be a source of input to the ventral pathway for object recognition? In the current study, we used high-density EEG-a technique with high temporal precision and spatial resolution sufficient to distinguish parietal and temporal lobes-to characterise the dynamics of dorsal and ventral pathways during object viewing. Using multivariate analyses, we found that category decoding in the dorsal pathway preceded that in the ventral pathway. Importantly, the dorsal pathway predicted the multivariate responses of the ventral pathway in a time-dependent manner, rather than the other way around. Together, these findings suggest that the dorsal pathway is a critical source of input to the ventral pathway for object recognition.

Functionally and structurally distinct fusiform face area(s) in over 1000 participants

The fusiform face area (FFA) is a widely studied region causally involved in face perception. Even though cognitive neuroscientists have been studying the FFA for over two decades, answers to foundational questions regarding the function, architecture, and connectivity of the FFA from a large (N>1000) group of participants are still lacking. To fill this gap in knowledge, we quantified these multimodal features of fusiform face-selective regions in 1053 participants in the Human Connectome Project. After manually defining over 4,000 fusiform face-selective regions, we report five main findings. First, 68.76% of hemispheres have two cortically separate regions (pFus-faces/FFA-1 and mFus-faces/FFA-2). Second, in 26.69% of hemispheres, pFus-faces/FFA-1 and mFus-faces/FFA-2 are spatially contiguous, yet are distinct based on functional, architectural, and connectivity metrics. Third, pFus-faces/FFA-1 is more face-selective than mFus-faces/FFA-2, and the two regions have distinct functional connectivity fingerprints. Fourth, pFus-faces/FFA-1 is cortically thinner and more heavily myelinated than mFus-faces/FFA-2. Fifth, face-selective patterns and functional connectivity fingerprints of each region are more similar in monozygotic than dizygotic twins and more so than architectural gradients. As we share our areal definitions with the field, future studies can explore how structural and functional features of these regions will inform theories regarding how visual categories are represented in the brain.

Functionally and structurally distinct fusiform face area(s) in over 1000 participants

The fusiform face area (FFA) is a widely studied region causally involved in face perception. Even though cognitive neuroscientists have been studying the FFA for over two decades, answers to foundational questions regarding the function, architecture, and connectivity of the FFA from a large (N>1000) group of participants are still lacking. To fill this gap in knowledge, we quantified these multimodal features of fusiform face-selective regions in 1053 participants in the Human Connectome Project. After manually defining over 4,000 fusiform face-selective regions, we report five main findings. First, 68.76% of hemispheres have two cortically separate regions (pFus-faces/FFA-1 and mFus-faces/FFA-2). Second, in 26.69% of hemispheres, pFus-faces/FFA-1 and mFus-faces/FFA-2 are spatially contiguous, yet are distinct based on functional, architectural, and connectivity metrics. Third, pFus-faces/FFA-1 is more face-selective than mFus-faces/FFA-2, and the two regions have distinct functional connectivity fingerprints. Fourth, pFus-faces/FFA-1 is cortically thinner and more heavily myelinated than mFus-faces/FFA-2. Fifth, face-selective patterns and functional connectivity fingerprints of each region are more similar in monozygotic than dizygotic twins and more so than architectural gradients. As we share our areal definitions with the field, future studies can explore how structural and functional features of these regions will inform theories regarding how visual categories are represented in the brain.

Evaluating the efficacy of multi-echo ICA denoising on model-based fMRI

fMRI is an indispensable tool for neuroscience investigation, but this technique is limited by multiple sources of physiological and measurement noise. These noise sources are particularly problematic for analysis techniques that require high signal-to-noise ratio for stable model fitting, such as voxel-wise modeling. Multi-echo data acquisition in combination with echo-time dependent ICA denoising (ME-ICA) represents one promising strategy to mitigate physiological and hardware-related noise sources as well as motion-related artifacts. However, most studies employing ME-ICA to date are resting-state fMRI studies, and therefore we have a limited understanding of the impact of ME-ICA on complex task or model-based fMRI paradigms. Here, we addressed this knowledge gap by comparing data quality and model fitting performance of data acquired during a visual population receptive field (pRF) mapping (N = 13 participants) experiment after applying one of three preprocessing procedures: ME-ICA, optimally combined multi-echo data without ICA-denoising, and typical single echo processing. As expected, multi-echo fMRI improved temporal signal-to-noise compared to single echo fMRI, with ME-ICA amplifying the improvement compared to optimal combination alone. However, unexpectedly, this boost in temporal signal-to-noise did not directly translate to improved model fitting performance: compared to single echo acquisition, model fitting was only improved after ICA-denoising. Specifically, compared to single echo acquisition, ME-ICA resulted in improved variance explained by our pRF model throughout the visual system, including anterior regions of the temporal and parietal lobes where SNR is typically low, while optimal combination without ICA did not. ME-ICA also improved reliability of parameter estimates compared to single echo and optimally combined multi-echo data without ICA-denoising. Collectively, these results suggest that ME-ICA is effective for denoising task-based fMRI data for modeling analyzes and maintains the integrity of the original data. Therefore, ME-ICA may be beneficial for complex fMRI experiments, including voxel-wise modeling and naturalistic paradigms.