Temporal Processing Capacity in High-Level Visual Cortex Is Domain Specific

A quantitative comparison of atlas parcellations on the human superior temporal sulcus

The superior temporal sulcus (STS) has a functional topography that has been difficult to characterize through traditional approaches. Automated atlas parcellations may be one solution while also being beneficial for both dimensional reduction and standardizing regions of interest, but they yield very different boundary definitions along the STS. Here we evaluate how well machine learning classifiers can correctly identify six social cognitive tasks from STS activation patterns dimensionally reduced using four popular atlases (Glasser et al., 2016; Gordon et al., 2016; Power et al., 2011 as projected onto the surface by Arslan et al., 2018; Schaefer et al., 2018). Functional data was summarized within each STS parcel in one of four ways, then subjected to leave-one-subject-out cross-validation SVM classification. We found that the classifiers could readily label conditions when data was parcellated using any of the four atlases, evidence that dimensional reduction to parcels did not compromise functional fingerprints. Mean activation for the social conditions was the most effective metric for classification in the right STS, whereas all the metrics classified equally well in the left STS. Interestingly, even atlases constructed from random parcellation schemes (null atlases) classified the conditions with high accuracy. We therefore conclude that the complex activation maps on the STS are readily differentiated at a coarse granular level, despite a strict topography having not yet been identified. Further work is required to identify what features have greatest potential to improve the utility of atlases in replacing functional localizers.

Contrastive learning explains the emergence and function of visual category-selective regions

Modular and distributed coding theories of category selectivity along the human ventral visual stream have long existed in tension. Here, we present a reconciling framework-contrastive coding-based on a series of analyses relating category selectivity within biological and artificial neural networks. We discover that, in models trained with contrastive self-supervised objectives over a rich natural image diet, category-selective tuning naturally emerges for faces, bodies, scenes, and words. Further, lesions of these model units lead to selective, dissociable recognition deficits, highlighting their distinct functional roles in information processing. Finally, these pre-identified units can predict neural responses in all corresponding face-, scene-, body-, and word-selective regions of human visual cortex, under a highly constrained sparse positive encoding procedure. The success of this single model indicates that brain-like functional specialization can emerge without category-specific learning pressures, as the system learns to untangle rich image content. Contrastive coding, therefore, provides a unifying account of object category emergence and representation in the human brain.

A practical guide for combining functional regions of interest and white matter bundles

Diffusion-weighted imaging (DWI) is the primary method to investigate macro- and microstructure of neural white matter in vivo. DWI can be used to identify and characterize individual-specific white matter bundles, enabling precise analyses on hypothesis-driven connections in the brain and bridging the relationships between brain structure, function, and behavior. However, cortical endpoints of bundles may span larger areas than what a researcher is interested in, challenging presumptions that bundles are specifically tied to certain brain functions. Functional MRI (fMRI) can be integrated to further refine bundles such that they are restricted to functionally-defined cortical regions. Analyzing properties of these Functional Sub-Bundles (FSuB) increases precision and interpretability of results when studying neural connections supporting specific tasks. Several parameters of DWI and fMRI analyses, ranging from data acquisition to processing, can impact the efficacy of integrating functional and diffusion MRI. Here, we discuss the applications of the FSuB approach, suggest best practices for acquiring and processing neuroimaging data towards this end, and introduce the FSuB-Extractor, a flexible open-source software for creating FSuBs. We demonstrate our processing code and the FSuB-Extractor on an openly-available dataset, the Natural Scenes Dataset.

Both mOTS-words and pOTS-words prefer emoji stimuli over text stimuli during a lexical judgment task

The visual word form area in the occipitotemporal sulcus (here OTS-words) is crucial for reading and shows a preference for text stimuli. We hypothesized that this text preference may be driven by lexical processing. Hence, we performed three fMRI experiments (n = 15), systematically varying participants' task and stimulus, and separately evaluated middle mOTS-words and posterior pOTS-words. Experiment 1 contrasted text with other visual stimuli to identify both OTS-words subregions. Experiment 2 utilized an fMRI adaptation paradigm, presenting compound words as texts or emojis. In experiment 3, participants performed a lexical or color judgment task on compound words in text or emoji format. In experiment 2, pOTS-words, but not mOTS-words, showed fMRI adaptation for compound words in both formats. In experiment 3, both subregions showed higher responses to compound words in emoji format. Moreover, mOTS-words showed higher responses during the lexical judgment task and a task-stimulus interaction. Multivariate analyses revealed that distributed responses in pOTS-words encode stimulus and distributed responses in mOTS-words encode stimulus and task. Together, our findings suggest that the function of the OTS-words subregions goes beyond the specific visual processing of text and that these regions are flexibly recruited whenever semantic meaning needs to be assigned to visual input.

White matter connections of human ventral temporal cortex are organized by cytoarchitecture, eccentricity, and category-selectivity from birth

Category-selective regions in ventral temporal cortex (VTC) have a consistent anatomical organization, which is hypothesized to be scaffolded by white matter connections. However, it is unknown how white matter connections are organized from birth. Here, we scanned newborn to 6-month-old infants and adults and used a data-driven approach to determine the organization of the white matter connections of VTC. We find that white matter connections are organized by cytoarchitecture, eccentricity, and category from birth. Connectivity profiles of functional regions in the same cytoarchitectonic area are similar from birth and develop in parallel, with decreases in endpoint connectivity to lateral occipital, and parietal, and somatosensory cortex, and increases to lateral prefrontal cortex. Additionally, connections between VTC and early visual cortex are organized topographically by eccentricity bands and predict eccentricity biases in VTC. These data have important implications for theories of cortical functional development and open new possibilities for understanding typical and atypical white matter development.

A unifying framework for functional organization in early and higher ventral visual cortex

A key feature of cortical systems is functional organization: the arrangement of functionally distinct neurons in characteristic spatial patterns. However, the principles underlying the emergence of functional organization in the cortex are poorly understood. Here, we develop the topographic deep artificial neural network (TDANN), the first model to predict several aspects of the functional organization of multiple cortical areas in the primate visual system. We analyze the factors driving the TDANN's success and find that it balances two objectives: learning a task-general sensory representation and maximizing the spatial smoothness of responses according to a metric that scales with cortical surface area. In turn, the representations learned by the TDANN are more brain-like than in spatially unconstrained models. Finally, we provide evidence that the TDANN's functional organization balances performance with between-area connection length. Our results offer a unified principle for understanding the functional organization of the primate ventral visual system.

Reading Reshapes Stimulus Selectivity in the Visual Word Form Area

Reading depends on a brain region known as the "visual word form area" (VWFA) in the left ventral occipitotemporal cortex. This region's function is debated because its stimulus selectivity is not absolute, it is modulated by a variety of task demands, and it is inconsistently localized. We used fMRI to characterize the combination of sensory and cognitive factors that activate word-responsive regions that we precisely localized in 16 adult humans (4 male). We then presented three types of character strings: English words, pseudowords, and unfamiliar characters with matched visual features. Participants performed three different tasks while viewing those stimuli: detecting real words, detecting color in the characters, and detecting color in the fixation mark. There were three primary findings about the VWFA's response: (1) It preferred letter strings over unfamiliar characters even when the stimuli were ignored during the fixation task. (2) Compared with those baseline responses, engaging in the word reading task enhanced the response to words but suppressed the response to unfamiliar characters. (3) Attending to the stimuli to judge their color had little effect on the response magnitudes. Thus, the VWFA is uniquely modulated by a cognitive signal that is specific to voluntary linguistic processing and is not additive. Functional connectivity analyses revealed that communication between the VWFA and a left frontal language area increased when the participant engaged in the linguistic task. We conclude that the VWFA is inherently selective for familiar orthography, but it falls under control of the language network when the task demands it.

Selective activations and functional connectivities to the sight of faces, scenes, body parts and tools in visual and non-visual cortical regions leading to the human hippocampus

Connectivity maps are now available for the 360 cortical regions in the Human Connectome Project Multimodal Parcellation atlas. Here we add function to these maps by measuring selective fMRI activations and functional connectivity increases to stationary visual stimuli of faces, scenes, body parts and tools from 956 HCP participants. Faces activate regions in the ventrolateral visual cortical stream (FFC), in the superior temporal sulcus (STS) visual stream for face and head motion; and inferior parietal visual (PGi) and somatosensory (PF) regions. Scenes activate ventromedial visual stream VMV and PHA regions in the parahippocampal scene area; medial (7m) and lateral parietal (PGp) regions; and the reward-related medial orbitofrontal cortex. Body parts activate the inferior temporal cortex object regions (TE1p, TE2p); but also visual motion regions (MT, MST, FST); and the inferior parietal visual (PGi, PGs) and somatosensory (PF) regions; and the unpleasant-related lateral orbitofrontal cortex. Tools activate an intermediate ventral stream area (VMV3, VVC, PHA3); visual motion regions (FST); somatosensory (1, 2); and auditory (A4, A5) cortical regions. The findings add function to cortical connectivity maps; and show how stationary visual stimuli activate other cortical regions related to their associations, including visual motion, somatosensory, auditory, semantic, and orbitofrontal cortex value-related, regions.

Reading reshapes stimulus selectivity in the visual word form area

Reading depends on a brain region known as the "visual word form area" (VWFA) in left ventral occipito-temporal cortex. This region's function is debated because its stimulus selectivity is not absolute, it is modulated by a variety of task demands, and it is inconsistently localized. We used fMRI to characterize the combination of sensory and cognitive factors that activate word-responsive regions that we precisely localized in 16 adult humans (4 male). We then presented three types of character strings: English words, pseudowords, and unfamiliar characters with matched visual features. Participants performed three different tasks while viewing those stimuli: detecting real words, detecting color in the characters, and detecting color in the fixation mark. There were three primary findings about the VWFA's response: (1) It preferred letter strings over unfamiliar characters even when the stimuli were ignored during the fixation task; (2) Compared to those baseline responses, engaging in the word reading task enhanced the response to words but suppressed the response to unfamiliar characters. (3) Attending to the stimuli to judge their font color had little effect on the response magnitudes. Thus, the VWFA is uniquely modulated by a cognitive signal that is specific to voluntary linguistic processing and is not additive. Functional connectivity analyses revealed that communication between the VWFA and a left frontal language area increased when the participant engaged in the linguistic task. We conclude that the VWFA is inherently selective for familiar orthography, but it falls under control of the language network when the task demands it.

Reading instruction causes changes in category-selective visual cortex

Education sculpts specialized neural circuits for skills like reading that are critical to success in modern society but were not anticipated by the selective pressures of evolution. Does the emergence of brain regions that selectively process novel visual stimuli like words occur at the expense of cortical representations of other stimuli like faces and objects? "Neuronal Recycling" predicts that learning to read should enhance the response to words in ventral occipitotemporal cortex (VOTC) and decrease the response to other visual categories such as faces and objects. To test this hypothesis, and more broadly to understand the changes that are induced by the early stages of literacy instruction, we conducted a randomized controlled trial with pre-school children (five years of age). Children were randomly assigned to intervention programs focused on either reading skills or oral language skills and magnetoencephalography (MEG) data collected before and after the intervention was used to measure visual responses to images of text, faces, and objects. We found that being taught reading versus oral language skills induced different patterns of change in category-selective regions of visual cortex, but that there was not a clear tradeoff between the response to words versus other categories. Within a predefined region of VOTC corresponding to the visual word form area (VWFA) we found that the relative amplitude of responses to text, faces, and objects changed, but increases in the response to words were not linked to decreases in the response to faces or objects. How these changes play out over a longer timescale is still unknown but, based on these data, we can surmise that high-level visual cortex undergoes rapid changes as children enter school and begin establishing new skills like literacy.

Natural scenes reveal diverse representations of 2D and 3D body pose in the human brain

Human pose, defined as the spatial relationships between body parts, carries instrumental information supporting the understanding of motion and action of a person. A substantial body of previous work has identified cortical areas responsive to images of bodies and different body parts. However, the neural basis underlying the visual perception of body part relationships has received less attention. To broaden our understanding of body perception, we analyzed high-resolution fMRI responses to a wide range of poses from over 4,000 complex natural scenes. Using ground-truth annotations and an application of three-dimensional (3D) pose reconstruction algorithms, we compared similarity patterns of cortical activity with similarity patterns built from human pose models with different levels of depth availability and viewpoint dependency. Targeting the challenge of explaining variance in complex natural image responses with interpretable models, we achieved statistically significant correlations between pose models and cortical activity patterns (though performance levels are substantially lower than the noise ceiling). We found that the 3D view-independent pose model, compared with two-dimensional models, better captures the activation from distinct cortical areas, including the right posterior superior temporal sulcus (pSTS). These areas, together with other pose-selective regions in the LOTC, form a broader, distributed cortical network with greater view-tolerance in more anterior patches. We interpret these findings in light of the computational complexity of natural body images, the wide range of visual tasks supported by pose structures, and possible shared principles for view-invariant processing between articulated objects and ordinary, rigid objects.

Pre-saccadic Preview Shapes Post-Saccadic Processing More Where Perception is Poor

The pre-saccadic preview of a peripheral target enhances the efficiency of its post-saccadic processing, termed the extrafoveal preview effect. Peripheral visual performance -and thus the quality of the preview- varies around the visual field, even at iso-eccentric locations: it is better along the horizontal than vertical meridian and along the lower than upper vertical meridian. To investigate whether these polar angle asymmetries influence the preview effect, we asked human participants (to preview four tilted gratings at the cardinals, until a central cue indicated to which one to saccade. During the saccade, the target orientation either remained or slightly changed (valid/invalid preview). After saccade landing, participants discriminated the orientation of the (briefly presented) second grating. Stimulus contrast was titrated with adaptive staircases to assess visual performance. Expectedly, valid previews increased participants' post-saccadic contrast sensitivity. This preview benefit, however, was inversely related to polar angle perceptual asymmetries; largest at the upper, and smallest at the horizontal meridian. This finding reveals that the visual system compensates for peripheral asymmetries when integrating information across saccades, by selectively assigning higher weights to the less-well perceived preview information. Our study supports the recent line of evidence showing that perceptual dynamics around saccades vary with eye movement direction.

Individual differences in face salience and rapid face saccades

Humans saccade to faces in their periphery faster than to other types of objects. Previous research has highlighted the potential importance of the upper face region in this phenomenon, but it remains unclear whether this is driven by the eye region. Similarly, it remains unclear whether such rapid saccades are exclusive to faces or generalize to other semantically salient stimuli. Furthermore, it is unknown whether individuals differ in their face-specific saccadic reaction times and, if so, whether such differences could be linked to differences in face fixations during free viewing. To explore these open questions, we invited 77 participants to perform a saccadic choice task in which we contrasted faces as well as other salient objects, particularly isolated face features and text, with cars. Additionally, participants freely viewed 700 images of complex natural scenes in a separate session, which allowed us to determine the individual proportion of first fixations falling on faces. For the saccadic choice task, we found advantages for all categories of interest over cars. However, this effect was most pronounced for images of full faces. Full faces also elicited faster saccades compared with eyes, showing that isolated eye regions are not sufficient to elicit face-like responses. Additionally, we found consistent individual differences in saccadic reaction times toward faces that weakly correlated with face salience during free viewing. Our results suggest a link between semantic salience and rapid detection, but underscore the unique status of faces. Further research is needed to resolve the mechanisms underlying rapid face saccades.

Perception and Memory Reinstatement Engage Overlapping Face-Selective Regions within Human Ventral Temporal Cortex

Humans have the remarkable ability to vividly retrieve sensory details of past events. According to the theory of sensory reinstatement, during remembering, brain regions specialized for processing specific sensory stimuli are reactivated to support content-specific retrieval. Recently, several studies have emphasized transformations in the spatial organization of these reinstated activity patterns. Specifically, studies of scene stimuli suggest a clear anterior shift in the location of retrieval activations compared with the activity observed during perception. However, it is not clear that such transformations occur universally, with inconsistent evidence for other important stimulus categories, particularly faces. One challenge in addressing this question is the careful delineation of face-selective cortices, which are interdigitated with other selective regions, in configurations that spatially differ across individuals. Therefore, we conducted a multisession neuroimaging study to first carefully map individual participants' (nine males and seven females) face-selective regions within ventral temporal cortex (VTC), followed by a second session to examine the activity patterns within these regions during face memory encoding and retrieval. While face-selective regions were expectedly engaged during face perception at encoding, memory retrieval engagement exhibited a more selective and constricted reinstatement pattern within these regions, but did not show any consistent direction of spatial transformation (e.g., anteriorization). We also report on unique human intracranial recordings from VTC under the same experimental conditions. These findings highlight the importance of considering the complex configuration of category-selective cortex in elucidating principles shaping the neural transformations that occur from perception to memory.

Functional Resilience of the Neural Visual Recognition System Post-Pediatric Occipitotemporal Resection

In the typically developing (TD) brain, neural representations for visual stimulus categories (e.g., faces, objects, and words) emerge in bilateral occipitotemporal cortex (OTC), albeit with weighted asymmetry; in parallel, recognition behavior continues to be refined. A fundamental question is whether two hemispheres are necessary or redundant for the emergence of neural representations and recognition behavior typically distributed across both hemispheres. The rare population of patients undergoing unilateral OTC resection in childhood offers a unique opportunity to evaluate whether neural computations for visual stimulus individuation suffice for recognition with only a single developing OTC. Here, using functional magnetic resonance imaging, we mapped category selectivity (CS) and neural representations for individual stimulus exemplars using repetition suppression (RS) in the non-resected hemisphere of pediatric OTC resection patients (n = 9) and control patients with resection outside of OTC (n = 12), as well as in both hemispheres of TD controls (n = 21). There were no univariate group differences in the magnitude of CS or RS or any multivariate differences (per representational similarity analysis) in neural activation to faces, objects, or words across groups. Notwithstanding their comparable neural profiles, accuracy of OTC resection patients on face and object recognition, but not word recognition, was statistically inferior to that of controls. The comparable neural signature of the OTC resection patients' preserved hemisphere and the other two groups highlights the resilience of the system following damage to the contralateral homologue. Critically, however, a single OTC does not suffice for normal behavior, and, thereby, implicates the necessity for two hemispheres.

Dissociable Contributions of the Medial Parietal Cortex to Recognition Memory

Human neuroimaging studies of episodic memory retrieval routinely observe the engagement of specific cortical regions beyond the medial temporal lobe. Of these, medial parietal cortex (MPC) is of particular interest given its distinct functional characteristics during different retrieval tasks. Specifically, while recognition and autobiographical recall tasks are both used to probe episodic retrieval, these paradigms consistently drive distinct spatial patterns of response within MPC. However, other studies have emphasized alternate MPC functional dissociations in terms of brain network connectivity profiles or stimulus category selectivity. As the unique contributions of MPC to episodic memory remain unclear, adjudicating between these different accounts can provide better consensus regarding MPC function. Therefore, we used a precision-neuroimaging dataset (7T functional magnetic resonance imaging) to examine how MPC regions are differentially engaged during recognition memory and how these task-related dissociations may also reflect distinct connectivity and stimulus category functional profiles. We observed interleaved, though spatially distinct, subregions of MPC where responses were sensitive to either recognition decisions or the semantic representation of stimuli. In addition, this dissociation was further accentuated by functional subregions displaying distinct profiles of connectivity with the hippocampus during task and rest. Finally, we show that recent observations of dissociable person and place selectivity within the MPC reflect category-specific responses from within identified semantic regions that are sensitive to mnemonic demands. Together, by examining precision functional mapping within individuals, these data suggest that previously distinct observations of functional dissociation within MPC conform to a common principle of organization throughout hippocampal-neocortical memory systems.

The transition from vision to language: Distinct patterns of functional connectivity for subregions of the visual word form area

Reading entails transforming visual symbols to sound and meaning. This process depends on specialized circuitry in the visual cortex, the visual word form area (VWFA). Recent findings suggest that this text-selective cortex comprises at least two distinct subregions: the more posterior VWFA-1 is sensitive to visual features, while the more anterior VWFA-2 processes higher level language information. Here, we explore whether these two subregions also exhibit different patterns of functional connectivity. To this end, we capitalize on two complementary datasets: Using the Natural Scenes Dataset (NSD), we identify text-selective responses in high-quality 7T adult data (N = 8), and investigate functional connectivity patterns of VWFA-1 and VWFA-2 at the individual level. We then turn to the Healthy Brain Network (HBN) database to assess whether these patterns replicate in a large developmental sample (N = 224; age 6-20 years), and whether they relate to reading development. In both datasets, we find that VWFA-1 is primarily correlated with bilateral visual regions. In contrast, VWFA-2 is more strongly correlated with language regions in the frontal and lateral parietal lobes, particularly the bilateral inferior frontal gyrus. Critically, these patterns do not generalize to adjacent face-selective regions, suggesting a specific relationship between VWFA-2 and the frontal language network. No correlations were observed between functional connectivity and reading ability. Together, our findings support the distinction between subregions of the VWFA, and suggest that functional connectivity patterns in the ventral temporal cortex are consistent over a wide range of reading skills.

Visual word processing engages a hierarchical, distributed, and bilateral cortical network

Although the Visual Word Form Area (VWFA) in left temporal cortex is considered the pre-eminent region in visual word processing, other regions are also implicated. We examined the entire text-selective circuit, using functional MRI. Ten regions of interest (ROIs) per hemisphere were defined, which, based on clustering, grouped into early vision, high-level vision, and language clusters. We analyzed the responses of the ROIs and clusters to words, inverted words, and consonant strings using univariate, multivariate, and functional connectivity measures. Bilateral modulation by stimulus condition was evident, with a stronger effect in left hemisphere regions. Last, using graph theory, we observed that the VWFA was equivalently connected with early visual and language clusters in both hemispheres, reflecting its role as a mediator in the circuit. Although the individual ROIs and clusters bilaterally were flexibly altered by the nature of the input, stability held at the level of global circuit connectivity, reflecting the complex hierarchical distributed system serving visual text perception.

Mechanisms underlying category learning in the human ventral occipito-temporal cortex

The human ventral occipito-temporal cortex (VOTC) has evolved into specialized regions that process specific categories, such as words, tools, and animals. The formation of these areas is driven by bottom-up visual and top-down nonvisual experiences. However, the specific mechanisms through which top-down nonvisual experiences modulate category-specific regions in the VOTC are still unknown. To address this question, we conducted a study in which participants were trained for approximately 13 h to associate three sets of novel meaningless figures with different top-down nonvisual features: the wordlike category with word features, the non-wordlike category with nonword features, and the visual familiarity condition with no nonvisual features. Pre- and post-training functional MRI (fMRI) experiments were used to measure brain activity during stimulus presentation. Our results revealed that training induced a categorical preference for the two training categories within the VOTC. Moreover, the locations of two training category-specific regions exhibited a notable overlap. Remarkably, within the overlapping category-specific region, training resulted in a dissociation in activation intensity and pattern between the two training categories. These findings provide important insights into how different nonvisual categorical information is encoded in the human VOTC.

Category-Selective Representation of Relationships in the Visual Cortex

Understanding social interaction requires processing social agents and their relationships. The latest results show that much of this process is visually solved: visual areas can represent multiple people encoding emergent information about their interaction that is not explained by the response to the individuals alone. A neural signature of this process is an increased response in visual areas, to face-to-face (seemingly interacting) people, relative to people presented as unrelated (back-to-back). This effect highlighted a network of visual areas for representing relational information. How is this network organized? Using functional MRI, we measured the brain activity of healthy female and male humans (N = 42), in response to images of two faces or two (head-blurred) bodies, facing toward or away from each other. Taking the facing > non-facing effect as a signature of relation perception, we found that relations between faces and between bodies were coded in distinct areas, mirroring the categorical representation of faces and bodies in the visual cortex. Additional analyses suggest the existence of a third network encoding relations between (nonsocial) objects. Finally, a separate occipitotemporal network showed the generalization of relational information across body, face, and nonsocial object dyads (multivariate pattern classification analysis), revealing shared properties of relations across categories. In sum, beyond single entities, the visual cortex encodes the relations that bind multiple entities into relationships; it does so in a category-selective fashion, thus respecting a general organizing principle of representation in high-level vision. Visual areas encoding visual relational information can reveal the processing of emergent properties of social (and nonsocial) interaction, which trigger inferential processes.

Both mOTS-words and pOTS-words prefer emoji stimuli over text stimuli during a reading task

The visual word form area in the occipitotemporal sulcus (OTS), here referred to as OTS-words, responds more strongly to text than other visual stimuli and is crucial for reading. We hypothesized, that this text preference may be driven by a preference for reading tasks, as in most prior fMRI studies only the text stimuli were readable. Hence, we performed three fMRI experiments (N=15) and systematically varied the participant's task and the stimulus, investigating mOTS-words and pOTS-words subregions. In experiment 1, we contrasted text stimuli with non-readable visual stimuli (faces, limbs, houses, objects). Experiment 2 utilized an fMRI adaptation paradigm, presenting compound words in text or emoji formats. In experiment 3, participants performed a reading or a color task on compound words in text or emoji format. Using experiment 1 data, we identified mOTS-words and pOTS-words by contrasting texts with non-readable stimuli. In experiment 2, pOTS-words, but not mOTS-words, showed fMRI adaptation for compound words in both text and emoji formats. In experiment 3, surprisingly, both subregions showed higher responses to compound words in emoji than text format. Moreover, mOTS-words showed higher responses during the reading than the color task and a task-stimulus interaction. Multivariate analyses revealed that distributed responses in pOTS-words encode the visual stimulus, while responses in mOTS-words encode both stimulus and task. Together, our findings suggest that the function of the OTS-words subregions goes beyond the specific visual processing of text and that these regions are flexibly recruited whenever semantic meaning needs to be assigned to visual input.

Converging evidence that left extrastriate body area supports visual sensitivity to social interactions

Navigating our complex social world requires processing the interactions we observe. Recent psychophysical and neuroimaging studies provide parallel evidence that the human visual system may be attuned to efficiently perceive dyadic interactions. This work implies, but has not yet demonstrated, that activity in body-selective cortical regions causally supports efficient visual perception of interactions. We adopt a multi-method approach to close this important gap. First, using a large fMRI dataset (n = 92), we found that the left hemisphere extrastriate body area (EBA) responds more to face-to-face than non-facing dyads. Second, we replicated a behavioral marker of visual sensitivity to interactions: categorization of facing dyads is more impaired by inversion than non-facing dyads. Third, in a pre-registered experiment, we used fMRI-guided transcranial magnetic stimulation to show that online stimulation of the left EBA, but not a nearby control region, abolishes this selective inversion effect. Activity in left EBA, thus, causally supports the efficient perception of social interactions.

Longitudinal development of category representations in ventral temporal cortex predicts word and face recognition

Regions in ventral temporal cortex that are involved in visual recognition of categories like words and faces undergo differential development during childhood. However, categories are also represented in distributed responses across high-level visual cortex. How distributed category representations develop and if this development relates to behavioral changes in recognition remains largely unknown. Here, we used functional magnetic resonance imaging to longitudinally measure the development of distributed responses across ventral temporal cortex to 10 categories in school-age children over several years. Our results reveal both strengthening and weakening of category representations with age, which was mainly driven by changes across category-selective voxels. Representations became particularly more distinct for words in the left hemisphere and for faces bilaterally. Critically, distinctiveness for words and faces across category-selective voxels in left and right lateral ventral temporal cortex, respectively, predicted individual children's word and face recognition performance. These results suggest that the development of distributed representations in ventral temporal cortex has behavioral ramifications and advance our understanding of prolonged cortical development during childhood.

Human brain responses are modulated when exposed to optimized natural images or synthetically generated images

Understanding how human brains interpret and process information is important. Here, we investigated the selectivity and inter-individual differences in human brain responses to images via functional MRI. In our first experiment, we found that images predicted to achieve maximal activations using a group level encoding model evoke higher responses than images predicted to achieve average activations, and the activation gain is positively associated with the encoding model accuracy. Furthermore, anterior temporal lobe face area (aTLfaces) and fusiform body area 1 had higher activation in response to maximal synthetic images compared to maximal natural images. In our second experiment, we found that synthetic images derived using a personalized encoding model elicited higher responses compared to synthetic images from group-level or other subjects' encoding models. The finding of aTLfaces favoring synthetic images than natural images was also replicated. Our results indicate the possibility of using data-driven and generative approaches to modulate macro-scale brain region responses and probe inter-individual differences in and functional specialization of the human visual system.

Dissociable contributions of the medial parietal cortex to recognition memory

Human neuroimaging studies of episodic memory retrieval routinely observe the engagement of specific cortical regions beyond the medial temporal lobe. Of these, medial parietal cortex (MPC) is of particular interest given its ubiquitous, and yet distinct, functional characteristics during different types of retrieval tasks. Specifically, while recognition memory and autobiographical recall tasks are both used to probe episodic retrieval, these paradigms consistently drive distinct patterns of response within MPC. This dissociation adds to growing evidence suggesting a common principle of functional organization across memory related brain structures, specifically regarding the control or content demands of memory-based decisions. To carefully examine this putative organization, we used a high-resolution fMRI dataset collected at ultra-high field (7T) while subjects performed thousands of recognition-memory trials to identify MPC regions responsive to recognition-decisions or semantic content of stimuli within and across individuals. We observed interleaving, though distinct, functional subregions of MPC where responses were sensitive to either recognition decisions or the semantic representation of stimuli, but rarely both. In addition, this functional dissociation within MPC was further accentuated by distinct profiles of connectivity bias with the hippocampus during task and rest. Finally, we show that recent observations of person and place selectivity within MPC reflect category specific responses from within identified semantic regions that are sensitive to mnemonic demands. Together, these data better account for how distinct patterns of MPC responses can occur as a result of task demands during episodic retrieval and may reflect a common principle of organization throughout hippocampal-neocortical memory systems.

A large-scale fMRI dataset for the visual processing of naturalistic scenes

One ultimate goal of visual neuroscience is to understand how the brain processes visual stimuli encountered in the natural environment. Achieving this goal requires records of brain responses under massive amounts of naturalistic stimuli. Although the scientific community has put a lot of effort into collecting large-scale functional magnetic resonance imaging (fMRI) data under naturalistic stimuli, more naturalistic fMRI datasets are still urgently needed. We present here the Natural Object Dataset (NOD), a large-scale fMRI dataset containing responses to 57,120 naturalistic images from 30 participants. NOD strives for a balance between sampling variation between individuals and sampling variation between stimuli. This enables NOD to be utilized not only for determining whether an observation is generalizable across many individuals, but also for testing whether a response pattern is generalized to a variety of naturalistic stimuli. We anticipate that the NOD together with existing naturalistic neuroimaging datasets will serve as a new impetus for our understanding of the visual processing of naturalistic stimuli.

Individual-specific memory reinstatement patterns within human face-selective cortex

Humans have the remarkable ability to vividly retrieve sensory details of past events. According to the theory of sensory reinstatement, during remembering, brain regions involved in the sensory processing of prior events are reactivated to support this perception of the past. Recently, several studies have emphasized potential transformations in the spatial organization of reinstated activity patterns. In particular, studies of scene stimuli suggest a clear anterior shift in the location of retrieval activations compared with those during perception. However, it is not clear that such transformations occur universally, with evidence lacking for other important stimulus categories, particularly faces. Critical to addressing these questions, and to studies of reinstatement more broadly, is the growing importance of considering meaningful variations in the organization of sensory systems across individuals. Therefore, we conducted a multi-session neuroimaging study to first carefully map individual participants face-selective regions within ventral temporal cortex (VTC), followed by a second session to examine the correspondence of activity patterns during face memory encoding and retrieval. Our results showed distinct configurations of face-selective regions within the VTC across individuals. While a significant degree of overlap was observed between face perception and memory encoding, memory retrieval engagement exhibited a more selective and constricted reinstatement pattern within these regions. Importantly, these activity patterns were consistently tied to individual-specific neural substrates, but did not show any consistent direction of spatial transformation (e.g., anteriorization). To provide further insight to these findings, we also report on unique human intracranial recordings from VTC under the same experimental conditions. Our findings highlight the importance of considering individual variations in functional neuroanatomy in the context of assessing the nature of cortical reinstatement. Consideration of such factors will be important for establishing general principles shaping the neural transformations that occur from perception to memory.

The effect of familiarity on behavioral oscillations in face perception

Studies on behavioral oscillations demonstrate that visual sensitivity fluctuates over time and visual processing varies periodically, mirroring neural oscillations at the same frequencies. Do these behavioral oscillations reflect fixed and relatively automatic sensory sampling, or top-down processes such as attention or predictive coding? To disentangle these theories, the current study used a dual-target rapid serial visual presentation paradigm, where participants indicated the gender of a face target embedded in streams of distractors presented at 30 Hz. On critical trials, two identical targets were presented with varied stimulus onset asynchrony from 200 to 833 ms. The target was either familiar or unfamiliar faces, divided into different blocks. We found a 4.6 Hz phase-coherent fluctuation in gender discrimination performance across both trial types, consistent with previous reports. In addition, however, we found an effect at the alpha frequency, with behavioral oscillations in the familiar blocks characterized by a faster high-alpha peak than for the unfamiliar face blocks. These results are consistent with the combination of both a relatively stable modulation in the theta band and faster modulation of the alpha oscillations. Therefore, the overall pattern of perceptual sampling in visual perception may depend, at least in part, on task demands. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on 16/08/2022. The protocol, as accepted by the journal, can be found at: https://doi.org/10.17605/OSF.IO/A98UF .

Spikiness and animacy as potential organizing principles of human ventral visual cortex

Considerable research has been devoted to understanding the fundamental organizing principles of the ventral visual pathway. A recent study revealed a series of 3-4 topographical maps arranged along the macaque inferotemporal (IT) cortex. The maps articulated a two-dimensional space based on the spikiness and animacy of visual objects, with "inanimate-spiky" and "inanimate-stubby" regions of the maps constituting two previously unidentified cortical networks. The goal of our study was to determine whether a similar functional organization might exist in human IT. To address this question, we presented the same object stimuli and images from "classic" object categories (bodies, faces, houses) to humans while recording fMRI activity at 7 Tesla. Contrasts designed to reveal the spikiness-animacy object space evoked extensive significant activation across human IT. However, unlike the macaque, we did not observe a clear sequence of complete maps, and selectivity for the spikiness-animacy space was deeply and mutually entangled with category-selectivity. Instead, we observed multiple new stimulus preferences in category-selective regions, including functional sub-structure related to object spikiness in scene-selective cortex. Taken together, these findings highlight spikiness as a promising organizing principle of human IT and provide new insights into the role of category-selective regions in visual object processing.

Tasks and their role in visual neuroscience

Vision is widely used as a model system to gain insights into how sensory inputs are processed and interpreted by the brain. Historically, careful quantification and control of visual stimuli have served as the backbone of visual neuroscience. There has been less emphasis, however, on how an observer's task influences the processing of sensory inputs. Motivated by diverse observations of task-dependent activity in the visual system, we propose a framework for thinking about tasks, their role in sensory processing, and how we might formally incorporate tasks into our models of vision.

A Texture Statistics Encoding Model Reveals Hierarchical Feature Selectivity across Human Visual Cortex

Midlevel features, such as contour and texture, provide a computational link between low- and high-level visual representations. Although the nature of midlevel representations in the brain is not fully understood, past work has suggested a texture statistics model, called the P-S model (Portilla and Simoncelli, 2000), is a candidate for predicting neural responses in areas V1-V4 as well as human behavioral data. However, it is not currently known how well this model accounts for the responses of higher visual cortex to natural scene images. To examine this, we constructed single-voxel encoding models based on P-S statistics and fit the models to fMRI data from human subjects (both sexes) from the Natural Scenes Dataset (Allen et al., 2022). We demonstrate that the texture statistics encoding model can predict the held-out responses of individual voxels in early retinotopic areas and higher-level category-selective areas. The ability of the model to reliably predict signal in higher visual cortex suggests that the representation of texture statistics features is widespread throughout the brain. Furthermore, using variance partitioning analyses, we identify which features are most uniquely predictive of brain responses and show that the contributions of higher-order texture features increase from early areas to higher areas on the ventral and lateral surfaces. We also demonstrate that patterns of sensitivity to texture statistics can be used to recover broad organizational axes within visual cortex, including dimensions that capture semantic image content. These results provide a key step forward in characterizing how midlevel feature representations emerge hierarchically across the visual system.SIGNIFICANCE STATEMENT Intermediate visual features, like texture, play an important role in cortical computations and may contribute to tasks like object and scene recognition. Here, we used a texture model proposed in past work to construct encoding models that predict the responses of neural populations in human visual cortex (measured with fMRI) to natural scene stimuli. We show that responses of neural populations at multiple levels of the visual system can be predicted by this model, and that the model is able to reveal an increase in the complexity of feature representations from early retinotopic cortex to higher areas of ventral and lateral visual cortex. These results support the idea that texture-like representations may play a broad underlying role in visual processing.

A Unifying Principle for the Functional Organization of Visual Cortex

A key feature of many cortical systems is functional organization: the arrangement of neurons with specific functional properties in characteristic spatial patterns across the cortical surface. However, the principles underlying the emergence and utility of functional organization are poorly understood. Here we develop the Topographic Deep Artificial Neural Network (TDANN), the first unified model to accurately predict the functional organization of multiple cortical areas in the primate visual system. We analyze the key factors responsible for the TDANN's success and find that it strikes a balance between two specific objectives: achieving a task-general sensory representation that is self-supervised, and maximizing the smoothness of responses across the cortical sheet according to a metric that scales relative to cortical surface area. In turn, the representations learned by the TDANN are lower dimensional and more brain-like than those in models that lack a spatial smoothness constraint. Finally, we provide evidence that the TDANN's functional organization balances performance with inter-area connection length, and use the resulting models for a proof-of-principle optimization of cortical prosthetic design. Our results thus offer a unified principle for understanding functional organization and a novel view of the functional role of the visual system in particular.

Multimodal deep neural decoding reveals highly resolved spatiotemporal profile of visual object representation in humans

Perception and categorization of objects in a visual scene are essential to grasp the surrounding situation. Recently, neural decoding schemes, such as machine learning in functional magnetic resonance imaging (fMRI), has been employed to elucidate the underlying neural mechanisms. However, it remains unclear as to how spatially distributed brain regions temporally represent visual object categories and sub-categories. One promising strategy to address this issue is neural decoding with concurrently obtained neural response data of high spatial and temporal resolution. In this study, we explored the spatial and temporal organization of visual object representations using concurrent fMRI and electroencephalography (EEG), combined with neural decoding using deep neural networks (DNNs). We hypothesized that neural decoding by multimodal neural data with DNN would show high classification performance in visual object categorization (faces or non-face objects) and sub-categorization within faces and objects. Visualization of the fMRI DNN was more sensitive than that in the univariate approach and revealed that visual categorization occurred in brain-wide regions. Interestingly, the EEG DNN valued the earlier phase of neural responses for categorization and the later phase of neural responses for sub-categorization. Combination of the two DNNs improved the classification performance for both categorization and sub-categorization compared with fMRI DNN or EEG DNN alone. These deep learning-based results demonstrate a categorization principle in which visual objects are represented in a spatially organized and coarse-to-fine manner, and provide strong evidence of the ability of multimodal deep learning to uncover spatiotemporal neural machinery in sensory processing.

Backward masking reveals coarse-to-fine dynamics in human V1

Natural images exhibit luminance variations aligned across a broad spectrum of spatial frequencies (SFs). It has been proposed that, at early stages of processing, the coarse signals carried by the low SF (LSF) of the visual input are sent rapidly from primary visual cortex (V1) to ventral, dorsal and frontal regions to form a coarse representation of the input, which is later sent back to V1 to guide the processing of fine-grained high SFs (i.e., HSF). We used functional resonance imaging (fMRI) to investigate the role of human V1 in the coarse-to-fine integration of visual input. We disrupted the processing of the coarse and fine content of full-spectrum human face stimuli via backward masking of selective SF ranges (LSFs: <1.75cpd and HSFs: >1.75cpd) at specific times (50, 83, 100 or 150ms). In line with coarse-to-fine proposals, we found that (1) the selective masking of stimulus LSF disrupted V1 activity in the earliest time window, and progressively decreased in influence, while (2) an opposite trend was observed for the masking of stimulus' HSF. This pattern of activity was found in V1, as well as in ventral (i.e. the Fusiform Face area, FFA), dorsal and orbitofrontal regions. We additionally presented subjects with contrast negated stimuli. While contrast negation significantly reduced response amplitudes in the FFA, as well as coupling between FFA and V1, coarse-to-fine dynamics were not affected by this manipulation. The fact that V1 response dynamics to strictly identical stimulus sets differed depending on the masked scale adds to growing evidence that V1 role goes beyond the early and quasi-passive transmission of visual information to the rest of the brain. It instead indicates that V1 may yield a 'spatially registered common forum' or 'blackboard' that integrates top-down inferences with incoming visual signals through its recurrent interaction with high-level regions located in the inferotemporal, dorsal and frontal regions.

Stacked regressions and structured variance partitioning for interpretable brain maps

Relating brain activity associated with a complex stimulus to different properties of that stimulus is a powerful approach for constructing functional brain maps. However, when stimuli are naturalistic, their properties are often correlated (e.g., visual and semantic features of natural images, or different layers of a convolutional neural network that are used as features of images). Correlated properties can act as confounders for each other and complicate the interpretability of brain maps, and can impact the robustness of statistical estimators. Here, we present an approach for brain mapping based on two proposed methods: stacking different encoding models and structured variance partitioning. Our stacking algorithm combines encoding models that each use as input a feature space that describes a different stimulus attribute. The algorithm learns to predict the activity of a voxel as a linear combination of the outputs of different encoding models. We show that the resulting combined model can predict held-out brain activity better or at least as well as the individual encoding models. Further, the weights of the linear combination are readily interpretable; they show the importance of each feature space for predicting a voxel. We then build on our stacking models to introduce structured variance partitioning, a new type of variance partitioning that takes into account the known relationships between features. Our approach constrains the size of the hypothesis space and allows us to ask targeted questions about the similarity between feature spaces and brain regions even in the presence of correlations between the feature spaces. We validate our approach in simulation, showcase its brain mapping potential on fMRI data, and release a Python package. Our methods can be useful for researchers interested in aligning brain activity with different layers of a neural network, or with other types of correlated feature spaces.

The transition from vision to language: distinct patterns of functional connectivity for sub-regions of the visual word form area

Reading entails transforming visual symbols to sound and meaning. This process depends on specialized circuitry in the visual cortex, the Visual Word Form Area (VWFA). Recent findings suggest that this word-selective cortex comprises at least two distinct subregions: the more posterior VWFA-1 is sensitive to visual features, while the more anterior VWFA-2 processes higher level language information. Here, we explore whether these two subregions exhibit different patterns of functional connectivity, and whether these patterns have relevance for reading development. We address these questions using two complementary datasets: Using the Natural Scenes Datasets (NSD; Allen et al, 2022) we identify word-selective responses in high-quality 7T individual adult data (N=8; 6 females), and investigate functional connectivity patterns of VWFA-1 and VWFA-2 at the individual level. We then turn to the Healthy Brain Network (HBN; Alexander et al., 2017) database to assess whether these patterns a) replicate in a large developmental sample (N=224; 98 females, age 5-21y), and b) are related to reading development. In both datasets, we find that VWFA-1 is more strongly correlated with bilateral visual regions including ventral occipitotemporal cortex and posterior parietal cortex. In contrast, VWFA-2 is more strongly correlated with language regions in the frontal and lateral parietal lobes, particularly bilateral inferior frontal gyrus (IFG). Critically, these patterns do not generalize to adjacent face-selective regions, suggesting a unique relationship between VWFA-2 and the frontal language network. While connectivity patterns increased with age, no correlations were observed between functional connectivity and reading ability. Together, our findings support the distinction between subregions of the VWFA, and portray the functional connectivity patterns of the reading circuitry as an intrinsic stable property of the brain.

The contribution of dynamics to macaque body and face patch responses

Previous functional imaging studies demonstrated body-selective patches in the primate visual temporal cortex, comparing activations to static bodies and static images of other categories. However, the use of static instead of dynamic displays of moving bodies may have underestimated the extent of the body patch network. Indeed, body dynamics provide information about action and emotion and may be processed in patches not activated by static images. Thus, to map with fMRI the full extent of the macaque body patch system in the visual temporal cortex, we employed dynamic displays of natural-acting monkey bodies, dynamic monkey faces, objects, and scrambled versions of these videos, all presented during fixation. We found nine body patches in the visual temporal cortex, starting posteriorly in the superior temporal sulcus (STS) and ending anteriorly in the temporal pole. Unlike for static images, body patches were present consistently in both the lower and upper banks of the STS. Overall, body patches showed a higher activation by dynamic displays than by matched static images, which, for identical stimulus displays, was less the case for the neighboring face patches. These data provide the groundwork for future single-unit recording studies to reveal the spatiotemporal features the neurons of these body patches encode. These fMRI findings suggest that dynamics have a stronger contribution to population responses in body than face patches.

Dyslexia: Causes and Concomitant Impairments

In recent decades, theories have been presented to explain the nature of dyslexia, but the causes of dyslexia remained unclear. Although the investigation of the causes of dyslexia presupposes a clear understanding of the concept of cause, such an understanding is missing. The present paper proposes the absence of at least one necessary condition or the absence of all sufficient conditions as causes for impaired reading. The causes of impaired reading include: an incorrect fixation location, too short a fixation time, the attempt to recognize too many letters simultaneously, too large saccade amplitudes, and too short verbal reaction times. It is assumed that a longer required fixation time in dyslexic readers results from a functional impairment of areas V1, V2, and V3 that require more time to complete temporal summation. These areas and areas that receive input from them, such as the fusiform gyrus, are assumed to be impaired in their ability to simultaneously process a string of letters. When these impairments are compensated by a new reading strategy, reading ability improves immediately.

White matter connections of high-level visual areas predict cytoarchitecture better than category-selectivity in childhood, but not adulthood

Ventral temporal cortex (VTC) consists of high-level visual regions that are arranged in consistent anatomical locations across individuals. This consistency has led to several hypotheses about the factors that constrain the functional organization of VTC. A prevailing theory is that white matter connections influence the organization of VTC, however, the nature of this constraint is unclear. Here, we test 2 hypotheses: (1) white matter tracts are specific for each category or (2) white matter tracts are specific to cytoarchitectonic areas of VTC. To test these hypotheses, we used diffusion magnetic resonance imaging to identify white matter tracts and functional magnetic resonance imaging to identify category-selective regions in VTC in children and adults. We find that in childhood, white matter connections are linked to cytoarchitecture rather than category-selectivity. In adulthood, however, white matter connections are linked to both cytoarchitecture and category-selectivity. These results suggest a rethinking of the view that category-selective regions in VTC have category-specific white matter connections early in development. Instead, these findings suggest that the neural hardware underlying the processing of categorical stimuli may be more domain-general than previously thought, particularly in childhood.

Reading the mind in the nose

Humans infer mental states and traits from faces and their expressions. Previous research focused on the role of eyes and mouths in this process, even though most observers fixate somewhere in between. Here, we report that ratings of the nose region are surprisingly consistent with those for the full face and even with subjective feelings of the nose bearer. We propose the nose as central to faces and their perception.

Selectivity for food in human ventral visual cortex

Visual cortex contains regions of selectivity for domains of ecological importance. Food is an evolutionarily critical category whose visual heterogeneity may make the identification of selectivity more challenging. We investigate neural responsiveness to food using natural images combined with large-scale human fMRI. Leveraging the improved sensitivity of modern designs and statistical analyses, we identify two food-selective regions in the ventral visual cortex. Our results are robust across 8 subjects from the Natural Scenes Dataset (NSD), multiple independent image sets and multiple analysis methods. We then test our findings of food selectivity in an fMRI "localizer" using grayscale food images. These independent results confirm the existence of food selectivity in ventral visual cortex and help illuminate why earlier studies may have failed to do so. Our identification of food-selective regions stands alongside prior findings of functional selectivity and adds to our understanding of the organization of knowledge within the human visual system.

Quick, eyes! Isolated upper face regions but not artificial features elicit rapid saccades

Human faces elicit faster saccades than objects or animals, resonating with the great importance of faces for our species. The underlying mechanisms are largely unclear. Here, we test two hypotheses based on previous findings. First, ultra-rapid saccades toward faces may not depend on the presence of the whole face, but the upper face region containing the eye region. Second, ultra-rapid saccades toward faces (and possibly face parts) may emerge from our extensive experience with this stimulus and thus extend to glasses and masks - artificial features frequently encountered as part of a face. To test these hypotheses, we asked 43 participants to complete a saccadic choice task, which contrasted images of whole, upper and lower faces, face masks, and glasses with car images. The resulting data confirmed ultra-rapid saccades for isolated upper face regions, but not for artificial facial features.

Functionally and structurally distinct fusiform face area(s) in over 1000 participants

The fusiform face area (FFA) is a widely studied region causally involved in face perception. Even though cognitive neuroscientists have been studying the FFA for over two decades, answers to foundational questions regarding the function, architecture, and connectivity of the FFA from a large (N>1000) group of participants are still lacking. To fill this gap in knowledge, we quantified these multimodal features of fusiform face-selective regions in 1053 participants in the Human Connectome Project. After manually defining over 4,000 fusiform face-selective regions, we report five main findings. First, 68.76% of hemispheres have two cortically separate regions (pFus-faces/FFA-1 and mFus-faces/FFA-2). Second, in 26.69% of hemispheres, pFus-faces/FFA-1 and mFus-faces/FFA-2 are spatially contiguous, yet are distinct based on functional, architectural, and connectivity metrics. Third, pFus-faces/FFA-1 is more face-selective than mFus-faces/FFA-2, and the two regions have distinct functional connectivity fingerprints. Fourth, pFus-faces/FFA-1 is cortically thinner and more heavily myelinated than mFus-faces/FFA-2. Fifth, face-selective patterns and functional connectivity fingerprints of each region are more similar in monozygotic than dizygotic twins and more so than architectural gradients. As we share our areal definitions with the field, future studies can explore how structural and functional features of these regions will inform theories regarding how visual categories are represented in the brain.

Functionally and structurally distinct fusiform face area(s) in over 1000 participants

The fusiform face area (FFA) is a widely studied region causally involved in face perception. Even though cognitive neuroscientists have been studying the FFA for over two decades, answers to foundational questions regarding the function, architecture, and connectivity of the FFA from a large (N>1000) group of participants are still lacking. To fill this gap in knowledge, we quantified these multimodal features of fusiform face-selective regions in 1053 participants in the Human Connectome Project. After manually defining over 4,000 fusiform face-selective regions, we report five main findings. First, 68.76% of hemispheres have two cortically separate regions (pFus-faces/FFA-1 and mFus-faces/FFA-2). Second, in 26.69% of hemispheres, pFus-faces/FFA-1 and mFus-faces/FFA-2 are spatially contiguous, yet are distinct based on functional, architectural, and connectivity metrics. Third, pFus-faces/FFA-1 is more face-selective than mFus-faces/FFA-2, and the two regions have distinct functional connectivity fingerprints. Fourth, pFus-faces/FFA-1 is cortically thinner and more heavily myelinated than mFus-faces/FFA-2. Fifth, face-selective patterns and functional connectivity fingerprints of each region are more similar in monozygotic than dizygotic twins and more so than architectural gradients. As we share our areal definitions with the field, future studies can explore how structural and functional features of these regions will inform theories regarding how visual categories are represented in the brain.

Invariance of object detection in untrained deep neural networks

The ability to perceive visual objects with various types of transformations, such as rotation, translation, and scaling, is crucial for consistent object recognition. In machine learning, invariant object detection for a network is often implemented by augmentation with a massive number of training images, but the mechanism of invariant object detection in biological brains—how invariance arises initially and whether it requires visual experience—remains elusive. Here, using a model neural network of the hierarchical visual pathway of the brain, we show that invariance of object detection can emerge spontaneously in the complete absence of learning. First, we found that units selective to a particular object class arise in randomly initialized networks even before visual training. Intriguingly, these units show robust tuning to images of each object class under a wide range of image transformation types, such as viewpoint rotation. We confirmed that this “innate” invariance of object selectivity enables untrained networks to perform an object-detection task robustly, even with images that have been significantly modulated. Our computational model predicts that invariant object tuning originates from combinations of non-invariant units via random feedforward projections, and we confirmed that the predicted profile of feedforward projections is observed in untrained networks. Our results suggest that invariance of object detection is an innate characteristic that can emerge spontaneously in random feedforward networks.

Shaping overnight consolidation via slow-oscillation closed-loop targeted memory reactivation

Sleep constitutes a privileged state for new memories to reactivate and consolidate. Previous work has demonstrated that consolidation can be bolstered experimentally either via delivery of reminder cues (targeted memory reactivation [TMR]) or via noninvasive brain stimulation geared toward enhancing endogenous sleep rhythms. Here, we combined both approaches, controlling the timing of TMR cues with respect to ongoing slow-oscillation (SO) phases. Prior to sleep, participants learned associations between unique words and a set of repeating images (e.g., car) while hearing a prototypical image sound (e.g., engine starting). Memory performance on an immediate test vs. a test the next morning quantified overnight memory consolidation. Importantly, two image sounds were designated as TMR cues, with one cue delivered at SO UP states and the other delivered at SO DOWN states. A novel sound was used as a TMR control condition. Behavioral results revealed a significant reduction of overnight forgetting for words associated with UP-state TMR compared with words associated with DOWN-state TMR. Electrophysiological results showed that UP-state cueing led to enhancement of the ongoing UP state and was followed by greater spindle power than DOWN-state cueing. Moreover, UP-state (and not DOWN-state) cueing led to reinstatement of target image representations. Together, these results unveil the behavioral and mechanistic effects of delivering reminder cues at specific phases of endogenous sleep rhythms and mark an important step for the endeavor to experimentally modulate memories during sleep.

Scene-selective brain regions respond to embedded objects of a scene

Objects are fundamental to scene understanding. Scenes are defined by embedded objects and how we interact with them. Paradoxically, scene processing in the brain is typically discussed in contrast to object processing. Using the BOLD5000 dataset (Chang et al., 2019), we examined whether objects within a scene predicted the neural representation of scenes, as measured by functional magnetic resonance imaging in humans. Stimuli included 1,179 unique scenes across 18 semantic categories. Object composition of scenes were compared across scene exemplars in different semantic scene categories, and separately, in exemplars of the same scene category. Neural representations in scene- and object-preferring brain regions were significantly related to which objects were in a scene, with the effect at times stronger in the scene-preferring regions. The object model accounted for more variance when comparing scenes within the same semantic category to scenes from different categories. Here, we demonstrate the function of scene-preferring regions includes the processing of objects. This suggests visual processing regions may be better characterized by the processes, which are engaged when interacting with the stimulus kind, such as processing groups of objects in scenes, or processing a single object in our foreground, rather than the stimulus kind itself.

Evidence for a theta-band behavioural oscillation in rapid face detection

Theories of rhythmic perception propose that perceptual sampling operates in a periodic way, with alternating moments of high and low responsiveness to sensory inputs. This rhythmic sampling is linked to neural oscillations and thought to produce fluctuations in behavioural outcomes. Previous studies have revealed theta- and alpha-band behavioural oscillations in low-level visual tasks and object categorization. However, less is known about fluctuations in face perception, for which the human brain has developed a highly specialized network. To investigate this, we ran an online study (N = 179) incorporating the dense sampling technique with a dual-target rapid serial visual presentation (RSVP) paradigm. In each trial, a stream of object images was presented at 30 Hz and participants were tasked with detecting whether or not there was a face image in the sequence. On some trials, one or two (identical) face images (the target) were embedded in each stream. On dual-target trials, the targets were separated by an interstimulus interval (ISI) that varied between 0 to 633 ms. The task was to indicate the presence of the target and its gender if present. Performance varied as a function of ISI, with a significant behavioural oscillation in the face detection task at 7.5 Hz, driven mainly by the male target faces. This finding is consistent with a high theta-band-based fluctuation in visual processing. Such fluctuations might reflect rhythmic attentional sampling or, alternatively, feedback loops involved in updating top-down predictions.

Behavioral and neural markers of visual configural processing in social scene perception

Research on face perception has revealed highly specialized visual mechanisms such as configural processing, and provided markers of interindividual differences -including disease risks and alterations- in visuo-perceptual abilities that traffic in social cognition. Is face perception unique in degree or kind of mechanisms, and in its relevance for social cognition? Combining functional MRI and behavioral methods, we address the processing of an uncharted class of socially relevant stimuli: minimal social scenes involving configurations of two bodies spatially close and face-to-face as if interacting (hereafter, facing dyads). We report category-specific activity for facing (vs. non-facing) dyads in visual cortex. That activity shows face-like signatures of configural processing -i.e., stronger response to facing (vs. non-facing) dyads, and greater susceptibility to stimulus inversion for facing (vs. non-facing) dyads-, and is predicted by performance-based measures of configural processing in visual perception of body dyads. Moreover, we observe that the individual performance in body-dyad perception is reliable, stable-over-time and correlated with the individual social sensitivity, coarsely captured by the Autism-Spectrum Quotient. Further analyses clarify the relationship between single-body and body-dyad perception. We propose that facing dyads are processed through highly specialized mechanisms -and brain areas-, analogously to other biologically and socially relevant stimuli such as faces. Like face perception, facing-dyad perception can reveal basic (visual) processes that lay the foundations for understanding others, their relationships and interactions.