Development of visual category selectivity in ventral visual cortex does not require visual experience

Distinct Yet Proximal Face- and Body-Selective Brain Regions Enable Clutter-Tolerant Representations of the Face, Body, and Whole Person

Faces and bodies are processed in separate but adjacent regions in the primate visual cortex. Yet, the functional significance of dividing the whole person into areas dedicated to its face and body components and their neighboring locations remains unknown. Here we hypothesized that this separation and proximity together with a normalization mechanism generate clutter-tolerant representations of the face, body, and whole person when presented in complex multi-category scenes. To test this hypothesis, we conducted a fMRI study, presenting images of a person within a multi-category scene to human male and female participants and assessed the contribution of each component to the response to the scene. Our results revealed a clutter-tolerant representation of the whole person in areas selective for both faces and bodies, typically located at the border between the two category-selective regions. Regions exclusively selective for faces or bodies demonstrated clutter-tolerant representations of their preferred category, corroborating earlier findings. Thus, the adjacent locations of face- and body-selective areas enable a hardwired machinery for decluttering of the whole person, without the need for a dedicated population of person-selective neurons. This distinct yet proximal functional organization of category-selective brain regions enhances the representation of the socially significant whole person, along with its face and body components, within multi-category scenes.

Neural correlates of individual facial recognition in a social wasp

Individual recognition is critical for social behavior across species. Whether recognition is mediated by circuits specialized for social information processing has been a matter of debate. Here we examine the neurobiological underpinning of individual visual facial recognition in Polistes fuscatus paper wasps. Front-facing images of conspecific wasps broadly increase activity across many brain regions relative to other stimuli. Notably, we identify a localized subpopulation of neurons in the protocerebrum which show specialized selectivity for front-facing wasp images, which we term wasp cells. These wasp cells encode information regarding the facial patterns, with ensemble activity correlating with facial identity. Wasp cells are strikingly analogous to face cells in primates, indicating that specialized circuits are likely an adaptive feature of neural architecture to support visual recognition.

Developing cortex is functionally pluripotent: Evidence from blindness

How rigidly does innate architecture constrain function of developing cortex? What is the contribution of early experience? We review insights into these questions from visual cortex function in people born blind. In blindness, occipital cortices are active during auditory and tactile tasks. What 'cross-modal' plasticity tells us about cortical flexibility is debated. On the one hand, visual networks of blind people respond to higher cognitive information, such as sentence grammar, suggesting drastic repurposing. On the other, in line with 'metamodal' accounts, sighted and blind populations show shared domain preferences in ventral occipito-temporal cortex (vOTC), suggesting visual areas switch input modality but perform the same or similar perceptual functions (e.g., face recognition) in blindness. Here we bring these disparate literatures together, reviewing and synthesizing evidence that speaks to whether visual cortices have similar or different functions in blind and sighted people. Together, the evidence suggests that in blindness, visual cortices are incorporated into higher-cognitive (e.g., fronto-parietal) networks, which are a major source long-range input to the visual system. We propose the connectivity-constrained experience-dependent account. Functional development is constrained by innate anatomical connectivity, experience and behavioral needs. Infant cortex is pluripotent, the same anatomical constraints develop into different functional outcomes.

Mechanisms underlying category learning in the human ventral occipito-temporal cortex

The human ventral occipito-temporal cortex (VOTC) has evolved into specialized regions that process specific categories, such as words, tools, and animals. The formation of these areas is driven by bottom-up visual and top-down nonvisual experiences. However, the specific mechanisms through which top-down nonvisual experiences modulate category-specific regions in the VOTC are still unknown. To address this question, we conducted a study in which participants were trained for approximately 13 h to associate three sets of novel meaningless figures with different top-down nonvisual features: the wordlike category with word features, the non-wordlike category with nonword features, and the visual familiarity condition with no nonvisual features. Pre- and post-training functional MRI (fMRI) experiments were used to measure brain activity during stimulus presentation. Our results revealed that training induced a categorical preference for the two training categories within the VOTC. Moreover, the locations of two training category-specific regions exhibited a notable overlap. Remarkably, within the overlapping category-specific region, training resulted in a dissociation in activation intensity and pattern between the two training categories. These findings provide important insights into how different nonvisual categorical information is encoded in the human VOTC.

Neural specialization for 'visual' concepts emerges in the absence of vision

Vision provides a key source of information about many concepts, including 'living things' (e.g., tiger) and visual events (e.g., sparkle). According to a prominent theoretical framework, neural specialization for different conceptual categories is shaped by sensory features, e.g., living things are neurally dissociable from navigable places because living things concepts depend more on visual features. We tested this framework by comparing the neural basis of 'visual' concepts across sighted (n=22) and congenitally blind (n=21) adults. Participants judged the similarity of words varying in their reliance on vision while undergoing fMRI. We compared neural responses to living things nouns (birds, mammals) and place nouns (natural, manmade). In addition, we compared visual event verbs (e.g., 'sparkle') to non-visual events (sound emission, hand motion, mouth motion). People born blind exhibited distinctive univariate and multivariate responses to living things in a temporo-parietal semantic network activated by nouns, including the precuneus (PC). To our knowledge, this is the first demonstration that neural selectivity for living things does not require vision. We additionally observed preserved neural signatures of 'visual' light events in the left middle temporal gyrus (LMTG+). Across a wide range of semantic types, neural representations of sensory concepts develop independent of sensory experience.

A unified model for cross-modal plasticity and skill acquisition

Historically, cross-modal plasticity following early blindness has been largely studied in the context of visual deprivation. However, more recently, there has been a shift in focus towards understanding cross-modal plasticity from the perspective of skill acquisition: the striking plasticity observed in early blind individuals reflects the extraordinary perceptual and cognitive challenges they solve. Here, inspired by two seminal papers on skill learning (the "cortical recycling" theory) and cross-modal plasticity (the "metamodal" hypothesis) respectively, we present a unified hypothesis of cortical specialization that describes how shared functional, algorithmic, and structural constraints might mediate both types of plasticity.

Longitudinal stability of individual brain plasticity patterns in blindness

The primary visual cortex (V1) in individuals born blind is engaged in a wide spectrum of tasks and sensory modalities, including audition, touch, language, and memory. This widespread involvement raises questions regarding the constancy of its role and whether it might exhibit flexibility in its function over time, connecting to diverse network functions in response to task-specific demands. This would suggest that reorganized V1 takes on a role similar to cognitive multiple-demand system regions. Alternatively, it is possible that the varying patterns of plasticity observed in the blind V1 can be attributed to individual factors, whereby different blind individuals recruit V1 for different functions, highlighting the immense idiosyncrasy of plasticity. In support of this second account, we have recently shown that V1 functional connectivity varies greatly across blind individuals. But do these represent stable individual patterns of plasticity or merely instantaneous changes, for a multiple-demand system now inhabiting V1? Here we tested if individual connectivity patterns from the visual cortex of blind individuals are stable over time. We show that over two years, fMRI functional connectivity from the primary visual cortex is unique and highly stable in a small sample of repeatedly sampled congenitally blind individuals. Further, using multivoxel pattern analysis, we demonstrate that the unique reorganization patterns of these individuals allow decoding of participant identity. Together with recent evidence for substantial individual differences in visual cortex connectivity, this indicates there may be a consistent role for the visual cortex in blindness, which may differ for each individual. Further, it suggests that the variability in visual reorganization in blindness across individuals could be used to seek stable neuromarkers for sight rehabilitation and assistive approaches.

The development of upright face perception depends on evolved orientation-specific mechanisms and experience

Here we examine whether our impressive ability to perceive upright faces arises from evolved orientation-specific mechanisms, our extensive experience with upright faces, or both factors. To do so, we tested Claudio, a man with a congenital joint disorder causing his head to be rotated back so that it is positioned between his shoulder blades. As a result, Claudio has seen more faces reversed in orientation to his own face than matched to it. Controls exhibited large inversion effects on all tasks, but Claudio performed similarly with upright and inverted faces in both detection and identity-matching tasks, indicating these abilities are the product of evolved mechanisms and experience. In contrast, he showed clear upright superiority when detecting "Thatcherized" faces (faces with vertically flipped features), suggesting experience plays a greater role in this judgment. Together, these findings indicate that both evolved orientation-specific mechanisms and experience contribute to our proficiency with upright faces.

Modeling Biological Face Recognition with Deep Convolutional Neural Networks

Deep convolutional neural networks (DCNNs) have become the state-of-the-art computational models of biological object recognition. Their remarkable success has helped vision science break new ground, and recent efforts have started to transfer this achievement to research on biological face recognition. In this regard, face detection can be investigated by comparing face-selective biological neurons and brain areas to artificial neurons and model layers. Similarly, face identification can be examined by comparing in vivo and in silico multidimensional "face spaces." In this review, we summarize the first studies that use DCNNs to model biological face recognition. On the basis of a broad spectrum of behavioral and computational evidence, we conclude that DCNNs are useful models that closely resemble the general hierarchical organization of face recognition in the ventral visual pathway and the core face network. In two exemplary spotlights, we emphasize the unique scientific contributions of these models. First, studies on face detection in DCNNs indicate that elementary face selectivity emerges automatically through feedforward processing even in the absence of visual experience. Second, studies on face identification in DCNNs suggest that identity-specific experience and generative mechanisms facilitate this particular challenge. Taken together, as this novel modeling approach enables close control of predisposition (i.e., architecture) and experience (i.e., training data), it may be suited to inform long-standing debates on the substrates of biological face recognition.

Multisensory perception constrains the formation of object categories: a review of evidence from sensory-driven and predictive processes on categorical decisions

Although object categorization is a fundamental cognitive ability, it is also a complex process going beyond the perception and organization of sensory stimulation. Here we review existing evidence about how the human brain acquires and organizes multisensory inputs into object representations that may lead to conceptual knowledge in memory. We first focus on evidence for two processes on object perception, multisensory integration of redundant information (e.g. seeing and feeling a shape) and crossmodal, statistical learning of complementary information (e.g. the 'moo' sound of a cow and its visual shape). For both processes, the importance attributed to each sensory input in constructing a multisensory representation of an object depends on the working range of the specific sensory modality, the relative reliability or distinctiveness of the encoded information and top-down predictions. Moreover, apart from sensory-driven influences on perception, the acquisition of featural information across modalities can affect semantic memory and, in turn, influence category decisions. In sum, we argue that both multisensory processes independently constrain the formation of object categories across the lifespan, possibly through early and late integration mechanisms, respectively, to allow us to efficiently achieve the everyday, but remarkable, ability of recognizing objects. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Loss of action-related function and connectivity in the blind extrastriate body area

The Extrastriate Body Area (EBA) participates in the visual perception and motor actions of body parts. We recently showed that EBA’s perceptual function develops independently of visual experience, responding to stimuli with body-part information in a supramodal fashion. However, it is still unclear if the EBA similarly maintains its action-related function. Here, we used fMRI to study motor-evoked responses and connectivity patterns in the congenitally blind brain. We found that, unlike the case of perception, EBA does not develop an action-related response without visual experience. In addition, we show that congenital blindness alters EBA’s connectivity profile in a counterintuitive way—functional connectivity with sensorimotor cortices dramatically decreases, whereas connectivity with perception-related visual occipital cortices remains high. To the best of our knowledge, we show for the first time that action-related functions and connectivity in the visual cortex could be contingent on visuomotor experience. We further discuss the role of the EBA within the context of visuomotor control and predictive coding theory.

Pulvinar Response Profiles and Connectivity Patterns to Object Domains

Distributed cortical regions show differential responses to visual objects belonging to different domains varying by animacy (e.g., animals vs tools), yet it remains unclear whether this is an organization principle also applying to the subcortical structures. Combining multiple fMRI activation experiments (two main experiments and six validation datasets; 12 females and 9 males in the main Experiment 1; 10 females and 10 males in the main Experiment 2), resting-state functional connectivity, and task-based dynamic causal modeling analysis in human subjects, we found that visual processing of images of animals and tools elicited different patterns of response in the pulvinar, with robust left lateralization for tools, and distinct, bilateral (with rightward tendency) clusters for animals. Such domain-preferring activity distribution in the pulvinar was associated with the magnitude with which the voxels were intrinsically connected with the corresponding domain-preferring regions in the cortex. The pulvinar-to-right-amygdala path showed a one-way shortcut supporting the perception of animals, and the modulation connection from pulvinar to parietal showed an advantage to the perception of tools. These results incorporate the subcortical regions into the object processing network and highlight that domain organization appears to be an overarching principle across various processing stages in the brain.SIGNIFICANCE STATEMENT Viewing objects belonging to different domains elicited different cortical regions, but whether the domain organization applied to the subcortical structures (e.g., pulvinar) was unknown. Multiple fMRI activation experiments revealed that object pictures belonging to different domains elicited differential patterns of response in the pulvinar, with robust left lateralization for tool pictures, and distinct, bilateral (with rightward tendency) clusters for animals. Combining the resting-state functional connectivity and dynamic causal modeling analysis on task-based fMRI data, we found domain-preferring activity distribution in the pulvinar aligned with that in cortical regions. These results highlight the need for coherent visual theories that explain the mechanisms underlying the domain organization across various processing stages.

CNNs reveal the computational implausibility of the expertise hypothesis

Face perception has long served as a classic example of domain specificity of mind and brain. But an alternative "expertise" hypothesis holds that putatively face-specific mechanisms are actually domain-general, and can be recruited for the perception of other objects of expertise (e.g., cars for car experts). Here, we demonstrate the computational implausibility of this hypothesis: Neural network models optimized for generic object categorization provide a better foundation for expert fine-grained discrimination than do models optimized for face recognition.

Calibrating Vision: Concepts and Questions

The idea that visual coding and perception are shaped by experience and adjust to changes in the environment or the observer is universally recognized as a cornerstone of visual processing, yet the functions and processes mediating these calibrations remain in many ways poorly understood. In this article we review a number of facets and issues surrounding the general notion of calibration, with a focus on plasticity within the encoding and representational stages of visual processing. These include how many types of calibrations there are - and how we decide; how plasticity for encoding is intertwined with other principles of sensory coding; how it is instantiated at the level of the dynamic networks mediating vision; how it varies with development or between individuals; and the factors that may limit the form or degree of the adjustments. Our goal is to give a small glimpse of an enormous and fundamental dimension of vision, and to point to some of the unresolved questions in our understanding of how and why ongoing calibrations are a pervasive and essential element of vision.

Face shape processing via visual-to-auditory sensory substitution activates regions within the face processing networks in the absence of visual experience

Previous evidence suggests that visual experience is crucial for the emergence and tuning of the typical neural system for face recognition. To challenge this conclusion, we trained congenitally blind adults to recognize faces via visual-to-auditory sensory-substitution (SDD). Our results showed a preference for trained faces over other SSD-conveyed visual categories in the fusiform gyrus and in other known face-responsive-regions of the deprived ventral visual stream. We also observed a parametric modulation in the same cortical regions, for face orientation (upright vs. inverted) and face novelty (trained vs. untrained). Our results strengthen the conclusion that there is a predisposition for sensory-independent and computation-specific processing in specific cortical regions that can be retained in life-long sensory deprivation, independently of previous perceptual experience. They also highlight that if the right training is provided, such cortical preference maintains its tuning to what were considered visual-specific face features.

Impact of blindness onset on the representation of sound categories in occipital and temporal cortices

The ventral occipito-temporal cortex (VOTC) reliably encodes auditory categories in people born blind using a representational structure partially similar to the one found in vision (Mattioni et al.,2020). Here, using a combination of uni- and multivoxel analyses applied to fMRI data, we extend our previous findings, comprehensively investigating how early and late acquired blindness impact on the cortical regions coding for the deprived and the remaining senses. First, we show enhanced univariate response to sounds in part of the occipital cortex of both blind groups that is concomitant to reduced auditory responses in temporal regions. We then reveal that the representation of the sound categories in the occipital and temporal regions is more similar in blind subjects compared to sighted subjects. What could drive this enhanced similarity? The multivoxel encoding of the ‘human voice’ category that we observed in the temporal cortex of all sighted and blind groups is enhanced in occipital regions in blind groups , suggesting that the representation of vocal information is more similar between the occipital and temporal regions in blind compared to sighted individuals. We additionally show that blindness does not affect the encoding of the acoustic properties of our sounds (e.g. pitch, harmonicity) in occipital and in temporal regions but instead selectively alter the categorical coding of the voice category itself. These results suggest a functionally congruent interplay between the reorganization of occipital and temporal regions following visual deprivation, across the lifespan.

Rethinking the representation of sound

Blindness triggers a reorganization of the visual and auditory cortices in the brain.

Face identity coding in the deep neural network and primate brain

A central challenge in face perception research is to understand how neurons encode face identities. This challenge has not been met largely due to the lack of simultaneous access to the entire face processing neural network and the lack of a comprehensive multifaceted model capable of characterizing a large number of facial features. Here, we addressed this challenge by conducting in silico experiments using a pre-trained face recognition deep neural network (DNN) with a diverse array of stimuli. We identified a subset of DNN units selective to face identities, and these identity-selective units demonstrated generalized discriminability to novel faces. Visualization and manipulation of the network revealed the importance of identity-selective units in face recognition. Importantly, using our monkey and human single-neuron recordings, we directly compared the response of artificial units with real primate neurons to the same stimuli and found that artificial units shared a similar representation of facial features as primate neurons. We also observed a region-based feature coding mechanism in DNN units as in human neurons. Together, by directly linking between artificial and primate neural systems, our results shed light on how the primate brain performs face recognition tasks.

More Than the Face: Representations of Bodies in the Inferior Temporal Cortex

Visual representations of bodies, in addition to those of faces, contribute to the recognition of con- and heterospecifics, to action recognition, and to nonverbal communication. Despite its importance, the neural basis of the visual analysis of bodies has been less studied than that of faces. In this article, I review what is known about the neural processing of bodies, focusing on the macaque temporal visual cortex. Early single-unit recording work suggested that the temporal visual cortex contains representations of body parts and bodies, with the dorsal bank of the superior temporal sulcus representing bodily actions. Subsequent functional magnetic resonance imaging studies in both humans and monkeys showed several temporal cortical regions that are strongly activated by bodies. Single-unit recordings in the macaque body patches suggest that these represent mainly body shape features. More anterior patches show a greater viewpoint-tolerant selectivity for body features, which may reflect a processing principle shared with other object categories, including faces. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Processing of visual and non-visual naturalistic spatial information in the "parahippocampal place area"

The "parahippocampal place area" (PPA) in the human ventral visual stream exhibits increased hemodynamic activity correlated with the perception of landscape photos compared to faces or objects. Here, we investigate the perception of scene-related, spatial information embedded in two naturalistic stimuli. The same 14 participants were watching a Hollywood movie and listening to its audio-description as part of the open-data resource studyforrest.org. We model hemodynamic activity based on annotations of selected stimulus features, and compare results to a block-design visual localizer. On a group level, increased activation correlating with visual spatial information occurring in the movie is overlapping with a traditionally localized PPA. Activation correlating with semantic spatial information occurring in the audio-description is more restricted to the anterior PPA. On an individual level, we find significant bilateral activity in the PPA of nine individuals and unilateral activity in one individual. Results suggest that activation in the PPA generalizes to spatial information embedded in a movie and an auditory narrative, and may call for considering a functional subdivision of the PPA.

Brain-like functional specialization emerges spontaneously in deep neural networks

The human brain contains multiple regions with distinct, often highly specialized functions, from recognizing faces to understanding language to thinking about what others are thinking. However, it remains unclear why the cortex exhibits this high degree of functional specialization in the first place. Here, we consider the case of face perception using artificial neural networks to test the hypothesis that functional segregation of face recognition in the brain reflects a computational optimization for the broader problem of visual recognition of faces and other visual categories. We find that networks trained on object recognition perform poorly on face recognition and vice versa and that networks optimized for both tasks spontaneously segregate themselves into separate systems for faces and objects. We then show functional segregation to varying degrees for other visual categories, revealing a widespread tendency for optimization (without built-in task-specific inductive biases) to lead to functional specialization in machines and, we conjecture, also brains.

Congenitally blind adults can learn to identify face-shapes via auditory sensory substitution and successfully generalize some of the learned features

Unlike sighted individuals, congenitally blind individuals have little to no experience with face shapes. Instead, they rely on non-shape cues, such as voices, to perform character identification. The extent to which face-shape perception can be learned in adulthood via a different sensory modality (i.e., not vision) remains poorly explored. We used a visual-to-auditory Sensory Substitution Device (SSD) that enables conversion of visual images to the auditory modality while preserving their visual characteristics. Expert SSD users were systematically taught to identify cartoon faces via audition. Following a tailored training program lasting ~ 12 h, congenitally blind participants successfully identified six trained faces with high accuracy. Furthermore, they effectively generalized their identification to the untrained, inverted orientation of the learned faces. Finally, after completing the extensive 12-h training program, participants learned six new faces within 2 additional hours of training, suggesting internalization of face-identification processes. Our results document for the first time that facial features can be processed through audition, even in the absence of visual experience across the lifespan. Overall, these findings have important implications for both non-visual object recognition and visual rehabilitation practices and prompt the study of the neural processes underlying auditory face perception in the absence of vision.

Early visual exposure primes future cross-modal specialization of the fusiform face area in tactile face processing in the blind

The fusiform face area (FFA) is a core cortical region for face information processing. Evidence suggests that its sensitivity to faces is largely innate and tuned by visual experience. However, how experience in different time windows shape the plasticity of the FFA remains unclear. In this study, we investigated the role of visual experience at different time points of an individual's early development in the cross-modal face specialization of the FFA. Participants (n = 74) were classified into five groups: congenital blind, early blind, late blind, low vision, and sighted control. Functional magnetic resonance imaging data were acquired when the participants haptically processed carved faces and other objects. Our results showed a robust and highly consistent face-selective activation in the FFA region in the early blind participants, invariant to size and level of abstraction of the face stimuli. The cross-modal face activation in the FFA was much less consistent in other groups. These results suggest that early visual experience primes cross-modal specialization of the FFA, and even after the absence of visual experience for more than 14 years in early blind participants, their FFA can engage in cross-modal processing of face information.

Naturalistic Audio-Movies reveal common spatial organization across "visual" cortices of different blind individuals

Occipital cortices of different sighted people contain analogous maps of visual information (e.g. foveal vs. peripheral). In congenital blindness, "visual" cortices respond to nonvisual stimuli. Do visual cortices of different blind people represent common informational maps? We leverage naturalistic stimuli and inter-subject pattern similarity analysis to address this question. Blindfolded sighted (n = 22) and congenitally blind (n = 22) participants listened to 6 sound clips (5-7 min each): 3 auditory excerpts from movies; a naturalistic spoken narrative; and matched degraded auditory stimuli (Backwards Speech, scrambled sentences), during functional magnetic resonance imaging scanning. We compared the spatial activity patterns evoked by each unique 10-s segment of the different auditory excerpts across blind and sighted people. Segments of meaningful naturalistic stimuli produced distinctive activity patterns in frontotemporal networks that were shared across blind and across sighted individuals. In the blind group only, segment-specific, cross-subject patterns emerged in visual cortex, but only for meaningful naturalistic stimuli and not Backwards Speech. Spatial patterns of activity within visual cortices are sensitive to time-varying information in meaningful naturalistic auditory stimuli in a broadly similar manner across blind individuals.

Visual object categorization in infancy

Categorization is the basis of thinking and reasoning. Through the analysis of infants’ gaze, we describe the trajectory through which visual object representations in infancy incrementally match categorical object representations as mapped onto adults’ visual cortex. Using a methodological approach that allows for a comparison of findings obtained with behavioral and brain measures in infants and adults, we identify the transition from visual exploration guided by perceptual salience to an organization of objects by categories, which begins with the animate–inanimate distinction in the first months of life and continues with a spurt of biologically relevant categories (human bodies, nonhuman bodies, nonhuman faces, small natural objects) through the second year of life.

Humans make sense of the world by organizing things into categories. When and how does this process begin? We investigated whether real-world object categories that spontaneously emerge in the first months of life match categorical representations of objects in the human visual cortex. Using eye tracking, we measured the differential looking time of 4-, 10-, and 19-mo-olds as they looked at pairs of pictures belonging to eight animate or inanimate categories (human/nonhuman, faces/bodies, real-world size big/small, natural/artificial). Taking infants’ looking times as a measure of similarity, for each age group, we defined a representational space where each object was defined in relation to others of the same or of a different category. This space was compared with hypothesis-based and functional MRI-based models of visual object categorization in the adults’ visual cortex. Analyses across different age groups showed that, as infants grow older, their looking behavior matches neural representations in ever-larger portions of the adult visual cortex, suggesting progressive recruitment and integration of more and more feature spaces distributed over the visual cortex. Moreover, the results characterize infants’ visual categorization as an incremental process with two milestones. Between 4 and 10 mo, visual exploration guided by saliency gives way to an organization according to the animate–inanimate distinction. Between 10 and 19 mo, a category spurt leads toward a mature organization. We propose that these changes underlie the coupling between seeing and thinking in the developing mind.

Selective responses to faces, scenes, and bodies in the ventral visual pathway of infants

Three of the most robust functional landmarks in the human brain are the selective responses to faces in the fusiform face area (FFA), scenes in the parahippocampal place area (PPA), and bodies in the extrastriate body area (EBA). Are the selective responses of these regions present early in development or do they require many years to develop? Prior evidence leaves this question unresolved. We designed a new 32-channel infant magnetic resonance imaging (MRI) coil and collected high-quality functional MRI (fMRI) data from infants (2-9 months of age) while they viewed stimuli from four conditions-faces, bodies, objects, and scenes. We find that infants have face-, scene-, and body-selective responses in the location of the adult FFA, PPA, and EBA, respectively, powerfully constraining accounts of cortical development.

Face detection in untrained deep neural networks

Face-selective neurons are observed in the primate visual pathway and are considered as the basis of face detection in the brain. However, it has been debated as to whether this neuronal selectivity can arise innately or whether it requires training from visual experience. Here, using a hierarchical deep neural network model of the ventral visual stream, we suggest a mechanism in which face-selectivity arises in the complete absence of training. We found that units selective to faces emerge robustly in randomly initialized networks and that these units reproduce many characteristics observed in monkeys. This innate selectivity also enables the untrained network to perform face-detection tasks. Intriguingly, we observed that units selective to various non-face objects can also arise innately in untrained networks. Our results imply that the random feedforward connections in early, untrained deep neural networks may be sufficient for initializing primitive visual selectivity.

Visuospatial coding as ubiquitous scaffolding for human cognition

For more than 100 years we have known that the visual field is mapped onto the surface of visual cortex, imposing an inherently spatial reference frame on visual information processing. Recent studies highlight visuospatial coding not only throughout visual cortex, but also brain areas not typically considered visual. Such widespread access to visuospatial coding raises important questions about its role in wider cognitive functioning. Here, we synthesise these recent developments and propose that visuospatial coding scaffolds human cognition by providing a reference frame through which neural computations interface with environmental statistics and task demands via perception-action loops.

One object, two networks? Assessing the relationship between the face and body-selective regions in the primate visual system

Faces and bodies are often treated as distinct categories that are processed separately by face- and body-selective brain regions in the primate visual system. These regions occupy distinct regions of visual cortex and are often thought to constitute independent functional networks. Yet faces and bodies are part of the same object and their presence inevitably covary in naturalistic settings. Here, we re-evaluate both the evidence supporting the independent processing of faces and bodies and the organizational principles that have been invoked to explain this distinction. We outline four hypotheses ranging from completely separate networks to a single network supporting the perception of whole people or animals. The current evidence, especially in humans, is compatible with all of these hypotheses, making it presently unclear how the representation of faces and bodies is organized in the cortex.

On the relationship between maps and domains in inferotemporal cortex

How does the brain encode information about the environment? Decades of research have led to the pervasive notion that the object-processing pathway in primate cortex consists of multiple areas that are each specialized to process different object categories (such as faces, bodies, hands, non-face objects and scenes). The anatomical consistency and modularity of these regions have been interpreted as evidence that these regions are innately specialized. Here, we propose that ventral-stream modules do not represent clusters of circuits that each evolved to process some specific object category particularly important for survival, but instead reflect the effects of experience on a domain-general architecture that evolved to be able to adapt, within a lifetime, to its particular environment. Furthermore, we propose that the mechanisms underlying the development of domains are both evolutionarily old and universal across cortex. Topographic maps are fundamental, governing the development of specializations across systems, providing a framework for brain organization.

Multi-scale structural rich-club organization of the brain in full-term newborns: a combined DWI and fMRI study

Objective.Our understanding of early brain development is limited due to rapid changes in white matter pathways after birth. In this study, we introduced a multi-scale cross-modal approach to investigate the rich club (RC) organization and topology of the structural brain networks in 40 healthy neonates using diffusion-weighted imaging and resting-state fMRI data.Approach.A group independent component analysis was first performed to identify eight resting state networks (RSNs) used as functional modules. A groupwise whole-brain functional parcellation was also performed at five scales comprising 100-900 parcels. The distribution of RC nodes was then investigated within and between the RSNs. We further assessed the distribution of short and long-range RC, feeder and local connections across different parcellation scales.Main results.Sharing the scale-free characteristic of small-worldness, the neonatal structural brain networks exhibited an RC organization at different nodal scales (NSs). The subcortical, sensory-motor and default mode networks were found to be strongly involved in the RC organization of the structural brain networks, especially in the zones where the RSNs overlapped, with an average cross-scale proportion of 45.9%, 28.5% and 10.5%, respectively. A large proportion of the connector hubs were found to be RC members for the coarsest (73%) to finest (92%) NSs. Our results revealed a prominent involvement of cortico-subcortical and cortico-cerebellar white matter pathways in the RC organization of the neonatal brain. Regardless of the NS, the majority (more than 65.2%) of the inter-RSN connections were long distance RC or feeder with an average physical connection of 105.5 and 97.4 mm, respectively. Several key RC regions were identified, including the insula and cingulate gyri, middle and superior temporal gyri, hippocampus and parahippocampus, fusiform gyrus, precuneus, superior frontal and precentral gyri, calcarine fissure and lingual gyrus.Significance.Our results emphasize the importance of the multi-scale connectivity analysis in assessing the cross-scale reproducibility of the connectivity results concerning the global and local topological properties of the brain networks. Our findings may improve our understanding of the early brain development.

Visual Influences on Auditory Behavioral, Neural, and Perceptual Processes: A Review

In a naturalistic environment, auditory cues are often accompanied by information from other senses, which can be redundant with or complementary to the auditory information. Although the multisensory interactions derived from this combination of information and that shape auditory function are seen across all sensory modalities, our greatest body of knowledge to date centers on how vision influences audition. In this review, we attempt to capture the state of our understanding at this point in time regarding this topic. Following a general introduction, the review is divided into 5 sections. In the first section, we review the psychophysical evidence in humans regarding vision's influence in audition, making the distinction between vision's ability to enhance versus alter auditory performance and perception. Three examples are then described that serve to highlight vision's ability to modulate auditory processes: spatial ventriloquism, cross-modal dynamic capture, and the McGurk effect. The final part of this section discusses models that have been built based on available psychophysical data and that seek to provide greater mechanistic insights into how vision can impact audition. The second section reviews the extant neuroimaging and far-field imaging work on this topic, with a strong emphasis on the roles of feedforward and feedback processes, on imaging insights into the causal nature of audiovisual interactions, and on the limitations of current imaging-based approaches. These limitations point to a greater need for machine-learning-based decoding approaches toward understanding how auditory representations are shaped by vision. The third section reviews the wealth of neuroanatomical and neurophysiological data from animal models that highlights audiovisual interactions at the neuronal and circuit level in both subcortical and cortical structures. It also speaks to the functional significance of audiovisual interactions for two critically important facets of auditory perception-scene analysis and communication. The fourth section presents current evidence for alterations in audiovisual processes in three clinical conditions: autism, schizophrenia, and sensorineural hearing loss. These changes in audiovisual interactions are postulated to have cascading effects on higher-order domains of dysfunction in these conditions. The final section highlights ongoing work seeking to leverage our knowledge of audiovisual interactions to develop better remediation approaches to these sensory-based disorders, founded in concepts of perceptual plasticity in which vision has been shown to have the capacity to facilitate auditory learning.

A Probabilistic Functional Atlas of Human Occipito-Temporal Visual Cortex

Human visual cortex contains many retinotopic and category-specific regions. These brain regions have been the focus of a large body of functional magnetic resonance imaging research, significantly expanding our understanding of visual processing. As studying these regions requires accurate localization of their cortical location, researchers perform functional localizer scans to identify these regions in each individual. However, it is not always possible to conduct these localizer scans. Here, we developed and validated a functional region of interest (ROI) atlas of early visual and category-selective regions in human ventral and lateral occipito-temporal cortex. Results show that for the majority of functionally defined ROIs, cortex-based alignment results in lower between-subject variability compared to nonlinear volumetric alignment. Furthermore, we demonstrate that 1) the atlas accurately predicts the location of an independent dataset of ventral temporal cortex ROIs and other atlases of place selectivity, motion selectivity, and retinotopy. Next, 2) we show that the majority of voxel within our atlas is responding mostly to the labeled category in a left-out subject cross-validation, demonstrating the utility of this atlas. The functional atlas is publicly available (download.brainvoyager.com/data/visfAtlas.zip) and can help identify the location of these regions in healthy subjects as well as populations (e.g., blind people, infants) in which functional localizers cannot be run.

Visual experience dependent plasticity in humans

While sensitive periods in brain development have often been studied by investigating the recovery of visual functions after a congenital phase of visual deprivation in non-human animals, research in humans who had recovered sight after a transient phase of congenital blindness is still scarce. Here, we discuss the hypothesis put forward based on non-human primate work which states that the effects of experience increase downstream the visual processing hierarchy. Recent results from behavioral and neuroscience studies in sight recovery individuals are discussed in the context of research findings from permanently congenitally blind humans as well as from prospective studies in infants and children.

Learning in Infancy Is Active, Endogenously Motivated, and Depends on the Prefrontal Cortices

A common view of learning in infancy emphasizes the role of incidental sensory experiences from which increasingly abstract statistical regularities are extracted. In this view, infant brains initially support basic sensory and motor functions, followed by maturation of higher-level association cortex. Here, we critique this view and posit that, by contrast and more like adults, infants are active, endogenously motivated learners who structure their own learning through flexible selection of attentional targets and active interventions on their environment. We further argue that the infant brain, and particularly the prefrontal cortex (PFC), is well equipped to support these learning behaviors. We review recent progress in characterizing the function of the infant PFC, which suggests that, as in adults, the PFC is functionally specialized and highly connected. Together, we present an integrative account of infant minds and brains, in which the infant PFC represents multiple intrinsic motivations, which are leveraged for active learning.

Social cognition in the blind brain: A coordinate-based meta-analysis

Social cognition skills are typically acquired on the basis of visual information (e.g., the observation of gaze, facial expressions, gestures). In light of this, a critical issue is whether and how the lack of visual experience affects neurocognitive mechanisms underlying social skills. This issue has been largely neglected in the literature on blindness, despite difficulties in social interactions may be particular salient in the life of blind individuals (especially children). Here we provide a meta-analysis of neuroimaging studies reporting brain activations associated to the representation of self and others' in early blind individuals and in sighted controls. Our results indicate that early blindness does not critically impact on the development of the "social brain," with social tasks performed on the basis of auditory or tactile information driving consistent activations in nodes of the action observation network, typically active during actual observation of others in sighted individuals. Interestingly though, activations along this network appeared more left-lateralized in the blind than in sighted participants. These results may have important implications for the development of specific training programs to improve social skills in blind children and young adults.

Neural mechanisms of visual sensitive periods in humans

Sensitive periods in brain development are phases of enhanced susceptibility to experience. Here we discuss research from human and non-human neuroscience studies which have demonstrated a) differences in the way infants vs. adults learn; b) how the brain adapts to atypical conditions, in particular a congenital vs. a late onset blindness (sensitive periods for atypical brain development); and c) the extent to which neural systems are capable of acquiring a typical brain organization after sight restoration following a congenital vs. late phase of pattern vision deprivation (sensitive periods for typical brain development). By integrating these three lines of research, we propose neural mechanisms characteristic of sensitive periods vs. adult neuroplasticity and learning.

The intrinsic neonatal hippocampal network: rsfMRI findings

Although both animal data and human data suggest that the hippocampus is immature at birth, to date, there are no direct assessments of human hippocampal functional connectivity (FC) very early in life. Our study explores the FC of the hippocampus to the cortex at birth, allowing insight into the development of human memory systems. In particular, we find that adults and neonates exhibit vastly different hippocampal connectivity profiles—a finding that likely has large developmental implications.

How visual is the « number sense »? Insights from the blind

Is vision a necessary building block for the foundations of mathematical cognition? A straightforward model to test the causal role visual experience plays in the development of numerical abilities is to study people born without sight. In this review we will demonstrate that congenitally blind people can develop numerical abilities that equal or even surpass those of sighted individuals, despite representing numbers using a qualitatively different representational format. We will also show that numerical thinking in blind people maps onto regions typically involved in visuo-spatial processing in the sighted, highlighting how intrinsic computational biases may constrain the reorganization of numerical networks in case of early visual deprivation. More generally, we will illustrate how the study of arithmetic abilities in congenitally blind people represents a compelling model to understand how sensory experience scaffolds the development of higher-level cognitive representations.

Innate connectivity patterns drive the development of the visual word form area

What determines the functional organization of cortex? One hypothesis is that innate connectivity patterns, either structural or functional connectivity, set up a scaffold upon which functional specialization can later take place. We tested this hypothesis by asking whether the visual word form area (VWFA), an experience-driven region, was already functionally connected to proto language networks in neonates scanned within one week of birth. Using the data from the Human Connectone Project (HCP) and the Developing Human Connectome Project (dHCP), we calculated intrinsic functional connectivity during resting-state functional magnetic resonance imaging (fMRI), and found that neonates showed similar functional connectivity patterns to adults. We observed that (1) language regions connected more strongly with the putative VWFA than other adjacent ventral visual regions that also show foveal bias, and (2) the VWFA connected more strongly with frontotemporal language regions than with regions adjacent to these language regions. These data suggest that the location of the VWFA is earmarked at birth due to its connectivity with the language network, providing evidence that innate connectivity instructs the later refinement of cortex.

Sulcal morphology of ventral temporal cortex is shared between humans and other hominoids

Hominoid-specific brain structures are of particular importance in understanding the evolution of human brain structure and function, as they are absent in mammals that are widely studied in the extended neuroscience field. Recent research indicates that the human fusiform gyrus (FG), which is a hominoid-specific structure critical for complex object recognition, contains a tertiary, longitudinal sulcus (mid-fusiform sulcus, MFS) that bisects the FG into lateral and medial parallel gyri. The MFS is a functional and architectonic landmark in the human brain. Here, we tested if the MFS is specific to the human FG or if the MFS is also identifiable in other hominoids. Using magnetic resonance imaging and cortical surface reconstructions in 30 chimpanzees and 30 humans, we show that the MFS is also present in chimpanzees. The MFS is relatively deeper and cortically thinner in chimpanzees compared to humans. Additional histological analyses reveal that the MFS is not only present in humans and chimpanzees, but also in bonobos, gorillas, orangutans, and gibbons. Taken together, these results reveal that the MFS is a sulcal landmark that is shared between humans and other hominoids. These results require a reconsideration of the sulcal patterning in ventral temporal cortex across hominoids, as well as revise the compensation theory of cortical folding.

Visual experience is not necessary for the development of face-selectivity in the lateral fusiform gyrus

The fusiform face area responds selectively to faces and is causally involved in face perception. How does face-selectivity in the fusiform arise in development, and why does it develop so systematically in the same location across individuals? Preferential cortical responses to faces develop early in infancy, yet evidence is conflicting on the central question of whether visual experience with faces is necessary. Here, we revisit this question by scanning congenitally blind individuals with fMRI while they haptically explored 3D-printed faces and other stimuli. We found robust face-selective responses in the lateral fusiform gyrus of individual blind participants during haptic exploration of stimuli, indicating that neither visual experience with faces nor fovea-biased inputs is necessary for face-selectivity to arise in the lateral fusiform gyrus. Our results instead suggest a role for long-range connectivity in specifying the location of face-selectivity in the human brain.

Combined Neural Tuning in Human Ventral Temporal Cortex Resolves the Perceptual Ambiguity of Morphed 2D Images

We have an amazing ability to categorize objects in the world around us. Nevertheless, how cortical regions in human ventral temporal cortex (VTC), which is critical for categorization, support this behavioral ability, is largely unknown. Here, we examined the relationship between neural responses and behavioral performance during the categorization of morphed silhouettes of faces and hands, which are animate categories processed in cortically adjacent regions in VTC. Our results reveal that the combination of neural responses from VTC face- and body-selective regions more accurately explains behavioral categorization than neural responses from either region alone. Furthermore, we built a model that predicts a person's behavioral performance using estimated parameters of brain-behavior relationships from a different group of people. Moreover, we show that this brain-behavior model generalizes to adjacent face- and body-selective regions in lateral occipitotemporal cortex. Thus, while face- and body-selective regions are located within functionally distinct domain-specific networks, cortically adjacent regions from both networks likely integrate neural responses to resolve competing and perceptually ambiguous information from both categories.

Decoding Natural Sounds in Early "Visual" Cortex of Congenitally Blind Individuals

Complex natural sounds, such as bird singing, people talking, or traffic noise, induce decodable fMRI activation patterns in early visual cortex of sighted blindfolded participants [1]. That is, early visual cortex receives non-visual and potentially predictive information from audition. However, it is unclear whether the transfer of auditory information to early visual areas is an epiphenomenon of visual imagery or, alternatively, whether it is driven by mechanisms independent from visual experience. Here, we show that we can decode natural sounds from activity patterns in early “visual” areas of congenitally blind individuals who lack visual imagery. Thus, visual imagery is not a prerequisite of auditory feedback to early visual cortex. Furthermore, the spatial pattern of sound decoding accuracy in early visual cortex was remarkably similar in blind and sighted individuals, with an increasing decoding accuracy gradient from foveal to peripheral regions. This suggests that the typical organization by eccentricity of early visual cortex develops for auditory feedback, even in the lifelong absence of vision. The same feedback to early visual cortex might support visual perception in the sighted [1] and drive the recruitment of this area for non-visual functions in blind individuals [2, 3].

•

Sounds can be decoded from early visual cortex activity in blind individuals

•

Sound decoding accuracy increases from foveal to peripheral early visual regions

•

Visual imagery is not necessary for auditory feedback to early visual cortex

•

Early visual cortex organization by eccentricity develops without visual experience

Hemispheric Organization for Visual Object Recognition: A Theoretical Account and Empirical Evidence

Despite the similarity in structure, the hemispheres of the human brain have somewhat different functions. A traditional view of hemispheric organization asserts that there are independent and largely lateralized domain-specific regions in ventral occipitotemporal (VOTC), specialized for the recognition of distinct classes of objects. Here, we offer an alternative account of the organization of the hemispheres, with a specific focus on face and word recognition. This alternative account relies on three computational principles: distributed representations and knowledge, cooperation and competition between representations, and topography and proximity. The crux is that visual recognition results from a network of regions with graded functional specialization that is distributed across both hemispheres. Specifically, the claim is that face recognition, which is acquired relatively early in life, is processed by VOTC regions in both hemispheres. Once literacy is acquired, word recognition, which is co-lateralized with language areas, primarily engages the left VOTC and, consequently, face recognition is primarily, albeit not exclusively, mediated by the right VOTC. We review psychological and neural evidence from a range of studies conducted with normal and brain-damaged adults and children and consider findings which challenge this account. Last, we offer suggestions for future investigations whose findings may further refine this account.

Connectivity at the origins of domain specificity in the cortical face and place networks

It is well established that the adult brain contains a mosaic of domain-specific networks. But how do these domain-specific networks develop? Here we tested the hypothesis that the brain comes prewired with connections that precede the development of domain-specific function. Using resting-state fMRI in the youngest sample of newborn humans tested to date, we indeed found that cortical networks that will later develop strong face selectivity (including the "proto" occipital face area and fusiform face area) and scene selectivity (including the "proto" parahippocampal place area and retrosplenial complex) by adulthood, already show domain-specific patterns of functional connectivity as early as 27 d of age (beginning as early as 6 d of age). Furthermore, we asked how these networks are functionally connected to early visual cortex and found that the proto face network shows biased functional connectivity with foveal V1, while the proto scene network shows biased functional connectivity with peripheral V1. Given that faces are almost always experienced at the fovea, while scenes always extend across the entire periphery, these differential inputs may serve to facilitate domain-specific processing in each network after that function develops, or even guide the development of domain-specific function in each network in the first place. Taken together, these findings reveal domain-specific and eccentricity-biased connectivity in the earliest days of life, placing new constraints on our understanding of the origins of domain-specific cortical networks.