Brain-wide functional connectivity of face patch neurons during rest

In resting state functional magnetic resonance imaging (fMRI), areas showing coherent hemodynamic fluctuations across the brain are operationally defined to be functionally connected. However, it is unknown how the activity of single units residing within a voxel contributes to this network structure. Here we demonstrate a shared but restricted pattern of functional connectivity among neighboring neurons residing in functionally defined face patches. Unexpectedly, such neurons also exhibited a prominent inverse correlation with thalamic structures and brainstem neuromodulatory centers. Single unit maps differed from analogous maps obtained with local field potentials and seed-based fMRI. These findings suggest that during rest, individual cortical neurons have a restricted set of functional connections, which is governed in part by anatomical projections and in part by neuromodulation.

The brain is a highly organized, dynamic system whose network architecture is often assessed through resting functional magnetic resonance imaging (fMRI) functional connectivity. The functional interactions between brain areas, including those observed during rest, are assumed to stem from the collective influence of action potentials carried by long-range neural projections. However, the contribution of individual neurons to brain-wide functional connectivity has not been systematically assessed. Here we developed a method to concurrently measure and compare the spiking activity of local neurons with fMRI signals measured across the brain during rest. We recorded spontaneous activity from neural populations in cortical face patches in the macaque during fMRI scanning sessions. Individual cells exhibited prominent, bilateral coupling with fMRI fluctuations in a restricted set of cortical areas inside and outside the face patch network, partially matching the pattern of known anatomical projections. Within each face patch population, a subset of neurons was positively coupled with the face patch network and another was negatively coupled. The same cells showed inverse correlations with distinct subcortical structures, most notably the lateral geniculate nucleus and brainstem neuromodulatory centers. Corresponding connectivity maps derived from fMRI seeds and local field potentials differed from the single unit maps, particularly in subcortical areas. Together, the results demonstrate that the spiking fluctuations of neurons are selectively coupled with discrete brain regions, with the coupling governed in part by anatomical network connections and in part by indirect neuromodulatory pathways.

Audiovisual integration in macaque face patch neurons

Primate social communication depends on the perceptual integration of visual and auditory cues, reflected in the multimodal mixing of sensory signals in certain cortical areas. The macaque cortical face patch network, identified through visual, face-selective responses measured with fMRI, is assumed to contribute to visual social interactions. However, whether face patch neurons are also influenced by acoustic information, such as the auditory component of a natural vocalization, remains unknown. Here, we recorded single-unit activity in the anterior fundus (AF) face patch, in the superior temporal sulcus, and anterior medial (AM) face patch, on the undersurface of the temporal lobe, in macaques presented with audiovisual, visual-only, and auditory-only renditions of natural movies of macaques vocalizing. The results revealed that 76% of neurons in face patch AF were significantly influenced by the auditory component of the movie, most often through enhancement of visual responses but sometimes in response to the auditory stimulus alone. By contrast, few neurons in face patch AM exhibited significant auditory responses or modulation. Control experiments in AF used an animated macaque avatar to demonstrate, first, that the structural elements of the face were often essential for audiovisual modulation and, second, that the temporal modulation of the acoustic stimulus was more important than its frequency spectrum. Together, these results identify a striking contrast between two face patches and specifically identify AF as playing a potential role in the integration of audiovisual cues during natural modes of social communication.

View-Tolerant Face Recognition and Hebbian Learning Imply Mirror-Symmetric Neural Tuning to Head Orientation

The primate brain contains a hierarchy of visual areas, dubbed the ventral stream, which rapidly computes object representations that are both specific for object identity and robust against identity-preserving transformations, like depth rotations [1, 2]. Current computational models of object recognition, including recent deep-learning networks, generate these properties through a hierarchy of alternating selectivity-increasing filtering and tolerance-increasing pooling operations, similar to simple-complex cells operations [3-6]. Here, we prove that a class of hierarchical architectures and a broad set of biologically plausible learning rules generate approximate invariance to identity-preserving transformations at the top level of the processing hierarchy. However, all past models tested failed to reproduce the most salient property of an intermediate representation of a three-level face-processing hierarchy in the brain: mirror-symmetric tuning to head orientation [7]. Here, we demonstrate that one specific biologically plausible Hebb-type learning rule generates mirror-symmetric tuning to bilaterally symmetric stimuli, like faces, at intermediate levels of the architecture and show why it does so. Thus, the tuning properties of individual cells inside the visual stream appear to result from group properties of the stimuli they encode and to reflect the learning rules that sculpted the information-processing system within which they reside.

Whole-agent selectivity within the macaque face-processing system

The primate brain contains a set of face-selective areas, which are thought to extract the rich social information that faces provide, such as emotional state and personal identity. The nature of this information raises a fundamental question about these face-selective areas: Do they respond to a face purely because of its visual attributes, or because the face embodies a larger social agent? Here, we used functional magnetic resonance imaging to determine whether the macaque face patch system exhibits a whole-agent response above and beyond its responses to individually presented faces and bodies. We found a systematic development of whole-agent preference through the face patches, from subadditive integration of face and body responses in posterior face patches to superadditive integration in anterior face patches. Superadditivity was not observed for faces atop nonbody objects, implying categorical specificity of face-body interaction. Furthermore, superadditivity was robust to visual degradation of facial detail, suggesting whole-agent selectivity does not require prior face recognition. In contrast, even the body patches immediately adjacent to anterior face areas did not exhibit superadditivity. This asymmetry between face- and body-processing systems may explain why observers attribute bodies' social signals to faces, and not vice versa. The development of whole-agent selectivity from posterior to anterior face patches, in concert with the recently described development of natural motion selectivity from ventral to dorsal face patches, identifies a single face patch, AF (anterior fundus), as a likely link between the analysis of facial shape and semantic inferences about other agents.

Curvature-processing network in macaque visual cortex

Our visual environment abounds with curved features. Thus, the goal of understanding visual processing should include the processing of curved features. Using functional magnetic resonance imaging in behaving monkeys, we demonstrated a network of cortical areas selective for the processing of curved features. This network includes three distinct hierarchically organized regions within the ventral visual pathway: a posterior curvature-biased patch (PCP) located in the near-foveal representation of dorsal V4, a middle curvature-biased patch (MCP) located on the ventral lip of the posterior superior temporal sulcus (STS) in area TEO, and an anterior curvature-biased patch (ACP) located just below the STS in anterior area TE. Our results further indicate that the processing of curvature becomes increasingly complex from PCP to ACP. The proximity of the curvature-processing network to the well-known face-processing network suggests a possible functional link between them.

Disparate substrates for head gaze following and face perception in the monkey superior temporal sulcus

Primates use gaze cues to follow peer gaze to an object of joint attention. Gaze following of monkeys is largely determined by head or face orientation. We used fMRI in rhesus monkeys to identify brain regions underlying head gaze following and to assess their relationship to the 'face patch' system, the latter being the likely source of information on face orientation. We trained monkeys to locate targets by either following head gaze or using a learned association of face identity with the same targets. Head gaze following activated a distinct region in the posterior STS, close to-albeit not overlapping with-the medial face patch delineated by passive viewing of faces. This 'gaze following patch' may be the substrate of the geometrical calculations needed to translate information on head orientation from the face patches into precise shifts of attention, taking the spatial relationship of the two interacting agents into account.