The bottom-up and top-down processing of faces in the human occipitotemporal cortex

Studying perceptual bias in favor of the from-above Necker cube perspective in a goal-directed behavior

When viewing a completely ambiguous image, different interpretations can switch involuntarily due to internal top-down processing. In the case of the Necker cube, an entirely ambiguous stimulus, observers often display a bias in perceptual switching between two interpretations based on their perspectives: one with a from-above perspective (FA) and the other with a from-below perspective (FB). Typically, observers exhibit a priori top-down bias in favor of the FA interpretation, which may stem from a statistical tendency in everyday life where we more frequently observe objects from above. However, it remains unclear whether this perceptual bias persists when individuals voluntarily decide on the Necker cube's interpretation in goal-directed behavior, and the impact of ambiguity in this context is not well-understood. In our study, we instructed observers to voluntarily identify the orientation of a Necker cube while manipulating its ambiguity from low (LA) to high (HA). Our investigation aimed to test two hypotheses: (i) whether the perspective (FA or FB) would result in a bias in response time, and (ii) whether this bias would depend on the level of stimulus ambiguity. Additionally, we analyzed electroencephalogram (EEG) signals to identify potential biomarkers that could explain the observed perceptual bias. The behavioral results confirmed a perceptual bias in favor of the from-above perspective, as indicated by shorter response times. However, this bias diminished for stimuli with high ambiguity. For the LA stimuli, the occipital theta-band power consistently exceeded the frontal theta-band power throughout most of the decision-making time. In contrast, for the HA stimuli, the frontal theta-band power started to exceed the occipital theta-band power during the 0.3-s period preceding the decision. We propose that occipital theta-band power reflects evidence accumulation, while frontal theta-band power reflects its evaluation and decision-making processes. For the FB perspective, occipital theta-band power exhibited higher values and dominated over a longer duration, leading to an overall increase in response time. These results suggest that more information and more time are needed to encode stimuli with a FB perspective, as this template is less common for the observers compared to the template for a cube with a FA perspective.

Positive hysteresis in emotion recognition: Face processing visual regions are involved in perceptual persistence, which mediates interactions between anterior insula and medial prefrontal cortex

Facial emotion perception can be studied from the point of view of dynamic systems whose output may depend not only on current input but also on prior history - a phenomenon known as hysteresis. In cognitive neuroscience, hysteresis has been described as positive (perceptual persistence) or negative (fatigue of current percept) depending on whether perceptual switching occurs later or earlier than actual physical stimulus changes. However, its neural correlates remain elusive. We used dynamic transitions between emotional expressions and combined behavioral assessment with functional magnetic resonance imaging (fMRI) to investigate the underlying circuitry of perceptual hysteresis in facial emotion recognition. Our findings revealed the involvement of face-selective visual areas - fusiform face area (FFA) and superior temporal sulcus (STS) - in perceptual persistence as well as the right anterior insula. Moreover, functional connectivity analyses revealed an interplay between the right anterior insula and medial prefrontal cortex, which showed to be dependent on the presence of positive hysteresis. Our results support the hypothesis that high-order regions are involved in perceptual stabilization and decision during perceptual persistence (positive hysteresis) and add evidence to the role of the anterior insula as a hub of sensory information in perceptual decision-making.

Twenty years of investigation with the case of prosopagnosia PS to understand human face identity recognition. Part II: Neural basis

Patient PS sustained her dramatic brain injury in 1992, the same year as the first report of a neuroimaging study of human face recognition. The present paper complements the review on the functional nature of PS's prosopagnosia (part I), illustrating how her case study directly, i.e., through neuroimaging investigations of her brain structure and activity, but also indirectly, through neural studies performed on other clinical cases and neurotypical individuals, inspired and constrained neural models of human face recognition. In the dominant right hemisphere for face recognition in humans, PS's main lesion concerns (inputs to) the inferior occipital gyrus (IOG), in a region where face-selective activity is typically found in normal individuals ('Occipital Face Area', OFA). Her case study initially supported the criticality of this region for face identity recognition (FIR) and provided the impetus for transcranial magnetic stimulation (TMS), intracerebral electrical stimulation, and cortical surgery studies that have generally supported this view. Despite PS's right IOG lesion, typical face-selectivity is found anteriorly in the middle portion of the fusiform gyrus, a hominoid structure (termed the right 'Fusiform Face Area', FFA) that is widely considered to be the most important region for human face recognition. This finding led to the original proposal of direct anatomico-functional connections from early visual cortices to the FFA, bypassing the IOG/OFA (lulu), a hypothesis supported by further neuroimaging studies of PS, other neurological cases and neuro-typical individuals with original visual stimulation paradigms, data recordings and analyses. The proposal of a lack of sensitivity to face identity in PS's right FFA due to defective reentrant inputs from the IOG/FFA has also been supported by other cases, functional connectivity and cortical surgery studies. Overall, neural studies of, and based on, the case of prosopagnosia PS strongly question the hierarchical organization of the human neural face recognition system, supporting a more flexible and dynamic view of this key social brain function.

Attention impedes neural representation of interpolated orientation during perceptual completion

One of the most remarkable functional feats accomplished by visual system is the interpolation of missing retinal inputs based on surrounding information, a process known as perceptual completion. Perceptual completion enables the active construction of coherent, vivid percepts from spatially discontinuous visual information that is prevalent in real-life visual scenes. Despite mounting evidence linking sensory activity enhancement and perceptual completion, surprisingly little is known about whether and how attention, a fundamental modulator of sensory activities, affects perceptual completion. Using EEG-based time-resolved inverted encoding model (IEM), we reconstructed the moment-to-moment representation of the illusory grating that resulted from spatially interpolating the orientation of surrounding inducers. We found that, despite manipulation of observers' attentional focus, the illusory grating representation unfolded in time in a similar manner. Critically, attention to the surrounding inducers simultaneously attenuated the illusory grating representation and delayed its temporal development. Our findings disclosed, for the first time, the suppressive role of selective attention in perceptual completion and were suggestive of a fast, automatic neural machinery that implements the interpolation of missing visual information.

Free-Living Physical Activity Measured With a Wearable Device Is Associated With Larger Hippocampus Volume and Greater Functional Connectivity in Healthy Older Adults: An Observational, Cross-Sectional Study in Northern Portugal

Several studies using neuroimaging techniques have established a positive relationship between physical activity (PA) and brain structure and function in older populations. However, the use of subjective measures of PA and the lack of multimodal neuroimaging approaches have limited the understanding of this association. This study aims to explore the associations between PA and brain structure and function by objectively evaluating PA. Community-dwelling cognitively healthy older adults (without diagnosed cognitive, neurological or degenerative disease) were recruited from local health centers and local gyms. In a cross-sectional design, participants were evaluated regarding cognitive, clinical, anthropometric, physical performance, and lifestyle characteristics. A 3 T magnetic resonance imaging (MRI) was performed for structural and functional brain measures. PA time and level was assessed via a Xiaomi Mi Band 2^® worn for 15 consecutive days. Participants (n = 110, after inclusion/exclusion criteria and completion of all evaluations) were 58 females (56%), with an average age of 68.42 years old (SD = 3.12), most were active. Multiple regression analysis revealed that higher time spent in vigorous PA associated with larger left parahippocampal gyrus and right hippocampus volumes. Furthermore, the analysis of the functional connectome indicated a greater functional connectivity (FC) between the frontal gyrus, cingulate gyrus, occipital inferior lobe for light, moderate, and total PA time, and sedentary time associated with lower FC in the same networks. Overall, the structural and functional findings may provide evidence on the relevant association between PA and brain health in aging.

A Cautionary Note on Predicting Social Judgments from Faces with Deep Neural Networks

People spontaneously infer other people’s psychology from faces, encompassing inferences of their affective states, cognitive states, and stable traits such as personality. These judgments are known to be often invalid, but nonetheless bias many social decisions. Their importance and ubiquity have made them popular targets for automated prediction using deep convolutional neural networks (DCNNs). Here, we investigated the applicability of this approach: how well does it generalize, and what biases does it introduce? We compared three distinct sets of features (from a face identification DCNN, an object recognition DCNN, and using facial geometry), and tested their prediction across multiple out-of-sample datasets. Across judgments and datasets, features from both pre-trained DCNNs provided better predictions than did facial geometry. However, predictions using object recognition DCNN features were not robust to superficial cues (e.g., color and hair style). Importantly, predictions using face identification DCNN features were not specific: models trained to predict one social judgment (e.g., trustworthiness) also significantly predicted other social judgments (e.g., femininity and criminal), and at an even higher accuracy in some cases than predicting the judgment of interest (e.g., trustworthiness). Models trained to predict affective states (e.g., happy) also significantly predicted judgments of stable traits (e.g., sociable), and vice versa. Our analysis pipeline not only provides a flexible and efficient framework for predicting affective and social judgments from faces but also highlights the dangers of such automated predictions: correlated but unintended judgments can drive the predictions of the intended judgments.

Global and high-level effects in crowding cannot be predicted by either high-dimensional pooling or target cueing

In visual crowding, the perception of a target deteriorates in the presence of nearby flankers. Traditionally, target-flanker interactions have been considered as local, mostly deleterious, low-level, and feature specific, occurring when information is pooled along the visual processing hierarchy. Recently, a vast literature of high-level effects in crowding (grouping effects and face-holistic crowding in particular) led to a different understanding of crowding, as a global, complex, and multilevel phenomenon that cannot be captured or explained by simple pooling models. It was recently argued that these high-level effects may still be captured by more sophisticated pooling models, such as the Texture Tiling model (TTM). Unlike simple pooling models, the high-dimensional pooling stage of the TTM preserves rich information about a crowded stimulus and, in principle, this information may be sufficient to drive high-level and global aspects of crowding. In addition, it was proposed that grouping effects in crowding may be explained by post-perceptual target cueing. Here, we extensively tested the predictions of the TTM on the results of six different studies that highlighted high-level effects in crowding. Our results show that the TTM cannot explain any of these high-level effects, and that the behavior of the model is equivalent to a simple pooling model. In addition, we show that grouping effects in crowding cannot be predicted by post-perceptual factors, such as target cueing. Taken together, these results reinforce once more the idea that complex target-flanker interactions determine crowding and that crowding occurs at multiple levels of the visual hierarchy.

High spatial frequency filtered primes hastens happy faces categorization in autistic adults

Coarse information of a visual stimulus is conveyed by Low Spatial Frequencies (LSF) and is thought to be rapidly extracted to generate predictions. This may guide fast recognition with the subsequent integration of fine information, conveyed by High Spatial Frequencies (HSF). In autism, emotional face recognition is challenging, and might be related to alterations in LSF predictive processes. We analyzed the data of 27 autistic and 34 non autistic (NA) adults on an emotional Stroop task (i.e., emotional face with congruent or incongruent emotional word) with spatially filtered primes (HSF vs.LSF). We hypothesized that LSF primes would generate predictions leading to faster categorization of the target face compared to HSF primes, in the NA group but not in autism. Surprisingly, HSF primes led to faster categorization than LSF primes in both groups. Moreover, the advantage of HSF vs.LSF primes was stronger for angry than happy faces in NA, but was stronger for happy than angry faces in autistic participants. Drift diffusion modelling confirmed HSF advantage and showed a longer non-decision time (e.g., encoding) in autism. Despite LSF predictive impairments in autism was not corroborated, our analyses suggest low level processing specificities in autism.

Gestalts at threshold could reveal Gestalts as predictions

We review and revisit the predictive processing inspired “Gestalts as predictions” hypothesis. The study of Gestalt phenomena at and below threshold can help clarify the role of higher-order object selective areas and feedback connections in mid-level vision. In two psychophysical experiments assessing manipulations of contrast and configurality we showed that: (1) Gestalt phenomena are robust against saliency manipulations across the psychometric function even below threshold (with the accuracy gains and higher saliency associated with Gestalts being present even around chance performance); and (2) peak differences between Gestalt and control conditions happened around the time where responses to Gestalts are starting to saturate (mimicking the differential contrast response profile of striate vs. extra-striate visual neurons). In addition, Gestalts are associated with steeper psychometric functions in all experiments. We propose that these results reflect the differential engagement of object-selective areas in Gestalt phenomena and of information- or percept-based processing, as opposed to energy- or stimulus-based processing, more generally. In addition, the presence of nonlinearities in the psychometric functions suggest differential top-down modulation of the early visual cortex. We treat this as a proof of principle study, illustrating that classic psychophysics can help assess possible involvement of hierarchical predictive processing in Gestalt phenomena.

Is Face Age Mapped Asymmetrically onto Space? Insights from a SNARC-like Task

The magnitude associated with a stimulus can be spatially connoted, with relatively smaller and larger magnitudes that would be represented on the left and on the right side of space, respectively. According to recent evidence, this space–magnitude association could reflect specific brain asymmetries. In this study, we explored whether such an association can also emerge for face age, assuming that responders should represent relatively younger and older adult faces on the left and on the right, respectively. A sample of young adults performed a speeded binary classification task aimed at categorising the age of a centrally placed adult face stimulus as either younger or older than the age of a reference face. A left-side and a right-side response key were used to collect manual responses. Overall, older faces were categorised faster than younger faces, and response latencies decreased with the absolute difference between the age of the target stimulus and the age of the reference, in line with a distance effect. However, no evidence of a left-to-right spatial representation of face age emerged. Taken together, these results suggest that face age is mapped onto space differently from other magnitudes.

Data-point-wise spatiotemporal mapping of human ventral visual areas: Use of spatial frequency/luminance-modulated chromatic faces

Visual information involving facial identity and expression is crucial for social communication. Although the influence of facial features such as spatial frequency (SF) and luminance on face processing in visual areas has been studied extensively using grayscale stimuli, the combined effects of other features in this process have not been characterized. To determine the combined effects of different SFs and color, we created chromatic stimuli with low, high or no SF components, which bring distinct SF and color information into the ventral stream simultaneously. To obtain neural activity data with high spatiotemporal resolution we recorded face-selective responses (M170) using magnetoencephalography. We used a permutation test procedure with threshold-free cluster enhancement to assess statistical significance while resolving problems related to multiple comparisons and arbitrariness found in traditional statistical methods. We found that time windows with statistically significant threshold levels were distributed differently among the stimulus conditions. Face stimuli containing any SF components evoked M170 in the fusiform gyrus (FG), whereas a significant emotional effect on M170 was only observed with the original images. Low SF faces elicited larger activation of the FG and the inferior occipital gyrus than the original images, suggesting an interaction between low and high SF information processing. Interestingly, chromatic face stimuli without SF first activated color-selective regions and then the FG, indicating that facial color was processed according to a hierarchy in the ventral stream. These findings suggest complex effects of SFs in the presence of color information, reflected in M170, and unveil the detailed spatiotemporal dynamics of face processing in the human brain.

Special Patterns of Dynamic Brain Networks Discriminate Between Face and Non-face Processing: A Single-Trial EEG Study

Face processing is a spatiotemporal dynamic process involving widely distributed and closely connected brain regions. Although previous studies have examined the topological differences in brain networks between face and non-face processing, the time-varying patterns at different processing stages have not been fully characterized. In this study, dynamic brain networks were used to explore the mechanism of face processing in human brain. We constructed a set of brain networks based on consecutive short EEG segments recorded during face and non-face (ketch) processing respectively, and analyzed the topological characteristic of these brain networks by graph theory. We found that the topological differences of the backbone of original brain networks (the minimum spanning tree, MST) between face and ketch processing changed dynamically. Specifically, during face processing, the MST was more line-like over alpha band in 0-100 ms time window after stimuli onset, and more star-like over theta and alpha bands in 100-200 and 200-300 ms time windows. The results indicated that the brain network was more efficient for information transfer and exchange during face processing compared with non-face processing. In the MST, the nodes with significant differences of betweenness centrality and degree were mainly located in the left frontal area and ventral visual pathway, which were involved in the face-related regions. In addition, the special MST patterns can discriminate between face and ketch processing by an accuracy of 93.39%. Our results suggested that special MST structures of dynamic brain networks reflected the potential mechanism of face processing in human brain.

Children with autism spectrum disorder show altered functional connectivity and abnormal maturation trajectories in response to inverted faces

The processing of information conveyed by faces is a critical component of social communication. While the neurophysiology of processing upright faces has been studied extensively in autism spectrum disorder (ASD), less is known about the neurophysiological abnormalities associated with processing inverted faces in ASD. We used magnetoencephalography (MEG) to study both long-range and local functional connectivity, with the latter assessed using local cross-frequency coupling, in response to inverted faces stimuli, in 7-18 years old individuals with ASD and age and IQ matched typically developing (TD) individuals. We found abnormally reduced coupling between the phase of the alpha rhythm and the amplitude of the gamma rhythm in the fusiform face area (FFA) in response to inverted faces, as well as reduced long-range functional connectivity between the FFA and the inferior frontal gyrus (IFG) in response to inverted faces in the ASD group. These group differences were absent in response to upright faces. The magnitude of functional connectivity between the FFA and the IFG was significantly correlated with the severity of ASD, and FFA-IFG long-range functional connectivity increased with age in TD group, but not in the ASD group. Our findings suggest that both local and long-range functional connectivity are abnormally reduced in children with ASD when processing inverted faces, and that the pattern of abnormalities associated with the processing of inverted faces differs from the pattern of upright faces in ASD, likely due to the presumed greater reliance on top-down regulations necessary for efficient processing of inverted faces. LAY SUMMARY: We found alterations in the neurophysiological responses to inverted faces in children with ASD, that were not reflected in the evoked responses, and were not observed in the responses to upright faces. These alterations included reduced local functional connectivity in the fusiform face area (FFA), and decreased long-range alpha-band modulated functional connectivity between the FFA and the left IFG. The magnitude of long-range functional connectivity between the FFA and the inferior frontal gyrus was correlated with the severity of ASD.

Perceptual difficulty modulates the direction of information flow in familiar face recognition

Humans are fast and accurate when they recognize familiar faces. Previous neurophysiological studies have shown enhanced representations for the dichotomy of familiar vs. unfamiliar faces. As familiarity is a spectrum, however, any neural correlate should reflect graded representations for more vs. less familiar faces along the spectrum. By systematically varying familiarity across stimuli, we show a neural familiarity spectrum using electroencephalography. We then evaluated the spatiotemporal dynamics of familiar face recognition across the brain. Specifically, we developed a novel informational connectivity method to test whether peri-frontal brain areas contribute to familiar face recognition. Results showed that feed-forward flow dominates for the most familiar faces and top-down flow was only dominant when sensory evidence was insufficient to support face recognition. These results demonstrate that perceptual difficulty and the level of familiarity influence the neural representation of familiar faces and the degree to which peri-frontal neural networks contribute to familiar face recognition.

A conceptual critique of brain lateralization models in emotional face perception: Toward a hemispheric functional-equivalence (HFE) model

The present review proposes a novel dynamic model of brain lateralization of emotional (happy, surprised, fearful, sad, angry, and disgusted) and neutral face perception. Evidence to date suggests that emotional face perception is lateralized in the brain. At least five prominent hypotheses of the lateralization of emotional face perception have been previously proposed; the right-hemisphere hypothesis; the valence-specific hypothesis; the modified valence-specific hypothesis; the motivational hypothesis; and behavioral activation/inhibition system hypothesis. However, a growing number of recent replication studies exploring those hypotheses frequently provide inconsistent or even contradictory results. The latest neuroimaging and behavioral studies strongly demonstrate the functional capacity of both hemispheres to process emotions relatively successfully. Moreover, the flexibility of emotional brain-networks in both hemispheres is functionally high even to the extent of a possible reversed asymmetry of the left and the right hemisphere performance under altered neurophysiological and psychological conditions. The present review aims to a) provide a critical conceptual analysis of prior and current hypotheses of brain lateralization of emotional and neutral face perception; b) propose an integrative introduction of a novel hemispheric functional-equivalence (HFE) model in emotional and neutral face perception based on the evaluation of theoretical considerations, behavioral and neuroimaging studies: the brain is initially right-biased in emotional and neutral face perception by default; however, altered psychophysiological conditions (e.g., acute stress, a demanding emotional task) activate a distributed brain-network of both hemispheres toward functional equivalence that results in relatively equalized behavioral performance in emotional and neutral face perception. The proposed novel model may provide a practical tool in further experimental investigation of brain lateralization of emotional face perception.