Reliability of individual differences in neural face identity discrimination

Super recognizers: Increased sensitivity or reduced biases? Insights from serial dependence

Super recognizers (SRs) are people that exhibit a naturally occurring superiority for processing facial identity. Despite the increase of SR research, the mechanisms underlying their exceptional abilities remain unclear. Here, we investigated whether the enhanced facial identity processing of SRs could be attributed to the lack of sequential effects, such as serial dependence. In serial dependence, perception of stimulus features is assimilated toward stimuli presented in previous trials. This constant error in visual perception has been proposed as a mechanism that promotes perceptual stability in everyday life. We hypothesized that an absence of this constant source of error in SRs could account for their superior processing-potentially in a domain-general fashion. We tested SRs (n = 17) identified via a recently proposed diagnostic framework (Ramon, 2021) and age-matched controls (n = 20) with two experiments probing serial dependence in the face and shape domains. In each experiment, observers were presented with randomly morphed face identities or shapes and were asked to adjust a face's identity or a shape to match the stimulus they saw. We found serial dependence in controls and SRs alike, with no difference in its magnitude across groups. Interestingly, we found that serial dependence impacted the performance of SRs more than that of controls. Taken together, our results show that enhanced face identity processing skills in SRs cannot be attributed to the lack of serial dependence. Rather, serial dependence, a beneficial nested error in our visual system, may in fact further stabilize the perception of SRs and thus enhance their visual processing proficiency.

The neuropsychological evaluation of face identity recognition

Facial identity recognition (FIR) is arguably the ultimate form of recognition for the adult human brain. Even if the term prosopagnosia is reserved for exceptionally rare brain-damaged cases with a category-specific abrupt loss of FIR at adulthood, subjective and objective impairments or difficulties of FIR are common in the neuropsychological population. Here we provide a critical overview of the evaluation of FIR both for clinicians and researchers in neuropsychology. FIR impairments occur following many causes that should be identified objectively by both general and specific, behavioral and neural examinations. We refute the commonly used dissociation between perceptual and memory deficits/tests for FIR, since even a task involving the discrimination of unfamiliar face images presented side-by-side relies on cortical memories of faces in the right-lateralized ventral occipito-temporal cortex. Another frequently encountered confusion is between specific deficits of the FIR function and a more general impairment of semantic memory (of people), the latter being most often encountered following anterior temporal lobe damage. Many computerized tests aimed at evaluating FIR have appeared over the last two decades, as reviewed here. However, despite undeniable strengths, they often suffer from ecological limitations, difficulties of instruction, as well as a lack of consideration for processing speed and qualitative information. Taking into account these issues, a recently developed behavioral test with natural images manipulating face familiarity, stimulus inversion, and correct response times as a key variable appears promising. The measurement of electroencephalographic (EEG) activity in the frequency domain from fast periodic visual stimulation also appears as a particularly promising tool to complete and enhance the neuropsychological assessment of FIR.

The time course of stimulus-specific perceptual learning

Practice on perceptual tasks can lead to long-lasting, stimulus-specific improvements. Rapid stimulus-specific learning, assessed 24 hours after practice, has been found with just 105 practice trials in a face identification task. However, a much longer time course for stimulus-specific learning has been found in other tasks. Here, we examined 1) whether rapid stimulus-specific learning occurs for unfamiliar, non-face stimuli in a texture identification task; 2) the effects of varying practice across a range from just 21 trials up to 840 trials; and 3) if rapid, stimulus-specific learning persists over a 1-week, as well as a 1-day, interval. Observers performed a texture identification task in two sessions separated by one day (Experiment 1) or 1 week (Experiment 2). Observers received varying amounts of practice (21, 63, 105, or 840 training trials) in session 1 and completed 840 trials in session 2. In session 2, one-half of the observers in each group performed the task with the same textures as in session 1, and one-half switched to novel textures (same vs. novel conditions). In both experiments we found that stimulus-specific learning - defined as the difference in response accuracy in the same and novel conditions - increased as a linear function of the log number of session 1 training trials and was statistically significant after approximately 100 training trials. The effects of stimulus novelty did not differ across experiments. These results support the idea that stimulus-specific learning in our task arises gradually and continuously through practice, perhaps concurrently with general learning.

Differences in eye movements for face recognition between Canadian and Chinese participants are not modulated by social orientation

Face recognition strategies do not generalize across individuals. Many studies have reported robust cultural differences between West Europeans/North Americans and East Asians in eye movement strategies during face recognition. The social orientation hypothesis posits that individualistic vs. collectivistic (IND/COL) value systems, respectively defining West European/North American and East Asian societies, would be at the root of many cultural differences in visual perception. Whether social orientation is also responsible for such cultural contrast in face recognition remains to be clarified. To this aim, we conducted two experiments with West European/North American and Chinese observers. In Experiment 1, we probed the existence of a link between IND/COL social values and eye movements during face recognition, by using an IND/COL priming paradigm. In Experiment 2, we dissected the latter relationship in greater depth, by using two IND/COL questionnaires, including subdimensions to those concepts. In both studies, cultural differences in fixation patterns were revealed between West European/North American and East Asian observers. Priming IND/COL values did not modulate eye movement visual sampling strategies, and only specific subdimensions of the IND/COL questionnaires were associated with distinct eye-movement patterns. Altogether, we show that the typical contrast between IND/COL cannot fully account for cultural differences in eye movement strategies for face recognition. Cultural differences in eye movements for faces might originate from mechanisms distinct from social orientation.

Spatial Resolution Evaluation Based on Experienced Visual Categories With Sweep Evoked Periodic EEG Activity

Purpose

Visual function is typically evaluated in clinical settings with visual acuity (VA), a test requiring to behaviorally match or name optotypes such as tumbling E or Snellen letters. The ability to recognize these symbols has little in common with the automatic and rapid visual recognition of socially important stimuli in real life. Here we use sweep visual evoked potentials to assess spatial resolution objectively based on the recognition of human faces and written words.

Methods

To this end, we tested unfamiliar face individuation1 and visual word recognition2 in 15 normally sighted adult volunteers with a 68-electrode electroencephalogram system.

Results

Unlike previous measures of low-level visual function including VA, the most sensitive electrode was found at an electrode different from Oz in a majority of participants. Thresholds until which faces and words could be recognized were evaluated at the most sensitive electrode defined individually for each participant. Word recognition thresholds corresponded with the VA level expected from normally sighted participants, and even a VA significantly higher than expected from normally sighted individuals for a few participants.

Conclusions

Spatial resolution can be evaluated based on high-level stimuli encountered in day-to-day life, such as faces or written words with sweep visual evoked potentials.

Stimulus size modulates idiosyncratic neural face identity discrimination

Humans show individual differences in neural facial identity discrimination (FID) responses across viewing positions. Critically, these variations have been shown to be reliable over time and to directly relate to observers' idiosyncratic preferences in facial information sampling. This functional signature in facial identity processing might relate to observer-specific diagnostic information processing. Although these individual differences are a valuable source of information for interpreting data, they can also be difficult to isolate when it is not possible to test many conditions. To address this potential issue, we explored whether reducing stimulus size would help decrease these interindividual variations in neural FID. We manipulated the size of face stimuli (covering 3°, 5°, 6.7°, 8.5°, and 12° of visual angle), as well as the fixation location (left eye, right eye, below the nasion, nose, and mouth) while recording electrophysiological responses. Same identity faces were presented with a base frequency of 6 Hz. Different identity faces were periodically inserted within this sequence to trigger an objective index of neural FID. Our data show robust and consistent individual differences in neural face identity discrimination across viewing positions for all face sizes. Nevertheless, FID was optimal for a larger number of observers when faces subtended 6.7° of visual angle and fixation was below the nasion. This condition is the most suited to reduce natural interindividual variations in neural FID patterns, defining an important benchmark to measure neural FID when it is not possible to assess and control for observers' idiosyncrasies.

Frequency tagging with infants: The visual oddball paradigm

Combining frequency tagging with electroencephalography (EEG) provides excellent opportunities for developmental research and is increasingly employed as a powerful tool in cognitive neuroscience within the last decade. In particular, the visual oddball paradigm has been employed to elucidate face and object categorization and intermodal influences on visual perception. Still, EEG research with infants poses special challenges that require consideration and adaptations of analyses. These challenges include limits to attentional capacity, variation in looking times, and presence of artefacts in the EEG signal. Moreover, potential differences between age-groups must be carefully evaluated. This manuscript evaluates challenges theoretically and empirically by (1) a systematic review of frequency tagging studies employing the oddball paradigm and (2) combining and re-analyzing data from seven-month-old infants (N = 124, 59 females) collected in a categorization task with artifical, unfamiliar stimuli. Specifically, different criteria for sequence retention and selection of harmonics, the influence of bins considered for baseline correction and the relation between fast periodic visual stimulation (FPVS) responses and looking time are analyzed. Overall, evidence indicates that analysis decisions should be tailored based on age-group to optimally capture the observed signal. Recommendations for infant frequency tagging studies are developed to aid researchers in selecting appropriate stimulation and analysis strategies in future work.

Trustworthiness perception is mandatory: Task instructions do not modulate fast periodic visual stimulation trustworthiness responses

Although it is often assumed that humans spontaneously respond to the trustworthiness of others’ faces, it is still unclear whether responses to facial trust are mandatory or can be modulated by instructions. Considerable scientific interest lies in understanding whether trust processing is mandatory, given the societal consequences of biased trusting behavior. We tested whether neural responses indexing trustworthiness discrimination depended on whether the task involved focusing on facial trustworthiness or not, using a fast periodic visual stimulation electroencephalography oddball paradigm with a neural marker of trustworthiness discrimination at 1 Hz. Participants judged faces on size without any reference to trust, explicitly formed impressions of facial trust, or were given a financial lending context that primed trust, without explicit trust judgement instructions. Significant trustworthiness discrimination responses at 1 Hz were found in all three conditions, demonstrating the robust nature of trustworthiness discrimination at the neural level. Moreover, no effect of task instruction was observed, with Bayesian analyses providing moderate to decisive evidence that task instruction did not affect trustworthiness discrimination. Our finding that visual trustworthiness discrimination is mandatory points to the remarkable spontaneity of trustworthiness processing, providing clues regarding why these often unreliable impressions are ubiquitous.

Brain responses to lexical attestedness and phonological well-formedness as revealed by fast periodic visual stimulation

We investigated the mechanisms underlying the online-processing of phonological constraints using oddball fast-periodic visual stimulation coupled with EEG. We focused on the Sonority Sequencing Principle and examined whether steady-state visual evoked potentials (SSVEPs) are sensitive to the sonority constraint on syllable onsets. Native French speakers were presented with streams of CCVC non-words (C: consonant, V: vowel) at a fixed 6-Hz base rate. We manipulated the phonological well-formedness and lexical attestedness of CC onsets in two conditions. SSVPs were observed at the base rate associated to visual stimuli. As expected, they did not differ between conditions. Oddball SSVEPs were observed at 1.2 Hz (and its harmonics) and differed in the two conditions. These results showed that SSVEPs are sensitive to sublexical features. They also suggest that the processing of phonological constraints rely on mechanisms which could be dissociated from those underlying the processing of statistical properties of the lexicon.

Characteristic fixation biases in Super-Recognizers

Neurotypical observers show large and reliable individual differences in gaze behavior along several semantic object dimensions. Individual gaze behavior toward faces has been linked to face identity processing, including that of neurotypical observers. Here, we investigated potential gaze biases in Super-Recognizers (SRs), individuals with exceptional face identity processing skills. Ten SRs, identified with a novel conservative diagnostic framework, and 43 controls freely viewed 700 complex scenes depicting more than 5000 objects. First, we tested whether SRs and controls differ in fixation biases along four semantic dimensions: faces, text, objects being touched, and bodies. Second, we tested potential group differences in fixation biases toward eyes and mouths. Finally, we tested whether SRs fixate closer to the theoretical optimal fixation point for face identification. SRs showed a stronger gaze bias toward faces and away from text and touched objects, starting from the first fixation onward. Further, SRs spent a significantly smaller proportion of first fixations and dwell time toward faces on mouths but did not differ in dwell time or first fixations devoted to eyes. Face fixation of SRs also fell significantly closer to the theoretical optimal fixation point for identification, just below the eyes. Our findings suggest that reliable superiority for face identity processing is accompanied by early fixation biases toward faces and preferred saccadic landing positions close to the theoretical optimum for face identification. We discuss future directions to investigate the functional basis of individual fixation behavior and face identity processing ability.

Varying Stimulus Duration Reveals Consistent Neural Activity and Behavior for Human Face Individuation

Establishing consistent relationships between neural activity and behavior is a challenge in human cognitive neuroscience research. We addressed this issue using variable time constraints in an oddball frequency-sweep design for visual discrimination of complex images (face exemplars). Sixteen participants viewed sequences of ascending presentation durations, from 25 to 333 ms (40-3 Hz stimulation rate) while their electroencephalogram (EEG) was recorded. Throughout each sequence, the same unfamiliar face picture was repeated with variable size and luminance changes while different unfamiliar facial identities appeared every 1 s (1 Hz). A neural face individuation response, tagged at 1 Hz and its unique harmonics, emerged over the occipito-temporal cortex at 50 ms stimulus duration (25-100 ms across individuals), with an optimal response reached at 170 ms stimulus duration. In a subsequent experiment, identity changes appeared non-periodically within fixed-frequency sequences while the same participants performed an explicit face individuation task. The behavioral face individuation response also emerged at 50 ms presentation time, and behavioral accuracy correlated with individual participants' neural response amplitude in a weighted middle stimulus duration range (50-125 ms). Moreover, the latency of the neural response peaking between 180 and 200 ms correlated strongly with individuals' behavioral accuracy in this middle duration range, as measured independently. These observations point to the minimal (50 ms) and optimal (170 ms) stimulus durations for human face individuation and provide novel evidence that inter-individual differences in the magnitude and latency of early, high-level neural responses are predictive of behavioral differences in performance at this function.

Combined frequency-tagging EEG and eye-tracking measures provide no support for the "excess mouth/diminished eye attention" hypothesis in autism

BACKGROUND

Scanning faces is important for social interactions. Difficulty with the social use of eye contact constitutes one of the clinical symptoms of autism spectrum disorder (ASD). It has been suggested that individuals with ASD look less at the eyes and more at the mouth than typically developing (TD) individuals, possibly due to gaze aversion or gaze indifference. However, eye-tracking evidence for this hypothesis is mixed. While gaze patterns convey information about overt orienting processes, it is unclear how this is manifested at the neural level and how relative covert attention to the eyes and mouth of faces might be affected in ASD.

METHODS

We used frequency-tagging EEG in combination with eye tracking, while participants watched fast flickering faces for 1-min stimulation sequences. The upper and lower halves of the faces were presented at 6 Hz and 7.5 Hz or vice versa in different stimulation sequences, allowing to objectively disentangle the neural saliency of the eyes versus mouth region of a perceived face. We tested 21 boys with ASD (8-12 years old) and 21 TD control boys, matched for age and IQ.

RESULTS

Both groups looked longer at the eyes than the mouth, without any group difference in relative fixation duration to these features. TD boys looked significantly more to the nose, while the ASD boys looked more outside the face. EEG neural saliency data partly followed this pattern: neural responses to the upper or lower face half were not different between groups, but in the TD group, neural responses to the lower face halves were larger than responses to the upper part. Face exploration dynamics showed that TD individuals mostly maintained fixations within the same facial region, whereas individuals with ASD switched more often between the face parts.

LIMITATIONS

Replication in large and independent samples may be needed to validate exploratory results.

CONCLUSIONS

Combined eye-tracking and frequency-tagged neural responses show no support for the excess mouth/diminished eye gaze hypothesis in ASD. The more exploratory face scanning style observed in ASD might be related to their increased feature-based face processing style.

Normative data for two challenging tests of face matching under ecological conditions

Background

Unfamiliar face processing is an ability that varies considerably between individuals. Numerous studies have aimed to identify its underlying determinants using controlled experimental procedures. While such tests can isolate variables that influence face processing, they usually involve somewhat unrealistic situations and optimized face images as stimulus material. As a consequence, the extent to which the performance observed under laboratory settings is informative for predicting real-life proficiency remains unclear.

Results

We present normative data for two ecologically valid but underused tests of face matching: the Yearbook Test (YBT) and the Facial Identity Card Sorting Test (FICST). The YBT (n = 252) measures identity matching across substantial age-related changes in facial appearance, while the FICST (n = 218) assesses the ability to process unfamiliar facial identity despite superficial image variations. To determine the predictive value of both tests, a subsample of our cohort (n = 181) also completed a commonly used test of face recognition and two tests of face perception (the long form of the Cambridge Face Memory Test (CFMT+), the Expertise in Facial Comparison Test (EFCT) and the Person Identification Challenge Test (PICT)).

Conclusions

Focusing on the top performers identified independently per test, we made two important observations: 1) YBT and FICST performance can predict CFMT+ scores and vice versa; and 2) EFCT and PICT scores neither reliably predict superior performance in ecologically meaningful and challenging tests of face matching, nor in the most commonly used test of face recognition. These findings emphasize the necessity for using challenging and ecologically relevant, and thus highly sensitive, tasks of unfamiliar face processing to identify high-performing individuals in the normal population.