Prioritized Detection of Personally Familiar Faces

Consciousness makes sense in the light of evolution

I believe consciousness is a property of advanced nervous systems, and as such a product of evolution. Thus, to understand consciousness we need to describe the trajectory leading to its evolution and the selective advantages conferred. A deeper understanding of the neurology would be a significant contribution, but other advanced functions, such as hearing and vision, are explained with a comparable lack of detailed knowledge of the brain processes responsible. In this paper, I try to add details and credence to a previously suggested, evolution-based model of consciousness. According to this model, the feature started to evolve in early amniotes (reptiles, birds, and mammals) some 320 million years ago. The reason was the introduction of feelings as a strategy for making behavioral decisions.

Pupil dilation reflects covert familiar face recognition under interocular suppression

In prosopagnosia, brain lesions impair overt face recognition, but not face detection, and may coexist with residual covert recognition of familiar faces. Previous studies that simulated covert recognition in healthy individuals have impaired face detection as well as recognition, thus not fully mirroring the deficits in prosopagnosia. We evaluated a model of covert recognition based on continuous flash suppression (CFS). Familiar and unfamiliar faces and houses were masked while participants performed two discrimination tasks. With increased suppression, face/house discrimination remained largely intact, but face familiarity discrimination deteriorated. Covert recognition was present across all masking levels, evinced by higher pupil dilation to familiar than unfamiliar faces. Pupil dilation was uncorrelated with overt performance across subjects. Thus, CFS can impede overt face recognition without disrupting covert recognition and face detection, mirroring critical features of prosopagnosia. CFS could be used to uncover shared neural mechanisms of covert recognition in prosopagnosic patients and neurotypicals.

Binocularly suppressed stimuli induce brain activities related to aesthetic emotions

Introduction

Aesthetic emotions are a class of emotions aroused by evaluating aesthetically appealing objects or events. While evolutionary aesthetics suggests the adaptive roles of these emotions, empirical assessments are lacking. Previous neuroscientific studies have demonstrated that visual stimuli carrying evolutionarily important information induce neural responses even when presented non-consciously. To examine the evolutionary importance of aesthetic emotions, we conducted a neuroscientific study using magnetoencephalography (MEG) to measure induced neural responses to non-consciously presented portrait paintings categorised as biological and non-biological and examined associations between the induced responses and aesthetic ratings.

Methods

MEG and pre-rating data were collected from 23 participants. The pre-rating included visual analogue scales for object saliency, facial saliency, liking, and beauty scores, in addition to 'biologi-ness,' which was used for subcategorising stimuli into biological and non-biological. The stimuli were presented non-consciously using a continuous flash suppression paradigm or consciously using binocular presentation without flashing masks, while dichotomic behavioural responses were obtained (beauty or non-beauty). Time-frequency decomposed MEG data were used for correlation analysis with pre-rating scores for each category.

Results

Behavioural data revealed that saliency scores of non-consciously presented stimuli influenced dichotomic responses (beauty or non-beauty). MEG data showed that non-consciously presented portrait paintings induced spatiotemporally distributed low-frequency brain activities associated with aesthetic ratings, which were distinct between the biological and non-biological categories and conscious and non-conscious conditions.

Conclusion

Aesthetic emotion holds evolutionary significance for humans. Neural pathways are sensitive to visual images that arouse aesthetic emotion in distinct ways for biological and non-biological categories, which are further influenced by consciousness. These differences likely reflect the diversity in mechanisms of aesthetic processing, such as processing fluency, active elaboration, and predictive processing. The aesthetic processing of non-conscious stimuli appears to be characterised by fluency-driven affective processing, while top-down regulatory processes are suppressed. This study provides the first empirical evidence supporting the evolutionary significance of aesthetic processing.

Spatial perspective and identity in visual awareness of the bodily self-other distinction

Spatial perspective and identity of visual bodily stimuli are two key cues for the self-other distinction. However, how they emerge into visual awareness is largely unknown. Here, self- or other-hands presented in first- or third-person perspective were compared in a breaking-Continuous Flash Suppression paradigm (Experiment 1) measuring the time the stimuli need to access visual awareness, and in a Binocular Rivalry paradigm (Experiment 2), measuring predominance in perceptual awareness. Results showed that, irrespectively of identity, first-person perspective speeded up the access, whereas the third-person one increased the dominance. We suggest that the effect of first-person perspective represents an unconscious prioritization of an egocentric body coding important for visuomotor control. On the other hand, the effect of third-person perspective indicates a conscious advantage of an allocentric body representation fundamental for detecting the presence of another intentional agent. Summarizing, the emergence of self-other distinction into visual awareness would strongly depend on the interplay between spatial perspectives, with an inverse prioritization before and after conscious perception. On the other hand, identity features might rely on post-perceptual processes.

Role of facial familiarity and emotional expression intensity in ensemble emotion perception

When looking at groups of people, we can extract information from the different faces to derive properties of the group, such as its average facial emotion, although how this average is computed remains a matter of debate. Here, we examined whether our participants' personal familiarity with the faces in the group, as well as the intensity of the facial expressions, biased ensemble perception. Participants judged the average emotional expression of ensembles of four different identities whose expressions depicted either neutral, angry, or happy emotions. For the angry and happy expressions, the intensity of the emotion could be either low (e.g., slightly happy) or high (very happy). When all the identities in the ensemble were unfamiliar, the presence of any high intensity emotional face biased ensemble perception towards its emotion. However, when a familiar face was present in the ensemble, perception was biased towards the familiar face's emotion regardless of its intensity. These findings reveal that how we perceive the average emotion of a group is influenced by both the emotional intensity and familiarity of the faces comprising the group, supporting the idea that different faces may be weighted differently in ensemble perception. These findings have important implications for the judgements we make about a group's overall emotional state may be biased by individuals within the group.

Familiar objects benefit more from transsaccadic feature predictions

The transsaccadic feature prediction mechanism associates peripheral and foveal information belonging to the same object to make predictions about how an object seen in the periphery would appear in the fovea or vice versa. It is unclear if such transsaccadic predictions require experience with the object such that only familiar objects benefit from this mechanism by virtue of having peripheral-foveal associations. In two experiments, we tested whether familiar objects have an advantage over novel objects in peripheral-foveal matching and transsaccadic change detection tasks. In both experiments, observers were unknowingly familiarized with a small set of stimuli by completing a sham orientation change detection task. In the first experiment, observers subsequently performed a peripheral-foveal matching task, where they needed to pick the foveal test object that matched a briefly presented peripheral target. In the second experiment, observers subsequently performed a transsaccadic object change detection task where a peripheral target was exchanged or not exchanged with another target after the saccade, either immediately or after a 300-ms blank period. We found an advantage of familiar objects over novel objects in both experiments. While foveal-peripheral associations explained the familiarity effect in the matching task of the first experiment, the second experiment provided evidence for the advantage of peripheral-foveal associations in transsaccadic object change detection. Introducing a postsaccadic blank improved change detection performance in general but more for familiar than for novel objects. We conclude that familiar objects benefit from additional object-specific predictions.

The effect of familiarity on behavioral oscillations in face perception

Studies on behavioral oscillations demonstrate that visual sensitivity fluctuates over time and visual processing varies periodically, mirroring neural oscillations at the same frequencies. Do these behavioral oscillations reflect fixed and relatively automatic sensory sampling, or top-down processes such as attention or predictive coding? To disentangle these theories, the current study used a dual-target rapid serial visual presentation paradigm, where participants indicated the gender of a face target embedded in streams of distractors presented at 30 Hz. On critical trials, two identical targets were presented with varied stimulus onset asynchrony from 200 to 833 ms. The target was either familiar or unfamiliar faces, divided into different blocks. We found a 4.6 Hz phase-coherent fluctuation in gender discrimination performance across both trial types, consistent with previous reports. In addition, however, we found an effect at the alpha frequency, with behavioral oscillations in the familiar blocks characterized by a faster high-alpha peak than for the unfamiliar face blocks. These results are consistent with the combination of both a relatively stable modulation in the theta band and faster modulation of the alpha oscillations. Therefore, the overall pattern of perceptual sampling in visual perception may depend, at least in part, on task demands. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on 16/08/2022. The protocol, as accepted by the journal, can be found at: https://doi.org/10.17605/OSF.IO/A98UF .

Familiarity Facilitates Detection of Angry Expressions

Personal familiarity facilitates rapid and optimized detection of faces. In this study, we investigated whether familiarity associated with faces can also facilitate the detection of facial expressions. Models of face processing propose that face identity and face expression detection are mediated by distinct pathways. We used a visual search paradigm to assess if facial expressions of emotion (anger and happiness) were detected more rapidly when produced by familiar as compared to unfamiliar faces. We found that participants detected an angry expression 11% more accurately and 135 ms faster when produced by familiar as compared to unfamiliar faces while happy expressions were detected with equivalent accuracies and at equivalent speeds for familiar and unfamiliar faces. These results suggest that detectors in the visual system dedicated to processing features of angry expressions are optimized for familiar faces.

Humans, fish, spiders and bees inherited working memory and attention from their last common ancestor

All brain processes that generate behaviour, apart from reflexes, operate with information that is in an "activated" state. This activated information, which is known as working memory (WM), is generated by the effect of attentional processes on incoming information or information previously stored in short-term or long-term memory (STM or LTM). Information in WM tends to remain the focus of attention; and WM, attention and STM together enable information to be available to mental processes and the behaviours that follow on from them. WM and attention underpin all flexible mental processes, such as solving problems, making choices, preparing for opportunities or threats that could be nearby, or simply finding the way home. Neither WM nor attention are necessarily conscious, and both may have evolved long before consciousness. WM and attention, with similar properties, are possessed by humans, archerfish, and other vertebrates; jumping spiders, honey bees, and other arthropods; and members of other clades, whose last common ancestor (LCA) is believed to have lived more than 600 million years ago. It has been reported that very similar genes control the development of vertebrate and arthropod brains, and were likely inherited from their LCA. Genes that control brain development are conserved because brains generate adaptive behaviour. However, the neural processes that generate behaviour operate with the activated information in WM, so WM and attention must have existed prior to the evolution of brains. It is proposed that WM and attention are widespread amongst animal species because they are phylogenetically conserved mechanisms that are essential to all mental processing, and were inherited from the LCA of vertebrates, arthropods, and some other animal clades.

The importance of awareness in face processing: A critical review of interocular suppression studies

Human faces convey essential information for understanding others' mental states and intentions. The importance of faces in social interaction has prompted suggestions that some relevant facial features such as configural information, emotional expression, and gaze direction may promote preferential access to awareness. This evidence has predominantly come from interocular suppression studies, with the most common method being the Breaking Continuous Flash Suppression (bCFS) procedure, which measures the time it takes different stimuli to overcome interocular suppression. However, the procedures employed in such studies suffer from multiple methodological limitations. For example, they are unable to disentangle detection from identification processes, their results may be confounded by participants' response bias and decision criteria, they typically use small stimulus sets, and some of their results attributed to detecting high-level facial features (e.g., emotional expression) may be confounded by differences in low-level visual features (e.g., contrast, spatial frequency). In this article, we review the evidence from the bCFS procedure on whether relevant facial features promote access to awareness, discuss the main limitations of this very popular method, and propose strategies to address these issues.

Top-down modulation and cortical-AMG/HPC interaction in familiar face processing

Humans can accurately recognize familiar faces in only a few hundred milliseconds, but the underlying neural mechanism remains unclear. Here, we recorded intracranial electrophysiological signals from ventral temporal cortex (VTC), superior/middle temporal cortex (STC/MTC), medial parietal cortex (MPC), and amygdala/hippocampus (AMG/HPC) in 20 epilepsy patients while they viewed faces of famous people and strangers as well as common objects. In posterior VTC and MPC, familiarity-sensitive responses emerged significantly later than initial face-selective responses, suggesting that familiarity enhances face representations after they are first being extracted. Moreover, viewing famous faces increased the coupling between cortical areas and AMG/HPC in multiple frequency bands. These findings advance our understanding of the neural basis of familiar face perception by identifying the top-down modulation in local face-selective response and interactions between cortical face areas and AMG/HPC.

Ingroup and outgroup differences in face detection

Humans show improved recognition for faces from their own social group relative to faces from another social group. Yet before faces can be recognized, they must first be detected in the visual field. Here, we tested whether humans also show an ingroup bias at the earliest stage of face processing - the point at which the presence of a face is first detected. To this end, we measured viewers' ability to detect ingroup (Black and White) and outgroup faces (Asian, Black, and White) in everyday scenes. Ingroup faces were detected with greater speed and accuracy relative to outgroup faces (Experiment 1). Removing face hue impaired detection generally, but the ingroup detection advantage was undiminished (Experiment 2). This same pattern was replicated by a detection algorithm using face templates derived from human data (Experiment 3). These findings demonstrate that the established ingroup bias in face processing can extend to the early process of detection. This effect is 'colour blind', in the sense that group membership effects are independent of general effects of image hue. Moreover, it can be captured by tuning visual templates to reflect the statistics of observers' social experience. We conclude that group bias in face detection is both a visual and a social phenomenon.

Holistic processing of gaze cues during interocular suppression

Direct eye contact is preferentially processed over averted gaze and has been shown to gain privileged access to conscious awareness during interocular suppression. This advantage might be driven by local features associated with direct gaze, such as the amount of visible sclera. Alternatively, a holistic representation of gaze direction, which depends on the integration of head and eye information, might drive the effects. Resolving this question is interesting because it speaks to whether the processing of higher-level social information in the visual system, such as facial characteristics that rely on holistic processing, is dependent on conscious awareness. The Wollaston Illusion is a visual illusion that allows researchers to manipulate perceived gaze direction while keeping local eye features constant. Here we used this illusion to elucidate the driving factor facilitating the direct gaze advantage during interocular suppression. Using continuous flash suppression, we rendered Wollaston faces with direct and averted gaze (initially) invisible. These faces conveyed different gaze directions but contained identical eye regions. Our results showed clear evidence for a direct gaze advantage with Wollaston faces, indicating that holistic representations of gaze direction may drive the direct gaze advantage during interocular suppression.

Gaze direction and face orientation modulate perceptual sensitivity to faces under interocular suppression

Faces convey information essential for social interaction. Their importance has prompted suggestions that some facial features may be processed unconsciously. Although some studies have provided empirical support for this idea, it remains unclear whether these findings were due to perceptual processing or to post-perceptual decisional factors. Evidence for unconscious processing of facial features has predominantly come from the Breaking Continuous Flash Suppression (b-CFS) paradigm, which measures the time it takes different stimuli to overcome interocular suppression. For example, previous studies have found that upright faces are reported faster than inverted faces, and direct-gaze faces are reported faster than averted-gaze faces. However, this procedure suffers from important problems: observers can decide how much information they receive before committing to a report, so their detection responses may be influenced by differences in decision criteria and by stimulus identification. Here, we developed a new procedure that uses predefined exposure durations, enabling independent measurement of perceptual sensitivity and decision criteria. We found higher detection sensitivity to both upright and direct-gaze (compared to inverted and averted-gaze) faces, with no effects on decisional factors. For identification, we found both greater sensitivity and more liberal criteria for upright faces. Our findings demonstrate that face orientation and gaze direction influence perceptual sensitivity, indicating that these facial features may be processed unconsciously.

Shared neural codes for visual and semantic information about familiar faces in a common representational space

Our brain processes faces of close others differently than faces of visually familiar individuals. While both types of faces activate similar visual areas, faces of close others activate areas involved in processing social and semantic information. Here, we used between-subject linear classifiers trained on hyperaligned brain data to investigate the neural code for visual and semantic information about familiar others. The identity of both visually and personally familiar faces could be decoded across participants from brain activity in visual areas. Instead, only the identity of personally familiar faces could be decoded in areas involved in social cognition. Our results suggest that individually distinctive information associated with familiar faces is embedded in a neural code that is shared across brains.

Processes evoked by seeing a personally familiar face encompass recognition of visual appearance and activation of social and person knowledge. Whereas visual appearance is the same for all viewers, social and person knowledge may be more idiosyncratic. Using between-subject multivariate decoding of hyperaligned functional magnetic resonance imaging data, we investigated whether representations of personally familiar faces in different parts of the distributed neural system for face perception are shared across individuals who know the same people. We found that the identities of both personally familiar and merely visually familiar faces were decoded accurately across brains in the core system for visual processing, but only the identities of personally familiar faces could be decoded across brains in the extended system for processing nonvisual information associated with faces. Our results show that personal interactions with the same individuals lead to shared neural representations of both the seen and unseen features that distinguish their identities.

Electrophysiological Chronometry of Graded Consciousness during the Attentional Blink

One of the ongoing debates about visual consciousness is whether it can be considered as an all-or-none or a graded phenomenon. While there is increasing evidence for the existence of graded states of conscious awareness based on paradigms such as visual masking, only little and mixed evidence is available for the attentional blink paradigm, specifically in regard to electrophysiological measures. Thereby, the all-or-none pattern reported in some attentional blink studies might have originated from specifics of the experimental design, suggesting the need to examine the generalizability of results. In the present event-related potential (ERP) study (N = 32), visual awareness of T2 face targets was assessed via subjective visibility ratings on a perceptual awareness scale in combination with ERPs time-locked to T2 onset (components P1, N1, N2, and P3). Furthermore, a classification task preceding visibility ratings allowed to track task performance. The behavioral results indicate a graded rather than an all-or-none pattern of visual awareness. Corresponding graded differences in the N1, N2, and P3 components were observed for the comparison of visibility levels. These findings suggest that conscious perception during the attentional blink can occur in a graded fashion.

Decoding category and familiarity information during visual imagery

Visual imagery relies on a widespread network of brain regions, partly engaged during the perception of external stimuli. Beyond the recruitment of category-selective areas (FFA, PPA), perception of familiar faces and places has been reported to engage brain areas associated with semantic information, comprising the precuneus, temporo-parietal junction (TPJ), medial prefrontal cortex (mPFC) and posterior cingulate cortex (PCC). Here we used multivariate pattern analyzes (MVPA) to examine to which degree areas of the visual imagery network, category-selective and semantic areas contain information regarding the category and familiarity of imagined stimuli. Participants were instructed via auditory cues to imagine personally familiar and unfamiliar stimuli (i.e. faces and places). Using region-of-interest (ROI)-based MVPA, we were able to distinguish between imagined faces and places within nodes of the visual imagery network (V1, SPL, aIPS), within category-selective inferotemporal regions (FFA, PPA) and across all brain regions of the extended semantic network (i.e. precuneus, mPFC, IFG and TPJ). Moreover, we were able to decode familiarity of imagined stimuli in the SPL and aIPS, and in some regions of the extended semantic network (in particular, right precuneus, right TPJ), but not in V1. Our results suggest that posterior visual areas - including V1 - host categorical representations about imagined stimuli, and that stimulus familiarity might be an additional aspect that is shared between perception and visual imagery.

A fast link between face perception and memory in the temporal pole

The question of how the brain recognizes the faces of familiar individuals has been important throughout the history of neuroscience. Cells linking visual processing to person memory have been proposed but not found. Here, we report the discovery of such cells through recordings from an area in the macaque temporal pole identified with functional magnetic resonance imaging. These cells responded to faces that were personally familiar. They responded nonlinearly to stepwise changes in face visibility and detail and holistically to face parts, reflecting key signatures of familiar face recognition. They discriminated between familiar identities, as fast as a general face identity area. The discovery of these cells establishes a new pathway for the fast recognition of familiar individuals.

Pupillary Responses to Faces Are Modulated by Familiarity and Rewarding Context

Observing familiar (known, recognisable) and socially relevant (personally important) faces elicits activation in the brain’s reward circuit. Although smiling faces are often used as social rewards in research, it is firstly unclear whether familiarity and social relevance modulate the processing of faces differently, and secondly whether this processing depends on the feedback context, i.e., if it is different when smiles are delivered depending on performance or in the absence of any action (passive viewing). In this preregistered study, we compared pupillary responses to smiling faces differing in subjective familiarity and social relevance. They were displayed in a passive viewing task and in an active task (a speeded visual short-term memory task). The pupils were affected only in the active task and only by subjective familiarity. Contrary to expectations, smaller dilations were observed in response to more familiar faces. Behavioural ratings supported the superior rewarding context of the active task, with higher reward ratings for the game than the passive task. This study offers two major insights. Firstly, familiarity plays a role in the processing of social rewards, as known and unknown faces influence the autonomic responses differently. Secondly, the feedback context is crucial in reward research as positive stimuli are rewarding when they are dependent on performance.

Rapid Neural Representations of Personally Relevant Faces

The faces of those most personally relevant to us are our primary source of social information, making their timely perception a priority. Recent research indicates that gender, age and identity of faces can be decoded from EEG/MEG data within 100 ms. Yet, the time course and neural circuitry involved in representing the personal relevance of faces remain unknown. We applied simultaneous EEG-fMRI to examine neural responses to emotional faces of female participants' romantic partners, friends, and a stranger. Combining EEG and fMRI in cross-modal representational similarity analyses, we provide evidence that representations of personal relevance start prior to structural encoding at 100 ms, with correlated representations in visual cortex, but also in prefrontal and midline regions involved in value representation, and monitoring and recall of self-relevant information. Our results add to an emerging body of research that suggests that models of face perception need to be updated to account for rapid detection of personal relevance in cortical circuitry beyond the core face processing network.

Continuous flash suppression: Known and unknowns

Studies utilizing continuous flash suppression (CFS) provide valuable information regarding conscious and nonconscious perception. There are, however, crucial unanswered questions regarding the mechanisms of suppression and the level of visual processing in the absence of consciousness with CFS. Research suggests that the answers to these questions depend on the experimental configuration and how we assess consciousness in these studies. The aim of this review is to evaluate the impact of different experimental configurations and the assessment of consciousness on the results of the previous CFS studies. We review studies that evaluated the influence of different experimental configuration on the depth of suppression with CFS and discuss how different assessments of consciousness may impact the results of CFS studies. Finally, we review behavioral and brain recording studies of CFS. In conclusion, previous studies provide evidence for survival of low-level visual information and complete impairment of high-level visual information under the influence of CFS. That is, studies suggest that nonconscious perception of lower-level visual information happens with CFS, but there is no evidence for nonconscious high-level recognition with CFS.

Synesthesia does not help to recover perceptual dominance following flash suppression

Grapheme-colour synesthesia occurs when letters or numbers elicit an abnormal colour sensation (e.g., printed black letters are perceived as coloured). This phenomenon is typically reported following explicit presentation of graphemes. Very few studies have investigated colour sensations in synesthesia in the absence of visual awareness. We took advantage of the dichoptic flash suppression paradigm to temporarily render a stimulus presented to one eye invisible. Synesthetic alphanumeric and non-synesthetic stimuli were presented to 21 participants (11 synesthetes) in achromatic and chromatic experimental conditions. The test stimulus was first displayed to one eye and then masked by a sudden presentation of visual noise in the other eye (flash suppression). The time for an image to be re-perceived following the onset of the suppressive noise was calculated. Trials where there was no flash suppression performed but instead mimicked the perceptual suppression of the flash were also tested. Results showed that target detection by synesthetes was significantly better than by controls in the absence of flash suppression. No difference was found between the groups in the flash suppression condition. Our findings suggest that synesthesia is associated with enhanced perception for overt recognition, but does not provide an advantage in recovering from a perceptual suppression. Further studies are needed to investigate synesthesia in relation to visual awareness.

Prosopagnosia and disorders of face processing

Face recognition is a form of expert visual processing. Acquired prosopagnosia is the loss of familiarity for facial identity and has several functional variants, namely apperceptive, amnestic, and associative forms. Acquired forms are usually caused by either occipitotemporal or anterior temporal lesions, right or bilateral in most cases. In addition, there is a developmental form, whose functional and structural origins are still being elucidated. Despite their difficulties with recognizing faces, some of these subjects still show signs of covert recognition, which may have a number of explanations. Other aspects of face perception can be spared in prosopagnosic subjects. Patients with other types of face processing difficulties have been described, including impaired expression processing, impaired lip-reading, false familiarity for faces, and a people-specific amnesia. Recent rehabilitative studies have shown some modest ability to improve face perception in prosopagnosic subjects through perceptual training protocols.

The self-face captures attention without consciousness: Evidence from the N2pc ERP component analysis

It is well established that stimuli representing or associated with ourselves, like our own name or an image of our own face, benefit from preferential processing. However, two key questions concerning the self-prioritization mechanism remain to be addressed. First, does it operate in an automatic manner during the early processing, or rather in a more controlled fashion at later processing stages? Second, is it specific to the self-related stimuli, or can it be activated also by other stimuli that are familiar or salient? We conducted a dot-probe experiment to investigate the mechanism behind the attentional prioritization of the self-face image and to tackle both questions. The former, by employing a backwards masking procedure to isolate the early and preconscious processing stages. The latter, by investigating whether a face that becomes visually familiar due to repeated presentations is able to capture attention in a similar manner as the self-face. Analysis of the N2pc ERP component revealed that the self-face image automatically captures attention, both when processed consciously and unconsciously. In contrast, the visually familiar face did not attract attention, neither in the conscious, nor in the unconscious condition. We conclude that the self-prioritization mechanism is early and automatic, and is not triggered by mere visual familiarity. More generally, our results provide further evidence for efficient unconscious processing of faces, and for dissociation between attention and consciousness.

Eye spy a liar: assessing the utility of eye fixations and confidence judgments for detecting concealed recognition of faces, scenes and objects

BACKGROUND

In criminal investigations, uncooperative witnesses might deny knowing a perpetrator, the location of a murder scene or knowledge of a weapon. We sought to identify markers of recognition in eye fixations and confidence judgments whilst participants told the truth and lied about recognising faces (Experiment 1) and scenes and objects (Experiment 2) that varied in familiarity. To detect recognition we calculated effect size differences in markers of recognition between familiar and unfamiliar items that varied in familiarity (personally familiar, newly learned).

RESULTS

In Experiment 1, recognition of personally familiar faces was reliably detected across multiple fixation markers (e.g. fewer fixations, fewer interest areas viewed, fewer return fixations) during honest and concealed recognition. In Experiment 2, recognition of personally familiar non-face items (scenes and objects) was detected solely by fewer fixations during honest and concealed recognition; differences in other fixation measures were not consistent. In both experiments, fewer fixations exposed concealed recognition of newly learned faces, scenes and objects, but the same pattern was not observed during honest recognition. Confidence ratings were higher for recognition of personally familiar faces than for unfamiliar faces.

CONCLUSIONS

Robust memories of personally familiar faces were detected in patterns of fixations and confidence ratings, irrespective of task demands required to conceal recognition. Crucially, we demonstrate that newly learned faces should not be used as a proxy for real-world familiarity, and that conclusions should not be generalised across different types of familiarity or stimulus class.

The developmental origins of subliminal face processing

Sensitive responding to facial information is of key importance during human social interactions. Research shows that adults glean much information from another person's face without conscious perception, attesting to the robustness of face processing in the service of adaptive social functioning. Until recently, it was unclear whether such subliminal face processing is an outcome of extensive learning, resulting in adult face processing skills, or an early defining feature of human face processing. Here, we review recent research examining the early ontogeny and brain correlates of subliminal face processing, demonstrating that subliminal face processing: (1) emerges during the first year of life; (2) is multifaceted in response to transient (gaze, emotion) and stable (trustworthiness) facial cues; (3) systematically elicits frontal brain responses linked to attention allocation. The synthesized research suggests that subliminal face processing emerges early in human development and thus may play a foundational role during human social interactions. This offers a fresh look at the ontogenetic origins of unconscious face processing and informs theoretical accounts of human sociality.

Hemodynamic response to familiar faces in children with ADHD

Background

School-age children with attention deficit hyperactivity disorder (ADHD) have difficulties in interpersonal relationships, in addition to impaired facial expression perception and recognition. For successful social interactions, the ability to discriminate between familiar and unfamiliar faces is critical. However, there are no published reports on the recognition of familiar and unfamiliar faces by children with ADHD.

Methods

We evaluated the neural correlates of familiar and unfamiliar facial recognition in children with ADHD compared to typically developing (TD) children. We used functional near-infrared spectroscopy (fNIRS) to measure hemodynamic responses on the bilateral temporal regions while participants looked at photographs of familiar and unfamiliar faces. Nine boys with ADHD and 14 age-matched TD boys participated in the study. fNIRS data were Z-scored prior to analysis.

Results

During familiar face processing, TD children only showed significant activity in the late phase, while ADHD children showed significant activity in both the early and late phases. Additionally, the boys with ADHD did not show right hemispheric lateralization to familiar faces.

Conclusions

This study is the first to assess brain activity during familiar face processing in boys with ADHD using fNIRS. These findings of atypical patterns of brain activity in boys with ADHD may be related to social cognitive impairments from ADHD.

Eye see through you! Eye tracking unmasks concealed face recognition despite countermeasures

Background

Criminal associates such as terrorist members are likely to deny knowing members of their network when questioned by police. Eye tracking research suggests that lies about familiar faces can be detected by distinct markers of recognition (e.g. fewer fixations and longer fixation durations) across multiple eye fixation parameters. However, the effect of explicit eye movement strategies to concealed recognition on such markers has not been examined. Our aim was to assess the impact of fixed-sequence eye movement strategies (across the forehead, ears, eyes, nose, mouth and chin) on markers of familiar face recognition. Participants were assigned to one of two groups: a standard guilty group who were simply instructed to conceal knowledge but with no specific instructions on how to do so; and a countermeasures group who were instructed to look at every familiar and unfamiliar face in the same way by executing a consistent sequence of fixations.

Results

In the standard guilty group, lies about recognition of familiar faces showed longer average fixation durations, a lower proportion of fixations to the inner face regions, and proportionately more viewing of the eyes than honest responses to genuinely unknown faces. In the countermeasures condition, familiar face recognition was detected by longer fixations durations, fewer fixations to the inner regions of the face, and fewer interest areas of the face viewed. Longer fixation durations were a consistent marker of recognition across both conditions for most participants; differences were detectable from the first fixation.

Conclusion

The results suggest that individuals can exert a degree of executive control over fixation patterns but that: the eyes are particularly attention-grabbing for familiar faces; the more viewers look around the face, the more they give themselves away; and attempts to deploy the same fixation patterns to familiar and unfamiliar faces were unsuccessful. The results suggest that the best strategy for concealing recognition might be to keep the eyes fixated in the centre of the screen but, even then, recognition is apparent in longer fixation durations. We discuss potential optimal conditions for detecting concealed knowledge of faces.

Electronic supplementary material

The online version of this article (10.1186/s41235-019-0169-0) contains supplementary material, which is available to authorized users.

Modelling face memory reveals task-generalizable representations

Current cognitive theories are cast in terms of information-processing mechanisms that use mental representations^1-4. For example, people use their mental representations to identify familiar faces under various conditions of pose, illumination and ageing, or to draw resemblance between family members. Yet, the actual information contents of these representations are rarely characterized, which hinders knowledge of the mechanisms that use them. Here, we modelled the three-dimensional representational contents of 4 faces that were familiar to 14 participants as work colleagues. The representational contents were created by reverse-correlating identity information generated on each trial with judgements of the face's similarity to the individual participant's memory of this face. In a second study, testing new participants, we demonstrated the validity of the modelled contents using everyday face tasks that generalize identity judgements to new viewpoints, age and sex. Our work highlights that such models of mental representations are critical to understanding generalization behaviour and its underlying information-processing mechanisms.

Familiarity revealed by involuntary eye movements on the fringe of awareness

Involuntary eye movements during fixation of gaze are typically transiently inhibited following stimulus onset. This oculomotor inhibition (OMI), which includes microsaccades and spontaneous eye blinks, is modulated by stimulus saliency and anticipation, but it is currently unknown whether it is sensitive to familiarity. To investigate this, we measured the OMI while observers passively viewed a slideshow of one familiar and 7 unfamiliar facial images presented briefly at 1 Hz in random order. Since the initial experiments indicated that OMI was occasionally insensitive to familiarity when the facial images were highly visible, and to prevent top-down strategies and potential biases, we limited visibility by backward masking making the faces barely visible or at the fringe of awareness. Under these conditions, we found prolonged inhibition of both microsaccades and eye-blinks, as well as earlier onset of microsaccade inhibition with familiarity. These findings demonstrate, for the first time, the sensitivity of OMI to familiarity. Because this is based on involuntary eye movements and can be measured on the fringe of awareness and in passive viewing, our results provide direct evidence that OMI can be used as a novel physiological measure for studying hidden memories with potential implications for health, legal, and security purposes.

One of us? how facial and symbolic cues to own- versus other-race membership influence access to perceptual awareness

Information that conveys racial group membership plays a powerful role in influencing people's information processing including perceptual, memory and evaluative judgments. Yet whether own- and other-race information can differentially impact people's perceptual awareness at a preconscious level remains unclear. Employing a breaking continuous flash suppression (b-CFS) paradigm, we investigated whether compared with other-race stimuli, participants' own-race stimuli would be prioritized to gain privileged access to perceptual awareness. Across five experiments (N = 136), we firstly found that participants' own-race faces enjoyed privileged access to perceptual awareness (Experiment 1). In Experiments 2-5, we employed an associative training task to establish associations between otherwise arbitrary visual stimuli and own- vs. other-racial groups. Although otherwise arbitrary visual stimuli were prioritized to represent one's own race (vs. other-race) during the training, own- and other-race representing stimuli did not differ in their potency in entering perceptual awareness. This dissociation was further corroborated by Bayesian analyses and an internal meta-analysis. Taken together, our findings suggest that people's perceptual expertise with their own-race members' faces plays a determining role in shaping perceptual awareness. In contrast, newly learned race-representing stimuli did not influence early perceptual selection processes as indicated by the time they take to emerge into perceptual awareness.

Pupil Size as a Gateway Into Conscious Interpretation of Brightness

Although retinal illumination is the main determinant of pupil size, evidence indicates that extra-retinal factors, including attention and contextual information, also modulate the pupillary response. For example, stimuli that evoke the idea of brightness (e.g., pictures of the sun) induce pupillary constriction compared to control stimuli of matched luminance. Is conscious appraisal of these stimuli necessary for the pupillary constriction to occur? Participants' pupil diameter was recorded while sun pictures and their phase-scrambled versions were shown to the left eye. A stream of Mondrian patterns was displayed to the right eye to produce continuous flash suppression, which rendered the left-eye stimuli invisible on some trials. Results revealed that when participants were aware of the sun pictures their pupils constricted relative to the control stimuli. This was not the case when the pictures were successfully suppressed from awareness, demonstrating that pupil size is highly sensitive to the contents of consciousness.

Continuous flash suppression operates in local spatial zones: Effects of mask size and contrast

Continuous flash suppression (CFS) is a technique in which presenting one eye with a dynamic Mondrian sequence prevents a low-contrast target in the other eye from being perceived for many seconds. Frequently used to study unconscious visual processing, CFS bears many similarities with binocular rivalry (BR), another popular dichoptic stimulation technique. It is therefore puzzling that the effect of mask size and contrast seem to differ between CFS and BR. To resolve this discrepancy, we conducted a systematic investigation on the effects of mask size and contrast in CFS. Also, building on findings from BR, we asked if the collinearity of the contours in the Mondrian masker play a role in CFS suppression. Our results showed a robust effect of mask contrast on suppression durations, and an effect of mask size that depended on collinearity. Specifically, higher mask contrasts produced longer suppression regardless of collinearity and mask size. Mask size, on the other hand, had little effect on suppression when collinearity was low and it weakened suppression when collinearity is high. These observations parallel prior findings in BR, further substantiating the close link between the two paradigms and demonstrating the usefulness of a shared explanatory framework describing both phenomena.

Can emotional content be extracted under interocular suppression?

Previous studies showed that emotional faces break through interocular suppression more easily compared to neutral faces under the continuous flash suppression (CFS) paradigm. However, there is controversy over whether emotional content or low-level properties contributed to the results. In this study, we directly manipulated the meaningfulness of facial expression to test the role of emotional content in breaking CFS (b-CFS). In addition, an explicit emotion judgment for different facial expressions (happy, neutral, and fearful) used in the b-CFS task was also conducted afterwards to examine the relationship between b-CFS time and emotion judgment. In Experiment 1, face orientation and luminance polarity were manipulated to generate upright-positive and inverted-negative faces. In Experiment 2, Asian and Caucasian faces were presented to Taiwanese participants so that these stimuli served as own-race and other-race faces, respectively. We found robust face familiarity effects in both experiments within the same experimental framework: upright-positive and own-race faces had shorter b-CFS times than inverted-negative and other-race faces, respectively. This indicates potential involvement of high-level processing under interocular suppression. In Experiment 1, different b-CFS times were found between emotional and neutral faces in both upright-positive and inverted-negative conditions. Furthermore, with sufficient duration (1000 ms) participants could still extract emotional content in explicit valence judgment even from inverted-negative faces, though with a smaller degree than upright-positive faces. In Experiment 2, differential b-CFS times were found between emotional and neutral faces with own-race but not other-race faces. Correlation analyses from both experiments showed that the magnitude of emotion judgment was correlated with b-CFS time only for familiar (upright-positive / own-race) but not unfamiliar (inverted-negative / other-race) faces. These results suggest that emotional content can be extracted under interocular suppression with familiar faces, and low-level properties in unfamiliar faces may play a major role in the b-CFS time.

Idiosyncratic, Retinotopic Bias in Face Identification Modulated by Familiarity

The perception of gender and age of unfamiliar faces is reported to vary idiosyncratically across retinal locations such that, for example, the same androgynous face may appear to be male at one location but female at another. Here, we test spatial heterogeneity for the recognition of the identity of personally familiar faces in human participants. We found idiosyncratic biases that were stable within participants and that varied more across locations for low as compared to high familiar faces. These data suggest that like face gender and age, face identity is processed, in part, by independent populations of neurons monitoring restricted spatial regions and that the recognition responses vary for the same face across these different locations. Moreover, repeated and varied social interactions appear to lead to adjustments of these independent face recognition neurons so that the same familiar face is eventually more likely to elicit the same recognition response across widely separated visual field locations. We provide a mechanistic account of this reduced retinotopic bias based on computational simulations.

Typical visual-field locations facilitate access to awareness for everyday objects

In real-world vision, humans are constantly confronted with complex environments that contain a multitude of objects. These environments are spatially structured, so that objects have different likelihoods of appearing in specific parts of the visual space. Our massive experience with such positional regularities prompts the hypothesis that the processing of individual objects varies in efficiency across the visual field: when objects are encountered in their typical locations (e.g., we are used to seeing lamps in the upper visual field and carpets in the lower visual field), they should be more efficiently perceived than when they are encountered in atypical locations (e.g., a lamp in the lower visual field and a carpet in the upper visual field). Here, we provide evidence for this hypothesis by showing that typical positioning facilitates an object's access to awareness. In two continuous flash suppression experiments, objects more efficiently overcame inter-ocular suppression when they were presented in visual-field locations that matched their typical locations in the environment, as compared to non-typical locations. This finding suggests that through extensive experience the visual system has adapted to the statistics of the environment. This adaptation may be particularly useful for rapid object individuation in natural scenes.

Distinct neural processes for the perception of familiar versus unfamiliar faces along the visual hierarchy revealed by EEG

Humans recognize faces with ease, despite the complexity of the task and of the visual system which underlies it. Different spatial regions, including both the core and extended face processing networks, and distinct temporal stages of processing have been implicated in face recognition, but there is ongoing controversy regarding the extent to which the mechanisms for recognizing a familiar face differ from those for an unfamiliar face. Here, we used electroencephalogram (EEG) and flicker SSVEP, a high signal-to-noise approach, and searchlight decoding methods to elucidate the mechanisms mediating the processing of familiar and unfamiliar faces in the time domain. Familiar and unfamiliar faces were presented periodically at 15 Hz, 6 Hz and 3.75 Hz either upright or inverted in separate blocks, with the rationale that faster frequencies require shorter processing times per image and tap into fundamentally different levels of visual processing. The 15 Hz trials, likely to reflect early visual processing, exhibited enhanced neural responses for familiar over unfamiliar face trials, but only when the faces were upright. In contrast, decoding methods revealed similar classification accuracies for upright and inverted faces for both familiar and unfamiliar faces. For the 6 Hz frequency, familiar faces had lower amplitude responses than unfamiliar faces, and decoding familiarity was more accurate for upright compared with inverted faces. Finally, the 3.75 Hz frequency revealed no main effects of familiarity, but decoding showed significant correlations with behavioral ratings of face familiarity, suggesting that activity evoked by this slow presentation frequency reflected higher-level, cognitive aspects of familiarity processing. This three-way dissociation between frequencies reveals that fundamentally different stages of the visual hierarchy are modulated by face familiarity. The combination of experimental and analytical approaches used here represent a novel method for elucidating spatio-temporal characteristics within the visual system.

The automaticity of face perception is influenced by familiarity

In this study, we explore the automaticity of encoding for different facial characteristics and ask whether it is influenced by face familiarity. We used a matching task in which participants had to report whether the gender, identity, race, or expression of two briefly presented faces was the same or different. The task was made challenging by allowing nonrelevant dimensions to vary across trials. To test for automaticity, we compared performance on trials in which the task instruction was given at the beginning of the trial, with trials in which the task instruction was given at the end of the trial. As a strong criterion for automatic processing, we reasoned that if perception of a given characteristic (gender, race, identity, or emotion) is fully automatic, the timing of the instruction should not influence performance. We compared automaticity for the perception of familiar and unfamiliar faces. Performance with unfamiliar faces was higher for all tasks when the instruction was given at the beginning of the trial. However, we found a significant interaction between instruction and task with familiar faces. Accuracy of gender and identity judgments to familiar faces was the same regardless of whether the instruction was given before or after the trial, suggesting automatic processing of these properties. In contrast, there was an effect of instruction for judgments of expression and race to familiar faces. These results show that familiarity enhances the automatic processing of some types of facial information more than others.

Self- or familiar-face recognition advantage? New insight using ambient images

Self-face recognition has been widely explored in the past few years. Nevertheless, the current literature relies on the use of standardized photographs which do not represent daily-life face recognition. Therefore, we aim for the first time to evaluate self-face processing in healthy individuals using natural/ambient images which contain variations in the environment and in the face itself. In total, 40 undergraduate and graduate students performed a forced delayed-matching task, including images of one's own face, friend, famous and unknown individuals. For both reaction time and accuracy, results showed that participants were faster and more accurate when matching different images of their own face compared to both famous and unfamiliar faces. Nevertheless, no significant differences were found between self-face and friend-face and between friend-face and famous-face. They were also faster and more accurate when matching friend and famous faces compared to unfamiliar faces. Our results suggest that faster and more accurate responses to self-face might be better explained by a familiarity effect - that is, (1) the result of frequent exposition to one's own image through mirror and photos, (2) a more robust mental representation of one's own face and (3) strong face recognition units as for other familiar faces.

Prioritized Identification of Attractive and Romantic Partner Faces in Rapid Serial Visual Presentation

People are sensitive to facial attractiveness because it is an important biological and social signal. As such, our perceptual and attentional system seems biased toward attractive faces. We tested whether attractive faces capture attention and enhance memory access in an involuntary manner using a dual-task rapid serial visual presentation (dtRSVP) paradigm, wherein multiple faces were successively presented for 120 ms. In Experiment 1, participants (N = 26) were required to identify two female faces embedded in a stream of animal faces as distractors. The results revealed that identification of the second female target (T2) was better when it was attractive compared to neutral or unattractive. In Experiment 2, we investigated whether perceived attractiveness affects T2 identification (N = 27). To this end, we performed another dtRSVP task involving participants in a romantic partnership with the opposite sex, wherein T2 was their romantic partner's face. The results demonstrated that a romantic partner's face was correctly identified more often than was the face of a friend or unknown person. Furthermore, the greater the intensity of passionate love participants felt for their partner (as measured by the Passionate Love Scale), the more often they correctly identified their partner's face. Our experiments indicate that attractive and romantic partners' faces facilitate the identification of the faces in an involuntary manner.

The neural representation of personally familiar and unfamiliar faces in the distributed system for face perception

Personally familiar faces are processed more robustly and efficiently than unfamiliar faces. The human face processing system comprises a core system that analyzes the visual appearance of faces and an extended system for the retrieval of person-knowledge and other nonvisual information. We applied multivariate pattern analysis to fMRI data to investigate aspects of familiarity that are shared by all familiar identities and information that distinguishes specific face identities from each other. Both identity-independent familiarity information and face identity could be decoded in an overlapping set of areas in the core and extended systems. Representational similarity analysis revealed a clear distinction between the two systems and a subdivision of the core system into ventral, dorsal and anterior components. This study provides evidence that activity in the extended system carries information about both individual identities and personal familiarity, while clarifying and extending the organization of the core system for face perception.

Separate requirements for detection and perceptual stability of motion in interocular suppression

In interocular masking, a stimulus presented to one eye (the mask) is made stronger in order to suppress from awareness the target stimulus presented to the other eye. We investigated whether matching the features of the target and the mask would lead to more effective suppression (feature-selective suppression), or not (i.e., non-selective suppression). To control the temporal characteristics of the stimuli, we used a dynamic interocular mask to suppress a moving target, and found that neither matching speed nor pattern of motion led to more effective suppression. Instead, a faster target was detected faster, regardless of the mask type or speed, while a relatively slow (about 1°/s) mask was more perceptually stable (i.e., maintained suppression longer) in a non-selective fashion. While the requirement for target detectability, i.e., salience, is well characterized, relatively little attention is given to the factors that make a mask percept more perceptually stable. Based on these results, we argue that there are separate requirements for detection and perceptual stability.

Familiarity facilitates feature-based face processing

Recognition of personally familiar faces is remarkably efficient, effortless and robust. We asked if feature-based face processing facilitates detection of familiar faces by testing the effect of face inversion on a visual search task for familiar and unfamiliar faces. Because face inversion disrupts configural and holistic face processing, we hypothesized that inversion would diminish the familiarity advantage to the extent that it is mediated by such processing. Subjects detected personally familiar and stranger target faces in arrays of two, four, or six face images. Subjects showed significant facilitation of personally familiar face detection for both upright and inverted faces. The effect of familiarity on target absent trials, which involved only rejection of unfamiliar face distractors, suggests that familiarity facilitates rejection of unfamiliar distractors as well as detection of familiar targets. The preserved familiarity effect for inverted faces suggests that facilitation of face detection afforded by familiarity reflects mostly feature-based processes.

Social Saliency of the Cue Slows Attention Shifts

Eye gaze is a powerful cue that indicates where another person's attention is directed in the environment. Seeing another person's eye gaze shift spontaneously and reflexively elicits a shift of one's own attention to the same region in space. Here, we investigated whether reallocation of attention in the direction of eye gaze is modulated by personal familiarity with faces. On the one hand, the eye gaze of a close friend should be more effective in redirecting our attention as compared to the eye gaze of a stranger. On the other hand, the social relevance of a familiar face might itself hold attention and, thereby, slow lateral shifts of attention. To distinguish between these possibilities, we measured the efficacy of the eye gaze of personally familiar and unfamiliar faces as directional attention cues using adapted versions of the Posner paradigm with saccadic and manual responses. We found that attention shifts were slower when elicited by a perceived change in the eye gaze of a familiar individual as compared to attention shifts elicited by unfamiliar faces at short latencies (100 ms). We also measured simple detection of change in direction of gaze in personally familiar and unfamiliar faces to test whether slower attention shifts were due to slower detection. Participants detected changes in eye gaze faster for familiar faces than for unfamiliar faces. Our results suggest that personally familiar faces briefly hold attention due to their social relevance, thereby slowing shifts of attention, even though the direction of eye movements are detected faster in familiar faces.