Ultra-coarse, single-glance human face detection in a dynamic visual stream

Hemispheric dominance in reading system alters contribution to face processing lateralization across development

Face processing dominates the right hemisphere. This lateralization can be affected by co-lateralization within the same system and influence between different systems, such as neural competition from reading acquisition. Yet, how the relationship pattern changes through development remains unknown. This study examined the lateralization of core face processing and word processing in different age groups. By comparing fMRI data from 36 school-aged children and 40 young adults, we investigated whether there are age and regional effects on lateralization, and how relationships between lateralization within and between systems change across development. Our results showed significant right hemispheric lateralization in the core face system and left hemispheric lateralization in reading-related areas for both age groups when viewing faces and texts passively. While all participants showed stronger lateralization in brain regions of higher functional hierarchy when viewing faces, only adults exhibited this lateralization when viewing texts. In both age cohorts, there was intra-system co-lateralization for face processing, whereas an inter-system relationship was only found in adults. Specifically, functional lateralization of Broca's area during reading negatively predicted functional asymmetry in the FFA during face perception. This study initially provides neuroimaging evidence for the reading-induced neural competition theory from a maturational perspective in Chinese cohorts.

Discrepancies in perceived humanness between spatially filtered and unfiltered faces and their associations with uncanny feelings

Human and artificial features that coexist in certain types of human-like robots create a discrepancy in perceived humanness and evoke uncanny feelings in human observers. However, whether this perceptual mismatch in humanness occurs for all faces, and whether it is related to the uncanny feelings toward them, is unknown. We investigated this by examining perceived humanness for a variety of natural images of robot and human faces with different spatial frequency (SF) information: that is, faces with only low SF, middle SF, and high SF information, and intact (spatially unfiltered) faces. Uncanny feelings elicited by these faces were also measured. The results showed perceptual mismatches that LSF, MSF, and HSF faces were perceived as more human than intact faces. This was particularly true for intact robot faces that looked slightly human, which tended to evoke strong uncanny feelings. Importantly, the mismatch in perceived humanness between the intact and spatially filtered faces was positively correlated with uncanny feelings toward intact faces. Given that the human visual system performs SF analysis when processing faces, the perceptual mismatches observed in this study likely occur in real life for all faces, and as such might be a ubiquitous source of uncanny feelings in real-life situations.

Visual periodicity reveals distinct attentional signatures for face and non-face categories

Observers can selectively deploy attention to regions of space, moments in time, specific visual features, individual objects, and even specific high-level categories-for example, when keeping an eye out for dogs while jogging. Here, we exploited visual periodicity to examine how category-based attention differentially modulates selective neural processing of face and non-face categories. We combined electroencephalography with a novel frequency-tagging paradigm capable of capturing selective neural responses for multiple visual categories contained within the same rapid image stream (faces/birds in Exp 1; houses/birds in Exp 2). We found that the pattern of attentional enhancement and suppression for face-selective processing is unique compared to other object categories: Where attending to non-face objects strongly enhances their selective neural signals during a later stage of processing (300-500 ms), attentional enhancement of face-selective processing is both earlier and comparatively more modest. Moreover, only the selective neural response for faces appears to be actively suppressed by attending towards an alternate visual category. These results underscore the special status that faces hold within the human visual system, and highlight the utility of visual periodicity as a powerful tool for indexing selective neural processing of multiple visual categories contained within the same image sequence.

Fast saccades to faces during the feedforward sweep

Saccadic choice tasks use eye movements as a response method, typically in a task where observers are asked to saccade as quickly as possible to an image of a prespecified target category. Using this approach, face-selective saccades have been observed within 100 ms poststimulus. When taking into account oculomotor processing, this suggests that faces can be detected in as little as 70 to 80 ms. It has therefore been suggested that face detection must occur during the initial feedforward sweep, since this latency leaves little time for feedback processing. In the current experiment, we tested this hypothesis using backward masking-a technique shown to primarily disrupt feedback processing while leaving feedforward activation relatively intact. Based on minimum saccadic reaction time, we found that face detection benefited from ultra-fast, accurate saccades within 110 to 160 ms and that these eye movements are obtainable even under extreme masking conditions that limit perceptual awareness. However, masking did significantly increase the median SRT for faces. In the manual responses, we found remarkable detection accuracy for faces and houses, even when participants indicated having no visual experience of the test images. These results provide evidence for the view that the saccadic bias to faces is initiated by coarse information used to categorize faces in the feedforward sweep but that, in most cases, additional processing is required to quickly reach the threshold for saccade initiation.

Voice categorization in the four-month-old human brain

Voices are the most relevant social sounds for humans and therefore have crucial adaptive value in development. Neuroimaging studies in adults have demonstrated the existence of regions in the superior temporal sulcus that respond preferentially to voices. Yet, whether voices represent a functionally specific category in the young infant's mind is largely unknown. We developed a highly sensitive paradigm relying on fast periodic auditory stimulation (FPAS) combined with scalp electroencephalography (EEG) to demonstrate that the infant brain implements a reliable preferential response to voices early in life. Twenty-three 4-month-old infants listened to sequences containing non-vocal sounds from different categories presented at 3.33 Hz, with highly heterogeneous vocal sounds appearing every third stimulus (1.11 Hz). We were able to isolate a voice-selective response over temporal regions, and individual voice-selective responses were found in most infants within only a few minutes of stimulation. This selective response was significantly reduced for the same frequency-scrambled sounds, indicating that voice selectivity is not simply driven by the envelope and the spectral content of the sounds. Such a robust selective response to voices as early as 4 months of age suggests that the infant brain is endowed with the ability to rapidly develop a functional selectivity to this socially relevant category of sounds.

Detection of emotional faces: The role of spatial frequencies and local features

Models of emotion processing suggest that threat-related stimuli such as fearful faces can be detected based on the rapid extraction of low spatial frequencies. However, this remains debated as other models argue that the decoding of facial expressions occurs with a more flexible use of spatial frequencies. The purpose of this study was to clarify the role of spatial frequencies and differences in luminance contrast between spatial frequencies, on the detection of facial emotions. We used a saccadic choice task in which emotional-neutral face pairs were presented and participants were asked to make a saccade toward the neutral or the emotional (happy or fearful) face. Faces were displayed either in low, high, or broad spatial frequencies. Results showed that participants were better to saccade toward the emotional face. They were also better for high or broad than low spatial frequencies, and the accuracy was higher with a happy target. An analysis of the eye and mouth saliency ofour stimuli revealed that the mouth saliency of the target correlates with participants' performance. Overall, this study underlines the importance of local more than global information, and of the saliency of the mouth region in the detection of emotional and neutral faces.

Spatial Resolution Evaluation Based on Experienced Visual Categories With Sweep Evoked Periodic EEG Activity

Purpose

Visual function is typically evaluated in clinical settings with visual acuity (VA), a test requiring to behaviorally match or name optotypes such as tumbling E or Snellen letters. The ability to recognize these symbols has little in common with the automatic and rapid visual recognition of socially important stimuli in real life. Here we use sweep visual evoked potentials to assess spatial resolution objectively based on the recognition of human faces and written words.

Methods

To this end, we tested unfamiliar face individuation1 and visual word recognition2 in 15 normally sighted adult volunteers with a 68-electrode electroencephalogram system.

Results

Unlike previous measures of low-level visual function including VA, the most sensitive electrode was found at an electrode different from Oz in a majority of participants. Thresholds until which faces and words could be recognized were evaluated at the most sensitive electrode defined individually for each participant. Word recognition thresholds corresponded with the VA level expected from normally sighted participants, and even a VA significantly higher than expected from normally sighted individuals for a few participants.

Conclusions

Spatial resolution can be evaluated based on high-level stimuli encountered in day-to-day life, such as faces or written words with sweep visual evoked potentials.

Automatic morpheme identification across development: Magnetoencephalography (MEG) evidence from fast periodic visual stimulation

The present study combined magnetoencephalography (MEG) recordings with fast periodic visual stimulation (FPVS) to investigate automatic neural responses to morphemes in developing and skilled readers. Native English-speaking children (N = 17, grade 5–6) and adults (N = 28) were presented with rapid streams of base stimuli (6 Hz) interleaved periodically with oddballs (i.e., every fifth item, oddball stimulation frequency: 1.2 Hz). In a manipulation-check condition, tapping into word recognition, oddballs featured familiar words (e.g., roll) embedded in a stream of consonant strings (e.g., ktlq). In the experimental conditions, the contrast between oddball and base stimuli was manipulated in order to probe selective stem and suffix identification in morphologically structured pseudowords (e.g., stem + suffix pseudowords such as softity embedded in nonstem + suffix pseudowords such as trumess). Neural responses at the oddball frequency and harmonics were analyzed at the sensor level using non-parametric cluster-based permutation tests. As expected, results in the manipulation-check condition revealed a word-selective response reflected by a predominantly left-lateralized cluster that emerged over temporal, parietal, and occipital sensors in both children and adults. However, across the experimental conditions, results yielded a differential pattern of oddball responses in developing and skilled readers. Children displayed a significant response that emerged in a mostly central occipital cluster for the condition tracking stem identification in the presence of suffixes (e.g., softity vs. trumess). In contrast, adult participants showed a significant response that emerged in a cluster located in central and left occipital sensors for the condition tracking suffix identification in the presence of stems (e.g., softity vs. stopust). The present results suggest that while the morpheme identification system in Grade 5–6 children is not yet adult-like, it is sufficiently mature to automatically analyze the morphemic structure of novel letter strings. These findings are discussed in the context of theoretical accounts of morphological processing across reading development.

Smell what you hardly see: Odors assist visual categorization in the human brain

Visual categorization is the brain ability to rapidly and automatically respond to a certain category of inputs. Whether category-selective neural responses are purely visual or can be influenced by other sensory modalities remains unclear. Here, we test whether odors modulate visual categorization, expecting that odors facilitate the neural categorization of congruent visual objects, especially when the visual category is ambiguous. Scalp electroencephalogram (EEG) was recorded while natural images depicting various objects were displayed in rapid 12-Hz streams (i.e., 12 images / second) and variable exemplars of a target category (either human faces, cars, or facelike objects in dedicated sequences) were interleaved every 9^th stimulus to tag category-selective responses at 12/9 = 1.33 Hz in the EEG frequency spectrum. During visual stimulation, participants (N = 26) were implicitly exposed to odor contexts (either body, gasoline or baseline odors) and performed an orthogonal cross-detection task. We identify clear category-selective responses to every category over the occipito-temporal cortex, with the largest response for human faces and the lowest for facelike objects. Critically, body odor boosts the response to the ambiguous facelike objects (i.e., either perceived as nonface objects or faces) over the right hemisphere, especially for participants reporting their presence post-stimulation. By contrast, odors do not significantly modulate other category-selective responses, nor the general visual response recorded at 12 Hz, revealing a specific influence on the categorization of congruent ambiguous stimuli. Overall, these findings support the view that the brain actively uses cues from the different senses to readily categorize visual inputs, and that olfaction, which has long been considered as poorly functional in humans, is well placed to disambiguate visual information.

Information redundancy across spatial scales modulates early visual cortical processing

Visual images contain redundant information across spatial scales where low spatial frequency contrast is informative towards the location and likely content of high spatial frequency detail. Previous research suggests that the visual system makes use of those redundancies to facilitate efficient processing. In this framework, a fast, initial analysis of low-spatial frequency (LSF) information guides the slower and later processing of high spatial frequency (HSF) detail. Here, we used multivariate classification as well as time-frequency analysis of MEG responses to the viewing of intact and phase scrambled images of human faces to demonstrate that the availability of redundant LSF information, as found in broadband intact images, correlates with a reduction in HSF representational dominance in both early and higher-level visual areas as well as a reduction of gamma-band power in early visual cortex. Our results indicate that the cross spatial frequency information redundancy that can be found in all natural images might be a driving factor in the efficient integration of fine image details.

Is human face recognition lateralized to the right hemisphere due to neural competition with left-lateralized visual word recognition? A critical review

The right hemispheric lateralization of face recognition, which is well documented and appears to be specific to the human species, remains a scientific mystery. According to a long-standing view, the evolution of language, which is typically substantiated in the left hemisphere, competes with the cortical space in that hemisphere available for visuospatial processes, including face recognition. Over the last decade, a specific hypothesis derived from this view according to which neural competition in the left ventral occipito-temporal cortex with selective representations of letter strings causes right hemispheric lateralization of face recognition, has generated considerable interest and research in the scientific community. Here, a systematic review of studies performed in various populations (infants, children, literate and illiterate adults, left-handed adults) and methodologies (behavior, lesion studies, (intra)electroencephalography, neuroimaging) offers little if any support for this reading lateralized neural competition hypothesis. Specifically, right-lateralized face-selective neural activity already emerges at a few months of age, well before reading acquisition. Moreover, consistent evidence of face recognition performance and its right hemispheric lateralization being modulated by literacy level during development or at adulthood is lacking. Given the absence of solid alternative hypotheses and the key role of neural competition in the sensory-motor cortices for selectivity of representations, learning, and plasticity, a revised language-related neural competition hypothesis for the right hemispheric lateralization of face recognition should be further explored in future research, albeit with substantial conceptual clarification and advances in methodological rigor.

Critical information thresholds underlying generic and familiar face categorisation at the same face encounter

Seeing a face in the real world provokes a host of automatic categorisations related to sex, emotion, identity, and more. Such individual facets of human face recognition have been extensively examined using overt categorisation judgements, yet their relative informational dependencies during the same face encounter are comparatively unknown. Here we used EEG to assess how increasing access to sensory input governs two ecologically relevant brain functions elicited by seeing a face: Distinguishing faces and nonfaces, and recognising people we know. Observers viewed a large set of natural images that progressively increased in either image duration (experiment 1) or spatial frequency content (experiment 2). We show that in the absence of an explicit categorisation task, the human brain requires less sensory input to categorise a stimulus as a face than it does to recognise whether that face is familiar. Moreover, where sensory thresholds for distinguishing faces/nonfaces were remarkably consistent across observers, there was high inter-individual variability in the lower informational bound for familiar face recognition, underscoring the neurofunctional distinction between these categorisation functions. By i) indexing a form of face recognition that goes beyond simple low-level differences between categories, and ii) tapping multiple recognition functions elicited by the same face encounters, the information minima we report bear high relevance to real-world face encounters, where the same stimulus is categorised along multiple dimensions at once. Thus, our finding of lower informational requirements for generic vs. familiar face recognition constitutes some of the strongest evidence to date for the intuitive notion that sensory input demands should be lower for recognising face category than face identity.

The roles of gaze and head orientation in face categorization during rapid serial visual presentation

Little is known about how perceived gaze direction and head orientation may influence human categorization of visual stimuli as faces. To address this question, a sequence of unsegmented natural images, each containing a random face or a non-face object, was presented in rapid succession (stimulus duration: 91.7 ms per image) during which human observers were instructed to respond immediately to every face presentation. Faces differed in gaze and head orientation in 7 combinations - full-front views with perceived gaze (1) directed to the observer, (2) averted to the left, or (3) averted to the right, left ¾ side views with (4) direct gaze or (5) averted gaze, and right ¾ side views with (6) direct gaze or (7) averted gaze - were presented randomly throughout the sequence. We found highly accurate and rapid behavioural responses to all kinds of faces. Crucially, both perceived gaze direction and head orientation had comparable, non-interactive effects on response times, where direct gaze was responded faster than averted gaze by 48 ms and full-front view faster than ¾ side view also by 48 ms on average. Presentations of full-front faces with direct gaze led to an additive speed advantage of 96 ms to ¾ faces with averted gaze. The results reveal that the effects of perceived gaze direction and head orientation on the speed of face categorization probably depend on the degree of social relevance of the face to the viewer.

Flexibility of emerging face categorization at different levels of abstraction

Categorization of visual stimuli at different levels of abstraction relies on the encoding of relevant diagnostic features present at different spatial scales. We used the Eidolon Factory, an image-manipulation algorithm that introduces random disarray fields across spatial scales, to study how such a process flexibly combines perceptual information to perform successful categorization depending on task demands. Images of animal faces, human faces, and everyday objects were disarrayed coherently (random fields correlated) or incoherently (random fields randomized) to create a family of 50 eidolons per stimulus image with increasing disarray. Participants (N = 243) viewed each family of eidolons in a smooth sequence from maximum disarray to no disarray and performed a category verification task either at the superordinate (any face type) or basic (human face only) levels at two levels of uncertainty: participants in one group used their gut feeling to respond, whereas another group had to be sure of their decision. When participants used their gut feeling to respond, we observed a superordinate-level advantage. When they were sure of their response, we observed a basic-level advantage. Coherently disarrayed sequences impaired target detection compared to incoherently disarrayed sequences for both levels of response certainty. Furthermore, participants' sensitivity in the Any Face condition increased when they observed coherently disarrayed sequences and had to be sure of their response. These results suggest that the visual system does not strictly adhere to feedforward processing but flexibly adjusts to the relevant perceptual information depending on task context.

Pinpointing the optimal spatial frequency range for automatic neural facial fear processing

Faces convey an assortment of emotional information via low and high spatial frequencies (LSFs and HSFs). However, there is no consensus on the role of particular spatial frequency (SF) information during facial fear processing. Comparison across studies is hampered by the high variability in cut-off values for demarcating the SF spectrum and by differences in task demands. We investigated which SF information is minimally required to rapidly detect briefly presented fearful faces in an implicit and automatic manner, by sweeping through an entire SF range without constraints of predefined cut-offs for LSFs and HSFs. We combined fast periodic visual stimulation with electroencephalography. We presented neutral faces at 6 ​Hz, periodically interleaved every 5th image with a fearful face, allowing us to quantify an objective neural index of fear discrimination at exactly 1.2 ​Hz. We started from a stimulus containing either only very low or very high SFs and gradually increased the SF content by adding higher or lower SF information, respectively, to reach the full SF spectrum over the course of 70 ​s. We found that faces require at least SF information higher than 5.93 cycles per image (cpi) to implicitly differentiate fearful from neutral faces. However, exclusive HSF faces, even in a restricted SF range between 94.82 and 189.63 cpi already carry the critical information to extract the emotional expression of the faces.

All-or-none face categorization in the human brain

Visual categorization is integral for our interaction with the natural environment. In this process, similar selective responses are produced to a class of variable visual inputs. Whether categorization is supported by partial (graded) or absolute (all-or-none) neural responses in high-level human brain regions is largely unknown. We address this issue with a novel frequency-sweep paradigm probing the evolution of face categorization responses between the minimal and optimal stimulus presentation times. In a first experiment, natural images of variable non-face objects were progressively swept from 120 to 3 ​Hz (8.33-333 ​ms duration) in rapid serial visual presentation sequences. Widely variable face exemplars appeared every 1 ​s, enabling an implicit frequency-tagged face-categorization electroencephalographic (EEG) response at 1 ​Hz. Face-categorization activity emerged with stimulus durations as brief as 17 ​ms (17-83 ​ms across individual participants) but was significant with 33 ​ms durations at the group level. The face categorization response amplitude increased until 83 ​ms stimulus duration (12 ​Hz), implying graded categorization responses. In a second EEG experiment, faces appeared non-periodically throughout such sequences at fixed presentation rates, while participants explicitly categorized faces. A strong correlation between response amplitude and behavioral accuracy across frequency rates suggested that dilution from missed categorizations, rather than a decreased response to each face stimulus, accounted for the graded categorization responses as found in Experiment 1. This was supported by (1) the absence of neural responses to faces that participants failed to categorize explicitly in Experiment 2 and (2) equivalent amplitudes and spatio-temporal signatures of neural responses to behaviorally categorized faces across presentation rates. Overall, these observations provide original evidence that high-level visual categorization of faces, starting at about 100 ​ms following stimulus onset in the human brain, is variable across observers tested under tight temporal constraints, but occurs in an all-or-none fashion.

Neurophysiological evidence for crossmodal (face-name) person-identity representation in the human left ventral temporal cortex

Putting a name to a face is a highly common activity in our daily life that greatly enriches social interactions. Although this specific person-identity association becomes automatic with learning, it remains difficult and can easily be disrupted in normal circumstances or neurological conditions. To shed light on the neural basis of this important and yet poorly understood association between different input modalities in the human brain, we designed a crossmodal frequency-tagging paradigm coupled to brain activity recording via scalp and intracerebral electroencephalography. In Experiment 1, 12 participants were presented with variable pictures of faces and written names of a single famous identity at a 4-Hz frequency rate while performing an orthogonal task. Every 7 items, another famous identity appeared, either as a face or a name. Robust electrophysiological responses were found exactly at the frequency of identity change (i.e., 4 Hz / 7 = 0.571 Hz), suggesting a crossmodal neural response to person identity. In Experiment 2 with twenty participants, two control conditions with periodic changes of identity for faces or names only were added to estimate the contribution of unimodal neural activity to the putative crossmodal face-name responses. About 30% of the response occurring at the frequency of crossmodal identity change over the left occipito-temporal cortex could not be accounted for by the linear sum of unimodal responses. Finally, intracerebral recordings in the left ventral anterior temporal lobe (ATL) in 7 epileptic patients tested with this paradigm revealed a small number of "pure" crossmodal responses, i.e., with no response to changes of identity for faces or names only. Altogether, these observations provide evidence for integration of verbal and nonverbal person identity-specific information in the human brain, highlighting the contribution of the left ventral ATL in the automatic retrieval of face-name identity associations.

Coarse-to-fine information integration in human vision

Coarse-to-fine theories of vision propose that the coarse information carried by the low spatial frequencies (LSF) of visual input guides the integration of finer, high spatial frequency (HSF) detail. Whether and how LSF modulates HSF processing in naturalistic broad-band stimuli is still unclear. Here we used multivariate decoding of EEG signals to separate the respective contribution of LSF and HSF to the neural response evoked by broad-band images. Participants viewed images of human faces, monkey faces and phase-scrambled versions that were either broad-band or filtered to contain LSF or HSF. We trained classifiers on EEG scalp-patterns evoked by filtered scrambled stimuli and evaluated the derived models on broad-band scrambled and intact trials. We found reduced HSF contribution when LSF was informative towards image content, indicating that coarse information does guide the processing of fine detail, in line with coarse-to-fine theories. We discuss the potential cortical mechanisms underlying such coarse-to-fine feedback.