Effects of band-pass spatial frequency filtering of face and object images on the amplitude of N170

Brain Source Correlates of Speech Perception and Reading Processes in Children With and Without Reading Difficulties

Neural correlates in reading and speech processing have been addressed extensively in the literature. While reading skills and speech perception have been shown to be associated with each other, their relationship remains debatable. In this study, we investigated reading skills, speech perception, reading, and their correlates with brain source activity in auditory and visual modalities. We used high-density event-related potentials (ERPs), fixation-related potentials (FRPs), and the source reconstruction method. The analysis was conducted on 12–13-year-old schoolchildren who had different reading levels. Brain ERP source indices were computed from frequently repeated Finnish speech stimuli presented in an auditory oddball paradigm. Brain FRP source indices were also computed for words within sentences presented in a reading task. The results showed significant correlations between speech ERP sources and reading scores at the P100 (P1) time range in the left hemisphere and the N250 time range in both hemispheres, and a weaker correlation for visual word processing N170 FRP source(s) in the posterior occipital areas, in the vicinity of the visual word form areas (VWFA). Furthermore, significant brain-to-brain correlations were found between the two modalities, where the speech brain sources of the P1 and N250 responses correlated with the reading N170 response. The results suggest that speech processes are linked to reading fluency and that brain activations to speech are linked to visual brain processes of reading. These results indicate that a relationship between language and reading systems is present even after several years of exposure to print.

The Time Sequence of Face Spatial Frequency Differs During Working Memory Encoding and Retrieval Stages

Previous studies have found that P1 and P2 components were more sensitive to configural and featural face processing, respectively, when attentional resources were sufficient, suggesting that face processing follows a coarse-to-fine sequence. However, the role of working memory (WM) load in the time course of configural and featural face processing is poorly understood, especially whether it differs during encoding and retrieval stages. This study employed a delayed recognition task with varying WM load and face spatial frequency (SF). Our behavioral and ERP results showed that WM load modulated face SF processing. Specifically, for the encoding stage, P1 and P2 were more sensitive to broadband SF (BSF) faces, while N170 was more sensitive to low SF (LSF) and BSF faces. For the retrieval stage, P1 on the right hemisphere was more sensitive to BSF faces relative to HSF faces, N170 was more sensitive to LSF faces than HSF faces, especially under the load 1 condition, while P2 was more sensitive to high SF (HSF) faces than HSF faces, especially under load 3 condition. These results indicate that faces are perceived less finely during the encoding stage, whereas face perception follows a coarse-to-fine sequence during the retrieval stage, which is influenced by WM load. The coarse and fine information were processed especially under the low and high load conditions, respectively.

Investigating the Single Trial Detectability of Cognitive Face Processing by a Passive Brain-Computer Interface

An automated recognition of faces enables machines to visually identify a person and to gain access to non-verbal communication, including mimicry. Different approaches in lab settings or controlled realistic environments provided evidence that automated face detection and recognition can work in principle, although applications in complex real-world scenarios pose a different kind of problem that could not be solved yet. Specifically, in autonomous driving-it would be beneficial if the car could identify non-verbal communication of pedestrians or other drivers, as it is a common way of communication in daily traffic. Automated identification from observation whether pedestrians or other drivers communicate through subtle cues in mimicry is an unsolved problem so far, as intent and other cognitive factors are hard to derive from observation. In contrast, communicating persons usually have clear understanding whether they communicate or not, and such information is represented in their mindsets. This work investigates whether the mental processing of faces can be identified through means of a Passive Brain-Computer Interface (pBCI). This then could be used to support the cars' autonomous interpretation of facial mimicry of pedestrians to identify non-verbal communication. Furthermore, the attentive driver can be utilized as a sensor to improve the context awareness of the car in partly automated driving. This work presents a laboratory study in which a pBCI is calibrated to detect responses of the fusiform gyrus in the electroencephalogram (EEG), reflecting face recognition. Participants were shown pictures from three different categories: faces, abstracts, and houses evoking different responses used to calibrate the pBCI. The resulting classifier could distinguish responses to faces from that evoked by other stimuli with accuracy above 70%, in a single trial. Further analysis of the classification approach and the underlying data identified activation patterns in the EEG that corresponds to face recognition in the fusiform gyrus. The resulting pBCI approach is promising as it shows better-than-random accuracy and is based on relevant and intended brain responses. Future research has to investigate whether it can be transferred from the laboratory to the real world and how it can be implemented into artificial intelligences, as used in autonomous driving.

Coarse-to-Fine(r) Automatic Familiar Face Recognition in the Human Brain

At what level of spatial resolution can the human brain recognize a familiar face in a crowd of strangers? Does it depend on whether one approaches or rather moves back from the crowd? To answer these questions, 16 observers viewed different unsegmented images of unfamiliar faces alternating at 6 Hz, with spatial frequency (SF) content progressively increasing (i.e., coarse-to-fine) or decreasing (fine-to-coarse) in different sequences. Variable natural images of celebrity faces every sixth stimulus generated an objective neural index of single-glanced automatic familiar face recognition (FFR) at 1 Hz in participants' electroencephalogram (EEG). For blurry images increasing in spatial resolution, the neural FFR response over occipitotemporal regions emerged abruptly with additional cues at about 6.3-8.7 cycles/head width, immediately reaching amplitude saturation. When the same images progressively decreased in resolution, the FFR response disappeared already below 12 cycles/head width, thus providing no support for a predictive coding hypothesis. Overall, these observations indicate that rapid automatic recognition of heterogenous natural views of familiar faces is achieved from coarser visual inputs than generally thought, and support a coarse-to-fine FFR dynamics in the human brain.

Sparsely Wiring Connectivity in the Upper Beta Band Characterizes the Brains of Top Swimming Athletes

Human brains are extremely energy costly in neural connections and activities. However, it is unknown what is the difference in the brain connectivity between top athletes with long-term professional trainings and age-matched controls. Here we ask whether long-term training can lower brain-wiring cost while have better performance. Since elite swimming requires athletes to move their arms and legs at different tempos in time with high coordination skills, we selected an eye-hand-foot complex reaction (CR) task to examine the relations between the task performance and the brain connections and activities, as well as to explore the energy cost-efficiency of top athletes. Twenty-one master-level professional swimmers and 23 age-matched non-professional swimmers as controls were recruited to perform the CR task with concurrent 8-channel EEG recordings. Reaction time and accuracy of the CR task were recorded. Topological network analysis of various frequency bands was performed using the phase lag index (PLI) technique to avoid volume conduction effects. The wiring number of connections and mean frequency were calculated to reflect the wiring and activity cost, respectively. Results showed that professional athletes demonstrated better eye-hand-foot coordination than controls when performing the CR task, indexing by faster reaction time and higher accuracy. Comparing to controls, athletes' brain demonstrated significantly less connections and weaker correlations in upper beta frequency band between the frontal and parietal regions, while demonstrated stronger connectivity in the low theta frequency band between sites of F3 and Cz/C4. Additionally, athletes showed highly stable and low eye-blinking rates across different reaction performance, while controls had high blinking frequency with high variance. Elite athletes' brain may be characterized with energy efficient sparsely wiring connections in support of superior motor performance and better cognitive performance in the eye-hand-foot complex reaction task.

Multi-modal Mapping of the Face Selective Ventral Temporal Cortex-A Group Study With Clinical Implications for ECS, ECoG, and fMRI

Face recognition is impaired in patients with prosopagnosia, which may occur as a side effect of neurosurgical procedures. Face selective regions on the ventral temporal cortex have been localized with electrical cortical stimulation (ECS), electrocorticography (ECoG), and functional magnetic resonance imagining (fMRI). This is the first group study using within-patient comparisons to validate face selective regions mapping, utilizing the aforementioned modalities. Five patients underwent surgical treatment of intractable epilepsy and joined the study. Subdural grid electrodes were implanted on their ventral temporal cortices to localize seizure foci and face selective regions as part of the functional mapping protocol. Face selective regions were identified in all patients with fMRI, four patients with ECoG, and two patients with ECS. From 177 tested electrode locations in the region of interest (ROI), which is defined by the fusiform gyrus and the inferior temporal gyrus, 54 face locations were identified by at least one modality in all patients. fMRI mapping showed the highest detection rate, revealing 70.4% for face selective locations, whereas ECoG and ECS identified 64.8 and 31.5%, respectively. Thus, 28 face locations were co-localized by at least two modalities, with detection rates of 89.3% for fMRI, 85.7% for ECoG and 53.6 % for ECS. All five patients had no face recognition deficits after surgery, even though five of the face selective locations, one obtained by ECoG and the other four by fMRI, were within 10 mm to the resected volumes. Moreover, fMRI included a quite large volume artifact on the ventral temporal cortex in the ROI from the anatomical structures of the temporal base. In conclusion, ECS was not sensitive in several patients, whereas ECoG and fMRI even showed activation within 10 mm to the resected volumes. Considering the potential signal drop-out in fMRI makes ECoG the most reliable tool to identify face selective locations in this study. A multimodal approach can improve the specificity of ECoG and fMRI, while simultaneously minimizing the number of required ECS sessions. Hence, all modalities should be considered in a clinical mapping protocol entailing combined results of co-localized face selective locations.

Implementation-Independent Representation for Deep Convolutional Neural Networks and Humans in Processing Faces

Deep convolutional neural networks (DCNN) nowadays can match human performance in challenging complex tasks, but it remains unknown whether DCNNs achieve human-like performance through human-like processes. Here we applied a reverse-correlation method to make explicit representations of DCNNs and humans when performing face gender classification. We found that humans and a typical DCNN, VGG-Face, used similar critical information for this task, which mainly resided at low spatial frequencies. Importantly, the prior task experience, which the VGG-Face was pre-trained to process faces at the subordinate level (i.e., identification) as humans do, seemed necessary for such representational similarity, because AlexNet, a DCNN pre-trained to process objects at the basic level (i.e., categorization), succeeded in gender classification but relied on a completely different representation. In sum, although DCNNs and humans rely on different sets of hardware to process faces, they can use a similar and implementation-independent representation to achieve the same computation goal.

Pinpointing the optimal spatial frequency range for automatic neural facial fear processing

Faces convey an assortment of emotional information via low and high spatial frequencies (LSFs and HSFs). However, there is no consensus on the role of particular spatial frequency (SF) information during facial fear processing. Comparison across studies is hampered by the high variability in cut-off values for demarcating the SF spectrum and by differences in task demands. We investigated which SF information is minimally required to rapidly detect briefly presented fearful faces in an implicit and automatic manner, by sweeping through an entire SF range without constraints of predefined cut-offs for LSFs and HSFs. We combined fast periodic visual stimulation with electroencephalography. We presented neutral faces at 6 ​Hz, periodically interleaved every 5th image with a fearful face, allowing us to quantify an objective neural index of fear discrimination at exactly 1.2 ​Hz. We started from a stimulus containing either only very low or very high SFs and gradually increased the SF content by adding higher or lower SF information, respectively, to reach the full SF spectrum over the course of 70 ​s. We found that faces require at least SF information higher than 5.93 cycles per image (cpi) to implicitly differentiate fearful from neutral faces. However, exclusive HSF faces, even in a restricted SF range between 94.82 and 189.63 cpi already carry the critical information to extract the emotional expression of the faces.

Time course of spatial frequency integration in face perception: An ERP study

Face perception is based on the processing and integration of multiple spatial frequency (SF) ranges. However, the temporal dynamics of SF integration to form an early face representation in the human brain is still a matter of debate. To address this issue, we recorded event-related potentials (ERPs) during the presentation of spatial frequency-manipulated facial images. Twenty-six participants performed a gender discrimination task on non-filtered, low-, high-, and band-pass filtered face images, corresponding, respectively, to the full range, spatial frequencies up to 8 cycles/image, above 32 cycles/image, and from 8 to 16 cycles/image. Behaviorally, the task related-performance was more accurate and faster for non-filtered (NF) and mid-range SF (MSF) than for low SF (LSF) and high SF (HSF) stimuli. At both behavioral and electrophysiological levels, response to MSF contained in faces did not differ from the responses to full spectrum non-filtered (NF) facial images. In ERPs, LSF facial images evoked the largest P1 amplitude while HSF facial images evoked the largest N170 amplitude compared with the other three conditions. Since LSFs and HSFs would transmit global and local information respectively, our observations lend further support to the "coarse-to-fine" processing theory of faces. Furthermore, they offer original evidence of the effectiveness and adequacy of the mid-range spatial frequency in face perception. Possible theoretical interpretations of our findings are discussed.

Early Left Parietal Activity Elicited by Direct Gaze: A High-Density EEG Study

Gaze is one of the most important cues for human communication and social interaction. In particular, gaze contact is the most primary form of social contact and it is thought to capture attention. A very early-differentiated brain response to direct versus averted gaze has been hypothesized. Here, we used high-density electroencephalography to test this hypothesis. Topographical analysis allowed us to uncover a very early topographic modulation (40-80 ms) of event-related responses to faces with direct as compared to averted gaze. This modulation was obtained only in the condition where intact broadband faces-as opposed to high-pass or low-pas filtered faces-were presented. Source estimation indicated that this early modulation involved the posterior parietal region, encompassing the left precuneus and inferior parietal lobule. This supports the idea that it reflected an early orienting response to direct versus averted gaze. Accordingly, in a follow-up behavioural experiment, we found faster response times to the direct gaze than to the averted gaze broadband faces. In addition, classical evoked potential analysis showed that the N170 peak amplitude was larger for averted gaze than for direct gaze. Taken together, these results suggest that direct gaze may be detected at a very early processing stage, involving a parallel route to the ventral occipito-temporal route of face perceptual analysis.

On the particular vulnerability of face recognition to aging: a review of three hypotheses

Age-related face recognition deficits are characterized by high false alarms to unfamiliar faces, are not as pronounced for other complex stimuli, and are only partially related to general age-related impairments in cognition. This paper reviews some of the underlying processes likely to be implicated in theses deficits by focusing on areas where contradictions abound as a means to highlight avenues for future research. Research pertaining to the three following hypotheses is presented: (i) perceptual deterioration, (ii) encoding of configural information, and (iii) difficulties in recollecting contextual information. The evidence surveyed provides support for the idea that all three factors are likely to contribute, under certain conditions, to the deficits in face recognition seen in older adults. We discuss how these different factors might interact in the context of a generic framework of the different stages implicated in face recognition. Several suggestions for future investigations are outlined.

Modulations of eye movement patterns by spatial filtering during the learning and testing phases of an old/new face recognition task

In two experiments, we examined the effects of varying the spatial frequency (SF) content of face images on eye movements during the learning and testing phases of an old/new recognition task. At both learning and testing, participants were presented with face stimuli band-pass filtered to 11 different SF bands, as well as an unfiltered baseline condition. We found that eye movements varied significantly as a function of SF. Specifically, the frequency of transitions between facial features showed a band-pass pattern, with more transitions for middle-band faces (≈5-20 cycles/face) than for low-band (≈<5 cpf) or high-band (≈>20 cpf) ones. These findings were similar for the learning and testing phases. The distributions of transitions across facial features were similar for the middle-band, high-band, and unfiltered faces, showing a concentration on the eyes and mouth; conversely, low-band faces elicited mostly transitions involving the nose and nasion. The eye movement patterns elicited by low, middle, and high bands are similar to those previous researchers have suggested reflect holistic, configural, and featural processing, respectively. More generally, our results are compatible with the hypotheses that eye movements are functional, and that the visual system makes flexible use of visuospatial information in face processing. Finally, our finding that only middle spatial frequencies yielded the same number and distribution of fixations as unfiltered faces adds more evidence to the idea that these frequencies are especially important for face recognition, and reveals a possible mediator for the superior performance that they elicit.

Configural and featural discriminations use the same spatial frequencies: a model observer versus human observer analysis

Previous work has shown mixed results regarding the role of different spatial frequency (SF) ranges in featural and configural processing of faces. Some studies suggest no special role of any given band for either type of processing, while others suggest that low SFs principally support configural analysis. Here we attempt to put this issue on a more rigorous footing by comparing human performance when making featural and configural discriminations with that of a model observer algorithm carrying out the same task. The model uses a simple algorithm that calculates the dot product of a stimulus image with each available potential match image to find the maximally likely match. It thus provides a principled way of analyzing available image information. We find human accuracy peaks at around 10 cycles per face (cpf) regardless of whether featural or configural manipulations are being detected. We also find accuracy peaks in the same part of the spectrum regardless of which feature is manipulated (ie eyes, nose, or mouth). Conversely, model observer performance, measured in terms of white noise tolerance, peaks at approximately 5 cpf, and this value again remains roughly constant regardless of the type of manipulation and feature manipulated. The ratio of the model's noise tolerance to a derived equivalent noise tolerance value for humans peaks at around 10 cpf, similar to the accuracy data. These results provide evidence that the human performance maxima at 10 cpf are not due simply to the physical characteristics of face stimuli, but rather arise due to an interaction between the available information in face images and human perceptual processing.

Multiple faces elicit augmented neural activity

How do our brains respond when we are being watched by a group of people?Despite the large volume of literature devoted to face processing, this question has received very little attention. Here we measured the effects on the face-sensitive N170 and other ERPs to viewing displays of one, two and three faces in two experiments. In Experiment 1, overall image brightness and contrast were adjusted to be constant, whereas in Experiment 2 local contrast and brightness of individual faces were not manipulated. A robust positive-negative-positive (P100-N170-P250) ERP complex and an additional late positive ERP, the P400, were elicited to all stimulus types. As the number of faces in the display increased, N170 amplitude increased for both stimulus sets, and latency increased in Experiment 2. P100 latency and P250 amplitude were affected by changes in overall brightness and contrast, but not by the number of faces in the display per se. In Experiment 1 when overall brightness and contrast were adjusted to be constant, later ERP (P250 and P400) latencies showed differences as a function of hemisphere. Hence, our data indicate that N170 increases its magnitude when multiple faces are seen, apparently impervious to basic low-level stimulus features including stimulus size. Outstanding questions remain regarding category-sensitive neural activity that is elicited to viewing multiple items of stimulus categories other than faces.