Learning to Discriminate Face Views

Profiles of visual perceptual learning in feature space

Visual perceptual learning (VPL), experience-induced gains in discriminating visual features, has been studied extensively and intensively for many years, its profile in feature space, however, remains unclear. Here, human subjects were trained to perform either a simple low-level feature (grating orientation) or a complex high-level object (face view) discrimination task over a long-time course. During, immediately after, and one month after training, all results showed that in feature space VPL in grating orientation discrimination was a center-surround profile; VPL in face view discrimination, however, was a monotonic gradient profile. Importantly, these two profiles can be emerged by a deep convolutional neural network with a modified AlexNet consisted of 7 and 12 layers, respectively. Altogether, our study reveals for the first time a feature hierarchy-dependent profile of VPL in feature space, placing a necessary constraint on our understanding of the neural computation of VPL.

Effect of facial emotion recognition learning transfers across emotions

Introduction

Perceptual learning of facial expression is shown specific to the train expression, indicating separate encoding of the emotional contents in different expressions. However, little is known about the specificity of emotional recognition training with the visual search paradigm and the sensitivity of learning to near-threshold stimuli.

Methods

In the present study, we adopted a visual search paradigm to measure the recognition of facial expressions. In Experiment 1 (Exp1), Experiment 2 (Exp2), and Experiment 3 (Exp3), subjects were trained for 8 days to search for a target expression in an array of faces presented for 950 ms, 350 ms, and 50 ms, respectively. In Experiment 4 (Exp4), we trained subjects to search for a target of a triangle, and tested them with the task of facial expression search. Before and after the training, subjects were tested on the trained and untrained facial expressions which were presented for 950 ms, 650 ms, 350 ms, or 50 ms.

Results

The results showed that training led to large improvements in the recognition of facial emotions only if the faces were presented long enough (Exp1: 85.89%; Exp2: 46.05%). Furthermore, the training effect could transfer to the untrained expression. However, when the faces were presented briefly (Exp3), the training effect was small (6.38%). In Exp4, the results indicated that the training effect could not transfer across categories.

Discussion

Our findings revealed cross-emotion transfer for facial expression recognition training in a visual search task. In addition, learning hardly affects the recognition of near-threshold expressions.

The development of upright face perception depends on evolved orientation-specific mechanisms and experience

Here we examine whether our impressive ability to perceive upright faces arises from evolved orientation-specific mechanisms, our extensive experience with upright faces, or both factors. To do so, we tested Claudio, a man with a congenital joint disorder causing his head to be rotated back so that it is positioned between his shoulder blades. As a result, Claudio has seen more faces reversed in orientation to his own face than matched to it. Controls exhibited large inversion effects on all tasks, but Claudio performed similarly with upright and inverted faces in both detection and identity-matching tasks, indicating these abilities are the product of evolved mechanisms and experience. In contrast, he showed clear upright superiority when detecting "Thatcherized" faces (faces with vertically flipped features), suggesting experience plays a greater role in this judgment. Together, these findings indicate that both evolved orientation-specific mechanisms and experience contribute to our proficiency with upright faces.

Identifying Long- and Short-Term Processes in Perceptual Learning

Practice makes perfect in almost all perceptual tasks, but how perceptual improvements accumulate remains unknown. Here, we developed a multicomponent theoretical framework to model contributions of both long- and short-term processes in perceptual learning. Applications of the framework to the block-by-block learning curves of 49 adult participants in seven perceptual tasks identified ubiquitous long-term general learning and within-session relearning in most tasks. More importantly, we also found between-session forgetting in the vernier-offset discrimination, face-view discrimination, and auditory-frequency discrimination tasks; between-session off-line gain in the visual shape search task; and within-session adaptation and both between-session forgetting and off-line gain in the contrast detection task. The main results of the vernier-offset discrimination and visual shape search tasks were replicated in a new experiment. The multicomponent model provides a theoretical framework to identify component processes in perceptual learning and a potential tool to optimize learning in normal and clinical populations.

Relative tuning of holistic face processing towards the fovea

Humans quickly detect and gaze at faces in the world, which reflects their importance in cognition and may lead to tuning of face recognition toward the central visual field. Although sometimes reported, foveal selectivity in face processing is debated: brain imaging studies have found evidence for a central field bias specific to faces, but behavioral studies have found little foveal selectivity in face recognition. These conflicting results are difficult to reconcile, but they could arise from stimulus-specific differences. Recent studies, for example, suggest that individual faces vary in the degree to which they require holistic processing. Holistic processing is the perception of faces as a whole rather than as a set of separate features. We hypothesized that the dissociation between behavioral and neuroimaging studies arises because of this stimulus-specific dependence on holistic processing. Specifically, the central bias found in neuroimaging studies may be specific to holistic processing. Here, we tested whether the eccentricity-dependence of face perception is determined by the degree to which faces require holistic processing. We first measured the holistic-ness of individual Mooney faces (two-tone shadow images readily perceived as faces). In a group of independent observers, we then used a gender discrimination task to measured recognition of these Mooney faces as a function of their eccentricity. Face gender was recognized across the visual field, even at substantial eccentricities, replicating prior work. Importantly, however, holistic face gender recognition was relatively tuned-slightly, but reliably stronger in the central visual field. Our results may reconcile the debate on the eccentricity-dependance of face perception and reveal a spatial inhomogeneity specifically in the holistic representations of faces.

Effects of High-Definition Transcranial Direct Current Stimulation Over the Left Fusiform Face Area on Face View Discrimination Depend on the Individual Baseline Performance

A basic human visual function is to identify objects from different viewpoints. Typically, the ability to discriminate face views based on in-depth orientation is necessary in daily life. Early neuroimaging studies have identified the involvement of the left fusiform face area (FFA) and the left superior temporal sulcus (STS) in face view discrimination. However, many studies have documented the important role of the right FFA in face processing. Thus, there remains controversy over whether one specific region or all of them are involved in discriminating face views. Thus, this research examined the influence of high-definition transcranial direct current stimulation (HD-tDCS) over the left FFA, left STS or right FFA on face view discrimination in three experiments. In experiment 1, eighteen subjects performed a face view discrimination task before and immediately, 10 min and 20 min after anodal, cathodal and sham HD-tDCS (20 min, 1.5 mA) over the left FFA in three sessions. Compared with sham stimulation, anodal and cathodal stimulation had no effects that were detected at the group level. However, the analyses at the individual level showed that the baseline performance negatively correlated with the degree of change after anodal tDCS, suggesting a dependence of the change amount on the initial performance. Specifically, tDCS decreased performance in the subjects with better baseline performance but increased performance in those with poorer baseline performance. In experiments 2 and 3, the same experimental protocol was used except that the stimulation site was the left STS or right FFA, respectively. Neither anodal nor cathodal tDCS over the left STS or right FFA influenced face view discrimination in group- or individual-level analyses. These results not only indicated the importance of the left FFA in face view discrimination but also demonstrated that individual initial performance should be taken into consideration in future research and practical applications.

The Effect of Visual Working Memory Training Could Transfer Across Stimuli

Background

Working memory, as a fundamental cognitive ability, has been shown to improve with learning. However, little is known about the learning effect of visual working memory training and its generalization to other stimuli and tasks.

Methods

In the present study, we utilized a delayed match-to-sample task to measure the working memory of faces and houses. Subjects were trained ten days on this task and were tested on the same task and a memory span task before and after the training.

Results

The results showed that training significantly increased the accuracy of visual working memory. More importantly, such a learning effect could partly transfer to a visual working memory task with different stimuli. However, the learning effect may not transfer to a memory span task.

Conclusion

Our findings demonstrate that training might influence the common processing of different stimuli in a visual working memory task.

Metacognitive Accuracy Improves With the Perceptual Learning of a Low- but Not High-Level Face Property

Experience with visual stimuli can improve their perceptual performance, a phenomenon termed visual perceptual learning (VPL). VPL has been found to improve metacognitive measures, suggesting increased conscious accessibility to the knowledge supporting perceptual decision-making. However, such studies have largely failed to control objective task accuracy, which typically correlates with metacognition. Here, using a staircase method to control this confound, we investigated whether VPL improves the metacognitive accuracy of perceptual decision-making. Across 3 days, subjects were trained to discriminate faces based on their high-level identity or low-level contrast. Holding objective accuracy constant across training days, perceptual thresholds decreased in both tasks, demonstrating VPL in our protocol. However, whilemetacognitive accuracy was not affected by face contrast VPL, it was decreased by face identity VPL. Our findings couldbe parsimoniously explained by a dual-stage signal detection theory-based model involving an initial perceptual decision-making stage and a second confidence judgment stage. Within this model, internal noise reductions for both stages accounts for our face contrast VPL result, while only first stage noise reductions accounts for our face identity VPL result. In summary, we found evidence suggesting that conscious knowledge accessibility was improved by the VPL of face contrast but not face identity.

Disentangling locus of perceptual learning in the visual hierarchy of motion processing

Visual perceptual learning (VPL) can lead to long-lasting perceptual improvements. One of the central topics in VPL studies is the locus of plasticity in the visual processing hierarchy. Here, we tackled this question in the context of motion processing. We took advantage of an established transition from component-dependent representations at the earliest level to pattern-dependent representations at the middle-level of cortical motion processing. Two groups of participants were trained on the same motion direction identification task using either grating or plaid stimuli. A set of pre- and post-training tests was used to determine the degree of learning specificity and generalizability. This approach allowed us to disentangle contributions from different levels of processing stages to behavioral improvements. We observed a complete bi-directional transfer of learning between component and pattern stimuli that moved to the same directions, indicating learning-induced plasticity associated with intermediate levels of motion processing. Moreover, we found that motion VPL is specific to the trained stimulus direction, speed, size, and contrast, diminishing the possibility of non-sensory decision-level enhancements. Taken together, these results indicate that, at least for the type of stimuli and the task used here, motion VPL most likely alters visual computation associated with signals at the middle stage of motion processing.

Perceptual learning and recognition confusion reveal the underlying relationships among the six basic emotions

The six basic emotions (disgust, anger, fear, happiness, sadness, and surprise) have long been considered discrete categories that serve as the primary units of the emotion system. Yet recent evidence indicated underlying connections among them. Here we tested the underlying relationships among the six basic emotions using a perceptual learning procedure. This technique has the potential of causally changing participants' emotion detection ability. We found that training on detecting a facial expression improved the performance not only on the trained expression but also on other expressions. Such a transfer effect was consistently demonstrated between disgust and anger detection as well as between fear and surprise detection in two experiments (Experiment 1A, n = 70; Experiment 1B, n = 42). Notably, training on any of the six emotions could improve happiness detection, while sadness detection could only be improved by training on sadness itself, suggesting the uniqueness of happiness and sadness. In an emotion recognition test using a large sample of Chinese participants (n = 1748), the confusion between disgust and anger as well as between fear and surprise was further confirmed. Taken together, our study demonstrates that the "basic" emotions share some common psychological components, which might be the more basic units of the emotion system.

Unsupervised visual discrimination learning of complex stimuli: Accuracy, bias and generalization

Through same-different judgements, we can discriminate an immense variety of stimuli and consequently, they are critical in our everyday interaction with the environment. The quality of the judgements depends on familiarity with stimuli. A way to improve the discrimination is through learning, but to this day, we lack direct evidence of how learning shapes the same-different judgments with complex stimuli. We studied unsupervised visual discrimination learning in 42 participants, as they performed same-different judgments with two types of unfamiliar complex stimuli in the absence of labeling or individuation. Across nine daily training sessions with equiprobable same and different stimuli pairs, participants increased the sensitivity and the criterion by reducing the errors with both same and different pairs. With practice, there was a superior performance for different pairs and a bias for different response. To evaluate the process underlying this bias, we manipulated the proportion of same and different pairs, which resulted in an additional proportion-induced bias, suggesting that the bias observed with equal proportions was a stimulus processing bias. Overall, these results suggest that unsupervised discrimination learning occurs through changes in the stimulus processing that increase the sensory evidence and/or the precision of the working memory. Finally, the acquired discrimination ability was fully transferred to novel exemplars of the practiced stimuli category, in agreement with the acquisition of a category specific perceptual expertise.

Towards a whole brain model of Perceptual Learning

A hallmark of modern Perceptual Learning (PL) is the extent to which learning is specific to the trained stimuli. Such specificity to orientation, spatial location and even eye of training has been used as psychophysical evidence of the neural basis of learning. This argument that specificity of PL implies regionalization of brain plasticity implicitly assumes that examination of a singular locus of PL is an appropriate approach to understand learning. However, recent research shows that learning effects once thought to be specific depend on subtleties of the training paradigm and that within even a simple training procedure there are multiple aspects of the task and stimuli that are learned simultaneously. Here, we suggest that learning on any task involves a broad network of brain regions undergoing changes in representations, read-out weights, decision rules, attention and feedback processes as well as oculomotor changes. However, importantly, the distribution of learning across the neural system depends upon the details of the training procedure and the characterstics of the individual being trained. We propose that to advance our understanding of PL, the field must move towards understanding how distributed brain processes jointly contribute to behavioral learning effects.

Neural mechanisms of motion perceptual learning in noise

Practice improves our perceptual ability. However, the neural mechanisms underlying this experience-dependent plasticity in adult brain remain unclear. Here, we studied the long-term neural correlates of motion perceptual learning. Subjects' behavioral performance and BOLD signals were tracked before, immediately after, and 2 weeks after practicing a motion direction discrimination task in noise over six daily sessions. Parallel to the specificity and persistency of the behavioral learning effect, we found that training sharpened the cortical tuning in MT, and enhanced the connectivity strength from MT to the intraparietal sulcus (IPS, a motion decision-making area). In addition, the decoding accuracy for the trained motion direction was improved in IPS 2 weeks after training. The dual changes in the sensory and the high-level cortical areas suggest that learning refines the neural representation of the trained stimulus and facilitates the information transmission in the decision process. Our findings are consistent with the functional specialization in the visual cortex, and provide empirical evidence to the reweighting theory of perceptual learning at a large spatial scale. Hum Brain Mapp 38:6029-6042, 2017. © 2017 Wiley Periodicals, Inc.

Human sensory cortex structure and top-down controlling brain network determine individual differences in perceptual alternations

Bistable perception is a type of subjective perception that spontaneously alternates between two perceptual interpretations of an ambiguous sensory input. Past functional magnetic resonance imaging (fMRI) studies have examined the activation patterns underlying bistable perception, yet the variability between individuals in the alternations is not well understood. Therefore, voxel-based morphometry (VBM) was introduced in this study to correlate the GM of the sensory cortex with the alternations of Rubin face-vase illusion in a large group of young adults. We found that the GM volume and density (GMV/GMD) of the left fusiform face area (FFA) were significantly positively correlated with the alternations. Next, Granger causality analysis (GCA) was introduced to investigate the top-down modulation from high-level areas to the sensory cortex using resting-state fMRI data. Correlations between the perceptual alternations and Granger causalities showed that the top-down modulations from high-level brain regions, such as the superior parietal lobule (SPL) to the left FFA, were positive. Together, these findings indicated that the anatomical structure of the face-selective area may determine individual alternations of the Rubin face-vase illusion. This process may be controlled by a high-level cortex associated with attentional modulation, such as the SPL or Posterior Cingulate Cortex (PCC).

Perceptual Learning of Facial Expressions

Perceptual learning is a phenomenon in which intensive training for a perceptual task may lead to significant improvement in the task performance. So far, the characteristics of the perceptual learning of facial expressions have not been investigated. In the current study, we trained subjects to distinguish facial expressions. With eight days of training, the subjects' discrimination performance improved significantly, and this improvement was generalized to faces with the same expression but different gender as the trained face. In the second experiment, we further examined the transfer of the learning effect between faces with different expression intensities. We found that the learning effect of happiness can be transferred from the high-intensity face to the low-intensity face, but the reverse was not true. Importantly, in all experiments, we measured the performance immediately after training and one month after training. The results showed that all learning effects and transfers were able to persist for at least one month, which implied that these findings revealed the long-term mechanisms of training. These results revealed the characteristics of facial expression learning and shed light on the mechanisms of perceptual learning for high-level vision.

The perception of a familiar face is no more than the sum of its parts

Why do faces become easier to recognize with repeated exposure? Previous research has suggested that familiarity may induce a qualitative shift in visual processing from an independent analysis of individual facial features to analysis that includes information about the relationships among features (Farah, Wilson, Drain, & Tanaka Psychological Review, 105, 482-498, 1998; Maurer, Grand, & Mondloch Trends in Cognitive Science, 6, 255-260, 2002). We tested this idea by using a "summation-at-threshold" technique (Gold, Mundy, & Tjan Psychological Science, 23, 427-434, 2012; Nandy & Tjan Journal of Vision, 8, 3.1-20, 2008), in which an observer's ability to recognize each individual facial feature shown independently is used to predict their ability to recognize all of the features shown in combination. We find that, although people are better overall at recognizing familiar as opposed to unfamiliar faces, their ability to integrate information across features is similar for unfamiliar and highly familiar faces and is well predicted by their ability to recognize each of the facial features shown in isolation. These results are consistent with the idea that familiarity has a quantitative effect on the efficiency with which information is extracted from individual features, rather than a qualitative effect on the process by which features are combined.

Learning faces from variability

Research on face learning has tended to use sets of images that vary systematically on dimensions such as pose and illumination. In contrast, we have proposed that exposure to naturally varying images of a person may be a critical part of the familiarization process. Here, we present two experiments investigating face learning with "ambient images"-relatively unconstrained photos taken from internet searches. Participants learned name and face associations for unfamiliar identities presented in high or low within-person variability-that is, images of the same person returned by internet search on their name (high variability) versus different images of the same person taken from the same event (low variability). In Experiment 1 we show more accurate performance on a speeded name verification task for identities learned in high than in low variability, when the test images are completely novel photos. In Experiment 2 we show more accurate performance on a face matching task for identities previously learned in high than in low variability. The results show that exposure to a large range of within-person variability leads to enhanced learning of new identities.

Task-irrelevant emotion facilitates face discrimination learning

We understand poorly how the ability to discriminate faces from one another is shaped by visual experience. The purpose of the present study is to determine whether face discrimination learning can be facilitated by facial emotions. To answer this question, we used a task-irrelevant perceptual learning paradigm because it closely mimics the learning processes that, in daily life, occur without a conscious intention to learn and without an attentional focus on specific facial features. We measured face discrimination thresholds before and after training. During the training phase (4 days), participants performed a contrast discrimination task on face images. They were not informed that we introduced (task-irrelevant) subtle variations in the face images from trial to trial. For the Identity group, the task-irrelevant features were variations along a morphing continuum of facial identity. For the Emotion group, the task-irrelevant features were variations along an emotional expression morphing continuum. The Control group did not undergo contrast discrimination learning and only performed the pre-training and post-training tests, with the same temporal gap between them as the other two groups. Results indicate that face discrimination improved, but only for the Emotion group. Participants in the Emotion group, moreover, showed face discrimination improvements also for stimulus variations along the facial identity dimension, even if these (task-irrelevant) stimulus features had not been presented during training. The present results highlight the importance of emotions for face discrimination learning.

Dissociable processes for orientation discrimination learning and contextual illusion magnitude

Previous research suggests an inverse relationship between human orientation discrimination sensitivity and tilt illusion magnitude. To test whether these perceptual functions are inherently linked, we measured both orientation discrimination sensitivity and the magnitude of the tilt illusion before and after participants had been trained for three days on an orientation discrimination task. Discrimination sensitivity improved with training and this improvement remained one month after the initial learning. However, tilt illusion magnitude remained unchanged before and after orientation training, at either trained or untrained orientations. Our results suggest that orientation discrimination sensitivity and illusion magnitude are not inherently linked. They also provide further evidence that, at least for the training periods we employed, perceptual learning of orientation discrimination may involve high-level processes.

Function and structure of human left fusiform cortex are closely associated with perceptual learning of faces

Training can lead to long-lasting improvement in our perceptual ability, which is referred to as perceptual learning. Unraveling its neural mechanisms has proved difficult. With functional and structural magnetic resonance imaging (MRI), we addressed this issue by searching for the neural correlates of perceptual learning of face views over a long time course. Human subjects were trained to perform a face view discrimination task. Their behavioral performance and MRI signals were measured before, immediately after, and 1 month after training. We found that, across individual subjects, their behavioral learning effects correlated with the stability improvement of spatial activity pattern in the left fusiform cortex immediately after and 1 month after training. We also found that the thickness of the left fusiform cortex before training could predict subjects' behavioral learning effects. These findings, for the first time, not only suggest that, remarkably, the improved pattern stability contributes to the long-term mechanisms of perceptual learning, but also provide strong and converging evidence for the pivotal role of the left fusiform cortex in adaptive face processing.

Implicit learning of geometric eigenfaces

The human visual system can implicitly extract a prototype of encountered visual objects (Posner & Keele, 1968). While learning a prototype provides an efficient way of encoding objects at the category level, discrimination among individual objects requires encoding of variations among them as well. Here we show that in addition to the prototype, human adults also implicitly learn the feature correlations that capture the most significant geometric variations among faces. After studying a group of synthetic faces, observers mistook as seen previously unseen faces representing the first two principal components (eigenfaces, Turk & Pentland, 1991) of the studied faces at significantly higher rates than the correct recognition of the faces actually studied. Implicit learning of the most significant eigenfaces provides an optimal way for encoding variations among faces. The data thus extend the types of summary statistics that can be implicitly extracted by the visual system to include several principal components.

The effect of spatial frequency on perceptual learning of inverted faces

We investigated the efficacy of training adults to recognize full spectrum inverted faces presented with different viewpoints. To examine the role of different spatial frequencies in any learning, we also used high-pass filtered faces that preserved featural information and low-pass filtered faces that severely reduced that featural information. Although all groups got faster over the 2 days of training, there was more improvement in accuracy for the group exposed to full spectrum faces than in the two groups exposed to filtered faces, both of which improved more modestly and only when the same faces were shown on the 2 days of training. For the group exposed to the full spectrum range and, to a lesser extent, for those exposed to high frequency faces, training generalized to a new set of full spectrum faces of a different size in a different task, but did not lead to evidence of holistic processing or improved sensitivity to feature shape or spacing in inverted faces. Overall these results demonstrate that only 2h of practice in recognizing full-spectrum inverted faces presented from multiple points of view is sufficient to improve recognition of the trained faces and to generalize to novel instances. Perceptual learning also occurred for low and high frequency faces but to a smaller extent.

Neural correlates of face gender discrimination learning

Using combined psychophysics and event-related potentials (ERPs), we investigated the effect of perceptual learning on face gender discrimination and probe the neural correlates of the learning effect. Human subjects were trained to perform a gender discrimination task with male or female faces. Before and after training, they were tested with the trained faces and other faces with the same and opposite genders. ERPs responding to these faces were recorded. Psychophysical results showed that training significantly improved subjects' discrimination performance and the improvement was specific to the trained gender, as well as to the trained identities. The training effect indicates that learning occurs at two levels-the category level (gender) and the exemplar level (identity). ERP analyses showed that the gender and identity learning was associated with the N170 latency reduction at the left occipital-temporal area and the N170 amplitude reduction at the right occipital-temporal area, respectively. These findings provide evidence for the facilitation model and the sharpening model on neuronal plasticity from visual experience, suggesting a faster processing speed and a sparser representation of face induced by perceptual learning.

Tilt aftereffect from orientation discrimination learning

Using psychophysical methods, the present study aims to investigate whether orientation discrimination learning can modify the visual cortex and how the modification is related to the development of perceptual learning (performance improvement). In Experiment 1, subjects were trained with an orientation discrimination task at the orientation of 15° right tilted from the vertical. The training not only improved subjects' orientation discrimination performance, but also shifted their perceived vertical toward the trained orientation, which resembles a well-known visual illusion-tilt aftereffect. Interestingly, the change of perceived vertical and performance improvement had different dynamics. Subjects' performance levels were maintained at a constant level, even 2 months after training. However, their perceived vertical changes reduced dramatically 1 week after training. In addition, Experiment 1 showed that there was only a weak transfer of the perceived vertical change from the trained location to the untrained location. Experiment 2 demonstrated that subjects' perceived vertical was not affected by training at 75°. These results suggest that the orientation discrimination training could modify neuronal responses in human retinotopic visual areas and the development of perceptual learning is not a single neural process.