Similarities and differences in the perceptual structure of facial expressions of children and adults

Motion's privilege in recognizing facial expressions following treatment for blindness

In his 1872 monograph, Charles Darwin posited that "… the habit of expressing our feelings by certain movements, though now rendered innate, had been in some manner gradually acquired."1 Nearly 150 years later, researchers are still teasing apart innate versus experience-dependent contributions to expression recognition. Indeed, studies have shown that face detection is surprisingly resilient to early visual deprivation,2,3,4,5 pointing to plasticity that extends beyond dogmatic critical periods.6,7,8 However, it remains unclear whether such resilience extends to downstream processing, such as the ability to recognize facial expressions. The extent to which innate versus experience-dependent mechanisms contribute to this ability has yet to be fully explored.9,10,11,12,13 To investigate the impact of early visual experience on facial-expression recognition, we studied children with congenital cataracts who have undergone sight-correcting treatment14,15 and tracked their longitudinal skill acquisition as they gain sight late in life. We introduce and explore two potential facilitators of late-life plasticity: the availability of newborn-like coarse visual acuity prior to treatment16 and the privileged role of motion following treatment.4,17,18 We find that early visual deprivation does not preclude partial acquisition of facial-expression recognition. While rudimentary pretreatment vision is sufficient to allow a low level of expression recognition, it does not facilitate post-treatment improvements. Additionally, only children commencing vision with high visual acuity privilege the use of dynamic cues. We conclude that skipping typical visual experience early in development and introducing high-resolution imagery late in development restricts, but does not preclude, facial-expression skill acquisition and that the representational mechanisms driving this learning differ from those that emerge during typical visual development.

Individual Differences in Emotional Appraisal during Development: Analysis of the Role of Age, Gender, and Appraisal Accuracy

Emotional appraisal is the process by which different responses are generated from subjective assessments of different stimuli. Children and adults can react differently to the same stimulus. Emotional appraisal is crucial to define emotional processing and its consequent response rather than the properties of the stimulus itself. Age and gender modulate emotional appraisal during development. This study analyzed emotional appraisal in children aged 4 to 8-years-old. The children's emotional appraisal responses and the appraisal accuracy to images with positive, negative, and neutral valence be compared between the different age groups, and according to the gender of the participants. About the appraisal accuracy, the comparisons indicated significant differences for positive and negative images according to the age of the participants, with no differences in the appraisal accuracy of neutral images. Differences in appraisal accuracy for negative images were observed between the 4-years-old group compared to the 7- and 8-years-old group, as well as in 5-years-old participants compared to 7-years-old. Significant differences were found for the positive and negative images between girls and boys, but no differences were found for neutral images. Girls had more appraisals accuracy for positive and negative images. This work highlights the importance of analyzing individual differences associated with emotional processing at different stages of development. This interrogates research findings in which emotional appraisal is carried out considering the valence assigned by adults and highlights the need for a methodological approach that includes individual differences in the appraisal of stimuli used during development.

Father-child dyads exhibit unique inter-subject synchronization during co-viewing of animation video stimuli

Inter-subject synchronization reflects the entrainment of two individuals to each other's brain signals. In parent-child dyads, synchronization indicates an attunement to each other's emotional states. Despite the ubiquity with which parents and their children watch screen media together, no study has investigated synchronization in father-child dyads during co-viewing. The present study examined whether father-child dyads would exhibit inter-subject synchronization that is unique to the dyad and hence would not be observed in control dyads (i.e., randomly paired signals). Hyperscanning fNIRS was used to record the prefrontal cortex (PFC) signals of 29 fathers and their preschool-aged children as they co-viewed children's shows. Three 1-min videos from "Brave", "Peppa Pig" and "The Incredibles" were presented to each dyad and children's ratings of video positivity and familiarity were obtained. Four PFC clusters were analyzed: medial left, medial right, frontal left and frontal right clusters. Results demonstrated that true father-child dyads showed significantly greater synchronization than control dyads in the medial left cluster during the emotionally arousing conflict scene. Dyads with older fathers displayed less synchrony and older fathers, compared to younger ones, exhibited greater activity. These findings suggest unique inter-subject synchronization in father-child dyads during co-viewing which is potentially modulated by parental age.

Flexible face processing: Holistic processing of facial identity is modulated by task-irrelevant facial expression

Holistic processing is a hallmark of face perception and has been observed separately for facial identity and expression. While the identity of a face remains constant regardless of any changes in facial expressions, to what extent is holistic processing of facial identity affected by task-irrelevant facial expressions? If holistic processing is flexible and integrates both identity and expression information, the magnitude of holistic processing of facial identity may be systematically modulated by different facial expressions, due to either visual or emotional differences among the expressions. In Experiment 1, participants matched the identities of target halves of two sequentially presented face composites, with both composites showing either positive (happy) or negative (angry) expressions. The presentation duration for the test composite was either short (200 ms) or long (until a response). With a short presentation duration, the magnitude of holistic processing of identity for happy and angry composites was comparable. In contrast, with a long presentation duration, holistic processing of identity was reduced for angry compared with happy face composites. Experiment 2 replicated the results and showed reduced holistic processing of identity for face composites with either angry or neutral expressions, compared with happy expressions, given a long presentation duration. Because the modulation of facial expressions on holistic processing of facial identity was observed with long, but not short, presentation durations, these results suggest that such influence is unlikely solely due to visual differences, but may instead arise from cognitive evaluation of the emotions conveyed by facial expressions.

Age-Related Differences in Sensitivity to Facial Trustworthiness: Perceptual Representation and the Role of Emotional Development

The ability to discriminate social signals from faces is a fundamental component of human social interactions whose developmental origins are still debated. In this study, 5-year-old (N = 29) and 7-year-old children (N = 31) and adults (N = 34) made perceptual similarity and trustworthiness judgments on a set of female faces varying in level of expressed trustworthiness. All groups represented perceived similarity of the faces as a function of trustworthiness intensity, but such representation becomes more fine-grained with development. Moreover, 5-year-olds' accuracy in choosing the more trustworthy face in a pair varied as a function of children's score at the Test of Emotion Comprehension, suggesting that the ability to perform face-to-trait inferences is related to the development of emotional understanding.

The Role of Emotional Valence for the Processing of Facial and Verbal Stimuli-Positivity or Negativity Bias?

Emotional valence is predominately conveyed in social interactions by words and facial expressions. The existence of broad biases which favor more efficient processing of positive or negative emotions is still a controversial matter. While so far this question has been investigated separately for each modality, in this narrative review of the literature we focus on valence effects in processing both words and facial expressions. In order to identify the factors underlying positivity and negativity effects, and to uncover whether these effects depend on modality and age, we present and analyze three representative overviews of the literature concerning valence effects in word processing, face processing, and combinations of word and face processing. Our analysis of word processing studies points to a positivity bias or a balanced processing of positive and negative words, whereas the analysis of face processing studies showed the existence of separate positivity and negativity biases depending on the experimental paradigm. The mixed results seem to be a product of the different methods and types of stimuli being used. Interestingly, we found that children exhibit a clear positivity advantage for both word and face processing, indicating similar processing biases in both modalities. Over the course of development, the initial positivity advantage gradually disappears, and in some face processing studies even reverses into a negativity bias. We therefore conclude that there is a need for future research that systematically analyses the impact of age and modality on the emergence of these valence effects. Finally, we discuss possible explanations for the presence of the early positivity advantage and its subsequent decrease.

Emotional Tears Communicate Sadness but Not Excessive Emotions Without Other Contextual Knowledge

Contexts of face perception are diverse. They range from the social environment to body postures, from the expresser’s gaze direction to the tone of voice. In extending the research on contexts of face perception, we investigated people’s perception of tears on a face. The act of shedding tears is often perceived as an expression of sad feelings aroused by experiencing loss, disappointment, or helplessness. Alternatively, tears may also represent the excessive intensity of any emotion, such as extreme fear during an unexpected encounter with a giant bear and extreme happiness when you win a competition. Investigating these competing interpretations of tears, we found that the addition of tears to different facial expressions made the expressions conceptually closer to sad expressions. In particular, the results of the similarity analysis showed that, after the addition of tears, patterns of ratings for anger, fear, disgust, and neutral facial expressions became more similar to those for sadness expressions. The effect of tears on the ratings of basic emotions and their patterns in facial expressions are discussed.

The importance of stimulus variability when studying face processing using fast periodic visual stimulation: A novel 'mixed-emotions' paradigm

Fast Periodic Visual Stimulation (FPVS) with oddball stimuli has been used to investigate discrimination of facial identity and emotion, with studies concluding that oddball responses indicate discrimination of faces at the conceptual level (i.e., discrimination of identity and emotion), rather than low-level perceptual (visual, image-based) discrimination. However, because previous studies have utilised identical images as base stimuli, physical differences between base and oddball stimuli, rather than recognition of identity or emotion, may have been responsible for oddball responses. This study tested two new FPVS paradigms designed to distinguish recognition of expressions of emotion from detection of visual change from the base stream. In both paradigms, the oddball emotional expression was different from that of the base stream images. However, in the 'fixed-emotion' paradigm, stimulus image varied at every presentation but the emotion in the base stream remained constant, and in the 'mixed-emotions' paradigm, both stimulus image and emotion varied at every presentation, with only the oddball emotion (disgust) remaining constant. In the fixed-emotion paradigm, typical inversion effects were observed at occipital sites. In the mixed-emotions paradigm, however, inversion effects in a central cluster (indicative of higher level emotion processing) were present in typical participants, but not those with alexithymia (who are impaired at emotion recognition), suggesting that only the mixed-emotions paradigm reflects emotion recognition rather than detection of a lower-level visual change from baseline. These results have significant methodological implications for future FPVS studies (of both facial emotion and identity), suggesting that it is crucial to vary base stimuli sufficiently, such that simple physical differences between base and oddball stimuli cannot give rise to neural oddball responses.

Sound iconicity of abstract concepts: Place of articulation is implicitly associated with abstract concepts of size and social dominance

The concept of sound iconicity implies that phonemes are intrinsically associated with non-acoustic phenomena, such as emotional expression, object size or shape, or other perceptual features. In this respect, sound iconicity is related to other forms of cross-modal associations in which stimuli from different sensory modalities are associated with each other due to the implicitly perceived correspondence of their primal features. One prominent example is the association between vowels, categorized according to their place of articulation, and size, with back vowels being associated with bigness and front vowels with smallness. However, to date the relative influence of perceptual and conceptual cognitive processing on this association is not clear. To bridge this gap, three experiments were conducted in which associations between nonsense words and pictures of animals or emotional body postures were tested. In these experiments participants had to infer the relation between visual stimuli and the notion of size from the content of the pictures, while directly perceivable features did not support–or even contradicted–the predicted association. Results show that implicit associations between articulatory-acoustic characteristics of phonemes and pictures are mainly influenced by semantic features, i.e., the content of a picture, whereas the influence of perceivable features, i.e., size or shape, is overridden. This suggests that abstract semantic concepts can function as an interface between different sensory modalities, facilitating cross-modal associations.

Facial emotion perception in patients with epilepsy: A systematic review with meta-analysis

Facial emotion perception is a fundamental social competency relying on a specialised, yet distributed, neural network. This review aimed to determine whether patients with epilepsy have facial emotion perception accuracy impairments overall, or for a subset of emotions (anger, disgust, happiness, sadness, fear, and surprise), and the relationship to epilepsy type, demographic/treatment variables, and brain organisation. Database searches used PRISMA guidelines with strict inclusion/exclusion criteria. Thirty included studies assessed patients with temporal lobe (TLE; n=709), frontocentral (FCE; n=22), and genetic generalised (GGE; n=48) epilepsy. Large deficits emerged in patients with epilepsy compared to controls (n=746; Hedges' g=0.908-1.076). Patients with TLE were significantly impaired on all emotions except surprise; patients with GGE were significantly impaired in anger, disgust, and fear perception. Meta-regression of patients with TLE revealed younger age at testing was associated with lower accuracy. This review provides evidence for marked global deficits of emotion perception in epilepsy, with differential emotion-specific impairment patterns in patients with TLE and GGE.

Body emotion recognition disproportionately depends on vertical orientations during childhood

Children’s ability to recognize emotional expressions from faces and bodies develops during childhood. However, the low-level features that support accurate body emotion recognition during development have not been well characterized. This is in marked contrast to facial emotion recognition, which is known to depend upon specific spatial frequency and orientation sub-bands during adulthood, biases that develop during childhood. Here, we examined whether children’s reliance on vertical vs. horizontal orientation energy for recognizing emotional expressions in static images of bodies changed during middle childhood (5 to 10 years old). We found that while children of all ages had an adult-like bias favoring vertical orientation energy, this effect was larger at younger ages. We conclude that in terms of information use, a key feature of the development of emotion recognition is improved performance with sub-optimal features for recognition – that is, learning to use less diagnostic features of the image is a slower process than learning to use more useful features.

Iconic faces are not real faces: enhanced emotion detection and altered neural processing as faces become more iconic

Iconic representations are ubiquitous; they fill children's cartoons, add humor to newspapers, and bring emotional tone to online communication. Yet, the communicative function they serve remains unaddressed by cognitive psychology. Here, we examined the hypothesis that iconic representations communicate emotional information more efficiently than their realistic counterparts. In Experiment 1, we manipulated low-level features of emotional faces to create five sets of stimuli that ranged from photorealistic to fully iconic. Participants identified emotions on briefly presented faces. Results showed that, at short presentation times, accuracy for identifying emotion on more "cartoonized" images was enhanced. In addition, increasing contrast and decreasing featural complexity benefited accuracy. In Experiment 2, we examined an event-related potential component, the P1, which is sensitive to low-level visual stimulus features. Lower levels of contrast and complexity within schematic stimuli were also associated with lower P1 amplitudes. These findings support the hypothesis that iconic representations differ from realistic images in their ability to communicate specific information, including emotion, quickly and efficiently, and that this effect is driven by changes in low-level visual features in the stimuli.

The integration of visual context information in facial emotion recognition in 5- to 15-year-olds

The current study investigated the role of congruent visual context information in the recognition of facial emotional expression in 190 participants from 5 to 15years of age. Children performed a matching task that presented pictures with different facial emotional expressions (anger, disgust, happiness, fear, and sadness) in two conditions: with and without a visual context. The results showed that emotions presented with visual context information were recognized more accurately than those presented in the absence of visual context. The context effect remained steady with age but varied according to the emotion presented and the gender of participants. The findings demonstrated for the first time that children from the age of 5years are able to integrate facial expression and visual context information, and this integration improves facial emotion recognition.

Tuning to the Positive: Age-Related Differences in Subjective Perception of Facial Emotion

Facial expressions aid social transactions and serve as socialization tools, with smiles signaling approval and reward, and angry faces signaling disapproval and punishment. The present study examined whether the subjective experience of positive vs. negative facial expressions differs between children and adults. Specifically, we examined age-related differences in biases toward happy and angry facial expressions. Young children (5–7 years) and young adults (18–29 years) rated the intensity of happy and angry expressions as well as levels of experienced arousal. Results showed that young children—but not young adults—rated happy facial expressions as both more intense and arousing than angry faces. This finding, which we replicated in two independent samples, was not due to differences in the ability to identify facial expressions, and suggests that children are more tuned to information in positive expressions. Together these studies provide evidence that children see unambiguous adult emotional expressions through rose-colored glasses, and suggest that what is emotionally relevant can shift with development.

A small-world network model of facial emotion recognition

Various models have been proposed to increase understanding of the cognitive basis of facial emotions. Despite those efforts, interactions between facial emotions have received minimal attention. If collective behaviours relating to each facial emotion in the comprehensive cognitive system could be assumed, specific facial emotion relationship patterns might emerge. In this study, we demonstrate that the frameworks of complex networks can effectively capture those patterns. We generate 81 facial emotion images (6 prototypes and 75 morphs) and then ask participants to rate degrees of similarity in 3240 facial emotion pairs in a paired comparison task. A facial emotion network constructed on the basis of similarity clearly forms a small-world network, which features an extremely short average network distance and close connectivity. Further, even if two facial emotions have opposing valences, they are connected within only two steps. In addition, we show that intermediary morphs are crucial for maintaining full network integration, whereas prototypes are not at all important. These results suggest the existence of collective behaviours in the cognitive systems of facial emotions and also describe why people can efficiently recognize facial emotions in terms of information transmission and propagation. For comparison, we construct three simulated networks--one based on the categorical model, one based on the dimensional model, and one random network. The results reveal that small-world connectivity in facial emotion networks is apparently different from those networks, suggesting that a small-world network is the most suitable model for capturing the cognitive basis of facial emotions.

Infants recognize the subtle happiness expression

Facial movement facilitates the recognition of facial expressions. While an intense expression is expressive enough to be recognized in a still image, a subtle expression can be recognized only in motion (Ambadar, Schooler, & Cohn, 2005, Psychological Science, 16, 403-410). The present study investigated whether infants recognize a subtle expression, and whether facial movement facilitates infants' recognition of a subtle expression. In experiment 1 4- to 7-month-old infants were tested for their spontaneous preference for a happy subtle expression rather than a neutral face, but they did not show a spontaneous preference. To confirm that infants did not recognize the static subtle expression, we conducted experiment 2 using the familiarization-novelty procedure. Infants were first familiarized with a static subtle happy expression. Following familiarization, they were presented with a pair of peak expressions of happiness and anger, but showed no significant novelty preference. In experiment 3 we presented the subtle expression dynamically. Infants were familiarized with a dynamic subtle expression and were tested for their novelty preference. The 6- to 7-month-olds showed a significant novelty preference, while 4- to 5-month-olds did not. These results suggest that infants can recognize the subtle expression only in dynamic presentation and that facial movement facilitates infants' recognition of facial expression.

Altered representation of facial expressions after early visual deprivation

We investigated the effects of early visual deprivation on the underlying representation of the six basic emotions. Using multi-dimensional scaling (MDS), we compared the similarity judgments of adults who had missed early visual input because of bilateral congenital cataracts to control adults with normal vision. Participants made similarity judgments of the six basic emotional expressions, plus neutral, at three different intensities. Consistent with previous studies, the similarity judgments of typical adults could be modeled with four underlying dimensions, which can be interpreted as representing pleasure, arousal, potency and intensity of expressions. As a group, cataract-reversal patients showed a systematic structure with dimensions representing pleasure, potency, and intensity. However, an arousal dimension was not obvious in the patient group's judgments. Hierarchical clustering analysis revealed a pattern in patients seen in typical 7-year-olds but not typical 14-year-olds or adults. There was also more variability among the patients than among the controls, as evidenced by higher stress values for the MDS fit to the patients' data and more dispersed weightings on the four dimensions. The findings suggest an important role for early visual experience in shaping the later development of the representations of emotions. Since the normal underlying structure for emotion emerges postnatally and continues to be refined until late childhood, the altered representation of emotion in adult patients suggests a sleeper effect.

Asymmetries of influence: differential effects of body postures on perceptions of emotional facial expressions

The accuracy and speed with which emotional facial expressions are identified is influenced by body postures. Two influential models predict that these congruency effects will be largest when the emotion displayed in the face is similar to that displayed in the body: the emotional seed model and the dimensional model. These models differ in whether similarity is based on physical characteristics or underlying dimensions of valence and arousal. Using a 3-alternative forced-choice task in which stimuli were presented briefly (Exp 1a) or for an unlimited time (Exp 1b) we provide evidence that congruency effects are more complex than either model predicts; the effects are asymmetrical and cannot be accounted for by similarity alone. Fearful postures are especially influential when paired with facial expressions, but not when presented in a flanker task (Exp 2). We suggest refinements to each model that may account for our results and suggest that additional studies be conducted prior to drawing strong theoretical conclusions.

Social skills improvement in children with high-functioning autism: a pilot randomized controlled trial

High-functioning autism (HFA) is characterized by persistent impairment in social interaction despite the absence of mental retardation. Although an increasing number of group-based programs for the improvement of social skills have been described, randomized controlled trials are needed to evaluate their efficacy. To compare the effect of a Social Skills Training Group-based Program (SST-GP) and a Leisure Activities Group-based Program (LA-GP) on the perception of facial emotions and quality of life (QoL) in young people with HFA. Eligible patients were children and adolescents with HFA. Participants were randomized to the SST or LA group. The primary outcome was defined as an improvement of 2 points in error rates for facial emotion labeling (DANVA2) from baseline. After the 6-month training period, the SST Group made fewer errors in labeling anger on adult faces, whereas error rates in the LA Group remained stable. Progress in the ability to recognize anger in the SST Group was due to better recognition of low intensity stimuli on adult faces. QoL increased in the SST Group in the dimension of school environment, as a marker of the transfer of skills acquired in the treatment setting to their use in the community. The SST-GP had higher efficacy than the LA-GP. Data justify replication using larger samples.

Recognition of facial expressions in children, young adults and elderly people

The present study evaluated the recognition of facial expressions in different ages, using groups composed of: 1) 21 children with a mean age of 7.7 years; 2) 19 young adults with a mean age of 20.1 years; and 3) 9 elderly people with a mean age of 74.7 years. In the tests, participants were asked to identify facial expressions of happiness, sadness, fear and anger of different emotional intensities. The results indicated that the young adults performed better in recognizing facial expressions when compared to the children and elderly people. The children presented a performance similar to the elderly people, supporting the hypothesis that the ability to recognize facial expressions improves in adulthood and diminishes in old age.

Universality Revisited

Evidence does not support the claim that observers universally recognize basic emotions from signals on the face. The percentage of observers who matched the face with the predicted emotion (matching score) is not universal, but varies with culture and language. Matching scores are also inflated by the commonly used methods: within-subject design; posed, exaggerated facial expressions (devoid of context); multiple examples of each type of expression; and a response format that funnels a variety of interpretations into one word specified by the experimenter. Without these methodological aids, matching scores are modest and subject to various explanations.

Ongoing development of social cognition in adolescence

Age differences in social cognition between adolescents and young adults were investigated. Two large groups of adolescents and young adults were given tasks of theory of mind and emotion recognition. In addition, to control for possibly related basic cognitive development, working memory, speed of processing, and verbal ability were assessed. A strong age effect was revealed across both measures of social cognition. Adolescents performed with a lower accuracy than adults. Further analyses indicated that those age differences remained significant even after controlling for basic cognitive abilities. Exploratory analyses indicated no influence of pubertal phase on social cognition. Results suggest ongoing development of social cognition across adolescence, independent of individual differences in more basic cognitive abilities.

Sad or fearful? The influence of body posture on adults' and children's perception of facial displays of emotion

The current research investigated the influence of body posture on adults' and children's perception of facial displays of emotion. In each of two experiments, participants categorized facial expressions that were presented on a body posture that was congruent (e.g., a sad face on a body posing sadness) or incongruent (e.g., a sad face on a body posing fear). Adults and 8-year-olds made more errors and had longer reaction times on incongruent trials than on congruent trials when judging sad versus fearful facial expressions, an effect that was larger in 8-year-olds. The congruency effect was reduced when faces and bodies were misaligned, providing some evidence for holistic processing. Neither adults nor 8-year-olds were affected by congruency when judging sad versus happy expressions. Evidence that congruency effects vary with age and with similarity of emotional expressions is consistent with dimensional theories and "emotional seed" models of emotion perception.

A comparison of spatial frequency tuning for the recognition of facial identity and facial expressions in adults and children

We measured contrast thresholds for the identification of faces and facial expressions as a function of the center spatial frequency of narrow-band additive noise. In adults, masking of mid spatial frequencies (11-16c/fw) caused the largest elevation in contrast threshold (Experiment 1). Ideal observer analysis revealed that adults were equally sensitive to available information at low and mid spatial frequencies, both of which they used more efficiently than high spatial frequencies. The drop-off of sensitivity at high spatial frequencies began at a lower spatial frequency for recognizing facial identity than for recognizing facial expression. As a result, the critical band was higher for expression than for identity. The critical band for both identity and expression shifted to slightly lower values as distance increased (Experiment 2), a pattern indicating only partial scale invariance. Children aged 10 and 14 years showed similar tuning but needed more contrast (Experiment 3). The patterns suggest that adults use finer details for recognizing facial expressions than for identifying faces, a tuning that appears as early as age 10.