Developing crossmodal expression recognition based on a deep neural model

A robot capable of understanding emotion expressions can increase its own capability of solving problems by using emotion expressions as part of its own decision-making, in a similar way to humans. Evidence shows that the perception of human interaction starts with an innate perception mechanism, where the interaction between different entities is perceived and categorized into two very clear directions: positive or negative. While the person is developing during childhood, the perception evolves and is shaped based on the observation of human interaction, creating the capability to learn different categories of expressions. In the context of human-robot interaction, we propose a model that simulates the innate perception of audio-visual emotion expressions with deep neural networks, that learns new expressions by categorizing them into emotional clusters with a self-organizing layer. The proposed model is evaluated with three different corpora: The Surrey Audio-Visual Expressed Emotion (SAVEE) database, the visual Bi-modal Face and Body benchmark (FABO) database, and the multimodal corpus of the Emotion Recognition in the Wild (EmotiW) challenge. We use these corpora to evaluate the performance of the model to recognize emotional expressions, and compare it to state-of-the-art research.

Sensitivity to Emotion Intensity and Recognition of Emotion Expression in Neurotypical Children

This study assessed two components of face emotion processing: emotion recognition and sensitivity to intensity of emotion expressions and their relation in children age 4 to 12 (N = 216). Results indicated a slower development in the accurate decoding of low intensity expressions compared to high intensity. Between age 4 and 12, children discriminated high intensity expressions better than low ones. The intensity of expression had a stronger impact on overall face expression recognition. High intensity happiness was better recognized than low intensity up to age 11, while children 4 to 12 had difficulties discriminating between high and low intensity sadness. Our results suggest that sensitivity to low intensity expressions acts as a complementary mediator between age and emotion expression recognition, while this was not the case for the recognition of high intensity expressions. These results could help in the development of specific interventions for populations presenting socio-cognitive and emotion difficulties.

Emotion knowledge in neurotypical children and in those with down syndrome

This research aimed to assess two components of emotion knowledge (EK): receptive EK with face emotion identification and matching tasks, and emotion situation knowledge with the emotion attribution task (EAT). Study 1 assessed the development of EK in 265 neurotypical (NT) children (4-11 years), divided into four age groups. Overall, results showed a significant improvement of EK with age in the NT population for the three tasks, especially between the ages of 4/5 and 6/7. Children were less successful at the EAT in comparison to the other two tasks, indicating that receptive EK develops earlier than emotion situation knowledge. The presence of visual context (EAT) does not help to improve our children's overall facial emotion recognition, especially for anger and sadness, while these emotions are well recognized in isolated facial expressions (emotion identification). Study 2 compared EK between 32 children with Down syndrome (CA: M = 13 years, SD = 2.13) and 32 NT children (CA: M = 5.3 years, SD = 1.36): matched on a vocabulary task. Children with DS had more difficulties in EK than NT children. They had lower performances on the identification and the EAT tasks, while exhibited similar performances to their NT controls on the emotion matching task. Moreover, good abilities to identify emotion expressions seem to be a prerequisite for successful face-context recognition in NT children, but not in children with DS. Difficulties encountered by children with DS could result from executive dysfunction when dealing with complex visual information in addition to emotion processing difficulties.