Multichannel Perception of Emotion in Speech, Voice, Facial Expression, and Gesture in Individuals With Autism: A Scoping Review

Auditory Challenges and Listening Effort in School-Age Children With Autism: Insights From Pupillary Dynamics During Speech-in-Noise Perception

PURPOSE

This study aimed to investigate challenges in speech-in-noise (SiN) processing faced by school-age children with autism spectrum conditions (ASCs) and their impact on listening effort.

METHOD

Participants, including 23 Mandarin-speaking children with ASCs and 19 age-matched neurotypical (NT) peers, underwent sentence recognition tests in both quiet and noisy conditions, with a speech-shaped steady-state noise masker presented at 0-dB signal-to-noise ratio in the noisy condition. Recognition accuracy rates and task-evoked pupil responses were compared to assess behavioral performance and listening effort during auditory tasks.

RESULTS

No main effect of group was found on accuracy rates. Instead, significant effects emerged for autistic trait scores, listening conditions, and their interaction, indicating that higher trait scores were associated with poorer performance in noise. Pupillometric data revealed significantly larger and earlier peak dilations, along with more varied pupillary dynamics in the ASC group relative to the NT group, especially under noisy conditions. Importantly, the ASC group's peak dilation in quiet mirrored that of the NT group in noise. However, the ASC group consistently exhibited reduced mean dilations than the NT group.

CONCLUSIONS

Pupillary responses suggest a different resource allocation pattern in ASCs: An initial sharper and larger dilation may signal an intense, narrowed resource allocation, likely linked to heightened arousal, engagement, and cognitive load, whereas a subsequent faster tail-off may indicate a greater decrease in resource availability and engagement, or a quicker release of arousal and cognitive load. The presence of noise further accentuates this pattern. This highlights the unique SiN processing challenges children with ASCs may face, underscoring the importance of a nuanced, individual-centric approach for interventions and support.

Incidence of Otolaryngological Manifestations in Individuals with Autism Spectrum Disorder: A Special Focus on Auditory Disorders

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by stereotyped and repetitive behavior patterns. In addition to neurological and behavioral problems, individuals with ASD commonly experience otolaryngological comorbidities. Individuals with ASD often have auditory disorders including hearing loss and auditory processing disorders such as central auditory processing disorder (CAPD), as well as both chronic and recurrent otitis media. These challenges negatively impact a person's ability to effectively communicate and may further impact their neurological functioning, particularly when not appropriately treated. Individuals diagnosed with ASD also have difficulty sleeping which contributes to increased irritability and may further aggravate the core behavioral symptoms of autism. The individuals with ASD also have a higher rate of sinusitis which contributes to the worsening of the autism behavior phenotype. The high prevalence of otolaryngological comorbidities in individuals with ASD warrants a better collaboration between their various healthcare providers and otolaryngologists with expertise in auditory, sleep, and sinus disorders in pursuit of improving the quality of life of affected individuals and their families/caregivers.

Mapping the Physical Language of Children Diagnosed with Autism: A Preliminary Study

Children diagnosed with autism spectrum disorder have a unique motor profile, characterized by, for example, unusual posture or compulsive use of the body. However, not much is known about specific characteristics of their physical language, such as their movement direction, their self-touch pattern, etc., and even less is known about these characteristics with regard to their typically developing siblings. In this first of its kind study, we attempted to map the physical language of children diagnosed with autism spectrum disorder and to compare it to their typically developing siblings. To this end, we recruited 12 pairs of siblings, comprising one sibling with a diagnosis of autism and one sibling who is typically developing. The siblings were asked to play for 10 min and were videotaped throughout the interaction. We evaluated the siblings' physical language using Laban's movement analysis. We found significant and substantial differences between the physical language of the children diagnosed with autism and their typically developing siblings. The results are discussed in terms of the implications of the differences in physical language between the two populations and how movement analysis could be important for interventions in order to improve the communication and social abilities of ASD children.

GCF2-Net: global-aware cross-modal feature fusion network for speech emotion recognition

Emotion recognition plays an essential role in interpersonal communication. However, existing recognition systems use only features of a single modality for emotion recognition, ignoring the interaction of information from the different modalities. Therefore, in our study, we propose a global-aware Cross-modal feature Fusion Network (GCF²-Net) for recognizing emotion. We construct a residual cross-modal fusion attention module (ResCMFA) to fuse information from multiple modalities and design a global-aware module to capture global details. More specifically, we first use transfer learning to extract wav2vec 2.0 features and text features fused by the ResCMFA module. Then, cross-modal fusion features are fed into the global-aware module to capture the most essential emotional information globally. Finally, the experiment results have shown that our proposed method has significant advantages than state-of-the-art methods on the IEMOCAP and MELD datasets, respectively.

Subjective Evaluation of Basic Emotions from Audio-Visual Data

Understanding of the perception of emotions or affective states in humans is important to develop emotion-aware systems that work in realistic scenarios. In this paper, the perception of emotions in naturalistic human interaction (audio-visual data) is studied using perceptual evaluation. For this purpose, a naturalistic audio-visual emotion database collected from TV broadcasts such as soap-operas and movies, called the IIIT-H Audio-Visual Emotion (IIIT-H AVE) database, is used. The database consists of audio-alone, video-alone, and audio-visual data in English. Using data of all three modes, perceptual tests are conducted for four basic emotions (angry, happy, neutral, and sad) based on category labeling and for two dimensions, namely arousal (active or passive) and valence (positive or negative), based on dimensional labeling. The results indicated that the participants' perception of emotions was remarkably different between the audio-alone, video-alone, and audio-video data. This finding emphasizes the importance of emotion-specific features compared to commonly used features in the development of emotion-aware systems.

Virtual Reality Technology as an Educational and Intervention Tool for Children with Autism Spectrum Disorder: Current Perspectives and Future Directions

The worldwide rising trend of autism spectrum disorder (ASD) calls for innovative and efficacious techniques for assessment and treatment. Virtual reality (VR) technology gains theoretical support from rehabilitation and pedagogical theories and offers a variety of capabilities in educational and interventional contexts with affordable products. VR is attracting increasing attention in the medical and healthcare industry, as it provides fully interactive three-dimensional simulations of real-world settings and social situations, which are particularly suitable for cognitive and performance training, including social and interaction skills. This review article offers a summary of current perspectives and evidence-based VR applications for children with ASD, with a primary focus on social communication, including social functioning, emotion recognition, and speech and language. Technology- and design-related limitations, as well as disputes over the application of VR to autism research and therapy, are discussed, and future directions of this emerging field are highlighted with regards to application expansion and improvement, technology enhancement, linguistic diversity, and the development of theoretical models and brain-based research.