Time course of selective attention to face regions in social anxiety: eye-tracking and computational modelling

Emotional Attention: From Eye Tracking to Computational Modeling

Attending selectively to emotion-eliciting stimuli is intrinsic to human vision. In this research, we investigate how emotion-elicitation features of images relate to human selective attention. We create the EMOtional attention dataset (EMOd). It is a set of diverse emotion-eliciting images, each with (1) eye-tracking data from 16 subjects, (2) image context labels at both object- and scene-level. Based on analyses of human perceptions of EMOd, we report an emotion prioritization effect: emotion-eliciting content draws stronger and earlier human attention than neutral content, but this advantage diminishes dramatically after initial fixation. We find that human attention is more focused on awe eliciting and aesthetic vehicle and animal scenes in EMOd. Aiming to model the above human attention behavior computationally, we design a deep neural network (CASNet II), which includes a channel weighting subnetwork that prioritizes emotion-eliciting objects, and an Atrous Spatial Pyramid Pooling (ASPP) structure that learns the relative importance of image regions at multiple scales. Visualizations and quantitative analyses demonstrate the model's ability to simulate human attention behavior, especially on emotion-eliciting content.

Aberrant neural correlates of multisensory processing of audiovisual social cues related to social anxiety: An electrophysiological study

BACKGROUND

Social anxiety disorder (SAD) is characterized by abnormal fear to social cues. Although unisensory processing to social stimuli associated with social anxiety (SA) has been well described, how multisensory processing relates to SA is still open to clarification. Using electroencephalography (EEG) measurement, we investigated the neural correlates of multisensory processing and related temporal dynamics in social anxiety disorder (SAD).

METHODS

Twenty-five SAD participants and 23 healthy control (HC) participants were presented with angry and neutral faces, voices and their combinations with congruent emotions and they completed an emotional categorization task.

RESULTS

We found that face-voice combinations facilitated auditory processing in multiple stages indicated by the acceleration of auditory N1 latency, attenuation of auditory N1 and P250 amplitudes, and decrease of theta power. In addition, bimodal inputs elicited cross-modal integrative activity which is indicated by the enhancement of visual P1, N170, and P3/LPP amplitudes and superadditive response of P1 and P3/LPP. More importantly, excessively greater integrative activity (at P3/LPP amplitude) was found in SAD participants, and this abnormal integrative activity in both early and late temporal stages was related to the larger interpretation bias of miscategorizing neutral face-voice combinations as angry.

CONCLUSION

The study revealed that neural correlates of multisensory processing was aberrant in SAD and it was related to the interpretation bias to multimodal social cues in multiple processing stages. Our findings suggest that deficit in multisensory processing might be an important factor in the psychopathology of SA.

Computational Process of Sharing Emotion: An Authentic Information Perspective

Although results of many psychology studies have shown that sharing emotion achieves dyadic interaction, no report has explained a study of the transmission of authentic information from emotional expressions that can strengthen perceivers. For this study, we used computational modeling, which is a multinomial processing tree, for formal quantification of the process of sharing emotion that emphasizes the perception of authentic information for expressers’ feeling states from facial expressions. Results indicated that the ability to perceive authentic information of feeling states from a happy expression has a higher probability than the probability of judging authentic information from anger expressions. Next, happy facial expressions can activate both emotional elicitation and sharing emotion in perceivers, where emotional elicitation alone is working rather than sharing emotion for angry facial expressions. Third, parameters to detect anger experiences were found to be correlated positively with those of happiness. No robust correlation was found between the parameters extracted from this experiment task and questionnaire-measured emotional contagion, empathy, and social anxiety. Results of this study revealed the possibility that a new computational approach contributes to description of emotion sharing processes.

Attentional processes during emotional face perception in social anxiety disorder: A systematic review and meta-analysis of eye-tracking findings

Background In recent years, a growing body of eye-tracking research has investigated gaze behavior in individuals with social anxiety during the visual perception of emotional stimuli. The aim of this article was to review and synthesize studies examining attention orientation in patients with clinical social anxiety by means of eye-tracking methodology. Methods Through a systematic search, 30 articles were identified, including 11 studies in which single emotional faces were used as stimuli and seven eligible studies in which threatening faces were paired with neutral stimuli. Meta-analyses were conducted to compare prolonged eye-contact behavior and early attentional biases to threats in individuals with social anxiety disorder (SAD) and healthy controls. Results Moderate group differences were revealed for single face viewing studies, with SAD patients showing significantly reduced eye contact with negative (Hedges' g = -0.67) and positive emotional faces (g = -0.49) compared to that of healthy participants. Type of task and duration of stimulus presentation were (marginally) significant moderators of between-study variance in effect size. Small but significant group differences were found for early attentional biases toward angry faces versus neutral stimuli (g = 0.21) but not toward happy faces versus neutral stimuli (g = 0.05). Preliminary evidence for a hyperscanning strategy in SAD patients relative to healthy controls emerged (g = 0.42). Limitations The number of included studies with face pairings was low, and two studies were excluded due to unavailable data. Conclusions Our results suggest that eye contact avoidance with emotional faces is a prominent feature in SAD patients. Patients might benefit from guidance to learn to make adequate eye contact during therapeutic interventions, such as exposure therapy. SAD patients demonstrated slightly heightened attention allocation toward angry faces relative to that of healthy participants during early processing stages. Threat biases can be potential targets for attention modification training as an adjuvant to other treatments. Future research on early attentional processes may benefit from improved arrangements of paired stimuli to increase the psychometric properties of initial attention indices.

A systematic review of visual avoidance of faces in socially anxious individuals: Influence of severity, type of social situation, and development

Although visual avoidance of faces is a hallmark feature of social anxiety disorder (SAD) on clinical and theoretical grounds, empirical support is equivocal. This review aims to clarify under which conditions socially anxious individuals display visual avoidance of faces. Through a systematic search in Web of Science and PubMed up to March 2019 we identified 61 publications that met the inclusion criteria. We discuss the influence of three factors on the extent to which socially anxious individuals avoid looking at faces: (a) severity of social anxiety symptoms (diagnosed SAD versus High Social Anxiety levels in community samples [HSA] or related characteristics [Shyness, Fear of Negative Evaluation]), (b) three types of social situation (computer facial-viewing tasks, speaking tasks, social interactions), and (c) development (age-group). Adults with SAD exhibit visual avoidance across all three types of social situations, whereas adults with HSA exhibit visual avoidance in speaking and interaction tasks but not in facial-viewing tasks. The relatively few studies with children and adolescents suggest that visual avoidance emerges during adolescence. The findings are discussed in the context of cognitive-behavioral and skills-deficit models. Suggestions for future research include the need for developmental studies and more fine-grained analyses of specific areas of the face.