Quantitative Evaluation of Facial Expression in a Patient With Minimally Conscious State After Severe Traumatic Brain Injury

The other person's smiling amount affects one's smiling response during face-to-face conversations

Introduction

Smiling during conversation occurs interactively between people and is known to build good interpersonal relationships. However, whether and how much the amount that an individual smiles is influenced by the other person's smile has remained unclear. This study aimed to quantify the amount of two individuals' smiles during conversations and investigate the dependency of one's smile amount (i.e., intensity and frequency) on that of the other.

Method

Forty participants (20 females) engaged in three-minute face-to-face conversations as speakers with a listener (male or female), under three conditions, where the amount of smiling response by listeners was controlled as "less," "moderate," and "greater." The amount of the smiles was quantified based on their facial movements through automated facial expression analysis.

Results

The results showed that the amount of smiling by the speaker changed significantly depending on the listener's smile amount; when the listeners smiled to a greater extent, the speakers tended to smile more, especially when they were of the same gender (i.e., male-male and female-female pairs). Further analysis revealed that the smiling intensities of the two individuals changed in a temporally synchronized manner.

Discussion

These results provide quantitative evidence for the dependence of one's smile on the other's smile, and the differential effect between gender pairs.

Positive Emotional Responses to Socially Assistive Robots in People With Dementia: Pilot Study

Background

Interventions and care that can evoke positive emotions and reduce apathy or agitation are important for people with dementia. In recent years, socially assistive robots used for better dementia care have been found to be feasible. However, the immediate responses of people with dementia when they are given multiple sensory modalities from socially assistive robots have not yet been sufficiently elucidated.

Objective

This study aimed to quantitatively examine the immediate emotional responses of people with dementia to stimuli presented by socially assistive robots using facial expression analysis in order to determine whether they elicited positive emotions.

Methods

This pilot study adopted a single-arm interventional design. Socially assistive robots were presented to nursing home residents in a three-step procedure: (1) the robot was placed in front of participants (visual stimulus), (2) the robot was manipulated to produce sound (visual and auditory stimuli), and (3) participants held the robot in their hands (visual, auditory, and tactile stimuli). Expression intensity values for "happy," "sad," "angry," "surprised," "scared," and "disgusted" were calculated continuously using facial expression analysis with FaceReader. Additionally, self-reported feelings were assessed using a 5-point Likert scale. In addition to the comparison between the subjective and objective emotional assessments, expression intensity values were compared across the aforementioned 3 stimuli patterns within each session. Finally, the expression intensity value for "happy" was compared between the different types of robots.

Results

A total of 29 participants (mean age 88.7, SD 6.2 years; n=27 female; Japanese version of Mini-Mental State Examination mean score 18.2, SD 5.1) were recruited. The expression intensity value for "happy" was the largest in both the subjective and objective assessments and increased significantly when all sensory modalities (visual, auditory, and tactile) were presented (median expression intensity 0.21, IQR 0.09-0.35) compared to the other 2 patterns (visual alone: median expression intensity 0.10, IQR 0.03-0.22; P<.001; visual and auditory: median expression intensity 0.10, IQR 0.04-0.23; P<.001). The comparison of different types of robots revealed a significant increase when all stimuli were presented by doll-type and animal-type robots, but not humanoid-type robots.

Conclusions

By quantifying the emotional responses of people with dementia, this study highlighted that socially assistive robots may be more effective in eliciting positive emotions when multiple sensory stimuli, including tactile stimuli, are involved. More studies, including randomized controlled trials, are required to further explore the effectiveness of using socially assistive robots in dementia care.

Covert consciousness

Covert consciousness is a state of residual awareness following severe brain injury or neurological disorder that evades routine bedside behavioral detection. Patients with covert consciousness have preserved awareness but are incapable of self-expression through ordinary means of behavior or communication. Growing recognition of the limitations of bedside neurobehavioral examination in reliably detecting consciousness, along with advances in neurotechnologies capable of detecting brain states or subtle signs indicative of consciousness not discernible by routine examination, carry promise to transform approaches to classifying, diagnosing, prognosticating and treating disorders of consciousness. Here we describe and critically evaluate the evolving clinical category of covert consciousness, including approaches to its diagnosis through neuroimaging, electrophysiology, and novel behavioral tools, its prognostic relevance, and open questions pertaining to optimal clinical management of patients with covert consciousness recovering from severe brain injury.