The Right Agent for the Job?

Embodied Conversational Agents Providing Motivational Interviewing to Improve Health-Related Behaviors: Scoping Review

BACKGROUND

Embodied conversational agents (ECAs) are advanced human-like interfaces that engage users in natural face-to-face conversations and interactions. These traits position ECAs as innovative tools for delivering interventions for promoting health-related behavior adoption. This includes motivational interviewing (MI), a therapeutic approach that combines brief interventions with motivational techniques to encourage the adoption of healthier behaviors.

OBJECTIVE

This study aims to identify the health issues addressed by ECAs delivering MI interventions, explore the key characteristics of these ECAs (eg, appearance, dialogue mechanism, emotional model), analyze the implementation of MI principles and techniques within ECAs, and examine the evaluation methods and primary outcomes of studies that use ECAs providing MI interventions.

METHODS

We conducted a scoping review following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) methodology. Our systematic search covered the PubMed, Scopus, IEEE Xplore, ACM Digital, and PsycINFO databases for papers published between January 2008 and December 2022. We included papers describing ECAs developed for delivering MI interventions targeting health-related behaviors and excluded articles that did not describe ECAs with human appearances and without the necessary evaluation or MI explanation. In a multistage process, 3 independent reviewers performed screening and data extraction, and the collected data were synthesized using a narrative approach.

RESULTS

The initial search identified 404 articles, of which 3.5% (n=14) were included in the review. ECAs primarily focused on reducing alcohol use (n=5, 36%), took on female representations (n=9, 64%), and gave limited consideration to user ethnicity (n=9, 64%). Most of them used rules-driven dialogue mechanisms (n=13, 93%), include emotional behavior to convey empathy (n=8, 57%) but without an automatic recognition of user emotions (n=12, 86%). Regarding MI implementation, of 14 studies, 3 (21%) covered all MI principles, 4 (29%) included all processes, and none covered all techniques. Most studies (8/14, 57%) conducted acceptability, usability, and user experience assessments, whereas a smaller proportion (4/14, 29%) used randomized controlled trials to evaluate behavior changes. Overall, the studies reported positive results regarding acceptability, usability, and user experience and showed promising outcomes in changes in attitudes, beliefs, motivation, and behavior.

CONCLUSIONS

This study revealed significant advancements in the use of ECAs for delivering MI interventions aimed at promoting healthier behaviors over the past 15 years. However, this review emphasizes the need for a more in-depth exploration of ECA characteristics. In addition, there is a need for the enhanced integration of MI principles, processes, and techniques into ECAs. Although acceptability and usability have received considerable attention, there is a compelling argument for placing a stronger emphasis on assessing changes in attitudes, beliefs, motivation, and behavior. Consequently, inclusion of more randomized controlled trials is essential for comprehensive intervention evaluations.

Do you like me? Behavioral and physical features for socially and emotionally engaging interactive systems

With the aim to give an overview of the most recent discoveries in the field of socially engaging interactive systems, the present paper discusses features affecting users' acceptance of virtual agents, robots, and chatbots. In addition, questionnaires exploited in several investigations to assess the acceptance of virtual agents, robots, and chatbots (voice only) are discussed and reported in the Supplementary material to make them available to the scientific community. These questionnaires were developed by the authors as a scientific contribution to the H2020 project EMPATHIC (http://www.empathic-project.eu/), Menhir (https://menhir-project.eu/), and the Italian-funded projects SIROBOTICS (https://www.exprivia.it/it-tile-6009-si-robotics/) and ANDROIDS (https://www.psicologia.unicampania.it/android-project) to guide the design and implementation of the promised assistive interactive dialog systems. They aimed to quantitatively evaluate Virtual Agents Acceptance (VAAQ), Robot Acceptance (RAQ), and Synthetic Virtual Agent Voice Acceptance (VAVAQ).

Acceptability, Adaptability, and Feasibility of a Novel Computer-Based Virtual Counselor-Delivered Alcohol Intervention: Focus Group and In-depth Interview Study Among Adults With HIV or Tuberculosis in Indian Clinical Settings

BACKGROUND

Unhealthy alcohol use is associated with increased morbidity and mortality among persons with HIV and tuberculosis (TB). Computer-based interventions (CBIs) can reduce unhealthy alcohol use, are scalable, and may improve outcomes among patients with HIV or TB.

OBJECTIVE

We assessed the acceptability, adaptability, and feasibility of a novel CBI for alcohol reduction in HIV and TB clinical settings in Pune, India.

METHODS

We conducted 10 in-depth interviews with persons with alcohol use disorder (AUD): TB (6/10), HIV (2/10), or HIV-TB co-infected (1/10) selected using convenience sampling method, no HIV or TB disease (1/10), 1 focus group with members of Alcoholics Anonymous (AA; n=12), and 2 focus groups with health care providers (HCPs) from a tertiary care hospital (n=22). All participants reviewed and provided feedback on a CBI for AUD delivered by a 3D virtual counselor. Qualitative data were analyzed using structured framework analysis.

RESULTS

The majority (9/10) of in-depth interview respondents were male, with median age 42 (IQR 38-45) years. AA focus group participants were all male (12/12), and HCP focus group participants were predominantly female (n=15). Feedback was organized into 3 domains: (1) virtual counselor acceptability, (2) intervention adaptability, and (3) feasibility of the CBI intervention in clinic settings. Overall, in-depth interview participants found the virtual counselor to be acceptable and felt comfortable honestly answering alcohol-related questions. All focus group participants preferred a human virtual counselor to an animal virtual counselor so as to potentially increase CBI engagement. Additionally, interaction with a live human counselor would further enhance the program's effectiveness by providing more flexible interaction. HCP focus group participants noted the importance of adding information on the effects of alcohol on HIV and TB outcomes because patients were not viewed as appreciating these linkages. For local adaptation, more information on types of alcoholic drinks, additional drinking triggers, motivators, and activities to substitute for drinking alcohol were suggested by all focus group participants. Intervention duration (about 20 minutes) and pace were deemed appropriate. HCPs reported that the CBI provides systematic, standardized counseling. All focus group and in-depth interview participants reported that the CBI could be implemented in Indian clinical settings with assistance from HIV or TB program staff.

CONCLUSIONS

With cultural tailoring to patients with HIV and TB in Indian clinical care settings, a virtual counselor-delivered alcohol intervention is acceptable and appears feasible to implement, particularly if coupled with person-delivered counseling.

The Terms of "You(s)": How the Term of Address Used by Conversational Agents Influences User Evaluations in French and German Linguaculture

Background: Conversational agents (CAs) are a novel approach to delivering digital health interventions. In human interactions, terms of address often change depending on the context or relationship between interlocutors. In many languages, this encompasses T/V distinction—formal and informal forms of the second-person pronoun “You”—that conveys different levels of familiarity. Yet, few research articles have examined whether CAs' use of T/V distinction across language contexts affects users' evaluations of digital health applications.

Methods: In an online experiment (N = 284), we manipulated a public health CA prototype to use either informal or formal T/V distinction forms in French (“tu” vs. “vous”) and German (“du” vs. “Sie”) language settings. A MANCOVA and post-hoc tests were performed to examine the effects of the independent variables (i.e., T/V distinction and Language) and the moderating role of users' demographic profile (i.e., Age and Gender) on eleven user evaluation variables. These were related to four themes: (i) Sociability, (ii) CA-User Collaboration, (iii) Service Evaluation, and (iv) Behavioral Intentions.

Results: Results showed a four-way interaction between T/V Distinction, Language, Age, and Gender, influencing user evaluations across all outcome themes. For French speakers, when the informal “T form” (“Tu”) was used, higher user evaluation scores were generated for younger women and older men (e.g., the CA felt more humanlike or individuals were more likely to recommend the CA), whereas when the formal “V form” (“Vous”) was used, higher user evaluation scores were generated for younger men and older women. For German speakers, when the informal T form (“Du”) was used, younger users' evaluations were comparable regardless of Gender, however, as individuals' Age increased, the use of “Du” resulted in lower user evaluation scores, with this effect more pronounced in men. When using the formal V form (“Sie”), user evaluation scores were relatively stable, regardless of Gender, and only increasing slightly with Age.

Conclusions: Results highlight how user CA evaluations vary based on the T/V distinction used and language setting, however, that even within a culturally homogenous language group, evaluations vary based on user demographics, thus highlighting the importance of personalizing CA language.

Designing Empathic Virtual Agents: Manipulating Animation, Voice, Rendering, and Empathy to Create Persuasive Agents

Designers of virtual agents have a combinatorically large space of choices for the look and behavior of their characters. We conducted two between-subjects studies to explore the systematic manipulation of animation quality, speech quality, rendering style, and simulated empathy, and its impact on perceptions of virtual agents in terms of naturalness, engagement, trust, credibility, and persuasion within a health counseling domain. In the first study, animation was varied between manually created, procedural, or no animations; voice quality was varied between recorded audio and synthetic speech; and rendering style was varied between realistic and toon-shaded. In the second study, simulated empathy of the agent was varied between no empathy, verbal-only empathic responses, and full empathy involving verbal, facial, and immediacy feedback. Results show that natural animations and recorded voice are more appropriate for the agent's general acceptance, trust, credibility, and appropriateness for the task. However, for a brief health counseling task, animation might actually be distracting from the persuasive message, with the highest levels of persuasion found when the amount of agent animation is minimized. Further, consistent and high levels of empathy improve agent perception but may interfere with forming a trusting bond with the agent.

Acceptability of Virtual Characters as a Social Skills Trainer (Preprint)

Background

Social skills training by human trainers is a well-established method to provide appropriate social interaction skills and strengthen social self-efficacy. In our previous work, we attempted to automate social skills training by developing a virtual agent that taught social skills through interaction. Previous research has not investigated the visual design of virtual agents for social skills training. Thus, we investigated the effect of virtual agent visual design on automated social skills training.

Objective

The 3 main purposes of this research were to investigate the effect of virtual agent appearance on automated social skills training, the relationship between acceptability and other measures (eg, likeability, realism, and familiarity), and the relationship between likeability and individual user characteristics (eg, gender, age, and autistic traits).

Methods

We prepared images and videos of a virtual agent, and 1218 crowdsourced workers rated the virtual agents through a questionnaire. In designing personalized virtual agents, we investigated the acceptability, likeability, and other impressions of the virtual agents and their relationship to individual characteristics.

Results

We found that there were differences between the virtual agents in all measures (P<.001). A female anime-type virtual agent was rated as the most likeable. We also confirmed that participants’ gender, age, and autistic traits were related to their ratings.

Conclusions

We confirmed the effect of virtual agent design on automated social skills training. Our findings are important in designing the appearance of an agent for use in personalized automated social skills training.

Virtual Agent Design for Social Skills Training Considering Autistic Traits

Social skills training by human coaches is a well-established method to obtain appropriate social interaction skills and strengthen social self-efficacy. Our previous works automated social skills training by developing a virtual agent that teaches social skills through interaction. This study attempts to investigate the effect of virtual agent design on automated social skills training. We prepared images and videos of a virtual agent, and a total of 912 crowdsourced workers rated the virtual agents by answering questions. We investigated the acceptability, likeability, and other impressions of the virtual agents and their relationship to the individuals' characteristics to design personalized virtual agents. As a result, a female anime-type virtual agent was rated as the most likable. We also confirmed that participants' gender, age, and autistic traits are related to the ratings. We believe our findings are important in designing a personalized virtual trainer.Clinical relevance- This study examines the effect of virtual agent design on social skills training. Our findings are important in designing a personalized virtual trainer.

Computer-Controlled Virtual Humans in Patient-Facing Systems: Systematic Review and Meta-Analysis

BACKGROUND

Virtual humans (VH) are computer-generated characters that appear humanlike and simulate face-to-face conversations using verbal and nonverbal cues. Unlike formless conversational agents, like smart speakers or chatbots, VH bring together the capabilities of both a conversational agent and an interactive avatar (computer-represented digital characters). Although their use in patient-facing systems has garnered substantial interest, it is unknown to what extent VH are effective in health applications.

OBJECTIVE

The purpose of this review was to examine the effectiveness of VH in patient-facing systems. The design and implementation characteristics of these systems were also examined.

METHODS

Electronic bibliographic databases were searched for peer-reviewed articles with relevant key terms. Studies were included in the systematic review if they designed or evaluated VH in patient-facing systems. Of the included studies, studies that used a randomized controlled trial to evaluate VH were included in the meta-analysis; they were then summarized using the PICOTS framework (population, intervention, comparison group, outcomes, time frame, setting). Summary effect sizes, using random-effects models, were calculated, and the risk of bias was assessed.

RESULTS

Among the 8,125 unique records identified, 53 articles describing 33 unique systems, were qualitatively, systematically reviewed. Two distinct design categories emerged - simple VH and VH augmented with health sensors and trackers. Of the 53 articles, 16 (26 studies) with 44 primary and 22 secondary outcomes were included in the meta-analysis. Meta-analysis of the 44 primary outcome measures revealed a significant difference between intervention and control conditions, favoring the VH intervention (SMD = .166, 95% CI .039-.292, P=.012), but with evidence of some heterogeneity, I2=49.3%. There were more cross-sectional (k=15) than longitudinal studies (k=11). The intervention was delivered using a personal computer in most studies (k=18), followed by a tablet (k=4), mobile kiosk (k=2), head-mounted display (k=1), and a desktop computer in a community center (k=1).

CONCLUSIONS

We offer evidence for the efficacy of VH in patient-facing systems. Considering that studies included different population and outcome types, more focused analysis is needed in the future. Future studies also need to identify what features of virtual human interventions contribute toward their effectiveness.

Perceived Usefulness, Satisfaction, Ease of Use and Potential of a Virtual Companion to Support the Care Provision for Older Adults

This article reports a study aiming to determine the perceptions of older adults needing formal care about the usefulness, satisfaction, and ease of use of CaMeLi, a virtual companion based on an embodied conversational agent, and the perceptions of formal caregivers about the potential of virtual companions to support care provision. An observational study involving older adults needing formal care was conducted to assess CaMeLi using a multi-method approach (i.e., an auto-reported questionnaire—the Usefulness, Satisfaction, and Ease of use questionnaire; a scale for the usability assessment based on the opinion of observers—the International Classification of Functioning Disability and Health-based Usability Scale; and critical incident registration). Moreover, a focus group was conducted to collect data regarding the perceived utility of virtual companions to support care provision. The observational study was conducted with 46 participants with an average age of 63.6 years, and the results were associated with a high level of usefulness, satisfaction, and ease of use of CaMeLi. Furthermore, the focus group composed of four care providers considered virtual companions a promising solution to support care provision and to prevent loneliness and social isolation. The results of both the observational study and the focus group revealed good perceptions regarding the role of virtual companions to support the care provision for older adults.

Acceptability of an Embodied Conversational Agent for Type 2 Diabetes Self-Management Education and Support via a Smartphone App: Mixed Methods Study

BACKGROUND

Embodied conversational agents (ECAs) are increasingly used in health care apps; however, their acceptability in type 2 diabetes (T2D) self-management apps has not yet been investigated.

OBJECTIVE

This study aimed to evaluate the acceptability of the ECA (Laura) used to deliver diabetes self-management education and support in the My Diabetes Coach (MDC) app.

METHODS

A sequential mixed methods design was applied. Adults with T2D allocated to the intervention arm of the MDC trial used the MDC app over a period of 12 months. At 6 months, they completed questions assessing their interaction with, and attitudes toward, the ECA. In-depth qualitative interviews were conducted with a subsample of the participants from the intervention arm to explore their experiences of using the ECA. The interview questions included the participants' perceptions of Laura, including their initial impression of her (and how this changed over time), her personality, and human character. The quantitative and qualitative data were interpreted using integrated synthesis.

RESULTS

Of the 93 intervention participants, 44 (47%) were women; the mean (SD) age of the participants was 55 (SD 10) years and the baseline glycated hemoglobin A1c level was 7.3% (SD 1.5%). Overall, 66 of the 93 participants (71%) provided survey responses. Of these, most described Laura as being helpful (57/66, 86%), friendly (57/66, 86%), competent (56/66, 85%), trustworthy (48/66, 73%), and likable (40/66, 61%). Some described Laura as not real (18/66, 27%), boring (26/66, 39%), and annoying (20/66, 30%). Participants reported that interacting with Laura made them feel more motivated (29/66, 44%), comfortable (24/66, 36%), confident (14/66, 21%), happy (11/66, 17%), and hopeful (8/66, 12%). Furthermore, 20% (13/66) of the participants were frustrated by their interaction with Laura, and 17% (11/66) of the participants reported that interacting with Laura made them feel guilty. A total of 4 themes emerged from the qualitative data (N=19): (1) perceived role: a friendly coach rather than a health professional; (2) perceived support: emotional and motivational support; (3) embodiment preference acceptability of a human-like character; and (4) room for improvement: need for greater congruence between Laura's words and actions.

CONCLUSIONS

These findings suggest that an ECA is an acceptable means to deliver T2D self-management education and support. A human-like character providing ongoing, friendly, nonjudgmental, emotional, and motivational support is well received. Nevertheless, the ECA can be improved by increasing congruence between its verbal and nonverbal communication and accommodating user preferences.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry CTRN12614001229662; https://tinyurl.com/yxshn6pd.

Participatory medicine: model based tools for engaging and empowering the individual

The long-term goal of the Virtual Physiological Human and Digital Patient projects is to run 'simulations' of health and disease processes on the virtual or 'digital' patient, and use the results to make predictions about real health and determine the best treatment specifically for an individual. This is termed 'personalized medicine', and is intended to be the future of healthcare. How will people interact and engage with their virtual selves, and how can virtual models be used to motivate people to actively participate in their own healthcare? We discuss these questions, and describe our current efforts to integrate and realistically embody psychobiological models of face-to-face interaction to enliven and increase engagement of virtual humans in healthcare. Overall, this paper highlights the need for attention to the design of human-machine interfaces to address patient engagement in healthcare.

Effects of Virtual Human Appearance Fidelity on Emotion Contagion in Affective Inter-Personal Simulations

Realistic versus stylized depictions of virtual humans in simulated inter-personal situations and their ability to elicit emotional responses in users has been an open question for artists and researchers alike. We empirically evaluated the effects of near visually realistic vs. non-realistic stylized appearance of virtual humans on the emotional response of participants in a medical virtual reality system that was designed to educate users in recognizing the signs and symptoms of patient deterioration. In a between-subjects experiment protocol, participants interacted with one of three different appearances of a virtual patient, namely visually realistic, cartoon-shaded and charcoal-sketch like conditions in a mixed reality simulation. Emotional impact were measured via a combination of quantitative objective measures were gathered using skin Electrodermal Activity (EDA) sensors, and quantitative subjective measures such as the Differential Emotion Survey (DES IV), Positive and Negative Affect Schedule (PANAS), and Social Presence questionnaire. The emotional states of the participants were analyzed across four distinct time steps during which the medical condition of the virtual patient deteriorated (an emotionally stressful interaction), and were contrasted to a baseline affective state. Objective EDA results showed that in all three conditions, male participants exhibited greater levels of arousal as compared to female participants. We found that negative affect levels were significantly lower in the visually realistic condition, as compared to the stylized appearance conditions. Furthermore, in emotional dimensions of interest-excitement, surprise, anger, fear and guilt participants in all conditions responded similarly. However, in social emotional constructs of shyness, presence, perceived personality, and enjoyment-joy, we found that participants responded differently in the visually realistic condition as compared to the cartoon and sketch conditions. Our study suggests that virtual human appearance can affect not only critical emotional reactions in affective inter-personal training scenarios. but also users' perceptions of personality and social characteristic of the virtual interlocutors.