Design and Evaluation of Personalized Motivational Messages by a Virtual Agent that Assists in Post-Traumatic Stress Disorder Therapy

Embodied Conversational Agents Providing Motivational Interviewing to Improve Health-Related Behaviors: Scoping Review

BACKGROUND

Embodied conversational agents (ECAs) are advanced human-like interfaces that engage users in natural face-to-face conversations and interactions. These traits position ECAs as innovative tools for delivering interventions for promoting health-related behavior adoption. This includes motivational interviewing (MI), a therapeutic approach that combines brief interventions with motivational techniques to encourage the adoption of healthier behaviors.

OBJECTIVE

This study aims to identify the health issues addressed by ECAs delivering MI interventions, explore the key characteristics of these ECAs (eg, appearance, dialogue mechanism, emotional model), analyze the implementation of MI principles and techniques within ECAs, and examine the evaluation methods and primary outcomes of studies that use ECAs providing MI interventions.

METHODS

We conducted a scoping review following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) methodology. Our systematic search covered the PubMed, Scopus, IEEE Xplore, ACM Digital, and PsycINFO databases for papers published between January 2008 and December 2022. We included papers describing ECAs developed for delivering MI interventions targeting health-related behaviors and excluded articles that did not describe ECAs with human appearances and without the necessary evaluation or MI explanation. In a multistage process, 3 independent reviewers performed screening and data extraction, and the collected data were synthesized using a narrative approach.

RESULTS

The initial search identified 404 articles, of which 3.5% (n=14) were included in the review. ECAs primarily focused on reducing alcohol use (n=5, 36%), took on female representations (n=9, 64%), and gave limited consideration to user ethnicity (n=9, 64%). Most of them used rules-driven dialogue mechanisms (n=13, 93%), include emotional behavior to convey empathy (n=8, 57%) but without an automatic recognition of user emotions (n=12, 86%). Regarding MI implementation, of 14 studies, 3 (21%) covered all MI principles, 4 (29%) included all processes, and none covered all techniques. Most studies (8/14, 57%) conducted acceptability, usability, and user experience assessments, whereas a smaller proportion (4/14, 29%) used randomized controlled trials to evaluate behavior changes. Overall, the studies reported positive results regarding acceptability, usability, and user experience and showed promising outcomes in changes in attitudes, beliefs, motivation, and behavior.

CONCLUSIONS

This study revealed significant advancements in the use of ECAs for delivering MI interventions aimed at promoting healthier behaviors over the past 15 years. However, this review emphasizes the need for a more in-depth exploration of ECA characteristics. In addition, there is a need for the enhanced integration of MI principles, processes, and techniques into ECAs. Although acceptability and usability have received considerable attention, there is a compelling argument for placing a stronger emphasis on assessing changes in attitudes, beliefs, motivation, and behavior. Consequently, inclusion of more randomized controlled trials is essential for comprehensive intervention evaluations.

Feasibility of generating structured motivational messages for tailored physical activity coaching

Tailored motivational messages are helpful to motivate people in eHealth applications for increasing physical activity, but it is not sufficiently clear how such messages can be effectively generated in advance. We, therefore, put forward a theory-driven approach to generating tailored motivational messages for eHealth applications for behavior change, and we examine its feasibility by assessing how motivating the resulting messages are perceived. For this, we designed motivational messages with a specific structure that was based on an adaptation of an existing ontology for tailoring motivational messages in the context of physical activity. To obtain tailored messages, experts in health psychology and coaching successfully wrote messages with this structure for personas in scenarios that differed with regard to the persona's mood, self-efficacy, and progress. Based on an experiment in which 60 participants each rated the perceived motivational impact of six generic and six tailored messages based on scenarios, we found credible support for our hypothesis that messages tailored to mood, self-efficacy, and progress are perceived as more motivating. A thematic analysis of people's free-text responses about what they found motivating and demotivating about motivational messages further supports the use of tailored messages, as well as messages that are encouraging and empathetic, give feedback about people's progress, and mention the benefits of physical activity. To aid future work on motivational messages, we make our motivational messages and corresponding scenarios publicly available.

Reducing social diabetes distress with a conversational agent support system: a three-week technology feasibility evaluation

Background

People with diabetes mellitus not only have to deal with physical health problems, but also with the psycho-social challenges their chronic disease brings. Currently, technological tools that support the psycho-social context of a patient have received little attention.

Objective

The objective of this work is to determine the feasibility and preliminary efficacy of an automated conversational agent to deliver, to people with diabetes, personalised psycho-education on dealing with (psycho-)social distress related to their chronic illness.

Methods

In a double-blinded between-subject study, 156 crowd-workers with diabetes received a social help program intervention in three sessions over three weeks. They were randomly assigned to receive support from either an interactive conversational support agent (n=79) or a self-help text from the book "Diabetes burnout" as a control condition (n=77). Participants completed the Diabetes Distress Scale (DDS) before and after the intervention, and after the intervention, the Client Satisfaction Questionnaire (CSQ-8), Feeling of Being Heard (FBH), and System Usability Scale (SUS).

Results

Results indicate that people using the conversational agent have a larger reduction in diabetes distress (M=-0.305, SD=0.865) than the control group (M=0.002, SD=0.743) and this difference is statistically significant (t(154)=2.377, p=0.019). A hypothesised mediation effect of "attitude to the social help program" was not observed.

Conclusions

An automated conversational agent can deliver personalised psycho-education on dealing with (psycho-)social distress to people with diabetes and reduce diabetes distress more than a self-help book.

Ethics Study Registration and Open Science

This study has been preregistered with the Open Science Foundation (osf.io/yb6vg) and has been accepted by the Human Research Ethics Committee - Delft University of Technology under application number 1130. The data and analysis script are available: https://surfdrive.surf.nl/files/index.php/s/4xSEHCrAu0HsJ4P.

A Mobile Application-Based Relational Agent as a Health Professional for COVID-19 Patients: Design, Approach, and Implications

Relational Agents' (RAs) ability to maintain socio-emotional relationships with users can be an asset to COVID-19 patients. The goal of this research was to identify principles for designing an RA that can act as a health professional for a COVID-19 patient. We first identified tasks that such an RA can provide by interviewing 33 individuals, who had recovered from COVID-19. The transcribed interviews were analyzed using qualitative thematic analysis. Based on the findings, four sets of hypothetical conversations were handcrafted to illustrate how the proposed RA will execute the identified tasks. These conversations were then evaluated by 43 healthcare professionals in a qualitative study. Thematic analysis was again used to identify characteristics that would be suitable for the proposed RA. The results suggest that the RA must: model clinical protocols; incorporate evidence-based interventions; inform, educate, and remind patients; build trusting relationships, and support their socio-emotional needs. The findings have implications for designing RAs for other healthcare contexts beyond the pandemic.

A Relational Agent for the COVID-19 Patients: Design, Approach, and Implications

BACKGROUND

Relational agents (RAs) have shown effectiveness in various health interventions with and without doctors and hospital facilities. We suggest that in situations such as a pandemic like the COVID-19 when healthcare professionals (HCPs) and facilities are unable to cope with increased demands, RAs can play a major role in ameliorating the situation.

OBJECTIVE

The goal of this research was to seek design validation on a prototypical RA to address healthcare needs of the COVID-19 patients.

METHODS

Therefore, RAs can deliver health interventions during COVID-19 pandemic, but they have not been well-explored in this domain. To address this gap, a prototypical RA is iteratively designed and developed in collaboration with infected patients (n=21) and two groups of HCPs (n=19 and n=16 respectively) to aid COVID-19 patients at various stages by performing four main tasks: testing guidance, support during self-isolation, handling emergency situations, and promoting post-recovery mental well-being.

RESULTS

A survey with 98 individuals was used to evaluate the usability of the prototype by system usability scale (SUS) and it received an average score of 58.82. Moreover, participants indicated perceived usefulness and acceptability of the system on Likert Scales where 89.65% perceived it to be helpful, 68.97% accepted it as a viable alternative to HCPs.

CONCLUSIONS

The prototypical RA received favorable feedback from the participants and they were inclined to accept it as an alternative to HCPs in non-life-threatening scenarios despite the usability rating falling below the acceptable threshold. Based on participants' feedback, we recommend further development of the RA with improved automation and emotional support, ability to provide information, tracking, and specific recommendations.

Using a conversational agent for thought recording as a cognitive therapy task: Feasibility, content, and feedback

E-mental health for depression is increasingly used in clinical practice, but patient adherence suffers as therapist involvement decreases. One reason may be the low responsiveness of existing programs: especially autonomous systems are lacking in their input interpretation and feedback-giving capabilities. Here, we explore (a) to what extent a more socially intelligent and, therefore, technologically advanced solution, namely a conversational agent, is a feasible means of collecting thought record data in dialog, (b) what people write about in their thought records, (c) whether providing content-based feedback increases motivation for thought recording, a core technique of cognitive therapy that helps patients gain an understanding of how their thoughts cause their feelings. Using the crowd-sourcing platform Prolific, 308 participants with subclinical depression symptoms were recruited and split into three conditions of varying feedback richness using the minimization method of randomization. They completed two thought recording sessions with the conversational agent: one practice session with scenarios and one open session using situations from their own lives. All participants were able to complete thought records with the agent such that the thoughts could be interpreted by the machine learning algorithm, rendering the completion of thought records with the agent feasible. Participants chose interpersonal situations nearly three times as often as achievement-related situations in the open chat session. The three most common underlying schemas were the Attachment, Competence, and Global Self-evaluation schemas. No support was found for a motivational effect of providing richer feedback. In addition to our findings, we publish the dataset of thought records for interested researchers and developers.

Adaptive Elements in Internet-Delivered Psychological Treatment Systems: Systematic Review

BACKGROUND

Internet-delivered psychological treatments (IDPTs) are built on evidence-based psychological treatment models, such as cognitive behavioral therapy, and are adjusted for internet use. The use of internet technologies has the potential to increase access to evidence-based mental health services for a larger proportion of the population with the use of fewer resources. However, despite extensive evidence that internet interventions can be effective in the treatment of mental health disorders, user adherence to such internet intervention is suboptimal.

OBJECTIVE

This review aimed to (1) inspect and identify the adaptive elements of IDPT for mental health disorders, (2) examine how system adaptation influences the efficacy of IDPT on mental health treatments, (3) identify the information architecture, adaptive dimensions, and strategies for implementing these interventions for mental illness, and (4) use the findings to create a conceptual framework that provides better user adherence and adaptiveness in IDPT for mental health issues.

METHODS

The review followed the guidelines from Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The research databases Medline (PubMed), ACM Digital Library, PsycINFO, CINAHL, and Cochrane were searched for studies dating from January 2000 to January 2020. Based on predetermined selection criteria, data from eligible studies were analyzed.

RESULTS

A total of 3341 studies were initially identified based on the inclusion criteria. Following a review of the title, abstract, and full text, 31 studies that fulfilled the inclusion criteria were selected, most of which described attempts to tailor interventions for mental health disorders. The most common adaptive elements were feedback messages to patients from therapists and intervention content. However, how these elements contribute to the efficacy of IDPT in mental health were not reported. The most common information architecture used by studies was tunnel-based, although a number of studies did not report the choice of information architecture used. Rule-based strategies were the most common adaptive strategies used by these studies. All of the studies were broadly grouped into two adaptive dimensions based on user preferences or using performance measures, such as psychometric tests.

CONCLUSIONS

Several studies suggest that adaptive IDPT has the potential to enhance intervention outcomes and increase user adherence. There is a lack of studies reporting design elements, adaptive elements, and adaptive strategies in IDPT systems. Hence, focused research on adaptive IDPT systems and clinical trials to assess their effectiveness are needed.