Evaluating the Therapeutic Alliance With a Free-Text CBT Conversational Agent (Wysa): A Mixed-Methods Study

The ethical aspects of integrating sentiment and emotion analysis in chatbots for depression intervention

Introduction

Digital health interventions specifically those realized as chatbots are increasingly available for mental health. They include technologies based on artificial intelligence that assess user's sentiment and emotions for the purpose of responding in an empathetic way, or for treatment purposes, e.g. for analyzing the expressed emotions and suggesting interventions.

Methods

In this paper, we study the ethical dimensions of integrating these technologies in chatbots for depression intervention using the digital ethics canvas and the DTx Risk Assessment Canvas.

Results

As result, we identified some specific risks associated with the integration of sentiment and emotion analysis methods into these systems related to the difficulty to recognize correctly the expressed sentiment or emotion from statements of individuals with depressive symptoms and the appropriate system reaction including risk detection. Depending on the realization of the sentiment or emotion analysis, which might be dictionary-based or machine-learning based, additional risks occur from biased training data or misinterpretations.

Discussion

While technology decisions during system development can be made carefully depending on the use case, other ethical risks cannot be prevented on a technical level, but by carefully integrating such chatbots into the care process allowing for supervision by health professionals. We conclude that a careful reflection is needed when integrating sentiment and emotion analysis into chatbots for depression intervention. Balancing risk factors is key to leveraging technology in mental health in a way that enhances, rather than diminishes, user autonomy and agency.

"It happened to be the perfect thing": experiences of generative AI chatbots for mental health

The global mental health crisis underscores the need for accessible, effective interventions. Chatbots based on generative artificial intelligence (AI), like ChatGPT, are emerging as novel solutions, but research on real-life usage is limited. We interviewed nineteen individuals about their experiences using generative AI chatbots for mental health. Participants reported high engagement and positive impacts, including better relationships and healing from trauma and loss. We developed four themes: (1) a sense of 'emotional sanctuary', (2) 'insightful guidance', particularly about relationships, (3) the 'joy of connection', and (4) comparisons between the 'AI therapist' and human therapy. Some themes echoed prior research on rule-based chatbots, while others seemed novel to generative AI. Participants emphasised the need for better safety guardrails, human-like memory and the ability to lead the therapeutic process. Generative AI chatbots may offer mental health support that feels meaningful to users, but further research is needed on safety and effectiveness.

Dreaming big with little therapy devices: automated therapy from India

This paper examines the aspirations, imaginaries and utopias of designers of an AI-based mental health app in India. By looking at automated therapy as both technological fix and sociotechnical object, I ask, What can we learn from engaging with psy technologists' imaginaries and practices of health care futures? What are the assumptions they encode in the app? How does automated therapy reconfigure the geographies and temporalities of care? While automated therapy as instantiated by Wysa provides, I argue, a modest mental health intervention, the scalar aspirations of designers are anything but small. The paper proceeds in three steps. First, it turns to designers' imaginaries of what it means to care for current mental health needs in digitally saturated lifeworlds and how they inscribe them into the app. It identifies nonjudgmental listening, anonymity, acceptance, reframing, and agency as key ideas encoded in Wysa's sociotechnical algorithms, along with a congruence between entrepreneurial and encoded ethics of care. Second, it situates automated therapy within anthropological scholarship on 'little' technical devices in global health to argue that automated therapy devices such as Wysa articulate dreams for minimalist interventions with macro effects. Finally, it explores the new geographies and temporalities of care that automated therapy spurs, tracing the ways the app bridges various spatial and temporal gaps and obstacles of human therapy and upends common global health pathways. This paper contributes to recent scholarship on aspirations, dreams and utopias and on digitization and datafication in global health.

User perceptions and utilisation of features of an AI-enabled workplace digital mental wellness platform 'mindline at work'

BACKGROUND

The working population encounters unique work-related stressors. Despite these challenges, accessibility to mental healthcare remains limited. Digital technology-enabled mental wellness tools can offer much-needed access to mental healthcare. However, existing literature has given limited attention to their relevance and user engagement, particularly for the working population.

AIM

This study aims to assess user perceptions and feature utilisation of mindline at work, a nationally developed AI-enabled digital platform designed to improve mental wellness in the working population.

METHODS

This study adopted a mixed-methods design comprising a survey (n=399) and semistructured interviews (n=40) with office-based working adults. Participants were asked to use mindline at work for 4 weeks. We collected data about utilisation of the platform features, intention for sustained use and perceptions of specific features.

RESULTS

Participants under 5 years of work experience reported lower utilisation of multimedia resources but higher utilisation of emotion self-assessment tools and the AI chatbot compared with their counterparts (p<0.001). The platform received a moderate level of satisfaction (57%) and positive intention for sustained use (58%). Participants regarded mindline at work as an 'essential' safeguard against workplace stress, valuing its secure and non-judgmental space and user anonymity. However, they wanted greater institutional support for office workers' mental wellness to enhance the uptake. The AI chatbot was perceived as useful for self-reflection and problem-solving, despite limited maturity.

CONCLUSION

Identifying the unique benefits of specific features for different segments of working adults can foster a personalised user experience and promote mental well-being. Increasing workplace awareness is essential for platform adoption.

Behavior Change Techniques Within Digital Interventions for the Treatment of Eating Disorders: Systematic Review and Meta-Analysis

BACKGROUND

Previous systematic reviews of digital eating disorder interventions have demonstrated effectiveness at improving symptoms of eating disorders; however, our understanding of how these interventions work and what contributes to their effectiveness is limited. Understanding the behavior change techniques (BCTs) that are most commonly included within effective interventions may provide valuable information for researchers and developers. Establishing whether these techniques have been informed by theory will identify whether they target those mechanisms of action that have been identified as core to changing eating disorder behaviors. It will also evaluate the importance of a theoretical approach to digital intervention design.

OBJECTIVE

This study aims to define the BCTs within digital self-management interventions or minimally guided self-help interventions for adults with eating disorders that have been evaluated within randomized controlled trials. It also assessed which of the digital interventions were grounded in theory and the range of modes of delivery included.

METHODS

A literature search identified randomized controlled trials of digital intervention for the treatment of adults with eating disorders with minimal therapist support. Each digital intervention was coded for BCTs using the established BCT Taxonomy v1; for the application of theory using an adapted version of the theory coding scheme (TCS); and for modes of delivery using the Mode of Delivery Ontology. A meta-analysis evaluated the evidence that any individual BCT moderated effect size or that other potential factors such as the application of theory or number of modes of delivery had an effect on eating disorder outcomes.

RESULTS

Digital interventions included an average of 14 (SD 2.6; range 9-18) BCTs. Self-monitoring of behavior was included in all effective interventions, with Problem-solving, Information about antecedents, Feedback on behavior, Self-monitoring of outcomes of behavior, and Action planning identified in >75% (13/17) of effective interventions. Social support and Information about health consequences were more evident in effective interventions at follow-up compared with postintervention measurement. The mean number of modes of delivery was 4 (SD 1.6; range 2-7) out of 12 possible modes, with most interventions (15/17, 88%) being web based. Digital interventions that had a higher score on the TCS had a greater effect size than those with a lower TCS score (subgroup differences: χ21=9.7; P=.002; I²=89.7%) within the meta-analysis. No other subgroup analyses had statistically significant results.

CONCLUSIONS

There was a high level of consistency in terms of the most common BCTs within effective interventions; however, there was no evidence that any specific BCT contributed to intervention efficacy. The interventions that were more strongly informed by theory demonstrated greater improvements in eating disorder outcomes compared to waitlist or treatment-as-usual controls. These results can be used to inform the development of future digital eating disorder interventions.

TRIAL REGISTRATION

PROSPERO CRD42023410060; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=410060.

The effect of mobile social media on the mental health status of Chinese international students: an empirical study on the chain mediation effect

PURPOSE

To explore the impact of mobile social media on the psychological well-being of Chinese international students and analyze the mechanisms of influence to enhance their overall psychological health and social interactions in a foreign environment.

METHODS

Convenience sampling was employed, using questionnaires on Mobile Social Media, Psychological Resilience, Body Image, Health Goal Setting, Physical Activity Level, and Mental Health Status as measurement tools. Data were gathered from 378 Chinese international students across 33 universities in South Korea, including Kangwon National University, Myongji University, Kunsan National University, Seoul National University, and Chonbuk National University. Confirmatory factor analysis, correlation analysis, common method bias testing, and chain mediation effect analysis were conducted using SPSS and AMOS 23.0.

RESULTS

Mobile social media has significant indirect effects on the mental health of international students through various factors: psychological resilience and physical activity level (effect 'adg' = 0.080, 95% CI [0.029, 0.144]), body image and physical activity level (effect 'beg' = 0.122, 95% CI [0.044, 0.247]), and health goal setting and physical activity level (effect 'cfg' = 0.255, 95% CI [0.123, 0.428]).

CONCLUSION

The study shows that mobile social media benefits the mental health of Chinese international students by enhancing psychological resilience, physical activity, body image perception, and health goal setting. Collaboration between educational institutions and social media platforms is recommended to promote physical activity among international students. This collaboration can involve sharing encouraging messages, joining health communities, setting goals, and providing accessible exercise resources. Utilizing mobile apps or social media for tracking progress and goal-setting can also improve self-management skills.

Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review

BACKGROUND

Chatbots, or conversational agents, have emerged as significant tools in health care, driven by advancements in artificial intelligence and digital technology. These programs are designed to simulate human conversations, addressing various health care needs. However, no comprehensive synthesis of health care chatbots' roles, users, benefits, and limitations is available to inform future research and application in the field.

OBJECTIVE

This review aims to describe health care chatbots' characteristics, focusing on their diverse roles in the health care pathway, user groups, benefits, and limitations.

METHODS

A rapid review of published literature from 2017 to 2023 was performed with a search strategy developed in collaboration with a health sciences librarian and implemented in the MEDLINE and Embase databases. Primary research studies reporting on chatbot roles or benefits in health care were included. Two reviewers dual-screened the search results. Extracted data on chatbot roles, users, benefits, and limitations were subjected to content analysis.

RESULTS

The review categorized chatbot roles into 2 themes: delivery of remote health services, including patient support, care management, education, skills building, and health behavior promotion, and provision of administrative assistance to health care providers. User groups spanned across patients with chronic conditions as well as patients with cancer; individuals focused on lifestyle improvements; and various demographic groups such as women, families, and older adults. Professionals and students in health care also emerged as significant users, alongside groups seeking mental health support, behavioral change, and educational enhancement. The benefits of health care chatbots were also classified into 2 themes: improvement of health care quality and efficiency and cost-effectiveness in health care delivery. The identified limitations encompassed ethical challenges, medicolegal and safety concerns, technical difficulties, user experience issues, and societal and economic impacts.

CONCLUSIONS

Health care chatbots offer a wide spectrum of applications, potentially impacting various aspects of health care. While they are promising tools for improving health care efficiency and quality, their integration into the health care system must be approached with consideration of their limitations to ensure optimal, safe, and equitable use.

The Role of Humanization and Robustness of Large Language Models in Conversational Artificial Intelligence for Individuals With Depression: A Critical Analysis

Unlabelled

Large language model (LLM)-powered services are gaining popularity in various applications due to their exceptional performance in many tasks, such as sentiment analysis and answering questions. Recently, research has been exploring their potential use in digital health contexts, particularly in the mental health domain. However, implementing LLM-enhanced conversational artificial intelligence (CAI) presents significant ethical, technical, and clinical challenges. In this viewpoint paper, we discuss 2 challenges that affect the use of LLM-enhanced CAI for individuals with mental health issues, focusing on the use case of patients with depression: the tendency to humanize LLM-enhanced CAI and their lack of contextualized robustness. Our approach is interdisciplinary, relying on considerations from philosophy, psychology, and computer science. We argue that the humanization of LLM-enhanced CAI hinges on the reflection of what it means to simulate "human-like" features with LLMs and what role these systems should play in interactions with humans. Further, ensuring the contextualization of the robustness of LLMs requires considering the specificities of language production in individuals with depression, as well as its evolution over time. Finally, we provide a series of recommendations to foster the responsible design and deployment of LLM-enhanced CAI for the therapeutic support of individuals with depression.

Considering the Role of Human Empathy in AI-Driven Therapy

Recent breakthroughs in artificial intelligence (AI) language models have elevated the vision of using conversational AI support for mental health, with a growing body of literature indicating varying degrees of efficacy. In this paper, we ask when, in therapy, it will be easier to replace humans and, conversely, in what instances, human connection will still be more valued. We suggest that empathy lies at the heart of the answer to this question. First, we define different aspects of empathy and outline the potential empathic capabilities of humans versus AI. Next, we consider what determines when these aspects are needed most in therapy, both from the perspective of therapeutic methodology and from the perspective of patient objectives. Ultimately, our goal is to prompt further investigation and dialogue, urging both practitioners and scholars engaged in AI-mediated therapy to keep these questions and considerations in mind when investigating AI implementation in mental health.

Social support of virtual characters reduces pain perception

BACKGROUND

Psychosocial factors, such as social support, can reduce pain. Virtual reality (VR) is a powerful tool to decrease pain, but social factors in VR-based pain analgesia have rarely been studied. Specifically, it is unclear whether social support by virtual characters can reduce pain and whether the perceived control behind virtual characters (agency) and varying degrees of social cues impact pain perception.

METHODS

Healthy participants (N = 97) received heat pain stimulation while undergoing four within-subject conditions in immersive VR: (1) virtual character with a low number of social cues (virtual figure) provided verbal support, (2) virtual character with a high number of social cues (virtual human) provided verbal support, (3) no social support (hearing neutral words), (4) no social support. Perceived agency of the virtual characters served as between-subjects factor. Participants in the avatar group were led to believe that another participant controlled the virtual characters. Participants in the agent group were told they interacted with a computer. However, in both conditions, virtual characters were computer-controlled. Pain ratings, psychophysiological measurements and presence ratings were recorded.

RESULTS

Virtual social support decreased pain intensity and pain unpleasantness ratings but had no impact on electrodermal activity nor heart rate. A virtual character with a high number of social cues led to lower pain unpleasantness and higher feelings of presence. Agency had no significant impact.

CONCLUSIONS

Virtual characters providing social support can reduce pain independent of perceived agency. A more human visual appearance can have beneficial effects on social pain modulation by virtual characters.

SIGNIFICANCE

Social influences are important factors in pain modulation. The current study demonstrated analgesic effects through verbal support provided by virtual characters and investigated modulating factors. A more human appearance of a virtual character resulted in a higher reduction of pain unpleasantness. Importantly, agency of the virtual characters had no impact. Given the increasing use of digital health interventions, the findings suggest a positive impact of virtual characters for digital pain treatments.

Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation

Large language models (LLMs) such as Open AI's GPT-4 (which power ChatGPT) and Google's Gemini, built on artificial intelligence, hold immense potential to support, augment, or even eventually automate psychotherapy. Enthusiasm about such applications is mounting in the field as well as industry. These developments promise to address insufficient mental healthcare system capacity and scale individual access to personalized treatments. However, clinical psychology is an uncommonly high stakes application domain for AI systems, as responsible and evidence-based therapy requires nuanced expertise. This paper provides a roadmap for the ambitious yet responsible application of clinical LLMs in psychotherapy. First, a technical overview of clinical LLMs is presented. Second, the stages of integration of LLMs into psychotherapy are discussed while highlighting parallels to the development of autonomous vehicle technology. Third, potential applications of LLMs in clinical care, training, and research are discussed, highlighting areas of risk given the complex nature of psychotherapy. Fourth, recommendations for the responsible development and evaluation of clinical LLMs are provided, which include centering clinical science, involving robust interdisciplinary collaboration, and attending to issues like assessment, risk detection, transparency, and bias. Lastly, a vision is outlined for how LLMs might enable a new generation of studies of evidence-based interventions at scale, and how these studies may challenge assumptions about psychotherapy.

Development and Evaluation of a Digital App for Patient Self-Management of Opioid Use Disorder: Usability, Acceptability, and Utility Study

BACKGROUND

Self-management of opioid use disorder (OUD) is an important component of treatment. Many patients receiving opioid agonist treatment in methadone maintenance treatment settings benefit from counseling treatments to help them improve their recovery skills but have insufficient access to these treatments between clinic appointments. In addition, many addiction medicine clinicians treating patients with OUD in a general medical clinic setting do not have consistent access to counseling referrals for their patients. This can lead to decreases in both treatment retention and overall progress in the patient's recovery from substance misuse. Digital apps may help to bridge this gap by coaching, supporting, and reinforcing behavioral change that is initiated and directed by their psychosocial and medical providers.

OBJECTIVE

This study aimed to conduct an acceptability, usability, and utility pilot study of the KIOS app to address these clinical needs.

METHODS

We developed a unique, patient-centered computational software system (KIOS; Biomedical Development Corporation) to assist in managing OUD in an outpatient, methadone maintenance clinic setting. KIOS tracks interacting self-reported symptoms (craving, depressed mood, anxiety, irritability, pain, agitation or restlessness, difficulty sleeping, absenteeism, difficulty with usual activities, and conflicts with others) to determine changes in both the trajectory and severity of symptom patterns over time. KIOS then applies a proprietary algorithm to assess the individual's patterns of symptom interaction in accordance with models previously established by OUD experts. After this analysis, KIOS provides specific behavioral advice addressing the individual's changing trajectory of symptoms to help the person self-manage their symptoms. The KIOS software also provides analytics on the self-reported data that can be used by patients, clinicians, and researchers to track outcomes.

RESULTS

In a 4-week acceptability, usability (mean System Usability Scale-Modified score 89.5, SD 9.2, maximum of 10.0), and utility (mean KIOS utility questionnaire score 6.32, SD 0.25, maximum of 7.0) pilot study of 15 methadone-maintained participants with OUD, user experience, usability, and software-generated advice received high and positive assessment scores. The KIOS clinical variables closely correlated with craving self-report measures. Therefore, managing these variables with advice generated by the KIOS software could have an impact on craving and ultimately substance use.

CONCLUSIONS

KIOS tracks key clinical variables and generates advice specifically relevant to the patient's current and changing clinical state. Patients in this pilot study assigned high positive values to the KIOS user experience, ease of use, and the appropriateness, relevance, and usefulness of the specific behavioral guidance they received to match their evolving experiences. KIOS may therefore be useful to augment in-person treatment of opioid agonist patients and help fill treatment gaps that currently exist in the continuum of care. A National Institute on Drug Abuse-funded randomized controlled trial of KIOS to augment in-person treatment of patients with OUD is currently being conducted.

Potential use of large language models for mitigating students' problematic social media use: ChatGPT as an example

The problematic use of social media has numerous negative impacts on individuals' daily lives, interpersonal relationships, physical and mental health, and more. Currently, there are few methods and tools to alleviate problematic social media, and their potential is yet to be fully realized. Emerging large language models (LLMs) are becoming increasingly popular for providing information and assistance to people and are being applied in many aspects of life. In mitigating problematic social media use, LLMs such as ChatGPT can play a positive role by serving as conversational partners and outlets for users, providing personalized information and resources, monitoring and intervening in problematic social media use, and more. In this process, we should recognize both the enormous potential and endless possibilities of LLMs such as ChatGPT, leveraging their advantages to better address problematic social media use, while also acknowledging the limitations and potential pitfalls of ChatGPT technology, such as errors, limitations in issue resolution, privacy and security concerns, and potential overreliance. When we leverage the advantages of LLMs to address issues in social media usage, we must adopt a cautious and ethical approach, being vigilant of the potential adverse effects that LLMs may have in addressing problematic social media use to better harness technology to serve individuals and society.

Evaluation framework for conversational agents with artificial intelligence in health interventions: a systematic scoping review

OBJECTIVES

Conversational agents (CAs) with emerging artificial intelligence present new opportunities to assist in health interventions but are difficult to evaluate, deterring their applications in the real world. We aimed to synthesize existing evidence and knowledge and outline an evaluation framework for CA interventions.

MATERIALS AND METHODS

We conducted a systematic scoping review to investigate designs and outcome measures used in the studies that evaluated CAs for health interventions. We then nested the results into an overarching digital health framework proposed by the World Health Organization (WHO).

RESULTS

The review included 81 studies evaluating CAs in experimental (n = 59), observational (n = 15) trials, and other research designs (n = 7). Most studies (n = 72, 89%) were published in the past 5 years. The proposed CA-evaluation framework includes 4 evaluation stages: (1) feasibility/usability, (2) efficacy, (3) effectiveness, and (4) implementation, aligning with WHO's stepwise evaluation strategy. Across these stages, this article presents the essential evidence of different study designs (n = 8), sample sizes, and main evaluation categories (n = 7) with subcategories (n = 40). The main evaluation categories included (1) functionality, (2) safety and information quality, (3) user experience, (4) clinical and health outcomes, (5) costs and cost benefits, (6) usage, adherence, and uptake, and (7) user characteristics for implementation research. Furthermore, the framework highlighted the essential evaluation areas (potential primary outcomes) and gaps across the evaluation stages.

DISCUSSION AND CONCLUSION

This review presents a new framework with practical design details to support the evaluation of CA interventions in healthcare research.

PROTOCOL REGISTRATION

The Open Science Framework (https://osf.io/9hq2v) on March 22, 2021.

Demographic and clinical characteristics associated with anxiety and depressive symptom outcomes in users of a digital mental health intervention incorporating a relational agent

BACKGROUND

Digital mental health interventions (DMHIs) may reduce treatment access issues for those experiencing depressive and/or anxiety symptoms. DMHIs that incorporate relational agents may offer unique ways to engage and respond to users and to potentially help reduce provider burden. This study tested Woebot for Mood & Anxiety (W-MA-02), a DMHI that employs Woebot, a relational agent that incorporates elements of several evidence-based psychotherapies, among those with baseline clinical levels of depressive or anxiety symptoms. Changes in self-reported depressive and anxiety symptoms over 8 weeks were measured, along with the association between each of these outcomes and demographic and clinical characteristics.

METHODS

This exploratory, single-arm, 8-week study of 256 adults yielded non-mutually exclusive subsamples with either clinical levels of depressive or anxiety symptoms at baseline. Week 8 Patient Health Questionnaire-8 (PHQ-8) changes were measured in the depressive subsample (PHQ-8 ≥ 10). Week 8 Generalized Anxiety Disorder-7 (GAD-7) changes were measured in the anxiety subsample (GAD-7 ≥ 10). Demographic and clinical characteristics were examined in association with symptom changes via bivariate and multiple regression models adjusted for W-MA-02 utilization. Characteristics included age, sex at birth, race/ethnicity, marital status, education, sexual orientation, employment status, health insurance, baseline levels of depressive and anxiety symptoms, and concurrent psychotherapeutic or psychotropic medication treatments during the study.

RESULTS

Both the depressive and anxiety subsamples were predominantly female, educated, non-Hispanic white, and averaged 38 and 37 years of age, respectively. The depressive subsample had significant reductions in depressive symptoms at Week 8 (mean change =-7.28, SD = 5.91, Cohen's d = -1.23, p < 0.01); the anxiety subsample had significant reductions in anxiety symptoms at Week 8 (mean change = -7.45, SD = 5.99, Cohen's d = -1.24, p < 0.01). No significant associations were found between sex at birth, age, employment status, educational background and Week 8 symptom changes. Significant associations between depressive and anxiety symptom outcomes and sexual orientation, marital status, concurrent mental health treatment, and baseline symptom severity were found.

CONCLUSIONS

The present study suggests early promise for W-MA-02 as an intervention for depression and/or anxiety symptoms. Although exploratory in nature, this study revealed potential user characteristics associated with outcomes that can be investigated in future studies.

TRIAL REGISTRATION

This study was retrospectively registered on ClinicalTrials.gov (#NCT05672745) on January 5th, 2023.

Development and Evaluation of a Smartphone-Based Chatbot Coach to Facilitate a Balanced Lifestyle in Individuals With Headaches (BalanceUP App): Randomized Controlled Trial

BACKGROUND

Primary headaches, including migraine and tension-type headaches, are widespread and have a social, physical, mental, and economic impact. Among the key components of treatment are behavior interventions such as lifestyle modification. Scalable conversational agents (CAs) have the potential to deliver behavior interventions at a low threshold. To our knowledge, there is no evidence of behavioral interventions delivered by CAs for the treatment of headaches.

OBJECTIVE

This study has 2 aims. The first aim was to develop and test a smartphone-based coaching intervention (BalanceUP) for people experiencing frequent headaches, delivered by a CA and designed to improve mental well-being using various behavior change techniques. The second aim was to evaluate the effectiveness of BalanceUP by comparing the intervention and waitlist control groups and assess the engagement and acceptance of participants using BalanceUP.

METHODS

In an unblinded randomized controlled trial, adults with frequent headaches were recruited on the web and in collaboration with experts and allocated to either a CA intervention (BalanceUP) or a control condition. The effects of the treatment on changes in the primary outcome of the study, that is, mental well-being (as measured by the Patient Health Questionnaire Anxiety and Depression Scale), and secondary outcomes (eg, psychosomatic symptoms, stress, headache-related self-efficacy, intention to change behavior, presenteeism and absenteeism, and pain coping) were analyzed using linear mixed models and Cohen d. Primary and secondary outcomes were self-assessed before and after the intervention, and acceptance was assessed after the intervention. Engagement was measured during the intervention using self-reports and usage data.

RESULTS

A total of 198 participants (mean age 38.7, SD 12.14 y; n=172, 86.9% women) participated in the study (intervention group: n=110; waitlist control group: n=88). After the intervention, the intention-to-treat analysis revealed evidence for improved well-being (treatment: β estimate=-3.28, 95% CI -5.07 to -1.48) with moderate between-group effects (Cohen d=-0.66, 95% CI -0.99 to -0.33) in favor of the intervention group. We also found evidence of reduced somatic symptoms, perceived stress, and absenteeism and presenteeism, as well as improved headache management self-efficacy, application of behavior change techniques, and pain coping skills, with effects ranging from medium to large (Cohen d=0.43-1.05). Overall, 64.8% (118/182) of the participants used coaching as intended by engaging throughout the coaching and completing the outro.

CONCLUSIONS

BalanceUP was well accepted, and the results suggest that coaching delivered by a CA can be effective in reducing the burden of people who experience headaches by improving their well-being.

TRIAL REGISTRATION

German Clinical Trials Register DRKS00017422; https://trialsearch.who.int/Trial2.aspx?TrialID=DRKS00017422.

The New Emerging Treatment Choice for Major Depressive Disorders: Digital Therapeutics

The chapter provides an in-depth analysis of digital therapeutics (DTx) as a revolutionary approach to managing major depressive disorder (MDD). It discusses the evolution and definition of DTx, their application across various medical fields, regulatory considerations, and their benefits and limitations. This chapter extensively covers DTx for MDD, including smartphone applications, virtual reality interventions, cognitive-behavioral therapy (CBT) platforms, artificial intelligence (AI) and chatbot therapies, biofeedback, wearable technologies, and serious games. It evaluates the effectiveness of these digital interventions, comparing them with traditional treatments and examining patient perspectives, compliance, and engagement. The integration of DTx into clinical practice is also explored, along with the challenges and barriers to their adoption, such as technological limitations, data privacy concerns, ethical considerations, reimbursement issues, and the need for improved digital literacy. This chapter concludes by looking at the future direction of DTx in mental healthcare, emphasizing the need for personalized treatment plans, integration with emerging modalities, and the expansion of access to these innovative solutions globally.

Qualitative analysis of mental health conversational agents messages about autism spectrum disorder: a call for action

Background

Conversational agents (CA's) have shown promise in increasing accessibility to mental health resources. This study aimed to identify common themes of messages sent to a mental health CA (Wysa) related to ASD by general users and users that identify as having ASD.

Methods

This study utilized retrospective data. Two thematic analyses were conducted, one focusing on user messages including the keywords (e.g., ASD, autism, Asperger), and the second one with messages from users who self-identified as having ASD.

Results

For the sample of general users, the most frequent themes were "others having ASD," "ASD diagnosis," and "seeking help." For the users that self-identified as having ASD (n = 277), the most frequent themes were "ASD diagnosis or symptoms," "negative reaction from others," and "positive comments." There were 3,725 emotion words mentioned by users who self-identified as having ASD. The majority had negative valence (80.3%), and few were positive (14.8%) or ambivalent (4.9%).

Conclusion

Users shared their experiences and emotions surrounding ASD with a mental health CA. Users asked about the ASD diagnosis, sought help, and reported negative reactions from others. CA's have the potential to become a source of support for those interested in ASD and/or identify as having ASD.

An Integrative Survey on Mental Health Conversational Agents to Bridge Computer Science and Medical Perspectives

Mental health conversational agents (a.k.a. chatbots) are widely studied for their potential to offer accessible support to those experiencing mental health challenges. Previous surveys on the topic primarily consider papers published in either computer science or medicine, leading to a divide in understanding and hindering the sharing of beneficial knowledge between both domains. To bridge this gap, we conduct a comprehensive literature review using the PRISMA framework, reviewing 534 papers published in both computer science and medicine. Our systematic review reveals 136 key papers on building mental health-related conversational agents with diverse characteristics of modeling and experimental design techniques. We find that computer science papers focus on LLM techniques and evaluating response quality using automated metrics with little attention to the application while medical papers use rule-based conversational agents and outcome metrics to measure the health outcomes of participants. Based on our findings on transparency, ethics, and cultural heterogeneity in this review, we provide a few recommendations to help bridge the disciplinary divide and enable the cross-disciplinary development of mental health conversational agents.

Behavioral Mechanisms That Mediate Mental and Physical Health Improvements in People With Chronic Pain Who Receive a Digital Health Intervention: Prospective Cohort Pilot Study

BACKGROUND

Preliminary evidence suggests that digital mental health intervention (Wysa for Chronic Pain) can improve mental and physical health in people with chronic musculoskeletal pain and coexisting symptoms of depression or anxiety. However, the behavioral mechanisms through which this intervention acts are not fully understood.

OBJECTIVE

The purpose of this study was to identify behavioral mechanisms that may mediate changes in mental and physical health associated with use of Wysa for Chronic Pain during orthopedic management of chronic musculoskeletal pain. We hypothesized that improved behavioral activation, pain acceptance, and sleep quality mediate improvements in self-reported mental and physical health.

METHODS

In this prospective cohort, pilot mediation analysis, adults with chronic (≥3 months) neck or back pain received the Wysa for Chronic Pain digital intervention, which uses a conversational agent and text-based access to human counselors to deliver cognitive behavioral therapy and related therapeutic content. Patient-reported outcomes and proposed mediators were collected at baseline and 1 month. The exposure of interest was participants' engagement (ie, total interactions) with the digital intervention. Proposed mediators were assessed using the Behavioral Activation for Depression Scale-Short Form, Chronic Pain Acceptance Questionnaire, and Athens Insomnia Scale. Outcomes included Patient-Reported Outcomes Measurement Information System Anxiety, Depression, Pain Interference, and Physical Function scores. A mediation analysis was conducted using the Baron and Kenny method, adjusting for age, sex, and baseline mediators and outcome values. P<.20 was considered significant for this pilot study.

RESULTS

Among 30 patients (mean age 59, SD 14, years; 21 [70%] female), the mediation effect of behavioral activation on the relationship between increased intervention engagement and improved anxiety symptoms met predefined statistical significance thresholds (indirect effect -0.4, 80% CI -0.7 to -0.1; P=.13, 45% of the total effect). The direction of mediation effect was generally consistent with our hypothesis for all other proposed mediator or outcome relationships, as well.

CONCLUSIONS

In a full-sized randomized controlled trial of patients with chronic musculoskeletal pain, behavioral activation, pain acceptance, and sleep quality may play an important role in mediating the relationship between use of a digital mental health intervention (Wysa for Chronic Pain) and improved mental and physical health.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05194722; https://clinicaltrials.gov/ct2/show/NCT05194722.

The Potential of Chatbots for Emotional Support and Promoting Mental Well-Being in Different Cultures: Mixed Methods Study

BACKGROUND

Artificial intelligence chatbot research has focused on technical advances in natural language processing and validating the effectiveness of human-machine conversations in specific settings. However, real-world chat data remain proprietary and unexplored despite their growing popularity, and new analyses of chatbot uses and their effects on mitigating negative moods are urgently needed.

OBJECTIVE

In this study, we investigated whether and how artificial intelligence chatbots facilitate the expression of user emotions, specifically sadness and depression. We also examined cultural differences in the expression of depressive moods among users in Western and Eastern countries.

METHODS

This study used SimSimi, a global open-domain social chatbot, to analyze 152,783 conversation utterances containing the terms "depress" and "sad" in 3 Western countries (Canada, the United Kingdom, and the United States) and 5 Eastern countries (Indonesia, India, Malaysia, the Philippines, and Thailand). Study 1 reports new findings on the cultural differences in how people talk about depression and sadness to chatbots based on Linguistic Inquiry and Word Count and n-gram analyses. In study 2, we classified chat conversations into predefined topics using semisupervised classification techniques to better understand the types of depressive moods prevalent in chats. We then identified the distinguishing features of chat-based depressive discourse data and the disparity between Eastern and Western users.

RESULTS

Our data revealed intriguing cultural differences. Chatbot users in Eastern countries indicated stronger emotions about depression than users in Western countries (positive: P<.001; negative: P=.01); for example, Eastern users used more words associated with sadness (P=.01). However, Western users were more likely to share vulnerable topics such as mental health (P<.001), and this group also had a greater tendency to discuss sensitive topics such as swear words (P<.001) and death (P<.001). In addition, when talking to chatbots, people expressed their depressive moods differently than on other platforms. Users were more open to expressing emotional vulnerability related to depressive or sad moods to chatbots (74,045/148,590, 49.83%) than on social media (149/1978, 7.53%). Chatbot conversations tended not to broach topics that require social support from others, such as seeking advice on daily life difficulties, unlike on social media. However, chatbot users acted in anticipation of conversational agents that exhibit active listening skills and foster a safe space where they can openly share emotional states such as sadness or depression.

CONCLUSIONS

The findings highlight the potential of chatbot-assisted mental health support, emphasizing the importance of continued technical and policy-wise efforts to improve chatbot interactions for those in need of emotional assistance. Our data indicate the possibility of chatbots providing helpful information about depressive moods, especially for users who have difficulty communicating emotions to other humans.

User Engagement Clusters of an 8-Week Digital Mental Health Intervention Guided by a Relational Agent (Woebot): Exploratory Study

BACKGROUND

With the proliferation of digital mental health interventions (DMHIs) guided by relational agents, little is known about the behavioral, cognitive, and affective engagement components associated with symptom improvement over time. Obtaining a better understanding could lend clues about recommended use for particular subgroups of the population, the potency of different intervention components, and the mechanisms underlying the intervention's success.

OBJECTIVE

This exploratory study applied clustering techniques to a range of engagement indicators, which were mapped to the intervention's active components and the connect, attend, participate, and enact (CAPE) model, to examine the prevalence and characterization of each identified cluster among users of a relational agent-guided DMHI.

METHODS

We invited adults aged 18 years or older who were interested in using digital support to help with mood management or stress reduction through social media to participate in an 8-week DMHI guided by a natural language processing-supported relational agent, Woebot. Users completed assessments of affective and cognitive engagement, working alliance as measured by goal and task working alliance subscale scores, and enactment (ie, application of therapeutic recommendations in real-world settings). The app passively collected data on behavioral engagement (ie, utilization). We applied agglomerative hierarchical clustering analysis to the engagement indicators to identify the number of clusters that provided the best fit to the data collected, characterized the clusters, and then examined associations with baseline demographic and clinical characteristics as well as mental health outcomes at week 8.

RESULTS

Exploratory analyses (n=202) supported 3 clusters: (1) "typical utilizers" (n=81, 40%), who had intermediate levels of behavioral engagement; (2) "early utilizers" (n=58, 29%), who had the nominally highest levels of behavioral engagement in week 1; and (3) "efficient engagers" (n=63, 31%), who had significantly higher levels of affective and cognitive engagement but the lowest level of behavioral engagement. With respect to mental health baseline and outcome measures, efficient engagers had significantly higher levels of baseline resilience (P<.001) and greater declines in depressive symptoms (P=.01) and stress (P=.01) from baseline to week 8 compared to typical utilizers. Significant differences across clusters were found by age, gender identity, race and ethnicity, sexual orientation, education, and insurance coverage. The main analytic findings remained robust in sensitivity analyses.

CONCLUSIONS

There were 3 distinct engagement clusters found, each with distinct baseline demographic and clinical traits and mental health outcomes. Additional research is needed to inform fine-grained recommendations regarding optimal engagement and to determine the best sequence of particular intervention components with known potency. The findings represent an important first step in disentangling the complex interplay between different affective, cognitive, and behavioral engagement indicators and outcomes associated with use of a DMHI incorporating a natural language processing-supported relational agent.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05672745; https://classic.clinicaltrials.gov/ct2/show/NCT05672745.

Waiting for a digital therapist: three challenges on the path to psychotherapy delivered by artificial intelligence

Growing demand for broadly accessible mental health care, together with the rapid development of new technologies, trigger discussions about the feasibility of psychotherapeutic interventions based on interactions with Conversational Artificial Intelligence (CAI). Many authors argue that while currently available CAI can be a useful supplement for human-delivered psychotherapy, it is not yet capable of delivering fully fledged psychotherapy on its own. The goal of this paper is to investigate what are the most important obstacles on our way to developing CAI systems capable of delivering psychotherapy in the future. To this end, we formulate and discuss three challenges central to this quest. Firstly, we might not be able to develop effective AI-based psychotherapy unless we deepen our understanding of what makes human-delivered psychotherapy effective. Secondly, assuming that it requires building a therapeutic relationship, it is not clear whether psychotherapy can be delivered by non-human agents. Thirdly, conducting psychotherapy might be a problem too complicated for narrow AI, i.e., AI proficient in dealing with only relatively simple and well-delineated tasks. If this is the case, we should not expect CAI to be capable of delivering fully-fledged psychotherapy until the so-called "general" or "human-like" AI is developed. While we believe that all these challenges can ultimately be overcome, we think that being mindful of them is crucial to ensure well-balanced and steady progress on our path to AI-based psychotherapy.

A relational agent for treating substance use in adults: Protocol for a randomized controlled trial with a psychoeducational comparator

BACKGROUND

Substance use disorders (SUDs) are prevalent and compromise health and wellbeing. Scalable solutions, such as digital therapeutics, may offer a population-based strategy for addressing SUDs. Two formative studies supported the feasibility and acceptability of the relational agent Woebot, an animated screen-based social robot, for treating SUDs (W-SUDs) in adults. Participants randomized to W-SUDs reduced their substance use occasions from baseline to end-of-treatment (EOT) relative to a waitlist control.

OBJECTIVE

To further develop the evidence base, the current randomized trial extends follow-up to 1-month post-treatment and will test the efficacy of W-SUDs relative to a psychoeducational control.

METHODS

This study will recruit, screen, and consent 400 adults online reporting problematic substance use. Following baseline assessment, participants will be randomized to 8 weeks of W-SUDs or a psychoeducational control. Assessments will be conducted at weeks 4, 8 (EOT), and 12 (1-month post-treatment). Primary outcome is past-month number of substance use occasions, summed across all substances. Secondary outcomes are number of heavy drinking days, the percent of days abstinent from all substances, substance use problems, thoughts about abstinence, cravings, confidence to resist substance use, symptoms of depression and anxiety, and work productivity. If significant group differences are found, we will explore moderators and mediators of treatment effects.

CONCLUSIONS

The current study builds upon emerging evidence of a digital therapeutic for reducing problematic substance use by examining sustained effects and testing against a psychoeducational control condition. If efficacious, the findings have implications for scalable mobile health interventions for reducing problematic substance use.

TRIAL REGISTRATION

NCT04925570.

Exploring the Perceptions of mHealth Interventions for the Prevention of Common Mental Disorders in University Students in Singapore: Qualitative Study

BACKGROUND

Mental health interventions delivered through mobile health (mHealth) technologies can increase the access to mental health services, especially among university students. The development of mHealth intervention is complex and needs to be context sensitive. There is currently limited evidence on the perceptions, needs, and barriers related to these interventions in the Southeast Asian context.

OBJECTIVE

This qualitative study aimed to explore the perception of university students and mental health supporters in Singapore about mental health services, campaigns, and mHealth interventions with a focus on conversational agent interventions for the prevention of common mental disorders such as anxiety and depression.

METHODS

We conducted 6 web-based focus group discussions with 30 university students and one-to-one web-based interviews with 11 mental health supporters consisting of faculty members tasked with student pastoral care, a mental health first aider, counselors, psychologists, a clinical psychologist, and a psychiatrist. The qualitative analysis followed a reflexive thematic analysis framework.

RESULTS

The following 6 main themes were identified: a healthy lifestyle as students, access to mental health services, the role of mental health promotion campaigns, preferred mHealth engagement features, factors that influence the adoption of mHealth interventions, and cultural relevance of mHealth interventions. The interpretation of our findings shows that students were reluctant to use mental health services because of the fear of stigma and a possible lack of confidentiality.

CONCLUSIONS

Study participants viewed mHealth interventions for mental health as part of a blended intervention. They also felt that future mental health mHealth interventions should be more personalized and capable of managing adverse events such as suicidal ideation.

Evaluating a Digital Mental Health Intervention (Wysa) for Workers' Compensation Claimants: Pilot Feasibility Study

OBJECTIVE

This study examines the feasibility and acceptability of an AI-led digital mental health intervention in a Workers' Compensation (WC) program, Wysa for Return to Work.

METHODS

Self-reported demographic data and responses to psychosocial screening questions were analyzed alongside participants' app usage through which four key outcomes were measured: recruitment rate, onboarding rate, retention, and engagement.

RESULTS

The data demonstrated a high need for psychosocial interventions among injured workers, especially women, young adults, and those with high severity injuries. Those with more psychosocial risk factors had a higher rate of onboarding, retention, and engagement, and those with severe injuries had higher retention.

CONCLUSIONS

Our study concluded that Wysa for Return to Work, the AI-led digital mental health intervention that delivers a recovery program using a digital conversational agent, is feasible and acceptable for a return-to-work population.

Understanding Digital Mental Health Needs and Usage With an Artificial Intelligence-Led Mental Health App (Wysa) During the COVID-19 Pandemic: Retrospective Analysis

BACKGROUND

There has been a surge in mental health concerns during the COVID-19 pandemic, which has prompted the increased use of digital platforms. However, there is little known about the mental health needs and behaviors of the global population during the pandemic. This study aims to fill this knowledge gap through the analysis of real-world data collected from users of a digital mental health app (Wysa) regarding their engagement patterns and behaviors, as shown by their usage of the service.

OBJECTIVE

This study aims to (1) examine the relationship between mental health distress, digital health uptake, and COVID-19 case numbers; (2) evaluate engagement patterns with the app during the study period; and (3) examine the efficacy of the app in improving mental health outcomes for its users during the pandemic.

METHODS

This study used a retrospective observational design. During the COVID-19 pandemic, the app's installations and emotional utterances were measured from March 2020 to October 2021 for the United Kingdom, the United States of America, and India and were mapped against COVID-19 case numbers and their peaks. The engagement of the users from this period (N=4541) with the Wysa app was compared to that of equivalent samples of users from a pre-COVID-19 period (1000 iterations). The efficacy was assessed for users who completed pre-post assessments for symptoms of depression (n=2061) and anxiety (n=1995) on the Patient Health Questionnaire-9 (PHQ-9) and Generalized Anxiety Disorder-7 (GAD-7) test measures, respectively.

RESULTS

Our findings demonstrate a significant positive correlation between the increase in the number of installs of the Wysa mental health app and the peaks of COVID-19 case numbers in the United Kingdom (P=.02) and India (P<.001). Findings indicate that users (N=4541) during the COVID period had a significantly higher engagement than the samples from the pre-COVID period, with a medium to large effect size for 80% of these 1000 iterative samples, as observed on the Mann-Whitney test. The PHQ-9 and GAD-7 pre-post assessments indicated statistically significant improvement with a medium effect size (PHQ-9: P=.57; GAD-7: P=.56).

CONCLUSIONS

This study demonstrates that emotional distress increased substantially during the pandemic, prompting the increased uptake of an artificial intelligence-led mental health app (Wysa), and also offers evidence that the Wysa app could support its users and its usage could result in a significant reduction in symptoms of anxiety and depression. This study also highlights the importance of contextualizing interventions and suggests that digital health interventions can provide large populations with scalable and evidence-based support for mental health care.

AI-based chatbot micro-intervention for parents: Meaningful engagement, learning, and efficacy

Introduction

Mental health issues have been on the rise among children and adolescents, and digital parenting programs have shown promising outcomes. However, there is limited research on the potential efficacy of utilizing chatbots to promote parental skills. This study aimed to understand whether parents learn from a parenting chatbot micro intervention, to assess the overall efficacy of the intervention, and to explore the user characteristics of the participants, including parental busyness, assumptions about parenting, and qualitative engagement with the chatbot.

Methods

A sample of 170 parents with at least one child between 2-11 years old were recruited. A randomized control trial was conducted. Participants in the experimental group accessed a 15-min intervention that taught how to utilize positive attention and praise to promote positive behaviors in their children, while the control group remained on a waiting list.

Results

Results showed that participants engaged with a brief AI-based chatbot intervention and were able to learn effective praising skills. Although scores moved in the expected direction, there were no significant differences by condition in the praising knowledge reported by parents, perceived changes in disruptive behaviors, or parenting self-efficacy, from pre-intervention to 24-hour follow-up.

Discussion

The results provided insight to understand how parents engaged with the chatbot and suggests that, in general, brief, self-guided, digital interventions can promote learning in parents. It is possible that a higher dose of intervention may be needed to obtain a therapeutic change in parents. Further research implications on chatbots for parenting skills are discussed.

User experience with a parenting chatbot micro intervention

Background

The use of chatbots to address mental health conditions have become increasingly popular in recent years. However, few studies aimed to teach parenting skills through chatbots, and there are no reports on parental user experience. Aim: This study aimed to assess the user experience of a parenting chatbot micro intervention to teach how to praise children in a Spanish-speaking country.

Methods

A sample of 89 parents were assigned to the chatbot micro intervention as part of a randomized controlled trial study. Completion rates, engagement, satisfaction, net promoter score, and acceptability were analyzed.

Results

66.3% of the participants completed the intervention. Participants exchanged an average of 49.8 messages (SD = 1.53), provided an average satisfaction score of 4.19 (SD = .79), and reported that they would recommend the chatbot to other parents (net promoter score = 4.63/5; SD = .66). Acceptability level was high (ease of use = 4.66 [SD = .73]; comfortability = 4.76 [SD = .46]; lack of technical problems = 4.69 [SD = .59]; interactivity = 4.51 [SD = .77]; usefulness for everyday life = 4.75 [SD = .54]).

Conclusions

Overall, users completed the intervention at a high rate, engaged with the chatbot, were satisfied, would recommend it to others, and reported a high level of acceptability. Chatbots have the potential to teach parenting skills however research on the efficacy of parenting chatbot interventions is needed.

Evaluating Conversational Agents for Mental Health: Scoping Review of Outcomes and Outcome Measurement Instruments (Preprint)

BACKGROUND

Rapid proliferation of mental health interventions delivered through conversational agents (CAs) calls for high-quality evidence to support their implementation and adoption. Selecting appropriate outcomes, instruments for measuring outcomes, and assessment methods are crucial for ensuring that interventions are evaluated effectively and with a high level of quality.

OBJECTIVE

We aimed to identify the types of outcomes, outcome measurement instruments, and assessment methods used to assess the clinical, user experience, and technical outcomes in studies that evaluated the effectiveness of CA interventions for mental health.

METHODS

We undertook a scoping review of the relevant literature to review the types of outcomes, outcome measurement instruments, and assessment methods in studies that evaluated the effectiveness of CA interventions for mental health. We performed a comprehensive search of electronic databases, including PubMed, Cochrane Central Register of Controlled Trials, Embase (Ovid), PsychINFO, and Web of Science, as well as Google Scholar and Google. We included experimental studies evaluating CA mental health interventions. The screening and data extraction were performed independently by 2 review authors in parallel. Descriptive and thematic analyses of the findings were performed.

RESULTS

We included 32 studies that targeted the promotion of mental well-being (17/32, 53%) and the treatment and monitoring of mental health symptoms (21/32, 66%). The studies reported 203 outcome measurement instruments used to measure clinical outcomes (123/203, 60.6%), user experience outcomes (75/203, 36.9%), technical outcomes (2/203, 1.0%), and other outcomes (3/203, 1.5%). Most of the outcome measurement instruments were used in only 1 study (150/203, 73.9%) and were self-reported questionnaires (170/203, 83.7%), and most were delivered electronically via survey platforms (61/203, 30.0%). No validity evidence was cited for more than half of the outcome measurement instruments (107/203, 52.7%), which were largely created or adapted for the study in which they were used (95/107, 88.8%).

CONCLUSIONS

The diversity of outcomes and the choice of outcome measurement instruments employed in studies on CAs for mental health point to the need for an established minimum core outcome set and greater use of validated instruments. Future studies should also capitalize on the affordances made available by CAs and smartphones to streamline the evaluation and reduce participants' input burden inherent to self-reporting.

Adherence and Engagement with a Cognitive Behavioral Therapy Based Conversational Agent (Wysa) in Adults with Chronic Pain: Survival Analysis

BACKGROUND

Digital applications are commonly used to support mental health and well-being. However, successfully retaining and engaging users to complete digital interventions is challenging, and comorbidities such as chronic pain further reduce user engagement. Digital conversational agents may improve user engagement by applying engagement principles that have been implemented within in-person care settings.

OBJECTIVE

To evaluate user retention and engagement with an artificial intelligence (AI)-led digital mental health application (app) that is customized for individuals managing mental health symptoms and coexisting chronic pain (Wysa for Chronic Pain).

METHODS

In this ancillary survival analysis of a clinical trial, participants included 51 adults who presented to a tertiary care center for chronic musculoskeletal pain, who endorsed coexisting symptoms of depression and/or anxiety (PROMIS Depression and/or Anxiety score ≥ 55), and initiated onboarding to an 8-week subscription of Wysa for Chronic Pain. The study outcomes were user retention, defined as revisiting the app each week and the last day of engagement, and user engagement, defined by the number of sessions the user completed.

RESULTS

Users engaged in a cumulative mean of 33.3 sessions during the eight-week study period. The survival analysis depicted a median user retention period (i.e., time to complete disengagement) of 51 days, with the usage of a morning check-in feature statistically significant in its relationship with a longer retention period (p = .001).

CONCLUSIONS

Our findings suggest that the user retention and engagement with a CBT-based conversational agent which is built for users with chronic pain is higher than standard industry metrics. These results have clear implications for addressing issues of suboptimal engagement of digital health interventions and improving access to care for chronic pain. Future work should use these findings to inform the design of evidence-based interventions for individuals with chronic pain and to enhance user retention and engagement of digital health interventions more broadly.

CLINICALTRIAL

NCT04640090, Clinicaltrials.gov.