1
|
Lee S, Yoon J, Cho Y, Chun J. A systematic review of chatbot-assisted interventions for substance use. Front Psychiatry 2024; 15:1456689. [PMID: 39319358 PMCID: PMC11420135 DOI: 10.3389/fpsyt.2024.1456689] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/29/2024] [Accepted: 08/19/2024] [Indexed: 09/26/2024] Open
Abstract
Objectives This study systematically reviewed research on the utilization of chatbot-related technologies for the prevention, assessment, and treatment of various substance uses, including alcohol, nicotine, and other drugs. Methods Following PRISMA guidelines, 28 articles were selected for final analysis from an initial screening of 998 references. Data were coded for multiple components, including study characteristics, intervention types, intervention contents, sample characteristics, substance use details, measurement tools, and main findings, particularly emphasizing the effectiveness of chatbot-assisted interventions on substance use and the facilitators and barriers affecting program effectiveness. Results Half of the studies specifically targeted smoking. Furthermore, over 85% of interventions were designed to treat substance use, with 7.14% focusing on prevention and 3.57% on assessment. Perceptions of effectiveness in quitting substance use varied, ranging from 25% to 50%, while for reduced substance use, percentages ranged from 66.67% to 83.33%. Among the studies assessing statistical effectiveness (46.43%), all experimental studies, including quasi-experiments, demonstrated significant and valid effects. Notably, 30% of studies emphasized personalization and providing relevant tips or information as key facilitators. Conclusion This study offers valuable insights into the development and validation of chatbot-assisted interventions, thereby establishing a robust foundation for their efficacy.
Collapse
Affiliation(s)
- Serim Lee
- Department of Social Welfare, Ewha Womans University, Seoul, Republic of Korea
- School of Public Health, University at Albany, State University of New York, Rensselaer, NY, United States
| | - Jiyoung Yoon
- Department of Social Welfare, Ewha Womans University, Seoul, Republic of Korea
| | - Yeonjee Cho
- Department of Social Welfare, Ewha Womans University, Seoul, Republic of Korea
| | - JongSerl Chun
- Department of Social Welfare, Ewha Womans University, Seoul, Republic of Korea
| |
Collapse
|
2
|
Suffoletto B. Deceptively Simple yet Profoundly Impactful: Text Messaging Interventions to Support Health. J Med Internet Res 2024; 26:e58726. [PMID: 39190427 PMCID: PMC11387917 DOI: 10.2196/58726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2024] [Revised: 05/30/2024] [Accepted: 07/15/2024] [Indexed: 08/28/2024] Open
Abstract
This paper examines the use of text message (SMS) interventions for health-related behavioral support. It first outlines the historical progress in SMS intervention research publications and the variety of funds from US government agencies. A narrative review follows, highlighting the effectiveness of SMS interventions in key health areas, such as physical activity, diet and weight loss, mental health, and substance use, based on published meta-analyses. It then outlines advantages of text messaging compared to other digital modalities, including the real-time capability to collect information and deliver microdoses of intervention support. Crucial design elements are proposed to optimize effectiveness and longitudinal engagement across communication strategies, psychological foundations, and behavior change tactics. We then discuss advanced functionalities, such as the potential for generative artificial intelligence to improve user interaction. Finally, major challenges to implementation are highlighted, including the absence of a dedicated commercial platform, privacy and security concerns with SMS technology, difficulties integrating SMS interventions with medical informatics systems, and concerns about user engagement. Proposed solutions aim to facilitate the broader application and effectiveness of SMS interventions. Our hope is that these insights can assist researchers and practitioners in using SMS interventions to improve health outcomes and reducing disparities.
Collapse
Affiliation(s)
- Brian Suffoletto
- Department of Emergency Medicine, Stanford University, Palo Alto, CA, United States
| |
Collapse
|
3
|
Villarreal-Zegarra D, Reategui-Rivera CM, García-Serna J, Quispe-Callo G, Lázaro-Cruz G, Centeno-Terrazas G, Galvez-Arevalo R, Escobar-Agreda S, Dominguez-Rodriguez A, Finkelstein J. Self-Administered Interventions Based on Natural Language Processing Models for Reducing Depressive and Anxiety Symptoms: Systematic Review and Meta-Analysis. JMIR Ment Health 2024; 11:e59560. [PMID: 39167795 PMCID: PMC11375382 DOI: 10.2196/59560] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Revised: 06/12/2024] [Accepted: 07/02/2024] [Indexed: 08/23/2024] Open
Abstract
BACKGROUND The introduction of natural language processing (NLP) technologies has significantly enhanced the potential of self-administered interventions for treating anxiety and depression by improving human-computer interactions. Although these advances, particularly in complex models such as generative artificial intelligence (AI), are highly promising, robust evidence validating the effectiveness of the interventions remains sparse. OBJECTIVE The aim of this study was to determine whether self-administered interventions based on NLP models can reduce depressive and anxiety symptoms. METHODS We conducted a systematic review and meta-analysis. We searched Web of Science, Scopus, MEDLINE, PsycINFO, IEEE Xplore, Embase, and Cochrane Library from inception to November 3, 2023. We included studies with participants of any age diagnosed with depression or anxiety through professional consultation or validated psychometric instruments. Interventions had to be self-administered and based on NLP models, with passive or active comparators. Outcomes measured included depressive and anxiety symptom scores. We included randomized controlled trials and quasi-experimental studies but excluded narrative, systematic, and scoping reviews. Data extraction was performed independently by pairs of authors using a predefined form. Meta-analysis was conducted using standardized mean differences (SMDs) and random effects models to account for heterogeneity. RESULTS In all, 21 articles were selected for review, of which 76% (16/21) were included in the meta-analysis for each outcome. Most of the studies (16/21, 76%) were recent (2020-2023), with interventions being mostly AI-based NLP models (11/21, 52%); most (19/21, 90%) delivered some form of therapy (primarily cognitive behavioral therapy: 16/19, 84%). The overall meta-analysis showed that self-administered interventions based on NLP models were significantly more effective in reducing both depressive (SMD 0.819, 95% CI 0.389-1.250; P<.001) and anxiety (SMD 0.272, 95% CI 0.116-0.428; P=.001) symptoms compared to various control conditions. Subgroup analysis indicated that AI-based NLP models were effective in reducing depressive symptoms (SMD 0.821, 95% CI 0.207-1.436; P<.001) compared to pooled control conditions. Rule-based NLP models showed effectiveness in reducing both depressive (SMD 0.854, 95% CI 0.172-1.537; P=.01) and anxiety (SMD 0.347, 95% CI 0.116-0.578; P=.003) symptoms. The meta-regression showed no significant association between participants' mean age and treatment outcomes (all P>.05). Although the findings were positive, the overall certainty of evidence was very low, mainly due to a high risk of bias, heterogeneity, and potential publication bias. CONCLUSIONS Our findings support the effectiveness of self-administered NLP-based interventions in alleviating depressive and anxiety symptoms, highlighting their potential to increase accessibility to, and reduce costs in, mental health care. Although the results were encouraging, the certainty of evidence was low, underscoring the need for further high-quality randomized controlled trials and studies examining implementation and usability. These interventions could become valuable components of public health strategies to address mental health issues. TRIAL REGISTRATION PROSPERO International Prospective Register of Systematic Reviews CRD42023472120; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42023472120.
Collapse
Affiliation(s)
- David Villarreal-Zegarra
- Instituto Peruano de Orientación Psicológica, Lima, Peru
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT, United States
| | - C Mahony Reategui-Rivera
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT, United States
| | | | | | | | | | | | | | | | - Joseph Finkelstein
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT, United States
| |
Collapse
|
4
|
Thornton JS, Hewitt C, Khan K, Speechley M, Ambrose A, Reilly K, Mountjoy ML, Gouttebarge V, Crossley K. Hang up your cleats and hope for the best? A cross-sectional study of five health domains in retired elite female rugby players. BMJ Open Sport Exerc Med 2024; 10:e001999. [PMID: 39286323 PMCID: PMC11404255 DOI: 10.1136/bmjsem-2024-001999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/18/2024] [Indexed: 09/19/2024] Open
Abstract
ABSTRACT Objectives To investigate retired elite female rugby players' health outcomes (and their relationships) in five key areas (musculoskeletal, cognitive, mental, reproductive/endocrinological and cardiovascular) and how those compare with the general population. Methods Female rugby players aged ≥18 years old and retired from elite competition ≥2 years were recruited via email or social media to complete a 179-item online questionnaire and neurocognitive assessment. Data from general population controls (matched for age and sex) were obtained where available. Results 159 participants responded (average age 43 (±5) years). 156 (98%) reported a hip/groin, knee, foot/ankle or lower back injury during their career, of which 104 (67%) reported ongoing pain. Participants reported worse hip and knee outcomes compared with the general population (p<0.0001). 146 (92%) reported sustaining one or more concussions. History of concussion was associated with lower-than-average scores on neurocognitive assessment. Compared with general population data, retired female rugby players reported less anxiety (OR=0.079 (95% CI 0.03 to 0.19)), depression (OR=0.67 (95% CI 0.57 to 0.78)) and distress (OR=0.17 (95% CI 0.15 to 0.19)). Amenorrhoea rates were higher compared with matched controls, and the age at menopause was younger. The prevalence of hypertension was higher. The rugby players perceived that their health decreased in retirement and cited a lack of physical activity as a main contributor. Conclusion Our findings point to the potential value of screening and monitoring, and identifying preventative measures during sporting careers to promote health and long-term quality of life for athletes.
Collapse
Affiliation(s)
- Jane S Thornton
- West Coast University-Ontario Campus, Ontario, California, USA
| | - Chloe Hewitt
- Epidemiology and Biostatistics, Western University, London, Ontario, Canada
| | - Karim Khan
- The University of British Columbia Faculty of Medicine, Vancouver, British Columbia, Canada
| | - Mark Speechley
- Departments of Epidemiology and Biostatistics, Western University, London, Ontario, Canada
| | - Ashley Ambrose
- Fowler Kennedy Sport Medicine Clinic, London, Ontario, Canada
| | | | | | - Vincent Gouttebarge
- Amsterdam UMC location University of Amsterdam, Department of Orthopedic Surgery and Sports Medicine, Meibergdreef 9, Amsterdam, Netherlands
- Amsterdam Collaboration for Health & Safety in Sports (ACHSS), IOC Research Center, Amsterdam, Netherlands
- Amsterdam Movement Sciences, Musculoskeletal Health, Sports, Amsterdam, Netherlands
- Section Sports Medicine, University of Pretoria, Pretoria, South Africa
| | - Kay Crossley
- La Trobe University, Melbourne, Victoria, Australia
| |
Collapse
|
5
|
Laymouna M, Ma Y, Lessard D, Schuster T, Engler K, Lebouché B. Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review. J Med Internet Res 2024; 26:e56930. [PMID: 39042446 PMCID: PMC11303905 DOI: 10.2196/56930] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 04/07/2024] [Accepted: 04/12/2024] [Indexed: 07/24/2024] Open
Abstract
BACKGROUND Chatbots, or conversational agents, have emerged as significant tools in health care, driven by advancements in artificial intelligence and digital technology. These programs are designed to simulate human conversations, addressing various health care needs. However, no comprehensive synthesis of health care chatbots' roles, users, benefits, and limitations is available to inform future research and application in the field. OBJECTIVE This review aims to describe health care chatbots' characteristics, focusing on their diverse roles in the health care pathway, user groups, benefits, and limitations. METHODS A rapid review of published literature from 2017 to 2023 was performed with a search strategy developed in collaboration with a health sciences librarian and implemented in the MEDLINE and Embase databases. Primary research studies reporting on chatbot roles or benefits in health care were included. Two reviewers dual-screened the search results. Extracted data on chatbot roles, users, benefits, and limitations were subjected to content analysis. RESULTS The review categorized chatbot roles into 2 themes: delivery of remote health services, including patient support, care management, education, skills building, and health behavior promotion, and provision of administrative assistance to health care providers. User groups spanned across patients with chronic conditions as well as patients with cancer; individuals focused on lifestyle improvements; and various demographic groups such as women, families, and older adults. Professionals and students in health care also emerged as significant users, alongside groups seeking mental health support, behavioral change, and educational enhancement. The benefits of health care chatbots were also classified into 2 themes: improvement of health care quality and efficiency and cost-effectiveness in health care delivery. The identified limitations encompassed ethical challenges, medicolegal and safety concerns, technical difficulties, user experience issues, and societal and economic impacts. CONCLUSIONS Health care chatbots offer a wide spectrum of applications, potentially impacting various aspects of health care. While they are promising tools for improving health care efficiency and quality, their integration into the health care system must be approached with consideration of their limitations to ensure optimal, safe, and equitable use.
Collapse
Affiliation(s)
- Moustafa Laymouna
- Department of Family Medicine, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
| | - Yuanchao Ma
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
- Department of Biomedical Engineering, Polytechnique Montréal, Montreal, QC, Canada
| | - David Lessard
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
| | - Tibor Schuster
- Department of Family Medicine, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
| | - Kim Engler
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
| | - Bertrand Lebouché
- Department of Family Medicine, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
| |
Collapse
|
6
|
Zhong W, Luo J, Zhang H. The therapeutic effectiveness of artificial intelligence-based chatbots in alleviation of depressive and anxiety symptoms in short-course treatments: A systematic review and meta-analysis. J Affect Disord 2024; 356:459-469. [PMID: 38631422 DOI: 10.1016/j.jad.2024.04.057] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/07/2023] [Revised: 04/10/2024] [Accepted: 04/14/2024] [Indexed: 04/19/2024]
Abstract
BACKGROUND The emergence of artificial intelligence-based chatbot has revolutionized the field of clinical psychology and psychotherapy, granting individuals unprecedented access to professional assistance, overcoming time constraints and geographical limitations with cost-effective convenience. However, despite its potential, there has been a noticeable gap in the literature regarding their effectiveness in addressing common mental health issues like depression and anxiety. This meta-analysis aims to evaluate the efficacy of AI-based chatbots in treating these conditions. METHODS A systematic search was executed across multiple databases, including PubMed, Cochrane Library, Web of Science, PsycINFO, and Embase on April 4th, 2024. The effect size of treatment efficacy was calculated using the standardized mean difference (Hedge's g). Quality assessment measures were implemented to ensure trial's quality. RESULTS In our analysis of 18 randomized controlled trials involving 3477 participants, we observed noteworthy improvements in depression (g = -0.26, 95 % CI = -0.34, -0.17) and anxiety (g = -0.19, 95 % CI = -0.29, -0.09) symptoms. The most significant benefits were evident after 8 weeks of treatment. However, at the three-month follow-up, no substantial effects were detected for either condition. LIMITATIONS Several limitations should be considered. These include the lack of diversity in the study populations, variations in chatbot design, and the use of different psychotherapeutic approaches. These factors may limit the generalizability of our findings. CONCLUSION This meta-analysis highlights the promising role of AI-based chatbot interventions in alleviating depressive and anxiety symptoms among adults. Our results indicate that these interventions can yield substantial improvements over a relatively brief treatment period.
Collapse
Affiliation(s)
- Wenjun Zhong
- Center for Studies of Education and Psychology of Ethnic Minorities in Southwest China, Southwest University, Chongqing, China
| | - Jianghua Luo
- Center for Studies of Education and Psychology of Ethnic Minorities in Southwest China, Southwest University, Chongqing, China.
| | - Hong Zhang
- Center for Psychological Health Education, Xinjiang University of Finance & Economics, Urumqi, Xinjiang, China
| |
Collapse
|
7
|
MacNeill AL, MacNeill L, Yi S, Goudreau A, Luke A, Doucet S. Depiction of conversational agents as health professionals: a scoping review. JBI Evid Synth 2024; 22:831-855. [PMID: 38482610 DOI: 10.11124/jbies-23-00029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/09/2024]
Abstract
OBJECTIVE The purpose of this scoping review was to examine the depiction of conversational agents as health professionals. We identified the professional characteristics that are used with these depictions and determined the prevalence of these characteristics among conversational agents that are used for health care. INTRODUCTION The depiction of conversational agents as health professionals has implications for both the users and the developers of these programs. For this reason, it is important to know more about these depictions and how they are implemented in practical settings. INCLUSION CRITERIA This review included scholarly literature on conversational agents that are used for health care. It focused on conversational agents designed for patients and health seekers, not health professionals or trainees. Conversational agents that address physical and/or mental health care were considered, as were programs that promote healthy behaviors. METHODS This review was conducted in accordance with JBI methodology for scoping reviews. The databases searched included MEDLINE (PubMed), Embase, CINAHL with Full Text (EBSCOhost), Scopus, Web of Science, ACM Guide to Computing Literature (Association for Computing Machinery Digital Library), and IEEE Xplore (IEEE). The main database search was conducted in June 2021, and an updated search was conducted in January 2022. Extracted data included characteristics of the report, basic characteristics of the conversational agent, and professional characteristics of the conversational agent. Extracted data were summarized using descriptive statistics. Results are presented in a narrative summary and accompanying tables. RESULTS A total of 38 health-related conversational agents were identified across 41 reports. Six of these conversational agents (15.8%) had professional characteristics. Four conversational agents (10.5%) had a professional appearance in which they displayed the clothing and accessories of health professionals and appeared in professional settings. One conversational agent (2.6%) had a professional title (Dr), and 4 conversational agents (10.5%) were described as having professional roles. Professional characteristics were more common among embodied vs disembodied conversational agents. CONCLUSIONS The results of this review show that the depiction of conversational agents as health professionals is not particularly common, although it does occur. More discussion is needed on the potential ethical and legal issues surrounding the depiction of conversational agents as health professionals. Future research should examine the impact of these depictions, as well as people's attitudes toward them, to better inform recommendations for practice.
Collapse
Affiliation(s)
- A Luke MacNeill
- Centre for Research in Integrated Care, University of New Brunswick, Saint John, NB, Canada
- Department of Nursing and Health Sciences, University of New Brunswick, Saint John, NB, Canada
| | - Lillian MacNeill
- Centre for Research in Integrated Care, University of New Brunswick, Saint John, NB, Canada
- Department of Nursing and Health Sciences, University of New Brunswick, Saint John, NB, Canada
| | - Sungmin Yi
- Centre for Research in Integrated Care, University of New Brunswick, Saint John, NB, Canada
- College of Pharmacy, Dalhousie University, Halifax, NS, Canada
| | - Alex Goudreau
- University of New Brunswick Libraries, Saint John, NB, Canada
- The University of New Brunswick (UNB) Saint John Collaboration for Evidence-Informed Healthcare: A JBI Centre of Excellence, Saint John, NB, Canada
| | - Alison Luke
- Centre for Research in Integrated Care, University of New Brunswick, Saint John, NB, Canada
- Department of Nursing and Health Sciences, University of New Brunswick, Saint John, NB, Canada
- The University of New Brunswick (UNB) Saint John Collaboration for Evidence-Informed Healthcare: A JBI Centre of Excellence, Saint John, NB, Canada
| | - Shelley Doucet
- Centre for Research in Integrated Care, University of New Brunswick, Saint John, NB, Canada
- Department of Nursing and Health Sciences, University of New Brunswick, Saint John, NB, Canada
- The University of New Brunswick (UNB) Saint John Collaboration for Evidence-Informed Healthcare: A JBI Centre of Excellence, Saint John, NB, Canada
| |
Collapse
|
8
|
Comulada WS, Rezai R, Sumstine S, Flores DD, Kerin T, Ocasio MA, Swendeman D, Fernández MI. A necessary conversation to develop chatbots for HIV studies: qualitative findings from research staff, community advisory board members, and study participants. AIDS Care 2024; 36:463-471. [PMID: 37253196 PMCID: PMC10687304 DOI: 10.1080/09540121.2023.2216926] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2022] [Accepted: 05/16/2023] [Indexed: 06/01/2023]
Abstract
Chatbots increase business productivity by handling customer conversations instead of human agents. Similar rationale applies to use chatbots in the healthcare sector, especially for health coaches who converse with clients. Chatbots are nascent in healthcare. Study findings have been mixed in terms of engagement and their impact on outcomes. Questions remain as to chatbot acceptability with coaches and other providers; studies have focused on clients.To clarify perceived benefits of chatbots in HIV interventions we conducted virtual focus groups with 13 research staff, eight community advisory board members, and seven young adults who were HIV intervention trial participants (clients). Our HIV healthcare context is important. Clients represent a promising age demographic for chatbot uptake. They are a marginalized population warranting consideration to avoid technology that limits healthcare access.Focus group participants expressed the value of chatbots for HIV research staff and clients. Staff discussed how chatbot functions, such as automated appointment scheduling and service referrals, could reduce workloads while clients discussed the after-hours convenience of these functions. Participants also emphasized that chatbots should provide relatable conversation, reliable functionality, and would not be appropriate for all clients. Our findings underscore the need to further examine appropriate chatbot functionality in HIV interventions.
Collapse
Affiliation(s)
- W. Scott Comulada
- Department of Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine, University of California, Los Angeles, CA
- Department of Health Policy and Management, Fielding School of Public Health, University of California, Los Angeles, CA
| | - Roxana Rezai
- Department of Epidemiology, Fielding School of Public Health, University of California, Los Angeles, CA
| | - Stephanie Sumstine
- Department of Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine, University of California, Los Angeles, CA
| | | | - Tara Kerin
- Department of Pediatrics, David Geffen School of Medicine, University of California, Los Angeles, CA
| | - Manuel A. Ocasio
- Department of Pediatrics, School of Medicine, Tulane University, New Orleans, LO
| | - Dallas Swendeman
- Department of Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine, University of California, Los Angeles, CA
| | | | | |
Collapse
|
9
|
Ding H, Simmich J, Vaezipour A, Andrews N, Russell T. Evaluation framework for conversational agents with artificial intelligence in health interventions: a systematic scoping review. J Am Med Inform Assoc 2024; 31:746-761. [PMID: 38070173 PMCID: PMC10873847 DOI: 10.1093/jamia/ocad222] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Revised: 11/04/2023] [Accepted: 11/24/2023] [Indexed: 02/18/2024] Open
Abstract
OBJECTIVES Conversational agents (CAs) with emerging artificial intelligence present new opportunities to assist in health interventions but are difficult to evaluate, deterring their applications in the real world. We aimed to synthesize existing evidence and knowledge and outline an evaluation framework for CA interventions. MATERIALS AND METHODS We conducted a systematic scoping review to investigate designs and outcome measures used in the studies that evaluated CAs for health interventions. We then nested the results into an overarching digital health framework proposed by the World Health Organization (WHO). RESULTS The review included 81 studies evaluating CAs in experimental (n = 59), observational (n = 15) trials, and other research designs (n = 7). Most studies (n = 72, 89%) were published in the past 5 years. The proposed CA-evaluation framework includes 4 evaluation stages: (1) feasibility/usability, (2) efficacy, (3) effectiveness, and (4) implementation, aligning with WHO's stepwise evaluation strategy. Across these stages, this article presents the essential evidence of different study designs (n = 8), sample sizes, and main evaluation categories (n = 7) with subcategories (n = 40). The main evaluation categories included (1) functionality, (2) safety and information quality, (3) user experience, (4) clinical and health outcomes, (5) costs and cost benefits, (6) usage, adherence, and uptake, and (7) user characteristics for implementation research. Furthermore, the framework highlighted the essential evaluation areas (potential primary outcomes) and gaps across the evaluation stages. DISCUSSION AND CONCLUSION This review presents a new framework with practical design details to support the evaluation of CA interventions in healthcare research. PROTOCOL REGISTRATION The Open Science Framework (https://osf.io/9hq2v) on March 22, 2021.
Collapse
Affiliation(s)
- Hang Ding
- RECOVER Injury Research Centre, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, QLD, Australia
- STARS Education and Research Alliance, Surgical Treatment and Rehabilitation Service (STARS), The University of Queensland and Metro North Health, Brisbane, QLD, Australia
| | - Joshua Simmich
- RECOVER Injury Research Centre, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, QLD, Australia
- STARS Education and Research Alliance, Surgical Treatment and Rehabilitation Service (STARS), The University of Queensland and Metro North Health, Brisbane, QLD, Australia
| | - Atiyeh Vaezipour
- RECOVER Injury Research Centre, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, QLD, Australia
- STARS Education and Research Alliance, Surgical Treatment and Rehabilitation Service (STARS), The University of Queensland and Metro North Health, Brisbane, QLD, Australia
| | - Nicole Andrews
- RECOVER Injury Research Centre, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, QLD, Australia
- STARS Education and Research Alliance, Surgical Treatment and Rehabilitation Service (STARS), The University of Queensland and Metro North Health, Brisbane, QLD, Australia
- The Tess Cramond Pain and Research Centre, Metro North Hospital and Health Service, Brisbane, QLD, Australia
- The Occupational Therapy Department, The Royal Brisbane and Women’s Hospital, Metro North Hospital and Health Service, Brisbane, QLD, Australia
| | - Trevor Russell
- RECOVER Injury Research Centre, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, QLD, Australia
- STARS Education and Research Alliance, Surgical Treatment and Rehabilitation Service (STARS), The University of Queensland and Metro North Health, Brisbane, QLD, Australia
| |
Collapse
|
10
|
Seo YC, Yong SY, Choi WW, Kim SH. Meta-Analysis of Studies on the Effects of Digital Therapeutics. J Pers Med 2024; 14:157. [PMID: 38392592 PMCID: PMC10890481 DOI: 10.3390/jpm14020157] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Revised: 01/25/2024] [Accepted: 01/29/2024] [Indexed: 02/24/2024] Open
Abstract
Digital therapeutics (DTx), novel treatment methods that have the potential to surpass traditional approaches such as pills, have received considerable research attention. Various efforts have been made to explore effective treatment methods that actively integrate DTx. This review investigates DTx treatment outcomes comprehensively through a meta-analysis. The analysis-a manual search of studies on "digital therapeutics"-includes DTx studies from January 2017 to October 2022. Hedges' g is used to quantify effect size for fifteen studies analyzed, encompassing eight control groups. Further, a quality assessment is performed using the Bias Risk Assessment Tool. The Hedges' g analysis results provide weighted average effect sizes across the eight control groups, revealing a substantial value of 0.91 (95% CI: 0.62 to 1.20); this signifies a moderate to large effect size. Further refinement, which excludes one study, yields an increased weighted average effect size of 1.13 (95% CI: 0.91 to 1.36). The quality assessment results consistently indicate a low risk of bias across studies. The meta-analysis results indicate that DTx can provide significant pivotal therapeutic impacts and offer a means to personalize treatment approaches and streamline the management of patients' treatment processes.
Collapse
Affiliation(s)
- Young-Chul Seo
- Wonju College of Medicine, Yonsei University, 20, Ilsan-ro, Wonju-si 26426, Gangwon-do, Republic of Korea
| | - Sang Yeol Yong
- Wonju College of Medicine, Yonsei University, 20, Ilsan-ro, Wonju-si 26426, Gangwon-do, Republic of Korea
| | - Won Woo Choi
- Department of Rehabilitation Medicine, Yonsei University, 20, Ilsan-ro, Wonju-si 26426, Kangwon-do, Republic of Korea
| | - Sung Hoon Kim
- Wonju College of Medicine, Yonsei University, 20, Ilsan-ro, Wonju-si 26426, Gangwon-do, Republic of Korea
| |
Collapse
|
11
|
Chiauzzi E, Williams A, Mariano TY, Pajarito S, Robinson A, Kirvin-Quamme A, Forman-Hoffman V. Demographic and clinical characteristics associated with anxiety and depressive symptom outcomes in users of a digital mental health intervention incorporating a relational agent. BMC Psychiatry 2024; 24:79. [PMID: 38291369 PMCID: PMC10826101 DOI: 10.1186/s12888-024-05532-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Accepted: 01/17/2024] [Indexed: 02/01/2024] Open
Abstract
BACKGROUND Digital mental health interventions (DMHIs) may reduce treatment access issues for those experiencing depressive and/or anxiety symptoms. DMHIs that incorporate relational agents may offer unique ways to engage and respond to users and to potentially help reduce provider burden. This study tested Woebot for Mood & Anxiety (W-MA-02), a DMHI that employs Woebot, a relational agent that incorporates elements of several evidence-based psychotherapies, among those with baseline clinical levels of depressive or anxiety symptoms. Changes in self-reported depressive and anxiety symptoms over 8 weeks were measured, along with the association between each of these outcomes and demographic and clinical characteristics. METHODS This exploratory, single-arm, 8-week study of 256 adults yielded non-mutually exclusive subsamples with either clinical levels of depressive or anxiety symptoms at baseline. Week 8 Patient Health Questionnaire-8 (PHQ-8) changes were measured in the depressive subsample (PHQ-8 ≥ 10). Week 8 Generalized Anxiety Disorder-7 (GAD-7) changes were measured in the anxiety subsample (GAD-7 ≥ 10). Demographic and clinical characteristics were examined in association with symptom changes via bivariate and multiple regression models adjusted for W-MA-02 utilization. Characteristics included age, sex at birth, race/ethnicity, marital status, education, sexual orientation, employment status, health insurance, baseline levels of depressive and anxiety symptoms, and concurrent psychotherapeutic or psychotropic medication treatments during the study. RESULTS Both the depressive and anxiety subsamples were predominantly female, educated, non-Hispanic white, and averaged 38 and 37 years of age, respectively. The depressive subsample had significant reductions in depressive symptoms at Week 8 (mean change =-7.28, SD = 5.91, Cohen's d = -1.23, p < 0.01); the anxiety subsample had significant reductions in anxiety symptoms at Week 8 (mean change = -7.45, SD = 5.99, Cohen's d = -1.24, p < 0.01). No significant associations were found between sex at birth, age, employment status, educational background and Week 8 symptom changes. Significant associations between depressive and anxiety symptom outcomes and sexual orientation, marital status, concurrent mental health treatment, and baseline symptom severity were found. CONCLUSIONS The present study suggests early promise for W-MA-02 as an intervention for depression and/or anxiety symptoms. Although exploratory in nature, this study revealed potential user characteristics associated with outcomes that can be investigated in future studies. TRIAL REGISTRATION This study was retrospectively registered on ClinicalTrials.gov (#NCT05672745) on January 5th, 2023.
Collapse
Affiliation(s)
- Emil Chiauzzi
- Woebot Health, 535 Mission Street, 14th Floor, San Francisco, CA, 94105, USA
| | - Andre Williams
- Woebot Health, 535 Mission Street, 14th Floor, San Francisco, CA, 94105, USA
| | - Timothy Y Mariano
- Woebot Health, 535 Mission Street, 14th Floor, San Francisco, CA, 94105, USA
- RR&D Center for Neurorestoration and Neurotechnology, VA Providence Healthcare System, Providence, RI, USA
- Department of Psychiatry and Human Behavior, Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Sarah Pajarito
- Woebot Health, 535 Mission Street, 14th Floor, San Francisco, CA, 94105, USA
| | - Athena Robinson
- Woebot Health, 535 Mission Street, 14th Floor, San Francisco, CA, 94105, USA
| | | | | |
Collapse
|
12
|
Islam A, Chaudhry BM, Islam A. RACares: a conceptual design to guide mHealth relational agent development based on a systematic review. Mhealth 2024; 10:11. [PMID: 38323144 PMCID: PMC10839507 DOI: 10.21037/mhealth-23-46] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 11/16/2023] [Indexed: 02/08/2024] Open
Abstract
Background A relational agent (RA) is a digital tool tailored to communicate with users, aiming to establish a sense of social ease and emotional bond, particularly focusing on their health and well-being concerns. A mobile health (mHealth) RA is particularly crafted to communicate with users within their mobile devices. As healthcare becomes increasingly digital, these mHealth RAs can serve as personal health assistants, e.g., guiding users through medical regimens, offering reminders for medication, providing emotional support during health crises, or even aiding in mental well-being exercises. Their accessibility, especially for those in remote areas, can bridge the gap between patients and immediate health assistance, revolutionizing the way healthcare is approached and delivered. Methods In this paper, our primary focus is introducing a conceptual design for mHealth RAs with the aim of enhanced user engagement, personalized health interventions, consistent support, data collection and monitoring, and enhanced multimodal accessibility. To develop this conceptual design, we employed an inductive approach. This involved conducting a qualitative analysis on data gathered from a systematic literature review of RAs. Consequently, this analysis allowed us to identify a taxonomy of key design features essential for RAs. Results This paper provides a conceptual design of mHealth RAs which includes five stages: user input receiving stage, input processing stage, data analysis stage, output processing stage, and output generation stage. A stage is a logical assembly of interconnected functionalities (components) that work together to accomplish a certain objective or set of goals. Each stage's outputs are used as inputs in the stages that follow after it. There is also a Data and Personalization Controller for aiding the data analysis stage. The stages are logically arranged one after another as follows: input, process, analysis, and output. Conclusions The conceptual design aims to create RAs for various mHealth applications, including patient education, mental health counseling, and chronic disease management. This design is crucial in digital health research as it enhances patient-RA interactions, potentially improving health outcomes and experiences in non-life-threatening scenarios where RAs can be an alternative to human healthcare professionals (HCPs).
Collapse
Affiliation(s)
- Ashraful Islam
- Center for Computational and Data Sciences, Independent University Bangladesh, Dhaka, Bangladesh
- Department of Computer Science and Engineering, Independent University Bangladesh, Dhaka, Bangladesh
| | - Beenish Moalla Chaudhry
- School of Computing and Informatics, University of Louisiana at Lafayette, Lafayette, LA, USA
| | - Aminul Islam
- School of Computing and Informatics, University of Louisiana at Lafayette, Lafayette, LA, USA
| |
Collapse
|
13
|
Li H, Zhang R, Lee YC, Kraut RE, Mohr DC. Systematic review and meta-analysis of AI-based conversational agents for promoting mental health and well-being. NPJ Digit Med 2023; 6:236. [PMID: 38114588 PMCID: PMC10730549 DOI: 10.1038/s41746-023-00979-5] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2023] [Accepted: 11/29/2023] [Indexed: 12/21/2023] Open
Abstract
Conversational artificial intelligence (AI), particularly AI-based conversational agents (CAs), is gaining traction in mental health care. Despite their growing usage, there is a scarcity of comprehensive evaluations of their impact on mental health and well-being. This systematic review and meta-analysis aims to fill this gap by synthesizing evidence on the effectiveness of AI-based CAs in improving mental health and factors influencing their effectiveness and user experience. Twelve databases were searched for experimental studies of AI-based CAs' effects on mental illnesses and psychological well-being published before May 26, 2023. Out of 7834 records, 35 eligible studies were identified for systematic review, out of which 15 randomized controlled trials were included for meta-analysis. The meta-analysis revealed that AI-based CAs significantly reduce symptoms of depression (Hedge's g 0.64 [95% CI 0.17-1.12]) and distress (Hedge's g 0.7 [95% CI 0.18-1.22]). These effects were more pronounced in CAs that are multimodal, generative AI-based, integrated with mobile/instant messaging apps, and targeting clinical/subclinical and elderly populations. However, CA-based interventions showed no significant improvement in overall psychological well-being (Hedge's g 0.32 [95% CI -0.13 to 0.78]). User experience with AI-based CAs was largely shaped by the quality of human-AI therapeutic relationships, content engagement, and effective communication. These findings underscore the potential of AI-based CAs in addressing mental health issues. Future research should investigate the underlying mechanisms of their effectiveness, assess long-term effects across various mental health outcomes, and evaluate the safe integration of large language models (LLMs) in mental health care.
Collapse
Affiliation(s)
- Han Li
- Department of Communications and New Media, National University of Singapore, Singapore, 117416, Singapore
| | - Renwen Zhang
- Department of Communications and New Media, National University of Singapore, Singapore, 117416, Singapore.
| | - Yi-Chieh Lee
- Department of Computer Science, National University of Singapore, Singapore, 117416, Singapore
| | - Robert E Kraut
- Human-Computer Interaction Institute Carnegie Mellon University, Pittsburgh, PA, 15213, USA
| | - David C Mohr
- Center for Behavioral Intervention Technologies, Department of Preventive Medicine, Northwestern University, Chicago, IL, 60611, USA
| |
Collapse
|
14
|
Cho YM, Rai S, Ungar L, Sedoc J, Guntuku SC. An Integrative Survey on Mental Health Conversational Agents to Bridge Computer Science and Medical Perspectives. PROCEEDINGS OF THE CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING. CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING 2023; 2023:11346-11369. [PMID: 38618627 PMCID: PMC11010238 DOI: 10.18653/v1/2023.emnlp-main.698] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/16/2024]
Abstract
Mental health conversational agents (a.k.a. chatbots) are widely studied for their potential to offer accessible support to those experiencing mental health challenges. Previous surveys on the topic primarily consider papers published in either computer science or medicine, leading to a divide in understanding and hindering the sharing of beneficial knowledge between both domains. To bridge this gap, we conduct a comprehensive literature review using the PRISMA framework, reviewing 534 papers published in both computer science and medicine. Our systematic review reveals 136 key papers on building mental health-related conversational agents with diverse characteristics of modeling and experimental design techniques. We find that computer science papers focus on LLM techniques and evaluating response quality using automated metrics with little attention to the application while medical papers use rule-based conversational agents and outcome metrics to measure the health outcomes of participants. Based on our findings on transparency, ethics, and cultural heterogeneity in this review, we provide a few recommendations to help bridge the disciplinary divide and enable the cross-disciplinary development of mental health conversational agents.
Collapse
|
15
|
Tate S, Fouladvand S, Chen JH, Chen CYA. The ChatGPT therapist will see you now: Navigating generative artificial intelligence's potential in addiction medicine research and patient care. Addiction 2023; 118:2249-2251. [PMID: 37735091 DOI: 10.1111/add.16341] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Accepted: 08/23/2023] [Indexed: 09/23/2023]
Affiliation(s)
- Steven Tate
- Department of Psychiatry and Behavioural Sciences, Stanford University School of Medicine, Palo Alto, California, USA
| | - Sajjad Fouladvand
- Department of Medicine, Stanford University School of Medicine, Palo Alto, California, USA
| | - Jonathan H Chen
- Department of Medicine, Stanford University School of Medicine, Palo Alto, California, USA
| | - Chwen-Yuen Angie Chen
- Department of Medicine, Stanford University School of Medicine, Palo Alto, California, USA
| |
Collapse
|
16
|
Wutz M, Hermes M, Winter V, Köberlein-Neu J. Factors Influencing the Acceptability, Acceptance, and Adoption of Conversational Agents in Health Care: Integrative Review. J Med Internet Res 2023; 25:e46548. [PMID: 37751279 PMCID: PMC10565637 DOI: 10.2196/46548] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 05/10/2023] [Accepted: 07/10/2023] [Indexed: 09/27/2023] Open
Abstract
BACKGROUND Conversational agents (CAs), also known as chatbots, are digital dialog systems that enable people to have a text-based, speech-based, or nonverbal conversation with a computer or another machine based on natural language via an interface. The use of CAs offers new opportunities and various benefits for health care. However, they are not yet ubiquitous in daily practice. Nevertheless, research regarding the implementation of CAs in health care has grown tremendously in recent years. OBJECTIVE This review aims to present a synthesis of the factors that facilitate or hinder the implementation of CAs from the perspectives of patients and health care professionals. Specifically, it focuses on the early implementation outcomes of acceptability, acceptance, and adoption as cornerstones of later implementation success. METHODS We performed an integrative review. To identify relevant literature, a broad literature search was conducted in June 2021 with no date limits and using all fields in PubMed, Cochrane Library, Web of Science, LIVIVO, and PsycINFO. To keep the review current, another search was conducted in March 2022. To identify as many eligible primary sources as possible, we used a snowballing approach by searching reference lists and conducted a hand search. Factors influencing the acceptability, acceptance, and adoption of CAs in health care were coded through parallel deductive and inductive approaches, which were informed by current technology acceptance and adoption models. Finally, the factors were synthesized in a thematic map. RESULTS Overall, 76 studies were included in this review. We identified influencing factors related to 4 core Unified Theory of Acceptance and Use of Technology (UTAUT) and Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) factors (performance expectancy, effort expectancy, facilitating conditions, and hedonic motivation), with most studies underlining the relevance of performance and effort expectancy. To meet the particularities of the health care context, we redefined the UTAUT2 factors social influence, habit, and price value. We identified 6 other influencing factors: perceived risk, trust, anthropomorphism, health issue, working alliance, and user characteristics. Overall, we identified 10 factors influencing acceptability, acceptance, and adoption among health care professionals (performance expectancy, effort expectancy, facilitating conditions, social influence, price value, perceived risk, trust, anthropomorphism, working alliance, and user characteristics) and 13 factors influencing acceptability, acceptance, and adoption among patients (additionally hedonic motivation, habit, and health issue). CONCLUSIONS This review shows manifold factors influencing the acceptability, acceptance, and adoption of CAs in health care. Knowledge of these factors is fundamental for implementation planning. Therefore, the findings of this review can serve as a basis for future studies to develop appropriate implementation strategies. Furthermore, this review provides an empirical test of current technology acceptance and adoption models and identifies areas where additional research is necessary. TRIAL REGISTRATION PROSPERO CRD42022343690; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=343690.
Collapse
Affiliation(s)
- Maximilian Wutz
- Center for Health Economics and Health Services Research, Schumpeter School of Business and Economics, University of Wuppertal, Wuppertal, Germany
| | - Marius Hermes
- Center for Health Economics and Health Services Research, Schumpeter School of Business and Economics, University of Wuppertal, Wuppertal, Germany
| | - Vera Winter
- Center for Health Economics and Health Services Research, Schumpeter School of Business and Economics, University of Wuppertal, Wuppertal, Germany
| | - Juliane Köberlein-Neu
- Center for Health Economics and Health Services Research, Schumpeter School of Business and Economics, University of Wuppertal, Wuppertal, Germany
| |
Collapse
|
17
|
Durden E, Pirner MC, Rapoport SJ, Williams A, Robinson A, Forman-Hoffman VL. Changes in stress, burnout, and resilience associated with an 8-week intervention with relational agent "Woebot". Internet Interv 2023; 33:100637. [PMID: 37635948 PMCID: PMC10457544 DOI: 10.1016/j.invent.2023.100637] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Revised: 06/08/2023] [Accepted: 06/12/2023] [Indexed: 08/29/2023] Open
Abstract
Background Research investigating the potential for digital mental health interventions with integrated relational agents to improve mental health outcomes is in its infancy. By delivering evidence-based mental health interventions through tailored, empathic conversations, relational agents have the potential to help individuals manage their stress and mood, and increase positive mental health. Aims The aims of this study were twofold: 1) to assess whether a smartphone app delivering mental health support through a relational agent, Woebot, is associated with changes in stress, burnout, and resilience over 8 weeks, and 2) to identify demographic and clinical factors associated with changes in these outcomes. Method This exploratory, non-randomized, single-armed, open-labeled trial was conducted from May to July 2022. A total of 256 adults (mean age 39 ± 13.35; 72 % females) recruited through social media advertising enrolled in the study. Participants completed an 8-week intervention period during which they were invited to use a smartphone app called Woebot-LIFE that delivers cognitive behavioral therapy through a relational agent called "Woebot". Participant-reported measures of stress, burnout, and resilience were collected at Baseline, and Week 8. Changes in these outcomes during the study period were assessed. Bivariate and stepwise multiple regression modeling was used to identify sociodemographic and clinical factors associated with observed changes over the 8-week study period. Results Exposure to Woebot-LIFE was associated with significant reductions in perceived stress and burnout and significantly increased resilience over the 8-week study period. A greater reduction in stress was observed among those with clinically elevated mood symptoms (i.e., Patient Health Questionnaire-8 or Generalized Anxiety Disorder 7-item scores ≥10) at baseline compared to those without; however, the differences in the improvements in resilience scores and burnout between the two groups were not statistically significant. Although a difference in the magnitude of change in stress was observed for participants with and without clinically elevated mood symptoms at baseline, significant improvements in stress, burnout, and resilience over the 8-week study period were observed for both groups. Bivariate analyses showed that race, insurance type, and baseline level of resilience were associated with changes in each of the outcomes, though baseline resilience was the only factor that remained significantly associated with changes in the outcomes in the stepwise multiple regression analyses. Conclusion Results of this single-arm, exploratory study suggest that conversational agent-guided mental health interventions such as Woebot-LIFE may be associated with reduced stress and burnout and increased resilience in both clinical and non-clinical populations.
Collapse
Affiliation(s)
- Emily Durden
- Woebot Health, 535 Mission Street, 14th Floor, San Francisco, CA 94105, United States of America
| | - Maddison C. Pirner
- Woebot Health, 535 Mission Street, 14th Floor, San Francisco, CA 94105, United States of America
| | - Stephanie J. Rapoport
- Woebot Health, 535 Mission Street, 14th Floor, San Francisco, CA 94105, United States of America
| | - Andre Williams
- Woebot Health, 535 Mission Street, 14th Floor, San Francisco, CA 94105, United States of America
| | - Athena Robinson
- Woebot Health, 535 Mission Street, 14th Floor, San Francisco, CA 94105, United States of America
| | | |
Collapse
|
18
|
Tong F, Lederman R, D'Alfonso S, Berry K, Bucci S. Conceptualizing the digital therapeutic alliance in the context of fully automated mental health apps: A thematic analysis. Clin Psychol Psychother 2023; 30:998-1012. [PMID: 37042076 DOI: 10.1002/cpp.2851] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 03/24/2023] [Accepted: 03/27/2023] [Indexed: 04/13/2023]
Abstract
Fully automated mental health apps provide a promising opportunity for increasing access to mental health care and resources. Given this opportunity, continued research into the utility and effectiveness of mental health apps is crucial. Therapeutic alliance (TA) refers to the relationship between a client and a healthcare professional, and has been shown to be an important predictor of clinical outcomes in face-to-face therapy. Given the significance of TA in traditional therapy, it is important to explore whether the notion of a digital therapeutic alliance (DTA) in the context of fully automated mental health apps also plays an important role in clinical outcomes. Current evidence shows that the conceptualization of DTA in the context of fully automated mental health apps can be potentially different to TA in face-to-face therapy. Thus, a new DTA conceptual model is necessary for comprehensively understanding the mechanisms underpinning DTA for fully automated mental health apps. To the best of our knowledge, this is the first study that qualitatively explored the dimensions of a DTA in the context of fully automated mental health apps. We conducted interviews with 20 users of mental health apps to explore the key dimensions comprising DTA in the context of fully automated mental health apps. We found that although conceptualizations of DTA and TA have shared dimensions, flexibility and emotional experiences are unique domains in DTA. On the other hand, although agreement on goals between a therapist and a client is important in face to face therapy, we found that users can have an alliance with an app without a goal. The importance of goal needs further investigations.
Collapse
Affiliation(s)
- Fangziyun Tong
- School of Computing and Information Systems, University of Melbourne, Parkville, Victoria, USA
- Division of Psychology and Mental Health, School of Health Sciences, Manchester Academic Health Sciences Centre, University of Manchester, Manchester, UK
| | - Reeva Lederman
- School of Computing and Information Systems, University of Melbourne, Parkville, Victoria, USA
| | - Simon D'Alfonso
- School of Computing and Information Systems, University of Melbourne, Parkville, Victoria, USA
| | - Katherine Berry
- Division of Psychology and Mental Health, School of Health Sciences, Manchester Academic Health Sciences Centre, University of Manchester, Manchester, UK
- Complex Trauma and Resilience Research Unit, Greater Manchester Mental Health NHS Foundation Trust, Manchester, UK
| | - Sandra Bucci
- Division of Psychology and Mental Health, School of Health Sciences, Manchester Academic Health Sciences Centre, University of Manchester, Manchester, UK
- Complex Trauma and Resilience Research Unit, Greater Manchester Mental Health NHS Foundation Trust, Manchester, UK
| |
Collapse
|
19
|
Suharwardy S, Ramachandran M, Leonard SA, Gunaseelan A, Lyell DJ, Darcy A, Robinson A, Judy A. Feasibility and impact of a mental health chatbot on postpartum mental health: a randomized controlled trial. AJOG GLOBAL REPORTS 2023; 3:100165. [PMID: 37560011 PMCID: PMC10407813 DOI: 10.1016/j.xagr.2023.100165] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/31/2023] Open
Abstract
BACKGROUND Perinatal mood disorders are common yet underdiagnosed and un- or undertreated. Barriers exist to accessing perinatal mental health services, including limited availability, time, and cost. Automated conversational agents (chatbots) can deliver evidence-based cognitive behavioral therapy content through text message-based conversations and reduce depression and anxiety symptoms in select populations. Such digital mental health technologies are poised to overcome barriers to mental health care access but need to be evaluated for efficacy, as well as for preliminary feasibility and acceptability among perinatal populations. OBJECTIVE To evaluate the acceptability and preliminary efficacy of a mental health chatbot for mood management in a general postpartum population. STUDY DESIGN An unblinded randomized controlled trial was conducted at a tertiary academic center. English-speaking postpartum women aged 18 years or above with a live birth and access to a smartphone were eligible for enrollment prior to discharge from delivery hospitalization. Baseline surveys were administered to all participants prior to randomization to a mental health chatbot intervention or to usual care only. The intervention group downloaded the mental health chatbot smartphone application with perinatal-specific content, in addition to continuing usual care. Usual care consisted of routine postpartum follow up and mental health care as dictated by the patient's obstetric provider. Surveys were administered during delivery hospitalization (baseline) and at 2-, 4-, and 6-weeks postpartum to assess depression and anxiety symptoms. The primary outcome was a change in depression symptoms at 6-weeks as measured using two depression screening tools: Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale. Secondary outcomes included anxiety symptoms measured using Generalized Anxiety Disorder-7, and satisfaction and acceptability using validated scales. Based on a prior study, we estimated a sample size of 130 would have sufficient (80%) power to detect a moderate effect size (d=.4) in between group difference on the Patient Health Questionnaire-9. RESULTS A total of 192 women were randomized equally 1:1 to the chatbot or usual care; of these, 152 women completed the 6-week survey (n=68 chatbot, n=84 usual care) and were included in the final analysis. Mean baseline mental health assessment scores were below positive screening thresholds. At 6-weeks, there was a greater decrease in Patient Health Questionnaire-9 scores among the chatbot group compared to the usual care group (mean decrease=1.32, standard deviation=3.4 vs mean decrease=0.13, standard deviation=3.01, respectively). 6-week mean Edinburgh Postnatal Depression Scale and Generalized Anxiety Disorder-7 scores did not differ between groups and were similar to baseline. 91% (n=62) of the chatbot users were satisfied or highly satisfied with the chatbot, and 74% (n=50) of the intervention group reported use of the chatbot at least once in 2 weeks prior to the 6-week survey. 80% of study participants reported being comfortable with the use of a mobile smartphone application for mood management. CONCLUSION Use of a chatbot was acceptable to women in the early postpartum period. The sample did not screen positive for depression at baseline and thus the potential of the chatbot to reduce depressive symptoms in this population was limited. This study was conducted in a general obstetric population. Future studies of longer duration in high-risk postpartum populations who screen positive for depression are needed to further understand the utility and efficacy of such digital therapeutics for that population.
Collapse
Affiliation(s)
- Sanaa Suharwardy
- Department of Obstetrics and Gynecology, Division of Maternal Fetal Medicine and Obstetrics, Stanford University, Stanford, CA (Dr. Suharwardy, Dr. Ramachandran, Dr. Leonard, Dr Gunaseelan, Dr Lyell, and Dr Judy)
| | - Maya Ramachandran
- Department of Obstetrics and Gynecology, Division of Maternal Fetal Medicine and Obstetrics, Stanford University, Stanford, CA (Dr. Suharwardy, Dr. Ramachandran, Dr. Leonard, Dr Gunaseelan, Dr Lyell, and Dr Judy)
| | - Stephanie A. Leonard
- Department of Obstetrics and Gynecology, Division of Maternal Fetal Medicine and Obstetrics, Stanford University, Stanford, CA (Dr. Suharwardy, Dr. Ramachandran, Dr. Leonard, Dr Gunaseelan, Dr Lyell, and Dr Judy)
| | - Anita Gunaseelan
- Department of Obstetrics and Gynecology, Division of Maternal Fetal Medicine and Obstetrics, Stanford University, Stanford, CA (Dr. Suharwardy, Dr. Ramachandran, Dr. Leonard, Dr Gunaseelan, Dr Lyell, and Dr Judy)
| | - Deirdre J. Lyell
- Department of Obstetrics and Gynecology, Division of Maternal Fetal Medicine and Obstetrics, Stanford University, Stanford, CA (Dr. Suharwardy, Dr. Ramachandran, Dr. Leonard, Dr Gunaseelan, Dr Lyell, and Dr Judy)
| | - Alison Darcy
- Woebot Health, San Francisco, CA (Drs Darcy and Robinson)
| | | | - Amy Judy
- Department of Obstetrics and Gynecology, Division of Maternal Fetal Medicine and Obstetrics, Stanford University, Stanford, CA (Dr. Suharwardy, Dr. Ramachandran, Dr. Leonard, Dr Gunaseelan, Dr Lyell, and Dr Judy)
| |
Collapse
|
20
|
Chun-Hung L, Guan-Hsiung L, Wu-Chuan Y, Yu-Hsin L. Chatbot-assisted therapy for patients with methamphetamine use disorder: a preliminary randomized controlled trial. Front Psychiatry 2023; 14:1159399. [PMID: 37484677 PMCID: PMC10359989 DOI: 10.3389/fpsyt.2023.1159399] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/05/2023] [Accepted: 06/16/2023] [Indexed: 07/25/2023] Open
Abstract
Background Methamphetamine (MA) use disorder is associated with a large public health burden. Despite the therapeutic effects of psychosocial interventions based on current evidence, finding an approach to retain patients in treatment remains a real-world challenge. The rapid development of mobile health (mHealth) systems suggests the potential to provide real-time personalized care at any time and from any location, minimize barriers to treatment, maximize use, and promote the dissemination of accessible therapeutic tools in at-risk populations. Our study aimed to investigate the feasibility and effectiveness of chatbots for the treatment of MA use disorder. Method The inclusion criteria were (a) a diagnosis of MA use disorder as defined by the DSM-5, (b) age between 18 and 65 years, (c) no acute exacerbation of severe mental illness during the initial assessment, such as schizophrenia or bipolar I disorder, (d) willingness to participate in standard outpatient treatment for ≥ 6 months, and (e) an Android phone. Participants were randomly allocated to either a chatbot-assisted therapy via smartphone (CAT) group or a control group following simple randomization procedures (computerized random numbers) without blinding. All participants were followed up for 6 months. Treatment retention and monthly urine test results were analyzed as outcome measures. Participants' satisfaction with CAT was also assessed. Results In total, 50 and 49 participants were allocated to the CAT and control groups, respectively. There were no significant differences in retention time between the two treatment groups (df = 1, p = 0.099). The CAT group had fewer MA-positive urine samples than the control group (19.5% vs. 29.6%, F = 9.116, p = 0.003). The proportion of MA-positive urine samples was positively correlated with the frequency of MA use (r = 0.323, p = 0.001), severity of MA use disorder (r = 0.364, p < 0.001), and polysubstance use (r = 0.212, p = 0.035), and negatively correlated with readiness to change (r = -0.330, p = 0.001). Totally 55 participants completed the study at the 6-month follow-up and 60% reported relative satisfaction. Conclusion Participants in this study had favorable acceptance and generally positive outcomes, which indicates that chatbot is feasible for treating people who use MA.
Collapse
Affiliation(s)
- Lee Chun-Hung
- Department of Information Engineering, I-Shou University, Kaohsiung, Taiwan
- Jianan Psychiatric Center, Ministry of Health and Welfare (MOHW), Tainan, Taiwan
| | - Liaw Guan-Hsiung
- Department of Information Engineering, I-Shou University, Kaohsiung, Taiwan
| | - Yang Wu-Chuan
- Department of Information Engineering, I-Shou University, Kaohsiung, Taiwan
| | - Liu Yu-Hsin
- King's College London, Florence Nightingale School of Nursing & Midwifery, London, United Kingdom
| |
Collapse
|
21
|
Darcy A, Beaudette A, Chiauzzi E, Daniels J, Goodwin K, Mariano TY, Wicks P, Robinson A. Anatomy of a Woebot® (WB001): agent guided CBT for women with postpartum depression. Expert Rev Med Devices 2023; 20:1035-1049. [PMID: 37938145 DOI: 10.1080/17434440.2023.2280686] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Accepted: 11/03/2023] [Indexed: 11/09/2023]
Abstract
INTRODUCTION Postpartum depression (PPD) is common, persistent, and stigmatized. There are insufficient trained professionals to deliver appropriate screening, diagnosis, and treatment. AREAS COVERED WB001 is a Software as a Medical Device (SaMD) based Agent-Guided Cognitive-Behavioral Therapy (AGCBT) program for the treatment of PPD, for which Breakthrough Device Designation was recently granted by the US Food and Drug Administration. WB001 combines therapeutic alliance, human-centered design, machine learning techniques, and established principles from CBT and interpersonal therapy (IPT). We introduce AGCBT as a new model of service delivery, whilst describing Woebot, the agent technology that enables guidance through the replication of some elements of human relationships. The profile describes the device's design principles, enabling technology, risk handling, and efficacy data in PPD. EXPERT OPINION WB001 is a dynamic and personalized tool with which patients may establish a therapeutic bond. Woebot is designed to augment (rather than replace) human healthcare providers, unlocking the therapeutic potency associated with guidance, whilst retaining the scalability and agency that characterizes self-help approaches. WB001 has the potential to improve both the quality and the scalability of care through providing support to patients on waiting lists, in between clinical encounters, and enabling automation of measurement-based-care.
Collapse
Affiliation(s)
| | | | | | | | | | - Timothy Y Mariano
- Woebot Health, San Francisco, CA, USA
- RR&D Center for Neurorestoration and Neurotechnology, VA Providence Healthcare System, Providence, RI, USA
- Department of Psychiatry and Human Behavior, Warren Alpert Medical School of Brown University, Providence, RI, USA
| | | | | |
Collapse
|
22
|
Grodniewicz JP, Hohol M. Waiting for a digital therapist: three challenges on the path to psychotherapy delivered by artificial intelligence. Front Psychiatry 2023; 14:1190084. [PMID: 37324824 PMCID: PMC10267322 DOI: 10.3389/fpsyt.2023.1190084] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Accepted: 05/15/2023] [Indexed: 06/17/2023] Open
Abstract
Growing demand for broadly accessible mental health care, together with the rapid development of new technologies, trigger discussions about the feasibility of psychotherapeutic interventions based on interactions with Conversational Artificial Intelligence (CAI). Many authors argue that while currently available CAI can be a useful supplement for human-delivered psychotherapy, it is not yet capable of delivering fully fledged psychotherapy on its own. The goal of this paper is to investigate what are the most important obstacles on our way to developing CAI systems capable of delivering psychotherapy in the future. To this end, we formulate and discuss three challenges central to this quest. Firstly, we might not be able to develop effective AI-based psychotherapy unless we deepen our understanding of what makes human-delivered psychotherapy effective. Secondly, assuming that it requires building a therapeutic relationship, it is not clear whether psychotherapy can be delivered by non-human agents. Thirdly, conducting psychotherapy might be a problem too complicated for narrow AI, i.e., AI proficient in dealing with only relatively simple and well-delineated tasks. If this is the case, we should not expect CAI to be capable of delivering fully-fledged psychotherapy until the so-called "general" or "human-like" AI is developed. While we believe that all these challenges can ultimately be overcome, we think that being mindful of them is crucial to ensure well-balanced and steady progress on our path to AI-based psychotherapy.
Collapse
|
23
|
He Y, Yang L, Qian C, Li T, Su Z, Zhang Q, Hou X. Conversational Agent Interventions for Mental Health Problems: Systematic Review and Meta-analysis of Randomized Controlled Trials. J Med Internet Res 2023; 25:e43862. [PMID: 37115595 PMCID: PMC10182468 DOI: 10.2196/43862] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Revised: 02/17/2023] [Accepted: 03/10/2023] [Indexed: 03/12/2023] Open
Abstract
BACKGROUND Mental health problems are a crucial global public health concern. Owing to their cost-effectiveness and accessibility, conversational agent interventions (CAIs) are promising in the field of mental health care. OBJECTIVE This study aims to present a thorough summary of the traits of CAIs available for a range of mental health problems, find evidence of efficacy, and analyze the statistically significant moderators of efficacy via a meta-analysis of randomized controlled trial. METHODS Web-based databases (Embase, MEDLINE, PsycINFO, CINAHL, Web of Science, and Cochrane) were systematically searched dated from the establishment of the database to October 30, 2021, and updated to May 1, 2022. Randomized controlled trials comparing CAIs with any other type of control condition in improving depressive symptoms, generalized anxiety symptoms, specific anxiety symptoms, quality of life or well-being, general distress, stress, mental disorder symptoms, psychosomatic disease symptoms, and positive and negative affect were considered eligible. This study followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Data were extracted by 2 independent reviewers, checked by a third reviewer, and pooled using both random effect models and fixed effects models. Hedges g was chosen as the effect size. RESULTS Of the 6900 identified records, a total of 32 studies were included, involving 6089 participants. CAIs showed statistically significant short-term effects compared with control conditions in improving depressive symptoms (g=0.29, 95% CI 0.20-0.38), generalized anxiety symptoms (g=0.29, 95% CI 0.21-0.36), specific anxiety symptoms (g=0.47, 95% CI 0.07-0.86), quality of life or well-being (g=0.27, 95% CI 0.16-0.39), general distress (g=0.33, 95% CI 0.20-0.45), stress (g=0.24, 95% CI 0.08-0.41), mental disorder symptoms (g=0.36, 95% CI 0.17-0.54), psychosomatic disease symptoms (g=0.62, 95% CI 0.14-1.11), and negative affect (g=0.28, 95% CI 0.05-0.51). However, the long-term effects of CAIs for the most mental health outcomes were not statistically significant (g=-0.04 to 0.39). Personalization and empathic response were 2 critical facilitators of efficacy. The longer duration of interaction with conversational agents was associated with the larger pooled effect sizes. CONCLUSIONS The findings show that CAIs are research-proven interventions that ought to be implemented more widely in mental health care. CAIs are effective and easily acceptable for those with mental health problems. The clinical application of this novel digital technology will conserve human health resources and optimize the allocation of mental health services. TRIAL REGISTRATION PROSPERO CRD42022350130; https://tinyurl.com/mvhk6w9p.
Collapse
Affiliation(s)
- Yuhao He
- Institute of Applied Psychology, College of Education, Tianjin University, Tianjin, China
- Laboratory of Suicidology, Tianjin Municipal Education Commission, Tianjin, China
| | - Li Yang
- Institute of Applied Psychology, College of Education, Tianjin University, Tianjin, China
- Laboratory of Suicidology, Tianjin Municipal Education Commission, Tianjin, China
| | - Chunlian Qian
- Institute of Applied Psychology, College of Education, Tianjin University, Tianjin, China
- Laboratory of Suicidology, Tianjin Municipal Education Commission, Tianjin, China
| | - Tong Li
- Institute of Applied Psychology, College of Education, Tianjin University, Tianjin, China
- Laboratory of Suicidology, Tianjin Municipal Education Commission, Tianjin, China
| | - Zhengyuan Su
- Institute of Applied Psychology, College of Education, Tianjin University, Tianjin, China
- Laboratory of Suicidology, Tianjin Municipal Education Commission, Tianjin, China
| | - Qiang Zhang
- Shenzhen School, Sun Yat-sen University, Shenzhen, China
| | - Xiangqing Hou
- Institute of Applied Psychology, College of Education, Tianjin University, Tianjin, China
- Laboratory of Suicidology, Tianjin Municipal Education Commission, Tianjin, China
| |
Collapse
|
24
|
Prochaska JJ, Vogel EA, Chieng A, Baiocchi M, Pajarito S, Pirner M, Darcy A, Robinson A. A relational agent for treating substance use in adults: Protocol for a randomized controlled trial with a psychoeducational comparator. Contemp Clin Trials 2023; 127:107125. [PMID: 36813084 PMCID: PMC10065942 DOI: 10.1016/j.cct.2023.107125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 02/16/2023] [Accepted: 02/17/2023] [Indexed: 02/22/2023]
Abstract
BACKGROUND Substance use disorders (SUDs) are prevalent and compromise health and wellbeing. Scalable solutions, such as digital therapeutics, may offer a population-based strategy for addressing SUDs. Two formative studies supported the feasibility and acceptability of the relational agent Woebot, an animated screen-based social robot, for treating SUDs (W-SUDs) in adults. Participants randomized to W-SUDs reduced their substance use occasions from baseline to end-of-treatment (EOT) relative to a waitlist control. OBJECTIVE To further develop the evidence base, the current randomized trial extends follow-up to 1-month post-treatment and will test the efficacy of W-SUDs relative to a psychoeducational control. METHODS This study will recruit, screen, and consent 400 adults online reporting problematic substance use. Following baseline assessment, participants will be randomized to 8 weeks of W-SUDs or a psychoeducational control. Assessments will be conducted at weeks 4, 8 (EOT), and 12 (1-month post-treatment). Primary outcome is past-month number of substance use occasions, summed across all substances. Secondary outcomes are number of heavy drinking days, the percent of days abstinent from all substances, substance use problems, thoughts about abstinence, cravings, confidence to resist substance use, symptoms of depression and anxiety, and work productivity. If significant group differences are found, we will explore moderators and mediators of treatment effects. CONCLUSIONS The current study builds upon emerging evidence of a digital therapeutic for reducing problematic substance use by examining sustained effects and testing against a psychoeducational control condition. If efficacious, the findings have implications for scalable mobile health interventions for reducing problematic substance use. TRIAL REGISTRATION NCT04925570.
Collapse
Affiliation(s)
- Judith J Prochaska
- Stanford Prevention Research Center, Department of Medicine, School of Medicine, Stanford University, United States of America.
| | - Erin A Vogel
- Stanford Prevention Research Center, Department of Medicine, School of Medicine, Stanford University, United States of America
| | - Amy Chieng
- Stanford Prevention Research Center, Department of Medicine, School of Medicine, Stanford University, United States of America
| | - Michael Baiocchi
- Department of Epidemiology & Population Health, School of Medicine, Stanford University, United States of America
| | | | | | | | | |
Collapse
|
25
|
Karnik NS, Kuhns LM, Hotton AL, Del Vecchio N, McNulty M, Schneider J, Donenberg G, Keglovitz Baker K, Diskin R, Muldoon A, Rivera J, Summersett Williams F, Garofalo R. Findings From the Step Up, Test Up Study of an Electronic Screening and Brief Intervention for Alcohol Misuse in Adolescents and Young Adults Presenting for HIV Testing: Randomized Controlled Efficacy Trial. JMIR Ment Health 2023; 10:e43653. [PMID: 36989027 PMCID: PMC10131684 DOI: 10.2196/43653] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Revised: 02/16/2023] [Accepted: 03/02/2023] [Indexed: 03/30/2023] Open
Abstract
BACKGROUND Substance use, particularly binge drinking of alcohol and noninjection substance use, is associated with increased risk for HIV infection among youth, but structured substance use screening and brief intervention are not often provided as part of HIV risk reduction. OBJECTIVE The purpose of the study was to test the efficacy of a fully automated electronic screening and brief intervention, called Step Up, Test Up, to reduce alcohol misuse among adolescents and young adults presenting for HIV testing. Secondary objectives were reduction in sexual risk and uptake of pre-exposure prophylaxis (PrEP) for HIV prevention. METHODS Youth aged 16 years to 25 years who presented for HIV testing at community-based locations were recruited for study participation. Those who screened at moderate to high risk on the Alcohol Use Disorders Identification Test were randomized (1:1) to either an electronic brief intervention or a time-attention control. The primary outcome was change in alcohol use at 1, 3, 6, and 12-month follow-ups. Negative binomial and log binomial regression analyses with generalized estimating equations were conducted to evaluate the intervention efficacy. RESULTS Among a sample of 329 youth, there were no significant differences in alcohol use outcomes between conditions over time or at the 1, 3, 6, or 12-month time points. In terms of secondary outcomes, there was evidence of reduction in condomless insertive anal sex under the influence of alcohol and drugs at 12 months compared with 3 months in the intervention versus the attention control condition (incidence rate ratio=0.15, 95% CI 0.05-0.44); however, there were no other significant differences in sexual risk and no difference in PrEP engagement. CONCLUSIONS We found no effect of electronic brief intervention to reduce alcohol use and some effect on sexual risk among youth aged 16 years to 25 years who present for HIV testing. TRIAL REGISTRATION ClinicalTrials.gov number NCT02703116; https://clinicaltrials.gov/ct2/show/NCT02703116. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) RR2-10.1186/s12889-020-8154-6.
Collapse
Affiliation(s)
- Niranjan S Karnik
- Institute for Juvenile Research, Department of Psychiatry, University of Illinois Chicago, Chicago, IL, United States
| | - Lisa M Kuhns
- The Potocsnak Family Division of Adolescent and Young Adult Medicine, Ann & Robert H Lurie Children's Hospital of Chicago, Chicago, IL, United States
| | - Anna L Hotton
- The Chicago Center for HIV Elimination, The University of Chicago, Chicago, IL, United States
| | - Natascha Del Vecchio
- The Chicago Center for HIV Elimination, The University of Chicago, Chicago, IL, United States
| | - Moira McNulty
- The Chicago Center for HIV Elimination, The University of Chicago, Chicago, IL, United States
| | - John Schneider
- The Chicago Center for HIV Elimination, The University of Chicago, Chicago, IL, United States
| | - Geri Donenberg
- Center for Dissemination and Implementation Science, University of Illinois Chicago, Chicago, IL, United States
| | | | - Rose Diskin
- The Potocsnak Family Division of Adolescent and Young Adult Medicine, Ann & Robert H Lurie Children's Hospital of Chicago, Chicago, IL, United States
| | - Abigail Muldoon
- The Potocsnak Family Division of Adolescent and Young Adult Medicine, Ann & Robert H Lurie Children's Hospital of Chicago, Chicago, IL, United States
| | - Juan Rivera
- Center for Education, Research & Advocacy, Howard Brown Health, Chicago, IL, United States
| | - Faith Summersett Williams
- The Potocsnak Family Division of Adolescent and Young Adult Medicine, Ann & Robert H Lurie Children's Hospital of Chicago, Chicago, IL, United States
| | - Robert Garofalo
- The Potocsnak Family Division of Adolescent and Young Adult Medicine, Ann & Robert H Lurie Children's Hospital of Chicago, Chicago, IL, United States
| |
Collapse
|
26
|
Nicol G, Wang R, Graham S, Dodd S, Garbutt J. Chatbot-Delivered Cognitive Behavioral Therapy in Adolescents With Depression and Anxiety During the COVID-19 Pandemic: Feasibility and Acceptability Study. JMIR Form Res 2022; 6:e40242. [PMID: 36413390 PMCID: PMC9683529 DOI: 10.2196/40242] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2022] [Revised: 10/10/2022] [Accepted: 10/25/2022] [Indexed: 11/23/2022] Open
Abstract
BACKGROUND Symptoms of depression and anxiety, suicidal ideation, and self-harm have escalated among adolescents to crisis levels during the COVID-19 pandemic. As a result, primary care providers (PCPs) are often called on to provide first-line care for these youth. Digital health interventions can extend mental health specialty care, but few are evidence based. We evaluated the feasibility of delivering an evidence-based mobile health (mHealth) app with an embedded conversational agent to deliver cognitive behavioral therapy (CBT) to symptomatic adolescents presenting in primary care settings during the pandemic. OBJECTIVE In this 12-week pilot study, we evaluated the feasibility of delivering the app-based intervention to adolescents aged 13 to 17 years with moderate depressive symptoms who were treated in a practice-based research network (PBRN) of academically affiliated primary care clinics. We also obtained preliminary estimates of app acceptability, effectiveness, and usability. METHODS This small, pilot randomized controlled trial (RCT) evaluated depressive symptom severity in adolescents randomized to the app or to a wait list control condition. The primary end point was depression severity at 4-weeks, measured by the 9-item Patient Health Questionnaire (PHQ-9). Data on acceptability, feasibility, and usability were collected from adolescents and their parent or legal guardian. Qualitative interviews were conducted with 13 PCPs from 11 PBRN clinics to identify facilitators and barriers to incorporating mental health apps in treatment planning for adolescents with depression and anxiety. RESULTS The pilot randomized 18 participants to the app (n=10, 56%) or to a wait list control condition (n=8, 44%); 17 participants were included in the analysis, and 1 became ineligible upon chart review due to lack of eligibility based on documented diagnosis. The overall sample was predominantly female (15/17, 88%), White (15/17, 88%), and privately insured (15/17, 88%). Mean PHQ-9 scores at 4 weeks decreased by 3.3 points in the active treatment group (representing a shift in mean depression score from moderate to mild symptom severity categories) and 2 points in the wait list control group (no shift in symptom severity category). Teen- and parent-reported usability, feasibility, and acceptability of the app was high. PCPs reported preference for introducing mHealth interventions like the one in this study early in the course of care for individuals presenting with mild or moderate symptoms. CONCLUSIONS In this small study, we demonstrated the feasibility, acceptability, usability, and safety of using a CBT-based chatbot for adolescents presenting with moderate depressive symptoms in a network of PBRN-based primary care clinics. This pilot study could not establish effectiveness, but our results suggest that further study in a larger pediatric population is warranted. Future study inclusive of rural, socioeconomically disadvantaged, and underrepresented communities is needed to establish generalizability of effectiveness and identify implementation-related adaptations needed to promote broader uptake in pediatric primary care. TRIAL REGISTRATION ClinicalTrials.gov NCT04603053; https://clinicaltrials.gov/ct2/show/NCT04603053.
Collapse
Affiliation(s)
- Ginger Nicol
- Division of Child and Adolescent Psychiatry, Department of Psychiatry, Washington University School of Medicine, St Louis, MO, United States
| | - Ruoyun Wang
- Division of Allergy, Immunology & Pulmonology, Department of Pediatrics, Washington University School of Medicine, St Louis, MO, United States
| | - Sharon Graham
- Division of Allergy, Immunology & Pulmonology, Department of Pediatrics, Washington University School of Medicine, St Louis, MO, United States
| | - Sherry Dodd
- Division of Allergy, Immunology & Pulmonology, Department of Pediatrics, Washington University School of Medicine, St Louis, MO, United States
| | - Jane Garbutt
- Division of Allergy, Immunology & Pulmonology, Department of Pediatrics, Washington University School of Medicine, St Louis, MO, United States
| |
Collapse
|
27
|
Abstract
PURPOSE OF REVIEW The purpose of this review is to provide an update of the recently published randomized controlled trials in the field of digital health interventions for substance use disorders. RECENT FINDINGS Over the past 2 years, five cannabis-specific and seven polysubstance-focused randomized controlled trials were published. No studies were found that focused on opioid or psychostimulant use disorders. Most studies examined feasibility but were underpowered to assess effectiveness. Given the optimistic results of the studies in regards to feasibility more fully powered trials should be conducted. In addition, the literature is in need for an increased focus on comorbidity and outcome standardization. SUMMARY Although the number of studies targeting new target groups, technologies and new delivery settings has increased - future studies should consider the identified gaps and suggestions to further strengthen the evidence of digital interventions targeting substance use disorders.
Collapse
|
28
|
Gong H, Wang M, Zhang H, Elahe MF, Jin M. An Explainable AI Approach for the Rapid Diagnosis of COVID-19 Using Ensemble Learning Algorithms. Front Public Health 2022; 10:874455. [PMID: 35801239 PMCID: PMC9253566 DOI: 10.3389/fpubh.2022.874455] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2022] [Accepted: 05/19/2022] [Indexed: 11/13/2022] Open
Abstract
Background Artificial intelligence-based disease prediction models have a greater potential to screen COVID-19 patients than conventional methods. However, their application has been restricted because of their underlying black-box nature. Objective To addressed this issue, an explainable artificial intelligence (XAI) approach was developed to screen patients for COVID-19. Methods A retrospective study consisting of 1,737 participants (759 COVID-19 patients and 978 controls) admitted to San Raphael Hospital (OSR) from February to May 2020 was used to construct a diagnosis model. Finally, 32 key blood test indices from 1,374 participants were used for screening patients for COVID-19. Four ensemble learning algorithms were used: random forest (RF), adaptive boosting (AdaBoost), gradient boosting decision tree (GBDT), and extreme gradient boosting (XGBoost). Feature importance from the perspective of the clinical domain and visualized interpretations were illustrated by using local interpretable model-agnostic explanations (LIME) plots. Results The GBDT model [area under the curve (AUC): 86.4%; 95% confidence interval (CI) 0.821–0.907] outperformed the RF model (AUC: 85.7%; 95% CI 0.813–0.902), AdaBoost model (AUC: 85.4%; 95% CI 0.810–0.899), and XGBoost model (AUC: 84.9%; 95% CI 0.803–0.894) in distinguishing patients with COVID-19 from those without. The cumulative feature importance of lactate dehydrogenase, white blood cells, and eosinophil counts was 0.145, 0.130, and 0.128, respectively. Conclusions Ensemble machining learning (ML) approaches, mainly GBDT and LIME plots, are efficient for screening patients with COVID-19 and might serve as a potential tool in the auxiliary diagnosis of COVID-19. Patients with higher WBC count, higher LDH level, or higher EOT count, were more likely to have COVID-19.
Collapse
Affiliation(s)
- Houwu Gong
- Department of Software Engineering, College of Computer Science and Electronic Engineering, Hunan University, Changsha, China
- Academy of Military Sciences, Beijing, China
| | - Miye Wang
- Engineering Research Center of Medical Information Technology, Ministry of Education, West China Hospital, Chengdu, China
- Information Center, West China Hospital, Chengdu, China
| | - Hanxue Zhang
- Department of Software Engineering, College of Computer Science and Electronic Engineering, Hunan University, Changsha, China
| | - Md Fazla Elahe
- Department of Software Engineering, College of Computer Science and Electronic Engineering, Hunan University, Changsha, China
| | - Min Jin
- Department of Software Engineering, College of Computer Science and Electronic Engineering, Hunan University, Changsha, China
- *Correspondence: Min Jin
| |
Collapse
|
29
|
Darcy A, Beaudette A, Chiauzzi E, Daniels J, Goodwin K, Mariano TY, Wicks P, Robinson A. Anatomy of a Woebot® (WB001): agent guided CBT for women with postpartum depression. Expert Rev Med Devices 2022; 19:287-301. [PMID: 35748029 DOI: 10.1080/17434440.2022.2075726] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2021] [Accepted: 05/05/2022] [Indexed: 11/04/2022]
Abstract
INTRODUCTION Postpartum Depression (PPD) is common, persistent, and stigmatized. There are insufficient trained professionals to deliver appropriate screening, diagnosis, and treatment. AREAS COVERED WB001 is a Software as a Medical Device (SaMD) based Agent-Guided Cognitive Behavioral Therapy (AGCBT) program for the treatment of PPD, for which Breakthrough Device Designation was recently granted by the US Food and Drug Administration. WB001 combines therapeutic alliance, human-centered design, machine learning techniques, and established principles from CBT and interpersonal therapy (IPT). We introduce AGCBT as a new model of service delivery, whilst describing Woebot, the agent technology that enables guidance through the replication of some elements of human relationships. The profile describes the device's design principles, enabling technology, risk handling, and efficacy data in PPD. EXPERT OPINION WB001 is a dynamic and personalized tool with which patients may establish a therapeutic bond. Woebot is designed to augment (rather than replace) human healthcare providers, unlocking the therapeutic potency associated with guidance, whilst retaining the scalability and agency that characterizes self-help approaches. WB001 has the potential to improve both the quality and the scalability of care through providing support to patients on waiting lists, in between clinical encounters, and enabling automation of measurement-based care.
Collapse
|
30
|
Tong F, Lederman R, D'Alfonso S, Berry K, Bucci S. Digital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review. Front Psychiatry 2022; 13:819623. [PMID: 35815030 PMCID: PMC9256980 DOI: 10.3389/fpsyt.2022.819623] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/22/2021] [Accepted: 05/30/2022] [Indexed: 11/29/2022] Open
Abstract
Fully automated mental health smartphone apps show strong promise in increasing access to psychological support. Therefore, it is crucial to understand how to make these apps effective. The therapeutic alliance (TA), or the relationship between healthcare professionals and clients, is considered fundamental to successful treatment outcomes in face-to-face therapy. Thus, understanding the TA in the context of fully automated apps would bring us insights into building effective smartphone apps which engage users. However, the concept of a digital therapeutic alliance (DTA) in the context of fully automated mental health smartphone apps is nascent and under-researched, and only a handful of studies have been published in this area. In particular, no published review paper examined the DTA in the context of fully automated apps. The objective of this review was to integrate the extant literature to identify research gaps and future directions in the investigation of DTA in relation to fully automated mental health smartphone apps. Our findings suggest that the DTA in relation to fully automated smartphone apps needs to be conceptualized differently to traditional face-to-face TA. First, the role of bond in the context of fully automated apps is unclear. Second, human components of face-to-face TA, such as empathy, are hard to achieve in the digital context. Third, some users may perceive apps as more non-judgmental and flexible, which may further influence DTA formation. Subdisciplines of computer science, such as affective computing and positive computing, and some human-computer interaction (HCI) theories, such as those of persuasive technology and human-app attachment, can potentially help to foster a sense of empathy, build tasks and goals and develop bond or an attachment between users and apps, which may further contribute to DTA formation in fully automated smartphone apps. Whilst the review produced a relatively limited quantity of literature, this reflects the novelty of the topic and the need for further research.
Collapse
Affiliation(s)
- Fangziyun Tong
- School of Computing and Information Systems, University of Melbourne, Parkville, VIC, Australia.,Division of Psychology and Mental Health, School of Health Sciences, Manchester Academic Health Sciences Centre, University of Manchester, Manchester, United Kingdom
| | - Reeva Lederman
- School of Computing and Information Systems, University of Melbourne, Parkville, VIC, Australia
| | - Simon D'Alfonso
- School of Computing and Information Systems, University of Melbourne, Parkville, VIC, Australia
| | - Katherine Berry
- Division of Psychology and Mental Health, School of Health Sciences, Manchester Academic Health Sciences Centre, University of Manchester, Manchester, United Kingdom.,Complex Trauma and Resilience Research Unit, Greater Manchester Mental Health NHS Foundation Trust, Manchester, United Kingdom
| | - Sandra Bucci
- Division of Psychology and Mental Health, School of Health Sciences, Manchester Academic Health Sciences Centre, University of Manchester, Manchester, United Kingdom.,Complex Trauma and Resilience Research Unit, Greater Manchester Mental Health NHS Foundation Trust, Manchester, United Kingdom
| |
Collapse
|
31
|
Hwang M, Kim SP, Chung D. Exploring the impacts of implicit context association and arithmetic booster in impulsivity reduction. Front Psychiatry 2022; 13:961484. [PMID: 36177221 PMCID: PMC9513136 DOI: 10.3389/fpsyt.2022.961484] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/04/2022] [Accepted: 08/22/2022] [Indexed: 12/02/2022] Open
Abstract
People have a higher preference for immediate over delayed rewards, and it is suggested that such an impulsive tendency is governed by one's ability to simulate future rewards. Consistent with this view, recent studies have shown that enforcing individuals to focus on episodic future thoughts reduces their impulsivity. Inspired by these reports, we hypothesized that administration of a simple cognitive task linked to future thinking might effectively modulate individuals' delay discounting. Specifically, we used one associative memory task targeting intervention of context information, and one working memory task targeting enhancement of individual's ability to construct a coherent future event. To measure whether each type of cognitive task reduces individuals' impulsivity, a classic intertemporal choice task was used to quantify individuals' baseline and post-intervention impulsivity. Across two experiments and data from 216 healthy young adult participants, we observed that the impacts of intervention tasks were inconsistent. Still, we observed a significant task repetition effect such that the participants showed more patient choices in the second impulsivity assessment. In conclusion, there was no clear evidence supporting that our suggested intervention tasks reduce individuals' impulsivity, and that the current results call attention to the importance of taking into account task repetition effects in studying the impacts of cognitive training and intervention.
Collapse
Affiliation(s)
- Minho Hwang
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan, South Korea
| | - Sung-Phil Kim
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan, South Korea
| | - Dongil Chung
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan, South Korea
| |
Collapse
|