The Use and Promise of Conversational Agents in Digital Health

The Feasibility and Acceptability of Using a Digital Conversational Agent (Chatbot) for Delivering Parenting Interventions: Systematic Review

BACKGROUND

Parenting interventions are crucial for promoting family well-being, reducing violence against children, and improving child development outcomes; however, scaling these programs remains a challenge. Prior reviews have characterized the feasibility, acceptability, and effectiveness of other more robust forms of digital parenting interventions (eg, via the web, mobile apps, and videoconferencing). Recently, chatbot technology has emerged as a possible mode for adapting and delivering parenting programs to larger populations (eg, Parenting for Lifelong Health, Incredible Years, and Triple P Parenting).

OBJECTIVE

This study aims to review the evidence of using chatbots to deliver parenting interventions and assess the feasibility of implementation, acceptability of these interventions, and preliminary outcomes.

METHODS

This review conducted a comprehensive search of databases, including Web of Science, MEDLINE, Scopus, ProQuest, and Cochrane Central Register of Controlled Trials. Cochrane Handbook for Systematic Review of Interventions and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were used to conduct the search. Eligible studies targeted parents of children aged 0 to 18 years; used chatbots via digital platforms, such as the internet, mobile apps, or SMS text messaging; and targeted improving family well-being through parenting. Implementation measures, acceptability, and any reported preliminary measures of effectiveness were included.

RESULTS

Of the 1766 initial results, 10 studies met the inclusion criteria. The included studies, primarily conducted in high-income countries (8/10, 80%), demonstrated a high mean retention rate (72.8%) and reported high acceptability (10/10, 100%). However, significant heterogeneity in interventions, measurement methods, and study quality necessitate cautious interpretation. Reporting bias, lack of clarity in the operationalization of engagement measures, and platform limitations were identified as limiting factors in interpreting findings.

CONCLUSIONS

This is the first study to review the implementation feasibility and acceptability of chatbots for delivering parenting programs. While preliminary evidence suggests that chatbots can be used to deliver parenting programs, further research, standardization of reporting, and scaling up of effectiveness testing are critical to harness the full benefits of chatbots for promoting family well-being.

Engagement With Conversational Agent-Enabled Interventions in Cardiometabolic Disease Management: Protocol for a Systematic Review

BACKGROUND

Cardiometabolic diseases (CMDs) are a group of interrelated conditions, including heart failure and diabetes, that increase the risk of cardiovascular and metabolic complications. The rising number of Australians with CMDs has necessitated new strategies for those managing these conditions, such as digital health interventions. The effectiveness of digital health interventions in supporting people with CMDs is dependent on the extent to which users engage with the tools. Augmenting digital health interventions with conversational agents, technologies that interact with people using natural language, may enhance engagement because of their human-like attributes. To date, no systematic review has compiled evidence on how design features influence the engagement of conversational agent-enabled interventions supporting people with CMDs. This review seeks to address this gap, thereby guiding developers in creating more engaging and effective tools for CMD management.

OBJECTIVE

The aim of this systematic review is to synthesize evidence pertaining to conversational agent-enabled intervention design features and their impacts on the engagement of people managing CMD.

METHODS

The review is conducted in accordance with the Cochrane Handbook for Systematic Reviews of Interventions and reported in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Searches will be conducted in the Ovid (Medline), Web of Science, and Scopus databases, which will be run again prior to manuscript submission. Inclusion criteria will consist of primary research studies reporting on conversational agent-enabled interventions, including measures of engagement, in adults with CMD. Data extraction will seek to capture the perspectives of people with CMD on the use of conversational agent-enabled interventions. Joanna Briggs Institute critical appraisal tools will be used to evaluate the overall quality of evidence collected.

RESULTS

This review was initiated in May 2023 and was registered with the International Prospective Register of Systematic Reviews (PROSPERO) in June 2023, prior to title and abstract screening. Full-text screening of articles was completed in July 2023 and data extraction began August 2023. Final searches were conducted in April 2024 prior to finalizing the review and the manuscript was submitted for peer review in July 2024.

CONCLUSIONS

This review will synthesize diverse observations pertaining to conversational agent-enabled intervention design features and their impacts on engagement among people with CMDs. These observations can be used to guide the development of more engaging conversational agent-enabled interventions, thereby increasing the likelihood of regular intervention use and improved CMD health outcomes. Additionally, this review will identify gaps in the literature in terms of how engagement is reported, thereby highlighting areas for future exploration and supporting researchers in advancing the understanding of conversational agent-enabled interventions.

TRIAL REGISTRATION

PROSPERO CRD42023431579; https://tinyurl.com/55cxkm26.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/52973.

Evaluating the Potential and Pitfalls of AI-Powered Conversational Agents as Humanlike Virtual Health Carers in the Remote Management of Noncommunicable Diseases: Scoping Review

BACKGROUND

The rising prevalence of noncommunicable diseases (NCDs) worldwide and the high recent mortality rates (74.4%) associated with them, especially in low- and middle-income countries, is causing a substantial global burden of disease, necessitating innovative and sustainable long-term care solutions.

OBJECTIVE

This scoping review aims to investigate the impact of artificial intelligence (AI)-based conversational agents (CAs)-including chatbots, voicebots, and anthropomorphic digital avatars-as human-like health caregivers in the remote management of NCDs as well as identify critical areas for future research and provide insights into how these technologies might be used effectively in health care to personalize NCD management strategies.

METHODS

A broad literature search was conducted in July 2023 in 6 electronic databases-Ovid MEDLINE, Embase, PsycINFO, PubMed, CINAHL, and Web of Science-using the search terms "conversational agents," "artificial intelligence," and "noncommunicable diseases," including their associated synonyms. We also manually searched gray literature using sources such as ProQuest Central, ResearchGate, ACM Digital Library, and Google Scholar. We included empirical studies published in English from January 2010 to July 2023 focusing solely on health care-oriented applications of CAs used for remote management of NCDs. The narrative synthesis approach was used to collate and summarize the relevant information extracted from the included studies.

RESULTS

The literature search yielded a total of 43 studies that matched the inclusion criteria. Our review unveiled four significant findings: (1) higher user acceptance and compliance with anthropomorphic and avatar-based CAs for remote care; (2) an existing gap in the development of personalized, empathetic, and contextually aware CAs for effective emotional and social interaction with users, along with limited consideration of ethical concerns such as data privacy and patient safety; (3) inadequate evidence of the efficacy of CAs in NCD self-management despite a moderate to high level of optimism among health care professionals regarding CAs' potential in remote health care; and (4) CAs primarily being used for supporting nonpharmacological interventions such as behavioral or lifestyle modifications and patient education for the self-management of NCDs.

CONCLUSIONS

This review makes a unique contribution to the field by not only providing a quantifiable impact analysis but also identifying the areas requiring imminent scholarly attention for the ethical, empathetic, and efficacious implementation of AI in NCD care. This serves as an academic cornerstone for future research in AI-assisted health care for NCD management.

TRIAL REGISTRATION

Open Science Framework; https://doi.org/10.17605/OSF.IO/GU5PX.

Artificial intelligence in perinatal mental health research: A scoping review

The intersection of Artificial Intelligence (AI) and perinatal mental health research presents promising avenues, yet uncovers significant challenges for innovation. This review explicitly focuses on this multidisciplinary field and undertakes a comprehensive exploration of existing research therein. Through a scoping review guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework, we searched relevant literature spanning a decade (2013-2023) and selected fourteen studies for our analysis. We first provide an overview of the main AI techniques and their development, including traditional methods across different categories, as well as recent emerging methods in the field. Then, through our analysis of the literature, we summarize the predominant AI and ML techniques adopted and their applications in perinatal mental health studies, such as identifying risk factors, predicting perinatal mental health disorders, voice assistants, and Q&A chatbots. We also discuss existing limitations and potential challenges that hinder AI technologies from improving perinatal mental health outcomes, and suggest several promising directions for future research to meet real needs in the field and facilitate the translation of research into clinical settings.

Exploring the Impact of Artificial Intelligence on Global Health and Enhancing Healthcare in Developing Nations

BACKGROUND

Artificial intelligence (AI), which combines computer science with extensive datasets, seeks to mimic human-like intelligence. Subsets of AI are being applied in almost all fields of medicine and surgery.

AIM

This review focuses on the applications of AI in healthcare settings in developing countries, designed to underscore its significance by comprehensively outlining the advancements made thus far, the shortcomings encountered in AI applications, the present status of AI integration, persistent challenges, and innovative strategies to surmount them.

METHODOLOGY

Articles from PubMed, Google Scholar, and Cochrane were searched from 2000 to 2023 with keywords including AI and healthcare, focusing on multiple medical specialties.

RESULTS

The increasing role of AI in diagnosis, prognosis prediction, and patient management, as well as hospital management and community healthcare, has made the overall healthcare system more efficient, especially in the high patient load setups and resource-limited areas of developing countries where patient care is often compromised. However, challenges, including low adoption rates and the absence of standardized guidelines, high installation and maintenance costs of equipment, poor transportation and connectivvity issues hinder AI's full use in healthcare.

CONCLUSION

Despite these challenges, AI holds a promising future in healthcare. Adequate knowledge and expertise of healthcare professionals for the use of AI technology in healthcare is imperative in developing nations.

Systematic review and meta-analysis of AI-based conversational agents for promoting mental health and well-being

Conversational artificial intelligence (AI), particularly AI-based conversational agents (CAs), is gaining traction in mental health care. Despite their growing usage, there is a scarcity of comprehensive evaluations of their impact on mental health and well-being. This systematic review and meta-analysis aims to fill this gap by synthesizing evidence on the effectiveness of AI-based CAs in improving mental health and factors influencing their effectiveness and user experience. Twelve databases were searched for experimental studies of AI-based CAs' effects on mental illnesses and psychological well-being published before May 26, 2023. Out of 7834 records, 35 eligible studies were identified for systematic review, out of which 15 randomized controlled trials were included for meta-analysis. The meta-analysis revealed that AI-based CAs significantly reduce symptoms of depression (Hedge's g 0.64 [95% CI 0.17-1.12]) and distress (Hedge's g 0.7 [95% CI 0.18-1.22]). These effects were more pronounced in CAs that are multimodal, generative AI-based, integrated with mobile/instant messaging apps, and targeting clinical/subclinical and elderly populations. However, CA-based interventions showed no significant improvement in overall psychological well-being (Hedge's g 0.32 [95% CI -0.13 to 0.78]). User experience with AI-based CAs was largely shaped by the quality of human-AI therapeutic relationships, content engagement, and effective communication. These findings underscore the potential of AI-based CAs in addressing mental health issues. Future research should investigate the underlying mechanisms of their effectiveness, assess long-term effects across various mental health outcomes, and evaluate the safe integration of large language models (LLMs) in mental health care.

Evaluation of the Current State of Chatbots for Digital Health: Scoping Review

BACKGROUND

Chatbots have become ubiquitous in our daily lives, enabling natural language conversations with users through various modes of communication. Chatbots have the potential to play a significant role in promoting health and well-being. As the number of studies and available products related to chatbots continues to rise, there is a critical need to assess product features to enhance the design of chatbots that effectively promote health and behavioral change.

OBJECTIVE

This scoping review aims to provide a comprehensive assessment of the current state of health-related chatbots, including the chatbots' characteristics and features, user backgrounds, communication models, relational building capacity, personalization, interaction, responses to suicidal thoughts, and users' in-app experiences during chatbot use. Through this analysis, we seek to identify gaps in the current research, guide future directions, and enhance the design of health-focused chatbots.

METHODS

Following the scoping review methodology by Arksey and O'Malley and guided by the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist, this study used a two-pronged approach to identify relevant chatbots: (1) searching the iOS and Android App Stores and (2) reviewing scientific literature through a search strategy designed by a librarian. Overall, 36 chatbots were selected based on predefined criteria from both sources. These chatbots were systematically evaluated using a comprehensive framework developed for this study, including chatbot characteristics, user backgrounds, building relational capacity, personalization, interaction models, responses to critical situations, and user experiences. Ten coauthors were responsible for downloading and testing the chatbots, coding their features, and evaluating their performance in simulated conversations. The testing of all chatbot apps was limited to their free-to-use features.

RESULTS

This review provides an overview of the diversity of health-related chatbots, encompassing categories such as mental health support, physical activity promotion, and behavior change interventions. Chatbots use text, animations, speech, images, and emojis for communication. The findings highlight variations in conversational capabilities, including empathy, humor, and personalization. Notably, concerns regarding safety, particularly in addressing suicidal thoughts, were evident. Approximately 44% (16/36) of the chatbots effectively addressed suicidal thoughts. User experiences and behavioral outcomes demonstrated the potential of chatbots in health interventions, but evidence remains limited.

CONCLUSIONS

This scoping review underscores the significance of chatbots in health-related applications and offers insights into their features, functionalities, and user experiences. This study contributes to advancing the understanding of chatbots' role in digital health interventions, thus paving the way for more effective and user-centric health promotion strategies. This study informs future research directions, emphasizing the need for rigorous randomized control trials, standardized evaluation metrics, and user-centered design to unlock the full potential of chatbots in enhancing health and well-being. Future research should focus on addressing limitations, exploring real-world user experiences, and implementing robust data security and privacy measures.

Adaptation of the Gabby conversational agent system to improve the sexual and reproductive health of young women in Lesotho

Introduction

Young women from the low-middle-income country of Lesotho in southern Africa frequently report limited knowledge regarding sexual and reproductive health issues and engage in risky sexual behaviors. The purpose of this study is to describe the adaptation of an evidence-based conversational agent system for implementation in Lesotho and provide qualitative data pertaining to the success of the said adaptation.

Methods

An embodied conversational agent system used to provide preconception health advice in the United States was clinically and culturally adapted for use in the rural country of Lesotho in southern Africa. Inputs from potential end users, health leaders, and district nurses guided the adaptations. Focus group discussions with young women aged 18-28 years who had used the newly adapted system renamed "Nthabi" for 3-4 weeks and key informant interviews with Ministry of Health leadership were conducted to explore their views of the acceptability of the said adaptation. Data were analyzed using NVivo software, and a thematic content analysis approach was employed in the study.

Results

A total of 33 women aged 18-28 years used Nthabi for 3-4 weeks; eight (24.2%) of them were able to download and use the app on their mobile phones and 25 (75.8%) of them used the app on a tablet provided to them. Focus group participants (n = 33) reported that adaptations were culturally appropriate and provided relevant clinical information. The participants emphasized that the physical characteristics, personal and non-verbal behaviors, utilization of Sesotho words and idioms, and sensitively delivered clinical content were culturally appropriate for Lesotho. The key informants from the Ministry leadership (n = 10) agreed that the adaptation was successful, and that the system holds great potential to improve the delivery of health education in Lesotho. Both groups suggested modifications, such as using the local language and adapting Nthabi for use by boys and young men.

Conclusions

Clinically tailored, culturally sensitive, and trustworthy content provided by Nthabi has the potential to improve accessibility of sexual and reproductive health information to young women in the low-middle-income country of Lesotho.

Understanding Trust Determinants in a Live Chat Service on Familial Cancer: Qualitative Triangulation Study With Focus Groups and Interviews in Germany

BACKGROUND

In dealing with familial cancer risk, seeking web-based health information can be a coping strategy for different stakeholder groups (ie, patients, relatives, and those suspecting an elevated familial cancer risk). In the vast digital landscape marked by a varied quality of web-based information and evolving technologies, trust emerges as a pivotal factor, guiding the process of health information seeking and interacting with digital health services. This trust formation in health information can be conceptualized as context dependent and multidimensional, involving 3 key dimensions: information seeker (trustor), information provider (trustee), and medium or platform (application). Owing to the rapid changes in the digital context, it is critical to understand how seekers form trust in new services, given the interplay among these different dimensions. An example of such a new service is a live chat operated by physicians for the general public with personalized cancer-related information and a focus on familial cancer risk.

OBJECTIVE

To gain a comprehensive picture of trust formation in a cancer-related live chat service, this study investigates the 3 dimensions of trust-trustor, trustee, and application-and their respective relevant characteristics based on a model of trust in web-based health information. In addition, the study aims to compare these characteristics across the 3 different stakeholder groups, with the goal to enhance the service's trustworthiness for each group.

METHODS

This qualitative study triangulated the different perspectives of medical cancer advisers, advisers from cancer support groups, and members of the public in interviews and focus group discussions to explore the 3 dimensions of trust-trustor, trustee, and application-and their determinants for a new live chat service for familial cancer risk to be implemented at the German Cancer Information Service.

RESULTS

The results indicate that experience with familial cancer risk is the key trustor characteristic to using, and trusting information provided by, the live chat service. The live chat might also be particularly valuable for people from minority groups who have unmet needs from physician-patient interactions. Participants highlighted trustee characteristics such as ability, benevolence, integrity, and humanness (ie, not a chatbot) as pivotal in a trustworthy cancer live chat service. Application-related characteristics, including the reputation of the institution, user-centric design, modern technology, and visual appeal, were also deemed essential. Despite the different backgrounds and sociodemographics of the 3 stakeholder groups, many overlaps were found among the 3 trust dimensions and their respective characteristics.

CONCLUSIONS

Trust in a live chat for cancer information is formed by different dimensions and characteristics of trust. This study underscores the importance of understanding trust formation in digital health services and suggests potential enhancements for effective, trustworthy interactions in live chat services (eg, by providing biographies of the human medical experts to differentiate them from artificial intelligence chatbots).

User-acceptability of an automated telephone call for post-operative follow-up after uncomplicated cataract surgery

BACKGROUND

Innovative technology is recommended to address the current capacity challenges facing the NHS. This study evaluates the patient acceptability of automated telephone follow-up after routine cataract surgery using Dora (Ufonia Limited, Oxford, United Kingdom), which to our knowledge is the first AI-powered clinical assistant to be used in the NHS. Dora has a natural-language, phone conversation with patients about their symptoms after cataract surgery.

METHODS

This is a prospective mixed-methods cohort study that was conducted at Buckinghamshire Healthcare NHS Foundation Trust. All patients who were followed up using Dora were asked to give a Net Promoter Score (NPS), and 24 patients were randomly selected to complete the validated Telephone Usability Questionnaire (TUQ) as well as extended semi-structured interviews that underwent thematic analysis.

RESULTS

A total of 170 autonomous calls were completed. The median NPS score was 9 out of 10. The TUQ (scored out of 5) showed high rates of acceptability, with an overall mean score of 4.0. Simplicity, time saving, and ease of use scored the highest with a median of 5, whilst 'speaking to Dora feels the same as speaking to a clinician' scored a median of 3. The main themes extracted from the qualitative data were 'I can see why you're doing it', 'It went quite well actually', 'I just trust human beings I suppose'.

CONCLUSION

We found high levels of patient acceptability when using Dora across three acceptability measures. Dora provides a potential solution to reduce pressure on hospital capacity whilst also providing a convenient service for patients.

Effects of personalization and source expertise on users' health beliefs and usage intention toward health chatbots: Evidence from an online experiment

Objective

Based on the heuristic–systematic model (HSM) and health belief model (HBM), this study aims to investigate how personalization and source expertise in responses from a health chatbot influence users’ health belief-related factors (i.e. perceived benefits, self-efficacy and privacy concerns) as well as usage intention.

Methods

A 2 (personalization vs. non-personalization) × 2 (source expertise vs. non-source expertise) online between-subject experiment was designed. Participants were recruited in China between April and May 2021. Data from 260 valid observations were used for the data analysis.

Results

Source expertise moderated the effects of personalization on health belief factors. Perceived benefits and self-efficacy mediated the relationship between personalization and usage intention when the source expertise cue was presented. However, the privacy concerns were not influenced by personalization and source expertise and did not significantly affect usage intention toward the health chatbot.

Discussion

This study verified that in the health chatbot context, source expertise as a heuristic cue may be a necessary condition for effects of the systematic cue (i.e. personalization), which supports the HSM's arguments. By introducing the HBM in the chatbot experiment, this study is expected to provide new insights into the acceptance of healthcare AI consulting services.

Effectiveness of a chatbot for eating disorders prevention: A randomized clinical trial

OBJECTIVE

Prevention of eating disorders (EDs) is of high importance. However, digital programs with human moderation are unlikely to be disseminated widely. The aim of this study was to test whether a chatbot (i.e., computer program simulating human conversation) would significantly reduce ED risk factors (i.e., weight/shape concerns, thin-ideal internalization) in women at high risk for an ED, compared to waitlist control, as well as whether it would significantly reduce overall ED psychopathology, depression, and anxiety and prevent ED onset.

METHOD

Women who screened as high risk for an ED were randomized (N = 700) to (1) chatbot based on the StudentBodies© program; or (2) waitlist control. Participants were followed for 6 months.

RESULTS

For weight/shape concerns, there was a significantly greater reduction in intervention versus control at 3- (d = -0.20; p = .03) and 6-m-follow-up (d = -0.19; p = .04). There were no differences in change in thin-ideal internalization. The intervention was associated with significantly greater reductions than control in overall ED psychopathology at 3- (d = -0.29; p = .003) but not 6-month follow-up. There were no differences in change in depression or anxiety. The odds of remaining nonclinical for EDs were significantly higher in intervention versus control at both 3- (OR = 2.37, 95% CI [1.37, 4.11]) and 6-month follow-ups (OR = 2.13, 95% CI [1.26, 3.59]).

DISCUSSION

Findings provide support for the use of a chatbot-based EDs prevention program in reducing weight/shape concerns through 6-month follow-up, as well as in reducing overall ED psychopathology, at least in the shorter-term. Results also suggest the intervention may reduce ED onset.

PUBLIC SIGNIFICANCE

We found that a chatbot, or a computer program simulating human conversation, based on an established, cognitive-behavioral therapy-based eating disorders prevention program, was successful in reducing women's concerns about weight and shape through 6-month follow-up and that it may actually reduce eating disorder onset. These findings are important because this intervention, which uses a rather simple text-based approach, can easily be disseminated in order to prevent these deadly illnesses.

TRIAL REGISTRATION

OSF Registries; https://osf.io/7zmbv.