Conversational Agents in Health Care: Scoping Review and Conceptual Analysis

Implementation of Anxiety UK's Ask Anxia Chatbot Service: Lessons Learned

Chatbots are increasingly being applied in the context of health care, providing access to services when there are constraints on human resources. Simple, rule-based chatbots are suited to high-volume, repetitive tasks and can therefore be used effectively in providing users with important health information. In this Viewpoint paper, we report on the implementation of a chatbot service called Ask Anxia as part of a wider provision of information and support services offered by the UK national charity, Anxiety UK. We reflect on the changes made to the chatbot over the course of approximately 18 months as the Anxiety UK team monitored its performance and responded to recurrent themes in user queries by developing further information and services. We demonstrate how corpus linguistics can contribute to the evaluation of user queries and the optimization of responses. On the basis of these observations of how Anxiety UK has developed its own chatbot service, we offer recommendations for organizations looking to add automated conversational interfaces to their services.

Bringing together multimodal and multilevel approaches to study the emergence of social bonds between children and improve social AI

This protocol paper outlines an innovative multimodal and multilevel approach to studying the emergence and evolution of how children build social bonds with their peers, and its potential application to improving social artificial intelligence (AI). We detail a unique hyperscanning experimental framework utilizing functional near-infrared spectroscopy (fNIRS) to observe inter-brain synchrony in child dyads during collaborative tasks and social interactions. Our proposed longitudinal study spans middle childhood, aiming to capture the dynamic development of social connections and cognitive engagement in naturalistic settings. To do so we bring together four kinds of data: the multimodal conversational behaviors that dyads of children engage in, evidence of their state of interpersonal rapport, collaborative performance on educational tasks, and inter-brain synchrony. Preliminary pilot data provide foundational support for our approach, indicating promising directions for identifying neural patterns associated with productive social interactions. The planned research will explore the neural correlates of social bond formation, informing the creation of a virtual peer learning partner in the field of Social Neuroergonomics. This protocol promises significant contributions to understanding the neural basis of social connectivity in children, while also offering a blueprint for designing empathetic and effective social AI tools, particularly for educational contexts.

Depiction of conversational agents as health professionals: a scoping review

OBJECTIVE

The purpose of this scoping review was to examine the depiction of conversational agents as health professionals. We identified the professional characteristics that are used with these depictions and determined the prevalence of these characteristics among conversational agents that are used for health care.

INTRODUCTION

The depiction of conversational agents as health professionals has implications for both the users and the developers of these programs. For this reason, it is important to know more about these depictions and how they are implemented in practical settings.

INCLUSION CRITERIA

This review included scholarly literature on conversational agents that are used for health care. It focused on conversational agents designed for patients and health seekers, not health professionals or trainees. Conversational agents that address physical and/or mental health care were considered, as were programs that promote healthy behaviors.

METHODS

This review was conducted in accordance with JBI methodology for scoping reviews. The databases searched included MEDLINE (PubMed), Embase, CINAHL with Full Text (EBSCOhost), Scopus, Web of Science, ACM Guide to Computing Literature (Association for Computing Machinery Digital Library), and IEEE Xplore (IEEE). The main database search was conducted in June 2021, and an updated search was conducted in January 2022. Extracted data included characteristics of the report, basic characteristics of the conversational agent, and professional characteristics of the conversational agent. Extracted data were summarized using descriptive statistics. Results are presented in a narrative summary and accompanying tables.

RESULTS

A total of 38 health-related conversational agents were identified across 41 reports. Six of these conversational agents (15.8%) had professional characteristics. Four conversational agents (10.5%) had a professional appearance in which they displayed the clothing and accessories of health professionals and appeared in professional settings. One conversational agent (2.6%) had a professional title (Dr), and 4 conversational agents (10.5%) were described as having professional roles. Professional characteristics were more common among embodied vs disembodied conversational agents.

CONCLUSIONS

The results of this review show that the depiction of conversational agents as health professionals is not particularly common, although it does occur. More discussion is needed on the potential ethical and legal issues surrounding the depiction of conversational agents as health professionals. Future research should examine the impact of these depictions, as well as people's attitudes toward them, to better inform recommendations for practice.

Achieving health equity through conversational AI: A roadmap for design and implementation of inclusive chatbots in healthcare

BACKGROUND

The rapid evolution of conversational and generative artificial intelligence (AI) has led to the increased deployment of AI tools in healthcare settings. While these conversational AI tools promise efficiency and expanded access to healthcare services, there are growing concerns ethically, practically and in terms of inclusivity. This study aimed to identify activities which reduce bias in conversational AI and make their designs and implementation more equitable.

METHODS

A qualitative research approach was employed to develop an analytical framework based on the content analysis of 17 guidelines about AI use in clinical settings. A stakeholder consultation was subsequently conducted with a total of 33 ethnically diverse community members, AI designers, industry experts and relevant health professionals to further develop a roadmap for equitable design and implementation of conversational AI in healthcare. Framework analysis was conducted on the interview data.

RESULTS

A 10-stage roadmap was developed to outline activities relevant to equitable conversational AI design and implementation phases: 1) Conception and planning, 2) Diversity and collaboration, 3) Preliminary research, 4) Co-production, 5) Safety measures, 6) Preliminary testing, 7) Healthcare integration, 8) Service evaluation and auditing, 9) Maintenance, and 10) Termination.

DISCUSSION

We have made specific recommendations to increase conversational AI's equity as part of healthcare services. These emphasise the importance of a collaborative approach and the involvement of patient groups in navigating the rapid evolution of conversational AI technologies. Further research must assess the impact of recommended activities on chatbots' fairness and their ability to reduce health inequalities.

Unpacking public resistance to health Chatbots: a parallel mediation analysis

Introduction

Despite the numerous potential benefits of health chatbots for personal health management, a substantial proportion of people oppose the use of such software applications. Building on the innovation resistance theory (IRT) and the prototype willingness model (PWM), this study investigated the functional barriers, psychological barriers, and negative prototype perception antecedents of individuals' resistance to health chatbots, as well as the rational and irrational psychological mechanisms underlying their linkages.

Methods

Data from 398 participants were used to construct a partial least squares structural equation model (PLS-SEM).

Results

Resistance intention mediated the relationship between functional barriers, psychological barriers, and resistance behavioral tendency, respectively. Furthermore, The relationship between negative prototype perceptions and resistance behavioral tendency was mediated by resistance intention and resistance willingness. Moreover, negative prototype perceptions were a more effective predictor of resistance behavioral tendency through resistance willingness than functional and psychological barriers.

Discussion

By investigating the role of irrational factors in health chatbot resistance, this study expands the scope of the IRT to explain the psychological mechanisms underlying individuals' resistance to health chatbots. Interventions to address people's resistance to health chatbots are discussed.

Patient Engagement with Conversational Agents in Health Applications 2016-2022: A Systematic Review and Meta-Analysis

Clinicians and patients seeking electronic health applications face challenges in selecting effective solutions due to a high market failure rate. Conversational agent applications ("chatbots") show promise in increasing healthcare user engagement by creating bonds between the applications and users. It is unclear if chatbots improve patient adherence or if past trends to include chatbots in electronic health applications were due to technology hype dynamics and competitive pressure to innovate. We conducted a systematic literature review using Preferred Reporting Items for Systematic reviews and Meta-Analyses methodology on health chatbot randomized control trials. The goal of this review was to identify if user engagement indicators are published in eHealth chatbot studies. A meta-analysis examined patient clinical trial retention of chatbot apps. The results showed no chatbot arm patient retention effect. The small number of studies suggests a need for ongoing eHealth chatbot research, especially given the claims regarding their effectiveness made outside the scientific literatures.

Dialogue agents for artificial intelligence-based conversational systems for cognitively disabled: a systematic review

PURPOSE

We present a systematic literature review of dialogue agents for Artificial Intelligence (AI) and agent-based conversational systems dealing with cognitive disability of aged and impaired people including dementia and Parkinson's disease. We analyze current applications, gaps, and challenges in the existing research body, and provide guidelines and recommendations for their future development and use.

MATERIALS AND METHODS

We perform this study by applying Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria. We performed a systematic search using relevant databases (ACM Digital Library, Google Scholar, IEEE Xplore, PubMed, and Scopus).

RESULTS

This study identified 468 articles on the use of conversational agents in healthcare. We finally selected 124 articles based on their objectives and content as directly related to our main topic.

CONCLUSION

We identified the main challenges in the field and analyzed the typical examples of the application of conversational agents in the healthcare domain, the desired characteristics of conversational agents, and chatbot support for aged people and people with cognitive disabilities. Our results contribute to a discussion on conversational health agents and emphasize current knowledge gaps and challenges for future research.IMPLICATIONS FOR REHABILITATIONA systematic literature review of dialogue agents for artificial intelligence and agent-based conversational systems dealing with cognitive disability of aged and impaired people.Main challenges and desired characteristics of the conversational agents, and chatbot support for aged people and people with cognitive disability.Current knowledge gaps and challenges for remote healthcare and rehabilitation.Guidelines and recommendations for future development and use of conversational systems.

The potential and limitations of large language models in identification of the states of motivations for facilitating health behavior change

IMPORTANCE

The study highlights the potential and limitations of the Large Language Models (LLMs) in recognizing different states of motivation to provide appropriate information for behavior change. Following the Transtheoretical Model (TTM), we identified the major gap of LLMs in responding to certain states of motivation through validated scenario studies, suggesting future directions of LLMs research for health promotion.

OBJECTIVES

The LLMs-based generative conversational agents (GAs) have shown success in identifying user intents semantically. Little is known about its capabilities to identify motivation states and provide appropriate information to facilitate behavior change progression.

MATERIALS AND METHODS

We evaluated 3 GAs, ChatGPT, Google Bard, and Llama 2 in identifying motivation states following the TTM stages of change. GAs were evaluated using 25 validated scenarios with 5 health topics across 5 TTM stages. The relevance and completeness of the responses to cover the TTM processes to proceed to the next stage of change were assessed.

RESULTS

3 GAs identified the motivation states in the preparation stage providing sufficient information to proceed to the action stage. The responses to the motivation states in the action and maintenance stages were good enough covering partial processes for individuals to initiate and maintain their changes in behavior. However, the GAs were not able to identify users' motivation states in the precontemplation and contemplation stages providing irrelevant information, covering about 20%-30% of the processes.

DISCUSSION

GAs are able to identify users' motivation states and provide relevant information when individuals have established goals and commitments to take and maintain an action. However, individuals who are hesitant or ambivalent about behavior change are unlikely to receive sufficient and relevant guidance to proceed to the next stage of change.

CONCLUSION

The current GAs effectively identify motivation states of individuals with established goals but may lack support for those ambivalent towards behavior change.

Use of automated conversational agents in improving young population mental health: a scoping review

Automated conversational agents (CAs) emerged as a promising solution in mental health interventions among young people. Therefore, the objective of this scoping review is to examine the current state of research into fully automated CAs mediated interventions for the emotional component of mental health among young people. Selected databases were searched in March 2023. Included studies were primary research, reporting on development, feasibility/usability, or evaluation of fully automated CAs as a tool to improve the emotional component of mental health among young population. Twenty-five studies were included (N = 1707). Most automated CAs applications were standalone preventions targeting anxiety and depression. Automated CAs were predominantly AI-based chatbots, using text as the main communication channel. Overall, the results of the current scoping review showed that automated CAs mediated interventions for emotional problems are acceptable, engaging and with high usability. However, the results for clinical efficacy are far less conclusive, since almost half of evaluation studies reported no significant effect on emotional mental health outcomes. Based on these findings, it can be concluded that there is a pressing need to improve the existing automated CAs applications to increase their efficacy as well as conducting more rigorous methodological research in this area.

A Microlearning-Based Self-directed Learning Chatbot on Medication Administration for New Nurses: A Feasibility Study

New nurses must acquire accurate knowledge of medication administration, as it directly affects patient safety. This study aimed to develop a microlearning-based self-directed learning chatbot on medication administration for novice nurses. Furthermore, the study had the objective of evaluating the chatbot feasibility. The chatbot covered two main topics: medication administration processes and drug-specific management, along with 21 subtopics. Fifty-eight newly hired nurses on standby were asked to use the chatbot over a 2-week period. Moreover, we evaluated the chatbot's feasibility through a survey that gauged changes in their confidence in medication administration knowledge, intrinsic learning motivation, satisfaction with the chatbot's learning content, and usability. After using the chatbot, participants' confidence in medication administration knowledge significantly improved in all topics (P < .001) except "Understanding a concept of 5Right" (P = .077). Their intrinsic learning motivation, satisfaction with the learning content, and usability scored above 5 out of 7 in all subdomains, except for pressure/tension (mean, 2.12; median, 1.90). They scored highest on ease of learning (mean, 6.69; median, 7.00). A microlearning-based chatbot can help new nurses improve their knowledge of medication administration through self-directed learning.

Human cues in eHealth to promote lifestyle change: An experimental field study to examine adherence to self-help interventions

eHealth lifestyle interventions without human support (self-help interventions) are generally less effective, as they suffer from lower adherence levels. To solve this, we investigated whether (1) using a text-based conversational agent (TCA) and applying human cues contribute to a working alliance with the TCA, and whether (2) adding human cues and establishing a positive working alliance increase intervention adherence. Participants (N = 121) followed a TCA-supported app-based physical activity intervention. We manipulated two types of human cues: visual (ie, message appearance) and relational (ie, message content). We employed a 2 (visual cues: yes, no) x 2 (relational cues: yes, no) between-subjects design, resulting in four experimental groups: (1) visual and relational cues, (2) visual cues only, (3) relational cues only, or (4) no human cues. We measured the working alliance with the Working Alliance Inventory Short Revised form and intervention adherence as the number of days participants responded to the TCA's messages. Contrary to expectations, the working alliance was unaffected by using human cues. Working alliance was positively related to adherence (t(78) = 3.606, p = .001). Furthermore, groups who received visual cues showed lower adherence levels compared to those who received relational cues only or no cues (U = 1140.5, z = -3.520, p < .001). We replicated the finding that establishing a working alliance contributes to intervention adherence, independently of the use of human cues in a TCA. However, we were unable to show that adding human cues impacted the working alliance and increased adherence. The results indicate that adding visual cues to a TCA may even negatively affect adherence, possibly because it may create confusion concerning the true nature of the coach, which may prompt unrealistic expectations.

Conversational Agents in Palliative Care: Potential Benefits, Risks, and Next Steps

Conversational agents (sometimes called chatbots) are technology-based systems that use artificial intelligence to simulate human-to-human conversations. Research on conversational agents in health care is nascent but growing, with recent reviews highlighting the need for more robust evaluations in diverse settings and populations. In this article, we consider how conversational agents might function in palliative care-not by replacing clinicians, but by interacting with patients around select uncomplicated needs while facilitating more targeted and appropriate referrals to specialty palliative care services. We describe potential roles for conversational agents aligned with the core domains of quality palliative care and identify risks that must be considered and addressed in the development and use of these systems for people with serious illness. With careful consideration of risks and benefits, conversational agents represent promising tools that should be explored as one component of a multipronged approach for improving patient and family outcomes in serious illness.

Attrition in Conversational Agent-Delivered Mental Health Interventions: Systematic Review and Meta-Analysis

BACKGROUND

Conversational agents (CAs) or chatbots are computer programs that mimic human conversation. They have the potential to improve access to mental health interventions through automated, scalable, and personalized delivery of psychotherapeutic content. However, digital health interventions, including those delivered by CAs, often have high attrition rates. Identifying the factors associated with attrition is critical to improving future clinical trials.

OBJECTIVE

This review aims to estimate the overall and differential rates of attrition in CA-delivered mental health interventions (CA interventions), evaluate the impact of study design and intervention-related aspects on attrition, and describe study design features aimed at reducing or mitigating study attrition.

METHODS

We searched PubMed, Embase (Ovid), PsycINFO (Ovid), Cochrane Central Register of Controlled Trials, and Web of Science, and conducted a gray literature search on Google Scholar in June 2022. We included randomized controlled trials that compared CA interventions against control groups and excluded studies that lasted for 1 session only and used Wizard of Oz interventions. We also assessed the risk of bias in the included studies using the Cochrane Risk of Bias Tool 2.0. Random-effects proportional meta-analysis was applied to calculate the pooled dropout rates in the intervention groups. Random-effects meta-analysis was used to compare the attrition rate in the intervention groups with that in the control groups. We used a narrative review to summarize the findings.

RESULTS

The systematic search retrieved 4566 records from peer-reviewed databases and citation searches, of which 41 (0.90%) randomized controlled trials met the inclusion criteria. The meta-analytic overall attrition rate in the intervention group was 21.84% (95% CI 16.74%-27.36%; I2=94%). Short-term studies that lasted ≤8 weeks showed a lower attrition rate (18.05%, 95% CI 9.91%- 27.76%; I2=94.6%) than long-term studies that lasted >8 weeks (26.59%, 95% CI 20.09%-33.63%; I2=93.89%). Intervention group participants were more likely to attrit than control group participants for short-term (log odds ratio 1.22, 95% CI 0.99-1.50; I2=21.89%) and long-term studies (log odds ratio 1.33, 95% CI 1.08-1.65; I2=49.43%). Intervention-related characteristics associated with higher attrition include stand-alone CA interventions without human support, not having a symptom tracker feature, no visual representation of the CA, and comparing CA interventions with waitlist controls. No participant-level factor reliably predicted attrition.

CONCLUSIONS

Our results indicated that approximately one-fifth of the participants will drop out from CA interventions in short-term studies. High heterogeneities made it difficult to generalize the findings. Our results suggested that future CA interventions should adopt a blended design with human support, use symptom tracking, compare CA intervention groups against active controls rather than waitlist controls, and include a visual representation of the CA to reduce the attrition rate.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42022341415; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022341415.

Virtual Assistants' Response to Queries About Nicotine Replacement Therapy: A Mixed-Method Analysis

This study focused on investigating the potential of Artificial Intelligent-powered Virtual Assistants (VAs) such as Amazon Alexa, Apple Siri, and Google Assistant as tools to help individuals seeking information about Nicotine Replacement Treatment (NRT) for smoking cessation. The researchers asked 40 NRT-related questions to each of the 3 VAs and evaluated the responses for voice recognition. The study used a cross-sectional mixed-method design with a total sample size of 360 responses. Inter-rater reliability and differences between VAs' responses were examined by SAS software, and qualitative assessments were conducted using NVivo software. Google Assistant achieved 100% voice recognition for NRT-related questions, followed by Apple Siri at 97.5%, and Amazon Alexa at 83.3%. Statistically significant differences were found between the responses of Amazon Alexa relative to both Google Assistant and Apple Siri. Researcher 1's ratings significantly differed from Researcher 2's (p = .001), but not from Researcher 3's (p = .11). Virtual Assistants occasionally struggled to understand the context or nuances of questions, lacked in-depth information in their responses, and provided generic or unrelated responses. Virtual Assistants have the potential to be incorporated into smoking cessation interventions and tobacco control initiatives, contingent upon improving their competencies.

Evaluation framework for conversational agents with artificial intelligence in health interventions: a systematic scoping review

OBJECTIVES

Conversational agents (CAs) with emerging artificial intelligence present new opportunities to assist in health interventions but are difficult to evaluate, deterring their applications in the real world. We aimed to synthesize existing evidence and knowledge and outline an evaluation framework for CA interventions.

MATERIALS AND METHODS

We conducted a systematic scoping review to investigate designs and outcome measures used in the studies that evaluated CAs for health interventions. We then nested the results into an overarching digital health framework proposed by the World Health Organization (WHO).

RESULTS

The review included 81 studies evaluating CAs in experimental (n = 59), observational (n = 15) trials, and other research designs (n = 7). Most studies (n = 72, 89%) were published in the past 5 years. The proposed CA-evaluation framework includes 4 evaluation stages: (1) feasibility/usability, (2) efficacy, (3) effectiveness, and (4) implementation, aligning with WHO's stepwise evaluation strategy. Across these stages, this article presents the essential evidence of different study designs (n = 8), sample sizes, and main evaluation categories (n = 7) with subcategories (n = 40). The main evaluation categories included (1) functionality, (2) safety and information quality, (3) user experience, (4) clinical and health outcomes, (5) costs and cost benefits, (6) usage, adherence, and uptake, and (7) user characteristics for implementation research. Furthermore, the framework highlighted the essential evaluation areas (potential primary outcomes) and gaps across the evaluation stages.

DISCUSSION AND CONCLUSION

This review presents a new framework with practical design details to support the evaluation of CA interventions in healthcare research.

PROTOCOL REGISTRATION

The Open Science Framework (https://osf.io/9hq2v) on March 22, 2021.

Barriers and facilitators to the implementation of family cancer history collection tools in oncology clinical practices

INTRODUCTION

This study aimed to identify barriers and facilitators to the implementation of family cancer history (FCH) collection tools in clinical practices and community settings by assessing clinicians' perceptions of implementing a chatbot interface to collect FCH information and provide personalized results to patients and providers.

OBJECTIVES

By identifying design and implementation features that facilitate tool adoption and integration into clinical workflows, this study can inform future FCH tool development and adoption in healthcare settings.

MATERIALS AND METHODS

Quantitative data were collected using survey to evaluate the implementation outcomes of acceptability, adoption, appropriateness, feasibility, and sustainability of the chatbot tool for collecting FCH. Semistructured interviews were conducted to gather qualitative data on respondents' experiences using the tool and recommendations for enhancements.

RESULTS

We completed data collection with 19 providers (n = 9, 47%), clinical staff (n = 5, 26%), administrators (n = 4, 21%), and other staff (n = 1, 5%) affiliated with the NCI Community Oncology Research Program. FCH was systematically collected using a wide range of tools at sites, with information being inserted into the patient's medical record. Participants found the chatbot tool to be highly acceptable, with the tool aligning with existing workflows, and were open to adopting the tool into their practice.

DISCUSSION AND CONCLUSIONS

We further the evidence base about the appropriateness of scripted chatbots to support FCH collection. Although the tool had strong support, the varying clinical workflows across clinic sites necessitate that future FCH tool development accommodates customizable implementation strategies. Implementation support is necessary to overcome technical and logistical barriers to enhance the uptake of FCH tools in clinical practices and community settings.

Implementation of Chatbot Technology in Health Care: Protocol for a Bibliometric Analysis

BACKGROUND

Chatbots have the potential to increase people's access to quality health care. However, the implementation of chatbot technology in the health care system is unclear due to the scarce analysis of publications on the adoption of chatbot in health and medical settings.

OBJECTIVE

This paper presents a protocol of a bibliometric analysis aimed at offering the public insights into the current state and emerging trends in research related to the use of chatbot technology for promoting health.

METHODS

In this bibliometric analysis, we will select published papers from the databases of CINAHL, IEEE Xplore, PubMed, Scopus, and Web of Science that pertain to chatbot technology and its applications in health care. Our search strategy includes keywords such as "chatbot," "virtual agent," "virtual assistant," "conversational agent," "conversational AI," "interactive agent," "health," and "healthcare." Five researchers who are AI engineers and clinicians will independently review the titles and abstracts of selected papers to determine their eligibility for a full-text review. The corresponding author (ZN) will serve as a mediator to address any discrepancies and disputes among the 5 reviewers. Our analysis will encompass various publication patterns of chatbot research, including the number of annual publications, their geographic or institutional distribution, and the number of annual grants supporting chatbot research, and further summarize the methodologies used in the development of health-related chatbots, along with their features and applications in health care settings. Software tool VOSViewer (version 1.6.19; Leiden University) will be used to construct and visualize bibliometric networks.

RESULTS

The preparation for the bibliometric analysis began on December 3, 2021, when the research team started the process of familiarizing themselves with the software tools that may be used in this analysis, VOSViewer and CiteSpace, during which they consulted 3 librarians at the Yale University regarding search terms and tentative results. Tentative searches on the aforementioned databases yielded a total of 2340 papers. The official search phase started on July 27, 2023. Our goal is to complete the screening of papers and the analysis by February 15, 2024.

CONCLUSIONS

Artificial intelligence chatbots, such as ChatGPT (OpenAI Inc), have sparked numerous discussions within the health care industry regarding their impact on human health. Chatbot technology holds substantial promise for advancing health care systems worldwide. However, developing a sophisticated chatbot capable of precise interaction with health care consumers, delivering personalized care, and providing accurate health-related information and knowledge remain considerable challenges. This bibliometric analysis seeks to fill the knowledge gap in the existing literature on health-related chatbots, entailing their applications, the software used in their development, and their preferred functionalities among users.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/54349.

The effect of the intensive care unit nurse manpower on care behaviours and stress level on the nurses

BACKGROUND

Nurses working in the intensive care units (ICU) regarding the work-index-nursing work environment, the effect level ICU environment has on the nurses' care behaviors and stress levels of the nurses should be determined.

OBJECTIVE

The study aimed to investigate the effect of the nurse manpower on care behaviours and stress level of the nurses working in the ICU.

METHODS

This was a cross-sectional and correlational study. The sample of the study consisted of 123 nurses working in the ICUs. The data were collected using the survey form, Distress Thermometer (DT), The Practice Work Environment Scale of the Nursing Work Index (PES-NWI), and Caring Behaviors Scale-24 (CBS-24).

RESULTS

The mean age of nurses in the ICU was 30.2±5.6 and the mean of working time in the intensive care unit of nurses in the ICU was 3.7±3.1 years. The mean of the DT was 4.8±3.4, and the mean score of PES-NWI was 2.6±1.0 and the mean score of CBS-24 was 4.7±1.1 in nurses in the ICU. The regression model which was studied to investigate the relationship between caring behaviors and stress and nurse manpower of nurses working in intensive care unit was significant.

CONCLUSION

Care behaviors and stress levels of nurses working in intensive care units are negatively affected by insufficient nurse manpower.

Closing the accessibility gap to mental health treatment with a personalized self-referral chatbot

Inequality in treatment access is a pressing issue in most healthcare systems across many medical disciplines. In mental healthcare, reduced treatment access for minorities is ubiquitous but remedies are sparse. Here we demonstrate that digital tools can reduce the accessibility gap by addressing several key barriers. In a multisite observational study of 129,400 patients within England's NHS services, we evaluated the impact of a personalized artificial intelligence-enabled self-referral chatbot on patient referral volume and diversity in ethnicity, gender and sexual orientation. We found that services that used this digital solution identified substantially increased referrals (15% increase versus 6% increase in control services). Critically, this increase was particularly pronounced in minorities, such as nonbinary (179% increase) and ethnic minority individuals (29% increase). Using natural language processing to analyze qualitative feedback from 42,332 individuals, we found that the chatbot's human-free nature and the patients' self-realization of their need for treatment were potential drivers for the observed improvement in the diversity of access. This provides strong evidence that digital tools may help overcome the pervasive inequality in mental healthcare.

Development and Evaluation of a Smartphone-Based Chatbot Coach to Facilitate a Balanced Lifestyle in Individuals With Headaches (BalanceUP App): Randomized Controlled Trial

BACKGROUND

Primary headaches, including migraine and tension-type headaches, are widespread and have a social, physical, mental, and economic impact. Among the key components of treatment are behavior interventions such as lifestyle modification. Scalable conversational agents (CAs) have the potential to deliver behavior interventions at a low threshold. To our knowledge, there is no evidence of behavioral interventions delivered by CAs for the treatment of headaches.

OBJECTIVE

This study has 2 aims. The first aim was to develop and test a smartphone-based coaching intervention (BalanceUP) for people experiencing frequent headaches, delivered by a CA and designed to improve mental well-being using various behavior change techniques. The second aim was to evaluate the effectiveness of BalanceUP by comparing the intervention and waitlist control groups and assess the engagement and acceptance of participants using BalanceUP.

METHODS

In an unblinded randomized controlled trial, adults with frequent headaches were recruited on the web and in collaboration with experts and allocated to either a CA intervention (BalanceUP) or a control condition. The effects of the treatment on changes in the primary outcome of the study, that is, mental well-being (as measured by the Patient Health Questionnaire Anxiety and Depression Scale), and secondary outcomes (eg, psychosomatic symptoms, stress, headache-related self-efficacy, intention to change behavior, presenteeism and absenteeism, and pain coping) were analyzed using linear mixed models and Cohen d. Primary and secondary outcomes were self-assessed before and after the intervention, and acceptance was assessed after the intervention. Engagement was measured during the intervention using self-reports and usage data.

RESULTS

A total of 198 participants (mean age 38.7, SD 12.14 y; n=172, 86.9% women) participated in the study (intervention group: n=110; waitlist control group: n=88). After the intervention, the intention-to-treat analysis revealed evidence for improved well-being (treatment: β estimate=-3.28, 95% CI -5.07 to -1.48) with moderate between-group effects (Cohen d=-0.66, 95% CI -0.99 to -0.33) in favor of the intervention group. We also found evidence of reduced somatic symptoms, perceived stress, and absenteeism and presenteeism, as well as improved headache management self-efficacy, application of behavior change techniques, and pain coping skills, with effects ranging from medium to large (Cohen d=0.43-1.05). Overall, 64.8% (118/182) of the participants used coaching as intended by engaging throughout the coaching and completing the outro.

CONCLUSIONS

BalanceUP was well accepted, and the results suggest that coaching delivered by a CA can be effective in reducing the burden of people who experience headaches by improving their well-being.

TRIAL REGISTRATION

German Clinical Trials Register DRKS00017422; https://trialsearch.who.int/Trial2.aspx?TrialID=DRKS00017422.

Consent-GPT: is it ethical to delegate procedural consent to conversational AI?

Obtaining informed consent from patients prior to a medical or surgical procedure is a fundamental part of safe and ethical clinical practice. Currently, it is routine for a significant part of the consent process to be delegated to members of the clinical team not performing the procedure (eg, junior doctors). However, it is common for consent-taking delegates to lack sufficient time and clinical knowledge to adequately promote patient autonomy and informed decision-making. Such problems might be addressed in a number of ways. One possible solution to this clinical dilemma is through the use of conversational artificial intelligence using large language models (LLMs). There is considerable interest in the potential benefits of such models in medicine. For delegated procedural consent, LLM could improve patients' access to the relevant procedural information and therefore enhance informed decision-making.In this paper, we first outline a hypothetical example of delegation of consent to LLMs prior to surgery. We then discuss existing clinical guidelines for consent delegation and some of the ways in which current practice may fail to meet the ethical purposes of informed consent. We outline and discuss the ethical implications of delegating consent to LLMs in medicine concluding that at least in certain clinical situations, the benefits of LLMs potentially far outweigh those of current practices.

Patient Acceptability of Symptom Screening and Patient Education Using a Chatbot for Autoimmune Inflammatory Diseases: Survey Study

BACKGROUND

Chatbots have the potential to enhance health care interaction, satisfaction, and service delivery. However, data regarding their acceptance across diverse patient populations are limited. In-depth studies on the reception of chatbots by patients with chronic autoimmune inflammatory diseases are lacking, although such studies are vital for facilitating the effective integration of chatbots in rheumatology care.

OBJECTIVE

We aim to assess patient perceptions and acceptance of a chatbot designed for autoimmune inflammatory rheumatic diseases (AIIRDs).

METHODS

We administered a comprehensive survey in an outpatient setting at a top-tier rheumatology referral center. The target cohort included patients who interacted with a chatbot explicitly tailored to facilitate diagnosis and obtain information on AIIRDs. Following the RE-AIM (Reach, Effectiveness, Adoption, Implementation and Maintenance) framework, the survey was designed to gauge the effectiveness, user acceptability, and implementation of the chatbot.

RESULTS

Between June and October 2022, we received survey responses from 200 patients, with an equal number of 100 initial consultations and 100 follow-up (FU) visits. The mean scores on a 5-point acceptability scale ranged from 4.01 (SD 0.63) to 4.41 (SD 0.54), indicating consistently high ratings across the different aspects of chatbot performance. Multivariate regression analysis indicated that having a FU visit was significantly associated with a greater willingness to reuse the chatbot for symptom determination (P=.01). Further, patients' comfort with chatbot diagnosis increased significantly after meeting physicians (P<.001). We observed no significant differences in chatbot acceptance according to sex, education level, or diagnosis category.

CONCLUSIONS

This study underscores that chatbots tailored to AIIRDs have a favorable reception. The inclination of FU patients to engage with the chatbot signifies the possible influence of past clinical encounters and physician affirmation on its use. Although further exploration is required to refine their integration, the prevalent positive perceptions suggest that chatbots have the potential to strengthen the bridge between patients and health care providers, thus enhancing the delivery of rheumatology care to various cohorts.

Using Conversational AI to Facilitate Mental Health Assessments and Improve Clinical Efficiency Within Psychotherapy Services: Real-World Observational Study

BACKGROUND

Most mental health care providers face the challenge of increased demand for psychotherapy in the absence of increased funding or staffing. To overcome this supply-demand imbalance, care providers must increase the efficiency of service delivery.

OBJECTIVE

In this study, we examined whether artificial intelligence (AI)-enabled digital solutions can help mental health care practitioners to use their time more efficiently, and thus reduce strain on services and improve patient outcomes.

METHODS

In this study, we focused on the use of an AI solution (Limbic Access) to support initial patient referral and clinical assessment within the UK's National Health Service. Data were collected from 9 Talking Therapies services across England, comprising 64,862 patients.

RESULTS

We showed that the use of this AI solution improves clinical efficiency by reducing the time clinicians spend on mental health assessments. Furthermore, we found improved outcomes for patients using the AI solution in several key metrics, such as reduced wait times, reduced dropout rates, improved allocation to appropriate treatment pathways, and, most importantly, improved recovery rates. When investigating the mechanism by which the AI solution achieved these improvements, we found that the provision of clinically relevant information ahead of clinical assessment was critical for these observed effects.

CONCLUSIONS

Our results emphasize the utility of using AI solutions to support the mental health workforce, further highlighting the potential of AI solutions to increase the efficiency of care delivery and improve clinical outcomes for patients.

Chatbots for HIV Prevention and Care: a Narrative Review

PURPOSE OF REVIEW

To explore the intersection of chatbots and HIV prevention and care. Current applications of chatbots in HIV services, the challenges faced, recent advancements, and future research directions are presented and discussed.

RECENT FINDINGS

Chatbots facilitate sensitive discussions about HIV thereby promoting prevention and care strategies. Trustworthiness and accuracy of information were identified as primary factors influencing user engagement with chatbots. Additionally, the integration of AI-driven models that process and generate human-like text into chatbots poses both breakthroughs and challenges in terms of privacy, bias, resources, and ethical issues. Chatbots in HIV prevention and care show potential; however, significant work remains in addressing associated ethical and practical concerns. The integration of large language models into chatbots is a promising future direction for their effective deployment in HIV services. Encouraging future research, collaboration among stakeholders, and bold innovative thinking will be pivotal in harnessing the full potential of chatbot interventions.

The Intersection of ChatGPT, Clinical Medicine, and Medical Education

As we progress deeper into the digital age, the robust development and application of advanced artificial intelligence (AI) technology, specifically generative language models like ChatGPT (OpenAI), have potential implications in all sectors including medicine. This viewpoint article aims to present the authors' perspective on the integration of AI models such as ChatGPT in clinical medicine and medical education. The unprecedented capacity of ChatGPT to generate human-like responses, refined through Reinforcement Learning with Human Feedback, could significantly reshape the pedagogical methodologies within medical education. Through a comprehensive review and the authors' personal experiences, this viewpoint article elucidates the pros, cons, and ethical considerations of using ChatGPT within clinical medicine and notably, its implications for medical education. This exploration is crucial in a transformative era where AI could potentially augment human capability in the process of knowledge creation and dissemination, potentially revolutionizing medical education and clinical practice. The importance of maintaining academic integrity and professional standards is highlighted. The relevance of establishing clear guidelines for the responsible and ethical use of AI technologies in clinical medicine and medical education is also emphasized.

AI Chatbots in Clinical Laboratory Medicine: Foundations and Trends

BACKGROUND

Artificial intelligence (AI) conversational agents, or chatbots, are computer programs designed to simulate human conversations using natural language processing. They offer diverse functions and applications across an expanding range of healthcare domains. However, their roles in laboratory medicine remain unclear, as their accuracy, repeatability, and ability to interpret complex laboratory data have yet to be rigorously evaluated.

CONTENT

This review provides an overview of the history of chatbots, two major chatbot development approaches, and their respective advantages and limitations. We discuss the capabilities and potential applications of chatbots in healthcare, focusing on the laboratory medicine field. Recent evaluations of chatbot performance are presented, with a special emphasis on large language models such as the Chat Generative Pre-trained Transformer in response to laboratory medicine questions across different categories, such as medical knowledge, laboratory operations, regulations, and interpretation of laboratory results as related to clinical context. We analyze the causes of chatbots' limitations and suggest research directions for developing more accurate, reliable, and manageable chatbots for applications in laboratory medicine.

SUMMARY

Chatbots, which are rapidly evolving AI applications, hold tremendous potential to improve medical education, provide timely responses to clinical inquiries concerning laboratory tests, assist in interpreting laboratory results, and facilitate communication among patients, physicians, and laboratorians. Nevertheless, users should be vigilant of existing chatbots' limitations, such as misinformation, inconsistencies, and lack of human-like reasoning abilities. To be effectively used in laboratory medicine, chatbots must undergo extensive training on rigorously validated medical knowledge and be thoroughly evaluated against standard clinical practice.

Conversational Agents in Health Care: Expert Interviews to Inform the Definition, Classification, and Conceptual Framework

BACKGROUND

Conversational agents (CAs), or chatbots, are computer programs that simulate conversations with humans. The use of CAs in health care settings is recent and rapidly increasing, which often translates to poor reporting of the CA development and evaluation processes and unreliable research findings. We developed and published a conceptual framework, designing, developing, evaluating, and implementing a smartphone-delivered, rule-based conversational agent (DISCOVER), consisting of 3 iterative stages of CA design, development, and evaluation and implementation, complemented by 2 cross-cutting themes (user-centered design and data privacy and security).

OBJECTIVE

This study aims to perform in-depth, semistructured interviews with multidisciplinary experts in health care CAs to share their views on the definition and classification of health care CAs and evaluate and validate the DISCOVER conceptual framework.

METHODS

We conducted one-on-one semistructured interviews via Zoom (Zoom Video Communications) with 12 multidisciplinary CA experts using an interview guide based on our framework. The interviews were audio recorded, transcribed by the research team, and analyzed using thematic analysis.

RESULTS

Following participants' input, we defined CAs as digital interfaces that use natural language to engage in a synchronous dialogue using ≥1 communication modality, such as text, voice, images, or video. CAs were classified by 13 categories: response generation method, input and output modalities, CA purpose, deployment platform, CA development modality, appearance, length of interaction, type of CA-user interaction, dialogue initiation, communication style, CA personality, human support, and type of health care intervention. Experts considered that the conceptual framework could be adapted for artificial intelligence-based CAs. However, despite recent advances in artificial intelligence, including large language models, the technology is not able to ensure safety and reliability in health care settings. Finally, aligned with participants' feedback, we present an updated iteration of the conceptual framework for health care conversational agents (CHAT) with key considerations for CA design, development, and evaluation and implementation, complemented by 3 cross-cutting themes: ethics, user involvement, and data privacy and security.

CONCLUSIONS

We present an expanded, validated CHAT and aim at guiding researchers from a variety of backgrounds and with different levels of expertise in the design, development, and evaluation and implementation of rule-based CAs in health care settings.

Factors Affecting the Adoption of Artificial Intelligence-Enabled Virtual Assistants for Leukemia Self-Management

AIM AND PURPOSE

The purpose of this study is to analyze the various influencing factors affecting the adoption of artificial intelligence (AI)-enabled virtual assistants (VAs) for self-management of leukemia.

METHODS

A cross-sectional survey design is adopted in this study. The questionnaire included eight factors (performance expectancy, effort expectancy, social influence, facilitating conditions, behavioral intention, trust, perceived privacy risk, and personal innovativeness) affecting the acceptance of AI-enabled virtual assistants. A total of 397 leukemia patients participated in the online survey.

RESULTS

Performance expectancy (μ = 3.14), effort expectancy (μ = 3.05), and personal innovativeness (μ = 3.14) were identified to be the major influencing factors of AI adoption. Statistically significant differences (p < .05) were observed between the gender-based and age groups of the participants in relation to the various factors. In addition, perceived privacy risks were negatively correlated with all other factors.

CONCLUSION

Although there are negative factors such as privacy risks and ethical issues in AI adoption, perceived effectiveness and ease of use among individuals are leading to greater adoption of AI-enabled VAs.

Acceptability of a Pain History Assessment and Education Chatbot (Dolores) Across Age Groups in Populations With Chronic Pain: Development and Pilot Testing

BACKGROUND

The delivery of education on pain neuroscience and the evidence for different treatment approaches has become a key component of contemporary persistent pain management. Chatbots, or more formally conversation agents, are increasingly being used in health care settings due to their versatility in providing interactive and individualized approaches to both capture and deliver information. Research focused on the acceptability of diverse chatbot formats can assist in developing a better understanding of the educational needs of target populations.

OBJECTIVE

This study aims to detail the development and initial pilot testing of a multimodality pain education chatbot (Dolores) that can be used across different age groups and investigate whether acceptability and feedback were comparable across age groups following pilot testing.

METHODS

Following an initial design phase involving software engineers (n=2) and expert clinicians (n=6), a total of 60 individuals with chronic pain who attended an outpatient clinic at 1 of 2 pain centers in Australia were recruited for pilot testing. The 60 individuals consisted of 20 (33%) adolescents (aged 10-18 years), 20 (33%) young adults (aged 19-35 years), and 20 (33%) adults (aged >35 years) with persistent pain. Participants spent 20 to 30 minutes completing interactive chatbot activities that enabled the Dolores app to gather a pain history and provide education about pain and pain treatments. After the chatbot activities, participants completed a custom-made feedback questionnaire measuring the acceptability constructs pertaining to health education chatbots. To determine the effect of age group on the acceptability ratings and feedback provided, a series of binomial logistic regression models and cumulative odds ordinal logistic regression models with proportional odds were generated.

RESULTS

Overall, acceptability was high for the following constructs: engagement, perceived value, usability, accuracy, responsiveness, adoption intention, esthetics, and overall quality. The effect of age group on all acceptability ratings was small and not statistically significant. An analysis of open-ended question responses revealed that major frustrations with the app were related to Dolores' speech, which was explored further through a comparative analysis. With respect to providing negative feedback about Dolores' speech, a logistic regression model showed that the effect of age group was statistically significant (χ22=11.7; P=.003) and explained 27.1% of the variance (Nagelkerke R2). Adults and young adults were less likely to comment on Dolores' speech compared with adolescent participants (odds ratio 0.20, 95% CI 0.05-0.84 and odds ratio 0.05, 95% CI 0.01-0.43, respectively). Comments were related to both speech rate (too slow) and quality (unpleasant and robotic).

CONCLUSIONS

This study provides support for the acceptability of pain history and education chatbots across different age groups. Chatbot acceptability for adolescent cohorts may be improved by enabling the self-selection of speech characteristics such as rate and personable tone.

Conversational AI and Vaccine Communication: Systematic Review of the Evidence

BACKGROUND

Since the mid-2010s, use of conversational artificial intelligence (AI; chatbots) in health care has expanded significantly, especially in the context of increased burdens on health systems and restrictions on in-person consultations with health care providers during the COVID-19 pandemic. One emerging use for conversational AI is to capture evolving questions and communicate information about vaccines and vaccination.

OBJECTIVE

The objective of this systematic review was to examine documented uses and evidence on the effectiveness of conversational AI for vaccine communication.

METHODS

This systematic review was conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. PubMed, Web of Science, PsycINFO, MEDLINE, Scopus, CINAHL Complete, Cochrane Library, Embase, Epistemonikos, Global Health, Global Index Medicus, Academic Search Complete, and the University of London library database were searched for papers on the use of conversational AI for vaccine communication. The inclusion criteria were studies that included (1) documented instances of conversational AI being used for the purpose of vaccine communication and (2) evaluation data on the impact and effectiveness of the intervention.

RESULTS

After duplicates were removed, the review identified 496 unique records, which were then screened by title and abstract, of which 38 were identified for full-text review. Seven fit the inclusion criteria and were assessed and summarized in the findings of this review. Overall, vaccine chatbots deployed to date have been relatively simple in their design and have mainly been used to provide factual information to users in response to their questions about vaccines. Additionally, chatbots have been used for vaccination scheduling, appointment reminders, debunking misinformation, and, in some cases, for vaccine counseling and persuasion. Available evidence suggests that chatbots can have a positive effect on vaccine attitudes; however, studies were typically exploratory in nature, and some lacked a control group or had very small sample sizes.

CONCLUSIONS

The review found evidence of potential benefits from conversational AI for vaccine communication. Factors that may contribute to the effectiveness of vaccine chatbots include their ability to provide credible and personalized information in real time, the familiarity and accessibility of the chatbot platform, and the extent to which interactions with the chatbot feel "natural" to users. However, evaluations have focused on the short-term, direct effects of chatbots on their users. The potential longer-term and societal impacts of conversational AI have yet to be analyzed. In addition, existing studies do not adequately address how ethics apply in the field of conversational AI around vaccines. In a context where further digitalization of vaccine communication can be anticipated, additional high-quality research will be required across all these areas.

"But can chatbots understand sex?" Attitudes towards artificial intelligence chatbots amongst sexual and reproductive health professionals: An exploratory mixed-methods study

BACKGROUND

Artificial Intelligence (AI)-enabled chatbots can offer anonymous education about sexual and reproductive health (SRH). Understanding chatbot acceptability and feasibility allows the identification of barriers to the design and implementation.

METHODS

In 2020, we conducted an online survey and qualitative interviews with SRH professionals recruited online to explore the views on AI, automation and chatbots. Qualitative data were analysed thematically.

RESULTS

Amongst 150 respondents (48% specialist doctor/consultant), only 22% perceived chatbots as effective and 24% saw them as ineffective for SRH advice [Mean = 2.91, SD = 0.98, range: 1-5]. Overall, there were mixed attitudes towards SRH chatbots [Mean = 4.03, SD = 0.87, range: 1-7]. Chatbots were most acceptable for appointment booking, general sexual health advice and signposting, but not acceptable for safeguarding, virtual diagnosis, and emotional support. Three themes were identified: "Moving towards a 'digital' age'", "AI improving access and service efficacy", and "Hesitancy towards AI".

CONCLUSIONS

Half of SRH professionals were hesitant about the use of chatbots in SRH services, attributed to concerns about patient safety, and lack of familiarity with this technology. Future studies should explore the role of AI chatbots as supplementary tools for SRH promotion. Chatbot designers need to address the concerns of health professionals to increase acceptability and engagement with AI-enabled services.

Factors Influencing the Acceptability, Acceptance, and Adoption of Conversational Agents in Health Care: Integrative Review

BACKGROUND

Conversational agents (CAs), also known as chatbots, are digital dialog systems that enable people to have a text-based, speech-based, or nonverbal conversation with a computer or another machine based on natural language via an interface. The use of CAs offers new opportunities and various benefits for health care. However, they are not yet ubiquitous in daily practice. Nevertheless, research regarding the implementation of CAs in health care has grown tremendously in recent years.

OBJECTIVE

This review aims to present a synthesis of the factors that facilitate or hinder the implementation of CAs from the perspectives of patients and health care professionals. Specifically, it focuses on the early implementation outcomes of acceptability, acceptance, and adoption as cornerstones of later implementation success.

METHODS

We performed an integrative review. To identify relevant literature, a broad literature search was conducted in June 2021 with no date limits and using all fields in PubMed, Cochrane Library, Web of Science, LIVIVO, and PsycINFO. To keep the review current, another search was conducted in March 2022. To identify as many eligible primary sources as possible, we used a snowballing approach by searching reference lists and conducted a hand search. Factors influencing the acceptability, acceptance, and adoption of CAs in health care were coded through parallel deductive and inductive approaches, which were informed by current technology acceptance and adoption models. Finally, the factors were synthesized in a thematic map.

RESULTS

Overall, 76 studies were included in this review. We identified influencing factors related to 4 core Unified Theory of Acceptance and Use of Technology (UTAUT) and Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) factors (performance expectancy, effort expectancy, facilitating conditions, and hedonic motivation), with most studies underlining the relevance of performance and effort expectancy. To meet the particularities of the health care context, we redefined the UTAUT2 factors social influence, habit, and price value. We identified 6 other influencing factors: perceived risk, trust, anthropomorphism, health issue, working alliance, and user characteristics. Overall, we identified 10 factors influencing acceptability, acceptance, and adoption among health care professionals (performance expectancy, effort expectancy, facilitating conditions, social influence, price value, perceived risk, trust, anthropomorphism, working alliance, and user characteristics) and 13 factors influencing acceptability, acceptance, and adoption among patients (additionally hedonic motivation, habit, and health issue).

CONCLUSIONS

This review shows manifold factors influencing the acceptability, acceptance, and adoption of CAs in health care. Knowledge of these factors is fundamental for implementation planning. Therefore, the findings of this review can serve as a basis for future studies to develop appropriate implementation strategies. Furthermore, this review provides an empirical test of current technology acceptance and adoption models and identifies areas where additional research is necessary.

TRIAL REGISTRATION

PROSPERO CRD42022343690; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=343690.

Can digital health researchers make a difference during the pandemic? Results of the single-arm, chatbot-led Elena+: Care for COVID-19 interventional study

Background

The current paper details findings from Elena+: Care for COVID-19, an app developed to tackle the collateral damage of lockdowns and social distancing, by offering pandemic lifestyle coaching across seven health areas: anxiety, loneliness, mental resources, sleep, diet and nutrition, physical activity, and COVID-19 information.

Methods

The Elena+ app functions as a single-arm interventional study, with participants recruited predominantly via social media. We used paired samples T-tests and within subjects ANOVA to examine changes in health outcome assessments and user experience evaluations over time. To investigate the mediating role of behavioral activation (i.e., users setting behavioral intentions and reporting actual behaviors) we use mixed-effect regression models. Free-text entries were analyzed qualitatively.

Results

Results show strong demand for publicly available lifestyle coaching during the pandemic, with total downloads (N = 7'135) and 55.8% of downloaders opening the app (n = 3,928) with 9.8% completing at least one subtopic (n = 698). Greatest areas of health vulnerability as assessed with screening measures were physical activity with 62% (n = 1,000) and anxiety with 46.5% (n = 760). The app was effective in the treatment of mental health; with a significant decrease in depression between first (14 days), second (28 days), and third (42 days) assessments: F2,38 = 7.01, p = 0.003, with a large effect size (η2G = 0.14), and anxiety between first and second assessments: t54 = 3.7, p = <0.001 with a medium effect size (Cohen d = 0.499). Those that followed the coaching program increased in net promoter score between the first and second assessment: t36 = 2.08, p = 0.045 with a small to medium effect size (Cohen d = 0.342). Mediation analyses showed that while increasing number of subtopics completed increased behavioral activation (i.e., match between behavioral intentions and self-reported actual behaviors), behavioral activation did not mediate the relationship to improvements in health outcome assessments.

Conclusions

Findings show that: (i) there is public demand for chatbot led digital coaching, (ii) such tools can be effective in delivering treatment success, and (iii) they are highly valued by their long-term user base. As the current intervention was developed at rapid speed to meet the emergency pandemic context, the future looks bright for other public health focused chatbot-led digital health interventions.

Assessing Mood With the Identifying Depression Early in Adolescence Chatbot (IDEABot): Development and Implementation Study

BACKGROUND

Mental health status assessment is mostly limited to clinical or research settings, but recent technological advances provide new opportunities for measurement using more ecological approaches. Leveraging apps already in use by individuals on their smartphones, such as chatbots, could be a useful approach to capture subjective reports of mood in the moment.

OBJECTIVE

This study aimed to describe the development and implementation of the Identifying Depression Early in Adolescence Chatbot (IDEABot), a WhatsApp-based tool designed for collecting intensive longitudinal data on adolescents' mood.

METHODS

The IDEABot was developed to collect data from Brazilian adolescents via WhatsApp as part of the Identifying Depression Early in Adolescence Risk Stratified Cohort (IDEA-RiSCo) study. It supports the administration and collection of self-reported structured items or questionnaires and audio responses. The development explored WhatsApp's default features, such as emojis and recorded audio messages, and focused on scripting relevant and acceptable conversations. The IDEABot supports 5 types of interactions: textual and audio questions, administration of a version of the Short Mood and Feelings Questionnaire, unprompted interactions, and a snooze function. Six adolescents (n=4, 67% male participants and n=2, 33% female participants) aged 16 to 18 years tested the initial version of the IDEABot and were engaged to codevelop the final version of the app. The IDEABot was subsequently used for data collection in the second- and third-year follow-ups of the IDEA-RiSCo study.

RESULTS

The adolescents assessed the initial version of the IDEABot as enjoyable and made suggestions for improvements that were subsequently implemented. The IDEABot's final version follows a structured script with the choice of answer based on exact text matches throughout 15 days. The implementation of the IDEABot in 2 waves of the IDEA-RiSCo sample (140 and 132 eligible adolescents in the second- and third-year follow-ups, respectively) evidenced adequate engagement indicators, with good acceptance for using the tool (113/140, 80.7% and 122/132, 92.4% for second- and third-year follow-up use, respectively), low attrition (only 1/113, 0.9% and 1/122, 0.8%, respectively, failed to engage in the protocol after initial interaction), and high compliance in terms of the proportion of responses in relation to the total number of elicited prompts (12.8, SD 3.5; 91% out of 14 possible interactions and 10.57, SD 3.4; 76% out of 14 possible interactions, respectively).

CONCLUSIONS

The IDEABot is a frugal app that leverages an existing app already in daily use by our target population. It follows a simple rule-based approach that can be easily tested and implemented in diverse settings and possibly diminishes the burden of intensive data collection for participants by repurposing WhatsApp. In this context, the IDEABot appears as an acceptable and potentially scalable tool for gathering momentary information that can enhance our understanding of mood fluctuations and development.

Curious thing, an artificial intelligence (AI)-based conversational agent for COVID-19 patient management

There are no clear guidelines or validated models for artificial intelligence (AI)-based approaches in the monitoring of coronavirus disease 2019 (COVID-19) patients who were isolated in the community, in order to identify early deterioration of their health symptoms. Developed in partnership with Curious Thing (CT), a Sydney-based AI conversational technology, a new care robot technology was introduced in South Western Sydney (SWS) in September 2021 to manage the large numbers of low-to-medium risk patients with a COVID-19 diagnosis and who were isolating at home. The CT interface made contact with patients via their mobile phone, following a locally produced script to obtain information recording physical condition, wellness and support. The care robot has engaged over 6323 patients between 2 September to 14 December 2021. The AI-assisted phone calls effectively identified the patients requiring further support, saved clinician time by monitoring less ailing patients remotely, and enabled them to spend more time on critically ill patients, thus ensuring that service and supply resources could be directed to those at greatest need. Engagement strategies had ensured stakeholders support of this technology to meet clinical and welfare needs of the identified patient group. Feedback from both the patients and healthcare staff was positive and had informed the ongoing formulation of a more patient-centred model of virtual care.

Scope, Characteristics, Behavior Change Techniques, and Quality of Conversational Agents for Mental Health and Well-Being: Systematic Assessment of Apps

BACKGROUND

Mental disorders cause substantial health-related burden worldwide. Mobile health interventions are increasingly being used to promote mental health and well-being, as they could improve access to treatment and reduce associated costs. Behavior change is an important feature of interventions aimed at improving mental health and well-being. There is a need to discern the active components that can promote behavior change in such interventions and ultimately improve users' mental health.

OBJECTIVE

This study systematically identified mental health conversational agents (CAs) currently available in app stores and assessed the behavior change techniques (BCTs) used. We further described their main features, technical aspects, and quality in terms of engagement, functionality, esthetics, and information using the Mobile Application Rating Scale.

METHODS

The search, selection, and assessment of apps were adapted from a systematic review methodology and included a search, 2 rounds of selection, and an evaluation following predefined criteria. We conducted a systematic app search of Apple's App Store and Google Play using 42matters. Apps with CAs in English that uploaded or updated from January 2020 and provided interventions aimed at improving mental health and well-being and the assessment or management of mental disorders were tested by at least 2 reviewers. The BCT taxonomy v1, a comprehensive list of 93 BCTs, was used to identify the specific behavior change components in CAs.

RESULTS

We found 18 app-based mental health CAs. Most CAs had <1000 user ratings on both app stores (12/18, 67%) and targeted several conditions such as stress, anxiety, and depression (13/18, 72%). All CAs addressed >1 mental disorder. Most CAs (14/18, 78%) used cognitive behavioral therapy (CBT). Half (9/18, 50%) of the CAs identified were rule based (ie, only offered predetermined answers) and the other half (9/18, 50%) were artificial intelligence enhanced (ie, included open-ended questions). CAs used 48 different BCTs and included on average 15 (SD 8.77; range 4-30) BCTs. The most common BCTs were 3.3 "Social support (emotional)," 4.1 "Instructions for how to perform a behavior," 11.2 "Reduce negative emotions," and 6.1 "Demonstration of the behavior." One-third (5/14, 36%) of the CAs claiming to be CBT based did not include core CBT concepts.

CONCLUSIONS

Mental health CAs mostly targeted various mental health issues such as stress, anxiety, and depression, reflecting a broad intervention focus. The most common BCTs identified serve to promote the self-management of mental disorders with few therapeutic elements. CA developers should consider the quality of information, user confidentiality, access, and emergency management when designing mental health CAs. Future research should assess the role of artificial intelligence in promoting behavior change within CAs and determine the choice of BCTs in evidence-based psychotherapies to enable systematic, consistent, and transparent development and evaluation of effective digital mental health interventions.

Integrating domain knowledge for biomedical text analysis into deep learning: A survey

The past decade has witnessed an explosion of textual information in the biomedical field. Biomedical texts provide a basis for healthcare delivery, knowledge discovery, and decision-making. Over the same period, deep learning has achieved remarkable performance in biomedical natural language processing, however, its development has been limited by well-annotated datasets and interpretability. To solve this, researchers have considered combining domain knowledge (such as biomedical knowledge graph) with biomedical data, which has become a promising means of introducing more information into biomedical datasets and following evidence-based medicine. This paper comprehensively reviews more than 150 recent literature studies on incorporating domain knowledge into deep learning models to facilitate typical biomedical text analysis tasks, including information extraction, text classification, and text generation. We eventually discuss various challenges and future directions.

Modeling the Conversation with Digital Health Assistants in Adherence Apps: Some Considerations on the Similarities and Differences with Familiar Medical Encounters

Digital health assistants (DHAs) are conversational agents incorporated into health systems' interfaces, exploiting an intuitive interaction format appreciated by the users. At the same time, however, their conversational format can evoke interactional practices typical of health encounters with human doctors that might misguide the users. Awareness of the similarities and differences between novel mediated encounters and more familiar ones helps designers avoid unintended expectations and leverage suitable ones. Focusing on adherence apps, we analytically discuss the structure of DHA-patient encounters against the literature on physician-patient encounters and the specific affordances of DHAs. We synthesize our discussion into a design checklist and add some considerations about DHA with unconstrained natural language interfaces.

Effectiveness and Acceptability of Conversational Agents for Smoking Cessation: A Systematic Review and Meta-analysis

INTRODUCTION

Conversational agents (CAs; computer programs that use artificial intelligence to simulate a conversation with users through natural language) have evolved considerably in recent years to support healthcare by providing autonomous, interactive, and accessible services, making them potentially useful for supporting smoking cessation. We performed a systematic review and meta-analysis to provide an overarching evaluation of their effectiveness and acceptability to inform future development and adoption.

AIMS AND METHODS

PsycInfo, Web of Science, ACM Digital Library, IEEE Xplore, Medline, EMBASE, Communication and Mass Media Complete, and CINAHL Complete were searched for studies examining the use of CAs for smoking cessation. Data from eligible studies were extracted and used for random-effects meta-analyses.

RESULTS

The search yielded 1245 publications with 13 studies eligible for systematic review (total N = 8236) and six studies for random-effects meta-analyses. All studies reported positive effects on cessation-related outcomes. A meta-analysis with randomized controlled trials reporting on abstinence yielded a sample-weighted odds ratio of 1.66 (95% CI = 1.33% to 2.07%, p < .001), favoring CAs over comparison groups. A narrative synthesis of all included studies showed overall high acceptability, while some barriers were identified from user feedback. Overall, included studies were diverse in design with mixed quality, and evidence of publication bias was identified. A lack of theoretical foundations was noted, as well as a clear need for relational communication in future designs.

CONCLUSIONS

The effectiveness and acceptability of CAs for smoking cessation are promising. However, standardization of reporting and designing of the agents is warranted for a more comprehensive evaluation.

IMPLICATIONS

This is the first systematic review to provide insight into the use of CAs to support smoking cessation. Our findings demonstrated initial promise in the effectiveness and user acceptability of these agents. We also identified a lack of theoretical and methodological limitations to improve future study design and intervention delivery.

Conversational Agents for Body Weight Management: Systematic Review

BACKGROUND

Obesity is a public health issue worldwide. Conversational agents (CAs), also frequently called chatbots, are computer programs that simulate dialogue between people. Owing to better accessibility, cost-effectiveness, personalization, and compassionate patient-centered treatments, CAs are expected to have the potential to provide sustainable lifestyle counseling for weight management.

OBJECTIVE

This systematic review aimed to critically summarize and evaluate clinical studies on the effectiveness and feasibility of CAs with unconstrained natural language input for weight management.

METHODS

PubMed, Embase, the Cochrane Library (CENTRAL), PsycINFO, and ACM Digital Library were searched up to December 2022. Studies were included if CAs were used for weight management and had a capability for unconstrained natural language input. No restrictions were imposed on study design, language, or publication type. The quality of the included studies was assessed using the Cochrane risk-of-bias assessment tool or the Critical Appraisal Skills Programme checklist. The extracted data from the included studies were tabulated and narratively summarized as substantial heterogeneity was expected.

RESULTS

In total, 8 studies met the eligibility criteria: 3 (38%) randomized controlled trials and 5 (62%) uncontrolled before-and-after studies. The CAs in the included studies were aimed at behavior changes through education, advice on food choices, or counseling via psychological approaches. Of the included studies, only 38% (3/8) reported a substantial weight loss outcome (1.3-2.4 kg decrease at 12-15 weeks of CA use). The overall quality of the included studies was judged as low.

CONCLUSIONS

The findings of this systematic review suggest that CAs with unconstrained natural language input can be used as a feasible interpersonal weight management intervention by promoting engagement in psychiatric intervention-based conversations simulating treatments by health care professionals, but currently there is a paucity of evidence. Well-designed rigorous randomized controlled trials with larger sample sizes, longer treatment duration, and follow-up focusing on CAs' acceptability, efficacy, and safety are warranted.

The Chatbots Are Invading Us: A Map Point on the Evolution, Applications, Opportunities, and Emerging Problems in the Health Domain

The inclusion of chatbots is potentially disruptive in society, introducing opportunities, but also important implications that need to be addressed on different domains. The aim of this study is to examine chatbots in-depth, by mapping out their technological evolution, current usage, and potential applications, opportunities, and emerging problems within the health domain. The study examined three points of view. The first point of view traces the technological evolution of chatbots. The second point of view reports the fields of application of the chatbots, giving space to the expectations of use and the expected benefits from a cross-domain point of view, also affecting the health domain. The third and main point of view is that of the analysis of the state of use of chatbots in the health domain based on the scientific literature represented by systematic reviews. The overview identified the topics of greatest interest with the opportunities. The analysis revealed the need for initiatives that simultaneously evaluate multiple domains all together in a synergistic way. Concerted efforts to achieve this are recommended. It is also believed to monitor both the process of osmosis between other sectors and the health domain, as well as the chatbots that can create psychological and behavioural problems with an impact on the health domain.

Conversational Agent Interventions for Mental Health Problems: Systematic Review and Meta-analysis of Randomized Controlled Trials

BACKGROUND

Mental health problems are a crucial global public health concern. Owing to their cost-effectiveness and accessibility, conversational agent interventions (CAIs) are promising in the field of mental health care.

OBJECTIVE

This study aims to present a thorough summary of the traits of CAIs available for a range of mental health problems, find evidence of efficacy, and analyze the statistically significant moderators of efficacy via a meta-analysis of randomized controlled trial.

METHODS

Web-based databases (Embase, MEDLINE, PsycINFO, CINAHL, Web of Science, and Cochrane) were systematically searched dated from the establishment of the database to October 30, 2021, and updated to May 1, 2022. Randomized controlled trials comparing CAIs with any other type of control condition in improving depressive symptoms, generalized anxiety symptoms, specific anxiety symptoms, quality of life or well-being, general distress, stress, mental disorder symptoms, psychosomatic disease symptoms, and positive and negative affect were considered eligible. This study followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Data were extracted by 2 independent reviewers, checked by a third reviewer, and pooled using both random effect models and fixed effects models. Hedges g was chosen as the effect size.

RESULTS

Of the 6900 identified records, a total of 32 studies were included, involving 6089 participants. CAIs showed statistically significant short-term effects compared with control conditions in improving depressive symptoms (g=0.29, 95% CI 0.20-0.38), generalized anxiety symptoms (g=0.29, 95% CI 0.21-0.36), specific anxiety symptoms (g=0.47, 95% CI 0.07-0.86), quality of life or well-being (g=0.27, 95% CI 0.16-0.39), general distress (g=0.33, 95% CI 0.20-0.45), stress (g=0.24, 95% CI 0.08-0.41), mental disorder symptoms (g=0.36, 95% CI 0.17-0.54), psychosomatic disease symptoms (g=0.62, 95% CI 0.14-1.11), and negative affect (g=0.28, 95% CI 0.05-0.51). However, the long-term effects of CAIs for the most mental health outcomes were not statistically significant (g=-0.04 to 0.39). Personalization and empathic response were 2 critical facilitators of efficacy. The longer duration of interaction with conversational agents was associated with the larger pooled effect sizes.

CONCLUSIONS

The findings show that CAIs are research-proven interventions that ought to be implemented more widely in mental health care. CAIs are effective and easily acceptable for those with mental health problems. The clinical application of this novel digital technology will conserve human health resources and optimize the allocation of mental health services.

TRIAL REGISTRATION

PROSPERO CRD42022350130; https://tinyurl.com/mvhk6w9p.

Exploring the potential of mobile health interventions to address behavioural risk factors for the prevention of non-communicable diseases in Asian populations: a qualitative study

BACKGROUND

Changing lifestyle patterns over the last decades have seen growing numbers of people in Asia affected by non-communicable diseases and common mental health disorders, including diabetes, cancer, and/or depression. Interventions targeting healthy lifestyle behaviours through mobile technologies, including new approaches such as chatbots, may be an effective, low-cost approach to prevent these conditions. To ensure uptake and engagement with mobile health interventions, however, it is essential to understand the end-users' perspectives on using such interventions. The aim of this study was to explore perceptions, barriers, and facilitators to the use of mobile health interventions for lifestyle behaviour change in Singapore.

METHODS

Six virtual focus group discussions were conducted with a total of 34 participants (mean ± SD; aged 45 ± 3.6 years; 64.7% females). Focus group recordings were transcribed verbatim and analysed using an inductive thematic analysis approach, followed by deductive mapping according to perceptions, barriers, facilitators, mixed factors, or strategies.

RESULTS

Five themes were identified: (i) holistic wellbeing is central to healthy living (i.e., the importance of both physical and mental health); (ii) encouraging uptake of a mobile health intervention is influenced by factors such as incentives and government backing; (iii) trying out a mobile health intervention is one thing, sticking to it long term is another and there are key factors, such as personalisation and ease of use that influence sustained engagement with mobile health interventions; (iv) perceptions of chatbots as a tool to support healthy lifestyle behaviour are influenced by previous negative experiences with chatbots, which might hamper uptake; and (v) sharing health-related data is OK, but with conditions such as clarity on who will have access to the data, how it will be stored, and for what purpose it will be used.

CONCLUSIONS

Findings highlight several factors that are relevant for the development and implementation of mobile health interventions in Singapore and other Asian countries. Recommendations include: (i) targeting holistic wellbeing, (ii) tailoring content to address environment-specific barriers, (iii) partnering with government and/or local (non-profit) institutions in the development and/or promotion of mobile health interventions, (iv) managing expectations regarding the use of incentives, and (iv) identifying potential alternatives or complementary approaches to the use of chatbots, particularly for mental health.

The development of a Chatbot technology to disseminate post-COVID condition information: a descriptive implementation study

BACKGROUND

Post-COVID condition has now affected millions of individuals, resulting in fatigue, neurocognitive symptoms and an impact on daily life. The uncertainty of knowledge around this condition, including its overall prevalence, pathophysiology, and management, along with the growing numbers of affected individuals have created an essential need in information and disease management. This has become even more critical in a time of abundant online misinformation and potential misleading of patients and healthcare professionals.

OBJECTIVE

The RAFAEL platform is an ecosystem created to address the information and management of post-COVID condition, integrating online information, webinars, and chatbot technology to answer to a large number of individuals in a time-limited and resources-limited setting. This paper describes the development and deployment of the RAFAEL platform and chatbot in addressing post-COVID condition in children and adults.

METHODS

The RAFAEL study takes place in Geneva, Switzerland. The RAFAEL platform and chatbot were made available online, and all users were considered participants to this study. The development phase started in December 2020 and consisted of developing the concept, the backend, and the frontend developments as well as beta testing. The specific strategy behind the RAFAEL chatbot balanced an accessible interactive approach with medical safety, aiming to relay correct and verified information in the management of post-COVID condition. Development was followed by deployment with the establishment of partnerships and communication strategies in the French speaking world. The use of the chatbot and the answers provided were continuously monitored by community moderators and healthcare professionals, creating a safe fallback for users.

RESULTS

To date, the RAFAEL chatbot has had 30488 interactions, with n=6471 (79.5%) matching rate and n=1795 positive feedback rate out of the 2 51 users who provided feedback. Overall, 5807 unique users interacted with the chatbot with 5.1 interactions on average per user, and 8061 stories triggered. Use of the chatbot and RAFAEL platform was additionally driven by the monthly thematic webinars as well as communication campaigns, with an average of 250 participants at each webinar. User queries included questions about post-COVID symptoms (n=5612; 69.2%), of which fatigue was the most predominant query (n=1255; 22.4% of symptoms-related stories). Additional queries included questions about consultations (n=598; 7.4%), treatment (n=527; 6.5%), and general information (n=510; 6.3%).

CONCLUSIONS

The RAFAEL chatbot is to our knowledge the first chatbot developed to address post-COVID condition in children and adults. Its innovation lies in the use of a scalable tool to disseminate verified information in a time- and resources-limited environment. Additionally, the use of machine-learning could help professionals gain knowledge about a new condition while concomitantly addressing patients' concerns. Lessons learned from the RAFAEL chatbot will further encourage a participative approach to learning and could potentially be applied to other chronic conditions.

CLINICALTRIAL

Exploring the Perceptions of mHealth Interventions for the Prevention of Common Mental Disorders in University Students in Singapore: Qualitative Study

BACKGROUND

Mental health interventions delivered through mobile health (mHealth) technologies can increase the access to mental health services, especially among university students. The development of mHealth intervention is complex and needs to be context sensitive. There is currently limited evidence on the perceptions, needs, and barriers related to these interventions in the Southeast Asian context.

OBJECTIVE

This qualitative study aimed to explore the perception of university students and mental health supporters in Singapore about mental health services, campaigns, and mHealth interventions with a focus on conversational agent interventions for the prevention of common mental disorders such as anxiety and depression.

METHODS

We conducted 6 web-based focus group discussions with 30 university students and one-to-one web-based interviews with 11 mental health supporters consisting of faculty members tasked with student pastoral care, a mental health first aider, counselors, psychologists, a clinical psychologist, and a psychiatrist. The qualitative analysis followed a reflexive thematic analysis framework.

RESULTS

The following 6 main themes were identified: a healthy lifestyle as students, access to mental health services, the role of mental health promotion campaigns, preferred mHealth engagement features, factors that influence the adoption of mHealth interventions, and cultural relevance of mHealth interventions. The interpretation of our findings shows that students were reluctant to use mental health services because of the fear of stigma and a possible lack of confidentiality.

CONCLUSIONS

Study participants viewed mHealth interventions for mental health as part of a blended intervention. They also felt that future mental health mHealth interventions should be more personalized and capable of managing adverse events such as suicidal ideation.

Older adults' experience with virtual conversational agents for health data collection

Introduction

Virtual conversational agents (i.e., chatbots) are an intuitive form of data collection. Understanding older adults' experiences with chatbots could help identify their usability needs. This quality improvement study evaluated older adults' experiences with a chatbot for health data collection. A secondary goal was to understand how perceptions differed based on length of chatbot forms.

Methods

After a demographic survey, participants (≥60 years) completed either a short (21 questions), moderate (30 questions), or long (66 questions) chatbot form. Perceived ease-of-use, usefulness, usability, likelihood to recommend, and cognitive load were measured post-test. Qualitative and quantitative analyses were used.

Results

A total of 260 participants reported on usability and satisfaction metrics including perceived ease-of-use (5.8/7), usefulness (4.7/7), usability (5.4/7), and likelihood to recommend (Net Promoter Score = 0). Cognitive load (12.3/100) was low. There was a statistically significant difference in perceived usefulness between groups, with a significantly higher mean perceived usefulness for Group 1 than Group 3. No other group differences were observed. The chatbot was perceived as quick, easy, and pleasant with concerns about technical issues, privacy, and security. Participants provided suggestions to enhance progress tracking, edit responses, improve readability, and have options to ask questions.

Discussion

Older adults found the chatbot to be easy, useful, and usable. The chatbot required low cognitive load demonstrating it could be an enjoyable health data collection tool for older adults. These results will inform the development of a health data collection chatbot technology.

Use of an Artificial Intelligence Conversational Agent (Chatbot) for Hip Arthroscopy Patients Following Surgery

Purpose

The purpose of this study was to evaluate the use of an AI conversational agent during the postoperative recovery of patients undergoing elective hip arthroscopy.

Methods

Patients undergoing hip arthroscopy were enrolled in a prospective cohort for their first 6 weeks following surgery. Patients used standard SMS text messaging to interact with an artificial intelligence (AI) chatbot ("Felix") used to initiate automated conversations regarding elements of postoperative recovery. Patient satisfaction was measured at 6 weeks after surgery using a Likert scale survey. Accuracy was determined by measuring the appropriateness of chatbot responses, topic recognition, and examples of confusion. Safety was measured by evaluating the chatbot's responses to any questions with potential medical urgency.

Results

Twenty-six patients were enrolled with a mean age of 36 years, and 58% (n = 15) were male. Overall, 80% of patients (n = 20) rated the helpfulness of Felix as good or excellent. In the postoperative period, 12/25 (48%) patients reported being worried about a complication but were reassured by Felix and, thus, did not seek medical attention. Of a total of 128 independent patient questions, Felix handled 101/128 questions appropriately (79%), either by addressing them independently, or facilitating contact with the care team. Felix was able to adequately answer the patient question independently 31% of the time (n = 40/128). Of 10 patient questions that were thought to potentially represent patient complications, in 3 cases Felix did not adequately address or recognize the health concern-none of these situations resulted in patient harm.

Conclusion

The results of this study demonstrate that the use of a chatbot or conversational agent can enhance the postoperative experience for hip arthroscopy patients, as demonstrated by high levels of patient satisfaction.

Levels of Evidence

Level IV, therapeutic case series.

Implementation Fidelity of Chatbot Screening for Social Needs: Acceptability, Feasibility, Appropriateness

OBJECTIVES

Patient and provider-facing screening tools for social determinants of health have been explored in a variety of contexts; however, effective screening and resource referral remain challenging, and less is known about how patients perceive chatbots as potential social needs screening tools. We investigated patient perceptions of a chatbot for social needs screening using three implementation outcome measures: acceptability, feasibility, and appropriateness.

METHODS

We implemented a chatbot for social needs screening at one large public hospital emergency department (ED) and used concurrent triangulation to assess perceptions of the chatbot use for screening. A total of 350 ED visitors completed the social needs screening and rated the chatbot on implementation outcome measures, and 22 participants engaged in follow-up phone interviews.

RESULTS

The screened participants ranged in age from 18 to 90 years old and were diverse in race/ethnicity, education, and insurance status. Participants (n = 350) rated the chatbot as an acceptable, feasible, and appropriate way of screening. Through interviews (n = 22), participants explained that the chatbot was a responsive, private, easy to use, efficient, and comfortable channel to report social needs in the ED, but wanted more information on data use and more support in accessing resources.

CONCLUSION

In this study, we deployed a chatbot for social needs screening in a real-world context and found patients perceived the chatbot to be an acceptable, feasible, and appropriate modality for social needs screening. Findings suggest that chatbots are a promising modality for social needs screening and can successfully engage a large, diverse patient population in the ED. This is significant, as it suggests that chatbots could facilitate a screening process that ultimately connects patients to care for social needs, improving health and well-being for members of vulnerable patient populations.

Development and testing of a multi-lingual Natural Language Processing-based deep learning system in 10 languages for COVID-19 pandemic crisis: A multi-center study

Purpose

The COVID-19 pandemic has drastically disrupted global healthcare systems. With the higher demand for healthcare and misinformation related to COVID-19, there is a need to explore alternative models to improve communication. Artificial Intelligence (AI) and Natural Language Processing (NLP) have emerged as promising solutions to improve healthcare delivery. Chatbots could fill a pivotal role in the dissemination and easy accessibility of accurate information in a pandemic. In this study, we developed a multi-lingual NLP-based AI chatbot, DR-COVID, which responds accurately to open-ended, COVID-19 related questions. This was used to facilitate pandemic education and healthcare delivery.

Methods

First, we developed DR-COVID with an ensemble NLP model on the Telegram platform (https://t.me/drcovid_nlp_chatbot). Second, we evaluated various performance metrics. Third, we evaluated multi-lingual text-to-text translation to Chinese, Malay, Tamil, Filipino, Thai, Japanese, French, Spanish, and Portuguese. We utilized 2,728 training questions and 821 test questions in English. Primary outcome measurements were (A) overall and top 3 accuracies; (B) Area Under the Curve (AUC), precision, recall, and F1 score. Overall accuracy referred to a correct response for the top answer, whereas top 3 accuracy referred to an appropriate response for any one answer amongst the top 3 answers. AUC and its relevant matrices were obtained from the Receiver Operation Characteristics (ROC) curve. Secondary outcomes were (A) multi-lingual accuracy; (B) comparison to enterprise-grade chatbot systems. The sharing of training and testing datasets on an open-source platform will also contribute to existing data.

Results

Our NLP model, utilizing the ensemble architecture, achieved overall and top 3 accuracies of 0.838 [95% confidence interval (CI): 0.826-0.851] and 0.922 [95% CI: 0.913-0.932] respectively. For overall and top 3 results, AUC scores of 0.917 [95% CI: 0.911-0.925] and 0.960 [95% CI: 0.955-0.964] were achieved respectively. We achieved multi-linguicism with nine non-English languages, with Portuguese performing the best overall at 0.900. Lastly, DR-COVID generated answers more accurately and quickly than other chatbots, within 1.12-2.15 s across three devices tested.

Conclusion

DR-COVID is a clinically effective NLP-based conversational AI chatbot, and a promising solution for healthcare delivery in the pandemic era.

Virtual Digital Psychotherapist App-Based Treatment in Patients With Methamphetamine Use Disorder (Echo-APP): Single-Arm Pilot Feasibility and Efficacy Study

BACKGROUND

Substance use disorder is one of the severe public health problems worldwide. Inequitable resources, discrimination, and physical distances limit patients' access to medical help. Automated conversational agents have the potential to provide in-home and remote therapy. However, automatic dialogue agents mostly use text and other methods to interact, which affects the interaction experience, treatment immersion, and clinical efficacy.

OBJECTIVE

The aim of this paper is to describe the design and development of Echo-APP, a tablet-based app with the function of a virtual digital psychotherapist, and to conduct a pilot study to explore the feasibility and preliminary efficacy results of Echo-APP for patients with methamphetamine use disorder.

METHODS

Echo-APP is an assessment and rehabilitation program developed for substance use disorder (SUD) by a team of clinicians, psychotherapists, and computer experts. The program is available for Android tablets. In terms of assessment, the focus is on the core characteristics of SUD, such as mood, impulsivity, treatment motivation, and craving level. In terms of treatment, Echo-APP provides 10 treatment units, involving awareness of addiction, motivation enhancement, emotion regulation, meditation, etc. A total of 47 patients with methamphetamine dependence were eventually enrolled in the pilot study to receive a single session of the Echo-APP-based motivational enhancement treatment. The outcomes were assessed before and after the patients' treatment, including treatment motivation, craving levels, self-perception on the importance of drug abstinence, and their confidence in stopping the drug use.

RESULTS

In the pilot study, scores on the Stages of Change Readiness and Treatment Eagerness Scale and the questionnaire on motivation for abstaining from drugs significantly increased after the Echo-APP-based treatment (P<.001, Cohen d=-0.60), while craving was reduced (P=.01, Cohen d=0.38). Patients' baseline Generalized Anxiety Disorder-7 assessment score (β=3.57; P<.001; 95% CI 0.80, 2.89) and Barratt Impulsiveness Scale (BIS)-motor impulsiveness score (β=-2.10; P=.04; 95% CI -0.94, -0.02) were predictive of changes in the patients' treatment motivation during treatment. Moreover, patients' baseline Generalized Anxiety Disorder-7 assessment score (β=-1.607; P=.03; 95% CI -3.08, -0.14), BIS-attentional impulsivity score (β=-2.43; P=.004; 95% CI -4.03, -0.83), and BIS-nonplanning impulsivity score (β=2.54; P=.002; 95% CI 0.98, 4.10) were predictive of changes in craving scores during treatment.

CONCLUSIONS

Echo-APP is a practical, accepted, and promising virtual digital psychotherapist program for patients with methamphetamine dependence. The preliminary findings lay a good foundation for further optimization of the program and the promotion of large-scale randomized controlled clinical studies for SUD.