Conversational Agents in the Treatment of Mental Health Problems: Mixed-Method Systematic Review

The first AI-based Chatbot to promote HIV self-management: A mixed methods usability study

BACKGROUND

We developed MARVIN, an artificial intelligence (AI)-based chatbot that provides 24/7 expert-validated information on self-management-related topics for people with HIV. This study assessed (1) the feasibility of using MARVIN, (2) its usability and acceptability, and (3) four usability subconstructs (perceived ease of use, perceived usefulness, attitude towards use, and behavioural intention to use).

METHODS

In a mixed-methods study conducted at the McGill University Health Centre, enrolled participants were asked to have 20 conversations within 3 weeks with MARVIN on predetermined topics and to complete a usability questionnaire. Feasibility, usability, acceptability, and usability subconstructs were examined against predetermined success thresholds. Qualitatively, randomly selected participants were invited to semi-structured focus groups/interviews to discuss their experiences with MARVIN. Barriers and facilitators were identified according to the four usability subconstructs.

RESULTS

From March 2021 to April 2022, 28 participants were surveyed after a 3-week testing period, and nine were interviewed. Study retention was 70% (28/40). Mean usability exceeded the threshold (69.9/68), whereas mean acceptability was very close to target (23.8/24). Ratings of attitude towards MARVIN's use were positive (+14%), with the remaining subconstructs exceeding the target (5/7). Facilitators included MARVIN's reliable and useful real-time information support, its easy accessibility, provision of convivial conversations, confidentiality, and perception as being emotionally safe. However, MARVIN's limited comprehension and the use of Facebook as an implementation platform were identified as barriers, along with the need for more conversation topics and new features (e.g., memorization).

CONCLUSIONS

The study demonstrated MARVIN's global usability. Our findings show its potential for HIV self-management and provide direction for further development.

Chatbot-Led Support Combined With Counselor-Led Support on Smoking Cessation in China: Protocol for a Pilot Randomized Controlled Trial

BACKGROUND

China has a large population of smokers, with half of them dependent on tobacco and in need of cessation assistance, indicating the need for mobile health (mHealth) to provide cessation support.

OBJECTIVE

The study aims to assess the feasibility and preliminary effectiveness of combining chatbot-led support with counselor-led support for smoking cessation among community smokers in China.

METHODS

This is a 2-arm, parallel, assessor-blinded, pilot randomized controlled trial nested in a smoke-free campus campaign in Zhuhai, China. All participants will receive brief face-to-face cessation advice and group cessation support led by a chatbot embedded in WeChat. In addition, participants in the intervention group will receive personalized WeChat-based counseling from trained counselors. Follow-up will occur at 1, 3, and 6 months after treatment initiation. The primary smoking outcome is bioverified abstinence (exhaled carbon monoxide <4 parts per million or salivary cotinine <30 ng/mL) at 6 months. Secondary outcomes include self-reported 7-day point prevalence of abstinence, smoking reduction rate, and quit attempts. Feasibility outcomes include eligibility rate, consent rate, intervention engagement, and retention rate. An intention-to-treat approach and regression models will be used for primary analyses.

RESULTS

Participant recruitment began in March 2023, and the intervention began in April 2023. The data collection was completed in June 2024. The results of the study will be published in peer-reviewed journals and presented at international conferences.

CONCLUSIONS

This study will provide novel insights into the feasibility and preliminary effectiveness of a chatbot-led intervention for smoking cessation in China. The findings of this study will inform the development and optimization of mHealth interventions for smoking cessation in China and other low- and middle-income countries.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05777005; https://clinicaltrials.gov/study/NCT05777005.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/58636.

Empathic Conversational Agent Platform Designs and Their Evaluation in the Context of Mental Health: Systematic Review

BACKGROUND

The demand for mental health (MH) services in the community continues to exceed supply. At the same time, technological developments make the use of artificial intelligence-empowered conversational agents (CAs) a real possibility to help fill this gap.

OBJECTIVE

The objective of this review was to identify existing empathic CA design architectures within the MH care sector and to assess their technical performance in detecting and responding to user emotions in terms of classification accuracy. In addition, the approaches used to evaluate empathic CAs within the MH care sector in terms of their acceptability to users were considered. Finally, this review aimed to identify limitations and future directions for empathic CAs in MH care.

METHODS

A systematic literature search was conducted across 6 academic databases to identify journal articles and conference proceedings using search terms covering 3 topics: "conversational agents," "mental health," and "empathy." Only studies discussing CA interventions for the MH care domain were eligible for this review, with both textual and vocal characteristics considered as possible data inputs. Quality was assessed using appropriate risk of bias and quality tools.

RESULTS

A total of 19 articles met all inclusion criteria. Most (12/19, 63%) of these empathic CA designs in MH care were machine learning (ML) based, with 26% (5/19) hybrid engines and 11% (2/19) rule-based systems. Among the ML-based CAs, 47% (9/19) used neural networks, with transformer-based architectures being well represented (7/19, 37%). The remaining 16% (3/19) of the ML models were unspecified. Technical assessments of these CAs focused on response accuracies and their ability to recognize, predict, and classify user emotions. While single-engine CAs demonstrated good accuracy, the hybrid engines achieved higher accuracy and provided more nuanced responses. Of the 19 studies, human evaluations were conducted in 16 (84%), with only 5 (26%) focusing directly on the CA's empathic features. All these papers used self-reports for measuring empathy, including single or multiple (scale) ratings or qualitative feedback from in-depth interviews. Only 1 (5%) paper included evaluations by both CA users and experts, adding more value to the process.

CONCLUSIONS

The integration of CA design and its evaluation is crucial to produce empathic CAs. Future studies should focus on using a clear definition of empathy and standardized scales for empathy measurement, ideally including expert assessment. In addition, the diversity in measures used for technical assessment and evaluation poses a challenge for comparing CA performances, which future research should also address. However, CAs with good technical and empathic performance are already available to users of MH care services, showing promise for new applications, such as helpline services.

Artificial Intelligence, the Digital Surgeon: Unravelling Its Emerging Footprint in Healthcare - The Narrative Review

Background

Artificial Intelligence (AI) holds transformative potential for the healthcare industry, offering innovative solutions for diagnosis, treatment planning, and improving patient outcomes. As AI continues to be integrated into healthcare systems, it promises advancements across various domains. This review explores the diverse applications of AI in healthcare, along with the challenges and limitations that need to be addressed. The aim is to provide a comprehensive overview of AI's impact on healthcare and to identify areas for further development and focus.

Main Applications

The review discusses the broad range of AI applications in healthcare. In medical imaging and diagnostics, AI enhances the accuracy and efficiency of diagnostic processes, aiding in early disease detection. AI-powered clinical decision support systems assist healthcare professionals in patient management and decision-making. Predictive analytics using AI enables the prediction of patient outcomes and identification of potential health risks. AI-driven robotic systems have revolutionized surgical procedures, improving precision and outcomes. Virtual assistants and chatbots enhance patient interaction and support, providing timely information and assistance. In the pharmaceutical industry, AI accelerates drug discovery and development by identifying potential drug candidates and predicting their efficacy. Additionally, AI improves administrative efficiency and operational workflows in healthcare, streamlining processes and reducing costs. AI-powered remote monitoring and telehealth solutions expand access to healthcare, particularly in underserved areas.

Challenges and Limitations

Despite the significant promise of AI in healthcare, several challenges persist. Ensuring the reliability and consistency of AI-driven outcomes is crucial. Privacy and security concerns must be navigated carefully, particularly in handling sensitive patient data. Ethical considerations, including bias and fairness in AI algorithms, need to be addressed to prevent unintended consequences. Overcoming these challenges is critical for the ethical and successful integration of AI in healthcare.

Conclusion

The integration of AI into healthcare is advancing rapidly, offering substantial benefits in improving patient care and operational efficiency. However, addressing the associated challenges is essential to fully realize the transformative potential of AI in healthcare. Future efforts should focus on enhancing the reliability, transparency, and ethical standards of AI technologies to ensure they contribute positively to global health outcomes.

A Chatbot (Juno) Prototype to Deploy a Behavioral Activation Intervention to Pregnant Women: Qualitative Evaluation Using a Multiple Case Study

BACKGROUND

Despite the increasing focus on perinatal care, preventive digital interventions are still scarce. Furthermore, the literature suggests that the design and development of these interventions are mainly conducted through a top-down approach that limitedly accounts for direct end user perspectives.

OBJECTIVE

Building from a previous co-design study, this study aimed to qualitatively evaluate pregnant women's experiences with a chatbot (Juno) prototype designed to deploy a preventive behavioral activation intervention.

METHODS

Using a multiple-case study design, the research aims to uncover similarities and differences in participants' perceptions of the chatbot while also exploring women's desires for improvement and technological advancements in chatbot-based interventions in perinatal mental health. Five pregnant women interacted weekly with the chatbot, operationalized in Telegram, following a 6-week intervention. Self-report questionnaires were administered at baseline and postintervention time points. About 10-14 days after concluding interactions with Juno, women participated in a semistructured interview focused on (1) their personal experience with Juno, (2) user experience and user engagement, and (3) their opinions on future technological advancements. Interview transcripts, comprising 15 questions, were qualitatively evaluated and compared. Finally, a text-mining analysis of transcripts was performed.

RESULTS

Similarities and differences have emerged regarding women's experiences with Juno, appreciating its esthetic but highlighting technical issues and desiring clearer guidance. They found the content useful and pertinent to pregnancy but differed on when they deemed it most helpful. Women expressed interest in receiving increasingly personalized responses and in future integration with existing health care systems for better support. Accordingly, they generally viewed Juno as an effective momentary support but emphasized the need for human interaction in mental health care, particularly if increasingly personalized. Further concerns included overreliance on chatbots when seeking psychological support and the importance of clearly educating users on the chatbot's limitations.

CONCLUSIONS

Overall, the results highlighted both the positive aspects and the shortcomings of the chatbot-based intervention, providing insight into its refinement and future developments. However, women stressed the need to balance technological support with human interactions, particularly when the intervention involves beyond preventive mental health context, to favor a greater and more reliable monitoring.

Effectiveness and User Experience of a Smoking Cessation Chatbot: Mixed Methods Study Comparing Motivational Interviewing and Confrontational Counseling

BACKGROUND

Cigarette smoking poses a major public health risk. Chatbots may serve as an accessible and useful tool to promote cessation due to their high accessibility and potential in facilitating long-term personalized interactions. To increase effectiveness and acceptability, there remains a need to identify and evaluate counseling strategies for these chatbots, an aspect that has not been comprehensively addressed in previous research.

OBJECTIVE

This study aims to identify effective counseling strategies for such chatbots to support smoking cessation. In addition, we sought to gain insights into smokers' expectations of and experiences with the chatbot.

METHODS

This mixed methods study incorporated a web-based experiment and semistructured interviews. Smokers (N=229) interacted with either a motivational interviewing (MI)-style (n=112, 48.9%) or a confrontational counseling-style (n=117, 51.1%) chatbot. Both cessation-related (ie, intention to quit and self-efficacy) and user experience-related outcomes (ie, engagement, therapeutic alliance, perceived empathy, and interaction satisfaction) were assessed. Semistructured interviews were conducted with 16 participants, 8 (50%) from each condition, and data were analyzed using thematic analysis.

RESULTS

Results from a multivariate ANOVA showed that participants had a significantly higher overall rating for the MI (vs confrontational counseling) chatbot. Follow-up discriminant analysis revealed that the better perception of the MI chatbot was mostly explained by the user experience-related outcomes, with cessation-related outcomes playing a lesser role. Exploratory analyses indicated that smokers in both conditions reported increased intention to quit and self-efficacy after the chatbot interaction. Interview findings illustrated several constructs (eg, affective attitude and engagement) explaining people's previous expectations and timely and retrospective experience with the chatbot.

CONCLUSIONS

The results confirmed that chatbots are a promising tool in motivating smoking cessation and the use of MI can improve user experience. We did not find extra support for MI to motivate cessation and have discussed possible reasons. Smokers expressed both relational and instrumental needs in the quitting process. Implications for future research and practice are discussed.

Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review

BACKGROUND

Chatbots, or conversational agents, have emerged as significant tools in health care, driven by advancements in artificial intelligence and digital technology. These programs are designed to simulate human conversations, addressing various health care needs. However, no comprehensive synthesis of health care chatbots' roles, users, benefits, and limitations is available to inform future research and application in the field.

OBJECTIVE

This review aims to describe health care chatbots' characteristics, focusing on their diverse roles in the health care pathway, user groups, benefits, and limitations.

METHODS

A rapid review of published literature from 2017 to 2023 was performed with a search strategy developed in collaboration with a health sciences librarian and implemented in the MEDLINE and Embase databases. Primary research studies reporting on chatbot roles or benefits in health care were included. Two reviewers dual-screened the search results. Extracted data on chatbot roles, users, benefits, and limitations were subjected to content analysis.

RESULTS

The review categorized chatbot roles into 2 themes: delivery of remote health services, including patient support, care management, education, skills building, and health behavior promotion, and provision of administrative assistance to health care providers. User groups spanned across patients with chronic conditions as well as patients with cancer; individuals focused on lifestyle improvements; and various demographic groups such as women, families, and older adults. Professionals and students in health care also emerged as significant users, alongside groups seeking mental health support, behavioral change, and educational enhancement. The benefits of health care chatbots were also classified into 2 themes: improvement of health care quality and efficiency and cost-effectiveness in health care delivery. The identified limitations encompassed ethical challenges, medicolegal and safety concerns, technical difficulties, user experience issues, and societal and economic impacts.

CONCLUSIONS

Health care chatbots offer a wide spectrum of applications, potentially impacting various aspects of health care. While they are promising tools for improving health care efficiency and quality, their integration into the health care system must be approached with consideration of their limitations to ensure optimal, safe, and equitable use.

Voice Assistants for Mental Health Services: Designing Dialogues with Homebound Older Adults

The number of older adults who are homebound with depressive symptoms is increasing. Due to their homebound status, they have limited access to trained mental healthcare support, which leaves this support often to untrained family caregivers. To increase access, a growing interest is placed on using technology-mediated solutions, such as voice-assisted intelligent personal assistants (VIPAs), to deliver mental health services to older adults. To better understand how older adults and family caregivers intend to interact with a VIPA for mental health interventions, we conducted a participatory design study during which 6 older adults and 7 caregivers designed VIPA-human dialogues for various scenarios. Using conversation style preferences as a starting point, we present aspects of human-likeness older adults and family caregivers perceived as helpful or uncanny, specifically in the context of the delivery of mental health interventions, which helps inform potential roles VIPAs can play in mental healthcare for older adults.

Artificial intelligence in perinatal mental health research: A scoping review

The intersection of Artificial Intelligence (AI) and perinatal mental health research presents promising avenues, yet uncovers significant challenges for innovation. This review explicitly focuses on this multidisciplinary field and undertakes a comprehensive exploration of existing research therein. Through a scoping review guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework, we searched relevant literature spanning a decade (2013-2023) and selected fourteen studies for our analysis. We first provide an overview of the main AI techniques and their development, including traditional methods across different categories, as well as recent emerging methods in the field. Then, through our analysis of the literature, we summarize the predominant AI and ML techniques adopted and their applications in perinatal mental health studies, such as identifying risk factors, predicting perinatal mental health disorders, voice assistants, and Q&A chatbots. We also discuss existing limitations and potential challenges that hinder AI technologies from improving perinatal mental health outcomes, and suggest several promising directions for future research to meet real needs in the field and facilitate the translation of research into clinical settings.

Rates and correlates of study enrolment and use of a chatbot aimed to promote mental health services use for eating disorders following online screening

OBJECTIVE

We developed a chatbot aimed to facilitate mental health services use for eating disorders (EDs) and offered the opportunity to enrol in a research study and use the chatbot to all adult respondents to a publicly available online ED screen who screened positive for clinical/subclinical EDs and reported not currently being in treatment. We examined the rates and correlates of enrolment in the study and uptake of the chatbot.

METHOD

Following screening, eligible respondents (≥18 years, screened positive for a clinical/subclinical ED, not in treatment for an ED) were shown the study opportunity. Chi-square tests and logistic regressions explored differences in demographics, ED symptoms, suicidality, weight, and probable ED diagnoses between those who enroled and engaged with the chatbot versus those who did not.

RESULTS

6747 respondents were shown the opportunity (80.0% of all adult screens). 3.0% enroled, of whom 90.2% subsequently used the chatbot. Enrolment and chatbot uptake were more common among respondents aged ≥25 years old versus those aged 18-24 and less common among respondents who reported engaging in regular dietary restriction.

CONCLUSIONS

Overall enrolment was low, yet uptake was high among those that enroled and did not differ across most demographics and symptom presentations. Future directions include evaluating respondents' attitudes towards treatment-promoting tools and removing barriers to uptake.

Effects of a Chatbot-Based Intervention on Stress and Health-Related Parameters in a Stressed Sample: Randomized Controlled Trial

BACKGROUND

Stress levels and the prevalence of mental disorders in the general population have been rising in recent years. Chatbot-based interventions represent novel and promising digital approaches to improve health-related parameters. However, there is a lack of research on chatbot-based interventions in the area of mental health.

OBJECTIVE

The aim of this study was to investigate the effects of a 3-week chatbot-based intervention guided by the chatbot ELME, specifically with respect to the ability to reduce stress and improve various health-related parameters in a stressed sample.

METHODS

In this multicenter two-armed randomized controlled trial, 118 individuals with medium to high stress levels were randomized to the intervention group (n=59) or the treatment-as-usual control group (n=59). The ELME chatbot guided participants of the intervention group through 3 weeks of training based on the topics stress, mindfulness, and interoception, with practical and psychoeducative elements delivered in two daily interactive intervention sessions via a smartphone (approximately 10-20 minutes each). The primary outcome (perceived stress) and secondary outcomes (mindfulness; interoception or interoceptive sensibility; subjective well-being; and emotion regulation, including the subfacets reappraisal and suppression) were assessed preintervention (T1), post intervention (T2; after 3 weeks), and at follow-up (T3; after 6 weeks). During both conditions, participants also underwent ecological momentary assessments of stress and interoceptive sensibility.

RESULTS

There were no significant changes in perceived stress (β03=-.018, SE=.329; P=.96) and momentary stress. Mindfulness and the subfacet reappraisal significantly increased in the intervention group over time, whereas there was no change in the subfacet suppression. Well-being and momentary interoceptive sensibility increased in both groups over time.

CONCLUSIONS

To gain insight into how the intervention can be improved to achieve its full potential for stress reduction, besides a longer intervention duration, specific sample subgroups should be considered. The chatbot-based intervention seems to have the potential to improve mindfulness and emotion regulation in a stressed sample. Future chatbot-based studies and interventions in health care should be designed based on the latest findings on the efficacy of rule-based and artificial intelligence-based chatbots.

TRIAL REGISTRATION

German Clinical Trials Register DRKS00027560; https://drks.de/search/en/trial/DRKS00027560.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-doi.org/10.3389/fdgth.2023.1046202.

A Digital Behavioral Activation Intervention (JuNEX) for Pregnant Women With Subclinical Depression Symptoms: Explorative Co-Design Study

BACKGROUND

Digital interventions are gaining increasing interest due to their structured nature, ready availability, and self-administered capabilities. Perinatal women have expressed a desire for such interventions. In this regard, behavioral activation interventions may be particularly suitable for digital administration.

OBJECTIVE

This study aims to exploratorily investigate and compare the feasibility of the internet-based self-help guided versus unguided version of the Brief Behavioral Activation Treatment for Depression-Revised, an empirically supported in-person behavioral activation protocol, targeting pregnant women with subclinical depression symptoms. A user-centered design is used, whereby data are collected with the intent of evaluating how to adjust the intervention in line with pregnant women's needs. Usability and user engagement were evaluated.

METHODS

A total of 11 Italian pregnant women with subclinical depressive symptoms based on the Patient Health Questionnaire-9 (scoring<15) participated in this study; of them, 6 (55%) women were randomly assigned to the guided group (age: mean 32.17, SD 4.36 years) and 5 (45%) to the unguided group (age: mean 31, SD 4.95 years). The Moodle platform was used to deliver the interventions in an e-learning format. It consisted of 6 core modules and 3 optional modules; the latter aimed at revising the content of the former. In the guided group, each woman had weekly chats with their assigned human guide to support them in the homework revisions. The intervention content included text, pictures, and videos. Semistructured interviews were conducted, and descriptive statistics were analyzed.

RESULTS

Collectively, the data suggest that the guided intervention was better accepted than the unguided one. However, the high rates of dropout (at T6: guided group: 3/6, 50%; unguided: 4/5, 80%) suggest that a digital replica of Behavioral Activation Treatment for Depression-Revised may not be feasible in an e-learning format. The reduced usability of the platform used was reported, and homework was perceived as too time-consuming and effort-intensive. Moreover, the 6 core modules were deemed sufficient for the intervention's goals, suggesting that the 3 optional modules could be eliminated. Nevertheless, participants from both groups expressed satisfaction with the content and found it relevant to their pregnancy experiences.

CONCLUSIONS

Overall, the findings have emphasized both the intervention's merits and shortcomings. Results highlight the unsuitability of replicating an in-person protocol digitally as well as of the use of nonprofessional tools for the implementation of self-help interventions, ultimately making the intervention not feasible. Pregnant women have nonetheless expressed a desire to receive psychological support and commented on the possibilities of digital psychosocial supports, particularly those that are app-based. The information collected and the issues identified here are important to guide the development and co-design of a more refined platform for the intervention deployment and to tailor the intervention's content to pregnant women's needs.

Dialogue agents for artificial intelligence-based conversational systems for cognitively disabled: a systematic review

PURPOSE

We present a systematic literature review of dialogue agents for Artificial Intelligence (AI) and agent-based conversational systems dealing with cognitive disability of aged and impaired people including dementia and Parkinson's disease. We analyze current applications, gaps, and challenges in the existing research body, and provide guidelines and recommendations for their future development and use.

MATERIALS AND METHODS

We perform this study by applying Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria. We performed a systematic search using relevant databases (ACM Digital Library, Google Scholar, IEEE Xplore, PubMed, and Scopus).

RESULTS

This study identified 468 articles on the use of conversational agents in healthcare. We finally selected 124 articles based on their objectives and content as directly related to our main topic.

CONCLUSION

We identified the main challenges in the field and analyzed the typical examples of the application of conversational agents in the healthcare domain, the desired characteristics of conversational agents, and chatbot support for aged people and people with cognitive disabilities. Our results contribute to a discussion on conversational health agents and emphasize current knowledge gaps and challenges for future research.IMPLICATIONS FOR REHABILITATIONA systematic literature review of dialogue agents for artificial intelligence and agent-based conversational systems dealing with cognitive disability of aged and impaired people.Main challenges and desired characteristics of the conversational agents, and chatbot support for aged people and people with cognitive disability.Current knowledge gaps and challenges for remote healthcare and rehabilitation.Guidelines and recommendations for future development and use of conversational systems.

Development and Evaluation of a Digital App for Patient Self-Management of Opioid Use Disorder: Usability, Acceptability, and Utility Study

BACKGROUND

Self-management of opioid use disorder (OUD) is an important component of treatment. Many patients receiving opioid agonist treatment in methadone maintenance treatment settings benefit from counseling treatments to help them improve their recovery skills but have insufficient access to these treatments between clinic appointments. In addition, many addiction medicine clinicians treating patients with OUD in a general medical clinic setting do not have consistent access to counseling referrals for their patients. This can lead to decreases in both treatment retention and overall progress in the patient's recovery from substance misuse. Digital apps may help to bridge this gap by coaching, supporting, and reinforcing behavioral change that is initiated and directed by their psychosocial and medical providers.

OBJECTIVE

This study aimed to conduct an acceptability, usability, and utility pilot study of the KIOS app to address these clinical needs.

METHODS

We developed a unique, patient-centered computational software system (KIOS; Biomedical Development Corporation) to assist in managing OUD in an outpatient, methadone maintenance clinic setting. KIOS tracks interacting self-reported symptoms (craving, depressed mood, anxiety, irritability, pain, agitation or restlessness, difficulty sleeping, absenteeism, difficulty with usual activities, and conflicts with others) to determine changes in both the trajectory and severity of symptom patterns over time. KIOS then applies a proprietary algorithm to assess the individual's patterns of symptom interaction in accordance with models previously established by OUD experts. After this analysis, KIOS provides specific behavioral advice addressing the individual's changing trajectory of symptoms to help the person self-manage their symptoms. The KIOS software also provides analytics on the self-reported data that can be used by patients, clinicians, and researchers to track outcomes.

RESULTS

In a 4-week acceptability, usability (mean System Usability Scale-Modified score 89.5, SD 9.2, maximum of 10.0), and utility (mean KIOS utility questionnaire score 6.32, SD 0.25, maximum of 7.0) pilot study of 15 methadone-maintained participants with OUD, user experience, usability, and software-generated advice received high and positive assessment scores. The KIOS clinical variables closely correlated with craving self-report measures. Therefore, managing these variables with advice generated by the KIOS software could have an impact on craving and ultimately substance use.

CONCLUSIONS

KIOS tracks key clinical variables and generates advice specifically relevant to the patient's current and changing clinical state. Patients in this pilot study assigned high positive values to the KIOS user experience, ease of use, and the appropriateness, relevance, and usefulness of the specific behavioral guidance they received to match their evolving experiences. KIOS may therefore be useful to augment in-person treatment of opioid agonist patients and help fill treatment gaps that currently exist in the continuum of care. A National Institute on Drug Abuse-funded randomized controlled trial of KIOS to augment in-person treatment of patients with OUD is currently being conducted.

Innovations in Medicine: Exploring ChatGPT's Impact on Rare Disorder Management

Artificial intelligence (AI) is rapidly transforming the field of medicine, announcing a new era of innovation and efficiency. Among AI programs designed for general use, ChatGPT holds a prominent position, using an innovative language model developed by OpenAI. Thanks to the use of deep learning techniques, ChatGPT stands out as an exceptionally viable tool, renowned for generating human-like responses to queries. Various medical specialties, including rheumatology, oncology, psychiatry, internal medicine, and ophthalmology, have been explored for ChatGPT integration, with pilot studies and trials revealing each field's potential benefits and challenges. However, the field of genetics and genetic counseling, as well as that of rare disorders, represents an area suitable for exploration, with its complex datasets and the need for personalized patient care. In this review, we synthesize the wide range of potential applications for ChatGPT in the medical field, highlighting its benefits and limitations. We pay special attention to rare and genetic disorders, aiming to shed light on the future roles of AI-driven chatbots in healthcare. Our goal is to pave the way for a healthcare system that is more knowledgeable, efficient, and centered around patient needs.

Use of automated conversational agents in improving young population mental health: a scoping review

Automated conversational agents (CAs) emerged as a promising solution in mental health interventions among young people. Therefore, the objective of this scoping review is to examine the current state of research into fully automated CAs mediated interventions for the emotional component of mental health among young people. Selected databases were searched in March 2023. Included studies were primary research, reporting on development, feasibility/usability, or evaluation of fully automated CAs as a tool to improve the emotional component of mental health among young population. Twenty-five studies were included (N = 1707). Most automated CAs applications were standalone preventions targeting anxiety and depression. Automated CAs were predominantly AI-based chatbots, using text as the main communication channel. Overall, the results of the current scoping review showed that automated CAs mediated interventions for emotional problems are acceptable, engaging and with high usability. However, the results for clinical efficacy are far less conclusive, since almost half of evaluation studies reported no significant effect on emotional mental health outcomes. Based on these findings, it can be concluded that there is a pressing need to improve the existing automated CAs applications to increase their efficacy as well as conducting more rigorous methodological research in this area.

Implementation of Chatbot Technology in Health Care: Protocol for a Bibliometric Analysis

BACKGROUND

Chatbots have the potential to increase people's access to quality health care. However, the implementation of chatbot technology in the health care system is unclear due to the scarce analysis of publications on the adoption of chatbot in health and medical settings.

OBJECTIVE

This paper presents a protocol of a bibliometric analysis aimed at offering the public insights into the current state and emerging trends in research related to the use of chatbot technology for promoting health.

METHODS

In this bibliometric analysis, we will select published papers from the databases of CINAHL, IEEE Xplore, PubMed, Scopus, and Web of Science that pertain to chatbot technology and its applications in health care. Our search strategy includes keywords such as "chatbot," "virtual agent," "virtual assistant," "conversational agent," "conversational AI," "interactive agent," "health," and "healthcare." Five researchers who are AI engineers and clinicians will independently review the titles and abstracts of selected papers to determine their eligibility for a full-text review. The corresponding author (ZN) will serve as a mediator to address any discrepancies and disputes among the 5 reviewers. Our analysis will encompass various publication patterns of chatbot research, including the number of annual publications, their geographic or institutional distribution, and the number of annual grants supporting chatbot research, and further summarize the methodologies used in the development of health-related chatbots, along with their features and applications in health care settings. Software tool VOSViewer (version 1.6.19; Leiden University) will be used to construct and visualize bibliometric networks.

RESULTS

The preparation for the bibliometric analysis began on December 3, 2021, when the research team started the process of familiarizing themselves with the software tools that may be used in this analysis, VOSViewer and CiteSpace, during which they consulted 3 librarians at the Yale University regarding search terms and tentative results. Tentative searches on the aforementioned databases yielded a total of 2340 papers. The official search phase started on July 27, 2023. Our goal is to complete the screening of papers and the analysis by February 15, 2024.

CONCLUSIONS

Artificial intelligence chatbots, such as ChatGPT (OpenAI Inc), have sparked numerous discussions within the health care industry regarding their impact on human health. Chatbot technology holds substantial promise for advancing health care systems worldwide. However, developing a sophisticated chatbot capable of precise interaction with health care consumers, delivering personalized care, and providing accurate health-related information and knowledge remain considerable challenges. This bibliometric analysis seeks to fill the knowledge gap in the existing literature on health-related chatbots, entailing their applications, the software used in their development, and their preferred functionalities among users.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/54349.

Development of an online service for coping with spousal loss by means of human-centered and stakeholder-inclusive design: the case of LEAVES

To support older mourners after the loss of their partner, LEAVES, an online self-help service that delivers the LIVIA spousal bereavement intervention, was developed. It integrates an embodied conversational agent and an initial risk assessment. Based on an iterative, human-centered, and stakeholder inclusive approach, interviews with older mourners and focus groups with stakeholders were conducted to understand their perspective on grief and on using LEAVES. Subsequently, the resulting technology and service model were evaluated by means of interviews, focus groups, and an online survey. While digital literacy remains a challenge, LEAVES shows promise of being supportive to the targeted end-users.

Demographic and clinical characteristics associated with anxiety and depressive symptom outcomes in users of a digital mental health intervention incorporating a relational agent

BACKGROUND

Digital mental health interventions (DMHIs) may reduce treatment access issues for those experiencing depressive and/or anxiety symptoms. DMHIs that incorporate relational agents may offer unique ways to engage and respond to users and to potentially help reduce provider burden. This study tested Woebot for Mood & Anxiety (W-MA-02), a DMHI that employs Woebot, a relational agent that incorporates elements of several evidence-based psychotherapies, among those with baseline clinical levels of depressive or anxiety symptoms. Changes in self-reported depressive and anxiety symptoms over 8 weeks were measured, along with the association between each of these outcomes and demographic and clinical characteristics.

METHODS

This exploratory, single-arm, 8-week study of 256 adults yielded non-mutually exclusive subsamples with either clinical levels of depressive or anxiety symptoms at baseline. Week 8 Patient Health Questionnaire-8 (PHQ-8) changes were measured in the depressive subsample (PHQ-8 ≥ 10). Week 8 Generalized Anxiety Disorder-7 (GAD-7) changes were measured in the anxiety subsample (GAD-7 ≥ 10). Demographic and clinical characteristics were examined in association with symptom changes via bivariate and multiple regression models adjusted for W-MA-02 utilization. Characteristics included age, sex at birth, race/ethnicity, marital status, education, sexual orientation, employment status, health insurance, baseline levels of depressive and anxiety symptoms, and concurrent psychotherapeutic or psychotropic medication treatments during the study.

RESULTS

Both the depressive and anxiety subsamples were predominantly female, educated, non-Hispanic white, and averaged 38 and 37 years of age, respectively. The depressive subsample had significant reductions in depressive symptoms at Week 8 (mean change =-7.28, SD = 5.91, Cohen's d = -1.23, p < 0.01); the anxiety subsample had significant reductions in anxiety symptoms at Week 8 (mean change = -7.45, SD = 5.99, Cohen's d = -1.24, p < 0.01). No significant associations were found between sex at birth, age, employment status, educational background and Week 8 symptom changes. Significant associations between depressive and anxiety symptom outcomes and sexual orientation, marital status, concurrent mental health treatment, and baseline symptom severity were found.

CONCLUSIONS

The present study suggests early promise for W-MA-02 as an intervention for depression and/or anxiety symptoms. Although exploratory in nature, this study revealed potential user characteristics associated with outcomes that can be investigated in future studies.

TRIAL REGISTRATION

This study was retrospectively registered on ClinicalTrials.gov (#NCT05672745) on January 5th, 2023.

Embodied Conversational Agents for Chronic Diseases: Scoping Review

BACKGROUND

Embodied conversational agents (ECAs) are computer-generated animated humanlike characters that interact with users through verbal and nonverbal behavioral cues. They are increasingly used in a range of fields, including health care.

OBJECTIVE

This scoping review aims to identify the current practice in the development and evaluation of ECAs for chronic diseases.

METHODS

We applied a methodological framework in this review. A total of 6 databases (ie, PubMed, Embase, CINAHL, ACM Digital Library, IEEE Xplore Digital Library, and Web of Science) were searched using a combination of terms related to ECAs and health in October 2023. Two independent reviewers selected the studies and extracted the data. This review followed the PRISMA-ScR (Preferred Reporting Items of Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) statement.

RESULTS

The literature search found 6332 papers, of which 36 (0.57%) met the inclusion criteria. Among the 36 studies, 27 (75%) originated from the United States, and 28 (78%) were published from 2020 onward. The reported ECAs covered a wide range of chronic diseases, with a focus on cancers, atrial fibrillation, and type 2 diabetes, primarily to promote screening and self-management. Most ECAs were depicted as middle-aged women based on screenshots and communicated with users through voice and nonverbal behavior. The most frequently reported evaluation outcomes were acceptability and effectiveness.

CONCLUSIONS

This scoping review provides valuable insights for technology developers and health care professionals regarding the development and implementation of ECAs. It emphasizes the importance of technological advances in the embodiment, personalized strategy, and communication modality and requires in-depth knowledge of user preferences regarding appearance, animation, and intervention content. Future studies should incorporate measures of cost, efficiency, and productivity to provide a comprehensive evaluation of the benefits of using ECAs in health care.

ChatGPT is not ready yet for use in providing mental health assessment and interventions

Background

Psychiatry is a specialized field of medicine that focuses on the diagnosis, treatment, and prevention of mental health disorders. With advancements in technology and the rise of artificial intelligence (AI), there has been a growing interest in exploring the potential of AI language models systems, such as Chat Generative Pre-training Transformer (ChatGPT), to assist in the field of psychiatry.

Objective

Our study aimed to evaluates the effectiveness, reliability and safeness of ChatGPT in assisting patients with mental health problems, and to assess its potential as a collaborative tool for mental health professionals through a simulated interaction with three distinct imaginary patients.

Methods

Three imaginary patient scenarios (cases A, B, and C) were created, representing different mental health problems. All three patients present with, and seek to eliminate, the same chief complaint (i.e., difficulty falling asleep and waking up frequently during the night in the last 2°weeks). ChatGPT was engaged as a virtual psychiatric assistant to provide responses and treatment recommendations.

Results

In case A, the recommendations were relatively appropriate (albeit non-specific), and could potentially be beneficial for both users and clinicians. However, as complexity of clinical cases increased (cases B and C), the information and recommendations generated by ChatGPT became inappropriate, even dangerous; and the limitations of the program became more glaring. The main strengths of ChatGPT lie in its ability to provide quick responses to user queries and to simulate empathy. One notable limitation is ChatGPT inability to interact with users to collect further information relevant to the diagnosis and management of a patient's clinical condition. Another serious limitation is ChatGPT inability to use critical thinking and clinical judgment to drive patient's management.

Conclusion

As for July 2023, ChatGPT failed to give the simple medical advice given certain clinical scenarios. This supports that the quality of ChatGPT-generated content is still far from being a guide for users and professionals to provide accurate mental health information. It remains, therefore, premature to conclude on the usefulness and safety of ChatGPT in mental health practice.

A mobile healthy lifestyle intervention to promote mental health in adolescence: a mixed-methods evaluation

BACKGROUND

A healthy lifestyle may improve mental health. It is yet not known whether and how a mobile intervention can be of help in achieving this in adolescents. This study investigated the effectiveness and perceived underlying mechanisms of the mobile health (mHealth) intervention #LIFEGOALS to promote healthy lifestyles and mental health. #LIFEGOALS is an evidence-based app with activity tracker, including self-regulation techniques, gamification elements, a support chatbot, and health narrative videos.

METHODS

A quasi-randomized controlled trial (N = 279) with 12-week intervention period and process evaluation interviews (n = 13) took place during the COVID-19 pandemic. Adolescents (12-15y) from the general population were allocated at school-level to the intervention (n = 184) or to a no-intervention group (n = 95). Health-related quality of life (HRQoL), psychological well-being, mood, self-perception, peer support, resilience, depressed feelings, sleep quality and breakfast frequency were assessed via a web-based survey; physical activity, sedentary time, and sleep routine via Axivity accelerometers. Multilevel generalized linear models were fitted to investigate intervention effects and moderation by pandemic-related measures. Interviews were coded using thematic analysis.

RESULTS

Non-usage attrition was high: 18% of the participants in the intervention group never used the app. An additional 30% stopped usage by the second week. Beneficial intervention effects were found for physical activity (χ21 = 4.36, P = .04), sedentary behavior (χ21 = 6.44, P = .01), sleep quality (χ21 = 6.11, P = .01), and mood (χ21 = 2.30, P = .02). However, effects on activity-related behavior were only present for adolescents having normal sports access, and effects on mood only for adolescents with full in-school education. HRQoL (χ22 = 14.72, P < .001), mood (χ21 = 6.03, P = .01), and peer support (χ21 = 13.69, P < .001) worsened in adolescents with pandemic-induced remote-education. Interviewees reported that the reward system, self-regulation guidance, and increased health awareness had contributed to their behavior change. They also pointed to the importance of social factors, quality of technology and autonomy for mHealth effectiveness.

CONCLUSIONS

#LIFEGOALS showed mixed results on health behaviors and mental health. The findings highlight the role of contextual factors for mHealth promotion in adolescence, and provide suggestions to optimize support by a chatbot and narrative episodes.

TRIAL REGISTRATION

ClinicalTrials.gov [NCT04719858], registered on 22/01/2021.

Exploring the characteristics of conversational agents in chronic disease management interventions: A scoping review

Objective

With the increasing global burden of chronic diseases, there is the potential for conversational agents (CAs) to assist people in actively managing their conditions. This paper reviews different types of CAs used for chronic condition management, delving into their characteristics and the chosen study designs. This paper also discusses the potential of these CAs to enhance the health and well-being of people with chronic conditions.

Methods

A search was performed in February 2023 on PubMed, ACM Digital Library, Scopus, and IEEE Xplore. Studies were included if they focused on chronic disease management or prevention and if systems were evaluated on target user groups.

Results

The 42 selected studies explored diverse types of CAs across 11 health conditions. Personalization varied, with 25 CAs not adapting message content, while others incorporated user characteristics and real-time context. Only 12 studies used medical records in conjunction with CAs for conditions like diabetes, mental health, cardiovascular issues, and cancer. Despite measurement method variations, the studies predominantly emphasized improved health outcomes and positive user attitudes toward CAs.

Conclusions

The results underscore the need for CAs to adapt to evolving patient needs, customize interventions, and incorporate human support and medical records for more effective care. It also highlights the potential of CAs to play a more active role in helping individuals manage their conditions and notes the value of linguistic data generated during user interactions. The analysis acknowledges its limitations and encourages further research into the use and potential of CAs in disease-specific contexts.

Factors influencing patient engagement in mental health chatbots: A thematic analysis of findings from a systematic review of reviews

Introduction

Mental health disorders affect millions of people worldwide. Chatbots are a new technology that can help users with mental health issues by providing innovative features. This article aimed to conduct a systematic review of reviews on chatbots in mental health services and synthesized the evidence on the factors influencing patient engagement with chatbots.

Methods

This study reviewed the literature from 2000 to 2024 using qualitative analysis. The authors conducted a systematic search of several databases, such as PubMed, Scopus, ProQuest, and Cochrane database of systematic reviews, to identify relevant studies on the topic. The quality of the selected studies was assessed using the Critical Appraisal Skills Programme appraisal checklist and the data obtained from the systematic review were subjected to a thematic analysis utilizing the Boyatzis's code development approach.

Results

The database search resulted in 1494 papers, of which 10 were included in the study after the screening process. The quality assessment of the included studies scored the papers within a moderate level. The thematic analysis revealed four main themes: chatbot design, chatbot outcomes, user perceptions, and user characteristics.

Conclusion

The research proposed some ways to use color and music in chatbot design. It also provided a systematic and multidimensional analysis of the factors, offered some insights for chatbot developers and researchers, and highlighted the potential of chatbots to improve patient-centered and person-centered care in mental health services.

My Health Coach: Community members' perspectives on a mobile health tool for adults with fetal alcohol spectrum disorders

Objectives

Fetal alcohol spectrum disorders (FASD) affect the health and development of people across the lifespan. Adults with FASD experience significant barriers to care. Accessible and scalable solutions are needed. In partnership with members of the International Adult Leadership Collaborative of FASD Changemakers, an international group of adults with FASD, we developed a mobile health (mHealth) application based on self-determination theory (SDT), called "My Health Coach," to promote self-management and health advocacy.

Methods

This project follows an established user-centered design approach to app development and evaluation, allowing for feedback loops promoting iterative change. Research staff and ALC members co-led online focus groups (n = 26) and an online follow-up survey (n = 26) with adults with FASD to elicit feedback on completed design prototypes. Focus group transcriptions and surveys underwent systemic thematic and theoretical framework analysis.

Results

Analyses show overall positive impressions of the My Health Coach app. Participants were enthusiastic about the proposed features and tools the app will provide. Discussions and free responses revealed SDT constructs (autonomy, competence, relatedness) are a strong fit with participants' perceived outcomes shared in their evaluation of the prototype. Interesting recommendations were made for additional features that would further promote SDT constructs.

Conclusions

This project demonstrates advantages of community-engaged partnerships in FASD research. Adults with FASD have a strong interest in scalable mHealth tools and described the acceptability of our initial design. App features and tools promoted SDT constructs.

An Integrative Survey on Mental Health Conversational Agents to Bridge Computer Science and Medical Perspectives

Mental health conversational agents (a.k.a. chatbots) are widely studied for their potential to offer accessible support to those experiencing mental health challenges. Previous surveys on the topic primarily consider papers published in either computer science or medicine, leading to a divide in understanding and hindering the sharing of beneficial knowledge between both domains. To bridge this gap, we conduct a comprehensive literature review using the PRISMA framework, reviewing 534 papers published in both computer science and medicine. Our systematic review reveals 136 key papers on building mental health-related conversational agents with diverse characteristics of modeling and experimental design techniques. We find that computer science papers focus on LLM techniques and evaluating response quality using automated metrics with little attention to the application while medical papers use rule-based conversational agents and outcome metrics to measure the health outcomes of participants. Based on our findings on transparency, ethics, and cultural heterogeneity in this review, we provide a few recommendations to help bridge the disciplinary divide and enable the cross-disciplinary development of mental health conversational agents.

Exploring Young Adults' Views About Aroha, a Chatbot for Stress Associated With the COVID-19 Pandemic: Interview Study Among Students

BACKGROUND

In March 2020, New Zealand was plunged into its first nationwide lockdown to halt the spread of COVID-19. Our team rapidly adapted our existing chatbot platform to create Aroha, a well-being chatbot intended to address the stress experienced by young people aged 13 to 24 years in the early phase of the pandemic. Aroha was made available nationally within 2 weeks of the lockdown and continued to be available throughout 2020.

OBJECTIVE

In this study, we aimed to evaluate the acceptability and relevance of the chatbot format and Aroha's content in young adults and to identify areas for improvement.

METHODS

We conducted qualitative in-depth and semistructured interviews with young adults as well as in situ demonstrations of Aroha to elicit immediate feedback. Interviews were recorded, transcribed, and analyzed using thematic analysis assisted by NVivo (version 12; QSR International).

RESULTS

A total of 15 young adults (age in years: median 20; mean 20.07, SD 3.17; female students: n=13, 87%; male students: n=2, 13%; all tertiary students) were interviewed in person. Participants spoke of the challenges of living during the lockdown, including social isolation, loss of motivation, and the demands of remote work or study, although some were able to find silver linings. Aroha was well liked for sounding like a "real person" and peer with its friendly local "Kiwi" communication style, rather than an authoritative adult or counselor. The chatbot was praised for including content that went beyond traditional mental health advice. Participants particularly enjoyed the modules on gratitude, being active, anger management, job seeking, and how to deal with alcohol and drugs. Aroha was described as being more accessible than traditional mental health counseling and resources. It was an appealing option for those who did not want to talk to someone in person for fear of the stigma associated with mental health. However, participants disliked the software bugs. They also wanted a more sophisticated conversational interface where they could express themselves and "vent" in free text. There were several suggestions for making Aroha more relevant to a diverse range of users, including developing content on navigating relationships and diverse chatbot avatars.

CONCLUSIONS

Chatbots are an acceptable format for scaling up the delivery of public mental health and well-being-enhancing strategies. We make the following recommendations for others interested in designing and rolling out mental health chatbots to better support young people: make the chatbot relatable to its target audience by working with them to develop an authentic and relevant communication style; consider including holistic health and lifestyle content beyond traditional "mental health" support; and focus on developing features that make users feel heard, understood, and empowered.

Factors Influencing the Acceptability, Acceptance, and Adoption of Conversational Agents in Health Care: Integrative Review

BACKGROUND

Conversational agents (CAs), also known as chatbots, are digital dialog systems that enable people to have a text-based, speech-based, or nonverbal conversation with a computer or another machine based on natural language via an interface. The use of CAs offers new opportunities and various benefits for health care. However, they are not yet ubiquitous in daily practice. Nevertheless, research regarding the implementation of CAs in health care has grown tremendously in recent years.

OBJECTIVE

This review aims to present a synthesis of the factors that facilitate or hinder the implementation of CAs from the perspectives of patients and health care professionals. Specifically, it focuses on the early implementation outcomes of acceptability, acceptance, and adoption as cornerstones of later implementation success.

METHODS

We performed an integrative review. To identify relevant literature, a broad literature search was conducted in June 2021 with no date limits and using all fields in PubMed, Cochrane Library, Web of Science, LIVIVO, and PsycINFO. To keep the review current, another search was conducted in March 2022. To identify as many eligible primary sources as possible, we used a snowballing approach by searching reference lists and conducted a hand search. Factors influencing the acceptability, acceptance, and adoption of CAs in health care were coded through parallel deductive and inductive approaches, which were informed by current technology acceptance and adoption models. Finally, the factors were synthesized in a thematic map.

RESULTS

Overall, 76 studies were included in this review. We identified influencing factors related to 4 core Unified Theory of Acceptance and Use of Technology (UTAUT) and Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) factors (performance expectancy, effort expectancy, facilitating conditions, and hedonic motivation), with most studies underlining the relevance of performance and effort expectancy. To meet the particularities of the health care context, we redefined the UTAUT2 factors social influence, habit, and price value. We identified 6 other influencing factors: perceived risk, trust, anthropomorphism, health issue, working alliance, and user characteristics. Overall, we identified 10 factors influencing acceptability, acceptance, and adoption among health care professionals (performance expectancy, effort expectancy, facilitating conditions, social influence, price value, perceived risk, trust, anthropomorphism, working alliance, and user characteristics) and 13 factors influencing acceptability, acceptance, and adoption among patients (additionally hedonic motivation, habit, and health issue).

CONCLUSIONS

This review shows manifold factors influencing the acceptability, acceptance, and adoption of CAs in health care. Knowledge of these factors is fundamental for implementation planning. Therefore, the findings of this review can serve as a basis for future studies to develop appropriate implementation strategies. Furthermore, this review provides an empirical test of current technology acceptance and adoption models and identifies areas where additional research is necessary.

TRIAL REGISTRATION

PROSPERO CRD42022343690; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=343690.

Can digital health researchers make a difference during the pandemic? Results of the single-arm, chatbot-led Elena+: Care for COVID-19 interventional study

Background

The current paper details findings from Elena+: Care for COVID-19, an app developed to tackle the collateral damage of lockdowns and social distancing, by offering pandemic lifestyle coaching across seven health areas: anxiety, loneliness, mental resources, sleep, diet and nutrition, physical activity, and COVID-19 information.

Methods

The Elena+ app functions as a single-arm interventional study, with participants recruited predominantly via social media. We used paired samples T-tests and within subjects ANOVA to examine changes in health outcome assessments and user experience evaluations over time. To investigate the mediating role of behavioral activation (i.e., users setting behavioral intentions and reporting actual behaviors) we use mixed-effect regression models. Free-text entries were analyzed qualitatively.

Results

Results show strong demand for publicly available lifestyle coaching during the pandemic, with total downloads (N = 7'135) and 55.8% of downloaders opening the app (n = 3,928) with 9.8% completing at least one subtopic (n = 698). Greatest areas of health vulnerability as assessed with screening measures were physical activity with 62% (n = 1,000) and anxiety with 46.5% (n = 760). The app was effective in the treatment of mental health; with a significant decrease in depression between first (14 days), second (28 days), and third (42 days) assessments: F2,38 = 7.01, p = 0.003, with a large effect size (η2G = 0.14), and anxiety between first and second assessments: t54 = 3.7, p = <0.001 with a medium effect size (Cohen d = 0.499). Those that followed the coaching program increased in net promoter score between the first and second assessment: t36 = 2.08, p = 0.045 with a small to medium effect size (Cohen d = 0.342). Mediation analyses showed that while increasing number of subtopics completed increased behavioral activation (i.e., match between behavioral intentions and self-reported actual behaviors), behavioral activation did not mediate the relationship to improvements in health outcome assessments.

Conclusions

Findings show that: (i) there is public demand for chatbot led digital coaching, (ii) such tools can be effective in delivering treatment success, and (iii) they are highly valued by their long-term user base. As the current intervention was developed at rapid speed to meet the emergency pandemic context, the future looks bright for other public health focused chatbot-led digital health interventions.

Feasibility and impact of a mental health chatbot on postpartum mental health: a randomized controlled trial

BACKGROUND

Perinatal mood disorders are common yet underdiagnosed and un- or undertreated. Barriers exist to accessing perinatal mental health services, including limited availability, time, and cost. Automated conversational agents (chatbots) can deliver evidence-based cognitive behavioral therapy content through text message-based conversations and reduce depression and anxiety symptoms in select populations. Such digital mental health technologies are poised to overcome barriers to mental health care access but need to be evaluated for efficacy, as well as for preliminary feasibility and acceptability among perinatal populations.

OBJECTIVE

To evaluate the acceptability and preliminary efficacy of a mental health chatbot for mood management in a general postpartum population.

STUDY DESIGN

An unblinded randomized controlled trial was conducted at a tertiary academic center. English-speaking postpartum women aged 18 years or above with a live birth and access to a smartphone were eligible for enrollment prior to discharge from delivery hospitalization. Baseline surveys were administered to all participants prior to randomization to a mental health chatbot intervention or to usual care only. The intervention group downloaded the mental health chatbot smartphone application with perinatal-specific content, in addition to continuing usual care. Usual care consisted of routine postpartum follow up and mental health care as dictated by the patient's obstetric provider. Surveys were administered during delivery hospitalization (baseline) and at 2-, 4-, and 6-weeks postpartum to assess depression and anxiety symptoms. The primary outcome was a change in depression symptoms at 6-weeks as measured using two depression screening tools: Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale. Secondary outcomes included anxiety symptoms measured using Generalized Anxiety Disorder-7, and satisfaction and acceptability using validated scales. Based on a prior study, we estimated a sample size of 130 would have sufficient (80%) power to detect a moderate effect size (d=.4) in between group difference on the Patient Health Questionnaire-9.

RESULTS

A total of 192 women were randomized equally 1:1 to the chatbot or usual care; of these, 152 women completed the 6-week survey (n=68 chatbot, n=84 usual care) and were included in the final analysis. Mean baseline mental health assessment scores were below positive screening thresholds. At 6-weeks, there was a greater decrease in Patient Health Questionnaire-9 scores among the chatbot group compared to the usual care group (mean decrease=1.32, standard deviation=3.4 vs mean decrease=0.13, standard deviation=3.01, respectively). 6-week mean Edinburgh Postnatal Depression Scale and Generalized Anxiety Disorder-7 scores did not differ between groups and were similar to baseline. 91% (n=62) of the chatbot users were satisfied or highly satisfied with the chatbot, and 74% (n=50) of the intervention group reported use of the chatbot at least once in 2 weeks prior to the 6-week survey. 80% of study participants reported being comfortable with the use of a mobile smartphone application for mood management.

CONCLUSION

Use of a chatbot was acceptable to women in the early postpartum period. The sample did not screen positive for depression at baseline and thus the potential of the chatbot to reduce depressive symptoms in this population was limited. This study was conducted in a general obstetric population. Future studies of longer duration in high-risk postpartum populations who screen positive for depression are needed to further understand the utility and efficacy of such digital therapeutics for that population.

The Use of Voice Assistant for Psychological Assessment Elicits Empathy and Engagement While Maintaining Good Psychometric Properties

This study aimed to use the Alexa vocal assistant as an administerer of psychometric tests, assessing the efficiency and validity of this measurement. A total of 300 participants were administered the Interpersonal Reactivity Index (IRI). After a week, the administration was repeated, but the participants were randomly divided into groups of 100 participants each. In the first, the test was administered by means of a paper version; in the second, the questionnaire was read to the participants in person, and the operator contemporaneously recorded the answers declared by the participants; in the third group, the questionnaire was directly administered by the Alexa voice device, after specific reprogramming. The third group was also administered, as a post-session survey, the Engagement and Perceptions of the Bot Scale (EPVS), a short version of the Communication Styles Inventory (CSI), the Marlowe-Crowne Social Desirability Scale (MCSDS), and an additional six items to measure degrees of concentration, ease, and perceived pressure at the beginning and at the end of the administration. The results confirmed that the IRI did keep measurement invariance within the three conditions. The administration through vocal assistant showed an empathic activation effect significantly superior to the conditions of pencil-paper and operator-in-presence. The results indicated an engagement and positive evaluation of the interactive experience, with reported perceptions of closeness, warmth, competence, and human-likeness associated with higher values of empathetic activation and lower values of personal discomfort.

Using Chatbot Technology to Improve Brazilian Adolescents' Body Image and Mental Health at Scale: Randomized Controlled Trial

BACKGROUND

Accessible, cost-effective, and scalable mental health interventions are limited, particularly in low- and middle-income countries, where disparities between mental health needs and services are greatest. Microinterventions (ie, brief, stand-alone, or digital approaches) aim to provide immediate reprieve and enhancements in mental health states and offer a novel and scalable framework for embedding evidence-based mental health promotion techniques into digital environments. Body image is a global public health issue that increases young peoples' risk of developing more severe mental and physical health issues. Embedding body image microinterventions into digital environments is one avenue for providing young people with immediate and short-term reprieve and protection from the negative exposure effects associated with social media.

OBJECTIVE

This 2-armed, fully remote, and preregistered randomized controlled trial assessed the impact of a body image chatbot containing microinterventions on Brazilian adolescents' state and trait body image and associated well-being outcomes.

METHODS

Geographically diverse Brazilian adolescents aged 13-18 years (901/1715, 52.54% girls) were randomized into the chatbot or an assessment-only control condition and completed web-based self-assessments at baseline, immediately after the intervention time frame, and at 1-week and 1-month follow-ups. The primary outcomes were mean change in state (at chatbot entry and at the completion of a microintervention technique) and trait body image (before and after the intervention), with the secondary outcomes being mean change in affect (state and trait) and body image self-efficacy between the assessment time points.

RESULTS

Most participants who entered the chatbot (258/327, 78.9%) completed ≥1 microintervention technique, with participants completing an average of 5 techniques over the 72-hour intervention period. Chatbot users experienced small significant improvements in primary (state: P<.001, Cohen d=0.30, 95% CI 0.25-0.34; and trait body image: P=.02, Cohen d range=0.10, 95% CI 0.01-0.18, to 0.26, 95% CI 0.13-0.32) and secondary outcomes across various time points (state: P<.001, Cohen d=0.28, 95% CI 0.22-0.33; trait positive affect: P=.02, Cohen d range=0.15, 95% CI 0.03-0.27, to 0.23, 95% CI 0.08-0.37; negative affect: P=.03, Cohen d range=-0.16, 95% CI -0.30 to -0.02, to -0.18, 95% CI -0.33 to -0.03; and self-efficacy: P=.02, Cohen d range=0.14, 95% CI 0.03-0.25, to 0.19, 95% CI 0.08-0.32) relative to the control condition. Intervention benefits were moderated by baseline levels of concerns but not by gender.

CONCLUSIONS

This is the first large-scale randomized controlled trial assessing a body image chatbot among Brazilian adolescents. Intervention attrition was high (531/858, 61.9%) and reflected the broader digital intervention literature; barriers to engagement were discussed. Meanwhile, the findings support the emerging literature that indicates microinterventions and chatbot technology are acceptable and effective web-based service provisions. This study also offers a blueprint for accessible, cost-effective, and scalable digital approaches that address disparities between health care needs and provisions in low- and middle-income countries.

TRIAL REGISTRATION

Clinicaltrials.gov NCT04825184; http://clinicaltrials.gov/ct2/show/NCT04825184.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1186/s12889-021-12129-1.

Effectiveness and Acceptability of Conversational Agents for Smoking Cessation: A Systematic Review and Meta-analysis

INTRODUCTION

Conversational agents (CAs; computer programs that use artificial intelligence to simulate a conversation with users through natural language) have evolved considerably in recent years to support healthcare by providing autonomous, interactive, and accessible services, making them potentially useful for supporting smoking cessation. We performed a systematic review and meta-analysis to provide an overarching evaluation of their effectiveness and acceptability to inform future development and adoption.

AIMS AND METHODS

PsycInfo, Web of Science, ACM Digital Library, IEEE Xplore, Medline, EMBASE, Communication and Mass Media Complete, and CINAHL Complete were searched for studies examining the use of CAs for smoking cessation. Data from eligible studies were extracted and used for random-effects meta-analyses.

RESULTS

The search yielded 1245 publications with 13 studies eligible for systematic review (total N = 8236) and six studies for random-effects meta-analyses. All studies reported positive effects on cessation-related outcomes. A meta-analysis with randomized controlled trials reporting on abstinence yielded a sample-weighted odds ratio of 1.66 (95% CI = 1.33% to 2.07%, p < .001), favoring CAs over comparison groups. A narrative synthesis of all included studies showed overall high acceptability, while some barriers were identified from user feedback. Overall, included studies were diverse in design with mixed quality, and evidence of publication bias was identified. A lack of theoretical foundations was noted, as well as a clear need for relational communication in future designs.

CONCLUSIONS

The effectiveness and acceptability of CAs for smoking cessation are promising. However, standardization of reporting and designing of the agents is warranted for a more comprehensive evaluation.

IMPLICATIONS

This is the first systematic review to provide insight into the use of CAs to support smoking cessation. Our findings demonstrated initial promise in the effectiveness and user acceptability of these agents. We also identified a lack of theoretical and methodological limitations to improve future study design and intervention delivery.

Meeting the Behavioral Health Needs of Health Care Workers During COVID-19 by Leveraging Chatbot Technology: Development and Usability Study

BACKGROUND

During the COVID-19 pandemic, health care systems were faced with the urgent need to implement strategies to address the behavioral health needs of health care workers. A primary concern of any large health care system is developing an easy-to-access, streamlined system of triage and support despite limited behavioral health resources.

OBJECTIVE

This study provides a detailed description of the design and implementation of a chatbot program designed to triage and facilitate access to behavioral health assessment and treatment for the workforce of a large academic medical center. The University of California, San Francisco (UCSF) Faculty, Staff, and Trainee Coping and Resiliency Program (UCSF Cope) aimed to provide timely access to a live telehealth navigator for triage and live telehealth assessment and treatment, curated web-based self-management tools, and nontreatment support groups for those experiencing stress related to their unique roles.

METHODS

In a public-private partnership, the UCSF Cope team built a chatbot to triage employees based on behavioral health needs. The chatbot is an algorithm-based, automated, and interactive artificial intelligence conversational tool that uses natural language understanding to engage users by presenting a series of questions with simple multiple-choice answers. The goal of each chatbot session was to guide users to services that were appropriate for their needs. Designers developed a chatbot data dashboard to identify and follow trends directly through the chatbot. Regarding other program elements, website user data were collected monthly and participant satisfaction was gathered for each nontreatment support group.

RESULTS

The UCSF Cope chatbot was rapidly developed and launched on April 20, 2020. As of May 31, 2022, a total of 10.88% (3785/34,790) of employees accessed the technology. Among those reporting any form of psychological distress, 39.7% (708/1783) of employees requested in-person services, including those who had an existing provider. UCSF employees responded positively to all program elements. As of May 31, 2022, the UCSF Cope website had 615,334 unique users, with 66,585 unique views of webinars and 601,471 unique views of video shorts. All units across UCSF were reached by UCSF Cope staff for special interventions, with >40 units requesting these services. Town halls were particularly well received, with >80% of attendees reporting the experience as helpful.

CONCLUSIONS

UCSF Cope used chatbot technology to incorporate individualized behavioral health triage, assessment, treatment, and general emotional support for an entire employee base (N=34,790). This level of triage for a population of this size would not have been possible without the use of chatbot technology. The UCSF Cope model has the potential to be scaled, adapted, and implemented across both academically and nonacademically affiliated medical settings.

A Survey of Conversational Agents and Their Applications for Self-Management of Chronic Conditions

Conversational agents have gained their ground in our daily life and various domains including healthcare. Chronic condition self-management is one of the promising healthcare areas in which conversational agents demonstrate significant potential to contribute to alleviating healthcare burdens from chronic conditions. This survey paper introduces and outlines types of conversational agents, their generic architecture and workflow, the implemented technologies, and their application to chronic condition self-management.

Waiting for a digital therapist: three challenges on the path to psychotherapy delivered by artificial intelligence

Growing demand for broadly accessible mental health care, together with the rapid development of new technologies, trigger discussions about the feasibility of psychotherapeutic interventions based on interactions with Conversational Artificial Intelligence (CAI). Many authors argue that while currently available CAI can be a useful supplement for human-delivered psychotherapy, it is not yet capable of delivering fully fledged psychotherapy on its own. The goal of this paper is to investigate what are the most important obstacles on our way to developing CAI systems capable of delivering psychotherapy in the future. To this end, we formulate and discuss three challenges central to this quest. Firstly, we might not be able to develop effective AI-based psychotherapy unless we deepen our understanding of what makes human-delivered psychotherapy effective. Secondly, assuming that it requires building a therapeutic relationship, it is not clear whether psychotherapy can be delivered by non-human agents. Thirdly, conducting psychotherapy might be a problem too complicated for narrow AI, i.e., AI proficient in dealing with only relatively simple and well-delineated tasks. If this is the case, we should not expect CAI to be capable of delivering fully-fledged psychotherapy until the so-called "general" or "human-like" AI is developed. While we believe that all these challenges can ultimately be overcome, we think that being mindful of them is crucial to ensure well-balanced and steady progress on our path to AI-based psychotherapy.

Conversational Agents for Body Weight Management: Systematic Review

BACKGROUND

Obesity is a public health issue worldwide. Conversational agents (CAs), also frequently called chatbots, are computer programs that simulate dialogue between people. Owing to better accessibility, cost-effectiveness, personalization, and compassionate patient-centered treatments, CAs are expected to have the potential to provide sustainable lifestyle counseling for weight management.

OBJECTIVE

This systematic review aimed to critically summarize and evaluate clinical studies on the effectiveness and feasibility of CAs with unconstrained natural language input for weight management.

METHODS

PubMed, Embase, the Cochrane Library (CENTRAL), PsycINFO, and ACM Digital Library were searched up to December 2022. Studies were included if CAs were used for weight management and had a capability for unconstrained natural language input. No restrictions were imposed on study design, language, or publication type. The quality of the included studies was assessed using the Cochrane risk-of-bias assessment tool or the Critical Appraisal Skills Programme checklist. The extracted data from the included studies were tabulated and narratively summarized as substantial heterogeneity was expected.

RESULTS

In total, 8 studies met the eligibility criteria: 3 (38%) randomized controlled trials and 5 (62%) uncontrolled before-and-after studies. The CAs in the included studies were aimed at behavior changes through education, advice on food choices, or counseling via psychological approaches. Of the included studies, only 38% (3/8) reported a substantial weight loss outcome (1.3-2.4 kg decrease at 12-15 weeks of CA use). The overall quality of the included studies was judged as low.

CONCLUSIONS

The findings of this systematic review suggest that CAs with unconstrained natural language input can be used as a feasible interpersonal weight management intervention by promoting engagement in psychiatric intervention-based conversations simulating treatments by health care professionals, but currently there is a paucity of evidence. Well-designed rigorous randomized controlled trials with larger sample sizes, longer treatment duration, and follow-up focusing on CAs' acceptability, efficacy, and safety are warranted.

The Chatbots Are Invading Us: A Map Point on the Evolution, Applications, Opportunities, and Emerging Problems in the Health Domain

The inclusion of chatbots is potentially disruptive in society, introducing opportunities, but also important implications that need to be addressed on different domains. The aim of this study is to examine chatbots in-depth, by mapping out their technological evolution, current usage, and potential applications, opportunities, and emerging problems within the health domain. The study examined three points of view. The first point of view traces the technological evolution of chatbots. The second point of view reports the fields of application of the chatbots, giving space to the expectations of use and the expected benefits from a cross-domain point of view, also affecting the health domain. The third and main point of view is that of the analysis of the state of use of chatbots in the health domain based on the scientific literature represented by systematic reviews. The overview identified the topics of greatest interest with the opportunities. The analysis revealed the need for initiatives that simultaneously evaluate multiple domains all together in a synergistic way. Concerted efforts to achieve this are recommended. It is also believed to monitor both the process of osmosis between other sectors and the health domain, as well as the chatbots that can create psychological and behavioural problems with an impact on the health domain.

Conversational Agent Interventions for Mental Health Problems: Systematic Review and Meta-analysis of Randomized Controlled Trials

BACKGROUND

Mental health problems are a crucial global public health concern. Owing to their cost-effectiveness and accessibility, conversational agent interventions (CAIs) are promising in the field of mental health care.

OBJECTIVE

This study aims to present a thorough summary of the traits of CAIs available for a range of mental health problems, find evidence of efficacy, and analyze the statistically significant moderators of efficacy via a meta-analysis of randomized controlled trial.

METHODS

Web-based databases (Embase, MEDLINE, PsycINFO, CINAHL, Web of Science, and Cochrane) were systematically searched dated from the establishment of the database to October 30, 2021, and updated to May 1, 2022. Randomized controlled trials comparing CAIs with any other type of control condition in improving depressive symptoms, generalized anxiety symptoms, specific anxiety symptoms, quality of life or well-being, general distress, stress, mental disorder symptoms, psychosomatic disease symptoms, and positive and negative affect were considered eligible. This study followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Data were extracted by 2 independent reviewers, checked by a third reviewer, and pooled using both random effect models and fixed effects models. Hedges g was chosen as the effect size.

RESULTS

Of the 6900 identified records, a total of 32 studies were included, involving 6089 participants. CAIs showed statistically significant short-term effects compared with control conditions in improving depressive symptoms (g=0.29, 95% CI 0.20-0.38), generalized anxiety symptoms (g=0.29, 95% CI 0.21-0.36), specific anxiety symptoms (g=0.47, 95% CI 0.07-0.86), quality of life or well-being (g=0.27, 95% CI 0.16-0.39), general distress (g=0.33, 95% CI 0.20-0.45), stress (g=0.24, 95% CI 0.08-0.41), mental disorder symptoms (g=0.36, 95% CI 0.17-0.54), psychosomatic disease symptoms (g=0.62, 95% CI 0.14-1.11), and negative affect (g=0.28, 95% CI 0.05-0.51). However, the long-term effects of CAIs for the most mental health outcomes were not statistically significant (g=-0.04 to 0.39). Personalization and empathic response were 2 critical facilitators of efficacy. The longer duration of interaction with conversational agents was associated with the larger pooled effect sizes.

CONCLUSIONS

The findings show that CAIs are research-proven interventions that ought to be implemented more widely in mental health care. CAIs are effective and easily acceptable for those with mental health problems. The clinical application of this novel digital technology will conserve human health resources and optimize the allocation of mental health services.

TRIAL REGISTRATION

PROSPERO CRD42022350130; https://tinyurl.com/mvhk6w9p.

Reinventing mental health care in youth through mobile approaches: Current status and future steps

In this perspective, we aim to bring together research on mobile assessments and interventions in the context of mental health care in youth. After the COVID-19 pandemic, one out of five young people is experiencing mental health problems worldwide. New ways to face this burden are now needed. Young people search for low-burden services in terms of costs and time, paired with high flexibility and easy accessibility. Mobile applications meet these principles by providing new ways to inform, monitor, educate, and enable self-help, thus reinventing mental health care in youth. In this perspective, we explore the existing literature reviews on mobile assessments and interventions in youth through data collected passively (e.g., digital phenotyping) and actively (e.g., using Ecological Momentary Assessments-EMAs). The richness of such approaches relies on assessing mental health dynamically by extending beyond the confines of traditional methods and diagnostic criteria, and the integration of sensor data from multiple channels, thus allowing the cross-validation of symptoms through multiple information. However, we also acknowledge the promises and pitfalls of such approaches, including the problem of interpreting small effects combined with different data sources and the real benefits in terms of outcome prediction when compared to gold-standard methods. We also explore a new promising and complementary approach, using chatbots and conversational agents, that encourages interaction while tracing health and providing interventions. Finally, we suggest that it is important to continue to move beyond the ill-being framework by giving more importance to intervention fostering well-being, e.g., using positive psychology.

Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review

BACKGROUND

Artificial intelligence (AI)-based chatbots can offer personalized, engaging, and on-demand health promotion interventions.

OBJECTIVE

The aim of this systematic review was to evaluate the feasibility, efficacy, and intervention characteristics of AI chatbots for promoting health behavior change.

METHODS

A comprehensive search was conducted in 7 bibliographic databases (PubMed, IEEE Xplore, ACM Digital Library, PsycINFO, Web of Science, Embase, and JMIR publications) for empirical articles published from 1980 to 2022 that evaluated the feasibility or efficacy of AI chatbots for behavior change. The screening, extraction, and analysis of the identified articles were performed by following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

RESULTS

Of the 15 included studies, several demonstrated the high efficacy of AI chatbots in promoting healthy lifestyles (n=6, 40%), smoking cessation (n=4, 27%), treatment or medication adherence (n=2, 13%), and reduction in substance misuse (n=1, 7%). However, there were mixed results regarding feasibility, acceptability, and usability. Selected behavior change theories and expert consultation were used to develop the behavior change strategies of AI chatbots, including goal setting, monitoring, real-time reinforcement or feedback, and on-demand support. Real-time user-chatbot interaction data, such as user preferences and behavioral performance, were collected on the chatbot platform to identify ways of providing personalized services. The AI chatbots demonstrated potential for scalability by deployment through accessible devices and platforms (eg, smartphones and Facebook Messenger). The participants also reported that AI chatbots offered a nonjudgmental space for communicating sensitive information. However, the reported results need to be interpreted with caution because of the moderate to high risk of internal validity, insufficient description of AI techniques, and limitation for generalizability.

CONCLUSIONS

AI chatbots have demonstrated the efficacy of health behavior change interventions among large and diverse populations; however, future studies need to adopt robust randomized control trials to establish definitive conclusions.

Comparing the Feasibility and Acceptability of a Virtual Human, Teletherapy, and an e-Manual in Delivering a Stress Management Intervention to Distressed Adult Women: Pilot Study

BACKGROUND

Virtual humans (VHs), teletherapy, and self-guided e-manuals may increase the accessibility of psychological interventions. However, there is limited research on how these technologies compare in terms of their feasibility and acceptability in delivering stress management interventions.

OBJECTIVE

We conducted a preliminary comparison of the feasibility and acceptability of a VH, teletherapy, and an e-manual at delivering 1 module of cognitive behavioral stress management (CBSM) to evaluate the feasibility of the trial methodology in preparation for a future randomized controlled trial (RCT).

METHODS

A pilot RCT was conducted with a parallel, mixed design. A community sample of distressed adult women were randomly allocated to receive 1 session of CBSM involving training in cognitive and behavioral techniques by a VH, teletherapy, or an e-manual plus homework over 2 weeks. Data were collected on the feasibility of the intervention technologies (technical support and homework access), trial methods (recruitment methods, questionnaire completion, and methodological difficulty observations), intervention acceptability (intervention completion, self-report ratings, therapist rapport, and trust), and acceptability of the trial methods (self-report ratings and observations). Qualitative data in the form of written responses to open-ended questions were collected to enrich and clarify the findings on intervention acceptability.

RESULTS

Overall, 38 participants' data were analyzed. A VH (n=12), teletherapy (n=12), and an e-manual (n=14) were found to be feasible and acceptable for delivering 1 session of CBSM to distressed adult women based on the overall quantitative and qualitative findings. Technical difficulties were minimal and did not affect intervention completion, and no significant differences were found between the conditions (P=.31). The methodology was feasible, although improvements were identified for a future trial. All conditions achieved good satisfaction and perceived engagement ratings, and no significant group differences were found (P>.40). Participants had similar willingness to recommend each technology (P=.64). There was a nonsignificant trend toward participants feeling more open to using the VH and e-manual from home than teletherapy (P=.10). Rapport (P<.001) and trust (P=.048) were greater with the human teletherapist than with the VH. The qualitative findings enriched the quantitative results by revealing the unique strengths and limitations of each technology that may have influenced acceptability.

CONCLUSIONS

A VH, teletherapy, and a self-guided e-manual were found to be feasible and acceptable methods of delivering 1 session of a stress management intervention to a community sample of adult women. The technologies were found to have unique strengths and limitations that may affect which works best for whom and in what circumstances. Future research should test additional CBSM modules for delivery by these technologies and conduct a larger RCT to compare their feasibility, acceptability, and effectiveness when delivering a longer home-based stress management program.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12620000859987; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=380114&isReview=true.

User experience with a parenting chatbot micro intervention

Background

The use of chatbots to address mental health conditions have become increasingly popular in recent years. However, few studies aimed to teach parenting skills through chatbots, and there are no reports on parental user experience. Aim: This study aimed to assess the user experience of a parenting chatbot micro intervention to teach how to praise children in a Spanish-speaking country.

Methods

A sample of 89 parents were assigned to the chatbot micro intervention as part of a randomized controlled trial study. Completion rates, engagement, satisfaction, net promoter score, and acceptability were analyzed.

Results

66.3% of the participants completed the intervention. Participants exchanged an average of 49.8 messages (SD = 1.53), provided an average satisfaction score of 4.19 (SD = .79), and reported that they would recommend the chatbot to other parents (net promoter score = 4.63/5; SD = .66). Acceptability level was high (ease of use = 4.66 [SD = .73]; comfortability = 4.76 [SD = .46]; lack of technical problems = 4.69 [SD = .59]; interactivity = 4.51 [SD = .77]; usefulness for everyday life = 4.75 [SD = .54]).

Conclusions

Overall, users completed the intervention at a high rate, engaged with the chatbot, were satisfied, would recommend it to others, and reported a high level of acceptability. Chatbots have the potential to teach parenting skills however research on the efficacy of parenting chatbot interventions is needed.

Finding the sweet spot: a qualitative study exploring patients' acceptability of chatbots in genetic service delivery

Chatbots, web-based artificial intelligence tools that simulate human conversation, are increasingly in use to support many areas of genomic medicine. However, patient preferences towards using chatbots across the range of clinical settings are unknown. We conducted a qualitative study with individuals who underwent genetic testing for themselves or their child. Participants were asked about their preferences for using a chatbot within the genetic testing journey. Thematic analysis employing interpretive description was used. We interviewed 30 participants (67% female, 50% 50 + years). Participants considered chatbots to be inefficient for very simple tasks (e.g., answering FAQs) or very complex tasks (e.g., explaining results). Chatbots were acceptable for moderately complex tasks where participants perceived a favorable return on their investment of time and energy. In addition to achieving this "sweet spot," participants anticipated that their comfort with chatbots would increase if the chatbot was used as a complement to but not a replacement for usual care. Participants wanted a "safety net" (i.e., access to a clinician) for needs not addressed by the chatbot. This study provides timely insights into patients' comfort with and perceived limitations of chatbots for genomic medicine and can inform their implementation in practice.

A chatbot-based intervention with ELME to improve stress and health-related parameters in a stressed sample: Study protocol of a randomised controlled trial

Background

Stress levels in the general population had already been increasing in recent years, and have subsequently been exacerbated by the global pandemic. One approach for innovative online-based interventions are "chatbots" - computer programs that can simulate a text-based interaction with human users via a conversational interface. Research on the efficacy of chatbot-based interventions in the context of mental health is sparse. The present study is designed to investigate the effects of a three-week chatbot-based intervention with the chatbot ELME, aiming to reduce stress and to improve various health-related parameters in a stressed sample.

Methods

In this multicenter, two-armed randomised controlled trial with a parallel design, a three-week chatbot-based intervention group including two daily interactive intervention sessions via smartphone (á 10-20 min.) is compared to a treatment-as-usual control group. A total of 130 adult participants with a medium to high stress levels will be recruited in Germany. Assessments will take place pre-intervention, post-intervention (after three weeks), and follow-up (after six weeks). The primary outcome is perceived stress. Secondary outcomes include self-reported interoceptive accuracy, mindfulness, anxiety, depression, personality, emotion regulation, psychological well-being, stress mindset, intervention credibility and expectancies, affinity for technology, and attitudes towards artificial intelligence. During the intervention, participants undergo ecological momentary assessments. Furthermore, satisfaction with the intervention, the usability of the chatbot, potential negative effects of the intervention, adherence, potential dropout reasons, and open feedback questions regarding the chatbot are assessed post-intervention.

Discussion

To the best of our knowledge, this is the first chatbot-based intervention addressing interoception, as well as in the context with the target variables stress and mindfulness. The design of the present study and the usability of the chatbot were successfully tested in a previous feasibility study. To counteract a low adherence of the chatbot-based intervention, a high guidance by the chatbot, short sessions, individual and flexible time points of the intervention units and the ecological momentary assessments, reminder messages, and the opportunity to postpone single units were implemented.

Trial registration

The trial is registered at the WHO International Clinical Trials Registry Platform via the German Clinical Trials Register (DRKS00027560; date of registration: 06 January 2022). This is protocol version No. 1. In case of important protocol modifications, trial registration will be updated.

Evaluation of chatbot-delivered interventions for self-management of depression: Content analysis

BACKGROUND

Conversational agents (CAs) or chatbots are increasingly used for depression, anxiety, and wellbeing management. CAs are considered acceptable and helpful. However, little is known about the adequacy of CA responses. This study assessed the structure, content, and user-customization of mental health CA dialogues with users with depression or at risk of suicide.

METHODS

We used content analysis to examine the dialogues of CAs previously included in three assessments of mental health apps (depression education, self-guided cognitive behavioural therapy, and suicide prevention) performed between 2019 and 2020. Two standardized user personas with depression were developed to interact with the CA. All conversations were saved as screenshots, transcribed verbatim, and coded inductively.

RESULTS

Nine CAs were included. Seven CAs (78%) had Android and iOS versions; five CAs (56%) had at least 500,000 downloads. The analysis generated eight categories: self-introduction, personalization, appropriateness of CA responses, conveying empathy, guiding users through mood-boosting activities, mood monitoring, suicide risk management, and others. CAs could engage in empathic, non-judgemental conversations with users, offer support, and guide psychotherapeutic exercises.

LIMITATIONS

CA evaluations were performed using standardized personas, not real-world users. CAs were included for evaluation only if retrieved in the search strategies associated with the previous assessment studies.

CONCLUSION

Assessed CAs offered anonymous, empathic, non-judgemental interactions that align with evidence for face-to-face psychotherapy. CAs from app stores are not suited to provide comprehensive suicide risk management. Further research should evaluate the effectiveness of CA-led interventions in mental health care and in enhancing suicide risk management strategies.

Evaluating Conversational Agents for Mental Health: Scoping Review of Outcomes and Outcome Measurement Instruments (Preprint)

BACKGROUND

Rapid proliferation of mental health interventions delivered through conversational agents (CAs) calls for high-quality evidence to support their implementation and adoption. Selecting appropriate outcomes, instruments for measuring outcomes, and assessment methods are crucial for ensuring that interventions are evaluated effectively and with a high level of quality.

OBJECTIVE

We aimed to identify the types of outcomes, outcome measurement instruments, and assessment methods used to assess the clinical, user experience, and technical outcomes in studies that evaluated the effectiveness of CA interventions for mental health.

METHODS

We undertook a scoping review of the relevant literature to review the types of outcomes, outcome measurement instruments, and assessment methods in studies that evaluated the effectiveness of CA interventions for mental health. We performed a comprehensive search of electronic databases, including PubMed, Cochrane Central Register of Controlled Trials, Embase (Ovid), PsychINFO, and Web of Science, as well as Google Scholar and Google. We included experimental studies evaluating CA mental health interventions. The screening and data extraction were performed independently by 2 review authors in parallel. Descriptive and thematic analyses of the findings were performed.

RESULTS

We included 32 studies that targeted the promotion of mental well-being (17/32, 53%) and the treatment and monitoring of mental health symptoms (21/32, 66%). The studies reported 203 outcome measurement instruments used to measure clinical outcomes (123/203, 60.6%), user experience outcomes (75/203, 36.9%), technical outcomes (2/203, 1.0%), and other outcomes (3/203, 1.5%). Most of the outcome measurement instruments were used in only 1 study (150/203, 73.9%) and were self-reported questionnaires (170/203, 83.7%), and most were delivered electronically via survey platforms (61/203, 30.0%). No validity evidence was cited for more than half of the outcome measurement instruments (107/203, 52.7%), which were largely created or adapted for the study in which they were used (95/107, 88.8%).

CONCLUSIONS

The diversity of outcomes and the choice of outcome measurement instruments employed in studies on CAs for mental health point to the need for an established minimum core outcome set and greater use of validated instruments. Future studies should also capitalize on the affordances made available by CAs and smartphones to streamline the evaluation and reduce participants' input burden inherent to self-reporting.

Validity of Chatbot Use for Mental Health Assessment: Experimental Study

BACKGROUND

Mental disorders in adolescence and young adulthood are major public health concerns. Digital tools such as text-based conversational agents (ie, chatbots) are a promising technology for facilitating mental health assessment. However, the human-like interaction style of chatbots may induce potential biases, such as socially desirable responding (SDR), and may require further effort to complete assessments.

OBJECTIVE

This study aimed to investigate the convergent and discriminant validity of chatbots for mental health assessments, the effect of assessment mode on SDR, and the effort required by participants for assessments using chatbots compared with established modes.

METHODS

In a counterbalanced within-subject design, we assessed 2 different constructs-psychological distress (Kessler Psychological Distress Scale and Brief Symptom Inventory-18) and problematic alcohol use (Alcohol Use Disorders Identification Test-3)-in 3 modes (chatbot, paper-and-pencil, and web-based), and examined convergent and discriminant validity. In addition, we investigated the effect of mode on SDR, controlling for perceived sensitivity of items and individuals' tendency to respond in a socially desirable way, and we also assessed the perceived social presence of modes. Including a between-subject condition, we further investigated whether SDR is increased in chatbot assessments when applied in a self-report setting versus when human interaction may be expected. Finally, the effort (ie, complexity, difficulty, burden, and time) required to complete the assessments was investigated.

RESULTS

A total of 146 young adults (mean age 24, SD 6.42 years; n=67, 45.9% female) were recruited from a research panel for laboratory experiments. The results revealed high positive correlations (all P<.001) of measures of the same construct across different modes, indicating the convergent validity of chatbot assessments. Furthermore, there were no correlations between the distinct constructs, indicating discriminant validity. Moreover, there were no differences in SDR between modes and whether human interaction was expected, although the perceived social presence of the chatbot mode was higher than that of the established modes (P<.001). Finally, greater effort (all P<.05) and more time were needed to complete chatbot assessments than for completing the established modes (P<.001).

CONCLUSIONS

Our findings suggest that chatbots may yield valid results. Furthermore, an understanding of chatbot design trade-offs in terms of potential strengths (ie, increased social presence) and limitations (ie, increased effort) when assessing mental health were established.

Designing a conversational agent to promote teamwork and collaborative practices using design thinking: An explorative study on user experiences

Appearance, voice features, and communication style affect users trust in conversational agents (chatbots), but few studies have assessed what features users like and dislike. Using design thinking, we developed Susa, a conversational agent, to help workplaces promote teamwork and collaborative practices. Design thinking prioritizes co-creation and multidisciplinary teamwork to develop innovative solutions to complex problems. The aim of this qualitative study was to explore users’ interactions with and reactions toward Susa and explain how we used user inputs to adapt and refine the first prototype. The employees and managers from four workplaces participated in three workshops to test and refine the agent. We applied an explorative thematic analysis of data collected via video recordings of the workshops. The results of the analyses revealed that visual identity, communication style and personality was important for acceptability. Users favored a more human like agent that primarily communicated with the team via text messages. Users disliked emoticons and humor because these features clashed with the seriousness of the topic. Finally, users highlighted that Susa helped structure organizational change processes, develop concrete action plans, and stay on track. It is a weakness that Susa is a simple robot based on a preprogrammed script that does not allow users to adapt the process.