Efficacy, Usability, and Acceptability of a Chatbot for Promoting COVID-19 Vaccination in Unvaccinated or Booster-Hesitant Young Adults: Pre-Post Pilot Study

A trial of a chat service for patients and their family members in an emergency department

BACKGROUND

Effective communication between patients and healthcare providers in the emergency department (ED) is challenging due to the dynamic nature of the ED environment. This study aimed to trial a chat service enabling patients in the ED and their family members to ask questions freely, exploring the service's feasibility and user experience.

OBJECTIVES

To identify the types of needs and inquiries from patients and family members in the ED that could be addressed through the chat service and to assess the user experience of the service.

METHODS

We enrolled patients and family members aged over 19 years in the ED, providing the chat service for up to 4 h per ED visit. Trained research nurses followed specific guidelines to respond to messages from the participants. After participation, participants were required to complete a survey. Those who agreed also participated in interviews to provide insights on their experiences with the ED chat service.

RESULTS

A total of 40 participants (20 patients and 20 family members) sent 305 messages (72 by patients and 233 by family members), with patients sending an average of 3.6 messages and family members 11.7. Research nurses resolved 41.4% of patient inquiries and 70.9% of family member inquiries without further healthcare provider involvement. High usability was reported, with positive feedback on communication with healthcare workers, information accessibility, and emotional support.

CONCLUSIONS

The ED chat service was found to be feasible and led to positive user experiences for both patients and their family members.

Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review

BACKGROUND

Chatbots, or conversational agents, have emerged as significant tools in health care, driven by advancements in artificial intelligence and digital technology. These programs are designed to simulate human conversations, addressing various health care needs. However, no comprehensive synthesis of health care chatbots' roles, users, benefits, and limitations is available to inform future research and application in the field.

OBJECTIVE

This review aims to describe health care chatbots' characteristics, focusing on their diverse roles in the health care pathway, user groups, benefits, and limitations.

METHODS

A rapid review of published literature from 2017 to 2023 was performed with a search strategy developed in collaboration with a health sciences librarian and implemented in the MEDLINE and Embase databases. Primary research studies reporting on chatbot roles or benefits in health care were included. Two reviewers dual-screened the search results. Extracted data on chatbot roles, users, benefits, and limitations were subjected to content analysis.

RESULTS

The review categorized chatbot roles into 2 themes: delivery of remote health services, including patient support, care management, education, skills building, and health behavior promotion, and provision of administrative assistance to health care providers. User groups spanned across patients with chronic conditions as well as patients with cancer; individuals focused on lifestyle improvements; and various demographic groups such as women, families, and older adults. Professionals and students in health care also emerged as significant users, alongside groups seeking mental health support, behavioral change, and educational enhancement. The benefits of health care chatbots were also classified into 2 themes: improvement of health care quality and efficiency and cost-effectiveness in health care delivery. The identified limitations encompassed ethical challenges, medicolegal and safety concerns, technical difficulties, user experience issues, and societal and economic impacts.

CONCLUSIONS

Health care chatbots offer a wide spectrum of applications, potentially impacting various aspects of health care. While they are promising tools for improving health care efficiency and quality, their integration into the health care system must be approached with consideration of their limitations to ensure optimal, safe, and equitable use.

The impact of Chatbot-Assisted Self Assessment (CASA) on intentions for sexual health screening in people from minoritised ethnic groups at risk of sexually transmitted infections

Background Sexually transmitted infections (STIs) present a significant global public health issue, with disparities in STI rates often observed across ethnic groups. The study investigates the impact of Chatbot-Assisted Self Assessment (CASA) on the intentions for sexual health screening within minoritised ethnic groups (MEGs) at risk of STIs as well as the subsequent use of a chatbot for booking STI screening. Methods A simulation within-subject design was utilised to evaluate the effect of CASA on intentions for STI/HIV screening, concern about STIs, and attitudes towards STI screening. Screening intentions served as the dependent variable, while demographic and behavioural factors related to STI/HIV risk were the independent variables. ANCOVA tests were conducted to measure the impact of CASA on these perceptions. Results Involving 548 participants (54% women, 66% black, average age=30years), the study found that CASA positively influenced screening intentions t (547)=-10.3, P t (544)=-4.96, P t (543)=-4.36, P Conclusion CASA increased motivations for STI screening intentions among ethnically diverse communities. The intervention's non-judgemental nature and the chatbot's ability to emulate sexual history-taking were critical in fostering an environment conducive to behavioural intention change. The study's high acceptability indicates the potential for broader application in digital health interventions. However, the limitation of not tracking actual post-intervention behaviour warrants further investigation into CASA's real-world efficacy.

Effectiveness of chatbots in increasing uptake, intention, and attitudes related to any type of vaccination: A systematic review and meta-analysis

This systematic review and meta-analysis analyzed and summarized the growing literature on the effectiveness of chatbot-delivered interventions in increasing uptake, intention, and attitudes related to any type of vaccination. We identified randomized controlled studies (RCTs), quasi-experimental studies, and non-experimental studies from the following platforms: PubMed, Web of Science, MEDLINE, Global Health, APA PsycInfo, and EMBASE databases. A total of 12 eligible studies published from 2019 to 2023 were analyzed and summarized. In particular, one RCT showed that a chatbot-delivered tailored intervention was more effective than a chatbot-delivered non-tailored intervention in promoting seasonal influenza vaccine uptake among older adults (50.5% versus 35.3%, p = 0.002). Six RCTs were included in the meta-analysis to evaluate the effectiveness of chatbot interventions to improve vaccination attitudes and intentions. The pooled standard mean difference (SMD) of overall attitude change was 0.34 (95% confidence intervals [CI]: 0.13, 0.55, p = 0.001). We found a non-significant trivial effect of chatbot interventions on improving intentions of vaccination (SMD: 0.11, 95% CI: -0.13, 0.34, p = 0.38). However, further evidence is needed to draw a more precise conclusion. Additionally, study participants reported high satisfaction levels of using the chatbot and were likely to recommend it to others. The development of chatbots is still nascent and rooms for improvement exist.

Chatbots and COVID-19: Taking Stock of the Lessons Learned

While digital innovation in health was already rapidly evolving, the COVID-19 pandemic has accelerated the generation of digital technology tools, such as chatbots, to help increase access to crucial health information and services to those who were cut off or had limited contact with health services. This theme issue titled "Chatbots and COVID-19" presents articles from researchers and practitioners across the globe, describing the development, implementation, and evaluation of chatbots designed to address a wide range of health concerns and services. In this editorial, we present some of the key challenges and lessons learned arising from the content of this theme issue. Most notably, we note that a stronger evidence base is needed to ensure that chatbots and other digital tools are developed to best serve the needs of population health.

A chatbot-delivered intervention for optimizing social media use and reducing perceived isolation among rural-living LGBTQ+ youth: Development, acceptability, usability, satisfaction, and utility

Background

Lesbian, gay, bisexual, transgender, and queer/questioning (LGBTQ+) youth are at higher risk of isolation and depression than their heterosexual peers. Having access to tailored mental health resources is a documented concern for rural living LGBTQ+ youth. Social media provides access to connections to a broader and like-minded community of peers, but it also is a vehicle for negative interactions. We developed REALbot, an automated, social media-based educational intervention to improve social media efficacy, reduce perceived isolation, and bolster connections for rural living LGBTQ+ youth. This report presents data on the acceptability, feasibility, and utility of REALbot among its target audience of rural living LGBTQ+ youth.

Methods

We conducted a week-long exploratory study with a single non-comparison group of 20 rural-living LGBTQ+ youth aged 14-19 recruited from social media to test our Facebook- and Instagram-delivered chatbot. We assessed pre- and post-test scores of social media self-efficacy, social isolation (4-item Patient-Reported Outcomes Measurement System - PROMIS), and depressive symptoms (Patient Health Questionnaire, Adolescent Version - PHQ-A). At post-test, we assessed acceptability (User Experience Questionnaire - UEQ-S), usability (Chatbot Usability Questionnaire -CUQ and Post-Study Satisfaction and Usability Questionnaire -PSSUQ), and satisfaction with the chatbot (Client Satisfaction Questionnaire - CSQ), along with two open-ended questions on 'likes' and 'dislikes' about the intervention. We compared pre- and post-test scores with standard univariate statistics. Means and standard deviations were calculated for usability, acceptability, and satisfaction. To analyze the responses to post-test open-end questions, we used a content analysis approach.

Results

Acceptability of REALbot was high with UEQ-S 5.3 out of 7 (SD = 1.1) and received high usability scores with CUQ and PSSUQ (mean score (M) = 78.0, SD = 14.5 and M = 86.9, SD = 25.2, respectively), as well as high user satisfaction with CSQ (M = 24.9, SD = 5.4). Themes related to user 'likes' and 'dislikes' were organized in two main categories: usability and content provided. Participants were engaged with the chatbot, sending an average of 49.3 messages (SD = 43.6, median = 30). Pre-/post- changes in scores of perceived isolation, depressive symptoms and social media self-efficacy were not significant (p's > 0.08).

Conclusion

REALbot deployment was found to be feasible and acceptable, with good usability and user satisfaction scores. Participants reported changes from pre- to post-test in most outcomes of interest and effect sizes were small to medium. Additional development and a formal evaluation of feasibility and engagement with behavioral targets is warranted.

Chatbot-Delivered Online Intervention to Promote Seasonal Influenza Vaccination During the COVID-19 Pandemic: A Randomized Clinical Trial

Importance

Receiving seasonal influenza vaccination (SIV) is important for adults during the COVID-19 pandemic. There are few robust evaluations of tailored interventions for improving SIV uptake among adults 65 years or older.

Objective

To evaluate the relative efficacy of a stages of change (SOC)-tailored online intervention compared with a standard, non-SOC-tailored online intervention in increasing SIV uptake among Hong Kong residents 65 years or older.

Design, Setting, and Participants

This nonblinded parallel-group randomized clinical trial was conducted between December 1, 2021, and July 31, 2022, in Hong Kong, China. Eligible participants were 65 years or older, had Cantonese- and/or Mandarin-speaking skills, were community-dwelling, had Hong Kong residency, were smartphone users, and had not received SIV for the 2021 to 2022 influenza season. Participants were recruited through random telephone calls, and those who completed the baseline telephone survey were randomized to either the intervention or control group. Both complete case and intention-to-treat (ITT) analyses were performed.

Intervention

In the intervention group, a simplified rule-based chatbot first assessed participants' SOC related to SIV uptake and then automatically selected and sent participants SOC-tailored online health promotion messages (videos) through a messaging application (WhatsApp; Meta) once every 2 weeks for 4 sessions. In the control group, the chatbot sent a link to access through the messaging application a standard online health promotion message (video) covering general SIV information every 2 weeks for 4 sessions.

Main Outcomes and Measures

The primary outcome was self-reported SIV uptake at month 6, which was validated by the research team. The secondary outcome was SOC measured at both baseline and month 6 by validated questions.

Results

A total of 396 participants (mean [SD] age of 70.2 [4.3] years; 249 females [62.9%]) were randomized to the intervention (n = 198) or control (n = 198) group. The ITT analysis showed that the validated SIV uptake rate was higher in the intervention group than the control group at month 6 (50.5% vs 35.3%; P = .002). The mean (SD) SOC score was higher in the intervention group than the control group (2.8 [1.4] vs 2.4 [1.4]; P = .02). More participants in the intervention group completed at least 1 episode of intervention than in the control group (77.3% vs 62.6%; P < .001).

Conclusions

Results of this trial indicate that the SOC-tailored online intervention was more effective than the non-SOC-tailored intervention and may be a sustainable new method in increasing SIV uptake among adults 65 years or older.

Trial Registration

ClinicalTrials.gov Identifier: NCT05155241.

Expressive Interviewing Agents to Support Health-Related Behavior Change: Randomized Controlled Study of COVID-19 Behaviors

BACKGROUND

Expressive writing and motivational interviewing are well-known approaches to help patients cope with stressful life events. Although these methods are often applied by human counselors, it is less well understood if an automated artificial intelligence approach can benefit patients. Providing an automated method would help expose a wider range of people to the possible benefits of motivational interviewing, with lower cost and more adaptability to sudden events like the COVID-19 pandemic.

OBJECTIVE

This study presents an automated writing system and evaluates possible outcomes among participants with respect to behavior related to the COVID-19 pandemic.

METHODS

We developed a rule-based dialogue system for "Expressive Interviewing" to elicit writing from participants on the subject of how COVID-19 has impacted their lives. The system prompts participants to describe their life experiences and emotions and provides topic-specific prompts in response to participants' use of topical keywords. In May 2021 and June 2021, we recruited participants (N=151) via Prolific to complete either the Expressive Interviewing task or a control task. We surveyed participants immediately before the intervention, immediately after the intervention, and again 2 weeks after the intervention. We measured participants' self-reported stress, general mental health, COVID-19-related health behavior, and social behavior.

RESULTS

Participants generally wrote long responses during the task (53.3 words per response). In aggregate, task participants experienced a significant decrease in stress in the short term (~23% decrease, P<.001) and a slight difference in social activity compared with the control group (P=.03). No significant differences in short-term or long-term outcomes were detected between participant subgroups (eg, male versus female participants) except for some within-condition differences by ethnicity (eg, higher social activity among African American people participating in Expressive Interviewing vs participants of other ethnicities). For short-term effects, participants showed different outcomes based on their writing. Using more anxiety-related words was correlated with a greater short-term decrease in stress (r=-0.264, P<.001), and using more positive emotion words was correlated with a more meaningful experience (r=0.243, P=.001). As for long-term effects, writing with more lexical diversity was correlated with an increase in social activity (r=0.266, P<.001).

CONCLUSIONS

Expressive Interviewing participants exhibited short-term, but not long-term, positive changes in mental health, and some linguistic metrics of writing style were correlated with positive change in behavior. Although there were no significant long-term effects observed, the positive short-term effects suggest that the Expressive Interviewing intervention could be used in cases in which a patient lacks access to traditional therapy and needs a short-term solution.

TRIAL REGISTRATION

Clincaltrials.gov NCT05949840; https://www.clinicaltrials.gov/study/NCT05949840.

Changes in Confidence, Feelings, and Perceived Necessity Concerning COVID-19 Booster

The COVID-19 booster first became available to all adults in the U.S. in November 2021 and a bivalent version in September 2022, but a large population remains booster-hesitant; only 17% of Americans have obtained the updated vaccine as of June 2023. We conducted two cross-sectional surveys in 2021 and 2022 (n = 1889 and 1319) to determine whether changes in booster-related feelings or perceptions had occurred and whether they altered vaccination rates over time. We found that both positive and negative emotions had grown stronger between the two years, with the prevalence of annoyance increasing the most (21.5% to 39.7%). The impact of trust on booster intention more than doubled (OR = 7.46 to 16.04). Although perceived risk of infection decreased, more participants in 2022 indicated uncertainty or unwillingness to obtain a new booster than in 2021, while the proportion refusing a booster remained constant at 22.5%. Confidence in the COVID-19 vaccine and feelings of hope from the booster motivated acceptance; both were stronger predictors of booster receptivity than prior vaccination history. Our findings signal a need to rebuild trust by informing people of their continued risk and appealing to positive, especially optimistic emotions to encourage booster uptake. Future research should explore longitudinal trends in behavior and feelings toward new booster doses and the impact of prolonged vaccine hesitancy on infection rates.

Examining an Altruism-Eliciting Video Intervention to Increase COVID-19 Vaccine Intentions in Younger Adults: A Qualitative Assessment Using the Realistic Evaluation Framework

COVID-19 vaccine-induced immunity wanes over time, and with the emergence of new variants, additional "booster" doses have been recommended in Canada. However, booster vaccination uptake has remained low, particularly amongst younger adults aged 18-39. A previous study by our research team found that an altruism-eliciting video increased COVID-19 vaccination intentions. Using qualitative methods, the present study aims to: (1) identify the factors that influence vaccine decision-making in Canadian younger adults; (2) understand younger adults' perceptions of an altruism-eliciting video designed to increase COVID-19 vaccine intentions; and (3) explore how the video can be improved and adapted to the current pandemic context. We conducted three focus groups online with participants who: (1) received at least one booster vaccine, (2) received the primary series without any boosters, or (3) were unvaccinated. We used deductive and inductive approaches to analyze data. Deductively, informed by the realist evaluation framework, we synthesized data around three main themes: context, mechanism, and intervention-specific suggestions. Within each main theme, we deductively created subthemes based on the health belief model (HBM). For quotes that could not be captured by these subthemes, additional themes were created inductively. We found multiple factors that could be important considerations in future messaging to increase vaccine acceptance, such as feeling empowered, fostering confidence in government and institutions, providing diverse (such as both altruism and individualism) messaging, and including concrete data (such as the prevalence of vulnerable individuals). These findings suggest targeted messaging tailored to these themes would be helpful to increase COVID-19 booster vaccination amongst younger adults.

Why Some People Are Hesitant to Receive COVID-19 Boosters: A Systematic Review

As the COVID-19 pandemic continues and transitions to an endemic stage, booster vaccines will play an important role in personal and public health. However, convincing people to take boosters continues to be a key obstacle. This study systematically analyzed research that examined the predictors of COVID-19 booster vaccine hesitancy. A search of PubMed, Medline, CINAHL, Web of Science, and Scopus uncovered 42 eligible studies. Globally, the average COVID-19 booster vaccination hesitancy rate was 30.72%. Thirteen key factors influencing booster hesitancy emerged from the literature: demographics (gender, age, education, income, occupation, employment status, ethnicity, and marital status), geographical influences (country, region, and residency), adverse events, perceived benefit/efficacy, perceived susceptibility, perceived severity, prior history of COVID-19 infection, vaccination status, vaccination recommendations, health status, knowledge and information, skepticism/distrust/conspiracy theories, and vaccine type. Vaccine communication campaigns and interventions for COVID boosters should focus on factors influencing booster confidence, complacency, and convenience.

Voice-Based Screening for SARS-CoV-2 Exposure in Cardiovascular Clinics (VOICE-COVID-19-II): Protocol for a Randomized Controlled Trial

BACKGROUND

The COVID-19 pandemic has disrupted the health care system, limiting health care resources such as the availability of health care professionals, patient monitoring, contact tracing, and continuous surveillance. As a result of this significant burden, digital tools have become an important asset in increasing the efficiency of patient care delivery. Digital tools can help support health care institutions by tracking transmission of the virus, aiding in the screening process, and providing telemedicine support. However, digital health tools face challenges associated with barriers to accessibility, efficiency, and privacy-related ethical issues.

OBJECTIVE

This paper describes the study design of an open-label, noninterventional, crossover, randomized controlled trial aimed at assessing whether interactive voice response systems can screen for SARS-CoV-2 in patients as accurately as standard screening done by people. The study aims to assess the concordance and interrater reliability of symptom screening done by Amazon Alexa compared to manual screening done by research coordinators. The perceived level of comfort of patients when interacting with voice response systems and their personal experience will also be evaluated.

METHODS

A total of 52 patients visiting the heart failure clinic at the Royal Victoria Hospital of the McGill University Health Center, in Montreal, Quebec, will be recruited. Patients will be randomly assigned to first be screened for symptoms of SARS-CoV-2 either digitally, by Amazon Alexa, or manually, by the research coordinator. Participants will subsequently be crossed over and screened either digitally or manually. The clinical setup includes an Amazon Echo Show, a tablet, and an uninterrupted power supply mounted on a mobile cart. The primary end point will be the interrater reliability on the accuracy of randomized screening data performed by Amazon Alexa versus research coordinators. The secondary end point will be the perceived level of comfort and app engagement of patients as assessed using 5-point Likert scales and binary mode responses.

RESULTS

Data collection started in May 2021 and is expected to be completed in fall 2022. Data analysis is expected to be completed in early 2023.

CONCLUSIONS

The use of voice-based assistants could improve the provision of health services and reduce the burden on health care personnel. Demonstrating a high interrater reliability between Amazon Alexa and health care coordinators may serve future digital tools to streamline the screening and delivery of care in the context of other conditions and clinical settings. The COVID-19 pandemic occurs during the first digital era using digital tools such as Amazon Alexa for disease screening, and it represents an opportunity to implement such technology in health care institutions in the long term.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04508972; https://clinicaltrials.gov/ct2/show/NCT04508972.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/41209.

Finding the sweet spot: a qualitative study exploring patients' acceptability of chatbots in genetic service delivery

Chatbots, web-based artificial intelligence tools that simulate human conversation, are increasingly in use to support many areas of genomic medicine. However, patient preferences towards using chatbots across the range of clinical settings are unknown. We conducted a qualitative study with individuals who underwent genetic testing for themselves or their child. Participants were asked about their preferences for using a chatbot within the genetic testing journey. Thematic analysis employing interpretive description was used. We interviewed 30 participants (67% female, 50% 50 + years). Participants considered chatbots to be inefficient for very simple tasks (e.g., answering FAQs) or very complex tasks (e.g., explaining results). Chatbots were acceptable for moderately complex tasks where participants perceived a favorable return on their investment of time and energy. In addition to achieving this "sweet spot," participants anticipated that their comfort with chatbots would increase if the chatbot was used as a complement to but not a replacement for usual care. Participants wanted a "safety net" (i.e., access to a clinician) for needs not addressed by the chatbot. This study provides timely insights into patients' comfort with and perceived limitations of chatbots for genomic medicine and can inform their implementation in practice.

Talking about the Vaccine after the Pandemic: A Cross-Sectional Study among Youth in Turkey and Ethical Issues

Examining the factors that affect the vaccination rate among young people in an ethical frame can support vaccination promotion. Therefore, this study will elaborate, through an ethical lens, on young people's hesitation about and decisions regarding getting vaccinated. The cross-sectional study was conducted with 2428 people aged 15-30 in Turkey in June 2022. The questionnaire included the following subtitles: psycho-social situation, health services and health policies, COVID-19 vaccine, and predictions about life and health after the pandemic. The average age was 22.9 years. In the study sample, 80% were vaccinated, while 20% were not. Vaccinated participants acted more cautiously to protect their health. Receiving accurate and sufficient information on proposed vaccines affects vaccination status. The primary reason for getting vaccinated was "to protect their health, families, and relatives", and the primary reason for not getting vaccinated was "not trusting the vaccine content or the country where the vaccine was produced". Specifically, those vaccinated felt more relaxed physically, psychologically, and socially. In addition, the expectations for the future of those vaccinated were significantly higher. Accurate and adequate information is essential for reducing vaccine hesitancy. In addition, promoting prosocial behaviors in young people and highlighting related values will support vaccination.

When Virtual Assistants Meet Teledermatology: Validation of a Virtual Assistant to Improve the Quality of Life of Psoriatic Patients

Teledermatology has given dermatologists a tool to track patients' responses to therapy using images. Virtual assistants, the programs that interact with users through text or voice messages, could be used in teledermatology to enhance the interaction of the tool with the patients and healthcare professionals and the overall impact of the medication and quality of life of patients. As such, this work aimed to investigate the effectiveness of using a virtual assistant for teledermatology and its impact on the quality of life. We conducted surveys with the participants and measured the usability of the system with the System Usability Scale (SUS). A total of 34 participants (30 patients diagnosed with moderate-severe psoriasis and 4 healthcare professionals) were included in the study. The measurement of the improvement of quality of life was done by analyzing Psoriasis Quality of Life (PSOLIFE) and Dermatology Life Quality Index (DLQI) questionnaires. The results showed that, on average, the quality of life improved (from 63.8 to 64.8 for PSOLIFE (with a p-value of 0.66 and an effect size of 0.06) and 4.4 to 2.8 for DLQI (with a p-value of 0.04 and an effect size of 0.31)). Patients also used the virtual assistant to do 52 medical consultations. Moreover, the usability is above average, with a SUS score of 70.1. As supported by MMAS-8 results, adherence also improved slightly. Our work demonstrates the improvement of the quality of life with the use of a virtual assistant in teledermatology, which could be attributed to the sense of security or peace of mind the patients get as they can contact their dermatologists directly within the virtual assistant-integrated system.