Chloe for COVID-19: Evolution of an Intelligent Conversational Agent to Address Infodemic Management Needs During the COVID-19 Pandemic

General Characteristics and Design Taxonomy of Chatbots for COVID-19: Systematic Review

BACKGROUND

A conversational agent powered by artificial intelligence, commonly known as a chatbot, is one of the most recent innovations used to provide information and services during the COVID-19 pandemic. However, the multitude of conversational agents explicitly designed during the COVID-19 pandemic calls for characterization and analysis using rigorous technological frameworks and extensive systematic reviews.

OBJECTIVE

This study aims to describe the general characteristics of COVID-19 chatbots and examine their system designs using a modified adapted design taxonomy framework.

METHODS

We conducted a systematic review of the general characteristics and design taxonomy of COVID-19 chatbots, with 56 studies included in the final analysis. This review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines to select papers published between March 2020 and April 2022 from various databases and search engines.

RESULTS

Results showed that most studies on COVID-19 chatbot design and development worldwide are implemented in Asia and Europe. Most chatbots are also accessible on websites, internet messaging apps, and Android devices. The COVID-19 chatbots are further classified according to their temporal profiles, appearance, intelligence, interaction, and context for system design trends. From the temporal profile perspective, almost half of the COVID-19 chatbots interact with users for several weeks for >1 time and can remember information from previous user interactions. From the appearance perspective, most COVID-19 chatbots assume the expert role, are task oriented, and have no visual or avatar representation. From the intelligence perspective, almost half of the COVID-19 chatbots are artificially intelligent and can respond to textual inputs and a set of rules. In addition, more than half of these chatbots operate on a structured flow and do not portray any socioemotional behavior. Most chatbots can also process external data and broadcast resources. Regarding their interaction with users, most COVID-19 chatbots are adaptive, can communicate through text, can react to user input, are not gamified, and do not require additional human support. From the context perspective, all COVID-19 chatbots are goal oriented, although most fall under the health care application domain and are designed to provide information to the user.

CONCLUSIONS

The conceptualization, development, implementation, and use of COVID-19 chatbots emerged to mitigate the effects of a global pandemic in societies worldwide. This study summarized the current system design trends of COVID-19 chatbots based on 5 design perspectives, which may help developers conveniently choose a future-proof chatbot archetype that will meet the needs of the public in the face of growing demand for a better pandemic response.

The Potential of Chatbots for Emotional Support and Promoting Mental Well-Being in Different Cultures: Mixed Methods Study

BACKGROUND

Artificial intelligence chatbot research has focused on technical advances in natural language processing and validating the effectiveness of human-machine conversations in specific settings. However, real-world chat data remain proprietary and unexplored despite their growing popularity, and new analyses of chatbot uses and their effects on mitigating negative moods are urgently needed.

OBJECTIVE

In this study, we investigated whether and how artificial intelligence chatbots facilitate the expression of user emotions, specifically sadness and depression. We also examined cultural differences in the expression of depressive moods among users in Western and Eastern countries.

METHODS

This study used SimSimi, a global open-domain social chatbot, to analyze 152,783 conversation utterances containing the terms "depress" and "sad" in 3 Western countries (Canada, the United Kingdom, and the United States) and 5 Eastern countries (Indonesia, India, Malaysia, the Philippines, and Thailand). Study 1 reports new findings on the cultural differences in how people talk about depression and sadness to chatbots based on Linguistic Inquiry and Word Count and n-gram analyses. In study 2, we classified chat conversations into predefined topics using semisupervised classification techniques to better understand the types of depressive moods prevalent in chats. We then identified the distinguishing features of chat-based depressive discourse data and the disparity between Eastern and Western users.

RESULTS

Our data revealed intriguing cultural differences. Chatbot users in Eastern countries indicated stronger emotions about depression than users in Western countries (positive: P<.001; negative: P=.01); for example, Eastern users used more words associated with sadness (P=.01). However, Western users were more likely to share vulnerable topics such as mental health (P<.001), and this group also had a greater tendency to discuss sensitive topics such as swear words (P<.001) and death (P<.001). In addition, when talking to chatbots, people expressed their depressive moods differently than on other platforms. Users were more open to expressing emotional vulnerability related to depressive or sad moods to chatbots (74,045/148,590, 49.83%) than on social media (149/1978, 7.53%). Chatbot conversations tended not to broach topics that require social support from others, such as seeking advice on daily life difficulties, unlike on social media. However, chatbot users acted in anticipation of conversational agents that exhibit active listening skills and foster a safe space where they can openly share emotional states such as sadness or depression.

CONCLUSIONS

The findings highlight the potential of chatbot-assisted mental health support, emphasizing the importance of continued technical and policy-wise efforts to improve chatbot interactions for those in need of emotional assistance. Our data indicate the possibility of chatbots providing helpful information about depressive moods, especially for users who have difficulty communicating emotions to other humans.

A scoping review of digital health interventions for combating COVID-19 misinformation and disinformation

OBJECTIVE

We provide a scoping review of Digital Health Interventions (DHIs) that mitigate COVID-19 misinformation and disinformation seeding and spread.

MATERIALS AND METHODS

We applied our search protocol to PubMed, PsychINFO, and Web of Science to screen 1666 articles. The 17 articles included in this paper are experimental and interventional studies that developed and tested public consumer-facing DHIs. We examined these DHIs to understand digital features, incorporation of theory, the role of healthcare professionals, end-user experience, and implementation issues.

RESULTS

The majority of studies (n = 11) used social media in DHIs, but there was a lack of platform-agnostic generalizability. Only half of the studies (n = 9) specified a theory, framework, or model to guide DHIs. Nine studies involve healthcare professionals as design or implementation contributors. Only one DHI was evaluated for user perceptions and acceptance.

DISCUSSION

The translation of advances in online social computing to interventions is sparse. The limited application of behavioral theory and cognitive models of reasoning has resulted in suboptimal targeting of psychosocial variables and individual factors that may drive resistance to misinformation. This affects large-scale implementation and community outreach efforts. DHIs optimized through community-engaged participatory methods that enable understanding of unique needs of vulnerable communities are urgently needed.

CONCLUSIONS

We recommend community engagement and theory-guided engineering of equitable DHIs. It is important to consider the problem of misinformation and disinformation through a multilevel lens that illuminates personal, clinical, cultural, and social pathways to mitigate the negative consequences of misinformation and disinformation on human health and wellness.

Development and testing of a multi-lingual Natural Language Processing-based deep learning system in 10 languages for COVID-19 pandemic crisis: A multi-center study

Purpose

The COVID-19 pandemic has drastically disrupted global healthcare systems. With the higher demand for healthcare and misinformation related to COVID-19, there is a need to explore alternative models to improve communication. Artificial Intelligence (AI) and Natural Language Processing (NLP) have emerged as promising solutions to improve healthcare delivery. Chatbots could fill a pivotal role in the dissemination and easy accessibility of accurate information in a pandemic. In this study, we developed a multi-lingual NLP-based AI chatbot, DR-COVID, which responds accurately to open-ended, COVID-19 related questions. This was used to facilitate pandemic education and healthcare delivery.

Methods

First, we developed DR-COVID with an ensemble NLP model on the Telegram platform (https://t.me/drcovid_nlp_chatbot). Second, we evaluated various performance metrics. Third, we evaluated multi-lingual text-to-text translation to Chinese, Malay, Tamil, Filipino, Thai, Japanese, French, Spanish, and Portuguese. We utilized 2,728 training questions and 821 test questions in English. Primary outcome measurements were (A) overall and top 3 accuracies; (B) Area Under the Curve (AUC), precision, recall, and F1 score. Overall accuracy referred to a correct response for the top answer, whereas top 3 accuracy referred to an appropriate response for any one answer amongst the top 3 answers. AUC and its relevant matrices were obtained from the Receiver Operation Characteristics (ROC) curve. Secondary outcomes were (A) multi-lingual accuracy; (B) comparison to enterprise-grade chatbot systems. The sharing of training and testing datasets on an open-source platform will also contribute to existing data.

Results

Our NLP model, utilizing the ensemble architecture, achieved overall and top 3 accuracies of 0.838 [95% confidence interval (CI): 0.826-0.851] and 0.922 [95% CI: 0.913-0.932] respectively. For overall and top 3 results, AUC scores of 0.917 [95% CI: 0.911-0.925] and 0.960 [95% CI: 0.955-0.964] were achieved respectively. We achieved multi-linguicism with nine non-English languages, with Portuguese performing the best overall at 0.900. Lastly, DR-COVID generated answers more accurately and quickly than other chatbots, within 1.12-2.15 s across three devices tested.

Conclusion

DR-COVID is a clinically effective NLP-based conversational AI chatbot, and a promising solution for healthcare delivery in the pandemic era.

Early detection of depression using a conversational AI bot: A non-clinical trial

BACKGROUND

Artificial intelligence (AI) has gained momentum in behavioural health interventions in recent years. However, a limited number of studies use or apply such methodologies in the early detection of depression. A large population needing psychological-intervention is left unidentified due to barriers such as cost, location, stigma and a global shortage of health workers. Therefore, it is essential to develop a mass screening integrative approach that can identify people with depression at its early stage to avoid a potential crisis.

OBJECTIVES

This study aims to understand the feasibility and efficacy of using AI-enabled chatbots in the early detection of depression.

METHODS

We use Dialogflow as a conversation interface to build a Depression Analysisn (DEPRA) chatbot. A structured and authoritative early detection depression interview guide, which contains 27 questions combining the structured interview guide for the Hamilton Depression Scale (SIGH-D) and the inventory of depressive symptomatology (IDS-C), underpins the design of the conversation flow. To attain better accuracy and a wide variety of responses, we train Dialogflow with the utterances collected from a focus group of 10 people. The occupation of the focus group members included academics and HDR candidates who are conscious, vigilant and have a clear understanding of the questions. In addition, DEPRA is integrated with a social media platform to provide practical access to all the participants. For the non-clinical trial, we recruited 50 participants aged between 18 and 80 from across Australia. To evaluate the practicability and performance of DEPRA, we also asked participants to submit a user satisfaction survey at the end of the conversation.

RESULTS

A sample of 50 participants, with an average age of 34.7 years, completed this non-clinical trial. More than half of the participants (54%) are male and the major ethnicities are Asian (63%), Middle Eastern (25%), and others 12%. The first group comprises professional academic staff and HDR candidates, the second and third groups comprise relatives, friends, and volunteers who were recruited via social media promotions. DEPRA uses two scientific scoring systems, QIDS-SR and IDS-SR to verify the results of early depression detection. As the results indicate, both scoring systems return a similar outcome with slight variations for different depression levels. According to IDS-SR, 30% of participants were healthy, 14% mild, 22% moderate, 14% severe, and 20% very severe. QIDS-SR suggests 32% were healthy, 18% mild, 10% moderate, 18% severe, and 22% very severe. Furthermore, the overall satisfaction rate of using DEPRA was 79% indicating that the participants had a high rate of user satisfaction and engagement.

CONCLUSION

DEPRA shows promises as a feasible option for developing a mass screening integrated approach for early detection of depression. Although the chatbot is not intended to replace the functionality of mental health professionals, it does show promise as a means of assisting with automation and concealed communication with verified scoring systems.

User-Chatbot Conversations During the COVID-19 Pandemic: Study Based on Topic Modeling and Sentiment Analysis

BACKGROUND

Chatbots have become a promising tool to support public health initiatives. Despite their potential, little research has examined how individuals interacted with chatbots during the COVID-19 pandemic. Understanding user-chatbot interactions is crucial for developing services that can respond to people's needs during a global health emergency.

OBJECTIVE

This study examined the COVID-19 pandemic-related topics online users discussed with a commercially available social chatbot and compared the sentiment expressed by users from 5 culturally different countries.

METHODS

We analyzed 19,782 conversation utterances related to COVID-19 covering 5 countries (the United States, the United Kingdom, Canada, Malaysia, and the Philippines) between 2020 and 2021, from SimSimi, one of the world's largest open-domain social chatbots. We identified chat topics using natural language processing methods and analyzed their emotional sentiments. Additionally, we compared the topic and sentiment variations in the COVID-19-related chats across countries.

RESULTS

Our analysis identified 18 emerging topics, which could be categorized into the following 5 overarching themes: "Questions on COVID-19 asked to the chatbot" (30.6%), "Preventive behaviors" (25.3%), "Outbreak of COVID-19" (16.4%), "Physical and psychological impact of COVID-19" (16.0%), and "People and life in the pandemic" (11.7%). Our data indicated that people considered chatbots as a source of information about the pandemic, for example, by asking health-related questions. Users turned to SimSimi for conversation and emotional messages when offline social interactions became limited during the lockdown period. Users were more likely to express negative sentiments when conversing about topics related to masks, lockdowns, case counts, and their worries about the pandemic. In contrast, small talk with the chatbot was largely accompanied by positive sentiment. We also found cultural differences, with negative words being used more often by users in the United States than by those in Asia when talking about COVID-19.

CONCLUSIONS

Based on the analysis of user-chatbot interactions on a live platform, this work provides insights into people's informational and emotional needs during a global health crisis. Users sought health-related information and shared emotional messages with the chatbot, indicating the potential use of chatbots to provide accurate health information and emotional support. Future research can look into different support strategies that align with the direction of public health policy.

Prospective mixed-methods study evaluating the potential of a voicebot (CovBot) to relieve German health authorities during the COVID-19 infodemic

Background

During the COVID-19 pandemic, telephone hotlines of local health authorities in Germany were overloaded due to information requests by the public.

Objective

Evaluating the use of a COVID-19-specific voicebot (CovBot) in local health authorities in Germany during the COVID-19 pandemic. This study investigates the performance of the CovBot by assessing a perceptible relief of staff in the hotline service.

Methods

This prospective mixed-methods study enrolled local health authorities in Germany from 01 February 2021 to 11 February 2022 to deploy the CovBot, which was mainly designed to answer frequently asked questions. To capture the user perspective and acceptance, we performed semistructured interviews and online surveys with their staff, conducted an online survey among callers, and analyzed the performance metrics of the CovBot.

Results

The CovBot was implemented in 20 local health authorities serving 6.1 million German citizens and processed almost 1.2 million calls during the study period. The overall assessment was that the CovBot contributed to a perceived relief of the hotline service. In a survey among callers, 79% indicated that a voicebot could not replace a human. The analyzed anonymous metadata revealed that 15% of calls hung up immediately, 32% after hearing an FAQ answer, and 51% of calls were forwarded to the local health authority offices.

Conclusions

A voicebot primarily answering FAQs can provide additional support to relieve the hotline service of local health authorities in Germany during the COVID-19 pandemic. For complex concerns, a forwarding option to a human proved to be an essential functionality.

Efficacy, Usability, and Acceptability of a Chatbot for Promoting COVID-19 Vaccination in Unvaccinated or Booster-Hesitant Young Adults: Pre-Post Pilot Study

Background

COVID-19 vaccines are highly effective in preventing severe disease and death but are underused. Interventions to address COVID-19 vaccine hesitancy are paramount to reducing the burden of COVID-19.

Objective

We aimed to evaluate the preliminary efficacy, usability, and acceptability of a chatbot for promoting COVID-19 vaccination and examine the factors associated with COVID-19 vaccine hesitancy.

Methods

In November 2021, we conducted a pre-post pilot study to evaluate “Vac Chat, Fact Check,” a web-based chatbot for promoting COVID-19 vaccination. We conducted a web-based survey (N=290) on COVID-19 vaccination at a university in Hong Kong. A subset of 46 participants who were either unvaccinated (n=22) or were vaccinated but hesitant to receive boosters (n=24) were selected and given access to the chatbot for a 7-day trial period. The chatbot provided information about COVID-19 vaccination (eg, efficacy and common side effects), debunked common myths about the vaccine, and included a decision aid for selecting vaccine platforms (inactivated and mRNA vaccines). The main efficacy outcome was changes in the COVID-19 Vaccine Hesitancy Scale (VHS) score (range 9-45) from preintervention (web-based survey) to postintervention (immediately posttrial). Other efficacy outcomes included changes in intention to vaccinate or receive boosters and willingness to encourage others to vaccinate on a scale from 1 (not at all) to 5 (very). Usability was assessed by the System Usability Scale (range 0-100). Linear regression was used to examine the factors associated with COVID-19 VHS scores in all survey respondents.

Results

The mean (SD) age of all survey respondents was 21.4 (6.3) years, and 61% (177/290) of respondents were female. Higher eHealth literacy (B=–0.26; P<.001) and perceived danger of COVID-19 (B=–0.17; P=.009) were associated with lower COVID-19 vaccine hesitancy, adjusting for age, sex, chronic disease status, previous flu vaccination, and perceived susceptibility to COVID-19. The main efficacy outcome of COVID-19 VHS score significantly decreased from 28.6 (preintervention) to 24.5 (postintervention), with a mean difference of –4.2 (P<.001) and an effect size (Cohen d) of 0.94. The intention to vaccinate increased from 3.0 to 3.9 (P<.001) in unvaccinated participants, whereas the intention to receive boosters increased from 1.9 to 2.8 (P<.001) in booster-hesitant participants. Willingness to encourage others to vaccinate increased from 2.7 to 3.0 (P=.04). At postintervention, the median (IQR) System Usability Scale score was 72.5 (65-77.5), whereas the median (IQR) recommendation score was 7 (6-8) on a scale from 0 to 10. In a post hoc 4-month follow-up, 82% (18/22) of initially unvaccinated participants reported having received the COVID-19 vaccine, whereas 29% (7/24) of booster-hesitant participants received boosters.

Conclusions

This pilot study provided initial evidence to support the efficacy, usability, and acceptability of a chatbot for promoting COVID-19 vaccination in young adults who were unvaccinated or booster-hesitant.

A Deep Learning Approach for Sentiment Analysis of COVID-19 Reviews

User-generated multi-media content, such as images, text, videos, and speech, has recently become more popular on social media sites as a means for people to share their ideas and opinions. One of the most popular social media sites for providing public sentiment towards events that occurred during the COVID-19 period is Twitter. This is because Twitter posts are short and constantly being generated. This paper presents a deep learning approach for sentiment analysis of Twitter data related to COVID-19 reviews. The proposed algorithm is based on an LSTM-RNN-based network and enhanced featured weighting by attention layers. This algorithm uses an enhanced feature transformation framework via the attention mechanism. A total of four class labels (sad, joy, fear, and anger) from publicly available Twitter data posted in the Kaggle database were used in this study. Based on the use of attention layers with the existing LSTM-RNN approach, the proposed deep learning approach significantly improved the performance metrics, with an increase of 20% in accuracy and 10% to 12% in precision but only 12–13% in recall as compared with the current approaches. Out of a total of 179,108 COVID-19-related tweets, tweets with positive, neutral, and negative sentiments were found to account for 45%, 30%, and 25%, respectively. This shows that the proposed deep learning approach is efficient and practical and can be easily implemented for sentiment classification of COVID-19 reviews.