Use of the Healthy Lifestyle Coaching Chatbot App to Promote Stair-Climbing Habits Among Office Workers: Exploratory Randomized Controlled Trial

The potential and limitations of large language models in identification of the states of motivations for facilitating health behavior change

IMPORTANCE

The study highlights the potential and limitations of the Large Language Models (LLMs) in recognizing different states of motivation to provide appropriate information for behavior change. Following the Transtheoretical Model (TTM), we identified the major gap of LLMs in responding to certain states of motivation through validated scenario studies, suggesting future directions of LLMs research for health promotion.

OBJECTIVES

The LLMs-based generative conversational agents (GAs) have shown success in identifying user intents semantically. Little is known about its capabilities to identify motivation states and provide appropriate information to facilitate behavior change progression.

MATERIALS AND METHODS

We evaluated 3 GAs, ChatGPT, Google Bard, and Llama 2 in identifying motivation states following the TTM stages of change. GAs were evaluated using 25 validated scenarios with 5 health topics across 5 TTM stages. The relevance and completeness of the responses to cover the TTM processes to proceed to the next stage of change were assessed.

RESULTS

3 GAs identified the motivation states in the preparation stage providing sufficient information to proceed to the action stage. The responses to the motivation states in the action and maintenance stages were good enough covering partial processes for individuals to initiate and maintain their changes in behavior. However, the GAs were not able to identify users' motivation states in the precontemplation and contemplation stages providing irrelevant information, covering about 20%-30% of the processes.

DISCUSSION

GAs are able to identify users' motivation states and provide relevant information when individuals have established goals and commitments to take and maintain an action. However, individuals who are hesitant or ambivalent about behavior change are unlikely to receive sufficient and relevant guidance to proceed to the next stage of change.

CONCLUSION

The current GAs effectively identify motivation states of individuals with established goals but may lack support for those ambivalent towards behavior change.

Approaches for the Use of AI in Workplace Health Promotion and Prevention: Systematic Scoping Review

BACKGROUND

Artificial intelligence (AI) is an umbrella term for various algorithms and rapidly emerging technologies with huge potential for workplace health promotion and prevention (WHPP). WHPP interventions aim to improve people's health and well-being through behavioral and organizational measures or by minimizing the burden of workplace-related diseases and associated risk factors. While AI has been the focus of research in other health-related fields, such as public health or biomedicine, the transition of AI into WHPP research has yet to be systematically investigated.

OBJECTIVE

The systematic scoping review aims to comprehensively assess an overview of the current use of AI in WHPP. The results will be then used to point to future research directions. The following research questions were derived: (1) What are the study characteristics of studies on AI algorithms and technologies in the context of WHPP? (2) What specific WHPP fields (prevention, behavioral, and organizational approaches) were addressed by the AI algorithms and technologies? (3) What kind of interventions lead to which outcomes?

METHODS

A systematic scoping literature review (PRISMA-ScR [Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews]) was conducted in the 3 academic databases PubMed, Institute of Electrical and Electronics Engineers, and Association for Computing Machinery in July 2023, searching for papers published between January 2000 and December 2023. Studies needed to be (1) peer-reviewed, (2) written in English, and (3) focused on any AI-based algorithm or technology that (4) were conducted in the context of WHPP or (5) an associated field. Information on study design, AI algorithms and technologies, WHPP fields, and the patient or population, intervention, comparison, and outcomes framework were extracted blindly with Rayyan and summarized.

RESULTS

A total of 10 studies were included. Risk prevention and modeling were the most identified WHPP fields (n=6), followed by behavioral health promotion (n=4) and organizational health promotion (n=1). Further, 4 studies focused on mental health. Most AI algorithms were machine learning-based, and 3 studies used combined deep learning algorithms. AI algorithms and technologies were primarily implemented in smartphone apps (eg, in the form of a chatbot) or used the smartphone as a data source (eg, Global Positioning System). Behavioral approaches ranged from 8 to 12 weeks and were compared to control groups. Additionally, 3 studies evaluated the robustness and accuracy of an AI model or framework.

CONCLUSIONS

Although AI has caught increasing attention in health-related research, the review reveals that AI in WHPP is marginally investigated. Our results indicate that AI is promising for individualization and risk prediction in WHPP, but current research does not cover the scope of WHPP. Beyond that, future research will profit from an extended range of research in all fields of WHPP, longitudinal data, and reporting guidelines.

TRIAL REGISTRATION

OSF Registries osf.io/bfswp; https://osf.io/bfswp.

Effectiveness and User Experience of a Smoking Cessation Chatbot: Mixed Methods Study Comparing Motivational Interviewing and Confrontational Counseling

BACKGROUND

Cigarette smoking poses a major public health risk. Chatbots may serve as an accessible and useful tool to promote cessation due to their high accessibility and potential in facilitating long-term personalized interactions. To increase effectiveness and acceptability, there remains a need to identify and evaluate counseling strategies for these chatbots, an aspect that has not been comprehensively addressed in previous research.

OBJECTIVE

This study aims to identify effective counseling strategies for such chatbots to support smoking cessation. In addition, we sought to gain insights into smokers' expectations of and experiences with the chatbot.

METHODS

This mixed methods study incorporated a web-based experiment and semistructured interviews. Smokers (N=229) interacted with either a motivational interviewing (MI)-style (n=112, 48.9%) or a confrontational counseling-style (n=117, 51.1%) chatbot. Both cessation-related (ie, intention to quit and self-efficacy) and user experience-related outcomes (ie, engagement, therapeutic alliance, perceived empathy, and interaction satisfaction) were assessed. Semistructured interviews were conducted with 16 participants, 8 (50%) from each condition, and data were analyzed using thematic analysis.

RESULTS

Results from a multivariate ANOVA showed that participants had a significantly higher overall rating for the MI (vs confrontational counseling) chatbot. Follow-up discriminant analysis revealed that the better perception of the MI chatbot was mostly explained by the user experience-related outcomes, with cessation-related outcomes playing a lesser role. Exploratory analyses indicated that smokers in both conditions reported increased intention to quit and self-efficacy after the chatbot interaction. Interview findings illustrated several constructs (eg, affective attitude and engagement) explaining people's previous expectations and timely and retrospective experience with the chatbot.

CONCLUSIONS

The results confirmed that chatbots are a promising tool in motivating smoking cessation and the use of MI can improve user experience. We did not find extra support for MI to motivate cessation and have discussed possible reasons. Smokers expressed both relational and instrumental needs in the quitting process. Implications for future research and practice are discussed.

Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review

BACKGROUND

Chatbots, or conversational agents, have emerged as significant tools in health care, driven by advancements in artificial intelligence and digital technology. These programs are designed to simulate human conversations, addressing various health care needs. However, no comprehensive synthesis of health care chatbots' roles, users, benefits, and limitations is available to inform future research and application in the field.

OBJECTIVE

This review aims to describe health care chatbots' characteristics, focusing on their diverse roles in the health care pathway, user groups, benefits, and limitations.

METHODS

A rapid review of published literature from 2017 to 2023 was performed with a search strategy developed in collaboration with a health sciences librarian and implemented in the MEDLINE and Embase databases. Primary research studies reporting on chatbot roles or benefits in health care were included. Two reviewers dual-screened the search results. Extracted data on chatbot roles, users, benefits, and limitations were subjected to content analysis.

RESULTS

The review categorized chatbot roles into 2 themes: delivery of remote health services, including patient support, care management, education, skills building, and health behavior promotion, and provision of administrative assistance to health care providers. User groups spanned across patients with chronic conditions as well as patients with cancer; individuals focused on lifestyle improvements; and various demographic groups such as women, families, and older adults. Professionals and students in health care also emerged as significant users, alongside groups seeking mental health support, behavioral change, and educational enhancement. The benefits of health care chatbots were also classified into 2 themes: improvement of health care quality and efficiency and cost-effectiveness in health care delivery. The identified limitations encompassed ethical challenges, medicolegal and safety concerns, technical difficulties, user experience issues, and societal and economic impacts.

CONCLUSIONS

Health care chatbots offer a wide spectrum of applications, potentially impacting various aspects of health care. While they are promising tools for improving health care efficiency and quality, their integration into the health care system must be approached with consideration of their limitations to ensure optimal, safe, and equitable use.

Digital Behavior Change Intervention Designs for Habit Formation: Systematic Review

BACKGROUND

With the development of emerging technologies, digital behavior change interventions (DBCIs) help to maintain regular physical activity in daily life.

OBJECTIVE

To comprehensively understand the design implementations of habit formation techniques in current DBCIs, a systematic review was conducted to investigate the implementations of behavior change techniques, types of habit formation techniques, and design strategies in current DBCIs.

METHODS

The process of this review followed the PRISMA (Preferred Reporting Item for Systematic Reviews and Meta-Analyses) guidelines. A total of 4 databases were systematically searched from 2012 to 2022, which included Web of Science, Scopus, ACM Digital Library, and PubMed. The inclusion criteria encompassed studies that used digital tools for physical activity, examined behavior change intervention techniques, and were written in English.

RESULTS

A total of 41 identified research articles were included in this review. The results show that the most applied behavior change techniques were the self-monitoring of behavior, goal setting, and prompts and cues. Moreover, habit formation techniques were identified and developed based on intentions, cues, and positive reinforcement. Commonly used methods included automatic monitoring, descriptive feedback, general guidelines, self-set goals, time-based cues, and virtual rewards.

CONCLUSIONS

A total of 32 commonly design strategies of habit formation techniques were summarized and mapped to the proposed conceptual framework, which was categorized into target-mediated (generalization and personalization) and technology-mediated interactions (explicitness and implicitness). Most of the existing studies use the explicit interaction, aligning with the personalized habit formation techniques in the design strategies of DBCIs. However, implicit interaction design strategies are lacking in the reviewed studies. The proposed conceptual framework and potential solutions can serve as guidelines for designing strategies aimed at habit formation within DBCIs.

Barriers and Facilitators of Implementing a Healthy Lifestyle Intervention at Workplaces in South Africa

Current evidence indicates that workplace health and wellness programmes provide numerous benefits concerning altering cardiovascular risk factor profiles. Implementing health programmes at workplaces provide an opportunity to engage adults towards positive and sustainable lifestyle choices. The first step in designing lifestyle interventions for the workplace is understanding the barriers and facilitators to implementing interventions in these settings. The barriers and facilitators to implementing lifestyle interventions in the workplace environment was qualitatively explored at two multinational consumer goods companies among seven workplaces in South Africa. Semi-structured in-depth interviews (IDIs) were conducted with ten workplace managers. Five focus group discussions (FGDs) were conducted among workplace employees. The IDI findings revealed that the main facilitators for participation in a lifestyle intervention programme were incentives and rewards, educational tools, workplace support, and engaging lessons. In contrast, the main facilitator of the FGDs was health and longevity. The main barriers from the IDIs included scheduling time for lifestyle interventions within production schedules at manufacturing sites, whereas time limitations, a lack of willpower and self-discipline were the main barriers identified from the FGDs. The findings of this study add to literature on the barriers and facilitators of implementing healthy lifestyle interventions at workplaces and suggest that there is a potential for successfully implementing intervention programmes to improve health outcomes, provided that such efforts are informed and guided through the engagement of workplace stakeholders, an assessment of the physical and food environment, and the availability of workplace resources.

Investigating the Potential of a Conversational Agent (Phyllis) to Support Adolescent Health and Overcome Barriers to Physical Activity: Co-Design Study

BACKGROUND

Conversational agents (CAs) are a promising solution to support people in improving physical activity (PA) behaviors. However, there is a lack of CAs targeted at adolescents that aim to provide support to overcome barriers to PA. This study reports the results of the co-design, development, and evaluation of a prototype CA called "Phyllis" to support adolescents in overcoming barriers to PA with the aim of improving PA behaviors. The study presents one of the first theory-driven CAs that use existing research, a theoretical framework, and a behavior change model.

OBJECTIVE

The aim of the study is to use a mixed methods approach to investigate the potential of a CA to support adolescents in overcoming barriers to PA and enhance their confidence and motivation to engage in PA.

METHODS

The methodology involved co-designing with 8 adolescents to create a relational and persuasive CA with a suitable persona and dialogue. The CA was evaluated to determine its acceptability, usability, and effectiveness, with 46 adolescents participating in the study via a web-based survey.

RESULTS

The co-design participants were students aged 11 to 13 years, with a sex distribution of 56% (5/9) female and 44% (4/9) male, representing diverse ethnic backgrounds. Participants reported 37 specific barriers to PA, and the most common barriers included a "lack of confidence," "fear of failure," and a "lack of motivation." The CA's persona, named "Phyllis," was co-designed with input from the students, reflecting their preferences for a friendly, understanding, and intelligent personality. Users engaged in 61 conversations with Phyllis and reported a positive user experience, and 73% of them expressed a definite intention to use the fully functional CA in the future, with a net promoter score indicating a high likelihood of recommendation. Phyllis also performed well, being able to recognize a range of different barriers to PA. The CA's persuasive capacity was evaluated in modules focusing on confidence and motivation, with a significant increase in students' agreement in feeling confident and motivated to engage in PA after interacting with Phyllis. Adolescents also expect to have a personalized experience and be able to personalize all aspects of the CA.

CONCLUSIONS

The results showed high acceptability and a positive user experience, indicating the CA's potential. Promising outcomes were observed, with increasing confidence and motivation for PA. Further research and development are needed to create further interventions to address other barriers to PA and assess long-term behavior change. Addressing concerns regarding bias and privacy is crucial for achieving acceptability in the future. The CA's potential extends to health care systems and multimodal support, providing valuable insights for designing digital health interventions including tackling global inactivity issues among adolescents.

Effects of habit formation interventions on physical activity habit strength: meta-analysis and meta-regression

BACKGROUND

Interventions aimed at promoting physical activity (PA) behavior through habit formation pathways are gaining popularity, as they differ from conventional interventions that rely on intention pathways. Past research has established a positive correlation between PA habits and behavior. However, the efficacy of current interventions designed to form PA habits and improve PA automaticity is not yet fully ascertained. Additionally, the intervention components that significantly impact the effectiveness of these interventions are yet to be determined.

METHODS

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. We conducted a search of three databases (PubMed, Embase, and Cochrane Library) from January 2000 to December 2022, with a focus on interventions for developing PA habits. Two independent authors conducted paper selection, quality assessment, data extraction, and coding of behavior change techniques (BCTs). The effect size of interventions was calculated using standardized mean difference. Subgroup analyses were carried out based on follow-up duration, delivery method, sample characteristics, and theory. Furthermore, we employed meta-regression to investigate the association between BCTs and PA habits.

RESULTS

Ten eligible studies with relatively high quality were included in the final data set. Characteristics of studies varied in intervention sample and delivery way. The habit formation interventions significantly increased PA habit (SMD = 0.31, 95% CI 0.14-0.48, P < .001) compared to the control groups. Subgroup analysis demonstrated that the duration of follow-up ≤ 12 weeks have a higher effect size on PA habit than the duration > 12 weeks. Meta-regression revealed that problem solving has a significant positive association with effectiveness improvement (β = 0.36, 95% CI 0.17-0.55), while social reward is linked with a reduction in effectiveness (β = -0.40, 95% CI -0.74-0.06).

CONCLUSIONS

Our findings reveal that habit formation interventions are effective in fostering PA habit. Future studies could leverage the insights form this study to optimize the intervention design and achieve better effectiveness.

Facilitators and barriers to physical activity among English adolescents in secondary schools: a mixed method study

Background

It is evident that physical activity (PA) programmes implemented in schools were not effective in improving PA behaviours among adolescents. This study investigated students' perceptions of barriers to PA among inactive English adolescents in secondary schools based on the Capability, Opportunity, Motivation, and Behaviour (COM-B) model, the Behaviour Change Wheel (BCW), and Theoretical Domains Framework (TDF). The study compared barriers faced by inactive and active groups participating in sports and PA in secondary schools to identify sources of behaviour contributing to inactivity.

Methods

A pre-intervention online survey was distributed to affiliated schools by 233 Teaching Schools Alliances (TSAs) as part of the monitoring and evaluation of the Secondary Teacher Training study. Data were cross-tabulated to analyse activity levels and behavioural barriers for active and inactive groups, using the COM-B domains. The research team followed a seven-step process to categorise barriers based on their relevant domain in the TDF mapped to the COM-B.

Results

The findings were derived from one of the most extensive surveys of adolescents ever undertaken involving 200,623 active and 8,231 inactive respondents. The study identified 52 barriers and 68 behaviours that prevent adolescents from participating in PA. Psychological and social barriers were found to affect all activity levels, genders, and ethnic groups, with a lack of confidence and self-consciousness being the most prevalent. Certain demographic groups, such as those from minority ethnic groups and disabled individuals, were found to be overrepresented among inactive populations. The finding of the study indicated that there were common barriers that affected both inactive and active groups, with further similarity when examining barriers between active and inactive girls. The study also found that girls were more likely to experience the main barriers compared to boys, while inactive boys were more likely to encounter these barriers compared to active boys. The findings suggest that common barriers could be addressed across the population, while recognising some differences in demographics, and the need to provide personalised support. Targeted interventions are also suggested for all girls and inactive boys.

Conclusion

This study highlights the range of barriers that impact adolescents and provides insight into potential mechanisms for behaviour change, including intervention functions, policy categories, and evidence-based behaviour change tools. The study highlights the need for further research to address the barriers to PA among adolescents, particularly those who are inactive. Utilising the findings of this study, future research should investigate the effectiveness of novel digital exercise interventions and policies in increasing PA levels among children and adolescents. Complex digital exercise interventions, including conversational AI solutions, could provide personalised tools to identify and revolutionise support around the multitude of barriers that impact adolescents globally."For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission."

Conversational Agents for Body Weight Management: Systematic Review

BACKGROUND

Obesity is a public health issue worldwide. Conversational agents (CAs), also frequently called chatbots, are computer programs that simulate dialogue between people. Owing to better accessibility, cost-effectiveness, personalization, and compassionate patient-centered treatments, CAs are expected to have the potential to provide sustainable lifestyle counseling for weight management.

OBJECTIVE

This systematic review aimed to critically summarize and evaluate clinical studies on the effectiveness and feasibility of CAs with unconstrained natural language input for weight management.

METHODS

PubMed, Embase, the Cochrane Library (CENTRAL), PsycINFO, and ACM Digital Library were searched up to December 2022. Studies were included if CAs were used for weight management and had a capability for unconstrained natural language input. No restrictions were imposed on study design, language, or publication type. The quality of the included studies was assessed using the Cochrane risk-of-bias assessment tool or the Critical Appraisal Skills Programme checklist. The extracted data from the included studies were tabulated and narratively summarized as substantial heterogeneity was expected.

RESULTS

In total, 8 studies met the eligibility criteria: 3 (38%) randomized controlled trials and 5 (62%) uncontrolled before-and-after studies. The CAs in the included studies were aimed at behavior changes through education, advice on food choices, or counseling via psychological approaches. Of the included studies, only 38% (3/8) reported a substantial weight loss outcome (1.3-2.4 kg decrease at 12-15 weeks of CA use). The overall quality of the included studies was judged as low.

CONCLUSIONS

The findings of this systematic review suggest that CAs with unconstrained natural language input can be used as a feasible interpersonal weight management intervention by promoting engagement in psychiatric intervention-based conversations simulating treatments by health care professionals, but currently there is a paucity of evidence. Well-designed rigorous randomized controlled trials with larger sample sizes, longer treatment duration, and follow-up focusing on CAs' acceptability, efficacy, and safety are warranted.

Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review

BACKGROUND

Artificial intelligence (AI)-based chatbots can offer personalized, engaging, and on-demand health promotion interventions.

OBJECTIVE

The aim of this systematic review was to evaluate the feasibility, efficacy, and intervention characteristics of AI chatbots for promoting health behavior change.

METHODS

A comprehensive search was conducted in 7 bibliographic databases (PubMed, IEEE Xplore, ACM Digital Library, PsycINFO, Web of Science, Embase, and JMIR publications) for empirical articles published from 1980 to 2022 that evaluated the feasibility or efficacy of AI chatbots for behavior change. The screening, extraction, and analysis of the identified articles were performed by following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

RESULTS

Of the 15 included studies, several demonstrated the high efficacy of AI chatbots in promoting healthy lifestyles (n=6, 40%), smoking cessation (n=4, 27%), treatment or medication adherence (n=2, 13%), and reduction in substance misuse (n=1, 7%). However, there were mixed results regarding feasibility, acceptability, and usability. Selected behavior change theories and expert consultation were used to develop the behavior change strategies of AI chatbots, including goal setting, monitoring, real-time reinforcement or feedback, and on-demand support. Real-time user-chatbot interaction data, such as user preferences and behavioral performance, were collected on the chatbot platform to identify ways of providing personalized services. The AI chatbots demonstrated potential for scalability by deployment through accessible devices and platforms (eg, smartphones and Facebook Messenger). The participants also reported that AI chatbots offered a nonjudgmental space for communicating sensitive information. However, the reported results need to be interpreted with caution because of the moderate to high risk of internal validity, insufficient description of AI techniques, and limitation for generalizability.

CONCLUSIONS

AI chatbots have demonstrated the efficacy of health behavior change interventions among large and diverse populations; however, future studies need to adopt robust randomized control trials to establish definitive conclusions.

Factors associated with long-term use of digital devices in the electronic Framingham Heart Study

Long-term use of digital devices is critical for successful clinical or research use, but digital health studies are challenged by a rapid drop-off in participation. A nested e-cohort (eFHS) is embedded in the Framingham Heart Study and uses three system components: a new smartphone app, a digital blood pressure (BP) cuff, and a smartwatch. This study aims to identify factors associated with the use of individual eFHS system components over 1-year. Among 1948 eFHS enrollees, we examine participants who returned surveys within 90 days (n = 1918), and those who chose to use the smartwatch (n = 1243) and BP cuff (n = 1115). For each component, we investigate the same set of candidate predictors for usage and use generalized linear mixed models to select predictors (P < 0.1, P value from Z test statistic), adjusting for age, sex, and time (app use: 3-month period, device use: weekly). A multivariable model with the predictors selected from initial testing is used to identify factors associated with use of components (P < 0.05, P value from Z test statistic) adjusting for age, sex, and time. In multivariable models, older age is associated with higher use of all system components. Female sex and higher education levels are associated with higher completion of app-based surveys whereas higher scores for depressive symptoms, and lower than excellent self-rated health are associated with lower use of the smartwatch over the 12-month follow-up. Our findings show that sociodemographic and health related factors are significantly associated with long-term use of digital devices. Future research is needed to test interventional strategies focusing on these factors to evaluate improvement in long-term engagement.

Let's (Tik) Talk About Fitness Trends

Several factors that follow the development of society affect physical inactivity, which primarily includes the development of technology and digitalization and the increasing choice of unhealthy lifestyle habits. However, certain shifts in the fitness industry have been noted in the last decade. The development of wearable technologies and artificial intelligence is one of the leading fitness trends and undoubtedly represents the future of the fitness industry. On the other hand, the significant influence of social media and networks affects the development and attitudes of people related to physical activity. Therefore, this review paper evaluates the advantages and disadvantages of wearable technologies and artificial intelligence, the positive and negative effects of social networks, and points out the problems accompanying these new fitness trends. The development of fitness trends follows humanity's needs, and one of the biggest challenges is incorporating these novelties in a mission to improve physical activity levels worldwide.

Usability Testing of a Social Media Chatbot for Increasing Physical Activity Behavior

Digital interventions for increasing physical activity behavior have shown great potential, especially those with social media. Chatbots, also known as conversational agents, have emerged in healthcare in relation to digital interventions and have proven effective in promoting physical activity among adults. The study’s objective is to explore users’ experiences with a social media chatbot. The concept and the prototype development of the social media chatbot MYA were realized in three steps: requirement analysis, concept development, and implementation. MYA’s design includes behavior change techniques effective in increasing physical activity through digital interventions. Participants in a usability study answered a survey with the Chatbot Usability Questionnaire (CUQ), which is comparable to the Systems Usability Scale. The mean CUQ score was below 68, the benchmark for average usability. The highest mean CUQ score was 64.5 for participants who thought MYA could help increase their physical activity behavior. The lowest mean CUQ score was 40.6 for participants aged between 50 and 69 years. Generally, MYA was considered to be welcoming, very easy to use, realistic, engaging, and informative. However, some technical issues were identified. A good and diversified user experience promotes prolonged chatbot use. Addressing identified issues will enhance users’ interaction with MYA.

Conversational agents in healthcare: a scoping review of their behavior change techniques and underpinning theory (Preprint)

Background

Conversational agents (CAs) are increasingly used in health care to deliver behavior change interventions. Their evaluation often includes categorizing the behavior change techniques (BCTs) using a classification system of which the BCT Taxonomy v1 (BCTTv1) is one of the most common. Previous studies have presented descriptive summaries of behavior change interventions delivered by CAs, but no in-depth study reporting the use of BCTs in these interventions has been published to date.

Objective

This review aims to describe behavior change interventions delivered by CAs and to identify the BCTs and theories guiding their design.

Methods

We searched PubMed, Embase, Cochrane’s Central Register of Controlled Trials, and the first 10 pages of Google and Google Scholar in April 2021. We included primary, experimental studies evaluating a behavior change intervention delivered by a CA. BCTs coding followed the BCTTv1. Two independent reviewers selected the studies and extracted the data. Descriptive analysis and frequent itemset mining to identify BCT clusters were performed.

Results

We included 47 studies reporting on mental health (n=19, 40%), chronic disorders (n=14, 30%), and lifestyle change (n=14, 30%) interventions. There were 20/47 embodied CAs (43%) and 27/47 CAs (57%) represented a female character. Most CAs were rule based (34/47, 72%). Experimental interventions included 63 BCTs, (mean 9 BCTs; range 2-21 BCTs), while comparisons included 32 BCTs (mean 2 BCTs; range 2-17 BCTs). Most interventions included BCTs 4.1 “Instruction on how to perform a behavior” (34/47, 72%), 3.3 “Social support” (emotional; 27/47, 57%), and 1.2 “Problem solving” (24/47, 51%). A total of 12/47 studies (26%) were informed by a behavior change theory, mainly the Transtheoretical Model and the Social Cognitive Theory. Studies using the same behavior change theory included different BCTs.

Conclusions

There is a need for the more explicit use of behavior change theories and improved reporting of BCTs in CA interventions to enhance the analysis of intervention effectiveness and improve the reproducibility of research.

A systematic review of artificial intelligence chatbots for promoting physical activity, healthy diet, and weight loss

BACKGROUND

This systematic review aimed to evaluate AI chatbot characteristics, functions, and core conversational capacities and investigate whether AI chatbot interventions were effective in changing physical activity, healthy eating, weight management behaviors, and other related health outcomes.

METHODS

In collaboration with a medical librarian, six electronic bibliographic databases (PubMed, EMBASE, ACM Digital Library, Web of Science, PsycINFO, and IEEE) were searched to identify relevant studies. Only randomized controlled trials or quasi-experimental studies were included. Studies were screened by two independent reviewers, and any discrepancy was resolved by a third reviewer. The National Institutes of Health quality assessment tools were used to assess risk of bias in individual studies. We applied the AI Chatbot Behavior Change Model to characterize components of chatbot interventions, including chatbot characteristics, persuasive and relational capacity, and evaluation of outcomes.

RESULTS

The database search retrieved 1692 citations, and 9 studies met the inclusion criteria. Of the 9 studies, 4 were randomized controlled trials and 5 were quasi-experimental studies. Five out of the seven studies suggest chatbot interventions are promising strategies in increasing physical activity. In contrast, the number of studies focusing on changing diet and weight status was limited. Outcome assessments, however, were reported inconsistently across the studies. Eighty-nine and thirty-three percent of the studies specified a name and gender (i.e., woman) of the chatbot, respectively. Over half (56%) of the studies used a constrained chatbot (i.e., rule-based), while the remaining studies used unconstrained chatbots that resemble human-to-human communication.

CONCLUSION

Chatbots may improve physical activity, but we were not able to make definitive conclusions regarding the efficacy of chatbot interventions on physical activity, diet, and weight management/loss. Application of AI chatbots is an emerging field of research in lifestyle modification programs and is expected to grow exponentially. Thus, standardization of designing and reporting chatbot interventions is warranted in the near future.

SYSTEMATIC REVIEW REGISTRATION

International Prospective Register of Systematic Reviews (PROSPERO): CRD42020216761 .

Feasibility, Usability, and Effectiveness of a Machine Learning-Based Physical Activity Chatbot: Quasi-Experimental Study

BACKGROUND

Behavioral eHealth and mobile health interventions have been moderately successful in increasing physical activity, although opportunities for further improvement remain to be discussed. Chatbots equipped with natural language processing can interact and engage with users and help continuously monitor physical activity by using data from wearable sensors and smartphones. However, a limited number of studies have evaluated the effectiveness of chatbot interventions on physical activity.

OBJECTIVE

This study aims to investigate the feasibility, usability, and effectiveness of a machine learning-based physical activity chatbot.

METHODS

A quasi-experimental design without a control group was conducted with outcomes evaluated at baseline and 6 weeks. Participants wore a Fitbit Flex 1 (Fitbit LLC) and connected to the chatbot via the Messenger app. The chatbot provided daily updates on the physical activity level for self-monitoring, sent out daily motivational messages in relation to goal achievement, and automatically adjusted the daily goals based on physical activity levels in the last 7 days. When requested by the participants, the chatbot also provided sources of information on the benefits of physical activity, sent general motivational messages, and checked participants' activity history (ie, the step counts/min that were achieved on any day). Information about usability and acceptability was self-reported. The main outcomes were daily step counts recorded by the Fitbit and self-reported physical activity.

RESULTS

Among 116 participants, 95 (81.9%) were female, 85 (73.3%) were in a relationship, 101 (87.1%) were White, and 82 (70.7%) were full-time workers. Their average age was 49.1 (SD 9.3) years with an average BMI of 32.5 (SD 8.0) kg/m2. Most experienced technical issues were due to an unexpected change in Facebook policy (93/113, 82.3%). Most of the participants scored the usability of the chatbot (101/113, 89.4%) and the Fitbit (99/113, 87.6%) as at least "OK." About one-third (40/113, 35.4%) would continue to use the chatbot in the future, and 53.1% (60/113) agreed that the chatbot helped them become more active. On average, 6.7 (SD 7.0) messages/week were sent to the chatbot and 5.1 (SD 7.4) min/day were spent using the chatbot. At follow-up, participants recorded more steps (increase of 627, 95% CI 219-1035 steps/day) and total physical activity (increase of 154.2 min/week; 3.58 times higher at follow-up; 95% CI 2.28-5.63). Participants were also more likely to meet the physical activity guidelines (odds ratio 6.37, 95% CI 3.31-12.27) at follow-up.

CONCLUSIONS

The machine learning-based physical activity chatbot was able to significantly increase participants' physical activity and was moderately accepted by the participants. However, the Facebook policy change undermined the chatbot functionality and indicated the need to use independent platforms for chatbot deployment to ensure successful delivery of this type of intervention.

Acceptance of an Informational Antituberculosis Chatbot Among Korean Adults: Mixed Methods Research

BACKGROUND

Tuberculosis (TB) is a highly infectious disease. Negative perceptions and insufficient knowledge have made its eradication difficult. Recently, mobile health care interventions, such as an anti-TB chatbot developed by the research team, have emerged in support of TB eradication programs. However, before the anti-TB chatbot is deployed, it is important to understand the factors that predict its acceptance by the population.

OBJECTIVE

This study aims to explore the acceptance of an anti-TB chatbot that provides information about the disease and its treatment to people vulnerable to TB in South Korea. Thus, we are investigating the factors that predict technology acceptance through qualitative research based on the interviews of patients with TB and homeless facility personnel. We are then verifying the extended Technology Acceptance Model (TAM) and predicting the factors associated with the acceptance of the chatbot.

METHODS

In study 1, we conducted interviews with potential chatbot users to extract the factors that predict user acceptance and constructed a conceptual framework based on the TAM. In total, 16 interviews with patients with TB and one focus group interview with 10 experts on TB were conducted. In study 2, we conducted surveys of potential chatbot users to validate the extended TAM. Survey participants were recruited among late-stage patients in TB facilities and members of web-based communities sharing TB information. A total of 123 responses were collected.

RESULTS

The results indicate that perceived ease of use and social influence were significantly predictive of perceived usefulness (P=.04 and P<.001, respectively). Perceived usefulness was predictive of the attitude toward the chatbot (P<.001), whereas perceived ease of use (P=.88) was not. Behavioral intention was positively predicted by attitude toward the chatbot and facilitating conditions (P<.001 and P=.03, respectively). The research model explained 55.4% of the variance in the use of anti-TB chatbots. The moderating effect of TB history was found in the relationship between attitude toward the chatbot and behavioral intention (P=.01) and between facilitating conditions and behavioral intention (P=.02).

CONCLUSIONS

This study can be used to inform future design of anti-TB chatbots and highlight the importance of services and the environment that empower people to use the technology.

Recruitment in a research study via chatbot versus telephone outreach: a randomized trial at a minority-serving institution

Chatbots are software applications to simulate a conversation with a person. The effectiveness of chatbots in facilitating the recruitment of study participants in research, specifically among racial and ethnic minorities, is unknown. The objective of this study is to compare a chatbot versus telephone-based recruitment in enrolling research participants from a predominantly minority patient population at an urban institution. We randomly allocated adults to receive either chatbot or telephone-based outreach regarding a study about vaccine hesitancy. The primary outcome was the proportion of participants who provided consent to participate in the study. In 935 participants, the proportion who answered contact attempts was significantly lower in the chatbot versus telephone group (absolute difference -21.8%; 95% confidence interval [CI] -27.0%, -16.5%; P < 0.001). The consent rate was also significantly lower in the chatbot group (absolute difference -3.4%; 95% CI -5.7%, -1.1%; P = 0.004). However, among participants who answered a contact attempt, the difference in consent rates was not significant. In conclusion, the consent rate was lower with chatbot compared to telephone-based outreach. The difference in consent rates was due to a lower proportion of participants in the chatbot group who answered a contact attempt.

Security, privacy, and healthcare-related conversational agents: a scoping review

Health chatbots interview patients and collect health data. This process makes demands on data security and data privacy. To identify how and to what extent security and privacy are considered in current health chatbots. We conducted a scoping review by searching three bibliographic databases (PubMed, ACM Digital Library, IEEExplore) for papers reporting on chatbots in healthcare. We extracted which, how, and where data is stored by health chatbots and identified which external services have access to the data. Out of 1026 retrieved papers, we included 70 studies in the qualitative synthesis. Most papers report on chatbots that collect and process personal health data, usually in the context of mental health coaching applications. The majority did not provide any information regarding security or privacy aspects. We were able to determine limitations in literature and identified concrete challenges, including data access and usage of (third-party) services, data storage, data security methods, use case peculiarities and data privacy, as well as legal requirements. Data privacy and security in health chatbots are still underresearched and related information is underrepresented in scientific literature. By addressing the five key challenges in future, the transfer of theoretical solutions into practice can be facilitated.

Promoting Physical Activity Through Conversational Agents: Mixed Methods Systematic Review

BACKGROUND

Regular physical activity (PA) is crucial for well-being; however, healthy habits are difficult to create and maintain. Interventions delivered via conversational agents (eg, chatbots or virtual agents) are a novel and potentially accessible way to promote PA. Thus, it is important to understand the evolving landscape of research that uses conversational agents.

OBJECTIVE

This mixed methods systematic review aims to summarize the usability and effectiveness of conversational agents in promoting PA, describe common theories and intervention components used, and identify areas for further development.

METHODS

We conducted a mixed methods systematic review. We searched seven electronic databases (PsycINFO, PubMed, Embase, CINAHL, ACM Digital Library, Scopus, and Web of Science) for quantitative, qualitative, and mixed methods studies that conveyed primary research on automated conversational agents designed to increase PA. The studies were independently screened, and their methodological quality was assessed using the Mixed Methods Appraisal Tool by 2 reviewers. Data on intervention impact and effectiveness, treatment characteristics, and challenges were extracted and analyzed using parallel-results convergent synthesis and narrative summary.

RESULTS

In total, 255 studies were identified, 7.8% (20) of which met our inclusion criteria. The methodological quality of the studies was varied. Overall, conversational agents had moderate usability and feasibility. Those that were evaluated through randomized controlled trials were found to be effective in promoting PA. Common challenges facing interventions were repetitive program content, high attrition, technical issues, and safety and privacy concerns.

CONCLUSIONS

Conversational agents hold promise for PA interventions. However, there is a lack of rigorous research on long-term intervention effectiveness and patient safety. Future interventions should be based on evidence-informed theories and treatment approaches and should address users' desires for program variety, natural language processing, delivery via mobile devices, and safety and privacy concerns.

Leveraging conversational technology to answer common COVID-19 questions

The rapidly evolving science about the Coronavirus Disease 2019 (COVID-19) pandemic created unprecedented health information needs and dramatic changes in policies globally. We describe a platform, Watson Assistant (WA), which has been used to develop conversational agents to deliver COVID-19 related information. We characterized the diverse use cases and implementations during the early pandemic and measured adoption through a number of users, messages sent, and conversational turns (ie, pairs of interactions between users and agents). Thirty-seven institutions in 9 countries deployed COVID-19 conversational agents with WA between March 30 and August 10, 2020, including 24 governmental agencies, 7 employers, 5 provider organizations, and 1 health plan. Over 6.8 million messages were delivered through the platform. The mean number of conversational turns per session ranged between 1.9 and 3.5. Our experience demonstrates that conversational technologies can be rapidly deployed for pandemic response and are adopted globally by a wide range of users.

A Process Evaluation Examining the Performance, Adherence, and Acceptability of a Physical Activity and Diet Artificial Intelligence Virtual Health Assistant

Artificial intelligence virtual health assistants are a promising emerging technology. This study is a process evaluation of a 12-week pilot physical activity and diet program delivered by virtual assistant “Paola”. This single-arm repeated measures study (n = 28, aged 45–75 years) was evaluated on technical performance (accuracy of conversational exchanges), engagement (number of weekly check-ins completed), adherence (percentage of step goal and recommended food servings), and user feedback. Paola correctly asked scripted questions and responded to participants during the check-ins 97% and 96% of the time, respectively, but correctly responded to spontaneous exchanges only 21% of the time. Participants completed 63% of weekly check-ins and conducted a total of 3648 exchanges. Mean dietary adherence was 91% and was lowest for discretionary foods, grains, red meat, and vegetables. Participants met their step goal 59% of the time. Participants enjoyed the program and found Paola useful during check-ins but not for spontaneous exchanges. More in-depth knowledge, personalized advice and spontaneity were identified as important improvements. Virtual health assistants should ensure an adequate knowledge base and ability to recognize intents and entities, include personality and spontaneity, and provide ongoing technical troubleshooting of the virtual assistant to ensure the assistant remains effective.

Artificial Intelligence Chatbot Behavior Change Model for Designing Artificial Intelligence Chatbots to Promote Physical Activity and a Healthy Diet: Viewpoint

BACKGROUND

Chatbots empowered by artificial intelligence (AI) can increasingly engage in natural conversations and build relationships with users. Applying AI chatbots to lifestyle modification programs is one of the promising areas to develop cost-effective and feasible behavior interventions to promote physical activity and a healthy diet.

OBJECTIVE

The purposes of this perspective paper are to present a brief literature review of chatbot use in promoting physical activity and a healthy diet, describe the AI chatbot behavior change model our research team developed based on extensive interdisciplinary research, and discuss ethical principles and considerations.

METHODS

We conducted a preliminary search of studies reporting chatbots for improving physical activity and/or diet in four databases in July 2020. We summarized the characteristics of the chatbot studies and reviewed recent developments in human-AI communication research and innovations in natural language processing. Based on the identified gaps and opportunities, as well as our own clinical and research experience and findings, we propose an AI chatbot behavior change model.

RESULTS

Our review found a lack of understanding around theoretical guidance and practical recommendations on designing AI chatbots for lifestyle modification programs. The proposed AI chatbot behavior change model consists of the following four components to provide such guidance: (1) designing chatbot characteristics and understanding user background; (2) building relational capacity; (3) building persuasive conversational capacity; and (4) evaluating mechanisms and outcomes. The rationale and evidence supporting the design and evaluation choices for this model are presented in this paper.

CONCLUSIONS

As AI chatbots become increasingly integrated into various digital communications, our proposed theoretical framework is the first step to conceptualize the scope of utilization in health behavior change domains and to synthesize all possible dimensions of chatbot features to inform intervention design and evaluation. There is a need for more interdisciplinary work to continue developing AI techniques to improve a chatbot's relational and persuasive capacities to change physical activity and diet behaviors with strong ethical principles.