Feasibility, Usability, and Effectiveness of a Machine Learning-Based Physical Activity Chatbot: Quasi-Experimental Study

Human Factors in AI-Driven Digital Solutions for Increasing Physical Activity: Scoping Review

BACKGROUND

Artificial intelligence (AI) has the potential to enhance physical activity (PA) interventions. However, human factors (HFs) play a pivotal role in the successful integration of AI into mobile health (mHealth) solutions for promoting PA. Understanding and optimizing the interaction between individuals and AI-driven mHealth apps is essential for achieving the desired outcomes.

OBJECTIVE

This study aims to review and describe the current evidence on the HFs in AI-driven digital solutions for increasing PA.

METHODS

We conducted a scoping review by searching for publications containing terms related to PA, HFs, and AI in the titles and abstracts across 3 databases-PubMed, Embase, and IEEE Xplore-and Google Scholar. Studies were included if they were primary studies describing an AI-based solution aimed at increasing PA, and results from testing the solution were reported. Studies that did not meet these criteria were excluded. Additionally, we searched the references in the included articles for relevant research. The following data were extracted from included studies and incorporated into a qualitative synthesis: bibliographic information, study characteristics, population, intervention, comparison, outcomes, and AI-related information. The certainty of the evidence in the included studies was evaluated using GRADE (Grading of Recommendations Assessment, Development, and Evaluation).

RESULTS

A total of 15 studies published between 2015 and 2023 involving 899 participants aged approximately between 19 and 84 years, 60.7% (546/899) of whom were female participants, were included in this review. The interventions lasted between 2 and 26 weeks in the included studies. Recommender systems were the most commonly used AI technology in digital solutions for PA (10/15 studies), followed by conversational agents (4/15 studies). User acceptability and satisfaction were the HFs most frequently evaluated (5/15 studies each), followed by usability (4/15 studies). Regarding automated data collection for personalization and recommendation, most systems involved fitness trackers (5/15 studies). The certainty of the evidence analysis indicates moderate certainty of the effectiveness of AI-driven digital technologies in increasing PA (eg, number of steps, distance walked, or time spent on PA). Furthermore, AI-driven technology, particularly recommender systems, seems to positively influence changes in PA behavior, although with very low certainty evidence.

CONCLUSIONS

Current research highlights the potential of AI-driven technologies to enhance PA, though the evidence remains limited. Longer-term studies are necessary to assess the sustained impact of AI-driven technologies on behavior change and habit formation. While AI-driven digital solutions for PA hold significant promise, further exploration into optimizing AI's impact on PA and effectively integrating AI and HFs is crucial for broader benefits. Thus, the implications for innovation management involve conducting long-term studies, prioritizing diversity, ensuring research quality, focusing on user experience, and understanding the evolving role of AI in PA promotion.

Usability and Utility of a Mobile App to Deliver Health-Related Content to an Older Adult Population: Pilot Noncontrolled Quasi-Experimental Study

BACKGROUND

Digital patient-centered interventions may be important tools for improving and promoting social interaction, health, and well-being among older adults. In this regard, we developed a mobile app called DigiAdherence for an older adult population, which consisted of easy-to-access short videos and messages, to improve health-related knowledge among them and prevent common health conditions, such as falls, polypharmacy, treatment adherence, nutritional problems, and physical inactivity.

OBJECTIVE

This study aimed to assess the usability and utility of the DigiAdherence app among Portuguese older adults 65 years or older.

METHODS

In this pilot noncontrolled quasi-experimental study, older adults who were patients at the primary health care center in Portimão, Portugal, and owned a smartphone or tablet were recruited. Participants were assessed at baseline, given access to the DigiAdherence app for 1 month, and assessed again immediately after 30 days (first assessment) and 60 days after stopping the use of the app (second assessment). App usability and utility (primary outcomes) were analyzed in the first follow-up assessment using a structured questionnaire with 8 items. In the second follow-up assessment, our focus was on knowledge acquired through the app. Secondary outcomes such as treatment adherence and health-related quality of life were also assessed.

RESULTS

The study included 26 older adults. Most participants rated the different functionalities of the app positively and perceived the app as useful, attractive, and user-friendly (median score of 6 on a 7-point Likert scale). In addition, after follow-up, participants reported having a sense of security and greater knowledge in preventing falls (16/24, 67%) and managing therapies and polypharmacy (16/26, 62%).

CONCLUSIONS

The DigiAdherence mobile app was useful and highly accepted by older adults, who developed more confidence regarding health-related knowledge.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/29675.

A scoping review of methodologies for applying artificial intelligence to physical activity interventions

PURPOSE

This scoping review aimed to offer researchers and practitioners an understanding of artificial intelligence (AI) applications in physical activity (PA) interventions; introduce them to prevalent machine learning (ML), deep learning (DL), and reinforcement learning (RL) algorithms; and encourage the adoption of AI methodologies.

METHODS

A scoping review was performed in PubMed, Web of Science, Cochrane Library, and EBSCO focusing on AI applications for promoting PA or predicting related behavioral or health outcomes. AI methodologies were summarized and categorized to identify synergies, patterns, and trends informing future research. Additionally, a concise primer on predominant AI methodologies within the realm of PA was provided to bolster understanding and broader application.

RESULTS

The review included 24 studies that met the predetermined eligibility criteria. AI models were found effective in detecting significant patterns of PA behavior and associations between specific factors and intervention outcomes. Most studies comparing AI models to traditional statistical approaches reported higher prediction accuracy for AI models on test data. Comparisons of different AI models yielded mixed results, likely due to model performance being highly dependent on the dataset and task. An increasing trend of adopting state-of-the-art DL and RL models over standard ML was observed, addressing complex human-machine communication, behavior modification, and decision-making tasks. Six key areas for future AI adoption in PA interventions emerged: personalized PA interventions, real-time monitoring and adaptation, integration of multimodal data sources, evaluation of intervention effectiveness, expanding access to PA interventions, and predicting and preventing injuries.

CONCLUSION

The scoping review highlights the potential of AI methodologies for advancing PA interventions. As the field progresses, staying informed and exploring emerging AI-driven strategies is essential for achieving significant improvements in PA interventions and fostering overall well-being.

"Better than my professor?" How to develop artificial intelligence tools for higher education

Artificial Intelligence (AI) tools are currently designed and tested in many fields to improve humans' ability to make decisions. One of these fields is higher education. For example, AI-based chatbots ("conversational pedagogical agents") could engage in conversations with students in order to provide timely feedback and responses to questions while the learning process is taking place and to collect data to personalize the delivery of course materials. However, many existent tools are able to perform tasks that human professionals (educators, tutors, professors) could perform, just in a timelier manner. While discussing the possible implementation of AI-based tools in our university's educational programs, we reviewed the current literature and identified a number of capabilities that future AI solutions may feature, in order to improve higher education processes, with a focus on distance higher education. Specifically, we suggest that innovative tools could influence the methodologies by which students approach learning; facilitate connections and information attainment beyond course materials; support the communication with the professor; and, draw from motivation theories to foster learning engagement, in a personalized manner. Future research should explore high-level opportunities represented by AI for higher education, including their effects on learning outcomes and the quality of the learning experience as a whole.

Attributes, Methods, and Frameworks Used to Evaluate Wearables and Their Companion mHealth Apps: Scoping Review

BACKGROUND

Wearable devices, mobile technologies, and their combination have been accepted into clinical use to better assess the physical fitness and quality of life of patients and as preventive measures. Usability is pivotal for overcoming constraints and gaining users' acceptance of technology such as wearables and their companion mobile health (mHealth) apps. However, owing to limitations in design and evaluation, interactive wearables and mHealth apps have often been restricted from their full potential.

OBJECTIVE

This study aims to identify studies that have incorporated wearable devices and determine their frequency of use in conjunction with mHealth apps or their combination. Specifically, this study aims to understand the attributes and evaluation techniques used to evaluate usability in the health care domain for these technologies and their combinations.

METHODS

We conducted an extensive search across 4 electronic databases, spanning the last 30 years up to December 2021. Studies including the keywords "wearable devices," "mobile apps," "mHealth apps," "physiological data," "usability," "user experience," and "user evaluation" were considered for inclusion. A team of 5 reviewers screened the collected publications and charted the features based on the research questions. Subsequently, we categorized these characteristics following existing usability and wearable taxonomies. We applied a methodological framework for scoping reviews and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist.

RESULTS

A total of 382 reports were identified from the search strategy, and 68 articles were included. Most of the studies (57/68, 84%) involved the simultaneous use of wearables and connected mobile apps. Wrist-worn commercial consumer devices such as wristbands were the most prevalent, accounting for 66% (45/68) of the wearables identified in our review. Approximately half of the data from the medical domain (32/68, 47%) focused on studies involving participants with chronic illnesses or disorders. Overall, 29 usability attributes were identified, and 5 attributes were frequently used for evaluation: satisfaction (34/68, 50%), ease of use (27/68, 40%), user experience (16/68, 24%), perceived usefulness (18/68, 26%), and effectiveness (15/68, 22%). Only 10% (7/68) of the studies used a user- or human-centered design paradigm for usability evaluation.

CONCLUSIONS

Our scoping review identified the types and categories of wearable devices and mHealth apps, their frequency of use in studies, and their implementation in the medical context. In addition, we examined the usability evaluation of these technologies: methods, attributes, and frameworks. Within the array of available wearables and mHealth apps, health care providers encounter the challenge of selecting devices and companion apps that are effective, user-friendly, and compatible with user interactions. The current gap in usability and user experience in health care research limits our understanding of the strengths and limitations of wearable technologies and their companion apps. Additional research is necessary to overcome these limitations.

Evaluation framework for conversational agents with artificial intelligence in health interventions: a systematic scoping review

OBJECTIVES

Conversational agents (CAs) with emerging artificial intelligence present new opportunities to assist in health interventions but are difficult to evaluate, deterring their applications in the real world. We aimed to synthesize existing evidence and knowledge and outline an evaluation framework for CA interventions.

MATERIALS AND METHODS

We conducted a systematic scoping review to investigate designs and outcome measures used in the studies that evaluated CAs for health interventions. We then nested the results into an overarching digital health framework proposed by the World Health Organization (WHO).

RESULTS

The review included 81 studies evaluating CAs in experimental (n = 59), observational (n = 15) trials, and other research designs (n = 7). Most studies (n = 72, 89%) were published in the past 5 years. The proposed CA-evaluation framework includes 4 evaluation stages: (1) feasibility/usability, (2) efficacy, (3) effectiveness, and (4) implementation, aligning with WHO's stepwise evaluation strategy. Across these stages, this article presents the essential evidence of different study designs (n = 8), sample sizes, and main evaluation categories (n = 7) with subcategories (n = 40). The main evaluation categories included (1) functionality, (2) safety and information quality, (3) user experience, (4) clinical and health outcomes, (5) costs and cost benefits, (6) usage, adherence, and uptake, and (7) user characteristics for implementation research. Furthermore, the framework highlighted the essential evaluation areas (potential primary outcomes) and gaps across the evaluation stages.

DISCUSSION AND CONCLUSION

This review presents a new framework with practical design details to support the evaluation of CA interventions in healthcare research.

PROTOCOL REGISTRATION

The Open Science Framework (https://osf.io/9hq2v) on March 22, 2021.

Construction and evaluation of a cloud follow-up platform for gynecological patients receiving chemotherapy

BACKGROUND

Patient follow-up is an essential component of hospital management. In the current information era, the patient follow-up scheme is expected to be replaced by Internet technology. This study constructed a cloud follow-up platform for gynecological chemotherapy patients and assessed its cost-effectiveness and patients' feedback.

METHODS

A total of 2,538 patients were followed up using a cloud follow-up system between January and October 2021. Prior to this, 690 patients were followed manually via telephone calls. Patients' characteristics, follow-up rate, satisfaction, and session duration were compared between the cloud follow-up and manual follow-up groups. In addition, the read rate of health education materials in the cloud follow-up group was analyzed.

RESULTS

General information, including age, education attainment, cancer stage, and disease category, and follow-up rate (cloud: 6,957/7,614, 91.4%; manual: 1,869/2,070, 90.3%; P = 0.13) did not significantly differ between the two groups. The follow-up satisfaction of the cloud follow-up patients was significantly better than that of the manual follow-up group (cloud: 7,192/7,614, 94.5%; manual: 1,532/2,070, 74.0%; P<0.001). The time spent on the follow-up was approximately 1.2 h for 100 patients in the cloud follow-up group and 10.5 h in the manual follow-up group. Multivariate analysis indicated that the cloud follow-up group had significantly greater follow-up satisfaction (odds ratio: 2.239, 95% CI: 1.237 ~ 5.219). Additionally, the average follow-up duration of the cloud follow-up group decreased by 9.287 h (coefficient: -9.287, 95% CI: -1.439~-0.165). The read rate of health education materials was 72.9% in the cloud follow-up group.

CONCLUSIONS

The follow-up effect of the cloud follow-up group was not inferior to that of the manual follow-up group. The cloud follow-up was more effective for prevention and control requirements in the post-epidemic era. Cloud follow-up can save medical resources, improve cost-effectiveness, provide sufficient health education resources for patients, and improve their satisfaction.

Patient Acceptability of Symptom Screening and Patient Education Using a Chatbot for Autoimmune Inflammatory Diseases: Survey Study

BACKGROUND

Chatbots have the potential to enhance health care interaction, satisfaction, and service delivery. However, data regarding their acceptance across diverse patient populations are limited. In-depth studies on the reception of chatbots by patients with chronic autoimmune inflammatory diseases are lacking, although such studies are vital for facilitating the effective integration of chatbots in rheumatology care.

OBJECTIVE

We aim to assess patient perceptions and acceptance of a chatbot designed for autoimmune inflammatory rheumatic diseases (AIIRDs).

METHODS

We administered a comprehensive survey in an outpatient setting at a top-tier rheumatology referral center. The target cohort included patients who interacted with a chatbot explicitly tailored to facilitate diagnosis and obtain information on AIIRDs. Following the RE-AIM (Reach, Effectiveness, Adoption, Implementation and Maintenance) framework, the survey was designed to gauge the effectiveness, user acceptability, and implementation of the chatbot.

RESULTS

Between June and October 2022, we received survey responses from 200 patients, with an equal number of 100 initial consultations and 100 follow-up (FU) visits. The mean scores on a 5-point acceptability scale ranged from 4.01 (SD 0.63) to 4.41 (SD 0.54), indicating consistently high ratings across the different aspects of chatbot performance. Multivariate regression analysis indicated that having a FU visit was significantly associated with a greater willingness to reuse the chatbot for symptom determination (P=.01). Further, patients' comfort with chatbot diagnosis increased significantly after meeting physicians (P<.001). We observed no significant differences in chatbot acceptance according to sex, education level, or diagnosis category.

CONCLUSIONS

This study underscores that chatbots tailored to AIIRDs have a favorable reception. The inclination of FU patients to engage with the chatbot signifies the possible influence of past clinical encounters and physician affirmation on its use. Although further exploration is required to refine their integration, the prevalent positive perceptions suggest that chatbots have the potential to strengthen the bridge between patients and health care providers, thus enhancing the delivery of rheumatology care to various cohorts.

Factors Influencing the Acceptability, Acceptance, and Adoption of Conversational Agents in Health Care: Integrative Review

BACKGROUND

Conversational agents (CAs), also known as chatbots, are digital dialog systems that enable people to have a text-based, speech-based, or nonverbal conversation with a computer or another machine based on natural language via an interface. The use of CAs offers new opportunities and various benefits for health care. However, they are not yet ubiquitous in daily practice. Nevertheless, research regarding the implementation of CAs in health care has grown tremendously in recent years.

OBJECTIVE

This review aims to present a synthesis of the factors that facilitate or hinder the implementation of CAs from the perspectives of patients and health care professionals. Specifically, it focuses on the early implementation outcomes of acceptability, acceptance, and adoption as cornerstones of later implementation success.

METHODS

We performed an integrative review. To identify relevant literature, a broad literature search was conducted in June 2021 with no date limits and using all fields in PubMed, Cochrane Library, Web of Science, LIVIVO, and PsycINFO. To keep the review current, another search was conducted in March 2022. To identify as many eligible primary sources as possible, we used a snowballing approach by searching reference lists and conducted a hand search. Factors influencing the acceptability, acceptance, and adoption of CAs in health care were coded through parallel deductive and inductive approaches, which were informed by current technology acceptance and adoption models. Finally, the factors were synthesized in a thematic map.

RESULTS

Overall, 76 studies were included in this review. We identified influencing factors related to 4 core Unified Theory of Acceptance and Use of Technology (UTAUT) and Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) factors (performance expectancy, effort expectancy, facilitating conditions, and hedonic motivation), with most studies underlining the relevance of performance and effort expectancy. To meet the particularities of the health care context, we redefined the UTAUT2 factors social influence, habit, and price value. We identified 6 other influencing factors: perceived risk, trust, anthropomorphism, health issue, working alliance, and user characteristics. Overall, we identified 10 factors influencing acceptability, acceptance, and adoption among health care professionals (performance expectancy, effort expectancy, facilitating conditions, social influence, price value, perceived risk, trust, anthropomorphism, working alliance, and user characteristics) and 13 factors influencing acceptability, acceptance, and adoption among patients (additionally hedonic motivation, habit, and health issue).

CONCLUSIONS

This review shows manifold factors influencing the acceptability, acceptance, and adoption of CAs in health care. Knowledge of these factors is fundamental for implementation planning. Therefore, the findings of this review can serve as a basis for future studies to develop appropriate implementation strategies. Furthermore, this review provides an empirical test of current technology acceptance and adoption models and identifies areas where additional research is necessary.

TRIAL REGISTRATION

PROSPERO CRD42022343690; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=343690.

Increasing physical activity using an just-in-time adaptive digital assistant supported by machine learning: A novel approach for hyper-personalised mHealth interventions

OBJECTIVE

Physical inactivity is a leading modifiable cause of death and disease worldwide. Population-based interventions to increase physical activity are needed. Existing automated expert systems (e.g., computer-tailored interventions) have significant limitations that result in low long-term effectiveness. Therefore, innovative approaches are needed. This special communication aims to describe and discuss a novel mHealth intervention approach that proactively offers participants with hyper-personalised intervention content adjusted in real-time.

METHODS

Using machine learning approaches, we propose a novel physical activity intervention approach that can learn and adapt in real-time to achieve high levels of personalisation and user engagement, underpinned by a likeable digital assistant. It will consist of three major components: (1) conversations: to increase user's knowledge on a wide range of activity-related topics underpinned by Natural Language Processing; (2) nudge engine: to provide users with hyper-personalised cues to action underpinned by reinforcement learning (i.e., contextual bandit) and integrating real-time data from activity tracking, GPS, GIS, weather, and user provided data; (3) Q&A: to facilitate users asking any physical activity related questions underpinned by generative AI (e.g., ChatGPT, Bard) for content generation.

RESULTS

The detailed concept of the proposed physical activity intervention platform demonstrates the practical application of a just-in-time adaptive intervention applying various machine learning techniques to deliver a hyper-personalised physical activity intervention in an engaging way. Compared to traditional interventions, the novel platform is expected to show potential for increased user engagement and long-term effectiveness due to: (1) using new variables to personalise content (e.g., GPS, weather), (2) providing behavioural support at the right time in real-time, (3) implementing an engaging digital assistant and (4) improving the relevance of content through applying machine learning algorithms.

CONCLUSION

The use of machine learning is on the rise in every aspect of today's society, however few attempts have been undertaken to harness its potential to achieve health behaviour change. By sharing our intervention concept, we contribute to the ongoing dialogue on creating effective methods for promoting health and well-being in the informatics research community. Future research should focus on refining these techniques and evaluating their effectiveness in controlled and real-world circumstances.

Systematic review and meta-analysis of the effectiveness of chatbots on lifestyle behaviours

Chatbots (also known as conversational agents and virtual assistants) offer the potential to deliver healthcare in an efficient, appealing and personalised manner. The purpose of this systematic review and meta-analysis was to evaluate the efficacy of chatbot interventions designed to improve physical activity, diet and sleep. Electronic databases were searched for randomised and non-randomised controlled trials, and pre-post trials that evaluated chatbot interventions targeting physical activity, diet and/or sleep, published before 1 September 2022. Outcomes were total physical activity, steps, moderate-to-vigorous physical activity (MVPA), fruit and vegetable consumption, sleep quality and sleep duration. Standardised mean differences (SMD) were calculated to compare intervention effects. Subgroup analyses were conducted to assess chatbot type, intervention type, duration, output and use of artificial intelligence. Risk of bias was assessed using the Effective Public Health Practice Project Quality Assessment tool. Nineteen trials were included. Sample sizes ranged between 25-958, and mean participant age ranged between 9-71 years. Most interventions (n = 15, 79%) targeted physical activity, and most trials had a low-quality rating (n = 14, 74%). Meta-analysis results showed significant effects (all p < 0.05) of chatbots for increasing total physical activity (SMD = 0.28 [95% CI = 0.16, 0.40]), daily steps (SMD = 0.28 [95% CI = 0.17, 0.39]), MVPA (SMD = 0.53 [95% CI = 0.24, 0.83]), fruit and vegetable consumption (SMD = 0.59 [95% CI = 0.25, 0.93]), sleep duration (SMD = 0.44 [95% CI = 0.32, 0.55]) and sleep quality (SMD = 0.50 [95% CI = 0.09, 0.90]). Subgroup analyses showed that text-based, and artificial intelligence chatbots were more efficacious than speech/voice chatbots for fruit and vegetable consumption, and multicomponent interventions were more efficacious than chatbot-only interventions for sleep duration and sleep quality (all p < 0.05). Findings from this systematic review and meta-analysis indicate that chatbot interventions are efficacious for increasing physical activity, fruit and vegetable consumption, sleep duration and sleep quality. Chatbot interventions were efficacious across a range of populations and age groups, with both short- and longer-term interventions, and chatbot only and multicomponent interventions being efficacious.

Conversational Agents for Body Weight Management: Systematic Review

BACKGROUND

Obesity is a public health issue worldwide. Conversational agents (CAs), also frequently called chatbots, are computer programs that simulate dialogue between people. Owing to better accessibility, cost-effectiveness, personalization, and compassionate patient-centered treatments, CAs are expected to have the potential to provide sustainable lifestyle counseling for weight management.

OBJECTIVE

This systematic review aimed to critically summarize and evaluate clinical studies on the effectiveness and feasibility of CAs with unconstrained natural language input for weight management.

METHODS

PubMed, Embase, the Cochrane Library (CENTRAL), PsycINFO, and ACM Digital Library were searched up to December 2022. Studies were included if CAs were used for weight management and had a capability for unconstrained natural language input. No restrictions were imposed on study design, language, or publication type. The quality of the included studies was assessed using the Cochrane risk-of-bias assessment tool or the Critical Appraisal Skills Programme checklist. The extracted data from the included studies were tabulated and narratively summarized as substantial heterogeneity was expected.

RESULTS

In total, 8 studies met the eligibility criteria: 3 (38%) randomized controlled trials and 5 (62%) uncontrolled before-and-after studies. The CAs in the included studies were aimed at behavior changes through education, advice on food choices, or counseling via psychological approaches. Of the included studies, only 38% (3/8) reported a substantial weight loss outcome (1.3-2.4 kg decrease at 12-15 weeks of CA use). The overall quality of the included studies was judged as low.

CONCLUSIONS

The findings of this systematic review suggest that CAs with unconstrained natural language input can be used as a feasible interpersonal weight management intervention by promoting engagement in psychiatric intervention-based conversations simulating treatments by health care professionals, but currently there is a paucity of evidence. Well-designed rigorous randomized controlled trials with larger sample sizes, longer treatment duration, and follow-up focusing on CAs' acceptability, efficacy, and safety are warranted.

Feasibility and Acceptability of Chatbots for Nutrition and Physical Activity Health Promotion Among Adolescents: Systematic Scoping Review With Adolescent Consultation

BACKGROUND

Reducing lifestyle risk behaviors among adolescents depends on access to age-appropriate health promotion information. Chatbots-computer programs designed to simulate conversations with human users-have the potential to deliver health information to adolescents to improve their lifestyle behaviors and support behavior change, but research on the feasibility and acceptability of chatbots in the adolescent population is unknown.

OBJECTIVE

This systematic scoping review aims to evaluate the feasibility and acceptability of chatbots in nutrition and physical activity interventions among adolescents. A secondary aim is to consult adolescents to identify features of chatbots that are acceptable and feasible.

METHODS

We searched 6 electronic databases from March to April 2022 (MEDLINE, Embase, Joanna Briggs Institute, the Cumulative Index to Nursing and Allied Health, the Association for Computing Machinery library, and the IT database Institute of Electrical and Electronics Engineers). Peer-reviewed studies were included that were conducted in the adolescent population (10-19 years old) without any chronic disease, except obesity or type 2 diabetes, and assessed chatbots used nutrition or physical activity interventions or both that encouraged individuals to meet dietary or physical activity guidelines and support positive behavior change. Studies were screened by 2 independent reviewers, with any queries resolved by a third reviewer. Data were extracted into tables and collated in a narrative summary. Gray literature searches were also undertaken. Results of the scoping review were presented to a diverse youth advisory group (N=16, 13-18 years old) to gain insights into this topic beyond what is published in the literature.

RESULTS

The search identified 5558 papers, with 5 (0.1%) studies describing 5 chatbots meeting the inclusion criteria. The 5 chatbots were supported by mobile apps using a combination of the following features: personalized feedback, conversational agents, gamification, and monitoring of behavior change. Of the 5 studies, 2 (40.0%) studies focused on nutrition, 2 (40.0%) studies focused on physical activity, and 1 (20.0%) focused on both nutrition and physical activity. Feasibility and acceptability varied across the 5 studies, with usage rates above 50% in 3 (60.0%) studies. In addition, 3 (60.0%) studies reported health-related outcomes, with only 1 (20.0%) study showing promising effects of the intervention. Adolescents presented novel concerns around the use of chatbots in nutrition and physical activity interventions, including ethical concerns and the use of false or misleading information.

CONCLUSIONS

Limited research is available on chatbots in adolescent nutrition and physical activity interventions, finding insufficient evidence on the acceptability and feasibility of chatbots in the adolescent population. Similarly, adolescent consultation identified issues in the design features that have not been mentioned in the published literature. Therefore, chatbot codesign with adolescents may help ensure that such technology is feasible and acceptable to an adolescent population.

Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review

BACKGROUND

Artificial intelligence (AI)-based chatbots can offer personalized, engaging, and on-demand health promotion interventions.

OBJECTIVE

The aim of this systematic review was to evaluate the feasibility, efficacy, and intervention characteristics of AI chatbots for promoting health behavior change.

METHODS

A comprehensive search was conducted in 7 bibliographic databases (PubMed, IEEE Xplore, ACM Digital Library, PsycINFO, Web of Science, Embase, and JMIR publications) for empirical articles published from 1980 to 2022 that evaluated the feasibility or efficacy of AI chatbots for behavior change. The screening, extraction, and analysis of the identified articles were performed by following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

RESULTS

Of the 15 included studies, several demonstrated the high efficacy of AI chatbots in promoting healthy lifestyles (n=6, 40%), smoking cessation (n=4, 27%), treatment or medication adherence (n=2, 13%), and reduction in substance misuse (n=1, 7%). However, there were mixed results regarding feasibility, acceptability, and usability. Selected behavior change theories and expert consultation were used to develop the behavior change strategies of AI chatbots, including goal setting, monitoring, real-time reinforcement or feedback, and on-demand support. Real-time user-chatbot interaction data, such as user preferences and behavioral performance, were collected on the chatbot platform to identify ways of providing personalized services. The AI chatbots demonstrated potential for scalability by deployment through accessible devices and platforms (eg, smartphones and Facebook Messenger). The participants also reported that AI chatbots offered a nonjudgmental space for communicating sensitive information. However, the reported results need to be interpreted with caution because of the moderate to high risk of internal validity, insufficient description of AI techniques, and limitation for generalizability.

CONCLUSIONS

AI chatbots have demonstrated the efficacy of health behavior change interventions among large and diverse populations; however, future studies need to adopt robust randomized control trials to establish definitive conclusions.

Experience and Satisfaction with a Family-Based Physical Activity Intervention Using Activity Trackers and Apps: A Qualitative Study

Wearable activity trackers and smartphone apps have been shown to increase physical activity in children and adults. However, interventions using activity trackers and apps have rarely been tested in whole families. This study examined the experience and satisfaction with an activity tracker and app intervention (Step it Up Family) to increase physical activity in whole families. Telephone interviews were conducted with Queensland-based families (n = 19) who participated in the Step it Up Family intervention (N = 40, single-arm, pre/post feasibility study) in 2017/2018. Using commercial activity trackers combined with apps, the intervention included an introductory session, individual and family-level goal setting, self-monitoring, family step challenges, and weekly motivational text messages. Qualitative content analysis was conducted to identify themes, categories and sub-categories. In summary, parents reported that children were engaged with the activity tracker and app features to reach their daily step goals. Some technical difficulties were experienced with app navigation, syncing of activity tracker data, and tracker band discomfort. Although families liked that the weekly text messages reminded them to be active, they did not find them very motivating. Using text messages for physical activity motivation in families requires further testing. Overall, the intervention was well-received by families for increasing physical activity motivation.

Engagement, acceptability, usability and satisfaction with Active for Life, a computer-tailored web-based physical activity intervention using Fitbits in older adults

BACKGROUND

Preliminary evidence suggests that web-based physical activity interventions with tailored advice and Fitbit integration are effective and may be well suited to older adults. Therefore, this study aimed to examine the engagement, acceptability, usability, and satisfaction with 'Active for Life,' a web-based physical activity intervention providing computer-tailored physical activity advice to older adults.

METHODS

Inactive older adults (n = 243) were randomly assigned into 3 groups: 1) tailoring + Fitbit, 2) tailoring only, or 3) a wait-list control. The tailoring + Fitbit group and the tailoring-only group received 6 modules of computer-tailored physical activity advice over 12 weeks. The advice was informed by objective Fitbit data in the tailoring + Fitbit group and self-reported physical activity in the tailoring-only group. This study examined the engagement, acceptability, usability, and satisfaction of Active for Life in intervention participants (tailoring + Fitbit n = 78, tailoring only n = 96). Wait-list participants were not included. Engagement (Module completion, time on site) were objectively recorded through the intervention website. Acceptability (7-point Likert scale), usability (System Usability Scale), and satisfaction (open-ended questions) were assessed using an online survey at post intervention. ANOVA and Chi square analyses were conducted to compare outcomes between intervention groups and content analysis was used to analyse program satisfaction.

RESULTS

At post-intervention (week 12), study attrition was 28% (22/78) in the Fitbit + tailoring group and 39% (37/96) in the tailoring-only group. Engagement and acceptability were good in both groups, however there were no group differences (module completions: tailoring + Fitbit: 4.72 ± 2.04, Tailoring-only: 4.23 ± 2.25 out of 6 modules, p = .14, time on site: tailoring + Fitbit: 103.46 ± 70.63, Tailoring-only: 96.90 ± 76.37 min in total, p = .56, and acceptability of the advice: tailoring + Fitbit: 5.62 ± 0.89, Tailoring-only: 5.75 ± 0.75 out of 7, p = .41). Intervention usability was modest but significantly higher in the tailoring + Fitbit group (tailoring + Fitbit: 64.55 ± 13.59, Tailoring-only: 57.04 ± 2.58 out of 100, p = .003). Participants reported that Active for Life helped motivate them, held them accountable, improved their awareness of how active they were and helped them to become more active. Conversely, many participants felt as though they would prefer personal contact, more detailed tailoring and more survey response options.

CONCLUSIONS

This study supports web-based physical activity interventions with computer-tailored advice and Fitbit integration as engaging and acceptable in older adults.

TRIAL REGISTRATION

Australian and New Zealand Clinical Trials Registry: ACTRN12618000646246. Registered April 23 2018, https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=374901.

Usability and Credibility of a COVID-19 Vaccine Chatbot for Young Adults and Health Workers in the United States: Formative Mixed Methods Study

BACKGROUND

The COVID-19 pandemic raised novel challenges in communicating reliable, continually changing health information to a broad and sometimes skeptical public, particularly around COVID-19 vaccines, which, despite being comprehensively studied, were the subject of viral misinformation. Chatbots are a promising technology to reach and engage populations during the pandemic. To inform and communicate effectively with users, chatbots must be highly usable and credible.

OBJECTIVE

We sought to understand how young adults and health workers in the United States assessed the usability and credibility of a web-based chatbot called Vira, created by the Johns Hopkins Bloomberg School of Public Health and IBM Research using natural language processing technology. Using a mixed method approach, we sought to rapidly improve Vira's user experience to support vaccine decision-making during the peak of the COVID-19 pandemic.

METHODS

We recruited racially and ethnically diverse young people and health workers, with both groups from urban areas of the United States. We used the validated Chatbot Usability Questionnaire to understand the tool's navigation, precision, and persona. We also conducted 11 interviews with health workers and young people to understand the user experience, whether they perceived the chatbot as confidential and trustworthy, and how they would use the chatbot. We coded and categorized emerging themes to understand the determining factors for participants' assessment of chatbot usability and credibility.

RESULTS

In all, 58 participants completed a web-based usability questionnaire and 11 completed in-depth interviews. Most questionnaire respondents said the chatbot was "easy to navigate" (51/58, 88%) and "very easy to use" (50/58, 86%), and many (45/58, 78%) said its responses were relevant. The mean Chatbot Usability Questionnaire score was 70.2 (SD 12.1) and scores ranged from 40.6 to 95.3. Interview participants felt the chatbot achieved high usability due to its strong functionality, performance, and perceived confidentiality and that the chatbot could attain high credibility with a redesign of its cartoonish visual persona. Young people said they would use the chatbot to discuss vaccination with hesitant friends or family members, whereas health workers used or anticipated using the chatbot to support community outreach, save time, and stay up to date.

CONCLUSIONS

This formative study conducted during the pandemic's peak provided user feedback for an iterative redesign of Vira. Using a mixed method approach provided multidimensional feedback, identifying how the chatbot worked well-being easy to use, answering questions appropriately, and using credible branding-while offering tangible steps to improve the product's visual design. Future studies should evaluate how chatbots support personal health decision-making, particularly in the context of a public health emergency, and whether such outreach tools can reduce staff burnout. Randomized studies should also be conducted to measure how chatbots countering health misinformation affect user knowledge, attitudes, and behavior.

Experiences and perceptions of final-year nursing students of using a chatbot in a simulated emergency situation: A qualitative study

AIM

The aim of this study is to explore the experiences and perceptions of final-year nursing students on the acceptability and feasibility of using a chatbot for clinical decision-making and patient safety.

BACKGROUND

The effective and inclusive use of new technologies such as conversational agents or chatbots could support nurses in increasing evidence-based care and decreasing low-quality services.

METHODS

A descriptive qualitative study was used through focus group interviews. The data analysis was conducted using a thematic analysis.

RESULTS

This study included 114 participants. After our data analysis, two main themes emerged: (i) experiences in the use of a chatbot service for clinical decision-making and and (ii) integrating conversational agents into the organizational safety culture.

CONCLUSIONS

The findings of our study provide preliminary support for the acceptability and feasibility of adopting SafeBot, a chatbot for clinical decision-making and patient safety. Our results revealed substantial recommendations for refining navigation, layout and content, as well as useful insights to support its acceptance in real nursing practice.

IMPLICATIONS FOR NURSING MANAGEMENT

Leaders and managers may well see artificial intelligence-based conversational agents like SafeBot as a potential solution in modern nursing practice for effective problem-solving resolution, innovative staffing and nursing care delivery models at the bedside and criteria for measuring and ensure quality and patient safety.