An iterative, interdisciplinary, collaborative framework for developing and evaluating digital behavior change interventions

Challenges and opportunities in mobile e-coaching

Background

Mobile e-health technologies have proven to provide tailored assessment, intervention, and coaching capabilities for various usage scenarios. Thanks to their spread and adoption, smartphones are one of the most important carriers for such applications.

Problem

However, the process of design, realization, evaluation, and implementation of these e-health solutions is wicked and challenging, requiring multiple stakeholders and expertise.

Method

Here, we present a tailorable intervention and interaction e-health solution that allows rapid prototyping, development, and evaluation of e-health interventions at scale. This platform allows researchers and clinicians to develop ecological momentary assessment, just-in-time adaptive interventions, ecological momentary intervention, cohort studies, and e-coaching and personalized interventions quickly, with no-code, and in a scalable way.

Result

The Twente Intervention and Interaction Instrument (TIIM) has been used by over 320 researchers in the last decade. We present the ecosystem and synthesize the main scientific output from clinical and research studies in different fields.

Discussion

The importance of mobile e-coaching for prediction, management, and prevention of adverse health outcomes is increasing. A profound e-health development strategyand strategic, technical, and operational investments are needed to prototype, develop, implement, and evaluate e-health solutions. TIIM ecosystem has proven to support these processes. This paper ends with the main research opportunities in mobile coaching, including intervention mechanisms, fine-grained monitoring, and inclusion of objective biomarker data.

A mobile healthy lifestyle intervention to promote mental health in adolescence: a mixed-methods evaluation

BACKGROUND

A healthy lifestyle may improve mental health. It is yet not known whether and how a mobile intervention can be of help in achieving this in adolescents. This study investigated the effectiveness and perceived underlying mechanisms of the mobile health (mHealth) intervention #LIFEGOALS to promote healthy lifestyles and mental health. #LIFEGOALS is an evidence-based app with activity tracker, including self-regulation techniques, gamification elements, a support chatbot, and health narrative videos.

METHODS

A quasi-randomized controlled trial (N = 279) with 12-week intervention period and process evaluation interviews (n = 13) took place during the COVID-19 pandemic. Adolescents (12-15y) from the general population were allocated at school-level to the intervention (n = 184) or to a no-intervention group (n = 95). Health-related quality of life (HRQoL), psychological well-being, mood, self-perception, peer support, resilience, depressed feelings, sleep quality and breakfast frequency were assessed via a web-based survey; physical activity, sedentary time, and sleep routine via Axivity accelerometers. Multilevel generalized linear models were fitted to investigate intervention effects and moderation by pandemic-related measures. Interviews were coded using thematic analysis.

RESULTS

Non-usage attrition was high: 18% of the participants in the intervention group never used the app. An additional 30% stopped usage by the second week. Beneficial intervention effects were found for physical activity (χ21 = 4.36, P = .04), sedentary behavior (χ21 = 6.44, P = .01), sleep quality (χ21 = 6.11, P = .01), and mood (χ21 = 2.30, P = .02). However, effects on activity-related behavior were only present for adolescents having normal sports access, and effects on mood only for adolescents with full in-school education. HRQoL (χ22 = 14.72, P < .001), mood (χ21 = 6.03, P = .01), and peer support (χ21 = 13.69, P < .001) worsened in adolescents with pandemic-induced remote-education. Interviewees reported that the reward system, self-regulation guidance, and increased health awareness had contributed to their behavior change. They also pointed to the importance of social factors, quality of technology and autonomy for mHealth effectiveness.

CONCLUSIONS

#LIFEGOALS showed mixed results on health behaviors and mental health. The findings highlight the role of contextual factors for mHealth promotion in adolescence, and provide suggestions to optimize support by a chatbot and narrative episodes.

TRIAL REGISTRATION

ClinicalTrials.gov [NCT04719858], registered on 22/01/2021.

The gamification and development of a chatbot to promote oral self-care by adopting behavior change wheel for Taiwanese children

Background

Oral health is closely related to general health and quality of life. School-aged children are at a critical stage for developing their self-care ability in oral health. Digital interventions can encourage and facilitate oral self-care in children.

Objective

This study aims to present the development of an educational chatbot for school-aged children to address their oral self-care and evaluate its usability.

Methods

The development and evaluation of the chatbot for oral self-care consisted of four stages: target behavior analysis, intervention design, system development, and the chatbot evaluation. The target behavior analysis identified barriers to children's engagement in oral self-care based on dentists' clinical observations; hence, the requirements for achieving the desired behavior were categorized according to the capability-opportunity-motivation behavior model. Interventional functions were created following the behavior change wheel. A menu-driven chatbot was created and evaluated for usability as well as likeability.

Results

The barriers and requirements for achieving good behavior in school-aged children's oral self-care were identified by the dental professionals. Intervention strategy incorporated specific functions enriched with gamification features to support school-aged children in developing their abilities for engaging in oral self-care. The intervention functions consist of capability establishment, motivation enhancement, and opportunity creation, which were designed to support children in their oral self-care practices. The designed chatbot was piloted with a convenient sample of 30 school-aged children and their accompanying parents at the pediatric dental clinic. The results indicated good usability, with a mean usability score of 79.91, and high likeability with a mean score of 4.32 out of 5 for the designed chatbot.

Conclusions

The educational chatbot incorporated a combination of clinical dentistry practice and guidelines, aiming to promote oral self-care behavior in school-aged children. The designed chatbot achieved high scores for its usability and user likability.

Consensus on Recommended Functions of a Smart Home System to Improve Self-Management Behaviors in People With Heart Failure: A Modified Delphi Approach

Background

Smart home systems could enhance clinical and self-management of chronic heart failure by supporting health monitoring and remote support, but evidence to guide the design of smart home system functionalities is lacking.

Objective

To identify consensus-based recommendations for functions of a smart home system that could augment clinical and self-management for people living with chronic heart failure in the community.

Methods

Healthcare professionals caring for people living with chronic heart failure participated in a two-round modified Delphi survey and a consensus workshop. Thirty survey items spanning eight chronic health failure categories were derived from international guidelines for the management of heart failure. In survey Round 1, participants rated the importance of all items using a 9-point Liket scale and suggested new functions to support people with chronic heart failure in their homes using a smart home system. The Likert scale scores ranged from 0 (not important) to 9 (very important) and scores were categorized into three groups: 1–3 = not important, 4–6 = important, and 7–9 = very important. Consensus agreement was defined a priori as ≥70% of respondents rating a score of ≥7 and ≤ 15% rating a score ≤ 3. In survey Round 2, panel members re-rated items where consensus was not reached, and rated the new items proposed in earlier round. Panel members were invited to an online consensus workshop to discuss items that had not reached consensus after Round 2 and agree on a set of recommendations for a smart home system.

Results

In Round 1, 15 experts agreed 24/30 items were “very important”, and suggested six new items. In Round 2, experts agreed 2/6 original items and 6/6 new items were “very important”. During the consensus workshop, experts endorsed 2/4 remaining items. Finally, the expert panel recommended 34 items as “very important” for a smart home system including, healthy eating, body weight and fluid intake, physical activity and sedentary behavior, heart failure symptoms, tobacco cessation and alcohol reduction, medication adherence, physiological monitoring, interaction with healthcare professionals, and mental health among others.

Conclusion

A panel of healthcare professional experts recommended 34-item core functions in smart home systems designed to support people with chronic heart failure for self-management and clinical support. Results of this study will help researchers to co-design and protyping solutions with consumers and healthcare providers to achieve these core functions to improve self-management and clinical outcomes in people with chronic heart failure.

Personalized Digital Health Communications to Increase COVID-19 Vaccination in Underserved Populations: A Double Diamond Approach to Behavioral Design

The COVID-19 pandemic exacerbated pre-existing health disparities. People of historically underserved communities, including racial and ethnic minority groups and people with lower incomes and educational attainments, experienced disproportionate premature mortality, access to healthcare, and vaccination acceptance and adoption. At the same time, the pandemic increased reliance on digital devices, offering a unique opportunity to leverage digital communication channels to address health inequities, particularly related to COVID-19 vaccination. We offer a real-world, systematic approach to designing personalized behavior change email and text messaging interventions that address individual barriers with evidence-based behavioral science inclusive of underserved populations. Integrating design processes such as the Double Diamond model with evidence-based behavioral science intervention development offers a unique opportunity to create equitable interventions. Further, leveraging behavior change artificial intelligence (AI) capabilities allows for both personalizing and automating that personalization to address barriers to COVID-19 vaccination at scale. The result is an intervention whose broad component library meets the needs of a diverse population and whose technology can deliver the right components for each individual.

Parents' Experience With a Mobile Health Intervention to Influence Human Papillomavirus Vaccination Decision Making: Mixed Methods Study

BACKGROUND

Human papillomavirus (HPV)-attributed cancers are preventable, yet HPV vaccination rates severely lag behind other adolescent vaccinations. HPVcancerFree (HPVCF) is a mobile health (mHealth) intervention developed to influence parental HPV vaccination decision making by raising awareness of HPV, reducing HPV vaccination barriers, and enabling HPV vaccination scheduling and reminders through a smartphone app. Evaluating the user experience of mHealth interventions is a vital component in assessing their quality and success but tends to be underreported in mHealth intervention evaluation.

OBJECTIVE

We aimed to evaluate the user experience of HPVCF, an HPV cancer prevention app designed for a pediatric clinic network, using mixed methods data collected from log files, survey measures, and qualitative feedback.

METHODS

Study data were evaluated from parents in a large US pediatric clinic network using HPVCF in the treatment study condition of a group randomized controlled trial. Log data captured HPVCF retention and use. Postintervention rating scales and items assessed HPVCF utility, usefulness, understandability, appeal, credibility, and perceived impact. Overall quality was evaluated using the user version of the Mobile Application Rating Scale (uMars). Open-ended responses assessed parent recommendations for HPVCF enhancement.

RESULTS

The 98 parents were mainly female (n=94, 96%), 41 (5.67) years of age, college educated (n=55, 56%), and White and non-Hispanic (n=55, 56%) and had private health insurance for their children (n=75, 77%). Parents used HPVCF 197 times, with the average visit duration approximating 3.5 minutes. The uMARS app quality score was positively skewed (4.2/5.0). Mean ratings were highest for information (4.46 [SD 0.53]) and lowest for engagement (3.74 [SD 0.69]). In addition, of 95 parents, 45 (47%) rated HPVCF as helpful in HPV vaccination decision making and 16 (17%) attributed HPV vaccine initiation to HPVCF. Parents reported that HPVCF increased their awareness (84/95, 88%), knowledge (84/95, 88%), and HPV vaccination intentions (64/95, 67%). Most of the 98 parents rated the 4 HPVCF components as useful (72-92 [73%-94%]). Parents also agreed that HPVCF is clear (86/95, 91%), accurate (86/95, 91%), and more helpful than other HPV vaccine information they had received (89/95, 94%) and that they would recommend it to others (81/95, 85%). In addition, parents suggested ways to increase awareness and engagement with the app, along with opportunities to enhance the content and functionality.

CONCLUSIONS

HPVCF was well received by parents and performed well on indicators of quality, usefulness, utility, credibility, and perceived impact. This study contributes a multimethod and multimeasure evaluation to the growing body of literature focused on assessing the user experience of patient-focused technology-mediated applications for HPV education.

A mobile app to promote alcohol and drug SBIRT skill translation among multi-disciplinary health care trainees: Results of a randomized controlled trial

BACKGROUND

Adherence to clinical practice guidelines for alcohol and drug screening, brief intervention, and referral to treatment (SBIRT) is often inadequate. Mobile apps developed as clinical translation tools could improve the delivery of high fidelity SBIRT.

Methods: This study tested the effectiveness of an SBIRT mobile app conceptually aligned with the Theory of Planned Behavior (TPB) to support SBIRT delivery by health care trainees (nursing, social work, internal medicine, psychiatry, and psychology) working in clinical settings (N = 101). Bivariate analyses examined the rate of SBIRT delivery between trainees assigned to the experimental (app) and control (no app) study conditions; as well as the relationship between TPB-based constructs, intention to deliver SBIRT, and screening rates.

Results: No significant differences were identified between the study conditions in SBIRT delivery. Significant correlations were found between intent to screen and TPB variables including attitudes/behavioral beliefs concerning substance use treatment (r = .49, p = .01); confidence in clinical skills (r = .36, p = .01); subjective norms (r = .54, p = .01) and perceived behavioral control over appointment time constraints (r = .42, p = .01). Also significant were correlations between percent of patients screened and confidence (r = .24, p = .05); subjective norms (r = .22, p = .05) and perceived behavioral control (r = .28, p = .01).

Conclusions: The negative results of the study condition comparisons indicate the need for further investigation of strategies to optimize mobile app utilization, engagement, and effectiveness as a clinical translation tool. Findings of significant correlations between substance use screening rates and both norms and confidence support the potential value of the TPB model in explaining behavior of health care learners in SBIRT delivery.

Informing Content and Feature Design of a Parent-Focused Human Papillomavirus Vaccination Digital Behavior Change Intervention: Synchronous Text-Based Focus Group Study

Background

Human papillomavirus (HPV) is a common and preventable sexually transmitted infection; however, vaccination rates in the United States among the target age group, which is 11-12 years, are lower than national goals. Interventions that address the barriers to and facilitators of vaccination are important for improving HPV vaccination rates. Web-based, text-based focus groups are becoming a promising method that may be well suited for conducting formative research to inform the design of digital behavior change intervention (DBCI) content and features that address HPV vaccination decision-making.

Objective

This study aims to explore parental HPV vaccination decision-making processes using a web-based, text-based focus group protocol to inform content and feature recommendations for an HPV prevention DBCI.

Methods

We conducted 4 web-based, text-based synchronous focus groups via Skype with the parents of patients aged 11-13 years within a large urban US pediatric clinic network.

Results

The 22 parents were mostly female, White, non-Hispanic college graduates, and they mostly had private health insurance for their children. Approximately half (14/25, 56%) of the parents' 11-13 year old children had initiated HPV vaccination. Most parents had experience using Skype (19/22, 86%). Approximately half (8/17, 47%) of parents expressed no preference for the focus group format, whereas 47% (8/17) requested a text-only chat format and 6% (1/17) requested an audiovisual format. The three main themes from the qualitative data were barriers to HPV vaccination, facilitators of HPV vaccination, and suggestions for improving the HPV vaccination clinic experience. A total of 11 intervention content and feature recommendations emerged from the themes, including addressing HPV knowledge barriers using trusted sources, designing for a family audience, focusing on the framing of messages, reporting reputable HPV research in a comprehensible format, and expanding the clinic visit experience.

Conclusions

Synchronous text-based focus groups are feasible for conducting formative research on HPV vaccination decision-making. Among well-educated and well-resourced parents, there are barriers such as misinformation and facilitators such as pediatrician recommendations that influence HPV vaccination decision-making. Parents want to conduct their own HPV research as well as receive relevant HPV vaccination advice from their child’s pediatrician. In addition, parents want an enhanced clinic visit experience that lets them access and connect to tailored information before and after clinic visits. The results gathered provide guidance for content and features that may inform a more responsive DBCI to address HPV vaccination decision-making among parents.

Black American and Latinx Parent/Caregiver Participation in Digital Health Obesity Interventions for Children: A Systematic Review

Parents/caregivers are consistently described as integral targets given their influential role in supporting and managing behaviors such as diet and physical activity. Identifying effective obesity prevention interventions to enhance and sustain parent participation is needed. Digital obesity prevention interventions are a promising strategy to improve parent/caregiver participation. Digital health interventions demonstrate acceptable participation and retention among parents/caregivers. However, our understanding of digital obesity prevention interventions targeting Black American and Latinx parents/caregivers is limited. This systematic review aims to identify Black American and Latinx parents'/caregivers' level of participation in digital obesity prevention and treatment interventions and determine the relationship between parent/caregiver participation and behavioral and weight status outcomes. This review adheres to PRISMA guidelines and is registered in PROSPERO. Eligibility criteria include: intervention delivered by digital technology, targeted Black American and Latinx parents/caregivers of young children (2-12 years), reported parent/caregiver participation outcomes, targeted diet or physical activity behaviors, and randomized controlled trial study design. Searches were conducted in September 2020 in ERIC, PsychInfo, PubMed, and Web of Science. Initial searches returned 499 results. Four reviewers screened records against eligibility criteria and 12 studies met inclusion criteria. Across all studies, parent/caregiver participation ranged from low to high. Only half of the included studies reported significant improvements in behavioral or weight status outcomes for parents/caregivers and/or children. Of these studies, three reported high parental/caregiver participation rates, and three reported high satisfaction rates. These findings suggest that participation and satisfaction may impact behavior change and weight status. The small number of studies indicates that additional research is needed to determine whether engagement or other factors predict responsiveness to the digital health intervention. Our results lay the groundwork for developing and testing future digital health interventions with the explicit goal of parental/caregiver participation and considers the need to expand our digital health intervention research methodologies to address obesity inequities among diverse families better.

Behavior science in the evolving world of digital health: considerations on anticipated opportunities and challenges

Digital health promises to increase intervention reach and effectiveness for a range of behavioral health outcomes. Behavioral scientists have a unique opportunity to infuse their expertise in all phases of a digital health intervention, from design to implementation. The aim of this study was to assess behavioral scientists' interests and needs with respect to digital health endeavors, as well as gather expert insight into the role of behavioral science in the evolution of digital health. The study used a two-phased approach: (a) a survey of behavioral scientists' current needs and interests with respect to digital health endeavors (n = 346); (b) a series of interviews with digital health stakeholders for their expert insight on the evolution of the health field (n = 15). In terms of current needs and interests, the large majority of surveyed behavioral scientists (77%) already participate in digital health projects, and from those who have not done so yet, the majority (65%) reported intending to do so in the future. In terms of the expected evolution of the digital health field, interviewed stakeholders anticipated a number of changes, from overall landscape changes through evolving models of reimbursement to more significant oversight and regulations. These findings provide a timely insight into behavioral scientists' current needs, barriers, and attitudes toward the use of technology in health care and public health. Results might also highlight the areas where behavioral scientists can leverage their expertise to both enhance digital health's potential to improve health, as well as to prevent the potential unintended consequences that can emerge from scaling the use of technology in health care.

"It depends:" a qualitative study on digital health academic-industry collaboration

BACKGROUND

Academic-industry collaborations (AICs) are endorsed to alleviate challenges in digital health, but partnership experiences remain understudied. The qualitative study's objective investigated collaboration experiences between academic institutions and digital health companies.

METHODS

A phenomenology methodology captured experiences of AICs, eliciting perspectives from academic researchers and industry affiliates (e.g., leadership, company investigators). Semi-structured interviews probed eligible collaborators about their experiences in digital health. Analysts coded and organized data into significant statements reaching thematic saturation.

RESULTS

Participants (N=20) were interviewed from 6 academic institutions and 14 unique industry partners. Seven themes emerged: (I) Collaboration evolves with time, relationships, funding, and evidence; (II) Collaboration demands strong relationships and interpersonal dynamics; (III) Operational processes vary across collaborations; (IV) Collaboration climate and context matters; (V) Shared expectations lead to a better understanding of success; (VI) Overcoming challenges with recommendations; (VII) Collaboration may help navigate the global pandemic.

CONCLUSIONS

Digital health academic industry collaboration demands strong relationships, requiring flexible mechanisms of collaboration and cultural fit. Diverse models of collaboration exist and remain dependent on contextual factors. While no collaboration conquers all challenges in digital health, AICs may serve as a facilitator for improved digital health products, thus advancing science, promoting public health, and benefiting the economy.

Development and Fidelity Testing of the Test@Work Digital Toolkit for Employers on Workplace Health Checks and Opt-In HIV Testing

Background: In the UK, few employers offer general health checks for employees, and opt-in HIV testing is rarely included. There is a need to provide evidence-based guidance and support for employers around health checks and HIV testing in the workplace. An Agile approach was used to develop and evaluate a digital toolkit to facilitate employers’ understanding about workplace health screening. Methods: The Test@Work toolkit development included an online survey (STAGE 1: n = 201), stakeholder consultation (STAGE 2: n = 19), expert peer review (STAGE 3: n = 24), and pilot testing (STAGE 4: n = 20). The toolkit includes employer guidance on workplace health promotion, workplace health screening, and confidential opt-in HIV testing with signposting to resources. Pilot testing included assessment of fidelity (delivery and engagement) and implementation qualities (attitudes, resources, practicality, acceptability, usability and cost). Results: STAGE 1: The vast majority of respondents would consider offering general health checks in the workplace that included confidential opt-in HIV testing, and this view was broadly comparable across organisation types (n = 201; public: 87.8%; private: 89.7%; third: 87.1%). STAGES 2 and 3: Stakeholders highlighted essential content considerations: (1) inclusion of the business case for workplace health initiatives, (2) clear pathways to employer responsibilities, and (3) presenting HIV-related information alongside other areas of health. With regards presentation, stakeholders proposed that the toolkit should be concise, with clear signposting and be hosted on a trusted portal. STAGE 4: Employers were satisfied with the toolkit content, usability and utility. The toolkit had high fidelity with regards to delivery and employer engagement. Assessment of implementation qualities showed high usability and practicality, with low perceived burden for completion and acceptable cost implications. Very few resource challenges were reported, and the toolkit was considered to be appropriate for any type of organisation, irrespective of size or resources. Conclusions: Employers perceived the Test@Work toolkit to be useful, meaningful and appropriate for their needs. This digital resource could be used to support employers to engage with health screening and opt-in HIV testing within the context of workplace health promotion.