A self-report measure of engagement with digital behavior change interventions (DBCIs): development and psychometric evaluation of the "DBCI Engagement Scale"

Testing behaviour change with an artificial intelligence chatbot in a randomized controlled study

Chatbots can effect large-scale behaviour change because they are accessible through social media, flexible, scalable, and gather data automatically. Yet research on the feasibility and effectiveness of chatbot-administered behaviour change interventions is sparse. The effectiveness of established behaviour change interventions when implemented in chatbots is not guaranteed, given the unique human-machine interaction dynamics. We pilot-tested chatbot-based behaviour change through information provision and embedded animations. We evaluated whether the chatbot could increase understanding and intentions to adopt protective behaviours during the pandemic. Fifty-nine culturally and linguistically diverse participants received a compassion intervention, an exponential growth intervention, or no intervention. We measured participants' COVID-19 testing intentions and measured their staying-home attitudes before and after their chatbot interaction. We found reduced uncertainty about protective behaviours. The exponential growth intervention increased participants' testing intentions. This study provides preliminary evidence that chatbots can spark behaviour change, with applications in diverse and underrepresented groups.

The feasibility and acceptability of an app-based intervention with brief behavioural support (APPROACH) to promote brisk walking in people diagnosed with breast, prostate and colorectal cancer in the UK

INTRODUCTION

Increased moderate to vigorous physical activity (MVPA) can improve clinical and psychosocial outcomes for people living with and beyond cancer (LWBC). This study aimed to assess the feasibility and acceptability of trial procedures in a pilot randomised controlled trial (RCT) of a theory-driven app-based intervention with behavioural support focused on promoting brisk walking (a form of MVPA) in people LWBC (APPROACH).

METHODS

Participants diagnosed with breast, prostate or colorectal cancer were recruited from a single UK hospital site. Assessments at baseline and 3 months included online questionnaires, device-measured brisk walking (activPAL accelerometer) and self-reported weight and height. Participants were randomised to intervention or control (care as usual). The intervention comprised a non-cancer-specific app to promote brisk walking (National Health Service 'Active 10') augmented with print information about habit formation, a walking planner and two behavioural support telephone calls. Feasibility and acceptability of trial procedures were explored. Initial estimates for physical activity informed a power calculation for a phase III RCT. A preliminary health economics analysis was conducted.

RESULTS

Of those medically eligible, 369/577 (64%) were willing to answer further eligibility questions and 90/148 (61%) of those eligible were enrolled. Feasibility outcomes, including retention (97%), assessment completion rates (>86%) and app download rates in the intervention group (96%), suggest that the trial procedures are acceptable and that the intervention is feasible. The phase III RCT will require 472 participants to be randomised. As expected, the preliminary health economic analyses indicate a high level of uncertainty around the cost-effectiveness of the intervention.

CONCLUSIONS

This pilot study demonstrates that a large trial of the brisk walking intervention with behavioural support is both feasible and acceptable to people LWBC. The results support progression onto a confirmatory phase III trial to determine the efficacy and cost-effectiveness of the intervention.

Time-varying model of engagement with digital self reporting: Evidence from smoking cessation longitudinal studies

Objective

Insufficient engagement is a critical barrier impacting the utility of digital interventions and mobile health assessments. As a result, engagement itself is increasingly becoming a target of studies and interventions. The purpose of this study is to investigate the dynamics of engagement in mobile health data collection by exploring whether, how, and why response to digital self-report prompts change over time in smoking cessation studies.

Method

Data from two ecological momentary assessment (EMA) studies of smoking cessation among diverse smokers attempting to quit (N = 573) with a total of 65,974 digital self-report prompts. We operationalize engagement with self-reporting in term of prompts delivered and prompt response to capture both broad and more granular engagement in self-reporting, respectively. The data were analyzed to describe trends in prompt delivered and prompt response over time. Time-varying effect modeling (TVEM) was employed to investigate the time-varying effects of response to previous prompt and the average response rate on the likelihood of current prompt response.

Results

Although prompt response rates were relatively stable over days in both studies, the proportion of participants with prompts delivered declined steadily over time in one of the studies, indicating that over time, fewer participants charged the device and kept it turned on (necessary to receive at least one prompt per day). Among those who did receive prompts, response rates were relatively stable. In both studies, there is a significant, positive and stable relationship between response to previous prompt and the likelihood of response to current prompt throughout all days of the study. The relationship between the average response rate prior to current prompt and the likelihood of responding to the current prompt was also positive, and increasing with time.

Conclusion

Our study highlights the importance of integrating various indicators to measure engagement in digital self-reporting. Both average response rate and response to previous prompt were highly predictive of response to the next prompt across days in the study. Dynamic patterns of engagement in digital self-reporting can inform the design of new strategies to promote and optimize engagement in digital interventions and mobile health studies.

Measuring Engagement with Mental Health and Behavior Change Interventions: an Integrative Review of Methods and Instruments

BACKGROUND

Engagement is a complex construct consisting of behavioral, cognitive, and affective dimensions, making engagement a difficult construct to measure. This integrative review aims to (1) present a multidisciplinary overview of measurement methods that are currently used to measure engagement with adult mental health and behavior change interventions, delivered in-person, blended, or digitally, and (2) provide a set of recommendations and considerations for researchers wishing to study engagement.

METHODS

We used an integrative approach and identified original studies and reviews on engagement with mental health or behavior change interventions that were delivered in-person, digitally, or blended.

RESULTS

Forty articles were analyzed in this review. Common methods to assess engagement were through objective usage data, questionnaire-based data, and qualitative data, with objective usage data being used most frequently. Based on the synthesis of engagement measures, we advise researchers to (1) predefine the operationalization of engagement for their specific research context, (2) measure behavioral, cognitive, and affective dimensions of engagement in all cases, and (3) measure engagement over time.

CONCLUSIONS

Current literature shows a bias towards behavioral measures of engagement in research, as most studies measured engagement exclusively through objective usage data, without including cognitive and affective measures of engagement. We hope that our recommendations will help to reduce this bias and to steer engagement research towards an integrated approach.

Workplace gaslighting: Conceptualization, development, and validation of a scale

Introduction

Gaslighting is a form of abuse that has transgressed the realms of romantic relationships to the relationships at work. Despite the growing literature on abuse at work, the conceptualization and measurement of gaslighting at work have received scarce attention. The study aimed to address this existing lacuna in the literature by conceptualizing and developing a measure of gaslighting at work.

Methods

By drawing upon and integrating existing works of literature on harmful leader behaviors, workplace abuse, and workplace mistreatment, the authors have conceptualized the concept of gaslighting in a new context, i.e., work settings, and delineated its dimensions and conceptual boundaries. Among three different samples (total N = 679) of employees, the study developed a new 12-item measure of gaslighting in work relationships, the Gaslighting at Work Questionnaire (GWQ). The study further tested the psychometric properties of GWQ, namely, internal consistency, face, and construct validity of GWQ. Additionally, a time-lagged study was used to validate the scale within a nomological net of conceptual relationships.

Results

Exploratory and confirmatory factor analysis supported a two-dimensional structure of gaslighting at work (trivialization and affliction). The psychometric properties of GWQ were established. Finally, using a time-lagged study, the scale was validated within a nomological net of conceptual relationships by showing the relationship of gaslighting at work with role conflict and job satisfaction.

Discussion

The GWQ scale offers new opportunities to understand and measure gaslighting behaviors of a supervisor toward their subordinates in the work context. It adds to the existing literature on harmful leader behaviors, workplace abuse, and mistreatment by highlighting the importance of identifying and measuring gaslighting at work.

Design Features Associated With Engagement in Mobile Health Physical Activity Interventions Among Youth: Systematic Review of Qualitative and Quantitative Studies

BACKGROUND

Globally, 81% of youth do not meet the physical activity (PA) guidelines. Youth of families with a low socioeconomic position are less likely to meet the recommended PA guidelines. Mobile health (mHealth) interventions are preferred by youth over traditional in-person approaches and are in line with their media preferences. Despite the promise of mHealth interventions in promoting PA, a common challenge is to engage users in the long term or effectively. Earlier reviews highlighted the association of different design features (eg, notifications and rewards) with engagement among adults. However, little is known about which design features are important for increasing engagement among youth.

OBJECTIVE

To inform the design process of future mHealth tools, it is important to investigate the design features that can yield effective user engagement. This systematic review aimed to identify which design features are associated with engagement in mHealth PA interventions among youth who were aged between 4 and 18 years.

METHODS

A systematic search was conducted in EBSCOhost (MEDLINE, APA PsycINFO, and Psychology & Behavioral Sciences Collection) and Scopus. Qualitative and quantitative studies were included if they documented design features associated with engagement. Design features and related behavior change techniques and engagement measures were extracted. Study quality was assessed according to the Mixed Method Assessment Tool, and one-third of all screening and data extraction were double coded by a second reviewer.

RESULTS

Studies (n=21) showed that various features were associated with engagement, such as a clear interface, rewards, multiplayer game mode, social interaction, variety of challenges with personalized difficulty level, self-monitoring, and variety of customization options among others, including self-set goals, personalized feedback, progress, and a narrative. In contrast, various features need to be carefully considered while designing mHealth PA interventions, such as sounds, competition, instructions, notifications, virtual maps, or self-monitoring, facilitated by manual input. In addition, technical functionality can be considered as a prerequisite for engagement. Research addressing youth from low socioeconomic position families is very limited with regard to engagement in mHealth apps.

CONCLUSIONS

Mismatches between different design features in terms of target group, study design, and content translation from behavior change techniques to design features are highlighted and set up in a design guideline and future research agenda.

TRIAL REGISTRATION

PROSPERO CRD42021254989; https://tinyurl.com/5n6ppz24.

Promoting Hand Hygiene During the COVID-19 Pandemic: Parallel Randomized Trial for the Optimization of the Soapp App

BACKGROUND

Hand hygiene is an effective behavior for preventing the spread of the respiratory disease COVID-19 and was included in public health guidelines worldwide. Behavior change interventions addressing hand hygiene have the potential to support the adherence to public health recommendations and, thereby, prevent the spread of COVID-19. However, randomized trials are largely absent during a pandemic; therefore, there is little knowledge about the most effective strategies to promote hand hygiene during an ongoing pandemic. This study addresses this gap by presenting the results of the optimization phase of a Multiphase Optimization Strategy of Soapp, a smartphone app for promoting hand hygiene in the context of the COVID-19 pandemic.

OBJECTIVE

This study aimed to identify the most effective combination and sequence of 3 theory- and evidence-based intervention modules (habit, motivation, and social norms) for promoting hand hygiene. To this end, 9 versions of Soapp were developed (conditions), and 2 optimization criteria were defined: the condition with the largest increase in hand hygiene at follow-up and condition with the highest engagement, usability, and satisfaction based on quantitative and qualitative analyses.

METHODS

This study was a parallel randomized trial with 9 intervention conditions defined by the combination of 2 intervention modules and their sequence. The trial was conducted from March to August 2021 with interested participants from the Swiss general population (N=232; randomized). Randomization was performed using Qualtrics (Qualtrics International Inc), and blinding was ensured. The duration of the intervention was 34 days. The primary outcome was self-reported hand hygiene at follow-up, which was assessed using an electronic diary. The secondary outcomes were user engagement, usability, and satisfaction assessed at follow-up. Nine participants were further invited to participate in semistructured exit interviews. A set of ANOVAs was performed to test the main hypotheses, whereas a thematic analysis was performed to analyze the qualitative data.

RESULTS

The results showed a significant increase in hand hygiene over time across all conditions. There was no interaction effect between time and intervention condition. Similarly, no between-group differences in engagement, usability, and satisfaction emerged. Seven themes (eg, "variety and timeliness of the task load" and "social interaction") were found in the thematic analysis.

CONCLUSIONS

The effectiveness of Soapp in promoting hand hygiene laid the foundation for the next evaluation phase of the app. More generally, the study supported the value of digital interventions in pandemic contexts. The findings showed no differential effect of intervention conditions involving different combinations and sequences of the habit, motivation, and social norms modules on hand hygiene, engagement, usability, and satisfaction. In the absence of quantitative differences, we relied on the results from the thematic analysis to select the best version of Soapp for the evaluation phase.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04830761; https://clinicaltrials.gov/ct2/show/NCT04830761.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2021-055971.

Observational cross-sectional study of the association of poor broadband provision with demographic and health outcomes: the Wolverhampton Digital ENablement (WODEN) programme

OBJECTIVES

The association between impaired digital provision, access and health outcomes has not been systematically studied. The Wolverhampton Digital ENablement programme (WODEN) is a multiagency collaborative approach to determine and address digital factors that may impact on health and social care in a single deprived multiethnic health economy. The objective of this study is to determine the association between measurable broadband provision and demographic and health outcomes in a defined population.

DESIGN

An observational cross-sectional whole local population-level study with cohorts defined according to broadband provision.

SETTING/PARTICIPANTS

Data for all residents of the City of Wolverhampton, totalling 269 785 residents.

PRIMARY OUTCOMES

Poor broadband provision is associated with variation in demographics and with increased comorbidity and urgent care needs.

RESULTS

Broadband provision was measured using the Broadband Infrastructure Index (BII) in 158 City localities housing a total of 269 785 residents. Lower broadband provision as determined by BII was associated with younger age (p<0.001), white ethnic status (p<0.001), lesser deprivation as measured by Index of Multiple Deprivation (p<0.001), a higher number of health comorbidities (p<0.001) and more non-elective urgent events over 12 months (p<0.001).

CONCLUSION

Local municipal and health authorities are advised to consider the variations in broadband provision within their locality and determine equal distribution both on a geographical basis but also against demographic, health and social data to determine equitable distribution as a platform for equitable access to digital resources for their residents.

Identifying essential factors that influence user engagement with digital mental health tools in clinical care settings: Protocol for a Delphi study

Introduction

Improving effective user engagement with digital mental health tools has become a priority in enabling the value of digital health. With increased interest from the mental health community in embedding digital health tools as part of care delivery, there is a need to examine and identify the essential factors in influencing user engagement with digital mental health tools in clinical care. The current study will use a Delphi approach to gain consensus from individuals with relevant experience and expertise (e.g. patients, clinicians and healthcare administrators) on factors that influence user engagement (i.e. an essential factor).

Methods

Participants will be invited to complete up to four rounds of online surveys. The first round of the Delphi study comprises of reviewing existing factors identified in literature and commenting on whether any factors they believe are important are missing from the list. Subsequent rounds will involve asking participants to rate the perceived impact of each factor in influencing user engagement with digital mental health tools in clinical care contexts. This work is expected to consolidate the perspectives from relevant stakeholders and the academic literature to identify a core set of factors considered essential in influencing user engagement with digital mental health tools in clinical care contexts.

Designing a Measurement Scale for Spiritual Health of the Elderly in Tehran/Iran (2019)

Background: Spiritual health is one of the important dimensions of health that causes the coherence and harmony of other dimensions of health in human beings. Due to the increasing number of elderly, considering the status of their health is important. Objectives: The present study was an attempt to design and evaluate a measurement instrument for spiritual health in the Iranian elderly, which is appropriate for the Iranian society. Methods: This mixed methods study used a sequential exploratory strategy. In the first phase, spiritual health items were extracted based on a review of the previous studies and interviews with experts and the elderly using direct content analysis. In the second phase, the standardized questionnaire was assessed by performing validity and reliability tests on 400 elderly residents of Tehran. The participants were selected based on the purposive sampling method from the elderly referring to nursing homes. To analyze the collected data, qualitative content analysis was employed. In the first phase, 45 items of the questionnaire were extracted based on the interviews. After quantitatively determining the face and content validity, six items were removed, and the questionnaire items were reduced to 39 items. Results: Exploratory factor analysis on this questionnaire identified five factors that explained a total of 52.2% of the total variance of the test. The Cronbach's alpha coefficient obtained confirmed the high internal consistency of the questionnaire (0.925). Also, a high correlation was reported between the test and retest with a 10-day interval (r = 0.997). In addition, a high and significant correlation was reported in the simultaneous implementation of the designed instrument with Paloutzin and Ellison’s spiritual health instrument (r = 0.76). Conclusions: In general, based on the present study's findings, the designed questionnaire has an acceptable level of validity and reliability and is usable for the elderly.

Measures of Engagement With mHealth Interventions in Patients With Heart Failure: Scoping Review

Background

Despite the potential of mobile health (mHealth) interventions to facilitate the early detection of signs of heart failure (HF) decompensation and provide personalized management of symptoms, the outcomes of such interventions in patients with HF have been inconsistent. As engagement with mHealth is required for interventions to be effective, poor patient engagement with mHealth interventions may be associated with mixed evidence. It is crucial to understand how engagement with mHealth interventions is measured in patients with HF, and the effects of engagement on HF outcomes.

Objective

In this review, we aimed to describe measures of patient engagement with mHealth interventions and the effects of engagement on HF outcomes.

Methods

We conducted a systematic literature search in 7 databases for relevant studies published in the English language from 2009 to September 2021 and reported the descriptive characteristics of the studies. We used content analysis to identify themes that described patient engagement with mHealth interventions in the qualitative studies included in the review.

Results

We synthesized 32 studies that operationalized engagement with mHealth interventions in 4771 patients with HF (3239/4771, 67.88%, male), ranging from a sample of 7 to 1571 (median 53.3) patients, followed for a median duration of 90 (IQR 45-180) days. Patient engagement with mHealth interventions was measured only quantitatively based on system usage data in 72% (23/32) of the studies, only qualitatively based on data from semistructured interviews and focus groups in 6% (2/32) of studies, and by a combination of both quantitative and qualitative data in 22% (7/32) of studies. System usage data were evaluated using 6 metrics of engagement: number of physiological parameters transmitted (19/30, 63% studies), number of HF questionnaires completed (2/30, 7% studies), number of log-ins (4/30, 13% studies), number of SMS text message responses (1/30, 3% studies), time spent (5/30, 17% studies), and the number of features accessed and screen viewed (4/30, 13% studies). There was a lack of consistency in how the system usage metrics were reported across studies. In total, 80% of the studies reported only descriptive characteristics of system usage data. The emotional, cognitive, and behavioral domains of patient engagement were identified through qualitative studies. Patient engagement levels ranged from 45% to 100% and decreased over time. The effects of engagement on HF knowledge, self-care, exercise adherence, and HF hospitalization were inconclusive.

Conclusions

The measures of patient engagement with mHealth interventions in patients with HF are underreported and lack consistency. The application of inferential analytical methods to engagement data is extremely limited. There is a need for a working group on mHealth that may consolidate the previous operational definitions of patient engagement into an optimal and standardized measure.

Measuring User Engagement with a Socially Connected, Gamified Health Promotion Mobile App

Participant engagement is an important consideration in mHealth interventions and there are no standardised measurements available to guide researchers. This paper describes the engagement index customised for the Milk Man app, a mobile app designed to engage fathers with breastfeeding and parenting information. Participants were recruited from maternity hospitals in Perth, Western Australia. An engagement index with scores ranging from 0 to 100 was calculated. Kaplan Meier survival analysis was used to determine difference in duration of exclusive breastfeeding, and Pearson's chi square analysis was conducted to investigate the association of engagement level with demographic characteristics and exclusive breastfeeding at 6 weeks. While overall, partners of participants who installed Milk Man were less likely to have ceased exclusive breastfeeding at any time point from birth to six weeks postpartum, this result was modest and of borderline significance (log rank test p = 0.052; Breslow p = 0.046; Tarone-Ware p = 0.049). The mean engagement score was 29.7% (range 1-80%), median 27.6%. Engagement level had no impact on duration of exclusive breastfeeding and demographic factors were not associated with engagement level. This research demonstrates a range of metrics that can be used to quantify participant engagement. However, more research is needed to identify ways of measuring effective engagement.

Assessing Engagement With Patient-Generated Health Data Recording and Its Impact on Health Behavior Changes in Multicomponent Interventions: Supplementary Analysis

Background

The use and sharing of patient-generated health data (PGHD) by clinicians or researchers is expected to enhance the remote monitoring of specific behaviors that affect patient health. In addition, PGHD use could support patients’ decision-making on preventive care management, resulting in reduced medical expenses. However, sufficient evidence on the use and sharing of PGHD is lacking, and the impact of PGHD recording on patients’ health behavior changes remains unclear.

Objective

This study aimed to assess patients’ engagement with PGHD recording and to examine the impact of PGHD recording on their health behavior changes.

Methods

This supplementary analysis used the data of 47 postpartum women who had been assigned to the intervention group of our previous study for managing urinary incontinence. To assess the patients’ engagement with PGHD recording during the intervention period (8 weeks), the fluctuation in the number of patients who record their PGHD (ie, PGHD recorders) was evaluated by an approximate curve. In addition, to assess adherence to the pelvic floor muscle training (PFMT), the weekly mean number of pelvic floor muscle contractions performed per day among 17 PGHD recorders was examined by latent class growth modeling (LCGM).

Results

The fluctuation in the number of PGHD recorders was evaluated using the sigmoid curve formula (R²=0.91). During the first week of the intervention, the percentage of PGHD recorders was around 64% (30/47) and then decreased rapidly from the second to the third week. After the fourth week, the percentage of PGHD recorders was 36% (17/47), which remained constant until the end of the intervention. When analyzing the data of these 17 PGHD recorders, PFMT adherence was categorized into 3 classes by LCGM: high (7/17, 41%), moderate (3/17, 18%), and low (7/17, 41%).

Conclusions

The number of PGHD recorders declined over time in a sigmoid curve. A small number of users recorded PGHD continuously; therefore, patients’ engagement with PGHD recording was low. In addition, more than half of the PGHD recorders (moderate- and low-level classes combined: 10/17, 59%) had poor PFMT adherence. These results suggest that PGHD recording does not always promote health behavior changes.

Engagement, Predictors, and Outcomes of a Trauma Recovery Digital Mental Health Intervention: Longitudinal Study

BACKGROUND

Worldwide, exposure to potentially traumatic events is extremely common, and many individuals develop posttraumatic stress disorder (PTSD) along with other disorders. Unfortunately, considerable barriers to treatment exist. A promising approach to overcoming treatment barriers is a digital mental health intervention (DMHI). However, engagement with DMHIs is a concern, and theoretically based research in this area is sparse and often inconclusive.

OBJECTIVE

The focus of this study is on the complex issue of DMHI engagement. On the basis of the social cognitive theory framework, the conceptualization of engagement and a theoretically based model of predictors and outcomes were investigated using a DMHI for trauma recovery.

METHODS

A 6-week longitudinal study with a national sample of survivors of trauma was conducted to measure engagement, predictors of engagement, and mediational pathways to symptom reduction while using a trauma recovery DMHI (time 1: N=915; time 2: N=350; time 3: N=168; and time 4: N=101).

RESULTS

Confirmatory factor analysis of the engagement latent constructs of duration, frequency, interest, attention, and affect produced an acceptable model fit (χ²₂=8.3; P=.02; comparative fit index 0.973; root mean square error of approximation 0.059; 90% CI 0.022-0.103). Using the latent construct, the longitudinal theoretical model demonstrated adequate model fit (comparative fit index 0.929; root mean square error of approximation 0.052; 90% CI 0.040-0.064), indicating that engagement self-efficacy (β=.35; P<.001) and outcome expectations (β=.37; P<.001) were significant predictors of engagement (R²=39%). The overall indirect effect between engagement and PTSD symptom reduction was significant (β=-.065; P<.001; 90% CI -0.071 to -0.058). This relationship was serially mediated by both skill activation self-efficacy (β=.80; P<.001) and trauma coping self-efficacy (β=.40; P<.001), which predicted a reduction in PTSD symptoms (β=-.20; P=.02).

CONCLUSIONS

The results of this study may provide a solid foundation for formalizing the nascent science of engagement. Engagement conceptualization comprised general measures of attention, interest, affect, and use that could be applied to other applications. The longitudinal research model supported 2 theoretically based predictors of engagement: engagement self-efficacy and outcome expectancies. A total of 2 task-specific self-efficacies-skill activation and trauma coping-proved to be significant mediators between engagement and symptom reduction. Taken together, this model can be applied to other DMHIs to understand engagement, as well as predictors and mechanisms of action. Ultimately, this could help improve the design and development of engaging and effective trauma recovery DMHIs.

Mobile intervention to promote correct hand hygiene at key times to prevent COVID-19 in the Swiss adult general population: study protocol of a multiphase optimisation strategy

INTRODUCTION

Behaviour change is key to the public health measures that have been issued in many countries worldwide to contain COVID-19. Public health measures will only take preventive effect if people adhere to them. Interventions taking health psychology approaches may promote adherence to public health measures. However, evidence from randomised controlled behaviour change trials is scarce during an ongoing pandemic. We aim to use the example of hand washing with soap to optimise and test a digital, theory-based and evidence-based behaviour change intervention to prevent the spread of COVID-19.

METHODS AND ANALYSIS

This protocol describes the multiphase optimisation strategy for the preparation, optimisation and evaluation of a theory-based and evidence-based intervention delivered via app. The app aims to promote correct hand hygiene at key times in the adult general population. The study will be conducted in German-speaking Switzerland. The preparation phase has identified relevant behavioural determinants of hand hygiene during a pandemic from health behaviour theories and formative research with focus groups (n=8). The optimisation phase will identify the most effective and acceptable combination and sequence of three intervention modules in a parallel randomised trial (n=387) with analysis of variance (ANOVA) and regression analysis. Additionally, thematic analysis of qualitative interview data (n=15) will be used to gain insights on the feasibility, usability and satisfaction of the intervention. The evaluation phase will test the optimised intervention against an active control group in a randomised controlled trial (n=205), analysing pre-post differences and 6-month follow-up effects with ANOVA and regression analysis.

ETHICS AND DISSEMINATION

The trial was approved by the Cantonal Ethics Commission Bern of the Swiss Association of Research Ethics Committees (protocol ID: 2021-00164). Final results will be presented in peer-reviewed journals and at conferences.

TRIAL REGISTRATION NUMBER

NCT04830761.

Exploring and Characterizing Patient Multibehavior Engagement Trails and Patient Behavior Preference Patterns in Pathway-Based mHealth Hypertension Self-Management: Analysis of Use Data

Background

Hypertension is a long-term medical condition. Mobile health (mHealth) services can help out-of-hospital patients to self-manage. However, not all management is effective, possibly because the behavior mechanism and behavior preferences of patients with various characteristics in hypertension management were unclear.

Objective

The purpose of this study was to (1) explore patient multibehavior engagement trails in the pathway-based hypertension self-management, (2) discover patient behavior preference patterns, and (3) identify the characteristics of patients with different behavior preferences.

Methods

This study included 863 hypertensive patients who generated 295,855 use records in the mHealth app from December 28, 2016, to July 2, 2020. Markov chain was used to infer the patient multibehavior engagement trails, which contained the type, quantity, time spent, sequence, and transition probability value (TP value) of patient behavior. K-means algorithm was used to group patients by the normalized behavior preference features: the number of behavioral states that a patient performed in each trail. The pages in the app represented the behavior states. Chi-square tests, Z-test, analyses of variance, and Bonferroni multiple comparisons were conducted to characterize the patient behavior preference patterns.

Results

Markov chain analysis revealed 3 types of behavior transition (1-way transition, cycle transition, and self-transition) and 4 trails of patient multibehavior engagement. In perform task trail (PT-T), patients preferred to start self-management from the states of task blood pressure (BP), task drug, and task weight (TP value 0.29, 0.18, and 0.20, respectively), and spent more time on the task food state (35.87 s). Some patients entered the states of task BP and task drug (TP value 0.20, 0.25) from the reminder item state. In the result-oriented trail (RO-T), patients spent more energy on the ranking state (19.66 s) compared to the health report state (13.25 s). In the knowledge learning trail (KL-T), there was a high probability of cycle transition (TP value 0.47, 0.31) between the states of knowledge list and knowledge content. In the support acquisition trail (SA-T), there was a high probability of self-transition in the questionnaire (TP value 0.29) state. Cluster analysis discovered 3 patient behavior preference patterns: PT-T cluster, PT-T and KL-T cluster, and PT-T and SA-T cluster. There were statistically significant associations between the behavior preference pattern and gender, education level, and BP.

Conclusions

This study identified the dynamic, longitudinal, and multidimensional characteristics of patient behavior. Patients preferred to focus on BP, medications, and weight conditions and paid attention to BP and medications using reminders. The diet management and questionnaires were complicated and difficult to implement and record. Competitive methods such as ranking were more likely to attract patients to pay attention to their own self-management states. Female patients with lower education level and poorly controlled BP were more likely to be highly involved in hypertension health education.

Participatory Development and Pilot Testing of an Adolescent Health Promotion Chatbot

Background: The use of chatbots may increase engagement with digital behavior change interventions in youth by providing human-like interaction. Following a Person-Based Approach (PBA), integrating user preferences in digital tool development is crucial for engagement, whereas information on youth preferences for health chatbots is currently limited.

Objective: The aim of this study was to gain an in-depth understanding of adolescents' expectations and preferences for health chatbots and describe the systematic development of a health promotion chatbot.

Methods: Three studies in three different stages of PBA were conducted: (1) a qualitative focus group study (n = 36), (2) log data analysis during pretesting (n = 6), and (3) a mixed-method pilot testing (n = 73).

Results: Confidentiality, connection to youth culture, and preferences when referring to other sources were important aspects for youth in chatbots. Youth also wanted a chatbot to provide small talk and broader support (e.g., technical support with the tool) rather than specifically in relation to health behaviors. Despite the meticulous approach of PBA, user engagement with the developed chatbot was modest.

Conclusion: This study highlights that conducting formative research at different stages is an added value and that adolescents have different chatbot preferences than adults. Further improvement to build an engaging chatbot for youth may stem from using living databases.

Understanding Uptake of Digital Health Products: Methodology Tutorial for a Discrete Choice Experiment Using the Bayesian Efficient Design

Understanding the preferences of potential users of digital health products is beneficial for digital health policy and planning. Stated preference methods could help elicit individuals' preferences in the absence of observational data. A discrete choice experiment (DCE) is a commonly used stated preference method-a quantitative methodology that argues that individuals make trade-offs when engaging in a decision by choosing an alternative of a product or a service that offers the greatest utility, or benefit. This methodology is widely used in health economics in situations in which revealed preferences are difficult to collect but is much less used in the field of digital health. This paper outlines the stages involved in developing a DCE. As a case study, it uses the application of a DCE to reveal preferences in targeting the uptake of smoking cessation apps. It describes the establishment of attributes, the construction of choice tasks of 2 or more alternatives, and the development of the experimental design. This tutorial offers a guide for researchers with no prior knowledge of this research technique.

Patterns for Patient Engagement with the Hypertension Management and Effects of Electronic Health Care Provider Follow-up on These Patterns: Cluster Analysis

Background

Hypertension is a long-term medical condition. Electronic and mobile health care services can help patients to self-manage this condition. However, not all management is effective, possibly due to different levels of patient engagement (PE) with health care services. Health care provider follow-up is an intervention to promote PE and blood pressure (BP) control.

Objective

This study aimed to discover and characterize patterns of PE with a hypertension self-management app, investigate the effects of health care provider follow-up on PE, and identify the follow-up effects on BP in each PE pattern.

Methods

PE was represented as the number of days that a patient recorded self-measured BP per week. The study period was the first 4 weeks for a patient to engage in the hypertension management service. K-means algorithm was used to group patients by PE. There was compliance follow-up, regular follow-up, and abnormal follow-up in management. The follow-up effect was calculated by the change in PE (CPE) and the change in systolic blood pressure (CSBP, SBP) before and after each follow-up. Chi-square tests and z scores were used to ascertain the distribution of gender, age, education level, SBP, and the number of follow-ups in each cluster. The follow-up effect was identified by analysis of variances. Once a significant effect was detected, Bonferroni multiple comparisons were further conducted to identify the difference between 2 clusters.

Results

Patients were grouped into 4 clusters according to PE: (1) PE started low and dropped even lower (PELL), (2) PE started high and remained high (PEHH), (3) PE started high and dropped to low (PEHL), and (4) PE started low and rose to high (PELH). Significantly more patients over 60 years old were found in the PEHH cluster (P≤.05). Abnormal follow-up was significantly less frequent (P≤.05) in the PELL cluster. Compliance follow-up and regular follow-up can improve PE. In the clusters of PEHH and PELH, the improvement in PE in the first 3 weeks and the decrease in SBP in all 4 weeks were significant after follow-up. The SBP of the clusters of PELL and PELH decreased more (–6.1 mmHg and –8.4 mmHg) after follow-up in the first week.

Conclusions

Four distinct PE patterns were identified for patients engaging in the hypertension self-management app. Patients aged over 60 years had higher PE in terms of recording self-measured BP using the app. Once SBP reduced, patients with low PE tended to stop using the app, and a continued decline in PE occurred simultaneously with the increase in SBP. The duration and depth of the effect of health care provider follow-up were more significant in patients with high or increased engagement after follow-up.

Associations between Digital Health Intervention Engagement and Dietary Intake: A Systematic Review

There has been a proliferation of digital health interventions (DHIs) targeting dietary intake. Despite their potential, the effectiveness of DHIs are thought to be dependent, in part, on user engagement. However, the relationship between engagement and the effectiveness of dietary DHIs is not well understood. The aim of this review is to describe the association between DHI engagement and dietary intake. A systematic search of four electronic databases and grey literature for records published before December 2019 was conducted. Studies were eligible if they examined a quantitative association between objective measures of engagement with a DHI (subjective experience or usage) and measures of dietary intake in adults (aged ≥18 years). From 10,653 citations, seven studies were included. Five studies included usage measures of engagement and two examined subjective experiences. Narrative synthesis, using vote counting, found mixed evidence of an association with usage measures (5 of 12 associations indicated a positive relationship, 7 were inconclusive) and no evidence regarding an association with subjective experience (both studies were inconclusive). The findings provide early evidence supporting an association between measures of usage and dietary intake; however, this was inconsistent. Further research examining the association between DHI engagement and dietary intake is warranted.

Targeting subjective engagement in experimental therapeutics for digital mental health interventions

Engagement is a multifaceted construct and a likely mechanism by which digital interventions achieve clinical improvements. To date, clinical research on digital mental health interventions (DMHIs) has overwhelmingly defined engagement and assessed its association with clinical outcomes through the objective/behavioral metrics of use of or interactions with a DMHI, such as number of log-ins or time spent using the technology. However, engagement also entails users' subjective experience. Research is largely lacking that tests the relationship between subjective metrics of engagement and clinical outcomes. The purpose of this study is to present a proof-of-concept exploratory evaluation of the association between subjective engagement measures of a mobile DMHI with changes in depression and anxiety. Adult primary care patients (N = 146) who screened positive for depression or anxiety were randomized to receive a DMHI, IntelliCare, immediately or following an 8-week waitlist. Subjective engagement was measured via the Usefulness, Satisfaction, and Ease of Use (USE) Questionnaire. Across both conditions, results showed that individuals who perceived a mobile intervention as more useful, easy to use and learn, and satisfying had greater improvements in depression and anxiety over eight weeks. Findings support our proposed experimental therapeutics framework that hypothesizes objective/behavioral and subjective engagement metrics as mechanisms that lead to changes in clinical outcomes, as well as support directing intervention design efforts for DMHIs to target the user experience.

Health on the Move (HOME) Study: Using a smartphone app to explore the health and wellbeing of migrants in the United Kingdom

Background/Aim: We have a limited understanding of the broader determinants of health of international migrants and how these change over time since migration to the United Kingdom (UK). To address this knowledge gap, we aim to conduct a prospective cohort study with data acquisition via a smartphone application (app). In this pilot study, we aim to 1) determine the feasibility of the use of an app for data collection in international migrants, 2) optimise app engagement by quantifying the impact of specific design features on the completion rates of survey questionnaires and on study retention, 3) gather preliminary profile health status data, to begin to examine how risk factors for health are distributed among migrants. Methods: We will recruit 275 participants through a social media campaign and through third sector organisations that work with or support migrants in the UK. Following consent and registration, data will be collected via surveys. To optimise app engagement and study retention, we will quantify the impact of specific design features (i.e. the frequency of survey requests, the time of day for app notifications, the frequency of notifications, and the wording of notifications) via micro-randomised process evaluations. The primary outcome for this study is survey completion rates with numerator as the number of surveys completed and denominator as the total number of available surveys. Secondary outcomes are study retention rates and ratings of interest after app usage. Ethics and dissemination: We have obtained approval to use consented patient identifiable data from the University College London Ethics Committee. Improving engagement with the app and gathering preliminary health profile data will help us identify accessibility and usability issues and other barriers to app and study engagement prior to moving to a larger study.

Digital Micro Interventions for Behavioral and Mental Health Gains: Core Components and Conceptualization of Digital Micro Intervention Care

Although many people access publicly available digital behavioral and mental health interventions, most do not invest as much effort in these interventions as hoped or intended by intervention developers, and ongoing engagement is often low. Thus, the impact of such interventions is minimized by a misalignment between intervention design and user behavior. Digital micro interventions are highly focused interventions delivered in the context of a person’s daily life with little burden on the individual. We propose that these interventions have the potential to disruptively expand the reach of beneficial therapeutics by lowering the bar for entry to an intervention and the effort needed for purposeful engagement. This paper provides a conceptualization of digital micro interventions, their component parts, and principles guiding their use as building blocks of a larger therapeutic process (ie, digital micro intervention care). The model represented provides a structure that could improve the design, delivery, and research on digital micro interventions and ultimately improve behavioral and mental health care and care delivery.

Psychometric Evaluation of the TWente Engagement with Ehealth Technologies Scale (TWEETS): Evaluation Study

Background

Engagement emerges as a predictor for the effectiveness of digital health interventions. However, a shared understanding of engagement is missing. Therefore, a new scale has been developed that proposes a clear definition and creates a tool to measure it. The TWente Engagement with Ehealth Technologies Scale (TWEETS) is based on a systematic review and interviews with engaged health app users. It defines engagement as a combination of behavior, cognition, and affect.

Objective

This paper aims to evaluate the psychometric properties of the TWEETS. In addition, a comparison is made with the experiential part of the Digital Behavior Change Intervention Engagement Scale (DBCI-ES-Ex), a scale that showed some issues in previous psychometric analyses.

Methods

In this study, 288 participants were asked to use any step counter app on their smartphones for 2 weeks. They completed online questionnaires at 4 time points: T0=baseline, T1=after 1 day, T2=1 week, and T3=2 weeks. At T0, demographics and personality (conscientiousness and intellect/imagination) were assessed; at T1-T3, engagement, involvement, enjoyment, subjective usage, and perceived behavior change were included as measures that are theoretically related to our definition of engagement. Analyses focused on internal consistency, reliability, and the convergent, divergent, and predictive validity of both engagement scales. Convergent validity was assessed by correlating the engagement scales with involvement, enjoyment, and subjective usage; divergent validity was assessed by correlating the engagement scales with personality; and predictive validity was assessed by regression analyses using engagement to predict perceived behavior change at later time points.

Results

The Cronbach alpha values of the TWEETS were .86, .86, and .87 on T1, T2, and T3, respectively. Exploratory factor analyses indicated that a 1-factor structure best fits the data. The TWEETS is moderately to strongly correlated with involvement and enjoyment (theoretically related to cognitive and affective engagement, respectively; P<.001). Correlations between the TWEETS and frequency of use were nonsignificant or small, and differences between adherers and nonadherers on the TWEETS were significant (P<.001). Correlations between personality and the TWEETS were nonsignificant. The TWEETS at T1 was predictive of perceived behavior change at T3, with an explained variance of 16%. The psychometric properties of the TWEETS and the DBCI-ES-Ex seemed comparable in some aspects (eg, internal consistency), and in other aspects, the TWEETS seemed somewhat superior (divergent and predictive validity).

Conclusions

The TWEETS performs quite well as an engagement measure with high internal consistency, reasonable test-retest reliability and convergent validity, good divergent validity, and reasonable predictive validity. As the psychometric quality of a scale is a reflection of how closely a scale matches the conceptualization of a concept, this paper is also an attempt to conceptualize and define engagement as a unique concept, providing a first step toward an acceptable standard of defining and measuring engagement.

Does the structure (tunneled vs. free-roam) and content (if-then plans vs. choosing strategies) of a brief online alcohol intervention effect engagement and effectiveness? A randomized controlled trial

Despite the potential of brief online interventions for reducing alcohol consumption, their effectiveness may be compromised by low levels of engagement and the inclusion of ineffective behavior change techniques. To test whether (i) a tunneled version of an intervention (where the content is delivered in a prespecified order) leads to greater engagement and greater reductions in alcohol consumption than a free-roam version (where the content can be viewed in any order) and (ii) forming if-then plans linking strategies to cut down with high-risk situations leads to greater reductions in alcohol consumption than only choosing strategies to cut down. Participants (N = 286 university staff and students) were randomly allocated to one of four versions of a brief online alcohol intervention in a 2 (structure: tunneled vs. free-roam) by 2 (planning: strategies vs. if-then plans) factorial design. Engagement (pages visited, time) was recorded automatically. Alcohol consumption (weekly units) was assessed at baseline and 1- and 6-month follow-up. Participants who received the tunneled version viewed significantly more pages and spent significantly more time on the website than those who received the free-roam version. Significant reductions in alcohol consumption were observed at follow-up; however, neither the structure of the intervention nor planning had a significant effect on reductions in alcohol consumption. Tunneled online interventions can increase engagement, but this may not translate into greater changes in behavior. Further experimental research using factorial designs is needed to identify the key behavior change techniques to include in brief online interventions.

Mobile Health App (AGRIPPA) to Prevent Relapse After Successful Interdisciplinary Treatment for Patients With Chronic Pain: Protocol for a Randomized Controlled Trial

BACKGROUND

To facilitate adherence to adaptive pain management behaviors after interdisciplinary multimodal pain treatment, we developed a mobile health app (AGRIPPA app) that contains two behavior regulation strategies.

OBJECTIVE

The aims of this project are (1) to test the effectiveness of the AGRIPPA app on pain disability; (2) to determine the cost-effectiveness; and (3) to explore the levels of engagement and usability of app users.

METHODS

We will perform a multicenter randomized controlled trial with two parallel groups. Within the 12-month inclusion period, we plan to recruit 158 adult patients with chronic pain during the initial stage of their interdisciplinary treatment program in one of the 6 participating centers. Participants will be randomly assigned to the standard treatment condition or to the enhanced treatment condition in which they will receive the AGRIPPA app. Patients will be monitored from the start of the treatment program until 12 months posttreatment. In our primary analysis, we will evaluate the difference over time of pain-related disability between the two conditions. Other outcome measures will include health-related quality of life, illness perceptions, pain self-efficacy, app system usage data, productivity loss, and health care expenses.

RESULTS

The study was approved by the local Medical Research Ethics Committee in October 2019. As of March 20, 2020, we have recruited 88 patients.

CONCLUSIONS

This study will be the first step in systematically evaluating the effectiveness and efficiency of the AGRIPPA app. After 3 years of development and feasibility testing, this formal evaluation will help determine to what extent the app will influence the maintenance of treatment gains over time. The outcomes of this trial will guide future decisions regarding uptake in clinical practice.

TRIAL REGISTRATION

Netherlands Trial Register NL8076; https://www.trialregister.nl/trial/8076.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/18632.

Exclusively Digital Health Interventions Targeting Diet, Physical Activity, and Weight Gain in Pregnant Women: Systematic Review and Meta-Analysis

BACKGROUND

Interventions to promote a healthy diet, physical activity, and weight management during pregnancy are increasingly embracing digital technologies. Although some interventions have combined digital with interpersonal (face-to-face or telephone) delivery, others have relied exclusively on digital delivery. Exclusively digital interventions have the advantages of greater cost-effectiveness and broader reach and as such can be a valuable resource for health care providers.

OBJECTIVE

This systematic review aims to focus on exclusively digital interventions to determine their effectiveness, identify behavior change techniques (BCTs), and investigate user engagement.

METHODS

A total of 6 databases (Medical Literature Analysis and Retrieval System Online [MEDLINE], Excerpta Medica dataBASE [EMBASE], PsycINFO, Cumulated Index to Nursing and Allied Health Literature [CINAHL] Plus, Web of Science, and ProQuest) were searched for randomized controlled trials or pilot control trials of exclusively digital interventions to encourage healthy eating, physical activity, or appropriate weight gain during pregnancy. The outcome measures were gestational weight gain (GWG) and changes in physical activity and dietary behaviors. Study quality was assessed using the Cochrane Risk of Bias tool 2.0. Where possible, pooled effect sizes were calculated using a random effects meta-analysis.

RESULTS

In total, 11 studies met the inclusion criteria. The risk of bias was mostly high (n=5) or moderate (n=3). Of the 11 studies, 6 reported on GWG as the primary outcome, 4 of which also measured changes in physical activity and dietary behaviors, and 5 studies focused either on dietary behaviors only (n=2) or physical activity only (n=3). The meta-analyses showed no significant benefit of interventions on total GWG for either intention-to-treat data (-0.28 kg; 95% CI -1.43 to 0.87) or per-protocol data (-0.65 kg; 95% CI -1.98 to 0.67). Substantial heterogeneity in outcome measures of change in dietary behaviors and physical activity precluded further meta-analyses. BCT coding identified 7 BCTs that were common to all effective interventions. Effective interventions averaged over twice as many BCTs from the goals and planning, and feedback and monitoring domains as ineffective interventions. Data from the 6 studies reporting on user engagement indicated a positive association between high engagement with key BCTs and greater intervention effectiveness. Interventions using proactive messaging and feedback appeared to have higher levels of engagement.

CONCLUSIONS

In contrast to interpersonal interventions, there is little evidence of the effectiveness of exclusively digital interventions to encourage a healthy diet, physical activity, or weight management during pregnancy. In this review, effective interventions used proactive messaging, such as reminders to engage in BCTs, feedback on progress, or tips, suggesting that interactivity may drive engagement and lead to greater effectiveness. Given the benefits of cost and reach of digital interventions, further research is needed to understand how to use advancing technologies to enhance user engagement and improve effectiveness.

Examining the Usage, User Experience, and Perceived Impact of an Internet-Based Cognitive Behavioral Therapy Program for Adolescents With Anxiety: Randomized Controlled Trial

BACKGROUND

Internet-based cognitive behavioral therapy (iCBT) increases treatment access for adolescents with anxiety; however, completion rates of iCBT programs are typically low. Understanding adolescents' experiences with iCBT, what program features and changes in anxiety (minimal clinically important difference [MCID]) are important to them, may help explain and improve iCBT program use and impact.

OBJECTIVE

Within a randomized controlled trial comparing a six-session iCBT program for adolescent anxiety, Being Real, Easing Anxiety: Tools Helping Electronically (Breathe), with anxiety-based resource webpages, we aimed to (1) describe intervention use among adolescents allocated to Breathe or webpages and those who completed postintervention assessments (Breathe or webpage respondents); (2) describe and compare user experiences between groups; and (3) calculate an MCID for anxiety and explore relationships between iCBT use, experiences, and treatment response among Breathe respondents.

METHODS

Enrolled adolescents with self-reported anxiety, aged 13 to 19 years, were randomly allocated to Breathe or webpages. Self-reported demographics and anxiety symptoms (Multidimensional Anxiety Scale for Children-2nd edition [MASC-2]) were collected preintervention. Automatically-captured Breathe or webpage use and self-reported symptoms and experiences (User Experience Questionnaire for Internet-based Interventions) were collected postintervention. Breathe respondents also reported their perceived change in anxiety (Global Rating of Change Scale [GRCS]) following program use. Descriptive statistics summarized usage and experience outcomes, and independent samples t tests and correlations examined relationships between them. The MCID was calculated using the mean MASC-2 change score among Breathe respondents reporting somewhat better anxiety on the GRCS.

RESULTS

Adolescents were mostly female (382/536, 71.3%), aged 16.6 years (SD 1.7), with very elevated anxiety (mean 92.2, SD 18.1). Intervention use was low for adolescents allocated to Breathe (mean 2.2 sessions, SD 2.3; n=258) or webpages (mean 2.1 visits, SD 2.7; n=278), but was higher for Breathe (median 6.0, range 1-6; 81/258) and webpage respondents (median 2.0, range 1-9; 148/278). Total user experience was significantly more positive for Breathe than webpage respondents (P<.001). Breathe respondents reported program design and delivery factors that may have challenged (eg, time constraints and program support) or facilitated (eg, demonstration videos, self-management activities) program use. The MCID was a mean MASC-2 change score of 13.8 (SD 18.1). Using the MCID, a positive treatment response was generated for 43% (35/81) of Breathe respondents. Treatment response was not correlated with respondents' experiences or use of Breathe (P=.32 to P=.88).

CONCLUSIONS

Respondents reported positive experiences and changes in their anxiety with Breathe; however, their reports were not correlated with program use. Breathe respondents identified program design and delivery factors that help explain their experiences and use of iCBT and inform program improvements. Future studies can apply our measures to compare user experiences between internet-based interventions, interpret treatment outcomes and improve treatment decision making for adolescents with anxiety.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02970734 https://clinicaltrials.gov/ct2/show/NCT02970734.

Assessing the Psychometric Properties of the Digital Behavior Change Intervention Engagement Scale in Users of an App for Reducing Alcohol Consumption: Evaluation Study

BACKGROUND

The level and type of engagement with digital behavior change interventions (DBCIs) are likely to influence their effectiveness, but validated self-report measures of engagement are lacking. The DBCI Engagement Scale was designed to assess behavioral (ie, amount, depth of use) and experiential (ie, attention, interest, enjoyment) dimensions of engagement.

OBJECTIVE

We aimed to assess the psychometric properties of the DBCI Engagement Scale in users of a smartphone app for reducing alcohol consumption.

METHODS

Participants (N=147) were UK-based, adult, excessive drinkers recruited via an online research platform. Participants downloaded the Drink Less app and completed the scale immediately after their first login in exchange for a financial reward. Criterion variables included the objectively recorded amount of use, depth of use, and subsequent login. Five types of validity (ie, construct, criterion, predictive, incremental, divergent) were examined in exploratory factor, correlational, and regression analyses. The Cronbach alpha was calculated to assess the scale's internal reliability. Covariates included motivation to reduce alcohol consumption.

RESULTS

Responses on the DBCI Engagement Scale could be characterized in terms of two largely independent subscales related to experience and behavior. The experiential and behavioral subscales showed high (α=.78) and moderate (α=.45) internal reliability, respectively. Total scale scores predicted future behavioral engagement (ie, subsequent login) with and without adjusting for users' motivation to reduce alcohol consumption (adjusted odds ratio [ORadj]=1.14; 95% CI 1.03-1.27; P=.01), which was driven by the experiential (ORadj=1.19; 95% CI 1.05-1.34; P=.006) but not the behavioral subscale.

CONCLUSIONS

The DBCI Engagement Scale assesses behavioral and experiential aspects of engagement. The behavioral subscale may not be a valid indicator of behavioral engagement. The experiential subscale can predict subsequent behavioral engagement with an app for reducing alcohol consumption. Further refinements and validation of the scale in larger samples and across different DBCIs are needed.