Predictors of Engagement, Response to Follow Up, and Extent of Alcohol Reduction in Users of a Smartphone App (Drink Less): Secondary Analysis of a Factorial Randomized Controlled Trial

Do engagement and behavioural mechanisms underpin the effectiveness of the Drink Less app?

This is a process evaluation of a large UK-based randomised controlled trial (RCT) (n = 5602) evaluating the effectiveness of recommending an alcohol reduction app, Drink Less, compared with usual digital care in reducing alcohol consumption in increasing and higher risk drinkers. The aim was to understand whether participants' engagement ('self-reported adherence') and behavioural characteristics were mechanisms of action underpinning the effectiveness of Drink Less. Self-reported adherence with both digital tools was over 70% (Drink Less: 78.0%, 95% CI = 77.6-78.4; usual digital care: 71.5%, 95% CI = 71.0-71.9). Self-reported adherence to the intervention (average causal mediation effect [ACME] = -0.250, 95% CI = -0.42, -0.11) and self-monitoring behaviour (ACME = -0.235, 95% CI = -0.44, -0.03) both partially mediated the effect of the intervention (versus comparator) on alcohol reduction. Following the recommendation (self-reported adherence) and the tracking (self-monitoring behaviour) feature of the Drink Less app appear to be important mechanisms of action for alcohol reduction among increasing and higher risk drinkers.

User engagement with organizational mHealth stress management intervention - A mixed methods study

Mobile health (mHealth) demonstrates great promise for providing effective and accessible interventions within an organizational context. Compared with traditional workplace interventions, mHealth solutions may be significantly more scalable and easier to standardize. However, inadequate user engagement is a major challenge with mHealth solutions that can negatively impact the potential benefits of an intervention. More research is needed to better understand how to ensure sufficient engagement, which is essential for designing and implementing effective interventions. To address this issue, this study employed a mixed methods approach to investigate what factors influence user engagement with an organizational mHealth intervention. Quantitative data were collected using surveys (n = 1267), and semi-structured interviews were conducted with a subset of participants (n = 17). Primary findings indicate that short and consistent interactions as well as user intention are key drivers of engagement. These results may inform future development of interventions to increase engagement and effectiveness.

Alcohol Treatment Access and Engagement Among Women in the USA: a Targeted Review of the Literature 2012-2022

Purpose of Review

The purpose of this review is to examine recent literature (2012-2022) on alcohol treatment access and engagement in women in the U.S. and propose future directions for research and clinical practice.

Recent Findings

A targeted literature review resulted in 27 studies encompassing screening and brief intervention (SBIRT), treatment utilization, treatment engagement, and barriers to treatment. Recent literature demonstrates overall low rates of screening and brief interventions and treatment utilization in the population with women less likely to be screened and utilize alcohol treatment. The magnitude of these gender differences varies with race/ethnicity. Extensive barriers to care include provider knowledge, structural barriers, and attitudinal barriers and these vary with service setting, gender, and race/ethnicity.

Summary

There is an increasing prevalence of alcohol use and Alcohol Use Disorder (AUD) in women with low rates of screening, brief treatment, treatment, and engagement which have resulted from extensive barriers to care. Possible areas of further inquiry include the impact of race/ethnicity on gender differences, improving provider and system level policies to promote SBIRT and treatment engagement and utilization, further developing digital interventions, and implementation research to investigate factors associated with optimizing effectiveness of gender-responsive and culturally tailored interventions that address the unique needs of women.

Examining engagement effects in an adaptive preventive intervention for college student drinking

OBJECTIVE

This study determined the characteristics of engagement and whether engagement in an adaptive preventive intervention (API) was associated with reduced binge drinking and alcohol-related consequences.

METHOD

Incoming students were recruited for a sequential multiple assignment randomized trial (SMART; N = 891, 62.4% female, 76.8% non-Hispanic White) with an assessment-only control group. The API occurred during the first semester of college, with outcomes assessed at the end of the semester. The API involved two stages. Stage 1 included universal intervention components (personalized normative feedback [PNF] and self-monitoring). Stage 2 bridged heavy drinkers to access additional resources. We estimated the effect of engagement in Stage 1 only and in the whole API (Stages 1 and 2) among the intervention group, and the effect of the API versus control had all students assigned an API engaged, on alcohol-related outcomes.

RESULTS

Precollege binge drinking, intention to pledge a fraternity/sorority, and higher conformity motives were most associated with lower odds of Stage 1 engagement. Action (readiness to change) and PNF engagement were associated with Stage 2 engagement. API engagement was associated with significant reductions in alcohol-related consequences among heavy drinkers. Compared to the control, we estimated the API would reduce the relative increase in alcohol-related consequences from baseline to follow-up by 25%, had all API students engaged.

CONCLUSIONS

Even partial engagement in each component of the "light-touch" API rendered benefits. Analyses suggested that had all students in the intervention group engaged, the API would significantly reduce the change in alcohol-related consequences over the first semester in college. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

User Engagement Clusters of an 8-Week Digital Mental Health Intervention Guided by a Relational Agent (Woebot): Exploratory Study

BACKGROUND

With the proliferation of digital mental health interventions (DMHIs) guided by relational agents, little is known about the behavioral, cognitive, and affective engagement components associated with symptom improvement over time. Obtaining a better understanding could lend clues about recommended use for particular subgroups of the population, the potency of different intervention components, and the mechanisms underlying the intervention's success.

OBJECTIVE

This exploratory study applied clustering techniques to a range of engagement indicators, which were mapped to the intervention's active components and the connect, attend, participate, and enact (CAPE) model, to examine the prevalence and characterization of each identified cluster among users of a relational agent-guided DMHI.

METHODS

We invited adults aged 18 years or older who were interested in using digital support to help with mood management or stress reduction through social media to participate in an 8-week DMHI guided by a natural language processing-supported relational agent, Woebot. Users completed assessments of affective and cognitive engagement, working alliance as measured by goal and task working alliance subscale scores, and enactment (ie, application of therapeutic recommendations in real-world settings). The app passively collected data on behavioral engagement (ie, utilization). We applied agglomerative hierarchical clustering analysis to the engagement indicators to identify the number of clusters that provided the best fit to the data collected, characterized the clusters, and then examined associations with baseline demographic and clinical characteristics as well as mental health outcomes at week 8.

RESULTS

Exploratory analyses (n=202) supported 3 clusters: (1) "typical utilizers" (n=81, 40%), who had intermediate levels of behavioral engagement; (2) "early utilizers" (n=58, 29%), who had the nominally highest levels of behavioral engagement in week 1; and (3) "efficient engagers" (n=63, 31%), who had significantly higher levels of affective and cognitive engagement but the lowest level of behavioral engagement. With respect to mental health baseline and outcome measures, efficient engagers had significantly higher levels of baseline resilience (P<.001) and greater declines in depressive symptoms (P=.01) and stress (P=.01) from baseline to week 8 compared to typical utilizers. Significant differences across clusters were found by age, gender identity, race and ethnicity, sexual orientation, education, and insurance coverage. The main analytic findings remained robust in sensitivity analyses.

CONCLUSIONS

There were 3 distinct engagement clusters found, each with distinct baseline demographic and clinical traits and mental health outcomes. Additional research is needed to inform fine-grained recommendations regarding optimal engagement and to determine the best sequence of particular intervention components with known potency. The findings represent an important first step in disentangling the complex interplay between different affective, cognitive, and behavioral engagement indicators and outcomes associated with use of a DMHI incorporating a natural language processing-supported relational agent.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05672745; https://classic.clinicaltrials.gov/ct2/show/NCT05672745.

The engagement problem: A review of engagement with digital mental health interventions and recommendations for a path forward

Purpose of the review

Digital mental health interventions (DMHIs) are an effective and accessible means of addressing the unprecedented levels of mental illness worldwide. Currently, however, patient engagement with DMHIs in real-world settings is often insufficient to see clinical benefit. In order to realize the potential of DMHIs, there is a need to better understand what drives patient engagement.

Recent findings

We discuss takeaways from the existing literature related to patient engagement with DMHIs and highlight gaps to be addressed through further research. Findings suggest that engagement is influenced by patient-, intervention- and systems-level factors. At the patient-level, variables such as sex, education, personality traits, race, ethnicity, age and symptom severity appear to be associated with engagement. At the intervention-level, integrating human support, gamification, financial incentives and persuasive technology features may improve engagement. Finally, although systems-level factors have not been widely explored, the existing evidence suggests that achieving engagement will require addressing organizational and social barriers and drawing on the field of implementation science.

Summary

Future research clarifying the patient-, intervention- and systems-level factors that drive engagement will be essential. Additionally, to facilitate improved understanding of DMHI engagement, we propose the following: (a) widespread adoption of a minimum necessary 5-element engagement reporting framework; (b) broader application of alternative clinical trial designs; and (c) directed efforts to build upon an initial parsimonious conceptual model of DMHI engagement.

Trajectory classes of engagement with an alcohol text message intervention and predictors of intervention engagement

Digital interventions have increased our capacity to reach young adults who have hazardous alcohol use. Alcohol text message interventions have shown small effects in reducing hazardous drinking, leaving room for improvement. An important challenge to address in improving digital interventions is maintaining engagement, which reflects the "dose" of intervention received. This study aimed to identify trajectory classes of engagement with an alcohol text message intervention, and baseline predictors of the trajectory classes to determine "for whom" the digital intervention was more versus less engaging, to guide further intervention tailoring. This secondary analysis examined data from a study that compared five 12-week alcohol text message interventions designed to reduce hazardous drinking in young adults (aged 18-25; N = 1,131, 68% female) recruited from Emergency Departments in Western Pennsylvania. Engagement with the intervention was based on response (present/absent) to text message queries delivered twice per week during 2-week run-in and 12-week intervention. Repeated measures latent profile analysis identified five latent trajectory classes as having the best fit to the data: "High engagement" (55.1%), "Slow decrease, moderate engagement" (23.2%); "Mid-way decrease in engagement" (8.9%), "Steadily decreasing engagement" (8.1%); and "Fluctuating, moderate engagement" (4.6%). Females and individuals enrolled in college were overrepresented in the high engagement trajectory class, whereas individuals higher in impulsivity were more likely to be in decreasing (versus high) engagement trajectory classes. Methods to boost engagement using, for example, motivational enhancement for young adults with higher levels of impulsivity, at specific time points, such as the mid-point of the intervention, warrant consideration.

A Smartphone Physical Activity App for Patients in Alcohol Treatment: Single-Arm Feasibility Trial

BACKGROUND

Alcohol use disorder (AUD) is a significant public health concern worldwide. Alcohol consumption is a leading cause of death in the United States and has a significant negative impact on individuals and society. Relapse following treatment is common, and adjunct intervention approaches to improve alcohol outcomes during early recovery continue to be critical. Interventions focused on increasing physical activity (PA) may improve AUD treatment outcomes. Given the ubiquity of smartphones and activity trackers, integrating this technology into a mobile app may be a feasible, acceptable, and scalable approach for increasing PA in individuals with AUD.

OBJECTIVE

This study aims to test the Fit&Sober app developed for patients with AUD. The goals of the app were to facilitate self-monitoring of PA engagement and daily mood and alcohol cravings, increase awareness of immediate benefits of PA on mood and cravings, encourage setting and adjusting PA goals, provide resources and increase knowledge for increasing PA, and serve as a resource for alcohol relapse prevention strategies.

METHODS

To preliminarily test the Fit&Sober app, we conducted an open pilot trial of patients with AUD in early recovery (N=22; 13/22, 59% women; mean age 43.6, SD 11.6 years). At the time of hospital admission, participants drank 72% of the days in the last 3 months, averaging 9 drinks per drinking day. The extent to which the Fit&Sober app was feasible and acceptable among patients with AUD during early recovery was examined. Changes in alcohol consumption, PA, anxiety, depression, alcohol craving, and quality of life were also examined after 12 weeks of app use.

RESULTS

Participants reported high levels of satisfaction with the Fit&Sober app. App metadata suggested that participants were still using the app approximately 2.5 days per week by the end of the intervention. Pre-post analyses revealed small-to-moderate effects on increase in PA, from a mean of 5784 (SD 2511) steps per day at baseline to 7236 (SD 3130) steps per day at 12 weeks (Cohen d=0.35). Moderate-to-large effects were observed for increases in percentage of abstinent days (Cohen d=2.17) and quality of life (Cohen d=0.58) as well as decreases in anxiety (Cohen d=-0.71) and depression symptoms (Cohen d=-0.58).

CONCLUSIONS

The Fit&Sober app is an acceptable and feasible approach for increasing PA in patients with AUD during early recovery. A future randomized controlled trial is necessary to determine the efficacy of the Fit&Sober app for long-term maintenance of PA, ancillary mental health, and alcohol outcomes. If the efficacy of the Fit&Sober app could be established, patients with AUD would have a valuable adjunct to traditional alcohol treatment that can be delivered in any setting and at any time, thereby improving the overall health and well-being of this population.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02958280; https://www.clinicaltrials.gov/ct2/show/NCT02958280.

Applying Collective Intelligence in Health Recommender Systems for Smoking Cessation: A Comparison Trial

Background: Health recommender systems (HRSs) are intelligent systems that can be used to tailor digital health interventions. We compared two HRSs to assess their impact providing smoking cessation support messages. Methods: Smokers who downloaded a mobile app to support smoking abstinence were randomly assigned to two interventions. They received personalized, ratable motivational messages on the app. The first intervention had a knowledge-based HRS (n = 181): it selected random messages from a subset matching the users’ demographics and smoking habits. The second intervention had a hybrid HRS using collective intelligence (n = 190): it selected messages applying the knowledge-based filter first, and then chose the ones with higher ratings provided by other similar users in the system. Both interventions were compared on: (a) message appreciation, (b) engagement with the system, and (c) one’s own self-reported smoking cessation status, as indicated by the last seven-day point prevalence report in different time intervals during a period of six months. Results: Both interventions had similar message appreciation, number of rated messages, and abstinence results. The knowledge-based HRS achieved a significantly higher number of active days, number of abstinence reports, and better abstinence results. The hybrid algorithm led to more quitting attempts in participants who completed their user profiles.

Engagement With a Mobile Phone-Based Life Skills Intervention for Adolescents and Its Association With Participant Characteristics and Outcomes: Tree-Based Analysis

Background

Mobile phone–delivered life skills programs are an emerging and promising way to promote mental health and prevent substance use among adolescents, but little is known about how adolescents actually use them.

Objective

The aim of this study is to determine engagement with a mobile phone–based life skills program and its different components, as well as the associations of engagement with adolescent characteristics and intended substance use and mental health outcomes.

Methods

We performed secondary data analysis on data from the intervention group (n=750) from a study that compared a mobile phone–based life skills intervention for adolescents recruited in secondary and upper secondary school classes with an assessment-only control group. Throughout the 6-month intervention, participants received 1 SMS text message prompt per week that introduced a life skills topic or encouraged participation in a quiz or individual life skills training or stimulated sharing messages with other program participants through a friendly contest. Decision trees were used to identify predictors of engagement (use and subjective experience). The stability of these decision trees was assessed using a resampling method and by graphical representation. Finally, associations between engagement and intended substance use and mental health outcomes were examined using logistic and linear regression analyses.

Results

The adolescents took part in half of the 50 interactions (mean 23.6, SD 15.9) prompted by the program, with SMS text messages being the most used and contests being the least used components. Adolescents who did not drink in a problematic manner and attended an upper secondary school were the ones to use the program the most. Regarding associations between engagement and intended outcomes, adolescents who used the contests more frequently were more likely to be nonsmokers at follow-up than those who did not (odds ratio 0.86, 95% CI 0.76-0.98; P=.02). In addition, adolescents who read the SMS text messages more attentively were less likely to drink in a problematic manner at follow-up (odds ratio 0.43, 95% CI 1.29-3.41; P=.003). Finally, participants who used the program the most and least were more likely to increase their well-being from baseline to 6-month follow-up compared with those with average engagement (βs=.39; t₅₈₆=2.66; P=.008; R²=0.24).

Conclusions

Most of the adolescents participating in a digital life skills program that aimed to prevent substance use and promote mental health engaged with the intervention. However, measures to increase engagement in problem drinkers should be considered. Furthermore, efforts must be made to ensure that interventions are engaging and powerful across different educational levels. First results indicate that higher engagement with digital life skills programs could be associated with intended outcomes. Future studies should apply further measures to improve the reach of lower-engaged participants at follow-up to establish such associations with certainty.

Engagement in Digital Mental Health Interventions: Can Monetary Incentives Help?

Digital mental health interventions (DMHI) are scalable and cost-effective strategies for increasing access to mental health care; however, dropout rates associated with digital interventions are high, particularly for open-access digital interventions. While some studies have focused on predictors of dropout from digital mental health programs, few studies have focused on engagement features that might improve engagement. In this perspective article, we discuss whether monetary incentives (MI) are one avenue to increasing user engagement in DMHI. We begin by reviewing the literature on the effects of MI for behavior change in health domains (e.g., dietary behaviors, substance use, and medication adherence). Then, drawing on a pilot study we conducted to test the effects of different levels of MI on usage and improvement in subjective well-being among users of a DMHI (Happify), we discuss the potential applications of MI for DMHI, the potential drawbacks of financial incentives in this context, and open questions for future research.

Predictors of Booster Engagement Following a Web-Based Brief Intervention for Alcohol Misuse Among National Guard Members: Secondary Analysis of a Randomized Controlled Trial

BACKGROUND

Alcohol misuse is a major health concern among military members. Reserve component members face unique barriers as they live off base with limited access to behavioral health services. Web and app-based brief interventions are a promising means to improve access to treatment for those who misuse alcohol, with the use of booster sessions to enhance effectiveness, solidify gains, and reinforce changes. However, little is known about who will engage in booster sessions.

OBJECTIVE

This study aims to evaluate booster engagement across booster delivery modalities (Web and Peer) and identify participant-specific factors associated with booster session engagement.

METHODS

Following a brief web-based alcohol misuse intervention in National Guard members (N=739), we examined engagement in a series of three booster sessions. Using unadjusted and adjusted models, demographic and clinical characteristics that may serve as predictors of booster session engagement were examined across the 2 arms of the trial with different types of booster sessions: peer-delivered (N=245) and web-delivered (N=246).

RESULTS

Booster session completion was greater for Peer than Web Booster sessions, with 142 (58%) service members in the Peer Booster arm completing all three boosters compared with only 108 (44%) of participants in the Web Booster arm (χ²₃=10.3; P=.006). In a model in which the 2 groups were combined, socioeconomic factors predicted booster engagement. In separate models, the demographic and clinical predictors of booster engagement varied between the 2 delivery modalities.

CONCLUSIONS

The use of peer-delivered boosters, especially among subsets of reserve members at risk of lack of engagement, may foster greater uptake and improve treatment outcomes.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02181283; https://clinicaltrials.gov/ct2/show/NCT02181283.

Examination of the relationship between lapses and weight loss in a smartphone-based just-in time adaptive intervention

We developed a smartphone-based just-in-time adaptive intervention (JITAI), called OnTrack, that provides personalized intervention to prevent dietary lapses (i.e., nonadherence from the behavioral weight loss intervention diet). OnTrack utilizes ecological momentary assessment (EMA; repeated electronic surveys) for self-reporting lapse triggers, predicts lapses using machine learning, and provides brief intervention to prevent lapse. We have established preliminary feasibility, acceptability, and efficacy of OnTrack, but no study has examined our hypothesized mechanism of action: reduced lapse frequency will be associated with greater weight loss while using OnTrack. This secondary analysis investigated the association between lapse frequency and the weekly percentage of weight loss. Post hoc analyses evaluated the moderating effect of OnTrack engagement on this association. Participants (N = 121) with overweight/obesity (MBMI = 34.51; 84.3% female; 69.4% White) used OnTrack with a digital weight loss program for 10 weeks. Engagement with OnTrack (i.e., EMA completed and interventions accessed) was recorded automatically, participants self-reported dietary lapses via EMA, and weighed weekly using Bluetooth scales. Linear mixed models with a random effect of subject and fixed effect of time revealed a nonsignificant association between weekly lapses and the percentage of weight loss. Post hoc analyses revealed a statistically significant moderation effect of OnTrack engagement such that fewer EMA and interventions completed conferred the expected associations between lapses and weight loss. Lapses were not associated with weight loss in this study and one explanation may be the influence of engagement levels on this relationship. Future research should investigate the role of engagement in evaluating JITAIs.

Identifying Content-Based Engagement Patterns in a Smoking Cessation Website and Associations With User Characteristics and Cessation Outcomes: A Sequence and Cluster Analysis

INTRODUCTION

Using WebQuit as a case study, a smoking cessation website grounded in Acceptance and Commitment Therapy, we aimed to identify sequence clusters of content usage and examine their associations with baseline characteristics, change to a key mechanism of action, and smoking cessation.

METHODS

Participants were adult smokers allocated to the WebQuit arm in a randomized controlled trial (n = 1,313). WebQuit contains theory-informed content including goal setting, self-monitoring and feedback, and values- and acceptance-based exercises. Sequence analysis was used to temporally order 30-s website usage segments for each participant. Similarities between sequences were assessed with the optimal matching distance algorithm and used as input in an agglomerative hierarchical clustering analysis. Associations between sequence clusters and baseline characteristics, acceptance of cravings at 3 months and self-reported 30-day point prevalence abstinence at 12 months were examined with linear and logistic regression.

RESULTS

Three qualitatively different sequence clusters were identified. "Disengagers" (576/1,313) almost exclusively used the goal-setting feature. "Tryers" (375/1,313) used goal setting and two of the values- and acceptance-based components ("Be Aware," "Be Willing"). "Committers" (362/1,313) primarily used two of the values- and acceptance-based components ("Be Willing," "Be Inspired"), goal setting, and self-monitoring and feedback. Compared with Disengagers, Committers demonstrated greater increases in acceptance of cravings (p = .01) and 64% greater odds of quit success (ORadj = 1.64, 95% CI = 1.18, 2.29, p = .003).

DISCUSSION

WebQuit users were categorized into Disengagers, Tryers, and Committers based on their qualitatively different content usage patterns. Committers saw increases in a key mechanism of action and greater odds of quit success.

IMPLICATIONS

This case study demonstrates how employing sequence and cluster analysis of usage data can help researchers and practitioners gain a better understanding of how users engage with a given eHealth intervention over time and use findings to test theory and/or to improve future iterations to the intervention. Future WebQuit users may benefit from being directed to the values- and acceptance-based and the self-monitoring and feedback components via reminders over the course of the program.

Usability and Acceptability of a Mobile App for the Self-Management of Alcohol Misuse Among Veterans (Step Away): Pilot Cohort Study

BACKGROUND

Alcohol misuse is common among Operation Enduring Freedom and Operation Iraqi Freedom veterans, yet barriers limit treatment participation. Mobile apps hold promise as means to deliver alcohol interventions to veterans who prefer to remain anonymous, have little time for conventional treatments, or live too far away to attend treatment in person.

OBJECTIVE

This pilot study evaluated the usability and acceptability of Step Away, a mobile app designed to reduce alcohol-related risks, and explored pre-post changes on alcohol use, psychological distress, and quality of life.

METHODS

This single-arm pilot study recruited Operation Enduring Freedom and Operation Iraqi Freedom veterans aged 18 to 55 years who exceeded National Institute on Alcohol Abuse and Alcoholism drinking guidelines and owned an iPhone. Enrolled veterans (N=55) completed baseline and 1-, 3-, and 6-month assessments. The System Usability Scale (scaled 1-100, ≥70 indicating acceptable usability) assessed the effectiveness, efficiency, and satisfaction dimensions of usability, while a single item (scaled 1-9) measured the attractiveness of 10 screenshots. Learnability was assessed by app use during week 1. App engagement (proportion of participants using Step Away, episodes of use, and minutes per episode per week) over 6 months measured acceptability. Secondary outcomes included pre-post change on heavy drinking days (men: ≥5 drinks per day; women: ≥4 drinks per day) and Short Inventory of Problems-Revised, Kessler-10, and brief World Health Organization Quality of Life Questionnaire scores.

RESULTS

Among the 55 veterans enrolled in the study, the mean age was 37.4 (SD 7.6), 16% (9/55) were women, 82% (45/55) were White, and 82% (45/55) had an alcohol use disorder. Step Away was used by 96% (53/55) of participants in week 1, 55% (30/55) in week 4, and 36% (20/55) in week 24. Step Away use averaged 55.1 minutes (SD 57.6) in week 1 and <15 minutes per week in weeks 2 through 24. Mean System Usability Scale scores were 69.3 (SD 19.7) and 71.9 (SD 15.8) at 1 and 3 months, respectively. Median attractiveness scores ranged from 5 to 8, with lower ratings for text-laden screens. Heavy drinking days decreased from 29.4% (95% CI 23.4%-35.4%) at baseline to 16.2% (95% CI 9.9%-22.4%) at 6 months (P<.001). Likewise, over 6 months, Short Inventory of Problems-Revised scores decreased from 6.3 (95% CI 5.1-7.5) to 3.6 (95% CI 2.4-4.9) (P<.001) and Kessler-10 scores decreased from 18.8 (95% CI 17.4-20.1) to 17.3 (95% CI 15.8-18.7) (P=.046). Changes were not detected on quality of life scores.

CONCLUSIONS

Operation Enduring Freedom and Operation Iraqi Freedom veterans found the usability of Step Away to be acceptable and engaged in the app over the 6-month study. Reductions were seen in heavy drinking days, alcohol-related problems, and Kessler-10 scores. A larger randomized trial is warranted to confirm our findings.

The impact of celebrity influence and national media coverage on users of an alcohol reduction app: a natural experiment

BACKGROUND

Smartphone apps are increasingly used for health-related behaviour change and people discover apps through different sources. However, it is unclear whether users differ by mode of app discovery. Drink Less is an alcohol reduction app that received national media coverage in the UK caused by celebrity influence (a male TV and radio national broadcaster, aged 51). Our aim was to compare users who discovered the app before and after this coverage.

METHODS

A natural experiment assessing the impact of media coverage of Drink Less on users' socio-demographic and drinking characteristics, app engagement levels, and extent of alcohol reduction. The study period was from 17th May 2017 to 23rd January 2019, with media coverage starting on 21st August 2018. Users were 18 years or over, based in the UK and interested in drinking less. Interrupted time series analyses using Generalised Additive Mixed Models were conducted for each outcome variable aggregated at the weekly level.

RESULTS

In 66 weeks prior to the media coverage, 8617 users downloaded the app and 18,959 in 23 weeks afterwards. There was a significant step-level increase in users' mean age (B = 8.17, p < .001) and a decrease in the percentage of female users (B = -27.71, p < .001), though these effects dissipated non-linearly over time. No effect of media coverage was detected on employment type or on the percentage of at-risk drinkers, though the mean Alcohol Use Disorders Identification Test score was lower after the media coverage (B = -1.43, p = .031). There was a step-level increase in app engagement - number of sessions (B = 3.45, p = .038) and number of days used (B = 2.30, p = .005) - which continued to increase over time following quadratic trends.

CONCLUSIONS

Celebrity influence leading to national media coverage in the UK of the Drink Less app was associated with more people downloading the app who were male, older and engaged with the app; and did not appear to impact employment inequality.

Engagement With a Web-Based Intervention to Reduce Harmful Drinking: Secondary Analysis of a Randomized Controlled Trial

Background

Engagement with digital behavior change interventions (DBCIs) is considered a prerequisite for intervention efficacy. However, in many trials on DBCIs, participants use the intervention either only little or not at all.

Objective

To analyze engagement with a web-based intervention to reduce harmful drinking, we explored (1) whether engagement with a web-based alcohol intervention is related to drinking outcomes, (2) which user characteristics are associated with measures of engagement, and (3) whether reported outcomes are associated with data captured by voluntary intervention questionnaires.

Methods

We analyzed data of the intervention arm of a randomized controlled trial on a DBCI to reduce risky alcohol consumption. Data were collected at baseline (T0), after 90 days (T1), and at the end of the 180-day usage period (T2). Engagement with the intervention was measured via system usage data as well as self-reported usage. Drinking behavior was measured as average daily alcohol consumption as well as the number of binge drinking days. User characteristics included demographics, baseline drinking behavior, readiness to change, alcohol-related outcome expectancies, and alcohol abstinence self-efficacy. Following a bivariate approach, we performed two-tailed Welch’s t tests and Wilcoxon signed rank/Mann-Whitney U tests or calculated correlation coefficients.

Results

The data of 306 users were analyzed. Time spent engaging with the intervention as measured by system usage did not match self-reported usage. Higher self-reported usage was associated with higher reductions in average daily alcohol consumption (T1: ρ=0.39, P<.001; T2: ρ=0.29, P=.015) and in binge drinking days (T1: ρ=0.62, P<.001; T2: ρ=0.3, P=.006). Higher usage was reported from users who were single (T1: P<.001; T2: P<.001), users without children (T1: P<.001; T2: P<.001), users who did not start or finish secondary education (T1: P<.001; T2: P<.001), users without academic education (T1: P<.001; T2: P<.001), and those who worked (T1: P=.001; T2: P=.004). Relationships between self-reported usage and clinical or psychological baseline characteristics were complex. For system usage, the findings were mixed. Reductions in drinking captured by intervention questionnaires were associated with reported outcomes.

Conclusions

Though self-reported usage could be consistently linked to better outcomes and multiple user characteristics, our findings add to the overall inconclusive evidence that can be found throughout the literature. Our findings indicate potential benefits of self-reports as measures of engagement and intervention questionnaires as a basis for tailoring of intervention content. Future studies should adopt a theory-driven approach to engagement research utilizing psychometrically sound self-report questionnaires and include short ecological momentary assessments within the DBCIs.

Trial Registration

German Clinical Trials Register DRKS00006104; https://tinyurl.com/y22oc5jo

Notifications to Improve Engagement With an Alcohol Reduction App: Protocol for a Micro-Randomized Trial

BACKGROUND

Drink Less is a behavior change app that aims to help users in the general adult population reduce hazardous and harmful alcohol consumption. The app includes a daily push notification, delivered at 11 am, asking users to "Please complete your mood and drinking diaries." Previous analysis of Drink Less engagement data suggests the current notification strongly influences how users engage with the app in the subsequent hour. To exploit a potential increase of vulnerability of excess drinking and opportunity to engage with the app in the evenings, we changed the delivery time from 11 am to 8 pm. We now aim to further optimise the content and sequence of notifications, testing 30 new evidence-informed notifications targeting the user's perceived usefulness of the app.

OBJECTIVE

The primary objective is to assess whether sending a notification at 8 pm increases behavioral engagement (opening the app) in the subsequent hour. Secondary objectives include comparing the effect of the new bank of messages with the standard message and effect moderation over time. We also aim to more generally understand the role notifications have on the overall duration, depth, and frequency of engagement with Drink Less over the first 30 days after download.

METHODS

This is a protocol for a micro-randomized trial with two additional parallel arms. Inclusion criteria are Drink Less users who (1) consent to participate in the trial; (2) self-report a baseline Alcohol Use Disorders Identification Test score of 8 or above; (3) reside in the United Kingdom; (4) age ≥18 years and; (5) report interest in drinking less alcohol. In the micro-randomized trial, participants will be randomized daily at 8 pm to receive no notification, a notification with text from the new message bank, or the standard message. The primary outcome is the time-varying, binary outcome of "Did the user open the app in the hour from 8 pm to 9 pm?". The primary analysis will estimate the marginal relative risk for the notifications using an estimator developed for micro-randomized trials with binary outcomes. Participants randomized to the parallel arms will receive no notifications (Secondary Arm A), or the standard notification delivered daily at 11 am (Secondary Arm B) over 30 days, allowing the comparison of overall engagement between different notification delivery strategies.

RESULTS

Approval was granted by the University College of London's Departmental Research Ethics Committee (CEHP/2016/556) on October 11, 2019, and The London School of Hygiene and Tropical Medicine Interventions Research Ethics Committee (17929) on November 27, 2019. Recruitment began on January 2, 2020, and is ongoing.

CONCLUSIONS

Understanding how push notifications may impact engagement with a behavior change app can lead to further improvements in engagement, and ultimately help users reduce their alcohol consumption. This understanding may also be generalizable to other apps that target a variety of behavior changes.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/18690.

Methodological Challenges in Web-Based Trials: Update and Insights From the Relatives Education and Coping Toolkit Trial

RR2-10.1136/bmjopen-2017-016965.

Cognitive-Behavioral Therapy in the Digital Age: Presidential Address

Our field has come a long way in establishing cognitive-behavioral therapy as the empirically supported treatment of choice for a wide range of mental and behavioral health problems. Nevertheless, most individuals with mental disorders do not receive any care at all, and those who do often have difficulty accessing care that is consistently high in quality. Addressing these issues is complex and costly and thus progress has been slow. We are entering an exciting stage in which emerging technologies might offer novel solutions to the treatment gap. This paper discusses a number of technology-enabled solutions to our field's challenges, including Internet-based and smartphone-based cognitive-behavioral therapy. Nevertheless, we must remain attentive to potential pitfalls of these emerging technologies. The paper incorporates suggestions for how the field may approach these potential pitfalls and provides a vision for how we might develop powerful, scalable, precisely timed, personalized interventions to enhance global mental health.

Do Daily Fluctuations in Psychological and App-Related Variables Predict Engagement With an Alcohol Reduction App? A Series of N-Of-1 Studies

BACKGROUND

Previous studies have identified psychological and smartphone app-related predictors of engagement with alcohol reduction apps at a group level. However, strategies to promote engagement need to be effective at the individual level. Evidence as to whether group-level predictors of engagement are also predictive for individuals is lacking.

OBJECTIVE

The aim of this study was to examine whether daily fluctuations in (1) the receipt of a reminder, (2) motivation to reduce alcohol, (3) perceived usefulness of the app, (4) alcohol consumption, and (5) perceived lack of time predicted within-person variability in the frequency and amount of engagement with an alcohol reduction app.

METHODS

We conducted a series of observational N-of-1 studies. The predictor variables were measured twice daily for 28 days via ecological momentary assessments. The outcome variables were measured through automated recordings of the participants' app screen views. A total of nine London-based adults who drank alcohol excessively and were willing to set a reduction goal took part. Each participant's dataset was analyzed separately using generalized additive mixed models to derive incidence rate ratios (IRRs) for the within-person associations of the predictor and outcome variables. Debriefing interviews, analyzed using thematic analysis, were used to contextualize the findings.

RESULTS

Predictors of the frequency and amount of engagement differed between individuals, and for the variables 'perceived usefulness of the app' and 'perceived lack of time', the direction of associations also differed between individuals. The most consistent predictors of within-person variability in the frequency of engagement were the receipt of a daily reminder (IRR=1.80-3.88; P<.05) and perceived usefulness of the app (IRR=0.82-1.42; P<.05). The most consistent predictors of within-person variability in the amount of engagement were motivation to reduce alcohol (IRR=1.67-3.45; P<.05) and perceived usefulness of the app (IRR=0.52-137.32; P<.05).

CONCLUSIONS

The utility of the selected psychological and app-related variables in predicting the frequency and amount of engagement with an alcohol reduction app differed at the individual level. This highlights that key within-person associations may be masked in group-level designs and suggests that different strategies to promote engagement may be required for different individuals.

Updating the evidence on the effectiveness of the alcohol reduction app, Drink Less: using Bayes factors to analyse trial datasets supplemented with extended recruitment

Background: A factorial experiment evaluating the Drink Less app found no clear evidence for main effects of enhanced versus minimal versions of five components but some evidence for an interaction effect. Bayes factors (BFs) showed the data to be insensitive. This study examined the use of BFs to update the evidence with further recruitment. Methods: A between-subject factorial experiment evaluated the main and two-way interaction effects of enhanced versus minimal version of five components of Drink Less. Participants were excessive drinkers, aged 18+, and living in the UK. After the required sample size was reached (n=672), additional data were collected for five months. Outcome measures were change in past week alcohol consumption and Alcohol Use Disorders Identification Test (AUDIT) score at one-month follow-up, amongst responders only (those who completed the questionnaire). BFs (with a half-normal distribution) were calculated (BF<0.33 indicate evidence for null hypothesis; 0.33 Collapse

Key Words

• Bayes Factors
• alcohol reduction
• digital interventions
• smartphone apps

Collapse

MESH Headings

• Adolescent
• Alcohol Abstinence
• Alcohol Drinking
• Alcoholism/therapy
• Bayes Theorem
• Female
• Humans
• Male
• Mobile Applications
• Outcome Assessment, Health Care
• Surveys and Questionnaires

Collapse

Grants

• 22962 Cancer Research UK
• MR/K023195/1 Medical Research Council
• C1417/A22962 Cancer Research UK

Collapse

Affiliation(s)

Claire Garnett Department of Behavioural Science and Health, University College London, London, UK
Susan Michie Department of Clinical, Education and Health Psychology, University College London, London, UK
Robert West Department of Behavioural Science and Health, University College London, London, UK
Jamie Brown Department of Behavioural Science and Health, University College London, London, UK

Collapse