Sample size calculations for micro-randomized trials in mHealth

Toward Increasing Engagement in Substance Use Data Collection: Development of the Substance Abuse Research Assistant App and Protocol for a Microrandomized Trial Using Adolescents and Emerging Adults

Background

Substance use is an alarming public health issue associated with significant morbidity and mortality. Adolescents and emerging adults are at particularly high risk because substance use typically initiates and peaks during this developmental period. Mobile health apps are a promising data collection and intervention delivery tool for substance-using youth as most teens and young adults own a mobile phone. However, engagement with data collection for most mobile health applications is low, and often, large fractions of users stop providing data after a week of use.

Objective

Substance Abuse Research Assistant (SARA) is a mobile application to increase or sustain engagement of substance data collection overtime. SARA provides a variety of engagement strategies to incentivize data collection: a virtual aquarium in the app grows with fish and aquatic resources; occasionally, funny or inspirational contents (eg, memes or text messages) are provided to generate positive emotions. We plan to assess the efficacy of SARA’s engagement strategies over time by conducting a micro-randomized trial, where the engagement strategies will be sequentially manipulated.

Methods

We aim to recruit participants (aged 14-24 years), who report any binge drinking or marijuana use in the past month. Participants are instructed to use SARA for 1 month. During this period, participants are asked to complete one survey and two active tasks every day between 6 pm and midnight. Through the survey, we assess participants’ daily mood, stress levels, loneliness, and hopefulness, while through the active tasks, we measure reaction time and spatial memory. To incentivize and support the data collection, a variety of engagement strategies are used. First, predata collection strategies include the following: (1) at 4 pm, a push notification may be issued with an inspirational message from a contemporary celebrity; or (2) at 6 pm, a push notification may be issued reminding about data collection and incentives. Second, postdata collection strategies include various rewards such as points which can be used to grow a virtual aquarium with fishes and other treasures and modest monetary rewards (up to US $12; US $1 for each 3-day streak); also, participants may receive funny or inspirational content as memes or gifs or visualizations of prior data. During the study, the participants will be randomized every day to receive different engagement strategies. In the primary analysis, we will assess whether issuing 4 pm push-notifications or memes or gifs, respectively, increases self-reporting on the current or the following day.

Results

The microrandomized trial started on August 21, 2017 and the trial ended on February 28, 2018. Seventy-three participants were recruited. Data analysis is currently underway.

Conclusions

To the best of our knowledge, SARA is the first mobile phone app that systematically manipulates engagement strategies in order to identify the best sequence of strategies that keep participants engaged in data collection. Once the optimal strategies to collect data are identified, future versions of SARA will use this data to provide just-in-time adaptive interventions to reduce substance use among youth.

Trial Registration

ClinicalTrials.gov NCT03255317; https://clinicaltrials.gov/show/NCT03255317 (Archived by WebCite at http://www.webcitation.org/70raGWV0e)

Registered Report Identifier

RR1-10.2196/9850

Optimizing Digital Integrated Care via Micro-Randomized Trials

Mobile health (mHealth) interventions are a promising tool in providing digitally mediated integrative care. They can extend care outside of the clinic by providing reminders to take medications, assisting in managing symptoms, and supporting healthy behaviors including physical activity, healthy eating, and stress management. mHealth interventions can adapt the delivery of care across time in order to optimize treatment effectiveness. Yet there exists limited empirical evidence useful to the development of adaptive mHealth interventions. This article describes a new randomized trial design, the Micro-Randomized Trial (MRT), for informing the development of mHealth interventions. We provide examples of scientific questions important to the development of an mHealth intervention, and describe how these questions can be answered using an MRT.

Engagement Strategies for Self-Monitoring Symptoms of Bipolar Disorder With Mobile and Wearable Technology: Protocol for a Randomized Controlled Trial

Background

Monitoring signs and symptoms in bipolar disorder (BP) is typically based on regular assessments from patient-clinician interactions. Mobile and wearable technology promises to make monitoring symptoms in BP easier, but little is known about how best to engage individuals with BP in monitoring symptoms.

Objective

The objective of this study was to provide the rationale and protocol for a randomized controlled trial that investigates engagement strategies for monitoring symptoms of BP, including the strategies of using activity trackers compared with self-reports and reviewing recorded symptoms weekly with an interviewer.

Methods

A total of 50 individuals with BP will be recruited from the Prechter Longitudinal Study of Bipolar Disorder at the University of Michigan to participate in a 6-week study. Participants will monitor their symptoms through an activity tracker (Fitbit Alta HR) and a mobile phone app designed for this study. In addition to monitoring symptoms, participants have a 50-50 chance of being assigned to an arm that reviews self-reports and activity information weekly. Statistical tests will be performed to test hypotheses that participants adhere to activity tracking significantly more than self-reporting, prefer activity tracking significantly more than self-reporting, and better adhere to both activity tracking and self-reporting when reviewing collected information weekly.

Results

Recruitment commenced in November 2017. The first group of participants began the study in January 2018.

Conclusions

This study aims to establish strategies to engage individuals with BP in monitoring their symptoms with mobile and wearable technology. Better engagement strategies are expected to aid current efforts in bipolar research and clinical care, from the development of new mobile phone apps to providing the right intervention to the right individual at the right moment.

Trial Registration

ClinicalTrials.gov NCT03358238; https://clinicaltrials.gov/ct2/show/NCT03358238 (Archived by WebCite at http://www.webcitation.org/6yebuNfz5)

Registered Report Identifier

RR1-10.2196/9899

Using Behavioral Analytics to Increase Exercise: A Randomized N-of-1 Study

INTRODUCTION

This intervention study used mobile technologies to investigate whether those randomized to receive a personalized "activity fingerprint" (i.e., a one-time tailored message about personal predictors of exercise developed from 6 months of observational data) increased their physical activity levels relative to those not receiving the fingerprint.

STUDY DESIGN

A 12-month randomized intervention study.

SETTING/PARTICIPANTS

From 2014 to 2015, 79 intermittent exercisers had their daily physical activity assessed by accelerometry (Fitbit Flex) and daily stress experience, a potential predictor of exercise behavior, was assessed by smartphone.

INTERVENTION

Data collected during the first 6 months of observation were used to develop a person-specific "activity fingerprint" (i.e., N-of-1) that was subsequently sent via email on a single occasion to randomized participants.

MAIN OUTCOME MEASURES

Pre-post changes in the percentage of days exercised were analyzed within and between control and intervention groups.

RESULTS

The control group significantly decreased their proportion of days exercised (10.5% decrease, p<0.0001) following randomization. By contrast, the intervention group showed a nonsignificant decrease in the proportion of days exercised (4.0% decrease, p=0.14). Relative to the decrease observed in the control group, receipt of the activity fingerprint significantly increased the likelihood of exercising in the intervention group (6.5%, p=0.04).

CONCLUSIONS

This N-of-1 intervention study demonstrates that a one-time brief message conveying personalized exercise predictors had a beneficial effect on exercise behavior among urban adults.

Evaluating the Impact of Physical Activity Apps and Wearables: Interdisciplinary Review

Background

Although many smartphone apps and wearables have been designed to improve physical activity, their rapidly evolving nature and complexity present challenges for evaluating their impact. Traditional methodologies, such as randomized controlled trials (RCTs), can be slow. To keep pace with rapid technological development, evaluations of mobile health technologies must be efficient. Rapid alternative research designs have been proposed, and efficient in-app data collection methods, including in-device sensors and device-generated logs, are available. Along with effectiveness, it is important to measure engagement (ie, users’ interaction and usage behavior) and acceptability (ie, users’ subjective perceptions and experiences) to help explain how and why apps and wearables work.

Objectives

This study aimed to (1) explore the extent to which evaluations of physical activity apps and wearables: employ rapid research designs; assess engagement, acceptability, as well as effectiveness; use efficient data collection methods; and (2) describe which dimensions of engagement and acceptability are assessed.

Method

An interdisciplinary scoping review using 8 databases from health and computing sciences. Included studies measured physical activity, and evaluated physical activity apps or wearables that provided sensor-based feedback. Results were analyzed using descriptive numerical summaries, chi-square testing, and qualitative thematic analysis.

Results

A total of 1829 abstracts were screened, and 858 articles read in full. Of 111 included studies, 61 (55.0%) were published between 2015 and 2017. Most (55.0%, 61/111) were RCTs, and only 2 studies (1.8%) used rapid research designs: 1 single-case design and 1 multiphase optimization strategy. Other research designs included 23 (22.5%) repeated measures designs, 11 (9.9%) nonrandomized group designs, 10 (9.0%) case studies, and 4 (3.6%) observational studies. Less than one-third of the studies (32.0%, 35/111) investigated effectiveness, engagement, and acceptability together. To measure physical activity, most studies (90.1%, 101/111) employed sensors (either in-device [67.6%, 75/111] or external [23.4%, 26/111]). RCTs were more likely to employ external sensors (accelerometers: P=.005). Studies that assessed engagement (52.3%, 58/111) mostly used device-generated logs (91%, 53/58) to measure the frequency, depth, and length of engagement. Studies that assessed acceptability (57.7%, 64/111) most often used questionnaires (64%, 42/64) and/or qualitative methods (53%, 34/64) to explore appreciation, perceived effectiveness and usefulness, satisfaction, intention to continue use, and social acceptability. Some studies (14.4%, 16/111) assessed dimensions more closely related to usability (ie, burden of sensor wear and use, interface complexity, and perceived technical performance).

Conclusions

The rapid increase of research into the impact of physical activity apps and wearables means that evaluation guidelines are urgently needed to promote efficiency through the use of rapid research designs, in-device sensors and user-logs to assess effectiveness, engagement, and acceptability. Screening articles was time-consuming because reporting across health and computing sciences lacked standardization. Reporting guidelines are therefore needed to facilitate the synthesis of evidence across disciplines.

Action Centered Contextual Bandits

Contextual bandits have become popular as they offer a middle ground between very simple approaches based on multi-armed bandits and very complex approaches using the full power of reinforcement learning. They have demonstrated success in web applications and have a rich body of associated theoretical guarantees. Linear models are well understood theoretically and preferred by practitioners because they are not only easily interpretable but also simple to implement and debug. Furthermore, if the linear model is true, we get very strong performance guarantees. Unfortunately, in emerging applications in mobile health, the time-invariant linear model assumption is untenable. We provide an extension of the linear model for contextual bandits that has two parts: baseline reward and treatment effect. We allow the former to be complex but keep the latter simple. We argue that this model is plausible for mobile health applications. At the same time, it leads to algorithms with strong performance guarantees as in the linear model setting, while still allowing for complex nonlinear baseline modeling. Our theory is supported by experiments on data gathered in a recently concluded mobile health study.

SARA: A Mobile App to Engage Users in Health Data Collection

Despite the recent progress in sensor technologies, many relevant health data can be only captured with manual input (e.g., food intake, stress appraisal, subjective emotion, substance use). A common problem of manual logging is that users often disengage within a short time because of high burden. In this work, we propose SARA, a novel app to engage users with ongoing tracking using timely rewards thereby reinforcing users for data input. SARA is developed for adolescents and emerging adults at risk for substance abuse. The rewards in SARA are designed to be developmentally and culturally appropriate to the target demographic and are theoretically grounded in the behavioral science literature. In this paper, we describe SARA and its rewards to increase data collection. We also briefly discuss future plans to evaluate SARA and develop just in time adaptive interventions for engagement and behavior change.

Toward Usable Evidence: Optimizing Knowledge Accumulation in HCI Research on Health Behavior Change

Over the last ten years, HCI researchers have introduced a range of novel ways to support health behavior change, from glanceable displays to sophisticated game dynamics. Yet, this research has not had as much impact as its originality warrants. A key reason for this is that common forms of evaluation used in HCI make it difficult to effectively accumulate-and use-knowledge across research projects. This paper proposes a strategy for HCI research on behavior change that retains the field's focus on novel technical contributions while enabling accumulation of evidence that can increase impact of individual research projects both in HCI and the broader behavior-change science. The core of this strategy is an emphasis on the discovery of causal effects of individual components of behavior-change technologies and the precise ways in which those effects vary with individual differences, design choices, and contexts in which those technologies are used.

Assessing Time-Varying Causal Effect Moderation in Mobile Health

In mobile health interventions aimed at behavior change and maintenance, treatments are provided in real time to manage current or impending high risk situations or promote healthy behaviors in near real time. Currently there is great scientific interest in developing data analysis approaches to guide the development of mobile interventions. In particular data from mobile health studies might be used to examine effect moderators-individual characteristics, time-varying context or past treatment response that moderate the effect of current treatment on a subsequent response. This paper introduces a formal definition for moderated effects in terms of potential outcomes, a definition that is particularly suited to mobile interventions, where treatment occasions are numerous, individuals are not always available for treatment, and potential moderators might be influenced by past treatment. Methods for estimating moderated effects are developed and compared. The proposed approach is illustrated using BASICS-Mobile, a smartphone-based intervention designed to curb heavy drinking and smoking among college students.

Technology-based interventions for weight management: current randomized controlled trial evidence and future directions

Obesity is a prevalent health care issue associated with disability, premature morality, and high costs. Behavioral weight management interventions lead to clinically significant weight losses in overweight and obese individuals; however, many individuals are not able to participate in these face-to-face treatments due to limited access, cost, and/or time constraints. Technological advances such as widespread access to the Internet, increased use of smartphones, and newer behavioral self-monitoring tools have resulted in the development of a variety of eHealth weight management programs. In the present paper, a summary of the most current literature is provided along with potential solutions to methodological challenges (e.g., high attrition, minimal participant racial/ethnic diversity, heterogeneity of technology delivery modes). Dissemination and policy implications will be highlighted as future directions for the field of eHealth weight management.

Adaptive Interventions in Child and Adolescent Mental Health

The treatment or prevention of child and adolescent mental health (CAMH) disorders often requires an individualized, sequential approach to intervention, whereby treatments (or prevention efforts) are adapted over time based on the youth's evolving status (e.g., early response, adherence). Adaptive interventions are intended to provide a replicable guide for the provision of individualized sequences of interventions in actual clinical practice. Recently, there has been great interest in the development of adaptive intervenions by investigators working in CAMH. The development of such replicable, real-world, individualized sequences of decision rules to guide the treatment or prevention of CAMH disorders represents an important "next step" in interventions research. The primary purpose of this special issue is to showcase some recent work on the science of adaptive interventions in CAMH. In this overview article, we review why individualized sequences of interventions are needed in CAMH, provide an introduction to adaptive interventions, briefly describe each of the articles included in this special issue, and describe some exciting areas of ongoing and future research. A hopeful outcome of this special issue is that it encourages other researchers in CAMH to pursue creative and significant research on adaptive interventions.

Illustrating idiographic methods for translation research: moderation effects, natural clinical experiments, and complex treatment-by-subgroup interactions

A critical juncture in translation research involves the preliminary studies of intervention tools, provider training programs, policies, and other mechanisms used to leverage knowledge garnered at one translation stage into another stage. Potentially useful for such studies are rigorous techniques for conducting within-subject clinical trials, which have advanced incrementally over the last decade. However, these methods have largely not been utilized within prevention or translation contexts. The purpose of this manuscript is to demonstrate the flexibility, wide applicability, and rigor of idiographic clinical trials for preliminary testing of intervention mechanisms. Specifically demonstrated are novel uses of state-space modeling for testing intervention mechanisms of short-term outcomes, identifying heterogeneity in and moderation of within-person treatment mechanisms, a horizontal line plot to refine sampling design during the course of a clinic-based experimental study, and the need to test a treatment's efficacy as treatment is administered along with (e.g., traditional 12-month outcomes).

Randomised trials for the Fitbit generation

Data from activity trackers and mobile phones can be used to craft personalised health interventions. But measuring the efficacy of these "treatments" requires a rethink of the traditional randomised trial.

Building health behavior models to guide the development of just-in-time adaptive interventions: A pragmatic framework

Advances in wireless devices and mobile technology offer many opportunities for delivering just-in-time adaptive interventions (JITAIs)-suites of interventions that adapt over time to an individual's changing status and circumstances with the goal to address the individual's need for support, whenever this need arises. A major challenge confronting behavioral scientists aiming to develop a JITAI concerns the selection and integration of existing empirical, theoretical and practical evidence into a scientific model that can inform the construction of a JITAI and help identify scientific gaps. The purpose of this paper is to establish a pragmatic framework that can be used to organize existing evidence into a useful model for JITAI construction. This framework involves clarifying the conceptual purpose of a JITAI, namely, the provision of just-in-time support via adaptation, as well as describing the components of a JITAI and articulating a list of concrete questions to guide the establishment of a useful model for JITAI construction. The proposed framework includes an organizing scheme for translating the relatively static scientific models underlying many health behavior interventions into a more dynamic model that better incorporates the element of time. This framework will help to guide the next generation of empirical work to support the creation of effective JITAIs.

Microrandomized trials: An experimental design for developing just-in-time adaptive interventions

OBJECTIVE

This article presents an experimental design, the microrandomized trial, developed to support optimization of just-in-time adaptive interventions (JITAIs). JITAIs are mHealth technologies that aim to deliver the right intervention components at the right times and locations to optimally support individuals' health behaviors. Microrandomized trials offer a way to optimize such interventions by enabling modeling of causal effects and time-varying effect moderation for individual intervention components within a JITAI.

METHOD

The article describes the microrandomized trial design, enumerates research questions that this experimental design can help answer, and provides an overview of the data analyses that can be used to assess the causal effects of studied intervention components and investigate time-varying moderation of those effects.

RESULTS

Microrandomized trials enable causal modeling of proximal effects of the randomized intervention components and assessment of time-varying moderation of those effects.

CONCLUSION

Microrandomized trials can help researchers understand whether their interventions are having intended effects, when and for whom they are effective, and what factors moderate the interventions' effects, enabling creation of more effective JITAIs.