Assessing Real-Time Moderation for Developing Adaptive Mobile Health Interventions for Medical Interns: Micro-Randomized Trial

Personalized interventions for behaviour change: A scoping review of just-in-time adaptive interventions

PURPOSE

Examine the development, implementation and evaluation of just-in-time adaptive interventions (JITAIs) in behaviour change and evaluate the quality of intervention reporting.

METHODS

A scoping review of JITAIs incorporating mobile health (mHealth) technologies to improve health-related behaviours in adults. We searched MEDLINE, Embase and PsycINFO using terms related to JITAIs, mHealth, behaviour change and intervention methodology. Narrative analysis assessed theoretical foundations, real-time data capturing and processing methods, outcome evaluation and summarized JITAI efficacy. Quality of intervention reporting was assessed using the template for intervention description and replication (TIDieR) checklist.

RESULTS

Sixty-two JITAIs across physical activity, sedentary behaviour, dietary behaviour, substance use, sexual behaviour, fluid intake, treatment adherence, social skills, gambling behaviour and self-management skills were included. The majority (71%) aimed to evaluate feasibility, acceptability and/or usability. Supporting evidence for JITAI development was identified in 46 studies, with 67% applying this to develop tailored intervention content. Over half (55%) relied solely on self-reported data for tailoring, and 13 studies used only passive monitoring data. While data processing methods were commonly reported, 44% did not specify their techniques. 89% of JITAI designs achieved full marks on the TIDieR checklist and provided sufficient details on JITAI components. Overall, JITAIs proved to be feasible, acceptable and user-friendly across behaviours and settings. Randomized trials showed tailored interventions were efficacious, though outcomes varied by behaviour.

CONCLUSIONS

JITAIs offer a promising approach to developing personalized interventions, with their potential effects continuously growing. The recommended checklist emphasizes the importance of reporting transparency in establishing robust intervention designs.

Optimization of a Mobile Transdiagnostic Emotion Regulation Intervention for University Students: A Micro-Randomized Trial

Increasing mental health problems among university students highlight the need for scalable, effective solutions. We have developed a transdiagnostic mobile intervention called ROOM, promoting adaptive emotion regulation (ER) skills among university students. Understanding how the intervention works and optimising content and delivery is essential for creating an effective and adaptive system. Therefore, this study aimed to optimise ROOM through a sequential explanatory mixed-methods design, combining a Micro-Randomized Trial (MRT), evaluating within-person effects using repeated randomisation, with user experience interviews. 161 university students (82% females) participated in a 3-week MRT to assess the intervention proximal outcomes, that is, participants' positive and negative emotional states after completing intervention exercises. Additionally, we evaluated impact on distal outcomes (i.e., distress symptoms and ER skills), and user experience by combining objective (e.g., exercise completion rates) and subjective (e.g., exercise likability and helpfulness ratings) engagement patterns with insights from the semi-structured interviews (n = 18). Upon receiving the intervention, positive emotional states increased and negative ones decreased. The effect on positive emotional states gradually decreased over time while the effect on negative emotional states remained stable throughout the 3-week intervention period. Distress symptoms and ER skills either remained stable or improved over the 3 weeks, which indicated the intervention's safety. Overall, engagement patterns and interview data show that the intervention was well received, students enjoyed this study design and found context-sensitive content recommendations highly relevant.

Text Messages to Promote Physical Activity in Patients With Cardiovascular Disease: A Micro-Randomized Trial of a Just-In-Time Adaptive Intervention

BACKGROUND

Text messages may enhance physical activity levels in patients with cardiovascular disease, including those enrolled in cardiac rehabilitation. However, the independent and long-term effects of text messages remain uncertain.

METHODS

The VALENTINE study (Virtual Application-supported Environment to Increase Exercise) was a micro-randomized trial that delivered text messages through a smartwatch (Apple Watch or Fitbit Versa) to participants initiating cardiac rehabilitation. Participants were randomized 4× per day over 6-months to receive no text message or a message encouraging low-level physical activity. Text messages were tailored on contextual factors (eg, weather). Our primary outcome was step count 60 minutes following a text message, and we used a centered and weighted least squares mean method to estimate causal effects. Given potential measurement differences between devices determined a priori, data were assessed separately for Apple Watch and Fitbit Versa users over 3 time periods corresponding to the initiation (0-30 days), maintenance (31-120 days), and completion (121-182 days) of cardiac rehabilitation.

RESULTS

One hundred eight participants were included with 70 552 randomizations over 6 months; mean age was 59.5 (SD, 10.7) years with 36 (32.4%) female and 68 (63.0%) Apple Watch participants. For Apple Watch participants, text messages led to a trend in increased step count by 10% in the 60-minutes following a message during days 1 to 30 (95% CI, -1% to +20%), with no effect from days 31 to 120 (+1% [95% CI, -4% to +5%]), and a significant 6% increase during days 121 to 182 (95% CI, +0% to +11%). For Fitbit users, text messages significantly increased step count by 17% (95% CI, +7% to +28%) in the 60-minutes following a message in the first 30 days of the study with no effect subsequently.

CONCLUSIONS

In patients undergoing cardiac rehabilitation, contextually tailored text messages may increase physical activity, but this effect varies over time and by device.

REGISTRATION

URL: https://www.clinicaltrials.gov; Unique identifier: NCT04587882.

Comparative Effectiveness of Three Digital Interventions for Adults Seeking Psychiatric Services: A Randomized Clinical Trial

Importance

There is a substantial gap between demand for and availability of mental health services. Digital mental health interventions (DMHIs) are promising tools for bridging this gap, yet little is known about their comparative effectiveness.

Objective

To assess whether patients randomized to a cognitive behavioral therapy (CBT)-based or mindfulness-based DMHI had greater improvements in mental health symptoms than patients randomized to the enhanced personalized feedback (EPF)-only DMHI.

DESIGN,

SETTING, AND PARTICIPANTS This randomized clinical trial was conducted between May 13, 2020, and December 12, 2022, with follow-up at 6 weeks. Adult patients of outpatient psychiatry services across various clinics within the University of Michigan Health System with a scheduled or recent outpatient psychiatry appointment were recruited. Eligible patients were randomized to an intervention arm. All analyses followed the intent-to-treat principle.

Interventions

Participants were randomized to 1 of 5 intervention arms: (1) EPF only; (2) Silvercloud only, a mobile application designed to deliver CBT strategies; (3) Silvercloud plus EPF; (4) Headspace only, a mobile application designed to train users in mindfulness practices; and (5) Headspace plus EPF.

Main Outcomes and Measures

The primary outcome was change in depressive symptoms as measured by the Patient Health Questionnaire-9 (PHQ-9; score range: 0-27, with higher scores indicating greater depression symptoms). Secondary outcomes included changes in anxiety, suicidality, and substance use symptoms.

Results

A total of 2079 participants (mean [SD] age, 36.8 [14.3] years; 1423 self-identified as women [68.4%]) completed the baseline survey. The baseline mean (SD) PHQ-9 score was 12.7 (6.4) and significantly decreased for all 5 intervention arms at 6 weeks (from -2.1 [95% CI, -2.6 to -1.7] to -2.9 [95% CI, -3.4 to -2.4]; n = 1885). The magnitude of change was not significantly different across the 5 arms (F4,1879 = 1.19; P = .31). Additionally, the groups did not differ in decrease in anxiety or substance use symptoms. However, the Headspace arms reported significantly greater improvements on a suicidality measure subscale compared with the Silvercloud arms (mean difference in mean change = 0.63; 95% CI, 0.20-1.06; P = .004).

Conclusions and Relevance

This randomized clinical trial found decreases in depression and anxiety symptoms across all DMHIs and minimal evidence that specific applications were better than others. The findings suggest that DMHIs may provide support for patients during waiting list-related delays in care.

Trial Registration

ClinicalTrials.gov Identifier: NCT04342494.

Just-in-Time Adaptive Intervention for Stabilizing Sleep Hours of Japanese Workers: Microrandomized Trial

BACKGROUND

Sleep disturbance is a major contributor to future health and occupational issues. Mobile health can provide interventions that address adverse health behaviors for individuals in a vulnerable health state in real-world settings (just-in-time adaptive intervention).

OBJECTIVE

This study aims to identify a subpopulation with vulnerable sleep state in daily life (study 1) and, immediately afterward, to test whether providing mobile health intervention improved habitual sleep behaviors and psychological wellness in real-world settings by conducting a microrandomized trial (study 2).

METHODS

Japanese workers (n=182) were instructed to collect data on their habitual sleep behaviors and momentary symptoms (including depressive mood, anxiety, and subjective sleep quality) using digital devices in a real-world setting. In study 1, we calculated intraindividual mean and variability of sleep hours, midpoint of sleep, and sleep efficiency to characterize their habitual sleep behaviors. In study 2, we designed and conducted a sleep just-in-time adaptive intervention, which delivered objective push-type sleep feedback messages to improve their sleep hours for a subset of participants in study 1 (n=81). The feedback messages were generated based on their sleep data measured on previous nights and were randomly sent to participants with a 50% chance for each day (microrandomization).

RESULTS

In study 1, we applied hierarchical clustering to dichotomize the population into 2 clusters (group A and group B) and found that group B was characterized by unstable habitual sleep behaviors (large intraindividual variabilities). In addition, linear mixed-effect models showed that the interindividual variability of sleep hours was significantly associated with depressive mood (β=3.83; P=.004), anxiety (β=5.70; P=.03), and subjective sleep quality (β=-3.37; P=.03). In study 2, we found that providing sleep feedback prolonged subsequent sleep hours (increasing up to 40 min; P=.01), and this effect lasted for up to 7 days. Overall, the stability of sleep hours in study 2 was significantly improved among participants in group B compared with the participants in study 1 (P=.001).

CONCLUSIONS

This is the first study to demonstrate that providing sleep feedback can benefit the modification of habitual sleep behaviors in a microrandomized trial. The findings of this study encourage the use of digitalized health intervention that uses real-time health monitoring and personalized feedback.

The effect of cognitive behavioral therapy text messages on mood: A micro-randomized trial

The StayWell at Home intervention, a 60-day text-messaging program based on Cognitive Behavioral Therapy (CBT) principles, was developed to help adults cope with the adverse effects of the global pandemic. Participants in StayWell at Home were found to show reduced depressive and anxiety symptoms after participation. However, it remains unclear whether the intervention improved mood and which intervention components were most effective at improving user mood during the pandemic. Thus, utilizing a micro-randomized trial (MRT) design, we examined two intervention components to inform the mechanisms of action that improve mood: 1) text messages delivering CBT-informed coping strategies (i.e., behavioral activation, other coping skills, or social support); 2) time at which messages were sent. Data from two independent trials of StayWell are included in this paper. The first trial included 303 adults aged 18 or older, and the second included 266 adults aged 18 or older. Participants were recruited via online platforms (e.g., Facebook ads) and partnerships with community-based agencies aiming to reach diverse populations, including low-income individuals and people of color. The results of this paper indicate that participating in the program improved and sustained self-reported mood ratings among participants. We did not find significant differences between the type of message delivered and mood ratings. On the other hand, the results from Phase 1 indicated that delivering any type of message in the 3 pm-6 pm time window improved mood significantly over sending a message in the 9 am-12 pm time window. The StayWell at Home program increases in mood ratings appeared more pronounced during the first two to three weeks of the intervention and were maintained for the remainder of the study period. The current paper provides evidence that low-burden text-message interventions may effectively address behavioral health concerns among diverse communities.

Association between sleep health and rates of self-reported medical errors in intern physicians: an ancillary analysis of the Intern Health Study

STUDY OBJECTIVES

Reduced sleep duration and work hour variability contribute to medical error and physician burnout. This study assesses the relationships between physician performance, burnout, and the dimensions of sleep beyond hours slept.

METHODS

This was an ancillary analysis of 3 years of data from an international prospective cohort study: the Intern Health Study. Actigraphy data from 3,654 intern physicians capturing sleep timing, regularity, efficiency, and duration were used individually and combined as a composite sleep health index to measure the association of multidimensional sleep patterns on self-reported medical errors and burnout.

RESULTS

From 2017-2019, interns' work hours decreased by 4 hours per week and total sleep time also decreased (6.7 to 5.99 hours), and sleep efficiency, timing, and regularity all worsened (all P < .05). In the 21.2% of participants who committed an error, there was no difference in sleep duration, timing, or regularity. Lower sleep efficiency was associated with higher odds of committing an error (P = .003) and higher burnout scores (P < .001). Although overall sleep quality was poor in the entire cohort, interns in the lowest quintile of sleep duration, regularity, and efficiency had higher burnout scores than those in the best quintile.

CONCLUSIONS

Sleep efficiency, not duration, was associated with increased self-reported medical errors and burnout in intern physicians. Overall sleep quality and duration worsened despite fewer hours worked. Future studies on physician burnout should measure all aspects of sleep health.

CITATION

Hassinger AB, Velez C, Wang J, Mador MJ, Wilding GE, Mishra A. Association between sleep health and rates of self-reported medical errors in intern physicians: an ancillary analysis of the Intern Health Study. J Clin Sleep Med. 2024;20(2):221-227.

Adherence Promotion With Tailored Motivational Messages: Proof of Concept and Message Preferences in Older Adults

This study examined the feasibility of using tailored text messages to promote adherence to longitudinal protocols and determined what facets of text message tone influence motivation. Forty-three older adults (Mage = 73.21, SD = 5.37) were recruited to engage in video-game-based cognitive training for 10 consecutive days. Participants received encouraging text messages each morning that matched their highest or lowest ranking reasons for participating in the study, after which they rated how effective each message was in motivating them to play the games that day. After 10 days, participants rated all possible messages and participated in semi-structured interviews to elicit their preferences for these messages. Results showed that messages matching participants' reasons for participating were more motivating than mismatched messages. Further, participants preferred messages that were personalized (i.e., use second person voice) and in formal tones. Messages consistent with these preferences were also rated as more motivating. These findings establish the feasibility of using message tailoring to promote adherence to longitudinal protocols and the relevance of tailoring messages to be personal and formal.

Unraveling the interplay of circadian rhythm and sleep deprivation on mood: A Real-World Study on first-year physicians

The interplay between circadian rhythms, time awake, and mood remains poorly understood in the real-world. Individuals in high-stress occupations with irregular schedules or nighttime shifts are particularly vulnerable to depression and other mood disorders. Advances in wearable technology have provided the opportunity to study these interactions outside of a controlled laboratory environment. Here, we examine the effects of circadian rhythms and time awake on mood in first-year physicians using wearables. Continuous heart rate, step count, sleep data, and daily mood scores were collected from 2,602 medical interns across 168,311 days of Fitbit data. Circadian time and time awake were extracted from minute-by-minute wearable heart rate and motion measurements. Linear mixed modeling determined the relationship between mood, circadian rhythm, and time awake. In this cohort, mood was modulated by circadian timekeeping (p<0.001). Furthermore, we show that increasing time awake both deteriorates mood (p<0.001) and amplifies mood's circadian rhythm nonlinearly. These findings demonstrate the contributions of both circadian rhythms and sleep deprivation to underlying mood and show how these factors can be studied in real-world settings using Fitbits. They underscore the promising opportunity to harness wearables in deploying chronotherapies for psychiatric illness.

Optimization of a Transdiagnostic Mobile Emotion Regulation Intervention for University Students: Protocol for a Microrandomized Trial

BACKGROUND

Many university students experience mental health problems such as anxiety and depression. To support their mental health, a transdiagnostic mobile app intervention has been developed. The intervention provides short exercises rooted in various approaches (eg, positive psychology, mindfulness, self-compassion, and acceptance and commitment therapy) that aim to facilitate adaptive emotion regulation (ER) to help students cope with the various stressors they encounter during their time at university.

OBJECTIVE

The goals of this study are to investigate whether the intervention and its components function as intended and how participants engage with them. In addition, this study aims to monitor changes in distress symptoms and ER skills and identify relevant contextual factors that may moderate the intervention's impact.

METHODS

A sequential explanatory mixed methods design combining a microrandomized trial and semistructured interviews will be used. During the microrandomized trial, students (N=200) will be prompted via the mobile app twice a day for 3 weeks to evaluate their emotional states and complete a randomly assigned intervention (ie, an exercise supporting ER) or a control intervention (ie, a health information snippet). A subsample of participants (21/200, 10.5%) will participate in interviews exploring their user experience with the app and the completed exercises. The primary outcomes will be changes in emotional states and engagement with the intervention (ie, objective and subjective engagement). Objective engagement will be evaluated through log data (eg, exercise completion time). Subjective engagement will be evaluated through exercise likability and helpfulness ratings as well as user experience interviews. The secondary outcomes will include the distal outcomes of the intervention (ie, ER skills and distress symptoms). Finally, the contextual moderators of intervention effectiveness will be explored (eg, the time of day and momentary emotional states).

RESULTS

The study commenced on February 9, 2023, and the data collection was concluded on June 13, 2023. Of the 172 eligible participants, 161 (93.6%) decided to participate. Of these 161 participants, 137 (85.1%) completed the first phase of the study. A subsample of participants (18/172, 10.5%) participated in the user experience interviews. Currently, the data processing and analyses are being conducted.

CONCLUSIONS

This study will provide insight into the functioning of the intervention and identify areas for improvement. Furthermore, the findings will shed light on potential changes in the distal outcomes of the intervention (ie, ER skills and distress symptoms), which will be considered when designing a follow-up randomized controlled trial evaluating the full-scale effectiveness of this intervention. Finally, the results and data gathered will be used to design and train a recommendation algorithm that will be integrated into the app linking students to relevant content.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05576883; https://www.clinicaltrials.gov/study/NCT05576883.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/46603.

Identifying Design Opportunities for Adaptive mHealth Interventions That Target General Well-Being: Interview Study With Informal Care Partners

BACKGROUND

Mobile health (mHealth) interventions can deliver personalized behavioral support to users in daily contexts. These interventions have been increasingly adopted to support individuals who require low-cost and low-burden support. Prior research has demonstrated the feasibility and acceptability of an mHealth intervention app (CareQOL) designed for use with informal care partners. To further optimize the intervention delivery, we need to investigate how care partners, many of whom lack the time for self-care, react and act in response to different behavioral messages.

OBJECTIVE

The goal of this study was to understand the factors that impact care partners' decision-making and actions in response to different behavioral messages. Insights from this study will help optimize future tailored and personalized behavioral interventions.

METHODS

We conducted semistructured interviews with participants who had recently completed a 3-month randomized controlled feasibility trial of the CareQOL mHealth intervention app. Of the 36 participants from the treatment group of the randomized controlled trial, 23 (64%) participated in these interviews. To prepare for each interview, the team first selected representative behavioral messages (eg, targeting different health dimensions) and presented them to participants during the interview to probe their influence on participants' thoughts and actions. The time of delivery, self-reported perceptions of the day, and user ratings of a message were presented to the participants during the interviews to assist with recall.

RESULTS

The interview data showed that after receiving a message, participants took various actions in response to different messages. Participants performed suggested behaviors or adjusted them either immediately or in a delayed manner (eg, sometimes up to a month later). We identified 4 factors that shape the variations in user actions in response to different behavioral messages: uncertainties about the workload required to perform suggested behaviors, concerns about one's ability to routinize suggested behaviors, in-the-moment willingness and ability to plan for suggested behaviors, and overall capability to engage with the intervention.

CONCLUSIONS

Our study showed that care partners use mHealth behavioral messages differently regarding the immediacy of actions and the adaptation to suggested behaviors. Multiple factors influence people's perceptions and decisions regarding when and how to take actions. Future systems should consider these factors to tailor behavioral support for individuals and design system features to support the delay or adaptation of the suggested behaviors. The findings also suggest extending the assessment of user adherence by considering the variations in user actions on behavioral support (ie, performing suggested or adjusted behaviors immediately or in a delayed manner).

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/32842.

The Dosing of Mobile-Based Just-in-Time Adaptive Self-Management Prompts for Caregivers: Preliminary Findings From a Pilot Microrandomized Study

BACKGROUND

Caregivers of people with chronic illnesses often face negative stress-related health outcomes and are unavailable for traditional face-to-face interventions due to the intensity and constraints of their caregiver role. Just-in-time adaptive interventions (JITAIs) have emerged as a design framework that is particularly suited for interventional mobile health studies that deliver in-the-moment prompts that aim to promote healthy behavioral and psychological changes while minimizing user burden and expense. While JITAIs have the potential to improve caregivers' health-related quality of life (HRQOL), their effectiveness for caregivers remains poorly understood.

OBJECTIVE

The primary objective of this study is to evaluate the dose-response relationship of a fully automated JITAI-based self-management intervention involving personalized mobile app notifications targeted at decreasing the level of caregiver strain, anxiety, and depression. The secondary objective is to investigate whether the effectiveness of this mobile health intervention was moderated by the caregiver group. We also explored whether the effectiveness of this intervention was moderated by (1) previous HRQOL measures, (2) the number of weeks in the study, (3) step count, and (4) minutes of sleep.

METHODS

We examined 36 caregivers from 3 disease groups (10 from spinal cord injury, 11 from Huntington disease, and 25 from allogeneic hematopoietic cell transplantation) in the intervention arm of a larger randomized controlled trial (subjects in the other arm received no prompts from the mobile app) designed to examine the acceptability and feasibility of this intensive type of trial design. A series of multivariate linear models implementing a weighted and centered least squares estimator were used to assess the JITAI efficacy and effect.

RESULTS

We found preliminary support for a positive dose-response relationship between the number of administered JITAI messages and JITAI efficacy in improving caregiver strain, anxiety, and depression; while most of these associations did not meet conventional levels of significance, there was a significant association between high-frequency JITAI and caregiver strain. Specifically, administering 5-6 messages per week as opposed to no messages resulted in a significant decrease in the HRQOL score of caregiver strain with an estimate of -6.31 (95% CI -11.76 to -0.12; P=.046). In addition, we found that the caregiver groups and the participants' levels of depression in the previous week moderated JITAI efficacy.

CONCLUSIONS

This study provides preliminary evidence to support the effectiveness of the self-management JITAI and offers practical guidance for designing future personalized JITAI strategies for diverse caregiver groups.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04556591; https://clinicaltrials.gov/ct2/show/NCT04556591.

ASSESSING TIME-VARYING CAUSAL EFFECT MODERATION IN THE PRESENCE OF CLUSTER-LEVEL TREATMENT EFFECT HETEROGENEITY AND INTERFERENCE

The micro-randomized trial (MRT) is a sequential randomized experimental design to empirically evaluate the effectiveness of mobile health (mHealth) intervention components that may be delivered at hundreds or thousands of decision points. MRTs have motivated a new class of causal estimands, termed "causal excursion effects", for which semiparametric inference can be conducted via a weighted, centered least squares criterion (Boruvka et al., 2018). Existing methods assume between-subject independence and non-interference. Deviations from these assumptions often occur. In this paper, causal excursion effects are revisited under potential cluster-level treatment effect heterogeneity and interference, where the treatment effect of interest may depend on cluster-level moderators. Utility of the proposed methods is shown by analyzing data from a multi-institution cohort of first year medical residents in the United States.

How Notifications Affect Engagement With a Behavior Change App: Results From a Micro-Randomized Trial

BACKGROUND

Drink Less is a behavior change app to help higher-risk drinkers in the United Kingdom reduce their alcohol consumption. The app includes a daily notification asking users to "Please complete your drinks and mood diary," yet we did not understand the causal effect of the notification on engagement nor how to improve this component of Drink Less. We developed a new bank of 30 new messages to increase users' reflective motivation to engage with Drink Less. This study aimed to determine how standard and new notifications affect engagement.

OBJECTIVE

Our objective was to estimate the causal effect of the notification on near-term engagement, to explore whether this effect changed over time, and to create an evidence base to further inform the optimization of the notification policy.

METHODS

We conducted a micro-randomized trial (MRT) with 2 additional parallel arms. Inclusion criteria were Drink Less users who consented to participate in the trial, self-reported a baseline Alcohol Use Disorders Identification Test score of ≥8, resided in the United Kingdom, were aged ≥18 years, and reported interest in drinking less alcohol. Our MRT randomized 350 new users to test whether receiving a notification, compared with receiving no notification, increased the probability of opening the app in the subsequent hour, over the first 30 days since downloading Drink Less. Each day at 8 PM, users were randomized with a 30% probability of receiving the standard message, a 30% probability of receiving a new message, or a 40% probability of receiving no message. We additionally explored time to disengagement, with the allocation of 60% of eligible users randomized to the MRT (n=350) and 40% of eligible users randomized in equal number to the 2 parallel arms, either receiving the no notification policy (n=98) or the standard notification policy (n=121). Ancillary analyses explored effect moderation by recent states of habituation and engagement.

RESULTS

Receiving a notification, compared with not receiving a notification, increased the probability of opening the app in the next hour by 3.5-fold (95% CI 2.91-4.25). Both types of messages were similarly effective. The effect of the notification did not change significantly over time. A user being in a state of already engaged lowered the new notification effect by 0.80 (95% CI 0.55-1.16), although not significantly. Across the 3 arms, time to disengagement was not significantly different.

CONCLUSIONS

We found a strong near-term effect of engagement on the notification, but no overall difference in time to disengagement between users receiving the standard fixed notification, no notification at all, or the random sequence of notifications within the MRT. The strong near-term effect of the notification presents an opportunity to target notifications to increase "in-the-moment" engagement. Further optimization is required to improve the long-term engagement.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/18690.

Individual-level interventions for reducing occupational stress in healthcare workers

BACKGROUND

Healthcare workers can suffer from work-related stress as a result of an imbalance of demands, skills and social support at work. This may lead to stress, burnout and psychosomatic problems, and deterioration of service provision. This is an update of a Cochrane Review that was last updated in 2015, which has been split into this review and a review on organisational-level interventions.  OBJECTIVES: To evaluate the effectiveness of stress-reduction interventions targeting individual healthcare workers compared to no intervention, wait list, placebo, no stress-reduction intervention or another type of stress-reduction intervention in reducing stress symptoms.  SEARCH METHODS: We used the previous version of the review as one source of studies (search date: November 2013). We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Embase, PsycINFO, CINAHL, Web of Science and a trials register from 2013 up to February 2022.

SELECTION CRITERIA

We included randomised controlled trials (RCT) evaluating the effectiveness of stress interventions directed at healthcare workers. We included only interventions targeted at individual healthcare workers aimed at reducing stress symptoms.  DATA COLLECTION AND ANALYSIS: Review authors independently selected trials for inclusion, assessed risk of bias and extracted data. We used standard methodological procedures expected by Cochrane. We categorised interventions into ones that: 1. focus one's attention on the (modification of the) experience of stress (thoughts, feelings, behaviour);  2. focus one's attention away from the experience of stress by various means of psychological disengagement (e.g. relaxing, exercise);  3. alter work-related risk factors on an individual level; and ones that 4. combine two or more of the above.  The crucial outcome measure was stress symptoms measured with various self-reported questionnaires such as the Maslach Burnout Inventory (MBI), measured at short term (up to and including three months after the intervention ended), medium term (> 3 to 12 months after the intervention ended), and long term follow-up (> 12 months after the intervention ended).  MAIN RESULTS: This is the second update of the original Cochrane Review published in 2006, Issue 4. This review update includes 89 new studies, bringing the total number of studies in the current review to 117 with a total of 11,119 participants randomised.  The number of participants per study arm was ≥ 50 in 32 studies. The most important risk of bias was the lack of blinding of participants.  Focus on the experience of stress versus no intervention/wait list/placebo/no stress-reduction intervention Fifty-two studies studied an intervention in which one's focus is on the experience of stress. Overall, such interventions may result in a reduction in stress symptoms in the short term (standardised mean difference (SMD) -0.37, 95% confidence interval (CI) -0.52 to -0.23; 41 RCTs; 3645 participants; low-certainty evidence) and medium term (SMD -0.43, 95% CI -0.71 to -0.14; 19 RCTs; 1851 participants; low-certainty evidence). The SMD of the short-term result translates back to 4.6 points fewer on the MBI-emotional exhaustion scale (MBI-EE, a scale from 0 to 54). The evidence is very uncertain (one RCT; 68 participants, very low-certainty evidence) about the long-term effect on stress symptoms of focusing one's attention on the experience of stress. Focus away from the experience of stress versus no intervention/wait list/placebo/no stress-reduction intervention  Forty-two studies studied an intervention in which one's focus is away from the experience of stress. Overall, such interventions may result in a reduction in stress symptoms in the short term (SMD -0.55, 95 CI -0.70 to -0.40; 35 RCTs; 2366 participants; low-certainty evidence) and medium term (SMD -0.41 95% CI -0.79 to -0.03; 6 RCTs; 427 participants; low-certainty evidence). The SMD on the short term translates back to 6.8 fewer points on the MBI-EE. No studies reported the long-term effect. Focus on work-related, individual-level factors versus no intervention/no stress-reduction intervention Seven studies studied an intervention in which the focus is on altering work-related factors. The evidence is very uncertain about the short-term effects (no pooled effect estimate; three RCTs; 87 participants; very low-certainty evidence) and medium-term effects and long-term effects (no pooled effect estimate; two RCTs; 152 participants, and one RCT; 161 participants, very low-certainty evidence) of this type of stress management intervention.  A combination of individual-level interventions versus no intervention/wait list/no stress-reduction intervention Seventeen studies studied a combination of interventions. In the short-term, this type of intervention may result in a reduction in stress symptoms (SMD -0.67 95%, CI -0.95 to -0.39; 15 RCTs; 1003 participants; low-certainty evidence). The SMD translates back to 8.2 fewer points on the MBI-EE. On the medium term, a combination of individual-level interventions may result in a reduction in stress symptoms, but the evidence does not exclude no effect (SMD -0.48, 95% CI -0.95 to 0.00; 6 RCTs; 574 participants; low-certainty evidence). The evidence is very uncertain about the long term effects of a combination of interventions on stress symptoms (one RCT, 88 participants; very low-certainty evidence). Focus on stress versus other intervention type  Three studies compared focusing on stress versus focusing away from stress and one study a combination of interventions versus focusing on stress. The evidence is very uncertain about which type of intervention is better or if their effect is similar.

AUTHORS' CONCLUSIONS

Our review shows that there may be an effect on stress reduction in healthcare workers from individual-level stress interventions, whether they focus one's attention on or away from the experience of stress. This effect may last up to a year after the end of the intervention. A combination of interventions may be beneficial as well, at least in the short term. Long-term effects of individual-level stress management interventions remain unknown. The same applies for interventions on (individual-level) work-related risk factors. The bias assessment of the studies in this review showed the need for methodologically better-designed and executed studies, as nearly all studies suffered from poor reporting of the randomisation procedures, lack of blinding of participants and lack of trial registration. Better-designed trials with larger sample sizes are required to increase the certainty of the evidence. Last, there is a need for more studies on interventions which focus on work-related risk factors.

Effectiveness of gamified team competition as mHealth intervention for medical interns: a cluster micro-randomized trial

Gamification, the application of gaming elements to increase enjoyment and engagement, has the potential to improve the effectiveness of digital health interventions, while the effectiveness of competition gamification components remains poorly understood on residency. To address this gap, we evaluate the effect of smartphone-based gamified team competition intervention on daily step count and sleep duration via a micro-randomized trial on medical interns. Our aim is to assess potential improvements in the factors (namely step count and sleep) that may help interns cope with stress and improve well-being. In 1779 interns, team competition intervention significantly increases the mean daily step count by 105.8 steps (SE 35.8, p = 0.03) relative to the no competition arm, while does not significantly affect the mean daily sleep minutes (p = 0.76). Moderator analyses indicate that the causal effects of competition on daily step count and sleep minutes decreased by 14.5 steps (SE 10.2, p = 0.16) and 1.9 minutes (SE 0.6, p = 0.003) for each additional week-in-study, respectively. Intra-institutional competition negatively moderates the causal effect of competition upon daily step count by -90.3 steps (SE 86.5, p = 0.30). Our results show that gamified team competition delivered via mobile app significantly increases daily physical activity which suggests that team competition can function as a mobile health intervention tool to increase short-term physical activity levels for medical interns. Future improvements in strategies of forming competition opponents and introducing occasional competition breaks may improve the overall effectiveness.

Microrandomized Trials: Developing Just-in-Time Adaptive Interventions for Better Public Health

Just-in-time adaptive interventions (JITAIs) represent an intervention design that adapts the provision and type of support over time to an individual's changing status and contexts, intending to deliver the right support on the right occasion. As a novel strategy for delivering mobile health interventions, JITAIs have the potential to improve access to quality care in underserved communities and, thus, alleviate health disparities, a significant public health concern. Valid experimental designs and analysis methods are required to inform the development of JITAIs. Here, we briefly review the cutting-edge design of microrandomized trials (MRTs), covering both the classical MRT design and its outcome-adaptive counterpart. Associated statistical challenges related to the design and analysis of MRTs are also discussed. Two case studies are provided to illustrate the aforementioned concepts and designs throughout the article. We hope our work leads to better design and application of JITAIs, advancing public health research and practice. (Am J Public Health. 2023;113(1):60-69. https://doi.org/10.2105/AJPH.2022.307150).

A caveat to using wearable sensor data for COVID-19 detection: The role of behavioral change after receipt of test results

BACKGROUND

Recent studies indicate that wearable sensors can capture subtle within-person changes caused by SARS-CoV-2 infection and play a role in detecting COVID-19 infections. However, in addition to direct effects of infection, wearable sensor data may capture changes in behavior after the receipt of COVID test results. At present, it remains unclear to what extent the observed discriminative performance of the wearable sensor data is affected by behavioral changes upon receipt of the test results.

METHODS

We conducted a retrospective study of wearable sensor data in a sample of medical interns who had symptoms and received COVID-19 test results from March to December 2020, and calculated wearable sensor metrics incorporating changes in step, sleep, and resting heart rate for interns who tested positive (cases, n = 22) and negative (controls, n = 83) after symptom onset. All these interns had wearable sensor data available for > 50% of the days in pre- and post-symptom onset periods. We assessed discriminative accuracy of the metrics via area under the curve (AUC) and tested the impact of behavior changes after receiving test results by comparing AUCs of three models: all data, pre-test-result-only data, and post-test-result-only data.

RESULTS

Wearable sensor metrics differentiated between symptomatic COVID-19 positive and negative individuals with good accuracy (AUC = 0.75). However, the discriminative capacity of the model with pre-test-result-only data substantially decreased (AUC from 0.75 to 0.63; change = -0.12, p = 0.013). The model with post-test-result-only data did not produce similar reductions in discriminative capacity.

CONCLUSIONS

Changes in wearable sensor data, especially physical activity and sleep, are robust indicators of COVID-19 infection, though they may be reflective of a person's behavior change after receiving a positive test result as opposed to a physiological signature of the virus. Thus, wearable sensor data could facilitate the monitoring of COVID-19 prevalence, but not yet replace SARS-CoV-2 testing.

An app-based just-in-time-adaptive self-management intervention for care partners: The CareQOL feasibility pilot study

PURPOSE/OBJECTIVE

The primary objective of this study was to establish the feasibility and acceptability of an intensive data collection protocol that involves the delivery of a personalized just-in-time adaptive intervention (JITAI) in three distinct groups of care partners (care partners of persons with spinal cord injury [SCI], Huntington's disease [HD], or hematopoietic cell transplantation [HCT]).

RESEARCH METHOD/DESIGN

Seventy care partners were enrolled in this study (n = 19 SCI; n = 21 HD, n = 30 HCT). This three-month (90 day) randomized control trial involved wearing a Fitbit to track sleep and steps, providing daily reports of health-related quality of life (HRQOL), and completing end of month HRQOL surveys. Care partners in the JITAI group also received personalized pushes (i.e., text-based phone notifications that include brief tips or suggestions for improving self-care). At the end of three-months, care partners in both groups completed a feasibility and acceptability questionnaire.

RESULTS

Most (98.6%) care partners completed the study, average compliance was 88% for daily HRQOL surveys, 96% for daily steps, and 85% for daily sleep (from wearing the Fitbit), and all monthly surveys were completed with the exception of one missed 3-month assessment. The acceptability of the protocol was high; ratings exceeded 80% agreement for the different elements of the study. Improvements were seen for the majority of the HRQOL measures. There was no evidence of measurement reactivity.

CONCLUSIONS/IMPLICATIONS

Findings provide strong support for the acceptability and feasibility of an intensive data collection protocol that involved the administration of a JITAI. Although this trial was not powered to establish efficacy, findings indicated improvements across a variety of different HRQOL measures (~1/3 of which were statistically significant). (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Utilizing daily mood diaries and wearable sensor data to predict depression and suicidal ideation among medical interns

BACKGROUND

Intensive longitudinal methods (ILMs) for collecting self-report (e.g., daily diaries, ecological momentary assessment) and passive data from smartphones and wearable sensors provide promising avenues for improved prediction of depression and suicidal ideation (SI). However, few studies have utilized ILMs to predict outcomes for at-risk, non-clinical populations in real-world settings.

METHODS

Medical interns (N = 2881; 57 % female; 58 % White) were recruited from over 300 US residency programs. Interns completed a pre-internship assessment of depression, were given Fitbit wearable devices, and provided daily mood ratings (scale: 1-10) via mobile application during the study period. Three-step hierarchical logistic regressions were used to predict depression and SI at the end of the first quarter utilizing pre-internship predictors in step 1, Fitbit sleep/step features in step 2, and daily diary mood features in step 3.

RESULTS

Passively collected Fitbit features related to sleep and steps had negligible predictive validity for depression, and no incremental predictive validity for SI. However, mean-level and variability in mood scores derived from daily diaries were significant independent predictors of depression and SI, and significantly improved model accuracy.

LIMITATIONS

Work schedules for interns may result in sleep and activity patterns that differ from typical associations with depression or SI. The SI measure did not capture intent or severity.

CONCLUSIONS

Mobile self-reporting of daily mood improved the prediction of depression and SI during a meaningful at-risk period under naturalistic conditions. Additional research is needed to guide the development of adaptive interventions among vulnerable populations.

Digital Phenotyping in Health Using Machine Learning Approaches: Scoping Review

BACKGROUND

Digital phenotyping is the real-time collection of individual-level active and passive data from users in naturalistic and free-living settings via personal digital devices, such as mobile phones and wearable devices. Given the novelty of research in this field, there is heterogeneity in the clinical use cases, types of data collected, modes of data collection, data analysis methods, and outcomes measured.

OBJECTIVE

The primary aim of this scoping review was to map the published research on digital phenotyping and to outline study characteristics, data collection and analysis methods, machine learning approaches, and future implications.

METHODS

We utilized an a priori approach for the literature search and data extraction and charting process, guided by the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-analyses Extension for Scoping Reviews). We identified relevant studies published in 2020, 2021, and 2022 on PubMed and Google Scholar using search terms related to digital phenotyping. The titles, abstracts, and keywords were screened during the first stage of the screening process, and the second stage involved screening the full texts of the shortlisted articles. We extracted and charted the descriptive characteristics of the final studies, which were countries of origin, study design, clinical areas, active and/or passive data collected, modes of data collection, data analysis approaches, and limitations.

RESULTS

A total of 454 articles on PubMed and Google Scholar were identified through search terms associated with digital phenotyping, and 46 articles were deemed eligible for inclusion in this scoping review. Most studies evaluated wearable data and originated from North America. The most dominant study design was observational, followed by randomized trials, and most studies focused on psychiatric disorders, mental health disorders, and neurological diseases. A total of 7 studies used machine learning approaches for data analysis, with random forest, logistic regression, and support vector machines being the most common.

CONCLUSIONS

Our review provides foundational as well as application-oriented approaches toward digital phenotyping in health. Future work should focus on more prospective, longitudinal studies that include larger data sets from diverse populations, address privacy and ethical concerns around data collection from consumer technologies, and build "digital phenotypes" to personalize digital health interventions and treatment plans.

Improving outcomes for care partners of persons with traumatic brain injury: Protocol for a randomized control trial of a just-in-time-adaptive self-management intervention

Informal family care partners of persons with traumatic brain injury (TBI) often experience intense stress resulting from their caregiver role. As such, there is a need for low burden, and easy to engage in interventions to improve health-related quality of life (HRQOL) for these care partners. This study is designed to evaluate the effectiveness of a personalized just-in-time adaptive intervention (JITAI) aimed at improving the HRQOL of care partners. Participants are randomized either to a control group, where they wear the Fitbit® and provide daily reports of HRQOL over a six-month (180 day) period (without the personalized feedback), or the JITAI group, where they wear the Fitbit®, provide daily reports of HRQOL and receive personalized self-management pushes for 6 months. 240 participants will be enrolled (n = 120 control group; n = 120 JITAI group). Outcomes are collected at baseline, 1-, 2-, 3-, 4-, 5- & 6-months, as well as 3- and 6-months post intervention. We hypothesize that the care partners who receive the intervention (JITAI group) will show improvements in caregiver strain (primary outcome) and mental health (depression and anxiety) after the 6-month (180 day) home monitoring period. Participant recruitment for this study started in November 2020. Data collection efforts should be completed by spring 2025; results are expected by winter 2025. At the conclusion of this randomized control trial, we will be able to identify care partners at greatest risk for negative physical and mental health outcomes, and will have demonstrated the efficacy of this JITAI intervention to improve HRQOL for these care partners. Trial registration: ClinicalTrial.gov NCT04570930; https://clinicaltrials.gov/ct2/show/NCT04570930.

Engagement With Personalized Feedback for Emotional Distress Among College Students at Elevated Suicide Risk

Depression and suicidal ideation have substantially increased among college students, yet many students with clinically significant symptoms do not perceive their distress as warranting mental health services. Personalized feedback (PF) interventions deliver objective data, often electronically, comparing an individual's reported symptoms or behaviors to a group norm. Several studies have shown promise for PF interventions in the context of mood and depression, yet little is known regarding how, and for whom, mood-focused PF interventions might be best deployed. The primary aim of this study was to examine the sociodemographic, clinical, and treatment-seeking factors associated with reviewing PF reports on emotional distress among college students (N = 1,673) screening positive for elevated suicide risk and not receiving mental health treatment. Results indicated that PF engagement was greatest among those with higher depression scores, and those reporting privacy/stigma concerns as barriers to treatment. Sexual minority students were more likely to review their PF than heterosexual students. Taken together, PF interventions may be a useful tool for engaging those with greater clinical acuity, and those hesitant to seek in-person care. Further research is warranted to examine the circumstances in which PF interventions might be used in isolation, or as part of a multitiered intervention strategy.

Risk Factors for COVID-19 in College Students Identified by Physical, Mental, and Social Health Reported During the Fall 2020 Semester: Observational Study Using the Roadmap App and Fitbit Wearable Sensors

BACKGROUND

The COVID-19 pandemic triggered a seismic shift in education to web-based learning. With nearly 20 million students enrolled in colleges across the United States, the long-simmering mental health crisis in college students was likely further exacerbated by the pandemic.

OBJECTIVE

This study leveraged mobile health (mHealth) technology and sought to (1) characterize self-reported outcomes of physical, mental, and social health by COVID-19 status; (2) assess physical activity through consumer-grade wearable sensors (Fitbit); and (3) identify risk factors associated with COVID-19 positivity in a population of college students prior to release of the vaccine.

METHODS

After completing a baseline assessment (ie, at Time 0 [T0]) of demographics, mental, and social health constructs through the Roadmap 2.0 app, participants were instructed to use the app freely, wear the Fitbit, and complete subsequent assessments at T1, T2, and T3, followed by a COVID-19 assessment of history and timing of COVID-19 testing and diagnosis (T4: ~14 days after T3). Continuous measures were described using mean (SD) values, while categorical measures were summarized as n (%) values. Formal comparisons were made on the basis of COVID-19 status. The multivariate model was determined by entering all statistically significant variables (P<.05) in univariable associations at once and then removing one variable at a time through backward selection until the optimal model was obtained.

RESULTS

During the fall 2020 semester, 1997 participants consented, enrolled, and met criteria for data analyses. There was a high prevalence of anxiety, as assessed by the State Trait Anxiety Index, with moderate and severe levels in 465 (24%) and 970 (49%) students, respectively. Approximately one-third of students reported having a mental health disorder (n=656, 33%). The average daily steps recorded in this student population was approximately 6500 (mean 6474, SD 3371). Neither reported mental health nor step count were significant based on COVID-19 status (P=.52). Our analyses revealed significant associations of COVID-19 positivity with the use of marijuana and alcohol (P=.02 and P=.046, respectively) and with lower belief in public health measures (P=.003). In addition, graduate students were less likely and those with ≥20 roommates were more likely to report a COVID-19 diagnosis (P=.009).

CONCLUSIONS

Mental health problems were common in this student population. Several factors, including substance use, were associated with the risk of COVID-19. These data highlight important areas for further attention, such as prioritizing innovative strategies that address health and well-being, considering the potential long-term effects of COVID-19 on college students.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04766788; https://clinicaltrials.gov/ct2/show/NCT04766788.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/29561.

An Ethics Checklist for Digital Health Research in Psychiatry: Viewpoint

BACKGROUND

Psychiatry has long needed a better and more scalable way to capture the dynamics of behavior and its disturbances, quantitatively across multiple data channels, at high temporal resolution in real time. By combining 24/7 data-on location, movement, email and text communications, and social media-with brain scans, genetics, genomics, neuropsychological batteries, and clinical interviews, researchers will have an unprecedented amount of objective, individual-level data. Analyzing these data with ever-evolving artificial intelligence could one day include bringing interventions to patients where they are in the real world in a convenient, efficient, effective, and timely way. Yet, the road to this innovative future is fraught with ethical dilemmas as well as ethical, legal, and social implications (ELSI).

OBJECTIVE

The goal of the Ethics Checklist is to promote careful design and execution of research. It is not meant to mandate particular research designs; indeed, at this early stage and without consensus guidance, there are a range of reasonable choices researchers may make. However, the checklist is meant to make those ethical choices explicit, and to require researchers to give reasons for their decisions related to ELSI issues. The Ethics Checklist is primarily focused on procedural safeguards, such as consulting with experts outside the research group and documenting standard operating procedures for clearly actionable data (eg, expressed suicidality) within written research protocols.

METHODS

We explored the ELSI of digital health research in psychiatry, with a particular focus on what we label "deep phenotyping" psychiatric research, which combines the potential for virtually boundless data collection and increasingly sophisticated techniques to analyze those data. We convened an interdisciplinary expert stakeholder workshop in May 2020, and this checklist emerges out of that dialogue.

RESULTS

Consistent with recent ELSI analyses, we find that existing ethical guidance and legal regulations are not sufficient for deep phenotyping research in psychiatry. At present, there are regulatory gaps, inconsistencies across research teams in ethics protocols, and a lack of consensus among institutional review boards on when and how deep phenotyping research should proceed. We thus developed a new instrument, an Ethics Checklist for Digital Health Research in Psychiatry ("the Ethics Checklist"). The Ethics Checklist is composed of 20 key questions, subdivided into 6 interrelated domains: (1) informed consent; (2) equity, diversity, and access; (3) privacy and partnerships; (4) regulation and law; (5) return of results; and (6) duty to warn and duty to report.

CONCLUSIONS

Deep phenotyping research offers a vision for vastly more effective care for people with, or at risk for, psychiatric disease. The potential perils en route to realizing this vision are significant; however, and researchers must be willing to address the questions in the Ethics Checklist before embarking on each leg of the journey.

A method for characterizing daily physiology from widely used wearables

Millions of wearable-device users record their heart rate (HR) and activity. We introduce a statistical method to extract and track six key physiological parameters from these data, including an underlying circadian rhythm in HR (CRHR), the direct effects of activity, and the effects of meals, posture, and stress through hormones like cortisol. We test our method on over 130,000 days of real-world data from medical interns on rotating shifts, showing that CRHR dynamics are distinct from those of sleep-wake or physical activity patterns and vary greatly among individuals. Our method also estimates a personalized phase-response curve of CRHR to activity for each individual, representing a passive and personalized determination of how human circadian timekeeping continually changes due to real-world stimuli. We implement our method in the "Social Rhythms" iPhone and Android app, which anonymously collects data from wearable-device users and provides analysis based on our method.

Power Constrained Bandits

Contextual bandits often provide simple and effective personalization in decision making problems, making them popular tools to deliver personalized interventions in mobile health as well as other health applications. However, when bandits are deployed in the context of a scientific study-e.g. a clinical trial to test if a mobile health intervention is effective-the aim is not only to personalize for an individual, but also to determine, with sufficient statistical power, whether or not the system's intervention is effective. It is essential to assess the effectiveness of the intervention before broader deployment for better resource allocation. The two objectives are often deployed under different model assumptions, making it hard to determine how achieving the personalization and statistical power affect each other. In this work, we develop general meta-algorithms to modify existing algorithms such that sufficient power is guaranteed while still improving each user's well-being. We also demonstrate that our meta-algorithms are robust to various model mis-specifications possibly appearing in statistical studies, thus providing a valuable tool to study designers.

Testing Digital Methods of Patient-Reported Outcomes Data Collection: A prospective, cluster randomized trial to test text messaging and mobile surveys. (Preprint)

Background

Health care delivery continues to evolve, with an effort being made to create patient-centered care models using patient-reported outcomes (PROs) data. Collecting PROs has remained challenging and an expanding landscape of digital health offers a variety of methods to engage patients.

Objective

The aim of this study is to prospectively investigate two common methods of remote PRO data collection. The study sought to compare response and engagement rates for bidirectional SMS text messaging and mobile surveys following orthopedic surgery.

Methods

The study was a prospective, block randomized trial of adults undergoing elective orthopedic procedures over 6 weeks. The primary objective was to determine if the method of digital patient engagement would impact response and completion rates. The primary outcome was response rate and total completion of PRO questionnaires.

Results

A total of 127 participants were block randomized into receiving a mobile survey (n=63) delivered as a hyperlink or responding to the same questions through an automated bidirectional SMS text messaging system (n=64). Gender, age, number of comorbidities, and opioid prescriptions were similar across messaging arms. Patients receiving the mobile survey were more likely to have had a knee-related surgery (n=50, 83.3% vs n=40, 62.5%; P=.02) but less likely to have had an invasive procedure (n=26, 41.3% vs n=39, 60.9%; P=.03). Overall engagement over the immediate postoperative period was similar. Prolonged engagement for patients taking opioids past postoperative day 4 was higher in the mobile survey arm at day 7 (18/19, 94.7% vs 9/16, 56.3%). Patients with more invasive procedures showed a trend toward being responsive at day 4 as compared to not responding (n=41, 59.4% vs n=24, 41.4%; P=.05).

Conclusions

As mobile patient engagement becomes more common in health care, testing the various options to engage patients to gather data is crucial to inform future care and research. We found that bidirectional SMS text messaging and mobile surveys were comparable in response and engagement rates; however, mobile surveys may trend toward higher response rates over longer periods of time.

Trial Registration

ClinicalTrials.gov NCT03532256; https://clinicaltrials.gov/ct2/show/NCT03532256

A Caveat to Using Wearable Sensor Data for COVID-19 Detection: The Role of Behavioral Change after Receipt of Test Results

Recent studies indicate that wearable sensors have the potential to capture subtle within-person changes that signal SARS-CoV-2 infection. However, it remains unclear the extent to which observed discriminative performance is attributable to behavioral change after receiving test results. We conducted a retrospective study in a sample of medical interns who received COVID-19 test results from March to December 2020. Our data confirmed that sensor data were able to differentiate between symptomatic COVID-19 positive and negative individuals with good accuracy (area under the curve (AUC) = 0.75). However, removing post-result data substantially reduced discriminative capacity (0.75 to 0.63; delta= -0.12, p=0.013). Removing data in the symptomatic period prior to receipt of test results did not produce similar reductions in discriminative capacity. These findings suggest a meaningful proportion of the discriminative capacity of wearable sensor data for SARS-CoV-2 infection may be due to behavior change after receiving test results.

Learning From Others Without Sacrificing Privacy: Simulation Comparing Centralized and Federated Machine Learning on Mobile Health Data

Background

The use of wearables facilitates data collection at a previously unobtainable scale, enabling the construction of complex predictive models with the potential to improve health. However, the highly personal nature of these data requires strong privacy protection against data breaches and the use of data in a way that users do not intend. One method to protect user privacy while taking advantage of sharing data across users is federated learning, a technique that allows a machine learning model to be trained using data from all users while only storing a user’s data on that user’s device. By keeping data on users’ devices, federated learning protects users’ private data from data leaks and breaches on the researcher’s central server and provides users with more control over how and when their data are used. However, there are few rigorous studies on the effectiveness of federated learning in the mobile health (mHealth) domain.

Objective

We review federated learning and assess whether it can be useful in the mHealth field, especially for addressing common mHealth challenges such as privacy concerns and user heterogeneity. The aims of this study are to describe federated learning in an mHealth context, apply a simulation of federated learning to an mHealth data set, and compare the performance of federated learning with the performance of other predictive models.

Methods

We applied a simulation of federated learning to predict the affective state of 15 subjects using physiological and motion data collected from a chest-worn device for approximately 36 minutes. We compared the results from this federated model with those from a centralized or server model and with the results from training individual models for each subject.

Results

In a 3-class classification problem using physiological and motion data to predict whether the subject was undertaking a neutral, amusing, or stressful task, the federated model achieved 92.8% accuracy on average, the server model achieved 93.2% accuracy on average, and the individual model achieved 90.2% accuracy on average.

Conclusions

Our findings support the potential for using federated learning in mHealth. The results showed that the federated model performed better than a model trained separately on each individual and nearly as well as the server model. As federated learning offers more privacy than a server model, it may be a valuable option for designing sensitive data collection methods.