Optimizing Adaptive Notifications in Mobile Health Interventions Systems: Reinforcement Learning from a Data-driven Behavioral Simulator

Just-in-Time Adaptive Intervention for Stabilizing Sleep Hours of Japanese Workers: Microrandomized Trial

BACKGROUND

Sleep disturbance is a major contributor to future health and occupational issues. Mobile health can provide interventions that address adverse health behaviors for individuals in a vulnerable health state in real-world settings (just-in-time adaptive intervention).

OBJECTIVE

This study aims to identify a subpopulation with vulnerable sleep state in daily life (study 1) and, immediately afterward, to test whether providing mobile health intervention improved habitual sleep behaviors and psychological wellness in real-world settings by conducting a microrandomized trial (study 2).

METHODS

Japanese workers (n=182) were instructed to collect data on their habitual sleep behaviors and momentary symptoms (including depressive mood, anxiety, and subjective sleep quality) using digital devices in a real-world setting. In study 1, we calculated intraindividual mean and variability of sleep hours, midpoint of sleep, and sleep efficiency to characterize their habitual sleep behaviors. In study 2, we designed and conducted a sleep just-in-time adaptive intervention, which delivered objective push-type sleep feedback messages to improve their sleep hours for a subset of participants in study 1 (n=81). The feedback messages were generated based on their sleep data measured on previous nights and were randomly sent to participants with a 50% chance for each day (microrandomization).

RESULTS

In study 1, we applied hierarchical clustering to dichotomize the population into 2 clusters (group A and group B) and found that group B was characterized by unstable habitual sleep behaviors (large intraindividual variabilities). In addition, linear mixed-effect models showed that the interindividual variability of sleep hours was significantly associated with depressive mood (β=3.83; P=.004), anxiety (β=5.70; P=.03), and subjective sleep quality (β=-3.37; P=.03). In study 2, we found that providing sleep feedback prolonged subsequent sleep hours (increasing up to 40 min; P=.01), and this effect lasted for up to 7 days. Overall, the stability of sleep hours in study 2 was significantly improved among participants in group B compared with the participants in study 1 (P=.001).

CONCLUSIONS

This is the first study to demonstrate that providing sleep feedback can benefit the modification of habitual sleep behaviors in a microrandomized trial. The findings of this study encourage the use of digitalized health intervention that uses real-time health monitoring and personalized feedback.

Mixed-Response State-Space Model for Analyzing Multi-Dimensional Digital Phenotypes

Digital technologies (e.g., mobile phones) can be used to obtain objective, frequent, and real-world digital phenotypes from individuals. However, modeling these data poses substantial challenges since observational data are subject to confounding and various sources of variabilities. For example, signals on patients' underlying health status and treatment effects are mixed with variation due to the living environment and measurement noises. The digital phenotype data thus shows extensive variabilities between- and within-patient as well as across different health domains (e.g., motor, cognitive, and speaking). Motivated by a mobile health study of Parkinson's disease (PD), we develop a mixed-response state-space (MRSS) model to jointly capture multi-dimensional, multi-modal digital phenotypes and their measurement processes by a finite number of latent state time series. These latent states reflect the dynamic health status and personalized time-varying treatment effects and can be used to adjust for informative measurements. For computation, we use the Kalman filter for Gaussian phenotypes and importance sampling with Laplace approximation for non-Gaussian phenotypes. We conduct comprehensive simulation studies and demonstrate the advantage of MRSS in modeling a mobile health study that remotely collects real-time digital phenotypes from PD patients.

Patient Acceptability of Home Monitoring for Neovascular Age-Related Macular Degeneration Reactivation: A Qualitative Study

Neovascular age-related macular degeneration (nAMD) is a chronic, progressive condition and the commonest cause of visual disability in older adults. This study formed part of a diagnostic test accuracy study to quantify the ability of three index home monitoring (HM) tests (one paper-based and two digital tests) to identify reactivation in nAMD. The aim of this qualitative research was to investigate patients' or participants' views about acceptability and explore adherence to weekly HM. Semi-structured interviews were held with 78/297 participants (26%), with close family members (n = 11) and with healthcare professionals involved in training participants in HM procedures (n = 9) (n = 98 in total). A directed thematic analytical approach was applied to the data using a deductive and inductive coding framework informed by theories of technology acceptance. Five themes emerged related to: 1. The role of HM; 2. Suitability of procedures and instruments; 3. Experience of HM; 4. Feasibility of HM in usual practice; and 5. Impediments to patient acceptability of HM. Various factors influenced acceptability including a patient's understanding about the purpose of monitoring. While initial training and ongoing support were regarded as essential for overcoming unfamiliarity with use of digital technology, patients viewed HM as relatively straightforward and non-burdensome. There is a need for further research about how use of performance feedback, level of support and nature of tailoring might facilitate further the implementation of routinely conducted HM. Home monitoring was acceptable to patients and they recognised its potential to reduce clinic visits during non-active treatment phases. Findings have implications for implementation of digital HM in the care of older people with nAMD and other long-term conditions.

Sensor Technology and Intelligent Systems in Anorexia Nervosa: Providing Smarter Healthcare Delivery Systems

Ubiquitous technology, big data, more efficient electronic health records, and predictive analytics are now at the core of smart healthcare systems supported by artificial intelligence. In the present narrative review, we focus on sensing technologies for the healthcare of Anorexia Nervosa (AN). We employed a framework inspired by the Interpersonal Neurobiology Theory (IPNB), which posits that human experience is characterized by a flow of energy and information both within us (within our whole body), and between us (in the connections we have with others and with nature). In line with this framework, we focused on sensors designed to evaluate bodily processes (body sensors such as implantable sensors, epidermal sensors, and wearable and portable sensors), human social interaction (sociometric sensors), and the physical environment (indoor and outdoor ambient sensors). There is a myriad of man-made sensors as well as nature-based sensors such as plants that can be used to design and deploy intelligent systems for human monitoring and healthcare. In conclusion, sensing technologies and intelligent systems can be employed for smarter healthcare of AN and help to relieve the burden of health professionals. However, there are technical, ethical, and environmental sustainability issues that must be considered prior to implementing these systems. A joint collaboration of professionals and other members of the society involved in the healthcare of individuals with AN can help in the development of these systems. The evolution of cyberphysical systems should also be considered in these collaborations.

Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-Implementation Guidelines

Online reinforcement learning (RL) algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints, and accounting for the complexity of the environment, e.g., a lack of accurate mechanistic models for the user dynamics. To guide how one can tackle these challenges, we extend the PCS (predictability, computability, stability) framework, a data science framework that incorporates best practices from machine learning and statistics in supervised learning to the design of RL algorithms for the digital interventions setting. Furthermore, we provide guidelines on how to design simulation environments, a crucial tool for evaluating RL candidate algorithms using the PCS framework. We show how we used the PCS framework to design an RL algorithm for Oralytics, a mobile health study aiming to improve users' tooth-brushing behaviors through the personalized delivery of intervention messages. Oralytics will go into the field in late 2022.

Just-in-Time Prompts for Running, Walking, and Performing Strength Exercises in the Built Environment: 4-Week Randomized Feasibility Study

BACKGROUND

App-based mobile health exercise interventions can motivate individuals to engage in more physical activity (PA). According to the Fogg Behavior Model, it is important that the individual receive prompts at the right time to be successfully persuaded into PA. These are referred to as just-in-time (JIT) interventions. The Playful Active Urban Living (PAUL) app is among the first to include 2 types of JIT prompts: JIT adaptive reminder messages to initiate a run or walk and JIT strength exercise prompts during a walk or run (containing location-based instruction videos). This paper reports on the feasibility of the PAUL app and its JIT prompts.

OBJECTIVE

The main objective of this study was to examine user experience, app engagement, and users' perceptions and opinions regarding the PAUL app and its JIT prompts and to explore changes in the PA behavior, intrinsic motivation, and the perceived capability of the PA behavior of the participants.

METHODS

In total, 2 versions of the closed-beta version of the PAUL app were evaluated: a basic version (Basic PAUL) and a JIT adaptive version (Smart PAUL). Both apps send JIT exercise prompts, but the versions differ in that the Smart PAUL app sends JIT adaptive reminder messages to initiate running or walking behavior, whereas the Basic PAUL app sends reminder messages at randomized times. A total of 23 participants were randomized into 1 of the 2 intervention arms. PA behavior (accelerometer-measured), intrinsic motivation, and the perceived capability of PA behavior were measured before and after the intervention. After the intervention, participants were also asked to complete a questionnaire on user experience, and they were invited for an exit interview to assess user perceptions and opinions of the app in depth.

RESULTS

No differences in PA behavior were observed (Z=-1.433; P=.08), but intrinsic motivation for running and walking and for performing strength exercises significantly increased (Z=-3.342; P<.001 and Z=-1.821; P=.04, respectively). Furthermore, participants increased their perceived capability to perform strength exercises (Z=2.231; P=.01) but not to walk or run (Z=-1.221; P=.12). The interviews indicated that the participants were enthusiastic about the strength exercise prompts. These were perceived as personal, fun, and relevant to their health. The reminders were perceived as important initiators for PA, but participants from both app groups explained that the reminder messages were often not sent at times they could exercise. Although the participants were enthusiastic about the functionalities of the app, technical issues resulted in a low user experience.

CONCLUSIONS

The preliminary findings suggest that the PAUL apps are promising and innovative interventions for promoting PA. Users perceived the strength exercise prompts as a valuable addition to exercise apps. However, to be a feasible intervention, the app must be more stable.

Mitigating feelings of loneliness and depression by means of web-based or print-based physical activity interventions: Pooled analysis of two community-based intervention trials (Preprint)

Background

Physical activity (PA) is associated with benefits, such as fewer depressive symptoms and loneliness. Web- and print-based PA interventions can help older individuals accordingly.

Objective

We aimed to test the following research questions: Do PA interventions delivered in a web- or print-based mode improve self-reported PA stage of change, social-cognitive determinants of PA, loneliness, and symptoms of depression? Is subjective age a mediator and stage of change a moderator of this effect?

Methods

Overall, 831 adults aged ≥60 years were recruited and either allocated to a print-based or web-based intervention group or assigned to a wait-list control group (WLCG) in 2 community-based PA intervention trials over 10 weeks. Missing value imputation using an expectation-maximization algorithm was applied. Frequency analyses, multivariate analyses of variance, and moderated mediation analyses were conducted.

Results

The web-based intervention outperformed (47/59, 80% of initially inactive individuals being adopters, and 396/411, 96.4% of initially active individuals being maintainers of the recommended PA behavior) the print-based intervention (20/25, 80% of adopters, and 63/69, 91% of maintainers) and the WLCG (5/7, 71% of adopters; 141/150, 94% of maintainers). The pattern regarding adopters was statistically significant (web vs print Z=–1.94; P=.02; WLCG vs web Z=3.8367; P=.01). The pattern was replicated with stages (χ²₄=79.1; P<.001; contingency coefficient 0.314; P<.001); in the WLCG, 40.1% (63/157) of the study participants moved to or remained in action stage. This number was higher in the groups receiving web-based (357/470, 76%) or print-based interventions (64/94, 68.1%). A significant difference was observed favoring the 2 intervention groups over and above the WLCG (F_{19, 701}=4.778; P<.001; η²=0.098) and a significant interaction of time and group (F_{19, 701}=2.778; P<.001; η²=0.070) for predictors of behavior. The effects of the interventions on subjective age, loneliness, and depression revealed that both between-group effects (F_{3, 717}=8.668; P<.001; η²=0.018) and the interaction between group and time were significant (F_{3, 717}=6.101; P<.001; η²=0.025). In a moderated mediation model, both interventions had a significant direct effect on depression in comparison with the WLCG (web-based: c′ path −0.86, 95% CI −1.58 to −0.13, SE 0.38; print-based: c′ path −1.96, 95% CI −2.99 to −0.92, SE 0.53). Furthermore, subjective age was positively related to depression (b path 0.14, 95% CI 0.05-0.23; SE 0.05). An indirect effect of the intervention on depression via subjective age was only present for participants who were in actor stage and received the web-based intervention (ab path −0.14, 95% CI −0.34 to −0.01; SE 0.09).

Conclusions

Web-based interventions appear to be as effective as print-based interventions. Both modes might help older individuals remain or become active and experience fewer depression symptoms, especially if they feel younger.

Trial Registration

German Registry of Clinical Trials DRKS00010052 (PROMOTE 1); https://tinyurl.com/nnzarpsu and DRKS00016073 (PROMOTE 2); https://tinyurl.com/4fhcvkwy

International Registered Report Identifier (IRRID)

RR2-10.2196/15168