Reinforcement Learning to Send Reminders at Right Moments in Smartphone Exercise Application: A Feasibility Study

User Experience of a Large-Scale Smartphone-Based Observational Study in Multiple Sclerosis: Global, Open-Access, Digital-Only Study

BACKGROUND

The Floodlight Open app is a digital health technology tool (DHTT) that comprises remote, smartphone sensor-based tests (daily activities) for assessing symptoms of multiple sclerosis (MS). User acquisition, engagement, and retention remain a barrier to successfully deploying such tools.

OBJECTIVE

This study aims to quantitatively and qualitatively investigate key user experience (UX) factors associated with the Floodlight Open app.

METHODS

Floodlight Open is a global, open-access, digital-only study designed to understand the drivers and barriers in deploying a DHTT in a naturalistic setting without supervision and onboarding by a clinician. Daily activities included tests assessing cognition (Information Processing Speed and Information Processing Speed Digit-Digit), hand-motor function (Pinching Test and Draw a Shape Test), and postural stability and gait (Static Balance Test, U-Turn Test, and Two-Minute Walk Test [2MWT]). All daily activities except the 2MWT were taken in a fixed sequence. Qualitative UX was studied through semistructured interviews in a substudy of US participants with MS. The quantitative UX analysis investigated the impact of new UX design features on user engagement and retention in US participants for 3 separate test series: all daily activities included in the fixed sequence (DA), all daily activities included in the fixed sequence except the Static Balance Test and U-Turn Test (DAx), and the 2MWT.

RESULTS

The qualitative UX substudy (N=22) revealed the need for 2 new UX design features: a more seamless user journey during the activation process that eliminates the requirement of switching back and forth between the app and the email that the participants received upon registration, and configurable reminders and push notifications to help plan and remind the participants to complete their daily activities. Both UX design features were assessed in the quantitative UX analysis. Introducing the more seamless user journey (original user journey: n=608; more seamless user journey: n=481) improved the conversion rate of participants who enrolled in the study and proceeded to successfully activate the app from 53.9% (328/608) to 74.6% (359/481). Introducing reminders and push notifications (with reminders and notifications: n=350; without reminders and notifications: n=172) improved continuous usage time (proportion of participants with ≥3 consecutive days of usage: DA and DAx: ~30% vs ~12%; 2MWT: ~30% vs ~20%); test completion rates (maximum number of test series completed: DA: 279 vs 64; DAx: 283 vs 126; 2MWT: 302 vs 76); and user retention rates (at day 30: DA: 53/172, 30.8% vs 34/350, 9.7%; DAx: 53/172, 30.8% vs 60/350, 17.1%; 2MWT: 39/172, 22.6% vs 22/350, 6.2%). Inactivity times remained comparable.

CONCLUSIONS

The remote assessment of MS with DHTTs is a relatively nascent but growing field of research. The continued assessment and improvement of UX design features can play a crucial role in the successful long-term adoption of new DHTTs.

Effectiveness of artificial intelligence vs. human coaching in diabetes prevention: a study protocol for a randomized controlled trial

BACKGROUND

Prediabetes is a highly prevalent condition that heralds an increased risk of progression to type 2 diabetes, along with associated microvascular and macrovascular complications. The Diabetes Prevention Program (DPP) is an established effective intervention for diabetes prevention. However, participation in this 12-month lifestyle change program has historically been low. Digital DPPs have emerged as a scalable alternative, accessible asynchronously and recognized by the Centers for Disease Control and Prevention (CDC). Yet, most digital programs still incorporate human coaching, potentially limiting scalability. Furthermore, existing effectiveness results of digital DPPs are primarily derived from per protocol, longitudinal non-randomized studies, or comparisons to control groups that do not represent the standard of care DPP. The potential of an AI-powered DPP as an alternative to the DPP is yet to be investigated. We propose a randomized controlled trial (RCT) to directly compare these two approaches.

METHODS

This open-label, multicenter, non-inferiority RCT will compare the effectiveness of a fully automated AI-powered digital DPP (ai-DPP) with a standard of care human coach-based DPP (h-DPP). A total of 368 participants with elevated body mass index (BMI) and prediabetes will be randomized equally to the ai-DPP (smartphone app and Bluetooth-enabled body weight scale) or h-DPP (referral to a CDC recognized DPP). The primary endpoint, assessed at 12 months, is the achievement of the CDC's benchmark for type 2 diabetes risk reduction, defined as any of the following: at least 5% weight loss, at least 4% weight loss and at least 150 min per week on average of physical activity, or at least a 0.2-point reduction in hemoglobin A1C. Physical activity will be objectively measured using serial actigraphy at baseline and at 1-month intervals throughout the trial. Secondary endpoints, evaluated at 6 and 12 months, will include changes in A1C, weight, physical activity measures, program engagement, and cost-effectiveness. Participants include adults aged 18-75 years with laboratory confirmed prediabetes, a BMI of ≥ 25 kg/m2 (≥ 23 kg/m2 for Asians), English proficiency, and smartphone users. This U.S. study is conducted at Johns Hopkins Medicine in Baltimore, MD, and Reading Hospital (Tower Health) in Reading, PA.

DISCUSSION

Prediabetes is a significant public health issue, necessitating scalable interventions for the millions affected. Our pragmatic clinical trial is unique in directly comparing a fully automated AI-powered approach without direct human coach interaction. If proven effective, it could be a scalable, cost-effective strategy. This trial will offer vital insights into both AI and human coach-based behavioral change strategies in real-world clinical settings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05056376. Registered on September 24, 2021, https://clinicaltrials.gov/study/NCT05056376.

Infusing behavior science into large language models for activity coaching

Large language models (LLMs) have shown promise for task-oriented dialogue across a range of domains. The use of LLMs in health and fitness coaching is under-explored. Behavior science frameworks such as COM-B, which conceptualizes behavior change in terms of capability (C), Opportunity (O) and Motivation (M), can be used to architect coaching interventions in a way that promotes sustained change. Here we aim to incorporate behavior science principles into an LLM using two knowledge infusion techniques: coach message priming (where exemplar coach responses are provided as context to the LLM), and dialogue re-ranking (where the COM-B category of the LLM output is matched to the inferred user need). Simulated conversations were conducted between the primed or unprimed LLM and a member of the research team, and then evaluated by 8 human raters. Ratings for the primed conversations were significantly higher in terms of empathy and actionability. The same raters also compared a single response generated by the unprimed, primed and re-ranked models, finding a significant uplift in actionability and empathy from the re-ranking technique. This is a proof of concept of how behavior science frameworks can be infused into automated conversational agents for a more principled coaching experience.

Quantitative versus qualitative emotion regulation goals: Differential effects on emotional responses

Emotion regulation (ER) involves both a goal (e.g., to feel less emotion) and a strategy (e.g., reappraisal). To clarify the impact of ER goals on emotional responding, we conducted a within-participant study (N = 156) in which we held the strategy constant (reappraisal) to isolate the impact of regulation goals. We compared the impact of a quantitative goal (changing emotion quantity/intensity) with that of a qualitative goal (changing emotion quality/type) on emotional responses to negative and positive pictures. We manipulated ER goals by cuing participants to continue viewing the picture (unregulated/no ER goal) or to reappraise it to decrease its predominant affective impact (quantitative goal) or increase its opposite-valence impact (qualitative goal). We assessed emotional responses through self-reported feelings and facial expressions (corrugator supercilii and zygomaticus major electromyography). Our findings suggest that the type of regulation goal has a differential effect on emotional responses, with qualitative goals being more effective in modulating both negative and positive emotions. For negative stimuli, attempts to use a quantitative goal decreased negative but not positive emotional responses (uncoupled negative deactivation). Conversely, attempts to use a qualitative goal decreased negative and increased positive emotional responses (reciprocal positive activation). For positive stimuli, the quantitative goal generated uncoupled positive deactivation, while the qualitative goal produced reciprocal negative activation. Results highlight the importance of considering specific regulation goals in shaping emotional responses. Future research in the field of ER may benefit from identifying and manipulating different goals and strategies to understand how to effectively regulate emotions.

Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care

While dental disease is largely preventable, professional advice on optimal oral hygiene practices is often forgotten or abandoned by patients. Therefore patients may benefit from timely and personalized encouragement to engage in oral self-care behaviors. In this paper, we develop an online reinforcement learning (RL) algorithm for use in optimizing the delivery of mobile-based prompts to encourage oral hygiene behaviors. One of the main challenges in developing such an algorithm is ensuring that the algorithm considers the impact of current actions on the effectiveness of future actions (i.e., delayed effects), especially when the algorithm has been designed to run stably and autonomously in a constrained, real-world setting characterized by highly noisy, sparse data. We address this challenge by designing a quality reward that maximizes the desired health outcome (i.e., high-quality brushing) while minimizing user burden. We also highlight a procedure for optimizing the hyperparameters of the reward by building a simulation environment test bed and evaluating candidates using the test bed. The RL algorithm discussed in this paper will be deployed in Oralytics. To the best of our knowledge, Oralytics is the first mobile health study utilizing an RL algorithm designed to prevent dental disease by optimizing the delivery of motivational messages supporting oral self-care behaviors.

Three levels at which the user's cognition can be represented in artificial intelligence

Artificial intelligence (AI) plays an important role in modern society. AI applications are omnipresent and assist many decisions we make in daily life. A common and important feature of such AI applications are user models. These models allow an AI application to adapt to a specific user. Here, we argue that user models in AI can be optimized by modeling these user models more closely to models of human cognition. We identify three levels at which insights from human cognition can be-and have been-integrated in user models. Such integration can be very loose with user models only being inspired by general knowledge of human cognition or very tight with user models implementing specific cognitive processes. Using AI-based applications in the context of education as a case study, we demonstrate that user models that are more deeply rooted in models of cognition offer more valid and more fine-grained adaptations to an individual user. We propose that such user models can also advance the development of explainable AI.

Just-in-Time Prompts for Running, Walking, and Performing Strength Exercises in the Built Environment: 4-Week Randomized Feasibility Study

BACKGROUND

App-based mobile health exercise interventions can motivate individuals to engage in more physical activity (PA). According to the Fogg Behavior Model, it is important that the individual receive prompts at the right time to be successfully persuaded into PA. These are referred to as just-in-time (JIT) interventions. The Playful Active Urban Living (PAUL) app is among the first to include 2 types of JIT prompts: JIT adaptive reminder messages to initiate a run or walk and JIT strength exercise prompts during a walk or run (containing location-based instruction videos). This paper reports on the feasibility of the PAUL app and its JIT prompts.

OBJECTIVE

The main objective of this study was to examine user experience, app engagement, and users' perceptions and opinions regarding the PAUL app and its JIT prompts and to explore changes in the PA behavior, intrinsic motivation, and the perceived capability of the PA behavior of the participants.

METHODS

In total, 2 versions of the closed-beta version of the PAUL app were evaluated: a basic version (Basic PAUL) and a JIT adaptive version (Smart PAUL). Both apps send JIT exercise prompts, but the versions differ in that the Smart PAUL app sends JIT adaptive reminder messages to initiate running or walking behavior, whereas the Basic PAUL app sends reminder messages at randomized times. A total of 23 participants were randomized into 1 of the 2 intervention arms. PA behavior (accelerometer-measured), intrinsic motivation, and the perceived capability of PA behavior were measured before and after the intervention. After the intervention, participants were also asked to complete a questionnaire on user experience, and they were invited for an exit interview to assess user perceptions and opinions of the app in depth.

RESULTS

No differences in PA behavior were observed (Z=-1.433; P=.08), but intrinsic motivation for running and walking and for performing strength exercises significantly increased (Z=-3.342; P<.001 and Z=-1.821; P=.04, respectively). Furthermore, participants increased their perceived capability to perform strength exercises (Z=2.231; P=.01) but not to walk or run (Z=-1.221; P=.12). The interviews indicated that the participants were enthusiastic about the strength exercise prompts. These were perceived as personal, fun, and relevant to their health. The reminders were perceived as important initiators for PA, but participants from both app groups explained that the reminder messages were often not sent at times they could exercise. Although the participants were enthusiastic about the functionalities of the app, technical issues resulted in a low user experience.

CONCLUSIONS

The preliminary findings suggest that the PAUL apps are promising and innovative interventions for promoting PA. Users perceived the strength exercise prompts as a valuable addition to exercise apps. However, to be a feasible intervention, the app must be more stable.

Optimizing Adaptive Notifications in Mobile Health Interventions Systems: Reinforcement Learning from a Data-driven Behavioral Simulator

Mobile health (mHealth) intervention systems can employ adaptive strategies to interact with users. Instead of designing such complex strategies manually, reinforcement learning (RL) can be used to adaptively optimize intervention strategies concerning the user’s context. In this paper, we focus on the issue of overwhelming interactions when learning a good adaptive strategy for the user in RL-based mHealth intervention agents. We present a data-driven approach integrating psychological insights and knowledge of historical data. It allows RL agents to optimize the strategy of delivering context-aware notifications from empirical data when counterfactual information (user responses when receiving notifications) is missing. Our approach also considers a constraint on the frequency of notifications, which reduces the interaction burden for users. We evaluated our approach in several simulation scenarios using real large-scale running data. The results indicate that our RL agent can deliver notifications in a manner that realizes a higher behavioral impact than context-blind strategies.