The potential impact of intelligent systems for mobile health self-management support: Monte Carlo simulations of text message support for medication adherence

The impact of using reinforcement learning to personalize communication on medication adherence: findings from the REINFORCE trial

Text messaging can promote healthy behaviors, like adherence to medication, yet its effectiveness remains modest, in part because message content is rarely personalized. Reinforcement learning has been used in consumer technology to personalize content but with limited application in healthcare. We tested a reinforcement learning program that identifies individual responsiveness ("adherence") to text message content and personalizes messaging accordingly. We randomized 60 individuals with diabetes and glycated hemoglobin A1c [HbA1c] ≥ 7.5% to reinforcement learning intervention or control (no messages). Both arms received electronic pill bottles to measure adherence. The intervention improved absolute adjusted adherence by 13.6% (95%CI: 1.7%-27.1%) versus control and was more effective in patients with HbA1c 7.5- < 9.0% (36.6%, 95%CI: 25.1%-48.2%, interaction p < 0.001). We also explored whether individual patient characteristics were associated with differential response to tested behavioral factors and unique clusters of responsiveness. Reinforcement learning may be a promising approach to improve adherence and personalize communication at scale.

An Automatically Adaptive Digital Health Intervention to Decrease Opioid-Related Risk While Conserving Counselor Time: Quantitative Analysis of Treatment Decisions Based on Artificial Intelligence and Patient-Reported Risk Measures

BACKGROUND

Some patients prescribed opioid analgesic (OA) medications for pain experience serious side effects, including dependence, sedation, and overdose. As most patients are at low risk for OA-related harms, risk reduction interventions requiring multiple counseling sessions are impractical on a large scale.

OBJECTIVE

This study evaluates whether an intervention based on reinforcement learning (RL), a field of artificial intelligence, learned through experience to personalize interactions with patients with pain discharged from the emergency department (ED) and decreased self-reported OA misuse behaviors while conserving counselors' time.

METHODS

We used data representing 2439 weekly interactions between a digital health intervention ("Prescription Opioid Wellness and Engagement Research in the ED" [PowerED]) and 228 patients with pain discharged from 2 EDs who reported recent opioid misuse. During each patient's 12 weeks of intervention, PowerED used RL to select from 3 treatment options: a brief motivational message delivered via an interactive voice response (IVR) call, a longer motivational IVR call, or a live call from a counselor. The algorithm selected session types for each patient each week, with the goal of minimizing OA risk, defined in terms of a dynamic score reflecting patient reports during IVR monitoring calls. When a live counseling call was predicted to have a similar impact on future risk as an IVR message, the algorithm favored IVR to conserve counselor time. We used logit models to estimate changes in the relative frequency of each session type as PowerED gained experience. Poisson regression was used to examine the changes in self-reported OA risk scores over calendar time, controlling for the ordinal session number (1st to 12th).

RESULTS

Participants on average were 40 (SD 12.7) years of age; 66.7% (152/228) were women and 51.3% (117/228) were unemployed. Most participants (175/228, 76.8%) reported chronic pain, and 46.2% (104/225) had moderate to severe depressive symptoms. As PowerED gained experience through interactions over a period of 142 weeks, it delivered fewer live counseling sessions than brief IVR sessions (P=.006) and extended IVR sessions (P<.001). Live counseling sessions were selected 33.5% of the time in the first 5 weeks of interactions (95% CI 27.4%-39.7%) but only for 16.4% of sessions (95% CI 12.7%-20%) after 125 weeks. Controlling for each patient's changes during the course of treatment, this adaptation of treatment-type allocation led to progressively greater improvements in self-reported OA risk scores (P<.001) over calendar time, as measured by the number of weeks since enrollment began. Improvement in risk behaviors over time was especially pronounced among patients with the highest risk at baseline (P=.02).

CONCLUSIONS

The RL-supported program learned which treatment modalities worked best to improve self-reported OA risk behaviors while conserving counselors' time. RL-supported interventions represent a scalable solution for patients with pain receiving OA prescriptions.

TRIAL REGISTRATION

Clinicaltrials.gov NCT02990377; https://classic.clinicaltrials.gov/ct2/show/NCT02990377.

Adaptive learning algorithms to optimize mobile applications for behavioral health: guidelines for design decisions

OBJECTIVE

Providing behavioral health interventions via smartphones allows these interventions to be adapted to the changing behavior, preferences, and needs of individuals. This can be achieved through reinforcement learning (RL), a sub-area of machine learning. However, many challenges could affect the effectiveness of these algorithms in the real world. We provide guidelines for decision-making.

MATERIALS AND METHODS

Using thematic analysis, we describe challenges, considerations, and solutions for algorithm design decisions in a collaboration between health services researchers, clinicians, and data scientists. We use the design process of an RL algorithm for a mobile health study "DIAMANTE" for increasing physical activity in underserved patients with diabetes and depression. Over the 1.5-year project, we kept track of the research process using collaborative cloud Google Documents, Whatsapp messenger, and video teleconferencing. We discussed, categorized, and coded critical challenges. We grouped challenges to create thematic topic process domains.

RESULTS

Nine challenges emerged, which we divided into 3 major themes: 1. Choosing the model for decision-making, including appropriate contextual and reward variables; 2. Data handling/collection, such as how to deal with missing or incorrect data in real-time; 3. Weighing the algorithm performance vs effectiveness/implementation in real-world settings.

CONCLUSION

The creation of effective behavioral health interventions does not depend only on final algorithm performance. Many decisions in the real world are necessary to formulate the design of problem parameters to which an algorithm is applied. Researchers must document and evaulate these considerations and decisions before and during the intervention period, to increase transparency, accountability, and reproducibility.

TRIAL REGISTRATION

clinicaltrials.gov, NCT03490253.

Preferences for mHealth Technology and Text Messaging Communication in Patients With Type 2 Diabetes: Qualitative Interview Study

Background

Individuals with diabetes need regular support to help them manage their diabetes on their own, ideally delivered via mechanisms that they already use, such as their mobile phones. One reason for the modest effectiveness of prior technology-based interventions may be that the patient perspective has been insufficiently incorporated.

Objective

This study aims to understand patients’ preferences for mobile health (mHealth) technology and how that technology can be integrated into patients’ routines, especially with regard to medication use.

Methods

We conducted semistructured qualitative individual interviews with patients with type 2 diabetes from an urban health care system to elicit and explore their perspectives on diabetes medication–taking behaviors, daily patterns of using mobile technology, use of mHealth technology for diabetes care, acceptability of text messages to support medication adherence, and preferred framing of information within text messages to support diabetes care. The interviews were digitally recorded and transcribed. The data were analyzed using codes developed by the study team to generate themes, with representative quotations selected as illustrations.

Results

We conducted interviews with 20 participants, of whom 12 (60%) were female and 9 (45%) were White; in addition, the participants’ mean glycated hemoglobin A_1c control was 7.8 (SD 1.1). Overall, 5 key themes were identified: patients try to incorporate cues into their routines to help them with consistent medication taking; many patients leverage some form of technology as a cue to support adherence to medication taking and diabetes self-management behaviors; patients value simplicity and integration of technology solutions used for diabetes care, managing medications, and communicating with health care providers; some patients express reluctance to rely on mobile technology for these diabetes care behaviors; and patients believe they prefer positively framed communication, but communication preferences are highly individualized.

Conclusions

The participants expressed some hesitation about using mobile technology in supporting diabetes self-management but have largely incorporated it or are open to incorporating it as a cue to make medication taking more automatic and less burdensome. When using technology to support diabetes self-management, participants exhibited individualized preferences, but overall, they preferred simple and positively framed communication. mHealth interventions may be improved by focusing on integrating them easily into daily routines and increasing the customization of content.

Beyond Adherence Thresholds: A Simulation Study of the Optimal Classification of Longitudinal Adherence Trajectories From Medication Refill Histories

Background: The description of adherence based on medication refill histories relies on the estimation of continuous medication availability (CMA) during an observation period. Thresholds to distinguish adherence from non-adherence typically refer to an aggregated value across the entire observation period, disregarding differences in adherence over time. Sliding windows to divide the observation period into smaller portions, estimating adherence for these increments, and classify individuals with similar trajectories into clusters can retain this temporal information. Optimal methods to estimate adherence trajectories to identify underlying patterns have not yet been established. This simulation study aimed to provide guidance for future studies by analyzing the effect of different longitudinal adherence estimates, sliding window parameters, and sample characteristics on the performance of a longitudinal clustering algorithm.

Methods: We generated samples of 250–25,000 individuals with one of six longitudinal refill patterns over a 2-year period. We used two longitudinal CMA estimates (LCMA1 and LCMA2) and their dichotomized variants (with a threshold of 80%) to create adherence trajectories. LCMA1 assumes full adherence until the supply ends while LCMA2 assumes constant adherence between refills. We assessed scenarios with different LCMA estimates and sliding window parameters for 350 independent samples. Individual trajectories were clustered with kml, an implementation of k-means for longitudinal data in R. We compared performance between the four LCMA estimates using the adjusted Rand Index (cARI).

Results: Cluster analysis with LCMA2 outperformed other estimates in overall performance, correct identification of groups, and classification accuracy, irrespective of sliding window parameters. Pairwise comparison between LCMA estimates showed a relative cARI-advantage of 0.12–0.22 (p < 0.001) for LCMA2. Sample size did not affect overall performance.

Conclusion: The choice of LCMA estimate and sliding window parameters has a major impact on the performance of a clustering algorithm to identify distinct longitudinal adherence trajectories. We recommend (a) to assume constant adherence between refills, (b) to avoid dichotomization based on a threshold, and (c) to explore optimal sliding windows parameters in simulation studies or selecting shorter non-overlapping windows for the identification of different adherence patterns from medication refill data.

The Impact of the mKidney mHealth System on Live Donor Follow-Up Compliance: Protocol for a Randomized Controlled Trial

Background

Every year, more than 5500 healthy people in the United States donate a kidney for the medical benefit of another person. The Organ Procurement and Transplantation Network (OPTN) requires transplant hospitals to monitor living kidney donors (LKDs) for 2 years postdonation. However, the majority (115/202, 57%) of transplant hospitals in the United States continue to fail to meet nationally mandated requirements for LKD follow-up. A novel method for collecting LKD follow-up is needed to ease both the transplant hospital-level and patient-level burden. We built mKidney—a mobile health (mHealth) system designed specifically to facilitate the collection and reporting of OPTN-required LKD follow-up data. The mKidney mobile app was developed on the basis of input elicited from LKDs, transplant providers, and thought leaders.

Objective

The primary objective of this study is to evaluate the impact of the mKidney smartphone app on LKD follow-up rates.

Methods

We will conduct a two-arm randomized controlled trial (RCT) with LKDs who undergo LKD transplantation at Methodist Specialty and Transplant Hospital in San Antonio, Texas. Eligible participants will be recruited in-person by a study team member at their 1-week postdonation clinical visit and randomly assigned to the intervention or control arm (1:1). Participants in the intervention arm will receive the mHealth intervention (mKidney), and participants in the control arm will receive the current standard of follow-up care. Our primary outcome will be policy-defined complete (all components addressed) and timely (60 days before or after the expected visit date) submission of LKD follow-up data at required 6-month, 1-year, and 2-year visits. Our secondary outcome will be hospital-level compliance with OPTN reporting requirements at each visit. Data analysis will follow the intention-to-treat principle. Additionally, we will collect quantitative and qualitative process data regarding the implementation of the mKidney system.

Results

We began recruitment for this RCT in May 2018. We plan to enroll 400 LKDs over 2 years and follow participants for the 2-year mandated follow-up period.

Conclusions

This pilot RCT will evaluate the impact of the mKidney system on rates of LKD and hospital compliance with OPTN-mandated LKD follow-up at a large LKD transplant hospital. It will provide valuable information on strategies for implementing such a system in a clinical setting and inform effect sizes for future RCT sample size calculations.

International Registered Report Identifier (IRRID)

DERR1-10.2196/11000

Assessing the Attitudes and Perceptions Regarding the Use of Mobile Health Technologies for Living Kidney Donor Follow-Up: Survey Study

BACKGROUND

In 2013, the Organ Procurement and Transplantation Network began requiring transplant centers in the United States to collect and report postdonation living kidney donor follow-up data at 6 months, 1 year, and 2 years. Despite this requirement, <50% of transplant centers have been able to collect and report the required data. Previous work identified a number of barriers to living kidney donor follow-up, including logistical and administrative barriers for transplant centers and cost and functional barriers for donors. Novel smartphone-based mobile health (mHealth) technologies might reduce the burden of living kidney donor follow-up for centers and donors. However, the attitudes and perceptions toward the incorporation of mHealth into postdonation care among living kidney donors are unknown. Understanding donor attitudes and perceptions will be vital to the creation of a patient-oriented mHealth system to improve living donor follow-up in the United States.

OBJECTIVE

The goal of this study was to assess living kidney donor attitudes and perceptions associated with the use of mHealth for follow-up.

METHODS

We developed and administered a cross-sectional 14-question survey to 100 living kidney donors at our transplant center. All participants were part of an ongoing longitudinal study of long-term outcomes in living kidney donors. The survey included questions on smartphone use, current health maintenance behaviors, accessibility to health information, and attitudes toward using mHealth for living kidney donor follow-up.

RESULTS

Of the 100 participants surveyed, 94 owned a smartphone (35 Android, 58 iPhone, 1 Blackberry), 37 had accessed their electronic medical record on their smartphone, and 38 had tracked their exercise and physical activity on their smartphone. While 77% (72/93) of participants who owned a smartphone and had asked a medical question in the last year placed the most trust with their doctors, nurses, or other health care professionals regarding answering a health-related question, 52% (48/93) most often accessed health information elsewhere. Overall, 79% (74/94) of smartphone-owning participants perceived accessing living kidney donor information and resources on their smartphone as useful. Additionally, 80% (75/94) perceived completing some living kidney donor follow-up via mHealth as useful. There were no significant differences in median age (60 vs 59 years; P=.65), median years since donation (10 vs 12 years; P=.45), gender (36/75, 36%, vs 37/75, 37%, male; P=.57), or race (70/75, 93%, vs 18/19, 95%, white; P=.34) between those who perceived mHealth as useful for living kidney donor follow-up and those who did not, respectively.

CONCLUSIONS

Overall, smartphone ownership was high (94/100, 94.0%), and 79% (74/94) of surveyed smartphone-owning donors felt that it would be useful to complete their required follow-up with an mHealth tool, with no significant differences by age, sex, or race. These results suggest that patients would benefit from an mHealth tool to perform living donor follow-up.

Can the artificial intelligence technique of reinforcement learning use continuously-monitored digital data to optimize treatment for weight loss?

Behavioral weight loss (WL) trials show that, on average, participants regain lost weight unless provided long-term, intensive-and thus costly-intervention. Optimization solutions have shown mixed success. The artificial intelligence principle of "reinforcement learning" (RL) offers a new and more sophisticated form of optimization in which the intensity of each individual's intervention is continuously adjusted depending on patterns of response. In this pilot, we evaluated the feasibility and acceptability of a RL-based WL intervention, and whether optimization would achieve equivalent benefit at a reduced cost compared to a non-optimized intensive intervention. Participants (n = 52) completed a 1-month, group-based in-person behavioral WL intervention and then (in Phase II) were randomly assigned to receive 3 months of twice-weekly remote interventions that were non-optimized (NO; 10-min phone calls) or optimized (a combination of phone calls, text exchanges, and automated messages selected by an algorithm). The Individually-Optimized (IO) and Group-Optimized (GO) algorithms selected interventions based on past performance of each intervention for each participant, and for each group member that fit into a fixed amount of time (e.g., 1 h), respectively. Results indicated that the system was feasible to deploy and acceptable to participants and coaches. As hypothesized, we were able to achieve equivalent Phase II weight losses (NO = 4.42%, IO = 4.56%, GO = 4.39%) at roughly one-third the cost (1.73 and 1.77 coaching hours/participant for IO and GO, versus 4.38 for NO), indicating strong promise for a RL system approach to weight loss and maintenance.

Mobile Phone Support for Diabetes Self-Care Among Diverse Adults: Protocol for a Three-Arm Randomized Controlled Trial

Background

Nonadherence to self-care is common among patients with type 2 diabetes (T2D) and often leads to severe complications. Moreover, patients with T2D who have low socioeconomic status and are racial/ethnic minorities disproportionately experience barriers to adherence and poor outcomes. Basic phone technology (text messages and phone calls) provides a practical medium for delivering content to address patients’ barriers to adherence; however, trials are needed to explore long-term and sustainable effects of mobile phone interventions among diverse patients.

Objective

The aim of this study is to evaluate the effects of mobile phone–based diabetes support interventions on self-care and hemoglobin A_1c (HbA_1c) among adults with T2D using a 3-arm, 15-month randomized controlled trial with a Type 1 hybrid effectiveness-implementation approach. The intervention arms are (1) Rapid Encouragement/Education And Communications for Health (REACH) and (2) REACH + Family-focused Add-on for Motivating Self-care (FAMS).

Methods

We recruited primary care patients with T2D (N=512) from Federally Qualified Health Centers and an academic medical center, prioritizing recruitment of publicly insured and minority patients from the latter. Eligible patients were prescribed daily diabetes medication and owned a cell phone with text messaging capability. We excluded patients whose most recent HbA_1c result within 12 months was <6.8% to support detection of intervention effects on HbA_1c. Participants were randomly assigned to REACH only, REACH + FAMS, or the control condition. REACH provides text messages tailored to address patient-specific barriers to medication adherence based on the Information-Motivation-Behavioral skills model, whereas FAMS provides monthly phone coaching with related text message content focused on family and friend barriers to diet and exercise adherence. We collect HbA_1c and self-reported survey data at baseline and at 3, 6, and 12 months, and again at 15 months to assess sustained changes. We will use generalized estimating equation models to test the effects of REACH (either intervention arm) on HbA_1c relative to the control group, the potential additive effects of FAMS, and effects of either intervention on adherence to self-care behaviors and diabetes self-efficacy.

Results

The trial is ongoing; recruitment closed December 2017. We plan to perform analyses on 6-month outcomes for FAMS in July 2018, and project to have 15-month data for REACH analyses in April 2019.

Conclusions

Our study will be one of the first to evaluate a long-term, theory-based text messaging intervention to promote self-care adherence among racially/ethnically and socioeconomically diverse adults with T2D. Moreover, our study will assess the feasibility of a family-focused intervention delivered via mobile phones and compare the effects of text messaging alone versus text messaging plus phone coaching. Findings will advance our understanding of how interventions delivered by phone can benefit diverse patients with chronic conditions.

Trial Registration

ClinicalTrials.gov NCT02409329; https://clinicaltrials.gov/ct2/show/NCT02409329 (Archived by WebCite at http://www.webcitation.org/6yHkg9SSl); NCT02481596; https://clinicaltrials.gov/ct2/show/NCT02481596 (Archived by WebCite at http://www.webcitation.org/6yHkj9XD4)

Mobile Health Devices as Tools for Worldwide Cardiovascular Risk Reduction and Disease Management

We examined evidence on whether mobile health (mHealth) tools, including interactive voice response calls, short message service, or text messaging, and smartphones, can improve lifestyle behaviors and management related to cardiovascular diseases throughout the world. We conducted a state-of-the-art review and literature synthesis of peer-reviewed and gray literature published since 2004. The review prioritized randomized trials and studies focused on cardiovascular diseases and risk factors, but included other reports when they represented the best available evidence. The search emphasized reports on the potential benefits of mHealth interventions implemented in low- and middle-income countries. Interactive voice response and short message service interventions can improve cardiovascular preventive care in developed countries by addressing risk factors including weight, smoking, and physical activity. Interactive voice response and short message service-based interventions for cardiovascular disease management also have shown benefits with respect to hypertension management, hospital readmissions, and diabetic glycemic control. Multimodal interventions including Web-based communication with clinicians and mHealth-enabled clinical monitoring with feedback also have shown benefits. The evidence regarding the potential benefits of interventions using smartphones and social media is still developing. Studies of mHealth interventions have been conducted in >30 low- and middle-income countries, and evidence to date suggests that programs are feasible and may improve medication adherence and disease outcomes. Emerging evidence suggests that mHealth interventions may improve cardiovascular-related lifestyle behaviors and disease management. Next-generation mHealth programs developed worldwide should be based on evidence-based behavioral theories and incorporate advances in artificial intelligence for adapting systems automatically to patients' unique and changing needs.

Understanding patients' health and technology attitudes for tailoring self-management interventions

Healthcare providers are moving towards tailoring self-management interventions to include the communication technologies patients use in daily life. Accurate understanding of patients' attitudes towards both technology and involvement in managing chronic conditions will be critical for informing effective self-management strategies. The tailoring of these interventions, however, could be undermined by providers' implicit biases based on patient age, race, and education level that have been shown to negatively affect care. To inform the design and tailoring of self-management interventions, we elicited attitudes toward technology use and participation in care of 40 participants in a maximum variation sample. The analysis revealed three participant clusters-"Proactive Techies," "Indie Self-Managers," and "Remind Me! Non-Techies"-that represent varying attitudes toward health behaviors and technologies that were independent of race, education level, and age. Our approach provides insight into how people prioritize important values related to health participation and technology.

Confirming the theoretical structure of expert-developed text messages to improve adherence to anti-hypertensive medications

BACKGROUND

Text messages can improve medication adherence and outcomes in several conditions. For this study, experts developed text messages addressing determinants of medication adherence: disease beliefs, medication necessity, medication concerns, and forgetfulness, as well as positive reinforcement messages for patients who were adherent.

OBJECTIVES

To validate expert-developed text messages to address medication non-adherence with a group of non-researchers.

METHODS

A two-wave, card-sorting activity was conducted with students and staff at the University of Michigan. In the first wave, 40 participants grouped 32 messages addressing barriers for medication adherence (disease beliefs, medication necessity, medication concerns, and forgetfulness) according to their perceived relationship. Messages with poor grouping agreement were deleted or modified. In the second wave, positive reinforcement messages were developed and tested along with the previous categories (36 messages) by 37 participants. Similarity and cluster analyses were used to assess agreement between experts and participants.

RESULTS

In the first card-sorting wave, participants grouped messages into between 2 and 13 separate categories. Similarity analysis showed four groupings of messages, however, some had an agreement below 50% and clusters appeared dispersed. In the second wave, and after messages being edited, participants grouped the messages into between 4 and 9 categories. Five groups (now including positive reinforcement messages) were identified with higher agreement in the similarity and cluster analyses.

CONCLUSIONS

The structure of expert-developed text messages to address medication adherence key barriers was confirmed. Messages will be used in future research to determine their impact on affecting medication adherence to anti-hypertensive medications using a reinforcement learning controlled text messaging service.

Personal health technology: A new era in cardiovascular disease prevention

Cardiovascular disease (CVD) remains the leading cause of morbidity and mortality worldwide yet the majority of related risk factors are largely preventable (primary prevention [PP]) and effectively treatable (secondary prevention [SP]) with healthy lifestyle behaviors. The use of information and communication technology (ICT) offers a unique approach to personal health and CVD prevention, as these mediums are relatively affordable, approachable, and accessible. The purpose of this review is to provide an overview of ICT-driven personal health technologies and their potential role in promoting and supporting self-care behaviors for PP and SP of CVD. In this review, we focus on technological interventions that have been successful at supporting positive behavior change in order to determine which tools, resources, and methods are most appropriate for delivering interventions geared towards CVD prevention. We conducted a literature search from a range of sources including scholarly, peer-reviewed journal articles indexed in PubMed and CINAHL, gray literature, and reputable websites and other Internet-based media. A synthesis of existing literature indicates that the overall efficacy of ICT-driven personal health technologies is largely determined by: 1) the educational resources provided and the extent to which the relayed information is customized or individually tailored; and 2) the degree of self-monitoring and levels of personalized feedback or other interactions (e.g. interpersonal communications). We conclude that virtually all the technological tools and resources identified (e.g. Internet-based communications including websites, weblogs and wikis, mobile devices and applications, social media, and wearable monitors) can be strategically leveraged to enhance self-care behaviors for CVD risk reduction and SP but further research is needed to evaluate their efficacy, cost-effectiveness, and long-term maintainability.