Beyond clicks and downloads: a call for a more comprehensive approach to measuring mobile-health app engagement

The Potential for Digital Phenotyping in Understanding Mindfulness App Engagement Patterns: A Pilot Study

Background: Low app engagement is a central barrier to digital mental health efficacy. With mindfulness-based mental health apps growing in popularity, there is a need for new understanding of factors influencing engagement. This study utilized digital phenotyping to understand real-time patterns of engagement around app-based mindfulness. Different engagement metrics are presented that measure both the total number of app-based activities participants completed each week, as well as the proportion of days that participants engaged with the app each week. Method: Data were derived from two iterations of a four-week study exploring app engagement in college students (n = 169). This secondary analysis investigated the relationships between general and mindfulness-based app engagement with passive data metrics (sleep duration, home time, and screen duration) at a weekly level, as well as the relationship between demographics and engagement. Additional clinically focused analysis was performed on three case studies of participants with high mindfulness activity completion. Results: Demographic variables such as gender, race/ethnicity, and age lacked a significant association with mindfulness app-based engagement. Passive data variables such as sleep and screen duration were significant predictors for different metrics of general and mindfulness-based app engagement at a weekly level. There was a significant interaction effect for screen duration between the number of mindfulness activities completed and whether or not the participant received a mindfulness notification. K-means clusters analyses using passive data features to predict mindfulness activity completion had low performance. Conclusions: While there are no simple solutions to predicting engagement with mindfulness apps, utilizing digital phenotyping approaches at a population and personal level offers new potential. The signal from digital phenotyping warrants more investigation; even small increases in engagement with mindfulness apps may have a tremendous impact given their already high prevalence of engagement, availability, and potential to engage patients across demographics.

Use and Engagement With Low-Intensity Cognitive Behavioral Therapy Techniques Used Within an App to Support Worry Management: Content Analysis of Log Data

BACKGROUND

Low-intensity cognitive behavioral therapy (LICBT) has been implemented by the Improving Access to Psychological Therapies services across England to manage excessive worry associated with generalized anxiety disorder and support emotional well-being. However, barriers to access limit scalability. A solution has been to incorporate LICBT techniques derived from an evidence-based protocol within the Iona Mind Well-being app for Worry management (IMWW) with support provided through an algorithmically driven conversational agent.

OBJECTIVE

This study aims to examine engagement with a mobile phone app to support worry management with specific attention directed toward interaction with specific LICBT techniques and examine the potential to reduce symptoms of anxiety.

METHODS

Log data were examined with respect to a sample of "engaged" users who had completed at least 1 lesson related to the Worry Time and Problem Solving in-app modules that represented the "minimum dose." Paired sample 2-tailed t tests were undertaken to examine the potential for IMWW to reduce worry and anxiety, with multivariate linear regressions examining the extent to which completion of each of the techniques led to reductions in worry and anxiety.

RESULTS

There was good engagement with the range of specific LICBT techniques included within IMWW. The vast majority of engaged users were able to interact with the cognitive behavioral therapy model and successfully record types of worry. When working through Problem Solving, the conversational agent was successfully used to support the user with lower levels of engagement. Several users engaged with Worry Time outside of the app. Forgetting to use the app was the most common reason for lack of engagement, with features of the app such as completion of routine outcome measures and weekly reflections having lower levels of engagement. Despite difficulties in the collection of end point data, there was a significant reduction in severity for both anxiety (t53=5.5; P<.001; 95% CI 2.4-5.2) and low mood (t53=2.3; P=.03; 95% CI 0.2-3.3). A statistically significant linear model was also fitted to the Generalized Anxiety Disorder-7 (F2,51=6.73; P<.001), while the model predicting changes in the Patient Health Questionnaire-8 did not reach significance (F2,51=2.33; P=.11). This indicates that the reduction in these measures was affected by in-app engagement with Worry Time and Problem Solving.

CONCLUSIONS

Engaged users were able to successfully interact with the LICBT-specific techniques informed by an evidence-based protocol although there were lower completion rates of routine outcome measures and weekly reflections. Successful interaction with the specific techniques potentially contributes to promising data, indicating that IMWW may be effective in the management of excessive worry. A relationship between dose and improvement justifies the use of log data to inform future developments. However, attention needs to be directed toward enhancing interaction with wider features of the app given that larger improvements were associated with greater engagement.

Self-Care Mental Health App Intervention for Post-Intensive Care Syndrome-Family: A Randomized Pilot Study

BACKGROUND

Post-intensive care syndrome-family (PICS-F) is a constellation of adverse psychological symptoms experienced by family members of critically ill patients during and after acute illness. Cognitive behavioral therapy delivered using smartphone technology is a novel approach for PICS-F symptom self-management.

OBJECTIVE

To determine the efficacy of smartphone delivery of cognitive behavioral therapy in reducing the prevalence and severity of PICS-F symptoms in family members of critically ill patients.

METHODS

The study had a randomized controlled longitudinal design with control and intervention groups composed of family members of patients admitted to 2 adult intensive care units. The intervention consisted of a mental health app loaded on participants' personal smartphones. The study time points were upon enrollment (within 5 days of intensive care unit admission; time 1), 30 days after enrollment (time 2), and 60 days after enrollment (time 3). Study measures included demographic data, PICS-F symptoms, mental health self-efficacy, health-related quality of life, and app use.

RESULTS

The study sample consisted of 60 predominantly White (72%) and female (78%) family members (30 intervention, 30 control). Anxiety and depression symptom severity decreased significantly over time in the intervention group but not in the control group. Family members logged in to the app a mean of 11.4 times (range, 1-53 times) and spent a mean of 50.16 minutes (range, 1.87-245.92 minutes) using the app.

CONCLUSIONS

Delivery of cognitive behavioral therapy to family members of critically ill patients via a smartphone app shows some efficacy in reducing PICS-F symptoms.

Identifying Design Opportunities for Adaptive mHealth Interventions That Target General Well-Being: Interview Study With Informal Care Partners

BACKGROUND

Mobile health (mHealth) interventions can deliver personalized behavioral support to users in daily contexts. These interventions have been increasingly adopted to support individuals who require low-cost and low-burden support. Prior research has demonstrated the feasibility and acceptability of an mHealth intervention app (CareQOL) designed for use with informal care partners. To further optimize the intervention delivery, we need to investigate how care partners, many of whom lack the time for self-care, react and act in response to different behavioral messages.

OBJECTIVE

The goal of this study was to understand the factors that impact care partners' decision-making and actions in response to different behavioral messages. Insights from this study will help optimize future tailored and personalized behavioral interventions.

METHODS

We conducted semistructured interviews with participants who had recently completed a 3-month randomized controlled feasibility trial of the CareQOL mHealth intervention app. Of the 36 participants from the treatment group of the randomized controlled trial, 23 (64%) participated in these interviews. To prepare for each interview, the team first selected representative behavioral messages (eg, targeting different health dimensions) and presented them to participants during the interview to probe their influence on participants' thoughts and actions. The time of delivery, self-reported perceptions of the day, and user ratings of a message were presented to the participants during the interviews to assist with recall.

RESULTS

The interview data showed that after receiving a message, participants took various actions in response to different messages. Participants performed suggested behaviors or adjusted them either immediately or in a delayed manner (eg, sometimes up to a month later). We identified 4 factors that shape the variations in user actions in response to different behavioral messages: uncertainties about the workload required to perform suggested behaviors, concerns about one's ability to routinize suggested behaviors, in-the-moment willingness and ability to plan for suggested behaviors, and overall capability to engage with the intervention.

CONCLUSIONS

Our study showed that care partners use mHealth behavioral messages differently regarding the immediacy of actions and the adaptation to suggested behaviors. Multiple factors influence people's perceptions and decisions regarding when and how to take actions. Future systems should consider these factors to tailor behavioral support for individuals and design system features to support the delay or adaptation of the suggested behaviors. The findings also suggest extending the assessment of user adherence by considering the variations in user actions on behavioral support (ie, performing suggested or adjusted behaviors immediately or in a delayed manner).

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/32842.

Identifying factors associated with user retention and outcomes of a digital intervention for substance use disorder: a retrospective analysis of real-world data

Objectives

Successful delivery of digital health interventions is affected by multiple real-world factors. These factors may be identified in routinely collected, ecologically valid data from these interventions. We propose ideas for exploring these data, focusing on interventions targeting complex, comorbid conditions.

Materials and Methods

This study retrospectively explores pre-post data collected between 2016 and 2019 from users of digital cognitive behavioral therapy (CBT)-containing psychoeducation and practical exercises-for substance use disorder (SUD) at UK addiction services. To identify factors associated with heterogenous user responses to the technology, we employed multivariable and multivariate regressions and random forest models of user-reported questionnaire data.

Results

The dataset contained information from 14 078 individuals of which 12 529 reported complete data at baseline and 2925 did so again after engagement with the CBT. Ninety-three percent screened positive for dependence on 1 of 43 substances at baseline, and 73% screened positive for anxiety or depression. Despite pre-post improvements independent of user sociodemographics, women reported more frequent and persistent symptoms of SUD, anxiety, and depression. Retention-minimum 2 use events recorded-was associated more with deployment environment than user characteristics. Prediction accuracy of post-engagement outcomes was acceptable (Area Under Curve [AUC]: 0.74-0.79), depending non-trivially on user characteristics.

Discussion

Traditionally, performance of digital health interventions is determined in controlled trials. Our analysis showcases multivariate models with which real-world data from these interventions can be explored and sources of user heterogeneity in retention and symptom reduction uncovered.

Conclusion

Real-world data from digital health interventions contain information on natural user-technology interactions which could enrich results from controlled trials.

The engagement problem: A review of engagement with digital mental health interventions and recommendations for a path forward

Purpose of the review

Digital mental health interventions (DMHIs) are an effective and accessible means of addressing the unprecedented levels of mental illness worldwide. Currently, however, patient engagement with DMHIs in real-world settings is often insufficient to see clinical benefit. In order to realize the potential of DMHIs, there is a need to better understand what drives patient engagement.

Recent findings

We discuss takeaways from the existing literature related to patient engagement with DMHIs and highlight gaps to be addressed through further research. Findings suggest that engagement is influenced by patient-, intervention- and systems-level factors. At the patient-level, variables such as sex, education, personality traits, race, ethnicity, age and symptom severity appear to be associated with engagement. At the intervention-level, integrating human support, gamification, financial incentives and persuasive technology features may improve engagement. Finally, although systems-level factors have not been widely explored, the existing evidence suggests that achieving engagement will require addressing organizational and social barriers and drawing on the field of implementation science.

Summary

Future research clarifying the patient-, intervention- and systems-level factors that drive engagement will be essential. Additionally, to facilitate improved understanding of DMHI engagement, we propose the following: (a) widespread adoption of a minimum necessary 5-element engagement reporting framework; (b) broader application of alternative clinical trial designs; and (c) directed efforts to build upon an initial parsimonious conceptual model of DMHI engagement.

Smartphone Applications (Apps) for Nutrition Education: A Qualitative Analysis of Outpatient Dietitian Perspectives

OBJECTIVE

To explore how outpatient dietitians select and use applications (apps) to support nutrition education.

METHODS

Qualitative analysis of 20 dietitians who participated in semistructured interviews investigating their app use and recommendation processes. Transcripts were analyzed using thematic analysis.

RESULTS

Four themes described how dietitians perceive apps for nutrition education: (1) nutrition education goals focus on long-term lifestyle behavior change while protecting patients' relationship with food, (2) attitudes toward tracking apps influence app selection, (3) dietitians differentiate among patients who will benefit from tracking vs information apps, and (4) barriers to optimal app use result in adaptations by dietitians.

CONCLUSIONS AND IMPLICATIONS

Barriers exist to optimal app use for nutrition education. However, accessible app design, app selection guides, and research expounding the effects of apps and their use by dietitians may improve how practitioners incorporate apps into nutrition education.

Human-to-Computer Interactivity Features Incorporated Into Behavioral Health mHealth Apps: Systematic Search

BACKGROUND

While there are thousands of behavioral health apps available to consumers, users often quickly discontinue their use, which limits their therapeutic value. By varying the types and number of ways that users can interact with behavioral health mobile health apps, developers may be able to support greater therapeutic engagement and increase app stickiness.

OBJECTIVE

The main objective of this analysis was to systematically characterize the types of user interactions that are available in behavioral health apps and then examine if greater interactivity was associated with greater user satisfaction, as measured by app metrics.

METHODS

Using a modified PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) methodology, we searched several different app clearinghouse websites and identified 76 behavioral health apps that included some type of interactivity. We then filtered the results to ensure we were examining behavioral health apps and further refined our search to include apps that identified one or more of the following terms: peer or therapist forum, discussion, feedback, professional, licensed, buddy, friend, artificial intelligence, chatbot, counselor, therapist, provider, mentor, bot, coach, message, comment, chat room, community, games, care team, connect, share, and support in the app descriptions. In the final group of 34 apps, we examined the presence of 6 types of human-machine interactivities: human-to-human with peers, human-to-human with providers, human-to-artificial intelligence, human-to-algorithms, human-to-data, and novel interactive smartphone modalities. We also downloaded information on app user ratings and visibility, as well as reviewed other key app features.

RESULTS

We found that on average, the 34 apps reviewed included 2.53 (SD 1.05; range 1-5) features of interactivity. The most common types of interactivities were human-to-data (n=34, 100%), followed by human-to-algorithm (n=15, 44.2%). The least common type of interactivity was human-artificial intelligence (n=7, 20.5%). There were no significant associations between the total number of app interactivity features and user ratings or app visibility. We found that a full range of therapeutic interactivity features were not used in behavioral health apps.

CONCLUSIONS

Ideally, app developers would do well to include more interactivity features in behavioral health apps in order to fully use the capabilities of smartphone technologies and increase app stickiness. Theoretically, increased user engagement would occur by using multiple types of user interactivity, thereby maximizing the benefits that a person would receive when using a mobile health app.

Is the uptake, engagement, and effectiveness of exclusively mobile interventions for the promotion of weight-related behaviors equal for all? A systematic review

Mobile health interventions are promising behavior change tools. However, there is a concern that they may benefit some populations less than others and thus widen inequalities in health. This systematic review investigated differences in uptake of, engagement with, and effectiveness of mobile interventions for weight-related behaviors (i.e., diet, physical activity, and sedentary behavior) based on a range of inequality indicators including age, gender, race/ethnicity, and socioeconomic status. The protocol was registered on PROSPERO (CRD42020192473). Six databases (CINAHL, EMBASE, ProQuest, PsycINFO, Pubmed, and Web of Science) were searched from inception to July 2021. Publications were eligible for inclusion if they reported the results of an exclusively mobile intervention and examined outcomes by at least one inequality indicator. Sixteen publications reporting on 13 studies were included with most reporting on multiple behaviors and inequality indicators. Uptake was investigated in one study with no differences reported by the inequality indicators studied. Studies investigating engagement (n = 7) reported differences by age (n = 1), gender (n = 3), ethnicity (n = 2), and education (n = 2), while those investigating effectiveness (n = 9) reported differences by age (n = 3), gender (n = 5), education (n = 2), occupation (n = 1), and geographical location (n = 1). Given the limited number of studies and their inconsistent findings, evidence of the presence of a digital divide in mobile interventions targeting weight-related behaviors is inconclusive. Therefore, we recommend that inequality indicators are specifically addressed, analyzed, and reported when evaluating mobile interventions.

User engagement in mobile apps for people with schizophrenia: A scoping review

Over the past decade, there has been an increase in the number of mobile apps designed for mental health proposes and mHealth has been perceived as a promising approach to help people with schizophrenia to manage their condition. However, adoption rates are relatively low and long-term user engagement is a major issue. The aim of this study is to identify and better understand what strategies and factors may influence user engagement and facilitate prolonged use of apps for people with schizophrenia to better manage their illness. A scoping review was conducted in accordance with the Arksey and O'Malley scoping review framework and following PRISMA ScR guidelines. The sources consisted of searching four electronic databases. Rayyan software was used for this study selection process and a narrative approach was used to synthesize the extracted data. A total of 28 studies which met the inclusion criteria were identified. The engagement strategies included push notifications, message prompts, personalization, application customization, goal setting, game-like features, use of different multimedia formats, social connectedness, support (peers and professionals), reliability of content and quality of feedback received. Some demographic factors may influence adherence such as age, gender, education level and socioeconomic status. Other factors also may play a role impacting engagement: health status, data privacy and security, involvement in design process, incentives for participation, app usage fitting in the user routines, initial training, and constant technical support. Included studies present high heterogeneity in outcome measures and thresholds criteria to assess engagement. Understanding what influences engagement and how to measure it is essential to enhance the design of mobile apps and deliver scalable solutions to help people with schizophrenia better manage their illness in their real-world uptake.

Understanding the Role of Support in Digital Mental Health Programs With Older Adults: Users' Perspective and Mixed Methods Study

BACKGROUND

Digital mental health interventions have the potential to increase mental health support among isolated older adults. However, the older adult population can experience several barriers to accessing and using digital health resources and may need extra support to experience its benefits.

OBJECTIVE

This paper aimed to understand what older adults experience as an important aspect of support during engagement in a digital mental health program. The program entailed 3 months of staff support to participate in digital literacy training and engage with the digital mental health platform myStrength, which offers support for a range of mental health challenges, including depression and anxiety.

METHODS

A total of 30 older adults participated in surveys and interviews to assess their experience of participating in a digital mental health program provided by county mental health services. As part of the program, participants attended 4 classes of digital literacy training, had access to the digital mental health platform myStrength for 2 months with staff support (and 10 months after the program without support), and received support from program staff during the entire 3-month program. Survey data were analyzed using descriptive statistics, and interview data were analyzed using thematic analysis.

RESULTS

A thematic analysis of the interview data revealed that participants valued ongoing support in 3 main areas: technical support to assist them in using technology, guided support to remind them to use myStrength and practice skills they had learned, and social support to enable them to connect with others through the program. Furthermore, participants reported that social connections was the most important aspect of the program and that they were mainly motivated to participate in the program because it was recommended to them by trusted others such as a community partner or because they believed it could potentially help others.

CONCLUSIONS

Our findings can be used to inform the design of future digital mental health programs for older adults who may have unique support needs in terms of dedicated technical support and ongoing guided support to use technology and social support to increase social connectedness.

Analyzing the Impact of Mobile App Engagement on Mental Health Outcomes: Secondary Analysis of the Unwinding Anxiety Program

BACKGROUND

App-based interventions provide a promising avenue for mitigating the burden on mental health services by complimenting therapist-led treatments for anxiety. However, it remains unclear how specific systems' use of app features may be associated with changes in mental health outcomes (eg, anxiety and worry).

OBJECTIVE

This study was a secondary analysis of engagement data from a stage 1 randomized controlled trial testing the impact of the Unwinding Anxiety mobile app among adults with generalized anxiety disorder. The aims of this study were 2-fold: to investigate whether higher microengagement with the primary intervention feature (ie, educational modules) is associated with positive changes in mental health outcomes at 2 months (ie, anxiety, worry, interoceptive awareness, and emotional reactivity) and to investigate whether the use of adjunctive app features is also associated with changes in mental health outcomes.

METHODS

We analyzed the intervention group during the stage 1 trial of the Unwinding Anxiety mobile app. The total use of specific mobile app features and the use specific to each feature were calculated. We used multivariate linear models with a priori significance of α=.05 to investigate the impact of cumulative app use on anxiety, worry, interoceptive awareness, and emotional regulation at 2 months, controlling for baseline scores, age, and education level in all models. Significant relationships between system use metrics and baseline participant characteristics were assessed for differences in use groupings using between-group testing (ie, 2-tailed t tests for continuous data and chi-square analyses for categorical data).

RESULTS

The sample was primarily female (25/27, 93%), and the average age was 42.9 (SD 15.6) years. Educational module completion, the central intervention component, averaged 20.2 (SD 11.4) modules out of 32 for the total sample. Multivariate models revealed that completing >75% of the program was associated with an average 22.6-point increase in interoceptive awareness (b=22.6; SE 8.32; P=.01; 95% CI 5.3-39.8) and an 11.6-point decrease in worry (b=-11.6; SE 4.12; P=.01; 95% CI -20.2 to -3.1). In addition, a single log unit change in the total number of meditations was associated with a 0.62-point reduction in the Generalized Anxiety Disorder-7 scale scores (b=0.62; SE 0.27; P=.005; 95% CI -1.2 to -0.6), whereas a single log unit use of the stress meter was associated with an average of a 0.5-point increase in emotional regulation scores (Five Facet Mindfulness Questionnaire; b=0.5; SE 0.21; P=.03; 95% CI 0.1-0.9).

CONCLUSIONS

This study offers a clearer understanding of the impact of engagement with app features on broader engagement with the health outcomes of interest. This study highlights the importance of comprehensive investigations of engagement during the development of evidence-based mobile apps.

Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management

Background

Individuals with bipolar disorder (BD) are increasingly turning to smartphone applications (apps) for health information and self-management support. While reviews have raised concerns regarding the effectiveness and safety of publicly available apps for BD, apps surveyed may not reflect what individuals with BD are using. The present study had two aims: first, to characterize the use of health apps to support mood and sleep amongst people with BD, and second, to evaluate the quality, safety and functionality of the most commonly used self-management apps.

Methods

A web-based survey was conducted to explore which apps people with BD reported using to support self-management of mood and sleep. The characteristics of the most commonly nominated apps were described using a standardized framework, including their privacy policy, clinical foundations, and functionality.

Results

Respondents (n = 919) were 77.9% female with a mean age of 36.9 years. 41.6% of participants (n = 382) reported using a self-management app to support mood or sleep. 110 unique apps were nominated in relation to mood, and 104 unique apps nominated in relation to sleep; however, most apps were only mentioned once. The nine most frequently nominated apps related to mood and sleep were subject to further evaluation. All reviewed apps offered a privacy policy, however user control over data was limited and the complexity of privacy policies was high. Only one app was developed for BD populations. Half of reviewed apps had published peer-reviewed evidence to support their claims of efficacy, but little research was specific to BD.

Conclusion

Findings illustrate the potential of smartphone apps to increase the reach of psychosocial interventions amongst people with BD. Apps were largely created by commercial developers and designed for the general population, highlighting a gap in the development and dissemination of evidence-informed apps for BD. There may be risks in using generic health apps for BD self-management; clinicians should enquire about patients’ app use to foster conversations about their particular benefits and limitations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40345-022-00256-6.

Use of smartphone apps in bipolar disorder: An international web-based survey of feature preferences and privacy concerns

BACKGROUND

Smartphone apps have potential for delivering interventions and supporting self-management in bipolar disorder (BD), however clinical trials of mental health apps have high dropout rates and low sustained use in real-world contexts. To support the development of app-based interventions, we explored use of and attitudes towards apps amongst people with BD, specifically concerns about privacy and preferences for various app features.

METHODS

An international web-based survey was used to investigate concerns about privacy and the perceived importance of various app features among people with BD. Quantitative findings were summarised using descriptive statistics. Qualitative content analysis was used to investigate free-text responses.

RESULTS

A total of 919 people completed the survey; 97.5% reported using smartphone apps in general. Concerns regarding data security were prevalent. Commonly prioritised mHealth features included content quality/accuracy, ease and flexibility of use, cost, and data security. The ability to share data with others, rewards for use, inter-app connectivity, and peer support were endorsed as important by fewer than half of respondents. Qualitative findings suggested that sustained app use could be supported by novel and positive content, customisation, meaningful use of data, interactivity, and perceived real-world benefits.

CONCLUSIONS

The findings of the present study offer important design considerations for the development and evaluation of future app-based interventions for BD. Importantly, some features that have previously been suggested as clinically beneficial or likely to support engagement were perceived ambivalently, emphasising the need for in-depth consultation with potential end users during app development.

The extent of engagement with telehealth approaches by patients with advanced cancer: A systematic review (Preprint)

Background

Telehealth approaches are increasingly being used to support patients with advanced diseases, including cancer. Evidence suggests that telehealth is acceptable to most patients; however, the extent of and factors influencing patient engagement remain unclear.

Objective

The aim of this review is to characterize the extent of engagement with telehealth interventions in patients with advanced, incurable cancer reported in the international literature.

Methods

This systematic review was registered with PROSPERO (International Prospective Register of Systematic Reviews) and is reported in line with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines. A comprehensive search of databases was undertaken for telehealth interventions (communication between a patient with advanced cancer and their health professional via telehealth technologies), including MEDLINE, Embase, CINAHL, PsycINFO, Cochrane Library, Sociological Abstracts, and Web of Science, from the inception of each electronic database up until December 31, 2020. A narrative synthesis was conducted to outline the design, population, and context of the studies. A conceptual framework of digital engagement comprising quantitative behavioral measures (frequency, amount, duration, and depth of use) framed the analysis of engagement with telehealth approaches. Frequency data were transformed to a percentage (actual patient engagement as a proportion of intended engagement), and the interventions were characterized by intensity (high, medium, and low intended engagement) and mode of delivery for standardized comparisons across studies.

Results

Of the 19,676 identified papers, 40 (0.2%) papers covering 39 different studies were eligible for inclusion, dominated by US studies (22/39, 56%), with most being research studies (26/39, 67%). The most commonly reported measure of engagement was frequency (36/39, 92%), with substantial heterogeneity in the way in which it was measured. A standardized percentage of actual patient engagement was derived from 17 studies (17/39, 44%; n=1255), ranging from 51% to 100% with a weighted average of 75.4% (SD 15.8%). A directly proportional relationship was found between intervention intensity and actual patient engagement. Higher engagement occurred when a tablet, computer, or smartphone app was the mode of delivery.

Conclusions

Understanding engagement for people with advanced cancer can guide the development of telehealth approaches from their design to monitoring as part of routine care. With increasing telehealth use, the development of meaningful and context- and condition-appropriate measures of telehealth engagement is needed to address the current heterogeneity in reporting while improving the understanding of optimal implementation of telehealth for oncology and palliative care.

Trial Registration

PROSPERO (International Prospective Register of Systematic Reviews) CRD42018117232; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018117232

Engagement with a digital therapeutic for smoking cessation designed for persons with psychiatric illness fully mediates smoking outcomes in a pilot randomized controlled trial

Understanding the mechanisms of change of digital therapeutics is a critical step to improve digital health outcomes and optimize their development. Access to and engagement with digital content is arguably a core mechanism of change of these interventions. However, the mediational role of app engagement has been largely unexamined. To evaluate the mediational effect of engaging with a digital therapeutic for smoking cessation designed for adults with psychiatric disorders. Secondary analysis of a pilot clinical trial of 62 adults with serious mental illness who were randomized to receive either a tailored digital therapeutic (Learn to Quit) or a digital therapeutic for the general public (NCI QuitGuide). Engagement was captured using background analytics of app utilization, including (a) number of interactions with app content, (b) minutes/day of app use, and (c) number of days used. The main outcome was reductions in cigarettes per day from baseline to the four-month endpoint. Mediational analysis followed the Preacher and Hayes bootstrap method. Number of application interactions fully mediated reductions in cigarettes per day in the Learn to Quit application but not in QuitGuide (Average Causal Mediation Effect = .31, p = .02). Minutes/day of app use played an uncertain role, and number of days used was not a significant mediator. Results suggest that one of the mechanisms of action of the Learn to Quit device, engagement with theory-based content, functioned as intended. Future research of digital therapeutics should emphasize granular approaches to evaluating apps' mechanisms of action.