Engagement with mobile health interventions for depression: A systematic review

A Randomized Evaluation of MoodFX, a Patient-Centred e-Health Tool to Support Outcome Measurement for Depression: Une évaluation randomisée de MoodFX, un outil de santé en ligne centré sur le patient pour soutenir la mesure du résultat dans la dépression

BACKGROUND

e-Health tools using validated questionnaires to assess outcomes may facilitate measurement-based care for psychiatric disorders. MoodFX was created as a free online symptom tracker to support patients for outcome measurement in their depression treatment. We conducted a pilot randomized evaluation to examine its usability, and clinical utility.

METHODS

Patients presenting with a major depressive episode (within a major depressive or bipolar disorder) were randomly assigned to receive either MoodFX or a health information website as the intervention and control condition, respectively, with follow-up assessment surveys conducted online at baseline, 8 weeks and 6 months. The primary usability outcomes included the percentage of patients with self-reported use of MoodFX 3 or more times during follow up (indicating minimally adequate usage) and usability measures based on the System Usability Scale (SUS). Secondary clinical outcomes included the Quick Inventory of Depressive Symptomatology, Self-Rated (QIDS-SR) and Patient Health Questionnaire (PHQ-9).

RESULTS

Forty-nine participants were randomized (24 to MoodFX and 25 to the control condition). Of the 23 participants randomized to MoodFX who completed the user survey, 18 (78%) used MoodFX 3 or more times over the 6 months of the study. The mean SUS score of 72.7 (65th-69th percentile) represents good usability. Compared to the control group, the MoodFX group had significantly better improvement on QIDS-SR and PHQ-9 scores, with large effect sizes and higher response rates at 6 months. There were no differences between conditions on other secondary outcomes such as functioning and quality of life.

CONCLUSION

MoodFX demonstrated good usability and was associated with reduction in depressive symptoms. This pilot study supports the use of digital tools in depression treatment.

The Potential for Digital Phenotyping in Understanding Mindfulness App Engagement Patterns: A Pilot Study

Background: Low app engagement is a central barrier to digital mental health efficacy. With mindfulness-based mental health apps growing in popularity, there is a need for new understanding of factors influencing engagement. This study utilized digital phenotyping to understand real-time patterns of engagement around app-based mindfulness. Different engagement metrics are presented that measure both the total number of app-based activities participants completed each week, as well as the proportion of days that participants engaged with the app each week. Method: Data were derived from two iterations of a four-week study exploring app engagement in college students (n = 169). This secondary analysis investigated the relationships between general and mindfulness-based app engagement with passive data metrics (sleep duration, home time, and screen duration) at a weekly level, as well as the relationship between demographics and engagement. Additional clinically focused analysis was performed on three case studies of participants with high mindfulness activity completion. Results: Demographic variables such as gender, race/ethnicity, and age lacked a significant association with mindfulness app-based engagement. Passive data variables such as sleep and screen duration were significant predictors for different metrics of general and mindfulness-based app engagement at a weekly level. There was a significant interaction effect for screen duration between the number of mindfulness activities completed and whether or not the participant received a mindfulness notification. K-means clusters analyses using passive data features to predict mindfulness activity completion had low performance. Conclusions: While there are no simple solutions to predicting engagement with mindfulness apps, utilizing digital phenotyping approaches at a population and personal level offers new potential. The signal from digital phenotyping warrants more investigation; even small increases in engagement with mindfulness apps may have a tremendous impact given their already high prevalence of engagement, availability, and potential to engage patients across demographics.

The conceptualisation and measurement of engagement in digital health

Digital tools are an increasingly important component of healthcare, but their potential impact is commonly limited by a lack of user engagement. Digital health evaluations of engagement are often restricted to system usage metrics, which cannot capture a full understanding of how and why users engage with an intervention. This study aimed to examine how theory-based, multifaceted measures of engagement with digital health interventions capture different components of engagement (affective, cognitive, behavioural, micro, and macro) and to consider areas that are unclear or missing in their measurement. We identified and compared two recently developed measures that met these criteria (the Digital Behaviour Change Intervention Engagement Scale and the TWente Engagement with Ehealth Technologies Scale). Despite having similar theoretical bases and being relatively strongly correlated, there are key differences in how these scales aim to capture engagement. We discuss the implications of our analysis for how affective, cognitive, and behavioural components of engagement can be conceptualised and whether there is value in distinguishing between them. We conclude with recommendations for the circumstances in which each scale may be most useful and for how future measure development could supplement existing scales.

Effectiveness of a chatbot in improving the mental wellbeing of health workers in Malawi during the COVID-19 pandemic: A randomized, controlled trial

We conducted a randomized, controlled trial (RCT) to investigate our hypothesis that the interactive chatbot, Vitalk, is more effective in improving mental wellbeing and resilience outcomes of health workers in Malawi than the passive use of Internet resources. For our 2-arm, 8-week, parallel RCT (ISRCTN Registry: trial ID ISRCTN16378480), we recruited participants from 8 professional cadres from public and private healthcare facilities. The treatment arm used Vitalk; the control arm received links to Internet resources. The research team was blinded to the assignment. Of 1,584 participants randomly assigned to the treatment and control arms, 215 participants in the treatment and 296 in the control group completed baseline and endline anxiety assessments. Six assessments provided outcome measures for: anxiety (GAD-7); depression (PHQ-9); burnout (OLBI); loneliness (ULCA); resilience (RS-14); and resilience-building activities. We analyzed effectiveness using mixed-effects linear models, effect size estimates, and reliable change in risk levels. Results support our hypothesis. Difference-in-differences estimators showed that Vitalk reduced: depression (-0.68 [95% CI -1.15 to -0.21]); anxiety (-0.44 [95% CI -0.88 to 0.01]); and burnout (-0.58 [95% CI -1.32 to 0.15]). Changes in resilience (1.47 [95% CI 0.05 to 2.88]) and resilience-building activities (1.22 [95% CI 0.56 to 1.87]) were significantly greater in the treatment group. Our RCT produced a medium effect size for the treatment and a small effect size for the control group. This is the first RCT of a mental health app for healthcare workers during the COVID-19 pandemic in Southern Africa combining multiple mental wellbeing outcomes and measuring resilience and resilience-building activities. A substantial number of participants could have benefited from mental health support (1 in 8 reported anxiety and depression; 3 in 4 suffered burnout; and 1 in 4 had low resilience). Such help is not readily available in Malawi. Vitalk has the potential to fill this gap.

Macro-engagement in mHealth: Exploring user engagement beyond the screen

While digital technology holds great promise for health and well-being, some users feel sceptical about the time they spend online and how they use their mobile devices. This attitude could hamper uptake of digital health technologies and engagement with them. This study uses the concept of macro-engagement as a starting point to investigate how users of digital behaviour change interventions (DBCIs) engage with their behaviour change goals beyond the screens of their tools. Thirty semi-structured interviews were conducted with individuals who take part in behaviour change processes in different ways (i.e. mental health professionals, digital health experts and users of DBCIs). A qualitative analysis of their data through a grounded theory approach highlighted a wide array of offscreen behaviors and strategies that complement a behavior change process offscreen. Furthermore, implications for designing technology that encourages progressive non-reliance on DBCI usage are drawn out.

Rates of attrition and engagement in randomized controlled trials of mindfulness apps: Systematic review and meta-analysis

Randomized controlled trials (RCT) have established the efficacy of mindfulness apps for numerous mental health and behavioral outcomes. However, studies of this kind are characterized by significant problems with attrition and engagement, making it challenging to interpret these findings and realize the full potential of mindfulness app-based solutions. This systematic review and meta-analysis aimed to examine rates of attrition from, and engagement in, mindfulness apps in 70 RCTs located through four databases. The weighted meta-analytic attrition rate from 9258 participants allocated to a mindfulness app was 24.7%. This estimate was higher (38.7%) when restricting analyses to larger studies (n ≥ 100 per condition) and for specific target samples (e.g., general population of adults). Studies that offered participants monetary compensation and required in-person enrolment methods were associated with the lowest attrition rates. Attrition was higher in mindfulness app conditions relative to waitlists, but was not different to placebo or active apps. Although engagement data were inadequately reported, qualitative synthesis highlighted clear problems with sustained user engagement with mindfulness apps. Findings have implications for powering, designing and appraising research trials of mindfulness apps, as well as the development of potentially effective retention and engagement strategies worthy of evaluation in future work.

Retention in an mHealth App Aiming to Promote the Use of HIV Pre-Exposure Prophylaxis Among Female Sex Workers in Dar es Salaam, Tanzania: Prospective Cohort Study

Background

Increasing access to smartphones in sub-Saharan Africa offers an opportunity to leverage mobile health (mHealth) technology to improve access to health care in underserved populations. In the domain of HIV prevention, mHealth interventions can potentially contribute to solving the challenges of suboptimal adherence to pre-exposure prophylaxis (PrEP) and low retention in PrEP services among populations most vulnerable to HIV acquisition. However, there is a gap in the knowledge about the use of such interventions in sub-Saharan Africa.

Objective

This study aims to evaluate the extent and predictors of retention in an mHealth app (Jichunge) that aims to promote adherence to PrEP and retention in PrEP care among female sex workers in Dar es Salaam, Tanzania.

Methods

A prospective cohort of female sex workers residing in Dar es Salaam were recruited, using respondent-driven sampling. All participants were provided with the Jichunge app as they started PrEP. A questionnaire was used to collect data on sociodemographics and other structural factors, while app use data for the 60-day period following the first 150 days of being in the intervention arm were extracted from the app's back end. A multivariable log-binomial model was used to determine predictors of 6-month retention in the Jichunge app.

Results

A total of 470 female sex workers were recruited. Nearly three-quarters of participants (206/284, 72.5%) who came to the 6-month follow-up interview no longer had the Jichunge app on their phones. The majority of these participants (193/206, 93.7%) no longer had access to the app because of issues related to their phones. Data extracted from the back end of the app showed that the use of the app declined over time, and only 13.4% (63/470) of the participants were retained (continued to use the app) after 6 months of intervention. At 6 months, women aged ≥35 years were >2 times more likely to use the app than women aged 18 to 24 years (adjusted risk ratio [aRR] 2.2, 95% CI 1.2-4.1; P=.01). Furthermore, retention in the app was higher among participants who demonstrated high PrEP awareness at baseline (aRR 1.8, 95% CI 1.1-3; P=.01) and among those who had experienced financial difficulties due to health care spending (aRR 1.9, 95% CI 1.2-3.2; P=.01).

Conclusions

Most female sex workers (206/284, 72.5%) who were enrolled in PrEP care in Tanzania no longer used the Jichunge app after 6 months. Retention in the app at 6 months was predicted by older age, high PrEP awareness, and financial difficulties due to health care spending. Strategies for the long-term retention of participants in mHealth apps, such as systems for reinstallations of apps, should be considered during the design phase.

The impact of guided versus supportive coaching on mental health app engagement and clinical outcomes

Although mobile mental health apps have the unique potential to increase access to care, evidence reveals engagement is low unless coupled with coaching. However, most coaching protocols are limited in their scalability. This study assesses how human support and guidance from a Digital Navigator (DN), a scalable coach, can impact mental health app engagement and effectiveness on anxiety and depressive symptoms. This study aims to detach components of coaching, specifically personalized recommendations versus general support, to inform scalability of coaching models for mental health apps. 156 participants were split into the DN Guide versus DN Support groups for the 6-week study. Both groups utilized the mindLAMP app for the duration of the study and had equal time with the DN, but the Guide group received personalized app recommendations. The Guide group completed significantly more activities than the Support group. 34% (49/139) of all participants saw a 25% decrease in PHQ-9 scores and 38% (53/141) saw a 25% decrease in GAD-7 scores. These findings show mental health apps, especially when supported by DNs, can reduce depression and anxiety symptoms when coupled with coaching, suggesting a feasible path for large-scale deployment.

Specifying the Efficacy of Digital Therapeutic Tools for Depression and Anxiety: Retrospective, 2-Cohort, Real-World Analysis

BACKGROUND

Depression and anxiety are the main sources of work and social disabilities as well as health-related problems around the world. Digital therapeutic solutions using cognitive behavioral therapy have demonstrated efficacy in depression and anxiety. A common goal of digital health apps is to increase user digital engagement to improve outcomes. However, there is a limited understanding of the association between digital platform components and clinical outcomes.

OBJECTIVE

The aim of the study is to investigate the contribution of specific digital engagement tools to mental health conditions. We hypothesized that participation in coaching sessions and breathing exercises would be associated with a reduction in depression and anxiety.

METHODS

Depression and general anxiety symptoms were evaluated in real-world data cohorts using the digital health platform for digital intervention and monitoring change. This retrospective real-world analysis of users on a mobile platform-based treatment followed two cohorts of people: (1) users who started with moderate levels of depression and completed at least 2 depression assessments (n=519) and (2) users who started with moderate levels of anxiety and completed at least 2 anxiety assessments (n=474). Levels of depression (Patient Health Questionnaire-9) and anxiety (Generalized Anxiety Disorder-7) were tracked throughout the first 16 weeks. A piecewise mixed-effects model was applied to model the trajectories of the Patient Health Questionnaire-9 and the Generalized Anxiety Disorder-7 mean scores in 2 segments (1-6 weeks and 7-16 weeks). Finally, simple slope analysis was used for the interpretation of the interactions probing the moderators: coaching sessions and breathing exercises in both depression and anxiety cohorts.

RESULTS

Analysis revealed a significant decrease in depression symptoms (β=-.37, 95% CI -0.46 to 0.28; P≤.001) during the period of weeks 1-6 of app use, which was maintained during the period of 7-16 weeks. Coach interaction significantly moderated the reduction in depression symptoms during the period of weeks 1-6 (β=-.03, 95% CI -0.05 to -0.001; P=.02). A significant decrease in anxiety symptoms (β=-.41, 95% CI -0.50 to -0.33; P≤.001) was revealed during the period of 1-6 weeks, which was maintained during the period of 7-16 weeks. Breathing exercises significantly moderated the reduction in anxiety symptoms during the period of 1-6 weeks (β=-.07, 95% CI -0.14 to -0.01; P=.04).

CONCLUSIONS

This study demonstrated general improvement followed by a period of stability of depression and anxiety symptoms associated with cognitive behavioral therapy-based digital intervention. Interestingly, engagement with a coaching session but not a breathing exercise was associated with a reduction in depression symptoms. Moreover, breathing exercise but not engagement with a coaching session was associated with a reduction of anxiety symptoms. These findings emphasize the importance of using a personalized approach to behavioral health during digital health interventions.

Assessing Patient Adherence to and Engagement With Digital Interventions for Depression in Clinical Trials: Systematic Literature Review

BACKGROUND

New approaches to the treatment of depression are necessary for patients who do not respond to current treatments or lack access to them because of barriers such as cost, stigma, and provider shortage. Digital interventions for depression are promising; however, low patient engagement could limit their effectiveness.

OBJECTIVE

This systematic literature review (SLR) assessed how participant adherence to and engagement with digital interventions for depression have been measured in the published literature, what levels of adherence and engagement have been reported, and whether higher adherence and increased engagement are linked to increased efficacy.

METHODS

We focused on a participant population of adults (aged ≥18 years) with depression or major depressive disorder as the primary diagnosis and included clinical trials, feasibility studies, and pilot studies of digital interventions for treating depression, such as digital therapeutics. We screened 756 unique records from Ovid MEDLINE, Embase, and Cochrane published between January 1, 2000, and April 15, 2022; extracted data from and appraised the 94 studies meeting the inclusion criteria; and performed a primarily descriptive analysis. Otsuka Pharmaceutical Development & Commercialization, Inc (Princeton, New Jersey, United States) funded this study.

RESULTS

This SLR encompassed results from 20,111 participants in studies using 47 unique web-based interventions (an additional 10 web-based interventions were not described by name), 15 mobile app interventions, 5 app-based interventions that are also accessible via the web, and 1 CD-ROM. Adherence was most often measured as the percentage of participants who completed all available modules. Less than half (44.2%) of the participants completed all the modules; however, the average dose received was 60.7% of the available modules. Although engagement with digital interventions was measured differently in different studies, it was most commonly measured as the number of modules completed, the mean of which was 6.4 (means ranged from 1.0 to 19.7) modules. The mean amount of time participants engaged with the interventions was 3.9 (means ranged from 0.7 to 8.4) hours. Most studies of web-based (34/45, 76%) and app-based (8/9, 89%) interventions found that the intervention group had substantially greater improvement for at least 1 outcome than the control group (eg, care as usual, waitlist, or active control). Of the 14 studies that investigated the relationship between engagement and efficacy, 9 (64%) found that increased engagement with digital interventions was significantly associated with improved participant outcomes. The limitations of this SLR include publication bias, which may overstate engagement and efficacy, and low participant diversity, which reduces the generalizability.

CONCLUSIONS

Patient adherence to and engagement with digital interventions for depression have been reported in the literature using various metrics. Arriving at more standardized ways of reporting adherence and engagement would enable more effective comparisons across different digital interventions, studies, and populations.

Integrating the "Quit and Stay Quit Monday" Model into Smoking Cessation Services for Smokers with Mental Health Conditions: A Pilot Randomized Controlled Trial

Introduction

People with mental health conditions (MHCs) are less likely to achieve long-term abstinence than people without MHCs. The Quit and Stay Quit Monday (QSQM) model offers a long-term approach to treating tobacco use by encouraging people to quit, requit, or recommit to quit smoking every Monday.

Aim

To evaluate the efficacy, patient satisfaction, and patient engagement with an intervention that integrated the QSQM model into multicomponent smoking cessation services among people with an MHC.

Methods

This was a randomized controlled pilot trial. Eligibility criteria were as follows: (1) ≥18 years old, (2) smoked a cigarette in the past 30 days, (3) diagnosis of an ICD-10 MHC, (4) interest in quitting smoking, (5) able to receive services in English, and (5) had an active email and a cell phone. The intervention group (n = 33) received QSQM-focused telephone coaching, a weekly QSQM email newsletter, a SmokefreeTXT anchored around a Monday quit date, and 4 weeks of nicotine replacement therapy (NRT). The control group (n = 36) received information about contacting their state Quitline for usual services. Primary outcomes were self-reported quit attempts, 7-day abstinence, and intervention satisfaction at 3 months.

Results

Twenty-four participants (73%) in the intervention group began telephone coaching, 26 (79%) enrolled in the QSQM email newsletter, 19 (58%) enrolled in SmokefreeTXT, and 15 (46%) used NRT. Using a penalized intent-to-treat approach, quit attempts in the intervention and control groups were 63.6% and 38.9% (OR 2.75, 95% CI 1.03-7.30), respectively. Seven-day abstinence in the two groups was 12.1% and 5.6% (OR 2.35, 95% CI 0.40-13.74), respectively. Of the 15 intervention group participants who set a quit date during the intervention, 13 (86.7%) selected a Monday quit day. Qualitative interviews revealed positive participant experiences with picking a Monday quit day. On follow-up surveys, 89.5%, 69.3%, and 64.3% of intervention participants reported that the counseling, QSQM email, and text messaging, respectively, were very or somewhat helpful.

Conclusions

The QSQM model was acceptable and potentially efficacious among people with MHCs, but intervention engagement and satisfaction were modest. Future research should adapt or develop new QSQM delivery approaches to improve patient engagement and potential efficacy of the model. This trial is registered with clinicaltrials.gov (NCT04512248).

Assessing the Dynamics of the Mental Health Apple and Android App Marketplaces

Barriers to mental health care, including stigma, costs, and mental health professional shortages, have been exacerbated by the COVID-19 pandemic. Smartphone apps have the potential to increase scalability and improve access to mental health information, support, and interventions. However, evaluating these apps and selecting ones for use in care remain challenging, especially as apps are often updating and changing. Recommending apps requires knowledge of how stable apps are as the experience of one user several months ago may or may not be the same. A sample of 347 apps of the 650 apps on the M-health Index and Navigation Database (MIND) https://mindapps.org were reviewed between September 1, 2021, and January 5, 2022. Apps were selected by time since their last review, with updates occurring on average approximately 4 months from the last review. Eleven trained app evaluators reviewed apps across 105 evaluation criteria in 9 categories. Results were compared to initial ratings, identifying the changes that occurred. The average app updates every 433 days, though 19% were updated in the last 3 months and some nearly weekly. Changes in privacy and features made up the highest percentage of changes, both at 38%. The most frequently observed privacy-related change was increased privacy policy reading level. Functionality parameters changed in 28% of apps. The most common functionality change was the removal of an accessibility feature. Clinical foundations changed in 18% of apps and 9% added supporting studies. Cost structure changed in 17% of apps, with 10% adding a fee for use of the app. Engagement features changed in 17% of the apps, with additions and removals of validated assessments or screeners most common. The dynamic nature of the app stores is reflected in app privacy, features, and functionality. These changes, reflected by the increased reading levels required to understand privacy policies, the decrease in accessibility features, and the additions of fees to access mobile apps, reflect the need to constantly review apps and understand how they are evolving. Patient and clinicians should use the most recent and updated possible when evaluating apps.

Smartphone app engagement and clinical outcomes in a hybrid clinic

Despite the growing prevalence of mental health-related smartphone apps, low real-world engagement has prevented these apps from transforming the mental health landscape. Integrating mental health apps into more traditional therapeutic models appears to support better clinical outcomes, but also raises questions about the relationship between app engagement, the app itself, and the coach or clinician. This study explores patient app engagement patterns and the associated clinical outcomes gathered from piloting a digital clinic. Patients with anxiety or depression completed eight clinical visits and coach visits over a median of 83 days with a standard deviation of 17.25 days. Between clinical visits, patients completed therapeutic activities on the mindLAMP app. Mean PHQ-9 and GAD-7 scores decreased from the intake visit to both visit 4 and visit 8. Patients had high app engagement, but engagement did not correlate with outcomes. From intake visit to visit 4, the interaction effects indicate significant differences in the change of both PHQ-9 and GAD-7 depending on participants' average app satisfaction and clinician/coach satisfaction (as measured by WAI-SR) with engagement. Overall, results support the feasibility of incorporating an app into a hybrid clinic.

Current challenges for evaluating mobile health applications

As mobile health applications continue to proliferate without clear regulation, the need for app evaluation frameworks to offer guidance to patients and clinicians also expands. However, this expanding number of app evaluation frameworks itself can be a source of confusion and often contradictory recommendations. In pursuit of better frameworks that offer innovation for app evaluation, we present 4 challenges that app evaluation frameworks must overcome as well as examples from our own experience toward overcoming them. The recommendations are applicable to all health apps from any field of medicine, although we use examples from mental health as they are illustrative.

Evaluating Conversational Agents for Mental Health: Scoping Review of Outcomes and Outcome Measurement Instruments (Preprint)

BACKGROUND

Rapid proliferation of mental health interventions delivered through conversational agents (CAs) calls for high-quality evidence to support their implementation and adoption. Selecting appropriate outcomes, instruments for measuring outcomes, and assessment methods are crucial for ensuring that interventions are evaluated effectively and with a high level of quality.

OBJECTIVE

We aimed to identify the types of outcomes, outcome measurement instruments, and assessment methods used to assess the clinical, user experience, and technical outcomes in studies that evaluated the effectiveness of CA interventions for mental health.

METHODS

We undertook a scoping review of the relevant literature to review the types of outcomes, outcome measurement instruments, and assessment methods in studies that evaluated the effectiveness of CA interventions for mental health. We performed a comprehensive search of electronic databases, including PubMed, Cochrane Central Register of Controlled Trials, Embase (Ovid), PsychINFO, and Web of Science, as well as Google Scholar and Google. We included experimental studies evaluating CA mental health interventions. The screening and data extraction were performed independently by 2 review authors in parallel. Descriptive and thematic analyses of the findings were performed.

RESULTS

We included 32 studies that targeted the promotion of mental well-being (17/32, 53%) and the treatment and monitoring of mental health symptoms (21/32, 66%). The studies reported 203 outcome measurement instruments used to measure clinical outcomes (123/203, 60.6%), user experience outcomes (75/203, 36.9%), technical outcomes (2/203, 1.0%), and other outcomes (3/203, 1.5%). Most of the outcome measurement instruments were used in only 1 study (150/203, 73.9%) and were self-reported questionnaires (170/203, 83.7%), and most were delivered electronically via survey platforms (61/203, 30.0%). No validity evidence was cited for more than half of the outcome measurement instruments (107/203, 52.7%), which were largely created or adapted for the study in which they were used (95/107, 88.8%).

CONCLUSIONS

The diversity of outcomes and the choice of outcome measurement instruments employed in studies on CAs for mental health point to the need for an established minimum core outcome set and greater use of validated instruments. Future studies should also capitalize on the affordances made available by CAs and smartphones to streamline the evaluation and reduce participants' input burden inherent to self-reporting.

Analyzing the Impact of Mobile App Engagement on Mental Health Outcomes: Secondary Analysis of the Unwinding Anxiety Program

BACKGROUND

App-based interventions provide a promising avenue for mitigating the burden on mental health services by complimenting therapist-led treatments for anxiety. However, it remains unclear how specific systems' use of app features may be associated with changes in mental health outcomes (eg, anxiety and worry).

OBJECTIVE

This study was a secondary analysis of engagement data from a stage 1 randomized controlled trial testing the impact of the Unwinding Anxiety mobile app among adults with generalized anxiety disorder. The aims of this study were 2-fold: to investigate whether higher microengagement with the primary intervention feature (ie, educational modules) is associated with positive changes in mental health outcomes at 2 months (ie, anxiety, worry, interoceptive awareness, and emotional reactivity) and to investigate whether the use of adjunctive app features is also associated with changes in mental health outcomes.

METHODS

We analyzed the intervention group during the stage 1 trial of the Unwinding Anxiety mobile app. The total use of specific mobile app features and the use specific to each feature were calculated. We used multivariate linear models with a priori significance of α=.05 to investigate the impact of cumulative app use on anxiety, worry, interoceptive awareness, and emotional regulation at 2 months, controlling for baseline scores, age, and education level in all models. Significant relationships between system use metrics and baseline participant characteristics were assessed for differences in use groupings using between-group testing (ie, 2-tailed t tests for continuous data and chi-square analyses for categorical data).

RESULTS

The sample was primarily female (25/27, 93%), and the average age was 42.9 (SD 15.6) years. Educational module completion, the central intervention component, averaged 20.2 (SD 11.4) modules out of 32 for the total sample. Multivariate models revealed that completing >75% of the program was associated with an average 22.6-point increase in interoceptive awareness (b=22.6; SE 8.32; P=.01; 95% CI 5.3-39.8) and an 11.6-point decrease in worry (b=-11.6; SE 4.12; P=.01; 95% CI -20.2 to -3.1). In addition, a single log unit change in the total number of meditations was associated with a 0.62-point reduction in the Generalized Anxiety Disorder-7 scale scores (b=0.62; SE 0.27; P=.005; 95% CI -1.2 to -0.6), whereas a single log unit use of the stress meter was associated with an average of a 0.5-point increase in emotional regulation scores (Five Facet Mindfulness Questionnaire; b=0.5; SE 0.21; P=.03; 95% CI 0.1-0.9).

CONCLUSIONS

This study offers a clearer understanding of the impact of engagement with app features on broader engagement with the health outcomes of interest. This study highlights the importance of comprehensive investigations of engagement during the development of evidence-based mobile apps.

A systematic review of engagement reporting in remote measurement studies for health symptom tracking

Remote Measurement Technologies (RMTs) could revolutionise management of chronic health conditions by providing real-time symptom tracking. However, the promise of RMTs relies on user engagement, which at present is variably reported in the field. This review aimed to synthesise the RMT literature to identify how and to what extent engagement is defined, measured, and reported, and to present recommendations for the standardisation of future work. Seven databases (Embase, MEDLINE and PsycINFO (via Ovid), PubMed, IEEE Xplore, Web of Science, and Cochrane Central Register of Controlled Trials) were searched in July 2020 for papers using RMT apps for symptom monitoring in adults with a health condition, prompting users to track at least three times during the study period. Data were synthesised using critical interpretive synthesis. A total of 76 papers met the inclusion criteria. Sixty five percent of papers did not include a definition of engagement. Thirty five percent included both a definition and measurement of engagement. Four synthetic constructs were developed for measuring engagement: (i) engagement with the research protocol, (ii) objective RMT engagement, (iii) subjective RMT engagement, and (iv) interactions between objective and subjective RMT engagement. The field is currently impeded by incoherent measures and a lack of consideration for engagement definitions. A process for implementing the reporting of engagement in study design is presented, alongside a framework for definition and measurement options available. Future work should consider engagement with RMTs as distinct from the wider eHealth literature, and measure objective versus subjective RMT engagement.Registration: This review has been registered on PROSPERO [CRD42020192652].

Assessment of Smartphone Apps for Common Neurologic Conditions (Headache, Insomnia, and Pain): Cross-sectional Study

BACKGROUND

There are thousands of apps for individuals struggling with headache, insomnia, and pain, but it is difficult to establish which of these apps are best suited for patients' specific needs. If clinicians were to have access to a platform that would allow them to make an informed decision on the efficacy and feasibility of smartphone apps for patient care, they would feel confident in prescribing specific apps.

OBJECTIVE

We sought to evaluate the quality of apps for some of the top common, disabling neurologic conditions (headache, insomnia, and pain) based on principles derived from the American Psychiatric Association's (APA) app evaluation model.

METHODS

We used the Mobile Health Index and Navigation database and expanded upon the database's current supported conditions by adding 177 new app entries. Each app was rated for consistency with the APA's app evaluation model, which includes 105 objective questions based on the following 5 major classes of consideration: (1) accessibility, (2) privacy and security, (3) clinical foundation, (4) engagement style, and (5) interoperability. These characteristics were evaluated to gain a broader understanding of the significant features of each app category in comparison against a control group.

RESULTS

Approximately 90% (187/201) of all apps evaluated were free to download, but only 50% (63/201) of headache- and pain-related apps were truly free. Most (87/106, 81%) sleep apps were not truly free to use. The apps had similar limitations with limited privacy, accessibility, and crisis management resources. For example, only 17% (35/201) of the apps were available in Spanish. The apps offered mostly self-help tools with little tailoring; symptom tracking was the most common feature in headache- (32/48, 67%) and pain-related apps (21/47, 45%), whereas mindfulness was the most common feature in sleep-related apps (73/106, 69%).

CONCLUSIONS

Although there are many apps for headache, pain, and insomnia, all 3 types of apps have room for improvement around accessibility and privacy. Pain and headache apps share many common features, whereas insomnia apps offer mostly mindfulness-based resources. Given the many available apps to pick from, clinicians and patients should seek apps that offer the highest-quality features, such as complete privacy, remedial features, and the ability to download the app at no cost. These results suggest that there are many opportunities for the improvement of apps centered on headache, insomnia, and pain.

Association between Care Modality and Utilization with Treatment Response among Members Accessing Virtual Mental Health Services: A Real-World Observational Study (Preprint)

Background

There is a growing bottleneck in mental health care, as the demand for services has outpaced the availability of mental health professionals. Consequently, many health systems have shifted to teletherapy as a scalable approach to increasing accessibility to care. Within these care models, various treatment modalities (eg, coaching and clinical care) are used to deliver support for anxiety and depression. However, more research is needed to better understand the differences in treatment responses.

Objective

The purpose of this study was to examine the association between different care modalities and the levels of use with symptom score changes for members seeking virtual care services.

Methods

We conducted an observational study of 4219 members who accessed Ginger, an on-demand mental health service, between September 2020 and September 2021. Using a mobile app, members can access text-based behavioral health coaching and virtual clinical services. This study focused on members with clinically elevated depression or anxiety levels at baseline. Logistic regressions were used to assess the association between care modalities and the levels of use with treatment response in depression and anxiety, using the Patient Health Questionnaire and Generalized Anxiety Disorder Assessment, respectively.

Results

Of the 4219 members, 1623 (38.47%) demonstrated a full response to depression, and 1684 (39.91%) demonstrated a full response to anxiety. Members who completed care (ie, text-based coaching, virtual clinical therapy, hybrid of coaching, and clinical care) beyond the introductory session showed significantly increased odds of a full response compared with those who completed only limited care. Members who completed a hybrid of care had the highest odds of improvement; the odds of showing a full response in depression were 2.31 times higher (95% CI 1.91-2.80; P<.001) and in anxiety were 2.23 times higher (95% CI 1.84-2.70; P<.001) compared with members who completed limited care. For members who completed only coaching or clinical care, the largest effects were observed among those with high use. For members who completed a hybrid care program, we observed similar treatment responses across all levels of use.

Conclusions

Our real-world study found that members who completed text-based coaching achieved full treatment responses at similar rates compared with members who completed virtual clinical care and members who completed a hybrid of care. There were no significant differences in the predicted probabilities of full treatment response between coaching and clinical care. Generally, the odds for a full response were highest among members with high use within each care modality; however, there were no differences in full-response treatment odds across levels of use with hybrid care. The results support the utility of digital behavioral health interventions and further highlight text-based coaching protocols as an accessible and suitable option when considering virtual care for treating anxiety and depression.