Use of a Mobile App to Augment Psychotherapy in a Community Psychiatric Clinic: Feasibility and Fidelity Trial

Evaluation of Virtual Enhanced Child Adult Relationship Enhancement in Primary Care Intervention

OBJECTIVE

Because of COVID-19 pandemic social distancing requirements, the in-person Child Adult Relationship Enhancement in Primary Care (PriCARE) positive parenting intervention was adapted for virtual delivery. Objective was to evaluate the efficacy of the virtual PriCARE program to improve parenting capacity, decrease child behavioral problems, and decrease child maltreatment risk.

METHODS

Caregivers of children 2 to 6 years old recruited from pediatric primary care were randomized to PriCARE (n = 92) or waitlist control (n = 90). Dysfunctional parenting, positive parenting skills, child behaviors, and child maltreatment risk were measured at baseline and 2 to 3 months after intervention using the Parenting Scale (PS), Dyadic Parent-Child Interaction Coding System (DPICS), Eyberg Child Behavior Inventory (ECBI), and Child Abuse Potential Inventory (CAPI). Kruskal-Wallis test compared median change scores from baseline to follow-up by treatment arm.

RESULTS

Of 182 enrolled caregivers, 92% (168) were mothers and 67% (122) completed study measures at baseline and follow-up. The median decrease (improvement) in total PS score was greater in the PriCARE group compared with the control group (-0.3 [IQR 0.69] vs -0.1 [IQR 0.56], p = 0.028) as was the median decrease (improvement) in ECBI problem score (-3 [IQR 9] vs -1 [IQR 7], p = 0.045) and ECBI intensity score (-9 [IQR 21] vs 0 [IQR 25], p = 0.006). Improvements in 4 positive parenting skills measured by DPICS were greater in the PriCARE group compared with the control group (all p< 0.003). Median decrease in CAPI abuse score did not differ significantly by study arm (p = 0.055).

CONCLUSION

The PriCARE virtual adaptation demonstrated promise in promoting positive parenting and decreasing child behavior problems.

Current challenges for evaluating mobile health applications

As mobile health applications continue to proliferate without clear regulation, the need for app evaluation frameworks to offer guidance to patients and clinicians also expands. However, this expanding number of app evaluation frameworks itself can be a source of confusion and often contradictory recommendations. In pursuit of better frameworks that offer innovation for app evaluation, we present 4 challenges that app evaluation frameworks must overcome as well as examples from our own experience toward overcoming them. The recommendations are applicable to all health apps from any field of medicine, although we use examples from mental health as they are illustrative.

Mobile App to Enhance Patient Activation and Patient-Provider Communication in Major Depressive Disorder Management: Collaborative, Randomized Controlled Pilot Study

BACKGROUND

Enhanced patient-provider engagement can improve patient health outcomes in chronic conditions, including major depressive disorder (MDD).

OBJECTIVE

We evaluated the impact of a digitally enabled care mobile app, Pathway, designed to improve MDD patient-provider engagement. Patients used a mobile interface to assess treatment progress and share this information with primary care providers (PCPs).

METHODS

In this 52-week, real-world effectiveness and feasibility study conducted in primary care clinics, 40 patients with MDD who were recently prescribed antidepressant monotherapy were randomized to use a mobile app with usual care (20/40, 50%) or usual care alone (20/40, 50%). Patients in the app arm engaged with the app daily for 18 weeks; a report was generated at 6-week intervals and shared with the PCPs to facilitate shared treatment decision-making discussions. The patients discontinued the app at week 18 and were followed through year 1. Coprimary outcome measures, assessed via research visits, included change from baseline in the 13-item Patient Activation Measure (PAM-13) and 7-item Patient-Provider Engagement Scale scores at week 18. Additional outcome measures included depression severity (9-item Patient Health Questionnaire [PHQ-9]) and cognitive symptoms (5-item Perceived Deficits Questionnaire-Depression).

RESULTS

All 37 patients (app arm: n=18, 49%; usual care arm: n=19, 51%) who completed the 18-week follow-up period (n=31, 84% female, mean age 36, SD 11.3 years) had moderate to moderately severe depression. Improvements in PAM-13 and PHQ-9 scores were observed in both arms. Increases in PAM-13 scores from baseline to 18 weeks were numerically greater in the app arm than in the usual care arm (mean 10.5, SD 13.2 vs mean 8.8, SD 9.4; P=.65). At 52 weeks, differences in PAM-13 scores from baseline demonstrated significantly greater improvements in the app arm than in the usual care arm (mean 20.2, SD 17.7 vs mean 1.6, SD 14.2; P=.04). Compared with baseline, PHQ-9 scores decreased in both the app arm and the usual care arm at 18 weeks (mean 7.8, SD 7.2 vs mean 7.0, SD 6.5; P=.73) and 52 weeks (mean 9.5, SD 4.0 vs mean 4.7, SD 6.0; P=.07). Improvements in 7-item Patient-Provider Engagement Scale and WHO-5 scores were observed in both arms at 18 weeks and were sustained through 52 weeks in the app arm. Improvements in WHO-5 scores at 52 weeks were significantly greater in the app arm than in the usual care arm (41.5 vs 20.0; P=.02).

CONCLUSIONS

Patients with MDD will engage with a mobile app designed to track treatment and disease progression. PCPs will use the data generated as part of their assessment to inform clinical care. The study results suggest that an app-enabled clinical care pathway may enhance patient activation and benefit MDD management.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03242213; https://clinicaltrials.gov/ct2/show/NCT03242213.

Acceptability of an mHealth App for Youth With Substance Use and Mental Health Needs: Iterative, Mixed Methods Design

BACKGROUND

Treating substance use disorders (SUDs) during adolescence can prevent adult addiction and improve youth outcomes. However, it can be challenging to keep adolescents with SUDs engaged in ongoing services, thus limiting potential benefits. Developmentally appropriate tools are needed to improve treatment engagement during and between sessions for youth with SUDs and mental health disorders. Mobile health apps may augment or replace psychotherapy components; however, few have been developed specifically for youth with SUDs following user-guided design principles, which may limit their appropriateness and utility. Formative research on acceptability to intended end users is needed before the efficacy of such tools can be examined.

OBJECTIVE

This study involves user-centered, iterative development and initial user testing of a web-based app for adolescents with SUDs and mental health concerns.

METHODS

Adolescents aged 14 to 17 years with past-year involvement in outpatient psychotherapy and behavioral health clinicians with adolescent SUD treatment caseloads were recruited. Across 2 assessment phases, 40 participants (alpha: 10 youths and 10 clinicians; beta: 10 youths and 10 clinicians) viewed an app demonstration and completed semistructured interviews and questionnaires about app content and functionality.

RESULTS

Participants expressed positive impressions of the app and its potential utility in augmenting outpatient therapy for youth with SUDs and mental health concerns. Noted strengths included valuable educational content, useful embedded resources, and a variety of activities. Adolescents and clinicians favored the app over conventional (paper-and-pencil) modalities, citing convenience and familiarity. The app was found to be user-friendly and likely to improve treatment engagement. Adolescents suggested the inclusion of privacy settings, and clinicians recommended more detailed instructions and simplified language.

CONCLUSIONS

The novel app developed here appears to be a promising, acceptable, and highly scalable resource to support adolescents with SUDs and mental health concerns. Future studies should test the efficacy of such apps in enhancing adolescent behavioral health treatment engagement and outcomes.

Tracking Lower Urinary Tract Symptoms and Tamsulosin Side Effects Among Older Men Using a Mobile App (PERSONAL): Feasibility and Usability Study

Background

Continuous α1a-blockade is the first-line treatment for lower urinary tract symptoms (LUTS) among older men with suspected benign prostatic hyperplasia. Variable efficacy and safety for individual men necessitate a more personalized, data-driven approach to prescribing and deprescribing tamsulosin for LUTS in older men.

Objective

We aim to evaluate the feasibility and usability of the PERSONAL (Placebo–Controlled, Randomized, Patient-Selected Outcomes, N-of-1 Trials) mobile app for tracking daily LUTS severity and medication side effects among older men receiving chronic tamsulosin therapy.

Methods

We recruited patients from the University of California, San Francisco health care system to participate in a 2-week pilot study. The primary objectives were to assess recruitment feasibility, study completion rates, frequency of symptom tracking, duration of tracking sessions, and app usability rankings measured using a follow-up survey. As secondary outcomes, we evaluated whether daily symptom tracking led to changes in LUTS severity, perceptions of tamsulosin, overall quality of life, medication adherence between baseline and follow-up surveys, and perceived app utility.

Results

We enrolled 19 men within 23 days, and 100% (19/19) of the participants completed the study. Each participant selected a unique combination of symptoms to track and recorded data in the PERSONAL app, with a median daily completion rate of 79% (11/14 days). The median duration of the app session was 44 (IQR 33) seconds. On a scale of 1 (strongly disagree) to 5 (strongly agree), the participants reported that the PERSONAL app was easy to use (mean 4.3, SD 1.0), that others could learn to use it quickly (mean 4.2, SD 0.9), and that they felt confident using the app (mean 4.4, SD 0.8). LUTS severity, quality of life, and medication adherence remained unchanged after the 2-week study period. Fewer men were satisfied with tamsulosin after using the app (14/19, 74% vs 17/19, 89% at baseline), although the perceived benefit from tamsulosin remained unchanged (18/19, 95% at baseline and at follow-up). In total, 58% (11/19) of the participants agreed that the PERSONAL app could help people like them manage their urinary symptoms.

Conclusions

This pilot study demonstrated the high feasibility and usability of the PERSONAL mobile app to track patient-selected urinary symptoms and medication side effects among older men taking tamsulosin to manage LUTS. We observed that daily symptom monitoring had no adverse effects on the secondary outcomes. This proof-of-concept study establishes a framework for future mobile app studies, such as digital n-of-1 trials, to collect comprehensive individual-level data for personalized LUTS management in older men.

Considerations of diversity, equity, and inclusion in mental health apps: A scoping review of evaluation frameworks

Mental health applications or apps are a promising approach to reduce the pervasive unmet mental health need observed among marginalized groups. However, these groups face unique challenges to engage in and benefit from these interventions, and apps that consider issues of diversity, equity, and inclusion (DEI) may be better positioned to reach them. App evaluation frameworks emerge as crucial tools for researchers, clinicians, and users to select evidence-based apps. Nevertheless, it is unclear the extent to which existing assessment tools capture DEI factors. The present scoping review identified 68 studies that employed one or more assessment tools to evaluate a mental health app, leading to 44 unique app evaluation frameworks. Results showed that most frameworks were developed after 2015, and only 58% of them considered at least one DEI criterion. Frameworks that performed exceptionally well were those that combined multiple standardized and validated measures. This lack of consideration for DEI variables may limit the ability of app-based interventions to serve marginalized communities, or even worse, create new disparities. Therefore, we provide recommendations to improve current app evaluation frameworks' cultural robustness and clinical utility, maximizing their effectiveness when working with individuals from marginalized communities.