Alliance With an Unguided Smartphone App: Validation of the Digital Working Alliance Inventory

Evidence for Bidirectional, Cross-Lagged Associations Between Alliance and Psychological Distress in an Unguided Mobile-Health Intervention

Bidirectional associations between changes in symptoms and alliance are established for in-person psychotherapy. Alliance may play an important role in promoting engagement and effectiveness within unguided mobile health (mHealth) interventions. Using models disaggregating alliance and psychological distress into within- and between-person components (random intercept cross-lagged panel model), we report bidirectional associations between alliance and distress over the course of a 4-week smartphone-based meditation intervention (n=302, 80.0% elevated depression/anxiety). Associations were stable across time with effect sizes similar to those observed for psychotherapy (βs=-.13 to -.14 and -.09 to -.10, for distress to alliance and alliance to distress, respectively). Alliance may be worth measuring to improve the acceptability and effectiveness of mHealth tools. Further empirical and theoretical work characterizing the role and meaning of alliance in unguided mHealth is warranted.

Revealing subgroup-specific mechanisms of change via moderated mediation: A meditation intervention example

OBJECTIVE

Effective psychosocial interventions exist for numerous mental health conditions. However, despite decades of research, limited progress has been made in clarifying the mechanisms that account for their beneficial effects. We know that many treatments work, but we know relatively little about why they work. Mechanisms of change may be obscured due to prior research collapsing across heterogeneous subgroups of patients with differing underlying mechanisms of response. Studies identifying baseline individual characteristics that predict differential response (i.e., moderation) may inform research on why (i.e., mediation) a particular subgroup has better outcomes to an intervention via tests of moderated mediation.

METHOD

In a recent randomized controlled trial comparing a 4-week meditation app with a control condition in school system employees (N = 662), we previously developed a "Personalized Advantage Index" (PAI) using baseline characteristics, which identified a subgroup of individuals who derived relatively greater benefit from meditation training. Here, we tested whether the effect of mindfulness acquisition in mediating group differences in outcome was moderated by PAI scores.

RESULTS

A significant index of moderated mediation (IMM = 1.22, 95% CI [0.30, 2.33]) revealed that the effect of mindfulness acquisition in mediating group differences in outcome was only significant among those individuals with PAI scores predicting relatively greater benefit from the meditation app.

CONCLUSIONS

Subgroups of individuals may differ meaningfully in the mechanisms that mediate their response to an intervention. Considering subgroup-specific mediators may accelerate progress on clarifying mechanisms of change underlying psychosocial interventions and may help inform which specific interventions are most beneficial for whom. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A Personalized, Transdiagnostic Smartphone Intervention (Mello) Targeting Repetitive Negative Thinking in Young People With Depression and Anxiety: Pilot Randomized Controlled Trial

BACKGROUND

Repetitive negative thinking (RNT) is a key transdiagnostic mechanism underpinning depression and anxiety. Using "just-in-time adaptive interventions" via smartphones may disrupt RNT in real time, providing targeted and personalized intervention.

OBJECTIVE

This pilot randomized controlled trial evaluates the feasibility, acceptability, and preliminary clinical outcomes and mechanisms of Mello-a fully automated, personalized, transdiagnostic, and mechanistic smartphone intervention targeting RNT in young people with depression and anxiety.

METHODS

Participants with heightened depression, anxiety, and RNT were recruited via social media and randomized to receive Mello or a nonactive control over a 6-week intervention period. Assessments were completed via Zoom sessions at baseline and at 3 and 6 weeks after baseline.

RESULTS

The findings supported feasibility and acceptability, with high rates of recruitment (N=55), uptake (55/64, 86% of eligible participants), and retention (52/55, 95% at 6 weeks). Engagement was high, with 90% (26/29) and 59% (17/29) of the participants in the Mello condition still using the app during the third and sixth weeks, respectively. Greater reductions in depression (Cohen d=0.50), anxiety (Cohen d=0.61), and RNT (Cohen d=0.87) were observed for Mello users versus controls. Mediation analyses suggested that changes in depression and anxiety were accounted for by changes in RNT.

CONCLUSIONS

The results indicate that mechanistic, targeted, and real-time technology-based solutions may provide scalable and effective interventions that advance the treatment of youth mental ill health.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12621001701819; http://tinyurl.com/4d3jfj9f.

German Version of the Mobile Agnew Relationship Measure: Translation and Validation Study

BACKGROUND

The mobile Agnew Relationship Measure (mARM) is a self-report questionnaire for the evaluation of digital mental health interventions and their interactions with users. With the global increase in digital mental health intervention research, translated measures are required to conduct research with local populations.

OBJECTIVE

The aim of this study was to translate and validate the original English version of the mARM into a German version (mARM-G).

METHODS

A total of 2 native German speakers who spoke English as their second language conducted forward translation of the original items. This version was then back translated by 2 native German speakers with a fluent knowledge of English. An independent bilingual reviewer then compared these drafts and created a final German version. The mARM-G was validated by 15 experts in the field of mobile app development and 15 nonexperts for content validity and face validity; 144 participants were recruited to conduct reliability testing as well as confirmatory factor analysis.

RESULTS

The content validity index of the mARM-G was 0.90 (expert ratings) and 0.79 (nonexperts). The face validity index was 0.89 (experts) and 0.86 (nonexperts). Internal consistency for the entire scale was Cronbach α=.91. Confirmatory factor analysis results were as follows: the chi-square statistic to df ratio was 1.66. Comparative Fit Index was 0.87 and the Tucker-Lewis Index was 0.86. The root mean square error of approximation was 0.07.

CONCLUSIONS

The mARM-G is a valid and reliable tool that can be used for future studies in German-speaking countries.

User engagement in a randomised controlled trial for a digital health intervention for early psychosis (Actissist 2.0 trial)

Digital Health Interventions (DHIs) can help support people with mental health problems. Achieving satisfactory levels of patient engagement is a crucial, yet often underexplored, pre-requisite for health improvement. Actissist is a co-produced DHI delivered via a smartphone app for people with early psychosis, based on Cognitive Behaviour Therapy principles. This study describes and compares engagement patterns among participants in the two arms of the Actissist 2.0 randomised controlled trial. Engagement frequency and duration were measured among participants using the Actissist app in the intervention arm (n = 87) and the ClinTouch symptom monitoring only app used as the control condition (n = 81). Overall, 47.1 % of Actissist and 45.7 % of ClinTouch users completed at least a third of scheduled alerts while active in the study. The mean frequency (77.1 versus 60.2 total responses) and the median duration (80 versus 75 days until last response) of engagement were not significantly higher among Actissist users compared to ClinTouch users. Older age, White ethnicity, using their own smartphone device and, among Actissist users, an increased sense of therapeutic alliance were significantly associated with increased engagement. Through exploiting detailed usage data, this study identifies possible participant-level and DHI-level predictors of engagement to inform the practical implementation of future DHIs.

The impact of guided versus supportive coaching on mental health app engagement and clinical outcomes

Although mobile mental health apps have the unique potential to increase access to care, evidence reveals engagement is low unless coupled with coaching. However, most coaching protocols are limited in their scalability. This study assesses how human support and guidance from a Digital Navigator (DN), a scalable coach, can impact mental health app engagement and effectiveness on anxiety and depressive symptoms. This study aims to detach components of coaching, specifically personalized recommendations versus general support, to inform scalability of coaching models for mental health apps. 156 participants were split into the DN Guide versus DN Support groups for the 6-week study. Both groups utilized the mindLAMP app for the duration of the study and had equal time with the DN, but the Guide group received personalized app recommendations. The Guide group completed significantly more activities than the Support group. 34% (49/139) of all participants saw a 25% decrease in PHQ-9 scores and 38% (53/141) saw a 25% decrease in GAD-7 scores. These findings show mental health apps, especially when supported by DNs, can reduce depression and anxiety symptoms when coupled with coaching, suggesting a feasible path for large-scale deployment.

The engagement problem: A review of engagement with digital mental health interventions and recommendations for a path forward

Purpose of the review

Digital mental health interventions (DMHIs) are an effective and accessible means of addressing the unprecedented levels of mental illness worldwide. Currently, however, patient engagement with DMHIs in real-world settings is often insufficient to see clinical benefit. In order to realize the potential of DMHIs, there is a need to better understand what drives patient engagement.

Recent findings

We discuss takeaways from the existing literature related to patient engagement with DMHIs and highlight gaps to be addressed through further research. Findings suggest that engagement is influenced by patient-, intervention- and systems-level factors. At the patient-level, variables such as sex, education, personality traits, race, ethnicity, age and symptom severity appear to be associated with engagement. At the intervention-level, integrating human support, gamification, financial incentives and persuasive technology features may improve engagement. Finally, although systems-level factors have not been widely explored, the existing evidence suggests that achieving engagement will require addressing organizational and social barriers and drawing on the field of implementation science.

Summary

Future research clarifying the patient-, intervention- and systems-level factors that drive engagement will be essential. Additionally, to facilitate improved understanding of DMHI engagement, we propose the following: (a) widespread adoption of a minimum necessary 5-element engagement reporting framework; (b) broader application of alternative clinical trial designs; and (c) directed efforts to build upon an initial parsimonious conceptual model of DMHI engagement.

Digital Phenotyping Models of Symptom Improvement in College Mental Health: Generalizability Across Two Cohorts

Smartphones can be used to gain insight into mental health conditions through the collection of survey and sensor data. However, the external validity of this digital phenotyping data is still being explored, and there is a need to assess if predictive models derived from this data are generalizable. The first dataset (V1) of 632 college students was collected between December 2020 and May 2021. The second dataset (V2) was collected using the same app between November and December 2021 and included 66 students. Students in V1 could enroll in V2. The main difference between the V1 and V2 studies was that we focused on protocol methods in V2 to ensure digital phenotyping data had a lower degree of missing data than in the V1 dataset. We compared survey response counts and sensor data coverage across the two datasets. Additionally, we explored whether models trained to predict symptom survey improvement could generalize across datasets. Design changes in V2, such as a run-in period and data quality checks, resulted in significantly higher engagement and sensor data coverage. The best-performing model was able to predict a 50% change in mood with 28 days of data, and models were able to generalize across datasets. The similarities between the features in V1 and V2 suggest that our features are valid across time. In addition, models must be able to generalize to new populations to be used in practice, so our experiments provide an encouraging result toward the potential of personalized digital mental health care.

Exploring how informed mental health app selection may impact user engagement and satisfaction

The prevalence of mental health app use by people suffering from mental health disorders is rapidly growing. The integration of mental health apps shows promise in increasing the accessibility and quality of treatment. However, a lack of continued engagement is one of the significant challenges of such implementation. In response, the M-health Index and Navigation Database (MIND)- derived from the American Psychiatric Association's app evaluation framework- was created to support patient autonomy and enhance engagement. This study aimed to identify factors influencing engagement with mental health apps and explore how MIND may affect user engagement around selected apps. We conducted a longitudinal online survey over six weeks after participants were instructed to find mental health apps using MIND. The survey included demographic information, technology usage, access to healthcare, app selection information, System Usability Scale, the Digital Working Alliance Inventory, and the General Self-Efficacy Scale questions. Quantitative analysis was performed to analyze the data. A total of 321 surveys were completed (178 at the initial, 90 at the 2-week mark, and 53 at the 6-week mark). The most influential factors when choosing mental health apps included cost (76%), condition supported by the app (59%), and app features offered (51%), while privacy and clinical foundation to support app claims were among the least selected filters. The top ten apps selected by participants were analyzed for engagement. Rates of engagement among the top-ten apps decreased by 43% from the initial to week two and 22% from week two to week six on average. In the context of overall low engagement with mental health apps, implementation of mental health app databases like MIND can play an essential role in maintaining higher engagement and satisfaction. Together, this study offers early data on how educational approaches like MIND may help bolster mental health apps engagement.

Increasing the value of digital phenotyping through reducing missingness: a retrospective review and analysis of prior studies

BACKGROUND

Digital phenotyping methods present a scalable tool to realise the potential of personalised medicine. But underlying this potential is the need for digital phenotyping data to represent accurate and precise health measurements.

OBJECTIVE

To assess the impact of population, clinical, research and technological factors on the digital phenotyping data quality as measured by rates of missing digital phenotyping data.

METHODS

This study analyses retrospective cohorts of mindLAMP smartphone application digital phenotyping studies run at Beth Israel Deaconess Medical Center between May 2019 and March 2022 involving 1178 participants (studies of college students, people with schizophrenia and people with depression/anxiety). With this large combined data set, we report on the impact of sampling frequency, active engagement with the application, phone type (Android vs Apple), gender and study protocol features on missingness/data quality.

FINDINGS

Missingness from sensors in digital phenotyping is related to active user engagement with the application. After 3 days of no engagement, there was a 19% decrease in average data coverage for both Global Positioning System and accelerometer. Data sets with high degrees of missingness can generate incorrect behavioural features that may lead to faulty clinical interpretations.

CONCLUSIONS

Digital phenotyping data quality requires ongoing technical and protocol efforts to minimise missingness. Adding run-in periods, education with hands-on support and tools to easily monitor data coverage are all productive strategies studies can use today.

CLINICAL IMPLICATIONS

While it is feasible to capture digital phenotyping data from diverse populations, clinicians should consider the degree of missingness in the data before using them for clinical decision-making.

Protocol for a randomised controlled trial evaluating the effectiveness of a CBT-based smartphone application for improving mental health outcomes in adolescents: the MobiliseMe study

BACKGROUND

Depression is a leading cause of disability in adolescents, however few receive evidence-based treatment. Despite having the potential to overcome barriers to treatment uptake and adherence, there are very few CBT-based smartphone apps for adolescents. To address this gap, we developed ClearlyMe®, a self-guided CBT smartphone app for adolescent depression and anxiety. ClearlyMe® consists of 37 brief lessons containing core CBT elements, accessed either individually or as part of a 'collection'. Here, we describe the protocol for a randomised controlled trial aiming to evaluate the effect of ClearlyMe® on depressive symptoms and secondary outcomes, including engagement, anxiety and wellbeing, when delivered with and without guided support compared to an attention matched control.

METHODS

We aim to recruit 489 adolescents aged 12-17 years with mild to moderately-severe depressive symptoms. Participants will be screened for inclusion, complete the baseline assessment and are then randomly allocated to receive ClearlyMe® (self-directed use), ClearlyMe® with guided SMS support (guided use) or digital psychoeducation (attention-matched control). Depressive symptoms and secondary outcomes will be assessed at 6-weeks (primary endpoint) and 4-months post-baseline (secondary endpoint). Engagement, conceptualised as uptake, adherence and completion, will also be assessed 6-weeks post-baseline. Mixed-effects linear modelling will be used to conduct intention-to-treat analyses to determine whether reductions in depressive symptoms and secondary outcomes are greater for conditions receiving ClearlyMe® relative to control at 6-weeks and 4-months post-baseline and greater for intervention adherers relative to non-adherers. To minimise risk, participants will be encouraged to use the Get Help section of the app and can also opt to receive a call from the team clinical psychologist at baseline, and at the 6-week and 4-month post-baseline assessments when reporting suicidal ideation.

DISCUSSION

This is the first clinical trial examining a CBT smartphone app specifically designed for adolescent depression. It will provide empirical evidence on the effects of ClearlyMe® on depressive symptoms when used with and without guided support.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry (ACTRN12622000131752).

UNIVERSAL TRIAL NUMBER

U1111-1271-8519.

A common factors perspective on mindfulness-based interventions

Mindfulness-based interventions (MBIs) have entered mainstream Western culture in the past four decades. There are now dozens of MBIs with varying degrees of empirical support and a variety of mindfulness-specific psychological mechanisms have been proposed to account for the beneficial effects of MBIs. Although it has long been acknowledged that non-specific or common factors might contribute to MBI efficacy, relatively little empirical work has directly investigated these aspects. In this Perspective, I suggest that situating MBIs within the broader psychotherapy research literature and emphasizing the commonalities rather than differences between MBIs and other treatments might help guide future MBI research. To that end, I summarize the evidence for MBI efficacy and several MBI-specific psychological mechanisms, contextualize MBI findings within the broader psychotherapy literature from a common factors perspective, and propose suggestions for future research based on innovations and challenges occurring within psychotherapy research.

Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials

Mobile phone-based interventions have been proposed as a means for reducing the burden of disease associated with mental illness. While numerous randomized controlled trials and meta-analyses have investigated this possibility, evidence remains unclear. We conducted a systematic meta-review of meta-analyses examining mobile phone-based interventions tested in randomized controlled trials. We synthesized results from 14 meta-analyses representing 145 randomized controlled trials and 47,940 participants. We identified 34 effect sizes representing unique pairings of participants, intervention, comparisons, and outcome (PICO) and graded the strength of the evidence as using umbrella review methodology. We failed to find convincing evidence of efficacy (i.e., n > 1000, p < 10^-6, I ² < 50%, absence of publication bias); publication bias was rarely assessed for the representative effect sizes. Eight effect sizes provided highly suggestive evidence (i.e., n > 1000, p < 10^-6), including smartphone interventions outperforming inactive controls on measures of psychological symptoms and quality of life (ds = 0.32 to 0.47) and text message-based interventions outperforming non-specific controls and active controls for smoking cessation (ds = 0.31 and 0.19, respectively). The magnitude of effects and strength of evidence tended to diminish as comparison conditions became more rigorous (i.e., inactive to active, non-specific to specific). Four effect sizes provided suggestive evidence, 14 effect sizes provided weak evidence, and eight effect sizes were non-significant. Despite substantial heterogeneity, no moderators were identified. Adverse effects were not reported. Taken together, results support the potential of mobile phone-based interventions and highlight key directions to guide providers, policy makers, clinical trialists, and meta-analysts working in this area.

Digital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review

Fully automated mental health smartphone apps show strong promise in increasing access to psychological support. Therefore, it is crucial to understand how to make these apps effective. The therapeutic alliance (TA), or the relationship between healthcare professionals and clients, is considered fundamental to successful treatment outcomes in face-to-face therapy. Thus, understanding the TA in the context of fully automated apps would bring us insights into building effective smartphone apps which engage users. However, the concept of a digital therapeutic alliance (DTA) in the context of fully automated mental health smartphone apps is nascent and under-researched, and only a handful of studies have been published in this area. In particular, no published review paper examined the DTA in the context of fully automated apps. The objective of this review was to integrate the extant literature to identify research gaps and future directions in the investigation of DTA in relation to fully automated mental health smartphone apps. Our findings suggest that the DTA in relation to fully automated smartphone apps needs to be conceptualized differently to traditional face-to-face TA. First, the role of bond in the context of fully automated apps is unclear. Second, human components of face-to-face TA, such as empathy, are hard to achieve in the digital context. Third, some users may perceive apps as more non-judgmental and flexible, which may further influence DTA formation. Subdisciplines of computer science, such as affective computing and positive computing, and some human-computer interaction (HCI) theories, such as those of persuasive technology and human-app attachment, can potentially help to foster a sense of empathy, build tasks and goals and develop bond or an attachment between users and apps, which may further contribute to DTA formation in fully automated smartphone apps. Whilst the review produced a relatively limited quantity of literature, this reflects the novelty of the topic and the need for further research.

Exploring the Neuropsychiatric Sequalae of Perceived COVID-19 Exposure in College Students: A Pilot Digital Phenotyping Study

The urgency to understand the long-term neuropsychiatric sequala of COVID-19, a part of the Post-Acute COVID-19 Syndrome (PACS), is expanding as millions of infected individuals experience new unexplained symptoms related to mood, anxiety, insomnia, headache, pain, and more. Much research on PACS involves cross sectional surveys which limits ability to understand the dynamic trajectory of this emerging phenomenon. In this secondary analysis, we analyzed data from a 4-week observational digital phenotyping study using the mindLAMP app for 695 college students with elevated stress who specified if they were exposed to COVID-19. Students also completed a biweekly survey of clinical assessments to obtain active data. Additionally, passive data streams like GPS, accelerometer, and screen state were extracted from phone sensors and through features the group built. Three hundred and eighty-second number participants successfully specified their COVID-19 exposure and completed the biweekly survey. From active smartphone data, we found significantly higher scores for the Prodromal Questionnaire (PQ) and the Pittsburgh Sleep Quality Index (PSQI) for students reporting exposure to COVID-19 compared to those who were not (ps < 0.05). Additionally, we found significantly decreased sleep duration as captured from the smartphone via passive data for the COVID-19 exposed group (p < 0.05). No significant differences were detected for other surveys or passive sensors. Smartphones can capture both self-reported symptoms and behavioral changes related to PACS. Our results around changes in sleep highlight how digital phenotyping methods can be used in a scalable and accessible manner toward better capturing the evolving phenomena of PACS. The present study further provides a foundation for future research to implement improving digital phenotyping methods.