User Engagement in Mental Health Apps: A Review of Measurement, Reporting, and Validity

Smartband-based smoking detection and real-time brief mindfulness intervention: findings from a feasibility clinical trial

BACKGROUND

Smartbands can be used to detect cigarette smoking and deliver real time smoking interventions. Brief mindfulness interventions have been found to reduce smoking.

OBJECTIVE

This single arm feasibility trial used a smartband to detect smoking and deliver brief mindfulness exercises.

METHODS

Daily smokers who were motivated to reduce their smoking wore a smartband for 60 days. For 21 days, the smartband monitored, detected and notified the user of smoking in real time. After 21 days, a 'mindful smoking' exercise was triggered by detected smoking. After 28 days, a 'RAIN' (recognize, allow, investigate, nonidentify) exercise was delivered to predicted smoking. Participants received mindfulness exercises by text message and online mindfulness training. Feasibility measures included treatment fidelity, adherence and acceptability.

RESULTS

Participants (N=155) were 54% female, 76% white non-Hispanic, and treatment starters (n=115) were analyzed. Treatment fidelity cutoffs were met, including for detecting smoking and delivering mindfulness exercises. Adherence was mixed, including moderate smartband use and low completion of mindfulness exercises. Acceptability was mixed, including high helpfulness ratings and mixed user experiences data. Retention of treatment starters was high (81.9%).

CONCLUSIONS

Findings demonstrate the feasibility of using a smartband to track smoking and deliver quit smoking interventions contingent on smoking.

Design of Digital Mental Health Platforms for Family Member Cocompletion: Scoping Review

BACKGROUND

The COVID-19 pandemic placed an additional mental health burden on individuals and families, resulting in widespread service access problems. Digital mental health interventions suggest promise for improved accessibility. Recent reviews have shown emerging evidence for individual use and early evidence for multiusers. However, attrition rates remain high for digital mental health interventions, and additional complexities exist when engaging multiple family members together.

OBJECTIVE

As such, this scoping review aims to detail the reported evidence for digital mental health interventions designed for family use with a focus on the build and design characteristics that promote accessibility and engagement and enable cocompletion by families.

METHODS

A systematic literature search of MEDLINE, Embase, PsycINFO, Web of Science, and CINAHL databases was conducted for articles published in the English language from January 2002 to March 2024. Eligible records included empirical studies of digital platforms containing some elements designed for cocompletion by related people as well as some components intended to be completed without therapist engagement. Platforms were included in cases in which clinical evidence had been documented.

RESULTS

Of the 9527 papers reviewed, 85 (0.89%) met the eligibility criteria. A total of 24 unique platforms designed for co-use by related parties were identified. Relationships between participants included couples, parent-child dyads, family caregiver-care recipient dyads, and families. Common platform features included the delivery of content via structured interventions with no to minimal tailoring or personalization offered. Some interventions provided live contact with therapists. User engagement indicators and findings varied and included user experience, satisfaction, completion rates, and feasibility. Our findings are more remarkable for what was absent in the literature than what was present. Contrary to expectations, few studies reported any design and build characteristics that enabled coparticipation. No studies reported on platform features for enabling cocompletion or considerations for ensuring individual privacy and safety. None examined platform build or design characteristics as moderators of intervention effect, and none offered a formative evaluation of the platform itself.

CONCLUSIONS

In this early era of digital mental health platform design, this novel review demonstrates a striking absence of information about design elements associated with the successful engagement of multiple related users in any aspect of a therapeutic process. There remains a large gap in the literature detailing and evaluating platform design, highlighting a significant opportunity for future cross-disciplinary research. This review details the incentive for undertaking such research; suggests design considerations when building digital mental health platforms for use by families; and offers recommendations for future development, including platform co-design and formative evaluation.

Study of a PST-trained voice-enabled artificial intelligence counselor for adults with emotional distress (SPEAC-2): Design and methods

BACKGROUND

Novel and scalable psychotherapies are urgently needed to address the depression and anxiety epidemic. Leveraging artificial intelligence (AI), a voice-based virtual coach named Lumen was developed to deliver problem solving treatment (PST). The first pilot trial showed promising changes in cognitive control measured by functional neuroimaging and improvements in depression and anxiety symptoms.

METHODS

To further validate Lumen in a 3-arm randomized clinical trial, 200 participants with mild-to-moderate depression and/or anxiety will be randomly assigned in a 2:1:1 ratio to receive Lumen-coached PST, human-coached PST as active treatment comparison, or a waitlist control condition where participants can receive Lumen after the trial period. Participants will be assessed at baseline and 18 weeks. The primary aim is to confirm neural target engagement by testing whether compared with waitlist controls, Lumen participants will show significantly greater improvements from baseline to 18 weeks in the a priori neural target for cognitive control, right dorsal lateral prefrontal cortex engaged by the go/nogo task (primary superiority hypothesis). A secondary hypothesis will test whether compared with human-coached PST participants, Lumen participants will show equivalent improvements (i.e., noninferiority) in the same neural target from baseline to 18 weeks. The second aim is to examine (1) treatment effects on depression and anxiety symptoms, psychosocial functioning, and quality of life outcomes, and (2) relationships of neural target engagement to these patient-reported outcomes.

CONCLUSIONS

This study offers potential to improve the reach and impact of psychotherapy, mitigating access, cost, and stigma barriers for people with depression and/or anxiety.

CLINICALTRIALS

gov #: NCT05603923.

The conceptualisation and measurement of engagement in digital health

Digital tools are an increasingly important component of healthcare, but their potential impact is commonly limited by a lack of user engagement. Digital health evaluations of engagement are often restricted to system usage metrics, which cannot capture a full understanding of how and why users engage with an intervention. This study aimed to examine how theory-based, multifaceted measures of engagement with digital health interventions capture different components of engagement (affective, cognitive, behavioural, micro, and macro) and to consider areas that are unclear or missing in their measurement. We identified and compared two recently developed measures that met these criteria (the Digital Behaviour Change Intervention Engagement Scale and the TWente Engagement with Ehealth Technologies Scale). Despite having similar theoretical bases and being relatively strongly correlated, there are key differences in how these scales aim to capture engagement. We discuss the implications of our analysis for how affective, cognitive, and behavioural components of engagement can be conceptualised and whether there is value in distinguishing between them. We conclude with recommendations for the circumstances in which each scale may be most useful and for how future measure development could supplement existing scales.

Outcomes of Best-Practice Guided Digital Mental Health Interventions for Youth and Young Adults with Emerging Symptoms: Part II. A Systematic Review of User Experience Outcomes

Although many young people demonstrate resilience and strength, research and clinical evidence highlight an upward trend in mental health concerns among those aged 12 to 25 years. Youth-specific digital mental health interventions (DMHIs) aim to address this trend by providing timely access to mental health support for young people (12-25 years). However, there is a considerable gap in understanding young people user experiences with digital interventions. This review, co-designed with Australia's leading mental health organization Beyond Blue, utilizes a systematic methodology to synthesize evidence on user experience in youth-oriented digital mental health interventions that are fully or partially guided. Five relevant online databases were searched for articles published from 2018 to 2023, yielding 22,482 articles for screening and 22 studies were included in the present analysis. User experience outcomes relating to satisfaction and engagement were assessed for each included intervention, with experience indicators relating to usefulness, usability, value, credibility, and desirability being examined. Elements associated with positive/negative outcomes were extracted. Elements shown to positively influence user experience included peer engagement, modern app-based delivery, asynchronous support, and personalized content. In contrast, users disliked static content, homework/log-keeping, the requirement for multiple devices, and social media integration. Asynchronous interventions showed high satisfaction but faced engagement issues, with combined asynchronous/synchronous interventions reporting better completion rates. DMHIs offer a promising platform for youth mental health support and has the potential to dramatically increase the reach of interventions through the adoption of technological and user experience best practices. While young people respond positively to many aspects of intervention modernization, such as interactive, app-based design, other concepts, such as social media integration, they need to be adopted by the field more cautiously to ensure trust and engagement.Trial Registration CRD42023405812.

Engagement With a Relaxation and Mindfulness Mobile App Among People With Cancer: Exploratory Analysis of Use Data and Self-Reports From a Randomized Controlled Trial

BACKGROUND

Mobile health (mHealth) apps offer unique opportunities to support self-care and behavior change, but poor user engagement limits their effectiveness. This is particularly true for fully automated mHealth apps without any human support. Human support in mHealth apps is associated with better engagement but at the cost of reduced scalability.

OBJECTIVE

This work aimed to (1) describe the theory-informed development of a fully automated relaxation and mindfulness app to reduce distress in people with cancer (CanRelax app 2.0), (2) describe engagement with the app on multiple levels within a fully automated randomized controlled trial over 10 weeks, and (3) examine whether engagement was related to user characteristics.

METHODS

The CanRelax app 2.0 was developed in iterative processes involving input from people with cancer and relevant experts. The app includes evidence-based relaxation exercises, personalized weekly coaching sessions with a rule-based conversational agent, 39 self-enactable behavior change techniques, a self-monitoring dashboard with gamification elements, highly tailored reminder notifications, an educational video clip, and personalized in-app letters. For the larger study, German-speaking adults diagnosed with cancer within the last 5 years were recruited via the web in Switzerland, Austria, and Germany. Engagement was analyzed in a sample of 100 study participants with multiple measures on a micro level (completed coaching sessions, relaxation exercises practiced with the app, and feedback on the app) and a macro level (relaxation exercises practiced without the app and self-efficacy toward self-set weekly relaxation goals).

RESULTS

In week 10, a total of 62% (62/100) of the participants were actively using the CanRelax app 2.0. No associations were identified between engagement and level of distress at baseline, sex assigned at birth, educational attainment, or age. At the micro level, 71.88% (3520/4897) of all relaxation exercises and 714 coaching sessions were completed in the app, and all participants who provided feedback (52/100, 52%) expressed positive app experiences. At the macro level, 28.12% (1377/4897) of relaxation exercises were completed without the app, and participants' self-efficacy remained stable at a high level. At the same time, participants raised their weekly relaxation goals, which indicates a potential relative increase in self-efficacy.

CONCLUSIONS

The CanRelax app 2.0 achieved promising engagement even though it provided no human support. Fully automated social components might have compensated for the lack of human involvement and should be investigated further. More than one-quarter (1377/4897, 28.12%) of all relaxation exercises were practiced without the app, highlighting the importance of assessing engagement on multiple levels.

Online Screening and Personalized Education to Identify Post-Deployment Mental Health Need and Facilitate Access to Care

INTRODUCTION

Reserve and National Guard (RNG) service members face increased risks for psychological and behavioral problems and are unlikely to seek mental health (MH) care after returning from military deployments. This article examines an online intervention (Web-Ed) with regard to participation, screening results, satisfaction, and intent to seek follow-up MH care, with comparisons by gender and post-deployment MH care receipt.

MATERIALS AND METHODS

This was a cross-sectional study of 414 RNG service members (214 women and 200 men), who returned from deployments to or in support of the Iraq or Afghanistan wars within the prior 36 months. Participants completed Web-Ed, which includes screening, personalized education, and links to Veterans Health Administration health care.

RESULTS

Positive post-deployment screening proportions, Web-Ed satisfaction, and intent to seek follow-up care were similar for men and women. Few had received MH care (33% women; 24% men), yet most screened positive on at least one screen (69% women; 72% men). Most indicated that they would recommend Web-Ed to other veterans (71% women; 67% men) and that they received useful information they may not have received otherwise (52% women; 53% men) regardless of gender or prior MH care. Almost half (40% women; 48% men) planned to seek further assessment from a health care provider as a direct result of Web-Ed.

CONCLUSIONS

Efforts to facilitate access to MH care among post-deployed RNG veterans should be ongoing, regardless of the length of time since deployment, Veterans Affairs enrollment status, prior MH care receipt, or gender. Online screening and personalized education engage veterans who have not sought MH care and provide new information to those who have.

Digital Mental Health for Schizophrenia and Other Severe Mental Illnesses: An International Consensus on Current Challenges and Potential Solutions

BACKGROUND

Digital approaches may be helpful in augmenting care to address unmet mental health needs, particularly for schizophrenia and severe mental illness (SMI).

OBJECTIVE

An international multidisciplinary group was convened to reach a consensus on the challenges and potential solutions regarding collecting data, delivering treatment, and the ethical challenges in digital mental health approaches for schizophrenia and SMI.

METHODS

The consensus development panel method was used, with an in-person meeting of 2 groups: the expert group and the panel. Membership was multidisciplinary including those with lived experience, with equal participation at all stages and coproduction of the consensus outputs and summary. Relevant literature was shared in advance of the meeting, and a systematic search of the recent literature on digital mental health interventions for schizophrenia and psychosis was completed to ensure that the panel was informed before the meeting with the expert group.

RESULTS

Four broad areas of challenge and proposed solutions were identified: (1) user involvement for real coproduction; (2) new approaches to methodology in digital mental health, including agreed standards, data sharing, measuring harms, prevention strategies, and mechanistic research; (3) regulation and funding issues; and (4) implementation in real-world settings (including multidisciplinary collaboration, training, augmenting existing service provision, and social and population-focused approaches). Examples are provided with more detail on human-centered research design, lived experience perspectives, and biomedical ethics in digital mental health approaches for SMI.

CONCLUSIONS

The group agreed by consensus on a number of recommendations: (1) a new and improved approach to digital mental health research (with agreed reporting standards, data sharing, and shared protocols), (2) equal emphasis on social and population research as well as biological and psychological approaches, (3) meaningful collaborations across varied disciplines that have previously not worked closely together, (4) increased focus on the business model and product with planning and new funding structures across the whole development pathway, (5) increased focus and reporting on ethical issues and potential harms, and (6) organizational changes to allow for true communication and coproduction with those with lived experience of SMI. This study approach, combining an international expert meeting with patient and public involvement and engagement throughout the process, consensus methodology, discussion, and publication, is a helpful way to identify directions for future research and clinical implementation in rapidly evolving areas and can be combined with measurements of real-world clinical impact over time. Similar initiatives will be helpful in other areas of digital mental health and similarly fast-evolving fields to focus research and organizational change and effect improved real-world clinical implementation.

Individuals' attitudes toward digital mental health apps and implications for adoption in Portugal: web-based survey

BACKGROUND

The literature is consensual regarding the academic community exhibiting higher levels of mental disorder prevalence than the general population. The potential of digital mental health apps for improving access to resources to cope with these issues is ample. However, studies have yet to be performed in Portugal on individuals' attitudes and perceptions toward digital mental health applications or their preferences and decision drivers on obtaining mental health care, self-assessment, or treatment.

OBJECTIVE

This study aims to understand the determinants of digital mental health applications use in the Portuguese academic community of Porto, along with potential adoption barriers and enablers.

METHODS

A cross-sectional, web-based survey was delivered via dynamic email to the University of Porto's academic community. Data collection occurred between September 20 and October 20, 2022. We used structural equation modeling to build three models, replicating a peer-reviewed and published study and producing a newly full mediation model shaped by the collected data. We tested the relationships between use of digital mental health apps and perceived stress, perceived need to seek help for mental health, perceived stigma, past use of mental health services, privacy concerns, and social influence.

RESULTS

Of the 539 participants, 169 (31.4%) reported having used digital mental health apps. Perceived stress and a latent variable, comprising perceptions of mental health problems and coping strategies, were positively associated with mental health app use, while privacy concerns regarding one's information being accessible to others were negatively associated. Perceived stigma, need to seek help, and close relationships did not have a statistically significant direct effect.

CONCLUSIONS

These findings can inform product and policy development of new, better-targeted digital mental health app interventions, with implications for researchers and academia, industry, and policymakers. Our study concludes that, to maximize adherence to these apps, they should have low to no financial charges, demonstrate evidence of their helpfulness and focus on the timely delivery of care. We also conclude that to foster digital mental health app use, there is a need to improve mental health literacy, namely regarding self-awareness of one's conditions, acceptable stress levels, and overall behavior towards mental health.

TRIAL REGISTRATION

RR2-10.2196/41040.

Patient Engagement with Conversational Agents in Health Applications 2016-2022: A Systematic Review and Meta-Analysis

Clinicians and patients seeking electronic health applications face challenges in selecting effective solutions due to a high market failure rate. Conversational agent applications ("chatbots") show promise in increasing healthcare user engagement by creating bonds between the applications and users. It is unclear if chatbots improve patient adherence or if past trends to include chatbots in electronic health applications were due to technology hype dynamics and competitive pressure to innovate. We conducted a systematic literature review using Preferred Reporting Items for Systematic reviews and Meta-Analyses methodology on health chatbot randomized control trials. The goal of this review was to identify if user engagement indicators are published in eHealth chatbot studies. A meta-analysis examined patient clinical trial retention of chatbot apps. The results showed no chatbot arm patient retention effect. The small number of studies suggests a need for ongoing eHealth chatbot research, especially given the claims regarding their effectiveness made outside the scientific literatures.

Content analysis of qualitative interviews for user-centered design of a prototype mobile health app for direct support professionals' resilience

BACKGROUND

Direct support professionals support people with intellectual and developmental disabilities with essential health and community living needs. Despite an unrelenting workforce shortage crisis, limited evidence-based resources exist for their support. Resilience, or the ability to thrive in the face of challenges, is a target for health care professional wellbeing initiatives. As an "invisible" workforce interspersed throughout various community-based settings, direct support professionals may benefit from resources provided via mobile health apps.

OBJECTIVES

To assess direct support professionals' acceptability of and preferences for the design of a mobile health app prototype to support their resilience, DSP Thrive.

METHOD

This concept generation and prototype design study employed a two-phase qualitative-descriptive design. Phase 1 assessed direct support professionals' acceptability of and preferences for a mobile health app for direct support professional resilience. Phase 2 elicited feedback on mock-up images of an initial prototype. Data were collected via qualitative interviews (n = 13), and transcripts were analyzed using content analysis.

RESULTS

Phase 1: Direct support professionals identified three primary opportunities for an app to support their resilience: learning via educational resources, practicing resilience strategies, and cultivating a peer support network. Based on these findings, the DSP Thrive app prototype was designed with Learn, Practice, and Connect functions. Phase 2: User feedback on walk-throughs of mock-up images indicated initial acceptability of the prototype design with considerations for further development identified.

CONCLUSIONS

Engaging direct support professionals in user-centered design of mobile health apps may help to bridge the gap to resilience resources for this essential workforce population.

The impact of mindfulness apps on psychological processes of change: a systematic review

Mindfulness-based interventions (MBIs) have demonstrated therapeutic efficacy for various psychological conditions, and smartphone apps that facilitate mindfulness practice can enhance the reach and impact of MBIs. The goal of this review was to summarize the published evidence on the impact of mindfulness apps on the psychological processes known to mediate transdiagnostic symptom reduction after mindfulness practice. A literature search from January 1, 1993, to August 7, 2023 was conducted on three databases, and 28 randomized controlled trials involving 5963 adults were included. Across these 28 studies, 67 outcome comparisons were made between a mindfulness app group and a control group. Between-group effects tended to favor the mindfulness app group over the control group in three psychological process domains: repetitive negative thinking, attention regulation, and decentering/defusion. Findings were mixed in other domains (i.e., awareness, nonreactivity, non-judgment, positive affect, and acceptance). The range of populations examined, methodological concerns across studies, and problems with sustained app engagement likely contributed to mixed findings. However, effect sizes tended to be moderate to large when effects were found, and gains tended to persist at follow-up assessments two to six months later. More research is needed to better understand the impact of these apps on psychological processes of change. Clinicians interested in integrating apps into care should consider app-related factors beyond evidence of a clinical foundation and use app databases to identify suitable apps for their patients, as highlighted at the end of this review.

Measuring digital intervention user experience with a novel ecological momentary assessment (EMA) method, CORTO

Digital interventions often suffer from low usage, which may reflect insufficient attention to user experience. Moreover, the existing evaluation methods have limited applicability in the remote study of user experience of complex interventions that have expansive content and that are used over an extensive period of time. To alleviate these challenges, we describe here a novel qualitative Ecological Momentary Assessment (EMA) method: the CORTO method (Contextual, One-item, Repeated, Timely, Open-ended). We used it to gather digital intervention user experience data from Finnish adults (n = 184) who lived with interview-confirmed major depressive disorder (MDD) and took part in a randomized controlled trial (RCT) that studied the efficacy of a novel 12-week game-based digital intervention for depression. A second dataset on user experience was gathered with retrospective interviews (n = 22). We inductively coded the CORTO method and retrospective interview data, which led to four user experience categories: (1) contextual use, (2) interaction-elicited emotional experience, (3) usability, and (4) technical issues. Then, we used the created user experience categories and Template Analysis to analyze both datasets together, and reported the results qualitatively. Finally, we compared the two datasets with each other. We found that the data generated with the CORTO method offered more insights into usability and technical categories than the interview data that particularly illustrated the contextual use. The emotional valence of the interview data was more positive compared with the CORTO data. Both the CORTO and interview data detected 55 % of the micro-level categories; 20 % of micro-level categories were only detected by the CORTO data and 25 % only by the interview data. We found that the during-intervention user experience measurement with the CORTO method can provide intervention-specific insights, and thereby further the iterative user-centered intervention development. Overall, these findings highlight the impact of evaluation methods on the categories and qualities of insights acquired in intervention research.

Symptom Reduction and Engagement in a Cognitive-Behavioral Mobile Phone App: A Study of User Profiling to Determine Prognostic Indicators

OBJECTIVE

We investigated the presence of latent transition profiles in a sample of users of a cognitive-behavioral mental health app for the general population. Users' baseline characteristics were used as predictors of the profiles. The role of engagement with the app in the transition profiles was examined.

METHOD

A total of 541 users completed the Patient Health Questionnaire-9 and the General Anxiety Disorder-7 when started using the app and 30 days after. Random-Intercept Latent Transition Analysis was implemented to identify users' profiles and transition patterns as classes. The age of the users and the Emotional Self-Awareness Scale-Revised (ESAS-R) were used as predictors of class membership at baseline. The Homework Rating Scale-Mobile Application (HRS-MA; as a measure of engagement) was used as a predictor of class membership at 30 days of app use.

RESULTS

A 3-class solution was obtained according to the severity of symptoms (from mild to moderately severe). Age and ESAS-R predicted class membership initially; the higher the age and ESAS-R, the higher the probability of starting using the app with lower distress levels. The HRS-MA predicted class membership at 30 days of app use; the higher the engagement for more symptomatic and younger users, the higher the probability of improvement. However, older users tended to engage less.

CONCLUSION

Our findings underpin the relevance of easily accessible digital interventions for young adults with mild to moderate mental health problems. Further studies and developments are required to enhance these apps for older cohorts.

Determinants and outcome correlates of engagement with a mobile mental health intervention for depression and anxiety in middle-aged and older adults

OBJECTIVES

To examine baseline factors (i.e., age, gender, mobile device proficiency, sensory impairment) associated with app engagement in a 12-week mental health app intervention and to explore whether app engagement predicts changes in depression and anxiety symptoms among middle-aged and older adults.

METHOD

Mobile device proficiency, sensory impairment, depression, and anxiety symptoms were measured using questionnaires. App engagement was defined by metrics characterizing the core intervention features (i.e., messages sent to therapist, mindfulness meditation minutes, action tasks completed). Multiple regressions and multilevel models were conducted.

RESULTS

Forty-nine participants (M age = 57.40, SD = 11.09 years) enrolled. Women (β = .35, p < .05) and participants with less sensory impairment completed more action tasks (β = -.40, p < .05). Depressive and anxiety symptoms measured within the app declined significantly across treatment. Clinical significant improvements were observed for depression in 48.9% and for anxiety in 40% of participants. App engagement metrics were not predictive of depression or anxiety symptoms, either incrementally in time-lagged models or cumulatively in hierarchical linear regression analyses.

CONCLUSION

App engagement is multifaceted; participants engaged differently by gender and ability. Participation in this digital mental health intervention reduced depression and anxiety symptoms, but these findings should be interpreted with caution as the study did not include a control condition. Our findings underscore the importance of considering individual factors that may influence use of a digital mental health intervention.

Methods for Assessing Health Outcomes Associated with Food Insecurity in the United States College Student Population: A Narrative Review

In the United States, college students experience disproportionate food insecurity (FI) rates compared to the national prevalence. The experience of acute and chronic FI has been associated with negative physical and mental health outcomes in this population. This narrative review aims to summarize the current methodologies for assessing health outcomes associated with the experience of FI in college students in the United States. To date, assessing the health outcomes of FI has predominately consisted of subjective assessments, such as self-reported measures of dietary intake, perceived health status, stress, depression, anxiety, and sleep behaviors. This review, along with the emergence of FI as an international public health concern, establishes the need for novel, innovative, and objective biomarkers to evaluate the short- and long-term impacts of FI on physical and mental health outcomes in college students. The inclusion of objective biomarkers will further elucidate the relationship between FI and a multitude of health outcomes to better inform strategies for reducing the pervasiveness of FI in the United States college student population.

Feasibility, Perceived Impact, and Acceptability of a Socially Assistive Robot to Support Emotion Regulation With Highly Anxious University Students: Mixed Methods Open Trial

BACKGROUND

Mental health difficulties among university students have been rising rapidly over the last decade, and the demand for university mental health services commonly far exceeds available resources. Digital interventions are seen as one potential solution to these challenges. However, as in other mental health contexts, digital programs often face low engagement and uptake, and the field lacks usable, engaging, evidence-supported mental health interventions that may be used flexibly when students need them most.

OBJECTIVE

The aim of this study is to investigate the feasibility and acceptability of a new, in situ intervention tool (Purrble) among university students experiencing anxiety. As an intervention, Purrble was designed to provide in situ support for emotion regulation (ER)-a well-known transdiagnostic construct-directly in the moments when individuals are facing emotionally challenging situations. A secondary aim is to consider the perceived impact of Purrble on youth mental health, as reported by students over a 7-week deployment.

METHODS

A mixed methods open trial was conducted with 78 under- and postgraduate students at Oxford University. Participants were recruited based on moderate to high levels of anxiety measured by Generalized Anxiety Disorder-7 at baseline (mean 16.09, SD 3.03). All participants had access to Purrble for 7 weeks during the spring term with data on their perceived anxiety, emotion dysregulation, ER self-efficacy, and engagement with the intervention collected at baseline (pre), week 4 (mid), and week 8 (postintervention). Qualitative responses were also collected at the mid- and postintervention points.

RESULTS

The findings demonstrated a sustained engagement with Purrble over the 7-week period, with the acceptability further supported by the qualitative data indicating that students accepted Purrble and that Purrble was well-integrated into their daily routines. Exploratory quantitative data analysis indicated that Purrble was associated with reductions in student anxiety (dz=0.96, 95% CI 0.62-1.29) and emotion dysregulation (dz=0.69, 95% CI 0.38-0.99), and with an increase in ER self-efficacy (dz=-0.56, 95% CI -0.86 to -0.26).

CONCLUSIONS

This is the first trial of a simple physical intervention that aims to provide ongoing ER support to university students. Both quantitative and qualitative data suggest that Purrble is an acceptable and feasible intervention among students, the engagement with which can be sustained at a stable level across a 7-week period while retaining a perceived benefit for those who use it (n=32, 61% of our sample). The consistency of use is particularly promising given that there was no clinician engagement or further support provided beyond Purrble being delivered to the students. These results show promise for an innovative intervention model, which could be complementary to the existing interventions.

Selecting and describing control conditions in mobile health randomized controlled trials: a proposed typology

Hundreds of randomized controlled trials (RCTs) have tested the efficacy of mobile health (mHealth) tools for a wide range of mental and behavioral health outcomes. These RCTs have used a variety of control condition types which dramatically influence the scientific inferences that can be drawn from a given study. Unfortunately, nomenclature across mHealth RCTs is inconsistent and meta-analyses commonly combine control conditions that differ in potentially important ways. We propose a typology of control condition types in mHealth RCTs. We define 11 control condition types, discuss key dimensions on which they differ, provide a decision tree for selecting and identifying types, and describe the scientific inferences each comparison allows. We propose a five-tier comparison strength gradation along with four simplified categorization schemes. Lastly, we discuss unresolved definitional, ethical, and meta-analytic issues related to the categorization of control conditions in mHealth RCTs.

Specifying the Efficacy of Digital Therapeutic Tools for Depression and Anxiety: Retrospective, 2-Cohort, Real-World Analysis

BACKGROUND

Depression and anxiety are the main sources of work and social disabilities as well as health-related problems around the world. Digital therapeutic solutions using cognitive behavioral therapy have demonstrated efficacy in depression and anxiety. A common goal of digital health apps is to increase user digital engagement to improve outcomes. However, there is a limited understanding of the association between digital platform components and clinical outcomes.

OBJECTIVE

The aim of the study is to investigate the contribution of specific digital engagement tools to mental health conditions. We hypothesized that participation in coaching sessions and breathing exercises would be associated with a reduction in depression and anxiety.

METHODS

Depression and general anxiety symptoms were evaluated in real-world data cohorts using the digital health platform for digital intervention and monitoring change. This retrospective real-world analysis of users on a mobile platform-based treatment followed two cohorts of people: (1) users who started with moderate levels of depression and completed at least 2 depression assessments (n=519) and (2) users who started with moderate levels of anxiety and completed at least 2 anxiety assessments (n=474). Levels of depression (Patient Health Questionnaire-9) and anxiety (Generalized Anxiety Disorder-7) were tracked throughout the first 16 weeks. A piecewise mixed-effects model was applied to model the trajectories of the Patient Health Questionnaire-9 and the Generalized Anxiety Disorder-7 mean scores in 2 segments (1-6 weeks and 7-16 weeks). Finally, simple slope analysis was used for the interpretation of the interactions probing the moderators: coaching sessions and breathing exercises in both depression and anxiety cohorts.

RESULTS

Analysis revealed a significant decrease in depression symptoms (β=-.37, 95% CI -0.46 to 0.28; P≤.001) during the period of weeks 1-6 of app use, which was maintained during the period of 7-16 weeks. Coach interaction significantly moderated the reduction in depression symptoms during the period of weeks 1-6 (β=-.03, 95% CI -0.05 to -0.001; P=.02). A significant decrease in anxiety symptoms (β=-.41, 95% CI -0.50 to -0.33; P≤.001) was revealed during the period of 1-6 weeks, which was maintained during the period of 7-16 weeks. Breathing exercises significantly moderated the reduction in anxiety symptoms during the period of 1-6 weeks (β=-.07, 95% CI -0.14 to -0.01; P=.04).

CONCLUSIONS

This study demonstrated general improvement followed by a period of stability of depression and anxiety symptoms associated with cognitive behavioral therapy-based digital intervention. Interestingly, engagement with a coaching session but not a breathing exercise was associated with a reduction in depression symptoms. Moreover, breathing exercise but not engagement with a coaching session was associated with a reduction of anxiety symptoms. These findings emphasize the importance of using a personalized approach to behavioral health during digital health interventions.

Potential associations between behavior change techniques and engagement with mobile health apps: a systematic review

Introduction

Lack of engagement is a common challenge for digital health interventions. To achieve their potential, it is necessary to understand how best to support users' engagement with interventions and target health behaviors. The aim of this systematic review was to identify the behavioral theories and behavior change techniques being incorporated into mobile health apps and how they are associated with the different components of engagement.

Methods

The review was structured using the PRISMA and PICOS frameworks and searched six databases in July 2022: PubMed, Embase, CINAHL, APA PsycArticles, ScienceDirect, and Web of Science. Risk of bias was evaluated using the Cochrane Collaboration Risk of Bias 2 and the Mixed Methods Appraisal Tools.

Analysis

A descriptive analysis provided an overview of study and app characteristics and evidence for potential associations between Behavior Change Techniques (BCTs) and engagement was examined.

Results

The final analysis included 28 studies. Six BCTs were repeatedly associated with user engagement: goal setting, self-monitoring of behavior, feedback on behavior, prompts/cues, rewards, and social support. There was insufficient data reported to examine associations with specific components of engagement, but the analysis indicated that the different components were being captured by various measures.

Conclusion

This review provides further evidence supporting the use of common BCTs in mobile health apps. To enable developers to leverage BCTs and other app features to optimize engagement in specific contexts and individual characteristics, we need a better understanding of how BCTs are associated with different components of engagement.

Systematic review registration

https://www.crd.york.ac.uk/prospero/, identifier CRD42022312596.

A systematic review of smartphone applications for cancer survivors

PURPOSE

Mobile phone applications are positioned to support, educate, and empower cancer survivors during post-treatment care. We undertook a review to assess the utility of such smartphone applications; determine whether their use correlates with improved quality of life and other self-reported outcomes; and understand the feasibility of integrating mobile apps into routine follow-up care.

METHODS

MEDLINE, EMBASE, Emcare, and PsycINFO databases were searched for studies evaluating apps that addressed at least one of the five Cancer Survivorship Care Quality Framework (CSCQF) domains published up until December 2021. Studies were narratively synthesized. Implementation barriers and facilitators were mapped against the Technology Acceptance Model.

RESULTS

Twenty-three primary studies were included in this review. Only three randomized controlled trials (RCTs) were identified. Studies generally found mobile apps to be feasible, acceptable, and well-placed to support survivorship care. Health promotion was the most predominant CSCQF domain with apps primarily aiming to support exercise and dietary changes. The domains of monitoring for cancer recurrence (n=5) and management of co-morbidities (n=1) were underrepresented. Barriers to app use included greater time since active treatment, lack of familiarity with technology, and content not tailored to the user.

CONCLUSIONS

Mobile apps are both feasible and acceptable in supporting the transition between active treatment and follow-up care. However, understanding the utility of such apps is limited by the low number of RCTs.

IMPLICATIONS FOR CANCER SURVIVORS

Mobile apps have the potential to be useful support tools for patients post-treatment. However, given the number of apps developed, targeted, and available to cancer survivors, practical guidance to help cancer survivors choose appropriate apps is needed.

The Development of Game-Based Digital Mental Health Interventions: Bridging the Paradigms of Health Care and Entertainment

Game elements are increasingly used to improve user engagement in digital mental health interventions, and specific game mechanics may yield therapeutic effects per se and thereby contribute to digital mental health intervention efficacy. However, only a few commercial game-based interventions are available. We suggest that the key challenge in their development reflects the tension between the 2 underlying paradigms, health care and entertainment, which have disparate goals and processes in digital development. We describe 3 approaches currently used to negotiate the 2 paradigms: the gamification of health care software, designing serious games, and purpose shifting existing entertainment games. We advanced an integrative framework to focus attention on 4 key themes in intervention development: target audience, engagement, mechanisms of action, and health-related effectiveness. On each theme, we show how the 2 paradigms contrast and can complement each other. Finally, we consider the 4 interdependent themes through the new product development phases from concept to production. Our viewpoint provides an integrative synthesis that facilitates the research, design, and development of game-based digital mental health interventions.

Acceptability and Utility of a Smartphone App to Support Adolescent Mental Health (BeMe): Program Evaluation Study

BACKGROUND

Adolescents face unprecedented mental health challenges, and technology has the opportunity to facilitate access and support digitally connected generations. The combination of digital tools and live human connection may hold particular promise for resonating with and flexibly supporting young people's mental health.

OBJECTIVE

This study aimed to describe the BeMe app-based platform to support adolescents' mental health and well-being and to examine app engagement, usability, and satisfaction.

METHODS

Adolescents in the United States, aged 13 to 20 years, were recruited via the web and enrolled between September 1 and October 31, 2022. App engagement, feature use, clinical functioning, and satisfaction with BeMe were examined for 30 days. BeMe provides content based on cognitive behavioral therapy, dialectical behavior therapy, motivational interviewing, and positive psychology; interactive activities; live text-based coaching; links to clinical services; and crisis support tools (digital and live).

RESULTS

The average age of the sample (N=13,421) was 15.04 (SD 1.7) years, and 56.72% (7612/13,421) identified with she/her pronouns. For the subsample that completed the in-app assessments, the mean scores indicated concern for depression (8-item Patient Health Questionnaire mean 15.68/20, SD 5.9; n=239), anxiety (7-item Generalized Anxiety Disorder Questionnaire mean 13.37/17, SD 5.0; n=791), and poor well-being (World Health Organization-Five Well-being Index mean 30.15/100, SD 16.1; n=1923). Overall, the adolescents engaged with BeMe for an average of 2.38 (SD 2.7) days in 7.94 (SD 24.1) sessions and completed 11.26 (SD 19.8) activities. Most adolescents engaged with BeMe's content (12,270/13,421, 91.42%), mood ratings (13,094/13,421, 97.56%), and interactive skills (10,098/13,421, 75.24%), and almost one-fifth of the adolescents engaged with coaching (2539/13,421, 18.92%), clinical resources (2411/13,421, 17.96%), and crisis support resources (2499/13,421, 18.62%). Overall app engagement (total activities) was highest among female and gender-neutral adolescents compared with male adolescents (all P<.001) and was highest among younger adolescents (aged 13-14 years) compared with all other ages (all P<.001). Satisfaction ratings were generally high for content (eg, 158/176, 89.8% rated as helpful and 1044/1139, 91.66% improved coping self-efficacy), activities (5362/8468, 63.32% helpful and 4408/6072, 72.6% useful in coping with big feelings), and coaching (747/894, 83.6% helpful and 747/894, 83.6% improved coping self-efficacy). Engagement (total activities completed) predicted the likelihood of app satisfaction (P<.001).

CONCLUSIONS

Many adolescents downloaded the BeMe app and completed multiple sessions and activities. Engagement with BeMe was higher among female and younger adolescents. Ratings of BeMe's content, activities, and coaching were very positive for cognitive precursors aimed at reducing depression and anxiety and improving well-being. The findings will inform future app development to promote more sustained engagement, and future evaluations will assess the effects of BeMe on changes in mental health outcomes.

Assessing Patient Adherence to and Engagement With Digital Interventions for Depression in Clinical Trials: Systematic Literature Review

BACKGROUND

New approaches to the treatment of depression are necessary for patients who do not respond to current treatments or lack access to them because of barriers such as cost, stigma, and provider shortage. Digital interventions for depression are promising; however, low patient engagement could limit their effectiveness.

OBJECTIVE

This systematic literature review (SLR) assessed how participant adherence to and engagement with digital interventions for depression have been measured in the published literature, what levels of adherence and engagement have been reported, and whether higher adherence and increased engagement are linked to increased efficacy.

METHODS

We focused on a participant population of adults (aged ≥18 years) with depression or major depressive disorder as the primary diagnosis and included clinical trials, feasibility studies, and pilot studies of digital interventions for treating depression, such as digital therapeutics. We screened 756 unique records from Ovid MEDLINE, Embase, and Cochrane published between January 1, 2000, and April 15, 2022; extracted data from and appraised the 94 studies meeting the inclusion criteria; and performed a primarily descriptive analysis. Otsuka Pharmaceutical Development & Commercialization, Inc (Princeton, New Jersey, United States) funded this study.

RESULTS

This SLR encompassed results from 20,111 participants in studies using 47 unique web-based interventions (an additional 10 web-based interventions were not described by name), 15 mobile app interventions, 5 app-based interventions that are also accessible via the web, and 1 CD-ROM. Adherence was most often measured as the percentage of participants who completed all available modules. Less than half (44.2%) of the participants completed all the modules; however, the average dose received was 60.7% of the available modules. Although engagement with digital interventions was measured differently in different studies, it was most commonly measured as the number of modules completed, the mean of which was 6.4 (means ranged from 1.0 to 19.7) modules. The mean amount of time participants engaged with the interventions was 3.9 (means ranged from 0.7 to 8.4) hours. Most studies of web-based (34/45, 76%) and app-based (8/9, 89%) interventions found that the intervention group had substantially greater improvement for at least 1 outcome than the control group (eg, care as usual, waitlist, or active control). Of the 14 studies that investigated the relationship between engagement and efficacy, 9 (64%) found that increased engagement with digital interventions was significantly associated with improved participant outcomes. The limitations of this SLR include publication bias, which may overstate engagement and efficacy, and low participant diversity, which reduces the generalizability.

CONCLUSIONS

Patient adherence to and engagement with digital interventions for depression have been reported in the literature using various metrics. Arriving at more standardized ways of reporting adherence and engagement would enable more effective comparisons across different digital interventions, studies, and populations.

Assessing Mood With the Identifying Depression Early in Adolescence Chatbot (IDEABot): Development and Implementation Study

BACKGROUND

Mental health status assessment is mostly limited to clinical or research settings, but recent technological advances provide new opportunities for measurement using more ecological approaches. Leveraging apps already in use by individuals on their smartphones, such as chatbots, could be a useful approach to capture subjective reports of mood in the moment.

OBJECTIVE

This study aimed to describe the development and implementation of the Identifying Depression Early in Adolescence Chatbot (IDEABot), a WhatsApp-based tool designed for collecting intensive longitudinal data on adolescents' mood.

METHODS

The IDEABot was developed to collect data from Brazilian adolescents via WhatsApp as part of the Identifying Depression Early in Adolescence Risk Stratified Cohort (IDEA-RiSCo) study. It supports the administration and collection of self-reported structured items or questionnaires and audio responses. The development explored WhatsApp's default features, such as emojis and recorded audio messages, and focused on scripting relevant and acceptable conversations. The IDEABot supports 5 types of interactions: textual and audio questions, administration of a version of the Short Mood and Feelings Questionnaire, unprompted interactions, and a snooze function. Six adolescents (n=4, 67% male participants and n=2, 33% female participants) aged 16 to 18 years tested the initial version of the IDEABot and were engaged to codevelop the final version of the app. The IDEABot was subsequently used for data collection in the second- and third-year follow-ups of the IDEA-RiSCo study.

RESULTS

The adolescents assessed the initial version of the IDEABot as enjoyable and made suggestions for improvements that were subsequently implemented. The IDEABot's final version follows a structured script with the choice of answer based on exact text matches throughout 15 days. The implementation of the IDEABot in 2 waves of the IDEA-RiSCo sample (140 and 132 eligible adolescents in the second- and third-year follow-ups, respectively) evidenced adequate engagement indicators, with good acceptance for using the tool (113/140, 80.7% and 122/132, 92.4% for second- and third-year follow-up use, respectively), low attrition (only 1/113, 0.9% and 1/122, 0.8%, respectively, failed to engage in the protocol after initial interaction), and high compliance in terms of the proportion of responses in relation to the total number of elicited prompts (12.8, SD 3.5; 91% out of 14 possible interactions and 10.57, SD 3.4; 76% out of 14 possible interactions, respectively).

CONCLUSIONS

The IDEABot is a frugal app that leverages an existing app already in daily use by our target population. It follows a simple rule-based approach that can be easily tested and implemented in diverse settings and possibly diminishes the burden of intensive data collection for participants by repurposing WhatsApp. In this context, the IDEABot appears as an acceptable and potentially scalable tool for gathering momentary information that can enhance our understanding of mood fluctuations and development.

Efficacy of stand-alone digital mental health applications for anxiety and depression: A meta-analysis of randomized controlled trials

BACKGROUND

Anxiety and depressive disorders affect 20% of the population, cause functional impairment, and represent a leading cause of disability. Although evidence-based treatments exist, the shortage of trained clinicians and high demand for mental health services have resulted in limited access to evidence-based care. Digital mental health applications (DMHA) present innovative, scalable, and sustainable solutions to address disparities in mental health care.

METHODS

The present study used meta-analytic techniques to evaluate the therapeutic effect of DMHAs in randomized controlled trials (RCTs) for individuals experiencing anxiety and/or depressive symptoms. Search terms were selected based on concepts related to digital mental health applications, mental health/wellness, intervention type, trial design, and anxiety and/or depression symptoms/diagnosis outcomes to capture all potentially eligible results. Potential demographic, DMHA, and trial design characteristics were examined as moderators of therapeutic effects.

RESULTS

Random effects meta-analyses found that stand-alone DMHAs produced a modest reduction in anxiety (g = 0.31) and depressive (g = 0.35) symptom severity. Several moderators influenced the therapeutic effects of DMHAs for anxiety and/or depressive symptoms including treatment duration, participant inclusion criteria, and outcome measures.

LIMITATIONS

Minimal information was available on DMHA usability and participant engagement with DMHAs within RCTs.

CONCLUSIONS

While DMHAs have the potential to be scalable and sustainable solutions to improve access and availability of evidence-based mental healthcare, moderator analyses highlight the considerations for implementation of DMHAs in practice. Further research is needed to understand factors that influence therapeutic effects of DMHAs and investigate strategies to optimize its implementation and overcome the extant research-to-practice gap.

Measuring the active elements of cognitive-behavioral therapies

Understanding how and for whom cognitive-behavioral therapies work is central to the development and improvement of mental health interventions. Suboptimal quantification of the active elements of cognitive-behavioral therapies has hampered progress in elucidating mechanisms of change. To advance process research on cognitive-behavioral therapies, we describe a theoretical measurement framework that focuses on the delivery, receipt, and application of the active elements of these interventions. We then provide recommendations for measuring the active elements of cognitive-behavioral therapies aligned with this framework. Finally, to support measurement harmonization and improve study comparability, we propose the development of a publicly available repository of assessment tools: the Active Elements of Cognitive-Behavioral Therapies Measurement Kit.

Human-to-Computer Interactivity Features Incorporated Into Behavioral Health mHealth Apps: Systematic Search

BACKGROUND

While there are thousands of behavioral health apps available to consumers, users often quickly discontinue their use, which limits their therapeutic value. By varying the types and number of ways that users can interact with behavioral health mobile health apps, developers may be able to support greater therapeutic engagement and increase app stickiness.

OBJECTIVE

The main objective of this analysis was to systematically characterize the types of user interactions that are available in behavioral health apps and then examine if greater interactivity was associated with greater user satisfaction, as measured by app metrics.

METHODS

Using a modified PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) methodology, we searched several different app clearinghouse websites and identified 76 behavioral health apps that included some type of interactivity. We then filtered the results to ensure we were examining behavioral health apps and further refined our search to include apps that identified one or more of the following terms: peer or therapist forum, discussion, feedback, professional, licensed, buddy, friend, artificial intelligence, chatbot, counselor, therapist, provider, mentor, bot, coach, message, comment, chat room, community, games, care team, connect, share, and support in the app descriptions. In the final group of 34 apps, we examined the presence of 6 types of human-machine interactivities: human-to-human with peers, human-to-human with providers, human-to-artificial intelligence, human-to-algorithms, human-to-data, and novel interactive smartphone modalities. We also downloaded information on app user ratings and visibility, as well as reviewed other key app features.

RESULTS

We found that on average, the 34 apps reviewed included 2.53 (SD 1.05; range 1-5) features of interactivity. The most common types of interactivities were human-to-data (n=34, 100%), followed by human-to-algorithm (n=15, 44.2%). The least common type of interactivity was human-artificial intelligence (n=7, 20.5%). There were no significant associations between the total number of app interactivity features and user ratings or app visibility. We found that a full range of therapeutic interactivity features were not used in behavioral health apps.

CONCLUSIONS

Ideally, app developers would do well to include more interactivity features in behavioral health apps in order to fully use the capabilities of smartphone technologies and increase app stickiness. Theoretically, increased user engagement would occur by using multiple types of user interactivity, thereby maximizing the benefits that a person would receive when using a mobile health app.

Rating analysis and BERTopic modeling of consumer versus regulated mHealth app reviews in Germany

Germany introduced prescription-based mobile health (mHealth) apps in October 2020, becoming the first country to offer them fully reimbursed by health insurance. These regulated apps, known as DiGAs, undergo a rigorous approval process similar to pharmaceuticals, including data protection measures and sometimes clinical trials. This study compares the user experience of DiGAs with non-prescription mHealth apps in Germany, analyzing both average app store ratings and written reviews. Our study pioneers the use of BERTopic for sentiment analysis and topic modeling in the mHealth research domain. The dataset comprises 15 DiGAs and 50 comparable apps, totaling 17,588 German-language reviews. Results reveal that DiGAs receive higher contemporary ratings than non-regulated apps (Android: 3.82 vs. 3.77; iOS: 3.78 vs. 3.53; p < 0.01; non-parametric Mann-Whitney-Wilcoxon test). Key factors contributing to positive user experience with DiGAs are customer service and personalization (15%) and ease of use (13%). However, challenges for DiGAs include software bugs (24%) and a cumbersome registration process (20%). Negative user reviews highlight concerns about therapy effectiveness (11%). Excessive pricing is the main concern for the non-regulated group (27%). Data privacy and security receive limited attention from users (DiGAs: 0.5%; comparators: 2%). In conclusion, DiGAs are generally perceived positively based on ratings and sentiment analysis of reviews. However, addressing pricing concerns in the non-regulated mHealth sector is crucial. Integrating user experience evaluation into the review process could improve adherence and health outcomes.

Feasibility and Acceptability of the Aboriginal and Islander Mental Health Initiative for Youth App: Nonrandomized Pilot With First Nations Young People

BACKGROUND

Despite young First Nations Australians being typically healthy, happy, and connected to family and culture, high rates of emotional distress, suicide, and self-harm are also observed. Differing worldviews of service providers and First Nations young people regarding illness and treatment practices, language differences, culturally inappropriate service models, geographical remoteness, and stigma can all inhibit access to appropriate mental health support. Mental health treatments delivered digitally (digital mental health; dMH) offer flexible access to evidence-based, nonstigmatizing, low-cost treatment and early intervention on a broad scale. There is a rapidly growing use and acceptance of these technologies among young First Nations people.

OBJECTIVE

The objective was to assess the feasibility, acceptability, and use of the newly developed Aboriginal and Islander Mental Health Initiative for Youth (AIMhi-Y) app and determine the feasibility of study procedures in preparation for future assessments of effectiveness.

METHODS

This was a nonrandomized pre-post study using mixed methods. First Nations young people aged 12-25 years who provided consent (with parental consent where appropriate) and possessed the ability to navigate a simple app with basic English literacy were included. Researchers conducted one face-to-face 20-minute session with participants to introduce and orient them to the AIMhi-Y app. The app integrates culturally adapted low-intensity cognitive behavioral therapy (CBT), psychoeducation, and mindfulness-based activities. Participants received supportive text messages weekly throughout the 4-week intervention period and completed assessments of psychological distress, depression, anxiety, substance misuse, help-seeking, service use, and parent-rated strengths and difficulties at baseline and 4 weeks. Qualitative interviews and rating scales were completed at 4 weeks to gain feedback on subjective experience, look and style, content, overall rating, check-ins, and involvement in the study. App use data were collected.

RESULTS

Thirty young people (17 males and 13 females) aged between 12 and 18 (mean 14.0, SD 1.55) years were assessed at baseline and 4 weeks. Repeated measures 2-tailed t tests showed improvements in well-being measures that were statistically and clinically significant for psychological distress (Kessler Psychological Distress Scale, 10-item) and depressive symptoms (Patient Health Questionnaire, 2-item). Participants spent on average 37 minutes in the app. The app was rated positively, with mean ratings of 4 out of 5 points (on scales of 1-5). Participants reported that they found the app easy to use, culturally relevant, and useful. The feasibility of the study was demonstrated with a 62% recruitment rate, a 90% retention rate, and high study acceptability ratings.

CONCLUSIONS

This study supports earlier research suggesting that dMH apps that are appropriately designed with and for the target populations are a feasible and acceptable means of lowering symptoms for mental health disorders among First Nations youth.

Exploring engagement with the CBT-informed Actissist smartphone application for early psychosis

BACKGROUND

Individuals with psychosis report favourable attitudes towards psychological interventions delivered via smartphone apps. Evidence for acceptability, safety, feasibility and efficacy is promising but in-depth reporting of app engagement in trials is sparse.

AIMS

To examine how people with psychosis engaged with the cognitive behaviour therapy (CBT)-informed Actissist app over a 12-week intervention period, and to examine factors associated with app engagement.

METHODS

Secondary data from participants in the intervention arm (n = 24) of a proof-of-concept randomised controlled trial of the Actissist app were analysed. The app prompted participants to engage with app-based CBT-informed material in five domains (voices, socialization, cannabis use, paranoia, perceived criticism) at pseudo-random intervals (three notifications per day, six days per week). Participants could self-initiate use any time. App use was financially incentivised.

RESULTS

Participants responded to 47% of app notifications. Most app engagements (87%) were app-initiated rather than self-initiated. Participants engaged most with the voices domain, then paranoia. Age and employment status were significantly associated with overall app engagement.

CONCLUSION

Individuals with psychosis engaged well with Actissist, particularly with areas focussing on voice-hearing and paranoia. App-generated reminders successfully prompted app engagement. As financial incentives may have increased app engagement, future studies of non-incentivized engagement in larger samples are needed.

Evaluation of a supportive care app for coordinating caring networks: an analysis of the first 19,000 users

Background

Major illnesses such as cancer, and other traumatic life events, can lead to sudden increases in supportive care needs. This study aimed to describe engagement, acceptability and satisfaction with a supportive care networking app under real-world conditions.

Methods

A total of 10,952 individuals used the app during the study period (2018-2022). The app is designed to enable "captains" to assemble a network of friends and family members to provide timely, and individually tailored, supportive care (including assistance with tasks such as taking children to school, cooking meals, grocery shopping, and transport to appointments). Engagement was determined from server data, whilst acceptability and satisfaction were captured using purposed-designed surveys.

Results

Users were mostly female (76%) and aged between 30 and 49 years (61%). The most common reason for using the app was sudden illness (web: 81%; mobile: 64%). An average of 42 tasks were requested per network, with a 32% acceptance rate. Significantly more tasks were requested (web: 52.2 tasks per network; mobile: 31.7 tasks per network; p < 0.001) and accepted (web: 43.2%; mobile: 20.2%; p < 0.001) in the web app vs. the mobile app. Task requests in the web app most commonly related to food (43% of requested tasks), social (15% of requested tasks) and children (13% of requested tasks). The task acceptance rate differed by task categories (p < 0.001), with tasks relating to transport, medical appointments and children accepted at the highest rates (56%, 52% and 49%, respectively). Acceptability and satisfaction data suggested that the app was well received and overall, participants were satisfied with the app.

Conclusion

Findings suggested that this support care networking app achieved widespread uptake for a wide variety of supportive care tasks. Future research focused on optimizing engagement with the mobile app and examining the effectiveness of the app for improving patient and hospital outcomes is warranted.

Validation of the Attitudes Towards Psychological Online Interventions Questionnaire Among Black Americans: Cross-cultural Confirmatory Factor Analysis

BACKGROUND

Acceptability of digital mental health interventions is a significant predictor of treatment-seeking behavior and engagement. However, acceptability has been conceptualized and operationalized in various ways, which decreases measurement precision and leads to heterogeneous conclusions about acceptability. Standardized self-report measures of acceptability have been developed, which have the potential to ameliorate these problems, but none have demonstrated evidence for validation among Black communities, which limits our understanding of attitudes toward these interventions among racially minoritized groups with well-documented barriers to mental health treatment.

OBJECTIVE

This study aims to examine the psychometric validity and reliability of one of the first and most widely used measures of acceptability, the Attitudes Towards Psychological Online Interventions Questionnaire, among a Black American sample.

METHODS

Participants (N=254) were recruited from a large southeastern university and the surrounding metropolitan area and completed the self-report measure via a web-based survey. A confirmatory factor analysis using mean and variance adjusted weighted least squares estimation was conducted to examine the validity of the underlying hierarchical 4-factor structure proposed by the original authors of the scale. An alternative, hierarchical 2-factor structure model and bifactor model were examined for comparative fit.

RESULTS

The findings indicated that the bifactor model demonstrated a superior fit (comparative fit index=0.96, Tucker-Lewis index=0.94, standardized root mean squared residual=0.03, and root mean square error of approximation=0.09) compared with both 2- and 4-factor hierarchical structure models.

CONCLUSIONS

The findings suggest that, within a Black American sample, there may be greater utility in interpreting the Attitudes Towards Psychological Online Interventions Questionnaire subscales as attitudinal constructs that are distinct from the global acceptability factor. The theoretical and practical implications for culturally responsive measurements were explored.

How do patients and families evaluate attitude of psychiatrists in Japan?: quantitative content analysis of open-ended items of patient responses from a large-scale questionnaire survey

BACKGROUND

Patient and Public Involvement (PPI) has been widely advocated in psychiatric fields. In Japan, however, PPI has not been implemented in clinical practice. In order to improve quality of psychiatric service in Japan, it is essential to understand psychiatrists' attitudes from the patients' perspective as a first step in practicing PPI. This study aimed to investigate the patients' evaluation of psychiatrists' attitudes by illustrating themes appeared in the questionnaire survey.

METHODS

This study used the data obtained from the questionnaire survey responded by 2,683 patients with family members who belong to the family associations for psychiatric patients in Japan. Three open-ended question items in this survey, "criteria for selecting a psychiatrist (784 patient responses, response rate 29.2%)," "attitude of the psychiatrist in charge (929 patient responses, response rate 34.6%)," and "communication skills of the psychiatrist in charge (739 patient responses, response rate 27.5%)" were analyzed by co-occurrence network using KH Coder software.

RESULTS

The common theme observed in all three items was whether psychiatrist took sufficient consultation time. The criteria for selecting a psychiatrist were summarized whether psychiatrist provided appropriate advices for patients' problems, whether psychiatrist cared about patients' demands and whether psychiatrist informed to patients about diseases and medications. The attitudes of the psychiatrists in charge that patients had most wanted their psychiatrists to improve were: psychiatrists only watch the computer, make diagnosis according to the patients' individual condition, and try to build a relationship of trust with the patient. The patients' demands regarding communication skills of the psychiatrist in charge included: whether the psychiatrist communicated in a way that improves the patient's psychological state, whether the psychiatrist was attentive to the patients' family, and whether the psychiatrist could control his/her own mood during the consultation.

CONCLUSION

The results reflected the patients' demands that do not appear in closed-ended items. It was suggested that patients'open-ended responses to questionnaires and their involvement in the psychiatric research (PPI) may provide more insight into improving pshchiaric care in Japan.

Characterizing observed and effective behavioral engagement with smartphone cognitive behavioral therapy for body dysmorphic disorder: A methods roadmap and use case

Smartphone psychotherapies are growing in popularity, yet little is understood about (1) how people prefer to engage with psychotherapy apps, or (2) which engagement patterns constitute effective engagement. The present study uses secondary data from a 12-week randomized waitlist-controlled trial of smartphone-delivered cognitive behavioral therapy (CBT) for body dysmorphic disorder (BDD) (N = 77) to address these aims. Additionally, using the present study as a use-case, we seek to provide a roadmap for how researchers may improve upon methodological limitations of existing smartphone psychotherapy engagement research. We measured behavioral engagement via 19 objective variables derived from phone analytics data, which we reduced via factor analysis into two factors: 1) use volume and frequency, and 2) session duration. Cluster analysis based on engagement factors yielded three engager types, which mapped onto "deep" users, "samplers," and "light" users. The clusters did not differ significantly in improvement in BDD severity across treatment, although deep users improved more than light users at a marginally significant level. Results suggest that varying patterns of preferred engagement may be efficacious. Moreover, the study's methods provide an example of how researchers can measure and study behavioral engagement comprehensively and objectively. Trial Registration: ClinicalTrials.gov Identifier: NCT04034693.

Measuring Engagement with Mental Health and Behavior Change Interventions: an Integrative Review of Methods and Instruments

BACKGROUND

Engagement is a complex construct consisting of behavioral, cognitive, and affective dimensions, making engagement a difficult construct to measure. This integrative review aims to (1) present a multidisciplinary overview of measurement methods that are currently used to measure engagement with adult mental health and behavior change interventions, delivered in-person, blended, or digitally, and (2) provide a set of recommendations and considerations for researchers wishing to study engagement.

METHODS

We used an integrative approach and identified original studies and reviews on engagement with mental health or behavior change interventions that were delivered in-person, digitally, or blended.

RESULTS

Forty articles were analyzed in this review. Common methods to assess engagement were through objective usage data, questionnaire-based data, and qualitative data, with objective usage data being used most frequently. Based on the synthesis of engagement measures, we advise researchers to (1) predefine the operationalization of engagement for their specific research context, (2) measure behavioral, cognitive, and affective dimensions of engagement in all cases, and (3) measure engagement over time.

CONCLUSIONS

Current literature shows a bias towards behavioral measures of engagement in research, as most studies measured engagement exclusively through objective usage data, without including cognitive and affective measures of engagement. We hope that our recommendations will help to reduce this bias and to steer engagement research towards an integrated approach.

Use of a Mobile Biofeedback Application to Provide Health Coaching for Stress Self-Management: Findings from a Pilot Quasi-Experiment

BACKGROUND

Mental health is an increasing concern among vulnerable populations, including college students and veterans.

OBJECTIVE

The purpose of this study was to determine if mobile Health (mHealth) technology combined with health coaching can better enable user to self-manage their mental health.

METHODS

This study evaluated a mobile Biofeedback app that provided health coaching on stress self-management for college student veterans' mental health concerns. Twenty-four college student veterans were recruited from a large public university in Texas during the spring 2020 semester, impacted by COVID-19. Ten participants were assigned to the intervention group where they used the mobile Biofeedback app on their smartphones and smartwatches, and 14 were assigned to the control group without the app; assignment was based on mobile phone compatibility. Both groups participated in one initial lab session where they learned a deep breathing exercise technique. The intervention group was then asked to use the mobile Biofeedback app during their daily lives using the Biofeedback app and smartwatch, and the control group was asked to perform the breathing exercises on their own. Both groups filled out Patient Health Questionnaire (PHQ-9) and Generalized Anxiety Disorder (GAD-7) self-assessments at two-week intervals. At the end of the semester, both groups were given an exit interview to provide user experience and perceived benefits of health coaching via the mobile Biofeedback app.

RESULTS

Deep breathing exercise in the initial lab session reduced stress in both groups. Over the course of the study, the app recorded 565 coached breathing exercises with a significant decrease (approximately 3 beats per minute) in participants' heart rate during the 6-minute time period immediately after conducting the breathing exercises [Spearman's rank correlation coefficient of -0.61 (P<.001 and S=9,816,176)]. There was no significant difference between the two groups for PHQ-9 and GAD-7 scores over the course of the semester. Exit interview responses indicate that participants perceived that the mobile Biofeedback app improved their health and helped them address stress challenges. All participants reported that the intervention helped them manage their stress better and expressed that health coaching via a mobile device would improve their overall health.

CONCLUSIONS

Participants reported a positive perception of the app for their mental health self-management during a stressful semester. Future work should examine long-term effects of the app with a larger sample size balanced between male and female participants, randomized participant allocation, real-time detection of mental health symptoms, and additional features of the app.

CLINICALTRIAL

Using a Safety Planning Mobile App to Address Suicidality in Young People Attending Community Mental Health Services in Ireland: Protocol for a Pilot Randomized Controlled Trial

BACKGROUND

Over 700,000 people die by suicide annually, making it the fourth leading cause of death among those aged 15-29 years globally. Safety planning is recommended best practice when individuals at risk of suicide present to health services. A safety plan, developed in collaboration with a health care practitioner, details the steps to be taken in an emotional crisis. SafePlan, a safety planning mobile app, was designed to support young people experiencing suicidal thoughts and behaviors and to record their plan in a way that is accessible immediately and in situ.

OBJECTIVE

The aim of this study is to assess the feasibility and acceptability of the SafePlan mobile app for patients experiencing suicidal thoughts and behaviors and their clinicians within Irish community mental health services, examine the feasibility of study procedures for both patients and clinicians, and determine if the SafePlan condition yields superior outcomes when compared with the control condition.

METHODS

A total of 80 participants aged 16-35 years accessing Irish mental health services will be randomized (1:1) to receive the SafePlan app plus treatment as usual or treatment as usual plus a paper-based safety plan. The feasibility and acceptability of the SafePlan app and study procedures will be evaluated using both qualitative and quantitative methodologies. The primary outcomes are feasibility outcomes and include the acceptability of the app to participants and clinicians, the feasibility of delivery in this setting, recruitment, retention, and app use. The feasibility and acceptability of the following measures in a full randomized controlled trial will also be assessed: the Beck Scale for Suicide Ideation, Columbia Suicide Severity Rating Scale, Coping Self-Efficacy Scale, Interpersonal Needs Questionnaire, and Client Service Receipt Inventory. A repeated measures design with outcome data collected at baseline, post intervention (8 weeks), and at 6-month follow-up will be used to compare changes in suicidal ideation for the intervention condition relative to the waitlist control condition. A cost-outcome description will also be undertaken. Thematic analyses will be used to analyze the qualitative data gathered through semistructured interviews with patients and clinicians.

RESULTS

As of January 2023, funding and ethics approval have been acquired, and clinician champions across mental health service sites have been established. Data collection is expected to commence by April 2023. The submission of completed manuscript is expected by April 2025.

CONCLUSIONS

The framework for Decision-making after Pilot and feasibility Trials will inform the decision to progress to a full trial. The results will inform patients, researchers, clinicians, and health services of the feasibility and acceptability of the SafePlan app in community mental health services. The findings will have implications for further research and policy regarding the broader integration of safety planning apps.

TRIAL REGISTRATION

OSF Registries osf.io/3y54m; https://osf.io/3y54m.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/44205.

Building Emotional Awareness and Mental Health (BEAM): an open-pilot and feasibility study of a digital mental health and parenting intervention for mothers of infants

BACKGROUND

Maternal mental health concerns and parenting stress in the first few years following childbirth are common and pose significant risks to maternal and child well-being. The COVID-19 pandemic has led to increases in maternal depression and anxiety and has presented unique parenting stressors. Although early intervention is crucial, there are significant barriers to accessing care.

METHODS

To inform a larger randomized controlled trial, the current open-pilot trial investigated initial evidence for the feasibility, acceptability, and efficacy of a newly developed online group therapy and app-based mental health and parenting program (BEAM) for mothers of infants. Forty-six mothers 18 years or older with clinically elevated depression scores, with an infant aged 6-17 months old, and who lived in Manitoba or Alberta were enrolled in the 10-week program (starting in July 2021) and completed self-report surveys.

RESULTS

The majority of participants engaged in each of the program components at least once and participants indicated relatively high levels of app satisfaction, ease of use, and usefulness. However, there was a high level of attrition (46%). Paired-sample t-tests indicated significant pre- to post-intervention change in maternal depression, anxiety, and parenting stress, and in child internalizing, but not externalizing symptoms. Effect sizes were in the medium to high range, with the largest effect size observed for depressive symptoms (Cohen's d = .93).

DISCUSSION

This study shows moderate levels of feasibility and strong preliminary efficacy of the BEAM program. Limitations to program design and delivery are being addressed for testing in adequately powered follow-up trials of the BEAM program for mothers of infants.

TRIAL REGISTRATION

NCT04772677 . Registered on February 26 2021.

The Feasibility of Implementing Remote Measurement Technologies in Psychological Treatment for Depression: Mixed Methods Study on Engagement

BACKGROUND

Remote measurement technologies (RMTs) such as smartphones and wearables can help improve treatment for depression by providing objective, continuous, and ecologically valid insights into mood and behavior. Engagement with RMTs is varied and highly context dependent; however, few studies have investigated their feasibility in the context of treatment.

OBJECTIVE

A mixed methods design was used to evaluate engagement with active and passive data collection via RMT in people with depression undergoing psychotherapy. We evaluated the effects of treatment on 2 different types of engagement: study attrition (engagement with study protocol) and patterns of missing data (engagement with digital devices), which we termed data availability. Qualitative interviews were conducted to help interpret the differences in engagement.

METHODS

A total of 66 people undergoing psychological therapy for depression were followed up for 7 months. Active data were gathered from weekly questionnaires and speech and cognitive tasks, and passive data were gathered from smartphone sensors and a Fitbit (Fitbit Inc) wearable device.

RESULTS

The overall retention rate was 60%. Higher-intensity treatment (χ21=4.6; P=.03) and higher baseline anxiety (t56.28=-2.80, 2-tailed; P=.007) were associated with attrition, but depression severity was not (t50.4=-0.18; P=.86). A trend toward significance was found for the association between longer treatments and increased attrition (U=339.5; P=.05). Data availability was higher for active data than for passive data initially but declined at a sharper rate (90%-30% drop in 7 months). As for passive data, wearable data availability fell from a maximum of 80% to 45% at 7 months but showed higher overall data availability than smartphone-based data, which remained stable at the range of 20%-40% throughout. Missing data were more prevalent among GPS location data, followed by among Bluetooth data, then among accelerometry data. As for active data, speech and cognitive tasks had lower completion rates than clinical questionnaires. The participants in treatment provided less Fitbit data but more active data than those on the waiting list.

CONCLUSIONS

Different data streams showed varied patterns of missing data, despite being gathered from the same device. Longer and more complex treatments and clinical characteristics such as higher baseline anxiety may reduce long-term engagement with RMTs, and different devices may show opposite patterns of missingness during treatment. This has implications for the scalability and uptake of RMTs in health care settings, the generalizability and accuracy of the data collected by these methods, feature construction, and the appropriateness of RMT use in the long term.

User engagement in mobile apps for people with schizophrenia: A scoping review

Over the past decade, there has been an increase in the number of mobile apps designed for mental health proposes and mHealth has been perceived as a promising approach to help people with schizophrenia to manage their condition. However, adoption rates are relatively low and long-term user engagement is a major issue. The aim of this study is to identify and better understand what strategies and factors may influence user engagement and facilitate prolonged use of apps for people with schizophrenia to better manage their illness. A scoping review was conducted in accordance with the Arksey and O'Malley scoping review framework and following PRISMA ScR guidelines. The sources consisted of searching four electronic databases. Rayyan software was used for this study selection process and a narrative approach was used to synthesize the extracted data. A total of 28 studies which met the inclusion criteria were identified. The engagement strategies included push notifications, message prompts, personalization, application customization, goal setting, game-like features, use of different multimedia formats, social connectedness, support (peers and professionals), reliability of content and quality of feedback received. Some demographic factors may influence adherence such as age, gender, education level and socioeconomic status. Other factors also may play a role impacting engagement: health status, data privacy and security, involvement in design process, incentives for participation, app usage fitting in the user routines, initial training, and constant technical support. Included studies present high heterogeneity in outcome measures and thresholds criteria to assess engagement. Understanding what influences engagement and how to measure it is essential to enhance the design of mobile apps and deliver scalable solutions to help people with schizophrenia better manage their illness in their real-world uptake.

Evaluating Conversational Agents for Mental Health: Scoping Review of Outcomes and Outcome Measurement Instruments (Preprint)

BACKGROUND

Rapid proliferation of mental health interventions delivered through conversational agents (CAs) calls for high-quality evidence to support their implementation and adoption. Selecting appropriate outcomes, instruments for measuring outcomes, and assessment methods are crucial for ensuring that interventions are evaluated effectively and with a high level of quality.

OBJECTIVE

We aimed to identify the types of outcomes, outcome measurement instruments, and assessment methods used to assess the clinical, user experience, and technical outcomes in studies that evaluated the effectiveness of CA interventions for mental health.

METHODS

We undertook a scoping review of the relevant literature to review the types of outcomes, outcome measurement instruments, and assessment methods in studies that evaluated the effectiveness of CA interventions for mental health. We performed a comprehensive search of electronic databases, including PubMed, Cochrane Central Register of Controlled Trials, Embase (Ovid), PsychINFO, and Web of Science, as well as Google Scholar and Google. We included experimental studies evaluating CA mental health interventions. The screening and data extraction were performed independently by 2 review authors in parallel. Descriptive and thematic analyses of the findings were performed.

RESULTS

We included 32 studies that targeted the promotion of mental well-being (17/32, 53%) and the treatment and monitoring of mental health symptoms (21/32, 66%). The studies reported 203 outcome measurement instruments used to measure clinical outcomes (123/203, 60.6%), user experience outcomes (75/203, 36.9%), technical outcomes (2/203, 1.0%), and other outcomes (3/203, 1.5%). Most of the outcome measurement instruments were used in only 1 study (150/203, 73.9%) and were self-reported questionnaires (170/203, 83.7%), and most were delivered electronically via survey platforms (61/203, 30.0%). No validity evidence was cited for more than half of the outcome measurement instruments (107/203, 52.7%), which were largely created or adapted for the study in which they were used (95/107, 88.8%).

CONCLUSIONS

The diversity of outcomes and the choice of outcome measurement instruments employed in studies on CAs for mental health point to the need for an established minimum core outcome set and greater use of validated instruments. Future studies should also capitalize on the affordances made available by CAs and smartphones to streamline the evaluation and reduce participants' input burden inherent to self-reporting.

Determination of Markers of Successful Implementation of Mental Health Apps for Young People: Systematic Review

Background

Smartphone apps have the potential to address some of the current issues facing service provision for young people’s mental health by improving the scalability of evidence-based mental health interventions. However, very few apps have been successfully implemented, and consensus on implementation measurement is lacking.

Objective

This review aims to determine the proportion of evidence-based mental health and well-being apps that have been successfully adopted and sustained in real-world settings. A secondary aim is to establish if key implementation determinants such as coproduction, acceptability, feasibility, appropriateness, and engagement contribute toward successful implementation and longevity.

Methods

Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, an electronic search of 5 databases in 2021 yielded 18,660 results. After full-text screening, 34 articles met the full eligibility criteria, providing data on 29 smartphone apps studied with individuals aged 15 to 25 years.

Results

Of 34 studies, only 10 (29%) studies were identified that were evaluating the effectiveness of 8 existing, commercially available mental health apps, and the remaining 24 (71%) studies reported the development and evaluation of 21 newly developed apps, of which 43% (9/21) were available, commercially or otherwise (eg, in mental health services), at the time of enquiry. Most studies addressed some implementation components including adoption, acceptability, appropriateness, feasibility, and engagement. Factors including high cost, funding constraints, and lengthy research processes impeded implementation.

Conclusions

Without addressing common implementation drivers, there is considerable redundancy in the translation of mobile mental health research findings into practice. Studies should embed implementation strategies from the outset of the planned research, build collaborations with partners already working in the field (academic and commercial) to capitalize on existing interventions and platforms, and modify and evaluate them for local contexts or target problems and populations.

Trial Registration

PROSPERO CRD42021224365; https://tinyurl.com/4umpn85f

Digital Mental Health Interventions for Depression: Scoping Review of User Engagement

BACKGROUND

While many digital mental health interventions (DMHIs) have been found to be efficacious, patient engagement with DMHIs has increasingly emerged as a concern for implementation in real-world clinical settings. To address engagement, we must first understand what standard engagement levels are in the context of randomized controlled trials (RCTs) and how these compare with other treatments.

OBJECTIVE

This scoping review aims to examine the state of reporting on intervention engagement in RCTs of mobile app-based interventions intended to treat symptoms of depression. We sought to identify what engagement metrics are and are not routinely reported as well as what the metrics that are reported reflect about standard engagement levels.

METHODS

We conducted a systematic search of 7 databases to identify studies meeting our eligibility criteria, namely, RCTs that evaluated use of a mobile app-based intervention in adults, for which depressive symptoms were a primary outcome of interest. We then extracted 2 kinds of information from each article: intervention details and indices of DMHI engagement. A 5-element framework of minimum necessary DMHI engagement reporting was derived by our team and guided our data extraction. This framework included (1) recommended app use as communicated to participants at enrollment and, when reported, app adherence criteria; (2) rate of intervention uptake among those assigned to the intervention; (3) level of app use metrics reported, specifically number of uses and time spent using the app; (4) duration of app use metrics (ie, weekly use patterns); and (5) number of intervention completers.

RESULTS

Database searching yielded 2083 unique records. Of these, 22 studies were eligible for inclusion. Only 64% (14/22) of studies included in this review specified rate of intervention uptake. Level of use metrics was only reported in 59% (13/22) of the studies reviewed. Approximately one-quarter of the studies (5/22, 23%) reported duration of use metrics. Only half (11/22, 50%) of the studies reported the number of participants who completed the app-based components of the intervention as intended or other metrics related to completion. Findings in those studies reporting metrics related to intervention completion indicated that between 14.4% and 93.0% of participants randomized to a DMHI condition completed the intervention as intended or according to a specified adherence criteria.

CONCLUSIONS

Findings suggest that engagement was underreported and widely varied. It was not uncommon to see completion rates at or below 50% (11/22) of those participants randomized to a treatment condition or to simply see completion rates not reported at all. This variability in reporting suggests a failure to establish sufficient reporting standards and limits the conclusions that can be drawn about level of engagement with DMHIs. Based on these findings, the 5-element framework applied in this review may be useful as a minimum necessary standard for DMHI engagement reporting.

The service user experience of SlowMo therapy: A co-produced thematic analysis of service users' subjective experience

OBJECTIVES

SlowMo is the first blended digital therapy for paranoia, showing significant small-moderate reductions in paranoia in a recent large-scale randomized controlled trial (RCT). This study explored the subjective service-user experience of the SlowMo therapy content and design; the experience of the blended therapy approach, including the triangle of the therapeutic alliance; and the experience of the digital aspects of the intervention.

DESIGN

Qualitative co-produced sub-study of an RCT.

METHODS

Participants were 22 adult service users with schizophrenia-spectrum psychosis and persistent distressing paranoia, who completed at least one SlowMo therapy session and a 24-week follow-up, at one of 3 sites in Oxford, London, and Sussex, UK. They were interviewed by peer researchers, using a topic guide co-produced by the Patient and Public Involvement (PPI) team. The transcribed data were analysed thematically. Multiple coding and triangulation, and lay peer researcher validation were used to reach a consensus on the final theme structure.

RESULTS

Six core themes were identified: (i) starting the SlowMo journey; (ii) the central role of the supportive therapist; (iii) slowing things down; (iv) value and learning from social connections; (v) approaches and challenges of technology; and (vi) improvements in paranoia and well-being.

CONCLUSIONS

For these service users, slowing down for a moment was helpful, and integrated into thinking over time. Learning from social connections reflected reduced isolation, and enhanced learning through videos, vignettes, and peers. The central role of the supportive therapist and the triangle of alliance between service user, therapist, and digital platform were effective in promoting positive therapeutic outcomes.

Smartphone-based interventions in bipolar disorder: Systematic review and meta-analyses of efficacy. A position paper from the International Society for Bipolar Disorders (ISBD) Big Data Task Force

BACKGROUND

The clinical effects of smartphone-based interventions for bipolar disorder (BD) have yet to be established.

OBJECTIVES

To examine the efficacy of smartphone-based interventions in BD and how the included studies reported user-engagement indicators.

METHODS

We conducted a systematic search on January 24, 2022, in PubMed, Scopus, Embase, APA PsycINFO, and Web of Science. We used random-effects meta-analysis to calculate the standardized difference (Hedges' g) in pre-post change scores between smartphone intervention and control conditions. The study was pre-registered with PROSPERO (CRD42021226668).

RESULTS

The literature search identified 6034 studies. Thirteen articles fulfilled the selection criteria. We included seven RCTs and performed meta-analyses comparing the pre-post change in depressive and (hypo)manic symptom severity, functioning, quality of life, and perceived stress between smartphone interventions and control conditions. There was significant heterogeneity among studies and no meta-analysis reached statistical significance. Results were also inconclusive regarding affective relapses and psychiatric readmissions. All studies reported positive user-engagement indicators.

CONCLUSION

We did not find evidence to support that smartphone interventions may reduce the severity of depressive or manic symptoms in BD. The high heterogeneity of studies supports the need for expert consensus to establish ideally how studies should be designed and the use of more sensitive outcomes, such as affective relapses and psychiatric hospitalizations, as well as the quantification of mood instability. The ISBD Big Data Task Force provides preliminary recommendations to reduce the heterogeneity and achieve more valid evidence in the field.

Appropriate Use and Operationalization of Adherence to Digital Cognitive Behavioral Therapy for Depression and Anxiety in Youth: Systematic Review

BACKGROUND

Digital, self-guided cognitive behavioral therapy (CBT) interventions circumvent many barriers to in-person therapy for young people (aged 12-24 years), although adherence to these interventions is low. The absence or insufficient disclosure of recommendations or instructions for appropriate use may account for this. As such, many young people may not self-administer these interventions appropriately or receive the optimal degree of treatment.

OBJECTIVE

This systematic review aims to synthesize the literature on digital CBT for depression and anxiety in young people to describe how appropriate use has been defined and communicated to users as instructions for use, to describe how adherence has been measured, and to determine the associations between adherence and treatment outcomes.

METHODS

A systematic review was conducted with 2 reviewers (SHL and MRA) extracting data independently. Overall, 4 electronic databases (Embase, MEDLINE, PsycINFO, and Cochrane Library) were searched in April 2021 for studies that met the following inclusion criteria: participants aged between 12 and 24 years, evaluated a digital CBT intervention targeting depression or anxiety, and reported instructions or recommendations for use or measures of adherence. Studies that evaluated non-CBT interventions or cognitive- or behavioral-only interventions were excluded. Methodological quality was assessed using the Cochrane Risk of Bias Tool and the Integrated Quality Criteria for the Review of Multiple Study Designs.

RESULTS

There were 32 manuscripts that met the inclusion criteria, of which 28 (88%) were unique studies (N=16,578 youths). Definitions of appropriate use varied among the different interventions in terms of intended recipients, duration and frequency of use, and the features used to support engagement and adherence to appropriate use definitions. Reporting of appropriate use definitions in studies was inconsistent, with no study systematically describing components of appropriate use or providing information on how recommendations for use were relayed to users. Most often, definitions of appropriate use were derived from the study protocol and descriptions of intervention features. Adherence was mostly operationalized as the degree of intervention completion; however, reporting of adherence data was heterogeneous. There was little evidence of an association between degree of use and outcomes in the 9 studies that examined this.

CONCLUSIONS

Definitions of appropriate use are unique to each digital CBT intervention. However, statements of appropriate use are not systematically reported in the literature. Furthermore, the extent to which recommendations for use are communicated to users is not routinely reported. Despite unique definitions of appropriate use, adherence was most often generically operationalized as the degree of intervention completion and was not consistently associated with outcomes. We proposed a framework to promote systematic reporting of definitions of appropriate use for digital interventions to provide guidance to users and to assist the development of appropriate and nuanced measures of adherence.

TRIAL REGISTRATION

PROSPERO CRD42020208668; https://tinyurl.com/4bu2yram.

Analyzing the Impact of Mobile App Engagement on Mental Health Outcomes: Secondary Analysis of the Unwinding Anxiety Program

BACKGROUND

App-based interventions provide a promising avenue for mitigating the burden on mental health services by complimenting therapist-led treatments for anxiety. However, it remains unclear how specific systems' use of app features may be associated with changes in mental health outcomes (eg, anxiety and worry).

OBJECTIVE

This study was a secondary analysis of engagement data from a stage 1 randomized controlled trial testing the impact of the Unwinding Anxiety mobile app among adults with generalized anxiety disorder. The aims of this study were 2-fold: to investigate whether higher microengagement with the primary intervention feature (ie, educational modules) is associated with positive changes in mental health outcomes at 2 months (ie, anxiety, worry, interoceptive awareness, and emotional reactivity) and to investigate whether the use of adjunctive app features is also associated with changes in mental health outcomes.

METHODS

We analyzed the intervention group during the stage 1 trial of the Unwinding Anxiety mobile app. The total use of specific mobile app features and the use specific to each feature were calculated. We used multivariate linear models with a priori significance of α=.05 to investigate the impact of cumulative app use on anxiety, worry, interoceptive awareness, and emotional regulation at 2 months, controlling for baseline scores, age, and education level in all models. Significant relationships between system use metrics and baseline participant characteristics were assessed for differences in use groupings using between-group testing (ie, 2-tailed t tests for continuous data and chi-square analyses for categorical data).

RESULTS

The sample was primarily female (25/27, 93%), and the average age was 42.9 (SD 15.6) years. Educational module completion, the central intervention component, averaged 20.2 (SD 11.4) modules out of 32 for the total sample. Multivariate models revealed that completing >75% of the program was associated with an average 22.6-point increase in interoceptive awareness (b=22.6; SE 8.32; P=.01; 95% CI 5.3-39.8) and an 11.6-point decrease in worry (b=-11.6; SE 4.12; P=.01; 95% CI -20.2 to -3.1). In addition, a single log unit change in the total number of meditations was associated with a 0.62-point reduction in the Generalized Anxiety Disorder-7 scale scores (b=0.62; SE 0.27; P=.005; 95% CI -1.2 to -0.6), whereas a single log unit use of the stress meter was associated with an average of a 0.5-point increase in emotional regulation scores (Five Facet Mindfulness Questionnaire; b=0.5; SE 0.21; P=.03; 95% CI 0.1-0.9).

CONCLUSIONS

This study offers a clearer understanding of the impact of engagement with app features on broader engagement with the health outcomes of interest. This study highlights the importance of comprehensive investigations of engagement during the development of evidence-based mobile apps.

Meaningful engagement: A crossfunctional framework for digital therapeutics

Digital mental health interventions, or digital therapeutics, have the potential to transform the field of mental health. They provide the opportunity for increased accessibility, reduced stigma, and daily integration with patient's lives. However, as the burgeoning field continues to expand, there is a growing concern regarding the level and type of engagement users have with these technologies. Unlike many traditional technology products that have optimized their user experience to maximize the amount of time users spend within the product, such engagement within a digital therapeutic is not sufficient if users are not experiencing an improvement in clinical outcomes. In fact, a primary challenge within digital therapeutics is user engagement. Digital therapeutics are only effective if users sufficiently engage with them and, we argue, only if users meaningfully engage with the product. Therefore, we propose a 4-step framework to assess meaningful engagement within digital therapeutics: (1) Define the measure of value (2) Operationalize meaningful engagement for your digital therapeutic (3) Implement solutions to increase meaningful engagement (4) Iteratively evaluate the solution's impact on meaningful engagement and clinical outcomes. We provide recommendations to the common challenges associated with each step. We specifically emphasize a cross-functional approach to assessing meaningful engagement and use an adolescent-focused example throughout to further highlight developmental considerations one should consider depending on their target users.

Improvements in Depression Outcomes Following a Digital Cognitive Behavioral Therapy Intervention in a Polychronic Population: Retrospective Study

Background

Digital mental health interventions have shown promise in reducing barriers to effective care for depression. Depression and related mental disorders are known to be highly comorbid with common chronic physical conditions, such as obesity and type 2 diabetes. While some research has explored the interaction dynamics of treating populations living with both mental and physical disorders, very little is known about such dynamics in digital care.

Objective

We aimed to examine the effectiveness of a 12-week, therapist-supported, app-based cognitive behavioral therapy program in improving symptoms of depression and anxiety. The studied population included adults with a heavy burden of chronic physical disease, including obesity and type 2 diabetes.

Methods

A total of 1512 participants with at least moderate depression were enrolled. The treatment cohort consisted of 831 (54.96%) participants who completed a follow-up assessment. The program included structured lessons and tools (ie, exercises and practices) and offered one-on-one weekly video counseling sessions with a licensed therapist for 12 weeks and monthly sessions thereafter. The clinically validated 8-item Patient Health Questionnaire (PHQ-8) and the 7-item Generalized Anxiety Disorder scale (GAD-7) were used to assess depression and anxiety, respectively. Linear mixed-effects modeling was employed to examine changes in depression and anxiety over time. Given correlation among various measures of program usage, a composite variable for depth of usage was used to analyze the correlation between usage and changes in depressive symptoms. Body weight changes from baseline were assessed primarily with digitally connected scales.

Results

Out of 831 participants in the treatment cohort, 74.5% (n=619) showed a clinically significant reduction in depressive symptom severity after 12 weeks, where follow-up PHQ-8 scores had shifted downward by at least one diagnostic category. In total, 67.5% (n=561) of the participants showed a reliable improvement in PHQ-8 scores as measured by the reliable change index. There was an average reduction of 5.9 (SD 5.2) points (P<.001) between baseline and follow-up. Greater program usage was correlated with greater likelihood of reliable improvement in depressive symptoms (odds ratio 1.3, 95% CI 1.1-1.5; P=.002). An exploratory analysis of body weight changes with a multilevel, mixed-effect model suggested that reliable improvement in depressive symptoms at follow-up was associated with significantly greater weight loss at 9 months (β=–1.11, P=.002).

Conclusions

The results provide further support that digital interventions can support clinically meaningful improvements in depression. Some form of synergy in treatment of comorbid depression and obesity or diabetes could be studied in future research. The study was limited by postintervention participant attrition as well as the retrospective observational study design.