Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis

Evaluation of the impact of a digital care navigator on increasing patient registration with digital mental health interventions in routine care

Background

Less than half of adults with mental health disorders in the United States receive appropriate or timely care. Digital Mental Health Interventions (DMHIs) have the potential to bridge this gap. However, real-world adoption of DMHIs is impeded by patient and provider-level technological barriers. Care navigators have the potential to address these challenges by providing technical support and enhancing patients' experience with DMHIs.

Objective

This study explores the effect of a digital care navigator (DCN) on patient registration latency and rates of DMHIs implemented as part of a digital-first behavioral health care model integrated within routine care at a large multispecialty group medical practice.

Methods

Data were collected from electronic medical records and DMHI registration data were obtained from the DMHI vendors. Descriptive statistics were used to describe the DCN's efforts to help patients register with their DMHIs. Moods median tests evaluated differences in registration latency and weekly registration rate pre- and post- DCN implementation. Change in registration likelihood as a function of DCN outreach latency was investigated using a Kaplan-Meier plot.

Results

During the first eight months, the DCN made 1306 phone calls to 680 unique patients, successfully connecting with 66 %. DCN implementation also increased the median registration rate from 61.9 % to 76.9 %. Results showed that the expediency by which the DCN outreached patients directly impacted registration rate such that of those who were outreached by the DCN on the day of their referral to a DMHI, 96.86 % registered. This number was reduced to 76.15 % if the DCN reached them 1-day following referral, and 41.39 % 5-days after their referral.

Conclusions

Use of a DCN shows promise for enhancing patient registration rates with DMHIs in routine healthcare settings.

Predictors of adherence to a publicly available self-guided digital mental health intervention

Low adherence to self-guided digital mental health interventions (DMHIs) have raised concerns about their real-world effectiveness. Naturalistic data from self-guided DMHIs are often not available, hindering our ability to assess adherence among real-world users. This study aimed to analyze 3 years of user data from the public launch of an empirically supported 12-session self-guided DMHI, to assess overall program adherence rates and explore predictors of adherence. Data from 984 registered users were analyzed. Results showed that only 14.8% of users completed all 12 modules and 68.6% completed less than half of the modules. Users who were younger, had milder depression, had never seen a mental health provider, and who rejected signing-up for weekly program emails completed significantly more modules. Results add to concerns about the generalizability of controlled research on DMHIs due to lower adherence outside of research trials. This study highlights the potential of user data in identifying key factors that may be related to adherence. By examining adherence patterns among different sub-sets of users, we can pinpoint and focus on individuals who may adhere and benefit more from self-guided programs. Findings could also have implications for guiding intervention personalization for individuals who struggle to complete DMHIs.

Young People's Barriers and Facilitators of Engagement with Web-Based Mental Health Interventions for Anxiety and Depression: A Qualitative Study

BACKGROUND

The prevalence of anxiety and depressive symptoms in young people have increased in many countries around the world. Web-based mental health interventions (or W-MHIs) have the potential to reduce anxiety and depression symptoms for young people. Although W-MHIs have become more widely used by young people since the coronavirus disease 2019 (COVID-19) pandemic, real-world engagement in these W-MHIs has remained low compared with engagement reported in research studies. Moreover, there are limited studies examining factors influencing engagement with W-MHIs in the post-COVID-19 pandemic years.

OBJECTIVE

This study aims to explore barriers and facilitators of engagement with W-MHIs for anxiety and depression among young people.

METHOD

Seventeen semi-structured interviews and one focus group with three participants were conducted online via Zoom between February and March 2023. Participants were young people aged 18-25 years who had self-reported experience of anxiety and/or depression in the past 6 months, lived in Australia, and considered using W-MHIs to manage their anxiety and/or depression symptoms. Inductive thematic analysis was performed to understand the key barriers and facilitators of young people's engagement with W-MHIs.

RESULTS

Both individual- and intervention-related factors influenced young people's engagement with W-MHIs. Facilitators of engagement included personal trust and beliefs in W-MHIs, ability to contact a health professional, programme suitability (e.g., affordability, content aligning with user needs), programme usability (e.g., user interface), and accessibility of the online platform. Barriers included concerns about online security, lack of human interaction and immediate responses from health professionals (if any), and negative experience with mental health programmes. Participants expressed greater willingness to pay if they could contact health professionals during the programme.

CONCLUSION

Better promotion strategies for mental health and W-MHI awareness are needed to increase the perceived importance and priority of mental health interventions among young people. Young people should be involved in the W-MHI co-design to enhance the programme suitability and usability for young people, fostering their engagement with W-MHIs.

Clinical Effects of Asynchronous Provider-Guided Practice Sessions During Blended Care Therapy for Anxiety and Depression: Pragmatic Retrospective Cohort Study

BACKGROUND

Blended care therapy models are intended to increase the efficiency and effectiveness of evidence-based psychotherapy by combining synchronous and asynchronous components of care.

OBJECTIVE

This retrospective cohort study evaluated the clinical effects of synchronous video therapy sessions and asynchronous guided practice session elements on anxiety and depression in a blended care therapy program, with a novel focus on asynchronous provider feedback messages.

METHODS

Participants were adults (N=33,492) with clinical symptoms of anxiety (Generalized Anxiety Disorder 7-item scale [GAD-7] score of ≥8) and depression (Patient Health Questionnaire 9-item scale [PHQ-9] score of ≥10) at intake. Symptom trajectories were evaluated via individual growth curve models. Time-varying covariates evaluated effects of synchronous video session attendance and the presence or absence of each asynchronous guided practice session element occurring within 7 days and 8-14 days prior to each clinical outcome assessment. Guided practice session elements included client digital lesson completion, client digital exercise completion, and feedback messages sent by providers.

RESULTS

Approximately 86.6% (29,012/33,492) of clients met criteria for clinical improvement by end of care (median 6, IQR 4-8 synchronous sessions). Synchronous video session attendance and client digital lesson completion in the past 7 days and in the past 8-14 days were each uniquely and significantly associated with lower GAD-7 scores (video session effects: bsession7=-0.82, bsession8-14=-0.58, P values<.001; digital lesson effects: blesson7=-0.18, blesson8-14=-0.26, P values <.001) and PHQ-9 scores (video session effects: bsession7=-0.89, bsession8-14=-0.67, P values <.001; digital lesson effects: blesson7=-0.12, blesson8-14=-0.30, P values <.001). Client digital exercise completion in the past 8-14 days was significantly associated with lower GAD-7 scores (bexercise8-14=-0.10; P<.001) but exercise completion in the 7 days prior to clinical outcome assessment was not (bexercise7=0.00; P=.89). Exercise completion in the past 7 days was significantly associated with lower PHQ-9 scores (bexercise7=-0.16; P<.001) but exercise completion in the past 8-14 days was not (bexercise8-14=-0.05; P=.09). Provider feedback messaging in the past 7 days and in the past 8-14 days was significantly associated with lower GAD-7 and PHQ-9 scores, respectively (GAD-7: bfeedback7=-0.12, P<.001; bfeedback8-14=-0.07, P=.004; PHQ-9: bfeedback7=-0.15, P<.001; bfeedback8-14=-0.08, P=.01).

CONCLUSIONS

Provider feedback between synchronous therapy sessions provided significant benefit for symptom reduction, beyond the effects of client digital engagement and synchronous video sessions. When guided practice sessions are well integrated into care, blended care therapy provides meaningful improvements upon the traditional, synchronous session-only therapy model. Provider guidance and feedback for clients between synchronous sessions support more efficient and effective mental health care overall.

Leveraging Personal Technologies in the Treatment of Schizophrenia Spectrum Disorders: Scoping Review

BACKGROUND

Digital mental health is a rapidly growing field with an increasing evidence base due to its potential scalability and impacts on access to mental health care. Further, within underfunded service systems, leveraging personal technologies to deliver or support specialized service delivery has garnered attention as a feasible and cost-effective means of improving access. Digital health relevance has also improved as technology ownership in individuals with schizophrenia has improved and is comparable to that of the general population. However, less digital health research has been conducted in groups with schizophrenia spectrum disorders compared to other mental health conditions, and overall feasibility, efficacy, and clinical integration remain largely unknown.

OBJECTIVE

This review aims to describe the available literature investigating the use of personal technologies (ie, phone, computer, tablet, and wearables) to deliver or support specialized care for schizophrenia and examine opportunities and barriers to integrating this technology into care.

METHODS

Given the size of this review, we used scoping review methods. We searched 3 major databases with search teams related to schizophrenia spectrum disorders, various personal technologies, and intervention outcomes related to recovery. We included studies from the full spectrum of methodologies, from development papers to implementation trials. Methods and reporting follow the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

RESULTS

This search resulted in 999 studies, which, through review by at least 2 reviewers, included 92 publications. Included studies were published from 2010 to 2023. Most studies examined multitechnology interventions (40/92, 43%) or smartphone apps (25/92, 27%), followed by SMS text messaging (16/92, 17%) and internet-based interventions (11/92, 12%). No studies used wearable technology on its own to deliver an intervention. Regarding the stage of research in the field, the largest number of publications were pilot studies (32/92, 35%), followed by randomized control trials (RCTs; 20/92, 22%), secondary analyses (16/92, 17%), RCT protocols (16/92, 17%), development papers (5/92, 5%), and nonrandomized or quasi-experimental trials (3/92, 3%). Most studies did not report on safety indices (55/92, 60%) or privacy precautions (64/92, 70%). Included studies tend to report consistent positive user feedback regarding the usability, acceptability, and satisfaction with technology; however, engagement metrics are highly variable and report mixed outcomes. Furthermore, efficacy at both the pilot and RCT levels report mixed findings on primary outcomes.

CONCLUSIONS

Overall, the findings of this review highlight the discrepancy between the high levels of acceptability and usability of these digital interventions, mixed efficacy results, and difficulties with sustained engagement. The discussion highlights common patterns that may underscore this observation in the field; however, as this was a scoping review, a more in-depth systematic review or meta-analysis may be required to better understand the trends outlined in this review.

Opportunities and challenges in leveraging digital technology for mental health system strengthening: a systematic review to inform interventions in the United Arab Emirates

Digital technology offers scalable, real-time interventions for mental health promotion and treatment. This systematic review explores the opportunities and challenges associated with the use of digital technology in mental health, with a focus on informing mental health system strengthening interventions in the United Arab Emirates (UAE). Following PRISMA guidelines, a systematic search of databases was conducted up to August 2023 and identified a total of 8479 citations of which 114 studies were included in the qualitative analysis. The included studies encompass diverse digital interventions, platforms, and modalities used across various mental health conditions. The review identifies feasible, acceptable, and efficacious interventions, ranging from telehealth and mobile apps to virtual reality and machine learning models. Opportunities for improving access to care, reducing patients' transfers, and utilizing real-world interaction data for symptom monitoring are highlighted. However, challenges such as digital exclusion, privacy concerns, and potential service replacement caution policymakers. This study serves as a valuable evidence base for policymakers and mental health stakeholders in the UAE to navigate the integration of digital technology in mental health services effectively.

Brief psychological intervention for suicide prevention based on problem-solving applied in different formats to people over 50 years old: protocol for a randomized controlled trial

BACKGROUND

Suicide is a major public health problem, especially among individuals over 50 years old. Despite the suitability of this life stage for prevention, research on the efficacy of psychological interventions is scarce and methodologically limited, affecting their clinical utility and efficacy. Brief, flexible interventions that can be applied both in-person and remotely are needed. This study aims to evaluate the efficacy of a brief problem-solving-based suicide prevention program applied through various modalities to individuals over 50 years old.

METHODS

A randomized controlled trial will be conducted. A sample of 212 adults aged 50 or older with suicidal ideation will be randomly assigned to a problem-solving-based psychological intervention administered face-to-face (PSPI-P; n = 53), by telephone multiconference (PSPI-M; n = 53), via a smartphone app (PSPI-A; n = 53), or to a usual care control group (UCCG; n = 53). The intervention will be delivered in 7 sessions or modules of 90 min each. Blind trained evaluators will conduct assessments at pre-intervention, post-intervention, and follow-ups at 3, 6, and 12 months. The primary outcome will be suicidal ideation evaluated using the Suicidal Ideation Scale (SSI) and the Columbia Suicide Severity Rating Scale (C-SSRS). Secondary outcomes will include hopelessness, anxiety and depression symptoms, reasons for living, impulsivity, problem-solving skills, social support, anger syndrome, gratitude, personality, dropouts, treatment adherence, and satisfaction with the intervention.

DISCUSSION

This study will provide evidence of the efficacy of a brief problem-solving-based intervention for suicide prevention in individuals over 50 years old, administered face-to-face, by telephone multiconference, and via a smartphone app. If results are favorable, it will indicate that an effective, accessible, clinically and socially useful suicide prevention intervention has been developed for affected individuals, families, and communities.

TRIAL REGISTRATION

ClinicalTrials.gov NCT06338904. Registered April 1, 2024.

Engagement challenges in digital mental health programs: hybrid approaches and user retention of an online self-knowledge journey in Brazil

Digital mental health interventions (DMHIs) have surged in popularity over the last few years. However, adherence to self-guided interventions remains a major hurdle to overcome. The current study utilized a phased implementation design, incorporating diverse samples and contexts to delve into the engagement challenges faced by a recently launched online mental health platform in Brazil with self-evaluation forms. Employing an iterative mixed-methods approach, including focus groups, online surveys, and think-aloud protocols, the research aims to evaluate user satisfaction, identify barriers to adherence, and explore potential hybrid solutions. Engagement in the platform was evaluated by descriptive statistics of the number of instruments completed, and qualitative interviews that were interpreted thematically. In the fully self-guided mode, 2,145 individuals registered, but a substantial majority (88.9%) engaged with the platform for only 1 day, and merely 3.3% completed all activities. In another sample of 50 participants were given a choice between online-only or a hybrid experience with face-to-face meetings. 40% of individuals from the hybrid group completed all activities, compared to 8% in the online-only format. Time constraints emerged as a significant barrier to engagement, with suggested improvements including app development, periodic reminders, and meetings with healthcare professionals. While the study identified weaknesses in the number and length of instruments, personalized results stood out as a major strength. Overall, the findings indicate high satisfaction with the mental health platform but underscore the need for improvements, emphasizing the promise of personalized mental health information and acknowledging persistent barriers in a digital-only setting.

The Role of Virtual Reality and Artificial Intelligence in Cognitive Pain Therapy: A Narrative Review

PURPOSE OF REVIEW

This review investigates the roles of artificial intelligence (AI) and virtual reality (VR) in enhancing cognitive pain therapy for chronic pain management. The work assesses current research, outlines benefits and limitations and examines their potential integration into existing pain management methods.

RECENT FINDINGS

Advances in VR have shown promise in chronic pain management through immersive cognitive therapy exercises, with evidence supporting VR's effectiveness in symptom reduction. AI's personalization of treatment plans and its support for mental health through AI-driven avatars are emerging trends. The integration of AI in hybrid programs indicates a future with real-time adaptive technology tailored to individual needs in chronic pain management. Incorporating AI and VR into chronic pain cognitive therapy represents a promising approach to enhance management by leveraging VR's immersive experiences and AI's personalized tactics, aiming to improve patient engagement and outcomes. Nonetheless, further empirical studies are needed to standardized methodologies, compare these technologies to traditional therapies and fully realize their clinical potential.

Evaluation of Digital Mental Health Technologies in the United States: Systematic Literature Review and Framework Synthesis

BACKGROUND

Digital mental health technologies (DMHTs) have the potential to enhance mental health care delivery. However, there is little information on how DMHTs are evaluated and what factors influence their use.

OBJECTIVE

A systematic literature review was conducted to understand how DMHTs are valued in the United States from user, payer, and employer perspectives.

METHODS

Articles published after 2017 were identified from MEDLINE, Embase, PsycINFO, Cochrane Library, the Health Technology Assessment Database, and digital and mental health congresses. Each article was evaluated by 2 independent reviewers to identify US studies reporting on factors considered in the evaluation of DMHTs targeting mental health, Alzheimer disease, epilepsy, autism spectrum disorder, or attention-deficit/hyperactivity disorder. Study quality was assessed using the Critical Appraisal Skills Program Qualitative and Cohort Studies Checklists. Studies were coded and indexed using the American Psychiatric Association's Mental Health App Evaluation Framework to extract and synthesize relevant information, and novel themes were added iteratively as identified.

RESULTS

Of the 4353 articles screened, data from 26 unique studies from patient, caregiver, and health care provider perspectives were included. Engagement style was the most reported theme (23/26, 88%), with users valuing DMHT usability, particularly alignment with therapeutic goals through features including anxiety management tools. Key barriers to DMHT use included limited internet access, poor technical literacy, and privacy concerns. Novel findings included the discreetness of DMHTs to avoid stigma.

CONCLUSIONS

Usability, cost, accessibility, technical considerations, and alignment with therapeutic goals are important to users, although DMHT valuation varies across individuals. DMHT apps should be developed and selected with specific user needs in mind.

Skill Enactment Among University Students Using a Brief Video-Based Mental Health Intervention: Mixed Methods Study Within a Randomized Controlled Trial

BACKGROUND

Mental health problems are common among university students, yet many students do not seek professional help. Digital mental health interventions can increase students' access to support and have been shown to be effective in preventing and treating mental health problems. However, little is known about the extent to which students implement therapeutic skills from these programs in everyday life (ie, skill enactment) or about the impact of skill enactment on outcomes.

OBJECTIVE

This study aims to assess the effects of a low-intensity video-based intervention, Uni Virtual Clinic Lite (UVC-Lite), in improving skill enactment relative to an attention-control program (primary aim) and examine whether skill enactment influences symptoms of depression and anxiety (secondary aim). The study also qualitatively explored participants' experiences of, and motivations for, engaging with the therapeutic techniques.

METHODS

We analyzed data from a randomized controlled trial testing the effectiveness of UVC-Lite for symptoms of depression and anxiety among university students with mild to moderate levels of psychological distress. Participants were recruited from universities across Australia and randomly assigned to 6 weeks of self-guided use of UVC-Lite (243/487, 49.9%) or an attention-control program (244/487, 50.1%). Quantitative data on skill enactment, depression, and anxiety were collected through baseline, postintervention, and 3- and 6-month follow-up surveys. Qualitative data were obtained from 29 intervention-group participants through open-ended questions during postintervention surveys (n=17, 59%) and semistructured interviews (n=12, 41%) after the intervention period concluded.

RESULTS

Mixed model repeated measures ANOVA demonstrated that the intervention did not significantly improve skill enactment (F3,215.36=0.50; P=.68). Skill enactment was also not found to influence change in symptoms of depression (F3,241.10=1.69; P=.17) or anxiety (F3,233.71=1.11; P=.35). However, higher levels of skill enactment were associated with lower symptom levels among both intervention and control group participants across time points (depression: F1,541.87=134.61; P<.001; anxiety: F1,535.11=73.08; P<.001). Inductive content analysis confirmed low levels of skill enactment among intervention group participants. Participants were motivated to use techniques and skills that were perceived to be personally relevant, easily integrated into daily life, and that were novel or had worked for them in the past.

CONCLUSIONS

The intervention did not improve skill enactment or mental health among students with mild to moderate psychological distress. Low adherence impacted our ability to draw robust conclusions regarding the intervention's impact on outcomes. Factors influencing skill enactment differed across individuals, suggesting that it may be necessary to tailor therapeutic skills and engagement strategies to the individual user. Theoretically informed research involving collaboration with end users is needed to understand the processes underlying skill enactment in digital mental health interventions.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12621000375853; https://tinyurl.com/7b9ar54r.

The effects of language learning on cognitive functioning and psychosocial well-being in cognitively healthy older adults: A semi-blind randomized controlled trial

This study investigated the impact of language learning in comparison to other complex learning activities on cognitive functioning and psychosocial well-being in cognitively healthy, community-dwelling older adults. In a randomized controlled trial, 43 Dutch functionally monolinguals aged 65-78 completed a three-month English course (n = 15), music training (n = 13), or a lecture series (n = 15). Cognitive functioning (global cognition, cognitive flexibility, episodic memory, working memory, verbal fluency, and attention) and psychosocial well-being were assessed before and immediately after the intervention, and at a four-month follow-up. The language learners significantly improved on episodic memory and cognitive flexibility. However, the magnitude of cognitive change did not significantly differ between the language learning and music training conditions, except for a larger positive change in cognitive flexibility for the language learners from pretest to follow-up. Our results suggest that language learning in later life can improve some cognitive functions and fluency in the additional language, but that its unique effects seem limited.

Skill enactment and knowledge acquisition among community users of digital mental health interventions: qualitative study with thematic analysis

BACKGROUND

The acquisition of knowledge and use of skills from digital mental health interventions (DMHIs) are considered important for effectiveness. However, our understanding of user experiences implementing skills learned from these interventions is limited, particularly outside of research trials. This qualitative study aimed to investigate how community users learn and apply knowledge and skills from DMHIs based on cognitive behavioural therapy (CBT) in daily life. The study also examined factors influencing the selection and use of skills and explored perceived changes in mental health resulting from the intervention.

METHODS

Thirteen adults aged 26 to 66 years (10 females) were recruited using social media advertising and participated in semi-structured interviews by telephone or videoconference. All participants were living in Australia and had used a digital CBT program within the past 3 months. Interviews lasted on average 45 min. Transcripts were analysed using theoretical thematic analysis.

RESULTS

Participants demonstrated high levels of program engagement. Findings were organised into three topics with six major themes. Participants reported that their chosen intervention reinforced existing knowledge and fostered new skills and insights (Topic 1, Theme 1: knowledge consolidation). Most described actively applying skills (Topic 1, Theme 2: active approach to skill enactment), although the extent of learning and range of skills enacted varied across participants. Influences on skill selection included the perceived relevance of intervention strategies to the user's needs and personal characteristics (Topic 2, Theme 1: relevance of intervention strategies), as well as the perceived or experienced effectiveness of those strategies (Topic 2, Theme 2: perceived and experienced benefit). Challenges to ongoing skill enactment included time scarcity, prioritisation difficulties, and lack of motivation (Topic 2, Theme 3: navigating time constraints and low motivation). Improvements in mental health were generally modest and attributed mainly to participants' proactive efforts (Topic 3, Theme 1: perceived changes).

CONCLUSIONS

DMHIs may reinforce existing understanding of psychotherapeutic strategies, offer new knowledge, and encourage the application of skills in everyday life among community users who actively engage with these interventions. Future research should prioritise personalising DMHIs and investigating methods to optimise the acquisition, retention, and sustained application of knowledge and skills.

Acceptability and Engagement of a Smartphone-Delivered Interpretation Bias Intervention in a Sample of Black and Latinx Adults: Open Trial

BACKGROUND

Access to evidence-based interventions is urgently required, especially for individuals of minoritized identities who experience unique barriers to mental health care. Digital mental health interventions have the potential to increase accessibility. Previous pilot studies testing HabitWorks, a smartphone app providing an interpretation bias intervention, have found strong engagement and adherence for HabitWorks; however, previous trials' samples consisted of predominantly non-Hispanic, White individuals.

OBJECTIVE

This study conducted an open trial of HabitWorks in a community sample of adults who identified as Black, Hispanic or Latinx, or both. This study aims to test safety, acceptability, and engagement with the HabitWorks app for Black and Latinx adults.

METHODS

Black, Hispanic or Latinx adults (mean age 32.83, SD 11.06 y; 22/31, 71% women) who endorsed symptoms of anxiety or depression were asked to complete interpretation modification exercises via HabitWorks 3 times per week for 1 month. Interpretation bias and anxiety and depression symptoms were assessed at baseline and posttreatment assessments. Participants completed qualitative interviews to assess overall perceptions of HabitWorks.

RESULTS

Of the 31 participants that downloaded the app, 27 (87%) used HabitWorks all 4 weeks. On average, participants completed 15.74 (SD 7.43) exercises out of the 12 prescribed, demonstrating high engagement. Acceptability ratings met all a priori benchmarks except for relevancy. Qualitative interviews also demonstrated high acceptability and few negative experiences. Significant improvements were found in interpretation style (t30=2.29; P<.001), with a large effect size (Cohen d=1.53); anxiety symptoms (t30=2.29; P=.03), with a small effect size (Cohen d=0.41); and depression symptoms (t30=3.065; P=.005), with a medium effect size (Cohen d=0.55).

CONCLUSIONS

This study adds to the literature evaluating digital mental health interventions in Black and Latinx adults. Preliminary results further support a future controlled trial testing the effectiveness of HabitWorks as an intervention.

The Impact of User Engagement With Exposure Components on Posttraumatic Stress Symptoms in an mHealth Mobile App: Secondary Analysis of a Randomized Controlled Trial

Background

Mobile mental health apps are a cost-effective option for managing mental health problems, such as posttraumatic stress disorder (PTSD). The efficacy of mobile health (mHealth) apps depends on engagement with the app, but few studies have examined how users engage with different features of mHealth apps for PTSD.

Objective

This study aims to examine the relationship between app engagement indices and PTSD symptom reduction using data from an unblinded pilot randomized controlled trial of "Renew" (Vertical Design), an exposure-based app for PTSD with and without coaching support. Because exposure is an effective approach for treating PTSD, we expected that engagement with exposure activities would be positively related to symptom reduction, over and above overall app usage.

Methods

Participants were veterans (N=69) with clinically significant PTSD symptoms who were recruited online using Facebook advertisements and invited to use the Renew app as often as they wanted over a 6-week period. Participants completed screening and assessments online but provided informed consent, toured the app, and completed feedback interviews via telephone. We assessed users' self-reported PTSD symptoms before and after a 6-week intervention period and collected app usage data using a research-instrumented dashboard. To examine overall app engagement, we used data on the total time spent in the app, the number of log-in days, and the number of points that the user gained in the app. To examine engagement with exposure components, we used data on total time spent completing exposure activities (both in vivo and imaginal), the number of in vivo exposure activities completed, and the number of characters written in response to imaginal exposure prompts. We used hierarchical regression analyses to test the effect of engagement indices on change in PTSD symptoms.

Results

Usage varied widely. Participants spent an average of 166.09 (SD 156.52) minutes using Renew, over an average of 14.7 (SD 10.71) mean log-in days. Engagement with the exposure components of the app was positively associated with PTSD symptom reduction (F6,62=2.31; P=.04). Moreover, this relationship remained significant when controlling for overall engagement with the app (ΔF3,62=4.42; P=.007). The number of characters written during imaginal exposure (β=.37; P=.009) and the amount of time spent completing exposure activities (β=.36; P=.03) were significant contributors to the model.

Conclusions

To our knowledge, this is the first study to show a relationship between symptom improvement and engagement with the active therapeutic components of an mHealth app (ie, exposure) for PTSD. This relationship held when controlling for overall app use, which suggests that it was engagement with exposure, specifically, that was associated with symptom change. Future work to identify ways of promoting greater engagement with self-guided exposure may help improve the effectiveness of mHealth apps for PTSD.

Comparative effectiveness of remote digital gamified and group CBT skills training interventions for anxiety and depression among college students: Results of a three-arm randomised controlled trial

Digital interventions can enhance access to healthcare in under-resourced settings. However, guided digital interventions may be costly for low- and middle-income countries, despite their effectiveness. In this randomised control trial, we evaluated the effectiveness of two digital interventions designed to address this issue: (1) a Cognitive Behavioral Therapy Skills Training (CST) intervention that increased scalability by using remote online group administration; and (2) the SuperBetter gamified self-guided CBT skills training app, which uses other participants rather than paid staff as guides. The study was implemented among anxious and/or depressed South African undergraduates (n = 371) randomised with equal allocation to Remote Group CST, SuperBetter, or a MoodFlow mood monitoring control. Symptoms were assessed with the Generalized Anxiety Disorder-7 (GAD-7) and the Patient Health Questionnaire-9 (PHQ-9). Intention-to-treat analysis found effect sizes at the high end of prior digital intervention trials, including significantly higher adjusted risk differences (ARD; primary outcome) in joint anxiety/depression remission at 3-months and 6-months for Remote Group CST (ARD = 23.3-18.9%, p = 0.001-0.035) and SuperBetter (ARD = 12.7-22.2%, p = 0.047-0.006) than MoodFlow and mean combined PHQ-9/GAD-7 scores (secondary outcome) significantly lower for Remote Group CST and SuperBetter than MoodFlow. These results illustrate how innovative delivery methods can increase the scalability of standard one-on-one guided digital interventions. PREREGISTRATION INTERNATIONAL STANDARD RANDOMISED CONTROLLED TRIAL NUMBER (ISRTCN) SUBMISSION #: 47,089,643.

Effectiveness of a chatbot in improving the mental wellbeing of health workers in Malawi during the COVID-19 pandemic: A randomized, controlled trial

We conducted a randomized, controlled trial (RCT) to investigate our hypothesis that the interactive chatbot, Vitalk, is more effective in improving mental wellbeing and resilience outcomes of health workers in Malawi than the passive use of Internet resources. For our 2-arm, 8-week, parallel RCT (ISRCTN Registry: trial ID ISRCTN16378480), we recruited participants from 8 professional cadres from public and private healthcare facilities. The treatment arm used Vitalk; the control arm received links to Internet resources. The research team was blinded to the assignment. Of 1,584 participants randomly assigned to the treatment and control arms, 215 participants in the treatment and 296 in the control group completed baseline and endline anxiety assessments. Six assessments provided outcome measures for: anxiety (GAD-7); depression (PHQ-9); burnout (OLBI); loneliness (ULCA); resilience (RS-14); and resilience-building activities. We analyzed effectiveness using mixed-effects linear models, effect size estimates, and reliable change in risk levels. Results support our hypothesis. Difference-in-differences estimators showed that Vitalk reduced: depression (-0.68 [95% CI -1.15 to -0.21]); anxiety (-0.44 [95% CI -0.88 to 0.01]); and burnout (-0.58 [95% CI -1.32 to 0.15]). Changes in resilience (1.47 [95% CI 0.05 to 2.88]) and resilience-building activities (1.22 [95% CI 0.56 to 1.87]) were significantly greater in the treatment group. Our RCT produced a medium effect size for the treatment and a small effect size for the control group. This is the first RCT of a mental health app for healthcare workers during the COVID-19 pandemic in Southern Africa combining multiple mental wellbeing outcomes and measuring resilience and resilience-building activities. A substantial number of participants could have benefited from mental health support (1 in 8 reported anxiety and depression; 3 in 4 suffered burnout; and 1 in 4 had low resilience). Such help is not readily available in Malawi. Vitalk has the potential to fill this gap.

Blended care in psychosis - A systematic review

Blended-care, a psychosocial intervention combining traditional, face-to-face therapy with digital mental health tools, has shown potential for improving therapeutic processes, fostering patient engagement, and augmenting clinical outcomes. This systematic review aimed to evaluate the development and effectiveness of blended-care interventions tailored for adults diagnosed with schizophrenia-spectrum disorders or other conditions with psychotic features. Our search strategy spanned three electronic databases (PsycINFO, Web of Science, and PubMed) in accordance with the reporting guidelines outlined by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses. We identified 11 papers, each examining the use of one of eight distinct blended-care interventions. Significantly, the majority of these papers (10/11, 91 %) examined these interventions exclusively within controlled research environments, demonstrating both acceptability and favourable impacts on symptomatology and recovery (e.g., 0.22 ≤ Cohen's ds ≤ 1.00). Only one intervention was examined in research settings and real-world conditions, and the shift resulted in low real-world uptake (e.g., only 50 % of practitioners were able to engage at least one of their clients with the intervention) and an inability to reproduce positive changes in clinical outcomes. Additional research is needed to determine the viability of successfully developing and implementing blended-care interventions for psychosis in real-world conditions. An exploration of the developmental processes that could facilitate the transition from research settings to routine clinical practice is vital.

The role of sociodemographic factors on the acceptability of digital mental health care: A scoping review protocol

INTRODUCTION

Many individuals experiencing mental health complications face barriers when attempting to access services. To bridge this care gap, digital mental health innovations (DMHI) have proven to be valuable additions to in-person care by enhancing access to care. An important aspect to consider when evaluating the utility of DMHI is perceived acceptability. However, it is unclear whether diverse sociodemographic groups differ in their degree of perceived acceptability of DMHI.

OBJECTIVE

This scoping review aims to synthesize evidence on the role of sociodemographic factors (e.g., age, gender) in the perceived acceptability of DMHI among individuals seeking mental health care.

METHODS

Guided by the JBI Manual of Evidence Synthesis, chapter on Scoping Review, a search strategy developed according to the PCC framework will be implemented in MEDLINE and then adapted to four electronic databases (i.e., CINAHL, MEDLINE, PsycINFO, and EMBASE). The study selection strategy will be piloted by two reviewers on subsets of 30 articles until agreement among reviewers reaches 90%, after which one reviewer will complete the remaining screening of titles and abstracts. The full-text screening, data extraction strategy, and charting tool will be completed by one reviewer and then validated by a second member of the team. Main findings will be presented using tables and figures.

EXPECTED CONTRIBUTIONS

This scoping review will examine the extent to which sociodemographic factors have been considered in the digital mental health literature. Also, the proposed review may help determine whether certain populations have been associated with a lower level of acceptability within the context of digital mental health care. This investigation aims to favor equitable access to DMHI among diverse populations.

Individuals' attitudes toward digital mental health apps and implications for adoption in Portugal: web-based survey

BACKGROUND

The literature is consensual regarding the academic community exhibiting higher levels of mental disorder prevalence than the general population. The potential of digital mental health apps for improving access to resources to cope with these issues is ample. However, studies have yet to be performed in Portugal on individuals' attitudes and perceptions toward digital mental health applications or their preferences and decision drivers on obtaining mental health care, self-assessment, or treatment.

OBJECTIVE

This study aims to understand the determinants of digital mental health applications use in the Portuguese academic community of Porto, along with potential adoption barriers and enablers.

METHODS

A cross-sectional, web-based survey was delivered via dynamic email to the University of Porto's academic community. Data collection occurred between September 20 and October 20, 2022. We used structural equation modeling to build three models, replicating a peer-reviewed and published study and producing a newly full mediation model shaped by the collected data. We tested the relationships between use of digital mental health apps and perceived stress, perceived need to seek help for mental health, perceived stigma, past use of mental health services, privacy concerns, and social influence.

RESULTS

Of the 539 participants, 169 (31.4%) reported having used digital mental health apps. Perceived stress and a latent variable, comprising perceptions of mental health problems and coping strategies, were positively associated with mental health app use, while privacy concerns regarding one's information being accessible to others were negatively associated. Perceived stigma, need to seek help, and close relationships did not have a statistically significant direct effect.

CONCLUSIONS

These findings can inform product and policy development of new, better-targeted digital mental health app interventions, with implications for researchers and academia, industry, and policymakers. Our study concludes that, to maximize adherence to these apps, they should have low to no financial charges, demonstrate evidence of their helpfulness and focus on the timely delivery of care. We also conclude that to foster digital mental health app use, there is a need to improve mental health literacy, namely regarding self-awareness of one's conditions, acceptable stress levels, and overall behavior towards mental health.

TRIAL REGISTRATION

RR2-10.2196/41040.

Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Mental Health

Digital mental health (DMH) interventions, such as text-message-based lessons and activities, offer immense potential for accessible mental health support. While these interventions can be effective, real-world experimental testing can further enhance their design and impact. Adaptive experimentation, utilizing algorithms like Thompson Sampling for (contextual) multi-armed bandit (MAB) problems, can lead to continuous improvement and personalization. However, it remains unclear when these algorithms can simultaneously increase user experience rewards and facilitate appropriate data collection for social-behavioral scientists to analyze with sufficient statistical confidence. Although a growing body of research addresses the practical and statistical aspects of MAB and other adaptive algorithms, further exploration is needed to assess their impact across diverse real-world contexts. This paper presents a software system developed over two years that allows text-messaging intervention components to be adapted using bandit and other algorithms while collecting data for side-by-side comparison with traditional uniform random non-adaptive experiments. We evaluate the system by deploying a text-message-based DMH intervention to 1100 users, recruited through a large mental health non-profit organization, and share the path forward for deploying this system at scale. This system not only enables applications in mental health but could also serve as a model testbed for adaptive experimentation algorithms in other domains.

The relation between usage of an eHealth intervention for stress urinary incontinence and treatment outcomes: an observational study

BACKGROUND

Stress urinary incontinence (SUI), though a prevalent condition among women, is undertreated in primary care. EHealth with pelvic floor muscle training is an evidence-based alternative to care-as-usual. It is unknown, however, how eHealth usage is related to treatment outcome, and this knowledge is required for general practitioners to implement eHealth in their practice. This study examines the relation between usage of eHealth for SUI and treatment outcomes by examining log data. Baseline factors were also explored for associations with treatment success.

METHOD

In this pre-post study, women with SUI participated in "Baasoverjeblaas.nl", a web-based intervention translated from the Swedish internet intervention "Tät®-treatment of stress urinary incontinence". Usage was based on log data and divided into three user groups (low, intermediate and high). Online questionnaires were sent before, after treatment and at six-months follow-up. The relation between usage and the primary outcome - treatment success (PGI-) - was studied with a binomial logistic regression analysis. Changes in the secondary outcomes - symptom severity (ICIQ-UI SF) and quality of life (ICIQ-LUTSqol) - were studied per user group with linear mixed model analysis.

RESULTS

Included were 515 users with a mean age of 50.5 years (12.0 SD). The majority were low users (n = 295, 57.3%). Treatment success (PGI-I) was reached by one in four women and was more likely in high and intermediate users than in low users (OR 13.2, 95% CI 6.1-28.5, p < 0.001 and OR 2.92, 95% CI 1.35-6.34, p = 0.007, respectively). Symptom severity decreased and quality of life improved significantly over time, especially among high users. The women's expected ability to train their pelvic floor muscles and the frequency of pelvic floor muscle exercises at baseline were associated with treatment success.

CONCLUSION

This study shows that usage of eHealth for SUI is related to all treatment outcomes. High users are more likely to have treatment success. Treatment success is more likely in women with higher expectations and pelvic floor muscle training at baseline. These findings indicate that general practitioners can select patients that would be more likely to benefit from eHealth treatment, and they can enhance treatment effect by stimulating eHealth usage.

TRIAL REGISTRATION

Landelijk Trial Register NL6570;  https://onderzoekmetmensen.nl/nl/trial/25463 .

Layperson-Supported, Web-Delivered Cognitive Behavioral Therapy for Depression in Older Adults: Randomized Controlled Trial

BACKGROUND

Depression is the most prevalent mental health condition in older adults. However, not all evidence-based treatments are easily accessible. Web-delivered cognitive behavioral therapy (wCBT) facilitated by laypersons is a viable treatment alternative.

OBJECTIVE

This randomized controlled trial aims to evaluate the efficacy of a novel wCBT program, Empower@Home, supported by trained lay coaches, against a waitlist attention control. Empower@Home is among the very few existing wCBT programs specifically designed for older adults. The primary objective was to assess the efficacy of the intervention compared with attention control. The secondary objective was to evaluate the program's impact on secondary psychosocial outcomes and explore potential change mechanisms.

METHODS

Older adults (N=70) were recruited via web-based research registries, social media advertisements, and community agency referrals and randomly assigned to either the intervention or control group in a 1:1 allocation ratio. The intervention group received access to Empower@Home, which included 9 web-delivered self-help lessons and weekly telephone coaching sessions by a trained layperson over 10 weeks. The control group received weekly friendly phone calls and depressive symptom monitoring. The primary clinical outcome was the severity of depressive symptoms assessed using the Patient Health Questionnaire-9. The secondary clinical outcomes included anxiety, anger, social isolation, insomnia, pain intensity, and quality of life. Linear mixed modeling was used to determine the treatment effects on depression, and 2-tailed t tests were used to assess within-group changes and between-group differences.

RESULTS

Most participants in the intervention group completed all 9 sessions (31/35, 89%). The usability and acceptability ratings were excellent. The intervention group had a large within-group change in depressive symptoms (Cohen d=1.22; P<.001), whereas the attention control group experienced a medium change (Cohen d=0.57; P<.001). The between-group effect size was significant, favoring the intervention group over the control group (Cohen d=0.72; P<.001). In the linear mixed model, the group-by-time interaction was statistically significant (b=-0.68, 95% CI -1.00 to -0.35; P<.001). The treatment effects were mediated by improvements in cognitive behavioral therapy skills acquisition; behavioral activation; and satisfaction with the basic psychological needs of autonomy, competence, and relatedness. Furthermore, the intervention group showed significant within-group improvements in secondary psychosocial outcomes, including anxiety (P=.001), anger (P<.001), social isolation (P=.02), insomnia (P=.007), and pain (P=.03). By contrast, the control group did not experience significant changes in these outcome domains. However, the between-group differences in secondary outcomes were not statistically significant, owing to the small sample size.

CONCLUSIONS

Empower@Home, a wCBT program supported by lay coaches, was more efficacious in reducing depressive symptoms than friendly telephone calls and depression symptom monitoring. Future studies should examine the effectiveness of the intervention in community and practice settings using nonclinician staff already present in these real-world settings as coaches.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05593276; https://clinicaltrials.gov/ct2/show/NCT05593276.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/44210.

Measuring digital intervention user experience with a novel ecological momentary assessment (EMA) method, CORTO

Digital interventions often suffer from low usage, which may reflect insufficient attention to user experience. Moreover, the existing evaluation methods have limited applicability in the remote study of user experience of complex interventions that have expansive content and that are used over an extensive period of time. To alleviate these challenges, we describe here a novel qualitative Ecological Momentary Assessment (EMA) method: the CORTO method (Contextual, One-item, Repeated, Timely, Open-ended). We used it to gather digital intervention user experience data from Finnish adults (n = 184) who lived with interview-confirmed major depressive disorder (MDD) and took part in a randomized controlled trial (RCT) that studied the efficacy of a novel 12-week game-based digital intervention for depression. A second dataset on user experience was gathered with retrospective interviews (n = 22). We inductively coded the CORTO method and retrospective interview data, which led to four user experience categories: (1) contextual use, (2) interaction-elicited emotional experience, (3) usability, and (4) technical issues. Then, we used the created user experience categories and Template Analysis to analyze both datasets together, and reported the results qualitatively. Finally, we compared the two datasets with each other. We found that the data generated with the CORTO method offered more insights into usability and technical categories than the interview data that particularly illustrated the contextual use. The emotional valence of the interview data was more positive compared with the CORTO data. Both the CORTO and interview data detected 55 % of the micro-level categories; 20 % of micro-level categories were only detected by the CORTO data and 25 % only by the interview data. We found that the during-intervention user experience measurement with the CORTO method can provide intervention-specific insights, and thereby further the iterative user-centered intervention development. Overall, these findings highlight the impact of evaluation methods on the categories and qualities of insights acquired in intervention research.

Examination of Brief Parent-Based Interventions to Reduce Drinking Outcomes on a Nationally Representative Sample of Teenagers

PURPOSE

Examine brief parent interventions (PBIs) on a nationally representative sample of teenagers (ages 15-18 years) to change drinking, teens declining (i.e., saying no) to ride with impaired drivers, and increase parent communication about alcohol.

METHODS

Parent-teen dyads (N = 1,176) recruited from a web panel completed online surveys at baseline (T1), 30 days (T2), 6 months (T3), and 12 months (T4) post baseline (Psex: 54% female; Tsex: 49% female; Tage: M = 16.33, SD = 1.09; 76% White, 27% Hispanic). Dyads were randomly assigned to: Active Control (n = 390, 33.1%), Surgeon General (SG; n = 393, 33.4%), or Mothers Against Drunk Driving PBI (MADD; n = 393, 33.4%). Aim 1 was an intent-to-treat analysis of the efficacy comparing the Active Control, SG, and MADD conditions on typical weekend drinking and declining riding with an impaired driver from T1 to T4. Aim 2 examined differences in parent communication at T2 by different levels of engagement. Aim 3 examined differences in efficacy comparing the SG and MADD engagement subgroups at T4. Aims were evaluated by comparing means using Tukey's test to reduce experiment-wise error.

RESULTS

Aim 1 revealed no differences in drinking, but participants in the SG and MADD conditions were able to maintain increases in declining riding with impaired drivers. Aim 2 revealed highly engaged participants reported higher parent communication than moderate and low participants. Aim 3 revealed some highly engaged participants reported less drinking and more declining riding with impaired drivers.

DISCUSSION

PBIs may be successful in helping teens decline rides with impaired drivers. Highly engaged parents reported more positive program outcomes than moderate and low engaged parents.

Determinants and outcome correlates of engagement with a mobile mental health intervention for depression and anxiety in middle-aged and older adults

OBJECTIVES

To examine baseline factors (i.e., age, gender, mobile device proficiency, sensory impairment) associated with app engagement in a 12-week mental health app intervention and to explore whether app engagement predicts changes in depression and anxiety symptoms among middle-aged and older adults.

METHOD

Mobile device proficiency, sensory impairment, depression, and anxiety symptoms were measured using questionnaires. App engagement was defined by metrics characterizing the core intervention features (i.e., messages sent to therapist, mindfulness meditation minutes, action tasks completed). Multiple regressions and multilevel models were conducted.

RESULTS

Forty-nine participants (M age = 57.40, SD = 11.09 years) enrolled. Women (β = .35, p < .05) and participants with less sensory impairment completed more action tasks (β = -.40, p < .05). Depressive and anxiety symptoms measured within the app declined significantly across treatment. Clinical significant improvements were observed for depression in 48.9% and for anxiety in 40% of participants. App engagement metrics were not predictive of depression or anxiety symptoms, either incrementally in time-lagged models or cumulatively in hierarchical linear regression analyses.

CONCLUSION

App engagement is multifaceted; participants engaged differently by gender and ability. Participation in this digital mental health intervention reduced depression and anxiety symptoms, but these findings should be interpreted with caution as the study did not include a control condition. Our findings underscore the importance of considering individual factors that may influence use of a digital mental health intervention.

The Effect of Digital Mental Health Literacy Interventions on Mental Health: Systematic Review and Meta-Analysis

BACKGROUND

Accelerated by technological advancements and the recent global pandemic, there is burgeoning interest in digital mental health literacy (DMHL) interventions that can positively affect mental health. However, existing work remains inconclusive regarding the effectiveness of DMHL interventions.

OBJECTIVE

This systematic review and meta-analysis investigated the components and modes of DMHL interventions, their moderating factors, and their long-term impacts on mental health literacy and mental health.

METHODS

We used a random-effects model to conduct meta-analyses and meta-regressions on moderating effects of DMHL interventions on mental health.

RESULTS

Using 144 interventions with 206 effect sizes, we found a moderate effect of DMHL interventions in enhancing distal mental health outcomes (standardized mean difference=0.42, 95% CI -0.10 to 0.73; P<.001) and a large effect in increasing proximal mental health literacy outcomes (standardized mean difference=0.65, 95% CI 0.59-0.74; P<.001). Uptake of DMHL interventions was comparable with that of control conditions, and uptake of DMHL interventions did not moderate the effects on both proximal mental health literacy outcomes and distal mental health outcomes. DMHL interventions were as effective as face-to-face interventions and did not differ by platform type or dosage. DMHL plus interventions (DMHL psychoeducation coupled with other active treatment) produced large effects in bolstering mental health, were more effective than DMHL only interventions (self-help DMHL psychoeducation), and were comparable with non-DMHL interventions (treatment as usual). DMHL interventions demonstrated positive effects on mental health that were sustained over follow-up assessments and were most effective in enhancing the mental health of emerging and older adults.

CONCLUSIONS

For theory building, our review and meta-analysis found that DMHL interventions are as effective as face-to-face interventions. DMHL interventions confer optimal effects on mental health when DMHL psychoeducation is combined with informal, nonprofessional active treatment components such as skills training and peer support, which demonstrate comparable effectiveness with that of treatment as usual (client-professional interactions and therapies). These effects, which did not differ by platform type or dosage, were sustained over time. Additionally, most DMHL interventions are found in Western cultural contexts, especially in high-income countries (Global North) such as Australia, the United States, and the United Kingdom, and limited research is conducted in low-income countries in Asia and in South American and African countries. Most of the DMHL studies did not report information on the racial or ethnic makeup of the samples. Future work on DMHL interventions that target racial or ethnic minority groups, particularly the design, adoption, and evaluation of the effects of culturally adaptive DMHL interventions on uptake and mental health functioning, is needed. Such evidence can drive the adoption and implementation of DMHL interventions at scale, which represents a key foundation for practice-changing impact in the provision of mental health resources for individuals and the community.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42023363995; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42023363995.

Development of a digital intervention for psychedelic preparation (DIPP)

Psychedelic substances induce profound alterations in consciousness. Careful preparation is therefore essential to limit adverse reactions, enhance therapeutic benefits, and maintain user safety. This paper describes the development of a self-directed, digital intervention for psychedelic preparation. Drawing on elements from the UK Medical Research Council (MRC) framework for developing complex interventions, the design was informed by a four-factor model of psychedelic preparedness, using a person-centred approach. Our mixed-methods investigation consisted of two studies. The first involved interviews with 19 participants who had previously attended a 'high-dose' psilocybin retreat, systematically exploring their preparation behaviours and perspectives on the proposed intervention. The second study engaged 28 attendees of an ongoing psilocybin retreat in co-design workshops, refining the intervention protocol using insights from the initial interviews. The outcome is a co-produced 21-day digital course (Digital Intervention for Psychedelic Preparation (DIPP)), that is organised into four modules: Knowledge-Expectation, Psychophysical-Readiness, Safety-Planning, and Intention-Preparation. Fundamental components of the course include daily meditation practice, supplementary exercises tied to the weekly modules, and mood tracking. DIPP provides a comprehensive and scalable solution to enhance psychedelic preparedness, aligning with the broader shift towards digital mental health interventions.

Barriers to and facilitators of user engagement with web-based mental health interventions in young people: a systematic review

Many young people (YP) are diagnosed with mental illnesses and require support. Web-based mental health interventions (W-MHIs) have been increasingly utilized by YP, healthcare providers, and parents due to reasons including convenience and anonymity. W-MHIs are effective in improving mental health in YP. However, real-world engagement with W-MHIs remains low. Therefore, understanding barriers/facilitators of user engagement with W-MHIs is necessary to promote W-MHIs and help users gain optimal benefits through higher engagement. This review aims to identify barriers/facilitators of user engagement with W-MHIs in YP aged 10-24 years. A systematic search of five databases for English language, peer-reviewed publications was conducted between January 2010 and February 2023. Studies examining factors influencing user engagement with W-MHIs, described as barriers or facilitators, were included. Study quality was assessed using the Mixed Methods Appraisal Tool. A narrative synthesis was performed. Of 4088 articles identified, 69 studies were included. Barriers/facilitators were reported by young people (63 studies), providers (17 studies), and parents/caregivers (8 studies). YP perceived that usefulness and connectedness were the most common facilitators, whereas low-perceived need was the most reported barrier. Both providers and parents reported that perceived usefulness for YP was the most common facilitator, whereas concerns about program effectiveness and privacy were noted as barriers. This review found that program- and individual-related factors were important determinants of engagement with W-MHIs. This review provides guidance on the future design and development of new interventions, narrowing the gap between existing W-MHIs and unmet needs of users.

Testing 3 Digital Health Platforms to Improve Mental Health Outcomes in Adults With Type 1 Diabetes: A Pilot Trial

OBJECTIVE

Our aim in this study was to examine the potential impact of a 3-pronged digital health pilot intervention (TRIFECTA) on diabetes distress and depressive symptoms among adults with type 1 diabetes (T1D) in British Columbia.

METHODS

We recruited 60 adults with T1D (mean age 38.9±15.1 years, 75% female, 77% Caucasian) who participated in the 6-month pilot intervention involving 3 digital health platforms: monthly, provider-led, group-based sessions over Zoom (virtual huddles); a WhatsApp peer texting group; and a web-based "Ask-the-expert" portal. Assessments were conducted at baseline and 6 months and measured diabetes distress (T1D Diabetes Distress Scale), depressive symptoms (9-item Personal Health Questionnaire), and TRIFECTA engagement metrics.

RESULTS

Participation in TRIFECTA was associated with significant reductions in Overall Distress (p=0.011) and 4 distress subscales: Powerlessness (p=0.006), Management Distress (p=0.001), Hypoglycemia Distress (p=0.029), and Eating Distress (p<0.001). A higher number of virtual huddles attended predicted lower Overall Distress (p=0.019) and Family/Friends Distress (p=0.023). A higher number of "Ask-the-expert" posts viewed predicted lower Overall Distress (p=0.046), whereas a higher number of WhatsApp messages posted predicted lower Management Distress (p=0.006). Furthermore, engagement in all 3 metrics was a predictor for lower Negative Social Perceptions Distress (p<0.05). No associations were seen in other distress subscales or for depressive symptoms.

CONCLUSIONS

Participation in TRIFECTA was linked to reduced diabetes distress levels, but not depressive symptoms, in a platform-dependent manner. This study provides promising pilot data for a subsequent large-scale and fully powered randomized controlled trial.

Can gamification reduce the burden of self-reporting in mHealth applications? A feasibility study using machine learning from smartwatch data to estimate cognitive load

The effectiveness of digital treatments can be measured by requiring patients to self-report their state through applications, however, it can be overwhelming and causes disengagement. We conduct a study to explore the impact of gamification on self-reporting. Our approach involves the creation of a system to assess cognitive load (CL) through the analysis of photoplethysmography (PPG) signals. The data from 11 participants is utilized to train a machine learning model to detect CL. Subsequently, we create two versions of surveys: a gamified and a traditional one. We estimate the CL experienced by other participants (13) while completing surveys. We find that CL detector performance can be enhanced via pre-training on stress detection tasks. For 10 out of 13 participants, a personalized CL detector can achieve an F1 score above 0.7. We find no difference between the gamified and non-gamified surveys in terms of CL but participants prefer the gamified version.

What does it mean to be engaged with digital health interventions? A qualitative study into the experiences of engaged users and the views of professionals

Objective

Digital health interventions (DHIs) hold promise for influencing health behaviors positively, but their widespread implementation and effectiveness remain limited. Engagement is crucial for DHI effectiveness, yet its conceptualization is debated. This qualitative study explores engagement from user and professional perspectives.

Methods

Twenty self-proclaimed engaged health app users participated in semistructured interviews, and 13 professionals working with DHIs completed an online survey.

Results

Interviews with health app users revealed three key components of their sense of engagement: behavioral, cognitive, and affective. Behavioral engagement includes routine, effortless, and dynamic usage; emphasizing the importance of the quality of fit between user and technology over frequency of use. Cognitive engagement encompasses the technology's utility as a tool for supporting behavior change, providing new insights, and enhancing motivation. Affective engagement involves enjoying progress, deriving pleasure from using the technology, and identifying with the technology. Notably, participants exhibited varying emphasis on these components. Professionals, in a parallel inquiry, agreed on the relevance of behavior, cognition, and affect in defining engagement. In their understanding, behavioral engagement is often associated with adherence and frequency of use, while cognitive engagement emphasizes understanding, motivation, and achieving cognitive outcomes. Affective engagement, although diverse, is recognized as a critical dimension. In addition, it was noticeable that users and professionals perceived microengagement (with the DHI) and macroengagement (with the target behavior) as interconnected.

Conclusion

To conclude, this study contributes a nuanced understanding of the multifaceted nature of engagement, informing future measurement of the concept, DHI design, and implementation strategies for improved user experiences and outcomes.

Guided versus non-guided digital psychological interventions for cancer patients: A systematic review and meta-analysis of engagement and efficacy

OBJECTIVE

To evaluate engagement with and efficacy of guided versus non-guided digital interventions targeting psychological symptoms of cancer via a systematic review of current evidence.

METHODS

PubMed, Scopus, PsychINFO, MEDLINE, and CINAHL databases were searched. Eligible publications were randomised controlled trials of guided or non-guided digital psychological interventions used in cancer settings reporting intervention efficacy and/or engagement. Study methodological quality was assessed using the Cochrane risk of bias (RoB) tool. Random effects meta-analyses were performed on outcomes with sufficient data, with sub-group analyses of intervention type and follow-up period.

RESULTS

Forty-three studies were included. Studies varied by level of guidance, type of technology used, duration, and outcomes assessed. Most studies had a high overall RoB. Meta-analysis indicated that guided interventions significantly reduced distress, anxiety, and fatigue, while non-guided did not. For depression and quality of life, both guided and non-guided interventions produced significant improvements. Guided interventions reported higher levels of intervention engagement than non-guided.

CONCLUSIONS

Guided digital psychological interventions were likely to be more effective than non-guided ones for cancer patients, particularly in reducing distress, anxiety, and fatigue. Whilst both types were found to improve depression and life quality, guided interventions were associated with higher patient engagement. These findings suggest digital interventions could supplement traditional cancer care, warranting further research concerning long-term effects and cost-efficiency.

Mind-body internet and mobile-based interventions for depression and anxiety in adults with chronic physical conditions: A systematic review of RCTs

This review summarizes the effectiveness of scalable mind-body internet and mobile-based interventions (IMIs) on depression and anxiety symptoms in adults living with chronic physical conditions. Six databases (MEDLINE, PsycINFO, SCOPUS, EMBASE, CINAHL, and CENTRAL) were searched for randomized controlled trials published from database inception to March 2023. Mind-body IMIs included cognitive behavioral therapy, breathwork, meditation, mindfulness, yoga or Tai-chi. To focus on interventions with a greater potential for scale, the intervention delivery needed to be online with no or limited facilitation by study personnel. The primary outcome was mean change scores for anxiety and depression (Hedges' g). In subgroup analyses, random-effects models were used to calculate pooled effect size estimates based on personnel support level, intervention techniques, chronic physical condition, and survey type. Meta-regression was conducted on age and intervention length. Fifty-six studies met inclusion criteria (sample size 7691, mean age of participants 43 years, 58% female): 30% (n = 17) neurological conditions, 12% (n = 7) cardiovascular conditions, 11% cancer (n = 6), 43% other chronic physical conditions (n = 24), and 4% (n = 2) multiple chronic conditions. Mind-body IMIs demonstrated statistically significant pooled reductions in depression (SMD = -0.33 [-0.40, -0.26], p<0.001) and anxiety (SMD = -0.26 [-0.36, -0.17], p<0.001). Heterogeneity was moderate. Scalable mind-body IMIs hold promise as interventions for managing anxiety and depression symptoms in adults with chronic physical conditions without differences seen with age or intervention length. While modest, the effect sizes are comparable to those seen with pharmacological therapy. The field would benefit from detailed reporting of participant demographics including those related to technological proficiency, as well as further evaluation of non-CBT interventions. Registration: The study is registered with PROSPERO ID #CRD42022375606.

Digital therapeutics in the hospital for suicide crisis - content and design recommendations from young people and hospital staff

Objective

Hospital emergency departments lack the resources to adequately support young people who present for suicidal crisis. Digital therapeutics could fill this service gap by providing psychological support without creating additional burden on hospital staff. However, existing research on what is needed for successful integration of digital therapeutics in hospital settings is scant. Thus, this study sought to identify key considerations for implementing digital therapeutics to manage acute suicidal distress in hospitals.

Method

Participants were 17 young people who recently presented at the hospital for suicide-related crisis, and 12 hospital staff who regularly interacted with young people experiencing mental ill-health in their day-to-day work. Interviews were conducted via videoconference. Framework analysis and reflexive thematic analysis were used to interpret the data obtained.

Results

Qualitative insights were centred around three major themes: hospital-specific content, therapeutic content, and usability. Digital therapeutics were seen as a useful means for facilitating hospital-based assessment and treatment planning, and for conducting post-discharge check-ins. Therapeutic content should be focused on helping young people self-manage suicide-related distress while they wait for in-person services. Features to promote usability, such as the availability of customisable features and the use of inclusive design or language, should be considered in the design of digital therapeutics.

Conclusions

Digital therapeutics in hospital settings need to benefit both patients and staff. Given the unique context of the hospital setting and acute nature of suicidal distress, creating specialty digital therapeutics may be more viable than integrating existing ones.

Digitalising mental health care: Practical recommendations from the European Psychiatric Association

The digitalisation of mental health care is expected to improve the accessibility and quality of specialised treatment services and introduce innovative methods to study, assess, and monitor mental health disorders. In this narrative review and practical recommendation of the European Psychiatric Association (EPA), we aim to help healthcare providers and policymakers to navigate this rapidly evolving field. We provide an overview of the current scientific and implementation status across two major domains of digitalisation: i) digital mental health interventions and ii) digital phenotyping, discuss the potential of each domain to improve the accessibility and outcomes of mental health services, and highlight current challenges faced by researchers, clinicians, and service users. Furthermore, we make several recommendations meant to foster the widespread adoption of evidence-based digital solutions for mental health care in the member states of the EPA. To realise the vision of a digitalised, patient-centred, and data-driven mental health ecosystem, a number of implementation challenges must be considered and addressed, spanning from human, technical, ethical-legal, and economic barriers. The list of priority areas and action points our expert panel has identified could serve as a playbook for this process.

Finding the Best Match - a Case Study on the (Text-)Feature and Model Choice in Digital Mental Health Interventions

With the need for psychological help long exceeding the supply, finding ways of scaling, and better allocating mental health support is a necessity. This paper contributes by investigating how to best predict intervention dropout and failure to allow for a need-based adaptation of treatment. We systematically compare the predictive power of different text representation methods (metadata, TF-IDF, sentiment and topic analysis, and word embeddings) in combination with supplementary numerical inputs (socio-demographic, evaluation, and closed-question data). Additionally, we address the research gap of which ML model types - ranging from linear to sophisticated deep learning models - are best suited for different features and outcome variables. To this end, we analyze nearly 16.000 open-text answers from 849 German-speaking users in a Digital Mental Health Intervention (DMHI) for stress. Our research proves that - contrary to previous findings - there is great promise in using neural network approaches on DMHI text data. We propose a task-specific LSTM-based model architecture to tackle the challenge of long input sequences and thereby demonstrate the potential of word embeddings (AUC scores of up to 0.7) for predictions in DMHIs. Despite the relatively small data set, sequential deep learning models, on average, outperform simpler features such as metadata and bag-of-words approaches when predicting dropout. The conclusion is that user-generated text of the first two sessions carries predictive power regarding patients' dropout and intervention failure risk. Furthermore, the match between the sophistication of features and models needs to be closely considered to optimize results, and additional non-text features increase prediction results.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-023-00148-z.

Mental Health for All: The Case for Investing in Digital Mental Health to Improve Global Outcomes, Access, and Innovation in Low-Resource Settings

Mental health disorders are an increasing global public health concern that contribute to morbidity, mortality, disability, and healthcare costs across the world. Biomedical and psychological research has come a long way in identifying the importance of mental health and its impact on behavioral risk factors, physiological health, and overall quality of life. Despite this, access to psychological and psychiatric services remains widely unavailable and is a challenge for many healthcare systems, particularly those in developing countries. This review article highlights the strengths and opportunities brought forward by digital mental health in narrowing this divide. Further, it points to the economic and societal benefits of effectively managing mental illness, making a case for investing resources into mental healthcare as a larger priority for large non-governmental organizations and individual nations across the globe.

Specifying the Efficacy of Digital Therapeutic Tools for Depression and Anxiety: Retrospective, 2-Cohort, Real-World Analysis

BACKGROUND

Depression and anxiety are the main sources of work and social disabilities as well as health-related problems around the world. Digital therapeutic solutions using cognitive behavioral therapy have demonstrated efficacy in depression and anxiety. A common goal of digital health apps is to increase user digital engagement to improve outcomes. However, there is a limited understanding of the association between digital platform components and clinical outcomes.

OBJECTIVE

The aim of the study is to investigate the contribution of specific digital engagement tools to mental health conditions. We hypothesized that participation in coaching sessions and breathing exercises would be associated with a reduction in depression and anxiety.

METHODS

Depression and general anxiety symptoms were evaluated in real-world data cohorts using the digital health platform for digital intervention and monitoring change. This retrospective real-world analysis of users on a mobile platform-based treatment followed two cohorts of people: (1) users who started with moderate levels of depression and completed at least 2 depression assessments (n=519) and (2) users who started with moderate levels of anxiety and completed at least 2 anxiety assessments (n=474). Levels of depression (Patient Health Questionnaire-9) and anxiety (Generalized Anxiety Disorder-7) were tracked throughout the first 16 weeks. A piecewise mixed-effects model was applied to model the trajectories of the Patient Health Questionnaire-9 and the Generalized Anxiety Disorder-7 mean scores in 2 segments (1-6 weeks and 7-16 weeks). Finally, simple slope analysis was used for the interpretation of the interactions probing the moderators: coaching sessions and breathing exercises in both depression and anxiety cohorts.

RESULTS

Analysis revealed a significant decrease in depression symptoms (β=-.37, 95% CI -0.46 to 0.28; P≤.001) during the period of weeks 1-6 of app use, which was maintained during the period of 7-16 weeks. Coach interaction significantly moderated the reduction in depression symptoms during the period of weeks 1-6 (β=-.03, 95% CI -0.05 to -0.001; P=.02). A significant decrease in anxiety symptoms (β=-.41, 95% CI -0.50 to -0.33; P≤.001) was revealed during the period of 1-6 weeks, which was maintained during the period of 7-16 weeks. Breathing exercises significantly moderated the reduction in anxiety symptoms during the period of 1-6 weeks (β=-.07, 95% CI -0.14 to -0.01; P=.04).

CONCLUSIONS

This study demonstrated general improvement followed by a period of stability of depression and anxiety symptoms associated with cognitive behavioral therapy-based digital intervention. Interestingly, engagement with a coaching session but not a breathing exercise was associated with a reduction in depression symptoms. Moreover, breathing exercise but not engagement with a coaching session was associated with a reduction of anxiety symptoms. These findings emphasize the importance of using a personalized approach to behavioral health during digital health interventions.

The Development of Game-Based Digital Mental Health Interventions: Bridging the Paradigms of Health Care and Entertainment

Game elements are increasingly used to improve user engagement in digital mental health interventions, and specific game mechanics may yield therapeutic effects per se and thereby contribute to digital mental health intervention efficacy. However, only a few commercial game-based interventions are available. We suggest that the key challenge in their development reflects the tension between the 2 underlying paradigms, health care and entertainment, which have disparate goals and processes in digital development. We describe 3 approaches currently used to negotiate the 2 paradigms: the gamification of health care software, designing serious games, and purpose shifting existing entertainment games. We advanced an integrative framework to focus attention on 4 key themes in intervention development: target audience, engagement, mechanisms of action, and health-related effectiveness. On each theme, we show how the 2 paradigms contrast and can complement each other. Finally, we consider the 4 interdependent themes through the new product development phases from concept to production. Our viewpoint provides an integrative synthesis that facilitates the research, design, and development of game-based digital mental health interventions.

The engagement problem: A review of engagement with digital mental health interventions and recommendations for a path forward

Purpose of the review

Digital mental health interventions (DMHIs) are an effective and accessible means of addressing the unprecedented levels of mental illness worldwide. Currently, however, patient engagement with DMHIs in real-world settings is often insufficient to see clinical benefit. In order to realize the potential of DMHIs, there is a need to better understand what drives patient engagement.

Recent findings

We discuss takeaways from the existing literature related to patient engagement with DMHIs and highlight gaps to be addressed through further research. Findings suggest that engagement is influenced by patient-, intervention- and systems-level factors. At the patient-level, variables such as sex, education, personality traits, race, ethnicity, age and symptom severity appear to be associated with engagement. At the intervention-level, integrating human support, gamification, financial incentives and persuasive technology features may improve engagement. Finally, although systems-level factors have not been widely explored, the existing evidence suggests that achieving engagement will require addressing organizational and social barriers and drawing on the field of implementation science.

Summary

Future research clarifying the patient-, intervention- and systems-level factors that drive engagement will be essential. Additionally, to facilitate improved understanding of DMHI engagement, we propose the following: (a) widespread adoption of a minimum necessary 5-element engagement reporting framework; (b) broader application of alternative clinical trial designs; and (c) directed efforts to build upon an initial parsimonious conceptual model of DMHI engagement.

The effects of a sleep-focused smartphone application on insomnia and depressive symptoms: a randomised controlled trial and mediation analysis

BACKGROUND

Rates of depression are increasing among adolescents. A novel way to reduce depression is by improving sleep. We evaluated whether an app-based intervention for insomnia improved sleep and depression, and whether changes in insomnia mediated changes in depression.

METHODS

We conducted a 2-arm single-blind randomised controlled trial at the Black Dog Institute in Australia. Adolescents 12-16 years experiencing insomnia symptoms were randomly allocated to receive Sleep Ninja, an app-delivered cognitive behavioural therapy program for insomnia, or to an active control group involving weekly text message sleep tips. Assessments took place at baseline, 6 weeks (post-intervention) and 14 weeks (post-baseline). Co-primary outcomes were symptoms of insomnia and depression at post-intervention (primary endpoint). Intent-to-treat analyses were conducted. The trial is registered with the Australian and New Zealand Clinical Trials Registry, number ACTRN12619001462178.

RESULTS

Between October 25, 2019, and September 6, 2020, 264 participants were randomised to receive Sleep Ninja (n = 131) or to the control group (n = 133). Relative to the control group, those allocated to the intervention reported a greater reduction in insomnia symptoms at 6 weeks (95% CI: -2.96 to -0.41, d = .41) and 14 weeks (95% CI: -3.34 to -0.19, d = .39), and a greater reduction in depression symptoms at 6 weeks (95% CI: -3.46 to -0.56, d = .28) but not 14 weeks (p < 1). Change in insomnia mediated change in depression. No adverse events were reported.

CONCLUSIONS

An app-delivered program for insomnia could be a practical, non-stigmatising and scalable way to reduce symptoms of insomnia and depression among adolescents experiencing difficulties getting enough good quality sleep.

Promoting Engagement With Smartphone Apps for Suicidal Ideation in Young People: Development of an Adjunctive Strategy Using a Lived Experience Participatory Design Approach

BACKGROUND

Suicide among young people is a worrying public health concern. Despite this, there is a lack of suitable interventions aligned with the needs of this priority population. Emerging evidence supports the effectiveness of digital interventions in alleviating the severity of suicidal thoughts. However, their efficacy may be undermined by poor engagement. Technology-supported strategies (eg, electronic prompts and reminders) have been deployed alongside digital interventions to increase engagement with the latter. However, evidence of their efficacy is inconclusive. User-centered design approaches may be key to developing feasible and effective engagement strategies. Currently, no study has been published on how such an approach might be expressly applied toward developing strategies for promoting engagement with digital interventions.

OBJECTIVE

This study aimed to detail the processes and activities involved in developing an adjunctive strategy for promoting engagement with LifeBuoy-a smartphone app that helps young people manage suicidal thoughts.

METHODS

Development of the engagement strategy took place in 2 phases. The discovery phase aimed to create an initial prototype by synthesizing earlier findings-from 2 systematic reviews and a cross-sectional survey of the broader mental health app user population-with qualitative insights from LifeBuoy users. A total of 16 web-based interviews were conducted with young people who participated in the LifeBuoy trial. Following the discovery phase, 3 interviewees were invited by the research team to take part in the workshops in the design phase, which sought to create a final prototype by making iterative improvements to the initial prototype. These improvements were conducted over 2 workshops. Thematic analysis was used to analyze the qualitative data obtained from the interviews and workshops.

RESULTS

Main themes from the interviews centered around the characteristics of the strategy, timing of notifications, and suitability of social media platforms. Subsequently, themes that emerged from the design workshops emphasized having a wider variety of content, greater visual consistency with LifeBuoy, and a component with more detailed information to cater to users with greater informational needs. Thus, refinements to the prototype were focused on (1) improving the succinctness, variety, and practical value of Instagram content, (2) creating a blog containing articles contributed by mental health professionals and young people with lived experience of suicide, and (3) standardizing the use of marine-themed color palettes across the Instagram and blog components.

CONCLUSIONS

This is the first study to describe the development of a technology-supported adjunctive strategy for promoting engagement with a digital intervention. It was developed by integrating perspectives from end users with lived experience of suicide with evidence from the existing literature. The development process documented in this study may be useful for guiding similar projects aimed at supporting the use of digital interventions for suicide prevention or mental health.

A systematic review and meta-analysis of digital interventions targeted at individuals with borderline personality disorder (BPD), emotionally unstable personality disorder (EUPD), and related symptoms

OBJECTIVES

The present review investigates the impact of digital interventions for individuals with features of borderline personality disorder (BPD)/emotional unstable personality disorder (EUPD) as digital interventions show promise as therapeutic tools in underserved groups. BPD/EUPD features are identified as clinically relevant, yet previous reviews on the use of digital interventions fail to include subthreshold symptomatology.

METHODS

Five online databases were searched for terminology in three categories: BPD/EUPD and related symptoms, mental-health interventions, and digital technology. Additionally, four relevant journals and two trial registers were searched for additional papers meeting the inclusion criteria.

RESULTS

Twelve articles met all inclusion criteria. Meta-analyses revealed statistically significant differences in symptom measures between intervention and control groups at postintervention and decreases in BPD/EUPD symptomatology and well-being from pre- to postintervention. Service users' engagement, satisfaction, and acceptability of interventions were high. Results support the previous literature on the value of using digital interventions in populations with BPD/EUPD.

CONCLUSION

Overall, it was identified that digital interventions show promise for successful implementation with this population.

Framework for Guiding the Development of High-Quality Conversational Agents in Healthcare

Evaluating conversational agents (CAs) that are supposed to be applied in healthcare settings and ensuring their quality is essential to avoid patient harm and ensure efficacy of the CA-delivered intervention. However, a guideline for a standardized quality assessment of health CAs is still missing. The objective of this work is to describe a framework that provides guidance for development and evaluation of health CAs. In previous work, consensus on categories for evaluating health CAs has been found. In this work, we identify concrete metrics, heuristics, and checklists for these evaluation categories to form a framework. We focus on a specific type of health CA, namely rule-based systems that are based on written input and output, have a simple personality without any kind of embodiment. First, we identified relevant metrics, heuristics, and checklists to be linked to the evaluation categories through a literature search. Second, five experts judged the metrics regarding their relevance to be considered within evaluation and development of health CAs. The final framework considers nine aspects from a general perspective, five aspects from a response understanding perspective, one aspect from a response generation perspective, and three aspects from an aesthetics perspective. Existing tools and heuristics specifically designed for evaluating CAs were linked to these evaluation aspects (e.g., Bot usability scale, design heuristics for CAs); tools related to mHealth evaluation were adapted when necessary (e.g., aspects from the ISO technical specification for mHealth Apps). The resulting framework comprises aspects to be considered not only as part of a system evaluation, but already during the development. In particular, aspects related to accessibility or security have to be addressed in the design phase (e.g., which input and output options are provided to ensure accessibility?) and have to be verified after the implementation phase. As a next step, transfer of the framework to other types of health CAs has to be studied. The framework has to be validated by applying it during health CA design and development.

Measuring Engagement with Mental Health and Behavior Change Interventions: an Integrative Review of Methods and Instruments

BACKGROUND

Engagement is a complex construct consisting of behavioral, cognitive, and affective dimensions, making engagement a difficult construct to measure. This integrative review aims to (1) present a multidisciplinary overview of measurement methods that are currently used to measure engagement with adult mental health and behavior change interventions, delivered in-person, blended, or digitally, and (2) provide a set of recommendations and considerations for researchers wishing to study engagement.

METHODS

We used an integrative approach and identified original studies and reviews on engagement with mental health or behavior change interventions that were delivered in-person, digitally, or blended.

RESULTS

Forty articles were analyzed in this review. Common methods to assess engagement were through objective usage data, questionnaire-based data, and qualitative data, with objective usage data being used most frequently. Based on the synthesis of engagement measures, we advise researchers to (1) predefine the operationalization of engagement for their specific research context, (2) measure behavioral, cognitive, and affective dimensions of engagement in all cases, and (3) measure engagement over time.

CONCLUSIONS

Current literature shows a bias towards behavioral measures of engagement in research, as most studies measured engagement exclusively through objective usage data, without including cognitive and affective measures of engagement. We hope that our recommendations will help to reduce this bias and to steer engagement research towards an integrated approach.

Design Features Associated With Engagement in Mobile Health Physical Activity Interventions Among Youth: Systematic Review of Qualitative and Quantitative Studies

BACKGROUND

Globally, 81% of youth do not meet the physical activity (PA) guidelines. Youth of families with a low socioeconomic position are less likely to meet the recommended PA guidelines. Mobile health (mHealth) interventions are preferred by youth over traditional in-person approaches and are in line with their media preferences. Despite the promise of mHealth interventions in promoting PA, a common challenge is to engage users in the long term or effectively. Earlier reviews highlighted the association of different design features (eg, notifications and rewards) with engagement among adults. However, little is known about which design features are important for increasing engagement among youth.

OBJECTIVE

To inform the design process of future mHealth tools, it is important to investigate the design features that can yield effective user engagement. This systematic review aimed to identify which design features are associated with engagement in mHealth PA interventions among youth who were aged between 4 and 18 years.

METHODS

A systematic search was conducted in EBSCOhost (MEDLINE, APA PsycINFO, and Psychology & Behavioral Sciences Collection) and Scopus. Qualitative and quantitative studies were included if they documented design features associated with engagement. Design features and related behavior change techniques and engagement measures were extracted. Study quality was assessed according to the Mixed Method Assessment Tool, and one-third of all screening and data extraction were double coded by a second reviewer.

RESULTS

Studies (n=21) showed that various features were associated with engagement, such as a clear interface, rewards, multiplayer game mode, social interaction, variety of challenges with personalized difficulty level, self-monitoring, and variety of customization options among others, including self-set goals, personalized feedback, progress, and a narrative. In contrast, various features need to be carefully considered while designing mHealth PA interventions, such as sounds, competition, instructions, notifications, virtual maps, or self-monitoring, facilitated by manual input. In addition, technical functionality can be considered as a prerequisite for engagement. Research addressing youth from low socioeconomic position families is very limited with regard to engagement in mHealth apps.

CONCLUSIONS

Mismatches between different design features in terms of target group, study design, and content translation from behavior change techniques to design features are highlighted and set up in a design guideline and future research agenda.

TRIAL REGISTRATION

PROSPERO CRD42021254989; https://tinyurl.com/5n6ppz24.

Exploring Retention, Usage, and Efficacy of Web-Based Delivery of Positive Emotion Regulation Skills During the COVID-19 Pandemic

COVID-19 prompted distress and increased reliance on digital mental health interventions, which previously demonstrated low rates of retention and adherence. This single-arm trial evaluated whether self-guided, web-based, positive affect regulation skills (PARK) were engaging and associated with changes in well-being during the pandemic. Over 6 weeks, PARK delivers brief lessons and practices in skills designed to increase positive emotions: noticing positive events, savoring, gratitude, mindfulness, positive reappraisal, personal strengths, and self-compassion. Patient-Reported Outcome Measurement Information System (PROMIS) computer adaptive tests of anxiety, depression, social isolation, positive affect, and meaning and purpose were administered at baseline, post-intervention, and 6 months after baseline. Retention and usage of PARK were measured by the web-based assessment and intervention platforms. The sample (n = 616) was predominantly female, non-Hispanic, white, and well-educated. Of those who completed baseline, only 42% completed a follow-up assessment; 30% never logged into PARK. Among those who did, 86% used at least one skill, but only 14% completed PARK. Across retention and usage metrics, older age predicted more engagement. In multivariable models, people of color and people with greater baseline anxiety were more likely to complete PARK. All well-being indicators improved over time, with greater improvements in anxiety and social isolation among participants who accessed at least one PARK skill compared to those who did not. Retention and usage rates mirrored pre-pandemic trends, but within this select sample, predictors of engagement differed from prior research. Findings underscore the need for additional efforts to ensure equitable access to digital mental health interventions and research. Trials registration: NCT04367922.

Building Emotional Awareness and Mental Health (BEAM): an open-pilot and feasibility study of a digital mental health and parenting intervention for mothers of infants

BACKGROUND

Maternal mental health concerns and parenting stress in the first few years following childbirth are common and pose significant risks to maternal and child well-being. The COVID-19 pandemic has led to increases in maternal depression and anxiety and has presented unique parenting stressors. Although early intervention is crucial, there are significant barriers to accessing care.

METHODS

To inform a larger randomized controlled trial, the current open-pilot trial investigated initial evidence for the feasibility, acceptability, and efficacy of a newly developed online group therapy and app-based mental health and parenting program (BEAM) for mothers of infants. Forty-six mothers 18 years or older with clinically elevated depression scores, with an infant aged 6-17 months old, and who lived in Manitoba or Alberta were enrolled in the 10-week program (starting in July 2021) and completed self-report surveys.

RESULTS

The majority of participants engaged in each of the program components at least once and participants indicated relatively high levels of app satisfaction, ease of use, and usefulness. However, there was a high level of attrition (46%). Paired-sample t-tests indicated significant pre- to post-intervention change in maternal depression, anxiety, and parenting stress, and in child internalizing, but not externalizing symptoms. Effect sizes were in the medium to high range, with the largest effect size observed for depressive symptoms (Cohen's d = .93).

DISCUSSION

This study shows moderate levels of feasibility and strong preliminary efficacy of the BEAM program. Limitations to program design and delivery are being addressed for testing in adequately powered follow-up trials of the BEAM program for mothers of infants.

TRIAL REGISTRATION

NCT04772677 . Registered on February 26 2021.

Providing Human Support for the Use of Digital Mental Health Interventions: Systematic Meta-review

BACKGROUND

Digital mental health interventions (DMHIs) have been increasingly deployed to bridge gaps in mental health care, particularly given their promising efficacy. Nevertheless, attrition among DMHI users remains high. In response, human support has been studied as a means of improving retention to and outcomes of DMHIs. Although a growing number of studies and meta-analyses have investigated the effects of human support for DMHIs on mental health outcomes, systematic empirical evidence of its effectiveness across mental health domains remains scant.

OBJECTIVE

We aimed to summarize the results of meta-analyses of human support versus no support for DMHI use across various outcome domains, participant samples, and support providers.

METHODS

We conducted a systematic meta-review of meta-analyses, comparing the effects of human support with those of no support for DMHI use, with the goal of qualitatively summarizing data across various outcome domains, participant samples, and support providers. We used MEDLINE, PubMed, and PsycINFO electronic databases. Articles were included if the study had a quantitative meta-analysis study design; the intervention targeted mental health symptoms and was delivered via a technology platform (excluding person-delivered interventions mediated through telehealth, text messages, or social media); the outcome variables included mental health symptoms such as anxiety, depression, stress, posttraumatic stress disorder symptoms, or a number of these symptoms together; and the study included quantitative comparisons of outcomes in which human support versus those when no or minimal human support was provided.

RESULTS

The results of 31 meta-analyses (505 unique primary studies) were analyzed. The meta-analyses reported 45 effect sizes; almost half (n=22, 48%) of them showed that human-supported DMHIs were significantly more effective than unsupported DMHIs. A total of 9% (4/45) of effect sizes showed that unsupported DMHIs were significantly more effective. No clear patterns of results emerged regarding the efficacy of human support for the outcomes assessed (including anxiety, depression, posttraumatic stress disorder, stress, and multiple outcomes). Human-supported DMHIs may be more effective than unsupported DMHIs for individuals with elevated mental health symptoms. There were no clear results regarding the type of training for those providing support.

CONCLUSIONS

Our findings highlight the potential of human support in improving the effects of DMHIs. Specifically, evidence emerged for stronger effects of human support for individuals with greater symptom severity. There was considerable heterogeneity across meta-analyses in the level of detail regarding the nature of the interventions, population served, and support delivered, making it difficult to draw strong conclusions regarding the circumstances under which human support is most effective. Future research should emphasize reporting detailed descriptions of sample and intervention characteristics and describe the mechanism through which they believe the coach will be most useful for the DMHI.