Usability, Feasibility, and Effect of a Biocueing Intervention in Addition to a Moderated Digital Social Therapy-Platform in Young People With Emerging Mental Health Problems: A Mixed-Method Approach

A "Next Generation" of Pediatric Mental Health Systems

An optimistic view of the future child/youth mental health system is that it will be oriented toward prevention and shaped by innovations in early detection and treatment of functional problems, coupled with the power of digital technology to provide new ways to help individuals and families monitor their well-being and seek or agree to help as it is needed. These innovations will be deployed within a community-based health care system, centered on primary care that fully implements ideas about continuity and comprehensiveness (including social determinants, substance use, and multigenerational care) that have been around for decades.

Putting the usability of wearable technology in forensic psychiatry to the test: a randomized crossover trial

Introduction

Forensic psychiatric patients receive treatment to address their violent and aggressive behavior with the aim of facilitating their safe reintegration into society. On average, these treatments are effective, but the magnitude of effect sizes tends to be small, even when considering more recent advancements in digital mental health innovations. Recent research indicates that wearable technology has positive effects on the physical and mental health of the general population, and may thus also be of use in forensic psychiatry, both for patients and staff members. Several applications and use cases of wearable technology hold promise, particularly for patients with mild intellectual disability or borderline intellectual functioning, as these devices are thought to be user-friendly and provide continuous daily feedback.

Method

In the current randomized crossover trial, we addressed several limitations from previous research and compared the (continuous) usability and acceptance of four selected wearable devices. Each device was worn for one week by staff members and patients, amounting to a total of four weeks. Two of the devices were general purpose fitness trackers, while the other two devices used custom made applications designed for bio-cueing and for providing insights into physiological reactivity to daily stressors and events.

Results

Our findings indicated significant differences in usability, acceptance and continuous use between devices. The highest usability scores were obtained for the two fitness trackers (Fitbit and Garmin) compared to the two devices employing custom made applications (Sense-IT and E4 dashboard). The results showed similar outcomes for patients and staff members.

Discussion

None of the devices obtained usability scores that would justify recommendation for future use considering international standards; a finding that raises concerns about the adaptation and uptake of wearable technology in the context of forensic psychiatry. We suggest that improvements in gamification and motivational aspects of wearable technology might be helpful to tackle several challenges related to wearable technology.

Outcomes of Best-Practice Guided Digital Mental Health Interventions for Youth and Young Adults with Emerging Symptoms: Part II. A Systematic Review of User Experience Outcomes

Although many young people demonstrate resilience and strength, research and clinical evidence highlight an upward trend in mental health concerns among those aged 12 to 25 years. Youth-specific digital mental health interventions (DMHIs) aim to address this trend by providing timely access to mental health support for young people (12-25 years). However, there is a considerable gap in understanding young people user experiences with digital interventions. This review, co-designed with Australia's leading mental health organization Beyond Blue, utilizes a systematic methodology to synthesize evidence on user experience in youth-oriented digital mental health interventions that are fully or partially guided. Five relevant online databases were searched for articles published from 2018 to 2023, yielding 22,482 articles for screening and 22 studies were included in the present analysis. User experience outcomes relating to satisfaction and engagement were assessed for each included intervention, with experience indicators relating to usefulness, usability, value, credibility, and desirability being examined. Elements associated with positive/negative outcomes were extracted. Elements shown to positively influence user experience included peer engagement, modern app-based delivery, asynchronous support, and personalized content. In contrast, users disliked static content, homework/log-keeping, the requirement for multiple devices, and social media integration. Asynchronous interventions showed high satisfaction but faced engagement issues, with combined asynchronous/synchronous interventions reporting better completion rates. DMHIs offer a promising platform for youth mental health support and has the potential to dramatically increase the reach of interventions through the adoption of technological and user experience best practices. While young people respond positively to many aspects of intervention modernization, such as interactive, app-based design, other concepts, such as social media integration, they need to be adopted by the field more cautiously to ensure trust and engagement.Trial Registration CRD42023405812.

Outcomes of Best-Practice Guided Digital Mental Health Interventions for Youth and Young Adults with Emerging Symptoms: Part I. A Systematic Review of Socioemotional Outcomes and Recommendations

Youth-specific digital mental health interventions (DMHI) represent an emerging field of study, and aim to increase access, improve socioemotional outcomes, and, where required, support triage to targeted interventions. However, prior reviews have reported inconsistent findings on the clinical effectiveness of such interventions in young adults (12-25 years). Further, shortfalls remain for the impact of guided interventions based on the mode of delivery and the type of human support personnel (e.g., professional or peer) guiding the intervention. In response, this systematic review, co-designed with Australia's leading mental health organization, aims to assess the effectiveness of guided digital programs in improving youth socioemotional outcomes. Included studies involve young people experiencing mental ill-health, receiving brief (i.e., 1-12 sessions), digitally delivered (at least partially) psychological interventions that were guided or partially guided, tested in a type of experimental study, with a socioemotional outcome. Specific socioemotional outcomes examined were depression, anxiety, stress, wellbeing, mindfulness, and quality of life. A systematic search of the contemporary published and grey literature identified 22,482 records with 32 relevant records published between 2018 and 2023. A narrative synthesis guided integration of findings. Results demonstrated strong evidence for the effectiveness of guided interventions on socioemotional outcomes (i.e., depression, anxiety, stress) yet these effects were short-lived. When factoring in the use of different control groups (i.e., active vs. inactive), inconsistent effects were observed for the socioemotional outcomes of depression, anxiety, and stress. The mode of delivery (i.e., asynchronous, synchronous, combined) and the type of human support personnel did not appear to impact socioemotional outcomes. Results indicate efficacious brief digital interventions for depression and anxiety include refresher/follow-up content, goal setting content, and relapse prevention content. In contrast, poor efficacy is associated with interventions that include homework tasks, self-monitoring, and log-keeping content.PROSPERO, ID CRD42023405812.

Methods, Indicators, and End-User Involvement in the Evaluation of Digital Health Interventions for the Public: Scoping Review

BACKGROUND

Digital health interventions (DHIs) have the potential to enable public end users, such as citizens and patients, to manage and improve their health. Although the number of available DHIs is increasing, examples of successfully established DHIs in public health systems are limited. To counteract the nonuse of DHIs, they should be comprehensively evaluated while integrating end users. Unfortunately, there is a wide variability and heterogeneity according to the approaches of evaluation, which creates a methodological challenge.

OBJECTIVE

This scoping review aims to provide an overview of the current established processes for evaluating DHIs, including methods, indicators, and end-user involvement. The review is not limited to a specific medical field or type of DHI but offers a holistic overview.

METHODS

This scoping review was conducted following the JBI methodology for scoping reviews based on the framework by Arksey & O'Malley and complies with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Three scientific databases (PubMed, Scopus, and Science Direct) were searched in April 2023. English and German studies between 2008 and 2023 were considered when evaluating DHIs that explicitly address public end users. The process of study selection was carried out by several researchers to avoid reviewer bias.

RESULTS

The search strategy identified 9618 publications, of which 160 were included. Among these included articles, 200 evaluations were derived and analyzed. The results showed that there is neither a consensus on the methods to evaluate DHIs nor a commonly agreed definition or usage of the evaluated indicators, which results in a broad variety of evaluation practices. This aligns with observations of the existing literature. It was found that there is a lack of references to existing frameworks for the evaluation of DHIs. The majority of the included studies referred to user-centered approaches and involved end users in the evaluation process. As assistance for people developing and evaluating DHIs and as a basis for thinking about appropriate ways to evaluate DHIs, a results matrix was created where the findings were combined per DHI cluster. Additionally, general recommendations for the evaluators of DHIs were formulated.

CONCLUSIONS

The findings of this scoping review offer a holistic overview of the variety and heterogeneity according to the approaches of evaluation of DHIs for public end users. Evaluators of these DHIs should be encouraged to reference established frameworks or measurements for justification. This would ease the transferability of the results among similar evaluation studies within the digital health sector, thereby enhancing the coherence and comparability of research in this area.

The Effects of a Digital, Transdiagnostic, Clinically and Peer-Moderated Treatment Platform for Young People With Emerging Mental Health Complaints: Repeated Measures Within-Subjects Study

BACKGROUND

To address the growing prevalence of youth mental health problems, early intervention is crucial to minimize individual, societal, and economic impacts. Indicative prevention aims to target emerging mental health complaints before the onset of a full-blown disorder. When intervening at this early stage, individuals are more responsive to treatment, resulting in cost-effective outcomes. The Moderated Online Social Therapy platform, which was successfully implemented and proven effective in Australia, is a digital, peer- and clinically moderated treatment platform designed for young people. The Netherlands was the first country outside Australia to implement this platform, under the name Engage Young People Early (ENYOY). It has the potential to reduce the likelihood of young people developing serious mental health disorders.

OBJECTIVE

This study aims to investigate the effects on young people using the ENYOY-platform in relation to psychological distress, psychosocial functioning, and positive health parameters.

METHODS

Dutch-speaking young people with emerging mental health complaints (N=131) participated in the ENYOY-platform for 6 months in a repeated measures within-subjects study. Psychological distress, psychosocial functioning, and positive health parameters were assessed at baseline and 3, 6, and 12 months. Repeated measures ANOVA was conducted and adjusted for age, sex, therapy, and community activity. The Reliable Change Index and Clinically Significant Index were computed to compare the baseline with the 6- and 12-month measurements. The missing data rate was 22.54% and the dropout rate 62.6% (82/131).

RESULTS

The primary analysis (77/131, 58.8%) showed that psychological distress decreased and psychosocial functioning improved over time with large effect sizes (P<.001 in both cases; ηp2=0.239 and 0.318, respectively) independent of age (P=.76 for psychological distress and P=.48 for psychosocial functioning), sex (P=.24 and P=.88, respectively), therapy activity (P=.49 and P=.80, respectively), or community activity (P=.59 and P=.48, respectively). Similarly, secondary analyses (51/131, 38.9%) showed significant effects of time on the quality of life, well-being, and meaningfulness positive health parameters (P<.05; ηp2=0.062, 0.140, and 0.121, respectively). Improvements in all outcome measures were found between baseline and 3 and 6 months (P≤.001-.01; d=0.23-0.62) and sustained at follow-up (P=.18-.97; d=0.01-0.16). The Reliable Change Index indicated psychological distress improvements in 38% (39/102) of cases, no change in 54.9% (56/102) of cases, and worsening in 5.9% (6/102) of cases. Regarding psychosocial functioning, the percentages were 50% (51/102), 43.1% (44/102), and 6.9% (7/102), respectively. The Clinically Significant Index demonstrated clinically significant changes in 75.5% (77/102) of cases for distress and 89.2% (91/102) for functioning.

CONCLUSIONS

This trial demonstrated that the ENYOY-platform holds promise as a transdiagnostic intervention for addressing emerging mental health complaints among young people in the Netherlands and laid the groundwork for further clinical research. It would be of great relevance to expand the population on and service delivery of the platform.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1186/s12888-021-03315-x.

Forensic Psychiatric Outpatients' and Therapists' Perspectives on a Wearable Biocueing App (Sense-IT) as an Addition to Aggression Regulation Therapy: Qualitative Focus Group and Interview Study

BACKGROUND

Given the increased use of smart devices and the advantages of individual behavioral monitoring and assessment over time, wearable sensor-based mobile health apps are expected to become an important part of future (forensic) mental health care. For successful implementation in clinical practice, consideration of barriers and facilitators is of utmost importance.

OBJECTIVE

The aim of this study was to provide insight into the perspectives of both psychiatric outpatients and therapists in a forensic setting on the use and implementation of the Sense-IT biocueing app in aggression regulation therapy.

METHODS

A combination of qualitative methods was used. First, we assessed the perspectives of forensic outpatients on the use of the Sense-IT biocueing app using semistructured interviews. Next, 2 focus groups with forensic therapists were conducted to gain a more in-depth understanding of their perspectives on facilitators of and barriers to implementation.

RESULTS

Forensic outpatients (n=21) and therapists (n=15) showed a primarily positive attitude toward the addition of the biocueing intervention to therapy, with increased interoceptive and emotional awareness as the most frequently mentioned advantage in both groups. In the semistructured interviews, patients mainly reported barriers related to technical or innovation problems (ie, connection and notification issues, perceived inaccuracy of the feedback, and limitations in the ability to personalize settings). In the focus groups with therapists, 92 facilitator and barrier codes were identified and categorized into technical or innovation level (n=13, 14%), individual therapist level (n=28, 30%), individual patient level (n=33, 36%), and environmental and organizational level (n=18, 20%). The predominant barriers were limitations in usability of the app, patients' motivation, and both therapists' and patients' knowledge and skills. Integration into treatment, expertise within the therapists' team, and provision of time and materials were identified as facilitators.

CONCLUSIONS

The chances of successful implementation and continued use of sensor-based mobile health interventions such as the Sense-IT biocueing app can be increased by considering the barriers and facilitators from patients' and therapists' perspectives. Technical or innovation-related barriers such as usability issues should be addressed first. At the therapist level, increasing integration into daily routines and enhancing affinity with the intervention are highly recommended for successful implementation. Future research is expected to be focused on further development and personalization of biocueing interventions considering what works for whom at what time in line with the trend toward personalizing treatment interventions in mental health care.

Moderated digital social therapy for young people with emerging mental health problems: A user-centered mixed-method design and usability study

Introduction

Over 25% of Dutch young people are psychologically unhealthy. Individual and societal consequences that follow from having mental health complaints at this age are substantial. Young people need care which is often unavailable. ENgage YOung people earlY (ENYOY) is a moderated digital social therapy-platform that aims to help youngsters with emerging mental health complaints. Comprehensive research is being conducted into the effects and to optimize and implement the ENYOY-platform throughout the Netherlands. The aim of this study is to explore the usability and user experience of the ENYOY-platform.

Methods

A user-centered mixed-method design was chosen. 26 young people aged 16-25 with emerging mental health complaints participated. Semi-structured interviews were conducted to explore usability, user-friendliness, impact, accessibility, inclusivity, and connection (Phase 1). Phase 2 assessed usability problems using the concurrent and retrospective Think Aloud-method. User experience and perceived helpfulness were assessed using a 10-point rating scale and semi-structured interviews (Phase 3). The Health Information Technology Usability Evaluation Scale (Health-ITUES; Phase 1) and System Usability Scale (SUS; Phase 2 and 3) were administered. Qualitative data was analyzed using thematic analysis. Task completion rate and time were tracked and usability problems were categorized using the Nielsen's rating scale (Phase 2).

Results

Adequate to high usability was found (Phase 1 Health-ITUES 4.0(0.34); Phase 2 SUS 69,5(13,70); Phase 3 SUS 71,6(5,63)). Findings from Phase 1 (N = 10) indicated that users viewed ENYOY as a user-friendly, safe, accessible, and inclusive initiative which helped them reduce their mental health complaints and improve quality of life. Phase 2 (N = 10) uncovered 18 usability problems of which 5 of major severity (e.g. troubles accessing the platform). Findings from Phase 3 (N = 6) suggested that users perceived the coaching calls the most helpful [9(0.71)] followed by the therapy content [6.25(1.41)]. Users liked the social networking aspect but rated it least helpful [6(2.1)] due to inactivity.

Conclusion

The ENYOY-platform has been found to have adequate to high usability and positive user experiences were reported. All findings will be transferred to the developmental team to improve the platform. Other evaluation methods and paring these with quantitative outcomes could provide additional insight in future research.