Routine Outcome Monitoring in CAMHS: How Can We Enable Implementation in Practice?

Youth Perspectives on 'Highly Personalised and Measurement-Based Care': Qualitative Co-Design of Education Materials

OBJECTIVES

Despite high levels of mental ill-health amongst young people (aged 15-30), this group demonstrates low help-seeking and high drop-out from mental health services (MHS). Whilst shared decision-making can assist people in receiving appropriate and effective health care, young people frequently report that they do not feel involved in treatment decisions. The current study focused on co-design of a clinical education and participant information programme for the Brain and Mind Centre Youth Model of Care. This model, which articulates a youth-focused form of highly personalised and measurement-based care, is designed to promote shared decision-making between young people and clinical service providers.

METHODS

We conducted workshops with 24 young people (16-31; MAge = 21.5) who had accessed mental health services. Participants were asked what advice they would give to young people entering services, before giving advice on existing materials. Workshops were conducted and transcripts were coded using thematic analysis by two lived experience researchers and a clinical researcher.

RESULTS

Young people found it empowering to be educated on transdiagnostic models of mental illness, namely clinical staging, which gives them a better understanding of why certain treatments may be inappropriate and ineffective, and thus reduce self-blame. Similarly, young people had limited knowledge of links between mental health and other life domains and found it helpful to be educated on multidisciplinary treatment options. Measurement-based care was seen as an important method of improving shared decision-making between young people and health professionals; however, to facilitate shared decision-making, young people also wanted better information on their rights in care and more support to share their expertise in their own needs, values and treatment preferences.

CONCLUSIONS

These findings will inform the delivery of the further development and implementation of a youth-specific clinical education and participant information programme for the BMC Youth Model.

PATIENT OR PUBLIC CONTRIBUTION

Workshops were facilitated by researchers with lived expertise in mental ill-health (A.H. and/or C.G.) and a clinical researcher (who has expertise as an academic and a clinical psychologist). A.H. and C.G. were also involved in conceptualisation, analysis, interpretation, review and editing of this paper.

New Brunswick's mental health action plan: A quantitative exploration of program efficacy in children and youth using the Canadian Community Health Survey

In 2011, the New Brunswick government released the New Brunswick Mental Health Action Plan 2011-2018 (Action Plan). Following the release of the Action Plan in 2011, two progress reports were released in 2013 and 2015, highlighting the implementation status of the Action Plan. While vague in their language, these reports indicated considerable progress in implementing the Action Plan, as various initiatives were undertaken to raise awareness and provide additional resources to facilitate early prevention and intervention in children and youth. However, whether these initiatives have yielded measurable improvements in population-level mental health outcomes in children and youth remains unclear. The current study explored the impact of the Action Plan by visualizing the trend in psychosocial outcomes and service utilization of vulnerable populations in New Brunswick before and after the implementation of the Action Plan using multiple datasets from the Canadian Community Health Survey. Survey-weighted ordinary least square regression analyses were performed to investigate measurable improvements in available mental health outcomes. The result revealed a declining trend in the mental wellness of vulnerable youth despite them consistently reporting higher frequencies of mental health service use. This study highlights the need for a concerted effort in providing effective mental health services to New Brunswick youth and, more broadly, Canadian youth, as well as ensuring rigorous routine outcome monitoring and evaluation plans are consistently implemented for future mental health strategies at the time of their initiation.

Systematic review of effectiveness and satisfaction evaluation in child and adolescent mental health services in Ireland

OBJECTIVES

Increasing numbers of youth experience mental illness, and also require and benefit from specialist child and adolescent mental health services (CAMHS). Worldwide, such services are underfunded and under-resourced, and services in Ireland are no different. It is vital that existing services are regularly reviewed for both efficacy and acceptability. Our objective was to review published studies evaluating service user satisfaction with CAMHS in Ireland and CAMHS therapeutic efficacy.

METHODS

MEDLINE, PsycINFO and CINAHL databases were systematically searched. Studies were included if they reported on service user satisfaction or an evaluation of CAMHS in Ireland.

RESULTS

From an initial 125 articles identified, 15 studies meet the inclusion criteria: four reporting on overall CAMHS satisfaction, three on satisfaction where a specific diagnosis was present, while eight evaluated various interventions offered. Whilst most service users perceived services to be satisfactory, important issues relating to accessibility were present. Evidence of efficacy was present for a small number of interventions, but studies were limited by methodological issues.

CONCLUSIONS

There is a dearth of studies evaluating CAMHS in Ireland. The extant literature suggests a positive experience once accessed, but long waiting times and poor collaboration are seen to limit services users' experience. More robust methodologically sound studies are urgently required. Given the expected increased demand linked to the current COVID-19 pandemic, coupled with the resultant compromised financial position, it is essential that scant resources are appropriately directed.

Assessing the feasibility of a web-based outcome measurement system in child and adolescent mental health services - myHealthE a randomised controlled feasibility pilot study

BACKGROUND

Interest in internet-based patient reported outcome measure (PROM) collection is increasing. The NHS myHealthE (MHE) web-based monitoring system was developed to address the limitations of paper-based PROM completion. MHE provides a simple and secure way for families accessing Child and Adolescent Mental Health Services to report clinical information and track their child's progress. This study aimed to assess whether MHE improves the completion of the Strengths and Difficulties Questionnaire (SDQ) compared with paper collection. Secondary objectives were to explore caregiver satisfaction and application acceptability.

METHODS

A 12-week single-blinded randomised controlled feasibility pilot trial of MHE was conducted with 196 families accessing neurodevelopmental services in south London to examine whether electronic questionnaires are completed more readily than paper-based questionnaires over a 3-month period. Follow up process evaluation phone calls with a subset (n = 8) of caregivers explored system satisfaction and usability.

RESULTS

MHE group assignment was significantly associated with an increased probability of completing an SDQ-P in the study period (adjusted hazard ratio (HR) 12.1, 95% CI 4.7-31.0; p = <.001). Of those caregivers' who received the MHE invitation (n = 68) 69.1% completed an SDQ using the platform compared to 8.8% in the control group (n = 68). The system was well received by caregivers, who cited numerous benefits of using MHE, for example, real-time feedback and ease of completion.

CONCLUSIONS

MHE holds promise for improving PROM completion rates. Research is needed to refine MHE, evaluate large-scale MHE implementation, cost effectiveness and explore factors associated with differences in electronic questionnaire uptake.

Use of the Strengths and Difficulties Questionnaire in child and school health services among children aged 4 and 6 years in Southern Norway: clinical considerations

BACKGROUND

Parent reported mental health can be assessed by the Strengths and Difficulties Questionnaire (SDQ). Currently, Norwegian norms for parent-reported SDQ do not exist, whereas Swedish, Danish, and United Kingdom (UK) norms have been published. We aimed to (1) describe parent-reported SDQ among children aged 4 and 6 years in Southern Norway, (2) evaluate empirical cutoff values within the context of the Starting Right^TM project in relation to the Swedish, Danish, and UK cutoffs, and (3) evaluate the representativeness of the study sample with regard to parental socioeconomic status.

METHODS

This study included parent-reported observations for 665 children (63% consent rate). Means and standard deviations were calculated for the domains of SDQ, and gender differences were assessed. Based on the Swedish, Danish, and UK cutoffs and the 80^th and 90^th percentile cutoff values within the study, we calculated the total number of children with borderline and abnormal scores.

RESULTS

Boys had higher mean total difficulties (7.3 vs 5.6) and impact scores (0.3 vs 0.1) and lower prosocial scores (8.3 vs 8.8) than girls. The differences in means were largest in the case of externalizing symptoms (5.0 vs 3.6) and hyperactivity subscore (3.2 vs 2.3). Using the UK cutoff values, 28 and 25 children had borderline and abnormal total difficulties scores, respectively. The corresponding numbers using the within study or Scandinavian cutoff values were 84-99 and 54-79, respectively. Overall, our study sample was well representative of the target population.

CONCLUSIONS

Our findings consistently indicated that girls had better SDQ scores than boys among children aged 4 and 6 years. Fewer children would be identified as having mental health difficulties using the UK cutoff values than using the Scandinavian age- and gender-relevant cutoff values.

Moving from development to implementation of digital innovations within the NHS: myHealthE, a remote monitoring system for tracking patient outcomes in child and adolescent mental health services

Objective

This paper aims to report our experience of developing, implementing, and evaluating myHealthE (MHE), a digital innovation for Child and Adolescents Mental Health Services (CAMHS), which automates the remote collection and reporting of Patient-Reported Outcome Measures (PROMs) into National Health Services (NHS) electronic healthcare records.

Methods

We describe the logistical and governance issues encountered in developing the MHE interface with patient-identifiable information, and the steps taken to overcome these development barriers. We describe the application's architecture and hosting environment to enable its operability within the NHS, as well as the capabilities needed within the technical team to bridge the gap between academic development and NHS operational teams.

Results

We present evidence on the feasibility and acceptability of this system within clinical services and the process of iterative development, highlighting additional functions that were incorporated to increase system utility.

Conclusion

This article provides a framework with which to plan, develop, and implement automated PROM collection from remote devices back to NHS infrastructure. The challenges and solutions described in this paper will be pertinent to other digital health innovation researchers aspiring to deploy interoperable systems within NHS clinical systems.

Experiences of Norwegian child and school health nurses with the "Starting Right™" child health assessment innovation: a qualitative interview study

Background

Although child health services are well established in Norway, the use of information technology for the systematic collection of evidence-based child- and proxy-reported health measures may be beneficial in the early identification of child development problems. The Norwegian “Starting Right™” health service innovation consists of parent- and child-reported online structured health assessments tools, including practical routines for child and school health assessments. The aim of this study was to explore the experiences of child and school health nurses with the Starting Right innovation.

Methods

We used a qualitative design and conducted three focus group interviews with 18 child and school health nurses from three child health centres one year after the implementation of the innovation.

Results

The experiences of professionals with the Starting Right innovation were captured by three themes: (1) the digital innovation could be used to obtain a good overview of a child’s health and development; (2) interpreting the questionnaires was a challenge; and (3) implementing the new digital innovation was time-consuming.

Conclusions

Overall, the child and school health nurses experienced that the Starting Right innovation was useful for providing a comprehensive overview of child development and health. The challenges related to interpreting the parents’ scores and follow-up of children, as well as providing the questionnaires in relevant foreign languages, should be addressed to allow all children and families to be reached.

Implementation facilitation strategies to promote routine progress monitoring among community therapists

Despite substantial support for the importance of routine progress monitoring (RPM) as part of evidence-based practice, few providers utilize measurement-based care. This study sought to identify the relative importance of facilitation strategies viewed as most helpful for increasing intention to use RPM among 388 ethnically diverse community therapists serving children and families. Four types of facilitation strategies were examined: language/interpretability, automation, staffing/access, and requirements. Mixed analyses of variance found that therapists' reported intentions to use RPM were more influenced by strategies of automating assessment administration, provision of clerical assistance, and agency requirements than by making linguistically appropriate measures available. However, the importance of strategies differed depending on therapist race/ethnicity and current RPM use. Language/interpretability of RPM assessments was less emphasized for non-Hispanic White therapists and therapists who have not yet or only minimally adopted RPM compared with ethnic minority therapists and therapists who regularly use RPM, respectively. Furthermore, therapists who were not current RPM users emphasized automation more than staffing/access. Results may inform prioritization of implementation facilitation strategies for agencies to encourage RPM. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Monitoring and Measurement in Child and Adolescent Mental Health: It's about More than Just Symptoms

Routine outcome monitoring (ROM) provides information to practitioners and others providing healthcare support to demonstrate the impact of interventions and for service evaluation [...].

Evaluation of an Online System for Routine Outcome Monitoring: Cross-sectional Survey Study

BACKGROUND

The use of routine outcome monitoring (ROM) in the treatment of mental health has emerged as a method of improving psychotherapy treatment outcomes. Despite this, very few clinicians regularly use ROM in clinical practice. Online ROM has been suggested as a solution to increase adoption.

OBJECTIVE

The aim of this study is to identify the influence of moving ROM online on client completion rates of self-reported outcome measures and to identify implementation and utilization barriers to online ROM by assessing clinicians' views on their experience using the online system over previous paper-based methods.

METHODS

Client completion rates of self-reported outcome measures were compared pre- and postimplementation of an online system of ROM. In addition, a survey questionnaire was administered to 324 mental health service providers regarding their perception of the benefits with an online system of ROM.

RESULTS

Client completion rates of self-reported measures increased from 15.62% (427/2734) to 53.98% (1267/2347) after they were moved online. Furthermore, 57% (56/98) of service providers found the new system less time-consuming than the previous paper-based ROM, and 64% (63/98) found that it helped monitor clients. However, the perceived value of the system remains in doubt as only 23% (23/98) found it helped them identify clients at risk for treatment failure, and only 18% (18/98) found it strengthened the therapeutic alliance.

CONCLUSIONS

Although the current study suggests mixed results regarding service providers' views on their experience using an online system for ROM, it has identified barriers and challenges that are actionable for improvement.

Making sense of child and adolescent mental health services (CAMHS): An audit of the referral journey and the use of routine outcome measures (ROMS)

BACKGROUND

There is increased emphasis on the national reporting of Routine Outcome Measures (ROMS) as a way of improving Child and Adolescent Mental Health Services (CAMHS). This data needs to be viewed in context so that reasons for outcome completion rates are understood and monitored over time.

METHOD

We undertook an in-depth prospective audit of consecutive referrals accepted into the Bath and North East Somerset, Swindon and Wiltshire (BSW) CAMHS service from November 2017 to January 2018 (n = 1074) and April to September 2019 (n = 1172).

RESULTS

Across both audits 90% of those offered an appointment were seen with three quarters completing baseline ROMS. One in three were not seen again with around 30% still being open to the service at the end of each audit. Of those closed to the service, paired ROMS were obtained for 46% to 60% of cases. There were few changes in referral problems or complexity factors over time.

CONCLUSION

Understanding the referral journey and the reasons for attrition will help to put nationally collected data in context and can inform and monitor service transformation over time.

Implementation of the norwegian 'Starting right' child health service innovation: implementation adjustments, adoption, and acceptability

BACKGROUND

An increased and/or stable proportion of the child and adolescent population reports symptoms of impaired health, and the symptoms can be identified early. Therefore, structured child- and parent-reported outcome measures need to be implemented in child and school health services for decision support and identification of children at risk. We aimed to (a) qualitatively examine adjustments of active implementation from the pilot implementation of the Norwegian 'Starting Right' health service innovation including an online child health assessment tool and practical routines, and (b) measure practitioners´ adoption and parental acceptability.

METHODS

We used a mixed-methods design to qualitatively examine adjustments from working notes and meeting memoranda, and quantitatively assess adoption and acceptability from user rates provided by the systems log. Twenty-one child and school health nurses (CSHNs) from two child health centers participated in the implementation pilot of online health assessments in children aged 2-, 4- and 6-year. We used a deductive and narrative analysis approach using Fixsen et al.´s core implementation components to code and sort adjustments.

RESULTS

Core implementation components were adjusted throughout the pilot implementation. Researchers´ increased their availability in reciprocity with staff evaluation to integrate active implementation adjustments. We launched a project for improved data systems integration. The overall CSHNs adoption rate was satisfactory and higher in center A, where a medical secretary supported the nurses through the entire pilot phase, than in center B (96 vs. 55 %). Parental acceptability rate was overall high (77 %) with increased rates among parents of 6-year-old children (98 %) compared with younger ones (78-85 %), and in cases where both parents received the questionnaires.

CONCLUSIONS

The 'Starting Right' health service innovation implementation was actively adjusted by integration of core implementation components mainly based on staff evaluation. The CSHNs adopted the innovation which was also acceptable to parents.

Challenges in Investigating the Effective Components of Feedback from Routine Outcome Monitoring (ROM) in Youth Mental Health Care

BackGround

Studies on feedback in youth mental health care are scarce and implementation of feedback into clinical practice is problematic.

Objective

To investigate potentially effective components of feedback from Routine Outcome Monitoring (ROM) in youth mental health care in the Netherlands through a three-arm, parallel-group, randomized controlled trial in which a literature-based, multi-faceted implementation strategy was used.

Method

Participants were randomly allocated to three conditions (basic feedback about symptoms and quality of life; basic feedback supplemented with clinical support tools; discussion of the feedback of the second condition with a colleague while following a standardized format for case consultation) using a block randomization procedure, stratified by location and participants’ age. The youth sample consisted of 225 participants (mean age = 15.08 years; 61.8% female) and the parent sample of 234 mothers and 54 fathers (mean age of children = 12.50 years; 47.2% female). Primary outcome was symptom severity. Secondary outcomes were quality of life and end-of-treatment variables. Additionally, we evaluated whether being Not On Track (NOT) moderated the association between condition and changes in symptom severity.

Results

No significant differences between conditions and no moderating effect of being NOT were found. This outcome can probably be attributed to limited power and implementation difficulties, such as infrequent ROM, unknown levels of viewing and sharing of feedback, and clinicians’ poor adherence to feedback conditions.

Conclusions

The study contributes to our limited knowledge about feedback from ROM and underscores the complexity of research on and implementation of ROM within youth mental health care.

Trial registration Dutch Trial Register NTR4234 .