Implementing a Measurement Feedback System: A Tale of Two Sites

Randomized Trial of an Organizational Implementation Strategy to Improve Measurement-Based Care Fidelity and Youth Outcomes in Community Mental Health

OBJECTIVE

Measurement-based care (MBC), which collects session-by-session symptom data from patients and provides clinicians with feedback on treatment response, is a highly generalizable evidence-based practice with significant potential to improve the outcomes of mental health treatment in youth when implemented with fidelity; however, it is rarely used in community settings. This study tested whether an implementation strategy targeting organizational leadership and organizational implementation climate could improve MBC fidelity and clinical outcomes for youth in outpatient mental health clinics.

METHOD

In a cluster randomized trial, 21 clinics were assigned to the Leadership and Organizational Change for Implementation strategy plus training and technical assistance in MBC (k = 11, n = 117) or training and technical assistance only (k = 10, n = 117). Primary outcomes of MBC fidelity (assessed via electronic metadata) and youth symptom improvement (assessed via caregiver-reported change on the Shortform Assessment for Children Total Problem Score) were collected for consecutively enrolled youths (ages 4-18 years) who initiated treatment in the 12 months following MBC training. Outcomes of each youth were assessed for 6 months following baseline.

RESULTS

A total of 234 youths were enrolled and included in intent-to-treat analyses. At baseline, there were no significant differences by condition in clinic, clinician, or youth characteristics. Youths in clinics using the Leadership and Organizational Change for Implementation strategy experienced significantly higher MBC fidelity compared with youths in control clinics (23.1% vs 3.4%, p = .014), and exhibited significantly greater reductions in symptoms from baseline to 6 months (d = 0.31, 95% CI: 0.04-0.58, p = .023).

CONCLUSION

Implementation strategies targeting organizational leadership and focused implementation climate can improve fidelity to evidence-based practices and clinical outcomes of youth mental health services.

PLAIN LANGUAGE SUMMARY

Measurement-based care is an evidence-based intervention that has been shown to improve the outcomes of mental health services in dozens of randomized trials; however, it is not widely used in routine practice. In this randomized controlled trial of 21 outpatient mental health clinics and 234 youth, the authors found an organizational intervention which trained clinic leaders in skills to support the use of measurement-based care increased the extent to which patients received measurement-based care compared to control clinics where clinic leaders did not receive this training (23% vs. 3%). Youth receiving services in the clinics that received the organizational intervention also had greater reductions in mental health symptoms.

CLINICAL TRIAL REGISTRATION INFORMATION

Working to Implement and Sustain Digital Outcome Measures (WISDOM); https://clinicaltrials.gov/; NCT04096274.

Prediction of adolescent psychotherapy outcomes using youth- and caregiver-reported symptoms data

OBJECTIVE

We used longitudinal youth- and caregiver-reports of adolescent psychological symptoms from three samples of youth receiving mental health services in routine treatment settings to derive expected change trajectories and identify cases at risk for treatment failure.

METHOD

Participants were 1906 youth (1053 caregivers) receiving treatment in community mental health settings, merged across three samples. The Symptoms and Functioning Severity Scale (SFSS) was used as an indicator of weekly clinical change. Multilevel modeling methods were used to develop expected change trajectories and identify cases at risk for treatment failure (not on track; NOT). Logistic regression was used to predict client improvement as a function of NOT status.

RESULTS

The SFSS was a reliable indicator of therapeutic change according to youth-reported symptoms. Caregiver reports were not as robust. Whereas predictive accuracy of NOT status yielded moderately high sensitivity in detecting improvement according to youth report, caregiver reports were not as predictive.

CONCLUSIONS

The youth-reported version of the SFSS-based algorithm seems appropriate for implementation in clinical care. Future studies should search for similarly predictive measures for caregivers.

Patient perspectives on systematic client feedback in Dutch outpatient mental healthcare, a qualitative case reports study

OBJECTIVE

The added value of systematic client feedback (SCF) to psychotherapy can be affected by patient perspectives, both in a positive and negative way, and is influenced by cultural factors as well. Current study explores patients' perspectives on use and optimization of SCF in Dutch outpatient mental healthcare. Primary aim of present study is to generate implications for daily practice and optimize SCF implementation, particularly for the Netherlands.

METHOD

Four patients suffering from mild to moderate psychological disorders were in-depth interviewed on their perspective on the use of SCF, when the Partners for Change Outcome Management System (PCOMS, high frequent), the Outcome Questionnaire (OQ-45, low frequent) and the Mental Health Continuum Short Form (MHC-SF, low frequent) was added to treatment as usual in two Dutch outpatient mental healthcare centers offering brief psychological treatment. Interview topics were (a) SCF in general; (b) type of questionnaires; (c) frequency of use; (d) effect of SCF on therapy; and (e) perceived added value of SCF. A SCF expert team analyzed the data through reflexive Thematic Analysis.

RESULTS

We identified three themes, all of which have two sub-themes: (a) Requirements to use SCF, with sub-themes (a1) Balance between effort to complete SCF and perceived validity, and (a2) Balance between used frequency and perceived validity; (b) Modifiers for test-taking attitude, with subthemes (b1) SCF as an embedded part of therapy, and (b2) Quality of Therapist-Client alignment; and (c) Effects on therapeutic process, with subthemes (c1) Focus on task and goals, and shared responsibility, and (c2) Effects on outcome and satisfaction.

CONCLUSIONS

Adding SCF to therapy can be perceived as helpful by patients in psychotherapy if two conditions are met: (1) Creating a right balance between effort and yield for SCF to be used at all; and (2) embedding SCF as an integral part of therapy, through therapist-client alignment. Throughout the progression of therapy, it might be useful to perceive SCF and therapy as communicating vessels; according to patients it is not only necessary to adjust therapy based on SCF, but also to adjust SCF based on the course of therapy.

TRIAL REGISTRATION

This trial was registered on September 30, 2015 in the Dutch Trial Register NTR5466. The Medical Ethics Committee of the University of Twente (Enschede) approved this study (registration number: K15-11, METC Twente).

Dose-Response Effects of MittEcho, a Measurement Feedback System, in an Indicated Mental Health Intervention for Children in Municipal and School Services in Norway

Including routine client feedback can increase the effectiveness of mental health interventions for children, especially when implemented as intended. Rate of implementation, or dose, of such feedback interventions has been shown to moderate results in some studies. Variation in implementation and use of client feedback may also contribute to the mixed results observed within the feedback literature. This study evaluates dose-response associations of client feedback using a novel Measurement Feedback System (MFS) within an indicated group intervention. The primary aim was to determine whether the rate of MFS implementation predicts symptom reduction in anxiety and depression among school-aged children. The secondary aim was to assess whether the rate of MFS implementation influences children's satisfaction with the group intervention or their dropout rates. Data were collected via a randomized factorial study (clinicaltrials.gov NCT04263558) across 58 primary schools in Norway. Children aged 8 to 12 years (N = 701) participated in a group-based, transdiagnostic intervention targeting elevated symptoms of anxiety or depression. Half of the child groups also received the feedback intervention using the MittEcho MFS. Group leaders (N = 83), recruited locally, facilitated the interventions. The MFS dose was measured using the Implementation Index, which combines the use of MFS by both children and providers (group leaders) into a single dose variable. Results showed no significant additional effect of dose of MFS on change in depression or anxiety scores, on user satisfaction with the intervention or on intervention dropout. The discussion addresses potential reasons for these non-significant findings and implications for MFS implementation in preventive, group-based interventions in school settings.

Using Progress Feedback to Enhance Treatment Outcomes: A Narrative Review

We face increasing demand for greater access to effective routine mental health services, including telehealth. However, treatment outcomes in routine clinical practice are only about half the size of those reported in controlled trials. Progress feedback, defined as the ongoing monitoring of patients' treatment response with standardized measures, is an evidence-based practice that continues to be under-utilized in routine care. The aim of the current review is to provide a summary of the current evidence base for the use of progress feedback, its mechanisms of action and considerations for successful implementation. We reviewed ten available meta-analyses, which report small to medium overall effect sizes. The results suggest that adding feedback to a wide range of psychological and psychiatric interventions (ranging from primary care to hospitalization and crisis care) tends to enhance the effectiveness of these interventions. The strongest evidence is for patients with common mental health problems compared to those with very severe disorders. Effect sizes for not-on-track cases, a subgroup of cases that are not progressing well, are found to be somewhat stronger, especially when clinical support tools are added to the feedback. Systematic reviews and recent studies suggest potential mechanisms of action for progress feedback include focusing the clinician's attention, altering clinician expectations, providing new information, and enhancing patient-centered communication. Promising approaches to strengthen progress feedback interventions include advanced systems with signaling technology, clinical problem-solving tools, and a broader spectrum of outcome and progress measures. An overview of methodological and implementation challenges is provided, as well as suggestions for addressing these issues in future studies. We conclude that while feedback has modest effects, it is a small and affordable intervention that can potentially improve outcomes in psychological interventions. Further research into mechanisms of action and effective implementation strategies is needed.

Optimizing indicated cognitive behavioral therapy to prevent child anxiety and depression: A cluster-randomized factorial trial

Identifying effective components can lead to interventions that are less resource-intensive and better suited for real-world needs. In this 2×2×2 cluster-randomized factorial trial (clinicaltrials.gov NCT04263558), we investigated the effects of three components of an indicated, transdiagnostic CBT intervention for children: 1) Intervention Delivery Format (child group format versus a blended format with group sessions and automated web-based sessions), 2) Parental Involvement in the intervention (group-based versus psychoeducational brochure), and 3) a Measurement Feedback System (MFS; on versus off). The intervention was delivered at schools in a group-based format. The participants (N = 701 children) were school children (age 8-12 years) with elevated symptoms of anxiety or depression, and their parents. The main outcomes were self-reported (N = 633) and parent-reported (N = 725) symptoms of child anxiety and depression post-intervention. The secondary outcome was children's user satisfaction with the intervention. We did not find significant main or interaction effects of Delivery Format, Parental Involvement, or MFS on children's symptom levels. There were no significant effects on children's user satisfaction. Results were compatible with retaining the least resource intensive combination (i.e., blended format, parental brochure, no MFS) in an optimized intervention.

Implementation Science and Practice-Oriented Research: Convergence and Complementarity

Implementation science is the scientific study of methods to promote the uptake of research findings and other evidence-based practices in routine care, with the goal of improving the quality and effectiveness of health services (Bauer et al., 2015). In addition to this common goal, practice-oriented psychotherapy research (and researchers) and implementation science (and scientists) share a common focus on the people and the places where treatment happens. Thus, there exists strong potential for combining these two approaches. In this article, we provide a primer on implementation science for psychotherapy researchers and highlight important areas and examples of convergence and complementarity between implementation science and practice-oriented psychotherapy research. Specifically, we (a) define and describe the core features of implementation science; (b) discuss similarities and areas of complementarity between implementation science and practice-oriented psychotherapy research; (c) discuss a case example that exemplifies the integration of implementation science and practice-oriented research; and (d) propose directions for future research and collaborations that leverage both implementation science and practice-oriented research.

Routine Outcome Monitoring and Clinical Feedback in Psychotherapy: Recent Advances and Future Directions

In the past decade, there has been an increase in research related to the routine collection and active use of standardized patient data in psychotherapy. Research has increasingly focused on personalization of care to patients, clinical skills and interventions that modulate treatment outcomes, and implementation strategies, all of which appear to enhance the beneficial effects of ROM and feedback. In this article, we summarize trends and recent advances in the research on this topic and identify several essential directions for the field in the short to medium term. We anticipate a broadening of research from the focus on average effects to greater specificity around what kinds of feedback, provided at what time, to which individuals, in what settings, are most beneficial. We also propose that the field needs to focus on issues of health equity, ensuring that ROM can be a vehicle for increased wellbeing for those who need it most. The complexity of mental healthcare systems means that there may be multiple viable measurement solutions with varying costs and benefits to diverse stakeholders in different treatment contexts, and research is needed to identify the most influential components in each of these contexts.

The Connections Matter: Bi-Directional Learning in Program Evaluation and Practice-Oriented Research

Intended for researchers and clinical leaders, this article suggests that embedded program evaluation is a good fit with the desired features of practice-oriented research. The systematic nature of evaluation that is built into the operational workflow of a practice setting may increase the diversity of methods available to explore processes and outcomes of interest. We propose a novel conceptual framework that uses a human-centered systems lens to foster such embedded evaluation in clinical routine. This approach emphasizes the evaluator-practitioner partnership to build confidence in the bi-directional learning of practice-based evidence with evidence-based practice. The iterative cycles inherent to design thinking are aimed at developing better evaluation questions. The attention to structure and context inherent to systems thinking is intended to support meaningful perspectives in the naturally complex world of health care. Importantly, the combined human-centered systems lens can create greater awareness of the influence of individual and systemic biases that exist in any endeavor or institution that involves people. Recommended tools and strategies include systems mapping, program theory development, and visual facilitation using a logic model to represent the complexity of mental health treatment for communication, shared understanding, and connection to the broader evidence base. To illustrate elements of the proposed conceptual framework, two case examples are drawn from routine outcome monitoring (ROM) and progress feedback. We conclude with questions for future collaboration and research that may strengthen the partnership of evaluators and practitioners as a community of learners in service of local and system-level improvement.

Measurement-Based Care in Youth: An Opportunity for Better Clinical Outcomes?

Measurement-based care (MBC) is a procedure in which systematic and routine assessments are performed. Through this practice, clinicians can verify the progress of the symptomatology of the patient and adapt the appointments and the intervention to the current symptoms. Studies have reflected on the importance and the benefits of this type of procedure in the adult population, and have shown positive results. However, there is a lack of evidence concerning the remaining populations. Regarding youth, for instance, few articles have evaluated the benefits of using this procedure in clinical practice. However, research focused on this topic has revealed positive results, especially when clinicians were loyal to the MBC procedures. Still, further research is needed. This letter aims to share the methodology used by our multidisciplinary team, composed of psychologists and psychiatrists, in a clinical context at the Hospital Cruz Vermelha, Lisboa, applied to the adult population; the objective is to share and discuss some alterations that could be made to our evaluation protocol to enable the same to be used with the youth population. We believe that implementing MBC for youth is crucial for several reasons, including enhanced treatment efficacy, more personalized treatment, a reduced reliance on subjectivity, and empowerment not only of patients but also families.

Data-Informed Communication: How Measurement-Based Care Can Optimize Child Psychotherapy

Measurement-based care (MBC) research and practice, including clinical workflows and systems to support MBC, are grounded in adult-serving mental health systems. MBC research evidence is building in child and adolescent services, but MBC practice is inherently more complex due to identified client age, the family system and the need to involve multiple reporters. This paper seeks to address a gap in the literature by providing practical guidance for youth-serving clinicians implementing MBC with children and their families. We focus on MBC as a data-informed, client-centered communication process, and present three key strategies to enhance usual care child and adolescent psychotherapy via developmentally-appropriate MBC. These strategies include (1) go beyond standardized measures; (2) lean into discrepancies; and (3) get curious together. Case-based examples drawn from various child-serving settings illustrate these key strategies of MBC in child psychotherapy.

Implementing Measurement-Based Care in a Youth Partial Hospital Setting: Leveraging Feedback for Sustainability

This paper describes the successful implementation of Measurement-Based Care (MBC) within a Partial Hospitalization Program (PHP) for children and adolescents. Measurement-based care (MBC), the practice of using patient-reported measures routinely to inform decision-making, is associated with improved clinical outcomes for behavioral health patients (Jong et al., Clinical Psychology Review 85, 2021; Fortney & Sladek, 2015). MBC holds great promise in partial hospital programs (PHP) to improve outcomes, yet implementation strategies are as complex as the setting itself. This paper provides a case study example of MBC implementation in a PHP for youth. Over the course of 18 months, an interdisciplinary staff of approximately 20 behavioral health professionals provided partial hospitalization level of care to 633 (39% in-person, 61% telehealth) youth from ages 5 to 18 years old. MBC in this setting incorporated daily patient self-report and weekly caregiver-reported measurements. This descriptive reconstruction, which includes examples of the data that were used during the implementation project, illustrates specific barriers and facilitators in a successful implementation in the local PHP setting. Implementation strategies to address workflow integration, leadership and supervision, and coaching are described, including evolution of these strategies over the course of implementation. Practical considerations for implementing MBC in youth PHP settings are discussed. Finally, the authors explore the potential relationships between the data-driven MBC model of decision making and the development of resilient organizations.

Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial

BACKGROUND

Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity.

METHODS

Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses.

RESULTS

LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045).

CONCLUSIONS

LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.

Process and Implementation Elements of Measurement Feedback Systems: A Systematic Review

Measurement feedback systems (MFS) can help guide treatment and improve clinical outcomes. Studies of MFS are heterogeneous both in execution and results, and the effects of MFS seem restricted by limited attention to process and implementation elements and by limited adoption by health professionals. The current systematic review mapped the use of process and implementation elements in MFS studies. An overview of therapists' use of and attitudes toward MFS is provided. Three-level meta-analyses were used to test theoretically informed process and implementation elements as moderators of the effects of MFS. Hypotheses and general propositions from Clinical Performance Feedback Intervention Theory (CP-FIT) were used to organize the elements of the studies and were used as moderator variables. Previous studies on MFS interventions have had a limited focus on implementation efforts and process elements that may increase the effects of MFS and their use among therapists. Efforts have sparsely been made to reduce barriers to MFS use, and several studies have reported limited engagement with MFS among therapists. Therapists' attitudes toward MFS, feedback, or standardized measures were heterogeneously reported, making data synthesis challenging. Identified process and implementation elements were not significantly associated with effect sizes in the studies and the results did not support the propositions of CP-FIT. The lack of statistically significant associations may be due to limited reporting of details about process and implementation aspects. More research designed to test hypotheses regarding process and implementation elements is needed to improve the use and effects of MFS. Future studies should aspire to report findings in a manner that allows for an understanding of the implementation process and therapists' adoption of these systems.

Consultation as an Implementation Strategy to Increase Fidelity of Measurement-Based Care Delivery in Community Mental Health Settings: An Observational Study

Regularly administering outcome measures to clients to inform clinical decision making, referred to as measurement-based care (MBC), has the potential to improve mental health treatment due to its applicability across disorders and treatment settings. Given the utility of MBC, understanding predictors of high-fidelity MBC implementation is important. Training and consultation are widely used implementation strategies to increase the fidelity of evidence-based practice delivery but have rarely been studied with MBC. The current study will examine the relationship between time clinicians spent discussing a case in consultation ("dosage") and MBC fidelity. Thirty clinicians and 56 youth were in the MBC condition of a randomized controlled trial and completed baseline questionnaires. Consultation dosage was extracted from call notes. MBC fidelity was measured using the implementation index, which combines rates of administering and viewing questionnaires, using objective data from the online MBC system. Multi-level modeling was used. Greater consultation dosage significantly predicted a higher implementation index (ß = 0.27, SE = 0.06, p < .001). For every 30 min spent discussing a case in consultation, the case's MBC fidelity increased by 8.1%. Greater consultation dosage significantly predicted higher rates of administration (ß = 0.07, SE = 0.03, p = 0.033) and feedback report viewing (ß = 0.24, SE = 0.06, p < .001). More consultation at the case level predicts greater MBC fidelity using objective measures. The greatest impact was increasing the clinician feedback report viewing rate. The results of this study can help inform future efforts to increase the fidelity with which MBC is delivered and to make consultation as efficient and effective as possible.

Study protocol: Novel Methods for Implementing Measurement-Based Care with youth in Low-Resource Environments (NIMBLE)

BACKGROUND

For youth receiving care in community mental health centers, comorbidities are the rule rather than the exception. Using measurement-based care (MBC), or the routine evaluation of symptoms to inform care decisions, as the foundation of treatment for youth with comorbid problems significantly improves the impact of psychotherapy by focusing care and building engagement and alliance. MBC increases the rate of symptom improvement, detects clients who would otherwise deteriorate, and alerts clinicians to non-responders. Despite its demonstrated utility, MBC is rarely implemented with fidelity; less than 15% of providers report using MBC per recommendations. Previous efforts to support MBC implementation have yielded suboptimal outcomes, in part, due to organizations' challenges with identifying and prioritizing barriers and selecting and developing strategies to overcome them. New methods are needed for identifying and prioritizing barriers, and matching strategies to barriers to optimize MBC implementation and treatment quality to improve youth mental health outcomes in community settings.

METHODS

Pragmatic implementation methods will be piloted in four diverse community mental health centers. Methods include (a) rapid evidence synthesis; (b) rapid ethnography; (c) design kits (e.g., kits with disposable cameras, journals, maps); (d) barrier prioritization, and (e) causal pathway diagramming. These activities will generate actionable barriers; subsequently, we will use facilitated group processes to prioritize barriers and develop causal pathway diagrams to match strategies to barriers to create implementation plans that optimize MBC fidelity (Aim 1). We will track strategy deployment for 6 months, then compare MBC fidelity for another 6 months post-implementation with data from 2 years of historical controls (Aim 2). Finally, we will co-design a toolkit for design kit methods with youth and the practice and scientific communities (Aim 3).

DISCUSSION

Optimizing MBC implementation in community mental health centers could transform youth mental health care by ensuring the most pressing symptoms are targeted early in treatment. The discussion section highlights expected challenges and limits to using the five methods, including recruitment and engagement given the high pressure on community mental health settings.

TRIAL REGISTRATION

Clinicaltrials.gov. NCT05644756 . Registered on 18 November 2022. This trial was retrospectively registered.

A Clinical Leadership Lens on Implementing Progress Feedback in Three Countries: Development of a Multidimensional Qualitative Coding Scheme

BACKGROUND

Progress feedback, also known as measurement-based care (MBC), is the routine collection of patient-reported measures to monitor treatment progress and inform clinical decision-making. Although a key ingredient to improving mental health care, sustained use of progress feedback is poor. Integration into everyday workflow is challenging, impacted by a complex interrelated set of factors across patient, clinician, organizational, and health system levels. This study describes the development of a qualitative coding scheme for progress feedback implementation that accounts for the dynamic nature of barriers and facilitators across multiple levels of use in mental health settings. Such a coding scheme may help promote a common language for researchers and implementers to better identify barriers that need to be addressed, as well as facilitators that could be supported in different settings and contexts.

METHODS

Clinical staff, managers, and leaders from two Dutch, three Norwegian, and four mental health organizations in the USA participated in semi-structured interviews on how intra- and extra-organizational characteristics interact to influence the use of progress feedback in clinical practice, supervision, and program improvement. Interviews were conducted in the local language, then translated to English prior to qualitative coding.

RESULTS

A team-based consensus coding approach was used to refine an a priori expert-informed and literature-based qualitative scheme to incorporate new understandings and constructs as they emerged. First, this hermeneutic approach resulted in a multi-level coding scheme with nine superordinate categories and 30 subcategories. Second-order axial coding established contextually sensitive categories for barriers and facilitators.

CONCLUSIONS

The primary outcome is an empirically derived multi-level qualitative coding scheme that can be used in progress feedback implementation research and development. It can be applied across contexts and settings, with expectations for ongoing refinement. Suggestions for future research and application in practice settings are provided. Supplementary materials include the coding scheme and a detailed playbook.

The effect of therapist characteristics on the use and outcome of systematic client feedback in outpatient mental healthcare

OBJECTIVE

Therapist characteristics are known to affect treatment outcome in general and could also influence the use of systematic client feedback (SCF). The current study explores the effect of feedback orientation, regulatory focus, self-efficacy, attitude towards feedback resources and perceived feedback validity on the use and outcome of SCF in outpatient mental healthcare.

METHOD

The data of therapists (n = 12) and patients (n = 504) of two outpatient centres offering brief psychological treatment were analysed when SCF, based on the Partners for Change Outcome Management System (PCOMS), was added to treatment as usual. The data of therapists were obtained through a therapist questionnaire composed of relevant characteristics from feedback studies in social and organizational psychology. The effect on the use of SCF was analysed using logistic regression; whereas, the effect on outcome was assessed using a two-level multilevel analysis. Regular use of SCF and the Outcome Questionnaire (OQ-45) were used as outcome variables. DSM-classification, sex and age of each patient were included as covariates.

RESULTS

High perceived feedback validity significantly increased the use of SCF. No significant therapist characteristics effects were found on outcome, but high promotion focus was associated with treating more complex patients.

CONCLUSIONS

The perceived feedback validity of SCF is likely to have an influence on its use and is probably affected by the changes in the organizational climate.

Caregiver Engagement in Outpatient Treatment for Adolescents in Community Settings: Construct and Predictive Validity of Family Therapy Techniques

Caregiver engagement and participation in community-based outpatient treatment services for adolescents is generally poor, which is problematic given the integral role of caregivers prescribed by evidence-based treatments across orientations. The current study explores the psychometric and predictive properties of a set of caregiver engagement techniques distilled from family therapy, used by community clinicians in routine care. It highlights relational engagement interventions and adds to growing work distilling core elements of family therapy. The study examined caregiver engagement techniques observed in 320 recorded sessions and outcome data from 152 cases treated by 45 therapists participating in one of three randomized trials investigating delivery of family therapy for adolescent behavior problems in community settings. Construct and predictive validity of caregiver engagement coding items were analyzed to understand the degree to which they cohered as a single factor and predicted outcomes in predictable ways. Results demonstrated item reliability and construct validity of a Caregiver Engagement Techniques factor. Greater use of these techniques was associated with decreased adolescent substance use. Unexpected results suggested greater use of techniques was associated with worsening internalizing symptoms and family cohesion per youth-report only. Post-hoc analyses revealed additional complexities in the association between engagement techniques and outcomes. Caregiver engagement practices tested in the current study represent a unified treatment factor that may contribute to positive therapeutic outcomes for adolescents in some clinical domains. Further research is needed to understand predictive effects.

Taking a Magnifying Glass to Measurement-Based Care Consultation Sessions: with What Issues Do Mental Health Clinicians Struggle?

PURPOSE

Ongoing consultation following initial training is one of the most commonly deployed implementation strategies to facilitate uptake of evidence-based practices, such as measurement-based care (MBC). Group consultation provides an interactive experience with an expert and colleagues to get feedback on actual issues faced, yet there is little research that unpacks the questions raised in consultation and what types of issues are important to address.

METHODS

The current study characterized the questions and concerns raised by community mental health clinicians (N = 38 across six clinics) during group consultation sessions completed as part of an MBC implementation trial. We conducted a qualitative content analysis of consultation forms completed by clinicians before each MBC consultation session.

RESULTS

Clinicians sought MBC consultation for clients across a range of ages and levels of depression severity. Qualitative results revealed five main questions and concerns in consultation sessions: (1) how to administer the PHQ-9, (2) how to review PHQ-9 scores, (3) how to respond to PHQ-9 score, (4) the types of clients for whom MBC would be appropriate, and (5) how MBC could impact a clinician's usual care.

CONCLUSION

Findings highlight the need for ongoing consultation and limitations of workshop training alone. Practical recommendations for addressing the common questions and concerns identified are presented to support MBC use.

Improving measurement-based care implementation in adult ambulatory psychiatry: a virtual focus group interview with multidisciplinary healthcare professionals

BACKGROUND

Measurement-Based Care (MBC) is an evidence-based practice shown to enhance patient care. Despite being efficacious, MBC is not commonly used in practice. While barriers and facilitators of MBC implementation have been described in the literature, the type of clinicians and populations studied vary widely, even within the same practice setting. The current study aims to improve MBC implementation in adult ambulatory psychiatry by conducting focus group interviews while utilizing a novel virtual brainwriting premortem method.

METHODS

Semi-structured focus group interviews were conducted with clinicians (n = 18) and staff (n = 7) to identify their current attitudes, facilitators, and barriers of MBC implementation in their healthcare setting. Virtual video-conferencing software was used to conduct focus groups, and based on transcribed verbatin, emergent barriers/facilitators and four themes were identified. Mixed methods approach was utilized for this study. Specifically, qualitative data was aggregated and re-coded separately by three doctoral-level coders. Quantitative analyses were conducted from a follow-up questionnaire surveying clinician attitudes and satisfaction with MBC.

RESULTS

The clinician and staff focus groups resulted in 291 and 91 unique codes, respectively. While clinicians identified a similar number of barriers (40.9%) and facilitators (44.3%), staff identified more barriers (67%) than facilitators (24.7%) for MBC. Four themes emerged from the analysis; (1) a description of current status/neutral opinion on MBC; (2) positive themes that include benefits of MBC, facilitators, enablers, or reasons on why they conduct MBC in their practice, (3) negative themes that include barriers or issues that hinder them from incorporating MBC into their practice, and (4) requests and suggestions for future MBC implementation. Both participant groups raised more negative themes highlighting critical challenges to MBC implementation than positive themes. The follow-up questionnaire regarding MBC attitudes showed the areas that clinicians emphasized the most and the least in their clinical practice.

CONCLUSION

The virtual brainwriting premortem focus groups provided critical information on the shortcomings and strengths of MBC in adult ambulatory psychiatry. Our findings underscore implementation challenges in healthcare settings and provide insight for both research and clinical practice in mental health fields. The barriers and facilitators identified in this study can inform future training to increase sustainability and better integrate MBC with positive downstream outcomes in patient care.

The results of clinician-focused implementation strategies on uptake and outcomes of Measurement-Based Care (MBC) in general mental health care

BACKGROUND

Measurement-Based Care (MBC) is the routine administration of measures, clinicians' review of the feedback and discussion of the feedback with their clients, and collaborative evaluation of the treatment plan. Although MBC is a promising way to improve outcomes in clinical practice, the implementation of MBC faces many barriers, and its uptake by clinicians is low. The purpose of this study was to investigate whether implementation strategies that were developed with clinicians and aimed at clinicians had an effect on (a) clinicians' uptake of MBC and (b) clients' outcomes of MBC.

METHODS

We used an effectiveness-implementation hybrid design based on Grol and Wensing's implementation framework to assess the impact of clinician-focused implementation strategies on both clinicians' uptake of MBC and outcomes obtained with MBC for clients in general mental health care. We hereby focused on the first and second parts of MBC, i.e., the administration of measures and use of feedback. Primary outcome measures were questionnaire completion rate and discussion of the feedback with clients. Secondary outcomes were treatment outcome, treatment length, and satisfaction with treatment.

RESULTS

There was a significant effect of the MBC implementation strategies on questionnaire completion rate (one part of clinicians' uptake), but no significant effect on the amount of discussion of the feedback (the other part of clinicians' uptake). Neither was there a significant effect on clients' outcomes (treatment outcome, treatment length, and satisfaction with treatment). Due to various study limitations, the results should be viewed as exploratory.

CONCLUSIONS

Establishing and sustaining MBC in real-world general mental health care is complex. This study helps to disentangle the effects of MBC implementation strategies on differential clinician uptake, but the effects of MBC implementation strategies on client outcomes need further examination.

A Systematic Review and Meta-Analysis of Measurement Feedback Systems in Treatment for Common Mental Health Disorders

To investigate the effects of measurement feedback systems (MFSs) in therapy on mental health outcomes through a literature review and meta-analysis. Using a three-level modeling approach, we conducted a meta-analysis of all effect sizes from randomized controlled studies of MFSs used in the treatment of common mental health disorders. Eighty-two effect sizes were extracted from the thirty-one included studies. Analyses were performed to consider the post-treatment effects of the MFS-assisted treatment compared to treatment as usual. A separate analysis was done for the subgroup "not-on-track" patients as it is theorized that MFSs will be clinically useful because they make therapists aware of patients who fail to progress. MFSs had a significant effect on mental health outcomes (d = 0.14, 95% CI [0.082-0.206], p < .001). Further analysis found a larger effect in patients identified as less respondent to therapy, the "not-on-track" group (d = 0.29, 95% CI [0.114, 0.464], p = .003). Moderation analyses indicated that the type of outcome measurement and type of feedback system used, and whether it was used for a child and youth or adult population, influenced effect sizes. MFSs seem to have a small positive effect on treatment outcomes. The effects seem to be larger for "not-on-track" patients, the group of patients that would usually not benefit much from treatment.

Systematic Client Feedback in Youth Mental Health and Addiction Care: A Controlled Study Comparing Two Treatment Cohorts

INTRODUCTION

We investigated the value of systematic client feedback in youth mental health and addiction care. In the present study, we examined whether a client feedback intervention would result in improved therapeutic alliance and treatment outcomes.

METHODS

Two hundred and four adolescents participated in the study using a - non-randomized - between-group A/B design. In the first study group, 127 patients were offered 4 months of treatment as usual (TAU), and in the second study group, 77 patients received the client feedback intervention as an add-on to TAU during 4 months.

RESULTS

Youths who received systematic client feedback in addition to TAU did not show better treatment outcomes or better alliance ratings after 4 months than youths receiving TAU only. Sensitivity analyses, in which we compared the more adherent patients of the second study group with patients receiving TAU, did not show significant beneficial effects of client feedback either. Also, the client feedback intervention did not result in lower rates of early treatment drop-out.

DISCUSSION/CONCLUSION

Our results cautiously suggest that client feedback does not have incremental effects on alliance and the treatment outcome for youth in mental health and addiction treatment. Moreover, our study highlights the challenges of implementing client feedback in clinical practice and the need for additional research addressing these challenges.

Moving from development to implementation of digital innovations within the NHS: myHealthE, a remote monitoring system for tracking patient outcomes in child and adolescent mental health services

Objective

This paper aims to report our experience of developing, implementing, and evaluating myHealthE (MHE), a digital innovation for Child and Adolescents Mental Health Services (CAMHS), which automates the remote collection and reporting of Patient-Reported Outcome Measures (PROMs) into National Health Services (NHS) electronic healthcare records.

Methods

We describe the logistical and governance issues encountered in developing the MHE interface with patient-identifiable information, and the steps taken to overcome these development barriers. We describe the application's architecture and hosting environment to enable its operability within the NHS, as well as the capabilities needed within the technical team to bridge the gap between academic development and NHS operational teams.

Results

We present evidence on the feasibility and acceptability of this system within clinical services and the process of iterative development, highlighting additional functions that were incorporated to increase system utility.

Conclusion

This article provides a framework with which to plan, develop, and implement automated PROM collection from remote devices back to NHS infrastructure. The challenges and solutions described in this paper will be pertinent to other digital health innovation researchers aspiring to deploy interoperable systems within NHS clinical systems.

Measurement feedback system implementation in public youth mental health treatment services: a mixed methods analysis

BACKGROUND

Prior studies indicate the effectiveness of measurement-based care (MBC), an evidence-based practice, in improving and accelerating positive outcomes for youth receiving behavioral health services. MBC is the routine collection and use of client-reported progress measures to inform shared decision-making and collaborative treatment adjustments and is a relatively feasible and scalable clinical practice, particularly well-suited for under-resourced community mental health settings. However, uptake of MBC remains low, so information on determinants related to MBC practice patterns is needed.

METHODS

Quantitative and qualitative data from N = 80 clinicians who implemented MBC using a measurement feedback system (MFS) were merged to understand and describe determinants of practice over three study phases. Quantitative, latent class analysis identified clinician groups based on participants' ratings of MFS acceptability, appropriateness, and feasibility and describes similarities/differences between classes in clinician-level characteristics (e.g., age; perceptions of implementation climate; reported MFS use; phase I). Qualitative analyses of clinicians' responses to open-ended questions about their MFS use and feedback about the MFS and implementation supports were conducted separately to understand multi-level barriers and facilitators to MFS implementation (phase II). Mixing occurred during interpretation, examining clinician experiences and opinions across groups to understand the needs of different classes of clinicians, describe class differences, and inform selection of implementation strategies in future research (phase III).

RESULTS

We identified two classes of clinicians: "Higher MFS" and "Lower MFS," and found similarities and differences in MFS use across groups. Compared to Lower MFS participants, clinicians in the Higher MFS group reported facilitators at a higher rate. Four determinants of practice were associated with the uptake of MBC and MFS in youth-serving community mental health settings for all clinicians: clarity, appropriateness, and feasibility of the MFS and its measures; clinician knowledge and skills; client preferences and behaviors; and incentives and resources (e.g., time; continuing educational support). Findings also highlighted the need for individual-level implementation strategies to target clinician needs, skills, and perceptions for future MBC and MFS implementation efforts.

CONCLUSION

This study has implications for the adoption of evidence-based practices, such as MBC, in the context of community-based mental health services for youth.

Characterizing measurement-based care implementation using therapist report

The effectiveness of measurement-based care (MBC), an evidence-based practice that uses regularly collected assessment data to guide clinical decision-making, is impacted by whether and how therapists use information from MBC tools in treatment. Improved characterization of how therapists use MBC in treatment sessions with youth is needed to guide implementation and understand variability in MBC effectiveness. To meet this need, this study examined therapists' sharing and discussion of MBC in treatment sessions. Thirty therapists were randomly assigned to the MBC condition as part of a comparative effectiveness trail of treatments for adolescent anxiety and depression. A qualitative content analysis was conducted on therapists' written explanations of changes made to the session based on the MBC data. Therapists reported sharing data with youth and caregivers in an average of 34.6% and 27.4% of sessions, respectively. Therapists reported incorporating MBC data in an average of 21.1% of sessions. When data were used, therapists predominately focused changes on short-term (e.g., current symptoms, treatment skill) rather than long-term (e.g., symptom progress, treatment goals) decision-making. Therapists inconsistently used MBC data, highlighting the need for improved training in and monitoring of how therapists use MBC in session to guide collaborative treatment decision-making with youth and caregivers.

To Utility and Beyond! Specifying and Advancing the Utility of Measurement-Based Care for Youth

Mental health organizations that serve youth are under pressure to adopt measurement-based care (MBC), defined as the continuous collection of client-report data used to support clinical decision-making as part of standard care. However, few frameworks exist to help leadership ascertain how to select an MBC approach for a clinical setting. This paper seeks to define how an MBC approach can display clinical utility to provide such a framework. Broadly, we define clinical utility as evidence that an MBC approach assists stakeholders in fulfilling clinical goals related to care quality (i.e., improve client-clinician alliance and clinical outcomes) at the client (i.e., youth and caregiver), clinician, supervisor, and administrator levels. More specifically, our definition of clinical utility is divided into two categories relevant to the usability and usefulness of an MBC approach for a specific setting: (a) implementability (i.e., evidence indicating ease of use in a clinical setting) and (b) usefulness in aiding clinical activities (i.e., evidence indicating the potential to improve communication and make clinical activities related to care quality easier or more effective). These categories provide valuable information about how easy an MBC approach is to use and the potential benefits that the MBC data will confer. To detail how we arrived at this definition, we review prior definitions of clinical utility, discuss how previous definitions inform our definition of clinical utility for MBC, and provide examples of how the concept of clinical utility can be applied to MBC. We finish with a discussion of future research directions.

Patients’ and Psychologists’ Preferences for Feedback Reports on Expected Mental Health Treatment Outcomes: A Discrete-Choice Experiment

In recent years, there has been an increasing focus on routine outcome monitoring (ROM) to provide feedback on patient progress during mental health treatment, with some systems also predicting the expected treatment outcome. The aim of this study was to elicit patients’ and psychologists’ preferences regarding how ROM system-generated feedback reports should display predicted treatment outcomes. In a discrete-choice experiment, participants were asked 12–13 times to choose between two ways of displaying an expected treatment outcome. The choices varied in four different attributes: representation, outcome, predictors, and advice. A conditional logistic regression was used to estimate participants’ preferences. A total of 104 participants (68 patients and 36 psychologists) completed the questionnaire. Participants preferred feedback reports on expected treatment outcome that included: (a) both text and images, (b) a continuous outcome or an outcome that is expressed in terms of a probability, (c) specific predictors, and (d) specific advice. For both patients and psychologists, specific predictors appeared to be most important, specific advice was second most important, a continuous outcome or a probability was third most important, and feedback that includes both text and images was fourth in importance. The ranking in importance of both the attributes and the attribute levels was identical for patients and psychologists. This suggests that, as long as the report is understandable to the patient, psychologists and patients can use the same ROM feedback report, eliminating the need for ROM administrators to develop different versions.

Measurement Feedback System for Intensive Neurorehabilitation after Severe Acquired Brain Injury

Outcome of acquired brain injury (ABI) and the potential for neurorehabilitation are subject to distinct heterogeneity between patients. Limited knowledge of the complex constellation of determinants at play interferes with the possibility to deploy precision medicine in neurorehabilitation. Measurement Feedback Systems (MFS) structure clinical data collection and deliver the measurement results as feedback to clinicians, thereby facilitating progress monitoring, promoting balanced patient-centered discussion and shared decision making. Accumulation of clinical data in the MFS also enables data-driven precision rehabilitation medicine. This article describes the development and implementation of a MFS for neurorehabilitation after ABI. The MFS consists of specialized measurement tracks which are developed together with representatives of each discipline in the multidisciplinary team. The MFS is built into a digital platform that automatically distributes measurements among clinicians, at predetermined time points during the inpatient treatment, outpatient treatment and follow-up. The results of all measurements are visualized in individual patient dashboards that are accessible for all clinicians involved in treatment. Since step-wise implementation, 124 patients have been registered on the MFS platform so far, providing an average of more than 200 new measurements per week. Currently, more than 15,000 clinical measurements are captured in the MFS. The current overall completion rate of measurements is 86,4%. This study shows that structured clinical assessment and feedback is feasible in the context of neurorehabilitation after severe ABI. The future directions are discussed for MFS data in our Health Intelligence Program, which aims at periodic care evaluation and the transition of neurorehabilitation care towards precision medicine.

Parenting interventions for families with refugee backgrounds: a randomized factorial, mixed-methods design study protocol

Background

Raising children in new social and cultural contexts can be challenging for parents. In order to help parents address these challenges, the Norwegian government has instituted a policy of mandatory parent training for families who settle in Norway as refugees. The Incredible Years (IY) and The International Child Development Programme (ICDP) have been widely adopted throughout Norway. They have similar aims: to improve parenting through positive parenting practices and development of attachment behaviors. We will evaluate the use of these programs and a measurement feedback system (MFS) to give regular feedback to interventionists about parents’ progress during the course of the parenting intervention.

Methods

The study is a mixed method, randomized factorial design aimed at evaluating the effect of parenting interventions and the use of feedback to address parental stress, child behavior, resilience, and parents’ mental health. Factor 1 is based on random assignment to one of the parenting interventions IY or ICDP. The parenting interventions are delivered over 15 weeks (IY) or 12 weeks (ICDP) in group-based settings. Factor 2 is based on random assignment of the parenting groups to the (a) with MFS or (b) without MFS condition. The MFS is answered weekly via a phone app, MittEcho, and results are sent to group leaders in the MFS condition. Additionally, the study explores the experiences of families settling in a new cultural context and participating with parenting programs via qualitative interviews. Participants will be recruited from a population of parents with children between the age of 6 and 12 years who settled in Norway as refugees within the previous 9 years. The target sample size is N = 360; n (IY) = 180, n (ICDP) = 180 families. This study is a collaboration between first-line, municipal services; their national governing agencies; family representatives; and a national network of research organizations.

Discussion

This study endeavors to provide information about what helps families with refugee background integrate successfully into new cultural contexts with different laws, norms, and expectations. Whether or not these interventions can help to normalize this experience, reduce stress, and provide parents with new tools to improve their parenting and the lives of their children are important questions which we address. These findings can lead to the further establishment of evidence-based practices in Norway.

Trial registration

ISRCTN35008070. Registered on February 24, 2020

Measurement-based care educational programmes for clinical trainees in mental healthcare: a scoping review protocol

INTRODUCTION

Measurement-based care (MBC) represents the approach of regularly using symptom rating scales to guide patient care decisions in mental healthcare. MBC is an effective, feasible and acceptable approach to enhance clinical outcomes in various disciplines, including medicine, psychology, social work and psychotherapy. Yet, it is infrequently used by clinicians, potentially due to limited education for care providers. The objective of this scoping review is to survey the characteristics of MBC educational programmes for undergraduate, graduate and postgraduate clinical trainees in mental healthcare.

METHODS AND ANALYSIS

Using database-tailored search strategies, we plan on searching Medline, PsycINFO, Embase, CINAHL and Cochrane Central for relevant studies. Thereafter, we will analyse the selected studies to extract information on the delivery of educational programmes, the clinical and educational outcomes of these programmes, and the potential enablers and barriers to MBC education. In this paper, we articulate the protocol for this scoping review.

ETHICS AND DISSEMINATION

This scoping review does not require research ethics approval. The findings from this scoping review will be incorporated into the creation of a novel MBC curriculum and handbook. Results will be disseminated at appropriate national or international conferences, as well as in a peer-reviewed journal publication.

A routine outcome measure for youth mental health: Clinically interpreting MyLifeTracker

AIM

MyLifeTracker is a session-by-session mental health outcome measure for young people aged 12 to 25 years. The aim of this study was to determine clinically significant change indexes for this measure that would identify developmentally appropriate thresholds. The study also aimed to determine expected change trajectories to enable clinicians to compare a client's progress against average rates of change.

METHODS

Participants comprised young people aged 12 to 25 years from both a clinical and a community sample from Australia. The clinical sample was 63 840 young people that attended a headspace centre. The non-clinical group was an Australian representative community sample of 4034 young people.

RESULTS

Clinically significant change indexes were developed for MyLifeTracker specific for age and gender groups by comparing clinical and non-clinical samples. Males and young people aged 12 to 14 years needed to reach higher scores to achieve clinically significant change compared to females and other age groups, respectively. MyLifeTracker expected change trajectories followed a cubic pattern for those with lower baseline scores of 0 to 50, whereas those with baseline scores of 51 and above had varying patterns. For those with lower baseline scores, expected change trajectories showed that stronger change was evident early in treatment, which then tapered off before accelerating again later in treatment.

CONCLUSIONS

The development of MyLifeTracker benchmarks allows the measure to be used for Feedback Informed Treatment by supporting treatment planning and decision-making. This information can help clinicians to identify clients who are not on track or deteriorating and identify when clients are improving.

Provider perceived barriers and facilitators to integrating routine outcome monitoring into practice in an urban community psychiatry clinic: A mixed-methods quality improvement project

BACKGROUND AND AIMS

Although mental health clinics are under increasing pressure to demonstrate value and routine outcome monitoring (ROM) has become a mandated component of care, providers have been slow to adopt ROM into practice, with some estimating that less than 20% of mental health clinicians use it consistently in the United States. This article explores perceived barriers and facilitators to integrating ROM into practice among clinicians and administrators in a large urban US community psychiatry clinic.

METHOD

One hundred and thirty-eight clinical and administrative staff were administered an anonymous web-based survey to elicit attitudes towards ROM. Responses were summarized descriptively and qualitatively synthesized into a conceptual model using inductive thematic analysis.

RESULTS

Common barriers to integration included insufficient time to collect and/or use measures, not knowing what measures to use, measures being difficult to access, and insufficient training. Facilitators included increased access/ease of use, training and support, measure relevance/validity, and accountability.

CONCLUSIONS

In order for psychiatry clinics to successfully implement ROM into practice, they must diagnose organization-side barriers and translate this knowledge into actionable quality improvement initiatives ranging from the infrastructural to the cultural.

Applying behaviour change theory to understand the barriers to implementing routine outcome monitoring

BACKGROUND

Routine outcome monitoring (ROM) is a valuable tool for monitoring client progress and pre-empting deterioration, however, there is considerable variation in how data are collected and recorded and uptake in clinical practice remains low. The aim of this study was to develop a self-report measure of practitioner attitudes to ROM in order to better understand the barriers to successful implementation in Child and Adolescent Mental Health Services (CAMHS).

METHODS

An anonymous survey was completed by 184 CAMHS practitioners in the United Kingdom. The survey was designed using the Capability, Opportunity, and Motivation Model of Behaviour (COM-B). Practitioners who reported using ROM frequently in their clinical work (53%) were compared to those who used ROM infrequently (47%) across dimensions of the COM-B survey subscales.

RESULTS

Confirmatory factor analysis confirmed the proposed four-factor structure, showing acceptable model fit, with high factor loadings and good reliability for all subscales. Frequent users of ROM exhibited significantly higher psychological capability, physical opportunity, social opportunity, and motivation, compared to infrequent users F (4, 140) = 14.76, p < .0001; Pillai's Trace = .297, partial η²  = .30. Results highlight several barriers to ROM, including the belief that there is not a strong evidence base for ROM, not receiving external training, and not discussing feedback and outcome data in supervision.

IMPLICATIONS

In the hope of improving the successful implementation of ROM, this research provides an evidence-based tool for assessing practitioners' attitudes to ROM, which map on to intervention functions and represent targets for future implementation efforts.

PRACTITIONER POINTS

The value of routine outcome monitoring (ROM) as a means to measure client progress and to elevate the efficiency and quality of mental health care is well-documented in the research literature, however, uptake in practice remains relatively low. This study applied behaviour change theory to develop a psychometrically sound self-report measure of practitioners' perspectives and practices to understand the barriers to implementation in child and adolescent mental health services in the United Kingdom. The complex and multifaceted nature of the barriers to implementation requires multilevel behaviour change strategies at the client, clinician, and organisational level. Recommendations for practice include the need for integrated, multilevel strategies aimed at improving practitioners' capabilities and motivations, strong organisational leadership and a culture of data gathering and sharing, and implementation interventions, which are tailored to target local barriers.

Inside the Clockwork of the ECHO Factorial Trial: A Conceptual Model With Proposed Mediators for Prevention of Emotional Problems in Children

Having interventions that are not only evidence-based and effective but also cost-effective and efficient is important for the prevention and treatment of child and adolescent emotional problems. A randomized clinical trial (RCT) tests the total interventions effect but does not address specific components of the intervention. In this article the hypothesis and a conceptual model of the ECHO study are presented and discussed. The ECHO intervention consists of three different components each containing two levels of intervention. By using a cluster randomized factorial design, children aged 8-12 at 40 schools across Norway will be randomized to eight different experimental conditions investigating the optimal balance between effect, cost-effectiveness, and efficiency. The article presents the design and the different components being tested and discusses how optimalization can be reached through this innovative design. The article also discusses how interventions can be improved by investigating and understanding the mechanisms of change within psychological interventions. For each of the three components in the study we consider the mediators that could be active within the intervention and how the study investigates such mediation. The results will contribute to a better understanding of how psychological interventions work and how we intend to optimize the EMOTION intervention.

Middle and High School Student Perspectives on Digitally-Delivered Mental Health Assessments and Measurement Feedback Systems

Implementation of measurement-based care (MBC) by child-serving community mental health providers, particularly school-based providers, is low. To inform user-centered design of measurement feedback systems (MFSs) and MBC implementation more broadly, semi-structured interviews were conducted with 61 middle and high school students. Interviews explored student preferences for and perceived helpfulness of different assessment methods and use of MFS in counseling. Results indicate that student preference for digitally-administered assessment is equivocal, with preferences being influenced by student perceptions of the ease of use, impersonalization, and confidentiality. Students with exposure to the MFS found it helpful when used by their provider to share assessment feedback.

MFA and ASA-MF: A Psychometric Analysis of Attitudes Towards Measurement-Based Care

Measurement based care (MBC) improves client outcomes by providing clinicians with routine mental health outcome data that can be used to inform treatment planning but is rarely used in practice. The Monitoring and Feedback Attitudes Scale (MFA) and Attitudes Towards Standardized Assessment Scales-Monitoring and Feedback (ASA-MF) (Jensen-Doss et al., 2016) may identify attitudinal barriers to MBC, which could help trainings and implementation strategies. This study examines the psychometric properties of the MFA and ASA-MF, including the factor structure, longitudinal invariance, and indicators of validity, in a sample of community mental health clinicians (N = 164). The measures demonstrate adequate fit to their factor structures across time and predict MBC use as captured in a client's electronic health record. Given that clinician attitudes are associated with MBC use, using instruments with psychometric support to assess attitudes fills a research to practice gap.

Exploration, Preparation, and Implementation of Standardized Assessment in a Multi-agency School Behavioral Health Network

School mental health treatment services offer broad public health impact and could benefit from more widespread implementation and sustainment of standardized assessments (SA). This demonstration study describes one approach to increase SA use in a large school behavioral health network using the Exploration, Preparation, Implementation and Sustainment (EPIS) framework. Mental health administrator interviews with four participating agencies and a multidisciplinary planning team informed SA measure selection and implementation supports. The SA initiative was implemented during one school year, including system-wide training and ongoing implementation supports for 95 clinicians. Linear mixed effect models revealed improvements in clinician attitudes about the SA for clinical utility and treatment planning immediately following the half-day training (N = 95, p < .001). Clinicians self-reported a significant increase in use of SA for new clients during intakes (p < .001) over time and 71.4% of expected SA data were submitted. Qualitative feedback, including recommendations to offer more SA choices and beginning new SA data collection earlier in the school year, was integrated to inform quality improvements and future sustainment efforts.

Assessing recovery in treatment as usual provided by community child and adolescent mental health services

BACKGROUND

Despite the importance of routinely assessing the outcomes of everyday practice, few studies have reported outcome metrics for child and adolescent mental health services (CAMHS).

AIMS

Our aim is to investigate reliable change and recovery rates for treatment as usual, provided by one community CAMHS over two time periods.

METHOD

We prospectively audited accepted consecutive referrals from November 2017 to January 2018, and April to September 2019. Cases with paired outcomes were identified, and reliable change and recovery rates were calculated.

RESULTS

Baseline outcome data were obtained for 672 (78.2%) and 744 (77.5%) young people in 2018 and 2019, respectively. Of eligible participants, 174 (59.2%) and 155 (45.7%) completed at least one follow-up outcome measure in 2018 and 2019, respectively. Pre- and post-test scores on the Revised Child Anxiety and Depression Scale (RCADS) and Strengths and Difficulties Questionnaire (SDQ) showed a reduction in symptoms. Total RCADS scores showed 21-25% of participants reliably improved, with 44-49% showing reliable improvement on one or more subscale. On the SDQ, 11 (15.5%) and 19 (25.3%) participants reported reliable improvement on at least one subscale in 2018 and 2019, respectively. Reliable recovery rates ranged from 48 to 51% for youth-completed and 40 to 42% for parent-completed RCADS.

CONCLUSIONS

Half of young people receiving treatment as usual from CAMHS reliably improved on at least one routine outcome measure subscale, improvement rates comparable with adult psychological therapies services. Our findings indicate that reliable change and recovery on subscale rather than total scores may be a better indication of outcomes.

Improving the patient-centred care of children with life-altering skin conditions using feedback from electronic patient-reported outcome measures: protocol for a hybrid effectiveness-implementation study (PEDS-ePROM)

INTRODUCTION

Using patient-reported outcome measures (PROMs) with children have been described as 'giving a voice to the child'. Few studies have examined the routine use of these measures as potentially therapeutic interventions. This study aims to investigate: (1) the effectiveness of feedback using graphical displays of information from electronic PROMs (ePROMs) that target health-related quality of life, to improve health outcomes, referrals and treatment satisfaction and (2) the implementation of ePROMs and graphical displays by assessing acceptability, sustainability, cost, fidelity and context of the intervention and study processes.

METHODS AND ANALYSIS

A hybrid II effectiveness-implementation study will be conducted from February 2020 with children with life-altering skin conditions attending two outpatient clinics at a specialist paediatric children's hospital. A pragmatic randomised controlled trial and mixed methods process evaluation will be completed. Randomisation will occur at the child participant level. Children or parent proxies completing baseline ePROMs will be randomised to: (1) completion of ePROMs plus graphical displays of ePROM results to treating clinicians in consultations, versus (2) completion of ePROMs without graphical display of ePROM results. The primary outcome of the effectiveness trial will be overall health-related quality of life of children. Secondary outcomes will include other health-related quality of life outcomes (eg, child psychosocial and physical health, parent psychosocial health), referrals and treatment satisfaction. Trial data will be primarily analysed using linear mixed-effects models; and implementation data using inductive thematic analysis of interviews, meeting minutes, observational field notes and study communication mapped to the Consolidated Framework for Implementation Research.

ETHICS AND DISSEMINATION

Ethical approval was obtained from Children's Health Queensland Human Research Ethics Committee (HREC/2019/QCHQ/56290), The University of Queensland (2019002233) and Queensland University of Technology (1900000847). Dissemination will occur through stakeholder groups, scientific meetings and peer-reviewed publications.

TRIAL REGISTRATION NUMBER

Australian New Zealand Clinical Trials Registry (ACTRN12620000174987).

Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: A multilevel meta-analysis

Progress feedback is an intervention aimed at enhancing patient outcomes in routine clinical practice. This study reports a comprehensive multilevel meta-analysis on the effectiveness of progress feedback in psychological treatments in curative care. The short- and long-term effects of feedback on symptom reduction were investigated using 58 (randomized and non-randomized) studies, analyzing 110 effect sizes in a total of 21,699 patients. Effects of feedback on dropout rate, percentage of deteriorated cases, and treatment duration were also examined. Moderation analyses were conducted for study and feedback characteristics. A small significant effect of progress feedback on symptom reduction (d = 0.15, 95% CI: [0.10, 0.20]) was found, compared to control groups. This was also true for not-on-track cases (d = 0.17, 95% CI: [0.11, 0.22]). In addition, feedback had a small favorable effect on dropout rates (OR = 1.19, 95% CI: [1.03, 1.38]). The moderation analyses identified several potentially interesting variables for further research, including feedback instrument, outcome instrument, type of feedback, feedback frequency, treatment intensity, and country in which the study was conducted. Future studies should report on these variables more consistently so that we can obtain a better understanding of when and why feedback improves outcomes.

Sociodemographic factors associated with routine outcome monitoring: a historical cohort study of 28,382 young people accessing child and adolescent mental health services

BACKGROUND

Patient-reported outcome measures (PROMs) are important tools to inform patients, clinicians and policy-makers about clinical need and the effectiveness of any given treatment. Consistent PROM use can promote early symptom detection, help identify unexpected treatment responses and improve therapeutic engagement. Very few studies have examined associations between patient characteristics and PROM data collection.

METHODS

We used the electronic mental health records for 28,382 children and young people (aged 4-17 years) accessing Child and Adolescent Mental Health Services (CAMHS) across four South London boroughs between the 1st of January 2008 to the 1st of October 2017. We examined the completion rates of the caregiver Strengths and Difficulties Questionnaire (SDQ), a ubiquitous PROM for CAMHS at baseline and 6-month follow-up.

RESULTS AND CONCLUSIONS

SDQs were present for approximately 40% (n = 11,212) of the sample at baseline, and from these, only 8% (n = 928) had a follow-up SDQ. Patterns of unequal PROM collection by sociodemographic factors were identified: males were more likely (aOR 1.07, 95% CI 1.01-1.13), whilst older age (aOR 0.87, 95% CI 0.87-0.88), Black (aOR 0.79 95% CI 0.74-0.84) and Asian ethnicity (aOR 0.75 95% CI 0.66-0.86) relative to White ethnicity, and residence within the most deprived neighbourhood (aOR 0.87 95% CI 0.80-0.94) were less likely to have a record of baseline SDQ. Similar results were found in the sub-group (n = 11,212) with follow-up SDQ collection. Our findings indicate systematic differences in the currently available PROMS data and highlights which groups require increased focus if we are to gain equitable PROM collection. We need to ensure representative PROM collection for all individuals accessing treatment, regardless of ethnic or socioeconomic background; biased data have adverse ramifications for policy and service level decision-making.

Factors affecting the implementation of an outcome measurement feedback system in youth mental health settings

Objective: Measurement feedback systems provide clinicians with regular snapshots of a client's mental health status, which can be used in treatment planning and client feedback. There are numerous barriers to clinicians using outcome measures routinely. This study aimed to investigate factors affecting the use of a measurement feedback system across youth mental health settings. Methods: The participants were 210 clinicians from headspace youth mental health services across Australia. They were surveyed on predictors and use of MyLifeTracker, a routine outcome measure. This was explored through three processes: looking at MyLifeTracker before session, using MyLifeTracker in treatment planning, and providing feedback of MyLifeTracker scores to clients. Results: Clinicians were more likely to look at MyLifeTracker before session, less likely to use it in treatment planning, and least likely to provide MyLifeTracker scores to clients. Each measurement feedback system process had a distinct group of predictors. Perceptions of MyLifeTracker's practicality was the only significant predictor of all three processes. Conclusion: Practically, organisations and supervisors can increase the use of measurement feedback systems through targeted supports.

Evidence Base for Measurement-Based Care in Child and Adolescent Psychiatry

Measurement-based care (MBC) is the routine collection of data using standardized, validated measures and use of these data to guide treatment. MBC has been implemented throughout medicine to improve patient outcomes, but its use in mental health care remains low. This article summarizes the evidence of MBC's efficacy in adults then reviews the sparser data in youth mental health care. The literature indicates that MBC must be administered immediately before or during every encounter, results must be available to and reviewed by providers immediately, results must be shared with the patient, and results must be used to guide treatment.

Improving measurement feedback systems for measurement-based care

Abstract Objectives: Measurement feedback systems (MFS) are a class of health information technologies developed to facilitate measurement-based care. The individual clinical decision support features within MFS are diverse and their influence on clinicians is largely unknown. This study tested the impact of MFS features on clinicians' progress assessments and treatment decisions in different scenarios. Method: Clinicians (n = 299) were randomly assigned to view one of six combinations of the following MFS features: graph, expected change trajectory line, alert, and treatment suggestions. The assigned feature combination was paired with three vignettes and clinical data representing three clinical scenarios: patient deterioration, no progress, and approaching remission. Clinicians answered questions after each vignette, and at the conclusion. Results: MFS features differentially impacted clinicians' progress assessment accuracy, their likelihood of making a treatment change, and their treatment choices. Which feature was most impactful varied depending on the clinical scenario. Clinicians reported graphs influenced their assessments and choices significantly more than the other features, and the majority stated they would prefer to use all of the features. Conclusions: Specific MFS features impact clinicians' assessments and choices to greater degrees, and the impact of those features can be influenced by the clinical state of the patient.

Challenges in Investigating the Effective Components of Feedback from Routine Outcome Monitoring (ROM) in Youth Mental Health Care

BackGround

Studies on feedback in youth mental health care are scarce and implementation of feedback into clinical practice is problematic.

Objective

To investigate potentially effective components of feedback from Routine Outcome Monitoring (ROM) in youth mental health care in the Netherlands through a three-arm, parallel-group, randomized controlled trial in which a literature-based, multi-faceted implementation strategy was used.

Method

Participants were randomly allocated to three conditions (basic feedback about symptoms and quality of life; basic feedback supplemented with clinical support tools; discussion of the feedback of the second condition with a colleague while following a standardized format for case consultation) using a block randomization procedure, stratified by location and participants’ age. The youth sample consisted of 225 participants (mean age = 15.08 years; 61.8% female) and the parent sample of 234 mothers and 54 fathers (mean age of children = 12.50 years; 47.2% female). Primary outcome was symptom severity. Secondary outcomes were quality of life and end-of-treatment variables. Additionally, we evaluated whether being Not On Track (NOT) moderated the association between condition and changes in symptom severity.

Results

No significant differences between conditions and no moderating effect of being NOT were found. This outcome can probably be attributed to limited power and implementation difficulties, such as infrequent ROM, unknown levels of viewing and sharing of feedback, and clinicians’ poor adherence to feedback conditions.

Conclusions

The study contributes to our limited knowledge about feedback from ROM and underscores the complexity of research on and implementation of ROM within youth mental health care.

Trial registration Dutch Trial Register NTR4234 .

Attitudes Towards the Use of Routine Outcome Monitoring of Psychological Therapies Among Mental Health Providers: The EBPAS-ROM

Implementation of routine outcome monitoring (ROM) in mental health care is progressing slowly. Knowledge about factors influencing ROM implementation, including health providers' attitudes towards ROM, is necessary. Based on a survey of 662 psychologists and nurses, this article describes (1) the development of a short instrument measuring provider attitudes towards ROM, derived from the Evidence-based Practice Attitude Scale (EBPAS), and (2) how attitudinal domains relate to clinicians' current use of standardized instruments for treatment evaluation. The EBPAS-ROM showed concurrent validity in predicting aspects important for the implementation of ROM, including perceived limitations and the value of organizational support.