Measurement-based care as a practice improvement tool: Clinical and organizational applications in youth mental health

Development and validation of the modified Yale Food Addiction Scale for Children 2.0

OBJECTIVE

Scientific interest in the concept of food addiction is growing, but more studies are needed in youth samples. Brief, psychometrically valid, and developmentally appropriate measures are needed to support the assessment of food addiction in large-scale studies of youth that need to minimize participant burden. While a brief version of the Yale Food Addiction Scale 2.0 (YFAS 2.0) exists for adults, no comparable scale exists for youth. Thus, the current study aimed to develop a brief and valid measure of food addiction for use in youth populations (the modified YFAS for children 2.0).

METHOD

Data stem from the Food Addiction Denmark (FADK) Project, where random samples of 559 adolescents from the general population and 413 adolescents from a population with mental disorder completed the 35-item original YFAS for children 2.0 (YFAS-C 2.0). An abbreviated 13-item version was developed, and the psychometric properties of the modified YFAS-C 2.0 (mYFAS-C 2.0) were then evaluated.

RESULTS

The mYFAS-C 2.0 exhibited a one-factor structure and was found to have sound psychometric properties regarding internal consistency and convergent, discriminant, and incremental validity. This was evident in both the general adolescent population sample and the sample of adolescents with mental disorders.

CONCLUSIONS

The mYFAS-C 2.0 provides a brief and psychometrically valid measure of food addiction for youth, which may be beneficial for study designs (e.g., epidemiological samples) that aim to minimize participant burden. Future research on the test-retest reliability of the mYFAS-C 2.0 and its psychometric properties in younger children are important next steps.

Using Progress Feedback to Enhance Treatment Outcomes: A Narrative Review

We face increasing demand for greater access to effective routine mental health services, including telehealth. However, treatment outcomes in routine clinical practice are only about half the size of those reported in controlled trials. Progress feedback, defined as the ongoing monitoring of patients' treatment response with standardized measures, is an evidence-based practice that continues to be under-utilized in routine care. The aim of the current review is to provide a summary of the current evidence base for the use of progress feedback, its mechanisms of action and considerations for successful implementation. We reviewed ten available meta-analyses, which report small to medium overall effect sizes. The results suggest that adding feedback to a wide range of psychological and psychiatric interventions (ranging from primary care to hospitalization and crisis care) tends to enhance the effectiveness of these interventions. The strongest evidence is for patients with common mental health problems compared to those with very severe disorders. Effect sizes for not-on-track cases, a subgroup of cases that are not progressing well, are found to be somewhat stronger, especially when clinical support tools are added to the feedback. Systematic reviews and recent studies suggest potential mechanisms of action for progress feedback include focusing the clinician's attention, altering clinician expectations, providing new information, and enhancing patient-centered communication. Promising approaches to strengthen progress feedback interventions include advanced systems with signaling technology, clinical problem-solving tools, and a broader spectrum of outcome and progress measures. An overview of methodological and implementation challenges is provided, as well as suggestions for addressing these issues in future studies. We conclude that while feedback has modest effects, it is a small and affordable intervention that can potentially improve outcomes in psychological interventions. Further research into mechanisms of action and effective implementation strategies is needed.

Implementation Science and Practice-Oriented Research: Convergence and Complementarity

Implementation science is the scientific study of methods to promote the uptake of research findings and other evidence-based practices in routine care, with the goal of improving the quality and effectiveness of health services (Bauer et al., 2015). In addition to this common goal, practice-oriented psychotherapy research (and researchers) and implementation science (and scientists) share a common focus on the people and the places where treatment happens. Thus, there exists strong potential for combining these two approaches. In this article, we provide a primer on implementation science for psychotherapy researchers and highlight important areas and examples of convergence and complementarity between implementation science and practice-oriented psychotherapy research. Specifically, we (a) define and describe the core features of implementation science; (b) discuss similarities and areas of complementarity between implementation science and practice-oriented psychotherapy research; (c) discuss a case example that exemplifies the integration of implementation science and practice-oriented research; and (d) propose directions for future research and collaborations that leverage both implementation science and practice-oriented research.

The Connections Matter: Bi-Directional Learning in Program Evaluation and Practice-Oriented Research

Intended for researchers and clinical leaders, this article suggests that embedded program evaluation is a good fit with the desired features of practice-oriented research. The systematic nature of evaluation that is built into the operational workflow of a practice setting may increase the diversity of methods available to explore processes and outcomes of interest. We propose a novel conceptual framework that uses a human-centered systems lens to foster such embedded evaluation in clinical routine. This approach emphasizes the evaluator-practitioner partnership to build confidence in the bi-directional learning of practice-based evidence with evidence-based practice. The iterative cycles inherent to design thinking are aimed at developing better evaluation questions. The attention to structure and context inherent to systems thinking is intended to support meaningful perspectives in the naturally complex world of health care. Importantly, the combined human-centered systems lens can create greater awareness of the influence of individual and systemic biases that exist in any endeavor or institution that involves people. Recommended tools and strategies include systems mapping, program theory development, and visual facilitation using a logic model to represent the complexity of mental health treatment for communication, shared understanding, and connection to the broader evidence base. To illustrate elements of the proposed conceptual framework, two case examples are drawn from routine outcome monitoring (ROM) and progress feedback. We conclude with questions for future collaboration and research that may strengthen the partnership of evaluators and practitioners as a community of learners in service of local and system-level improvement.

Measurement-Based Care in Youth: An Opportunity for Better Clinical Outcomes?

Measurement-based care (MBC) is a procedure in which systematic and routine assessments are performed. Through this practice, clinicians can verify the progress of the symptomatology of the patient and adapt the appointments and the intervention to the current symptoms. Studies have reflected on the importance and the benefits of this type of procedure in the adult population, and have shown positive results. However, there is a lack of evidence concerning the remaining populations. Regarding youth, for instance, few articles have evaluated the benefits of using this procedure in clinical practice. However, research focused on this topic has revealed positive results, especially when clinicians were loyal to the MBC procedures. Still, further research is needed. This letter aims to share the methodology used by our multidisciplinary team, composed of psychologists and psychiatrists, in a clinical context at the Hospital Cruz Vermelha, Lisboa, applied to the adult population; the objective is to share and discuss some alterations that could be made to our evaluation protocol to enable the same to be used with the youth population. We believe that implementing MBC for youth is crucial for several reasons, including enhanced treatment efficacy, more personalized treatment, a reduced reliance on subjectivity, and empowerment not only of patients but also families.

Data-Informed Communication: How Measurement-Based Care Can Optimize Child Psychotherapy

Measurement-based care (MBC) research and practice, including clinical workflows and systems to support MBC, are grounded in adult-serving mental health systems. MBC research evidence is building in child and adolescent services, but MBC practice is inherently more complex due to identified client age, the family system and the need to involve multiple reporters. This paper seeks to address a gap in the literature by providing practical guidance for youth-serving clinicians implementing MBC with children and their families. We focus on MBC as a data-informed, client-centered communication process, and present three key strategies to enhance usual care child and adolescent psychotherapy via developmentally-appropriate MBC. These strategies include (1) go beyond standardized measures; (2) lean into discrepancies; and (3) get curious together. Case-based examples drawn from various child-serving settings illustrate these key strategies of MBC in child psychotherapy.

Implementing Measurement-Based Care in a Youth Partial Hospital Setting: Leveraging Feedback for Sustainability

This paper describes the successful implementation of Measurement-Based Care (MBC) within a Partial Hospitalization Program (PHP) for children and adolescents. Measurement-based care (MBC), the practice of using patient-reported measures routinely to inform decision-making, is associated with improved clinical outcomes for behavioral health patients (Jong et al., Clinical Psychology Review 85, 2021; Fortney & Sladek, 2015). MBC holds great promise in partial hospital programs (PHP) to improve outcomes, yet implementation strategies are as complex as the setting itself. This paper provides a case study example of MBC implementation in a PHP for youth. Over the course of 18 months, an interdisciplinary staff of approximately 20 behavioral health professionals provided partial hospitalization level of care to 633 (39% in-person, 61% telehealth) youth from ages 5 to 18 years old. MBC in this setting incorporated daily patient self-report and weekly caregiver-reported measurements. This descriptive reconstruction, which includes examples of the data that were used during the implementation project, illustrates specific barriers and facilitators in a successful implementation in the local PHP setting. Implementation strategies to address workflow integration, leadership and supervision, and coaching are described, including evolution of these strategies over the course of implementation. Practical considerations for implementing MBC in youth PHP settings are discussed. Finally, the authors explore the potential relationships between the data-driven MBC model of decision making and the development of resilient organizations.

Integration of Measurement-Based Care for Youth Depression and Suicidality Using VitalSign6

Depression and suicidality are prevalent in youth and are associated with a range of negative outcomes. The current study aimed to evaluate a measurement-based care (MBC) software (VitalSign6) tool to improve the screening and treatment of depression and suicidality in youth aged 8-17 years within a rural, underserved population. To assess for depression and suicidality, the Patient Health Questionnaire-2 was administered as an initial screen, and the Patient Health Questionnaire-9 Modified for Adolescents (PHQ-9-A) was administered if the initial screen was positive. Data were collected at medical clinics over one year, and descriptive statistics and t-tests or Wilcoxon-Mann-Whitney tests were conducted. A total of 1,984 youth were initially screened (mean age of 13 years; 51.6% female); 24.2% screened positive for depression, and 14.9% endorsed suicidality. Of those who screened positive, the mean PHQ-9-A score was 12.8; 66.9% had PHQ-9-A scores in the moderate to severe range, and 44.2% endorsed suicidality. Almost half of the youth who screened positive for depression had at least one follow-up assessment, and about one quarter achieved remission 4 months after initial screening. Adolescents (12-17 years) had higher PHQ-9-A scores, higher suicidality, and more follow-up assessments than younger youth (8-11 years). Younger youth had higher rates of remission. The widespread use of MBC was feasible in this setting. It is important to utilize MBC to identify and treat youth with depression and suicidality and to do so in younger populations to improve their trajectory over time; VitalSign6 is one tool to help achieve these goals.

Provider Use of, Attitudes Towards, and Self-efficacy with Key Measurement-based Care Practices in Youth Mental Health Treatment: A Multi-site Examination

Measurement-based care (MBC) is an underutilized evidence-based practice, and current implementation efforts demonstrate limited success in increasing MBC use. A better understanding of MBC implementation determinants is needed to improve these efforts, particularly from studies examining the full range of MBC practices and that span multiple samples of diverse providers using different MBC systems. This study addressed these limitations by conducting a multi-site survey examining MBC predictors and use in youth treatment. Participants were 159 clinicians and care coordinators working in youth mental health care settings across the United States. Participants were drawn from three program evaluations of MBC implementation. Providers completed measures assessing use of five MBC practices (administering measures, viewing feedback, reviewing feedback in supervision, sharing feedback with clients in session, and using feedback to plan treatment), MBC self-efficacy, and MBC attitudes. Despite expectations that MBC should be standard care for all clients, providers reported only administering measures to 40-60% of clients on average, with practices related to the use of feedback falling in the 1-39% range. Higher MBC self-efficacy and more positive views of MBC practicality predicted higher MBC use, although other attitude measures were not significant predictors. Effects of predictors were not moderated by site, suggesting consistent predictors across implementation settings. Implications of study findings for future research and for the implementation of MBC are discussed.

Putting measurement-based care into action: A mixed methods study of the benefits of integrating routine client feedback in coordinated specialty care programs for early psychosis

Background

Measurement-based care (MBC) is an effective tool in the delivery of evidence-based practices (EBPs). MBC utilizes feedback loops to share information and drive changes throughout a learning healthcare system. Few studies have demonstrated this practice in team-based care for people with early psychosis. This paper describes the development of a personalized feedback report derived from routine assessments that is shared with clients and clinicians as part of a MBC process.

Methods

We used a quasi pre-post comparison design with mixed methods to evaluate the implementation of a personalized feedback report at 5 early psychosis coordinated specialty care programs (CSC). We compared clients enrolled in CSC who did and did not receive a feedback report over the first 6 months of treatment. The sample included 204 clients: 146 who did not receive the feedback report and were enrolled over 2 years, and 58 who received the feedback report. A subset of 67 clients completed measures at both intake and 6-month follow-up, including 42 who received the report and 25 who did not. We compared the two groups with regard to self-reported symptoms, likelihood of completing treatment, and perception of shared decision making. We conducted qualitative interviews with 5 clients and 5 clinicians to identify the benefits and challenges associated with the personalized feedback report.

Results

People who received a personalized feedback report reported significant improvements in shared decision-making and had greater improvements over time in their intent to attend future treatment sessions. They engaged in more sessions for Supported Employment and Education (SEE), case management, and peer support, and fewer medication visits over the first 6 months of treatment. Both groups showed significant improvement in symptoms and functioning. Results from the qualitative analysis indicated that the experience of receiving the reports was valuable and validating for both patients and clinicians.

Conclusions

A personalized feedback report was integrated into standard of care for early psychosis programs. This process may improve shared decision-making, strengthen the likelihood to stay in treatment, and increase engagement in psychosocial interventions. We posit that this process facilitates strengths-focused discussions, enhances intrinsic motivation, and strengthens the therapeutic alliance.

A Pilot Project to Integrate Individualized Measurement Into Measurement-Based Care in a Child Partial Hospitalization Program

Measurement-based care (MBC), or the regular use of progress measures to inform clinical decision-making, improves quality of care and clinical outcomes. MBC typically focuses on standardized rather than individualized outcome measurement. In this pilot study, we examined the clinical utility of integrating individualized measurement with existing standardized outcome monitoring in a children's partial hospitalization program. Participants were 48 youth (M age 10.13 ± 1.39; 54.2% male, 41.7% female, 4.2% transgender or nonbinary). Comorbidity was common; 83.4% of youth had more than one diagnosis at intake. Using the Youth Top Problems for individualized outcome measurement, we examined Top Problem content and clinical improvement over time. Finally, we examined completion rates and describe implementation issues. Top Problems were heterogeneous and sensitive to change. Of the 144 problems, 107 (74%) had a focus consistent with measures used in program, while 37 (26%) were not captured by standardized measures used in program. Effect sizes from admission to final measurement ranged from Cohen's d = .75 - 1.00. Initial adoption of the MBC was strong, but sustained use of the system over the treatment course was challenging. Individualized outcome measurement in children's partial hospitalization programs is feasible to administer and sensitive to clinical change that is unique from change captured in standardized measures. Parents were able to self-identify clinically meaningful, highly individualized Top Problems. Challenges of implementation and clinical assessment in acute settings and potential strategies for improving implementation are discussed.

Consultation as an Implementation Strategy to Increase Fidelity of Measurement-Based Care Delivery in Community Mental Health Settings: An Observational Study

Regularly administering outcome measures to clients to inform clinical decision making, referred to as measurement-based care (MBC), has the potential to improve mental health treatment due to its applicability across disorders and treatment settings. Given the utility of MBC, understanding predictors of high-fidelity MBC implementation is important. Training and consultation are widely used implementation strategies to increase the fidelity of evidence-based practice delivery but have rarely been studied with MBC. The current study will examine the relationship between time clinicians spent discussing a case in consultation ("dosage") and MBC fidelity. Thirty clinicians and 56 youth were in the MBC condition of a randomized controlled trial and completed baseline questionnaires. Consultation dosage was extracted from call notes. MBC fidelity was measured using the implementation index, which combines rates of administering and viewing questionnaires, using objective data from the online MBC system. Multi-level modeling was used. Greater consultation dosage significantly predicted a higher implementation index (ß = 0.27, SE = 0.06, p < .001). For every 30 min spent discussing a case in consultation, the case's MBC fidelity increased by 8.1%. Greater consultation dosage significantly predicted higher rates of administration (ß = 0.07, SE = 0.03, p = 0.033) and feedback report viewing (ß = 0.24, SE = 0.06, p < .001). More consultation at the case level predicts greater MBC fidelity using objective measures. The greatest impact was increasing the clinician feedback report viewing rate. The results of this study can help inform future efforts to increase the fidelity with which MBC is delivered and to make consultation as efficient and effective as possible.

A Clinical Leadership Lens on Implementing Progress Feedback in Three Countries: Development of a Multidimensional Qualitative Coding Scheme

BACKGROUND

Progress feedback, also known as measurement-based care (MBC), is the routine collection of patient-reported measures to monitor treatment progress and inform clinical decision-making. Although a key ingredient to improving mental health care, sustained use of progress feedback is poor. Integration into everyday workflow is challenging, impacted by a complex interrelated set of factors across patient, clinician, organizational, and health system levels. This study describes the development of a qualitative coding scheme for progress feedback implementation that accounts for the dynamic nature of barriers and facilitators across multiple levels of use in mental health settings. Such a coding scheme may help promote a common language for researchers and implementers to better identify barriers that need to be addressed, as well as facilitators that could be supported in different settings and contexts.

METHODS

Clinical staff, managers, and leaders from two Dutch, three Norwegian, and four mental health organizations in the USA participated in semi-structured interviews on how intra- and extra-organizational characteristics interact to influence the use of progress feedback in clinical practice, supervision, and program improvement. Interviews were conducted in the local language, then translated to English prior to qualitative coding.

RESULTS

A team-based consensus coding approach was used to refine an a priori expert-informed and literature-based qualitative scheme to incorporate new understandings and constructs as they emerged. First, this hermeneutic approach resulted in a multi-level coding scheme with nine superordinate categories and 30 subcategories. Second-order axial coding established contextually sensitive categories for barriers and facilitators.

CONCLUSIONS

The primary outcome is an empirically derived multi-level qualitative coding scheme that can be used in progress feedback implementation research and development. It can be applied across contexts and settings, with expectations for ongoing refinement. Suggestions for future research and application in practice settings are provided. Supplementary materials include the coding scheme and a detailed playbook.

Taking a Magnifying Glass to Measurement-Based Care Consultation Sessions: with What Issues Do Mental Health Clinicians Struggle?

PURPOSE

Ongoing consultation following initial training is one of the most commonly deployed implementation strategies to facilitate uptake of evidence-based practices, such as measurement-based care (MBC). Group consultation provides an interactive experience with an expert and colleagues to get feedback on actual issues faced, yet there is little research that unpacks the questions raised in consultation and what types of issues are important to address.

METHODS

The current study characterized the questions and concerns raised by community mental health clinicians (N = 38 across six clinics) during group consultation sessions completed as part of an MBC implementation trial. We conducted a qualitative content analysis of consultation forms completed by clinicians before each MBC consultation session.

RESULTS

Clinicians sought MBC consultation for clients across a range of ages and levels of depression severity. Qualitative results revealed five main questions and concerns in consultation sessions: (1) how to administer the PHQ-9, (2) how to review PHQ-9 scores, (3) how to respond to PHQ-9 score, (4) the types of clients for whom MBC would be appropriate, and (5) how MBC could impact a clinician's usual care.

CONCLUSION

Findings highlight the need for ongoing consultation and limitations of workshop training alone. Practical recommendations for addressing the common questions and concerns identified are presented to support MBC use.

Using Digital Measurement-Based Care for the Treatment of Anxiety and Depression in Children and Adolescents: Observational Retrospective Analysis of Bend Health Data

BACKGROUND

A growing body of evidence supports the efficacy of measurement-based care (MBC) for children and adolescents experiencing mental health concerns, particularly anxiety and depression. In recent years, MBC has increasingly transitioned to web-based spaces in the form of digital mental health interventions (DMHIs), which render high-quality mental health care more accessible nationwide. Although extant research is promising, the emergence of MBC DMHIs means that much is unknown regarding their effectiveness as a treatment for anxiety and depression, particularly among children and adolescents.

OBJECTIVE

This study uses preliminary data from children and adolescents participating in an MBC DMHI administered by Bend Health Inc, a mental health care provider that uses a collaborative care model to assess changes in anxiety and depressive symptoms during participation in the MBC DMHI.

METHODS

Caregivers of children and adolescents participating in Bend Health Inc for anxiety or depressive symptoms reported measures of their children's symptoms every 30 days throughout the duration of participation in Bend Health Inc. Data from 114 children (age 6-12 years) and adolescents (age 13-17 years) were used for the analyses (anxiety symptom group: n=98, depressive symptom group: n=61).

RESULTS

Among children and adolescents participating in care with Bend Health Inc, 73% (72/98) exhibited improvements in anxiety symptoms and 73% (44/61) exhibited improvement in depressive symptoms, as indicated by either a decrease in symptom severity or screening out of completing the complete assessment. Among those with complete assessment data, group-level anxiety symptom T-scores exhibited a moderate decrease of 4.69 points (P=.002) from the first to the last assessment. However, members' depressive symptom T-scores remained largely stable throughout their involvement.

CONCLUSIONS

As increasing numbers of young people and families seek DMHIs over traditional mental health treatments due to their accessibility and affordability, this study offers promising early evidence that youth anxiety symptoms decrease during involvement in an MBC DMHI such as Bend Health Inc. However, further analyses with enhanced longitudinal symptom measures are necessary to determine whether depressive symptoms show similar improvements among those involved in Bend Health Inc.

Using digital measurement-based care to address symptoms of inattention, hyperactivity, and opposition in youth: A retrospective analysis of Bend Health (Preprint)

BACKGROUND

Attention-deficit/hyperactivity disorder (ADHD) and associated behavioral disorders are highly prevalent in children and adolescents, yet many of them do not receive the care they need. Digital mental health interventions (DMHIs) may address this need by providing accessible and high-quality care. Given the necessity for high levels of caregiver and primary care practitioner involvement in addressing ADHD symptoms and behavioral problems, collaborative care interventions that adopt a whole-family approach may be particularly well suited to reduce symptoms of inattention, hyperactivity, and opposition in children and adolescents.

OBJECTIVE

The purpose of this study is to use member (ie, child and adolescent) data from Bend Health, Inc, a collaborative care DMHI that uses a whole-family approach to address child and adolescent mental health concerns, to (1) determine the effects of a collaborative care DMHI on inattention, hyperactivity, and oppositional symptoms in children and adolescents and (2) assess whether the effects of a collaborative care DMHI vary across ADHD subtypes and demographic factors.

METHODS

Caregivers of children and adolescents with elevated symptoms of inattention, hyperactivity, or opposition assessed their children's symptom severity approximately every 30 days while participating in Bend Health, Inc. Data from 107 children and adolescents aged 6-17 years who exhibited clinically elevated symptoms at baseline were used to assess symptom severity across monthly assessments (inattention symptom group: n=91, 85.0%; hyperactivity symptom group: n=48, 44.9%; oppositional symptom group: n=70, 65.4%). The majority of the sample exhibited elevated symptoms of at least 2 symptom types at baseline (n=67, 62.6%).

RESULTS

Members received care for up to 5.52 months and attended between 0 and 10 coaching, therapy, or psychiatry sessions through Bend Health, Inc. For those with at least 2 assessments, 71.0% (n=22) showed improvements in inattention symptoms, 60.0% (n=9) showed improvements in hyperactivity symptoms, and 60.0% (n=12) showed improvements in oppositional symptoms. When considering group-level change over time, symptom severity decreased over the course of treatment with Bend Health, Inc, for inattention (average decrease=3.51 points, P=.001) and hyperactivity (average decrease=3.07 points, P=.049) but not for oppositional symptoms (average decrease=0.70 points, P=.26). There was a main effect of the duration of care on symptom severity (P<.001) such that each additional month of care was associated with lower symptom scores.

CONCLUSIONS

This study offers promising early evidence that collaborative care DHMIs may facilitate improvements in ADHD symptoms among children and adolescents, addressing the growing need for accessible and high-quality care for behavioral health problems in the United States. However, additional follow-up studies bolstered by larger samples and control groups are necessary to further establish the robustness of these findings.

Prescribing Psychotropic Medications for Justice-Involved Juveniles

Psychiatric disorders are exceedingly common in justice-involved youth. Relevant justice systems are varied, evolving, complex, and underserved. The intent of this article is to highlight the research and best practices related to managing the psychiatric disorders of justice-involved youths with a particular focus on pharmacotherapy. We review relevant features of the justice system and related case law, how prescribing for these individuals varies from both community settings and for incarcerated adults, effective assessment and medication selection, informed consent, management of nonadherence and medication diversion, and applied evidence for specific psychiatric problems common in these patients. Psychiatrists caring for justice-involved youth will benefit from tools, like this article, for training and orientation to provide timely and effective treatment services and consultation in these settings.

What is the Status of Multi-Informant Treatment Fidelity Research?

OBJECTIVE

The precise measurement of treatment fidelity (quantity and quality in the delivery of treatment strategies in an intervention) is essential for intervention development, evaluation, and implementation. Various informants are used in fidelity assessment (e.g., observers, practitioners [clinicians, teachers], clients), but these informants often do not agree on ratings. This scoping review aims to ascertain the state of science around multi-informant assessment of treatment fidelity.

METHOD

A literature search of articles published through December 2021 identified 673 articles. Screening reduced the number of articles to 44, and the final study set included 35 articles.

RESULTS

There was substantial variability across studies regarding study design, how fidelity was operationalized, and how reliability was defined and assessed. Most studies evaluated the agreement between independent observers and practitioner-report, though several other informant pairs were assessed. Overall, findings suggest that concordance across fidelity informants was low to moderate, with a few key exceptions.

CONCLUSIONS

It is difficult to draw clear conclusions about the degree to which single versus multiple informant assessment is needed to produce an accurate and complete picture of treatment fidelity. The field needs to take steps to determine how to leverage multi-informant assessment to accurately assess treatment fidelity.

Organization-level variation in therapists' attitudes toward and use of measurement-based care

PURPOSE

Despite significant interest in improving behavioral health therapists' implementation of measurement-based care (MBC)-and widespread acknowledgment of the potential importance of organization-level determinants-little is known about the extent to which therapists' use of, and attitudes toward, MBC vary across and within provider organizations or the multilevel factors that predict this variation.

METHODS

Data were collected from 177 therapists delivering psychotherapy to youth in 21 specialty outpatient clinics in the USA. Primary outcomes were use of MBC for progress monitoring and treatment modification, measured by the nationally-normed Current Assessment of Practice Evaluation-Revised. Secondary outcomes were therapist attitudes towards MBC. Linear multilevel regression models tested the association of theory-informed clinic and therapist characteristics with these outcomes.

RESULTS

Use of MBC varied significantly across clinics, with means on progress monitoring ranging from values at the 25th to 93rd percentiles and means on treatment modification ranging from the 18th to 71st percentiles. At the clinic level, the most robust predictor of both outcomes was clinic climate for evidence-based practice implementation; at the therapist level, the most robust predictors were: attitudes regarding practicality, exposure to MBC in graduate training, and prior experience with MBC. Attitudes were most consistently related to clinic climate for evidence-based practice implementation, exposure to MBC in graduate training, and prior experience with MBC.

CONCLUSIONS

There is important variation in therapists' attitudes toward and use of MBC across clinics. Implementation strategies that target clinic climate for evidence-based practice implementation, graduate training, and practicality may enhance MBC implementation in behavioral health.

Characterizing measurement-based care implementation using therapist report

The effectiveness of measurement-based care (MBC), an evidence-based practice that uses regularly collected assessment data to guide clinical decision-making, is impacted by whether and how therapists use information from MBC tools in treatment. Improved characterization of how therapists use MBC in treatment sessions with youth is needed to guide implementation and understand variability in MBC effectiveness. To meet this need, this study examined therapists' sharing and discussion of MBC in treatment sessions. Thirty therapists were randomly assigned to the MBC condition as part of a comparative effectiveness trail of treatments for adolescent anxiety and depression. A qualitative content analysis was conducted on therapists' written explanations of changes made to the session based on the MBC data. Therapists reported sharing data with youth and caregivers in an average of 34.6% and 27.4% of sessions, respectively. Therapists reported incorporating MBC data in an average of 21.1% of sessions. When data were used, therapists predominately focused changes on short-term (e.g., current symptoms, treatment skill) rather than long-term (e.g., symptom progress, treatment goals) decision-making. Therapists inconsistently used MBC data, highlighting the need for improved training in and monitoring of how therapists use MBC in session to guide collaborative treatment decision-making with youth and caregivers.

To Utility and Beyond! Specifying and Advancing the Utility of Measurement-Based Care for Youth

Mental health organizations that serve youth are under pressure to adopt measurement-based care (MBC), defined as the continuous collection of client-report data used to support clinical decision-making as part of standard care. However, few frameworks exist to help leadership ascertain how to select an MBC approach for a clinical setting. This paper seeks to define how an MBC approach can display clinical utility to provide such a framework. Broadly, we define clinical utility as evidence that an MBC approach assists stakeholders in fulfilling clinical goals related to care quality (i.e., improve client-clinician alliance and clinical outcomes) at the client (i.e., youth and caregiver), clinician, supervisor, and administrator levels. More specifically, our definition of clinical utility is divided into two categories relevant to the usability and usefulness of an MBC approach for a specific setting: (a) implementability (i.e., evidence indicating ease of use in a clinical setting) and (b) usefulness in aiding clinical activities (i.e., evidence indicating the potential to improve communication and make clinical activities related to care quality easier or more effective). These categories provide valuable information about how easy an MBC approach is to use and the potential benefits that the MBC data will confer. To detail how we arrived at this definition, we review prior definitions of clinical utility, discuss how previous definitions inform our definition of clinical utility for MBC, and provide examples of how the concept of clinical utility can be applied to MBC. We finish with a discussion of future research directions.

Psychometric evaluation of a caregiver-report adaptation of the Overall Anxiety Severity and Impairment Scale (OASIS) for use with youth populations

Background Despite progress in youth anxiety assessment, there is need for a measure that is simultaneously (a) free, (b) brief, (c) focused broadly on anxiety and avoidance severity, frequency, and interference, and (d) concerned with the past week. The adult overall anxiety severity and impairment scale (OASIS) was adapted to yield a caregiver-report of past week youth anxiety and interference (OASIS-Y). Methods In a sample of diverse youth seeking anxiety services (N = 132; 67% racial/ethnic minority) and their caregivers, analyses examined the OASIS-Y factor structure, internal consistency, and convergent and divergent validity. Hierarchical linear modeling in a participant subset examined OASIS-Y sensitivity to treatment-related change. Results OASIS-Y internal consistency was high and confirmatory factor analysis supported a single-factor structure similar to that found in adults. OASIS-Y convergent validity was supported by a medium-sized association with an established, commercially available measure of youth anxiety, and divergent validity was supported by the absence of unique associations with measures of youth attention and externalizing problems. In a sample subset, session-by-session OASIS-Y scores significantly declined across treatment, and declined at a steeper rate among treatment "responders" versus "non-responders," providing evidence of OASIS-Y sensitivity to treatment-related change. Limitations This study focused on a clinical sample and cannot speak to OASIS-Y performance in community settings. Shared method-variance may have also influenced findings. Conclusions This study offers the first psychometric evaluation of the OASIS-Y, and underscores the promising clinical utility of the measure for assessing past week youth anxiety and impairment and for supporting routine outcome monitoring.

Measurement-Based Care in the Adolescent Partial Hospital Setting: Implementation, Challenges, & Future Directions

In this paper, we describe the process of implementing measurement-based care (MBC) in the adolescent partial hospital program setting. First, we outline the rationale for incorporating MBC in this treatment setting. Second, we describe the partial hospital setting in which implementation took place, including the patient population, treatment providers, and structure of programming. Next, we outline the initial implementation of standardized assessments into our programming, including key initial considerations and challenges during implementation. We describe the importance of considering the primary symptom presentations of the patient population when selecting assessment tools, the importance of leveraging existing electronic health record tools to efficiently track and record data collection, and the ability to integrate assessments into clinical workflows. Fourth, we present data describing compliance with implementation, patient outcomes, and providers' attitudes towards and knowledge of MBC following implementation. We found after the initial implementation period, compliance was high. We also found providers had an overall positive perception of the use of MBC, reporting they perceived it to be helpful to both their clinical practice and patient outcomes. Finally, we discuss future directions for best utilizing standardized assessments in intensive treatment settings.

Improving the Quality of Children's Mental Health Care with Progress Measures: A Mixed-Methods Study of PCIT Therapist Attitudes

Progress measures are an evidence-based technique for improving the quality of mental health care, however, clinicians rarely incorporate them into treatment. Research into how measure type impacts clinician preference has been recommended to help improve measure implementation. Parent–Child Interaction Therapy (PCIT) is an assessment-driven treatment that serves as an ideal intervention through which to investigate measure preferences given its routine use of two types of assessments, a behavioral observation (the Dyadic Parent–Child Interaction Coding System) and a parent-report measure (the Eyberg Child Behavior Inventory). This study investigated PCIT therapist attitudes towards progress measures used within PCIT and children’s mental health treatment generally. A mixed-method (QUAN + QUAL) study design examined PCIT therapist attitudes towards two types of progress measures and measures used in two contexts (PCIT and general practice). Multi-level modeling of a survey distributed to 324 PCIT therapists identified predictors of therapist attitudes towards measures, while qualitative interviews with 23 therapists expanded and clarified the rationale for differing perceptions. PCIT therapists reported more positive attitudes towards a behavioral observation measure, the DPICS, than a parent-report measure, the ECBI, and towards measures used in PCIT than in general practice. Clinician race/ethnicity was significantly related to measure-specific attitudes. Qualitative interviews highlighted how perceptions of measure reliability, type of data offered, ease of use, utility in guiding sessions and motivating clients, and embeddedness in treatment protocol impact therapist preferences. Efforts to implement progress monitoring should consider preferences for particular types of measures, as well as how therapists are trained to embed measures in treatment.

Applying behaviour change theory to understand the barriers to implementing routine outcome monitoring

BACKGROUND

Routine outcome monitoring (ROM) is a valuable tool for monitoring client progress and pre-empting deterioration, however, there is considerable variation in how data are collected and recorded and uptake in clinical practice remains low. The aim of this study was to develop a self-report measure of practitioner attitudes to ROM in order to better understand the barriers to successful implementation in Child and Adolescent Mental Health Services (CAMHS).

METHODS

An anonymous survey was completed by 184 CAMHS practitioners in the United Kingdom. The survey was designed using the Capability, Opportunity, and Motivation Model of Behaviour (COM-B). Practitioners who reported using ROM frequently in their clinical work (53%) were compared to those who used ROM infrequently (47%) across dimensions of the COM-B survey subscales.

RESULTS

Confirmatory factor analysis confirmed the proposed four-factor structure, showing acceptable model fit, with high factor loadings and good reliability for all subscales. Frequent users of ROM exhibited significantly higher psychological capability, physical opportunity, social opportunity, and motivation, compared to infrequent users F (4, 140) = 14.76, p < .0001; Pillai's Trace = .297, partial η²  = .30. Results highlight several barriers to ROM, including the belief that there is not a strong evidence base for ROM, not receiving external training, and not discussing feedback and outcome data in supervision.

IMPLICATIONS

In the hope of improving the successful implementation of ROM, this research provides an evidence-based tool for assessing practitioners' attitudes to ROM, which map on to intervention functions and represent targets for future implementation efforts.

PRACTITIONER POINTS

The value of routine outcome monitoring (ROM) as a means to measure client progress and to elevate the efficiency and quality of mental health care is well-documented in the research literature, however, uptake in practice remains relatively low. This study applied behaviour change theory to develop a psychometrically sound self-report measure of practitioners' perspectives and practices to understand the barriers to implementation in child and adolescent mental health services in the United Kingdom. The complex and multifaceted nature of the barriers to implementation requires multilevel behaviour change strategies at the client, clinician, and organisational level. Recommendations for practice include the need for integrated, multilevel strategies aimed at improving practitioners' capabilities and motivations, strong organisational leadership and a culture of data gathering and sharing, and implementation interventions, which are tailored to target local barriers.