Using Measurement-Based Care to Enhance Any Treatment

Measurement-based care (MBC) can be defined as the practice of basing clinical care on client data collected throughout treatment. MBC is considered a core component of numerous evidence-based practices (e.g., Beck & Beck, 2011; Klerman, Weissman, Rounsaville, & Chevron, 1984) and has emerging empirical support as an evidence-based framework that can be added to any treatment (Lambert et al., 2003, Trivedi et al., 2007). The observed benefits of MBC are numerous. MBC provides insight into treatment progress, highlights ongoing treatment targets, reduces symptom deterioration, and improves client outcomes (Lambert et al., 2005). Moreover, as a framework to guide treatment, MBC has transtheoretical and transdiagnostic relevance with broad reach across clinical settings. Although MBC has primarily focused on assessing symptoms (e.g., depression, anxiety), MBC can also be used to assess valuable information about (a) symptoms, (b) functioning and satisfaction with life, (c) putative mechanisms of change (e.g., readiness to change), and (d) the treatment process (e.g., session feedback, working alliance). This paper provides an overview of the benefits and challenges of MBC implementation when conceptualized as a transtheoretical and transdiagnostic framework for evaluating client therapy progress and outcomes across these four domains. The empirical support for MBC use is briefly reviewed, an adult case example is presented to serve as a guide for successful implementation of MBC in clinical practice, and future directions to maximize MBC utility are discussed.

Modeling Oral Reading Fluency Development in Latino Students: A Longitudinal Study Across Second and Third Grade

This study examines growth in oral reading fluency across 2nd and 3rd grade for Latino students grouped in 3 English proficiency levels: students receiving English as a second language (ESL) services (n = 2,182), students exited from ESL services (n = 965), and students never designated as needing services (n = 1,857). An important focus was to learn whether, within these 3 groups, proficiency levels and growth were reliably related to special education status. Using hierarchical linear modeling, the authors compared proficiency levels and growth in oral reading fluency in English between and within groups and then to state reading benchmarks. Findings indicate that oral reading fluency scores reliably distinguished between students with learning disabilities and typically developing students within each group (effect sizes ranging from 0.96 to 1.51). The growth trajectory included a significant quadratic trend (generally slowing over time). These findings support the effectiveness of using oral reading fluency in English to screen and monitor reading progress under Response to Intervention models, but also suggest caution in interpreting oral reading fluency data as part of the process in identifying students with learning disabilities.

Efficacy of client feedback in group psychotherapy with soldiers referred for substance abuse treatment

This study investigated whether routine monitoring of client progress, often called "client feedback," via an abbreviated version of the Partners for Change Outcome Management System (PCOMS) resulted in improved outcomes for soldiers receiving group treatment at an Army Substance Abuse Outpatient Treatment Program (ASAP). Participants (N = 263) were active-duty male and female soldiers randomized into a group feedback condition (n = 137) or a group treatment-as-usual (TAU) condition (n = 126). Results indicated that clients in the feedback condition achieved significantly more improvement on the outcome rating scale (d = 0.28), higher rates of clinically significant change, higher percentage of successful ratings by both clinicians and commanders, and attended significantly more sessions compared to the TAU condition. Despite a reduced PCOMS protocol and a limited duration of intervention, preliminary results suggest that the benefits of client feedback appear to extend to group psychotherapy with clients in the military struggling with substance abuse.

From distal to proximal: Routine educational data monitoring in school-based mental health

Research and practice in school-based mental health (SBMH) typically includes educational variables only as distal outcomes, resulting from improvements in mental health symptoms rather than directly from mental health intervention. Although sometimes appropriate, this approach also has the potential to inhibit the integration of mental health and schools. The current paper applies an existing model of data-driven decision making (Daleiden & Chorpita, 2005) to detail how SBMH can better integrate routine monitoring of school and academic outcomes into four evidence bases: general services research evidence, case histories, local aggregate, and causal mechanisms. The importance of developing new consultation protocols specific to data-driven decision making in SBMH as well as supportive infrastructure (e.g., measurement feedback systems) to support the collection and use of educational data is also described.

The neural basis of monitoring goal progress

The neural basis of progress monitoring has received relatively little attention compared to other sub-processes that are involved in goal directed behavior such as motor control and response inhibition. Studies of error-monitoring have identified the dorsal anterior cingulate cortex (dACC) as a structure that is sensitive to conflict detection, and triggers corrective action. However, monitoring goal progress involves monitoring correct as well as erroneous events over a period of time. In the present research, 20 healthy participants underwent functional magnetic resonance imagining (fMRI) while playing a game that involved monitoring progress toward either a numerical or a visuo-spatial target. The findings confirmed the role of the dACC in detecting situations in which the current state may conflict with the desired state, but also revealed activations in the frontal and parietal regions, pointing to the involvement of processes such as attention and working memory (WM) in monitoring progress over time. In addition, activation of the cuneus was associated with monitoring progress toward a specific target presented in the visual modality. This is the first time that activation in this region has been linked to higher-order processing of goal-relevant information, rather than low-level anticipation of visual stimuli. Taken together, these findings identify the neural substrates involved in monitoring progress over time, and how these extend beyond activations observed in conflict and error monitoring.

Daily monitoring of temporal trajectories of suicidal ideation predict self-injury: A novel application of patient progress monitoring

OBJECTIVE

The interpersonal theory of suicide argues that suicidal ideation predicts self-injury. We hypothesized that distinct patterns of suicidal ideation could be identified and these ratings could allow early identification of self-injury.

METHOD

The sample consisted of 562 psychiatric inpatients who reported suicidal ideation.

RESULTS

Latent growth class analysis identified five classes of change in suicidal ideation. Patients who displayed prolonged suicidal ideation could be identified with improved sensitivity (89.66%) and negative predictive power (94%), compared to a model based on routine ratings of suicidality at admission (sensitivity = 50%; negative predictive power = 74%). These patients had a fourfold increased risk of self-injury.

CONCLUSIONS

Daily measurement of suicidal ideation may identify inpatients at risk and inform clinical decision-making.

Psychometric Properties of Maze Tasks in Middle School Students

Maze tasks have appealing properties as progress-monitoring tools, but there is a need for a thorough examination of the psychometric properties of Maze tasks among middle school students. We evaluated form effects, reliability, validity, and practice effects of Maze among students in Grades 6 through 8. We administered the same (familiar) and novel Maze passages for progress monitoring of a reading intervention among typical readers (n = 588), struggling readers receiving researcher-provided intervention (n = 471), and struggling readers not receiving intervention (n = 284). Form effects accounted for significant variance in Maze performance. Familiar passages had greater test-retest reliability than novel passages. Both administrative conditions had similar, moderate correlations (validity coefficients) with other measures of reading fluency and comprehension. There were also significant practice effects. Students who read the same passage showed steeper slopes in Maze performance than students who read different passages over time. Practice effects were influenced by beginning levels of reading comprehension and by intervention status.

Symptom Severity at Week 4 of Cognitive-Behavior Therapy Predicts Depression Remission

Early response has been shown to predict psychotherapy outcome. We examined the strength of the relationship between early response and remission in 82 patients who received naturalistic cognitive-behavior therapy in a private practice setting, and 158 patients who received protocol cognitive therapy in a research setting. We predicted that the relationship between early response and remission would be substantial enough to guide clinical decision making in both samples, and that a simple model of severity at Week 4 of treatment would predict remission as effectively as a more complex change score. Logistic regressions showed that a simple model based on the Week 4 Beck Depression Inventory (BDI) score was as predictive of remission as more complex models of early change. A receiver operating characteristics analysis showed that BDI score at Week 4 was substantially predictive of remission in both the naturalistic and research protocol samples; the area under the curve was .80 and .84 in the naturalistic and protocol samples, respectively. To guide clinical decision making, we identified threshold scores on the BDI corresponding to various negative predictive values (probability of nonremission when nonremission is predicted). Our results indicate that depressed patients who remain severely depressed at Week 4 of cognitive therapy are unlikely to reach remission at the end of relatively brief (maximum 20 sessions) treatment. We discuss implications of our findings for clinical decision making and treatment development.

Understanding routine outcome monitoring and clinical feedback in context: Introduction to the special section

The practice of routine outcome monitoring and providing clinical feedback has been widely studied within psychotherapy. Nevertheless, there are many outstanding questions regarding this practice. Is it an evidence-based adjunct to ongoing psychotherapies, or an ineffective complication of treatment? If it is effective, through what mechanism(s) does it act? Is it effective with all patient populations, treatment types, and service delivery mechanisms, or does its impact vary across context? What choices in the implementation process affect the utility of patient-reported data feedback on psychotherapy outcomes? The studies in this special section explore these questions using a wide variety of methods and significantly expand the reach of studies on feedback. Together, these studies represent a snapshot of a maturing field of study: Initial discoveries are developed into more robust theories and applied in a wider range of contexts, while the limits of that theory are tested. They also signal directions for future clinical and research work that may improve patient care in psychosocial interventions into the future.

Factors affecting the implementation of an outcome measurement feedback system in youth mental health settings

Objective: Measurement feedback systems provide clinicians with regular snapshots of a client's mental health status, which can be used in treatment planning and client feedback. There are numerous barriers to clinicians using outcome measures routinely. This study aimed to investigate factors affecting the use of a measurement feedback system across youth mental health settings. Methods: The participants were 210 clinicians from headspace youth mental health services across Australia. They were surveyed on predictors and use of MyLifeTracker, a routine outcome measure. This was explored through three processes: looking at MyLifeTracker before session, using MyLifeTracker in treatment planning, and providing feedback of MyLifeTracker scores to clients. Results: Clinicians were more likely to look at MyLifeTracker before session, less likely to use it in treatment planning, and least likely to provide MyLifeTracker scores to clients. Each measurement feedback system process had a distinct group of predictors. Perceptions of MyLifeTracker's practicality was the only significant predictor of all three processes. Conclusion: Practically, organisations and supervisors can increase the use of measurement feedback systems through targeted supports.

Trans Collaborations Clinical Check-In (TC3): Initial Validation of a Clinical Measure for Transgender and Gender Diverse Adults Receiving Psychological Services

One key aspect of evidence-based psychological services is monitoring progress to inform treatment decision making, often using a brief self-report measure. However, no such measure exists to support measurement-based care, given the distinct needs of transgender and gender diverse people (TGD), a group facing large documented health disparities and marginalization in health care. The purpose of the present study was to develop and provide initial psychometric validation of a short, behavioral health progress monitoring self-report measure, the Trans Collaborations Clinical Check-in (TC3). TGD communities, providers identified as TGD-affirmative, and relevant academic experts contributed to item and scale development. The final 18-item version was administered to 215 TGD adults (75 transfeminine, 76 transmasculine, 46 nonbinary, 18 unknown; mean age of 30 with a range of 19 to 73), who were recruited for an online study, with other questionnaires assessing negative affect, well-being, gender dysphoria, gender minority stressors, and resilience. Higher scores on the TC3 (indicating better adjustment and comfort with gender) were generally associated with lower depression, anxiety, minority stress, and gender dysphoria and greater life satisfaction, body congruence, and positive aspects of being TGD such as pride in identity and community belongingness. These results support the validity of the TC3 as a brief measure to be used as a clinical tool for TGD people receiving mental health services. Additional research is needed on the reliability and validity of the TC3 across multiple time points to determine utility as a progress monitoring measure. The TC3 should also be further validated with more culturally diverse samples.

Why Do People Act Like the Proverbial Ostrich? Investigating the Reasons That People Provide for Not Monitoring Their Goal Progress

Two studies examined peoples' reasons for not monitoring their progress toward their personal goals—a phenomenon that has been termed “the ostrich problem” (Webb et al., 2013). Study 1 used factor analysis to organize the reasons that people gave for not monitoring their goal progress, resulting in 10 factors. The most strongly endorsed reasons were: (a) that information on goal progress would demand a change in beliefs, or (b) undesired action; (c) that progress was poor, and (d) that thinking about and/or working on the goal was associated with negative emotions. Study 2 adopted a prospective design and investigated whether the reasons identified in Study 1 predicted: (a) the likelihood that participants would decline an opportunity to monitor their goal progress, and (b) the frequency with which participants monitored their goal progress. We found evidence that some of the most strongly endorsed reasons from Study 1 also predicted the avoidance of monitoring in Study 2; however, the belief that information about goal progress was likely to be inaccurate and not useful, and perceived control over goal attainment also reliably predicted the avoidance of monitoring in Study 2. Taken together, the findings explain why people do not monitor their goal progress and point to potential avenues for intervention.

Predicting Dropout from Inpatient Substance Use Disorder Treatment: A Prospective Validation Study of the OQ-Analyst

Background and Aims:

There is an urgent need for tools allowing therapists to identify patients at risk of dropout. The OQ-Analyst, an increasingly popular computer-based system, is used to track patient progress and predict dropout. However, we have been unable to find empirical documentation regarding the ability of OQ-Analyst to predict dropout. The aim of the present study was to perform the first direct test of the ability of the OQ-Analyst to predict dropout.

Design:

Patients were consecutively enlisted in a naturalistic, prospective, longitudinal clinical trial. As interventions based on feedback from the OQ-Analyst could alter the outcome and potentially render the prediction wrong, feedback was withheld from patients and therapists.

Setting:

The study was carried out during 2011–2013 in an inpatient substance use disorder clinic in Oslo, Norway.

Participants:

Patients aged 18 to 28 years who met criteria for a principal diagnosis of mental or behavioural disorder due to psychoactive substance use (ICD 10; F10.2–F19.2).

Measurements:

Red signal (predictions of high risk) from the Norwegian version of the OQ-Analyst were compared with dropouts identified using patient medical records as the standard for predictive accuracy.

Findings:

A total of 40 patients completed 647 OQ assessments resulting in 46 red signals. There were 27 observed dropouts, only one of which followed after a red signal. Patients indicated by the OQ-Analyst as being at high risk of dropping out were no more likely to do so than those indicated as being at low risk. Random intercept logistic regression predicting dropout from a red signal was statistically nonsignificant. Bayes factor supports no association.

Conclusions:

The study does not support the predictive ability of the OQ-Analyst for the present patient population. In the absence of empirical evidence of predictive ability, it may be better not to assume such ability.

Exploring Growth in Expressive Communication of Infants and Toddlers With Autism Spectrum Disorder

One of the earliest indicators of autism spectrum disorder (ASD) is delay in language and social communication. Despite consensus on the benefits of earlier diagnosis and intervention, our understanding of the language growth of children with ASD during the first years of life remains limited. Therefore, this study compared communication growth patterns of infants and toddlers with ASD to growth benchmarks of a standardized language assessment. We conducted a retrospective analysis of growth on the Early Communication Indicator (ECI) of 23 infants and toddlers who received an ASD diagnosis in the future. At 42 months of age, children with ASD had significantly lower rates of gestures, single words, and multiple words, but significantly higher rates of nonword vocalizations. Children with ASD had significantly slower growth of single and multiple words, but their rate of vocalization growth was significantly greater than benchmark. Although more research is needed with larger samples, because the ECI was designed for practitioners to monitor children's response to intervention over time, these findings show promise for the ECI's use as a progress monitoring measure for young children with ASD. Limitations and the need for future research are discussed.

Development and validation of a progress monitoring tool tailored for use in intensive eating disorder treatment

OBJECTIVE

Despite calls for routine use of progress and outcome monitoring in private and intensive treatment centres for eating disorders (EDs), existing measures have limited relevance to these supervised treatment settings. This study sought to develop and validate the progress monitoring tool for eating disorders, a multidimensional measure for progress monitoring in the context of intensive ED treatment.

METHOD

Thirty-seven items were generated by a team of content experts, clinicians, and administrative staff from the target treatment setting. Adolescent and adult females (N = 531) seeking residential ED treatment completed the items at admission as part of the clinic's routine assessment battery; 83% were retained for repeat assessment at discharge. Exploratory factor analysis was conducted for preliminary measure development.

RESULTS

Results yielded a five-factor, 26-item structure explaining 50% of total variance. Final construct domains included weight and shape concern, ED behaviours and urges, emotion avoidance, adaptive coping, and relational connection. The measure demonstrated adequate internal consistency, sensitivity to change during treatment, and convergence with validated assessment measures.

CONCLUSIONS

Preliminary data support the progress monitoring tool for eating disorders as a novel and valid multidimensional measure of treatment-relevant constructs. This measure may have utility in measuring treatment progress for patients receiving intensive treatment for EDs.

Advancing Stage 2 Research on Measures for Monitoring Kindergarten Reading Progress

Although several measures exist for frequently monitoring early reading progress, little research has specifically investigated their technical properties when administered on a frequent basis with kindergarten students. In this study, kindergarten students ( N = 137) of whom the majority was receiving supplemental intervention for reading skills were monitored using Letter Sound Fluency, Phoneme Segmentation Fluency, Word Reading Fluency, Nonsense Word Fluency, Highly Decodable Passages, and Spelling on a biweekly basis between February and May. Acceptable reliability was observed for all measures. Analyses of slope validity using latent growth models, latent change score models, and slope differences according to level of year-end achievement indicated that the relation of slope to overall reading skills varied across the measures. A suggested approach to kindergarten students' reading progress is offered that includes Letter Sound Fluency and a measure of word-reading skills to provide a comprehensive picture of student growth toward important year-end reading outcomes.

Examining Predictive Validity of Oral Reading Fluency Slope in Upper Elementary Grades Using Quantile Regression

Within multitiered instructional delivery models, progress monitoring is a key mechanism for determining whether a child demonstrates an adequate response to instruction. One measure commonly used to monitor the reading progress of students is oral reading fluency (ORF). This study examined the extent to which ORF slope predicts reading comprehension outcomes for fifth-grade struggling readers ( n = 102) participating in an intensive reading intervention. Quantile regression models showed that ORF slope significantly predicted performance on a sentence-level fluency and comprehension assessment, regardless of the students' reading skills, controlling for initial ORF performance. However, ORF slope was differentially predictive of a passage-level comprehension assessment based on students' reading skills when controlling for initial ORF status. Results showed that ORF explained unique variance for struggling readers whose posttest performance was at the upper quantiles at the end of the reading intervention, but slope was not a significant predictor of passage-level comprehension for students whose reading problems were the most difficult to remediate.

More than de minimis: FAPE in the Post Endrew F. Era

Federal regulations for special education services have focused primarily on procedural issues since the Rowley decision, which held that Individualized Education Programs (IEPs) need only be reasonably calculated to yield educational benefit. However, the minimum threshold for benefit has changed with the recent Endrew F. decision as IEPs must yield more than de minimis progress. To ensure sufficient progress toward the achievement of ambitious goals, schools must develop IEPs that meet procedural and substantive requirements, employ interventions with clear evidence of effectiveness, effectively measure student response to services, and to communicate this information with parents/guardians so that they can actively participate in this process. Manuscripts invited for this special issue include investigations of IEP quality; co-teaching; intervention studies in reading, writing, and mathematics; meta-analytic findings regarding social studies education; and a discussion of the implications of Endrew F. for different student disability populations. These papers discuss challenges faced by stakeholders with vested interests in students with disabilities as well as areas of continued development and refinement in evidence-based practice.

What does this mean for graduate education in marriage and family therapy? Commentary on "The divide between 'evidenced-based' approaches and practitioners of traditional theories of family therapy"

The Dattilio, Piercy, and Davis article is a welcome addition to the conversation focusing on how to bridge the divide in the MFT field between research and practice. The present commentary challenges us to see the divide as an indictment of our training programs resulting from a lack of focus on MFT research. Suggestions for increasing expectations for students to monitor client progress, get involved in research at all levels, and for doctoral students to expect to be able to conduct independent, fundable research in the MFT field when they leave their programs are offered.

Shaky Student Growth? A Comparison of Robust Bayesian Learning Progress Estimation Methods

Monitoring the progress of student learning is an important part of teachers' data-based decision making. One such tool that can equip teachers with information about students' learning progress throughout the school year and thus facilitate monitoring and instructional decision making is learning progress assessments. In practical contexts and research, estimating learning progress has relied on approaches that seek to estimate progress either for each student separately or within overarching model frameworks, such as latent growth modeling. Two recently emerging lines of research for separately estimating student growth have examined robust estimation (to account for outliers) and Bayesian approaches (as opposed to commonly used frequentist methods). The aim of this work was to combine these approaches (i.e., robust Bayesian estimation) and extend these lines of research to the framework of linear latent growth models. In a sample of N = 4970 second-grade students who worked on the quop-L2 test battery (to assess reading comprehension) at eight measurement points, we compared three Bayesian linear latent growth models: (a) a Gaussian model, (b) a model based on Student's t-distribution (i.e., a robust model), and (c) an asymmetric Laplace model (i.e., Bayesian quantile regression and an alternative robust model). Based on leave-one-out cross-validation and posterior predictive model checking, we found that both robust models outperformed the Gaussian model, and both robust models performed comparably well. While the Student's t model performed statistically slightly better (yet not substantially so), the asymmetric Laplace model yielded somewhat more realistic posterior predictive samples and a higher degree of measurement precision (i.e., for those estimates that were either associated with the lowest or highest degree of measurement precision). The findings are discussed for the context of learning progress assessment.

Progress Monitoring in Inclusive Preschools: Using Children's School Success+Curriculum Framework

Progress monitoring in inclusive preschool classrooms should describe all children's progress towards general curriculum outcomes and individual children's unique outcomes or IEP goals. This research study used the CSS+ Curriculum Framework and progress monitoring process to assess the outcomes of 73 children on these dimensions. Children's progress monitoring data were analyzed within groupings based on instructional need level (i.e., low, medium, or high) in academic content and social domains. Progress monitoring findings for both the academic and social support level of need groups showed significant progress pre- to post-test on most academic outcomes, but some variation with less consistent gains within the social emotional domain. Goal attainment scaling data demonstrated children's gains toward achieving their social goals (IEP or specific learning goals) were at the expected level between 50% and 71% of the time. Academic-focused goal attainment was at or above the expected level of between 54% and 76% of the time, based on the learning grouping. Teacher implementation of CSS+ Curriculum Framework appeared to impact change in classroom and instructional practices pre-post intervention.

Graph Out Loud: Pre-Service Teachers' Data Decisions and Interpretations of CBM Progress Graphs

Data-based instruction (DBI) is a process in which teachers use progress data to make ongoing instructional decisions for students with learning disabilities. Curriculum-based measurement (CBM) is a common form of progress monitoring, and CBM data are placed on a graph to guide decision-making. Despite the central role that graph interpretation plays in the successful implementation of DBI, relatively little attention has been devoted to investigating this skill among special education teachers. In the present study, we examined the data decisions of 32 U.S. pre-service special education teachers (29 females and 3 males). Participants viewed data presented sequentially on CBM progress graphs and used a think-aloud procedure to explain their reasoning each time they indicated they would make instructional changes. We also asked participants to make the same type of decisions in response to static CBM progress graphs depicting 10 weeks of data. Overall, there was inconsistency in pre-service teachers' responses related to when or why they would make an instructional change. Decisions were often influenced by graph-related features, such as variability in the data. Furthermore, responses suggested misunderstandings that led to premature instructional change decisions and reliance on individual data points.

Construct Validity of Outcome Rating Scale (ORS) Scores in Clinical Samples: Extension of Harris, Murphy, and Rakes' (2019) Narrative Review

PURPOSE

A recent review of 24 studies primarily evaluated psychometric properties of the Outcome Rating Scale (ORS) related to coefficient alpha and convergent validity.

METHOD

To provide a fuller picture, this review examined effect size (ES) estimates and the full range of correlations between ORS scores and related measures.

RESULTS

Mean ORS ESs equaled 1.00 for 44 comparison periods. ORS scores correlated highest with measures of depression and evidenced larger ESs in depression treatments.

DISCUSSION

The ORS functions as a robust measure for detecting client progress in clinical samples that were primarily young, female, and Caucasian. Some results suggest ORS scores primarily reflect a rapid response to interventions.

CONCLUSION AND LIMITATIONS

Future research should assess ORS scores' psychometric properties in response to interventions with more heterogeneous client samples as well as its capacity to monitor change beyond initial sessions.

Educator Perspectives on Data-Based Decision Making in Germany and the United States

The purpose of this explorative study was to examine the use and understanding of key components of data-based decision making by educators in two countries-Germany and the United States. Educators responded to a survey that asked about data use and characteristics related to data-based decision making (DBDM). Results suggest educators in both countries are focused on using data to monitor progress, although less so in Germany. Educators in both countries noted similar understanding of important features (e.g., psychometric properties) of data. Educators in the United States reported they used data for decision making at the classroom level almost twice as often as their counterparts in Germany, while German educators focused on decision making at the student level. These findings will influence future research, including joint studies that could use the best practices of both countries, and professional learning opportunities for educators in Germany and the United States.