Developing a Measure of Therapist Adherence to Contingency Management: An Application of the Many-Facet Rasch Model

Toward Efficient, Sustainable, and Scalable Methods of Treatment Characterization: An Investigation of Coding Clinical Practice from Chart Notes

PURPOSE

Chart notes provide a low-cost data source that could help characterize what occurs in treatment with sufficient precision to improve management of care. This study assessed the interrater reliability of treatment content coded from chart notes and evaluated its concordance with content coded from transcribed treatment sessions.

METHOD

Fifty randomly selected and digitally recorded treatment events were transcribed and coded for practice content. Independent coders then applied the same code system to chart notes for these same treatment events.

ANALYSIS

We measured reliability and concordance of practice occurrence and extensiveness at two levels of specificity: practices (full procedures) and steps (subcomponents of those procedures).

RESULTS

For chart notes, practices had moderate interrater reliability (M k = 0.50, M ICC = 0.56) and steps had moderate (M ICC = 0.74) to substantial interrater reliability (M k = 0.78). On average, 2.54 practices and 5.64 steps were coded per chart note and 4.53 practices and 13.10 steps per transcript. Across sources, ratings for 64% of practices and 41% of steps correlated significantly, with those with significant correlations generally demonstrating moderate concordance (practice M r = 0.48; step M r = 0.47). Forty one percent of practices and 34% of steps from transcripts were also identified in the corresponding chart notes.

CONCLUSION

Chart notes provide an accessible data source for evaluating treatment content, with different levels of specificity posing tradeoffs for validity and reliability, which in turn may have implications for chart note interfaces, training, and new metrics to support accurate, reliable, and efficient measurement of clinical practice.

What is the Status of Multi-Informant Treatment Fidelity Research?

OBJECTIVE

The precise measurement of treatment fidelity (quantity and quality in the delivery of treatment strategies in an intervention) is essential for intervention development, evaluation, and implementation. Various informants are used in fidelity assessment (e.g., observers, practitioners [clinicians, teachers], clients), but these informants often do not agree on ratings. This scoping review aims to ascertain the state of science around multi-informant assessment of treatment fidelity.

METHOD

A literature search of articles published through December 2021 identified 673 articles. Screening reduced the number of articles to 44, and the final study set included 35 articles.

RESULTS

There was substantial variability across studies regarding study design, how fidelity was operationalized, and how reliability was defined and assessed. Most studies evaluated the agreement between independent observers and practitioner-report, though several other informant pairs were assessed. Overall, findings suggest that concordance across fidelity informants was low to moderate, with a few key exceptions.

CONCLUSIONS

It is difficult to draw clear conclusions about the degree to which single versus multiple informant assessment is needed to produce an accurate and complete picture of treatment fidelity. The field needs to take steps to determine how to leverage multi-informant assessment to accurately assess treatment fidelity.

Implementation of the Kidney Team at Home Intervention: Evaluating Generalizability, Implementation Process, and Effects

Research has shown that a home-based educational intervention for patients with chronic kidney disease results in better knowledge and communication, and more living donor kidney transplantations (LDKT). Implementation research in the field of renal care is almost non-existent. The aims of this study were (1) to demonstrate generalizability, (2) evaluate the implementation process, and (3) to assess the relationship of intervention effects on LDKT-activity. Eight hospitals participated in the project. Patients eligible for all kidney replacement therapies (KRT) were invited to participate. Effect outcomes were KRT-knowledge and KRT-communication, and treatment choice. Feasibility, fidelity and intervention costs were assessed as part of the process evaluation. 332 patients completed the intervention. There was a significant increase in KRT-knowledge and KRT-communication among participants. 129 out of 332 patients (39%) had LDKT-activity, which was in line with the results of the clinical trials. Protocol adherence, knowledge and age were correlated with LDKT-activity. This unique implementation study shows that the results in practice are comparable to the previous trials, and show that the intervention can be implemented, while maintaining quality. Results from the project resulted in the uptake of the intervention in standard care. We urge other countries to investigate the uptake of the intervention.

Capacity of juvenile probation officers in low-resourced, rural settings to deliver an evidence-based substance use intervention to adolescents

Substance use is a major public health problem with a host of negative outcomes. Justice-involved youth have even higher risks and lack access to evidence-based interventions, particularly in rural communities. Task-shifting, or redistribution of tasks downstream to an existing workforce with less training, may be an innovative strategy to increase access to evidence-based interventions. Initial findings are presented from a services research trial conducted primarily in rural communities in which an existing workforce, juvenile probation/parole officers (JPOs), were randomized either to learn and deliver contingency management (CM) or to continue delivering probation services as usual (PAU). This study used the prevailing version of CM for adolescents, that is, family-based with behavior modification and cognitive behavioral components. Data included JPOs' self-reports, as well as audio-recorded youth/family sessions with JPOs rated by expert and trained observational coders. Data also included ratings from a comparison study in which therapists were trained and supervised by experts to deliver CM to justice-involved youth/families. Results showed JPOs can feasibly incorporate CM into their services. When adherence of CM JPOs was compared against CM therapists, JPOs delivered significantly more cognitive behavioral components of CM and similar levels of behavior modification components of CM. These findings suggest that JPOs can be leveraged to provide evidence-based substance use interventions like CM in similar, or even greater, capacities to clinically trained therapists. This task-shifting approach could dramatically expand service access for these high-risk youth, particularly in rural areas where substance use services are limited or nonexistent. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Cost-effectiveness of a home-based group educational programme on renal replacement therapies: a study protocol

INTRODUCTION

Living donor kidney transplantation (LDKT) is the optimal treatment for most patients with end-stage renal disease (ESRD). However, there are numerous patients who cannot find a living kidney donor. Randomised controlled trials have shown that home-based education for patients with ESRD and their family/friends leads to four times more LDKTs. This educational intervention is currently being implemented in eight hospitals in the Netherlands. Supervision and quality assessment are being employed to maintain the quality of the intervention. In this study, we aim to: (1) conduct a cost-effectiveness analysis of the educational programme and its quality assurance system; (2) investigate the relationship between the quality of the implementation of the intervention and the outcomes knowledge, communication and LDKT activities; and (3) investigate policy implications.

METHODS AND DESIGN

Patients with ESRD who do not have a living kidney donor are eligible to receive the home-based educational intervention. This is carried out by allied health transplantation professionals and psychologists across eight hospitals in the Netherlands. The cost-effectiveness analysis will be conducted with a Markov model. Cost data will be obtained from the literature. We will obtain the quality of life data from the patients who participate in the educational programme. Questionnaires on knowledge and communication will be used to measure the outcomes of the programme. Data on LDKT activities will be obtained from medical records up to 24 months after the education. A protocol adherence measure will be assessed by a third party by means of a telephone interview with the patients and the invitees.

ETHICS AND DISSEMINATION

Ethical approval was obtained through all participating hospitals. Results will be disseminated through peer-reviewed publications and scientific presentations. Results of the cost-effectiveness of the educational programme will also be disseminated to the Dutch National Health Care Institute.

TRIAL REGISTRATION NUMBER

NL6529.

A Web-Delivered Multicomponent Intervention for Adolescents with Poorly Controlled Type 1 Diabetes: A Pilot Randomized Controlled Trial

Background

Type 1 diabetes is associated with significant mortality and economic cost. Management of type 1 diabetes involves completing multiple daily adherence behaviors, and many adolescents struggle with self-management and show poor glycemic control.

Purpose

The purpose was to conduct an unblinded pilot randomized controlled parallel-group study of a web-delivered multicomponent intervention targeting self-monitoring of blood glucose, working memory, and parent supervision of diabetes care among adolescents with type 1 diabetes. Intervention components included high magnitude incentives for adolescents and parents, motivational and cognitive behavioral therapy and working memory training for adolescents, and training in contingency contracting for parents.

Methods

Adolescents (N = 114) with poorly controlled type 1 diabetes were screened, and N = 61 were randomized using minimum likelihood allocation to usual care (usual care, N = 31) or to a 25-week/15-session web-delivered intervention (WebRx, N = 30).

Results

At the end of treatment, adolescents in WebRx had higher self-monitoring of blood glucose (d = 0.58) (primary outcome), better visual spatial working memory (d = 0.48) and inhibition (d = 0.98), and lower HbA1c (d = 0.45) than those in usual care. WebRx parents reported more frequent review of the adolescent's glucometer (d = 1.30) and reduced family conflict (d = 0.56). Between-condition differences were maintained 6 months later in self-monitoring of blood glucose (d = 0.42), visual spatial working memory (d = 0.76), family conflict (d = 0.50), and HbA1c (d = 0.44).

Conclusions

Results showing sustained effects on self-monitoring of blood glucose and HbA1c support moving forward with a larger trial to test this innovative web-delivered and multicomponent intervention. ClinicalTrials.gov Number (NCT01722643).

Developing an instrument for assessing fidelity of motivational care planning: The Aboriginal and Islander Mental health initiative adherence scale

Purpose

The aim of this study was to design and trial an Adherence Scale to measure fidelity of Motivational Care Planning (MCP) within a clinical trial. This culturally adapted therapy MCP uses a client centered holistic approach that emphasises family and culture to motivate healthy life style changes.

Methods

The Motivational Care Planning-Adherence Scale (MCP-AS) was developed through consultation with Aboriginal and Islander Mental Health Initiative (AIMhi) Indigenous and non-Indigenous trainers, and review of MCP training resources. The resultant ten-item scale incorporates a 9-Point Likert Scale with a supporting protocol manual and uses objective, behaviourally anchored criteria for each scale point. A fidelity assessor piloted the tool through analysis of four audio-recordings of MCP (conducted by Indigenous researchers within a study in remote communities in Northern Australia). File audits of the remote therapy sessions were utilised as an additional source of information. A Gold Standard Motivational Care Planning training video was also assessed using the MCP-AS.

Results

The Motivational Care Planning-Adherence Scale contains items measuring both process and content of therapy sessions. This scale was used successfully to assess therapy through observation of audio or video-recorded sessions and review of clinical notes. Treatment fidelity measured by the MCP-AS within the pilot study indicated high fidelity ratings. Ratings were high across the three domains of rapport, motivation, and self-management with especially high ratings for positive feedback and engagement, review of stressors and goal setting.

Conclusions

The Motivational Care Planning-Adherence Scale has the potential to provide a measure of quality of delivery of Motivation Care Planning. The pilot findings suggest that despite challenges within the remote Indigenous community setting, Indigenous therapists delivered therapy that was of high fidelity. While developed as a research tool, the scale has the potential to support fidelity of delivery of Motivation Care Planning in clinical, supervision and training settings. Larger studies are needed to establish inter-rater reliability and internal and external validity.

Evaluating training methods for transporting contingency management to therapists

The effects of three increasingly intensive training methods on therapist use, knowledge, and implementation adherence of contingency management (CM) with substance abusing adolescents were evaluated. Ten public sector substance abuse or mental health provider organizations were randomized to one of three training conditions: workshop and resources (WS+), WS+and computer assisted training (WS+/CAT), or WS+/CAT and supervisory support (WS+/CAT/SS). Across conditions, 161 therapists participated in the training experiences, and measures were obtained at baseline and 2-month intervals for 12 months following workshop participation. Across training conditions, therapists reported increased CM use, knowledge, and implementation adherence through the 12-month follow-up. The findings show that community-based practitioners are amenable to the adoption of evidence-based treatments when provided access to useful resources. Moreover, high quality workshops in combination with resource access can increase knowledge of the evidence-based treatment and might enhance intervention adherence to a level needed to improve youth outcomes.

Comparison of youth, caregiver, therapist, trained, and treatment expert raters of therapist adherence to a substance abuse treatment protocol

OBJECTIVE

This study evaluated the accuracy of youth, caregiver, therapist, and trained raters relative to treatment experts on ratings of therapist adherence to a substance abuse treatment protocol for adolescents.

METHOD

Adherence ratings were provided by youth and caregivers in an ongoing trial evaluating a Contingency Management (CM) intervention for youth in juvenile drug court. These ratings were compared to those provided by therapists and trained raters, and each rater type was compared to ratings provided by CM treatment experts. Data were analyzed using item-response-theory-based Many-Facet Rasch Models.

RESULTS

Relative to treatment experts, youth and caregivers were significantly more likely to endorse the occurrence of CM components. In contrast, therapists and trained raters were much more consistent with treatment experts. In terms of practical significance, youth and caregivers each had a 97% estimated probability of indicating that a typical treatment component had occurred. By comparison, the probability was 31%, 19%, and 26% for therapists, trained raters, and treatment experts, respectively.

CONCLUSIONS

Youth and caregivers were highly inaccurate relative to treatment experts, whereas, therapists and trained raters were generally consistent with treatment experts. The implications of these findings for therapist adherence measurement are considered.

Therapeutic Alliance Between Youth and Staff in Residential Group Care: Psychometrics of the Therapeutic Alliance Quality Scale

Therapeutic alliance has been frequently studied in individual counseling sessions; however, research on therapeutic alliance in residential settings for youth with mental health diagnoses has been limited. This may be due, in part, to the presence of multiple service providers often in caregiving roles. The purpose of this study was to examine the psychometric quality of a widely utilized measure of therapeutic alliance used in psychotherapy with youth in residential care where the treatment is provided by a trained married couple. We also compared the relationship between youth ratings of their male and female service provider, as well as examined correlations in ratings between youth and staff on therapeutic alliance. Finally, we investigated the direction, magnitude, and trajectory of change in therapeutic alliance over a 12-month period following admission into residential care. The method was a longitudinal assessment of 135 youth and 124 staff regarding therapeutic alliance over the course of 12 months or discharge from services. Results indicated strong psychometric properties and high correlations for youth ratings of both their male and female service providers. However, the correlation was low between youth and service provider ratings of alliance. Longitudinal analyses indicated that rates of therapeutic alliance changed over time.

System-level effects of integrating a promising treatment into juvenile drug courts

This study examined the system-level effects of implementing a promising treatment for adolescent substance abuse in juvenile drug courts (JDCs). Six JDCs were randomized to receive training in the experimental intervention (contingency management-family engagement [CM-FAM]) or to continue their usual services (US). Participants were 104 families served by the courts, 51 therapists, and 74 JDC stakeholders (e.g., judges, prosecutors, defense attorneys). Assessments included repeated measurements of CM-FAM implementation by therapists and therapist and stakeholder perceptions of incentive-based interventions and organizational characteristics. Results revealed greater use of CM and family engagement techniques among CM-FAM relative to US therapists. In addition, therapists and stakeholders in the CM-FAM condition reported more favorable attitudes toward the use of incentives and greater improvement on several domains of organizational functioning relative to US counterparts. Taken together, these findings suggest that JDC professionals are amenable to the adoption and implementation of a treatment model that holds promise for improving youth outcomes.

A review of treatment adherence measurement methods

Fidelity measurement is critical for testing the effectiveness and implementation in practice of psychosocial interventions. Adherence is a critical component of fidelity. The purposes of this review were to catalogue adherence measurement methods and assess existing evidence for the valid and reliable use of the scores that they generate and the feasibility of their use in routine care settings. A systematic literature search identified articles published between 1980-2008 reporting studies of evidence-based psychosocial treatments for child or adult mental health problems, including mention of adherence or fidelity assessment. Coders abstracted data on the measurement methods and clinical contexts of their use. Three hundred forty-one articles were reviewed, in which 249 unique adherence measurement methods were identified. These methods assessed many treatment models, although more than half (59%) assessed cognitive behavioral treatments. The measurement methods were used in studies with diverse clientele and clinicians. The majority (71.5%) of methods were observational. Information about psychometric properties was reported for 35% of the measurement methods, but adherence-outcomes relationships were reported for only 10%. Approximately 1/3 of the measures were used in community-based settings. Many adherence measurement methods have been used in treatment research; however, little reliability and validity evidence exists for the use of these methods. That some methods were used in routine care settings suggests the feasibility of their use in practice; however, information about the operational details of measurement, scoring, and reporting is sorely needed to inform and evaluate strategies to embed fidelity measurement in implementation support and monitoring systems.

Contingency management in substance abuse treatment: a structured review of the evidence for its transportability

AIMS

Extant literature on contingency management (CM) transportability, or its transition from academia to community practice, is reviewed. The Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009) guides the examination of this material.

METHODS

PsychInfo and Medline database searches identified 27 publications, with reviewed reference lists garnering 22 others. These 49 sources were examined according to CFIR domains of the intervention, outer setting, inner setting, clinicians, and implementation processes.

RESULTS

Intervention characteristics were focal in 59% of the identified literature, with less frequent focus on clinicians (34%), inner setting (32%), implementation processes (18%), and outer setting (8%). As intervention characteristics, adaptability and trialability most facilitate transportability whereas non-clinical origin, perceived inefficacy or disadvantages, and costs are impediments. Clinicians with a managerial focus and greater clinic tenure and CM experience are candidates to curry organizational readiness for implementation, and combat staff disinterest or philosophical objection. A clinic's technology comfort, staff continuity, and leadership advocacy are inner setting characteristics that prompt effective implementation. Implementation processes in successful demonstration projects include careful fiscal/logistical planning, role-specific staff engagement, practical adaptation in execution, and evaluation via fidelity-monitoring and cost-effectiveness analyses. Outer setting characteristics-like economic policies and inter-agency networking or competition-are salient, often unrecognized influences.

CONCLUSIONS

As most implementation constructs are still moving targets, CM transportability is in its infancy and warrants further scientific attention. More effective dissemination may necessitate that future research weight emphasis on external validity, and utilize models of implementation science.

A randomized trial of contingency management delivered by community therapists

OBJECTIVE

Contingency management (CM) is an evidence-based treatment, but few clinicians deliver this intervention in community-based settings.

METHOD

Twenty-three clinicians from 3 methadone maintenance clinics received training in CM. Following a didactics seminar and a training and supervision period in which clinicians delivered CM to pilot patients, a randomized trial evaluated the efficacy of CM when delivered entirely by clinicians. Sixteen clinicians treated 130 patients randomized to CM or standard care. In both conditions, urine and breath samples were collected twice weekly for 12 weeks. In the CM condition, patients earned the opportunity to win prizes ranging in value from $1 to $100 for submitting samples negative for cocaine and alcohol. Primary treatment outcomes were retention, longest continuous period of abstinence, and proportion of negative samples submitted.

RESULTS

Patients randomized to CM remained in the study longer (9.5 ± 3.6 vs. 6.7 ± 5.0 weeks), achieved greater durations of abstinence (4.7 ± 4.7 vs. 1.7 ± 2.7 weeks), and submitted a higher proportion of negative samples (57.7% ± 40.0% vs. 29.4% ± 33.3%) than those assigned to standard care.

CONCLUSIONS

These data indicate that, with appropriate training, community-based clinicians can effectively administer CM. This study suggests that resources ought to be directed toward training and supervising community-based providers in delivering CM, as patient outcomes can be significantly improved by integrating CM in methadone clinics.

Enhancing the effectiveness of juvenile drug courts by integrating evidence-based practices

OBJECTIVE

The primary purpose of this study was to test a relatively efficient strategy for enhancing the capacity of juvenile drug courts (JDC) to reduce youth substance use and criminal behavior by incorporating components of evidence-based treatments into their existing services.

METHOD

Six JDCs were randomized to a condition in which therapists were trained to deliver contingency management in combination with family engagement strategies (CM-FAM) or to continue their usual services (US). Participants included 104 juvenile offenders (average age = 15.4 years; 83% male; 57% White, 40% African American, 3% Biracial). Eighty-six percent of the youths met criteria for at least 1 substance use disorder, and co-occurring psychiatric diagnoses were highly prevalent. Biological and self-report measures of substance use and self-reported delinquency were assessed from baseline through 9 months postrecruitment.

RESULTS

CM-FAM was significantly more effective than US at reducing marijuana use, based on urine drug screens, and at reducing both crimes against persons and property offenses. Such favorable outcomes, however, were not observed for the self-report measure of substance use. Although some variation in outcomes was observed between courts, the outcomes were not moderated by demographic characteristics or co-occurring psychiatric disorders.

CONCLUSIONS

The findings suggest that JDC practices can be enhanced to improve outcomes for participating juvenile offenders. A vehicle for promoting such enhancements might pertain to the development and implementation of program certification standards that support the use of evidence-based interventions by JDCs. Such standards have been fundamental to the successful transport of evidence-based treatments of juvenile offenders.

Therapist training in empirically supported treatments: a review of evaluation methods for short- and long-term outcomes

Therapist training efforts have been assessed using several outcomes. A model for evaluating therapist training in empirically supported treatments is presented, adapted from Kirkpatrick's (in: Craig and Bittel (eds.) Training and development handbook, 1967) training evaluation model. The adapted framework includes short-term outcomes, such as reactions to training and changes in attitude, knowledge, or skills, and longer-term outcomes, such as changes in therapist behavior in practice or client outcomes. Evaluation methods for these outcomes are reviewed, with information on their validity, reliability, and feasibility. An agenda for further research to improve therapist training evaluation is presented, with discussion of how evaluation can inform other areas of the field.

Intensive Quality Assurance of Therapist Adherence to Behavioral Interventions for Adolescent Substance Use Problems

This study was a crosscultural replication of a study that investigated therapist adherence to behavioral interventions as a result of an intensive quality assurance system which was integrated into Multisystemic Therapy. Thirty-three therapists and eight supervisors participated in the study and were block randomized to either an Intensive Quality Assurance or a Workshop Only condition. Twenty-one of these therapists treated 41 cannabis-abusing adolescents and their families. Therapist adherence and youth drug screens were collected during a five-month baseline period prior to the workshop on contingency management and during 12 months post workshop. The results replicated the previous finding that therapist adherence to the cognitive-behavioral interventions, but not to contingency management, showed a strong positive difference in trend in favor of the intensive quality assurance condition. While the clinical impact of such quality assurance may be delayed and remains to be demonstrated, cannabis abstinence increased as a function of time in therapy, and was more likely with stronger therapy adherence to contingency management, but did not differ across quality assurance interventions.

Psychometric properties of the contingency management competence scale

Contingency management (CM) is an evidence-based treatment, and clinicians are beginning to implement this intervention in practice. However, little research exists on methods for assuring appropriate implementation of CM. This study describes the development and psychometric properties of the 12-item CM Competence Scale (CMCS). Thirty-five therapists from nine community-based clinics participated; following a training period, a randomized trial evaluated the efficacy of CM in cocaine abusing patients. Analyses of the CMCS are based on ratings from 1613 audiotapes of therapist interactions with 78 patients enrolled in the training phase and 103 patients in the randomized phase. Inter-rater reliability from 11 raters and internal consistency of items on the CMCS was good to excellent. Items loaded onto two factors: one contained items specific to discussions of the outcomes of urine testing and reinforcement, and the other contained general items related to use of praise, communication of confidence, empathy, skillfulness, and maintaining session structure, as well as discussions of self-reports of drug use when they occurred. During the training phase in CM delivery, scores on the CMCS rose significantly between earlier and later training sessions, and during the randomized phase, CM sessions were rated more highly than non-CM sessions. Scores on the subscale assessing general items were significantly correlated with indices of the therapeutic alliance and predictive of durations of cocaine abstinence achieved. These data suggest that the CMCS is reliable and valid in assessing delivery of CM and that competence in CM delivery is associated with improved patient outcomes.

Statewide adoption and initial implementation of contingency management for substance-abusing adolescents

Four hundred thirty-two public sector therapists attended a workshop in contingency management (CM) and were interviewed monthly for the following 6 months to assess their adoption and initial implementation of CM to treat substance-abusing adolescent clients. Results showed that 58% (n = 131) of the practitioners with at least one substance-abusing adolescent client (n = 225) adopted CM. Rates of adoption varied with therapist service sector (mental health vs. substance abuse), educational background, professional experience, and attitudes toward treatment manuals and evidence-based practices. Competing clinical priorities and client resistance were most often reported as barriers to adopting CM, whereas unfavorable attitudes toward and difficulty in implementing CM were rarely cited as barriers. The fidelity of initial CM implementation among adopters was predicted by organizational characteristics as well as by several demographic, professional experience, attitudinal, and service sector characteristics. Overall, the findings support the amenability of public sector practitioners to adopt evidence-based practices and suggest that the predictors of adoption and initial implementation are complex and multifaceted.

Promoting the implementation of an evidence-based intervention for adolescent marijuana abuse in community settings: testing the use of intensive quality assurance

The development and evaluation of effective strategies for transporting evidence-based practices to community-based clinicians has become a research and policy priority. Using multisystemic therapy programs as a platform, an experimental design examined the capacity of an Intensive Quality Assurance (IQA) system to promote therapist implementation of contingency management (CM) for adolescent marijuana abuse. Participants included 30 therapists assigned to Workshop Only (WSO) versus IQA training conditions, and 70 marijuana-abusing youths and their caregivers who were treated by these clinicians. Analyses showed that IQA was more effective than WSO at increasing practitioner implementation of CM cognitive-behavioral techniques in the short-term based on youth and caregiver reports, and these increases were sustained based on youth reports. On the other hand, IQA did not increase therapist use of CM monitoring techniques relative to WSO, likely because of an unanticipated ceiling effect. Both sets of findings contribute to the emerging literature on the transport of evidence-based practice to real-world clinical settings.