Systematic literature review and meta-analysis of the relationship between adherence, competence and outcome in psychotherapy for children and adolescents

Provider Adherence to Modular Cognitive Behavioral Therapy for Children and Adolescents

This study explored predictors of community-based providers' adherence to MATCH, a modular cognitive behavioral therapy for children and adolescents. Provider-reported adherence to MATCH was measured using three increasingly strict criteria: (1) session content (whether the session covered MATCH content consistent with the client's target problem), (2) session content and sequencing (whether the session covered MATCH content in the expected sequence for the client's target problem), and (3) session content, sequencing, and participant (whether the session covered MATCH content in the expected sequence and with the expected participant(s) for the client's target problem). Session, client, provider, and organizational predictors of adherence to MATCH were assessed using multilevel modeling. Results revealed that nearly all providers delivered MATCH content that corresponded to the target problem, but only one-third of providers delivered MATCH content in the expected sequence and with the expected participant for the client's target problem. This difference underscores the need for nuanced adherence measurement to capture important implementation information that broad operationalizations of adherence miss. Regardless of the criteria used providers were most adherent to MATCH during sessions when clients presented with interfering comorbid mental health symptoms. This suggests that the design of MATCH, which offers flexibility and structured guidance to address comorbid mental health problems, may allow providers to personalize treatment to address interfering comorbidity symptoms while remaining adherent to evidence-based practices. Additional guidance for providers on managing other types of session interference (e.g., unexpected events) may improve treatment integrity in community settings.

Supervisory Alliance as a Moderator of the Effects of Behavioral Rehearsal on TF-CBT Fidelity: Results from a Randomized Trial of Supervision Strategies

Exposure is an important element of treatment for many evidence-based treatments but can be challenging to implement. Supervision strategies to support exposure delivery may be an important tool to facilitate the use of exposure techniques; however, they must be considered and used in the context of the supervisory alliance. The present study examined relations between supervisory alliance and fidelity to the trauma narrative (TN; i.e., imaginal exposure) component of Trauma-Focused Cognitive Behavioral Therapy. We also examined how supervisory alliance moderated the effect of behavioral rehearsal use in supervision on TN fidelity. We analyzed data from a randomized controlled trial, in which forty-two supervisors and their clinicians (N = 124) from 28 Washington State community-based mental health offices participated. Clinicians were randomized to receive one of two supervision conditions-symptom and fidelity monitoring (SFM) or SFM with behavioral rehearsal (SFM + BR). Supervisory alliance alone did not predict delivery (i.e., occurrence) or extensiveness of delivery of the trauma narrative. Client-focused supervisory alliance moderated the effectiveness of behavioral rehearsal-as client-focused alliance increased, the odds of delivering the TN also increased significantly. Future research should further investigate how to appropriately match supervision techniques with supervisory dyads and explore the interplay of alliance with supervision techniques a supervisor might employ.

Mediators of Outcome in Trauma-Focused Psychotherapy with Youth: A Systematic Review

This article aimed to provide a systematic narrative synthesis of existing studies on the mediators of change in psychotherapy with adolescents (10-19 years) and transition age youth (TAY) (20-29 years) who have experienced trauma-related symptoms or posttraumatic disorder. Additionally, we were interested in identifying psychotherapy-, trauma type-, and clients' age- and gender-specific mediators of treatment outcome. Following the preferred reporting items for systematic reviews and meta-analyses, a total of 3,723 studies published in PubMed and PsycINFO databases were screened against inclusion criteria, revealing 15 eligible studies. No studies with only TAY were found; therefore, all results were limited to therapy with adolescents. Cognitive mediators were tested in 66% of selected studies, followed by parents/family-related, mental-health-related, therapy-related, and behavioral mediators. Moderate evidence was found for posttraumatic cognitions, whereas therapeutic alliance seemed to be a promising candidate for future research. Striking absence of non-cognitive-behavioral therapy interventions, emotional and adolescent-specific mediators, as well as studies with males and in non-Western societies was evident. Future original studies would benefit from applying methodological rigor in respect to mediation testing.

Practitioner Adherence and Competence in MEYA, a Free Online Self-Instruction Program in Modular Psychotherapy and Counseling for Children's Autism-Related Clinical Needs

The quality of care in public schools and other community settings for school-aged youths on the autism spectrum is variable and often not evidence-based. Training practitioners in these settings to deliver evidence-based practices (EBPs) may improve the quality of care. We developed a free internet-based training and clinical guidance system synthesizing multiple EBPs for youth on the autism spectrum addressing a range of mental health needs and autism-related behaviors, entitled Modular EBPs for Youth on the Autism Spectrum (MEYA; meya.ucla.edu). A multiple baseline study was conducted with seven practitioners recruited from mental health practice settings across the United States who were providing services to children on the autism spectrum (aged 6 to 17 years). Practitioners were randomly assigned to undergo baseline conditions of 2 to 8 weeks. Once online training in MEYA commenced, practitioners engaged in algorithm-guided self-instruction in EBPs for autism. Participants video-recorded sessions. Independent coders used the MEYA Fidelity Scale (MEYA-FS) to rate adherence and competence in EBPs for autism. Practitioners also completed measures pertaining to implementation outcomes and parents rated youth outcomes on personalized target behaviors. Five of seven practitioners increased their adherence to MEYA practices (i.e., MEYA-FS scores) following MEYA training. Findings for competence were similar, though somewhat less robust. Practitioners generally viewed MEYA as feasible, understandable, and acceptable. Most youth outcomes improved during MEYA. A randomized, controlled trial of MEYA would be helpful in characterizing its effectiveness for supporting practitioner EBP implementation and youth outcomes in school and community service settings.

Therapist competence, homework engagement, and client characteristics in CBT for youth depression: A study of mediation and moderation in a community-based trial

OBJECTIVE

Prior studies of Cognitive Behavioral Therapy (CBT) have focused on the quantity and quality of clients' homework completion and only rarely have considered the role of therapist competence.

METHODS

The present study examined (a) therapist competence across the entire process of integrating homework into CBT, including the review, design, and planning of tasks; (b) homework engagement, including client appraisals of the difficulty and obstacles encountered in task completion using the Homework Rating Scale - Revised (HRS-II); (c) pre-post symptom reduction as the index of outcome; and (d) considered client factors such as suicide risk in a community-based trial for adolescent depression. Trained independent observers assessed therapist competence and engagement with homework at two consecutive sessions of CBT for N = 80 young people (Mage = 19.61, SD = 2.60).

RESULTS

Significant complementary mediation effects were obtained; there was an indirect mediation effect of HRS-II Beliefs (b = 1.03, SE B = 0.42, 95% BCa CI [0.35, 2.03]) and HRS-II Perceived Consequences on the Competence-Engagement relationship (b = 0.85, SE B = 0.31, 95% BCa CI [0.39, 1.61]). High levels of suicidal ideation were also shown to moderate this relationship.

CONCLUSIONS

The present findings contribute to the growing body of CBT process research designed to examine the complex interrelationships of client and therapist variables, in a manner that reflects the actual process of therapy, and advances beyond studies of isolated predictors of symptom change.

Opening the Black Box of Family-Based Treatments: An Artificial Intelligence Framework to Examine Therapeutic Alliance and Therapist Empathy

The evidence-based treatment (EBT) movement has primarily focused on core intervention content or treatment fidelity and has largely ignored practitioner skills to manage interpersonal process issues that emerge during treatment, especially with difficult-to-treat adolescents (delinquent, substance-using, medical non-adherence) and those of color. A chief complaint of "real world" practitioners about manualized treatments is the lack of correspondence between following a manual and managing microsocial interpersonal processes (e.g. negative affect) that arise in treating "real world clients." Although family-based EBTs share core similarities (e.g. focus on family interactions, emphasis on practitioner engagement, family involvement), most of these treatments do not have an evidence base regarding common implementation and treatment process problems that practitioners experience in delivering particular models, especially in mid-treatment when demands on families to change their behavior is greatest in treatment - a lack that characterizes the field as a whole. Failure to effectively address common interpersonal processes with difficult-to-treat families likely undermines treatment fidelity and sustained use of EBTs, treatment outcome, and contributes to treatment dropout and treatment nonadherence. Recent advancements in wearables, sensing technologies, multivariate time-series analyses, and machine learning allow scientists to make significant advancements in the study of psychotherapy processes by looking "under the skin" of the provider-client interpersonal interactions that define therapeutic alliance, empathy, and empathic accuracy, along with the predictive validity of these therapy processes (therapeutic alliance, therapist empathy) to treatment outcome. Moreover, assessment of these processes can be extended to develop procedures for training providers to manage difficult interpersonal processes while maintaining a physiological profile that is consistent with astute skills in psychotherapeutic processes. This paper argues for opening the "black box" of therapy to advance the science of evidence-based psychotherapy by examining the clinical interior of evidence-based treatments to develop the next generation of audit- and feedback- (i.e., systemic review of professional performance) supervision systems.

Setting and Provider Predictors of Implementation Success for an Eating Disorder Prevention Program Delivered by College Peer Educators

INTRODUCTION

College students face increased risk for a variety of mental health problems but experience barriers to treatment access. Prevention programs, including those implemented by peer educators, may decrease treatment needs and increase service access. We examined the implementation of an evidence-based eating disorder prevention program, Body Project, delivered by college peer educators at 63 colleges/universities, comparing three levels of implementation support: (1) Train-the-Trainer (TTT) training; (2) TTT plus a technical assistance workshop (TTT + TA); and (3) TTT + TA with one year of quality assurance calls (TTT + TA + QA). The present study tested the degree to which indicators proposed by the Consolidated Framework for Implementation Research (CFIR) were associated with core implementation outcomes.

METHOD

We tested whether indices of CFIR domains (i.e., perceived intervention characteristics, outer and inner setting factors, provider characteristics, and implementation process) were correlated with three implementation outcomes (program reach, fidelity, effectiveness) during a 1-year implementation period.

RESULTS

Greater program reach was associated with implementation process, specifically the completion of more implementation activities (β = 0.46). Greater program fidelity was associated with higher positive (β = 0.44) and lower negative (β = - 0.43) perceptions of the Body Project characteristics, and greater reported general support for evidence-based practices (β = 0.41). Greater effectiveness was associated with lower negative perceptions of Body Project characteristics (d = 0.49).

CONCLUSIONS

Several implementation determinants proposed by the CFIR model predicted outcomes, especially intervention fidelity. Across the outcomes of interest, implementation determinants related to peer educator and supervisor perceived characteristics of the specific intervention and general attitudes towards evidence-based practices emerged as robust predictors to inform future work investigating ongoing implementation and sustainability of programs in university settings.

Disparities in Youth and Family Experiences of System-of-Care Principles by Level of Youth Need

The extent to which mental health services for youths embody system-of-care (SOC) principles is an important quality indicator. This study tested whether youth and family experiences of SOC principles varied depending on youths' level of need after adjusting for sociodemographic and treatment factors. The relationship to caregiver-reported clinical outcomes was also examined. Using administrative data and cross-sectional surveys from a stratified random sample of 1124 caregivers of youths ages 5-20 within a statewide system, adjusted analyses indicated caregivers of youths with the most intensive needs were significantly less likely to report receiving care that embodied SOC principles, with deficits on six of nine items. Youths whose services embodied SOC principles experienced significantly greater improvement in caregiver-reported functioning even after adjusting for level of need. Results highlight disparities in SOC principles for youths with intensive needs and the need for policy and intervention development to improve care for this population.

A Comparison of the Readiness of Youth Service Agencies to Implement a Technology-Based Toolkit to Support Treatment Delivery

Implementation initiatives and technology-based resources aim to address barriers to Evidence-Based Practice (EBP) use by creating generalizable techniques that can be used for a variety of youth-serving agencies. However, research has not carefully examined unique differences between agency types or individual programs in readiness to use such technologies and implementation strategies. The current study explored differences between community mental health clinics and child advocacy centers on organizational cultural factors (e.g., ability to change and commitment for change) to implement a novel technology-based toolkit to support delivery of Trauma Focused Cognitive Behavioral Therapy (TF-CBT). Results indicated that TF-CBT providers from child advocacy centers reported greater commitment to change and more support to use the technology-based system than those from community mental health centers. Findings suggest that implementation initiatives should address the needs of individual agencies and service settings and adaptations should be explored to best meet the needs of these settings.

Comprehensive Prevention: An Evaluation of Peripheral Outcomes of a School-based Prevention Program

By age 18, 22-27% of adolescents have experienced depressive symptoms increasing their risk of peripheral mental health and social issues. Despite the development of effective depression prevention programs, issues related to dissemination persist. This study aims to identify ways of increasing the likelihood of dissemination by a) investigating how prevention effects differ based on the professional background of the prevention program group leader and b) evaluating adolescent depression prevention in terms of comprehensive prevention - prevention with the breadth to reduce peripheral mental health and social issues. This cluster-randomized trial included 646 eighth-grade students recruited from German secondary schools. Adolescents were randomized into three conditions: teacher-led prevention, psychologist-led prevention, or school-as-usual. Results from hierarchical linear models reveal differences in effects based on implementation type and adolescent gender and provide preliminary evidence for a wider reach of depression prevention such that, regardless of implementation type or gender, the tested program was effective in reducing hyperactivity over time. Taken together, our findings warrant further research and suggest that depression prevention programs may have an effect on some peripheral outcomes, but not others, and that these effects may differ based on the profession of the group leader and adolescent gender. With continued empirical research investigating the efficaciousness of comprehensive prevention, this type of prevention has the potential to impact a larger proportion of the population and improve the cost-benefit ratio of prevention, thus increasing the likelihood of dissemination.

A Psychometric Analysis of the Patterns of Adapted Learning Scales (PALS) with a Group Psychotherapy Patient Sample: A Measure of Motivation

Previous research has examined the role of clients' goal orientation, or the motives one uses to achieve specific tasks or goals in individual therapy highlighting the importance of clients' goal orientation within a therapeutic context. Constructing and validating a measure of clients' achievement goal orientation in group psychotherapy is needed to allow for additional methodologically sound research to examine the role of achievement goal orientation in the process and outcome of group psychotherapy. Data from 99 adult group members across 23 interpersonal process groups were used to investigate the psychometric properties of the Patterns of Adapted Learning Scales-Group Psychotherapy Version (PALS-GP), including internal reliability, structural validity, and concurrent and criterion-related validity. The PALS-GP scores revealed an overall good internal consistency (α = .76-.86) and yielded a robust three-factor structure. Evidence of concurrent and predictive validity was also demonstrated with a measure of group alliance and client improvement, respectively. The PALS-GP appears to be a promising instrument for measuring goal orientation with a clinical population participating in group therapy.

Empirical progression criteria thresholds for feasibility outcomes in HIV clinical trials: a methodological study

INTRODUCTION

Pilot and feasibility trials use predetermined thresholds for feasibility outcomes to decide if a larger trial is feasible. These thresholds may be derived from the literature, observational data, or clinical experience. The aim of this study was to determine empirical estimates for feasibility outcomes to inform future HIV pilot randomized trials.

METHODS

We conducted a methodological study of HIV clinical trials indexed in the past 5 years (2017-2021) in the PubMed database. We included trials of people living with HIV individually randomized to any type of intervention and excluded pilot trials and cluster randomized trials. Screening and data extraction were conducted in duplicate. We computed estimates for recruitment, randomization, non-compliance, lost to follow-up, discontinuation, and the proportion analyzed using a random effects meta-analysis of proportions and reported these estimates according to the following subgroups: use of medication, intervention type, trial design, income level, WHO region, participant type, comorbidities, and source of funding. We report estimates with 95% confidence intervals.

RESULTS

We identified 2122 studies in our search, of which 701 full texts were deemed relevant, but only 394 met our inclusion criteria. We found the following estimates: recruitment (64.1%; 95% CI 57.7 to 70.3; 156 trials); randomization (97.1%; 95% CI 95.8 to 98.3; 187 trials); non-compliance (3.8%; 95% CI 2.8 to 4.9; 216 trials); lost to follow-up (5.8%; 95% CI 4.9 to 6.8; 251 trials); discontinuation (6.5%; 95% CI 5.5 to 7.5; 215 trials); analyzed (94.2%; 95% CI 92.9 to 95.3; 367 trials). There were differences in estimates across most subgroups.

CONCLUSION

These estimates may be used to inform the design of HIV pilot randomized trials with careful consideration of variations due to some of the subgroups investigated.

Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation

BACKGROUND

Implementing evidence-based programs in community service settings introduces the challenge of ensuring sustained fidelity to the original program. We employ a fidelity measure based on direct observation of practitioners' competence and adherence to the evidence-based parenting program (EBPP) GenerationPMTO following installation in national and international sites. Fidelity monitoring is crucial, especially when the program purveyor transfers administration of the program to the community as was done in this case. In previous studies, the Fidelity of Implementation rating system (FIMP) was used to evaluate practitioners' fidelity to the GenerationPMTO intervention in six countries following implementation showing high levels of adherence up to 17 years post certification. Other studies showed FIMP to have predictive validity. The present study provides inter-rater reliability data for this fidelity tool across teams of the purveyor, Implementation Sciences International, Inc./ISII, and national and international sites over a five-year period.

METHODS

Data assess inter-rater reliability in terms of percent agreement and intraclass correlation (ICC) for the purveyor's two fidelity teams and the fidelity teams in seven implementation sites.

RESULTS

Results report stable good to excellent levels of inter-rater reliability and ICCs as well as good attendance at fidelity meetings for all fidelity teams.

CONCLUSIONS

This observational method of assessing fidelity post implementation is a promising approach to enable EBPPs to be transferred safely from purveyors to communities while maintaining reliable fidelity to the intervention.

The Therapy Process Observational Coding System: Group cohesion scale in youth anxiety treatment: Psychometric properties

BACKGROUND

There are no well-established measures of group cohesion, defined as the collaborative bond between group members, in group cognitive behavioral therapy (GCBT) with youth. We therefore examined the Therapy Process Observational Coding System for Child Psychotherapy-Group Cohesion Scale (TPOCS-GC), which has previously only been used with adult samples, in a youth sample.

METHODS

Observers coded 32 sessions from 16 groups with 83 youth aged 8 to 15 years (90.7% European White). Youth had anxiety disorders and received manualized GCBT in community clinics. We examined psychometric properties of the TPOCS-GC and its' construct validity in terms of relations with pretreatment variables, alliance and fidelity during treatment, and post-treatment variables. Group cohesion was measured twice during treatment (early and late).

RESULTS

The TPOCS-GC was internally consistent (α = 0.72) and was reliably coded (M ICC = 0.61). Higher clinical severity at pretreatment predicted lower early group cohesion. Higher youth age, higher clinical severity at pretreatment, and higher youth-rated early alliance predicted lower late group cohesion. Higher therapist-rated early alliance predicted higher early group cohesion. Higher therapist-rated late alliance predicted higher late group cohesion. Higher late group cohesion predicted lower clinical severity and higher client treatment satisfaction at post-treatment. Early group cohesion did not predict any post-treatment variables.

CONCLUSIONS

A four-item version of the TPOCS-GC can be reliably used in youth GCBT. The TPOCS-GC is distinct from, but associated with, multiple clinical variables.

Adherence, Competence, and Alliance as Predictors of Long-term Outcomes of Cognitive Behavioral Therapy for Youth Anxiety Disorders

The present study investigated therapist adherence, therapist competence, and patient-therapist alliance as predictors of long-term outcomes of cognitive behavioral therapy (CBT) for anxiety disorders in youth. Potential differential effects for group versus individual CBT, for therapists with or without formal CBT training, and based on youth symptom severity were examined. Videotapes (n = 181) from treatment sessions in a randomized controlled effectiveness trial comprising youth (N = 170, M age = 11.6 years, SD = 2.1) with anxiety disorders were assessed for therapist adherence and competence. Alliance was rated by therapists and youth. Participants completed a diagnostic interview and an anxiety symptom measure at pre-treatment, post-treatment, one-year follow-up, and long-term follow-up (M = 3.9 years post-treatment, SD = 0.8, range = 2.2-5.9 years). The change in anxiety symptoms or diagnostic status from pre-treatment to long-term follow-up was not significantly related to any predictor variables. However, several interaction effects were found. For loss of principal diagnosis, therapist competence predicted positive outcome when therapist adherence also was high. Adherence was found to predict positive outcome if CBT was provided individually. Therapist-rated alliance was related to both loss of principal diagnosis and loss of all diagnoses when CBT was provided in groups. Interaction effects suggested that therapists displaying both high adherence and high competence produced better long-term outcomes. Further, the alliance may be particularly important for outcomes in group CBT, whereas adherence may be particularly important for outcomes in individual CBT.

Pragmatic fidelity measurement in youth service settings

Background

Fidelity measurement is critical for developing, evaluating, and implementing evidence-based treatments (EBTs). However, traditional fidelity measurement tools are often not feasible for community-based settings. We developed a short fidelity rating form for the Collaborative Problem Solving (CPS) approach from an existing manualized coding system that requires extensive training. We examined the reliability and accuracy of this short form when completed by trained observers, untrained observers, and self-reporting providers to evaluate multiple options for reducing barriers to fidelity measurement in community-based settings.

Methods

Community-based treatment providers submitted recordings of youth service sessions in which they did, or did not, use CPS. For 60 recordings, we compared short-form fidelity ratings assigned by trained observers and untrained observers to those provided by trained observers on the manualized coding system. For 141 recordings, we compared providers' self-reported fidelity on the short form to ratings provided by trained observers on the manualized coding system and examined providers' accuracy as a function of their global fidelity.

Results & Conclusions

The short form was reliable and accurate for trained observers. An assigned global integrity score and a calculated average of component scores on the short form, but not component scores themselves, were reliable and accurate for observers who had CPS expertise but no specific training on rating CPS fidelity. When providers self-reported fidelity on the short form, their global integrity score was a reliable estimate of their CPS integrity; however, providers with better CPS fidelity were most accurate in their self-reports. We discuss the costs and benefits of these more pragmatic fidelity measurement options in community-based settings.

Prevalence and predictors of medication non-adherence in children with inflammatory bowel disease in China: A cross-sectional study

Background: Non-adherence to physician-prescribed medications, especially oral medications, is common in children with inflammatory bowel disease (IBD), and medication non-adherence is associated with poorer outcomes in IBD. Therefore, understanding and improving medication adherence in children with IBD is critical for optimizing treatment and improving treatment outcomes. Despite the relatively high prevalence of IBD in children in China, to date, very little is known about medication adherence in these patients.

Objective: The aim of this study was to investigate the prevalence of medication non-adherence and its risk factors in children with IBD in China to provide a basis for developing adherence improvement strategies.

Methods: A cross-sectional design was employed. Children (aged <18 years) with IBD who visited the Children’s Hospital, Zhejiang University School of Medicine, from September 2020 to December 2021 were included. Data were collected by a general information questionnaire, the 4-item Medication Adherence Report Scale (MARS-4) and Crohn’s and Colitis Knowledge (CCKNOW) questionnaire. Subsequently, forward stepwise binary logistic regression analysis was performed to determine independent predictors of medication non-adherence.

Results: A total of 119children were included in this study. The results showed that 33 (27.73%) and 86 (72.27%) children had poor and good medication adherence, respectively. Of these, 83 (69.75%) had forgotten to take their medications sometimes, often, or always. On binary logistic regression, we found that the incidence of medication non-adherence in children with IBD course of 3 years and above [OR 4.82 (95%CI: 1.47-15.88); p = 0.01] was significantly higher than that in children with course of 3 months to 1 year, whereas children with higher parental CCKNOW scores [OR 0.77 (95%CI: 0.67-0.88); p = 0.00] had significantly lower incidence of medication non-adherence than those with lower parental CCKNOW scores, and the results of the correlation between parental knowledge scores of the four categories and children’s medication adherence showed that drug knowledge scores (r = 0.36, p < 0.00) and complication knowledge scores (r = 0.24, p = 0.01) were positively correlated with medication adherence.

Conclusion: Poor medication adherence in children with IBD in China was common, and forgetting to take medication was the main barrier. Longer disease duration (3 years and above) in children could act as a risk factor for medication adherence, whereas higher level of parental knowledge about IBD could act as a protective factor, and one interesting novel finding was that the level of parental knowledge about drug and complication were significantly correlated with medication adherence in children with IBD. Our results may inform on the design and implementation of medication adherence interventions for children with IBD.

Implementing service transformation for children and adolescents with eating disorders across England: the theory, politics, and pragmatics of large-scale service reform

BACKGROUND

Eating disorders are among the most serious mental health problems affecting children and young people and without appropriate treatment often have a protracted course with high levels of morbidity and mortality. While considerable progress has been made in recent years in developing effective evidence-based outpatient treatments, these are not always readily available. In England, until recently, the usual care pathway for young people with an eating disorder was referral from primary care to local generic Child and Adolescent Mental Health Services with varying levels of expertise in eating disorders and a mix of outpatient treatments available. Poor treatment progress or physical deterioration would usually result in inpatient admission. Admission rates were high, with children and young people with an eating disorder accounting for nearly a quarter of all child and adolescent psychiatric hospital admissions. Inpatient treatment is costly and has high relapse rates with some evidence that it may contribute to poorer long-term outcomes in eating disorders. Accumulating clinical and research evidence that early expert outpatient treatment can significantly reduce the need for inpatient care indicates,+ that investing in dedicated community-based eating disorders services is likely to be both clinically and economically beneficial.

OVERVIEW OF PAPER

This paper describes a large-scale transformation programme following a major government investment (initially £30 million/year, since then increased to over £50 million/year) aimed at service level change in the provision of eating disorder services for children and adolescents in England. We describe the history, background, political context, and clinical and research evidence that contributed to the government's decision to invest in eating disorders. We also provide a brief account of the implementation of an England-wide whole team training to support the creation of a network of over 70 dedicated community-based eating disorders services for children and young people.

Measurement and assessment of fidelity and competence in nonspecialist-delivered, evidence-based behavioral and mental health interventions: A systematic review

Nonspecialists have increasingly been used to deliver evidence-based, mental health and behavioral interventions in lower resource settings where there is a dearth of specialized providers and a corresponding gap in service delivery. Recent literature acknowledges that nonspecialist-delivered interventions are shown to be effective. However, few studies report on the fidelity (the degree to which an intervention was implemented as intended) and/or competence (general skills of nonspecialists), key concepts that measure quality of evidence-based intervention delivery. This study seeks to understand how both fidelity and competence have been assessed in nonspecialist-delivered, evidence-based interventions with an intended social or psychological behavior-change outcome. Our search results originally yielded 2317 studies, and ultimately, 16 were included in our final analysis. Generally, results from a narrative synthesis indicated that tools used in the studies demonstrated sufficient inter-rater reliability and intra-class correlation components. Included studies used and described a range of fidelity and competence tools. However, the ENhancing Assessment of Common Therapeutic factors tool was the most commonly used tool that measures competence of nonspecialists, and has been adapted to several other settings. The roles of supervisors in mentoring, monitoring, and supervising nonspecialists emerged as a key ingredient for ensuring fidelity. Most studies assessing fidelity were limited by small sample sizes due to low numbers of nonspecialists implementing interventions, however, more advanced statistical methods may not be needed and may actually impede community-based organizations from assessing fidelity data. Our results suggest interventions can share resources, tools, and compare findings regardless with proper supervision. While the two terms "fidelity" and "competence" are often used interchangeably, their differences are noteworthy. Ultimately, both competency and fidelity are critical for delivering evidence-based interventions, and nonspecialists are most effective when they can be evaluated and mentored on both throughout the course of the intervention.

A common factors perspective on mindfulness-based interventions

Mindfulness-based interventions (MBIs) have entered mainstream Western culture in the past four decades. There are now dozens of MBIs with varying degrees of empirical support and a variety of mindfulness-specific psychological mechanisms have been proposed to account for the beneficial effects of MBIs. Although it has long been acknowledged that non-specific or common factors might contribute to MBI efficacy, relatively little empirical work has directly investigated these aspects. In this Perspective, I suggest that situating MBIs within the broader psychotherapy research literature and emphasizing the commonalities rather than differences between MBIs and other treatments might help guide future MBI research. To that end, I summarize the evidence for MBI efficacy and several MBI-specific psychological mechanisms, contextualize MBI findings within the broader psychotherapy literature from a common factors perspective, and propose suggestions for future research based on innovations and challenges occurring within psychotherapy research.

Evaluation of competency-driven training for facilitators delivering a psychological intervention for children in Lebanon: a proof-of-concept study

AIMS

The mounting evidence for effective delivery of psychological interventions by non-specialists in low- and middle-income settings has led to a rapid expansion of mental health and psychosocial support trainings globally. As such, there is a demand for strategies on how to train and implement these services to attain adequate quality. This study aims to evaluate the added value of a competency-driven approach to training of facilitators for a group intervention for children with severe emotional distress in Lebanon.

METHODS

In a controlled before and after study, 24 trainees were randomly allocated to participate in either a competency-driven training (CDT) or training-as-usual (TAU) (1 : 1) for a psychological intervention for children with severe emotional distress. We assessed the change in demonstrated competencies, using standardised role-plays, before and after the training. Measures included the 13-item Working with children-Assessment of Competencies Tool (WeACT), the 15-item ENhancing Assessment of Common Therapeutic factors (ENACT) and the 6-item Group facilitation: Assessment of Competencies Tool (GroupACT). The trainer in the experimental arm used pre-training and during training competency assessment scores to make real-time adjustment to training delivery. Due to COVID-19 pandemic restrictions, all activities were done remotely.

RESULTS

CDT resulted in significantly better outcomes on increasing competencies on the WeACT (repeated measures analysis of variance; F_{(1, 22)} = 6.49, p < 0.018) and on the GroupACT (Mann-Whitney U = 22, p < 0.003), though not statistically significant on the ENACT. There is no significant between-group difference on the reduction of harmful behaviours, mainly because both forms of training appear equally successful in eliminating such behaviours.

CONCLUSIONS

This proof-of-concept study demonstrates the potential of CDT, using standardised assessment of trainee competencies, to contribute to better training outcomes without extending the duration of training. CDT can result in up to 18% greater increase in adequate competency, when compared to TAU. The study also yields recommendations for further enhancing the benefits of competency-driven strategies. A fully powered trial is needed to confirm these findings.

Self-Coding of Fidelity as a Potential Active Ingredient of Consultation to Improve Clinicians' Fidelity

A key goal for implementation science is the identification of evidence-based consultation protocols and the active ingredients within these protocols that drive clinician behavior change. The current study examined clinicians' self-coding of fidelity as a potential active ingredient of consultation for the Attachment and Biobehavioral Catch-up (ABC) intervention. It also examined two other potential predictors of clinician fidelity in response to consultation: dosage of consultation and working alliance. Twenty-nine clinicians (97% female, 62% White, M age = 34 years) participated in a year of weekly fidelity-focused ABC consultation sessions, for which clinicians self-coded fidelity and received consultant feedback on both their coding and their fidelity. Data from the ABC fidelity measure were available for 1067 sessions coded by consultants, and clinicians' self-coding accuracy was calculated from 1044 sessions coded by both clinicians and consultants. Alliance was measured with the Working Alliance Inventory-Trainee and Supervisor Versions. The study was observational, and fidelity and self-coding accuracy were modeled across time using hierarchical linear modeling. Clinicians' ABC fidelity, as well as their self-coding accuracy, increased over the course of consultation. Clinicians' self-coding accuracy predicted their initial fidelity and growth in fidelity. Working alliance was also linked to fidelity and self-coding accuracy. These results suggest that clinician self-coding should be further examined as an active ingredient of consultation. The study has important implications for the design of consultation procedures and fidelity assessments.

Performance of a Supervisor Observational Coding System and an Audit and Feedback Intervention

Workplace-based clinical supervision is common in community based mental health care for youth and families and could be a leveraged to scale and improve the implementation of evidence-based treatment (EBTs). Accurate methods are needed to measure, monitor, and support supervisor performance with limited disruption to workflow. Audit and Feedback (A&F) interventions may offer some promise in this regard. The study-a randomized controlled trial with 60 clinical supervisors measured longitudinally for 7 months-had two parts: (1) psychometric evaluation of an observational coding system for measuring adherence and competence of EBT supervision and (2) evaluation of an experimental Supervisor Audit and Feedback (SAF) intervention on outcomes of supervisor adherence and competence. All supervisors recorded and uploaded weekly supervision sessions for 7 months, and those in the experimental condition were provided a single, monthly web-based feedback report. Psychometric performance was evaluated using measurement models based in Item Response Theory, and the effect of the SAF intervention was evaluated using mixed-effects regression models. The observational instrument performed well across psychometric indicators of dimensionality, rating scale functionality, and item fit; however, coder reliability was lower for competence than for adherence. Statistically significant A&F effects were largely in the expected directions and consistent with hypotheses. The observational coding system performed well, and a monthly electronic feedback report showed promise in maintaining or improving community-based clinical supervisors' adherence and, to a lesser extent, competence. Limitations discussed include unknown generalizability to the supervision of other EBTs.

A method for defining the CORE of a psychosocial intervention to guide adaptation in practice: Reciprocal imitation teaching as a case example

LAY ABSTRACT

Interventions that support social communication include several "components," or parts (e.g. strategies for working with children and families, targeting specific skills). Some of these components may be essential for the intervention to work, while others may be recommended or viewed as helpful but not necessary for the intervention to work. "Recommended" components are often described as "adaptable" because they can be changed to improve fit in different settings where interventions are offered or with different individuals. We need to understand which parts of an intervention are essential (and which are adaptable) when translating interventions from research to community settings, but it is challenging to do this before studying an intervention in the community. This article presents the CORE (COmponents & Rationales for Effectiveness) Fidelity Method-a new method for defining the essential components of evidence-based interventions-and applies it to a case example of Reciprocal Imitation Teaching, an intervention that parents are taught to deliver with their young children with social communication delays. The CORE Fidelity Method involves three steps: (1) gathering information from multiple sources; (2) integrating information from previous research and theory; and (3) drafting a CORE model for ongoing use. The benefits of using the CORE Fidelity Method may include: (1) improving consistency in intervention and research materials to help all providers emphasize the most important skills or strategies; (2) clarifying which parts of the intervention can be adapted; and (3) supporting future research that evaluates which intervention components work and how they work.

Can a Single Measure Estimate Protocol Adherence for Two Psychosocial Treatments for Youth Anxiety Delivered in Community Mental Health Settings?

Treatment adherence measurement can be time and resource-intensive in clinical trials, so the ability to measure protocol adherence for two distinct treatment programs with a single measure may benefit the field. The present study sought to determine if the Therapy Process Observational Coding System - Revised Strategies Scale (TPOCS-RS) could assess protocol adherence to two youth treatment programs. Treatment sessions (N = 796) from 55 youth (M age = 9.89 years, SD = 1.71; range 7-15 years; 55.0% White; 46.0% female) with primary anxiety problems treatment by 39 clinicians (M age = 40.54 years, SD = 9.56; 50.0% White; 80.0% female) were independently scored by coders using observational treatment adherence and alliance measures. The youth received one of three treatments: (a) Standard (i.e., cognitive-behavioral treatment program), (b) Modular (i.e., a program with cognitive-behavioral and parent training components), or (c) Usual Care. Consultants filled out a self-report measure of protocol adherence within the Standard and Modular conditions. Interrater reliability, ICC(2,2) for the various items for the full sample ranged from .17 to .92 (M ICC = .67; SD = .17). Scores from a TPOCS-RS subscale that mapped onto the specific content of the treatment protocols used in the Standard and Modular conditions evidenced convergent validity with the consultant-report adherence measure and discriminant validity with the alliance measure. The model-specific TPOCS-RS subscales also discriminated between the Standard and Modular treatments and Usual Care. This study provides initial evidence that (a) the TPOCS-RS has utility in estimating protocol adherence in different treatment programs and (b) support the score validity of the self-report consultation records.

Clinical practice during the COVID-19 pandemic: a qualitative study among child and adolescent psychiatrists across the world

BACKGROUND

The COVID-19 pandemic has directly impacted the field of child and adolescent psychiatry, affecting all aspects of the lives of children and their families and increasing their risk of distress and mental health issues, especially among children with preexisting psychiatric disorders. Child and adolescent psychiatrists (CAPs) across the world have had to adapt their practice, due to lockdown and social distancing measures. This study aimed to explore how CAPs experienced their clinical practice in these singular conditions.

METHODS

This exploratory international qualitative study used the Inductive Process to analyse the Structure of lived Experience (IPSE) approach, which is a five-stage inductive process used to explore the lived experience of participants in depth and to analyze their structure of lived experience. This study took place from March through July 2020 through individual in-depth video interviews. The sample size was determined according to the principles of theoretical sufficiency.

RESULTS

39 CAPs from 26 countries participated (age range 32-70 years; 23 women). Data analysis produced a structure of lived experience comprising three central axes of experience: (1) lost in space, lost in time, describing CAPs' experience of disorganization of their clinical practice in the dimensions of lived time and lived space, (2) the body-of CAPs and patients-underlining their disconcerting experience of both sensory aspects and the non-embodied encounter during clinical practice, and (3) unpleasant emotions, with angst and loneliness the two main feelings coloring their clinical practice experience.

CONCLUSIONS

This analysis of the structure of lived experience of CAPs went beyond the sole context of the pandemic and revealed key aspects of what usually organizes CAP clinical practice. It identified two blind spots or conceptual voids within the child and adolescent psychiatry field: first, the intrinsic therapeutic function of a CAP clinical practice and, second, the important diagnostic and therapeutic function of the embodied encounter during CAP consultations. Beyond the context of COVID-19, further research should investigate these aspects to better define what a CAP does in practice and to increase both attractiveness and recruitment in this specialty.

Examining the Relation Between Technical and Global Competence in Two Treatments for Youth Anxiety

Although technical (quality of delivering techniques from a specific treatment) and global (general clinical expertise) competence are believed to be important ingredients of successful psychosocial treatment with youth, there have been few empirical efforts to measure both dimensions. Efforts to understand the role that each competence dimension plays in the process and outcome of youth treatment starts with determining whether the dimensions can be measured separately. This study examined whether scores from measures designed to assess technical and global competence were distinct. Treatment sessions (N = 603) from 38 youths (M age = 9.84 years, SD = 1.65; 60.5% White; 52.6% male) treated for primary anxiety problems within a randomized effectiveness trial were coded. Four coders used observational measures designed to assess technical competence, global competence, protocol adherence, and the alliance. Mean item interrater reliability was .70 (SD = .09) for technical competence and .66 (SD = .05) for global competence. While most components of global competence were distinct from technical competence scores, two components showed redundancy (r > .70). Scores on both competence measures were empirically distinct (r < .70) from scores on measures of protocol adherence and the alliance. Although the measures did not fully distinguish between technical and global competence, our findings do indicate that some components of technical and global competence may provide unique information about competence.

Development and Initial Psychometrics of a Generic Treatment Integrity Measure Designed to Assess Practice Elements Targeting Social, Emotional, and Behavioral Outcomes in Early Childhood Settings

Though treatment integrity measurement is important for research intended to promote social and behavioral outcomes of children at risk for emotional and behavioral disorders (EBDs) in early childhood settings, measurement gaps exist in the field. This paper reports on the development and preliminary psychometric assessment of the treatment integrity measure for early childhood settings (TIMECS), an observational measure designed to address existing measurement gaps related to treatment integrity with tier 2 interventions in the early childhood field. To assess the preliminary score reliability (interrater) and validity (construct, discriminant) of the TIMECS, live observations (N = 650) in early childhood classrooms from 54 teachers (92.6% female, 7.4% male; 61.1% White) and 91 children (M age = 4.53 years, SD = .44; 45.1% female, 54.9% male; 45.1% Black) at risk for EBDs were scored by 12 coders using the TIMECS and an observational measure designed to assess teacher-child interactions. Teachers also self-reported on the quality of the teacher-child relationship. Interrater reliability (intraclass correlation coefficients, ICC [2,2]) for the quantity (i.e., adherence) item scores had a mean of .81 (SD = .07; range from .68 to .95), and the quality (i.e., competence) item scores had a mean of .69 (SD = .08; range from .52 to .80). Scores on the TIMECS Quantity and Quality items and scales showed evidence of construct validity, with the magnitude of the correlations suggesting that the quantity and quality items assess distinct components of treatment integrity. A TIMECS quantity scale also showed promise for intervention evaluation research by discriminating between teachers who had and had not been trained in a specific evidence-based intervention targeting social and behavioral skills in early childhood. The findings support the potential of the TIMECS to assess treatment integrity of teacher-delivered practices designed to address child social and behavioral outcomes of children at risk for EBDs in early childhood settings.

Role-Play Assessment of Therapist Adherence and Skill in Implementation of Trauma-Focused Cognitive-Behavioral Therapy

Numerous efforts are underway to train clinicians in evidence-based practices. Unfortunately, the field has few practical measures of therapist adherence and skill with which to judge the success of these training and implementation efforts. One possible assessment method is using behavioral rehearsal, or role-play, as an analogue for therapist in-session behavior. The current study describes aspects of reliability, validity and utility of a behavioral role-play assessment developed to evaluate therapist adherence and skill in implementing Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT). TF-CBT role-play assessments were conducted with a sample of 43 therapists as part of a larger training study. The TF-CBT role-play assessments were independently coded for TF-CBT adherence and skill by a certified TF-CBT trainer and three clinical psychology doctoral students. Findings indicated good interrater reliability for the individual items (ICC: M = .71, SD = .15). Regarding utility, 67.19% (n = 43/64) of contacted therapists completed the role-play assessment, which took an average of 30 min (M = 31.42, SD = 5.65) to complete and 60 min (M = 62.84, SD = 11.31) to code. Therapists with a master's degree were more likely to complete the role-play assessment than those with other degrees but no other differences in demographic variables, practice characteristics, or TF-CBT knowledge or training were found between participants and nonparticipants. Role-play assessments may offer an alternative to observational coding for assessing therapist adherence and skill, particularly in contexts where session recordings are not feasible.

Challenging rigidity in Anorexia (treatment, training and supervision): questioning manual adherence in the face of complexity

BACKGROUND

Anorexia Nervosa is a debilitating illness. While there have been many advancements to treatment protocols and outcomes for people with eating disorders, the field acknowledges there remains considerable room for improvement. This timely Special Edition of the Journal of Eating Disorders has invited those of us in the field to consider a range of topics in aid of this task, including potential modifications and implementation of evidence-based practice, specific and common psychotherapy factors, treatment manuals, adherence and individualising treatment approaches for individuals and families. BODY: In this paper, we briefly outline the key manualised treatments currently available to treat children, adolescents and adults with Anorexia Nervosa, considering the benefits, potential reasons for adaptations and limitations. We then review the current evidence for training strict adherence to treatment manuals which is often a key focus in training and supervision, questioning the association of increased treatment adherence with improved therapeutic outcome. We then summarise some key evidence behind other therapeutic factors which have been demonstrated to affect outcome regardless of which manual is implemented, such as readiness to change and therapeutic alliance.

CONCLUSION

The paper concludes with implications and considerations for future research, clinical guidelines, training and supervision, highlighting the need to consider the therapeutic relationship and processes alongside manual content to conduct best evidence-informed practice.

Community Implementation of MI-Enhanced Behavior Therapy for Adolescent ADHD: Linking Fidelity to Effectiveness

Evidence-based behavior therapy for adolescent ADHD faces implementation challenges in real-world settings. The purpose of this trial was to investigate the relationship between implementation fidelity and outcomes among adolescents receiving services in the active treatment arm (N = 114; Motivational Interviewing [MI]-enhanced parent-teen behavior therapy) of a community-based randomized trial of adolescent ADHD treatment. Participants received therapy from community clinicians (N = 44) at four agencies in a large, ethnically diverse metropolitan setting. Therapists provided self-report of session-by-session adherence to content fidelity checklists and audio recordings of sample sessions that were coded for MI integrity. Parents provided report of ADHD symptoms and family impairment at baseline, posttreatment, and follow-up, while academic records were obtained directly from the local school district. Results indicated that content fidelity significantly waned across the 10 manualized sessions (d = -1.23); these trends were steepest when therapy was delivered outside the office-setting and parent attendance was low. Community therapist self-report of content fidelity predicted significantly greater improvements in academic impairment from baseline to follow-up. MI delivery quality was not associated with improved outcomes; contrary to hypotheses, lower MI relational scores predicted significantly greater improvements in family impairment over time. Findings indicate that community-based outcomes for evidence-based ADHD treatment are enhanced when treatment is implemented with fidelity. Future work should revise community-based implementation strategies for adolescent ADHD treatment to prevent declines in fidelity over time, thereby improving outcomes.

Experience-Based Competency Training in Child and Adolescent Psychotherapy

Abstract. Background: Compared to adult psychotherapy training, there is little knowledge about the use of standardized actor-patients (SPs) for the development of professional competencies for younger patient populations. Objective: We assessed the effectiveness of two university-based training programs for diagnostic as well as intervention skills in child and adolescent psychotherapy. Methods: Psychology students took part in two courses on the development of diagnostic competencies ( N = 59) and on intervention skills ( N = 51). We applied active and experience-based learning strategies, including exercises on self-awareness, mindfulness, peer role-play, and the use of SPs. Competence gain as well as a general evaluation of the seminars were assessed by self-report. Results: For both seminars, self-reported competence increased significantly with a large effect. The retrospective evaluation of learning success and acceptance ratings of the didactic formats were high. Conclusions: Experience-based trainings that include SPs are a feasible option for child and adolescent psychotherapy. The findings are discussed from a conceptual as well as a practical perspective.

Treatment Fidelity in Brief Versus Standard-Length School-Based Interventions for Youth with Anxiety

To examine treatment fidelity in a randomized controlled trial of indicated school-based cognitive-behavioral therapy (CBT) delivered in groups to youth with anxiety. We investigated whether adherence and competence (a) differed across brief and standard-length CBT, and (b) if adherence and competence predicted change in anxiety symptoms and impairment. Method: Sessions were observationally coded with the Competence and Adherence Scale for CBT for Anxiety Disorders in Youth. Coders (N = 7) rated 104 sessions from 52 groups delivered by 32 facilitators (M age = 43.2 years, SD = 8.1) to 295 youth (M age = 14.0 years, SD = 0.8). Outcomes were youth- and parent-reported anxiety symptoms and impairment at post-intervention and 1-year follow-up. Linear mixed effect models were used to analyze whether fidelity predicted clinical outcomes. Results: Levels of adherence and competence were adequate in both programs, but higher in brief compared to standard-length CBT p < .001 and p = .010, respectively). Neither adherence nor competence predicted clinical outcomes at any timepoints. Conclusion: Higher levels of adherence and competence in brief CBT suggest that it may be easier for novice CBT providers to achieve fidelity in simplified and less flexible interventions. Contrary to expectation, adherence and competence did not predict clinical outcomes.

Fidelity Assessment of the Treatnet family (TF): A family-based intervention for adolescents with substance use disorders and their families

•

There was a high level of agreement between practitioners’ and supervisors’ ratings on the practitioner’s use of family-based therapeutic skills.

•

In working with adolescents who are affected by substance use disorders, the practitioners showed very good adherence to the Treatnet Family approach.

•

There was a good level of agreement between the adolescents and their family member in terms of how interactive they found the Treatnet Family session.

Introduction

In transporting family-based interventions to community settings, establishing and maintaining fidelity to intervention is important. This exploratory study was implemented in the framework of a United Nations Office on Drugs and Crime (UNODC) global programme on Drug Dependence Treatment and Care. It is the first to examine an evidence-informed family-based intervention (“Treatnet Family”; TF) adherence for the treatment of adolescents with substance use disorders (SUD) among practitioners in community settings in Jakarta, Indonesia.

Method

Twenty-three practitioners at five community-based counselling centres were trained in Treatnet Family and delivered it to 19 adolescents with SUD and their family members over a 6-week period. One of the five local Treatnet Family-trained supervisors randomly selected one session of the family-based intervention (TF) and observed the extent to which the practitioner’s adhered to the TF manual.

Results

According to the supervisors’ observation, all the practitioners used the Treatnet Family core skills such as positive reframing, positive relational reframing, perspective taking, relational questions, and going with resistance. There was a high level of agreement between practitioners’ and supervisors’ rating on the practitioners’ use of specific therapeutic skills as measured using the Inventory of Therapy Techniques (ITT).

Conclusion

Results suggest that Treatnet Family can be delivered with adherence by practitioners in community-based settings.

Exploring alternatives for adolescent anorexia nervosa: adolescent and parent treatment (APT) as a novel intervention prospect

Recovery and remission rates of adolescent anorexia nervosa (AN) following Family Based Treatment (FBT) have seen a relative decline over recent years. While reasonably successful in achieving physical recovery (i.e. weight restoration), both empirical and anecdotal accounts highlight a lack of attention to the psychological recovery of the adolescent within manualised FBT. As such, there is a need for innovation to explore treatment variations and alternatives for the proportion of adolescents with AN who do not respond favourably to this first-line treatment. This paper introduces a new treatment framework to the field for clinical consideration and empirical assessment. Adolescent and Parent Treatment (APT) for adolescent AN draws from both family-based and individual treatment models, applying a developmental lens. APT attends to physical and psychological recovery simultaneously and from the start of treatment, with capacity to tailor individual psychological modules to the adolescent formulation. While clearly in its infancy, APT provides an exciting new avenue for exploration within the field, as we seek new avenues to support young people and their families to effectively combat this deadly illness.

Competence and Adherence Scale for Cognitive Behavioural Therapy (CAS-CBT) for anxiety disorders in youth: reliability and factor structure

BACKGROUND

There has been increased research interest into the concept of treatment integrity within psychotherapy research. The Competence and Adherence Scale for Cognitive Behavioural Therapy (CAS-CBT) was developed to measure therapists' competence and adherence in cognitive behavioural therapy (CBT), when delivered to children and youth with anxiety disorders.

AIMS

The aim of this study was to evaluate the psychometric properties of the CAS-CBT in a naturalistic treatment setting.

METHOD

Ratings of 212 randomly selected sessions from a clinical effectiveness trial for children with anxiety disorders (n = 165, mean age = 10.46 years, SD = 1.49) were analysed to assess the psychometric properties of CAS-CBT. Therapy format included both individual sessions and group sessions.

RESULTS

Internal consistency for the CAS-CBT was excellent (Cronbach's alpha = .88). Factor analysis suggested a two-factor solution for the total sample, where the first factor was related to CBT structure and session goals, and the second factor was associated with process and relational skills. The individual CBT treatment condition (ICBT) and group CBT treatment condition (GCBT) showed the same factor solution.

CONCLUSION

The CAS-CBT is a feasible and reliable measure for assessing competence and adherence to CBT in the treatment of anxious children. Future research is needed to further assess the generalizability of this scale, its psychometric properties in different treatment populations and with other treatment approaches, and ideally with larger sample sizes.

Making Implementation Last: The Impact of Training Design on the Sustainability of an Evidence-Based Treatment in a Randomized Controlled Trial

Although advances have been made in facilitating the implementation of evidence-based treatments, little is known about the most effective way to sustain their use over time. The current study examined the sustainability of one evidence-based treatment, Parent-Child Interaction Therapy (PCIT), following a statewide implementation trial testing three training methods: Cascading Model, Learning Collaborative, and Distance Education. Participants included 100 clinicians and 50 administrators from 50 organizations across Pennsylvania. Clinicians and administrators reported on sustainability at 24-months, as measured by the number of clients receiving PCIT and the continued use of the PCIT protocol. Multi-level path analysis was utilized to examine the role of training on sustainability. Clinicians and administrators reported high levels of sustainability at 24-months. Clinicians in the Cascading Model reported greater average PCIT caseloads at 24-months, whereas clinicians in the Learning Collaborative reported greater full use of the PCIT protocol at 24-months. Attending consultation calls was associated with delivering PCIT to fewer families. Implications for the sustainable delivery of PCIT beyond the training year as well as for the broader field of implementation science are discussed.

Mediators and theories of change in psychotherapy with adolescents: a systematic review protocol

INTRODUCTION

Approximately 75% of mental disorders emerge before the age of 25 years but less than half receive appropriate treatment. Little is known about the mechanisms underlying the therapeutic change of adolescents in psychotherapy. The 'European Network of Individualised Psychotherapy Treatment of Young People with Mental Disorders', funded by the European Cooperation in Science and Technology, will conduct the first systematic review to summarise the existing knowledge on mediators and theories of change in psychotherapy for adolescents.

METHOD

A systematic review will be conducted, conforming to the reporting guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement recommendations. Electronic databases (PubMed and PsycINFO) have been systematically searched on 23 February 2020, for prospective, longitudinal and case-control designs which examine mediators of change. Participants will be adolescents between 10 and 19 years of age who suffer from a mental disorder or psychological difficulties and receive an intervention that aims at preventing, ameliorating and/or treating psychological problems.

ETHICS AND DISSEMINATION

Ethical approval is not required for this systematic review as no primary data will be collected. The results will be published in a peer-reviewed journals and at conference presentations and will be shared with stakeholder groups. The whole data set will be offered to other research groups following recommendations of the open science initiative. Databases with the systematic search will be made openly available following open science initiatives.

PROSPERO REGISTRATION NUMBER

CRD42020177535.

Developing a competence framework for cognitive analytic therapy

OBJECTIVE

This paper describes the development and summarizes the content of a competence framework for delivery of cognitive analytic therapy (CAT).

DESIGN

The framework was developed using the evidence-based method developed by Roth and Pilling (2008, Behavioural and Cognitive Psychotherapy, 36, 129).

METHODS

A review of the CAT outcome literature identified where CAT interventions had evidence of efficacy. Standard texts on CAT were primary sources for details of theory and practice. This process was supported by an expert reference group (ERG). The role of the ERG was to provide professional advice on areas where the evidence base was lacking, but where CAT interventions were commonly used by therapists trained in the model.

RESULTS

A framework was produced and structured in terms of core knowledge, core skills, and meta-competences (which require therapeutic judgement rather than simple adherence to a treatment protocol).

CONCLUSIONS

The framework enables trainees, service users, service managers, and commissioners to better understand a) the core features of CAT and b) what competences need to be in place for CAT to be skilfully delivered in practice.

PRACTITIONER POINTS

It is possible to define the core competences of CAT. Whilst generic competences are important, there are five CAT-specific domains of competence. The CAT-specific competences reflect the three-phase structure of the therapy: reformulation, recognition, and revision.

Assessing fidelity to family-based treatment: an exploratory examination of expert, therapist, parent, and peer ratings

INTRODUCTION

Fidelity is an essential component for evaluating the clinical and implementation outcomes related to delivery of evidence-based practices (EBPs). Effective measurement of fidelity requires clinical buy-in, and as such, requires a process that is not burdensome for clinicians and managers. As part of a larger implementation study, we examined fidelity to Family-Based Treatment (FBT) measured by several different raters including an expert, a peer, therapists themselves, and parents, with a goal of determining a pragmatic, reliable and efficient method to capture treatment fidelity to FBT.

METHODS

Each therapist audio-recorded at least one FBT case and submitted recordings from session 1, 2, and 3 from phase 1, plus one additional session from phase 1, two sessions from phase 2, and one session from phase 3. These submitted files were rated by an expert and a peer rater using a validated FBT fidelity measure. As well, therapists and parents rated fidelity immediately following each session and submitted ratings to the research team. Inter-observer reliability was calculated for each item using the intraclass correlation coefficient (ICC), comparing the expert ratings to ratings from each of the other raters (parents, therapists, and peer). Mean scale scores were compared using repeated measures ANOVA.

RESULTS

Intraclass correlation coefficients revealed that agreement was the best between expert and peer, with excellent, good, or fair agreement in 7 of 13 items from session 1, 2 and 3. There were only four such values when comparing expert to parent agreement, and two such values comparing expert to therapist ratings. The rest of the ICC values indicated poor agreement. Scale level analysis indicated that expert fidelity ratings for phase 1 treatment sessions scores were significantly higher than the peer ratings and, that parent fidelity ratings tended to be significantly higher than the other raters across all three treatment phases. There were no significant differences between expert and therapist mean scores.

CONCLUSIONS

There may be challenges inherent in parents rating fidelity accurately. Peer rating or therapist self-rating may be considered pragmatic, efficient, and reliable approaches to fidelity assessment for real-world clinical settings.

Assessment of service provider competency for child and adolescent psychological treatments and psychosocial services in global mental health: evaluation of feasibility and reliability of the WeACT tool in Gaza, Palestine

BACKGROUND

There is a scarcity of evaluated tools to assess whether non-specialist providers achieve minimum levels of competency to effectively and safely deliver psychological interventions in low- and middle-income countries. The objective of this study was to evaluate the reliability and utility of the newly developed Working with children - Assessment of Competencies Tool (WeACT) to assess service providers' competencies in Gaza, Palestine.

METHODS

The study evaluated; (1) psychometric properties of the WeACT based on observed role-plays by trainers/supervisors (N = 8); (2) sensitivity to change among service provider competencies (N = 25) using pre-and-post training WeACT scores on standardized role-plays; (3) in-service competencies among experienced service providers (N = 64) using standardized role-plays.

RESULTS

We demonstrated moderate interrater reliability [intraclass correlation coefficient, single measures, ICC = 0.68 (95% CI 0.48-0.86)] after practice, with high internal consistency (α = 0.94). WeACT assessments provided clinically relevant information on achieved levels of competencies (55% of the competencies were scored as adequate pre-training; 71% post-training; 62% in-service). Pre-post training assessment saw significant improvement in competencies (W = -3.64; p < 0.001).

CONCLUSION

This study demonstrated positive results on the reliability and utility of the WeACT, with sufficient inter-rater agreement, excellent internal consistency, sensitivity to assess change, and providing insight needs for remedial training. The WeACT holds promise as a tool for monitoring quality of care when implementing evidence-based care at scale.

Scaling up mental health care and psychosocial support in low-resource settings: a roadmap to impact

AIMS

Despite recent global attention to mental health and psychosocial support services and a growing body of evidence-support interventions, few mental health services have been established at a regional or national scale in low- and middle-income countries (LMIC). There are myriad challenges and barriers ranging from testing interventions that do not target priority needs of populations or policymakers to interventions that cannot achieve adequate coverage to decrease the treatment gap in LMIC.

METHOD

We propose a 'roadmap to impact' process that guides planning for interventions to move from the research space to the implementation space.

RESULTS

We establish four criteria and nine associated indicators that can be evaluated in low-resource settings to foster the greatest likelihood of successfully scaling mental health and psychosocial interventions. The criteria are relevance (indicators: population need, cultural and contextual fit), effectiveness (change in mental health outcome, change in hypothesised mechanism of action), quality (adherence, competence, attendance) and feasibility (coverage, cost). In the research space, relevance and effectiveness need to be established before moving into the implementation space. In the implementation space, ongoing monitoring of quality and feasibility is required to achieve and maintain a positive public health impact. Ultimately, a database or repository needs to be developed with these criteria and indicators to help researchers establish and monitor minimum benchmarks for the indicators, and for policymakers and practitioners to be able to select what interventions will be most likely to succeed in their settings.

CONCLUSION

A practicable roadmap with a sequence of measurable indicators is an important step to delivering interventions at scale and reducing the mental health treatment gap around the world.

Examining fidelity in the INFORM trial: a complex team-based behavioral intervention

Background

Fidelity in complex behavioral interventions is underexplored. This study examines the fidelity of the INFORM trial and explores the relationship between fidelity, study arm, and the trial’s primary outcome—care aide involvement in formal team communications about resident care.

Methods

A concurrent process evaluation of implementation fidelity was conducted in 33 nursing homes in Western Canada (Alberta and British Columbia). Study participants were from 106 clinical care units clustered in 33 nursing homes randomized to the Basic and Enhanced-Assisted Feedback arms of the INFORM trial.

Results

Fidelity of the INFORM intervention was moderate to high, with fidelity delivery and receipt higher than fidelity enactment for both study arms. Higher enactment teams experienced a significantly larger improvement in formal team communications between baseline and follow-up than lower enactment teams (F(1, 70) = 4.27, p = .042).

Conclusions

Overall fidelity enactment was associated with improvements in formal team communications, but the study arm was not. This suggests that the intensity with which an intervention is offered and delivered may be less important than the intensity with which intervention participants enact the core components of an intervention. Greater attention to fidelity assessment and publication of fidelity results through studies such as this one is critical to improving the utility of published trials.

Assessing Adherence, Competence and Differentiation in a Stepped-Wedge Randomised Clinical Trial of a Complex Behaviour Change Intervention

BACKGROUND

A key challenge in behavioural medicine is developing interventions that can be delivered adequately (i.e., with fidelity) within real-world consultations. Accordingly, clinical trials should (but tend not to) report what is actually delivered (adherence), how well (competence) and the distinction between intervention and comparator conditions (differentiation).

PURPOSE

To address this important clinical and research priority, we apply best practice guidelines to evaluate fidelity within a real-world, stepped-wedge evaluation of "EAT: Eating As Treatment", a new dietitian delivered health behaviour change intervention designed to reduce malnutrition in head and neck cancer (HNC) patients undergoing radiotherapy.

METHODS

Dietitians (n = 18) from five Australian hospitals delivered a period of routine care and following a randomly determined order each site received training and began delivering the EAT Intervention. A 20% random stratified sample of audio-recorded consultations (control n = 196; intervention n = 194) was coded by trained, independent, raters using a study specific checklist and the Behaviour Change Counselling Inventory. Intervention adherence and competence were examined relative to apriori benchmarks. Differentiation was examined by comparing control and intervention sessions (adherence, competence, non-specific factors, and dose), via multiple linear regression, logistic regression, or mixed-models.

RESULTS

Achievement of adherence benchmarks varied. The majority of sessions attained competence. Post-training consultations were clearly distinct from routine care regarding motivational and behavioural, but not generic, skills.

CONCLUSIONS

Although what level of fidelity is "good enough" remains an important research question, findings support the real-world feasibility of integrating EAT into dietetic consultations with HNC patients and provide a foundation for interpreting treatment effects.

How do personal and professional characteristics influence the development of psychotherapists in training: Results from a longitudinal study

This study examined the professional development of psychotherapy trainees over three years of training. The first objective was to investigate the long-term change of work involvement (Healing and Stressful Involvement) during psychotherapy training. The second objective was to investigate possible predictors of professional development from the areas of training context as well as professional and personal attributes of trainees. A total of 184 psychotherapy trainees with psychodynamic, psychoanalytic and cognitive behavioral orientation participated in the study. The development of work involvement was assessed over three years of training using the Work Involvement Scales. The set of possible predictors for work involvement included training context variables (training orientation, supervision), professional attributes of trainees (theoretical breadth, work satisfaction), and personal attributes of trainees (introject affiliation, attachment strategies, personality traits). Hierarchical Linear Modeling was conducted to investigate the change over time and the individual predictors of work involvement. Over three years of training Healing Involvement improved whereas Stressful Involvement did not change over time. Healing Involvement was mostly predicted by training context variables and professional attributes (therapeutic orientation, job satisfaction) as well as extraversion. Stressful Involvement was only predicted by personal attributes of trainees (age, neuroticism, conscientiousness, introject affiliation). The results imply two distinct sets of predictors for Healing and Stressful Involvement that will be discussed with regard to their implications for psychotherapy training and trainee selection.

Use of role plays to assess therapist competency and its association with client outcomes in psychological interventions: A scoping review and competency research agenda

A major challenge in scaling-up psychological interventions worldwide is how to evaluate competency among new workforces engaged in psychological services. One approach to measuring competency is through standardized role plays. Role plays have the benefits of standardization and reliance on observed behavior rather than written knowledge. However, role plays are also resource intensive and dependent upon inter-rater reliability. We undertook a two-part scoping review to describe how competency is conceptualized in studies evaluating the relationship of competency with client outcomes. We focused on use of role plays including achieving inter-rater reliability and the association with client outcomes. First, we identified 4 reviews encompassing 61 studies evaluating the association of competency with client outcomes. Second, we identified 39 competency evaluation tools, of which 21 were used in comparisons with client outcomes. Inter-rater reliability (intraclass correlation coefficient) was reported for 15 tools and ranged from 0.53 to 0.96 (mean ICC = 0.77). However, we found that none of the outcome comparison studies measured competency with standardized role plays. Instead, studies typically used therapy quality (i.e., session ratings with actual clients) as a proxy for competency. This reveals a gap in the evidence base for competency and its role in predicting client outcomes. We therefore propose a competency research agenda to develop an evidence-base for objective, standardized role plays to measure competency and its association with client outcomes. OPEN SCIENCE REGISTRATION #: https://osf.io/nqhu7/.