The Operations Triad Model and Youth Mental Health Assessments: Catalyzing a Paradigm Shift in Measurement Validation

Is There a Place for Cognitive Restructuring in Brief, Self-Guided Interventions? Randomized Controlled Trial of a Single-Session, Digital Program for Adolescents

OBJECTIVE

Self-guided digital mental health interventions (DMHIs) teaching empirically supported skills (e.g. behavioral activation) have demonstrated efficacy for improving youth mental health, but we lack evidence for the complex skill of cognitive restructuring (CR).

METHOD

We conducted the first-ever RCT testing a CR DMHI ("Project Think") against an active control (supportive therapy; "Project Share") in collaboration with public schools. Pre-registered outcomes were DMHI acceptability and helpfulness post-intervention, as well as internalizing symptoms and CR skills use from baseline to seven-month follow-up, in the full sample and the subsample with elevated symptoms.

RESULTS

Participants (N = 597; MAge = 11.99; 48% female; 68% White) rated both programs highly on acceptability and helpfulness. Both conditions were associated with significant internalizing symptom reductions across time in both samples, with no significant condition differences. CR skills use declined significantly across time for Project Share youths but held steady across time for Project Think youths in both samples; this pattern produced a significant condition difference favoring Project Think within the elevated sample at seven-month follow-up.

CONCLUSION

Internalizing symptoms declined comparably for Think and Share participants. Consequently, future research should examine whether encouraging youths to share their feelings produces symptom improvements, and whether a single-session, self-guided CR DMHI produces beneficial effects relative to more inert control conditions. Further, the decline in CR skills use for Project Share youths versus sustained CR use by Project Think youths raises questions about the natural time course of youths' CR use and the impact of these DMHIs on that course. ClinicalTrials.gov Registration: NCT04806321.

Interpersonal Mechanisms Between Child Maltreatment Timing and Young Adult Internalizing and Externalizing Symptomatology

OBJECTIVE

Children who experience maltreatment are prone to exhibit interpersonal deficits and lack secure attachment, which can lead to internalizing and externalizing symptomatology. This study investigated timing and chronicity of maltreatment and its impacts on psychopathology outcomes in young adulthood. Two interpersonal mediators were examined: problems with peers and childhood attachment security.

METHOD

Children with and without maltreatment exposure were recruited to take part in a 1-week research summer camp (N = 697; mean [SD] age = 11.29 [0.97] years; 71.3% Black or African American; 50.5% male). Participants were recontacted in young adulthood to complete a second wave of assessments (n = 427; mean [SD] age = 19.67 [1.16] years; 78.0% Black or African American; 48.9% male). Structural equation modeling was used to estimate indirect effects from child maltreatment timing to young adult internalizing and externalizing symptomatology via childhood attachment security and peer problems.

RESULTS

Findings highlighted the detrimental impact of chronic maltreatment, which was associated with higher levels of peer problems (β = .24, p < .001) and less secure attachment (β = -.13, p < .01) in childhood. Also, lower attachment quality in childhood mediated the association between chronic maltreatment and self-reported internalizing (a × b = 0.02, p < .05) and externalizing symptomatology (a × b = 0.02, p < .05). Additionally, childhood peer problems mediated the association between chronic maltreatment and caregiver-reported internalizing problems (a × b = 0.04, p < .05).

CONCLUSION

Chronic maltreatment is particularly harmful for interpersonal outcomes of children. Mediation findings differed by who reported on psychopathology, showing the importance of considering multireporter measures of psychopathology.

PLAIN LANGUAGE SUMMARY

Child maltreatment is an interpersonal stressor that may impact developmental outcomes throughout a youth's life. In this study of 697 children who participated in a 1-week research summer camp and were contacted in young adulthood, the authors found that chronic maltreatment significantly increased the risk of difficulties in forming secure attachments as well as having problems with peers during childhood. In turn, these challenges were linked to emotional and behavioral symptomology in young adulthood. These findings emphasize the need to support children facing chronic maltreatment to improve their long-term mental health.

Developmental cascades from early childhood attachment security to adolescent level of personality functioning among high-risk youth

This study examines associations between early childhood attachment security and adolescent personality functioning in a high-risk sample within a developmental psychopathology framework. Data from 2,268 children (1165 male; 1103 female) and caregivers participating in Future of Families and Child Well-Being Study (FFCWS) were used to examine (1) effects of genetic polymorphisms of the serotonin transporter (5-HTTLPR) and dopamine D4 receptor (DRD4) genes and adverse childhood experiences (ACEs) on attachment security and emotional and behavioral dysregulation in early childhood and (2) longitudinal associations and transactional relationships among attachment security, dysregulation, negative parenting attitudes and behaviors, social competence, and adolescent personality functioning. Results revealed that ACEs predicted attachment security over and above sex or the genetic risk, and gene × environment interactions did not increment prediction. Results of cascade models showed that greater early childhood attachment security predicted higher adolescent level of personality functioning via pathways through intermediary variables. Limitations and future research directions are discussed.

The General Psychopathology 'p' Factor in Adolescence: Multi-Informant Assessment and Computerized Adaptive Testing

Accumulating evidence supports the presence of a general psychopathology dimension, the p factor ('p'). Despite growing interest in the p factor, questions remain about how p is assessed. Although multi-informant assessment of psychopathology is commonplace in clinical research and practice with children and adolescents, almost no research has taken a multi-informant approach to studying youth p or has examined the degree of concordance between parent and youth reports. Further, estimating p requires assessment of a large number of symptoms, resulting in high reporter burden that may not be feasible in many clinical and research settings. In the present study, we used bifactor multidimensional item response theory models to estimate parent- and adolescent-reported p in a large community sample of youth (11-17 years) and parents (N = 5,060 dyads). We examined agreement between parent and youth p scores and associations with assessor-rated youth global functioning. We also applied computerized adaptive testing (CAT) simulations to parent and youth reports to determine whether adaptive testing substantially alters agreement on p or associations with youth global functioning. Parent-youth agreement on p was moderate (r =.44) and both reports were negatively associated with youth global functioning. Notably, 7 out of 10 of the highest loading items were common across reporters. CAT reduced the average number of items administered by 57%. Agreement between CAT-derived p scores was similar to the full form (r =.40) and CAT scores were negatively correlated with youth functioning. These novel results highlight the promise and potential clinical utility of a multi-informant p factor approach.

Utility of Multiple Specifiers for Subtyping Oppositional Defiant Problems: Investigating Multiple Psychopathy Dimensions as Specifiers

Past research has speculated that using grandiose-manipulative (GM) and daring-impulsive (DI) traits, rather than solely callous-unemotional (CU) traits, could better inform subtyping youths with behavioral problems. However, this approach remains largely untested. To further enhance our understanding of this matter, the current study utilized self-report (n = 2,381) and parent-report (n = 1,846) data to examine the utility of three psychopathic dimensions as specifiers for oppositional defiant problems (ODP). Findings showed that ODP was a significant predictor of negative outcomes, while the GM, CU, and DI traits emerged as significant predictors of theoretically and clinically relevant external variables (e.g., conduct problems and proactive aggression), independent of ODP (and age, gender, and socioeconomic status), forming the potential foundation for the development of specifiers for ODP. Furthermore, children presenting ODP alongside heightened levels of the three psychopathic traits demonstrated notably higher scores in important external variables compared to other groups, including but not confined to the subgroup of children with ODP and high CU trait levels. Findings provide preliminary support for employing GM and DI traits as specifiers for ODP. However, more comprehensive evaluations of these specifiers' utility would help inform ongoing discussions before definitively concluding they should be included in diagnostic manuals.

Adolescent social anxiety is associated with diminished discrimination of anticipated threat and safety in the bed nucleus of the stria terminalis

Social anxiety-which typically emerges in adolescence-lies on a continuum and, when extreme, can be devastating. Socially anxious individuals are prone to heightened fear, anxiety, and the avoidance of contexts associated with potential social scrutiny. Yet most neuroimaging research has focused on acute social threat. Much less attention has been devoted to understanding the neural systems recruited during the uncertain anticipation of potential encounters with social threat. Here we used a novel fMRI paradigm to probe the neural circuitry engaged during the anticipation and acute presentation of threatening faces and voices in a racially diverse sample of 66 adolescents selectively recruited to encompass a range of social anxiety and enriched for clinically significant levels of distress and impairment. Results demonstrated that adolescents with more severe social anxiety symptoms experience heightened distress when anticipating encounters with social threat, and reduced discrimination of uncertain social threat and safety in the bed nucleus of the stria terminalis (BST), a key division of the central extended amygdala (EAc). Although the EAc-including the BST and central nucleus of the amygdala-was robustly engaged by the acute presentation of threatening faces and voices, the degree of EAc engagement was unrelated to the severity of social anxiety. Together, these observations provide a neurobiologically grounded framework for conceptualizing adolescent social anxiety and set the stage for the kinds of prospective-longitudinal and mechanistic research that will be necessary to determine causation and, ultimately, to develop improved interventions for this often-debilitating illness.

Examining profiles of convergence and divergence in reports of parental warmth: Links to adolescent developmental problems

Parental warmth during the transition from childhood to adolescence is a key protective factor against a host of adolescent problems, including substance use, maladjustment, and diminished well-being. Moreover, adolescents and parents often disagree in their perceptions of parenting quality, and these discrepancies may confer risk for problem outcomes. The current study applies latent profile analysis to a sample of 687 mother-father-6th grade adolescent triads to identify patterns of adolescent-parent convergence and divergence in perceptions of parental warmth. Five profiles were identified, and associations with adolescent positive well-being, substance use, and maladjustment outcomes in 9th grade were assessed. Patterns of divergence in which adolescents had a pronounced negative perception of parental warmth compared to parents, as well as those wherein pronounced divergence was present in only one adolescent-parent dyad, were associated with diminished positive well-being compared to adolescents who had more positive perceptions of warmth than parents. Having more negative perceptions of warmth compared to parents was also associated with elevated risk for alcohol and marijuana initiation, but only when the divergence was pronounced rather than more moderate. These findings add nuance to findings from previous between-family investigations of informant discrepancies, calling for further family-centered methods for investigating multiple perspectives.

Bivalent fears of evaluation: A developmentally-informed, multi-informant, and multi-modal examination of associations with safety behaviors

Fears of negative (FNE) and positive (FPE) evaluation and safety behaviors feature prominently in cognitive-behavioral models of social anxiety. However, we have a poor understanding of their associations, particularly given evidence that they both vary in form and function. This study aimed to identify the factor structure of safety behaviors and explore their differential associations with FNE and FPE. We addressed these aims across samples that varied in developmental stage, informant, and assessment modality. We collected self-reported data from college students (n = 349; Mage = 19.42) and adolescent-parent dyads (n = 134; Mage_adolescents = 14.49, Mage_parents = 45.01); parents also completed an ecologically-valid evaluation task. We confirmed a two-factor structure of safety behaviors (i.e., avoidance and impression management) that fit the data well for college students, adolescents, and parents' self-report, but not for parents' report about adolescents. Associations between avoidance and impression management and FNE/FPE were significant within-informants but not between-informants. For parents, in-the-moment arousal following receipt of negative, but not positive, feedback was associated with avoidance and impression management. Findings have implications for integrated measurement of FNE, FPE, and safety behaviors, as well as treatments that target social anxiety through each of these domains.

Evidence Base Update on the Assessment of Irritability, Anger, and Aggression in Youth

OBJECTIVE

Irritability, anger, and aggression have garnered significant attention from youth mental health researchers and clinicians; however, fundamental challenges of conceptualization and measurement persist. This article reviews the evidence base for assessing these transdiagnostic constructs in children and adolescents.

METHOD

We conducted a preregistered systematic review of the evidence behind instruments used to measure irritability, anger, aggression, and related problems in youth. Searches were conducted in PsycINFO and PubMed, identifying 4,664 unique articles. Eligibility criteria focused on self- and proxy-report measures with peer-reviewed psychometric evidence from studies in English with youths ages 3-18. Additional measures were found through ancillary search strategies (e.g. book chapters, review articles, test publishers). Measures were screened and coded by multiple raters with acceptable reliability.

RESULTS

Overall, 68 instruments met criteria for inclusion, with scales covering irritability (n = 15), anger (n = 19), aggression (n = 45), and/or general overt externalizing problems (n = 27). Regarding overall psychometric support, 6 measures (8.8%) were classified as Excellent, 46 (67.6%) were Good, and 16 (23.5%) were Adequate. Descriptive information (e.g. informants, scales, availability, translations) and psychometric properties (e.g. reliability, validity, norms) are summarized.

CONCLUSIONS

Numerous instruments for youth irritability, anger, and aggression exist with varying degrees of empirical support for specific applications. Although some measures were especially strong, none had uniformly excellent properties across all dimensions, signaling the need for further research in particular areas. Findings promote conceptual clarity while also producing a well-characterized toolkit for researchers and clinicians addressing transdiagnostic problems affecting youth.

A Multi-Method Examination of Peer Functioning in Children with and without Cognitive Disengagement Syndrome

OBJECTIVE

Cognitive disengagement syndrome (CDS) includes excessive daydreaming, mental confusion, and hypoactive behaviors that are distinct from attention-deficit/hyperactivity disorder inattentive (ADHD-IN) symptoms. A growing number of studies indicate that CDS symptoms may be associated with ratings of social withdrawal. However, it is important to examine this association in children specifically recruited for the presence or absence of CDS, and to incorporate multiple methods including direct observations of peer interactions. The current study builds on previous research by recruiting children with and without clinically elevated CDS symptoms and using a multi-method, multi-informant design including recess observations and parent, teacher, and child rating scales.

METHOD

Participants were 207 children in grades 2-5 (63.3% male), including 103 with CDS and 104 without CDS, closely matched on grade and sex.

RESULTS

Controlling for family income, medication status, internalizing symptoms, and ADHD-IN severity, children with CDS were observed during recess to spend more time alone or engaging in parallel play, as well as less time involved in direct social interactions, than children without CDS. Children with CDS were also rated by teachers as being more asocial, shy, and socially disinterested than children without CDS. Although children with and without CDS did not differ on parent- or self-report ratings of shyness or social disinterest, children with CDS rated themselves as lonelier than children without CDS.

CONCLUSIONS

Findings indicate that children with CDS have a distinct profile of peer functioning and point to the potential importance of targeting withdrawal in interventions for youth with elevated CDS symptoms.

Therapist's effect on children's therapeutic alliance: A naturalistic study

OBJECTIVES

The therapeutic alliance (TA) has been shown to be a predictor of psychotherapy treatment success. In the case of psychotherapy with children, there is a dearth of information on TA's role. The aims of the paper are: (1) To estimate the therapist effects on children TA; (2) to investigate if therapists' TA predicts children's TA; (3) to analyze if children's age and sex predict children's TA; (4) to evaluate if the therapist's characteristics predict children's TA.

METHODS

The sample consisted of 77 children undergoing psychotherapy in Argentina, and the therapists (N = 29) providing services to those children. The assessment tools utilized for the study included the Therapy Alliance Scale for Children and the Personal Style of the Therapist Questionnaire (PST-Q).

RESULTS

Findings indicated that 2% of the children's TA was explained by the therapists (ICC = 0.02), while 17% of the therapists' TA was explained by the therapists (ICC = 0.17). Therapists' TA predicted children's TA. Children's age and sex did not have an effect on their own TA. Moreover, therapists with more experience achieved higher scores of children's TA. Finally, the Operative dimension of the PST had a negative effect on children's TA (i.e., therapists who prefer more spontaneous interventions over structured ones may experience higher levels of therapeutic alliance with child patients).

CONCLUSION

We found a positive effect of the therapist's TA on children's TA, especially in the preference for using more spontaneous intervention techniques. We discuss the implications of the findings on the training of psychotherapists who provide services to children.

Retrospective Psychometrics and Effect Heterogeneity in Integrated Data Analysis: Commentary on the Special Issue

The current special issue of Prevention Science indicates that momentum in using individual participant data (IPD) and integrative data analysis (IDA) to combine and synthesize findings in prevention science has accelerated over the past decade. In this commentary, we focus on two general themes involving methods for harmonizing measures and findings of effect heterogeneity. We describe methods for harmonization as retrospective psychometrics, requiring that we attend to the assumptions necessary for accurate measurement, but adjust our methods given the constraints of working with existing datasets that often involve different measures in different studies. We point to novel approaches for increasing confidence that semantic matching and empirical modeling used in these studies will yield accurate and valid measurements that can be combined in IDA. We also review findings about effect heterogeneity, emphasizing the importance of using etiologic and action theories to identify and evaluate sources of such effects. We note that all of the papers in this issue deserve careful attention, as they illustrate how prevention scientists are approaching the complexities of IDA and exploring novel methods for overcoming its challenges.

Future Directions in Youth Irritability Research

Research on irritability in children and adolescents has proliferated over the last 20 years. The evidence shows the clinical and developmental significance of irritable mood and behavior in youth, and it has led to significant changes in mental health classification, diagnosis, and services. At the same time, this research (including our own) has led to relatively little new in terms of practical, empirically based guidance to improve interventions and outcomes. In this article, we briefly summarize some of these developments and current evidence-based practices. We then put forth two key substantive challenges (the "whats") for future research to address: (a) the need for more effective treatments, especially evaluating and adapting evidence-based treatments that are already well-established for problems related to irritability (e.g., cognitive-behavioral therapies for internalizing and externalizing problems); and (b) the need for a better mechanistic understanding of irritability's phenomenology (e.g., phasic vs. tonic irritability, how frustration unfolds) and putative underlying mechanisms (e.g., cognitive control, threat and reward dysfunction). Lastly, we suggest three methodological approaches (the "hows") that may expedite progress in such areas: (a) ecological momentary assessment, (b) digital health applications, and (c) leveraging existing datasets. We hope this article will be useful for students and early-career researchers interested in tackling some of these important questions to better meet the needs of severely irritable youth.

What is the Best Source of Information for Psychopathic Traits in Youth? A Review and Meta-analysis of Self- and Other-Reported Psychopathic Traits and Their Association with Negative Outcomes

Child psychopathic traits appear to be associated with negative outcomes. Despite the study of youth psychopathy often relying on multiple reporters (e.g., child, caregivers, teachers), there is limited insight into how much information these various sources contribute and moreover, how this information is integrated. The present study sought to address this gap in the literature by examining the magnitude of relationships between self- and other-reported youth psychopathy and negative outcomes (e.g., delinquency, aggression) using a meta-analytic approach. Results revealed a moderate association between psychopathic traits and negative outcomes. Moderator analyses showed a greater relationship for other- than self-reported psychopathy, although not to a substantive extent. Results further indicated the magnitude of the overall psychopathy-negative outcomes association was stronger for externalizing than internalizing outcomes. Study findings can inform improvements in the assessment of youth psychopathy across research and practice, in addition to advancing our understanding of the utility of psychopathic traits in the prediction of clinically relevant outcomes. This review also provides guidance for future multisource raters and source-specific information in the study of psychopathy in youth.

Parent-youth convergence (and divergence) in reports about pediatric quality of life

PURPOSE

Researchers and service providers typically assess pediatric Health-Related Quality of Life (HRQOL) by collecting independent reports from parents and youth. An emerging body of work indicates that patterns of parent-youth reports yield information germane to understanding youth outcomes. We identified patterns of HRQOL among youth and their parents seeking mental health treatment and examined links between agreement patterns and mental and physical health functioning.

METHODS

Participants included 227 youth (mean age = 14.40 years, SD = 2.42; 63% female) and parent dyads presenting at a mood disorders clinic between 2013 and 2020. We assessed HRQOL using parallel youth and parent forms of the Pediatric Quality of Life Inventory Generic Core Scales. We also assessed youth clinical correlates of depression, suicidal ideation, and impairment, as well as health information via electronic health record (e.g., psychotropic medication usage, BMI).

RESULTS

Latent class analysis showed three parent-youth reporting patterns: Low-Low (LL), High-High (HH), and Parent Low-Youth High (PL-YH). Relative to youth in the HH group, youth in the LL and PL-YH groups reported significantly greater depressive symptoms and had higher rates of suicidal ideation and psychotropic medication use. In addition, youth in the LL group reported significantly greater levels of impairment.

CONCLUSIONS

Parent-youth patterns of HRQOL reporting can reveal clinically meaningful information and indicate poorer functioning for certain groups (LL, PL-YH) of youth. These findings have implications for improving accuracy of risk assessments that leverage HRQOL data.

The Proposed Specifiers for Conduct Disorder - Parent (PSCD-P): Convergent Validity, Incremental Validity, and Reactions to Unfamiliar Peer Confederates

Youth who experience psychopathy display multiple impairments across interpersonal (grandiose-manipulative [GM]), affective (callous-unemotional [CU]), lifestyle (daring-impulsive [DI]), and potentially antisocial and behavioral features. Recently, it has been acknowledged that the inclusion of psychopathic features can offer valuable information in relation to the etiology of Conduct Disorder (CD). Yet, prior work largely focuses on the affective component of psychopathy, namely CU. This focus creates uncertainty in the literature on the incremental value of a multicomponent approach to understanding CD-linked domains. Consequently, researchers developed the Proposed Specifiers for Conduct Disorder (PSCD; Salekin & Hare, 2016) as a multicomponent approach to assess GM, CU, and DI features in combination with CD symptoms. The notion of considering the wider set of psychopathic features for CD specification requires testing whether multiple personality dimensions predict domain-relevant criterion outcomes above-and-beyond a CU-based approach. Thus, we tested the psychometric properties of parents' reports on the PSCD (PSCD-P) in a mixed clinical/community sample of 134 adolescents (M_age = 14.49, 66.4% female). Confirmatory factor analyses resulted in a 19-item PSCD-P displaying acceptable reliability estimates and a bifactor solution consisting of GM, CU, DI, and CD factors. Findings supported the incremental validity of scores taken from the PSCD-P across multiple criterion variables, including (a) an established survey measure of parent-adolescent conflict; and (b) trained independent observers' ratings of adolescents' behavioral reactions to laboratory controlled tasks designed to simulate social interactions with unfamiliar peers. These findings have important implications for future research on the PSCD and links to adolescents' interpersonal functioning.

Future Directions in Youth and Family Treatment Engagement: Finishing the Bridge Between Science and Service

The field has spent more than 50 years investing in the quality of youth mental healthcare, with intervention science yielding roughly 1,300 efficacious treatments. In the latter half of this period, concurrent efforts in implementation science have developed effective methods for supporting front-line service organizations and therapists to begin to bridge the science to service gap. However, many youths and families still do not benefit fully from these strategic investments due to low treatment engagement: nearly half of youths in need of services pursue them, and among those who do, roughly another half terminate prematurely. The negative impact of low engagement is substantial, and is disproportionally and inequitably so for many. We contend that to build a robust and "finished" bridge connecting science and service, the field must go beyond its two historical foci of designing interventions and preparing therapists to deliver them, to include an intentional focus on the youths and families who participate in these interventions and who work with those therapists. In this paper, we highlight the significance of treatment engagement in youth mental healthcare and discuss the current state of the literature related to four priorities: conceptualization, theory, measurement, and interventions. Next, we offer an example from our own program of research as one illustration for advancing these priorities. Finally, we propose recommendations to act on these priorities.

Multi-Informant Ratings of Childhood Limited Prosocial Emotions: Mother, Father, and Teacher Perspectives

OBJECTIVE

Modest agreement between mothers', fathers', and teachers' reports of child psychopathology can cause diagnostic ambiguity. Despite this, there is little research on informant perspectives of youth's limited prosocial emotions (LPEs). We examined the relationship between mother-, father-, and teacher-reported LPE in a clinical sample of elementary school-aged children.

METHOD

The sample included 207 primarily Caucasian (n = 175, 84.5%) children (136 boys; 65.7%) aged 6-13 years (M = 8.35, SD = 2.04) referred to an outpatient child diagnostic clinic focused on externalizing problems. We report the percentage of youth meeting LPE criteria as a function of informant perspective(s). Utilizing standard scores, we report distributions of informant dyads in agreement/disagreement regarding child LPE, followed up by polynomial regressions to further interrogate the relationship between mother-, father-, and teacher-reported LPE as it relates to conduct problems (CPs).

RESULTS

The prevalence of child LPE was approximately twice as large when compared to those reported in community samples; mothers and fathers generally agreed on their child's LPE symptoms (55% agreement). Higher-order nonlinear interactions between mothers and fathers, as well as parents and teachers, emerged; discrepancies between informants, characterized by low levels of LPE reported by the child's mother, were predictive of youth at the highest risk for CPs.

CONCLUSIONS

Our findings emphasize the clinical utility of gathering multiple reports of LPE when serious CPs are suspected. It may be beneficial for clinicians to give significant consideration to teacher reported LPE when interpreting multiple-informant reports of LPE.

What is the Status of Multi-Informant Treatment Fidelity Research?

OBJECTIVE

The precise measurement of treatment fidelity (quantity and quality in the delivery of treatment strategies in an intervention) is essential for intervention development, evaluation, and implementation. Various informants are used in fidelity assessment (e.g., observers, practitioners [clinicians, teachers], clients), but these informants often do not agree on ratings. This scoping review aims to ascertain the state of science around multi-informant assessment of treatment fidelity.

METHOD

A literature search of articles published through December 2021 identified 673 articles. Screening reduced the number of articles to 44, and the final study set included 35 articles.

RESULTS

There was substantial variability across studies regarding study design, how fidelity was operationalized, and how reliability was defined and assessed. Most studies evaluated the agreement between independent observers and practitioner-report, though several other informant pairs were assessed. Overall, findings suggest that concordance across fidelity informants was low to moderate, with a few key exceptions.

CONCLUSIONS

It is difficult to draw clear conclusions about the degree to which single versus multiple informant assessment is needed to produce an accurate and complete picture of treatment fidelity. The field needs to take steps to determine how to leverage multi-informant assessment to accurately assess treatment fidelity.