Advancing equity in access to school mental health through multiple informant decision-making

There has been a substantial increase in the number of students with mental health needs, yet significant discrepancies exist in access to timely intervention. Traditional gatekeeping to intervention has been the provenance of single information sources. Multi-informant decision-making is a promising mechanism to improve equitable access. However, critical advancements are necessary to improve decision-making relating to (a) who is identified, (b) what type of need is determined, (c) the type of intervention necessary, and (d) where or under what circumstances to implement the intervention. We review critical components of effective mental health decision-making, contributors to inequities in school mental health services, and offer future directions for research and practice to increase equitable student outcomes.

Parent-youth convergence (and divergence) in reports about pediatric quality of life

PURPOSE

Researchers and service providers typically assess pediatric Health-Related Quality of Life (HRQOL) by collecting independent reports from parents and youth. An emerging body of work indicates that patterns of parent-youth reports yield information germane to understanding youth outcomes. We identified patterns of HRQOL among youth and their parents seeking mental health treatment and examined links between agreement patterns and mental and physical health functioning.

METHODS

Participants included 227 youth (mean age = 14.40 years, SD = 2.42; 63% female) and parent dyads presenting at a mood disorders clinic between 2013 and 2020. We assessed HRQOL using parallel youth and parent forms of the Pediatric Quality of Life Inventory Generic Core Scales. We also assessed youth clinical correlates of depression, suicidal ideation, and impairment, as well as health information via electronic health record (e.g., psychotropic medication usage, BMI).

RESULTS

Latent class analysis showed three parent-youth reporting patterns: Low-Low (LL), High-High (HH), and Parent Low-Youth High (PL-YH). Relative to youth in the HH group, youth in the LL and PL-YH groups reported significantly greater depressive symptoms and had higher rates of suicidal ideation and psychotropic medication use. In addition, youth in the LL group reported significantly greater levels of impairment.

CONCLUSIONS

Parent-youth patterns of HRQOL reporting can reveal clinically meaningful information and indicate poorer functioning for certain groups (LL, PL-YH) of youth. These findings have implications for improving accuracy of risk assessments that leverage HRQOL data.

Robust caregiver-youth discrepancies in irritability ratings on the affective reactivity index: An investigation of its origins

OBJECTIVE

The Affective Reactivity Index (ARI) is widely used to assess young people's irritability symptoms, but youth and caregivers often diverge in their assessments. Such informant discrepancy might be rooted in poor psychometric properties, the differential conceptualization of irritability across informants, or reflect sociodemographic and clinical characteristics. We use an out-of-sample replication approach and leverage longitudinal data, available for a subset of the participants, to test these hypotheses.

METHOD

Across two independent samples (N_Cohort-1 = 765, 8-21 years; N_Cohort-2 = 1910, 6-21 years), we investigate the reliability and measurement invariance of the ARI, examine sociodemographic and clinical predictors of discrepant reporting and probe the utility of a bifactor model for cross-informant integration.

RESULTS

Despite good internal consistency and 6-week-retest-reliability of parent (Cohort-1: α = 0.92, ICC = 0.85; Cohort-2: α = 0.93) and youth forms (Cohort-1: α = 0.88, ICC = 0.78; Cohort-2: α = 0.82), we confirm substantial informant discrepancy in ARI ratings (3 points on a scale from 0 to 12), which is stable over six weeks (ICC = 0.53). Measurement invariance across informants was weak, indicating that parents and youth may interpret ARI items differently. Irritability severity and diagnostic status predicted informant-discrepancy, albeit in opposing directions: higher severity was linked to relative, higher irritability-ratings by youth (Cohort-1: β = -0.06, p < .001; Cohort-2: β = -0.06, p < .001), while diagnoses of Disruptive Mood Dysregulation Disorder (Cohort-1: β = 0.44, p < .001; Cohort-2: β = 0.84, p < .001) and Oppositional Defiant Disorder (Cohort-1: β = 0.41, p < .001; Cohort-2: β = 0.42, p < .001) predicted relative higher irritability-ratings by caregivers. In both datasets, a bifactor model parsing informant-specific from shared irritability-related variance fit the data well (CFI = 0.99, RMSEA = 0.05; N₂: CFI = 0.99; RMSEA = 0.04).

CONCLUSION

Parent and youth ARI reports and their discrepancy are reliable and reflect different interpretations of the scale items; hence they should not be averaged. This finding also suggests that irritability is not a unitary construct. Future work should investigate and model how different aspects of irritability might differ in their impact on the responses of specific informants.

The Operations Triad Model and Youth Mental Health Assessments: Catalyzing a Paradigm Shift in Measurement Validation

Researchers strategically assess youth mental health by soliciting reports from multiple informants. Typically, these informants (e.g., parents, teachers, youth themselves) vary in the social contexts where they observe youth. Decades of research reveal that the most common data conditions produced with this approach consist of discrepancies across informants' reports (i.e., informant discrepancies). Researchers should arguably treat these informant discrepancies as domain-relevant information: data relevant to understanding youth mental health domains (e.g., anxiety, depression, aggression). Yet, historically, in youth mental health research as in many other research areas, one set of paradigms has guided interpretations of informant discrepancies: Converging Operations and the Multi-Trait Multi-Method Matrix (MTMM). These paradigms (a) emphasize shared or common variance observed in multivariate data, and (b) inspire research practices that treat unique variance (i.e., informant discrepancies) as measurement confounds, namely random error and/or rater biases. Several yearsw ago, the Operations Triad Model emerged to address a conceptual problem that Converging Operations does not address: Some informant discrepancies might reflect measurement confounds, whereas others reflect domain-relevant information. However, addressing this problem requires more than a conceptual paradigm shift beyond Converging Operations. This problem necessitates a paradigm shift in measurement validation. We advance a paradigm (Classifying Observations Necessitates Theory, Epistemology, and Testing [CONTEXT]) that addresses problems with using the MTMM in youth mental health research. CONTEXT optimizes measurement validity by guiding researchers to leverage (a) informants that produce domain-relevant informant discrepancies, (b) analytic procedures that retain domain-relevant informant discrepancies, and (c) study designs that facilitate detecting domain-relevant informant discrepancies.

Discrepancies between mother and father ratings of child behavior after early mild traumatic brain injury

Mild traumatic brain injuries (mTBI) are highly prevalent during early childhood and can lead to behavioral difficulties. Parent report questionnaires are widely used to assess children's behavior, but they are subject to parental bias. The aim of this study was to investigate parental discrepancies in internalized and externalized behavior ratings of children who sustain mTBI in early childhood (i.e., between 18 and 60 months) and to determine if parenting stress or family burden related to the injury contribute to parental discrepancies. Mothers and fathers of 85 children with mTBI, 58 orthopedic injured (OI), and 82 typically developing children (TDC) completed the Child Behavior Checklist 6 months after the injury. The primary caregiver completed the Parental Distress subscale of the Parenting Stress Index and the Family Burden of Injury Interview. Mothers reported more internalized and externalized behavior problems than fathers in the mTBI group. No group difference was found in the OI or TDC groups. Neither parenting stress nor family burden related to the injury predicted discrepancies in behavior ratings. Mothers' and fathers' perceptions of behavior after their young child sustains mTBI appear to differ, suggesting that both parents' views are useful in understanding outcome. This difference was not found in either of the comparison groups indicating that factors related to mTBI may underlie the rating discrepancies.

A Meta-Analysis on Differences and Associations between Alliance Ratings in Child and Adolescent Psychotherapy

OBJECTIVE

The alliance in child and adolescent psychotherapy is widely recognized as an important factor in therapy. Studies on the alliance have increasingly focused on assessment of the alliance as a dyadic construct, measuring both client and therapist alliance ratings. However, cross-informant reports of the alliance in child psychotherapy have not yet been subjected to meta-analysis. Therefore, the present meta-analysis aims to increase knowledge on the degree of convergence and divergence between child, parent, therapist, and observer alliance ratings in child and adolescent psychotherapy.

METHODS

A series of three-level meta-analyses of 78 studies was conducted to investigate differences and associations between child, parent, therapist, and observer alliance ratings in child and adolescent psychotherapy.

RESULTS

Findings indicated that children and parents in general rated the alliance more positively than their therapists (d = 0.35, d = 0.72, respectively), and that child-therapist and parent-therapist alliance ratings were moderately correlated (r = .32, r = .23, respectively). Associations between child and therapist ratings and observer ratings were moderate to large (r = .43, r = .53, respectively).

CONCLUSIONS

It can be concluded that children and parents generally report more positively on the alliance compared to their therapists, which is consistent with research on the alliance in adult populations. The small to moderate associations between alliance ratings indicate that individuals to some extent have a shared perspective on their alliance, and that the various perspectives on alliance should be acknowledged when dealing with children and parents in therapy. Implications for future research are discussed.

Introduction to the Special Issue. A Dozen Years of Demonstrating That Informant Discrepancies are More Than Measurement Error: Toward Guidelines for Integrating Data from Multi-Informant Assessments of Youth Mental Health

Validly characterizing youth mental health phenomena requires evidence-based approaches to assessment. An evidence-based assessment cannot rely on a "gold standard" instrument but rather, batteries of instruments. These batteries include multiple modalities of instrumentation (e.g., surveys, interviews, performance-based tasks, physiological readings, structured clinical observations). Among these instruments are those that require soliciting reports from multiple informants: People who provide psychometrically sound data about youth mental health (e.g., parents, teachers, youth themselves). The January 2011 issue of the Journal of Clinical Child and Adolescent Psychology (JCCAP) included a Special Section devoted to the most common outcome of multi-informant assessments of youth mental health, namely discrepancies across informants' reports (i.e., informant discrepancies). The 2011 JCCAP Special Section revolved around a critical question: Might informant discrepancies contain data relevant to understanding youth mental health (i.e., domain-relevant information)? This Special Issue is a "sequel" to the 2011 Special Section. Since 2011, an accumulating body of work indicates that informant discrepancies often contain domain-relevant information. Ultimately, we designed this Special Issue to lay the conceptual, methodological, and empirical foundations of guidelines for integrating multi-informant data when informant discrepancies contain domain-relevant information. In this introduction to the Special Issue, we briefly review the last 12 years of research and theory on informant discrepancies. This review highlights limitations inherent to the most commonly used strategies for integrating multi-informant data in youth mental health. We also describe contributions to the Special Issue, including articles about informant discrepancies that traverse multiple content areas (e.g., autism, implementation science, measurement validation, suicide).

Assessing Peer-Related Impairments Linked to Adolescent Social Anxiety: Strategic Selection of Informants Optimizes Prediction of Clinically Relevant Domains

Socially anxious adolescents commonly experience impaired interpersonal functioning with unfamiliar, same-age peers. Yet, we lack short screening tools for assessing peer-related impairments. Recent work revealed that a parent-reported, three-item screening tool produced scores that uniquely related to social anxiety concerns. However, this tool ought to go beyond linking impairments to service needs (i.e., social anxiety symptoms). This tool should also inform the goals of services, in particular by linking impairments to key domains relevant to therapeutically addressing adolescents' anxiety-related needs, such as social skills when interacting with unfamiliar peers. This requires an assessment approach that involves strategic selection of informants who vary in their expertise for observing anxiety-related needs, as well as the therapeutic goals for addressing anxiety-related impairments (e.g., social skills within peer interactions). We leveraged parents' reports to link impairments to social anxiety-related needs. To link impairments to social skills, we leveraged informants (i.e., unfamiliar untrained observers [UUOs]) who observed adolescents within tasks designed to simulate interactions with same-age, unfamiliar peers. We tested this approach using a mixed-clinical/community sample of 134 adolescents, ages 14- to 15 years old. We leveraged multi-informant survey reports to assess adolescent social anxiety, and trained independent observers rated adolescents' social skills within unfamiliar peer interactions. Parents' reports performed best when distinguishing adolescents on referral status and predicting survey-reported social anxiety, whereas only UUOs' reports predicted independent observers' social skills ratings. These findings inform the strategic selection of informants for assessing impairments that commonly prompt the need for adolescents to access mental health services for social anxiety.

How Robust Is the p Factor? Using Multitrait-Multimethod Modeling to Inform the Meaning of General Factors of Youth Psychopathology

We used multitrait-multimethod (MTMM) modeling to examine general factors of psychopathology in three samples of youth (Ns = 2119, 303, 592) for whom three informants reported on the youth's psychopathology (e.g., child, parent, teacher). Empirical support for the p-factor diminished in multi-informant models compared with mono-informant models: the correlation between externalizing and internalizing factors decreased and the general factor in bifactor models essentially reflected externalizing. Widely used MTMM-informed approaches for modeling multi-informant data cannot distinguish between competing interpretations of the patterns of effects we observed, including that the p-factor reflects, in part, evaluative consistency bias or that psychopathology manifests differently across contexts (e.g., home vs. school). Ultimately, support for the p-factor may be stronger in mono-informant designs, although it is does not entirely vanish in multi-informant models. Instead, the general factor of psychopathology in any given mono-informant model likely reflects a complex mix of variances, some substantive and some methodological.

Empirically based dimensions of externalizing symptoms in children and adolescents: a multitrait-multisource approach

The present study sought to refine knowledge about the structure underlying externalizing dimensions. From a “top-down” ICD/DSM-based perspective, externalizing symptoms can be categorized into attention-deficit/hyperactivity disorder (ADHD), oppositional defiant disorder (ODD), and conduct disorder (CD). From a “bottom-up” developmental theory-based perspective, disruptive behaviors can be meaningfully described as aggressive (AGG) and rule-breaking (RB) behaviors. We analyzed five large data sets comprising externalizing symptom ratings obtained with a screening instrument using different sources (parents, teachers, self-ratings) from different samples. Using confirmatory factor analyses, we evaluated several factor models (unidimensional; correlated factors; bifactor (S-1) models) derived from an ICD/DSM- and theory-based perspective. Our optimally fitting models were assessed for measurement invariance across all sources, sample settings, and sex. Following several model-based criteria (model fit indices; factor loadings; omega statistics; model parsimony), we discarded our models stepwise and concluded that both the ICD/DSM-based model with three correlated factors (ADHD, ODD, CD) and the developmental theory-based model with three correlated factors (ADHD, AGG, RB) displayed a statistically sound factor structure and allowed for straightforward interpretability. Furthermore, these two models demonstrated metric invariance across all five samples and across sample settings (community, clinical), as well as scalar invariance across sources and sex. While the dimensions AGG and RB may depict a more empirically coherent view than the categorical perspective of ODD and CD, at this point we cannot clearly determine whether one perspective really outperforms the other. Implications for model selection according to our model-based criteria and clinical research are discussed.

Social learning in the digital age: Associations between technoference, mother-child attachment, and child social skills

Technoference refers to incidents in which technology use interferes with interpersonal exchanges (e.g., conversations, playing). Although research on technoference is in its infancy, there is preliminary evidence that mothers believe technoference has a detrimental impact on the social-emotional functioning of their child. The current study investigated the degree to which technoference was associated with attachment between mothers and their elementary-aged children. A second aim was to determine if the relationship between technoference and children's social-emotional functioning may be moderated by mother-child attachment. Surveys were completed by a sample of 80 mothers and their elementary-aged children. This study is unique in asking elementary-aged children to report their perceptions of parental technoference and the impact it has on their relationship with their mother and their own social-emotional functioning. More frequent technoference was associated with less secure mother-child attachment as rated by children, but not as rated by mothers. That is, frequent technoference may not significantly influence a mother's attachment to their child, but it is associated with a child's attachment to their mother. More frequent technoference was associated with decreased ratings by mothers regarding their child's social-emotional functioning. Furthermore, maternal attachment moderated the relationship between technoference and child externalizing behaviors, such that a more secure attachment served as a protective factor against the negative impact of technoference on child externalizing behaviors. However, attachment did not moderate the relationship between technoference and most social skills assessed in our study. Implications from this study are discussed, including ways to increase awareness of technoference among school personnel, parents, and youth.

Using Evaluative Criteria to Review Youth Anxiety Measures, Part II: Parent-Report

This Evidence Base Update of parent-report measures of youth anxiety symptoms is a companion piece to our update on youth self-report anxiety symptom measures (Etkin et al., 2021). We rate the psychometric properties of the parent-report measures as Adequate, Good, or Excellent using criteria developed by Hunsley and Mash (2008) and Youngstrom et al. (2017). Our review reveals that the evidence base for parent-report measures is considerably less developed compared with the evidence base for youth self-report measures. Nevertheless, several measures, the parent-report Screen for Child Anxiety-Related Emotional Disorders, Multidimensional Anxiety Scale for Children, and Spence Children's Anxiety Scale, were found to have Good to Excellent psychometric properties. We conclude our review with suggestions about which parent-report youth anxiety measures are best suited to perform different assessment functions and directions for additional research to expand and strengthen the evidence base.

The Use of Immersive Environments for the Early Detection and Treatment of Neuropsychiatric Disorders

Neuropsychiatric disorders are highly prevalent conditions with significant individual, societal, and economic impacts. A major challenge in the diagnosis and treatment of these conditions is the lack of sensitive, reliable, objective, quantitative tools to inform diagnosis, and measure symptom severity. Currently available assays rely on self-reports and clinician observations, leading to subjective analysis. As a step toward creating quantitative assays of neuropsychiatric symptoms, we propose an immersive environment to track behaviors relevant to neuropsychiatric symptomatology and to systematically study the effect of environmental contexts on certain behaviors. Moreover, the overarching theme leads to connected tele-psychiatry which can provide effective assessment.

Assessing the Mental Health of Maltreated Youth with Child Welfare Involvement Using Multi-Informant Reports

Researchers often question the validity of multi-informant assessments among adolescents with child welfare involvement. Yet, within other clinical populations, prior research finds that multi-informant reports have a discernable structure characterized by discrete patterns of agreement and disagreement. This structure "tracks" contextual displays of behavior and clinical severity. We examined the structure of multi-informant reports (i.e., adolescent, caregiver, teacher) of adolescent externalizing and internalizing problems in a sample of adolescents with a history of child welfare involvement. Across problem domains and informants, reporting patterns mirrored those observed in other clinical populations, and displayed characteristics robustly present in meta-analytic work on cross-informant correspondence. Specifically, informants agreed more on reports of externalizing problems than internalizing problems and caregiver-teacher dyads agreed more than adolescent-caregiver dyads. Overall, we found robust, replicable patterns of multi-informant reports among child welfare involved adolescents. These reporting patterns may facilitate use and interpretation of multi-informant evidence-based assessments among this population.

Informant discrepancies in assessments of psychosocial functioning in school-based services and research: Review and directions for future research

Psychosocial functioning plays a key role in students' wellbeing and performance inside and outside of school. As such, techniques designed to measure and improve psychosocial functioning factor prominently in school-based service delivery and research. Given that the different contexts (e.g., school, home, community) in which students exist vary in the degree to which they influence psychosocial functioning, educators and researchers often rely on multiple informants to characterize intervention targets, monitor intervention progress, and inform the selection of evidence-based services. These informants include teachers, students, and parents. Across research teams, domains, and measurement methodologies, researchers commonly observe discrepancies among informants' reports. We review theory and research-occurring largely outside of school-based service delivery and research-that demonstrates how patterns of informant discrepancies represent meaningful differences that can inform our understanding of psychosocial functioning. In turn, we advance a research agenda to improve use and interpretation of informant discrepancies in school-based services and research.