The needs-to-goals gap: How informant discrepancies in youth mental health assessments impact service delivery

Advancing equity in access to school mental health through multiple informant decision-making

There has been a substantial increase in the number of students with mental health needs, yet significant discrepancies exist in access to timely intervention. Traditional gatekeeping to intervention has been the provenance of single information sources. Multi-informant decision-making is a promising mechanism to improve equitable access. However, critical advancements are necessary to improve decision-making relating to (a) who is identified, (b) what type of need is determined, (c) the type of intervention necessary, and (d) where or under what circumstances to implement the intervention. We review critical components of effective mental health decision-making, contributors to inequities in school mental health services, and offer future directions for research and practice to increase equitable student outcomes.

Cross-informant ratings on emotional and behavioral problems in Nepali adolescents: A comparison of adolescents' self-reports with parents' and teachers' reports

BACKGROUND

Studies on cross-informant agreement on adolescents' emotional and behavioral problems (EBPs) are sparse in low- and middle-income countries. This study aimed to assess parent-adolescent and teacher-adolescent agreement on EBPs and associated factors in Nepal.

METHODS

This cross-sectional survey included 1904 school-going adolescents aged 11-18, enrolled in government and private schools located in sixteen districts of Nepal. The Nepali versions of the Youth Self Report, Child Behavior Checklist, and Teacher's Report Form were administered to assess EBPs reported by adolescents, their parents, and teachers, respectively. Repeated measures analysis of variance (ANOVA) was done to assess mean differences in problem scores. Pearson's correlation was used to assess cross-informant agreement. Linear regression analysis was used to explore factors associated with cross-informant discrepancies in EBPs.

RESULTS

Adolescents reported significantly more problems than their parents and teachers. Mean Total Problem scores for the 90 common items in the adolescents' self-reports, parent reports, and teacher reports were 34.5 (standard deviation [SD]: 21.4), 24.1 (SD = 19.2), and 20.2 (SD = 17.5) respectively. Parent-adolescent agreement on Total Problems was moderate, whereas teacher-adolescent agreement was low. The parent-adolescent agreement was moderate to low for the two broadband scales and all syndrome scales, whereas the teacher-adolescent agreement was low for all scales. Female gender and ethnic minority status impacted both parent-adolescent and teacher-adolescent discrepancies. Family stress/conflicts impacted parent-adolescent discrepancies, while academic performance impacted teacher-adolescent discrepancies.

CONCLUSIONS

Nepali adolescents reported more EBPs than their parents and teachers. The agreement between adolescents' self-reports and reports by their parents and teachers was moderate to low. Gender, caste/ethnicity, family stress/conflicts, and academic performance were associated with cross-informant discrepancies. It is crucial to collect information from different sources, consider context-specific needs, and discern factors influencing cross-informant discrepancies to accurately assess adolescents' EBPs and develop personalized approaches to treatment planning.

Data-Informed Communication: How Measurement-Based Care Can Optimize Child Psychotherapy

Measurement-based care (MBC) research and practice, including clinical workflows and systems to support MBC, are grounded in adult-serving mental health systems. MBC research evidence is building in child and adolescent services, but MBC practice is inherently more complex due to identified client age, the family system and the need to involve multiple reporters. This paper seeks to address a gap in the literature by providing practical guidance for youth-serving clinicians implementing MBC with children and their families. We focus on MBC as a data-informed, client-centered communication process, and present three key strategies to enhance usual care child and adolescent psychotherapy via developmentally-appropriate MBC. These strategies include (1) go beyond standardized measures; (2) lean into discrepancies; and (3) get curious together. Case-based examples drawn from various child-serving settings illustrate these key strategies of MBC in child psychotherapy.

Systematic Integration of Multi-Informant Externalizing Ratings in Clinical Settings

Best practice clinical assessment of externalizing problems often necessitates collection of information from parents, youth themselves, and teachers. The present study tested the predictive validity of a psychometrically-driven scoring procedure to integrate multi-informant, dimensional ratings of externalizing problems. Participants were 2264 clinic-referred youth ages 6-18. Parents, teachers, and youth completed questionnaire ratings of externalizing problems (hyperactivity-inattention, conduct problems, and oppositionality-defiance) prior to an initial clinical appointment. The predictive validity of simple (highest informant rating; and all informant ratings separately) and more complex (latent S-1 bifactor model with specific informant factors; and moderated nonlinear factor analysis accounting for child age and sex) methods of informant integration was tested in predicting impairment, comorbidity, and number of clinical encounters. A simple model, in which all informant ratings were included, showed the best predictive validity across outcomes, performing as well or better than the use of the highest informant ratings or more complex latent variable models. The addition of child age and sex as moderators in the factor model did not improve predictive validity. Each informant (parent, teacher, and youth) contributes important information to the prediction of clinically-relevant outcomes. There is insufficient evidence at present to suggest that complex latent variable models should be favored over simpler models that preserve each informant's ratings.

Parent-Youth Attachment Insecurity and Informant Discrepancies of Intrafamilial Aggression

This study investigated how youth attachment anxiety and avoidance are associated with informant discrepancies of intrafamilial aggression within families where youth have clinically significant mental health challenges (N = 510 youth-parent dyads). Using polynomial regressions, we tested whether youth attachment avoidance and anxiety moderated the absolute magnitude of the association between youth- and parent-reports of aggression toward each other. Furthermore, difference scores were computed to test whether youth attachment was associated with the direction of youths' reports of the frequency of aggression relative to parents (i.e., did youth under- or over-report). Dyads' reports of youth-to-parent aggression were more strongly related at high than low levels of attachment anxiety. Results also revealed that youth attachment anxiety was associated with youth over-reporting of youth-to-parent and parent-to-youth aggression (relative to parents), whereas attachment avoidance was associated with youth over-reporting parent-to-youth aggression (relative to parents). These findings highlight the importance of understanding the source of informant discrepancies in social-emotional development and family functioning.

Identify adolescents' help-seeking intention on suicide through self- and caregiver's assessments of psychobehavioral problems: deep clustering of the Tokyo TEEN Cohort study

Background

Psychopathological and behavioral problems in adolescence are highly comorbid, making their developmental trajectories complex and unclear partly due to technical limitations. We aimed to classify these trajectories using deep learning and identify predictors of cluster membership.

Methods

We conducted a population-based cohort study on 3171 adolescents from three Tokyo municipalities, with 2344 pairs of adolescents and caregivers participating at all four timepoints (ages 10, 12, 14, and 16) from 2012 to 2021. Adolescent psychopathological and behavioral problems were assessed by using self-report questionnaires. Both adolescents and caregivers assessed depression/anxiety and psychotic-like experiences. Caregivers assessed obsession/compulsion, dissociation, sociality problem, hyperactivity/inattention, conduct problem, somatic symptom, and withdrawal. Adolescents assessed desire for slimness, self-harm, and suicidal ideation. These trajectories were clustered with variational deep embedding with recurrence, and predictors were explored using multinomial logistic regression.

Findings

Five clusters were identified: unaffected (60.5%), minimal problems; internalizing (16.2%), persistent or worsening internalizing problems; discrepant (9.9%), subjective problems overlooked by caregivers; externalizing (9.6%), persistent externalizing problems; and severe (3.9%), chronic severe problems across symptoms. Stronger autistic traits and experience of bullying victimization commonly predicted the four "affected" clusters. The discrepant cluster, showing the highest risks for self-harm and suicidal ideation, was predicted by avoiding help-seeking for depression. The severe cluster predictors included maternal smoking during pregnancy, not bullying others, caregiver's psychological distress, and adolescent's dissatisfaction with family.

Interpretation

Approximately 40% of adolescents were classified as "affected" clusters. Proactive societal attention is warranted toward adolescents in the discrepant cluster whose suicidality is overlooked and who have difficulty seeking help.

Funding

Japan Ministry of Health, Labor and Welfare, Japan Agency for Medical Research and Development, and Japan Science and Technology Agency.

Evidence-Based Assessment of DSM-5 Disruptive, Impulse Control, and Conduct Disorders

The assessment of oppositional defiant disorder, conduct disorder, antisocial personality disorder, and intermittent explosive disorder-the Disruptive, Impulse Control and Conduct Disorders-can be affected by biases in clinical judgment, including overestimating concerns about distinguishing symptoms from normative behavior and stigma associated with diagnosing antisocial behavior. Recent nosological changes call for special attention during assessment to symptom dimensions of limited prosocial emotions and chronic irritability. The present review summarizes best practices for evidence-based assessment of these disorders and discusses tools to identify their symptoms. Despite the focus on disruptive behavior disorders, their high degree of overlap with disruptive mood dysregulation disorder can complicate assessment. Thus, the latter disorder is also included for discussion here. Good practice in the assessment of disruptive behavior disorders involves using several means of information gathering (e.g., clinical interview, standardized rating scales or checklists), ideally via multiple informants (e.g., parent-, teacher-, and self-report). A commitment to providing a full and accurate diagnostic assessment, with careful and attentive reference to diagnostic guidelines, will mitigate concerns regarding biases.

Youth-Therapist and Parent-Therapist Match and Mismatch on Internalizing and Externalizing Treatment Goals as Predictors of Treatment Engagement

Therapist-client cognitive match upon key constructs such as treatment goals is purported to be an important component of culturally competent care. For adolescent clients, treatment may involve both youths and their parents, suggesting the need to consider both youth-therapist and parent-therapist perspectives. This longitudinal study examined broadband youth-therapist and parent-therapist treatment goal matching and mismatching in relationship to treatment engagement in a culturally diverse sample of 245 outpatient mental health service-using youth. Although goal matching/mismatching did not uniformly predict treatment engagement as measured by a total score, youth-therapist internalizing goal matching predicted better youth engagement, and parent-therapist externalizing goal mismatch marginally predicted worse parent engagement. When selected post hoc analyses examined relationships to four individual engagement dimensions, youth-therapist internalizing goal matches positively predicted youth Client-therapist interaction, Communication/Openness, Client's perceived usefulness of therapy, and Collaboration with treatment, while parent-therapist externalizing goal mismatch negatively predicted parent Collaboration with treatment. Findings support the importance of cognitive match on treatment goals as well as the consideration of both parent and youth perspectives, matched and mismatched goals, internalizing and externalizing goals, and examining individual dimensions in addition to total scores of engagement.

Predictors of school-based cognitive behavior therapy outcome for youth with anxiety

Meta-analyses of school-based CBT have shown that prevention for anxiety symptoms typically report small but significant effects. There is limited knowledge regarding which youths may benefit most and least from such programs, and characteristics of youth who respond differentially to interventions of different intensity. The present study examined predictors of school-based CBT outcomes among 302 youths (mean age 14.0 years, SD 0.8, 84% female) who participated in a randomized waitlist-controlled trial comparing a 10-session and a 5-session group intervention. Potential predictors included youth and parental factors, and credibility and expectancy of the interventions. Pre-intervention anxiety and depression levels, and clinician rated severity were examined as moderators of intervention effects. Outcomes were youth-, and parent-reported youth anxiety and depressive symptoms at post-intervention and 1-year follow-up. Higher parent-reported impairment from youth anxiety predicted larger parent-reported anxiety and depressive symptom change, whereas higher caregiver strain was associated with less symptom change. Higher parent rated credibility and expectancy was associated with improved outcomes at post-intervention. At 1-year follow-up, no predictors of outcome were identified. No moderators were identified. Families with high levels of caregiver strain associated with youth anxiety may need extra support regardless of length of intervention program. Parents' credibility and expectancy of interventions should be targeted to optimize school-based CBT.

Effectiveness of a transdiagnostic computerized self-applied program targeting children with emotional problems: A randomized controlled trial

BACKGROUND

Super Skills for Life (SSL) is an eight-session transdiagnostic program based on cognitive-behavioral therapy (CBT), aimed at the indicated prevention of childhood emotional problems, which has been implemented with positive short- and long-term results. The present study aimed to examine the effects of a self-applied computerized program based on SSL that maintains the same objectives and contents as the face-to-face program.

METHODS

In this randomized controlled study, 75 children (49.3 % female) aged 8-12 years (Mage = 9.45, SD = 1.31), selected for exhibiting emotional symptoms, were randomly assigned to either the intervention (n = 35) or the waiting list control (WLC) group (n = 40). Pre- and post-intervention data were collected through self-reports and the report of parents who completed parallel versions of the same measures of emotional and behavioral problems.

RESULTS

Overall, compared to the WLC group, the intervention group showed positive effects on targeted emotional symptomatology in the short term. Based on parents' reports, a significant reduction was found in outcomes such as anxiety, depression, emotional symptoms, and internalizing problems, while self-reported results were similar except for anxiety. In addition, a positive impact was found on symptoms related to other types of difficulties (e.g., externalizing problems and general difficulties measured).

LIMITATIONS

Small sample size, non-inclusion of follow-up assessment and other informants (e.g., teachers).

CONCLUSIONS

In conclusion, this research provides novel and promising data on the self-applied computerized adapted version of the SSL program, within a multi-informant approach, suggesting that it may be a useful tool for the indicated prevention of childhood emotional problems.

The Proposed Specifiers for Conduct Disorder - Parent (PSCD-P): Convergent Validity, Incremental Validity, and Reactions to Unfamiliar Peer Confederates

Youth who experience psychopathy display multiple impairments across interpersonal (grandiose-manipulative [GM]), affective (callous-unemotional [CU]), lifestyle (daring-impulsive [DI]), and potentially antisocial and behavioral features. Recently, it has been acknowledged that the inclusion of psychopathic features can offer valuable information in relation to the etiology of Conduct Disorder (CD). Yet, prior work largely focuses on the affective component of psychopathy, namely CU. This focus creates uncertainty in the literature on the incremental value of a multicomponent approach to understanding CD-linked domains. Consequently, researchers developed the Proposed Specifiers for Conduct Disorder (PSCD; Salekin & Hare, 2016) as a multicomponent approach to assess GM, CU, and DI features in combination with CD symptoms. The notion of considering the wider set of psychopathic features for CD specification requires testing whether multiple personality dimensions predict domain-relevant criterion outcomes above-and-beyond a CU-based approach. Thus, we tested the psychometric properties of parents' reports on the PSCD (PSCD-P) in a mixed clinical/community sample of 134 adolescents (M_age = 14.49, 66.4% female). Confirmatory factor analyses resulted in a 19-item PSCD-P displaying acceptable reliability estimates and a bifactor solution consisting of GM, CU, DI, and CD factors. Findings supported the incremental validity of scores taken from the PSCD-P across multiple criterion variables, including (a) an established survey measure of parent-adolescent conflict; and (b) trained independent observers' ratings of adolescents' behavioral reactions to laboratory controlled tasks designed to simulate social interactions with unfamiliar peers. These findings have important implications for future research on the PSCD and links to adolescents' interpersonal functioning.

"Peas in a pod": Oral History Reflections on Autistic Identity in Family and Community by Late-Diagnosed Adults

In this paper, we report on a participatory oral history study documenting the lives of late-diagnosed autistic adults in Australia. We interviewed 26 autistic adults about their life history and the impact of late diagnosis. All were diagnosed after the age of 35, growing up in an era when autism was not well known. Using reflexive thematic analysis, we uncovered a rich body of reflections on shared Autistic identity and identified three major themes within that data set: 'conceptualising the Autistic family', 'creating Autistic community', and 'contesting Autistic identity'. Overall, the study provides insights into the active creation of shared Autistic identity and the importance of Autistic community to these late-diagnosed autistic adults.

The Operations Triad Model and Youth Mental Health Assessments: Catalyzing a Paradigm Shift in Measurement Validation

Researchers strategically assess youth mental health by soliciting reports from multiple informants. Typically, these informants (e.g., parents, teachers, youth themselves) vary in the social contexts where they observe youth. Decades of research reveal that the most common data conditions produced with this approach consist of discrepancies across informants' reports (i.e., informant discrepancies). Researchers should arguably treat these informant discrepancies as domain-relevant information: data relevant to understanding youth mental health domains (e.g., anxiety, depression, aggression). Yet, historically, in youth mental health research as in many other research areas, one set of paradigms has guided interpretations of informant discrepancies: Converging Operations and the Multi-Trait Multi-Method Matrix (MTMM). These paradigms (a) emphasize shared or common variance observed in multivariate data, and (b) inspire research practices that treat unique variance (i.e., informant discrepancies) as measurement confounds, namely random error and/or rater biases. Several yearsw ago, the Operations Triad Model emerged to address a conceptual problem that Converging Operations does not address: Some informant discrepancies might reflect measurement confounds, whereas others reflect domain-relevant information. However, addressing this problem requires more than a conceptual paradigm shift beyond Converging Operations. This problem necessitates a paradigm shift in measurement validation. We advance a paradigm (Classifying Observations Necessitates Theory, Epistemology, and Testing [CONTEXT]) that addresses problems with using the MTMM in youth mental health research. CONTEXT optimizes measurement validity by guiding researchers to leverage (a) informants that produce domain-relevant informant discrepancies, (b) analytic procedures that retain domain-relevant informant discrepancies, and (c) study designs that facilitate detecting domain-relevant informant discrepancies.

Multi-Informant Ratings of Childhood Limited Prosocial Emotions: Mother, Father, and Teacher Perspectives

OBJECTIVE

Modest agreement between mothers', fathers', and teachers' reports of child psychopathology can cause diagnostic ambiguity. Despite this, there is little research on informant perspectives of youth's limited prosocial emotions (LPEs). We examined the relationship between mother-, father-, and teacher-reported LPE in a clinical sample of elementary school-aged children.

METHOD

The sample included 207 primarily Caucasian (n = 175, 84.5%) children (136 boys; 65.7%) aged 6-13 years (M = 8.35, SD = 2.04) referred to an outpatient child diagnostic clinic focused on externalizing problems. We report the percentage of youth meeting LPE criteria as a function of informant perspective(s). Utilizing standard scores, we report distributions of informant dyads in agreement/disagreement regarding child LPE, followed up by polynomial regressions to further interrogate the relationship between mother-, father-, and teacher-reported LPE as it relates to conduct problems (CPs).

RESULTS

The prevalence of child LPE was approximately twice as large when compared to those reported in community samples; mothers and fathers generally agreed on their child's LPE symptoms (55% agreement). Higher-order nonlinear interactions between mothers and fathers, as well as parents and teachers, emerged; discrepancies between informants, characterized by low levels of LPE reported by the child's mother, were predictive of youth at the highest risk for CPs.

CONCLUSIONS

Our findings emphasize the clinical utility of gathering multiple reports of LPE when serious CPs are suspected. It may be beneficial for clinicians to give significant consideration to teacher reported LPE when interpreting multiple-informant reports of LPE.

Introduction to the Special Issue. A Dozen Years of Demonstrating That Informant Discrepancies are More Than Measurement Error: Toward Guidelines for Integrating Data from Multi-Informant Assessments of Youth Mental Health

Validly characterizing youth mental health phenomena requires evidence-based approaches to assessment. An evidence-based assessment cannot rely on a "gold standard" instrument but rather, batteries of instruments. These batteries include multiple modalities of instrumentation (e.g., surveys, interviews, performance-based tasks, physiological readings, structured clinical observations). Among these instruments are those that require soliciting reports from multiple informants: People who provide psychometrically sound data about youth mental health (e.g., parents, teachers, youth themselves). The January 2011 issue of the Journal of Clinical Child and Adolescent Psychology (JCCAP) included a Special Section devoted to the most common outcome of multi-informant assessments of youth mental health, namely discrepancies across informants' reports (i.e., informant discrepancies). The 2011 JCCAP Special Section revolved around a critical question: Might informant discrepancies contain data relevant to understanding youth mental health (i.e., domain-relevant information)? This Special Issue is a "sequel" to the 2011 Special Section. Since 2011, an accumulating body of work indicates that informant discrepancies often contain domain-relevant information. Ultimately, we designed this Special Issue to lay the conceptual, methodological, and empirical foundations of guidelines for integrating multi-informant data when informant discrepancies contain domain-relevant information. In this introduction to the Special Issue, we briefly review the last 12 years of research and theory on informant discrepancies. This review highlights limitations inherent to the most commonly used strategies for integrating multi-informant data in youth mental health. We also describe contributions to the Special Issue, including articles about informant discrepancies that traverse multiple content areas (e.g., autism, implementation science, measurement validation, suicide).

Assessing Peer-Related Impairments Linked to Adolescent Social Anxiety: Strategic Selection of Informants Optimizes Prediction of Clinically Relevant Domains

Socially anxious adolescents commonly experience impaired interpersonal functioning with unfamiliar, same-age peers. Yet, we lack short screening tools for assessing peer-related impairments. Recent work revealed that a parent-reported, three-item screening tool produced scores that uniquely related to social anxiety concerns. However, this tool ought to go beyond linking impairments to service needs (i.e., social anxiety symptoms). This tool should also inform the goals of services, in particular by linking impairments to key domains relevant to therapeutically addressing adolescents' anxiety-related needs, such as social skills when interacting with unfamiliar peers. This requires an assessment approach that involves strategic selection of informants who vary in their expertise for observing anxiety-related needs, as well as the therapeutic goals for addressing anxiety-related impairments (e.g., social skills within peer interactions). We leveraged parents' reports to link impairments to social anxiety-related needs. To link impairments to social skills, we leveraged informants (i.e., unfamiliar untrained observers [UUOs]) who observed adolescents within tasks designed to simulate interactions with same-age, unfamiliar peers. We tested this approach using a mixed-clinical/community sample of 134 adolescents, ages 14- to 15 years old. We leveraged multi-informant survey reports to assess adolescent social anxiety, and trained independent observers rated adolescents' social skills within unfamiliar peer interactions. Parents' reports performed best when distinguishing adolescents on referral status and predicting survey-reported social anxiety, whereas only UUOs' reports predicted independent observers' social skills ratings. These findings inform the strategic selection of informants for assessing impairments that commonly prompt the need for adolescents to access mental health services for social anxiety.

The Importance of Parent-Teacher Informant Discrepancy in Characterizing Autistic Youth: A Replication Latent Profile Analysis

OBJECTIVE

Research about autism spectrum disorder (ASD) supports variation in symptom presentations across settings, and there is a growing literature that explicates how this variability may improve characterization of the autism phenotype. Capitalizing on a well-established literature on informant discrepancy as an index of contextual variability, research suggests that differing parent and teacher perceptions may impact treatment or education-related outcomes. A prior investigation by Lerner and colleagues suggests that parent-teacher discrepancies in ASD symptom ratings define discrete and clinically meaningful subgroups. However, replication in a larger sample is important to support the validity and utility of the subgroups for use in research and practice.

METHOD

The present paper used latent profile analysis (LPA) to (1) replicate the previous study by Lerner and colleagues in a larger sample of 514 clinic-referred autistic youth (aged 6-18, 83.2% male, 90.4% White, IQ 19-140) and (2) determine if parent-teacher informant discrepancies relate to clinical and functional correlates. We hypothesized that parent-teacher discrepancies in ASD symptom severity ratings would validly characterize ASD subgroups and predict clinical and functional correlates.

RESULTS

The results of the LPA supported a 4-profile solution made up of two parent-teacher agreement groups (high parent-teacher, 21.2%, and low parent-teacher, 34.2%) and two parent-teacher discrepancy groups (high parent-low teacher, 18.1%, and moderate parent-high teacher, 26.5%), replicating findings from Lerner and colleagues. Latent profile membership differentially predicted IQ, age, and educational outcomes of participants.

CONCLUSIONS

Unique, clinically useful information about the taxonomy and impact of ASD is obtained by considering informant discrepancies in symptom severity ratings, which underscores the importance of considering contextual variability assessed through multiple informants.

Primary and middle-school children’s drawings of the lockdown in Italy

This retrospective-descriptive study investigated how primary and middle-school children perceived the first COVID-19 lockdown in Italy (March–May 2020) as manifested in their drawings. Once school restarted after the first COVID-19 wave, and as part of a structured school re-entry program run in their class in September 2020, 900 Italian children aged 7–13 were asked to draw a moment of their life during the lockdown. The drawings were coded and quantitatively and qualitatively analyzed; several pictorial examples are illustrated in this article. Most children used colorful and full-body representations of the self, but in almost half of the pictures drawn by older students, the self was either missing or represented without the face visible. Most children drew the interior of their houses, and the outside world was completely invisible in over half of the pictures. The most represented activities among younger students were playing or sport, followed by screentime or technology-use. Domestic routines and distance learning were also depicted. Most children, but predominantly girls, drew characters showing emotional cohesion clues, and more younger pupils and girls depicted contentment as their main emotion. Conflicting emotions were virtually non-existent. Our data suggest that children coped with the lockdown through play, screen, and technology use. The high incidence of the missing self-representation in preadolescents could indicate how the enforced loneliness and lack of direct physical contact with others impacted their perception of the self. The findings presented here deepen our knowledge of the dynamics connected to the effects of the COVID crisis on children and young people and show how drawings can provide a valuable window into children’s emotions and perceptions.

Conceptual, methodological, and measurement factors that disqualify use of measurement invariance techniques to detect informant discrepancies in youth mental health assessments

On page 1 of his classic text, Millsap (2011) states, “Measurement invariance is built on the notion that a measuring device should function the same way across varied conditions, so long as those varied conditions are irrelevant [emphasis added] to the attribute being measured.” By construction, measurement invariance techniques require not only detecting varied conditions but also ruling out that these conditions inform our understanding of measured domains (i.e., conditions that do not contain domain-relevant information). In fact, measurement invariance techniques possess great utility when theory and research inform their application to specific, varied conditions (e.g., cultural, ethnic, or racial background of test respondents) that, if not detected, introduce measurement biases, and, thus, depress measurement validity (e.g., academic achievement and intelligence). Yet, we see emerging bodies of work where scholars have “put the cart before the horse” when it comes to measurement invariance, and they apply these techniques to varied conditions that, in fact, may reflect domain-relevant information. These bodies of work highlight a larger problem in measurement that likely cuts across many areas of scholarship. In one such area, youth mental health, researchers commonly encounter a set of conditions that nullify the use of measurement invariance, namely discrepancies between survey reports completed by multiple informants, such as parents, teachers, and youth themselves (i.e., informant discrepancies). In this paper, we provide an overview of conceptual, methodological, and measurement factors that should prevent researchers from applying measurement invariance techniques to detect informant discrepancies. Along the way, we cite evidence from the last 15 years indicating that informant discrepancies reflect domain-relevant information. We also apply this evidence to recent uses of measurement invariance techniques in youth mental health. Based on prior evidence, we highlight the implications of applying these techniques to multi-informant data, when the informant discrepancies observed within these data might reflect domain-relevant information. We close by calling for a moratorium on applying measurement invariance techniques to detect informant discrepancies in youth mental health assessments. In doing so, we describe how the state of the science would need to fundamentally “flip” to justify applying these techniques to detect informant discrepancies in this area of work.

Integrating multi-informant reports of youth mental health: A construct validation test of Kraemer and colleagues’ (2003) Satellite Model

Accurately assessing youth mental health involves obtaining reports from multiple informants who typically display low levels of correspondence. This low correspondence may reflect situational specificity. That is, youth vary as to where they display mental health concerns and informants vary as to where and from what perspective they observe youth. Despite the frequent need to understand and interpret these informant discrepancies, no consensus guidelines exist for integrating informants’ reports. The path to building these guidelines starts with identifying factors that reliably predict the level and form of these informant discrepancies, and do so for theoretically and empirically relevant reasons. Yet, despite the knowledge of situational specificity, few approaches to integrating multi-informant data are well-equipped to account for these factors in measurement, and those that claim to be well-positioned to do so have undergone little empirical scrutiny. One promising approach was developed roughly 20 years ago by Kraemer and colleagues (2003). Their Satellite Model leverages principal components analysis (PCA) and strategic selection of informants to instantiate situational specificity in measurement, namely components reflecting variance attributable to the context in which informants observe behavior (e.g., home/non-home), the perspective from which they observe behavior (e.g., self/other), and behavior that manifests across contexts and perspectives (i.e., trait). The current study represents the first construct validation test of the Satellite Model. A mixed-clinical/community sample of 134 adolescents and their parents completed six parallel surveys of adolescent mental health. Adolescents also participated in a series of simulated social interactions with research personnel trained to act as same-age, unfamiliar peers. A third informant (unfamiliar untrained observer) viewed these interactions and completed the same surveys as parents and adolescents. We applied the Satellite Model to each set of surveys and observed high internal consistency estimates for each of the six-item trait (α = 0.90), context (α = 0.84), and perspective (α = 0.83) components. Scores reflecting the trait, context, and perspective components displayed distinct patterns of relations to a battery of criterion variables that varied in the context, perspective, and source of measurement. The Satellite Model instantiates situational specificity in measurement and facilitates unifying conceptual and measurement models of youth mental health.

Future Directions on BIPOC Youth Mental Health: The Importance of Cultural Rituals in the COVID-19 Pandemic

Culture plays an important role in the development of mental health, especially during childhood and adolescence. However, less is known about how participation in cultural rituals is related to the wellbeing of youth who are Black, Indigenous, and People of Color (BIPOC), and part of the Global Majority. This is crucial amid the COVID-19 pandemic, a global event that has disproportionally affected BIPOC youth and disrupted participation in rituals. The goal of this paper is to promote advances in clinical child and adolescent psychology focused on rituals. We begin by defining culture and rituals and examining their role on development. We illustrate these issues with the Lunar New Year in China, Maya rituals in México, Ramadan in Turkey, and Black graduations and Latinx funerals in the United States. We discuss how the pandemic has affected participation in these rituals and their potential impact on BIPOC children and adolescents' mental health. We propose future directions and recommendations for research.

Multi-Informant Assessment of Adolescents’ Social–Emotional Skills: Patterns of Agreement and Discrepancy among Teachers, Parents, and Students

Objectives: This study explores the patterns of agreement and discrepancy among informants (teachers, parents, and students) in the domains of the Social Emotional Skills Scale Assessment System—Social Skills Scales (SESAS-SS), which is a translation of the Social Skills Improvement System—Rating Scales (SSIS-RS) for use in Spain. Methods: The sample is composed of students, 88 teachers, and 98 parents from Spanish secondary schools. Inter-rater agreements have been assessed, calculating the Pearson correlation coefficients among pairs of raters, effect size indices, and intraclass correlation coefficients at the subscale and total scale level. Results: The convergent validity coefficients were stronger than the divergent ones, with the highest level of agreement between teachers and parents in social skills, particularly for total social skills, engagement, empathy, and communication. The patterns of discrepancies confirmed weaker agreements between teachers and parents in self-control and between parents and students in empathy. Significant differences were also found in students’ estimates depending on gender. Conclusions: The SESAS-SS provides support for previous studies on inter-rater agreements for SS, extending the focus on the degree of agreement in the estimate of dyads of raters when considering the students’ gender.