Is Parent-Child Disagreement on Child Anxiety Explained by Differences in Measurement Properties? An Examination of Measurement Invariance Across Informants and Time

Do youth anxiety measures assess the same construct consistently throughout treatment? Results are...complicated

Interventionists interpret changes in symptoms as reflecting response to treatment. However, changes in symptom functioning and the measurement of the underlying constructs may be reflected in reported change. Longitudinal measurement invariance (LMI) is a statistical approach that assesses the degree to which measures consistently capture the same construct over time. We examined LMI in measures of anxiety severity/symptoms [i.e., Pediatric Anxiety Rating Scale (PARS), Multidimensional Anxiety Scale for Children (MASC), Screen for Child Anxiety and Related Disorders (SCARED)] in anxious youth at baseline and posttreatment. Initial fit was inadequate for 27 of 38 baseline and posttreatment models, but model modifications resulted in acceptable fit. Tests of LMI supported scalar invariance for the PARS and many, but not all, MASC and SCARED subscales. Findings suggest that the PARS, and many MASC and SCARED subscales can accurately be used to measure change over time, however, others may reflect changes in measurement properties.

Terror catastrophizing: association with anxiety, depression, and transgenerational effects

Background & Objectives: Terror catastrophizing, defined as an ongoing fear of future terrorist attacks, is associated with a higher incidence of anxiety disorders, among other psychological impacts. However, previous studies examining terror catastrophizing's relationship to other mental health disorders are limited. The current study sought to determine if patients diagnosed with anxiety and depression would experience increased terror catastrophizing. Additionally, this study aimed to investigate whether parental terror catastrophizing increases children's internalizing symptoms.Design & Methods: Individuals were randomly drawn from the Danish Civil Registration System and invited to complete a series of questionnaires to measure terror catastrophizing tendency, lifetime parental trauma, and children's internalizing symptoms. In total, n = 4,175 invitees completed the survey of which 933 reported on a child between 6 and 18 years. Responses were analyzed using a generalized linear regression model.Results: Participants diagnosed with anxiety alone or comorbid with depression were more likely to experience symptoms of terror catastrophizing than undiagnosed participants (β = 0.10, p < .001; β = 0.07, p = .012). Furthermore, the parental tendency to catastrophize terror was associated with higher internalizing symptoms in children (β = 0.09, p = .006), even after taking parental diagnoses, as well as lifetime and childhood trauma into account.Conclusion: The results can inform clinical practices to account for a patient's potential to exhibit increased terror catastrophizing tendencies or be more affected by traumatic events. Additionally, they can offer insights for designing novel preventative interventions for the whole family, due to the relation between parental tendencies for terror catastrophizing and the internalizing symptoms observed in children.

Assessing Adolescent Functioning from Different Perspectives: Extending the Validation of the Adolescent Functioning Scale (AFS)

The study aimed to provide further evidence for the validity of the 33-item Adolescent Functioning Scale (AFS) as a parent- and adolescent-report scale of adolescent adjustment. In separate samples of parents (N = 542; 88% female) and adolescents (N = 303; 60% female), confirmatory factor analyses supported the original 4-factor structure of the AFS. Analyses produced a 28-item parent measure, and a 27-item adolescent measure. Parent and adolescent versions included positive development, oppositional behaviour, antisocial behaviour and emotional problems subscales. Evidence for convergent and construct validity was provided through correlations with existing measures of adolescent functioning and parenting. The AFS demonstrated configural and metric invariance, but not scalar variance. The study provided support for the validity and reliability of the revised AFS for parents and adolescents. The strong psychometric properties, and brief and multi-dimensional nature of the AFS means that it will have utility in research and applied contexts.

The associations among sleep-related difficulties, anxiety, and error-related brain activity in youth

Given the high prevalence of anxiety disorders and their associated impairment, elucidating neural mechanisms related to these disorders has been increasingly prioritized. The error-related negativity (ERN) has been identified as a neural marker that indexes risk for anxiety across development. The ERN seems to confer risk for developing anxiety, especially in the context of stressful life events. The present study sought to examine sleep-related difficulties as another stressful factor that might impact the ERN. In a sample of 221 girls, aged 8 to 15 years old, we first examined the relationship between longer-term (i.e., over the past month) and shorter-term (i.e., over the past week) sleep difficulties and the ERN. We then investigated whether specific sleep difficulties uniquely predict the ERN. In exploratory analyses, we assessed whether sleep difficulties moderate the relationship between the ERN and anxiety. Results indicated that youth who report longer-term lower sleep duration, longer-term worse sleep, and shorter-term lower sleep duration on school days over the past week have a larger (i.e., more negative) ERN. Additionally, only shorter-term sleep duration on school days over the past week uniquely predicted the ERN. Finally, an elevated ERN predicted greater clinical anxiety in the context of longer-term sleep difficulties. Future studies should clarify the direction of these associations via longitudinal designs.

Parent-adolescent informant discrepancy on the Strengths and Difficulties Questionnaire in the UK Millennium Cohort Study

BACKGROUND

Developmental researchers often use a multi-informant approach to measure adolescent behaviour and adjustment, but informant discrepancies are common. In general population samples, it is often found that parents report more positive and less negative outcomes than adolescents themselves. This study aimed to investigate factors associated with informant discrepancy, including adolescent sex, and parental level of psychological distress and education.

METHODS

Informant discrepancy on the Strengths and Difficulties Questionnaire was investigated using a Latent Difference Score (LDS) approach, which estimates the true difference between parent and adolescent reports in a structural equation model. The sample were parent-adolescent dyads from the seventh wave of the UK Millennium Cohort Study (N = 6947, 49.3% female, aged 17 years).

RESULTS

Parents reported lower levels of difficulties (emotion symptoms, peer problems, conduct problems), and higher levels of pro-social behaviour than adolescents themselves. Conditional effects were found, as discrepancy was greater amongst parent-daughter dyads for emotion and peer problems, and greater amongst parent-son dyads for conduct problems and pro-social behaviour. Parent-adolescent discrepancy was also greater generally if parents had a lower level of psychological distress or a higher level of education.

CONCLUSIONS

In a large general population sample from the UK, it was found that adolescents tended to report more negative and less positive outcomes than parents reported about them. Conditional effects were found at the parent and adolescent level suggesting that specific informant biases are likely to impact the measurement of adolescent behaviour and adjustment across reporters.

The Operations Triad Model and Youth Mental Health Assessments: Catalyzing a Paradigm Shift in Measurement Validation

Researchers strategically assess youth mental health by soliciting reports from multiple informants. Typically, these informants (e.g., parents, teachers, youth themselves) vary in the social contexts where they observe youth. Decades of research reveal that the most common data conditions produced with this approach consist of discrepancies across informants' reports (i.e., informant discrepancies). Researchers should arguably treat these informant discrepancies as domain-relevant information: data relevant to understanding youth mental health domains (e.g., anxiety, depression, aggression). Yet, historically, in youth mental health research as in many other research areas, one set of paradigms has guided interpretations of informant discrepancies: Converging Operations and the Multi-Trait Multi-Method Matrix (MTMM). These paradigms (a) emphasize shared or common variance observed in multivariate data, and (b) inspire research practices that treat unique variance (i.e., informant discrepancies) as measurement confounds, namely random error and/or rater biases. Several yearsw ago, the Operations Triad Model emerged to address a conceptual problem that Converging Operations does not address: Some informant discrepancies might reflect measurement confounds, whereas others reflect domain-relevant information. However, addressing this problem requires more than a conceptual paradigm shift beyond Converging Operations. This problem necessitates a paradigm shift in measurement validation. We advance a paradigm (Classifying Observations Necessitates Theory, Epistemology, and Testing [CONTEXT]) that addresses problems with using the MTMM in youth mental health research. CONTEXT optimizes measurement validity by guiding researchers to leverage (a) informants that produce domain-relevant informant discrepancies, (b) analytic procedures that retain domain-relevant informant discrepancies, and (c) study designs that facilitate detecting domain-relevant informant discrepancies.

Testing Invariance of Measures of Internalizing Symptoms Before and After a Major Life Stressor: The Impact of COVID-19 in an Adolescent and Young Adult Sample

There are reports of increases in levels of internalizing psychopathology during the COVID-19 pandemic. However, these studies presume that measurement properties of these constructs remained unchanged from before the pandemic. In this study, we examined longitudinal measurement invariance of assessments of depression, anxiety, and intolerance of uncertainty (IU) in adolescents and young adults from ongoing longitudinal studies. We found consistent support for configural and metric invariance across all constructs, but scalar invariance was unsupported for depression and IU. Thus, it is necessary to interpret pandemic-associated mean-level changes in depression and IU cautiously. In contrast, mean-level comparisons of panic, generalized, and social anxiety symptoms were not compromised. These findings are limited to the specific measures examined and the developmental period of the sample. We acknowledge that there is tremendous distress accompanying disruptions due to the COVID-19 outbreak. However, for some instruments, comparisons of symptom levels before and during the pandemic may be limited.

Conceptual, methodological, and measurement factors that disqualify use of measurement invariance techniques to detect informant discrepancies in youth mental health assessments

On page 1 of his classic text, Millsap (2011) states, "Measurement invariance is built on the notion that a measuring device should function the same way across varied conditions, so long as those varied conditions are irrelevant [emphasis added] to the attribute being measured." By construction, measurement invariance techniques require not only detecting varied conditions but also ruling out that these conditions inform our understanding of measured domains (i.e., conditions that do not contain domain-relevant information). In fact, measurement invariance techniques possess great utility when theory and research inform their application to specific, varied conditions (e.g., cultural, ethnic, or racial background of test respondents) that, if not detected, introduce measurement biases, and, thus, depress measurement validity (e.g., academic achievement and intelligence). Yet, we see emerging bodies of work where scholars have "put the cart before the horse" when it comes to measurement invariance, and they apply these techniques to varied conditions that, in fact, may reflect domain-relevant information. These bodies of work highlight a larger problem in measurement that likely cuts across many areas of scholarship. In one such area, youth mental health, researchers commonly encounter a set of conditions that nullify the use of measurement invariance, namely discrepancies between survey reports completed by multiple informants, such as parents, teachers, and youth themselves (i.e., informant discrepancies). In this paper, we provide an overview of conceptual, methodological, and measurement factors that should prevent researchers from applying measurement invariance techniques to detect informant discrepancies. Along the way, we cite evidence from the last 15 years indicating that informant discrepancies reflect domain-relevant information. We also apply this evidence to recent uses of measurement invariance techniques in youth mental health. Based on prior evidence, we highlight the implications of applying these techniques to multi-informant data, when the informant discrepancies observed within these data might reflect domain-relevant information. We close by calling for a moratorium on applying measurement invariance techniques to detect informant discrepancies in youth mental health assessments. In doing so, we describe how the state of the science would need to fundamentally "flip" to justify applying these techniques to detect informant discrepancies in this area of work.

Integrating multi-informant reports of youth mental health: A construct validation test of Kraemer and colleagues' (2003) Satellite Model

Accurately assessing youth mental health involves obtaining reports from multiple informants who typically display low levels of correspondence. This low correspondence may reflect situational specificity. That is, youth vary as to where they display mental health concerns and informants vary as to where and from what perspective they observe youth. Despite the frequent need to understand and interpret these informant discrepancies, no consensus guidelines exist for integrating informants' reports. The path to building these guidelines starts with identifying factors that reliably predict the level and form of these informant discrepancies, and do so for theoretically and empirically relevant reasons. Yet, despite the knowledge of situational specificity, few approaches to integrating multi-informant data are well-equipped to account for these factors in measurement, and those that claim to be well-positioned to do so have undergone little empirical scrutiny. One promising approach was developed roughly 20 years ago by Kraemer and colleagues (2003). Their Satellite Model leverages principal components analysis (PCA) and strategic selection of informants to instantiate situational specificity in measurement, namely components reflecting variance attributable to the context in which informants observe behavior (e.g., home/non-home), the perspective from which they observe behavior (e.g., self/other), and behavior that manifests across contexts and perspectives (i.e., trait). The current study represents the first construct validation test of the Satellite Model. A mixed-clinical/community sample of 134 adolescents and their parents completed six parallel surveys of adolescent mental health. Adolescents also participated in a series of simulated social interactions with research personnel trained to act as same-age, unfamiliar peers. A third informant (unfamiliar untrained observer) viewed these interactions and completed the same surveys as parents and adolescents. We applied the Satellite Model to each set of surveys and observed high internal consistency estimates for each of the six-item trait (α = 0.90), context (α = 0.84), and perspective (α = 0.83) components. Scores reflecting the trait, context, and perspective components displayed distinct patterns of relations to a battery of criterion variables that varied in the context, perspective, and source of measurement. The Satellite Model instantiates situational specificity in measurement and facilitates unifying conceptual and measurement models of youth mental health.

Differences in Parent and Child Report on the Screen for Child Anxiety-Related Emotional Disorders (SCARED): Implications for Investigations of Social Anxiety in Adolescents

Social anxiety typically emerges by adolescence and is one of the most common anxiety disorders. Many clinicians and researchers utilize the Screen for Child Anxiety Related Disorders (SCARED) to quantify anxiety symptoms, including social anxiety, throughout childhood and adolescence. The SCARED can be administered to both children and their parents, though reports from each informant tend to only moderately correlate. Here, we investigated parent-child concordance on the SCARED in a sample of adolescents (N = 360, M_age = 13.2) using a multi-trait multi-method (MTMM) model. Next, in a selected sample of the adolescents, we explored relations among child report, parent report, and latent social anxiety scores with two laboratory tasks known to elicit signs of social anxiety in the presence of unfamiliar peers: a speech task and a "Get to Know You" task. Findings reveal differences in variance of the SCARED accounted for by parent and child report. Parent report of social anxiety is a better predictor of anxiety signs elicited by a structured speech task, whereas child report of social anxiety is a better predictor of anxiety signs during the naturalistic conversation with unfamiliar peers. Moreover, while latent social anxiety scores predict both observed anxiety measures, parent report more closely resembles latent scores in relation to the speech task, whereas child report functions more similarly to latent scores in relation to the peer conversation. Thus, while latent scores relate to either observed anxiety measure, parent and child report on the SCARED each provide valuable information that differentially relate to naturalistic social anxiety-related behaviors.

Teacher Versus Parent Informant Measurement Invariance of the Strengths and Difficulties Questionnaire

Background and Objectives

Obtaining a multi-informant perspective is important when assessing mental health issues in childhood and adolescence. Obtaining ratings from both parents and teachers also facilitates the evaluation of similarities and contrasts in the nature and severity of symptoms across home and school contexts. However, these informants may differ in their interpretations of observed behaviors, raising questions about the validity of comparing parents’ and teachers’ ratings.

Methods

We evaluated the cross-informant measurement invariance of one of the most widely used measures of child and adolescent mental health: The Strengths and Difficulties Questionnaire (SDQ). Using data from the UK-population representative Millennium Cohort Study, we evaluated configural, metric, and scalar measurement invariance across parents and teachers when children were aged 7 (N = 10,221) and 11 (N = 10,543).

Results

Scalar measurement invariance held at both ages. Parents reported higher levels of symptoms in all domains measured at both ages as well as higher prosociality.

Conclusions

For a UK sample, valid comparisons of parent and teacher SDQ ratings at ages 7 and 11 appear to be possible, facilitating the evaluation of contextual differences in child mental health problems. Further, parents report more problem and prosocial behavior in their children than teachers attribute to them.

Family Processes and Child Psychopathology: A Between- and Within-Family/Child Analysis

A vast array of family processes is linked to child mental development, among which (1) low parental acceptance and (2) high family conflict are known as transdiagnostic risk factors for child internalizing and externalizing psychopathology. In contrast to most prior research adopting cross-sectional or lagged designs, the current study applied fine-grained multilevel modeling to elucidate the complex relationships among parental acceptance, family conflict, and child psychopathology, considering the nesting structure of children within families and longitudinal changes within children. We focused on preadolescents from the two-wave Adolescent Brain Cognitive Development Study (N = 4,953; aged 9-12) and accounted for parental psychopathology and sex differences. Our findings suggest that consistent between-family and between-child differences in parental acceptance play a transdiagnostic role for both child internalizing and externalizing psychopathology, whereas family conflict is only significantly associated with externalizing psychopathology. Additionally, short-term within-family and within-child improvements in parental acceptance and family conflict across one year were associated with decreased externalizing, but not internalizing, psychopathology. These findings support the potential importance and feasibility of targeting these family process factors for child externalizing problems outside of an intensive treatment setting. We further discussed how such findings serve as a foundation for future research on family processes and child internalizing problems. The varying results across different grouping levels highlight the importance of decomposing within- from between-family/child effects in future studies on family processes and child psychopathology.

Experience Versus Report: Where Are Changes Seen After Exposure-Based Cognitive-Behavioral Therapy? A Randomized Controlled Group Treatment of Childhood Social Anxiety Disorder

A considerable number of children and adolescents with social anxiety disorder (SAD) do not benefit from treatment as much as expected. However, treatment success should not be measured with social anxiety reports alone; the cognitive, behavioral, and physiological components of social stress should also be assessed. The authors examined an exposure-based SAD-specific group cognitive behavioral therapy (CBT) in a randomized controlled trial (N = 67, age 9-13 years, blind randomized allocation to treatment [CBT; n = 31] and waitlist control [WLC; n = 36] groups). Success was operationalized as a clinically significant reduction of symptoms measured with SAD-specific questionnaires, structured interviews, and changes in response to the Trier Social Stress Test (TSST). In the CBT group, there was a trend toward a significant increase in positive cognitions in the TSST after treatment (d = 0.37), whereas these positive cognitions decreased in the WLC group (d = 0.40). No significant results involving group appeared for negative cognitions, behavior and physiology. Children in the CBT group, but not parents, further reported less social anxiety in one questionnaire from pre- to post-treatment (d = 0.89). A structured interview confirmed a decrease in severity of SAD in the CBT group. While the gold standard of a blind interview showed efficacy of treatment, not all trait and state measures demonstrated similar success patterns.Trial registration Eligibility criteria and some of the dependent variables (cognitions, physiology) for treatment success were registered with the German Research Foundation (TU 78/5-2, HE 3342/4-2) prior to recruitment. Clinical assessment of diagnosis and behavioral data were not a priori planned as outcome measures for this trial and therefore analyzed in a post-hoc approach.

Replication of Associations With Psychotic-Like Experiences in Middle Childhood From the Adolescent Brain Cognitive Development (ABCD) Study

The fields of psychology and psychiatry are increasingly recognizing the importance of replication efforts. The current study aimed to replicate previous findings examining the construct validity and psychometric properties of a psychotic-like experiences (PLEs) measure in middle childhood using an independent subset of the baseline Adolescent Brain Cognitive Development (ABCD) sample. Using a remainder baseline sample of 7013 nine- to eleven-year-old children with complete data, we examined measurement invariance across race/ethnicity and sex, and examined the associations between the Prodromal Questionnaire Brief-Child Version (PQ-BC) and other measures of PLEs, internalizing symptoms, neuropsychological test performance, and developmental milestones, to determine whether previously obtained results replicated in this nonoverlapping baseline sample subset. The results replicated measurement invariance across ethnicity and sex, and analyses again found higher PQ-BC scores for African American (β = .364, 95% CI = 0.292, 0.435) and Hispanic (β = .255, 95% CI = 0.185, 0.324) groups. We also replicated that higher PQ-BC scores were associated with psychosis risk measures, higher rates of child-reported internalizing symptoms (Distress: β = .378, 95% CI = 0.357,0.398), neuropsychological test performance deficits (eg, working memory; Distress: β = -.069, 95% CI = -0.096, -0.042), and motor (Distress: β = .026, 95% CI = 0.003, 0.049) and speech (Distress: β = .042, 95% CI = 0.018, 0.065) developmental milestone delays. The current results replicated many findings from the original study examining the PQ-BC. We replicated evidence for mean differences in race/ethnicity, and associations with other PLE measures, greater internalizing symptoms, cognitive impairments, and developmental milestone delays. These findings indicate robust and reliable associations between PLEs and hypothesized correlates can be found in middle childhood nonclinical samples.

Informant discrepancies in assessments of psychosocial functioning in school-based services and research: Review and directions for future research

Psychosocial functioning plays a key role in students' wellbeing and performance inside and outside of school. As such, techniques designed to measure and improve psychosocial functioning factor prominently in school-based service delivery and research. Given that the different contexts (e.g., school, home, community) in which students exist vary in the degree to which they influence psychosocial functioning, educators and researchers often rely on multiple informants to characterize intervention targets, monitor intervention progress, and inform the selection of evidence-based services. These informants include teachers, students, and parents. Across research teams, domains, and measurement methodologies, researchers commonly observe discrepancies among informants' reports. We review theory and research-occurring largely outside of school-based service delivery and research-that demonstrates how patterns of informant discrepancies represent meaningful differences that can inform our understanding of psychosocial functioning. In turn, we advance a research agenda to improve use and interpretation of informant discrepancies in school-based services and research.

A Psychometric Analysis of the Social Anxiety Scale for Adolescents Among Youth With Autism Spectrum Disorder: Caregiver-Adolescent Agreement, Factor Structure, and Validity

Social anxiety is common among adolescents with autism spectrum disorder (ASD). An ongoing challenge for both research and clinical practice in ASD is the assessment of anxious symptomatology. Despite its widespread use in samples of youth with ASD, the Social Anxiety Scale for Adolescents (SAS-A) has not received psychometric evaluation within this population; thus, the validity of its use in research and clinical practice for ASD remains unclear. The present study conducted a psychometric analysis of caregiver and adolescent SAS-A forms in a sample of adolescents with ASD (N = 197). Results revealed (1) poor caregiver-adolescent item-level agreement, (2) a two-factor structure, (3) lack of measurement invariance between reporters, and (4) modest evidence for convergent and discriminant validity. Overall, findings suggest that this measure demonstrates reasonable psychometric properties in an ASD sample. Lack of measurement invariance, however, calls for careful interpretation of research involving the SAS-A in ASD samples, particularly when the primary goal is to compare adolescent and caregiver reports. The implications of these findings for future research and clinical practice are discussed.