Psychometric defensibility of the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) Teacher Rating Scale and multiple gating procedure within elementary and middle school samples

Multi-informant Assessment of Internalizing Concerns: Rater Concordance and Implications for Decision-Making

Rising rates of mental health challenges among youths have become a significant concern following the COVID 19 pandemic. Although strong evidence supports the implementation of universal screening as a preventative approach to address unmet mental health concerns, the research is less clear surrounding the use of such data in decision-making processes when significant discrepancies between informants (e.g., students and teachers) exist. The purpose of the study was twofold. First, the study aimed to determine the degree of rater concordance between teachers and students on students' internalizing concerns. The second objective was to determine whether concordance on internalizing behaviors differs across ages/grades and if this differentially impacts distal (i.e., academic) outcomes. Results indicated that teachers and students demonstrated limited agreement on ratings of internalizing behaviors. However, when students and teachers agreed, higher and more positive emotional behaviors were linked to higher reading/math performance. Furthermore, patterns of informant dis/agreement and relationships between internalizing concerns and academic outcomes were similar across grade levels. Implications and areas for future research are discussed.

The Utility of the Behavior Assessment System for Children-2 Behavioral and Emotional Screening System and Strengths and Difficulties Questionnaire in Predicting Mental Disorders in the Project to Learn About Youth-Mental Health

We examined the predictive utility of the Behavior Assessment System for Children-2 Behavioral and Emotional Screening System (BASC-2-BESS) and Strengths and Difficulties Questionnaire (SDQ) in identifying students with a mental disorder. Data were collected in a two-stage study over 34 months with kindergarten-12th grade (K-12) students (aged 5-19 years) in four U.S. school districts. In Stage 1, teachers completed the BASC-2-BESS and the SDQ. In Stage 2, parents of 1,054 children completed a structured diagnostic interview to determine presence of a mental disorder. Results suggest that teacher versions of the BASC-2-BESS and SDQ have modest utility in identifying children meeting criteria for a mental disorder based on parent report. Area Under the Curve (AUC) statistics representing prediction of any externalizing disorder (.73 for both measures) were higher than the AUCs predicting any internalizing disorder (.58 for both measures). Findings can inform the use of teacher report in mental health screening, specifically the selection of measures when implementing screening procedures.

Risk Profiles of Korean Adolescents in Relations With Contextual Factors: Implications for Multi-Tiered Systems of Support

INTRODUCTION

Although prior studies have supported the effectiveness of Multi-Tiered Systems of Support (MTSS) on addressing social, emotional, behavioral, and academic challenges faced by youth at-risk, educators using MTSS often do not consider contextual factors which may also influence youth at-risk and the interventions targeting them. This study thus aimed to identify youth at-risk who should be referred to targeted instructions within MTSS by examining the risk profiles of Korean adolescents. Based on the identified risk profiles, we also tried to investigate the effect of contextual factors on deciding youth at-risk and confirm whether and/or what contextual factors should be considered when implementing targeted interventions for them.

METHOD

To accomplish the research goal, a latent profile analysis on risk factors of Korean adolescents was performed, using the first year data of "Korean Children and Youth Panel Study (KCYPS) 2018."

RESULTS

Four risk profiles were identified, using low academic motivation, low academic behavior, attention deficit, aggression, social withdrawal, and depression as indicators: the high risk, M-SEB (Moderate-social, emotional, & behavioral) risk, M-ACA (Moderate-academic) risk, and low risk group. The covariates of this study, home and school environmental variables, worked as predictors of adolescents included in the high group.

CONCLUSION

The results of this study suggest students in the high risk group (16.8%) should be given targeted instructions combining academic and SEB support within MTSS so as to prevent negative outcomes in the future among all adolescents. Those instructions need to be planned with consideration of contextual factors accompanied by teacher's careful understanding of social dynamics surrounding each student.

The Development and Validation of a New Multidimensional Test Anxiety Scale (MTAS)

Abstract. Although test anxiety has a long history in the educational and psychological literature there is a lack of census over its dimensionality. The aim of the present study was to clarify the dimensionality of test anxiety and develop a new instrument to reflect this dimensionality. Across two empirical studies we tested and refined a new multidimensional instrument comprising of two cognitive dimensions (Worry and Cognitive Interference) and two affective-physiological dimensions (Tension and Physiological Indicators). In both studies, four-correlated-factors and higher order models showed a good fit to the data. Test anxiety was positively related to an existing test anxiety measure (the Test Anxiety Inventory) and an elevated risk of mental health problems, and negatively related to school well-being and examination performance. This new instrument will prove a welcome addition for practitioners, to assist in the identification of highly test anxious students who may require support or intervention, and test anxiety researchers.

Multi-informant universal screening: Evaluation of rater, item, and construct variance using a trifactor model

Universal screening is a proactive method for identifying student risk, yet remains under-utilized in school systems. Instead, many schools rely on teacher reports and referrals without accounting for different informant perspectives. In the current study, multi-informant universal screening in evaluated using a trifactor model. The study utilized the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS), specifically the teacher (SAEBRS-TRS) and student (mySAEBRS) self-report forms, with items indicating risk for social, academic, and emotional behavior. Data from a national sample of over 24,000 K-12 teacher-student dyads were used to examine the extent and variance of discrepant reports between students and teachers of common, perspective, and item factors. Results demonstrated that informant perspective factors were a strong predictor for student and teacher emotional behavior item ratings. Whereas age had a positive effect on younger student reports of risk on the behavior items compared to older student reports, teachers showed the opposite effect. The teacherperspective of social and emotional behaviors of students was predicted by gender. Implications and directions for future research are further discussed.

Predicting School Suspension Risk from Eighth through Tenth Grade Using the Strengths and Difficulties Questionnaire

The current study examined (1) if the Strengths and Difficulties Questionnaire (SDQ) would yield alternative factor structures related to either symptoms or strengths with early adolescent students when an exploratory factor analysis (EFA) is used; (2) which scales best predicted suspensions of typically developing early adolescents; and (3) what cut-off scores were useful for identifying youth at risk for suspensions. The current study included 321 parent-student dyads, who were followed from the middle of eighth grade until the end of tenth grade. A symptoms-based EFA yielded three factors: Misbehavior, Isolation, and Agitation. A strength-based EFA yielded three factors, as, well: Emotional, Social, and Moral competence. Logistic regression path analyses were used to predict risk of any suspension at the end of eighth, ninth, and tenth grades. The predictor variables were the original SDQ Conduct Problems and Hyperactivity scales in one model, the Misbehavior and Agitation scales in a second model, and the Emotional and Moral competence scales in the third model. Only the Misbehavior scale consistently predicted suspensions across each grade (b = .27, OR = 1.32, p < .001; b = .15, OR = 1.18, p = .029; b = .17, OR = 1.18, p = .029, respectively). For the Misbehavior scale, cut-off scores were established that reflected the 75th and 90th percentile; however, each cut-off demonstrated strengths and weaknesses for identifying at-risk students. The expectation of screening to identify youth at-risk for suspensions, a complex school discipline decision, is discussed.

Test use and assessment practices of school psychologists in the United States: Findings from the 2017 National Survey

Although several studies have examined the specific instruments and procedures used by school psychologists when conducting comprehensive psychoeducational evaluations, the last one was published over 20 years ago (viz., Wilson & Reschly, 1996). Given the substantial theoretical and practical advances in assessment since then, the purpose of the current study was to examine the test use and assessment practices of contemporary school psychologists in the United States. Data from the 2017 National Survey of Assessment Practices in School Psychology revealed that test use and assessment practices have evolved significantly. Much of this change consists of the substitution of tests and practices with limited reliability and validity with those with greater psychometric support. Results of this study also indicate that school psychologists regularly conduct multi-method assessments to prevent, identify, monitor, and remediate child and adolescent learning difficulties and other presenting problems in the schools.

Evidence for the interpretation of Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) scores: An argument-based approach to screener validation

In accordance with an argument-based approach to validation, the purpose of the current study was to yield evidence relating to Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) score interpretation. Bifactor item response theory analyses were performed to examine SAEBRS item functioning. Structural equation modeling (SEM) was used to simultaneously evaluate intra- and inter-scale relationships, expressed through (a) a measurement model specifying a bifactor structure to SAEBRS items, and (b) a structural model specifying convergent and discriminant relations with an outcome measure (i.e., Behavioral and Emotional Screening System [BESS]). Finally, hierarchical omega coefficients were calculated in evaluating the model-based internal reliability of each SAEBRS scale. IRT analyses supported the adequate fit of the bifactor model, indicating items adequately discriminated moderate and high-risk students. SEM results further supported the fit of the latent bifactor measurement model, yielding superior fit relative to alternative models (i.e., unidimensional and correlated factors). SEM analyses also indicated the latent SAEBRS-Total Behavior factor was a statistically significant predictor of all BESS subscales, the SAEBRS-Academic Behavior predicted BESS Adaptive Skills subscales, and the SAEBRS-Emotional Behavior predicted the BESS Internalizing Problems subscale. Hierarchical omega coefficients indicated the SAEBRS-Total Behavior factor was associated with adequate reliability. In contrast, after accounting for the total scale, each of the SAEBRS subscales was associated with somewhat limited reliability, suggesting variability in these scores is largely driven by the Total Behavior scale. Implications for practice and future research are discussed.

A universal behavioral screener linked to personalized classroom interventions: Psychometric characteristics in a large sample of German schoolchildren

The current study represents the first psychometric evaluation of an American English into German translation of a school-based universal screening measure designed to assess academic and disruptive behavior problems. This initial study examines the factor structure and diagnostic accuracy of the newly translated measure in a large sample of 1009 German schoolchildren attending grades 1-6 in Western Germany. Confirmatory factor analysis supported a two-factor model for both male- and female- students. Configural invariance was supported between male- and female-samples. However scalar invariance was not supported, with higher thresholds for ratings of female students. Results of receiver operating characteristic (ROC) curve analyses were indicative of good to excellent diagnostic accuracy with areas under the curve ranging from 0.89 to 0.93. Optimal cut-off scores were 10, 5, and 13 for the Academic Productivity/Disorganization, Oppositional/Disruptive, and the Total Problems Composite scores of the Integrated System Teacher Rating Form respectively. This initial study of the newly translated measure supports further investigations into its utility for universal screening in German speaking schools.

Measurement equivalence: A non-technical primer on categorical multi-group confirmatory factor analysis in school psychology

Evidence-based interventions (EBIs) have become a central component of school psychology research and practice, but EBIs are dependent upon the availability and use of evidence-based assessments (EBAs) with diverse student populations. Multi-group confirmatory factor analysis (MG-CFA) is an analytical tool that can be used to examine the validity and measurement equivalence/invariance of scores across diverse groups. The objective of this article is to provide a conceptual and procedural overview of categorical MG-CFA, as well as an illustrated example based on data from the Social and Academic Behavior Risk Screener (SABRS) - a tool designed for use in school-based interventions. This article serves as a non-technical primer on the topic of MG-CFA with ordinal (rating scale) data and does so through the framework of examining equivalence of measures used for EBIs within multi-tiered models - an understudied topic. To go along with the illustrated example, we have provided supplementary files that include sample data, Mplus input code, and an annotated guide for understanding the input code (http://dx.doi.org/10.1016/j.jsp.2016.11.002). Data needed to reproduce analyses in this article are available as supplemental materials (online only) in the Appendix of this article.