Bootstrap Exploratory Graph Analysis of the WISC-V with a Clinical Sample

One important aspect of construct validity is structural validity. Structural validity refers to the degree to which scores of a psychological test are a reflection of the dimensionality of the construct being measured. A factor analysis, which assumes that unobserved latent variables are responsible for the covariation among observed test scores, has traditionally been employed to provide structural validity evidence. Factor analytic studies have variously suggested either four or five dimensions for the WISC-V and it is unlikely that any new factor analytic study will resolve this dimensional dilemma. Unlike a factor analysis, an exploratory graph analysis (EGA) does not assume a common latent cause of covariances between test scores. Rather, an EGA identifies dimensions by locating strongly connected sets of scores that form coherent sub-networks within the overall network. Accordingly, the present study employed a bootstrap EGA technique to investigate the structure of the 10 WISC-V primary subtests using a large clinical sample (N = 7149) with a mean age of 10.7 years and a standard deviation of 2.8 years. The resulting structure was composed of four sub-networks that paralleled the first-order factor structure reported in many studies where the fluid reasoning and visual-spatial dimensions merged into a single dimension. These results suggest that discrepant construct and scoring structures exist for the WISC-V that potentially raise serious concerns about the test interpretations of psychologists who employ the test structure preferred by the publisher.

Long-term stability of Wechsler Intelligence Scale for Children-fifth edition scores in a clinical sample

This study investigated the stability of Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V) scores for 225 children and adolescents from an outpatient neuropsychological clinic across, on average, a 2.6 year test-retest interval. WISC-V mean scores were relatively constant but subtest stability score coefficients were all below 0.80 (M = 0.66) and only the Verbal Comprehension Index (VCI), Visual Spatial Index (VSI), and omnibus Full Scale IQ (FSIQ) stability coefficients exceeded 0.80. Neither intraindividual subtest difference scores nor intraindividual composite difference scores were stable across time (M = 0.26 and 0.36, respectively). Rare and unusual subtest and composite score differences as well as subtest and index scatter at initial testing were unlikely to be repeated at retest (kappa = 0.03 to 0.49). It was concluded that VCI, VSI, and FSIQ scores might be sufficiently stable to support normative comparisons but that none of the intraindividual (i.e. idiographic, ipsative, or person-relative) measures were stable enough for confident clinical decision making.

Factor Analysis and Variance Partitioning in Intelligence Test Research: Clarifying Misconceptions

This article addresses conceptual and methodological shortcomings regarding conducting and interpreting intelligence test factor analytic research that appeared in the Decker, S. L., Bridges, R. M., Luedke, J. C., & Eason, M. J. (2020). Dimensional evaluation of cognitive measures: Methodological confounds and theoretical concerns. Journal of Psychoeducational Assessment. Advance online publication article.

Measurement Invariance of the Wechsler Intelligence Scale for Children, Fifth Edition 10-Subtest Primary Battery: Can Index Scores be Compared across Age, Sex, and Diagnostic Groups?

Measurement invariance of the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V) 10 subtest primary battery was evaluated across sex, age (6–8, 9–11, 12–14, and 15–16 year-olds), and three diagnostic (attention-deficit/hyperactivity disorder, anxiety, and encephalopathy) groups within a large clinical sample ( N = 5359) referred to a children’s specialty hospital. Competing models were tested using confirmatory factor analysis (CFA), and a five-factor oblique model corresponding to the publisher’s hypothesized first-order measurement model (e.g., verbal comprehension, fluid reasoning, visual-spatial, working memory, and processing speed) was found to have the best model fit. Multigroup CFA was subsequently used to evaluate progressively more restrictive constraints on the measurement model. Results indicated that full metric invariance was attained across the three groups studied. Full scalar invariance was attained for sex and diagnostic groups. Partial scalar invariance was attained for age-group. The results of this study provide support for the first-order scoring structure of the five WISC-V factors in the 10 subtest primary battery with this large clinical sample.

DAS-II Cognitive Profiles Are Not Diagnostically Meaningful For Autism: A ROC Analysis

Intelligence assessment is an integral part of a comprehensive autism evaluation. Many past studies have described a cognitive profile of autistic individuals characterized by higher nonverbal than verbal IQ scores. The diagnostic utility of this profile, however, remains unknown. We leveraged receiver operating characteristic methods to determine the sensitivity, specificity, and area under the curve (AUC) of three different IQ profiles in a large sample of children who have an autism spectrum disorder diagnosis (N = 1,228, Simons Simplex Collection) who completed the Differential Ability Scales-Second Edition (DAS-II), School Age compared to the normative sample provided by the DAS-II publisher (N = 2,200). The frequently discussed nonverbal > verbal IQ profile performed near chance at distinguishing ASD from normative individuals (AUC: 0.54, 95% CI [0.52-0.56]), and performed significantly worse for females than males (AUC: females: 0.46 [0.41-0.52]; males: 0.55 [0.53-0.58]). All cognitive profiles showed AUC < 0.56. We conclude that while significant differences between verbal and nonverbal IQ scores exist at the group level, these differences are small in an absolute sense and not meaningful at an individual level. We do not recommend using cognitive profiles to aid in autism diagnostic decision-making. LAY SUMMARY: Some researchers and clinicians have reported an "autistic cognitive profile" of higher nonverbal intelligence than verbal intelligence. In an analysis of over 1,000 autistic children, we found that the group's average nonverbal intelligence is usually higher than their verbal intelligence. However, this pattern should not be used by clinicians to make an individual diagnosis of autism because our results show it is not helpful nor accurate.

Does the Factor Structure of IQ Differ Between the Differential Ability Scales (DAS-II) Normative Sample and Autistic Children?

The Differential Abilities Scales, 2nd edition (DAS-II) is frequently used to assess intelligence in autism spectrum disorder (ASD). However, it remains unknown whether the DAS-II measurement model (e.g., factor structure, loadings), which was developed on a normative sample, holds for the autistic population or requires alternative score interpretations. We obtained DAS-II data from 1,316 autistic individuals in the Simons Simplex Consortium and 2,400 individuals in the normative data set. We combined ASD and normative data sets for multigroup confirmatory factor analyses to assess different levels of measurement invariance, or how well the same measurement model fit both data sets: "weak" or metric, "strong" or scalar, and partial scalar if full scalar was not achieved. A weak invariance model showed excellent fit (Confirmatory Fit Index [CFI] > 0.995, Tucker Lewis Index [TLI] > 0.995, root mean square error of approximation [RMSEA] < 0.025), but a strong invariance model demonstrated a significant deterioration in fit during permutation testing (all p's<0.001), suggesting measurement bias, meaning systematic error when assessing autistic children. Fit improved significantly, and partial scalar invariance was achieved when either of the two spatial subtest (Recall of Designs or Pattern Construction) intercepts was permitted to vary between the ASD and normative groups, pinpointing these subtests as the source of bias. The DAS-II appears to measure verbal and nonverbal-but not spatial-intelligence in autistic children similarly as in normative sample children. These results may be driven by Pattern Construction, which shows higher scores than other subtests in the ASD sample. Clinicians assessing autistic children with the DAS-II should interpret verbal and nonverbal reasoning composite scores over the spatial score or General Composite Ability. Autism Res 2020, 13: 1184-1194. © 2020 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: The Differential Abilities Scales, 2nd edition (DAS-II) is a popular intelligence quotient (IQ) test for assessing children with autism. This article shows that the DAS-II spatial standardized scores should be interpreted with caution because they hold a different meaning for autistic children. Verbal and nonverbal reasoning scores appear valid and to hold the same meaning for those with and without autism spectrum disorder.

Confirmatory factor analyses of the Baylor Revision of the Motivation to Read Survey (B-MRS) with middle school students

Reading depends on the integration of both cognitive and motivational factors, yet reading motivation has received little attention in the research literature. The purpose of the present study was to evaluate the psychometric properties of the Baylor Revision of the Motivation to Read Survey (B-MRS; Watkins & Browning, 2015) with a sample of 731 students in Grades 6-8. The results of exploratory and confirmatory factor analyses supported the oblique two-factor structure of the B-MRS, which was invariant across gender, and regression analyses indicated that gender and grade were not significant predictors of reading motivation. These results add to the growing body of literature providing preliminary support for the B-MRS. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Construct Validity of the WISC-V in Clinical Cases: Exploratory and Confirmatory Factor Analyses of the 10 Primary Subtests

Independent exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) research with the Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V) standardization sample has failed to provide support for the five group factors proposed by the publisher, but there have been no independent examinations of the WISC-V structure among clinical samples. The present study examined the latent structure of the 10 WISC-V primary subtests with a large (N = 2,512), bifurcated clinical sample (EFA, n = 1,256; CFA, n = 1,256). EFA did not support five factors as there were no salient subtest factor pattern coefficients on the fifth extracted factor. EFA indicated a four-factor model resembling the WISC-IV with a dominant general factor. A bifactor model with four group factors was supported by CFA as suggested by EFA. Variance estimates from both EFA and CFA found that the general intelligence factor dominated subtest variance and omega-hierarchical coefficients supported interpretation of the general intelligence factor. In both EFA and CFA, group factors explained small portions of common variance and produced low omega-hierarchical subscale coefficients, indicating that the group factors were of poor interpretive value.

Dimensions of Anxiety, Age, and Gender: Assessing Dimensionality and Measurement Invariance of the State-Trait for Cognitive and Somatic Anxiety (STICSA) in an Italian Sample

The State-Trait Inventory for Cognitive and Somatic Anxiety (STICSA) is a widely used measure of state and trait anxiety that permits a specific assessment of cognitive and somatic anxiety. Previous research provided inconsistent findings about its factor structure in non-clinical samples (e.g., hierarchical or bi-factor structure). To date, no psychometric validation of the Italian version of the STICSA has been conducted. Our study aimed to determine the psychometric functioning of the Italian version of the STICSA, including its dimensionality, gender and age measurement equivalence, and convergent/divergent validity in a large sample of community-dwelling participants (N = 2,938; 55.9% female). Through confirmatory factor analysis, the multidimensional structure of both State and Trait STICSA scales, with each including Cognitive and Somatic dimensions, was supported. Factor structure invariance was tested and established at configural, metric, and scalar levels for males and females. Additionally, full factorial measurement invariance was supported for the State scale across young, middle age, and old adult groups whereas the Trait scale was partially invariant across age groups. The STICSA also showed good convergent validity with concurrent anxiety measures (State-Trait Anxiety Inventory and Beck Anxiety Inventory), and satisfactory internal discriminant validity with two depression measures (Teate Depression Inventory and Beck Depression Inventory-II). Results provided support for the multidimensionality of the STICSA, as well as the generalizability of the State and Trait scales as independent measures of Cognitive and Somatic symptomatology across gender in the general population. Implications for research and personality and clinical assessment are discussed.

Exploratory Factor Analysis: A Guide to Best Practice

Exploratory factor analysis (EFA) is a multivariate statistical method that has become a fundamental tool in the development and validation of psychological theories and measurements. However, researchers must make several thoughtful and evidence-based methodological decisions while conducting an EFA, and there are a number of options available at each decision point, some better than others. Reviews of the professional literature have consistently found that many applications of EFA are marked by an injudicious choice of methods and incomplete reports. This article provides a systematic, evidence-based guide to the conduct of EFA studies that can be followed by researchers with modest statistical training, supplemented with an example to illustrate its application.

Latent national subpopulations of early education classroom disengagement of children from underresourced families

This research examined the latent developmental patterns for early classroom disengagement among children from some of the most underresourced families in the nation. Based on standardized teacher observations from the Head Start Impact Study, a nationally representative sample of children (N=1377) was assessed for manifestations of reticent/withdrawn and low energy behavior over four years spanning prekindergarten through first grade. For each form of disengagement, latent growth mixture modeling revealed three distinct subpopulations of change patterns featuring a dominant class associated with generally good classroom adjustment, a medial class that varied close to the population average over time, and a more extreme class (about 10% of the population) whose adjustment was relatively marginal and sometimes reached problematic levels. Whereas reticent/withdrawn behavior ordinarily subsided over time, low energy behavior increased. More extreme low energy behaviors tended to dissipate through schooling and extreme reticence/withdrawal became more accentuated, with both types associated with later academic and social problems. Attendant risk and protective factors are identified and mitigating assessment and prevention measures are discussed.

Evaluation of the ADHD Rating Scale in Youth with Autism

Scientists and clinicians regularly use clinical screening tools for attention deficit/hyperactivity disorder (ADHD) to assess comorbidity without empirical evidence that these measures are valid in youth with autism spectrum disorder (ASD). We examined the prevalence of youth meeting ADHD criteria on the ADHD rating scale fourth edition (ADHD-RS-IV), the relationship of ADHD-RS-IV ratings with participant characteristics and behaviors, and its underlying factor structure in 386, 7-17 year olds with ASD without intellectual disability. Expected parent prevalence rates, relationships with age and externalizing behaviors were observed, but confirmatory factor analyses revealed unsatisfactory fits for one-, two-, three-factor models. Exploratory analyses revealed several items cross-loading on multiple factors. Implications of screening ADHD in youth with ASD using current diagnostic criteria are discussed.

A Cross-Lagged Panel Analysis of Psychometric Intelligence and Achievement in Reading and Math

A cross-lagged panel analysis of Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV) intelligence test scores and reading and math achievement test scores of 337 students twice assessed for special education eligibility across a test-retest interval of 2.85 years was conducted. General intelligence (g) was loaded by the four WISC-IV factor index scores whereas reading and math were composite scores. After confirming measurement invariance, it was found that g, reading, and math were stable across time and synchronously correlated. The cross-lagged paths from g at time 1 to reading and math at time 2 (0.26 and 0.39, respectively) were both significantly greater than zero whereas the paths from reading and math at time 1 to g at time 2 (0.03 and 0.23, respectively) were not statistically significant. Given this pattern of relationships and extant research on the correlates of general intelligence, it was tentatively inferred that general intelligence was the temporal precursor to reading and math achievement.

Structural Validity of the WISC-IV for Students With ADHD

OBJECTIVE

To evaluate the structural validity of the Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV).

METHOD

Confirmatory factor analyses were applied to a sample of 233 students diagnosed with ADHD by school multidisciplinary evaluation teams to evaluate the relative fit of the following competing models: (a) one factor, (b) two oblique verbal and nonverbal factors, (c) three oblique verbal, perceptual, and combined working memory/processing speed factors, (d) four oblique verbal, perceptual, working memory, and processing speed factors, (e) a higher-order model with four first-order factors, and (f) a bifactor model with four domain-specific factors.

RESULTS

A higher-order four-factor model fit the data best, which was composed of a general intelligence factor and four domain-specific factors that matched the four factors specified in the WISC-IV technical and interpretive manual. Moreover, the general intelligence factor explained more than two times the total variance contributed by all four domain-specific factors combined.

CONCLUSIONS

Results substantiate previous research on the WISC-IV, indicating that the general intelligence factor contributes the most reliable information. Consequently, it is recommended that interpretation of the WISC-IV remain at the Full-Scale IQ score level.

Interrater Agreement of the Adjustment Scales for Children and Adolescents

Standardized behavior rating scales and checklists offer unobtrusive evaluations of students' behavior in natural social environments. This study investigated the interrater agreement of the Adjustment Scales for Children and Adolescents (ASCA), a behavior rating scale used in school settings. Participants were 71 students enrolled in a variety of special programs who were rated by 29 observers in 24 classrooms. Resulting interrater reliability coefficients were substantial, and level differences, although significant, were not clinically meaningful. It was concluded that the ASCA produced acceptable levels of interrater agreement when educational professionals and paraprofessionals observed exceptional students within a common environment.

Long-Term Stability of Membership in a Wechsler Intelligence Scale for Children–Third Edition (WISC-III) Subtest Core Profile Taxonomy

Although often applied in practice, clinically based cognitive subtest profile analysis has failed to achieve empirical support. Nonlinear multivariate subtest profile analysis may have benefits over clinically based techniques, but the psychometric properties of these methods must be studied prior to their implementation and interpretation. The current study posed the following question: Is Wechsler Intelligence Scale for Children–Third Edition (WISC-III) cluster membership based on nonlinear multivariate subtest profile analysis stable over a 3-year period? Membership stability to a subtest taxonomy, including constancy of displaying an unusual profile, was based on data from 585 students. General (.39) and partial (.26 to .51) kappa coefficients either failed to reach statistical significance or indicated poor classification stability, with the exception of two profile types. It was concluded that, with these two possible exceptions, profile-type membership to an empirically derived WISC-III subtest taxonomy should not be used in interpretation or educational decision making.

Confir Rmatory Factor Analysis of the WISC-III for Students with Learning Disabilities

This study conducted confirmatory factor analyses to examine 12 competing models that attempt to explain the underlying latent constructs measured by the Wechsler Intelligence Scale for Children-Third Edition (WNISC-III) in a sample of 1,201 students with learning disabilities (721 White and 480 Nonwhite). Models were selected based upon theoretical explanations of the WISC-III factor structure as well as from previously conducted empirical research conducted on the W7ISC-III normative sample and with independent samples of students with exceptionalities. For both White and Nonwhite groups, four models could not be statistically distinguished from one another based upon overall fit statistics. Plausible models included the traditional first-order four-factor oblique structure of the standardization sample, the hierarchical second-order model with four first-order factors favored by Keith and Witta (1997), a bifactor second-order model with four first-order factors described by Gustafsson and Undheim (1996), and a bifactor second-order model with three first-order factors. Current results add to a growing body of evidence suggesting that WISC-III Verbal Comprehension, Perceptual Organization. and Processing Speed factors are robust across samples but the Freedom from Distractibility factor demonstrates tenuous construct validity.

Cross-Context Agreement of the Adjustment Scales for Children and Adolescents

Interobserver agreement of children's problem behavior was assessed using two samples of special education students ages 5 to 18 years. The first sample had observers from the same setting (N= 71); the second sample (N= 182) had observers from different settings with no concurrent observation. Regular and special education teachers and aides completed the Adjustment Scales for Children and Adolescents (ASCA; McDermott, Marston, & Stott, 1993). Inter- and intraclass correlations were generally significant for both samples, with some exceptions. Substantial interobserver agreement was found for the same-setting sample; however, agreement coefficients were lower for the different-setting sample and some level effects were noted. Overall, interobserver agreement for the ASCA was supported in common settings, but rating variability was evident across classrooms and appears indicative of contextual influences on behavior.

Long-Term Stability of the Wechsler Intelligence Scale for Children-Third Edition among Demographic Subgroups: Gender, Race/Ethnicity, and Age

Long-term stability of the Wechsler Intelligence Scale for Children-Third Edition was investigated separately across gender, race/ethnicity, and age subgroups. Participants were 642 students from 33 states evaluated twice for special education eligibility over a mean test-retest interval of 2.83 years. Gender, race/ethnicity, and age produced few or no differential effects on long-term stability coefficients. Most of the demographic subgroup stability coefficients for VIQ, PIQ FSIQ VCI, and POI scores demonstrated satisfactory long-term stability. However, stability coefficients for FDI, PSI, and VIQ-PIQ discrepancy scores were not adequate. Mean differences from first testing to second testing were either not statistically significant or not clinically meaningful for all groups, except Hispanic/Latino youths. Analysis of individual change scores indicated that only the FSIQ was sufficiently stable for use with individual students. Results extended those of Canivez and Watkins (1998), supporting long-term stability for the WISC-III among most demographic subgroups studied.

Assessing Early Literacy Skills with the Mountain Shadows Phonemic Awareness Scale (MS-PAS)

Considerable evidence suggests that phonemic awareness is associated with the development of skilled reading. Consequently, it is recommended that beginning readers be assessed to ensure adequate development of phonemic awareness skills. When choosing an assessment method, reliability and validity, ease of administration and scoring, and cost-effectiveness should be considered. To meet these standards, a new phonemic awareness assessment measure, the Mountain Shadows Phonemic Awareness Scale (MS-PAS), was presented. This 20-item sound categorization task takes 20 minutes for an entire first-grade classroom. Reliability and validity data were good to excellent. It was recommended that the MS-PAS be used to identify students in need of explicit phonological awareness instruction or those who require more extensive individualized assessment.

Diagnostic Utility of WISC-III Subtest Variability among Students with Learning Disabilities

Psychologists have long conjectured that Wechsler subtest variability may be an indicator of learning disability. Research on intersubtest scatter (range) with previous Wechsler scales indicated that scatter could not reliably discriminate between disabled and nondisabled children. The Profile Variability Index (PVI) also failed to demonstrate adequate diagnostic utility with the WISC-R. The present study assessed the diagnostic utility of subtest variability indices on the WISC-III by comparing 684 student with learning disabilities to the 2,200 children in the standardization sample. Results of Receiver Operating Curve (ROC) analyses indicated that WISC-III subtest variability as quantified by range and variance exhibited no diagnostic utility in distinguishing between children with learning disabilities and children from the standardization sample. It was concluded that interpretation of WISC-III subtest variability has no place in the scientific practice of school psychology.

Structural Validity of the WISC-III for a National Sample of Native American Students

Test bias research with Native American participants is uncommon, although individual tests of intelligence are often used with Native American students to determine eligibility for special education services. Only two studies with minimally adequate sample sizes have addressed the structural validity of major tests of intelligence in Native American populations. It is unfortunate that both used an obsolete test and included students from only two tribes. This study used confirmatory factor analyses to examine the structure of the Wechsler Intelligence Scale for Children—Third Edition (WISC-III) among 344 Native American students representing 12 Bureau of Indian Affairs (BIA) Nations attending BIA schools in 11 states. Results indicated that Wechsler's four-factor oblique model exhibited the best overall statistical fit. Thus, the underlying factor structure of the WISC-III with a national sample of Native Americans was similar to that found in the normative sample. Implications for school psychologists are presented and recommendations for further research are provided.

Role of the School Psychologist

School psychologists often report that they would like to reduce their involvement in assessment activities in favour of an increase in other service delivery roles. Surveys of teachers and administrators reveal that they do not want a reduction in assessment services. Rather, they want more school psychology services of all kinds. In this study, 522 school staff from a suburban southwestern school district in the USA were surveyed regarding the importance of nine school psychology services. School staff gave ‘very important’ ratings to six services: assessment, special education input, consultation, counselling, crisis intervention and behaviour management. School-community liaison and parent education roles were rated as ‘fairly important’ and staff development was considered to be ‘somewhat important.’ School staff also wanted a school psychologist to be available on a daily basis at their schools. These results are discussed in terms of the ‘paradox of school psychology’ and the skills required of school psychologists who wish to deliver diverse school psychological services.

Structural Validity of the WISC-IV for Students With Learning Disabilities

The structural validity of the Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV) was evaluated using confirmatory factor analysis for a clinical sample of 1,537 students diagnosed with specific learning disabilities (SLD) by school psychologists in two large southwestern school districts. Results indicated that a bifactor model consisting of four first-order domain specific factors and a general intelligence breadth factor fit the data best. Consequently, the structural validity of the WISC-IV for students with SLD was supported by the results of the present study. The general intelligence factor contributed the most information, accounting for 48% of the common variance. Given this structure, it was recommended that score interpretation should emphasize the Full-Scale IQ score because of the marginal contributions of the first-order domain-specific factors and their low precision of measurement independent of the general factor.