Monte Carlo Modeling of Contemporary Intelligence Test (IQ) Factor Structure: Implications for IQ Assessment, Interpretation, and Theory

Autism Symptom Presentation and Hierarchical Models of Intelligence

BACKGROUND

There is a substantial history studying the relationship between general intelligence and the core symptoms of autism. However, a gap in knowledge is how dimensional autism symptomatology associates with different components of clinically-relevant hierarchical models of intelligence.

METHOD

We examined correlations between autism diagnostic symptom magnitude (Autism Diagnostic Observational Schedule; ADOS) and a hierarchical statistical model of intelligence. One autistic cohort was tested on the fourth edition of Wechsler Intelligence Scale for Children (WISC-IV; N = 131), and another on the fifth edition (WISC-V; N = 83). We anticipated a convergent pattern of results between cohorts.

RESULTS

On WISC-IV, ADOS scores were correlated significantly with g and three out of four intermediate factor scores, which was a broader pattern of correlations than anticipated from the literature. In the WISC-V cohort, only one intermediate factor correlated significantly with the ADOS; correlations with g and the other intermediate factors were less statistically certain. ADOS-factor correlations were larger in the WISC-IV than WISC-V cohort; this difference was significant at the 90% level.

CONCLUSIONS

WISC-IV shows dimensional relationships with ADOS at multiple points in the hierarchical model of intelligence. Moreover, the current results provide evidence that relationship between core autism symptomatology and the construct of general intelligence may depend on how intelligence is measured. Known cohort effects in the relationship between categorical autism diagnosis and general intelligence have previously been attributed to changes in autism diagnostic practices. To our knowledge, this is the first evidence that differing versions of IQ tests may be implicated.

Bootstrap Exploratory Graph Analysis of the WISC-V with a Clinical Sample

One important aspect of construct validity is structural validity. Structural validity refers to the degree to which scores of a psychological test are a reflection of the dimensionality of the construct being measured. A factor analysis, which assumes that unobserved latent variables are responsible for the covariation among observed test scores, has traditionally been employed to provide structural validity evidence. Factor analytic studies have variously suggested either four or five dimensions for the WISC-V and it is unlikely that any new factor analytic study will resolve this dimensional dilemma. Unlike a factor analysis, an exploratory graph analysis (EGA) does not assume a common latent cause of covariances between test scores. Rather, an EGA identifies dimensions by locating strongly connected sets of scores that form coherent sub-networks within the overall network. Accordingly, the present study employed a bootstrap EGA technique to investigate the structure of the 10 WISC-V primary subtests using a large clinical sample (N = 7149) with a mean age of 10.7 years and a standard deviation of 2.8 years. The resulting structure was composed of four sub-networks that paralleled the first-order factor structure reported in many studies where the fluid reasoning and visual-spatial dimensions merged into a single dimension. These results suggest that discrepant construct and scoring structures exist for the WISC-V that potentially raise serious concerns about the test interpretations of psychologists who employ the test structure preferred by the publisher.

Autism and Hierarchical Models of Intelligence

BACKGROUND

The Wechsler Intelligence Scale for Children (WISC) employs a hierarchical model of general intelligence in which index scores separate out different clinically-relevant aspects of intelligence; the test is designed such that index scores are statistically independent from one another within the normative sample. Whether or not the existing index scores meet the desired psychometric property of being statistically independent within autistic samples is unknown.

METHOD

We conducted a factor analysis on WISC fifth edition (WISC-V) (N = 83) and WISC fourth edition (WISC-IV) (N = 131) subtest data in children with autism. We compared the data-driven exploratory factor analysis with the manual-derived index scores, including in a typically developing (TD) WISC-IV cohort (N = 209).

RESULTS

The WISC-IV TD cohort showed the expected 1:1 relationship between empirically derived factors and manual-derived index scores. We observed less unique correlations between our data-driven factors and manualized IQ index scores in both ASD samples (WISC-IV and WISC-V). In particular, in both WISC-IV and -V, working memory (WM) influenced index scores in autistic individuals that do not load on WM in the normative sample.

CONCLUSIONS

WISC index scores do not show the desired statistical independence within autistic samples, as judged against an empirically-derived exploratory factor analysis. In particular, within the currently used WISC-V version, WM influences multiple index scores.

The Validity of WISC-V Profiles of Strengths and Weaknesses

The Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V; Wechsler, 2014 ) provides a general intelligence score, representing g, and five index scores, reflecting underlying broad factors. Within person differences between the overall performance across subtests and index scores, denoted as index difference scores, are often used to examine profiles of strengths and weaknesses. In this study, the validity of such profiles was examined for the Dutch WISC-V. In line with previous studies, broad factors explained little variance in index scores. A simulation study showed that variation in index difference scores also reflected little broad factor variance. The simulation study further revealed that, as a consequence, a significant discrepancy between an index score and overall performance was accompanied in only 40%–74% of the cases by a discrepancy on the underlying broad factor. Overall, these results provide little support for the validity and thereby clinical use of WISC-V profiles.

Long-term stability of Wechsler Intelligence Scale for Children-fifth edition scores in a clinical sample

This study investigated the stability of Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V) scores for 225 children and adolescents from an outpatient neuropsychological clinic across, on average, a 2.6 year test-retest interval. WISC-V mean scores were relatively constant but subtest stability score coefficients were all below 0.80 (M = 0.66) and only the Verbal Comprehension Index (VCI), Visual Spatial Index (VSI), and omnibus Full Scale IQ (FSIQ) stability coefficients exceeded 0.80. Neither intraindividual subtest difference scores nor intraindividual composite difference scores were stable across time (M = 0.26 and 0.36, respectively). Rare and unusual subtest and composite score differences as well as subtest and index scatter at initial testing were unlikely to be repeated at retest (kappa = 0.03 to 0.49). It was concluded that VCI, VSI, and FSIQ scores might be sufficiently stable to support normative comparisons but that none of the intraindividual (i.e. idiographic, ipsative, or person-relative) measures were stable enough for confident clinical decision making.

Factor Analysis and Variance Partitioning in Intelligence Test Research: Clarifying Misconceptions

This article addresses conceptual and methodological shortcomings regarding conducting and interpreting intelligence test factor analytic research that appeared in the Decker, S. L., Bridges, R. M., Luedke, J. C., & Eason, M. J. (2020). Dimensional evaluation of cognitive measures: Methodological confounds and theoretical concerns. Journal of Psychoeducational Assessment. Advance online publication article.