Exploratory bifactor analysis of the Wechsler Intelligence Scale for Children—Fifth Edition with the 16 primary and secondary subtests

Executive function deficits in attention-deficit/hyperactivity disorder and autism spectrum disorder

Executive function deficits have been reported in both autism spectrum disorder (ASD) and attention-deficit/hyperactivity disorder (ADHD). However, little is known regarding which, if any, of these impairments are unique vs. shared in children with ADHD versus ASD. In this Review, we provide an overview of the current literature with a critical eye toward diagnostic, measurement, and third-variable considerations that should be leveraged to provide more definitive answers. We conclude that the field's understanding of ASD and ADHD executive function profiles is highly limited because most research on one disorder has failed to account for their co-occurrence and the presence of symptoms of the other disorder; a vast majority of studies have relied on traditional neuropsychological tests and/or informant-rated executive function scales that have poor specificity and construct validity; and most studies have been unable to account for the well-documented between-person heterogeneity within and across disorders. Currently, the most parsimonious conclusion is that children with ADHD and/or ASD tend to perform moderately worse than neurotypical children on a broad range of neuropsychological tests. However, the extent to which these difficulties are unique vs. shared, or attributable to impairments in specific executive functions subcomponents, remains largely unknown. We end with focused recommendations for future research that we believe will advance this important line of inquiry.

The Validity of WISC-V Profiles of Strengths and Weaknesses

The Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V; Wechsler, 2014 ) provides a general intelligence score, representing g, and five index scores, reflecting underlying broad factors. Within person differences between the overall performance across subtests and index scores, denoted as index difference scores, are often used to examine profiles of strengths and weaknesses. In this study, the validity of such profiles was examined for the Dutch WISC-V. In line with previous studies, broad factors explained little variance in index scores. A simulation study showed that variation in index difference scores also reflected little broad factor variance. The simulation study further revealed that, as a consequence, a significant discrepancy between an index score and overall performance was accompanied in only 40%–74% of the cases by a discrepancy on the underlying broad factor. Overall, these results provide little support for the validity and thereby clinical use of WISC-V profiles.

The Optimal Design of Bifactor Multidimensional Computerized Adaptive Testing with Mixed-format Items

Multidimensional computerized adaptive testing (MCAT) using mixed-format items holds great potential for the next-generation assessments. Two critical factors in the mixed-format test design (i.e., the order and proportion of polytomous items) and item selection were addressed in the context of mixed-format bifactor MCAT. For item selection, this article presents the derivation of the Fisher information matrix of the bifactor graded response model and the application of the bifactor dimension reduction method to simplify the computation of the mutual information (MI) item selection method. In a simulation study, different MCAT designs were compared with varying proportions of polytomous items (0.2-0.6, 1), different item-delivering formats (DPmix: delivering polytomous items at the final stage; RPmix: random delivering), three bifactor patterns (low, middle, and high), and two item selection methods (Bayesian D-optimality and MI). Simulation results suggested that a) the overall estimation precision increased with a higher bifactor pattern; b) the two item selection methods did not show substantial differences in estimation precision; and c) the RPmix format always led to more precise interim and final estimates than the DPmix format. The proportions of 0.3 and 0.4 were recommended for the RPmix and DPmix formats, respectively.

Characterizing and comparing adaptive and academic functioning in children with low average and below average intellectual abilities

Objective The recent American Academy of Clinical Neuropsychology (AACN) consensus statement on uniform labeling of performance test scores places children who were previously characterized as having "borderline intellectual functioning" within the low average (LA; full scale intellectual quotient (FSIQ) between 80-89) or below average (BA; FSIQ between 70-79) categories. Given limited research examining functional differences across FSIQ groups using AACN's uniform labeling, this study examined adaptive and academic functioning by FSIQ group in youth referred for (neuro)psychological evaluation. Primary comparisons of interest were between LA and BA groups. MethodParticipants were 2,516 children between 6 to 13 years with standardized measures of intellectual, adaptive, and academic functioning. Participants were included if their FSIQ ranged from average to exceptionally low. Group differences in adaptive functioning and academic achievement were examined. ResultsThe LA group did not differ from the BA group in overall adaptive functioning and several domains of adaptive functioning (i.e. social, practical), but demonstrated slightly stronger adaptive skills in the conceptual domain. While the LA group evidenced slightly better word reading and math computation scores than the BA group, these statistically significant differences were not clinically -meaningful. ConclusionsIn this clinically referred sample, children with LA and BA intellectual abilities demonstrated similar adaptive skills, but slightly different academic achievement. Both groups demonstrated lower adaptive and academic functioning than children with average range FSIQs. These results suggest that adaptive functioning should be assessed during (neuro)psychological evaluations even when children do not have extremely low FSIQs.

Diversity of Intelligence is the Norm Within the Autism Spectrum: Full Scale Intelligence Scores Among Children with ASD

Although previous research helped to define differences in intelligence between neurotypicals and those with ASD, results were limited by small sample sizes or restricted subtests. Using data from the NIMH Data Archive, this study examined the intelligence of children with ASD (N = 671). Results demonstrate an average standard deviation of 25.75, which is 1.72 times greater than that of the normative sample for the WISC-III. Moreover, students with ASD are 12 times more likely than the general population of students to score within the intellectual disability range, but are also 1.5 times more likely to score in the superior range, suggesting that more students with ASD should be considered for giftedness. Determining the diversity of intelligence among those with ASD has implications for research, clinical practice, and neurological understanding.

Revisiting the Factor Structure of the German WISC-V for Clinical Interpretability: An Exploratory and Confirmatory Approach on the 10 Primary Subtests

With the exception of a recently published study and the analyses provided in the test manual, structural validity is mostly uninvestigated for the German version of the Wechsler Intelligence Scale for Children - Fifth Edition (WISC-V). Therefore, the main aim of the present study was to examine the latent structure of the 10 WISC-V primary subtests on a bifurcated extended population-representative German standardization sample (N=1,646) by conducting both exploratory (EFA; n=823) and confirmatory (CFA; n=823) factor analyses on the original data. Since no more than one salient subtest loading could be found on the Fluid Reasoning (FR) factor in EFA, results indicated a four-factor rather than a five-factor model solution when the extraction of more than two suggested factors was forced. Likewise, a bifactor model with four group factors was found to be slightly superior in CFA. Variance estimation from both EFA and CFA revealed that the general factor dominantly accounted for most of the subtest variance and construct reliability estimates further supported interpretability of the Full Scale Intelligence Quotient (FSIQ). In both EFA and CFA, most group factors explained rather small proportions of common subtest variance and produced low construct replicability estimates, suggesting that the WISC-V primary indexes were of lower interpretive value and should be evaluated with caution. Clinical interpretation should thus be primarily based on the FSIQ and include a comprehensive analysis of the cognitive profile derived from the WISC-V primary indexes rather than analyses of each single primary index.

Dimensional Evaluation of Cognitive Measures: Methodological Confounds and Theoretical Concerns

The current study provides a methodological review of studies supporting a general factor of intelligence as the primary model for contemporary measures of cognitive abilities. A further evaluation is provided by an empirical evaluation that compares statistical estimates using different approaches in a large sample of children (ages 9–13 years, N = 780) administered a comprehensive battery of cognitive measures. Results from this study demonstrate the ramifications of using the bifactor and Schmid–Leiman (BF/SL) technique and suggest that using BF/SL methods limit interpretation of cognitive abilities to only a general factor. The inadvertent use of BF/SL methods is demonstrated to impact both model dimensionality and variance estimates for specific measures. As demonstrated in this study, conclusions from both exploratory and confirmatory studies using BF/SL methods are significantly questioned, especially for studies with a questionable theoretical basis. Guidelines for the interpretation of cognitive test scores in applied practice are discussed.

Use of translated and adapted versions of the WISC-V: Caveat emptor

The Wechsler Intelligence Scale for Children (WISC) is the most widely used intelligence test in the world. Now in its fifth edition, the WISC-V has been translated and adapted for use in nearly a dozen countries. Despite its popularity, numerous concerns have been raised about some of the procedures used to develop and validate translated and adapted versions of the test around the world. The purpose of this article is to survey the most salient of those methodological and statistical limitations. In particular, empirical data are presented that call into question the equating procedures used to validate the WISC-V Spanish, suggesting cautious use of that instrument. It is believed that the issues raised in the present article will be instructive for school psychologists engaged in the clinical assessment of intelligence with the WISC-V Spanish and with other translated and adapted versions around the world.

Application of Dimension Reduction to CAT Item Selection Under the Bifactor Model

Multidimensional computerized adaptive testing (MCAT) based on the bifactor model is suitable for tests with multidimensional bifactor measurement structures. Several item selection methods that proved to be more advantageous than the maximum Fisher information method are not practical for bifactor MCAT due to time-consuming computations resulting from high dimensionality. To make them applicable in bifactor MCAT, dimension reduction is applied to four item selection methods, which are the posterior-weighted Fisher D-optimality (PDO) and three non-Fisher information-based methods-posterior expected Kullback-Leibler information (PKL), continuous entropy (CE), and mutual information (MI). They were compared with the Bayesian D-optimality (BDO) method in terms of estimation precision. When both the general and group factors are the measurement objectives, BDO, PDO, CE, and MI perform equally well and better than PKL. When the group factors represent nuisance dimensions, MI and CE perform the best in estimating the general factor, followed by the BDO, PDO, and PKL. How the bifactor pattern and test length affect estimation accuracy was also discussed.

Monte Carlo Modeling of Contemporary Intelligence Test (IQ) Factor Structure: Implications for IQ Assessment, Interpretation, and Theory

Researchers continue to debate the constructs measured by commercial ability tests. Factor analytic investigations of these measures have been used to develop and refine widely adopted psychometric theories of intelligence particularly the Cattell-Horn-Carroll (CHC) model. Even so, this linkage may be problematic as many of these investigations examine a particular instrument in isolation and CHC model specification across tests and research teams has not been consistent. To address these concerns, the present study used Monte Carlo resampling to investigate the latent structure of four of the most widely used intelligence tests for children and adolescents. The results located the approximate existence of the publisher posited CHC theoretical group factors in the Differential Abilities Scales-Second edition and the Kaufman Assessment Battery for Children-Second edition but not in the Wechsler Intelligence Scale for Children-Fifth edition or the Woodcock-Johnson IV Tests of Cognitive Abilities. Instead, the results supported alternative conceptualizations from independent factor analytic research. Additionally, whereas a bifactor model produced superior fit indices in two instruments (Wechsler Intelligence Scale for Children-Fifth edition and Woodcock-Johnson IV Tests of Cognitive Abilities), a higher order structure was found to be superior in the Kaufman Assessment Battery for Children-Second edition and the Differential Abilities Scales-Second edition. Regardless of the model employed, the general factor captured a significant portion of each instrument's variance. Implications for IQ test assessment, interpretation, and theory are discussed.

Exploratory Factor Analyses of the Intelligence and Development Scales-2: Implications for Theory and Practice

The factor structure of the intelligence and scholastic skills domains of the Intelligence and Development Scales-2 was examined using exploratory factor analyses with the standardization and validation sample (N = 2,030, aged 5 to 20 years). Results partly supported the seven proposed intelligence group factors. However, the theoretical factors Visual Processing and Abstract Reasoning as well as Verbal Reasoning and Long-Term Memory collapsed, resulting in a five-factor structure for intelligence. Adding the three scholastic skills subtests resulted in an additional factor Reading/Writing and in Logical-Mathematical Reasoning showing a loading on abstract Visual Reasoning and the highest general factor loading. A data-driven separation of intelligence and scholastic skills is not evident. Omega reliability estimates based on Schmid-Leiman transformations revealed a strong general factor that accounted for most of the true score variance both overall and at the group factor level. The possible usefulness of factor scores is discussed.

Construct Validity of the WISC-V in Clinical Cases: Exploratory and Confirmatory Factor Analyses of the 10 Primary Subtests

Independent exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) research with the Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V) standardization sample has failed to provide support for the five group factors proposed by the publisher, but there have been no independent examinations of the WISC-V structure among clinical samples. The present study examined the latent structure of the 10 WISC-V primary subtests with a large (N = 2,512), bifurcated clinical sample (EFA, n = 1,256; CFA, n = 1,256). EFA did not support five factors as there were no salient subtest factor pattern coefficients on the fifth extracted factor. EFA indicated a four-factor model resembling the WISC-IV with a dominant general factor. A bifactor model with four group factors was supported by CFA as suggested by EFA. Variance estimates from both EFA and CFA found that the general intelligence factor dominated subtest variance and omega-hierarchical coefficients supported interpretation of the general intelligence factor. In both EFA and CFA, group factors explained small portions of common variance and produced low omega-hierarchical subscale coefficients, indicating that the group factors were of poor interpretive value.

Investigating the Theoretical Structure of the Differential Ability Scales—Second Edition Through Hierarchical Exploratory Factor Analysis

When the Differential Ability Scales–Second Edition (DAS-II) was developed, the instrument’s content, structure, and theoretical orientation were amended. Despite these changes, the Technical Handbook did not report results from exploratory factor analytic investigations, and confirmatory factor analyses were implemented using selected subtests across the normative age groups from the total battery. To address these omissions, the present study investigated the theoretical structure of the DAS-II using principal axis factoring followed by the Schmid–Leiman procedure with participants from the 5- to 8-year-old age range to determine the degree to which the DAS-II theoretical structure proposed in the Technical Handbook could be replicated. Unlike other age ranges investigated where at most 14 subtests were administered, the entire DAS-II battery was normed on participants aged 5 to 8 years, making it well suited to test the full instrument’s alignment with theory. Results suggested a six-factor solution that was essentially consistent with the Cattell–Horn–Carroll (CHC)-based theoretical structure suggested by the test publisher and simple structure was attained. The only exception involved two subtests (Picture Similarities and Early Number Concepts) that did not saliently load on a group factor. Implications for clinical practice are discussed.

The Structure and Measurement of Unusual Sensory Experiences in Different Modalities: The Multi-Modality Unusual Sensory Experiences Questionnaire (MUSEQ)

Hallucinations and other unusual sensory experiences (USE) can occur in all modalities in the general population. Yet, the existing literature is dominated by investigations into auditory hallucinations (“voices”), while other modalities remain under-researched. Furthermore, there is a paucity of measures which can systematically assess different modalities, which limits our ability to detect individual and group differences across modalities. The current study explored such differences using a new scale, the Multi-Modality Unusual Sensory Experiences Questionnaire (MUSEQ). The MUSEQ is a 43-item self-report measure which assesses USE in six modalities: auditory, visual, olfactory, gustatory, bodily sensations, and sensed presence. Scale development and validation involved a total of 1,300 participants, which included: 513 students and community members for initial development, 32 individuals with schizophrenia spectrum disorder or bipolar disorder for validation, 659 students for factor replication, and 96 students for test-retest reliability. Confirmatory factor analyses showed that a correlated-factors model and bifactor model yielded acceptable model fit, while a unidimensional model fitted poorly. These findings were confirmed in the replication sample. Results showed contributions from a general common factor, as well as modality-specific factors. The latter accounted for less variance than the general factor, but could still detect theoretically meaningful group differences. The MUSEQ showed good reliability, construct validity, and could discriminate non-clinical and clinical groups. The MUSEQ offers a reliable means of measuring hallucinations and other USE in six different modalities.

Scores in Space: Multidimensional Scaling of the WISC-V

The purpose of this study was to use multidimensional scaling (MDS) to investigate relations among scores from the standardization sample of the Wechsler Intelligence Scale for Children–Fifth edition (WISC-V; Wechsler, 2014). Nonmetric two-dimensional MDS maps were selected for interpretation. The most cognitively complex subtests and indexes were located near the center of the maps. Subtests were also grouped together in the way that they are organized in the primary and complementary indexes, and not necessarily on content surface features. Naming Speed and Symbol Translation scores may be better off kept as separate indexes, and Digit Span Sequencing appeared to add more complexity to the Digit Span subtest. One implication related to score interpretation is that General Ability Index (GAI) and Cognitive Proficiency Index (CPI) may be distinguished by the level of complexity of the subtests included in each index (i.e., more complex subtests included in GAI and less complex subtests included in the CPI).