Towards the Validation of Executive Functioning Assessments: A Clinical Study

Neuropsychological assessment needs a more profound grounding in psychometric theory. Specifically, psychometrically reliable and valid tools are required, both in patient care and in scientific research. The present study examined convergent and discriminant validity of some of the most popular indicators of executive functioning (EF). A sample of 96 neurological inpatients (aged 18-68 years) completed a battery of standardized cognitive tests (Raven's matrices, vocabulary test, Wisconsin Card Sorting Test, verbal fluency test, figural fluency test). Convergent validity of indicators of intelligence (Raven's matrices, vocabulary test) and of indicators of EF (Wisconsin Card Sorting Test, verbal fluency test, figural fluency) were calculated. Discriminant validity of indicators of EF against indicators of intelligence was also calculated. Convergent validity of indicators of intelligence (Raven's matrices, vocabulary test) was good (r_xtyt = 0.727; R² = 0.53). Convergent validity of fluency indicators of EF against executive cognition as indicated by performance on the Wisconsin Card Sorting Test was poor (0.087 ≤ r_xtyt ≤ 0.304; 0.008 ≤ R² ≤ 0.092). Discriminant validity of indicators of EF against indicators of intelligence was good (0.106 ≤ r_xtyt ≤ 0.548; 0.011 ≤ R² ≤ 0.300). Our conclusions from these data are clear-cut: apparently dissimilar indicators of intelligence converge on general intellectual ability. Apparently dissimilar indicators of EF (mental fluency, executive cognition) do not converge on general executive ability. Executive abilities, although non-unitary, can be reasonably well distinguished from intellectual ability. The present data contribute to the hitherto meager evidence base regarding the validity of popular indicators of EF.

Assessing Cognitive Abilities Using the WAIS-IV: An Item Response Theory Approach

Background: The Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) has been adapted to 28 different cultures and there has been considerable interest in examining its structure through exploratory and confirmatory factor analysis. This study investigates item and scale properties of the Egyptian WAIS-IV using item response theory (IRT) models. Methods: The sample consisted of 250 adults from Egypt. The item-level and subtest statistical properties of the Egyptian WAIS-IV were established using a combination of four dichotomous IRT models and four polytomous IRT models. In addition, factor analysis was performed to investigate the dimensionality of each subtest. Results: Factor analysis indicated the unidimensionality of each subtest. Among IRT models, the two-parameter logistic model provided a good fit for dichotomous subtests, while the graded response model fitted the polytomous data. Most items of the Egyptian WAIS-IV showed high discrimination, and the scale was adequately informative across the levels of latent traits (i.e., cognitive variables). However, each subtest included at least some items with limited ability to distinguish between individuals with differing levels of the cognitive variable being measured. Furthermore, most subtests have items that do not follow the difficulty rank they are ascribed in the WAIS-IV manual. Conclusions: Overall, the results suggest that the Egyptian WAIS-IV offers a highly valid assessment of intellectual abilities, despite the need for some improvements.

Frequency of occurrence of four- and five-factor WAIS-IV profiles

Interpretive strategies for the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) include Wechsler's four-factor structure and the five-factor Cattell-Horn-Carroll (CHC) model. The frequency of profile occurrence and the contribution of demographic- and ability-related variables to their incidence are unknown. Current participants were 291 referrals (males = 134; female = 157) for neuropsychological evaluation with mean years for age and education of 34.94 (SD = 13.53) and 12.74 (SD = 2.46), respectively. Lichtenberger and Kaufman's guidelines for selecting each model were applied. Of the total, 67.3% were four-factor and 32.6% were five-factor profiles. The same pattern emerged when participants were subdivided by gender, education, ethnicity, IQ, and diagnosis. A noteworthy association between IQ and profile type emerged. When IQ increased, four-factor profiles declined and five-factors increased. A logistic regression, using demographics, IQ, and diagnosis as predictors, correctly classified 64.8% of participants. The average subtest intercorrelations and g saturations in the four-factor group were substantially larger than those for five-factor participants. These findings were consistent with Spearman's differentiation by ability hypothesis.

Executive functions and intelligence from the CHC theory perspective: Investigating the correspondence between the WAIS-IV and the NAB Executive Functions Module

Substantial overlap between executive functions (EFs) and intelligence has been reported. Yet further research is required with respect to how the specific components of the two constructs are interrelated. The present study was aimed at exploring the relationship between EFs and intelligence by examining the latent structure of the Wechsler Adult Intelligence Scale (WAIS-IV) and the Executive Functions Module from the Neuropsychological Assessment Battery (NAB). The aim was also to test the correspondence of the two measures to the Cattell-Horn-Carroll (CHC) theory. 205 healthy participants, aged 18-89 years, were administered the NAB and WAIS-IV. Confirmatory factor analysis (CFA) was employed to explore the latent constructs underlying the two test batteries. Models based on the current WAIS-IV index structure showed worse model fit than models based on the CHC theory. In particular, factors representing broad CHC abilities comprehension knowledge (Gc), fluid reasoning (Gf), visual processing (Gv), processing speed (Gs), and working memory capacity (Gwm) proved model fit improvement. The best model fit, however, provided a five-factor model combining the Gf and Gwm abilities into one factor and additionally including a retrieval fluency (Gr) factor. The results demonstrate strong relationships between the WAIS-IV and the NAB Executive Functions Module and suggest substantial overlap between EFs and intelligence within the CHC framework. Nevertheless, according to the recent update of the CHC theory, the NAB fluency tasks may represent a distinct EF factor comprising Gr abilities independent from the WAIS-IV.

A Meta-Analysis of Relationships between Measures of Wisconsin Card Sorting and Intelligence

The Wisconsin Card Sorting Test (WCST) represents a widely utilized neuropsychological assessment technique for executive function. This meta-analysis examined the discriminant validity of the WCST for the assessment of mental shifting, considered as an essential subcomponent of executive functioning, against traditional psychometric intelligence tests. A systematic search was conducted, resulting in 72 neuropsychological samples for the meta-analysis of relationships between WCST scores and a variety of intelligence quotient (IQ) domains. The study revealed low to medium-sized correlations with IQ domains across all WCST scores that could be investigated. Verbal/crystallized IQ and performance/fluid IQ were indistinguishably associated with WCST scores. To conclude, the WCST assesses cognitive functions that might be partially separable from common conceptualizations of intelligence. More vigorous initiatives to validate putative indicators of executive function against intelligence are required.

Furthering the Understanding of Wechsler Adult Intelligence Scale-Fourth Edition Factor Structure in a Clinical Sample

The Wechsler Adult Intelligence Scale, Fourth Edition (WAIS-IV) is a commonly administered battery for assessing intellectual and cognitive abilities. Despite its popularity, construct validation studies primarily utilize the WAIS-IV normative sample rather than ecologically-valid clinical samples. The current study expands the literature on the validity of WAIS-IV by testing a bifactor model in such a sample. We examined archival data from 300 concurrent psychological evaluations performed at a university-based community clinic. Participants received the full WAIS-IV standard battery. Consistent with recent literature, confirmatory factor analyses (CFAs) favored a direct hierarchical model, where the g factor has a direct influence on WAIS-IV subtest performance and index scores explain only modest degrees of residual variance. Results challenge traditional intelligence nosologies and suggest consideration of a two-step method of WAIS-IV interpretation in clinical samples, whereby the Full Scale IQ score (FSIQ) score is examined first and individual subtest scores are analyzed second.

Exploring the behavioral and cognitive phenotype of KBG syndrome

KBG syndrome is a neurodevelopmental disorder, caused by dominant mutations in ANKRD11, that is characterized by developmental delay/intellectual disability, mild craniofacial dysmorphisms, and short stature. Behavior and cognition have hardly been studied, but anecdotal evidence suggests higher frequencies of ADHD‐symptoms and social‐emotional impairments. In this study, the behavioral and cognitive profile of KBG syndrome will be investigated in order to examine if and how cognitive deficits contribute to behavioral difficulties. A total of 18 patients with KBG syndrome and a control group consisting of 17 patients with other genetic disorders with comparable intelligence levels, completed neuropsychological assessment. Age‐appropriate tasks were selected, covering overall intelligence, attention, memory, executive functioning, social cognition and visuoconstruction. Results were compared using Cohen's d effect sizes. As to behavior, fewer difficulties in social functioning and slightly more attentional problems, hyperactivity, oppositional defiant behavior and conduct problems were found in the KBG syndrome group. Regarding cognitive functioning, inspection of the observed differences shows that patients with KBG syndrome showed lower scores on sustained attention, cognitive flexibility, and visuoconstruction. In contrast, the KBG syndrome group demonstrated higher scores on visual memory, social cognition and emotion recognition. The cognitive profile of KBG syndrome in this sample indicates problems in attention and executive functioning that may underlie the behavior profile which primarily comprises impulsive behavior. Contrary to expectations based on previous (case) reports, no deficits were found in social cognitive functioning. These findings are important for counseling purposes, for tailored education planning, and for the development of personalized intervention.

Inhibition, shifting and updating in relation to psychometric intelligence across ability groups in the psychiatric population

BACKGROUND

Assessment of intelligence and executive function (EF) is common in complex neuropsychiatric practice. Although previous studies have shown that EF and intelligence are related, it is unknown whether these constructs relate to one another in a similar manner across different ability groups (mild intellectual disability, borderline intellectual disability and normal/high intelligence). This study therefore examines the relation between three EFs (inhibition, shifting and updating) and intelligence in a heterogeneous psychiatric sample. It is hypothesised that the strength of the relation between intelligence and the three EFs decreases when the level of intelligence increases, in accordance with Spearman's Law of Diminishing Returns.

METHODS

In a cross-sectional, between and within subject design, one of the three intelligence tests (Kaufman Adolescent and Adult Intelligence Test and Wechsler Adult Intelligence Scale - third and fourth editions) and several EF tests (Stroop Colour-Word Test, Trail Making Test and Spatial Working Memory task) were administered to 250 neuropsychiatric inpatients and outpatients (Mage  = 39.8, standard deviation = 14.3, 52.8% male). Based upon their full-scale IQ score, patients were divided into three ability groups (mild intellectual disability, borderline intellectual disability or normal/high intelligence). The relation between EF and intelligence was assessed through analyses of the correlation pattern; groups were compared using analysis of covariance.

RESULTS

Analyses showed significant correlations between the constructs of EF and intelligence. A significant interaction effect was found for shifting, with highest correlations in the normal to high intelligence group, but not for inhibition and updating.

CONCLUSIONS

Results support a specific role for shifting in this EF-intelligence relation. The correlational pattern of updating and intelligence, as well as the differential relation of shifting and intelligence across ability groups, suggests that EF tasks may not measure distinct EFs in lower intellectual ability but rely on cognitive primitives such as processing speed. EF tasks can be considered less valid indicators of EF ability. Implications in terms of the need for development of specific tasks to measure cognition in low intellectual ability are discussed.

Intellectual Profiles in KBG-Syndrome: A Wechsler Based Case-Control Study

KBG syndrome is a neurodevelopmental disorder (NDD) caused by loss-of-function of the ANKRD11 gene. The core phenotype comprises developmental delay (DD)/ intellectual disability (ID) and several specific facial dysmorphisms. In addition, both ADHD- and ASD-related symptoms have been mentioned. For the correct understanding of these developmental and behavioral characteristics however, it is of great importance to apply objective measures, which seldom has been done in patients with KBG syndrome. In this study, intelligence profiles of patients with KBG syndrome (n = 18) were compared with a control group comprising patients with NDD caused by various other genetic defects (n = 17), by means of the Wechsler scales. These scales were also used to measure speed of information processing, working memory, verbal comprehension and perceptual reasoning. No significant differences were found in the global level of intelligence of patients with KBG syndrome as compared to the patient genetic control group. The same was true for Wechsler subtest results. Hence, behavioral problems associated with KBG syndrome cannot directly be related to or explained by a specific intelligence profile. Instead, specific assessment of neurocognitive functions should be performed to clarify the putative behavioral problems as observed in this syndrome.

Hierarchical Structure of the Wechsler Adult Intelligence Scale–Fourth Edition with an Egyptian Sample

The Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV) is one of the most well-known tests in the field of adult intelligence assessment. This study explores the validity of the Egyptian adaptation for the subscales of the WAIS-IV. Confirmatory factor analysis (CFA) indicated that the first-order, second-order, and bifactor models of both the five-factor Cattel–Horn–Carroll (CHC) and the four-factor structures fitted with the WAIS-IV. When the Arithmetic subscale was pathed to Perceptual Reasoning and Working Memory, the modified four first-order factor showed a better fit than any other model. Estimates of internal consistency revealed that Cronbach’s alpha was very high (.91) for the WAIS-IV scale and for four-factor structures (ranging from .81 to .95), while [Formula: see text] was low (0.61) for the WAIS-IV scale and for four-factor structures, ranging from 0.20 to 0.68. Overall, these findings provided adding evidence about the dimensionality of WAIS-IV with a new sample.

Predictive Value of Traditional Measures of Executive Function on Broad Abilities of the Cattell–Horn–Carroll Theory of Cognitive Abilities

The neuropsychological construct of executive functions (EFs), and the psychometric Cattell–Horn–Carroll (CHC) theory of cognitive abilities are both approaches that attempt to describe cognitive functioning. The coherence between EF and CHC abilities has been mainly studied using factor-analytical techniques. Through multivariate regression analysis, the current study now assesses the integration of these latent constructs in clinical assessment. The predictive power of six widely used executive tasks on five CHC measures (crystallized and fluid intelligence, visual processing, short-term memory, and processing speed) is examined. Results indicate that executive tasks—except for the Stroop and the Tower of London—predict overall performance on the intelligence tests. Differentiation in predicting performance between the CHC abilities is limited, due to a high shared variance between these abilities. It is concluded that executive processes such as planning and inhibition have a unique variance that is not well-represented in intelligence tests. Implications for the use of EF tests and operationalization of CHC measures in clinical practice are discussed.