Do Raven's Colored Progressive Matrices function in the same way in typical and clinical populations? Insights from the intellectual disability field

A Psychometric Analysis of Raven’s Colored Progressive Matrices: Evaluating Guessing and Carelessness Using the 4PL Item Response Theory Model

The purpose of the present study was to evaluate the psychometric properties of Raven’s colored progressive matrices by estimating the presence of pseudo-guessing and pseudo-carelessness. Participants were 1127 children from ages 5 to 11. Guessing and carelessness were assessed using the lower and upper asymptotes of the 3PL and 4PL item response theory (IRT) models, respectively. Optimal model fit was judged using difference loglikelihood tests and information criteria. Results indicated that guessing, but not carelessness, were evident in the AB and B forms of the CPM, with successful guessing being more prevalent in the AB form. It is concluded that nonverbal IQ estimation in CPM should include variable estimation methods so that aptitude scores are estimated with the highest possible accuracy.

Intellectual abilities, language comprehension, speech, and motor function in children with spinal muscular atrophy type 1

Background

Spinal muscular atrophy (SMA) is a chronic, neuromuscular disease characterized by degeneration of spinal cord motor neurons, resulting in progressive muscular atrophy and weakness. SMA1 is the most severe form characterized by significant bulbar, respiratory, and motor dysfunction. SMA1 prevents children from speaking a clearly understandable and fluent language, with their communication being mainly characterized by eye movements, guttural sounds, and anarthria (type 1a); severe dysarthria (type 1b); and nasal voice and dyslalia (type 1c).

The aim of this study was to analyze for the first time cognitive functions, language comprehension, and speech in natural history SMA1 children according to age and subtypes, to develop cognitive and language benchmarks that provide outcomes for the clinical medication trials that are changing SMA1 course/trajectory.

Methods

This is a retrospective study including 22 children with SMA1 (10 affected by subtype 1a-1b: AB and 12 by 1c: C) aged 3–11 years in clinical stable condition with a coded way to communicate “yes” and “no”. Data from the following assessments have been retrieved from patient charts: one-dimensional Raven test (RCPM), to evaluate cognitive development (IQ); ALS Severity Score (ALSSS) to evaluate speech disturbances; Brown Bellugy modified for Italian standards (TCGB) to evaluate language comprehension; and Children’s Hospital of Philadelphia Infant Test of Neuromuscular Disorders (CHOP-INTEND) to assess motor functioning.

Results

SMA 1AB and 1C children were similar in age, with the former characterized by lower CHOP-INTEND scores compared to the latter. All 22 children had collaborated to RCPM and their median IQ was 120 with no difference (p = 0.945) between AB and C. Global median score of the speech domain of the ALSSS was 5; however, it was 2 in AB children, being significantly lower than C (6.5, p < 0.001).

TCGB test had been completed by 13 children, with morphosyntactic comprehension being in the normal range (50). Although ALSSS did not correlate with both IQ and TCGB, it had a strong (p < 0.001) correlation with CHOP-INTEND described by an exponential rise to maximum.

Conclusions

Although speech and motor function were severely compromised, children with SMA1 showed general intelligence and language comprehension in the normal range. Speech impairment was strictly related to global motor impairment.

On the Developmental Trajectories of Relational Concepts Among Children and Adolescents With Intellectual Disability of Undifferentiated Etiology

The aim of this study was to examine the developmental trajectories of comprehension of relational concepts among 557 participants with intellectual disability (ID) of undifferentiated etiology (M age  =  12.20 years, SD  =  3.18) and 557 typically developing (TD) participants (M age  =  4.57 years, SD  =  0.80). Logistic regression analyses, with nonverbal cognitive level entered first in the equations, showed only negligible differences with regard to the discriminative power of each of the 72 concepts used as outcome variables, and moderate differences in difficulty for only three items. A moderate mixed effect (i.e., combining a group difference in difficulty and discriminative power) was observed for a fourth item. It is concluded that the developmental trajectories of relational concepts are similar for participants with or without ID. The implications and limitations of the study are discussed.

Measurement Invariance and Psychometric Analysis of Oxford Happiness Inventory Scale across Gender and Marital Status

Background

The Oxford Happiness Inventory (OHI) is a self-report tool to measure happiness. A brief review of previous studies on OHI showed the lack of evaluation of OHI fairness/equivalence in measuring happiness among identified groups.

Methods

To examine the psychometric properties and measurement invariance of the OHI, responses of 500 university students were analyzed using item response theory and ordinal logistic regression (OLR). Relevant measures of effect size were utilized to interpret the results. Differential test functioning was also evaluated to determine whether there is an overall bias at the test level.

Results

OLR analysis detected four items across gender and two items across marital status to function differentially. An assessment of effect sizes implied negligible differences for practical considerations.

Conclusions

This study was a significant step towards providing theoretical and practical information regarding the assessment of happiness by presenting adequate evidence regarding the psychometric properties of OHI.

Differential Item Functioning on Raven’s SPM+ Amongst Two Convenience Samples of Yakuts and Russians

National IQ estimates are based on psychometric measurements carried out in a variety of cultural contexts and are often obtained from Raven’s Progressive Matrices tests. In a series of studies, J. Philippe Rushton et al. have argued that these tests are not biased with respect to ethnicity or race. Critics claimed their methods were inappropriate and suggested differential item functioning (DIF) analysis as a more suitable alternative. In the present study, we conduct a DIF analysis on Raven’s Standard Progressive Matrices Plus (SPM+) tests administered to convenience samples of Yakuts and ethnic Russians. The Yakuts scored lower than the Russians by 4.8 IQ points, a difference that can be attributed to the selectiveness of the Russian sample. Data from the Yakut (n = 518) and Russian (n = 956) samples were analyzed for DIF using logistic regression. Although items B9, B10, B11, B12, and C11 were identified as having uniform DIF, all of these DIF effects can be regarded as negligible (R2 <0.13). This is consistent with Rushton et al.’s arguments that the Raven’s Progressive Matrices tests are ethnically unbiased.

Measurement invariance and psychometric analysis of Kutcher Adolescent Depression Scale across gender and marital status

BACKGROUND

The Kutcher Adolescent Depression Scale (KADS) has been studied across Canada and some other countries during the past almost 15 years. The scale is a self-report tool to diagnose and monitor clinical depression in adolescents. A brief review of previous studies on KADS showed the lack of evaluation of KADS fairness/equivalence in measuring depression among identified groups.

METHODS

To examine the psychometric properties and measurement invariance of the KADS, responses of 407 participants were analyzed using item response theory (IRT) and ordinal logistic regression (OLR). Relevant measures of effect size were utilized to interpret the results.

RESULTS

Findings of the parallel factor analysis confirmed unidimensionality of the KADS and the partial credit IRT model found to be the best fitting model for analyzing the scale. OLR analysis detected three items across gender and one item across marital status to function differentially. An assessment of effect sizes implied negligible differences for practical considerations.

LIMITATIONS

A note of caution is necessary with respect to interpreting results of measurement invariance across Gender. The sample analyzed in this study was predominantly female and this might have affected our findings. A similar analysis with a more balanced sample is recommended.

CONCLUSIONS

This study was a significant step towards providing theoretical and practical information regarding the assessment of depression among adolescents by presenting adequate evidence regarding the psychometric properties of KADS-11. Future studies may look at different methods for assessing invariance and different groups for strengthening conclusions with respect to the KADS.

Does the Development of Syntax Comprehension Show a Premature Asymptote Among Persons With Down Syndrome? A Cross-Sectional Analysis

Uncertainty persists regarding the post-childhood trajectory of syntactic acquisition of persons with Down syndrome (DS). In some studies, asymptote is reached in the early teens, whereas others find syntax continuing to develop at least into early adulthood. This study addressed the issue using a cross-sectional approach. Receptive syntax and vocabulary were tested in 62 children, adolescents and young adults with DS matched on chronological age and cognitive level with 62 participants with intellectual disability (ID) of undifferentiated etiology. On both tests there were significant effects of chronological age and diagnosis, but the chronological age × diagnosis interactions were nonsignificant. We concluded that comprehension of vocabulary and syntax does not asymptote prematurely in individuals with DS relative to those with other forms of ID.

Emotion Recognition as a Real Strength in Williams Syndrome: Evidence From a Dynamic Non-verbal Task

The hypersocial profile characterizing individuals with Williams syndrome (WS), and particularly their attraction to human faces and their desire to form relationships with other people, could favor the development of their emotion recognition capacities. This study seeks to better understand the development of emotion recognition capacities in WS. The ability to recognize six emotions was assessed in 15 participants with WS. Their performance was compared to that of 15 participants with Down syndrome (DS) and 15 typically developing (TD) children of the same non-verbal developmental age, as assessed with Raven’s Colored Progressive Matrices (RCPM; Raven et al., 1998). The analysis of the three groups’ results revealed that the participants with WS performed better than the participants with DS and also than the TD children. Individuals with WS performed at a similar level to TD participants in terms of recognizing different types of emotions. The study of development trajectories confirmed that the participants with WS presented the same development profile as the TD participants. These results seem to indicate that the recognition of emotional facial expressions constitutes a real strength in people with WS.

Psychometric Investigation of the Raven's Colored Progressive Matrices Test in a Sample of Preschool Children

The present study investigated the psychometric properties of the Raven's Colored Progressive Matrices (CPM) test in a sample of preschoolers from Brazil (n = 582; age: mean = 57 months, SD = 7 months; 46% female). We investigated the plausibility of unidimensionality of the items (confirmatory factor analysis) and differential item functioning (DIF) for sex and age (multiple indicators multiple causes method). We tested four unidimensional models and the one with the best-fit index was a reduced form of the Raven's CPM. The DIF analysis was carried out with the reduced form of the test. A few items presented DIF (two for sex and one for age), confirming that the Raven's CPM items are mostly measurement invariant. There was no effect of sex on the general factor, but increasing age was associated with higher values of the g factor. Future research should indicate if the reduced form is suitable for evaluating the general ability of preschoolers.

Measuring intellectual ability in cerebral palsy: The comparison of three tests and their neuroimaging correlates

Standard intelligence scales require both verbal and manipulative responses, making it difficult to use in cerebral palsy and leading to underestimate their actual performance. This study aims to compare three intelligence tests suitable for the heterogeneity of cerebral palsy in order to identify which one(s) could be more appropriate to use. Forty-four subjects with bilateral dyskinetic cerebral palsy (26 male, mean age 23 years) conducted the Raven's Coloured Progressive Matrices (RCPM), the Peabody Picture Vocabulary Test-3rd (PPVT-III) and the Wechsler Nonverbal Scale of Ability (WNV). Furthermore, a comprehensive neuropsychological battery and magnetic resonance imaging were assessed. The results show that PPVT-III gives limited information on cognitive performance and brain correlates, getting lower intelligence quotient scores. The WNV provides similar outcomes as RCPM, but cases with severe motor impairment were unable to perform it. Finally, the RCPM gives more comprehensive information on cognitive performance, comprising not only visual but also verbal functions. It is also sensitive to the structural state of the brain, being related to basal ganglia, thalamus and white matter areas such as superior longitudinal fasciculus. So, the RCPM may be considered a standardized easy-to-administer tool with great potential in both clinical and research fields of bilateral cerebral palsy.

Eye-tracking controlled cognitive function tests in patients with amyotrophic lateral sclerosis: a controlled proof-of-principle study

Amyotrophic lateral sclerosis (ALS) primarily affects motor and speech abilities. In addition, cognitive functions are impaired in a subset of patients. There is a need to establish an eye movement-based method of neuropsychological assessment suitable for severely physically impaired patients with ALS. Forty-eight ALS patients and thirty-two healthy controls matched for age, sex and education performed a hand and speech motor-free version of the Raven's coloured progressive matrices (CPM) and the D2-test which had been especially adapted for eye-tracking control. Data were compared to a classical motor-dependent paper-pencil version. The association of parameters of the eye-tracking and the paper-pencil version of the tests and the differences between and within groups were studied. Subjects presented similar results in the eye-tracking and the corresponding paper-pencil versions of the CPM and D2-test: a correlation between performance accuracy for the CPM was observed for ALS patients (p < 0.001) and controls (p < 0.001) and in the D2-test for controls (p = 0.048), whereas this correlation did not reach statistical significance for ALS patients (p = 0.096). ALS patients performed worse in the CPM than controls in the eye-tracking (p = 0.053) and the paper-pencil version (p = 0.042). Most importantly, eye-tracking versions of the CPM (p < 0.001) and the D2-test (p = 0.024) reliably distinguished between more and less cognitively impaired patients. Eye-tracking-based neuropsychological testing is a promising approach for assessing cognitive deficits in patients who are unable to speak or write such as patients with severe ALS.

A qualitative analysis of general receptive vocabulary of adolescents with Down syndrome

This study aimed to discover whether general receptive vocabulary is qualitatively phenotypical in Down syndrome. Sixty-two participants with Down syndrome (M age=16.74 years, SD=3.28) were individually matched on general vocabulary raw total score with 62 participants with intellectual disability of undifferentiated etiology (M age=16.20 years, SD=3.08) and 62 typical children (M age=5.32 years, SD=0.82). Item analyses using the transformed item difficulties method to detect differential item functioning across groups showed that the groups' rank orders of item difficulty were highly similar. It was concluded that the general receptive vocabulary of older children and adolescents with Down syndrome is not qualitatively distinguished when its overall size is held constant. Methodological and theoretical implications of this finding are discussed.

On the difficulty of relational concepts among participants with Down syndrome

The aim of the study was to compare the difficulty of relational concepts among participants with and without intellectual disability. The French versions of the Boehm Tests of Basic Concepts Third Edition (Preschool and Kindergarten to 2nd grade) were administered to three groups of 47 participants individually matched on their total raw score on the tests. The first group comprised participants with intellectual disability of undifferentiated etiology, the second, participants with Down syndrome and the third, typical children. Item analyses using the transformed item difficulties method to detect differential item functioning across groups showed that the groups' rank-orders of item difficulty were highly similar. It is concluded that, all things being equal, relational concepts are of comparable difficulty and follow a similar sequence of development whatever the cognitive and etiological status of participants. Methodological and theoretical implications of these findings are discussed.

Beyond matching on the mean in developmental disabilities research

The matching of groups is a traditional way to control for confounding variables in developmental disabilities research. The equivalency of means across groups is routinely checked for these variables, but not the homogeneity of their variances or the shapes of their distributions. In the present paper, it is argued that group matching can go seriously wrong unless it directly confronts the distributional concerns by the use of well-known statistical indices and very simple graphical displays of the distributions. The question of the equivalency of item response profiles is also addressed since two participants or two groups of participants can obtain the same overall score on the matching variable by passing different items. In this case, the matching cannot be considered satisfactory because of poor concordance between the molar (overall score) and molecular (item scores) levels of matching. Angoff's Delta plot method, a statistical approach for detecting differential item functioning across small groups is described. It is promising as a simple way to prove whole test/individual item correspondence and, in addition, a useful tool for making post hoc statistical analyses at the item level on the dependent variables.