Consistent g- as well as consistent verbal-, numerical- and figural-factors in nested factor models? Confirmatory factor analyses using three test batteries

Cognitive Dyadic Measurements: A Game-Changer? Construction and First Validation of Three Cognitively Demanding Competitive Tasks

Competition among individuals is a natural mode of determining who is fittest. While in nature, economics, and sports, it is common to infer ability or aptitude from the outcome of competitions, our knowledge on its effects in regard to psychological/educational assessment is scarce. In the present pilot study, we explore a measurement approach for assessing individual differences in interpersonal, face-to-face competitions, based on a set of cognitively demanding, competitive, fast-paced, two-opponent tasks. For initial task evaluation, we conducted comprehensive reliability and construct validation analyses, considering cognitive ability, motivation, and personality measures. Moreover, using structural equation models we conducted a simultaneous factorization of the tasks with the other validation measures. The results suggest that the newly developed tasks measure both cognitive ability (intelligence) as well as a competition-specific component. The competition-specific component was positively associated with experience in competitive gaming and negatively correlated with neuroticism. While the pattern of validities was promising, the measurements’ reliabilities were yet unsatisfactory. Implications for future research as well as the design of competition-based measurements are discussed.

Full-information item bifactor model for mathematical ability assessment in Chinese compulsory education quality monitoring

This study focuses on the measurement of mathematical ability in the Chinese Compulsory Education Qualification Monitoring (CCEQM) framework using bifactor theory. First, we propose a full-information item bifactor (FIBF) model for the measurement of mathematical ability. Second, the performance of the FIBF model is empirically studied using a data set from three representative provinces were selected from CCEQM 2015-2017. Finally, Monte Carlo simulations are conducted to demonstrate the accuracy of the model evaluation indices and parameter estimation methods used in the empirical study. The obtained results are as follows: (1) The results for the four used model selection indices (AIC, SABIC, HQ, BIC) consistently showed that the fit of the FIBF model is better than that of the UIRT; (2) All of the estimated general and domain-specific abilities of the FIBF model have reasonable interpretations; (3) The model evaluation indices and parameter estimation methods exhibit excellent accuracy, indicating that the application of the FIBF model is technically feasible in large-scale testing projects.

Intelligenzdiagnostik der Zukunft

Zusammenfassung. Die menschliche Intelligenz ist eines der am besten erforschten und validierten Konstrukte innerhalb der Psychologie. Dennoch wird die Validität von Intelligenztests im gruppen- und insbesondere kulturvergleichenden Kontext regelmäßig und berechtigterweise kritisch hinterfragt. Obwohl verschiedene Alternativen und Weiterentwicklungen der Intelligenzdiagnostik vorgeschlagen wurden (z. B. kulturfaire Tests), sind fundamentale Probleme in der vergleichenden Intelligenzdiagnostik noch immer ungelöst und die Validitäten entsprechender Verfahren unklar. In dem vorliegenden Positionspapier wird diese Thematik aus der Perspektive der Kognitionspsychologie und der kognitiven Neurowissenschaften beleuchtet und eine prozessorientierte und biologisch inspirierte Form der Intelligenzdiagnostik als potentieller Lösungsansatz vorgeschlagen. Wir zeigen die Bedeutung elementarer kognitiver Prozesse auf (insbesondere Arbeitsgedächtniskapazität, Aufmerksamkeit, Verarbeitungsgeschwindigkeit), die individuellen Leistungsunterschieden zu Grunde liegen, und betonen, dass der Unterscheidung zwischen Inhalten und Prozessen eine zentrale, jedoch oft vernachlässigte Rolle in der Diagnostik allgemeiner kognitiver Leistungsunterschiede zukommt. Während aus kognitions- und neuropsychologischer Sicht davon ausgegangen werden kann, dass sich insbesondere Prozesse für interkulturelle Vergleiche eignen, sollten Inhalte als stärker kulturspezifisch verstanden werden. Darauf aufbauend diskutieren wir drei verschiedene Ansätze zur Verbesserung interkultureller Vergleichbarkeit der Intelligenzdiagnostik sowie deren Grenzen. Wir postulieren, dass sich die Intelligenzforschung im Austausch mit verschiedenen Disziplinen stärker auf die Identifikation von generellen kognitiven Prozessen fokussieren sollte und diskutieren das Potenzial zukünftiger Forschung hin zu einer prozessorientierten und biologisch inspirierten Intelligenzdiagnostik. Schließlich zeigen wir derzeitige Möglichkeiten auf, gehen aber auch auf etwaige Herausforderungen ein und beleuchten Implikationen für die zukünftige Intelligenzdiagnostik und -forschung.

The Relation of Personality and Intelligence-What Can the Brunswik Symmetry Principle Tell Us?

Personality and intelligence are defined as hierarchical constructs, ranging from broad g-factors to (domain-)specific constructs. The present study investigated whether different combinations of hierarchical levels lead to different personality-intelligence correlations. Based on the integrative data analysis approach, we combined a total of five data sets. The focus of the first study (N = 682) was an elaborated measurement of personality (NEO-PI-R), which was applied with a relatively short intelligence test (Intelligence Structure Test 2000 R). In the second study (N = 413), a comprehensive measurement of intelligence (Berlin Intelligence Structure test) was used with a shorter personality questionnaire (NEO-FFI). In line with the Brunswik symmetry principle, the findings emphasize that personality-intelligence correlations varied greatly across the hierarchical levels of constructs considered in the analysis. On average, Openness showed the largest relation with intelligence. We recommend for future studies to investigate personality-intelligence relations at more fine-grained levels based on elaborated measurements of both personality and intelligence.

Evaluation of multidimensional models of WAIS-IV subtest performance

OBJECTIVE

The present study examined the extent to which the covariance structure of the WAIS-IV is best accounted for by models that assume that test performance is the result of group-level factors and multiple independent general factors.

METHOD

Structural models with one to four general factors were evaluated with either four or five group-level factors. Simulations based on four general factors were run to clarify the adequacy of the estimates of the allocation of covariance by the models.

RESULTS

Four independent general factors provided better fit than a single general factor for either model with four or five group-level factors. While one of the general factors had much larger loadings than all other factors, simulation results suggested that this might be an artifact of the statistical procedure rather than a reflection of the nature of individual differences in cognitive abilities.

CONCLUSIONS

These results argue against the contention that clinical interpretation of cognitive test batteries should primarily be at the level of general intelligence. It is a fallacy to assume that factor analysis can reveal the structure of human abilities. Test validity should not be based solely on the results of modeling the covariance of test batteries.

Modeling General and Specific Abilities

The present study examined issues related to structural modeling of abilities by the use of simulated data as well as analysis of the standardization data from the Woodcock–Johnson-III. In both cases, results were evaluated with cross-validation. Simulation results showed that cross-validation with an independent data set was more successful in identifying the model that was used to generate test scores than were several fit indices. Analysis of the Woodcock–Johnson-III standardization data with cross-validation showed that bifactor models provided better fit than hierarchical or correlated factor models. This was true considering both fit indices and cross-validation. General and specific factors shared a considerable amount of variance as evaluated by using the bifactor models to partition variance. The results of the present study suggest that there is a certain degree of ambiguity in determining the exact amount of covariance in test performance accounted for by general and specific factors. This calls in to question the practice of adjusting or controlling for general abilities when evaluating measures of specific abilities. Evidence for the validity of a construct should not be limited to factor analysis of tests purported to measure that construct.

Zusammenhänge allgemeiner und spezifischer Intelligenzfaktoren mit allgemeinen und spezifischen Schulleistungen im Nested-Factor-Modell

Zur Klärung des Zusammenhangsgefüges von allgemeinen sowie spezifischen Intelligenzfaktoren und Schulleistungen (Zensuren) wurden zwei Intelligenztestbatterien (KFT 4-12+R – Kurzform, BIS-4 – Kurzform) N = 562 Neuntklässlern (Gymnasium) administriert und die Zeugniszensuren in sechs Fächern erhoben. Im kombinierten Nested-Factor-Modell wurden die Zusammenhänge eines allgemeinen (g) sowie spezifischer Intelligenzfaktoren (verbal, numerisch, figural) mit einem allgemeinen Schulleistungsfaktor sowie spezifischen (sprachlich, mathematisch/naturwissenschaftlich) Schulleistungsfaktoren spezifiziert. Der allgemeine Schulleistungsfaktor hing mit dem Intelligenz-g-Faktor substanziell zusammen (r = .44). Zusätzlich zeigten sich zwischen den spezifischen verbalen und numerischen Intelligenzfaktoren mit den spezifischen Schulleistungsfaktoren konvergente (r ≈ .50) und divergente (|r| ≤ .18) Validitätshinweise. Der spezifische figurale Intelligenzfaktor hing sowohl mit der sprachlichen als auch der mathematisch/naturwissenschaftlichen Schulleistung zusammen (r ≈ .30). Die Ergebnisse werden hinsichtlich der Relevanz einer systematischen Betrachtung der Zusammenhänge zwischen der allgemeinen Intelligenz und spezifischen Intelligenzfaktoren mit allgemeinen und spezifischen Schulleistungen diskutiert.