Specific Aptitude Theory Revisited: Is There Incremental Validity for Training Performance?

Relative Contributions of g and Basic Domain-Specific Mathematics Skills to Complex Mathematics Competencies

Meta-analytic structural equation modeling was used to estimate the relative contributions of general cognitive ability or g (defined by executive functions, short-term memory, and intelligence) and basic domain-specific mathematical abilities to performance in more complex mathematics domains. The domain-specific abilities included mathematics fluency (e.g., speed of retrieving basic facts), computational skills (i.e., accuracy at solving multi-step arithmetic, algebra, or geometry problems), and word problems (i.e., mathematics problems presented in narrative form). The core analysis included 448 independent samples and 431,344 participants and revealed that g predicted performance in all three mathematics domains. Mathematics fluency contributed to the prediction of computational skills, and both mathematics fluency and computational skills predicted word problem performance, controlling g. The relative contribution of g was consistently larger than basic domain-specific abilities, although the latter may be underestimated. The patterns were similar across younger and older individuals, individuals with and without a disability (e.g., learning disability), concurrent and longitudinal assessments, and family socioeconomic status, and have implications for fostering mathematical development.

Predicting academic success with the Big 5 rated from different points of view: Self‐rated, other rated and faked

Self‐ratings of personality predict academic success above general intelligence. The present study replicated these findings and investigated the increment of other‐ratings or intentionally distorted self‐ratings. Participants (N = 145) had to compile a personality questionnaire twice. First they were given neutral instructions. The second time they were asked to imagine a specific applicant setting. Furthermore, two peers rated each participant. Additionally, verbal, numerical and figural reasoning scores were obtained. Grades on a statistics exam obtained 2 months later served as the criterion. Results replicated prior findings and showed incremental validity for self‐ and other‐rated personality, which was stable after controlling for intelligence. Faking had no impact on the domain‐score level, but results on the facet‐score level were less encouraging. Copyright © 2010 John Wiley & Sons, Ltd.

Conceptual Model for Predicting Academic Success in Prelicensure Nursing Programs Through Expanded Cognitive Aptitude Assessment

BACKGROUND

Attrition from prelicensure nursing programs is a serious issue with implications for students, nursing programs, and the nursing workforce. Academic failure due to insufficient cognitive aptitude often contributes to this problematic attrition rate. Thus, cognitive aptitude could be included in admission criteria, as a predictor of academic success and a strategy to minimize attrition.

PROBLEM

Currently, admissions practices are often incomplete and lack standardization for measuring cognitive aptitude.

APPROACH

The Nursing Cognitive Aptitude Model (NCAM) presented in this article is an innovative conceptual model to guide educators in expanding current admissions processes.

CONCLUSION

The NCAM model depicts the cognitive domains involved in student academic success including current scholastic knowledge, previous academic performance, and critical-thinking ability. Together, these three domains predict student academic success and are a useful reference for the admissions process.

Non-g Factors Predict Educational and Occupational Criteria: More than g

In a prior issue of the Journal of Intelligence, I argued that the most important scientific issue in intelligence research was to identify specific abilities with validity beyond g (i.e., variance common to mental tests) (Coyle, T.R. Predictive validity of non-g residuals of tests: More than g. Journal of Intelligence 2014, 2, 21–25.). In this Special Issue, I review my research on specific abilities related to non-g factors. The non-g factors include specific math and verbal abilities based on standardized tests (SAT, ACT, PSAT, Armed Services Vocational Aptitude Battery). I focus on two non-g factors: (a) non-g residuals, obtained after removing g from tests, and (b) ability tilt, defined as within-subject differences between math and verbal scores, yielding math tilt (math > verbal) and verbal tilt (verbal > math). In general, math residuals and tilt positively predict STEM criteria (college majors, jobs, GPAs) and negatively predict humanities criteria, whereas verbal residuals and tilt show the opposite pattern. The paper concludes with suggestions for future research, with a focus on theories of non-g factors (e.g., investment theories, Spearman’s Law of Diminishing Returns, Cognitive Differentiation-Integration Effort Model) and a magnification model of non-g factors.

Specific Abilities in the Workplace: More Important Than g?

A frequently reported finding is that general mental ability (GMA) is the best single psychological predictor of job performance. Furthermore, specific abilities often add little incremental validity beyond GMA, suggesting that they are not useful for predicting job performance criteria once general intelligence is accounted for. We review these findings and their historical background, along with different approaches to studying the relative influence of g and narrower abilities. Then, we discuss several recent studies that used relative importance analysis to study this relative influence and that found that specific abilities are equally good, and sometimes better, predictors of work performance than GMA. We conclude by discussing the implications of these findings and sketching future areas for research.

What Can We Learn from "Not Much More than g"?

A series of papers showing that measures of general cognitive ability predicted performance on the job and in training and that measures of specific cognitive abilities rarely made an incremental contribution to prediction led to a premature decline in research on the roles of specific abilities in the workplace. Lessons learned from this research include the importance of choosing the right general cognitive measures and variables, the relative roles of prediction vs. understanding and the need for a wide range of criteria when evaluating the contribution of specific skills such as complex problem solving. In particular, research published since the “not much more than g” era suggests that distinguishing between fluid and crystallized intelligence is important for understanding the development and the contribution of complex problem solving.

Secondary education as a predictor of aptitude: Implications for selection in the automotive sector

Orientation: Details of applicants’ secondary education (incorporating subject choice) could be a useful screening tool when processing large applicant pools. Here, the relationships between secondary education (incorporating subject choice) and the reasoning and visual perceptual speed components of the Differential Aptitude Test are explored.Research purpose: The objective of the study was to determine whether type of secondary education (incorporating subject choice) could be used as a substitute for reasoning (verbal and non-verbal) and/or visual perceptual speed aptitudes in the selection of operators for an automotive plant in South Africa.Motivation for the study: The motivation for this study arose from the evident gap in academic literature as well as the selection needs of the automotive industry.Research design, approach and method: This research adopted a quantitative approach. It involved a non-probability convenience quota sample of 2463 work-seeking applicants for an automotive operator position in South Africa. Participants completed a biographical questionnaire and three subtests from the Differential Aptitude Test battery. The Chi-square test was used to determine the relationship between type of secondary education (incorporating subject choice) and selected cognitive aptitudes.Main findings: The study’s findings revealed statistically and practically significant relationships between type of secondary education (incorporating subject choice), verbal reasoning, non-verbal reasoning and visual perceptual speed. Broad performance levels in the three aptitude subtests employed in this study were significantly associated with the type of matriculation certificate held by applicants. The findings specifically indicated that the secondary education types that included the subjects mathematics or both mathematics and science were associated with higher levels of performance in the three aptitudes. This had consequences for these applicants’ success in the screening process which could lead to enhanced chances of employability.Practical and managerial implications: Applicants’ type of secondary education (incorporating subject choice) could be regarded as a key criterion in human resource selection and be instructive in the screening process. This could reduce the candidate pool prior to more costly psychometric assessments.Contribution or value-add: The findings are specifically relevant to the South African automotive industry in terms of their human resource selection practices. The insights gained from the findings may also be used as a guide to human resource practitioners in the selection of similar level employees in other working contexts. The study makes a case for a multiple-hurdle approach to selection.

Pervasiveness of Dominant General Factors in Organizational Measurement

General factors are found in the measurement of many human traits. The concept of dominant general factors (DGFs) is introduced to represent the magnitude of general factors within numerous content domains. DGFs are defined as coming from the largest sources of reliable variance and influencing every variable measuring the construct. Although these factors are most frequently found in measures of cognitive ability, they are not limited to cognitive abilities. Examples are provided for a variety of construct and content domains along with estimates of their DGF percentages, ranging from 38% to 92%. Several reasons for these results are offered, and a call for concerted research is made. Research that ignores DGFs by treating specific factors or constructs within a domain as if they were distinct and uncorrelated can lead to errors in interpretation.

Validation Is Like Motor Oil: Synthetic Is Better

Although synthetic validation has long been suggested as a practical and defensible approach to establishing validity evidence, synthetic validation techniques are infrequently used and not well understood by the practitioners and researchers they could most benefit. Therefore, we describe the assumptions, origins, and methods for establishing validity evidence of the two primary types of synthetic validation techniques: (a) job component validity and (b) job requirements matrix. We then present the case for synthetic validation as the best approach for many situations and address the potential limitations of synthetic validation. We conclude by proposing the development of a comprehensive database to build prediction equations for use in synthetic validation of jobs across the U.S. economy and reviewing potential obstacles to the creation of such a database. We maintain that synthetic validation is a practically useful methodology that has great potential to advance the science and practice of industrial and organizational psychology.

Content Validation Is Useful for Many Things, but Validity Isn't One of Them

Content-oriented validation strategies establish the validity of selection tests as predictors of performance by comparing the content of the tests with the content of the job. These comparisons turn out to have little if any bearing on the predictive validity of selection tests. There is little empirical support for the hypothesis that the match between job content and test content influences validity, and there are often structural factors in selection (e.g., positive correlations among selection tests) that strongly limit the possible influence of test content on validity. Comparisons between test content and job content have important implications for the acceptability of testing, the defensibility of tests in legal proceedings, and the transparency of test development and validation, but these comparisons have little if any bearing on validity.

Use and interpretation of test scores from limited cognitive test batteries: how g + Gc can equal g

Single scores from limited and unbalanced test batteries of cognitive ability can be ambiguous to interpret theoretically. In this study, a limited verbally and knowledge-loaded cognitive test battery, from applicants to the Swedish police academies (N = 1,344), was examined to provide foundations for the use and interpretation of test scores. Three measurement models were compared: one single factor model and two bifactor models, which decomposed the variance of the battery into orthogonal components. The models were evaluated by fit indices and omega coefficients, and then applied to the prediction of academic performance. The overall prediction of all models was similar, although specific abilities also were found to provide substantial predictive validity over and above general intelligence (g). The findings provide support for the use of single scores in applied settings (selection), but suggest that it may be more appropriate to interpret such scores as composites of substantive components, and not just as measures of g.

Specific Cognitive Abilities at Work

Recent findings suggest that the role of specific cognitive abilities in predicting work-related criteria may be critical and may add to the widely demonstrated importance of general mental ability. To summarize and organize these findings, the current paper puts forward two perspectives on the role of specific cognitive abilities in predicting work-related outcomes. Similarities and discrepancies of these perspectives are outlined together with suggestions for boundary conditions of the dominance of general versus specific cognitive abilities. Finally, avenues for future research within and across the two perspectives are discussed.

Examining the Incremental Validity and Relative Importance of Specific Cognitive Abilities in a Training Context

We examine the extent to which g and specific abilities predict learning in a training context by juxtaposing contrasting theories of cognitive ability (i.e., Spearmanian theory and nested-factors theory) and using an analytical strategy (i.e., multidimensional importance analysis) that aligns with underlying theoretical assumptions of both theories. When conducting incremental validity analysis (Spearmanian approach), specific abilities did not add much to the prediction of learning. However, when conducting relative weights analysis (nested-factors approach), specific abilities were the dominant predictors of learning. Results suggest different theoretical and analytical approaches can lead to different results. Results also suggest specific abilities may provide predictive utility beyond g in the training context, but recognizing this utility depends on one’s theoretical assumptions and methodological approach.

The Validity of Specific Cognitive Abilities for the Prediction of Training Success in Germany

General mental ability (GMA) has long been considered one of the best predictors of training success and considerably better than specific cognitive abilities (SCAs). Recently, however, researchers have provided evidence that SCAs may be of similar importance for training success, a finding supporting personnel selection based on job-related requirements. The present meta-analysis therefore seeks to assess validities of SCAs for training success in various occupations in a sample of German primary studies. Our meta-analysis (k = 72) revealed operational validities between ρ = .18 and ρ = .26 for different SCAs. Furthermore, results varied by occupational category, supporting a job-specific benefit of SCAs.

A Theory of Sex Differences in Technical Aptitude and Some Supporting Evidence

In this article, I present a theory that explains the origin of sex differences in technical aptitudes. The theory takes as proven that there are no sex differences in general mental ability (GMA), and it postulates that sex differences in technical aptitude (TA) stem from differences in experience in technical areas, which is in turn based on sex differences in technical interests. Using a large data set, I tested and found support for four predictions made by this theory: (a) the construct level correlation between technical aptitude and GMA is larger for females than males, (b) the observed and true score variability of technical aptitude is greater among males than females, (c) at every level of GMA females have lower levels of technical aptitude, and (d) technical aptitude measures used as estimates of GMA for decision purposes would result in underestimation of GMA levels for girls and women. Given that GMA carries the weight of prediction of job performance, the support found for this last prediction suggests that, for many jobs, technical aptitude tests may underpredict the job performance of female applicants and employees. Future research should examine this question.

The Other Side of Method Bias: The Perils of Distinct Source Research Designs

Common source bias has been the focus of much attention. To minimize the problem, researchers have sometimes been advised to take measurements of predictors from one observer and measurements of outcomes from another observer or to use separate occasions of measurement. We propose that these efforts to eliminate biases due to common source variance create serious problems. To demonstrate the problems of using what we term the "distinct sources" measurement design, we provide an integrative review of the literature regarding both contamination and deficiency of measures. Building on this theme, the article uses simulated data to demonstrate how using data from distinct observers or occasions of measurement can distort estimates of predictor importance at least as much as common source variance. Alternative multisource designs are advocated and examined for tractability by simulating various numbers of observations and sources in the research design.

Personal Philosophy and Personnel Achievement: Belief in Free Will Predicts Better Job Performance

Do philosophic views affect job performance? The authors found that possessing a belief in free will predicted better career attitudes and actual job performance. The effect of free will beliefs on job performance indicators were over and above well-established predictors such as conscientiousness, locus of control, and Protestant work ethic. In Study 1, stronger belief in free will corresponded to more positive attitudes about expected career success. In Study 2, job performance was evaluated objectively and independently by a supervisor. Results indicated that employees who espoused free will beliefs were given better work performance evaluations than those who disbelieve in free will, presumably because belief in free will facilitates exerting control over one’s actions.

A comprehensive model for the selection of medical students

BACKGROUND

Medical schools have a need to select their students from an excess of applicants. Selection procedures have evolved piecemeal: Academic thresholds have risen, written tests have been incorporated and interview protocols are developed.

AIM

To develop and offer for critical review and, ultimately, present for adoption by medical schools, an evidence-based and defensible model for medical student selection.

METHODS

We have described here a comprehensive model for selecting medical students which is grounded on the theoretical and empirical selection and assessment literature, and has been shaped by our own research and experience.

RESULTS

The model includes the following selection criteria: Informed self-selection, academic achievement, general cognitive ability (GCA) and aspects of personality and interpersonal skills. A psychometrically robust procedure by which cognitive and non-cognitive test scores can be used to make selection decisions is described. Using de-identified data (n = 1000) from actual selection procedures, we demonstrate how the model and the procedure can be used in practice.

CONCLUSION

The model presented is based on a currently best-practice approach and uses measures and methods that maximise the probability of making accurate, fair and defensible selection decisions.