The Prediction of Training Proficiency in Firefighters: A Study of Predictive Validity in Spain

Cognitive Reflection and General Mental Ability as Predictors of Job Performance

This paper presents a study with four independent samples on the validity of cognitive reflection (CR) for predicting job performance and academic outcomes. The results showed that CR was a valid predictor of academic outcomes (i.e., grades and exam marks) and job performance (i.e., assessment center ratings). The magnitude of the CR validity was similar to the validity of GMA. Moreover, we found that CR and GMA were moderately correlated, and multiple regression analyses showed that CR added a small amount of validity over GMA validity for predicting both types of performance. Finally, we discuss the implications of the findings for the theory and practice of predicting performance and organizational sustainability.

Meta-Analysis of the Validity of General Mental Ability for Five Performance Criteria: Hunter and Hunter (1984) Revisited

This paper presents a series of meta-analyses of the validity of general mental ability (GMA) for predicting five occupational criteria, including supervisory ratings of job performance, production records, work sample tests, instructor ratings, and grades. The meta-analyses were conducted with a large database of 467 technical reports of the validity of the General Aptitude Test Battery (GATB) which included 630 independent samples. GMA showed to be a consistent predictor of the five criteria, but the magnitude of the operational validity was not the same across the five criteria. Results also showed that job complexity is a moderator of the GMA validity for the performance criteria. We also found that the GMA validity estimates are slightly smaller than the previous ones obtained by Hunter and Hunter (1984). Finally, we discuss the implications of these findings for the research and practice of personnel selection.

Meta-Analysis of Interrater Reliability of Supervisory Performance Ratings: Effects of Appraisal Purpose, Scale Type, and Range Restriction

Objectives: This reliability generalization study aimed to estimate the mean and variance of the interrater reliability coefficients (r _yy ) of supervisory ratings of overall, task, contextual, and positive job performance. The moderating effect of the appraisal purpose and the scale type was examined. It was hypothesized that the ratings collected for research purposes and multi-item scales have higher r _yy . It was also examined whether r _yy was similar for the four performance dimensions. Method: A database consisting of 224 independent samples was created and hierarchical sub-grouping meta-analyses were conducted. Results: The appraisal purpose was a moderator of r _yy for the four performance dimensions. Scale type was a moderator of r _yy for overall and task performance collected for research purposes. The findings also suggest that supervisors seem to have less difficulty evaluating overall job performance than task, contextual, and positive performance. The best estimates of the observed r _yy for overall job performance are 0.61 for research-collected ratings and 0.45 for administrative-collected ratings. Conclusions: (1) Appraisal purpose moderates r _yy and researchers and practitioners should be aware of its effects before collecting ratings or using empirically-derived interrater reliability distributions, (2) Scale type seems to moderate r _yy in the case of the ratings collected for research purposes, only, (3) overall job performance is more reliably rated than task, contextual, and positive performance. Implications for research and practice are discussed.