The "g" in Faking: Doublethink the Validity of Personality Self-Report Measures for Applicant Selection

The desirability bias in personality-related syllogistic reasoning

INTRODUCTION

The belief-bias effect is a tendency to evaluate syllogistic statements based on believability rather than on formal logic validity. Following this rationale, the study examines desirability bias as the tendency to evaluate syllogistic conclusions based on their desirability when reasoning is conducted on personality-relevant categorical syllogisms.

METHODS

For this purpose, 60 syllogisms were constructed based on the items of the Big Five questionnaire. Syllogisms were subsequently categorized as desirable (e.g., "I empathize with others") and undesirable (e.g., "I am passive") based on their conclusion. In each task, the second premise and the conclusion were formulated in the first person to increase a respondent's identification with the content. A total of 300 university students (Mage = 20.08, SD = 2.02) participated in the study.

RESULTS

A 2 (syllogism validity: valid, invalid) × 2 (syllogism desirability: desirable, undesirable) repeated measures ANOVA was employed. The analysis showed a greater tendency to accept desirable conclusions on valid syllogisms (valid desirable rather than valid undesirable) and reject undesirable conclusions on invalid syllogisms (invalid undesirable rather than invalid desirable).

CONCLUSION

These findings have implications for socially desirable responding in cognitive tasks, which may be further developed as a source of self-relevant content as well as for further extension of belief bias in the form of desirability bias.

Examining SEB skills' incremental validity over personality traits in predicting academic achievement

Personality traits and social, emotional, and behavioral (SEB) skills share the same behavioral referents, but whereas traits refer to a person's typical or average performance, skills refer to their capacity or maximal performance. Given their shared behavioral foundations, an important question to address is whether personality traits and SEB skills independently predict important outcomes. In this study (N = 642), we examined whether subscales of the Behavioral, Emotional, and Social Skills Inventory (BESSI), a measure of SEB skills, provided incremental validity in the prediction of the ACT composite score, an important academic outcome for American adolescents, over the Big Five personality traits. Consistent with our expectations, on average, SEB skills showed stronger associations with ACT achievement scores than personality traits. Moreover, SEB skills added incremental validity over and above personality traits in predicting ACT achievement scores. The findings reinforce the importance of conceptually distinguishing and measuring traits and skills.

Lying on the Dissection Table: Anatomizing Faked Responses

Research has shown that even experts cannot detect faking above chance, but recent studies have suggested that machine learning may help in this endeavor. However, faking differs between faking conditions, previous efforts have not taken these differences into account, and faking indices have yet to be integrated into such approaches. We reanalyzed seven data sets (N = 1,039) with various faking conditions (high and low scores, different constructs, naïve and informed faking, faking with and without practice, different measures [self-reports vs. implicit association tests; IATs]). We investigated the extent to which and how machine learning classifiers could detect faking under these conditions and compared different input data (response patterns, scores, faking indices) and different classifiers (logistic regression, random forest, XGBoost). We also explored the features that classifiers used for detection. Our results show that machine learning has the potential to detect faking, but detection success varies between conditions from chance levels to 100%. There were differences in detection (e.g., detecting low-score faking was better than detecting high-score faking). For self-reports, response patterns and scores were comparable with regard to faking detection, whereas for IATs, faking indices and response patterns were superior to scores. Logistic regression and random forest worked about equally well and outperformed XGBoost. In most cases, classifiers used more than one feature (faking occurred over different pathways), and the features varied in their relevance. Our research supports the assumption of different faking processes and explains why detecting faking is a complex endeavor.

Challenges in Conducting Empirical Epidemiological Research with Truck and Bus Drivers in Diverse Settings in North America

Over 6.5 million commercial vehicle drivers were operating a large truck or bus in the United States in 2020. This career often has high stress and long working hours, with few opportunities for physical activity. Previous research has linked these factors to adverse health conditions. Adverse health conditions affect not only the professional drivers' wellbeing but potentially also commercial motor vehicle (CMV) operators' safe driving ability and public safety for others sharing the roadway. The prevalence of health conditions with high impact on roadway safety in North American CMV drivers necessitates empirical epidemiological research to better understand and improve driver health. The paper presents four challenges in conducting epidemiological research with truck and bus drivers in North America and potential resolutions identified in past and current research. These challenges include (1) the correlation between driving performance, driving experience, and driver demographic factors; (2) the impact of medical treatment status on the relationship between health conditions and driver risk; (3) capturing accurate data in self-report data collection methods; and (4) reaching the CMV population for research. These challenges are common and influential in epidemiological research of this population, as drivers face severe health issues, health-related federal regulations, and the impact of vehicle operation on the safety of themselves and others using the roadways.

Detecting Careless Responding in Survey Data Using Stochastic Gradient Boosting

Careless responding is a bias in survey responses that disregards the actual item content, constituting a threat to the factor structure, reliability, and validity of psychological measurements. Different approaches have been proposed to detect aberrant responses such as probing questions that directly assess test-taking behavior (e.g., bogus items), auxiliary or paradata (e.g., response times), or data-driven statistical techniques (e.g., Mahalanobis distance). In the present study, gradient boosted trees, a state-of-the-art machine learning technique, are introduced to identify careless respondents. The performance of the approach was compared with established techniques previously described in the literature (e.g., statistical outlier methods, consistency analyses, and response pattern functions) using simulated data and empirical data from a web-based study, in which diligent versus careless response behavior was experimentally induced. In the simulation study, gradient boosting machines outperformed traditional detection mechanisms in flagging aberrant responses. However, this advantage did not transfer to the empirical study. In terms of precision, the results of both traditional and the novel detection mechanisms were unsatisfactory, although the latter incorporated response times as additional information. The comparison between the results of the simulation and the online study showed that responses in real-world settings seem to be much more erratic than can be expected from the simulation studies. We critically discuss the generalizability of currently available detection methods and provide an outlook on future research on the detection of aberrant response patterns in survey research.

Review of Racially Equitable Admissions Practices in STEM Doctoral Programs

This study reviews literature on racially equitable admissions practices relevant to graduate programs in STEM. Graduate Record Exam (GRE) scores correlate more strongly with race, gender, and socioeconomic status than performance metrics for research during or after graduate school. Structural changes to admissions processes that can improve equity of admissions decisions and reduce correlations between admissions decisions and demographic data include using holistic review or composite scores that quantize more components of an application, removing hard limits on course requirements, admitting students as a cohort instead of to individual faculty sponsors, and diversifying admissions committees. Some alternative scoring methods attempt to measure personality traits, but performing these measurements during admissions may present difficulties. Bridge programs—whether they are implemented as collaborations with a minority-serving institution, a personalized educational program for each student admitted to a program, or a stand-alone program before the doctoral degree program—may be able to improve both recruitment and retention of students with underrepresented racial and ethnic identities in their field of study. Finally, financial barriers to applications can disproportionately affect underrepresented applicants due to systemic racism. We end with recommendations for graduate programs to improve equity in admissions processes.

The Good, the Bad, and the Clever: Faking Ability as a Socio-Emotional Ability?

Socio-emotional abilities have been proposed as an extension to models of intelligence, but earlier measurement approaches have either not fulfilled criteria of ability measurement or have covered only predominantly receptive abilities. We argue that faking ability-the ability to adjust responses on questionnaires to present oneself in a desired manner-is a socio-emotional ability that can broaden our understanding of these abilities and intelligence in general. To test this theory, we developed new instruments to measure the ability to fake bad (malingering) and administered them jointly with established tests of faking good ability in a general sample of n = 134. Participants also completed multiple tests of emotion perception along with tests of emotion expression posing, pain expression regulation, and working memory capacity. We found that individual differences in faking ability tests are best explained by a general factor that had a large correlation with receptive socio-emotional abilities and had a zero to medium-sized correlation with different productive socio-emotional abilities. All correlations were still small after controlling these effects for shared variance with general mental ability as indicated by tests of working memory capacity. We conclude that faking ability is indeed correlated meaningfully with other socio-emotional abilities and discuss the implications for intelligence research and applied ability assessment.

Intelligence and Personality

Abstract. This replication and extension of DeYoung, Quilty, Peterson, and Gray’s (2014) study aimed to assess the unique variance of each of the 10 aspects of the Big Five personality traits ( DeYoung, Quilty, & Peterson, 2007 ) associated with intelligence and its dimensions. Personality aspects and intelligence were assessed in a sample of French-Canadian adults from real-life assessment settings ( n = 213). Results showed that the Intellect aspect was independently associated with g, verbal, and nonverbal intelligence while its counterpart Openness was independently related to verbal intelligence only, thus replicating the results of the original study. Independent associations were also found between Withdrawal, Industriousness and Assertiveness aspects and verbal intelligence, as well as between Withdrawal and Politeness aspects and nonverbal intelligence. Possible explanations for these associations are discussed.

The Motivation and Opportunity for Socially Desirable Responding Does Not Alter the General Factor of Personality

Socially desirable responding may affect the factor structure of personality questionnaires and may be one of the reasons for the common variance among personality traits. In this study, we test this hypothesis by investigating the influence of the motivational test-taking context (development vs. selection) and the opportunity to distort responses (forced-choice vs. Likert response format) on personality questionnaire scores. Data from real selection and assessment candidates (total N = 3,980) matched on gender, age, and educational level were used. Mean score differences were found between the selection and development groups, with smaller differences for the FC version. Yet, exploratory structural equation models showed that the overall factor structures as well as the general factor were highly similar across the four groups. Thus, although socially desirable responding may affect mean scores on personality traits, it does not appear to affect factor structures. This study further suggests that the common variance in personality questionnaires is consistent and appears to be little influenced by motivational pressures for response distortion.