A Meta-Analysis of the Faking Resistance of Forced-Choice Personality Inventories

Equally vs. unequally keyed blocks in forced-choice questionnaires: Implications on validity and reliability

Forced-choice (FC) questionnaires have gained scientific interest over the last decades. However, the inclusion of unequally keyed item pairs in FC questionnaires remains a subject of debate, as there is evidence supporting both their usage and avoidance. Designing unequally keyed pairs may be more difficult when considering social desirability, as they might allow the identification of ideal responses. Nevertheless, they may enhance the reliability and the potential for normative interpretation of scores. To empirically investigate this topic, data were collected from 1,125 undergraduate Psychology students who completed a personality item pool measuring the Big Five personality traits in Likert-type format and two FC questionnaires (with and without unequally keyed pairs). These questionnaires were compared in terms of reliability, convergent and criterion validity, and ipsativity of the scores, along with insights on the construction process. While constructing questionnaires with unequally keyed blocks presented challenges in matching items on their social desirability, the differences observed in terms of reliability, validity, or ipsativity were sporadic and lacked systematic patterns. This suggests that neither questionnaire format exhibited a clear superiority. Given these results, it is recommended using only equally keyed blocks to minimize potential validity issues associated with response biases.

Predicting technostress: The Big Five model of personality and subjective well-being

The main goal of the current study is to broaden the knowledge on the association between personality, subjective well-being (SWB) and technostress in an academic context. This research specifically examines the prevalence of technostress in a European university sample. It also explores the relationship between technostress and its dimensions with the Big Five model of personality and with SWB and its affective and cognitive components. Finally, the combined predictive validity of the Big Five and SWB on technostress is tested. The sample was composed of 346 undergraduate students. Correlational and multiple regression analyses were carried out. Results show that fatigue and anxiety are the most frequently experienced dimensions of technostress. Emotional stability, openness to experience, and SWB are negatively and significantly correlated to technostress. Multiple regression analyses show that the Big Five factors and SWB account for technostress variance, the main predictor being the affective component of SWB. These results contribute to a more comprehensive understanding of technostress and suggest that personality traits and SWB are important factors in its prediction. The theoretical and practical implications will be discussed.

On Bank Assembly and Block Selection in Multidimensional Forced-Choice Adaptive Assessments

Multidimensional forced-choice (FC) questionnaires have been consistently found to reduce the effects of socially desirable responding and faking in noncognitive assessments. Although FC has been considered problematic for providing ipsative scores under the classical test theory, item response theory (IRT) models enable the estimation of nonipsative scores from FC responses. However, while some authors indicate that blocks composed of opposite-keyed items are necessary to retrieve normative scores, others suggest that these blocks may be less robust to faking, thus impairing the assessment validity. Accordingly, this article presents a simulation study to investigate whether it is possible to retrieve normative scores using only positively keyed items in pairwise FC computerized adaptive testing (CAT). Specifically, a simulation study addressed the effect of (a) different bank assembly (with a randomly assembled bank, an optimally assembled bank, and blocks assembled on-the-fly considering every possible pair of items), and (b) block selection rules (i.e., T, and Bayesian D and A-rules) over the estimate accuracy and ipsativity and overlap rates. Moreover, different questionnaire lengths (30 and 60) and trait structures (independent or positively correlated) were studied, and a nonadaptive questionnaire was included as baseline in each condition. In general, very good trait estimates were retrieved, despite using only positively keyed items. Although the best trait accuracy and lowest ipsativity were found using the Bayesian A-rule with questionnaires assembled on-the-fly, the T-rule under this method led to the worst results. This points out to the importance of considering both aspects when designing FC CAT.

Large studies reveal how reference bias limits policy applications of self-report measures

There is growing policy interest in identifying contexts that cultivate self-regulation. Doing so often entails comparing groups of individuals (e.g., from different schools). We show that self-report questionnaires-the most prevalent modality for assessing self-regulation-are prone to reference bias, defined as systematic error arising from differences in the implicit standards by which individuals evaluate behavior. In three studies, adolescents (N = 229,685) whose peers performed better academically rated themselves lower in self-regulation and held higher standards for self-regulation. This effect was not observed for task measures of self-regulation and led to paradoxical predictions of college persistence 6 years later. These findings suggest that standards for self-regulation vary by social group, limiting the policy applications of self-report questionnaires.

Counterproductive Academic Behaviors and Academic Performance: A Meta-Analysis and a Path Analysis Model

Counterproductive academic behaviors (CAB) are a complex phenomenon that affects academic institutions in multiple geographical areas with different cultures, values, and social norms. The high incidence of CAB causes problems of critical importance that transcend the educational domain. The current study aims to contribute to the knowledge of the CAB consequences by focusing on its impact on academic performance (AP). For this purpose, a meta-analysis was conducted in order to examine the relationship between CAB, its facets, and AP. The results show that overall CAB and students' performance are negatively related with a true effect size of ρ = −0.40 (K = 231, N = 127,269). Particularly, absenteeism appeared to be the facet most strongly related to AP (ρ = −0.48, K = 117, N = 69,453). A meta-analytic path analysis model was carried out in order to test the predictive validity of CAB, students' personality characteristics, and intelligence on AP. Results show that conscientiousness and cognitive intelligence have a negative relationship with CAB (β = −0.28 and β = −0.20, respectively), and that conscientiousness, openness to experience, intelligence, and CAB can explain 58% of AP true variance. Meta-analyses of moderator variables and hierarchical meta-analyses are also presented. The implications for research and practice are discussed at the end.