Though Forced, Still Valid: Examining the Psychometric Performance of Forced-Choice Measurement of Personality in Children and Adolescents

Unveiling the roles personality plays during childhood and adolescence necessitates its accurate measurement, commonly using traditional Likert-type (LK) scales. However, this format is susceptible to various response biases, which can be particularly prevalent in children and adolescents, thus likely undermining measurement accuracy. Forced-choice (FC) scales appear to be a promising alternative because they are largely free from these biases by design. However, some argue that the FC format may not perform satisfactorily in children and adolescents due to its complexity. Little empirical evidence exists regarding the suitability of the FC format for children and adolescents. As such, the current study examined the psychometric performance of an FC measure of the Big Five personality factors in three children and adolescent samples: 5th to 6th graders (N = 428), 7th to 8th graders (N = 449), and 10th to 11th graders (N = 555). Across the three age groups, the FC scale demonstrated a better fit to the Big Five model and better discriminant validity in comparison to the LK counterpart. Personality scores from the FC scale also converged well with those from the LK scale and demonstrated high reliability as well as sizable criterion-related validity. Furthermore, the FC scale had more invariant statements than its LK counterpart across age groups. Overall, we found good evidence showing that FC measurement of personality is suitable for children and adolescents.

Symptom coaching and symptom validity tests: An analog study using the structured inventory of malingered symptomatology, Self-Report Symptom Inventory, and Inventory of Problems-29

In this pilot and exploratory study, we tested the robustness of three self-report symptom validity tests (SVTs) to symptom coaching for depression, with and without additional information available on the Internet. Specifically, we divided our sample (N = 193) so that each subject received either the Structured Inventory of Malingered Symptomatology (SIMS; n = 64), the Self-Report Symptom Inventory (SRSI; n = 66), or the Inventory of Problems-29 (IOP-29; n = 63). Within each of the three subgroups, approximately one third of participants were instructed to respond honestly (Genuine Condition, nSIMS = 21; nSRSI = 24; nIOP-29 = 26) and approximately two-thirds were instructed to feign depression. One half of the feigners were presented with a vignette to increase their compliance with instructions and were given information about symptoms of depression (Coached Feigning, nSIMS = 25; nSRSI = 18; nIOP-29 = 21), and the other half were given the same vignette and information about symptoms of depression, plus two Internet links to review before completing the test (Internet-Coached Feigning, nSIMS = 18; nSRSI = 24; nIOP-29 = 16). Overall, the results showed that the genuine conditions yielded the lowest total scores on all three measures, while the two feigning conditions did not significantly differ from each other. Looking at the detection rates for all feigning participants, all three measures showed satisfactory results, with IOP-29 performing slightly better than SIMS and SIMS performing slightly better than SRSI. Internet-Coached Feigners scored slightly lower on all three measures than feigners who were coached without the Internet links. Taken together, the results of this preliminary and exploratory study suggest that all three SVTs examined are sensitive to feigned depression even in the presence of symptom coaching, both with and without additional Internet-based information.

Sources of measurement error in pediatric intelligence testing

Pediatric intelligence tests, such as the Wechsler Intelligence Scale for Children, are commonly used diagnostic tools used in the process of diagnosing learning and behavior disabilities. Decisions concerning treatment are made based on the results of these tests and they are frequently used in educational and vocational contexts for important decisions that impact persons’ academic or professional lives. Research has however shown that important errors may occur despite the application of validation processes and adherence to quality criteria for psychometric tests. At the same time this evidence seems not to be pervasively acknowledged in psychological practice and research. In this article, I will showcase research that places attention on sources of measurement error in pediatric intelligence testing, discuss a process-performance approach to measurement in intelligence testing, and propose the “pretest methods,” methods stemming from the field of survey methodology commonly used in questionnaire construction, as a method to help address the problem of sources of measurement error in pediatric intelligence testing and improve the development of these intelligence tests.

“All or nothing”

In the current study, we investigated the unique associations between dichotomous thinking, extreme response style (ERS), and the Dark Triad of personality, namely, Machiavellianism, narcissism, and psychopathy. We hypothesized that dichotomous thinking would exhibit a positive effect on ERS, and that dichotomous thinking would be positively associated with the Dark Triad even after accounting for ERS. Participants were 488 adults with a mean age of 29.54 years (SD = 10.38). Results confirmed dichotomous thinking positively predicts ERS, that the relationship between dichotomous thinking and the Dark Triad domains remains positive and significant even after accounting for ERS. Moreover, ERS manifested weak negative associations with the Dark Triad, with a significant relationship only with psychopathy. Findings from the current study help expand the understanding of both the substantive nature of response styles and the cognitive processes underlying the Dark Triad of personality.

Modeling Wording Effects Does Not Help in Recovering Uncontaminated Person Scores: A Systematic Evaluation With Random Intercept Item Factor Analysis

The item wording (or keying) effect consists of logically inconsistent answers to positively and negatively worded items that tap into similar (but polarly opposite) content. Previous research has shown that this effect can be successfully modeled through the random intercept item factor analysis (RIIFA) model, as evidenced by the improvements in the model fit in comparison to models that only contain substantive factors. However, little is known regarding the capability of this model in recovering the uncontaminated person scores. To address this issue, the study analyzes the performance of the RIIFA approach across three types of wording effects proposed in the literature: carelessness, item verification difficulty, and acquiescence. In the context of unidimensional substantive models, four independent variables were manipulated, using Monte Carlo methods: type of wording effect, amount of wording effect, sample size, and test length. The results corroborated previous findings by showing that the RIIFA models were consistently able to account for the variance in the data, attaining an excellent fit regardless of the amount of bias. Conversely, the models without the RIIFA factor produced increasingly a poorer fit with greater amounts of wording effects. Surprisingly, however, the RIIFA models were not able to better estimate the uncontaminated person scores for any type of wording effect in comparison to the substantive unidimensional models. The simulation results were then corroborated with an empirical dataset, examining the relationship between learning strategies and personality with grade point average in undergraduate studies. The apparently paradoxical findings regarding the model fit and the recovery of the person scores are explained, considering the properties of the factor models examined.

Forced-Choice Versus Likert Responses on an Occupational Big Five Questionnaire

Abstract. Conventional self-report measures are prone to response biases, which distort measurement in any applied assessment. The forced-choice (FC) format was proposed as a potential remedy for these biases. The purpose of these studies was to develop and evaluate a FC questionnaire for the occupational context based on the five factor model of personality. A single-stimulus Likert questionnaire was contextualized for occupational settings and psychometrically optimized in Study 1 ( N = 401). Considering optimal design strategies, we subsequently used this questionnaire to construct and validate a FC questionnaire in Study 2 ( N = 517). Methodological add-ons to established approaches were applied to achieve decent confirmatory model fit. The new questionnaire shows good psychometric qualities and strong validity. We make suggestions for further applications and studies.

The "g" in Faking: Doublethink the Validity of Personality Self-Report Measures for Applicant Selection

The meta-analytic finding that faking does not affect the criterion validity of self-report measures in applicant selection suggests cognitive abilities are crucial to fake personality to an expected optimal profile in self-report measures. Previous studies in this field typically focus on how the extent of faking changes self-report measurement. However, the effect of faking ability is rarely considered. In Study 1 (n = 151), we link two questionnaires, the WSQ and the NEO-PI-R, to use them for later faking ability tasks. With O^∗NET expert ratings and the linked questionnaires, we establish veridical responses of optimal personality profiles for both questionnaires. Based on this, in Study 2, we develop six faking ability task employing both questionnaires and three common jobs to fake for. To score the tasks, we introduce profile similarity metrics that compare faked response vectors to optimal profile vectors. The faking ability tasks were administered to a community sample (n = 210) who additionally completed measures of cognitive abilities, namely general mental ability, crystallized intelligence, and interpersonal abilities. For all, based on previous research, it can be argued that they should predict individual differences in faking ability. We establish a measurement model of faking ability and its relation to the other cognitive abilities. Using structural equations modeling, we find the strongest effect for crystallized intelligence and weaker effects for general mental ability and interpersonal abilities, all positively predicting faking ability. We show for the first time that we can measure faking ability with psychometrically sound techniques, establish a confirmatory factor model of faking ability and that it is largely explained by other cognitive abilities. We conclude that research supporting a positive link between self-reported personality and job performance is presumably confounded by cognitive abilities, because they are predictive of both faking self-reported personality and job performance. We recommend researchers to broaden their measurements with assessments of faking ability or other cognitive abilities (besides general mental ability) in research regarding applicant selection.