Does online psychological test administration facilitate faking?

Questionário para Rastreio de Sinais Precoces do Transtorno do Espectro Autista: evidências de validade e consistência interna

RESUMO Objetivo Analisar evidências de validade e a consistência interna do Questionário para Rastreio de Sinais Precoces do Transtorno do Espectro Autista (QR-TEA). Métodos Para a etapa do conteúdo, contou-se com quatro profissionais da saúde com mais de cinco anos de experiência clínica no tratamento de crianças com diagnóstico do TEA. Para a etapa de análise de estrutura interna e consistência interna, contou-se com uma amostra de 133 pais/responsáveis de crianças entre 24-36 meses de idade de todo o Brasil. Os participantes responderam ao questionário por meio de entrevista com profissionais da saúde. Para conhecer a estrutura fatorial, foi utilizado o procedimento de análise fatorial exploratória. Adicionalmente, para avaliar a consistência interna do instrumento, foi efetuado o cálculo da fidedignidade composta. Resultados Na etapa do conteúdo, encontrou-se i-IVC = 1,00 para a maioria dos itens; Kappa entre 0,51 e 0,94; p < 0,05; CCI = 0,90; p < 0,05. A análise fatorial exploratória apontou uma solução de dois fatores para o QR-TEA. A fidedignidade composta foi adequada (acima de 0,70) para os dois fatores. A medida de replicabilidade da estrutura fatorial sugeriu que os dois fatores podem ser replicáveis em estudos futuros (H > 0,80). Conclusões Os dados obtidos indicam evidências iniciais de validade e consistência interna adequada. Os achados apoiam, ainda, a adequação psicométrica do QR-TEA em consonância com o modelo de dois fatores do DSM-5. Recomenda-se a condução de novos estudos para buscar outras evidências de validade e verificar a sensibilidade e especificidade do instrumento.

Construct and Predictive Validity of an Assessment Game to Measure Honesty-Humility

Research on commercial computer games has demonstrated that in-game behavior is related to the players’ personality profiles. However, this potential has not yet been fully utilized for personality assessments. Hence, we developed an applied (i.e., serious) assessment game to assess the Honesty–Humility personality trait. In two studies, we demonstrate that this game adequately assesses Honesty–Humility. In Study 1 (N = 116), we demonstrate convergent validity of the assessment game with self-reported Honesty–Humility and divergent validity with the other HEXACO traits and cognitive ability. In Study 2 (N = 287), we replicate the findings from Study 1, and also demonstrate that the assessment game shows incremental validity—beyond self-reported personality—in the prediction of cheating for financial gain, but not of counterproductive work and unethical behaviors. The findings demonstrate that assessment games are promising tools for personality measurement in applied contexts.

Predicting instructed simulation and dissimulation when screening for depressive symptoms

The intentional distortion of test results presents a fundamental problem to self-report-based psychiatric assessment, such as screening for depressive symptoms. The first objective of the study was to clarify whether depressed patients like healthy controls possess both the cognitive ability and motivation to deliberately influence results of commonly used screening measures. The second objective was the construction of a method derived directly from within the test takers' responses to systematically detect faking behavior. Supervised machine learning algorithms posit the potential to empirically learn the implicit interconnections between responses, which shape detectable faking patterns. In a standardized design, faking bad and faking good were experimentally induced in a matched sample of 150 depressed and 150 healthy subjects. Participants completed commonly used questionnaires to detect depressive and associated symptoms. Group differences throughout experimental conditions were evaluated using linear mixed-models. Machine learning algorithms were trained on the test results and compared regarding their capacity to systematically predict distortions in response behavior in two scenarios: (1) differentiation of authentic patient responses from simulated responses of healthy participants; (2) differentiation of authentic patient responses from dissimulated patient responses. Statistically significant convergence of the test scores in both faking conditions suggests that both depressive patients and healthy controls have the cognitive ability as well as the motivational compliance to alter their test results. Evaluation of the algorithmic capability to detect faking behavior yielded ideal predictive accuracies of up to 89%. Implications of the findings, as well as future research objectives are discussed. Trial Registration The study was pre-registered at the German registry for clinical trials (Deutsches Register klinischer Studien, DRKS; DRKS00007708).

Comparing Job Applicants to Non–Applicants Using An Item–Level Bifactor Model on the Hexaco Personality Inventory

The present study evaluated the ability of item–level bifactor models (a) to provide an alternative explanation to current theories of higher order factors of personality and (b) to explain socially desirable responding in both job applicant and non–applicant contexts. Participants (46% male; mean age = 42 years, SD = 11) completed the 200–item HEXACO Personality Inventory–Revised either as part of a job application ( n = 1613) or as part of low–stakes research ( n = 1613). A comprehensive set of invariance tests were performed. Applicants scored higher than non–applicants on honesty–humility ( d = 0.86), extraversion ( d = 0.73), agreeableness ( d = 1.06), and conscientiousness ( d = 0.77). The bifactor model provided improved model fit relative to a standard correlated factor model, and loadings on the evaluative factor of the bifactor model were highly correlated with other indicators of item social desirability. The bifactor model explained approximately two–thirds of the differences between applicants and non–applicants. Results suggest that rather than being a higher order construct, the general factor of personality may be caused by an item–level evaluative process. Results highlight the importance of modelling data at the item–level. Implications for conceptualizing social desirability, higher order structures in personality, test development, and job applicant faking are discussed. Copyright © 2017 European Association of Personality Psychology

Study Protocol on Intentional Distortion in Personality Assessment: Relationship with Test Format, Culture, and Cognitive Ability

Self-report personality questionnaires, traditionally offered in a graded-scale format, are widely used in high-stakes contexts such as job selection. However, job applicants may intentionally distort their answers when filling in these questionnaires, undermining the validity of the test results. Forced-choice questionnaires are allegedly more resistant to intentional distortion compared to graded-scale questionnaires, but they generate ipsative data. Ipsativity violates the assumptions of classical test theory, distorting the reliability and construct validity of the scales, and producing interdependencies among the scores. This limitation is overcome in the current study by using the recently developed Thurstonian item response theory model. As online testing in job selection contexts is increasing, the focus will be on the impact of intentional distortion on personality questionnaire data collected online. The present study intends to examine the effect of three different variables on intentional distortion: (a) test format (graded-scale versus forced-choice); (b) culture, as data will be collected in three countries differing in their attitudes toward intentional distortion (the United Kingdom, Serbia, and Turkey); and (c) cognitive ability, as a possible predictor of the ability to choose the more desirable responses. Furthermore, we aim to integrate the findings using a comprehensive model of intentional distortion. In the Anticipated Results section, three main aspects are considered: (a) the limitations of the manipulation, theoretical approach, and analyses employed; (b) practical implications for job selection and for personality assessment in a broader sense; and (c) suggestions for further research.

Mode of administration does not cause bias in patient-reported outcome results: a meta-analysis

PURPOSE

Technological advances in recent decades have led to the availability of new modes to administer patient-reported outcomes (PROs). To aid selecting optimal modes of administration (MOA), we undertook a systematic review to determine whether differences in bias (both size and direction) exist among modes.

METHODS

We searched five electronic databases from 2004 (date of last comprehensive review on this topic) to April 2014, cross-referenced and searched reference lists. Studies that compared two or more MOA for a health-related PRO measure in adult samples were included. Two reviewers independently applied inclusion and quality criteria and extracted findings. Meta-analyses and meta-regressions were conducted using random-effects models.

RESULTS

Of 5100 papers screened, 222 were considered potentially relevant and 56 met eligibility criteria. No evidence of bias was found for: (1) paper versus electronic self-complete; and (2) self-complete versus assisted MOA. Heterogeneity for paper versus electronic comparison was explained by type of construct (i.e. physical vs. psychological). Heterogeneity for self-completion versus assisted modes was in part explained by setting (clinic vs. home); the largest bias was introduced when assisted completion occurred in the clinic and follow-up was by self-completion (either electronic or paper) in the home.

CONCLUSIONS

Self-complete paper and electronic MOA can be used interchangeably for research in clinic and home settings. Self-completion and assisted completion produce equivalent scores overall, although heterogeneity may be induced by setting. These results support the use of mixed MOAs within a research study, which may be a useful strategy for reducing missing PRO data.

Psychometric evaluation of the Swedish version of the pure procrastination scale, the irrational procrastination scale, and the susceptibility to temptation scale in a clinical population

Background

Procrastination is a prevalent self-regulatory failure associated with stress and anxiety, decreased well-being, and poorer performance in school as well as work. One-fifth of the adult population and half of the student population describe themselves as chronic and severe procrastinators. However, despite the fact that it can become a debilitating condition, valid and reliable self-report measures for assessing the occurrence and severity of procrastination are lacking, particularly for use in a clinical context. The current study explored the usefulness of the Swedish version of three Internet-administered self-report measures for evaluating procrastination; the Pure Procrastination Scale, the Irrational Procrastination Scale, and the Susceptibility to Temptation Scale, all having good psychometric properties in English.

Methods

In total, 710 participants were recruited for a clinical trial of Internet-based cognitive behavior therapy for procrastination. All of the participants completed the scales as well as self-report measures of depression, anxiety, and quality of life. Principal Component Analysis was performed to assess the factor validity of the scales, and internal consistency and correlations between the scales were also determined. Intraclass Correlation Coefficient, Minimal Detectable Change, and Standard Error of Measurement were calculated for the Irrational Procrastination Scale.

Results

The Swedish version of the scales have a similar factor structure as the English version, generated good internal consistencies, with Cronbach’s α ranging between .76 to .87, and were moderately to highly intercorrelated. The Irrational Procrastination Scale had an Intraclass Correlation Coefficient of .83, indicating excellent reliability. Furthermore, Standard Error of Measurement was 1.61, and Minimal Detectable Change was 4.47, suggesting that a change of almost five points on the scale is necessary to determine a reliable change in self-reported procrastination severity.

Conclusions

The current study revealed that the Pure Procrastination Scale, the Irrational Procrastination Scale, and the Susceptibility to Temptation Scale are both valid and reliable from a psychometric perspective, and that they might be used for assessing the occurrence and severity of procrastination via the Internet.

Trial registration

The current study is part of a clinical trial assessing the efficacy of Internet-based cognitive behavior therapy for procrastination, and was registered 04/22/2013 on ClinicalTrials.gov (NCT01842945).

Willing to give but not to forgive: borderline personality features and cooperative behavior

Impaired cooperation has been proposed to contribute to social dysfunctioning in borderline personality disorder (BPD). However, prior research has not distinguished active from reactive cooperation (i.e., nonexploitation versus non-retaliation)--two aspects of cooperative behavior associated with different basic personality traits (viz. Honesty-Humility and Agreeableness). The authors hypothesized that, due to low levels of Agreeableness, but normal levels of Honesty-Humility, borderline personality (BP) features are related to impaired reactive cooperation, but unrelated to active cooperation. Participants (N = 559) hypothetically played both the allocator in the dictator game and the recipient in the ultimatum game. High levels of BP features predicted impaired reactive cooperation in the ultimatum game, which was mediated through low Agreeableness. In contrast, BP features did not predict active cooperation in the dictator game-mirroring the nonassociation between BP features and Honesty-Humility. The findings highlight the importance of focusing on aspects of forgiveness in interventions aiming to improve cooperativeness in BPD.

Faked depression: comparing malingering via the internet, pen-and-paper, and telephone administration modes

BACKGROUND

The Internet and telephone are valuable resources for mental health practitioners when access to face-to-face assessment or therapy is difficult. However, despite the high stakes (such as access to medication or financial compensation) often involved in psychological testing, little is known about how test administration mode influences test scores when patients are malingering. The aim of this research was to examine for the first time the equivalence of a malingered measure of depression when administered over the Internet, over the telephone, and with pen and paper.

SUBJECTS AND METHODS

A mixed experimental design was used with participants (n=91) randomly allocated to Internet, telephone, or pen-and-paper groups. Participants completed the Edinburgh Depression Scale, first under standard instructions and then under instructions to fake as if experiencing severe depression. A manipulation check was included to ensure participants had understood and followed instructions to fake.

RESULTS

As predicted, participants were able to significantly increase their depression scores when asked to fake depression. Participants reported significantly lower faked depression scores over the telephone than in pen-and-paper format. Faked depression scores in the online group did not differ significantly from those in the telephone and pen-and-paper groups. However, participants in all groups met criteria indicative of a provisional diagnosis of depression.

CONCLUSIONS

Overall, the results indicate that there may be no meaningful difference between malingering across administration modes. If an individual is malingering depression, administration mode has minimal influence. These findings provide preliminary support for the use of online/telephone technologies in the assessment of depression.