Introduction to special issue: Current trends in empirical examinations of performance and symptom validity

EMG measured reaction time as a predictor of invalid symptom report in psychosomatic patients

Background: Symptom validity tests (SVTs) and performance validity tests (PVTs) are important tools in sociomedical assessments, especially in the psychosomatic context where diagnoses mainly depend on clinical observation and self-report measures. This study examined the relationship between reaction times (RTs) and scores on the Structured Inventory of Malingered Symptomatology (SIMS). It was proposed that slower RTs and larger standard deviations of reaction times (RTSDs) would be observed in participants who scored above the SIMS cut-off (>16). Methods: Direct surface electromyography (EMG) was used to capture RTs during a computer-based RT test in 152 inpatients from a psychosomatic rehabilitation clinic in Germany. Correlation analyses and Mann-Whitney U were used to examine the relationship between RTs and SIMS scores and to assess the potential impact of covariates such as demographics, medical history, and vocational challenges on RTs. Therefore, dichotomized groups based on each potential covariate were compared. Results: Significantly longer RTs and larger RTSDs were found in participants who scored above the SIMS cut-off. Current treatment with psychopharmacological medication, diagnosis of depression, and age had no significant influence on the RT measures. However, work-related problems had a significant impact on RTSDs. Conclusion: There was a significant relationship between longer and more inconsistent RTs and indicators of exaggerated or feigned symptom report on the SIMS in psychosomatic rehabilitation inpatients. Findings from this study provide a basis for future research developing a new RT-based PVT.

Specificity of Embedded Performance Validity Tests in Elderly Veterans with Mild and Major Neurocognitive Disorder

OBJECTIVE

This study explored the specificity of four embedded performance validity tests (PVTs) derived from common neuropsychological tasks in a sample of older veterans with verified cognitive decline and whose performance was deemed valid by licensed psychologists.

METHOD

Participants were 180 veterans who underwent comprehensive neuropsychological evaluation, were determined to have valid performance following profile analysis/conceptualization, and were diagnosed with mild neurocognitive disorder (i.e., MCI; n = 64) or major neurocognitive disorder (i.e., Dementia; n = 116). All participants completed at least one of four embedded PVTs: Reliable Digit Span (RDS), California Verbal Learning Test-2nd ed. Short Form (CVLT-II SF) Forced choice, Trails B:A, and Delis-Kaplan Executive Function System (DKEFS) Letter and Category Fluency.

RESULTS

Adequate specificity (i.e., ≥90%) was achieved at modified cut-scores for all embedded PVTs across MCI and Dementia groups. Trails B:A demonstrated near perfect specificity at its traditional cut-score (Trails B:A < 1.5). RDS ≤ 5 and CVLT-II SF Forced Choice ≤7 led to <10% false positive classification errors across MCI and dementia groups. DKEFS Letter and Category Fluency achieved 90% specificity at extremely low normative cut-scores.

CONCLUSIONS

RDS, Trails B:A, and CVLT-II SF Forced Choice reflect promising embedded PVTs in the context of dementia evaluations. DKEFS Letter and Category Fluency appear too sensitive to genuine neurocognitive decline and, therefore, are inappropriate PVTs in adults with MCI or dementia. Additional research into embedded PVT sensitivity (via known-groups or analogue designs) in MCI and dementia is needed.

Racial differences in positive findings on embedded performance validity tests

INTRODUCTION

Embedded performance validity tests (PVTs) may show increased positive findings in racially diverse examinees. This study examined positive findings in an older adult sample of African American (AA) and European American (EA) individuals recruited as part of a study on aging and cognition.

METHOD

The project involved secondary analysis of deidentified National Alzheimer's Coordinating Center data (N = 22,688). Exclusion criteria included diagnosis of dementia (n = 5,550), mild cognitive impairment (MCI; n = 5,160), impaired but not MCI (n = 1,126), other race (n = 864), and abnormal Mini Mental State Examination (MMSE < 25; n = 135). The initial sample included 9,853 participants (16.4% AA). Propensity score matching matched AA and EA participants on age, education, sex, and MMSE score. The final sample included 3,024 individuals with 50% of participants identifying as AA. Premorbid ability estimates were calculated based on demographics. Failure rates on five raw score and six age-adjusted scaled score PVTs were examined by race.

RESULTS

Age, education, sex, MMSE, and premorbid ability estimate were not significantly different by race. Thirteen percent of AA and 3.8% of EA participants failed two or more raw score PVTs (p < .0001). On age-adjusted PVTs, 20.6% of AA and 5.9% of EA participants failed two or more (p < .0001).

CONCLUSIONS

PVT failure rates were significantly higher among AA participants. Findings indicate a need for cautious interpretation of embedded PVTs with underrepresented groups. Adjustments to embedded PVT cutoffs may need to be considered to improve diagnostic accuracy.

Utility of WAIS-IV Digit Span indices as measures of performance validity in moderate to severe traumatic brain injury

Objective: The addition of Sequencing to WAIS-IV Digit Span (DS) brought about new Reliable Digit Span (RDS) indices and an Age-Corrected Scaled Score that includes Sequencing trials. Reports have indicated that these new performance validity tests (PVTs) are superior to the traditional RDS; however, comparisons in the context of known neurocognitive impairment are sparse. This study compared DS-derived PVT classification accuracies in a design that included adults with verified TBI. Methods: Participants included 64 adults with moderate-to-severe TBI (TBI), 51 healthy adults coached to simulate TBI (SIM), and 78 healthy comparisons (HC). Participants completed the WAIS-IV DS subtest in the context of a larger test battery. Results: Kruskal-Wallis tests indicated that all DS indices differed significantly across groups. Post hoc contrasts revealed that only RDS Forward and the traditional RDS differed significantly between SIM and TBI. ROC analyses indicated that RDS variables were comparable predictors of SIM vs. HC; however, the traditional RDS showed the highest sensitivity when approximating 90% specificity for SIM vs. TBI. A greater percentage of TBI scored RDS Sequencing < 1 compared to SIM and HC. Conclusion: In the context of moderate-to-severe TBI, the DS-derived PVTs showed comparable discriminability. However, the Greiffenstein et al. traditional RDS demonstrated the best classification accuracy with respect to specificity/sensitivity balance. This relative superiority may reflect that individuals with verified TBI are more likely to perseverate on prior instructions during DS Sequencing. Findings highlight the importance of including individuals with verified TBI when evaluating and developing PVTs.

What are the predictors of TOMM failure in clinical TBI populations? A retrospective analysis

OBJECTIVES

To determine base rates of invalid performance on the Test of Memory Malingering (TOMM) in patients with traumatic brain injury (TBI) undertaking rehabilitation who were referred for clinical assessment, and the factors contributing to TOMM failure.

METHODS

Retrospective file review of consecutive TBI referrals for neuropsychological assessment over seven years. TOMM failure was conventionally defined as performance <45/50 on Trial 2 or Retention Trial. Demographic, injury, financial compensation, occupational, and medical variables were collected.

RESULTS

Four hundred and ninety one TBI cases (Median age = 40 years [IQR = 26-52], 79% male, 82% severe TBI) were identified. Overall, 48 cases (9.78%) failed the TOMM. Logistic regression analyses revealed that use of an interpreter during the assessment (adjusted odds ratio [aOR] = 8.25, 95%CI = 3.96-17.18), outpatient setting (aOR = 4.80, 95%CI = 1.87-12.31) and post-injury psychological distress (aOR = 2.77, 95%CI = 1.35-5.70) were significant multivariate predictors of TOMM failure. The TOMM failure rate for interpreter cases was 49% (21/43) in the outpatient setting vs. 7% (2/30) in the inpatient setting. By comparison, 9% (21/230) of non-interpreter outpatient cases failed the TOMM vs. 2% (4/188) of inpatient cases.

CONCLUSIONS

TOMM failure very rarely occurs in clinical assessment of TBI patients in the inpatient rehabilitation setting. It is more common in the outpatient setting, particularly in non-English-speaking people requiring an interpreter. The findings reinforce the importance of routinely administering stand-alone performance validity tests in assessments of clinical TBI populations, particularly in outpatient settings, to ensure that neuropsychological test results can be interpreted with a high degree of confidence.

Demographic differences in performance validity test failure

OBJECTIVE

The present study investigated demographic differences in performance validity test (PVT) failure in a Veteran sample.

METHOD

Data were extracted from clinical neuropsychological evaluations. Only veterans who identified as men, as either European American/White (EA) or African American/Black (AA) were included (n = 1261). We investigated whether performance on two frequently used PVTs, the Test of Memory Malingering (TOMM), and the Medical Symptom Validity Test (MSVT), differed by age, education, and race using separate logistic regressions.

RESULTS

Veterans with younger age, less education, and Veterans Affairs (VA) service-connected disability were significantly more likely to fail both PVTs. Race was not a significant predictor of MSVT failure, but AA patients were significantly more likely than EA patients to fail the TOMM. For all significant demographic predictors in the models, effects were small. In a subsample of patients who were given both PVTs (n = 461), the effects of race on performance remained.

CONCLUSIONS

Performance on the TOMM and MSVT differed by age and level of education. Performance on the TOMM differed between EA and AA patients, whereas performance on the MSVT did not. These results suggest that demographic factors may play a small but measurable role in performance on specific PVTs.

American Academy of Clinical Neuropsychology (AACN) 2021 consensus statement on validity assessment: Update of the 2009 AACN consensus conference statement on neuropsychological assessment of effort, response bias, and malingering

Objective: Citation and download data pertaining to the 2009 AACN consensus statement on validity assessment indicated that the topic maintained high interest in subsequent years, during which key terminology evolved and relevant empirical research proliferated. With a general goal of providing current guidance to the clinical neuropsychology community regarding this important topic, the specific update goals were to: identify current key definitions of terms relevant to validity assessment; learn what experts believe should be reaffirmed from the original consensus paper, as well as new consensus points; and incorporate the latest recommendations regarding the use of validity testing, as well as current application of the term 'malingering.' Methods: In the spring of 2019, four of the original 2009 work group chairs and additional experts for each work group were impaneled. A total of 20 individuals shared ideas and writing drafts until reaching consensus on January 21, 2021. Results: Consensus was reached regarding affirmation of prior salient points that continue to garner clinical and scientific support, as well as creation of new points. The resulting consensus statement addresses definitions and differential diagnosis, performance and symptom validity assessment, and research design and statistical issues. Conclusions/Importance: In order to provide bases for diagnoses and interpretations, the current consensus is that all clinical and forensic evaluations must proactively address the degree to which results of neuropsychological and psychological testing are valid. There is a strong and continually-growing evidence-based literature on which practitioners can confidently base their judgments regarding the selection and interpretation of validity measures.

Do You Know People Who Feign? Proxy Respondents About Feigned Symptoms

We asked students, clinicians, and people from the general population attending a public university lecture (n = 401) whether they knew others who (had) feigned symptoms. We also asked about the type of symptoms and the motives involved. A slight majority of proxy respondents (59%) reported that they knew a person who (had) feigned symptoms, and 34% knew a person who had admitted to them having feigned symptoms. According to our respondents, the most often feigned symptoms were headache/migraine, common cold/fever, and stomachache/nausea, and the most important reasons for doing so were sick leave from work, excusing a failure, and seeking attention from others. We conclude that feigning is part of the normal behavioral repertoire of people and has little to do with deviant personality traits and/or criminal motives. Also, the current emphasis in the neuropsychological literature on malingering, i.e., feigning motivated by external incentives, might be one-sided given that psychological motives, notably seeking attention from others and excuse making, seem to be important determinants of everyday feigning.