The relationship between performance validity testing, external incentives, and cognitive functioning in long COVID

INTRODUCTION

Performance validity test (PVT) failures occur in clinical practice and at higher rates with external incentives. However, little PVT research has been applied to the Long COVID population. This study aims to address this gap.

METHODS

Participants were 247 consecutive individuals with Long COVID seen for neuropsychological evaluation who completed 4 PVTs and a standardized neuropsychological battery. The sample was 84.2% White and 66% female. The mean age was 51.16 years and mean education was 14.75 years. Medical records were searched for external incentive (e.g., disability claims). Three groups were created based on PVT failures (Pass [no failures], Intermediate [1 failure], and Fail [2+ failures]).

RESULTS

A total of 8.9% participants failed 2+ PVTs, 6.4% failed one PVT, and 85% passed PVTs. From the full sample, 25.1% were identified with external incentive. However, there was a significant difference between the rates of external incentives in the Fail group (54.5%) compared to the Pass (22.1%) and Intermediate (20%) groups. Further, the Fail group had lower cognitive scores and higher frequency of impaired range scores, consistent with PVT research in other populations. External incentives were uncorrelated with cognitive performance.

CONCLUSIONS

Consistent with other populations, results suggest Long COVID cases are not immune to PVT failure and external incentives are associated with PVT failure. Results indicated that individuals in the Pass and Intermediate groups showed no evidence for significant cognitive deficits, but the Fail group had significantly poorer cognitive performance. Thus, PVTs should be routinely administered in Long COVID cases and research.

Cross-validation of the Trail Making Test as a non-memory-based embedded performance validity test among veterans with and without cognitive impairment

OBJECTIVE

This study cross-validated multiple Trail Making Test (TMT) Parts A and B scores as non-memory-based embedded performance validity tests (PVTs) for detecting invalid neuropsychological performance among veterans with and without cognitive impairment.

METHOD

Data were collected from a demographically and diagnostically diverse mixed clinical sample of 100 veterans undergoing outpatient neuropsychological evaluation at a Southwestern VA Medical Center. As part of a larger battery of neuropsychological tests, all veterans completed TMT A and B and four independent criterion PVTs, which were used to classify veterans into valid (n = 75) and invalid (n = 25) groups. Among the valid group 47% (n = 35) were cognitively impaired.

RESULTS

Among the overall sample, all embedded PVTs derived from TMT A and B raw and demographically corrected T-scores significantly differed between validity groups (ηp2 = .21-.31) with significant areas under the curve (AUCs) of .72-.78 and 32-48% sensitivity (≥91% specificity) at optimal cut-scores. When subdivided by cognitive impairment status (i.e., valid-unimpaired vs. invalid; valid-impaired vs. invalid), all TMT scores yielded significant AUCs of .80-.88 and 56%-72% sensitivity (≥90% specificity) at optimal cut-scores. Among veterans with cognitive impairment, neither TMT A or B raw scores were able to significantly differentiate the invalid from the valid-cognitively impaired group; however, demographically corrected T-scores were able to significantly differentiate groups but had poor classification accuracy (AUCs = .66-.68) and reduced sensitivity of 28%-44% (≥91% specificity).

CONCLUSIONS

Embedded PVTs derived from TMT Parts A and B raw and T-scores were able to accurately differentiate valid from invalid neuropsychological performance among veterans without cognitive impairment; however, the demographically corrected T-scores generally were more robust and consistent with prior studies compared to raw scores. By contrast, TMT embedded PVTs had poor accuracy and low sensitivity among veterans with cognitive impairment, suggesting limited utility as PVTs among populations with cognitive dysfunction.

Optimal per test cutoff scores and combinations of failure on multiple embedded performance validity tests in detecting performance invalidity in a mixed clinical sample

We tested the usefulness of six embedded performance validity tests (EPVTs) in identifying performance invalidity in a mixed clinical sample. Using a retrospective design, 181 adults were classified as valid (n = 146) or invalid (n = 35) performance based upon their performance on one of three standalone PVTs (Test of Memory Malingering, Victoria Symptom Validity Test, Dot Counting Test). Multiple cutoffs were identified corresponding to predetermined false positive rates of 0, 5, 10, and 15% for each of six EPVTs. EPVT cutoffs corresponding to the predetermined false positive benchmarks were generally more conservative than currently established scores. Sensitivity was low (.0%-42.9%) for individual EPVTs across these cutoffs and was moderately improved by the combination of multiple EPVT failures. The optimal number of EPVT failures using the 10% false positive rate was ≥ 2. Although the overall classification accuracy of 80.7% and specificity of 89.0% were comparable to prior research, the sensitivity of 45.7% was more modest than previous estimates. Low sensitivities indicate that this combination of EPVTs failed to detect a majority of invalid performers.

Feigning Adult ADHD on a Comprehensive Neuropsychological Test Battery: An Analogue Study

The evaluation of performance validity is an essential part of any neuropsychological evaluation. Validity indicators embedded in routine neuropsychological tests offer a time-efficient option for sampling performance validity throughout the assessment while reducing vulnerability to coaching. By administering a comprehensive neuropsychological test battery to 57 adults with ADHD, 60 neurotypical controls, and 151 instructed simulators, we examined each test's utility in detecting noncredible performance. Cut-off scores were derived for all available outcome variables. Although all ensured at least 90% specificity in the ADHD Group, sensitivity differed significantly between tests, ranging from 0% to 64.9%. Tests of selective attention, vigilance, and inhibition were most useful in detecting the instructed simulation of adult ADHD, whereas figural fluency and task switching lacked sensitivity. Five or more test variables demonstrating results in the second to fourth percentile were rare among cases of genuine adult ADHD but identified approximately 58% of instructed simulators.

Cross-validation of RAVLT performance validity indicators and the RAVLT/RO discriminant function in a large known groups sample

OBJECTIVE

To cross-validate RAVLT performance validity cut-offs and the RAVLT/RO discriminant function in a large neuropsychological sample.

METHOD

RAVLT scores and the RAVLT/RO discriminant function were compared in credible (n = 100) and noncredible (n = 353) neuropsychology referrals.

RESULTS

Noncredible patients scored lower than credible patients on RAVLT scores and the RAVLT/RO discriminant function. With cut-offs set to ≥90% specificity, highest sensitivities were observed for the discriminant function (cut-off ≤.064; 55.8%), recognition total (cut-off ≤9; 53.1%), the recognition combination score (≤10; 47.7%), and total learning across trials (cut-off ≤31; 45.3%). Individuals with histories of learning difficulties were over-represented in the 10% of credible patients exceeding cut-offs. When these individuals were removed, cut-offs could be tightened while still maintaining at least 90% specificity, and thereby increasing sensitivity (e.g., recognition total cut-off ≤10, 65% sensitivity; RAVLT/RO discriminant function cut-off ≤.176, 58% sensitivity). When three of the most sensitive, non-overlapping scores were considered in combination, 17% of credible patients failed ≥1 of the three cut-offs, while 3% failed two, and only 1% failed all three. In contrast, in the noncredible sample, more than two-thirds failed one or more of the three cut-offs, nearly half failed ≥2, and nearly a quarter failed all three.

CONCLUSIONS

RAVLT PVT cut-offs and the RAVLT/RO discriminant function achieve approximately 50% sensitivity, and approach 65% sensitivity when cut-offs specific to samples without histories of learning problems are employed, confirming that RAVLT cut-offs and the RAVLT/RO discriminant function continue to be valuable techniques in the identification of performance invalidity.

Examining the Clinical Utility of Selected Memory-Based Embedded Performance Validity Tests in Neuropsychological Assessment of Patients with Multiple Sclerosis

Within the neuropsychological assessment, clinicians are responsible for ensuring the validity of obtained cognitive data. As such, increased attention is being paid to performance validity in patients with multiple sclerosis (pwMS). Experts have proposed batteries of neuropsychological tests for use in this population, though none contain recommendations for standalone performance validity tests (PVTs). The California Verbal Learning Test, Second Edition (CVLT-II) and Brief Visuospatial Memory Test, Revised (BVMT-R)—both of which are included in the aforementioned recommended neuropsychological batteries—include previously validated embedded PVTs (which offer some advantages, including expedience and reduced costs), with no prior work exploring their utility in pwMS. The purpose of the present study was to determine the potential clinical utility of embedded PVTs to detect the signal of non-credibility as operationally defined by below criterion standalone PVT performance. One hundred thirty-three (133) patients (M age = 48.28; 76.7% women; 85.0% White) with MS were referred for neuropsychological assessment at a large, Midwestern academic medical center. Patients were placed into “credible” (n = 100) or “noncredible” (n = 33) groups based on a standalone PVT criterion. Classification statistics for four CVLT-II and BVMT-R PVTs of interest in isolation were poor (AUCs = 0.58–0.62). Several arithmetic and logistic regression-derived multivariate formulas were calculated, all of which similarly demonstrated poor discriminability (AUCs = 0.61–0.64). Although embedded PVTs may arguably maximize efficiency and minimize test burden in pwMS, common ones in the CVLT-II and BVMT-R may not be psychometrically appropriate, sufficiently sensitive, nor substitutable for standalone PVTs in this population. Clinical neuropsychologists who evaluate such patients are encouraged to include standalone PVTs in their assessment batteries to ensure that clinical care conclusions drawn from neuropsychological data are valid.

Pediatric validity assessment

BACKGROUND

Youth have been assumed historically to be less capable of deception than adults, even though acts of deception in childhood are not uncommon. Relatively little attention has focused on how frequently they feign or exaggerate during healthcare evaluations.

PURPOSE

The current article reviews the literature relevant to using validity tests in children and adolescents, as well as provides a case example of a young adolescent providing noncredible effort and exaggerated symptomatology during neuropsychological evaluation after a mild traumatic brain injury.

CONCLUSION

Numerous case reports and case series have documented clearly that medical and neuropsychological noncredible presentations occur in children, likely more often than many practitioners believe. Thus far, research has found that the base rates of pediatric noncredible presentations are highest in children seen frequently by rehabilitation providers (i.e., children with persistent problems after mild traumatic brain injury and children from families seeking disability benefits on their behalf). Subjective clinical judgment is apt to be ineffective in consistently detecting noncredible presentations. Fortunately, recent research supports the use of several stand-alone validity tests in identifying noncredible pediatric data including the Test of Memory Malingering, Word Memory Test, and Medical Symptom Validity Test. If feigning and exaggeration are not considered in work-ups by rehabilitation practitioners, mismanagement and iatrogenic harm to the child can result.