Sensitivity of Green’s Word Memory Test Genuine Memory Impairment Profile to Temporal Pathology: A Study in Patients With Temporal Lobe Epilepsy

Performance validity tests in people with epilepsy: A review of the literature

OBJECTIVE

Professional bodies recommend the use of performance validity tests (PVTs) to aid the interpretation of scores obtained in neuropsychological assessments, but base rates of failure differ according to neurological diagnosis and the associated impairments. This review summarises the PVT literature in people with epilepsy with the aim of establishing base rates of PVT failure and the factors associated with PVT performance in this population.

METHODS

Ovid and PubMed databases were searched for studies reporting PVT test performance in people with epilepsy.

RESULTS

The search strategy identified 29 studies reporting the performance of people with epilepsy on 23 PVTs. The most commonly reported PVTs were memory-based tasks including the Test of Memory Malingering (n=15) and the Word Memory Test (n=9). Highly variable failure rates were reported on these tasks using cut-offs established in other neurological groups. Factors associated with apparent failure on PVTs in people with epilepsy included older age, lower IQ, attention deficits, impaired verbal and working memory and epilepsy-related factors including a left sided seizure focus, the presence of interictal EEG abnormalities and recent seizure activity.

CONCLUSIONS

Epilepsy related factors are associated with apparent failures on some PVTs. Caution should be employed when interpreting scores on memory based PVTs in this population.

Performance Validity Test Failure in the Clinical Population: A Systematic Review and Meta-Analysis of Prevalence Rates

Performance validity tests (PVTs) are used to measure the validity of the obtained neuropsychological test data. However, when an individual fails a PVT, the likelihood that failure truly reflects invalid performance (i.e., the positive predictive value) depends on the base rate in the context in which the assessment takes place. Therefore, accurate base rate information is needed to guide interpretation of PVT performance. This systematic review and meta-analysis examined the base rate of PVT failure in the clinical population (PROSPERO number: CRD42020164128). PubMed/MEDLINE, Web of Science, and PsychINFO were searched to identify articles published up to November 5, 2021. Main eligibility criteria were a clinical evaluation context and utilization of stand-alone and well-validated PVTs. Of the 457 articles scrutinized for eligibility, 47 were selected for systematic review and meta-analyses. Pooled base rate of PVT failure for all included studies was 16%, 95% CI [14, 19]. High heterogeneity existed among these studies (Cochran's Q = 697.97, p < .001; I2 = 91%; τ2 = 0.08). Subgroup analysis indicated that pooled PVT failure rates varied across clinical context, presence of external incentives, clinical diagnosis, and utilized PVT. Our findings can be used for calculating clinically applied statistics (i.e., positive and negative predictive values, and likelihood ratios) to increase the diagnostic accuracy of performance validity determination in clinical evaluation. Future research is necessary with more detailed recruitment procedures and sample descriptions to further improve the accuracy of the base rate of PVT failure in clinical practice.

An exploratory study of the convergent validity of the Test of Effort (TOE) in adults with acquired brain injury

PRIMARY OBJECTIVE

To examine the convergent validity of the Test of Effort (TOE), a performance validity test (PVT) currently under development that employs a two-subtest (one verbal, one visual), forced-choice recognition memory format.

RESEARCH DESIGN

A descriptive, correlational design was employed to describe performance on the TOE and examine the convergent validity between the TOE and comparison measures.

METHODS AND PROCEDURES

A sample of 53 individuals with chronic acquired brain injury (ABI) were administered the TOE and three well-validated PVTs (Reliable Digit Span [RDS], Test of Memory Malingering [TOMM] and Dot Counting Test [DCT]).

MAIN OUTCOMES AND RESULTS

The TOE appeared more difficult than it actually was, suggesting adequate face validity. Medium-to-large correlations were observed between the TOE and established PVTs, suggesting good convergent validity. Provisional cutoff scores are offered based on performance of a subgroup of participants with "sufficient effort."

CONCLUSIONS

Overall, the TOE shows promise as a PVT measure for clinical use. Future studies with larger and more diverse samples are needed to more fully determine the psychometric characteristics of the TOE.

Evaluation of the classification accuracy of multiple performance validity tests in a mixed clinical sample

The Test of Memory Malingering (TOMM) and Word Memory Test (WMT) are among the most well-known performance validity tests (PVTs) and regarded as gold standard measures. Due to the many factors that impact PVT selection, it is imperative that clinicians make informed clinical decisions with respect to additional or alternative PVTs that demonstrate similar classification accuracy as these well-validated measures. The present archival study evaluated the agreement/classification accuracy of a large battery consisting of multiple other freestanding/embedded PVTs in a mixed clinical sample of 126 veterans. We examined failure rates for all standalone/embedded PVTs using established cut-scores and calculated pass/fail agreement rates and diagnostic odds ratios for various combinations of PVTs using the TOMM and WMT as criterion measures. TOMM and WMT demonstrated the best agreement, followed by Word Choice Test (WCT). The Rey Fifteen Item Test had an excessive number of false-negative errors and reduced classification accuracy. The Digit Span age-corrected scaled score (DS-ACSS) had highest agreement. Findings lend further support to the use of a combination of embedded and standalone PVTs in identifying suboptimal performance. Results provide data to enhance clinical decision making for neuropsychologists who implement combinations of PVTs in a larger clinical battery.

Classification accuracy of the word memory test genuine memory impairment index

OBJECTIVE

The Word Memory Test (WMT) assesses non-credible performance in neuropsychological assessment. To mitigate risk of false positives among patients with severe cognitive dysfunction, the Genuine Memory Impairment Profile was derived. Only a modest number of investigations has evaluated classification accuracy among clinical samples, leaving the GMIP's accuracy largely uncertain. Accordingly, a simulation experiment evaluated the classification accuracy of the GMIP in a group of healthy individuals coached to simulate mild traumatic brain injury (TBI) related memory impairment on the WMT.

PARTICIPANTS AND METHODS

Eighty healthy individuals were randomly assigned to one of the four experimental groups. One group was provided superficial information concerning TBI symptoms (naïve simulators), another was provided extensive information concerning TBI symptoms (sophisticated simulators), and a third group was provided extensive TBI symptom information and tactics to evade detection by performance validity tests (PVT) (test-coached). An honest responding control group was directed to give their best performance. All participants were administered the California Verbal Learning Test-2 (CVLT-2) and the WMT.

RESULTS

Among the TBI simulators, 90% of the test-coached, 95% of the sophisticated simulators, and 100% of the naïve simulators were correctly classified as exaggerating memory impairment on the primary WMT indices. The simulator groups performed worse than the honest responding group on the CVLT-2. Of those who exceeded the WMT cutoffs, 60%, 27%, and 6% of the naïve-, sophisticated-, and test-coached simulators manifested the GMIP profile, respectively.

CONCLUSIONS

The GMIP is apt to misclassify individuals as having genuine memory impairment, especially if a naïve or unsophisticated effort is made to exert non-credible performance. Indeed, individuals who employ the least sophisticated efforts to exaggerate cognitive impairment appear most likely to manifest the GMIP. The GMIP should be used cautiously to discriminate genuine impairment from non-credible performance, especially among people with mild TBI.

Comparing the North American Adult Reading Test (NAART) and the Test of Premorbid Functioning (TOPF) to estimate premorbid Wechsler Adult Intelligence Scale - 4th edition FSIQ in a clinical sample with epilepsy

Estimating premorbid general cognitive functioning is an essential component to the neuropsychological evaluation process. The North American Adult Reading Test (NAART) is a method to predict premorbid general cognitive functioning based on word reading skills developed using the Wechsler Adult Intelligence Scale-Revised (WAIS-R), which is currently in its fourth edition (WAIS-IV). The Test of Premorbid Function (TOPF) was developed using the WAIS-IV, based on the same method as the NAART, to estimate premorbid intellectual ability. There is a paucity of research comparing estimates of premorbid general intellectual ability between the NAART and TOPF. This study evaluated the clinical utility of premorbid estimates of FSIQ derived from the NAART and TOPF in a sample of 101 patients with temporal lobe epilepsy (TLE). Differences between NAART-derived premorbid FSIQs and TOPF simple demographic predicted FSIQs were significant (p < .001) with large effect sizes. NAART estimated premorbid FSIQ (M = 104.04, SD = 8.42) was significantly greater than TOPF premorbid estimates (M = 99.83, SD = 9.26). Results suggested NAART-derived estimates of premorbid FSIQ may be more accurate than TOPF-based estimates, which likely underestimated premorbid FSIQ in this sample of patients with epilepsy. Limitations and future directions are discussed.

Validation of the Advanced Clinical Solutions Word Choice Test (WCT) in a Mixed Clinical Sample: Establishing Classification Accuracy, Sensitivity/Specificity, and Cutoff Scores

This study examined the Word Choice Test’s (WCT) utility as a performance validity test in a mixed clinical sample of veterans referred for neuropsychological evaluation. Participants completed Green’s Word Memory Test (WMT), WCT, and Test of Memory Malingering (TOMM) Trial 1. Using the WMT as the criterion for valid performance, logistic regressions examined the WCT and TOMM’s classification accuracy for those with and without cognitive impairment (CI). Receiver operating characteristic curves were used to establish cut scores which maximized the sensitivity/specificity of each measure. In those without CI, both tests showed good classification accuracy (86.7% and 85.0%, respectively). Among those with CI, the TOMM retained good classification accuracy (82.3%), while the WCT’s decreased considerably (69.4%). Optimal WCT cut scores differed based on impairment status, with a higher sensitivity/specificity trade-off among those with CI. Successful performance on the WCT appeared to rely more heavily on cognitive processes unrelated to performance validity.

Utility of Green's Word Memory Test Free Recall Subtest as a Measure of Verbal Memory: Initial Evidence from a Temporal Lobe Epilepsy Clinical Sample

This study investigated the Word Memory Test (WMT) Free Recall (FR) subtest as a conventional memory measure. Nineteen participants with pharmacoresistant left temporal lobe epilepsy (LTLE) and 16 with right temporal lobe epilepsy (RTLE) completed the WMT, Rey Auditory Verbal Learning Test (RAVLT), and Wechsler Memory Scale-Fourth Edition Logical Memory (LM) subtest during presurgical evaluation. LTLE participants performed significantly worse on FR subtest (p < .05, [Formula: see text]) and RAVLT Trial 7 (p < .01, [Formula: see text]), but not on LM subtest. Age was a significant covariate for FR (p < .01, [Formula: see text]). Logistic regression revealed FR plus age and RAVLT age-adjusted T-scores both yielded 77.1% classification accuracy and respective diagnostic odds ratios of 11.36 and 11.84. Receiver operating characteristic curves to classify seizure laterality found that RAVLT and FR were significant (area under the curve [AUC] = 0.82 and 0.74), whereas LM was nonsignificant (AUC = 0.67). Cut scores and positive/negative predictive values were established for improved clinical classification.