Convergent, Discriminant, and Concurrent Validity of Nonmemory-Based Performance Validity Tests

To supplement memory-based Performance Validity Tests (PVTs) in identifying noncredible performance, we examined the validity of the two most commonly used nonmemory-based PVTs—Dot Counting Test (DCT) and Wechsler Adult Intelligence Scale–Fourth edition (WAIS-IV) Reliable Digit Span (RDS)—as well as two alternative WAIS-IV Digit Span (DS) subtest PVTs. Examinees completed DCT, WAIS-IV DS, and the following criterion PVTs: Test of Memory Malingering, Word Memory Test, and Word Choice Test. Validity groups were determined by passing 3 (valid; n = 69) or failing ⩾2 (noncredible; n = 30) criterion PVTs. DCT, RDS, RDS–Revised (RDS-R), and WAIS-IV DS Age-Corrected Scaled Score (ACSS) were significantly correlated (but uncorrelated with memory-based PVTs). Combining RDS, RDS-R, and ACSS with DCT improved classification accuracy (particularly for DCT/ACSS) for detecting noncredible performance among valid-unimpaired, but largely not valid-impaired examinees. Combining DCT with ACSS may uniquely assess and best supplement memory-based PVTs to identify noncredible neuropsychological test performance in cognitively unimpaired examinees.

Differentiating epilepsy from psychogenic nonepileptic seizures using neuropsychological test data

OBJECTIVE

Differentiating epileptic seizures (ES) from psychogenic nonepileptic seizures (PNES) represents a challenging differential diagnosis with important treatment implications. This study was designed to explore the utility of neuropsychological test scores in differentiating ES from PNES.

METHOD

Psychometric data from 72 patients with ES and 33 patients with PNES were compared on various tests of cognitive ability and performance validity. Individual measures that best discriminated the diagnoses were then entered as predictors in a logistic regression equation with group membership (ES vs. PNES) as the criterion.

RESULTS

On most tests of cognitive ability, the PNES sample outperformed the ES sample (medium-large effect) and was less likely to fail the Reliable Digit Span. However, patients with PNES failed two embedded validity indicators at significantly higher rates (risk ratios (RR): 2.45-4.16). There were no group differences on the Test of Memory Malingering (TOMM). A logistic regression equation based on seven neuropsychological tests correctly classified 85.1% of patients. The cutoff with perfect specificity was associated with 0.47 sensitivity.

CONCLUSIONS

Consistent with previous research, the utility of psychometric methods of differential diagnosis is limited by the complex neurocognitive profiles associated with ES and PNES. Although individual measures might help differentiate ES from PNES, multivariate assessment models have superior discriminant power. The strongest psychometric evidence for PNES appears to be a consistent lack of impairment on tests sensitive to diffuse neurocognitive deficits such as processing speed, working memory, and verbal fluency. While video-electroencephalogram (EEG) monitoring is the gold standard of differential diagnosis, psychometric testing has the potential to enhance clinical decision-making, particularly in complex or unclear cases such as patients with nondiagnostic video-EEGs. Adopting a standardized, fixed neuropsychological battery at epilepsy centers would advance research on the differential diagnostic power of psychometric testing.

Neurocognitive dysfunctioning and the impact of comorbid depression and anxiety in patients with somatic symptom and related disorders: a cross-sectional clinical study

BACKGROUND

The prevalence and severity of neurocognitive dysfunctioning of patients with somatic symptom and related disorders (SSRD) is unknown. Furthermore, the influence of comorbid depression and anxiety has not been evaluated. This study examines neurocognitive dysfunctioning of patients with SSRD and explores if comorbid depression and anxiety is associated with specific neurocognitive dysfunctioning.

METHODS

Cross-sectional study with consecutive patients suffering from SSRD visiting an outpatient specialty mental health care Centre of Excellence for SSRD. Extensive neuropsychological assessment and assessment of depression and anxiety symptom levels using the Patient-Health-Questionnaire-9 and General Anxiety Disorder questionnaire-7 were performed at intake. Multivariate analysis was performed.

RESULTS

The study sample consisted of 201 SSRD patients, with a mean age of 43 years (Standard deviation = 13) years; 37.8% were male. Neurocognitive dysfunction in the domains information processing speed, sustained and divided attention, working memory, verbal and visual memory were reported, compared with normative data. Comorbid depression and anxiety occurred frequently within the sample (75.1% and 65.7%, respectively). Neurocognitive dysfunctioning was worse in patients suffering from comorbid depression [multivariate F (7,161) = 2.839, p = 0.008] but not in patients with comorbid anxiety.

CONCLUSIONS

Poor neurocognitive performance of patients with SSRD is common and worsens in case of comorbid depression. This may explain treatment dropout of patients with SSRD from neurocognitive behavioral therapy. Research on novel interventions is needed targeting neurocognitive functioning of patients with SSRD, particularly those with comorbid depression.

The Association Between PTSD and Functional Outcome Is Mediated by Perception of Cognitive Problems Rather Than Objective Neuropsychological Test Performance

Posttraumatic stress disorder (PTSD) has been consistently linked to poorer functional outcomes, including quality of life, health problems, and social and occupational functioning. Less is known about the potential mechanisms by which PTSD leads to poorer functional outcomes. We hypothesized that neurocognitive functioning and perception of cognitive problems would both mediate the relationship between PTSD diagnosis and functioning. In a sample of 140 veterans of the recent wars and conflicts in Iraq and Afghanistan, we assessed PTSD symptoms, history of traumatic brain injury (TBI), depression, self-report measures of quality of life, social and occupational functioning, and reintegration to civilian life, as well as perception of cognitive problems. Veterans also completed a comprehensive neuropsychological battery of tests. Structural equation modeling revealed that perception of cognitive problems, but not objective neuropsychological performance, mediated the relationship between PTSD diagnosis and functional outcomes after controlling for TBI, depression, education, and a premorbid IQ estimate, b = -6.29, 95% bias-corrected bootstrapped confidence interval [-11.03, -2.88], showing a large effect size. These results highlight the importance of addressing appraisals of posttrauma cognitive functioning in treatment as a means of improving functional outcomes.

Validation of the Advanced Clinical Solutions Word Choice Test (WCT) in a Mixed Clinical Sample: Establishing Classification Accuracy, Sensitivity/Specificity, and Cutoff Scores

This study examined the Word Choice Test’s (WCT) utility as a performance validity test in a mixed clinical sample of veterans referred for neuropsychological evaluation. Participants completed Green’s Word Memory Test (WMT), WCT, and Test of Memory Malingering (TOMM) Trial 1. Using the WMT as the criterion for valid performance, logistic regressions examined the WCT and TOMM’s classification accuracy for those with and without cognitive impairment (CI). Receiver operating characteristic curves were used to establish cut scores which maximized the sensitivity/specificity of each measure. In those without CI, both tests showed good classification accuracy (86.7% and 85.0%, respectively). Among those with CI, the TOMM retained good classification accuracy (82.3%), while the WCT’s decreased considerably (69.4%). Optimal WCT cut scores differed based on impairment status, with a higher sensitivity/specificity trade-off among those with CI. Successful performance on the WCT appeared to rely more heavily on cognitive processes unrelated to performance validity.

Performance validity testing in a clinical sample of adults with sickle cell disease

OBJECTIVE

Neuropsychologists utilize performance validity tests (PVTs) as objective means for drawing inferences about performance validity. The Test of Memory Malingering (TOMM) is a well-validated, stand-alone PVT and the Reliable Digit Span (RDS) and Reliable Digit Span-Revised (RDS-R) from the Digit Span subtest of the WAIS-IV are commonly employed, embedded PVTs. While research has demonstrated the utility of these PVTs with various clinical samples, no research has investigated their use in adults with sickle cell disease (SCD), a condition associated with multiple neurological, physical, and psychiatric symptoms. Thus, the purpose of this study was to explore PVT performance in adults with SCD.

METHOD

Fifty-four adults with SCD (M_age = 40.61, SD = 12.35) were consecutively referred by their hematologist for a routine clinical outpatient neuropsychological evaluation. During the evaluation, participants were administered the TOMM (Trials 1 and 2), neuropsychological measures including the WAIS-IV Digit Span subtest, and mood and behavioral questionnaires.

RESULTS

The average score on the TOMM was 47.70 (SD = 3.47, range = 34-50) for Trial 1 and 49.69 (SD = 1.66, range = 38-50) for Trial 2. Only one participant failed Trial 2 of the TOMM, yielding a 98.1% pass rate for the sample. Pass rates at various RDS and RDS-R values were calculated with TOMM Trial 2 performance as an external criterion.

CONCLUSIONS

Results support the use of the TOMM as a measure of performance validity for individuals with SCD, while RDS and RDS-R should be interpreted with caution in this population.

CVLT-II Forced Choice Recognition Trial as an Embedded Validity Indicator: A Systematic Review of the Evidence

OBJECTIVES

The Forced Choice Recognition (FCR) trial of the California Verbal Learning Test, 2nd edition, was designed as an embedded performance validity test (PVT). To our knowledge, this is the first systematic review of classification accuracy against reference PVTs.

METHODS

Results from peer-reviewed studies with FCR data published since 2002 encompassing a variety of clinical, research, and forensic samples were summarized, including 37 studies with FCR failure rates (N=7575) and 17 with concordance rates with established PVTs (N=4432).

RESULTS

All healthy controls scored >14 on FCR. On average, 16.9% of the entire sample scored ≤14, while 25.9% failed reference PVTs. Presence or absence of external incentives to appear impaired (as identified by researchers) resulted in different failure rates (13.6% vs. 3.5%), as did failing or passing reference PVTs (49.0% vs. 6.4%). FCR ≤14 produced an overall classification accuracy of 72%, demonstrating higher specificity (.93) than sensitivity (.50) to invalid performance. Failure rates increased with the severity of cognitive impairment.

CONCLUSIONS

In the absence of serious neurocognitive disorder, FCR ≤14 is highly specific, but only moderately sensitive to invalid responding. Passing FCR does not rule out a non-credible presentation, but failing FCR rules it in with high accuracy. The heterogeneity in sample characteristics and reference PVTs, as well as the quality of the criterion measure across studies, is a major limitation of this review and the basic methodology of PVT research in general. (JINS, 2016, 22, 851-858).

Neuropsychological Functioning in Veterans with Posttraumatic Stress Disorder: Associations with Performance Validity, Comorbidities, and Functional Outcomes

OBJECTIVES

Numerous studies have shown that individuals with posttraumatic stress disorder (PTSD) display reduced performances on neuropsychological tests, although most prior research has not adequately accounted for comorbidities or performance validity concerns that are common in this population and could partially account for the observed neurocognitive findings. Moreover, few studies have examined the functional implications of neuropsychological results in PTSD.

METHODS

We examined neuropsychological functioning in 44 veterans with PTSD and 40 veteran trauma comparison (TC) participants with combat exposure and no PTSD.

RESULTS

After excluding four veterans with PTSD for performance validity concerns, multivariate analyses of variance by neurocognitive domain revealed significantly worse performance by the PTSD group in the domains of speed of information processing (p=.035) and executive functions (p=.017), but no group differences in attention/working memory, verbal/language functioning, visuoconstruction, or episodic memory. Group differences by PTSD status were still present after covarying for depression, a history of head injuries, and substance use disorders. Executive functioning performance was associated with poorer self-reported occupational functioning and physical health-related quality of life, while speed of information processing performance was associated with poorer physical health-related quality of life.

DISCUSSION

These results are generally consistent with a fronto-limbic conceptualization of PTSD-associated neuropsychological dysfunction and show that cognitive functioning may be associated with critical functional outcomes. Taken together, results suggest that consideration of neurocognitive functioning may enhance the clinical management of individuals with PTSD.

Neuropsychological assessment of a veteran with a large arachnoid cyst

Arachnoid cysts are benign, congenital masses that are believed to form when the arachnoid membrane splits or is duplicated and the resulting space fills with fluid. Despite their potentially alarming appearance on brain imaging, congenital cysts discovered in adulthood are usually silent and do not result in functional impairment. A left-handed male veteran with mild memory complaints was discovered to have a large (16.4 cm × 7.7 cm), left-sided arachnoid cyst. Magnetic resonance imaging (MRI) revealed significant displacement of brain structures including the hippocampus, Sylvan fissure, and splenium. Viewing brain MRI images in only 1 plane was misleading and could have erroneously resulted in assuming some structures were absent. Viewing multiple planes of section revealed significant structural displacement and provided a better 3-dimensional conceptualization of an abnormal brain. A clinical interview indicated excellent premorbid functioning, and neuropsychological test results were within the normal range with the exception of mildly impaired scores on tests reliant on processing speed and lower-than-expected visual memory scores. Results were consistent with previous research noting retained verbal abilities and low-average visual skills. Low-average and mildly impaired scores were potentially secondary to microvascular changes, slowed visual scanning, psychiatric conditions, and testing base rates.

Motivated to do well: an examination of the relationships between motivation, effort, and cognitive performance in schizophrenia

The uncertain relationship between negative symptoms, and specifically motivational deficits, with cognitive dysfunction in schizophrenia is in need of further elucidation as it pertains to the interpretation of cognitive test results. Findings to date have suggested a possible mediating role of motivational deficits on cognitive test measures, although findings from formal examinations of effort using performance validity measures have been inconsistent. The aim of this study was to examine the relationships between motivation, effort exerted during cognitive testing, and cognitive performance in schizophrenia. Sixty-nine outpatients with schizophrenia or schizoaffective disorder were evaluated for psychopathology, severity of motivational deficits, effort exerted during cognitive testing, and cognitive performance. Motivation and degree of effort exerted during cognitive testing were significantly related to cognitive performance, specifically verbal fluency, verbal and working memory, attention and processing speed, and reasoning and problem solving. Further, effort accounted for 15% of the variance in cognitive performance, and partially mediated the relationship between motivation and cognitive performance. Examining cognitive performance profiles for individuals exerting normal or reduced effort revealed significant differences in global cognition, as well as attention/processing speed and reasoning and problem solving. These findings suggest that cognitive domains may be differentially affected by impairments in motivation and effort, and highlight the importance of understanding the interplay between motivation and cognitive performance deficits, which may guide the appropriate selection of symptom targets for promoting recovery in patients.

Efficacy of test of memory malingering Trial 1, Trial 2, the Retention Trial, and the Albany Consistency Index in a criterion group forensic neuropsychological sample

The Test of Memory Malingering is one of the most popular and heavily researched validity tests available for use in neuropsychological evaluations. Recent research has suggested, however, that the original indices and cutoffs may require modifications to increase sensitivity rates. Some of these modifications lack cross-validation and no study has examined all indices in a single sample. This study compares Trial 1, Trial 2, the Retention Trial, and the newly created Albany Consistency Index in a criterion group forensic neuropsychological sample. Findings lend support for the newly created indices and cutoff scores. Implications and cautionary statements are provided and discussed.