Neuropsychological test validity in Veterans presenting with subjective complaints of 'very severe' cognitive symptoms following mild traumatic brain injury

Symptom attribution is a stronger predictor of PVT-failure than symptom endorsement in treatment-seeking Veterans with remote mTBI history: A pilot study

OBJECTIVE

To examine relationships between performance validity testing (PVT), neurobehavioral symptom endorsement, and symptom attribution in Veterans with a history of mild traumatic brain injury (mTBI).

METHOD

Participants included treatment-seeking Veterans (n = 37) with remote mTBI histories who underwent a neuropsychological assessment and completed a modified version of the Neurobehavioral Symptom Inventory (NSI) to assess symptom endorsement and symptom attribution (the latter evaluated by having Veterans indicate whether they believed each NSI symptom was caused by their mTBI). Veterans were divided into two subgroups, PVT-Valid (n = 25) and PVT-Invalid (n = 12).

RESULTS

Independent samples t-tests showed that two of five symptom endorsement variables and all five symptom attribution variables were significantly different between PVT groups (PVT-Invalid > PVT-Valid; Cohen's d = 0.67-1.02). Logistic regression analyses adjusting for PTSD symptoms showed that symptom endorsement (Nagelkerke's R2 = .233) and symptom attribution (Nagelkerke's R2 = .279) significantly distinguished between PVT groups. According to the Wald criterion, greater symptom endorsement (OR = 1.09) and higher attribution of symptoms to mTBI (OR = 1.21) each reliably predicted PVT-failure.

CONCLUSIONS

While both symptom endorsement and symptom attribution were significantly associated with PVT-failure, our preliminary results suggest that symptom attribution is a stronger predictor of PVT-failure. Results highlight the importance of assessing symptom attribution to mTBI in this population.

Benefits and challenges of using logistic regression to assess neuropsychological performance validity: Evidence from a simulation study

Logistic regression (LR) is recognized as a promising method for making decisions about neuropsychological performance validity by integrating information across multiple measures. However, this method has yet to be widely adopted in clinical practice, likely because several open questions remain about its utility relative to simpler methods, its effectiveness across different clinical contexts, and its feasibility at sample sizes common in the field. The current study addresses these questions by assessing classification performance of logistic regression and alternative methods across an array of simulated data sets.

We simulated scores of valid and invalid performers on 6 tests designed to mimic the psychometric and distributional properties of real performance validity measures. Out-of-sample predictive performance of LR and a commonly used alternative ("vote counting") was assessed across different base rates, validity measure properties, and sample sizes.

LR improved classification accuracy by 2%-12% across simulation conditions, primarily by improving sensitivity. False positives and negatives can be further reduced when LR predictions are interpreted as continuous, rather than binary. LR made robust predictions at sample sizes feasible for neuropsychology research (N = 307) and when as few as 2 tests with good psychometric properties were used.

Although training and test data sets of at least several hundred individuals may be required to develop and evaluate LR models for use in clinical practice, LR promises to be an efficient and powerful tool for improving judgements about performance validity. We offer several recommendations for model development and LR interpretation in a clinical setting.

Relationship of psychological/mild traumatic brain injury (mTBI) history and invalid reporting to self-reported executive function

Individuals with mild traumatic brain injury (mTBI) often complain of executive functioning (EF) difficulties. There is a discrepancy between self-reported EF impairment and EF deficits on neuropsychological tests, with some arguing that self-report EF is more related to real-world functioning than EF tests. However, research suggests that self-reported EF may be related more to emotional distress and is vulnerable to invalid reporting. We examined the vulnerability of the short form Barkley Deficits in Executive Functioning Scale (BDEFS) to invalid reporting, using a simulated mTBI paradigm. We included four groups: individuals simulating mTBI with (N = 24) and without (N = 21) histories of mTBI/other psychological conditions and controls with (N = 21) and without (N = 25) histories of mTBI/other psychological conditions. As hypothesized, simulators performed worse on the BDEFS Total Score and EF Symptom Count relative to controls; however, this effect was larger within those who had no self-reported history of mTBI/other psychological conditions. We identified a preliminary cutoff on the EF Symptom Count that detected 42.8% of simulators, with 95% specificity relative to the controls with histories of mTBI/other psychological conditions. The present study emphasizes the need for validity scales on self-report EF measures such as the BDEFS.

Coordinating Global Multi-Site Studies of Military-Relevant Traumatic Brain Injury: Opportunities, Challenges, and Harmonization Guidelines

Traumatic brain injury (TBI) is common among military personnel and the civilian population and is often followed by a heterogeneous array of clinical, cognitive, behavioral, mood, and neuroimaging changes. Unlike many neurological disorders that have a characteristic abnormal central neurologic area(s) of abnormality pathognomonic to the disorder, a sufficient head impact may cause focal, multifocal, diffuse or combination of injury to the brain. This inconsistent presentation makes it difficult to establish or validate biological and imaging markers that could help improve diagnostic and prognostic accuracy in this patient population. The purpose of this manuscript is to describe both the challenges and opportunities when conducting military-relevant TBI research and introduce the Enhancing NeuroImaging Genetics through Meta-Analysis (ENIGMA) Military Brain Injury working group. ENIGMA is a worldwide consortium focused on improving replicability and analytical power through data sharing and collaboration. In this paper, we discuss challenges affecting efforts to aggregate data in this patient group. In addition, we highlight how "big data" approaches might be used to understand better the role that each of these variables might play in the imaging and functional phenotypes of TBI in Service member and Veteran populations, and how data may be used to examine important military specific issues such as return to duty, the late effects of combat-related injury, and alteration of the natural aging processes.

Performance Validity Testing in Justice-Involved Adults with Fetal Alcohol Spectrum Disorder

OBJECTIVES

A number of commonly used performance validity tests (PVTs) may be prone to high failure rates when used for individuals with severe neurocognitive deficits. This study investigated the validity of 10 PVT scores in justice-involved adults with fetal alcohol spectrum disorder (FASD), a neurodevelopmental disability stemming from prenatal alcohol exposure and linked with severe neurocognitive deficits.

METHOD

The sample comprised 80 justice-involved adults (ages 19-40) including 25 with confirmed or possible FASD and 55 where FASD was ruled out. Ten PVT scores were calculated, derived from Word Memory Test, Genuine Memory Impairment Profile, Advanced Clinical Solutions (Word Choice), the Wechsler Adult Intelligence Scale - Fourth Edition (Reliable Digit Span and age-corrected scaled scores (ACSS) from Digit Span, Coding, Symbol Search, Coding - Symbol Search, Vocabulary - Digit Span), and the Wechsler Memory Scale - Fourth Edition (Logical Memory II Recognition).

RESULTS

Participants with diagnosed/possible FASD were more likely to fail any single PVT, and failed a greater number of PVTs overall, compared to those without FASD. They were also more likely to fail based on Word Memory Test, Digit Span ACSS, Coding ACSS, Symbol Search ACSS, and Logical Memory II Recognition, compared to controls (35-76%). Across both groups, substantially more participants with IQ <70 failed two or more PVTs (90%), compared to those with an IQ ≥70 (44%).

CONCLUSIONS

Results highlight the need for additional research examining the use of PVTs in justice-involved populations with FASD.

Neurobehavioral symptom validity and performance validity in veterans: Evidence for distinct outcomes across data types

Self-reported traumatic brain injury (TBI) is common in combat veterans, and identification of psychiatric and neuropsychological consequences following TBI has become a priority for veteran healthcare. Given the importance of accurately capturing symptoms potentially related to TBI in VA settings, validity metrics are frequently used to evaluate both neuropsychological testing validity and the validity of symptom self-reports. The Validity-10 of the Neurobehavioral Symptom Inventory is one such metric that was designed to evaluate symptom over-reporting and thus identify individuals who may produce inconclusive testing profiles. However, the Validity-10's ability to predict objective effort during neuropsychological testing has not been sufficiently explored in veterans. Clinical evaluation data were collected from 295 veterans seeking treatment in a VA TBI clinic. We examined whether the Validity-10 can predict invalid performance on a battery of neuropsychological tests. Validity-10 was a poor predictor of performance validity metrics. Results provide a conceptual replication of earlier work demonstrating that performance and symptom validity are divergent. As such, separate evaluation of these domains is warranted during evaluations conducted in veteran TBI populations.