Effort in acute traumatic brain injury: considering more than pass/fail

Forced choice test score of p ≤ .20 and failures on ≥ six performance validity tests results in similar Overall Test Battery Means

Objective: To determine if similar levels of performance on the Overall Test Battery Mean (OTBM) occur at different forced choice test (FCT) p-value score failures. Second, to determine the OTBM levels that are associated with failures at above chance on various performance validity (PVT) tests. Method: OTBMs were computed from archival data obtained from four practices. We calculated each examinee's Estimated Premorbid Global Ability (EPGA) and OTBM. The sample size was 5,103 examinees with 282 (5.5%) of these scoring below chance at p ≤ .20 on at least one FCT. Results: The OTBM associated with a failure at p ≤ .20 was equivalent to the OTBM that was associated with failing 6 or more PVTs at above-chance cutoffs. The mean OTBMs relative to increasingly strict FCT p cutoffs were similar (T scores in the 30s). As expected, there was an inverse relationship between the number of PVTs failed and examinees' OTBMs. Conclusions: The data support the use of p ≤ .20 as the probability level for testing the significance of below chance performance on FCTs. The OTBM can be used to index the influence of invalid performance on outcomes, especially when an examinee scores below chance.

An Observational Cohort Study of the Role of Level of Effort in Post-Acute Brain Injury Rehabilitation

OBJECTIVE

To investigate the role of participant level of effort (LoE) on outcome in post-acute brain injury rehabilitation with the hypothesis that greater effort is associated with more positive outcomes.

DESIGN

Observational cohort study.

SETTING

Comprehensive integrated rehabilitation program for brain injury within a skilled nursing facility.

PARTICIPANTS

Consecutive admissions with acquired brain injury (N=101).

INTERVENTIONS

Individualized interdisciplinary brain injury rehabilitation; therapist rating of participant LoE with Acquired Brain Injury LoE Scale (ABI-LoES) during physical therapy, occupational therapy, and speech and language pathology sessions.

MAIN OUTCOME MEASURES

Mayo-Portland Adaptability Inventory, fourth edition (MPAI-4); Supervision Rating Scale (SRS).

RESULTS

Linear regression showed that discharge MPAI-4 Total T scores were significantly associated with mean ABI-LoES rating, admission MPAI-4 Total T scores, age at admission, and days from injury but not with standard deviation of ABI-LoES rating, sex, injury type, length of stay, or treatment before or during the COVID-19 pandemic. Discharge SRS scores were significantly associated with mean ABI-LoES rating, admission SRS scores, and age. A 1-unit increase in mean ABI-LoES rating was associated with 5.1-unit lower discharge MPAI-4 Total T scores and 1.5 lower discharge SRS scores, after controlling for other variables. Logistic regression showed that the odds of achieving a minimal clinically important difference on the MPAI-4 were 8.34 times higher with each 1-unit increase in mean ABI-LoES rating after controlling for other variables. Admission MPAI-4 was negatively associated with mean ABI-LoES rating (β=-0.07, t=-8.85, P<.0001).

CONCLUSIONS

After controlling for nonmodifiable variables, average ABI-LoES rating is positively associated with outcome. Initial level of disability is negatively associated with mean ABI-LoES rating.

Multivariate Models of Performance Validity: The Erdodi Index Captures the Dual Nature of Non-Credible Responding (Continuous and Categorical)

This study was designed to examine the classification accuracy of the Erdodi Index (EI-5), a novel method for aggregating validity indicators that takes into account both the number and extent of performance validity test (PVT) failures. Archival data were collected from a mixed clinical/forensic sample of 452 adults referred for neuropsychological assessment. The classification accuracy of the EI-5 was evaluated against established free-standing PVTs. The EI-5 achieved a good combination of sensitivity (.65) and specificity (.97), correctly classifying 92% of the sample. Its classification accuracy was comparable with that of another free-standing PVT. An indeterminate range between Pass and Fail emerged as a legitimate third outcome of performance validity assessment, indicating that the underlying construct is an inherently continuous variable. Results support the use of the EI model as a practical and psychometrically sound method of aggregating multiple embedded PVTs into a single-number summary of performance validity. Combining free-standing PVTs with the EI-5 resulted in a better separation between credible and non-credible profiles, demonstrating incremental validity. Findings are consistent with recent endorsements of a three-way outcome for PVTs (Pass, Borderline, and Fail).

A Scoping Review of Cognitive Assessment in Adults With Acute Traumatic Brain Injury

PURPOSE

The purpose of this study was to describe and synthesize the current research regarding the prevailing cognitive domains impacted by acute traumatic brain injury (TBI) in adults. Standardized and nonstandardized assessments of cognitive function and comorbidities influencing cognitive function during the initial stages of recovery are presented to help guide clinical assessment.

METHOD

A scoping review, guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses framework, was used to explore four electronic databases. Searches identified peer-reviewed empirical literature addressing aspects of cognitive domains impacted after TBI, cognitive assessment, and comorbidities impacting assessment in adults after acute TBI.

RESULTS

A total of 1,072 records were identified and reduced to 75 studies based on inclusion criteria. The cognitive domains most impacted in acute TBI were memory and executive function. The Glasgow Coma Scale (GCS) was the most frequently used tool to assess cognitive abilities, despite it being a measurement of consciousness, not of cognition. Psychological changes were the most commonly noted comorbidity impacting cognitive assessment.

CONCLUSIONS

Assessment of cognition after acute TBI requires a multifaceted approach that considers the typical profile of cognitive impairment, as well as patient-specific factors influencing cognitive abilities following initial brain injury. The present results support the generally held view that memory and executive function deficits are common cognitive difficulties associated with acute TBI in adults. The GCS remains the most widely used tool to assess function, though numerous tools are available that specifically address cognitive domains. Acute medical comorbidities common within this stage of injury are highlighted, as well as gaps of clinical knowledge that remain.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.18372086.

A Meta-Analysis of Neuropsychological Effort Test Performance in Psychotic Disorders

Psychotic disorders are characterized by a generalized neurocognitive deficit (i.e., performance 1.5 SD below controls across neuropsychological domains with no specific profile of differential deficits). A motivational account of the generalized neurocognitive deficit has been proposed, which attributes poor neuropsychological testing performance to low effort. However, findings are inconsistent regarding effort test failure rate in individuals with psychotic disorders across studies (0-72%), and moderators are unclear, making it difficult to know whether the motivational explanation is viable. To address these issues, a meta-analysis was performed on data from 2205 individuals with psychotic disorders across 19 studies with 24 independent effects. Effort failure rate was examined along with moderators of effort test type, forensic status, IQ, positive symptoms, negative symptoms, diagnosis, age, gender, education, and antipsychotic use. The pooled weighted effort test failure rate was 18% across studies and there was a moderate pooled association between effort failure rate and global neurocognitive performance (r = .57). IQ and education significantly moderated failure rate. Collectively, these findings suggest that a nontrivial proportion of individuals with a psychotic disorder fail effort testing, and failure rate is associated with global neuropsychological impairment. However, given that effort tests are not immune to the effects of IQ in psychotic disorders, these results cannot attest to the viability of the motivational account of the generalized neurocognitive deficit. Furthermore, the significant moderating effect of IQ and education on effort test performance suggests that effort tests have questionable validity in this population and should be interpreted with caution.

Performance Validity, Neurocognitive Disorder, and Post-concussion Symptom Reporting in Service Members with a History of Mild Traumatic Brain Injury

Objective

To examine the influence of different performance validity test (PVT) cutoffs on neuropsychological performance, post-concussion symptoms, and rates of neurocognitive disorder and postconcussional syndrome following mild traumatic brain injury (MTBI) in active duty service members.

Method

Participants were 164 service members (Age: M = 28.1 years [SD = 7.3]) evaluated on average 4.1 months (SD = 5.0) following injury. Participants were divided into three mutually exclusive groups using original and alternative cutoff scores on the Test of Memory Malingering (TOMM) and the Effort Index (EI) from the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): (a) PVT-Pass, n = 85; (b) Alternative PVT-Fail, n = 53; and (c) Original PVT-Fail, n = 26. Participants also completed the Neurobehavioral Symptom Inventory.

Results

The PVT-Pass group performed better on cognitive testing and reported fewer symptoms than the two PVT-Fail groups. The Original PVT-Fail group performed more poorly on cognitive testing and reported more symptoms than the Alternative PVT-Fail group. Both PVT-Fail groups were more likely to meet DSM-5 Category A criteria for mild and major neurocognitive disorder and symptom reporting criteria for postconcussional syndrome than the PVT-Pass group. When alternative PVT cutoffs were used instead of original PVT cutoffs, the number of participants with valid data meeting cognitive testing criteria for neurocognitive disorder or postconcussional syndrome decreased dramatically.

Conclusion

PVT performance is significantly and meaningfully related to overall neuropsychological outcome. By using only original cutoffs, clinicians and researchers may miss people with invalid performances.

White Matter Associations With Performance Validity Testing in Veterans With Mild Traumatic Brain Injury: The Utility of Biomarkers in Complicated Assessment

OBJECTIVE

Failure on performance validity tests (PVTs) is common in Veterans with histories of mild traumatic brain injury (mTBI), leading to questionable validity of clinical presentations.

PARTICIPANTS

Using diffusion tensor imaging, we investigated white matter (WM) integrity and cognition in 79 Veterans with history of mTBI who passed PVTs (n = 43; traumatic brain injury [TBI]-passed), history of mTBI who failed at least 1 PVT (n = 13; TBI-failed), and military controls (n = 23; MCs) with no history of TBI.

RESULTS

The TBI-failed group demonstrated significantly lower cognitive scores relative to MCs and the TBI-passed group; however, no such differences were observed between MCs and the TBI-passed group. On a global measure of WM integrity (ie, WM burden), the TBI-failed group showed more overall WM abnormalities than the other groups. However, no differences were observed between the MCs and TBI-passed group on WM burden. Interestingly, regional WM analyses revealed abnormalities in the anterior internal capsule and cingulum of both TBI subgroups relative to MCs. Moreover, compared with the TBI-passed group, the TBI-failed group demonstrated significantly decreased WM integrity in the corpus callosum.

CONCLUSIONS

Findings revealed that, within our sample, WM abnormalities are evident in those who fail PVTs. This study adds to the burgeoning PVT literature by suggesting that poor PVT performance does not negate the possibility of underlying WM abnormalities in military personnel with history of mTBI.

Aggregating validity indicators: The salience of domain specificity and the indeterminate range in multivariate models of performance validity assessment

This study was designed to examine the "domain specificity" hypothesis in performance validity tests (PVTs) and the epistemological status of an "indeterminate range" when evaluating the credibility of a neuropsychological profile using a multivariate model of performance validity assessment. While previous research suggests that aggregating PVTs produces superior classification accuracy compared to individual instruments, the effect of the congruence between the criterion and predictor variable on signal detection and the issue of classifying borderline cases remain understudied. Data from a mixed clinical sample of 234 adults referred for cognitive evaluation (M_Age = 46.6; M_Education = 13.5) were collected. Two validity composites were created: one based on five verbal PVTs (EI-5_VER) and one based on five nonverbal PVTs (EI-5_NV) and compared against several other PVTs. Overall, language-based tests of cognitive ability were more sensitive to elevations on the EI-5_VER compared to visual-perceptual tests; whereas, the opposite was observed with the EI-5_NV. However, the match between predictor and criterion variable had a more complex relationship with classification accuracy, suggesting the confluence of multiple factors (sensory modality, cognitive domain, testing paradigm). An "indeterminate range" of performance validity emerged that was distinctly different from both the Pass and the Fail group. Trichotomized criterion PVTs (Pass-Borderline-Fail) had a negative linear relationship with performance on tests of cognitive ability, providing further support for an "in-between" category separating the unequivocal Pass and unequivocal Fail classification range. The choice of criterion variable can influence classification accuracy in PVT research. Establishing a Borderline range between Pass and Fail more accurately reflected the distribution of scores on multiple PVTs. The traditional binary classification system imposes an artificial dichotomy on PVTs that was not fully supported by the data. Accepting "indeterminate" as a legitimate third outcome of performance validity assessment has the potential to improve the clinical utility of PVTs and defuse debates regarding "near-Passes" and "soft Fails."