Symptom Versus Performance Validity in Patients with Mild TBI: Independent Sources of Non-credible Responding

Neurocognitive and psychiatric outcomes associated with postacute COVID-19 infection without severe medical complication: a meta-analysis

BACKGROUND

Cognitive symptoms are often reported by those with a history of COVID-19 infection. No comprehensive meta-analysis of neurocognitive outcomes related to COVID-19 exists despite the influx of studies after the COVID-19 pandemic. This study meta-analysed observational research comparing cross-sectional neurocognitive outcomes in adults with COVID-19 (without severe medical/psychiatric comorbidity) to healthy controls (HCs) or norm-referenced data.

METHODS

Data were extracted from 54 studies published between January 2020 and June 2023. Hedges' g was used to index effect sizes, which were pooled using random-effects modelling. Moderating variables were investigated using meta-regression and subgroup analyses.

RESULTS

Omnibus meta-analysis of 696 effect sizes extracted across 54 studies (COVID-19 n=6676, HC/norm-reference n=12 986; average time since infection=~6 months) yielded a small but significant effect indicating patients with COVID-19 performed slightly worse than HCs on cognitive measures (g=-0.36; 95% CI=-0.45 to -0.28), with high heterogeneity (Q=242.30, p<0.001, τ=0.26). Significant within-domain effects was yielded by cognitive screener (g=-0.55; 95% CI=-0.75 to -0.36), processing speed (g=-0.44; 95% CI=-0.57 to -0.32), global cognition (g=-0.40; 95% CI=-0.71 to -0.09), simple/complex attention (g=-0.38; 95% CI=-0.46 to -0.29), learning/memory (g=-0.34; 95% CI=-0.46 to -0.22), language (g=-0.34; 95% CI=-0.45 to -0.24) and executive function (g=-0.32; 95% CI=-0.43 to -0.21); but not motor (g=-0.40; 95% CI=-0.89 to 0.10), visuospatial/construction (g=-0.09; 95% CI=-0.23 to 0.05) and orientation (g=-0.02; 95% CI=-0.17 to 0.14). COVID-19 samples with elevated depression, anxiety, fatigue and disease severity yielded larger effects.

CONCLUSION

Mild cognitive deficits are associated with COVID-19 infection, especially as detected by cognitive screeners and processing speed tasks. We failed to observe clinically meaningful cognitive impairments (as measured by standard neuropsychological instruments) in people with COVID-19 without severe medical or psychiatric comorbidities.

The classification accuracy of Warrington's recognition memory test (words) as a performance validity Test in a neurorehabilitation setting

This study was designed to evaluate the classification accuracy of the Warrington's Recognition Memory Test (RMT) in 167 patients (97 or 58.1% men; MAge = 40.4; MEducation= 13.8) medically referred for neuropsychological evaluation against five psychometrically defined criterion groups. At the optimal cutoff (≤42), the RMT produced an acceptable combination of sensitivity (.36-.60) and specificity (.85-.95), correctly classifying 68.4-83.3% of the sample. Making the cutoff more conservative (≤41) improved specificity (.88-.95) at the expense of sensitivity (.30-.60). Lowering the cutoff to ≤40 achieved uniformly high specificity (.91-.95) but diminished sensitivity (.27-.48). RMT scores were unrelated to lateral dominance, education, or gender. The RMT was sensitive to a three-way classification of performance validity (Pass/Borderline/Fail), further demonstrating its discriminant power. Despite a notable decline in research studies focused on its classification accuracy within the last decade, the RMT remains an effective free-standing PVT that is robust to demographic variables. Relatively low sensitivity is its main liability. Further research is needed on its cross-cultural validity (sensitivity to limited English proficiency).

Using the MMPI-2-RF, IOP-29, IOP-M, and FIT in the In-Person and Remote Administration Formats: A Simulation Study on Feigned mTBI

Our study compared the impact of administering Symptom Validity Tests (SVTs) and Performance Validity Tests (PVTs) in in-person versus remote formats and assessed different approaches to combining validity test results. Using the MMPI-2-RF, IOP-29, IOP-M, and FIT, we assessed 164 adults, with half instructed to feign mild traumatic brain injury (mTBI) and half to respond honestly. Within each subgroup, half completed the tests in person, and the other half completed them online via videoconferencing. Results from 2 ×2 analyses of variance showed no significant effects of administration format on SVT and PVT scores. When comparing feigners to controls, the MMPI-2-RF RBS exhibited the largest effect size (d = 3.05) among all examined measures. Accordingly, we conducted a series of two-step hierarchical logistic regression models by entering the MMPI-2-RF RBS first, followed by each other SVT and PVT individually. We found that the IOP-29 and IOP-M were the only measures that yielded incremental validity beyond the effects of the MMPI-2-RF RBS in predicting group membership. Taken together, these findings suggest that administering these SVTs and PVTs in-person or remotely yields similar results, and the combination of MMPI and IOP indexes might be particularly effective in identifying feigned mTBI.

Performance Validity Test Failure in the Clinical Population: A Systematic Review and Meta-Analysis of Prevalence Rates

Performance validity tests (PVTs) are used to measure the validity of the obtained neuropsychological test data. However, when an individual fails a PVT, the likelihood that failure truly reflects invalid performance (i.e., the positive predictive value) depends on the base rate in the context in which the assessment takes place. Therefore, accurate base rate information is needed to guide interpretation of PVT performance. This systematic review and meta-analysis examined the base rate of PVT failure in the clinical population (PROSPERO number: CRD42020164128). PubMed/MEDLINE, Web of Science, and PsychINFO were searched to identify articles published up to November 5, 2021. Main eligibility criteria were a clinical evaluation context and utilization of stand-alone and well-validated PVTs. Of the 457 articles scrutinized for eligibility, 47 were selected for systematic review and meta-analyses. Pooled base rate of PVT failure for all included studies was 16%, 95% CI [14, 19]. High heterogeneity existed among these studies (Cochran's Q = 697.97, p < .001; I2 = 91%; τ2 = 0.08). Subgroup analysis indicated that pooled PVT failure rates varied across clinical context, presence of external incentives, clinical diagnosis, and utilized PVT. Our findings can be used for calculating clinically applied statistics (i.e., positive and negative predictive values, and likelihood ratios) to increase the diagnostic accuracy of performance validity determination in clinical evaluation. Future research is necessary with more detailed recruitment procedures and sample descriptions to further improve the accuracy of the base rate of PVT failure in clinical practice.

Examination of the relationship between symptom and performance validity measures across referral subtypes

INTRODUCTION

The extent to which performance validity (PVT) and symptom validity (SVT) tests measure separate constructs is unclear. Prior research using the Minnesota Multiphasic Personality Inventory (MMPI-2 & RF) suggested that PVTs and SVTs are separate but related constructs. However, the relationship between Personality Assessment Inventory (PAI) SVTs and PVTs has not been explored. This study aimed to replicate previous MMPI research using the PAI, exploring the relationship between PVTs and overreporting SVTs across three subsamples, neurodevelopmental (attention deficit-hyperactivity disorder (ADHD)/learning disorder), psychiatric, and mild traumatic brain injury (mTBI).

METHODS

Participants included 561 consecutive referrals who completed the Test of Memory Malingering (TOMM) and the PAI. Three subgroups were created based on referral question. The relationship between PAI SVTs and the PVT was evaluated through multiple regression analysis.

RESULTS

The results demonstrated the relationship between PAI symptom overreporting SVTs, including Negative Impression Management (NIM), Malingering Index (MAL), and Cognitive Bias Scale (CBS), and PVTs varied by referral subgroup. Specifically, overreporting on CBS but not NIM and MAL significantly predicted poorer PVT performance in the full sample and the mTBI sample. In contrast, none of the overreporting SVTs significantly predicted PVT performance in the ADHD/learning disorder sample but conversely, all SVTs predicted PVT performance in the psychiatric sample.

CONCLUSIONS

The results partially replicated prior research comparing SVTs and PVTs and suggested that constructs measured by SVTs and PVTs vary depending upon population. The results support the necessity of both PVTs and SVTs in clinical neuropsychological practice.

Performance and symptom validity assessment in attention deficit/hyperactivity disorder: Base rates of invalidity, concordance, and relative impact on cognitive performance

Objective: Differential diagnosis of attention deficit/hyperactivity disorder (ADHD) is one of the most common referral questions for neuropsychological evaluation but is complicated by the presence of external incentives. Validity assessment is therefore critical in such evaluations, employing symptom validity tests (SVTs) and performance validity tests (PVTs) to assess the validity of reported symptoms and cognitive test performance, respectively. This study aimed to establish the base rate of symptom and performance invalidity in adults referred for ADHD, compare concordance between performance and symptom validity, and assess the impact of each type of validity on cognitive test performance. Method: This consecutive case series included data from 392 demographically-diverse adults who underwent outpatient neuropsychological evaluation for ADHD. All patients were administered the Clinical Assessment of Attention Deficit-Adult (CAT-A) and a uniform cognitive test battery, including seven PVTs. Results: Invalid symptom reporting and PVT performance were found in 22% and 16% of the sample, respectively. Sixty-eight percent had concordantly valid SVTs/PVTs and 6% had invalid SVTs/PVTs, whereas the remaining 26% had either invalid SVTs or PVTs (but not both). Invalid PVT performance resulted in a significant decrease across all cognitive test scores, with generally large effects (ηp2=.01-.18). Invalid symptom reporting had minimal effects on cognitive test performance (ηp2= ≤.04). Conclusions: PVTs and SVTs are dissociable and therefore should not be used interchangeably in the context of adult ADHD evaluations. Rather, symptom and performance validity should continue to be assessed independently as they provide largely non-redundant information.

Grooved Pegboard adds incremental value over memory-apparent performance validity tests in predicting psychiatric symptom report

The present study evaluated whether Grooved Pegboard (GPB), when used as a performance validity test (PVT), can incrementally predict psychiatric symptom report elevations beyond memory-apparent PVTs. Participants (N = 111) were military personnel and were predominantly White (84%), male (76%), with a mean age of 43 (SD = 12) and having on average 16 years of education (SD = 2). Individuals with disorders potentially compromising motor dexterity were excluded. Participants were administered GPB, three memory-apparent PVTs (Medical Symptom Validity Test, Non-Verbal Medical Symptom Validity Test, Reliable Digit Span), and a symptom validity test (Personality Assessment Inventory Negative Impression Management [NIM]). Results from the three memory-apparent PVTs were entered into a model for predicting NIM, where failure of two or more PVTs was categorized as evidence of non-credible responding. Hierarchical regression revealed that non-dominant hand GPB T-score incrementally predicted NIM beyond memory-apparent PVTs (F(2,108) = 16.30, p < .001; R² change = .05, β = -0.24, p < .01). In a second hierarchical regression, GPB performance was dichotomized into pass or fail, using T-score cutoffs (≤29 for either hand, ≤31 for both). Non-dominant hand GPB again predicted NIM beyond memory-apparent PVTs (F(2,108) = 18.75, p <.001; R² change = .08, β = -0.28, p < .001). Results indicated that noncredible/failing GPB performance adds incremental value over memory-apparent PVTs in predicting psychiatric symptom report.

Multivariate models provide an effective psychometric solution to the variability in classification accuracy of D-KEFS Stroop performance validity cutoffs

ObjectiveThe study was designed to expand on the results of previous investigations on the D-KEFS Stroop as a performance validity test (PVT), which produced diverging conclusions. Method The classification accuracy of previously proposed validity cutoffs on the D-KEFS Stroop was computed against four different criterion PVTs in two independent samples: patients with uncomplicated mild TBI (n = 68) and disability benefit applicants (n = 49). Results Age-corrected scaled scores (ACSSs) ≤6 on individual subtests often fell short of specificity standards. Making the cutoffs more conservative improved specificity, but at a significant cost to sensitivity. In contrast, multivariate models (≥3 failures at ACSS ≤6 or ≥2 failures at ACSS ≤5 on the four subtests) produced good combinations of sensitivity (.39-.79) and specificity (.85-1.00), correctly classifying 74.6-90.6% of the sample. A novel validity scale, the D-KEFS Stroop Index correctly classified between 78.7% and 93.3% of the sample. Conclusions A multivariate approach to performance validity assessment provides a methodological safeguard against sample- and instrument-specific fluctuations in classification accuracy, strikes a reasonable balance between sensitivity and specificity, and mitigates the invalid before impaired paradox.

Using the Grooved Pegboard Test as an Embedded Validity Indicator in a Mixed Neuropsychiatric Sample with Varying Cognitive Impairment: Cross-Validation Problems

Embedded validity indicators (EVIs) derived from motor tests have received less empirical attention than those derived from tests of other neuropsychological abilities, particularly memory. Preliminary evidence suggests that the Grooved Pegboard Test (GPB) may function as an EVI, but existing studies were largely conducted using simulators and population samples without cognitive impairment. In this study we aimed to evaluate the GPB's classification accuracy as an EVI among a mixed clinical neuropsychiatric sample with and without cognitive impairment. This cross-sectional study comprised 223 patients clinically referred for neuropsychological testing. GPB raw and T-scores for both dominant and nondominant hands were examined as EVIs. A known-groups design, based on ≤1 failure on a battery of validated, independent criterion PVTs, showed that GPB performance differed significantly by validity group. Within the valid group, receiver operating characteristic curve analyses revealed that only the dominant hand raw score displayed acceptable classification accuracy for detecting invalid performance (area under curve [AUC] = .72), with an optimal cut-score of ≥106 seconds (33% sensitivity/88% specificity). All other scores had marginally lower classification accuracy (AUCs = .65-.68) for differentiating valid from invalid performers. Therefore, the GPB demonstrated limited utility as an EVI in a clinical sample containing patients with bona fide cognitive impairment.

Take Their Word for It: The Inventory of Problems Provides Valuable Information on Both Symptom and Performance Validity

This study was designed to compare the validity of the Inventory of Problems (IOP-29) and its newly developed memory module (IOP-M) in 150 patients clinically referred for neuropsychological assessment. Criterion groups were psychometrically derived based on established performance and symptom validity tests (PVTs and SVTs). The criterion-related validity of the IOP-29 was compared to that of the Negative Impression Management scale of the Personality Assessment Inventory (NIM_PAI) and the criterion-related validity of the IOP-M was compared to that of Trial-1 on the Test of Memory Malingering (TOMM-1). The IOP-29 correlated significantly more strongly (z = 2.50, p = .01) with criterion PVTs than the NIM_PAI (r_IOP-29 = .34; r_NIM-PAI = .06), generating similar overall correct classification values (OCC_IOP-29: 79-81%; OCC_NIM-PAI: 71-79%). Similarly, the IOP-M correlated significantly more strongly (z = 2.26, p = .02) with criterion PVTs than the TOMM-1 (r_IOP-M = .79; r_TOMM-1 = .59), generating similar overall correct classification values (OCC_IOP-M: 89-91%; OCC_TOMM-1: 84-86%). Findings converge with the cumulative evidence that the IOP-29 and IOP-M are valuable additions to comprehensive neuropsychological batteries. Results also confirm that symptom and performance validity are distinct clinical constructs, and domain specificity should be considered while calibrating instruments.

Objective Cognitive Outcomes and Subjective Emotional Sequelae in Litigating Adults with a Traumatic Brain Injury: The Impact of Performance and Symptom Validity Measures

OBJECTIVE

This study examined the relative contribution of performance and symptom validity in litigating adults with traumatic brain injury (TBI), as a function of TBI severity, and examined the relationship between self-reported emotional symptoms and cognitive tests scores while controlling for validity test performance.

METHOD

Participants underwent neuropsychological assessment between January 2012 and June 2021 in the context of compensation-seeking claims related to a TBI. All participants completed a cognitive test battery, the Personality Assessment Inventory (including symptom validity tests; SVTs), and multiple performance validity tests (PVTs). Data analyses included independent t-tests, one-way ANOVAs, correlation analyses, and hierarchical multiple regression.

RESULTS

A total of 370 participants were included. Atypical PVT and SVT performance were associated with poorer cognitive test performance and higher emotional symptom report, irrespective of TBI severity. PVTs and SVTs had an additive effect on cognitive test performance for uncomplicated mTBI, but less so for more severe TBI. The relationship between emotional symptoms and cognitive test performance diminished substantially when validity test performance was controlled, and validity test performance had a substantially larger impact than emotional symptoms on cognitive test performance.

CONCLUSION

Validity test performance has a significant impact on the neuropsychological profiles of people with TBI, irrespective of TBI severity, and plays a significant role in the relationship between emotional symptoms and cognitive test performance. Adequate validity testing should be incorporated into every neuropsychological assessment, and associations between emotional symptoms and cognitive outcomes that do not consider validity testing should be interpreted with extreme caution.

Assessing Negative Response Bias Using Self-Report Measures: New Articles, New Issues

In psychological injury and related forensic evaluations, two types of tests are commonly used to assess Negative Response Bias (NRB): Symptom Validity Tests (SVTs) and Performance Validity Tests (PVTs). SVTs assess the credibility of self-reported symptoms, whereas PVTs assess the credibility of observed performance on cognitive tasks. Compared to the large and ever-growing number of published PVTs, there are still relatively few validated self-report SVTs available to professionals for assessing symptom validity. In addition, while several studies have examined how to combine and integrate the results of multiple independent PVTs, there are few studies to date that have addressed the combination and integration of information obtained from multiple self-report SVTs. The Special Issue of Psychological Injury and Law introduced in this article aims to help fill these gaps in the literature by providing readers with detailed information about the convergent and incremental validity, strengths and weaknesses, and applicability of a number of selected measures of NRB under different conditions and in different assessment contexts. Each of the articles in this Special Issue focuses on a particular self-report SVT or set of SVTs and summarizes their conditions of use, strengths, weaknesses, and possible cut scores and relative hit rates. Here, we review the psychometric properties of the 19 selected SVTs and discuss their advantages and disadvantages. In addition, we make tentative proposals for the field to consider regarding the number of SVTs to be used in an assessment, the number of SVT failures required to invalidate test results, and the issue of redundancy when selecting multiple SVTs for an assessment.

Failing Performance Validity Cutoffs on the Boston Naming Test (BNT) Is Specific, but Insensitive to Non-Credible Responding

This study was designed to examine alternative validity cutoffs on the Boston Naming Test (BNT).Archival data were collected from 206 adults assessed in a medicolegal setting following a motor vehicle collision. Classification accuracy was evaluated against three criterion PVTs.The first cutoff to achieve minimum specificity (.87-.88) was T ≤ 35, at .33-.45 sensitivity. T ≤ 33 improved specificity (.92-.93) at .24-.34 sensitivity. BNT validity cutoffs correctly classified 67-85% of the sample. Failing the BNT was unrelated to self-reported emotional distress. Although constrained by its low sensitivity, the BNT remains a useful embedded PVT.

Detecting Negative Response Bias Within the Trauma Symptom Inventory–2 (TSI-2): a Review of the Literature

This systematic review was performed to summarize existing research on the symptom validity scales within the Trauma Symptom Inventory–Second Edition (TSI-2), a relatively new self-report measure designed to assess the psychological sequelae of trauma. The TSI-2 has built-in symptom validity scales to monitor response bias and alert the assessor of non-credible symptom profiles. The Atypical Response scale (ATR) was designed to identify symptom exaggeration or fabrication. Proposed cutoffs on the ATR vary from ≥ 7 to ≥ 15, depending on the assessment context. The limited evidence available suggests that ATR has the potential to serve as measure of symptom validity, although its classification accuracy is generally inferior compared to well-established scales. While the ATR seems sufficiently sensitive to symptom over-reporting, significant concerns about its specificity persist. Therefore, it is proposed that the TSI-2 should not be used in isolation to determine the validity of the symptom presentation. More research is needed for development of evidence-based guidelines about the interpretation of ATR scores.