The classification accuracy of Warrington's recognition memory test (words) as a performance validity Test in a neurorehabilitation setting

This study was designed to evaluate the classification accuracy of the Warrington's Recognition Memory Test (RMT) in 167 patients (97 or 58.1% men; MAge = 40.4; MEducation= 13.8) medically referred for neuropsychological evaluation against five psychometrically defined criterion groups. At the optimal cutoff (≤42), the RMT produced an acceptable combination of sensitivity (.36-.60) and specificity (.85-.95), correctly classifying 68.4-83.3% of the sample. Making the cutoff more conservative (≤41) improved specificity (.88-.95) at the expense of sensitivity (.30-.60). Lowering the cutoff to ≤40 achieved uniformly high specificity (.91-.95) but diminished sensitivity (.27-.48). RMT scores were unrelated to lateral dominance, education, or gender. The RMT was sensitive to a three-way classification of performance validity (Pass/Borderline/Fail), further demonstrating its discriminant power. Despite a notable decline in research studies focused on its classification accuracy within the last decade, the RMT remains an effective free-standing PVT that is robust to demographic variables. Relatively low sensitivity is its main liability. Further research is needed on its cross-cultural validity (sensitivity to limited English proficiency).

Performance Validity Test Failure in the Clinical Population: A Systematic Review and Meta-Analysis of Prevalence Rates

Performance validity tests (PVTs) are used to measure the validity of the obtained neuropsychological test data. However, when an individual fails a PVT, the likelihood that failure truly reflects invalid performance (i.e., the positive predictive value) depends on the base rate in the context in which the assessment takes place. Therefore, accurate base rate information is needed to guide interpretation of PVT performance. This systematic review and meta-analysis examined the base rate of PVT failure in the clinical population (PROSPERO number: CRD42020164128). PubMed/MEDLINE, Web of Science, and PsychINFO were searched to identify articles published up to November 5, 2021. Main eligibility criteria were a clinical evaluation context and utilization of stand-alone and well-validated PVTs. Of the 457 articles scrutinized for eligibility, 47 were selected for systematic review and meta-analyses. Pooled base rate of PVT failure for all included studies was 16%, 95% CI [14, 19]. High heterogeneity existed among these studies (Cochran's Q = 697.97, p < .001; I2 = 91%; τ2 = 0.08). Subgroup analysis indicated that pooled PVT failure rates varied across clinical context, presence of external incentives, clinical diagnosis, and utilized PVT. Our findings can be used for calculating clinically applied statistics (i.e., positive and negative predictive values, and likelihood ratios) to increase the diagnostic accuracy of performance validity determination in clinical evaluation. Future research is necessary with more detailed recruitment procedures and sample descriptions to further improve the accuracy of the base rate of PVT failure in clinical practice.

Virtuous victims: Disability claimants who over- and under-report

Objective: The present study was the first to investigate the test performance and symptom reports of individuals who engage in both over-reporting (i.e., exaggerating or fabricating symptoms) and under-reporting (i.e., exaggerating positive qualities or denying shortcomings) in the context of a forensic evaluation. We focused on comparing individuals who over- and under-reported (OR + UR) with those who only over-reported (OR-only) on the MMPI-3. Method: Using a disability claimant sample referred for comprehensive psychological evaluations (n = 848), the present study first determined the rates of possible over-reporting (MMPI-3 F ≥ 75 T, Fp ≥ 70 T, Fs ≥ 100 T, or FBS or RBS ≥ 90 T) with (n = 42) and without (n = 332) under-reporting (L ≥ 65 T). Next, we examined group mean differences on MMPI-3 substantive scale scores and scores on several additional measures completed by the disability claimant sample during their evaluation. Results: The small group of individuals identified as both over-reporting and under-reporting (OR + UR) scored meaningfully higher than the OR-only group on several over- and under-reporting symptom validity tests, as well as on measures of emotional and cognitive/somatic complaints, but lower on externalizing measures. The OR + UR group also performed significantly worse than the OR-only group on several performance validity tests and measures of cognitive ability. Conclusions: The present study indicated that disability claimants who engage in simultaneous over- and under-reporting portray themselves as having greater levels of dysfunction but fewer externalizing tendencies relative to claimants who only over-report; however, these portrayals are likely less accurate reflections of their true functioning.

Logical memory, visual reproduction, and verbal paired associates are effective embedded validity indicators in patients with traumatic brain injury

OBJECTIVE

This study was design to evaluate the potential of the recognition trials for the Logical Memory (LM), Visual Reproduction (VR), and Verbal Paired Associates (VPA) subtests of the Wechsler Memory Scales-Fourth Edition (WMS-IV) to serve as embedded performance validity tests (PVTs).

METHOD

The classification accuracy of the three WMS-IV subtests was computed against three different criterion PVTs in a sample of 103 adults with traumatic brain injury (TBI).

RESULTS

The optimal cutoffs (LM ≤ 20, VR ≤ 3, VPA ≤ 36) produced good combinations of sensitivity (.33-.87) and specificity (.92-.98). An age-corrected scaled score of ≤5 on either of the free recall trials on the VPA was specific (.91-.92) and relatively sensitive (.48-.57) to psychometrically defined invalid performance. A VR I ≤ 5 or VR II ≤ 4 had comparable specificity, but lower sensitivity (.25-.42). There was no difference in failure rate as a function of TBI severity.

CONCLUSIONS

In addition to LM, VR, and VPA can also function as embedded PVTs. Failing validity cutoffs on these subtests signals an increased risk of non-credible presentation and is robust to genuine neurocognitive impairment. However, they should not be used in isolation to determine the validity of an overall neurocognitive profile.

Low rate of performance validity failures among individuals with bipolar disorder

OBJECTIVE

Assessing performance validity is imperative in both clinical and research contexts as data interpretation presupposes adequate participation from examinees. Performance validity tests (PVTs) are utilized to identify instances in which results cannot be interpreted at face value. This study explored the hit rates for two frequently used PVTs in a research sample of individuals with and without histories of bipolar disorder (BD).

METHOD

As part of an ongoing longitudinal study of individuals with BD, we examined the performance of 736 individuals with BD and 255 individuals with no history of mental health disorder on the Test of Memory Malingering (TOMM) and the California Verbal Learning Test forced choice trial (CVLT-FC) at three time points.

RESULTS

Undiagnosed individuals demonstrated 100% pass rate on PVTs and individuals with BD passed over 98% of the time. A mixed effects model adjusting for relevant demographic variables revealed no significant difference in TOMM scores between the groups, a = .07, SE = .07, p = .31. On the CVLT-FC, no clinically significant differences were observed (ps < .001).

CONCLUSIONS

Perfect PVT scores were obtained by the majority of individuals, with no differences in failure rates between groups. The tests have approximately >98% specificity in BD and 100% specificity among non-diagnosed individuals. Further, nearly 90% of individuals with BD obtained perfect scores on both measures, a trend observed at each time point.

Utility of CVLT-3 response bias as a measure of performance validity after traumatic brain injury

OBJECTIVE

We sought to determine the utility of a new performance validity index that was recently proposed. In particular, we wanted to determine if this index would be associated with a specificity of at least .90, a sensitivity of at least .40, and an Area Under the Curve of at least .70 in a traumatic brain injury (TBI) sample.

METHOD

We used logistic regression to investigate how well this new index could distinguish persons with TBI (n = 148) who were evaluated within 1-36 months after injury. All participants had been classified on the basis of at least two independent performance validity tests as having provided valid performance (n = 128) or invalid performance (n = 20).

RESULTS

The new performance validity index had acceptable specificity (.96) but had suboptimal sensitivity (.35) and Area Under the Curve (.66). It was concerning that almost half (5/12) of the cases that were identified by this index as providing invalid effort were false positives. Although a slightly more liberal cut-off improved sensitivity, the problem with poor positive predictive power remained. The conventional Forced Choice index had relatively better classification accuracy.

CONCLUSION

Differences in base rates between the original sample of Martin et al. and the current one most likely affected positive predictive power of the new index. Although their performance validity has excellent specificity, the current results do not support the application of this index in the clinical evaluation of patients with traumatic brain injury when base rates of invalid performance differ markedly from those in the original study.

An alternative approach to TOMM cutoff scores using a large sample of military personnel

The accuracy of neuropsychological assessments relies on participants exhibiting their true abilities during administration. The Test of Memory Malingering (TOMM) is a popular performance validity test used to determine whether an individual is providing honest answers. While the TOMM has proven to be highly sensitive to those who are deliberately exaggerating their symptoms, there is a limited explanation regarding the significance of using 45 as a cutoff score. The present study aims to further investigate this question by examining TOMM scores obtained in a large sample of active-duty military personnel (N = 859, M = 26 years, SD = 6.14, 97.31% males, 72.44% white). Results indicated that no notable discrepancies existed between the frequency of participants who scored a 45 and those who scored slightly below a 45 on the TOMM. The sensitivity and specificity of the TOMM were derived using the forced-choice recognition (FCR) scores obtained by participants on the California Verbal Learning Test, Second Edition (CVLT-II). The sensitivity for each trial of the TOMM was 0.84, 0.55, and 0.63, respectively; the specificity for each trial of the TOMM was 0.69, 0.93, and 0.92, respectively. Because sensitivity and specificity rates are both of importance in this study, balanced accuracy scores were also reported. Results suggested that various alternative cutoff scores produced a more accurate classification compared to the traditional cutoff of 45. Further analyses using Fisher's exact test also indicated that there were no significant performance differences on the FCR of the CVLT-II between individuals who received a 44 and individuals who received a 45 on the TOMM. The current study provides evidence on why the traditional cutoff may not be the most effective score. Future research should consider employing alternative methods which do not rely on a single score.

Examination of the MMPI-3 over-reporting scales in a forensic disability sample

Objective: The aim of this investigation was to provide information about the utility of the newly revised and renormed Minnesota Multiphasic Personality Inventory-3 (MMPI-3) over-reporting scales in a forensic disability sample. Method: Participants consisted of 550 non-head injury disability-related referrals (i.e. 95.6% for worker's compensation) and were primarily diagnosed with an adjustment disorder, depressive disorder, or posttraumatic stress disorder. Criterion measures included performance validity indicators and non-MMPI symptom validity indicators. Results: Correlation analyses showed that validity scale F was most strongly associated with non-MMPI symptom validity indicators, whereas F, Fs, FBS, and RBS were comparable to each other in their associations with performance validity indicators. Group mean comparisons between Pass versus Fail PVT groups showed that RBS consistently yielded the largest effect sizes. Using established structured criteria for Malingered Neurocognitive Dysfunction (MND), additional group mean comparisons showed that RBS, followed by Fs, F, and FBS, performed well in differentiating genuine responders from MND examinees. Classification accuracy estimates indicated that the MMPI-3 over-reporting scales performed well in the prediction of Probable/Definite MND and, as expected, to a lesser degree of Possible MND. Conclusions: Practical applications, study limitations, and directions for future research are discussed. The overall findings from this study provide empirical support for the utility of the MMPI-3 over-reporting scales in detecting negative response bias in forensic disability evaluations.

Take Their Word for It: The Inventory of Problems Provides Valuable Information on Both Symptom and Performance Validity

This study was designed to compare the validity of the Inventory of Problems (IOP-29) and its newly developed memory module (IOP-M) in 150 patients clinically referred for neuropsychological assessment. Criterion groups were psychometrically derived based on established performance and symptom validity tests (PVTs and SVTs). The criterion-related validity of the IOP-29 was compared to that of the Negative Impression Management scale of the Personality Assessment Inventory (NIM_PAI) and the criterion-related validity of the IOP-M was compared to that of Trial-1 on the Test of Memory Malingering (TOMM-1). The IOP-29 correlated significantly more strongly (z = 2.50, p = .01) with criterion PVTs than the NIM_PAI (r_IOP-29 = .34; r_NIM-PAI = .06), generating similar overall correct classification values (OCC_IOP-29: 79-81%; OCC_NIM-PAI: 71-79%). Similarly, the IOP-M correlated significantly more strongly (z = 2.26, p = .02) with criterion PVTs than the TOMM-1 (r_IOP-M = .79; r_TOMM-1 = .59), generating similar overall correct classification values (OCC_IOP-M: 89-91%; OCC_TOMM-1: 84-86%). Findings converge with the cumulative evidence that the IOP-29 and IOP-M are valuable additions to comprehensive neuropsychological batteries. Results also confirm that symptom and performance validity are distinct clinical constructs, and domain specificity should be considered while calibrating instruments.

CVLT-II short form forced choice recognition in a clinical dementia sample: Cautions for performance validity assessment

Performance validity tests are susceptible to false positives from genuine cognitive impairment (e.g., dementia); this has not been explored with the short form of the California Verbal Learning Test II (CVLT-II-SF). In a memory clinic sample, we examined whether CVLT-II-SF Forced Choice Recognition (FCR) scores differed across diagnostic groups, and how the severity of impairment [Clinical Dementia Rating Sum of Boxes (CDR-SOB) or Mini-Mental State Examination (MMSE)] modulated test performance. Three diagnostic groups were identified: subjective cognitive impairment (SCI; n = 85), amnestic mild cognitive impairment (a-MCI; n = 17), and dementia due to Alzheimer's Disease (AD; n = 50). Significant group differences in FCR were observed using one-way ANOVA; post-hoc analysis indicated the AD group performed significantly worse than the other groups. Using multiple regression, FCR performance was modeled as a function of the diagnostic group, severity (MMSE or CDR-SOB), and their interaction. Results yielded significant main effects for MMSE and diagnostic group, with a significant interaction. CDR-SOB analyses were non-significant. Increases in impairment disproportionately impacted FCR performance for persons with AD, adding caution to research-based cutoffs for performance validity in dementia. Caution is warranted when assessing performance validity in dementia populations. Future research should examine whether CVLT-II-SF-FCR is appropriately specific for best-practice testing batteries for dementia.

Assessing Negative Response Bias Using Self-Report Measures: New Articles, New Issues

In psychological injury and related forensic evaluations, two types of tests are commonly used to assess Negative Response Bias (NRB): Symptom Validity Tests (SVTs) and Performance Validity Tests (PVTs). SVTs assess the credibility of self-reported symptoms, whereas PVTs assess the credibility of observed performance on cognitive tasks. Compared to the large and ever-growing number of published PVTs, there are still relatively few validated self-report SVTs available to professionals for assessing symptom validity. In addition, while several studies have examined how to combine and integrate the results of multiple independent PVTs, there are few studies to date that have addressed the combination and integration of information obtained from multiple self-report SVTs. The Special Issue of Psychological Injury and Law introduced in this article aims to help fill these gaps in the literature by providing readers with detailed information about the convergent and incremental validity, strengths and weaknesses, and applicability of a number of selected measures of NRB under different conditions and in different assessment contexts. Each of the articles in this Special Issue focuses on a particular self-report SVT or set of SVTs and summarizes their conditions of use, strengths, weaknesses, and possible cut scores and relative hit rates. Here, we review the psychometric properties of the 19 selected SVTs and discuss their advantages and disadvantages. In addition, we make tentative proposals for the field to consider regarding the number of SVTs to be used in an assessment, the number of SVT failures required to invalidate test results, and the issue of redundancy when selecting multiple SVTs for an assessment.

Failing Performance Validity Cutoffs on the Boston Naming Test (BNT) Is Specific, but Insensitive to Non-Credible Responding

This study was designed to examine alternative validity cutoffs on the Boston Naming Test (BNT).Archival data were collected from 206 adults assessed in a medicolegal setting following a motor vehicle collision. Classification accuracy was evaluated against three criterion PVTs.The first cutoff to achieve minimum specificity (.87-.88) was T ≤ 35, at .33-.45 sensitivity. T ≤ 33 improved specificity (.92-.93) at .24-.34 sensitivity. BNT validity cutoffs correctly classified 67-85% of the sample. Failing the BNT was unrelated to self-reported emotional distress. Although constrained by its low sensitivity, the BNT remains a useful embedded PVT.

Introducing the ImPACT-5: An Empirically Derived Multivariate Validity Composite

OBJECTIVE

To create novel Immediate Post-Concussion and Cognitive Testing (ImPACT)-based embedded validity indicators (EVIs) and to compare the classification accuracy to 4 existing EVIImPACT.

METHOD

The ImPACT was administered to 82 male varsity football players during preseason baseline cognitive testing. The classification accuracy of existing EVIImPACT was compared with a newly developed index (ImPACT-5A and B). The ImPACT-5A represents the number of cutoffs failed on the 5 ImPACT composite scores at a liberal cutoff (0.85 specificity); ImPACT-5B is the sum of failures on conservative cutoffs (≥0.90 specificity).

RESULTS

ImPACT-5A ≥1 was sensitive (0.81), but not specific (0.49) to invalid performance, consistent with EVIImPACT developed by independent researchers (0.68 sensitivity at 0.73-0.75 specificity). Conversely, ImPACT-5B ≥3 was highly specific (0.98), but insensitive (0.22), similar to Default EVIImPACT (0.04 sensitivity at 1.00 specificity). ImPACT-5A ≥3 or ImPACT-5B ≥2 met forensic standards of specificity (0.91-0.93) at 0.33 to 0.37 sensitivity. Also, the ImPACT-5s had the strongest linear relationship with clinically meaningful levels of invalid performance of existing EVIImPACT.

CONCLUSIONS

The ImPACT-5s were superior to the standard EVIImPACT and comparable to existing aftermarket EVIImPACT, with the flexibility to optimize the detection model for either sensitivity or specificity. The wide range of ImPACT-5 cutoffs allows for a more nuanced clinical interpretation.

Examining the Clinical Utility of Selected Memory-Based Embedded Performance Validity Tests in Neuropsychological Assessment of Patients with Multiple Sclerosis

Within the neuropsychological assessment, clinicians are responsible for ensuring the validity of obtained cognitive data. As such, increased attention is being paid to performance validity in patients with multiple sclerosis (pwMS). Experts have proposed batteries of neuropsychological tests for use in this population, though none contain recommendations for standalone performance validity tests (PVTs). The California Verbal Learning Test, Second Edition (CVLT-II) and Brief Visuospatial Memory Test, Revised (BVMT-R)—both of which are included in the aforementioned recommended neuropsychological batteries—include previously validated embedded PVTs (which offer some advantages, including expedience and reduced costs), with no prior work exploring their utility in pwMS. The purpose of the present study was to determine the potential clinical utility of embedded PVTs to detect the signal of non-credibility as operationally defined by below criterion standalone PVT performance. One hundred thirty-three (133) patients (M age = 48.28; 76.7% women; 85.0% White) with MS were referred for neuropsychological assessment at a large, Midwestern academic medical center. Patients were placed into “credible” (n = 100) or “noncredible” (n = 33) groups based on a standalone PVT criterion. Classification statistics for four CVLT-II and BVMT-R PVTs of interest in isolation were poor (AUCs = 0.58–0.62). Several arithmetic and logistic regression-derived multivariate formulas were calculated, all of which similarly demonstrated poor discriminability (AUCs = 0.61–0.64). Although embedded PVTs may arguably maximize efficiency and minimize test burden in pwMS, common ones in the CVLT-II and BVMT-R may not be psychometrically appropriate, sufficiently sensitive, nor substitutable for standalone PVTs in this population. Clinical neuropsychologists who evaluate such patients are encouraged to include standalone PVTs in their assessment batteries to ensure that clinical care conclusions drawn from neuropsychological data are valid.

Critical Item Analysis Enhances the Classification Accuracy of the Logical Memory Recognition Trial as a Performance Validity Indicator

OBJECTIVE

: Replicate previous research on Logical Memory Recognition (LM_Recog) and perform a critical item analysis.

METHOD

: Performance validity was psychometrically operationalized in a mixed clinical sample of 213 adults. Classification of the LM_Recog and nine critical items (CR-9) was computed.

RESULTS

: LM_Recog ≤20 produced a good combination of sensitivity (.30-.35) and specificity (.89-.90). CR-9 ≥5 and ≥6 had comparable classification accuracy. CR-9 ≥5 increased sensitivity by 4% over LM_Recog ≤20; CR-9 ≥6 increased specificity by 6-8% over LM_Recog ≤20; CR-9 ≥7 increased specificity by 8-15%.

CONCLUSIONS

: Critical item analysis enhances the classification accuracy of the optimal LM_Recog cutoff (≤20).

Five shades of gray: Conceptual and methodological issues around multivariate models of performance validity

OBJECTIVE

This study was designed to empirically investigate the signal detection profile of various multivariate models of performance validity tests (MV-PVTs) and explore several contested assumptions underlying validity assessment in general and MV-PVTs specifically.

METHOD

Archival data were collected from 167 patients (52.4%male; MAge = 39.7) clinicially evaluated subsequent to a TBI. Performance validity was psychometrically defined using two free-standing PVTs and five composite measures, each based on five embedded PVTs.

RESULTS

MV-PVTs had superior classification accuracy compared to univariate cutoffs. The similarity between predictor and criterion PVTs influenced signal detection profiles. False positive rates (FPR) in MV-PVTs can be effectively controlled using more stringent multivariate cutoffs. In addition to Pass and Fail, Borderline is a legitimate third outcome of performance validity assessment. Failing memory-based PVTs was associated with elevated self-reported psychiatric symptoms.

CONCLUSIONS

Concerns about elevated FPR in MV-PVTs are unsubstantiated. In fact, MV-PVTs are psychometrically superior to individual components. Instrumentation artifacts are endemic to PVTs, and represent both a threat and an opportunity during the interpretation of a given neurocognitive profile. There is no such thing as too much information in performance validity assessment. Psychometric issues should be evaluated based on empirical, not theoretical models. As the number/severity of embedded PVT failures accumulates, assessors must consider the possibility of non-credible presentation and its clinical implications to neurorehabilitation.

Performance validity assessment using response time on the Warrington Recognition Memory Test

OBJECTIVE

The present study tested the incremental utility of response time (RT) on the Warrington Recognition Memory Test - Words (RMT-W) in classifying bona fide versus feigned TBI.

METHOD

Participants were 173 adults: 55 with moderate to severe TBI, 69 healthy comparisons (HC) instructed to perform their best, and 49 healthy adults coached to simulate TBI (SIM). Participants completed a computerized version of the RMT-W in the context of a comprehensive neuropsychological battery. Groups were compared on RT indices including mean RT (overall, correct trials, incorrect trials) and variability, as well as the traditional RMT-W accuracy score.

RESULTS

Several RT indices differed significantly across groups, although RMT-W accuracy predicted group membership more strongly than any individual RT index. SIM showed longer average RT than both TBI and HC. RT variability and RT for incorrect trials distinguished SIM-HC but not SIM-TBI comparisons. In general, results for SIM-TBI comparisons were weaker than SIM-HC results. For SIM-HC comparisons, classification accuracy was excellent for all multivariable models incorporating RMT-W accuracy with one of the RT indices. For SIM-TBI comparisons, classification accuracies for multivariable models ranged from acceptable to excellent discriminability. In addition to mean RT and RT on correct trials, the ratio of RT on correct items to incorrect items showed incremental predictive value to accuracy.

CONCLUSION

Findings support the growing body of research supporting the value of combining RT with PVTs in discriminating between verified and feigned TBI. The diagnostic accuracy of the RMT-W can be improved by incorporating RT.

This will only take a minute: Time cutoffs are superior to accuracy cutoffs on the forced choice recognition trial of the Hopkins Verbal Learning Test - Revised

OBJECTIVE

This study was designed to evaluate the classification accuracy of the recently introduced forced-choice recognition trial to the Hopkins Verbal Learning Test - Revised (FCR_HVLT-R) as a performance validity test (PVT) in a clinical sample. Time-to-completion (T2C) for FCR_HVLT-R was also examined.

METHOD

Forty-three students were assigned to either the control or the experimental malingering (expMAL) condition. Archival data were collected from 52 adults clinically referred for neuropsychological assessment. Invalid performance was defined using expMAL status, two free-standing PVTs and two validity composites.

RESULTS

Among students, FCR_HVLT-R ≤11 or T2C ≥45 seconds was specific (0.86-0.93) to invalid performance. Among patients, an FCR_HVLT-R ≤11 was specific (0.94-1.00), but relatively insensitive (0.38-0.60) to non-credible responding0. T2C ≥35 s produced notably higher sensitivity (0.71-0.89), but variable specificity (0.83-0.96). The T2C achieved superior overall correct classification (81-86%) compared to the accuracy score (68-77%). The FCR_HVLT-R provided incremental utility in performance validity assessment compared to previously introduced validity cutoffs on Recognition Discrimination.

CONCLUSIONS

Combined with T2C, the FCR_HVLT-R has the potential to function as a quick, inexpensive and effective embedded PVT. The time-cutoff effectively attenuated the low ceiling of the accuracy scores, increasing sensitivity by 19%. Replication in larger and more geographically and demographically diverse samples is needed before the FCR_HVLT-R can be endorsed for routine clinical application.

Stronger Together: The Wechsler Adult Intelligence Scale-Fourth Edition as a Multivariate Performance Validity Test in Patients with Traumatic Brain Injury

OBJECTIVE

This study was designed to evaluate the classification accuracy of a multivariate model of performance validity assessment using embedded validity indicators (EVIs) within the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV).

METHOD

Archival data were collected from 100 adults with traumatic brain injury (TBI) consecutively referred for neuropsychological assessment in a clinical setting. The classification accuracy of previously published individual EVIs nested within the WAIS-IV and a composite measure based on six independent EVIs were evaluated against psychometrically defined non-credible performance.

RESULTS

Univariate validity cutoffs based on age-corrected scaled scores on Coding, Symbol Search, Digit Span, Letter-Number-Sequencing, Vocabulary minus Digit Span, and Coding minus Symbol Search were strong predictors of psychometrically defined non-credible responding. Failing ≥3 of these six EVIs at the liberal cutoff improved specificity (.91-.95) over univariate cutoffs (.78-.93). Conversely, failing ≥2 EVIs at the more conservative cutoff increased and stabilized sensitivity (.43-.67) compared to univariate cutoffs (.11-.63) while maintaining consistently high specificity (.93-.95).

CONCLUSIONS

In addition to being a widely used test of cognitive functioning, the WAIS-IV can also function as a measure of performance validity. Consistent with previous research, combining information from multiple EVIs enhanced the classification accuracy of individual cutoffs and provided more stable parameter estimates. If the current findings are replicated in larger, diagnostically and demographically heterogeneous samples, the WAIS-IV has the potential to become a powerful multivariate model of performance validity assessment.

BRIEF SUMMARY

Using a combination of multiple performance validity indicators embedded within the subtests of theWechsler Adult Intelligence Scale, the credibility of the response set can be establishedwith a high level of confidence. Multivariatemodels improve classification accuracy over individual tests. Relying on existing test data is a cost-effective approach to performance validity assessment.

Short-term effects of cannabis consumption on cognitive performance in medical cannabis patients

This observational study examined the acute cognitive effects of cannabis. We hypothesized that cognitive performance would be negatively affected by acute cannabis intoxication. Twenty-two medical cannabis patients from Southwestern Ontario completed the study. The majority (n = 13) were male. Mean age was 36.0 years, and mean level of education was 13.7 years. Participants were administered the same brief neurocognitive battery three times during a six-hour period: at baseline ("Baseline"), once after they consumed a 20% THC cannabis product ("THC"), and once again several hours later ("Recovery"). The average self-reported level of cannabis intoxication prior to the second assessment (i.e., during THC) was 5.1 out of 10. Contrary to expectations, performance on neuropsychological tests remained stable or even improved during the acute intoxication stage (THC; d: .49-.65, medium effect), and continued to increase during Recovery (d: .45-.77, medium-large effect). Interestingly, the failure rate on performance validity indicators increased during THC. Contrary to our hypothesis, there was no psychometric evidence for a decline in cognitive ability following THC intoxication. There are several possible explanations for this finding but, in the absence of a control group, no definitive conclusion can be reached at this time.