Effort Indicators within the California Verbal Learning Test-II (CVLT-II)

Standalone Performance Validity Tests May Be Differentially Related to Measures of Working Memory, Processing Speed, and Verbal Memory in Patients With Multiple Sclerosis

Cognitive functioning may account for minimal levels (i.e., 5%-14%) of variance of performance validity test (PVT) scores in clinical examinees. The present study extended this research twofold: (a) by determining the variance cognitive functioning explains within three distinct PVTs (b) in a sample of patients with multiple sclerosis (pwMS). Seventy-five pwMS (Mage = 48.50, 70.6% female, 80.9% White) completed the Victoria Symptom Validity Test (VSVT), Word Choice Test (WCT), Dot Counting Test (DCT), and three objective measures of working memory, processing speed, and verbal memory as part of clinical neuropsychological assessment. Regression analyses in credible groups (ns ranged from 54 to 63) indicated that cognitive functioning explained 24% to 38% of the variance in logarithmically transformed PVT variables. Variance from cognitive testing differed across PVTs: verbal memory significantly influenced both VSVT and WCT scores; working memory influenced VSVT and DCT scores; and processing speed influenced DCT scores. The WCT appeared least related to cognitive functioning of the included PVTs. Alternative plausible explanations, including the apparent domain/modality specificity hypothesis of PVTs versus the potential sensitivity of these PVTs to neurocognitive dysfunction in pwMS were discussed. Continued psychometric investigations into factors affecting performance validity, especially in multiple sclerosis, are warranted.

Replicating a Meta-Analysis: The Search for the Optimal Word Choice Test Cutoff Continues

This study was designed to expand on a recent meta-analysis that identified ≤42 as the optimal cutoff on the Word Choice Test (WCT). We examined the base rate of failure and the classification accuracy of various WCT cutoffs in four independent clinical samples (N = 252) against various psychometrically defined criterion groups. WCT ≤ 47 achieved acceptable combinations of specificity (.86-.89) at .49 to .54 sensitivity. Lowering the cutoff to ≤45 improved specificity (.91-.98) at a reasonable cost to sensitivity (.39-.50). Making the cutoff even more conservative (≤42) disproportionately sacrificed sensitivity (.30-.38) for specificity (.98-1.00), while still classifying 26.7% of patients with genuine and severe deficits as non-credible. Critical item (.23-.45 sensitivity at .89-1.00 specificity) and time-to-completion cutoffs (.48-.71 sensitivity at .87-.96 specificity) were effective alternative/complementary detection methods. Although WCT ≤ 45 produced the best overall classification accuracy, scores in the 43 to 47 range provide comparable objective psychometric evidence of non-credible responding. Results question the need for designating a single cutoff as "optimal," given the heterogeneity of signal detection environments in which individual assessors operate. As meta-analyses often fail to replicate, ongoing research is needed on the classification accuracy of various WCT cutoffs.

Review of Statistical and Methodological Issues in the Forensic Prediction of Malingering from Validity Tests: Part I: Statistical Issues

Forensic neuropsychological examinations with determination of malingering have tremendous social, legal, and economic consequences. Thousands of studies have been published aimed at developing and validating methods to diagnose malingering in forensic settings, based largely on approximately 50 validity tests, including embedded and stand-alone performance validity tests. This is the first part of a two-part review. Part I explores three statistical issues related to the validation of validity tests as predictors of malingering, including (a) the need to report a complete set of classification accuracy statistics, (b) how to detect and handle collinearity among validity tests, and (c) how to assess the classification accuracy of algorithms for aggregating information from multiple validity tests. In the Part II companion paper, three closely related research methodological issues will be examined. Statistical issues are explored through conceptual analysis, statistical simulations, and through reanalysis of findings from prior validation studies. Findings suggest extant neuropsychological validity tests are collinear and contribute redundant information to the prediction of malingering among forensic examinees. Findings further suggest that existing diagnostic algorithms may miss diagnostic accuracy targets under most realistic conditions. The review makes several recommendations to address these concerns, including (a) reporting of full confusion table statistics with 95% confidence intervals in diagnostic trials, (b) the use of logistic regression, and (c) adoption of the consensus model on the "transparent reporting of multivariate prediction models for individual prognosis or diagnosis" (TRIPOD) in the malingering literature.

Logical memory, visual reproduction, and verbal paired associates are effective embedded validity indicators in patients with traumatic brain injury

OBJECTIVE

This study was design to evaluate the potential of the recognition trials for the Logical Memory (LM), Visual Reproduction (VR), and Verbal Paired Associates (VPA) subtests of the Wechsler Memory Scales-Fourth Edition (WMS-IV) to serve as embedded performance validity tests (PVTs).

METHOD

The classification accuracy of the three WMS-IV subtests was computed against three different criterion PVTs in a sample of 103 adults with traumatic brain injury (TBI).

RESULTS

The optimal cutoffs (LM ≤ 20, VR ≤ 3, VPA ≤ 36) produced good combinations of sensitivity (.33-.87) and specificity (.92-.98). An age-corrected scaled score of ≤5 on either of the free recall trials on the VPA was specific (.91-.92) and relatively sensitive (.48-.57) to psychometrically defined invalid performance. A VR I ≤ 5 or VR II ≤ 4 had comparable specificity, but lower sensitivity (.25-.42). There was no difference in failure rate as a function of TBI severity.

CONCLUSIONS

In addition to LM, VR, and VPA can also function as embedded PVTs. Failing validity cutoffs on these subtests signals an increased risk of non-credible presentation and is robust to genuine neurocognitive impairment. However, they should not be used in isolation to determine the validity of an overall neurocognitive profile.

Using the Grooved Pegboard Test as an Embedded Validity Indicator in a Mixed Neuropsychiatric Sample with Varying Cognitive Impairment: Cross-Validation Problems

Embedded validity indicators (EVIs) derived from motor tests have received less empirical attention than those derived from tests of other neuropsychological abilities, particularly memory. Preliminary evidence suggests that the Grooved Pegboard Test (GPB) may function as an EVI, but existing studies were largely conducted using simulators and population samples without cognitive impairment. In this study we aimed to evaluate the GPB's classification accuracy as an EVI among a mixed clinical neuropsychiatric sample with and without cognitive impairment. This cross-sectional study comprised 223 patients clinically referred for neuropsychological testing. GPB raw and T-scores for both dominant and nondominant hands were examined as EVIs. A known-groups design, based on ≤1 failure on a battery of validated, independent criterion PVTs, showed that GPB performance differed significantly by validity group. Within the valid group, receiver operating characteristic curve analyses revealed that only the dominant hand raw score displayed acceptable classification accuracy for detecting invalid performance (area under curve [AUC] = .72), with an optimal cut-score of ≥106 seconds (33% sensitivity/88% specificity). All other scores had marginally lower classification accuracy (AUCs = .65-.68) for differentiating valid from invalid performers. Therefore, the GPB demonstrated limited utility as an EVI in a clinical sample containing patients with bona fide cognitive impairment.

CVLT-3 response bias as an indicator of performance validity in a litigating population

This study examined the efficacy of CVLT-3 response bias (i.e., parametric and nonparametric response bias) indices in differentiating between a clinical sample with traumatic brain injury and a litigating sample with poor performance validity.

Participants included 106 individuals, divided into two groups: clinical group with TBI (n = 56) and a litigating group who demonstrated inadequate performance validity (n = 50), as measured by failure on at least two performance validity tests. Archival CVLT-II data was rescored utilizing the CVLT-3 scoring and normative data. Receiver operator characteristic (ROC) curve analysis was used to evaluate the diagnostic discriminability of the two response bias indices.

Both parametric and nonparametric bias indices showed acceptable levels of diagnostic discrimination: AUC = .791 for parametric response bias and AUC = .753 for nonparametric response bias.

Parametric response bias' discrimination was statistically superior to the nonparametric responses bias' discrimination. The CVLT-3 response bias score demonstrated good sensitivity and specificity when differentiating between individuals in a clinical sample with TBI and individuals in litigation who demonstrated inadequate performance validity.

Benefits and challenges of using logistic regression to assess neuropsychological performance validity: Evidence from a simulation study

Logistic regression (LR) is recognized as a promising method for making decisions about neuropsychological performance validity by integrating information across multiple measures. However, this method has yet to be widely adopted in clinical practice, likely because several open questions remain about its utility relative to simpler methods, its effectiveness across different clinical contexts, and its feasibility at sample sizes common in the field. The current study addresses these questions by assessing classification performance of logistic regression and alternative methods across an array of simulated data sets.

We simulated scores of valid and invalid performers on 6 tests designed to mimic the psychometric and distributional properties of real performance validity measures. Out-of-sample predictive performance of LR and a commonly used alternative ("vote counting") was assessed across different base rates, validity measure properties, and sample sizes.

LR improved classification accuracy by 2%-12% across simulation conditions, primarily by improving sensitivity. False positives and negatives can be further reduced when LR predictions are interpreted as continuous, rather than binary. LR made robust predictions at sample sizes feasible for neuropsychology research (N = 307) and when as few as 2 tests with good psychometric properties were used.

Although training and test data sets of at least several hundred individuals may be required to develop and evaluate LR models for use in clinical practice, LR promises to be an efficient and powerful tool for improving judgements about performance validity. We offer several recommendations for model development and LR interpretation in a clinical setting.

Validation of the Personality Assessment Inventory (PAI) scale of scales in a mixed clinical sample

Objective: This exploratory study examined the classification accuracy of three derived scales aimed at detecting cognitive response bias in neuropsychological samples. The derived scales are composed of existing scales from the Personality Assessment Inventory (PAI). A mixed clinical sample of consecutive outpatients referred for neuropsychological assessment at a large Midwestern academic medical center was utilized. Participants and Methods: Participants included 332 patients who completed study's embedded and free-standing performance validity tests (PVTs) and the PAI. PASS and FAIL groups were created based on PVT performance to evaluate the classification accuracy of the derived scales. Three new scales, Cognitive Bias Scale of Scales 1-3, (CB-SOS1-3) were derived by combining existing scales by either summing the scales together and dividing by the total number of scales summed, or by logistically deriving a variable from the contributions of several scales. Results: All of the newly derived scales significantly differentiated between PASS and FAIL groups. All of the derived SOS scales demonstrated acceptable classification accuracy (i.e. CB-SOS1 AUC = 0.72; CB-SOS2 AUC = 0.73; CB-SOS3 AUC = 0.75). Conclusions: This exploratory study demonstrates that attending to scale-level PAI data may be a promising area of research in improving prediction of PVT failure.

Assessing Negative Response Bias Using Self-Report Measures: New Articles, New Issues

In psychological injury and related forensic evaluations, two types of tests are commonly used to assess Negative Response Bias (NRB): Symptom Validity Tests (SVTs) and Performance Validity Tests (PVTs). SVTs assess the credibility of self-reported symptoms, whereas PVTs assess the credibility of observed performance on cognitive tasks. Compared to the large and ever-growing number of published PVTs, there are still relatively few validated self-report SVTs available to professionals for assessing symptom validity. In addition, while several studies have examined how to combine and integrate the results of multiple independent PVTs, there are few studies to date that have addressed the combination and integration of information obtained from multiple self-report SVTs. The Special Issue of Psychological Injury and Law introduced in this article aims to help fill these gaps in the literature by providing readers with detailed information about the convergent and incremental validity, strengths and weaknesses, and applicability of a number of selected measures of NRB under different conditions and in different assessment contexts. Each of the articles in this Special Issue focuses on a particular self-report SVT or set of SVTs and summarizes their conditions of use, strengths, weaknesses, and possible cut scores and relative hit rates. Here, we review the psychometric properties of the 19 selected SVTs and discuss their advantages and disadvantages. In addition, we make tentative proposals for the field to consider regarding the number of SVTs to be used in an assessment, the number of SVT failures required to invalidate test results, and the issue of redundancy when selecting multiple SVTs for an assessment.

Evidence for neuropsychological health disparities in Black Americans with HIV disease

OBJECTIVE

Black Americans are at high risk for HIV disease and associated morbidity. The impact and clinical correlates of HIV-associated neurocognitive impairment among Black Americans is not fully understood. The current study uses a full factorial design to examine the independent and combined effects of race and HIV disease on neurocognitive functioning, including its associations with everyday functioning and clinical disease markers in Black and White persons with HIV (PWH).

METHOD

Participants included 40 Black PWH, 83 White PWH, 28 Black HIV- and 64 White HIV- individuals. Neurocognition was measured by raw sample-based z-scores from a clinical battery. Everyday functioning was assessed using self- and clinician-rated measures of cognitive symptoms and activities of daily living. HIV-associated neurocognitive disorders were also classified using demographically adjusted normative standards and the Frascati criteria.

RESULTS

We observed a significant three-way interaction between HIV, race, and domain on raw neurocognitive z-scores. This omnibus effect was driven by medium and large effect size decrements in processing speed and semantic memory, respectively, in Black PWH compared to other study groups. Black PWH also demonstrated higher frequencies of HIV-associated neurocognitive disorders as compared to White PWH. Unexpectedly, global neurocognitive performance was negatively related to everyday functioning impairments for White PWH, but not for Black PWH.

CONCLUSIONS

Systemic disadvantages for Black Americans may combine with HIV disease to compound some neurocognitive impairments in this under-served population. Prospective studies are needed to identify better ways to prevent, measure, diagnose, and manage HIV-associated neurocognitive disorders among Black Americans.

Critical Item Analysis Enhances the Classification Accuracy of the Logical Memory Recognition Trial as a Performance Validity Indicator

OBJECTIVE

: Replicate previous research on Logical Memory Recognition (LM_Recog) and perform a critical item analysis.

METHOD

: Performance validity was psychometrically operationalized in a mixed clinical sample of 213 adults. Classification of the LM_Recog and nine critical items (CR-9) was computed.

RESULTS

: LM_Recog ≤20 produced a good combination of sensitivity (.30-.35) and specificity (.89-.90). CR-9 ≥5 and ≥6 had comparable classification accuracy. CR-9 ≥5 increased sensitivity by 4% over LM_Recog ≤20; CR-9 ≥6 increased specificity by 6-8% over LM_Recog ≤20; CR-9 ≥7 increased specificity by 8-15%.

CONCLUSIONS

: Critical item analysis enhances the classification accuracy of the optimal LM_Recog cutoff (≤20).

Neuropsychological deficits in Posterior Cortical Atrophy and typical Alzheimer's disease: A meta-analytic review

AIMS

To identify cognitive tests that best differentiate between Posterior Cortical Atrophy (PCA) and typical Alzheimer's Disease (tAD), as well as PCA and healthy control (HC) participants.

METHOD

Medline, PsycInfo and Web of Science were systematically searched using terms related to PCA, tAD, and cognitive testing. Seventeen studies were identified, including 441 PCA, 391 tAD, and 284 HC participants. Standardised effect sizes of mean scores were calculated to measure performance differences on cognitive tests for PCA versus tAD and PCA versus HC groups. Meta-analyses used a random effects model.

RESULTS

The most discriminating cognitive tests for PCA and tAD presentations were measures of visuospatial function and verbal memory. Large, significant effect sizes were produced for all measures of visuospatial function, most notably for Rey-Osterrieth Copy (Hedges' g = -2.79), VOSP Fragmented letters (Hedges' g = -1.73), VOSP Dot Counting (Hedges' g = -1.74), and VOSP Cube Analysis (Hedges' g = -1.98). For measures of verbal memory, the RAVLT delay and Digit Span Backwards produced significant medium effects (Hedges' g = .62 and -.56, respectively).

CONCLUSION

Establishing a common framework for testing individuals with PCA has important implications for diagnosis and treatment, and forms a practical objective for future research. Findings from this meta-analysis suggest that measures of visuospatial function and verbal memory would form an important part of this framework.

Five shades of gray: Conceptual and methodological issues around multivariate models of performance validity

OBJECTIVE

This study was designed to empirically investigate the signal detection profile of various multivariate models of performance validity tests (MV-PVTs) and explore several contested assumptions underlying validity assessment in general and MV-PVTs specifically.

METHOD

Archival data were collected from 167 patients (52.4%male; MAge = 39.7) clinicially evaluated subsequent to a TBI. Performance validity was psychometrically defined using two free-standing PVTs and five composite measures, each based on five embedded PVTs.

RESULTS

MV-PVTs had superior classification accuracy compared to univariate cutoffs. The similarity between predictor and criterion PVTs influenced signal detection profiles. False positive rates (FPR) in MV-PVTs can be effectively controlled using more stringent multivariate cutoffs. In addition to Pass and Fail, Borderline is a legitimate third outcome of performance validity assessment. Failing memory-based PVTs was associated with elevated self-reported psychiatric symptoms.

CONCLUSIONS

Concerns about elevated FPR in MV-PVTs are unsubstantiated. In fact, MV-PVTs are psychometrically superior to individual components. Instrumentation artifacts are endemic to PVTs, and represent both a threat and an opportunity during the interpretation of a given neurocognitive profile. There is no such thing as too much information in performance validity assessment. Psychometric issues should be evaluated based on empirical, not theoretical models. As the number/severity of embedded PVT failures accumulates, assessors must consider the possibility of non-credible presentation and its clinical implications to neurorehabilitation.

Stronger Together: The Wechsler Adult Intelligence Scale-Fourth Edition as a Multivariate Performance Validity Test in Patients with Traumatic Brain Injury

OBJECTIVE

This study was designed to evaluate the classification accuracy of a multivariate model of performance validity assessment using embedded validity indicators (EVIs) within the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV).

METHOD

Archival data were collected from 100 adults with traumatic brain injury (TBI) consecutively referred for neuropsychological assessment in a clinical setting. The classification accuracy of previously published individual EVIs nested within the WAIS-IV and a composite measure based on six independent EVIs were evaluated against psychometrically defined non-credible performance.

RESULTS

Univariate validity cutoffs based on age-corrected scaled scores on Coding, Symbol Search, Digit Span, Letter-Number-Sequencing, Vocabulary minus Digit Span, and Coding minus Symbol Search were strong predictors of psychometrically defined non-credible responding. Failing ≥3 of these six EVIs at the liberal cutoff improved specificity (.91-.95) over univariate cutoffs (.78-.93). Conversely, failing ≥2 EVIs at the more conservative cutoff increased and stabilized sensitivity (.43-.67) compared to univariate cutoffs (.11-.63) while maintaining consistently high specificity (.93-.95).

CONCLUSIONS

In addition to being a widely used test of cognitive functioning, the WAIS-IV can also function as a measure of performance validity. Consistent with previous research, combining information from multiple EVIs enhanced the classification accuracy of individual cutoffs and provided more stable parameter estimates. If the current findings are replicated in larger, diagnostically and demographically heterogeneous samples, the WAIS-IV has the potential to become a powerful multivariate model of performance validity assessment.

BRIEF SUMMARY

Using a combination of multiple performance validity indicators embedded within the subtests of theWechsler Adult Intelligence Scale, the credibility of the response set can be establishedwith a high level of confidence. Multivariatemodels improve classification accuracy over individual tests. Relying on existing test data is a cost-effective approach to performance validity assessment.

Verbal memory measurement towards digital perspectives in first-episode psychosis: A review

BACKGROUND

Even in the early phases of psychotic spectrum illnesses such as schizophrenia, patients can experience cognitive decline or deficits prior to the onset of psychotic symptoms such as delusions and hallucinations. In this systematic review, we assessed which verbal memory assessments are most widely used in first-episode psychosis and may be applied via digital technologies (smartphone applications, etc.) for use in early detection.

METHODS

In November 2019, we searched for studies measuring verbal memory in first episode psychosis or schizophrenia over the past 10 years on PubMed and PsycINFO. We screened abstracts of these studies and excluded review studies. Full-texts of included studies were used to identify the verbal memory measurement tests, follow-up frequencies, and sample sizes.

RESULTS

We screened 233 reports and found that 120 original research studies measured verbal memory in first episode psychosis over the past 10 years. Four of these studies specified using a computer, 24 (20%) used a paper-pen format, 1(1%) used both, and 91 (76%) studies did not specify their administration tools or suggest there were offered in digital formats. Thirty-five (30%) studies had follow-up measurements of verbal memory, while 85 (70%) had only a single verbal memory measurement.

DISCUSSION

While many scales are commonly used to measure verbal memory in first episode psychosis, they are not often administered via digital technology. There is an emerging opportunity to administer these and other tests via digital technologies for expanding access to early detection of cognitive decline in clinical high risk and first-episode psychosis.

Refined Measurement of Verbal Learning and Memory: Application of Item Response Theory to California Verbal Learning Test - Second Edition (CVLT-II) Learning Trials

OBJECTIVE

This study was conducted to investigate how items contained within the California Verbal Learning Test - Second Edition (CVLT-II; Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (2000). California Verbal Learning Test (2nd ed.). San Antonio, TX: Psychological Corporation) learning trials function and to evaluate whether weighted scoring approaches might improve quantification of verbal memory.

METHOD

Archival data from 577 individuals (338 healthy young adults, 239 medical or psychiatric patients referred to a neuropsychology clinic) were obtained and evaluated using item response theory.

RESULTS

The serial position effect was evident across trials, but was most evident in Trials 1, 2, and 3. CVLT-II Trial 5 was the most effective among the five learning trials in quantifying verbal memory, although it was most effective when measuring lower memory ability levels. In contrast, CVLT-II Trial 1 items had, on average, the highest difficulty levels. Various weighted scoring approaches did not appear incrementally helpful in improving prediction of memory performance.

CONCLUSION

Specific items and trials differentially discriminate between examinees with low levels of memory ability; it is important to thoroughly evaluate item properties of tests used in clinical decision-making.

Clinical utility of WAIS-IV 'excessive decline from premorbid functioning' scores to detect invalid test performance following traumatic brain injury

Objective: Excessive Decline from Premorbid Functioning (EDPF), an atypical discrepancy between demographically predicted and obtained Wechsler Adult Intelligence Scale-4th Edition (WAIS-IV) scores, has been recently proposed as a potential embedded performance validity test (PVT). This study examined the clinical utility of EDPF scores to detect invalid test performance following traumatic brain injury (TBI).Methods: Participants were 194 U.S. military service members who completed neuropsychological testing on average 2.4 years (SD = 4.0) following uncomplicated mild, complicated mild, moderate, severe, or penetrating TBI (Age: M = 34.0, SD = 9.9). Using TBI severity and PVT performance (i.e., PVT Pass/Fail), participants were classified into three groups: Uncomplicated Mild TBI-PVT Fail (MTBI-Fail; n = 21), Uncomplicated Mild TBI-PVT Pass (MTBI-Pass; n = 94), and Complicated Mild to Severe/Penetrating TBI-PVT Pass (CM/STBI-Pass; n = 79). Seven EDPF measures were calculated by subtracting WAIS-IV obtained index scores from the demographically predicted scores from the Test of Premorbid Functioning (TOPF). Cutoff scores to detect invalid test performance were examined for each EDPF measure separately.Results: The MTBI-Fail group had higher scores than the MTBI-Pass and CM/STBI-Pass groups on five of the seven EDPF measures (p<.05). Overall, the EDPF measure using the Processing Speed Index (EDPF-PSI) was the most useful score to detect invalid test performance. However, sensitivity was only low to moderate depending on the cutoff score used.Conclusions: These findings provide support for the use of EDPF as an embedded PVT to be considered along with other performance validity data when administering the WAIS-IV.

Impact of criterion measures on the classification accuracy of TOMM-1

This study was designed to examine the effect of various criterion measures on the classification accuracy of Trial 1 of the Test of Memory Malingering (TOMM-1), a free-standing performance validity test (PVT). Archival data were collected from a case sequence of 91 (M _Age = 42.2 years; M _Education = 12.7) patients clinically referred for neuropsychological assessment. Trials 2 and Retention of the TOMM, the Word Choice Test, and three validity composites were used as criterion PVTs. Classification accuracy varied systematically as a function of criterion PVT. TOMM-1 ≤ 43 emerged as the optimal cutoff, resulting in a wide range of sensitivity (.47-1.00), with perfect overall specificity. Failing the TOMM-1 was unrelated to age, education or gender, but was associated with elevated self-reported depression. Results support the utility of TOMM-1 as an independent, free-standing, single-trial PVT. Consistent with previous reports, the choice of criterion measure influences parameter estimates of the PVT being calibrated. The methodological implications of modality specificity to PVT research and clinical/forensic practice should be considered when evaluating cutoffs or interpreting scores in the failing range.

Utility of the Personality Assessment Inventory in detecting feigned Attention-Deficit/Hyperactivity Disorder (ADHD): The Feigned Adult ADHD index

Objective

The high potential for secondary gain among college students presenting for Attention-Deficit/Hyperactivity Disorder (ADHD) evaluations highlights the need for psychometrically sound embedded validity indicators. The purpose of this study was to develop new validity indicators specific to feigned ADHD for the Personality Assessment Inventory (PAI) and compare them to preexisting imbedded PAI validity measures.

Method

PAI scales that were theoretically related to feigned ADHD were evaluated. A binomial (ADHD simulators, n = 138, and genuine ADHD, n = 142) logistic regression was conducted with selected PAI scales and subscales. Classification rates were compared between the new and existing validity scales. A similar approach was used for item-level data in a second study in a subgroup of the original sample.

Results

The derived PAI scale-based logistic regression had a sensitivity of 54% and specificity of 92%. This algorithm accurately identified 97% of healthy controls as not feigning ADHD and correctly identified 98% of a no diagnosis group and 75% of a mood/anxiety disorders group. Classification accuracy of the new index was superior to the majority of existing PAI validity scales across groups. An item-level PAI algorithm had a sensitivity of 85% and specificity of 97% for identifying feigned ADHD.

Conclusions

New validity measures were compared to existing PAI validity indicators and performed better than many of them in this study. The algorithms developed in this study of ADHD simulators and genuine ADHD cases have adequate sensitivity and good specificity and appear to function differently than other PAI symptom validity scales.

Sleep Biomarkers, Health Comorbidities, and Neurocognition in Obstructive Sleep Apnea

OBJECTIVES

Obstructive sleep apnea (OSA) is associated with cognitive impairment but the relationships between specific biomarkers and neurocognitive domains remain unclear. The present study examined the influence of common health comorbidities on these relationships. Adults with suspected OSA (N=60; 53% male; M age=52 years; SD=14) underwent neuropsychological evaluation before baseline polysomnography (PSG). Apneic syndrome severity, hypoxic strain, and sleep architecture disturbance were assessed through PSG.

METHODS

Depression (Center for Epidemiological Studies Depression Scale, CESD), pain, and medical comorbidity (Charlson Comorbidity Index) were measured via questionnaires. Processing speed, attention, vigilance, memory, executive functioning, and motor dexterity were evaluated with cognitive testing. A winnowing approach identified 9 potential moderation models comprised of a correlated PSG variable, comorbid health factor, and cognitive performance.

RESULTS

Regression analyses identified one significant moderation model: average blood oxygen saturation (AVO2) and depression predicting recall memory, accounting for 31% of the performance variance, p<.001. Depression was a significant predictor of recall memory, p<.001, but AVO2 was not a significant predictor. The interaction between depression and AVO2 was significant, accounting for an additional 10% of the variance, p<.001. The relationship between low AVO2 and low recall memory performance emerged when depression severity ratings approached a previously established clinical cutoff score (CESD=16).

CONCLUSIONS

This study examined sleep biomarkers with specific neurocognitive functions among individuals with suspected OSA. Findings revealed that depression burden uniquely influence this pathophysiological relationship, which may aid clinical management. (JINS, 2018, 28, 864-875).

Illness Perceptions Predict Cognitive Performance Validity

OBJECTIVES

The aim of this study was to investigate the relationship of psychological variables to cognitive performance validity test (PVT) results in mixed forensic and nonforensic clinical samples.

METHODS

Participants included 183 adults who underwent comprehensive neuropsychological examination. Criterion groups were formed, that is, Credible Group or Noncredible Group, based upon their performance on the Word Memory Test and other stand-alone and embedded PVT measures.

RESULTS

Multivariate logistic regression analysis identified three significant predictors of cognitive performance validity. These included two psychological constructs, for example, Cogniphobia (perception that cognitive effort will exacerbate neurological symptoms), and Symptom Identity (perception that current symptoms are the result of illness or injury), and one contextual factor (forensic). While there was no interaction between these factors, elevated scores were most often observed in the forensic sample, suggesting that these independently contributing intrinsic psychological factors are more likely to occur in a forensic environment.

CONCLUSIONS

Illness perceptions were significant predictors of cognitive performance validity particularly when they reached very elevated levels. Extreme elevations were more common among participants in the forensic sample, and potential reasons for this pattern are explored. (JINS, 2018, 24, 735-745).

White Matter Associations With Performance Validity Testing in Veterans With Mild Traumatic Brain Injury: The Utility of Biomarkers in Complicated Assessment

OBJECTIVE

Failure on performance validity tests (PVTs) is common in Veterans with histories of mild traumatic brain injury (mTBI), leading to questionable validity of clinical presentations.

PARTICIPANTS

Using diffusion tensor imaging, we investigated white matter (WM) integrity and cognition in 79 Veterans with history of mTBI who passed PVTs (n = 43; traumatic brain injury [TBI]-passed), history of mTBI who failed at least 1 PVT (n = 13; TBI-failed), and military controls (n = 23; MCs) with no history of TBI.

RESULTS

The TBI-failed group demonstrated significantly lower cognitive scores relative to MCs and the TBI-passed group; however, no such differences were observed between MCs and the TBI-passed group. On a global measure of WM integrity (ie, WM burden), the TBI-failed group showed more overall WM abnormalities than the other groups. However, no differences were observed between the MCs and TBI-passed group on WM burden. Interestingly, regional WM analyses revealed abnormalities in the anterior internal capsule and cingulum of both TBI subgroups relative to MCs. Moreover, compared with the TBI-passed group, the TBI-failed group demonstrated significantly decreased WM integrity in the corpus callosum.

CONCLUSIONS

Findings revealed that, within our sample, WM abnormalities are evident in those who fail PVTs. This study adds to the burgeoning PVT literature by suggesting that poor PVT performance does not negate the possibility of underlying WM abnormalities in military personnel with history of mTBI.

Aggregating validity indicators: The salience of domain specificity and the indeterminate range in multivariate models of performance validity assessment

This study was designed to examine the "domain specificity" hypothesis in performance validity tests (PVTs) and the epistemological status of an "indeterminate range" when evaluating the credibility of a neuropsychological profile using a multivariate model of performance validity assessment. While previous research suggests that aggregating PVTs produces superior classification accuracy compared to individual instruments, the effect of the congruence between the criterion and predictor variable on signal detection and the issue of classifying borderline cases remain understudied. Data from a mixed clinical sample of 234 adults referred for cognitive evaluation (M_Age = 46.6; M_Education = 13.5) were collected. Two validity composites were created: one based on five verbal PVTs (EI-5_VER) and one based on five nonverbal PVTs (EI-5_NV) and compared against several other PVTs. Overall, language-based tests of cognitive ability were more sensitive to elevations on the EI-5_VER compared to visual-perceptual tests; whereas, the opposite was observed with the EI-5_NV. However, the match between predictor and criterion variable had a more complex relationship with classification accuracy, suggesting the confluence of multiple factors (sensory modality, cognitive domain, testing paradigm). An "indeterminate range" of performance validity emerged that was distinctly different from both the Pass and the Fail group. Trichotomized criterion PVTs (Pass-Borderline-Fail) had a negative linear relationship with performance on tests of cognitive ability, providing further support for an "in-between" category separating the unequivocal Pass and unequivocal Fail classification range. The choice of criterion variable can influence classification accuracy in PVT research. Establishing a Borderline range between Pass and Fail more accurately reflected the distribution of scores on multiple PVTs. The traditional binary classification system imposes an artificial dichotomy on PVTs that was not fully supported by the data. Accepting "indeterminate" as a legitimate third outcome of performance validity assessment has the potential to improve the clinical utility of PVTs and defuse debates regarding "near-Passes" and "soft Fails."

CVLT-II Forced Choice Recognition Trial as an Embedded Validity Indicator: A Systematic Review of the Evidence

OBJECTIVES

The Forced Choice Recognition (FCR) trial of the California Verbal Learning Test, 2nd edition, was designed as an embedded performance validity test (PVT). To our knowledge, this is the first systematic review of classification accuracy against reference PVTs.

METHODS

Results from peer-reviewed studies with FCR data published since 2002 encompassing a variety of clinical, research, and forensic samples were summarized, including 37 studies with FCR failure rates (N=7575) and 17 with concordance rates with established PVTs (N=4432).

RESULTS

All healthy controls scored >14 on FCR. On average, 16.9% of the entire sample scored ≤14, while 25.9% failed reference PVTs. Presence or absence of external incentives to appear impaired (as identified by researchers) resulted in different failure rates (13.6% vs. 3.5%), as did failing or passing reference PVTs (49.0% vs. 6.4%). FCR ≤14 produced an overall classification accuracy of 72%, demonstrating higher specificity (.93) than sensitivity (.50) to invalid performance. Failure rates increased with the severity of cognitive impairment.

CONCLUSIONS

In the absence of serious neurocognitive disorder, FCR ≤14 is highly specific, but only moderately sensitive to invalid responding. Passing FCR does not rule out a non-credible presentation, but failing FCR rules it in with high accuracy. The heterogeneity in sample characteristics and reference PVTs, as well as the quality of the criterion measure across studies, is a major limitation of this review and the basic methodology of PVT research in general. (JINS, 2016, 22, 851-858).

Rates of Abnormally Low TOPF Word Reading Scores in Individuals Failing Versus Passing Performance Validity Testing

The present study examined the impact of performance validity test (PVT) failure on the Test of Premorbid Functioning (TOPF) in a sample of 252 neuropsychological patients. Word reading performance differed significantly according to PVT failure status, and number of PVTs failed accounted for 7.4% of the variance in word reading performance, even after controlling for education. Furthermore, individuals failing ≥2 PVTs were twice as likely as individuals passing all PVTs (33% vs. 16%) to have abnormally low obtained word reading scores relative to demographically predicted scores when using a normative base rate of 10% to define abnormality. When compared with standardization study clinical groups, those failing ≥2 PVTs were twice as likely as patients with moderate to severe traumatic brain injury and as likely as patients with Alzheimer’s dementia to obtain abnormally low TOPF word reading scores. Findings indicate that TOPF word reading based estimates of premorbid functioning should not be interpreted in individuals invalidating cognitive testing.

False-Positive Error Rates for Reliable Digit Span and Auditory Verbal Learning Test Performance Validity Measures in Amnestic Mild Cognitive Impairment and Early Alzheimer Disease

OBJECTIVE

The objective is to examine failure on three embedded performance validity tests [Reliable Digit Span (RDS), Auditory Verbal Learning Test (AVLT) logistic regression, and AVLT recognition memory] in early Alzheimer disease (AD; n = 178), amnestic mild cognitive impairment (MCI; n = 365), and cognitively intact age-matched controls (n = 206).

METHOD

Neuropsychological tests scores were obtained from subjects participating in the Alzheimer's Disease Neuroimaging Initiative (ADNI).

RESULTS

RDS failure using a ≤7 RDS threshold was 60/178 (34%) for early AD, 52/365 (14%) for MCI, and 17/206 (8%) for controls. A ≤6 RDS criterion reduced this rate to 24/178 (13%) for early AD, 15/365 (4%) for MCI, and 7/206 (3%) for controls. AVLT logistic regression probability of ≥.76 yielded unacceptably high false-positive rates in both clinical groups [early AD = 149/178 (79%); MCI = 159/365 (44%)] but not cognitively intact controls (13/206, 6%). AVLT recognition criterion of ≤9/15 classified 125/178 (70%) of early AD, 155/365 (42%) of MCI, and 18/206 (9%) of control scores as invalid, which decreased to 66/178 (37%) for early AD, 46/365 (13%) for MCI, and 10/206 (5%) for controls when applying a ≤5/15 criterion. Despite high false-positive rates across individual measures and thresholds, combining RDS ≤ 6 and AVLT recognition ≤9/15 classified only 9/178 (5%) of early AD and 4/365 (1%) of MCI patients as invalid performers.

CONCLUSIONS

Embedded validity cutoffs derived from mixed clinical groups produce unacceptably high false-positive rates in MCI and early AD. Combining embedded PVT indicators lowers the false-positive rate.