Victoria Symptom Validity Test: A Systematic Review and Cross-Validation Study

Standalone Performance Validity Tests May Be Differentially Related to Measures of Working Memory, Processing Speed, and Verbal Memory in Patients With Multiple Sclerosis

Cognitive functioning may account for minimal levels (i.e., 5%-14%) of variance of performance validity test (PVT) scores in clinical examinees. The present study extended this research twofold: (a) by determining the variance cognitive functioning explains within three distinct PVTs (b) in a sample of patients with multiple sclerosis (pwMS). Seventy-five pwMS (Mage = 48.50, 70.6% female, 80.9% White) completed the Victoria Symptom Validity Test (VSVT), Word Choice Test (WCT), Dot Counting Test (DCT), and three objective measures of working memory, processing speed, and verbal memory as part of clinical neuropsychological assessment. Regression analyses in credible groups (ns ranged from 54 to 63) indicated that cognitive functioning explained 24% to 38% of the variance in logarithmically transformed PVT variables. Variance from cognitive testing differed across PVTs: verbal memory significantly influenced both VSVT and WCT scores; working memory influenced VSVT and DCT scores; and processing speed influenced DCT scores. The WCT appeared least related to cognitive functioning of the included PVTs. Alternative plausible explanations, including the apparent domain/modality specificity hypothesis of PVTs versus the potential sensitivity of these PVTs to neurocognitive dysfunction in pwMS were discussed. Continued psychometric investigations into factors affecting performance validity, especially in multiple sclerosis, are warranted.

Utility of the Brief Assessment of Cognitive Health (BACH) computerized screening tool in identifying MS-related cognitive impairment

BACKGROUND

Current guidelines recommend that individuals with MS are screened annually for processing speed deficits, often using the Symbol Digit Modalities Test (SDMT). However, given the heterogeneity of cognitive deficits in individuals with MS, other screening measures that assess a range of cognitive domains are necessary. The current cross-sectional study aimed to examine the ability of the computerized, self-administered Brief Assessment of Cognitive Health (BACH) screening measure to detect the presence of cognitive impairment in adults with MS as determined by performance on a standard neuropsychological test battery.

METHODS

Seventy-two individuals with MS completed the BACH and a comprehensive neuropsychological test battery. Receiver operating characteristic (ROC) analyses were conducted to investigate the ability of the BACH to identify cognitively impaired and cognitively intact individuals. ROC analyses were also conducted to compare the ability of the SDMT to discriminate between cognitively intact and cognitively impaired groups as a comparison with the BACH.

RESULTS

Cognitive impairment was observed in 56 % of the sample. The BACH showed acceptable ability to discriminate between cognitively intact and cognitively impaired groups (AUC = 0.78). Additionally, the BACH was able to adequately predict cognitive impairment in domains other than processing speed (AUC = 0.71). The SDMT also demonstrated adequate utility in identifying individuals with cognitive impairment (AUC = 0.73); however, the SDMT was not able to adequately predict cognitive impairment in domains other than processing speed (AUC = 0.56).

CONCLUSION

The BACH showed adequate ability to detect cognitive impairment in individuals with MS. The BACH was able to identify impairments across various assessed cognitive domains, including individuals with and without processing speed deficits.

Cross-validation of the Trail Making Test as a non-memory-based embedded performance validity test among veterans with and without cognitive impairment

OBJECTIVE

This study cross-validated multiple Trail Making Test (TMT) Parts A and B scores as non-memory-based embedded performance validity tests (PVTs) for detecting invalid neuropsychological performance among veterans with and without cognitive impairment.

METHOD

Data were collected from a demographically and diagnostically diverse mixed clinical sample of 100 veterans undergoing outpatient neuropsychological evaluation at a Southwestern VA Medical Center. As part of a larger battery of neuropsychological tests, all veterans completed TMT A and B and four independent criterion PVTs, which were used to classify veterans into valid (n = 75) and invalid (n = 25) groups. Among the valid group 47% (n = 35) were cognitively impaired.

RESULTS

Among the overall sample, all embedded PVTs derived from TMT A and B raw and demographically corrected T-scores significantly differed between validity groups (ηp2 = .21-.31) with significant areas under the curve (AUCs) of .72-.78 and 32-48% sensitivity (≥91% specificity) at optimal cut-scores. When subdivided by cognitive impairment status (i.e., valid-unimpaired vs. invalid; valid-impaired vs. invalid), all TMT scores yielded significant AUCs of .80-.88 and 56%-72% sensitivity (≥90% specificity) at optimal cut-scores. Among veterans with cognitive impairment, neither TMT A or B raw scores were able to significantly differentiate the invalid from the valid-cognitively impaired group; however, demographically corrected T-scores were able to significantly differentiate groups but had poor classification accuracy (AUCs = .66-.68) and reduced sensitivity of 28%-44% (≥91% specificity).

CONCLUSIONS

Embedded PVTs derived from TMT Parts A and B raw and T-scores were able to accurately differentiate valid from invalid neuropsychological performance among veterans without cognitive impairment; however, the demographically corrected T-scores generally were more robust and consistent with prior studies compared to raw scores. By contrast, TMT embedded PVTs had poor accuracy and low sensitivity among veterans with cognitive impairment, suggesting limited utility as PVTs among populations with cognitive dysfunction.

CT image segmentation of meat sheep Loin based on deep learning

There are no clear boundaries between internal tissues in sheep Computerized Tomography images, and it is difficult for traditional methods to meet the requirements of image segmentation in application. Deep learning has shown excellent performance in image analysis. In this context, we investigated the Loin CT image segmentation of sheep based on deep learning models. The Fully Convolutional Neural Network (FCN) and 5 different UNet models were applied in image segmentation on the data set of 1471 CT images including the Loin part from 25 Australian White rams and Dolper rams using the method of 5-fold cross validation. After 10 independent runs, different evaluation metrics were applied to assess the performances of the models. All models showed excellent results in terms evaluation metrics. There were slight differences among the results from the six models, and Attention-UNet outperformed others methods with 0.998±0.009 in accuracy, 4.391±0.338 in AVER_HD, 0.90±0.012 in MIOU and 0.95±0.007 in DICE, respectively, while the optimal value of LOSS was 0.029±0.018 from Channel-UNet, and the running time of ResNet34-UNet is the shortest.

"It's on the tip of my tongue!" exploring confrontation naming difficulties in patients with multiple sclerosis

BACKGROUND

Naming difficulty is commonly reported by patients with multiple sclerosis (pwMS). Though many cognitive batteries recommended for pwMS include fluency tasks, they do not include naming tasks. The aim of this study was to examine the prevalence of naming impairment in pwMS by using a measure of confrontation naming and to identify correlates with neuroimaging.

METHODS

One-hundred-eighty-five pwMS (M_age = 48.75 ± 11.23) completed neuropsychological testing and fifty had brain MRI scans within one year of neuropsychological testing. Controlling for demographic variables, partial correlations and hierarchical regressions with language tests as the outcome variables and neuroimaging variables as predictors were performed.

RESULTS

Performance on language tasks ranged within low average to average, with impairment most frequently found on a measure of confrontation naming (Boston Naming Test [BNT];27.6%), followed by a measure of phonemic fluency (Controlled Oral Word Association Test [COWAT]; 24.3%) and semantic fluency (animals [AF]; 18.3%). In the subset of patients with neuroimaging, thalamic volume had the strongest relationship with language variables, followed by white matter volume and T2 lesion volume. Language variables had no association with fractional gray matter volume. Of the language measures, BNT demonstrated the strongest relationship with MRI variables, followed by AF. There were no significant associations between neuroimaging variables and COWAT. Regression results revealed that fractional thalamic volume significantly contributed to BNT scores after adjusting for demographics, while T2 lesion volume predicted AF and no neuroimaging variables emerged as predictors for COWAT after controlling for demographics.

CONCLUSIONS

Objective naming impairment is common in pwMS and are more strongly associated with neuroimaging of MS brain pathology than verbal fluency tasks that are commonly used in cognitive batteries for pwMS. Continued research on language (especially naming) deficits and neuroimaging correlates (particularly thalamic involvement) in pwMS is needed.

Cognitive Performance and Psychiatric Self-Reports Across Adult Cognitive Disengagement Syndrome and ADHD Diagnostic Groups

OBJECTIVE

Cognitive disengagement syndrome (CDS) is characterized by inattention, under-arousal, and fatigue and frequently co-occurs with attention-deficit/hyperactivity disorder (ADHD). Although CDS is associated with cognitive complaints, its association with objective cognitive performance is less well understood.

METHOD

This study investigated neuropsychological correlates of CDS symptoms among 169 adults (M_age = 29.4) referred for outpatient neuropsychological evaluation following inattention complaints. We evaluated cognitive and self-report differences across four high/low CDS and positive/negative ADHD groups, and cognitive and self-report correlates of CDS symptomology.

RESULTS

There were no differences in cognitive performance, significant differences in self-reported psychiatric symptoms (greater CDS symptomatology, impulsivity among the high CDS groups; greater inattention among the positive ADHD/high CDS groups; greater hyperactivity among the positive ADHD groups), significant intercorrelations within cognitive and self-report measures, nonsignificant correlations between cognitive measures and self-report measures.

CONCLUSION

Findings support prior work demonstrating weak to null associations between ADHD and CDS symptoms and cognitive performance among adults.

Pain Influences Neuropsychological Performance Following Electrical Injury: A Cross-Sectional Study

OBJECTIVE

Electrical injury (EI) is a significant, multifaceted trauma often with multi-domain cognitive sequelae, even when the expected current path does not pass through the brain. Chronic pain (CP) research suggests pain may affect cognition directly and indirectly by influencing emotional distress which then impacts cognitive functioning. As chronic pain may be critical to understanding EI-related cognitive difficulties, the aims of the current study were: examine the direct and indirect effects of pain on cognition following EI and compare the relationship between pain and cognition in EI and CP populations.

METHOD

This cross-sectional study used data from a clinical sample of 50 patients with EI (84.0% male; M_age = 43.7 years) administered standardized measures of pain (Pain Patient Profile), depression, and neurocognitive functioning. A CP comparison sample of 93 patients was also included.

RESULTS

Higher pain levels were associated with poorer attention/processing speed and executive functioning performance among patients with EI. Depression was significantly correlated with pain and mediated the relationship between pain and attention/processing speed in patients with EI. When comparing the patients with EI and CP, the relationship between pain and cognition was similar for both clinical groups.

CONCLUSIONS

Findings indicate that pain impacts mood and cognition in patients with EI, and the influence of pain and its effect on cognition should be considered in the assessment and treatment of patients who have experienced an electrical injury.

Failing The b Test: The influence of cutoff scores and criterion group approaches in a sample of adults referred for psychoeducational evaluation

OBJECTIVE

Previous research has shown that both criterion grouping approaches and cutoff scores can impact PVT classification accuracy statistics. This study aimed to examine the influence of cutoff scores and criterion grouping approaches on The b Test, a measure designed to identify feigned impairment in visual scanning, processing speed, and letter identification.

METHOD

Two hundred ninety-seven adults referred for psychoeducational testing were included with the majority of individuals seeking academic accommodations (n = 215). Cutoff scores of ≥82, ≥90, and ≥120 were utilized along with two different criterion group approaches, 0 PVT failures vs. ≥2 PVT failures and 0 PVT failures versus ≥ 1 PVT failures.

RESULTS

Failure rates for The b Test in the overall sample ranged from 12.5% to 16.2%. Subgroup analyses in those referred specifically for ADHD revealed failure rates for The b Test ranging from 10.5% to 14.2%. ROC curves within the full sample and ADHD subsample demonstrated significant AUCs utilizing both criterion group approaches (AUC = .66 - .78). Sensitivity and specificity varied as a function of criterion group approach and cutoff score, with 0 PVT failures vs. ≥ 2 PVT failures resulting in the greatest sensitivity when maximizing specificity at ≥.90 in the full sample and ADHD sample.

CONCLUSIONS

The results demonstrate that criterion approaches and cutoff scores impact classification accuracy of The b Test with 0 PVT vs. ≥ 2 PVT failures demonstrating the greatest classification accuracy. Special considerations should be made with regard to clinical decision making in the context of psychoeducational evaluations given that a large portion of individuals seeking accommodations fail only one PVT. Limitations of this study are also discussed.

A Direct Comparison of 10 WAIS-IV Digit Span Embedded Validity Indicators among a Mixed Neuropsychiatric Sample with Varying Degrees of Cognitive Impairment

Objective

Reliable Digit Span (RDS), RDS-Revised (RDS-R), and age-corrected scaled score (ACSS) have been previously validated as embedded performance validity tests (PVTs) from the Wechsler Adult Intelligence Scale-IV Digit Span subtest (WAIS-IV DS). However, few studies have directly compared the relative utility of these and other proposed WAIS-IV DS validity indicators within a single sample.

Method

This study compared classification accuracies of 10 WAIS-IV DS indices in a mixed neuropsychiatric sample of 227 outpatients who completed a standardized neuropsychological battery. Participants with ≤1 PVT failures of the four, freestanding criterion PVTs constituted the valid group (n = 181), whereas those with ≥2 PVT failures formed the invalid group (n = 46). Among the valid group, 113 met criteria for mild cognitive impairment (MCI).

Results

Classification accuracies for all DS indicators were statistically significant across the overall sample and subsamples with and without MCI, apart from indices derived from the Forward trial in the MCI sample. DS Sequencing ACSS, working memory RDS (wmRDS), and DS ACSS emerged as the most effective predictors of validity status, with acceptable to excellent classification accuracy for the overall sample (AUCs = 0.792–0.816; 35%–50% sensitivity/88%–96% specificity).

Conclusions

Although most DS indices demonstrated clinical utility as embedded PVTs, DS Sequencing ACSS, wmRDS, and DS ACSS may be particularly robust to cognitive impairment, minimizing risk of false positive errors while identifying noncredible performance. Moreover, DS indices incorporating data from multiple trials (i.e., wmRDS, DS ACSS) also generally yielded greater classification accuracy than those derived from a single trial.

Disentangling subjective symptom complaints and objective cognitive performance in veterans: Impact of posttraumatic stress disorder and lifetime traumatic brain injury burden

Self-reported histories of mild traumatic brain injury (mTBI) and posttraumatic stress disorder (PTSD) symptoms are prevalent among post-9/11 veterans. Both are associated with subjective and often overlapping symptom complaints, but variably with objective neuropsychological test performances. These outcomes are seldom explored in relation to lifetime mTBI burden. This cross-sectional study examined associations of PTSD and lifetime mTBI with subjective (persistent symptoms after concussion and cognitive complaints) and objective (performances across five cognitive domains) measures among 46 veterans. Within this sample, 20 veterans had Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) confirmed PTSD (PTSD+), whereas 26 demographically-similar participants did not meet criteria (PTSD-). The Boston Assessment of Traumatic Brain Injury-Lifetime (BAT-L) yielded total mTBI scores ranging from 0 to 8. Regressions showed PTSD was associated with increased subjective symptoms/cognitive complaints, along with reduced verbal fluency, visuospatial memory, and processing speed performances. Lifetime mTBI burden was associated with subjective symptoms, but not with objective cognitive test performance, after controlling for PTSD. No significant interactions were observed. Exploratory correlations suggested that all PTSD symptom clusters were generally associated with the subjective and objective measures. However, fewer significant associations emerged within the PTSD+/- groups separately, with each group yielding somewhat different patterns of relationships. PTSD and increasing mTBI burden are consistently associated with negative subjective symptoms, including cognitive complaints. Each condition likely explains some degree of unique variance in symptom reporting. PTSD is associated with poorer objective cognition on some tasks, including processing speed, executive functioning, and learning/memory. Implications are explored.

Cross-Validation of Multiple WAIS-IV Digit Span Embedded Performance Validity Indices Among a Large Sample of Adult Attention Deficit/Hyperactivity Disorder Clinical Referrals

This study investigated the utility of four WAIS-IV Digit Span (DS) indices (traditional Reliable Digit Span [RDS], RDS-Working Memory [RDS-WM], RDS-Revised [RDS-R], and DS Age-Corrected Scaled Score [ACSS]) as embedded performance validity tests (PVTs) among a sample of 342 consecutive adults referred for neuropsychological evaluation of ADHD. All DS indices had acceptable classification accuracy (areas under the curve: .73–.76) for detecting invalid performance with optimal cut-scores of RDS ≤7 (35% sensitivity/93% specificity), RDS-WM ≤7 (56% sensitivity/86% specificity), RDS-R ≤12 (48% sensitivity/85% specificity), and ACSS ≤7 (46% sensitivity/87% specificity). Although all indices were able to detect invalid performance, DS indices incorporating the more complex working memory trials of the task yielded the best accuracy for identification of invalid test performance among adults referred for ADHD evaluation.

The impact of visual memory impairment on Victoria Symptom Validity Test performance: A known-groups analysis

OBJECTIVE

We assessed the effect of visual learning and recall impairment on Victoria Symptom Validity Test (VSVT) accuracy and response latency for Easy, Difficult, and Total Items.

METHOD

A sample of 163 adult patients administered the VSVT and Brief Visuospatial Memory Test-Revised were classified as valid (114/163) or invalid (49/163) groups via independent criterion performance validity tests (PVTs). Classification accuracies for all VSVT indices were examined for the overall sample, and separately for subgroups based on visual memory functioning.

RESULTS

In the overall sample, all indices produced acceptable classification accuracy (areas under the curve [AUCs] ≥ 0.79). When stratified by visual learning/recall impairment, accuracy indices yielded acceptable classification for both the unimpaired (AUCs ≥0.79) and impaired subsamples (AUCs ≥0.75). Latency indices had acceptable classification accuracy for the unimpaired subsample (AUCs ≥0.74), but accuracy and sensitivity dropped for the impaired sample (AUCs ≥0.67).

CONCLUSIONS

VSVT accuracy and response latency yielded acceptable classification accuracies in the overall sample, and this effect was maintained in those with and without visual learning/recall impairment for the accuracy indices. Findings indicate that the VSVT is a psychometrically robust PVT with largely invariant cut-scores, even in the presence of bona fide visual learning/recall impairment.

Examining the Clinical Utility of Selected Memory-Based Embedded Performance Validity Tests in Neuropsychological Assessment of Patients with Multiple Sclerosis

Within the neuropsychological assessment, clinicians are responsible for ensuring the validity of obtained cognitive data. As such, increased attention is being paid to performance validity in patients with multiple sclerosis (pwMS). Experts have proposed batteries of neuropsychological tests for use in this population, though none contain recommendations for standalone performance validity tests (PVTs). The California Verbal Learning Test, Second Edition (CVLT-II) and Brief Visuospatial Memory Test, Revised (BVMT-R)—both of which are included in the aforementioned recommended neuropsychological batteries—include previously validated embedded PVTs (which offer some advantages, including expedience and reduced costs), with no prior work exploring their utility in pwMS. The purpose of the present study was to determine the potential clinical utility of embedded PVTs to detect the signal of non-credibility as operationally defined by below criterion standalone PVT performance. One hundred thirty-three (133) patients (M age = 48.28; 76.7% women; 85.0% White) with MS were referred for neuropsychological assessment at a large, Midwestern academic medical center. Patients were placed into “credible” (n = 100) or “noncredible” (n = 33) groups based on a standalone PVT criterion. Classification statistics for four CVLT-II and BVMT-R PVTs of interest in isolation were poor (AUCs = 0.58–0.62). Several arithmetic and logistic regression-derived multivariate formulas were calculated, all of which similarly demonstrated poor discriminability (AUCs = 0.61–0.64). Although embedded PVTs may arguably maximize efficiency and minimize test burden in pwMS, common ones in the CVLT-II and BVMT-R may not be psychometrically appropriate, sufficiently sensitive, nor substitutable for standalone PVTs in this population. Clinical neuropsychologists who evaluate such patients are encouraged to include standalone PVTs in their assessment batteries to ensure that clinical care conclusions drawn from neuropsychological data are valid.

Considerations regarding noncredible performance in the neuropsychological assessment of patients with multiple sclerosis: A case series

Determining the validity of data during clinical neuropsychological assessment is crucial for proper interpretation, and extensive literature has emphasized myriad methods of doing so in diverse samples. However, little research has considered noncredible presentation in persons with multiple sclerosis (pwMS). PwMS often experience one or more factors known to impact validity of data, including major neurocognitive impairment, psychological distress/psychogenic interference, and secondary gain. This case series aimed to illustrate the potential relationships between these factors and performance validity testing in pwMS. Six cases from an IRB-approved database containing pwMS referred for neuropsychological assessment at a large, academic medical center involving at least one of the above-stated factors were identified. Backgrounds, neuropsychological test data, and clinical considerations for each were reviewed. Interestingly, no pwMS diagnosed with major neurocognitive impairment was found to have noncredible performance, nor was any patient with noncredible performance in the absence of notable psychological distress. Given the variability of noncredible performance and multiplicity of factors affecting performance validity in pwMS, clinicians are strongly encouraged to consider psychometrically appropriate methods for evaluating validity of cognitive data in pwMS. Additional research aiming to elucidate base rates of, mechanisms begetting, and methods for assessing noncredible performance in pwMS is imperative.

Nonmemory Composite Embedded Performance Validity Formulas in Patients with Multiple Sclerosis

OBJECTIVE

Research regarding performance validity tests (PVTs) in patients with multiple sclerosis (MS) is scant, with recommended batteries for neuropsychological evaluations in this population lacking suggestions to include PVTs. Moreover, limited work has examined embedded PVTs in this population. As previous investigations indicated that nonmemory-based embedded PVTs provide clinical utility in other populations, this study sought to determine if a logistic regression-derived PVT formula can be identified from selected nonmemory variables in a sample of patients with MS.

METHOD

A total of 184 patients (M age = 48.45; 76.6% female) with MS were referred for neuropsychological assessment at a large, Midwestern academic medical center. Patients were placed into "credible" (n = 146) or "noncredible" (n = 38) groups according to performance on standalone PVT. Missing data were imputed with HOTDECK.

RESULTS

Classification statistics for a variety of embedded PVTs were examined, with none appearing psychometrically appropriate in isolation (areas under the curve [AUCs] = .48-.64). Four exponentiated equations were created via logistic regression. Six, five, and three predictor equations yielded acceptable discriminability (AUC = .71-.74) with modest sensitivity (.34-.39) while maintaining good specificity (≥.90). The two predictor equation appeared unacceptable (AUC = .67).

CONCLUSIONS

Results suggest that multivariate combinations of embedded PVTs may provide some clinical utility while minimizing test burden in determining performance validity in patients with MS. Nonetheless, the authors recommend routine inclusion of several PVTs and utilization of comprehensive clinical judgment to maximize signal detection of noncredible performance and avoid incorrect conclusions. Clinical implications, limitations, and avenues for future research are discussed.

Establishing the base rate of performance invalidity in a clinical electrical injury sample: Implications for neuropsychological test performance

Objective: The base rate of neuropsychological performance invalidity in electrical injury, a clinically-distinct and frequently compensation-seeking population, is not well established. This study determined the base rate of performance invalidity in a large electrical injury sample, and examined patient characteristics, injury parameters, and neuropsychological test performance based on validity status.Method: This cross-sectional study included data from 101 patients with electrical injury consecutively referred for post-acute neuropsychological evaluation. Eighty-five percent of the sample was compensation-seeking. Multiple performance validity tests (PVTs) were administered as part of standard clinical evaluation. For patients with four or more PVTs, valid performance was operationalized as less than or equal to one PVT failure and invalid performance as two or more failures.Results: Frequency analysis revealed 66% (n = 67) had valid performance while 29% (n = 29) demonstrated probable invalid performance; the remaining 5% (n = 5) had indeterminate validity. No significant differences in demographics or injury parameters emerged between validity groups (0 vs. 1 vs. ≥2 PVT failures). In contrast, the electrical injury group with invalid performance performed significantly worse across tests of processing speed and executive abilities than those with valid performance (ps< .05, η_p² = .19-.25).Conclusions: The current study is the first to establish the base rate of neuropsychological performance invalidity in electrical injury survivors using empirical methods and current practice standards. Patient and clinical variables, including compensation-seeking status, did not differ between validity groups; however, neuropsychological test performance did, supporting the need for multi-method, objective performance validity assessment.