Young Child With Severe Brain Volume Loss Easily Passes the Word Memory Test and Medical Symptom Validity Test: Implications for Mild TBI

Validation of the personality assessment inventory (PAI) cognitive bias (CBS) and cognitive bias scale of scales (CB-SOS) in a post-deployment veteran sample

Objective: The present study evaluated the function of four cognitive, symptom validity scales on the Personality Assessment Inventory (PAI), the Cognitive Bias Scale (CBS) and the Cognitive Bias Scale of Scales (CB-SOS) 1, 2, and 3 in a sample of Veterans who volunteered for a study of neurocognitive functioning. Method: 371 Veterans (88.1% male, 66.1% White) completed a battery including the Miller Forensic Assessment of Symptoms Test (M-FAST), the Word Memory Test (WMT), and the PAI. Independent samples t-tests compared mean differences on cognitive bias scales between valid and invalid groups on the M-FAST and WMT. Area under the curve (AUC), sensitivity, specificity, and hit rate across various scale point-estimates were used to evaluate classification accuracy of the CBS and CB-SOS scales. Results: Group differences were significant with moderate effect sizes for all cognitive bias scales between the WMT-classified groups (d = .52-.55), and large effect sizes between the M-FAST-classified groups (d = 1.27-1.45). AUC effect sizes were moderate across the WMT-classified groups (.650-.676) and large across M-FAST-classified groups (.816-.854). When specificity was set to .90, sensitivity was higher for M-FAST and the CBS performed the best (sensitivity = .42). Conclusion: The CBS and CB-SOS scales seem to better detect symptom invalidity than performance invalidity in Veterans using cutoff scores similar to those found in prior studies with non-Veterans.

Adaptation and initial validation of the Arabic version of the Word Memory Test (WMTARB)

BACKGROUND

The feigning of cognitive impairment is common in neuropsychological assessments, especially in a medicolegal setting. The Word Memory Test (WMT) is a forced-choice recognition memory performance validity test (PVT) which is widely used to detect noncredible performance. Though translated to several languages, this was not done for one of the most common languages, Arabic. The aim of the current study was to evaluate the convergent validity of the Arabic adaptation of the WMT (WMT_ARB) among Israeli Arabic speakers.

METHODS

We adapted the WMT to Arabic using the back-translation method and in accordance with relevant guidelines. We then randomly assigned healthy Arabic speaking adults (N = 63) to either a simulation or honest control condition. The participants then performed neuropsychological tests which included the WMT_ARB and the Test of Memory Malingering (TOMM), a well-validated nonverbal PVT.

RESULTS

The WMT_ARB had high split-half reliability and its measures were significantly correlated with that of the TOMM (p < .001). High concordance was found in classification of participants using the WMT_ARB and TOMM (specificity = 94.29% and sensitivity = 100% using the conventional TOMM trial 2 cutoff as gold standard). As expected, simulators' accuracy on the WMT_ARB was significantly lower than that of honest controls. None of the demographic variables significantly correlated with WMT_ARB measures.

CONCLUSION

The WMT_ARB shows initial evidence of reliability and validity, emphasizing its potential use in the large population of Arabic speakers and universality in detecting noncredible performance. The findings, however, are preliminary and mandate validation in clinical settings.

Multivariate Models of Performance Validity: The Erdodi Index Captures the Dual Nature of Non-Credible Responding (Continuous and Categorical)

This study was designed to examine the classification accuracy of the Erdodi Index (EI-5), a novel method for aggregating validity indicators that takes into account both the number and extent of performance validity test (PVT) failures. Archival data were collected from a mixed clinical/forensic sample of 452 adults referred for neuropsychological assessment. The classification accuracy of the EI-5 was evaluated against established free-standing PVTs. The EI-5 achieved a good combination of sensitivity (.65) and specificity (.97), correctly classifying 92% of the sample. Its classification accuracy was comparable with that of another free-standing PVT. An indeterminate range between Pass and Fail emerged as a legitimate third outcome of performance validity assessment, indicating that the underlying construct is an inherently continuous variable. Results support the use of the EI model as a practical and psychometrically sound method of aggregating multiple embedded PVTs into a single-number summary of performance validity. Combining free-standing PVTs with the EI-5 resulted in a better separation between credible and non-credible profiles, demonstrating incremental validity. Findings are consistent with recent endorsements of a three-way outcome for PVTs (Pass, Borderline, and Fail).

Five shades of gray: Conceptual and methodological issues around multivariate models of performance validity

OBJECTIVE

This study was designed to empirically investigate the signal detection profile of various multivariate models of performance validity tests (MV-PVTs) and explore several contested assumptions underlying validity assessment in general and MV-PVTs specifically.

METHOD

Archival data were collected from 167 patients (52.4%male; MAge = 39.7) clinicially evaluated subsequent to a TBI. Performance validity was psychometrically defined using two free-standing PVTs and five composite measures, each based on five embedded PVTs.

RESULTS

MV-PVTs had superior classification accuracy compared to univariate cutoffs. The similarity between predictor and criterion PVTs influenced signal detection profiles. False positive rates (FPR) in MV-PVTs can be effectively controlled using more stringent multivariate cutoffs. In addition to Pass and Fail, Borderline is a legitimate third outcome of performance validity assessment. Failing memory-based PVTs was associated with elevated self-reported psychiatric symptoms.

CONCLUSIONS

Concerns about elevated FPR in MV-PVTs are unsubstantiated. In fact, MV-PVTs are psychometrically superior to individual components. Instrumentation artifacts are endemic to PVTs, and represent both a threat and an opportunity during the interpretation of a given neurocognitive profile. There is no such thing as too much information in performance validity assessment. Psychometric issues should be evaluated based on empirical, not theoretical models. As the number/severity of embedded PVT failures accumulates, assessors must consider the possibility of non-credible presentation and its clinical implications to neurorehabilitation.

Assessing Neuropsychological Functions in Middle Childhood: a Narrative Review of Measures and Their Psychometric Properties Across Context

Background

There is a significant number of neuropsychological measures for use among children aged 6–12 years. However, most of these tests have been developed in high-income contexts (HICs). To avoid or to at least to minimize bias in assessment, most researchers carry out cultural adaptations of these tools. In selecting sub-tests to adapt before using the entire test battery, researchers would benefit from having a reference source summarizing available tools and how easily they can be used in different context. This is where the paper makes a contribution. This narrative review has a twofold aim: first, to identify tools commonly used among 6–12-year-olds; second, to summarize the psychometric properties of these tools especially emphasizing their usage across different cultural contexts.

Methods

We searched the literature from 1 January 1987 to 31 December 2017 for tools used among children aged 6 to 12 years. An extensive search of PubMed, Psych Info and Web of Science using the keywords (i) neuropsychological or neurocognitive with (ii) assessment or test was done.

Results

A hundred and forty-five papers out of 306 reported on psychometric properties of different tools including Behavior Rating Inventory of Executive Functioning—BRIEF (count = 6), Visual-Motor Integration—VMI (count = 6), the Test of Memory Malingering—TOMM (count = 6), MSVT (count = 6) and Continuous Performance Tests—CPT (count = 6). Forty-six percent of the papers reported studies conducted in the USA. Most of these studies were based in high-income countries, which further highlights the need to validate these measures for use in lower- and middle-income countries. Psychometric check was adequate in most tests for measuring executive functioning such as BRIEF, although tests such as CPT that measure complex attention had mixed findings. Moreover, we found that these studies addressed certain aspects of validity and or reliability while leaving out others; thus, a comprehensive picture is lacking.

Conclusion

We propose further studies to thoroughly investigate and report the psychometric properties of these measures, especially in lower- and middle-income countries.

Distress Tolerance and Symptom Severity as Mediators of Symptom Validity Failure in Veterans With PTSD

OBJECTIVE

Performance validity tests (PVTs) and symptom validity tests (SVTs) are necessary in clinical and research contexts. The extent to which psychiatric distress contributes to failure on these tests is unclear. The authors hypothesized that the relation between posttraumatic stress disorder (PTSD) and validity would be serially mediated by distress tolerance and symptom severity.

METHODS

Participants included 306 veterans, 110 of whom met full criteria for current PTSD. PVTs included the Medical Symptom Validity Test (MSVT) and b Test. The Structured Inventory of Malingered Symptomatology (SIMS) was used to measure symptom validity.

RESULTS

MSVT failure was significantly and directly associated with PTSD severity (B=0.05, CI=0.01, 0.08) but not distress tolerance or PTSD diagnosis. b Test performance was not significantly related to any variable. SIMS failure was significantly associated with PTSD diagnosis (B=0.71, CI=0.05, 1.37), distress tolerance (B=-0.04, CI=-0.07, -0.01), and symptom severity (B=0.07, CI=0.04, 0.09). The serial mediation model significantly predicted all SIMS subscales.

CONCLUSIONS

PTSD severity was associated with failing a memory-based PVT but not an attention-based PVT. Neither PVT was associated with distress tolerance or PTSD diagnosis. SVT failure was associated with PTSD diagnosis, poor distress tolerance, and high symptomatology. For veterans with PTSD, difficulty managing negative emotional states may contribute to symptom overreporting. This may reflect exaggeration or an inability to tolerate stronger negative affect, rather than a "cry for help."

Psychologists' Role in Concussion Assessments for Children and Adolescents in Pediatric Practice

An estimated 1.1 to 1.9 million children and adolescents in the United States are treated for a sports- or recreationally-related concussion each year. The importance of formalized assessment and measurement of concussion symptoms has been widely recognized as a component of best-practice treatment. The present paper reviews a sample of the most commonly used measures of concussion symptomology and explores psychologists' role in their application in a pediatric practice. In addition, other issues such as accessibility and the appropriateness of application with child and adolescent patients are discussed. Literature is reviewed from journals pertaining to pediatric and adolescent medicine, sports medicine, neuropsychology, and testing and measurement.

Introduction to special issue on effort testing in children and adolescents

This manuscript reviews the past history of effort testing in children and adolescents. Results in children and adults are briefly described on tests such as the Word Memory Test (WMT), the Medical Symptom Validity Test (MSVT), the Nonverbal Medical Symptom Validity Test (NV-MSVT) and Test of Memory Malingering (TOMM). Articles in the Special Issue are also briefly described.

The use of the MSVT in children and adolescents with epilepsy

Pediatric neuropsychologists are increasingly recognizing the importance of performance validity testing during evaluations. The use of such measures to detect insufficient effort is of particular importance in pediatric epilepsy evaluations, where test results are often used to guide surgical decisions and failure to detect poor task engagement can result in postsurgical cognitive decline. The present investigation assesses the utility of the Medical Symptom Validity Test (MSVT) in 104 clinically referred children and adolescents with epilepsy. Though the overall failure rate was 15.4% of the total group, children with 2nd grade or higher reading skills (a requirement of the task) passed at a very high rate (96.6%). Of the three failures, two were unequivocally deemed true positives, while the third failed due to extreme somnolence during testing. Notably, for those with ≥2nd grade reading levels, MSVT validity indices were unrelated to patient age, intellectual functioning, or age of epilepsy onset, while modest relations were seen with specific memory measures, number of epilepsy medications, and seizure frequency. Despite these associations, however, this did not result in more failures in this population of children and adolescents with substantial neurologic involvement, as pass rates exceeded 92% for those with intellectual disability, high seizure frequency, high medication burden, and even prior surgical resection of critical memory structures.

Comparison of the Word Memory Test and the Test of Memory Malingering in detecting invalid performance in neuropsychological testing

Given the prevalence of compensation seeking patients who exaggerate or fabricate their symptoms, the assessment of performance and symptom validity throughout testing is vital in neuropsychological evaluations. Two of the most commonly utilized performance validity tests (PVTs) are the Word Memory Test (WMT) and the Test of Memory Malingering (TOMM). While both have proven successful in detecting invalid performance, some studies suggest greater sensitivity in the WMT relative to the TOMM. To improve upon previous research, this study compared performance in individuals who completed both the WMT and TOMM during a neuropsychological evaluation. Participants included 268 cases from a clinical private practice consisting of primarily disability claimants. One-way multivariate analysis of variance (MANOVA) compared neuropsychological performance of participants who passed both PVTs (n = 198) versus those who failed the WMT but passed the TOMM (n = 70). Global suppression of neuropsychological scores was found for participants who failed the WMT but passed the TOMM, as well as more psychiatric symptoms reported on questionnaires, relative to those who passed both PVTs. These findings suggest that those passing the TOMM but failing the WMT demonstrated performance invalidity, which illustrates the WMT's enhanced sensitivity.

Performance validity test failure predicts suppression of neuropsychological test results in developmentally disabled children

There is increasing awareness of the need to use Performance Validity Tests (PVTs) when assessing cognitive abilities in children. Since 1996, there has been an ongoing clinical study of the use of three PVTs with a consecutive series of 1,285 developmentally disabled children. In this study, we report on the results of these PVTs in children of many diagnostic categories. Failure rates on all three tests were very low. The mean scores on the effort measures in those passing the PVTs were extremely high. Failure on each PVT was found to be associated with a significant and widespread suppression of scores across a neuropsychological battery. Failure on even one PVT significantly suppresses ability test scores.

Utility of Green's Word Memory Test Free Recall Subtest as a Measure of Verbal Memory: Initial Evidence from a Temporal Lobe Epilepsy Clinical Sample

This study investigated the Word Memory Test (WMT) Free Recall (FR) subtest as a conventional memory measure. Nineteen participants with pharmacoresistant left temporal lobe epilepsy (LTLE) and 16 with right temporal lobe epilepsy (RTLE) completed the WMT, Rey Auditory Verbal Learning Test (RAVLT), and Wechsler Memory Scale-Fourth Edition Logical Memory (LM) subtest during presurgical evaluation. LTLE participants performed significantly worse on FR subtest (p < .05, [Formula: see text]) and RAVLT Trial 7 (p < .01, [Formula: see text]), but not on LM subtest. Age was a significant covariate for FR (p < .01, [Formula: see text]). Logistic regression revealed FR plus age and RAVLT age-adjusted T-scores both yielded 77.1% classification accuracy and respective diagnostic odds ratios of 11.36 and 11.84. Receiver operating characteristic curves to classify seizure laterality found that RAVLT and FR were significant (area under the curve [AUC] = 0.82 and 0.74), whereas LM was nonsignificant (AUC = 0.67). Cut scores and positive/negative predictive values were established for improved clinical classification.

Effort, symptom validity testing, performance validity testing and traumatic brain injury

BACKGROUND

To understand the neurocognitive effects of brain injury, valid neuropsychological test findings are paramount.

REVIEW

This review examines the research on what has been referred to a symptom validity testing (SVT). Above a designated cut-score signifies a 'passing' SVT performance which is likely the best indicator of valid neuropsychological test findings. Likewise, substantially below cut-point performance that nears chance or is at chance signifies invalid test performance. Significantly below chance is the sine qua non neuropsychological indicator for malingering. However, the interpretative problems with SVT performance below the cut-point yet far above chance are substantial, as pointed out in this review. This intermediate, border-zone performance on SVT measures is where substantial interpretative challenges exist. Case studies are used to highlight the many areas where additional research is needed. Historical perspectives are reviewed along with the neurobiology of effort. Reasons why performance validity testing (PVT) may be better than the SVT term are reviewed.

CONCLUSIONS

Advances in neuroimaging techniques may be key in better understanding the meaning of border zone SVT failure. The review demonstrates the problems with rigidity in interpretation with established cut-scores. A better understanding of how certain types of neurological, neuropsychiatric and/or even test conditions may affect SVT performance is needed.