Results From Three Performance Validity Tests in Children With Intellectual Disability

Objective and subjective neurocognitive functioning in functional motor symptoms and functional seizures: preliminary findings

INTRODUCTION

This study aimed to provide a preliminary assessment of objective and subjective neurocognitive functioning in individuals with functional motor symptoms (FMS) and/or functional seizures (FS). We tested the hypotheses that the FMS/FS group would display poorer objective attentional and executive functioning, altered social cognition, and reduced metacognitive accuracy.

METHOD

Individuals with FMS/FS (n = 16) and healthy controls (HCs, n = 17) completed an abbreviated CANTAB battery, and measures of intellectual functioning, subjective cognitive complaints, performance validity, and comorbid symptoms. Subjective performance ratings were obtained to assess local metacognitive accuracy.

RESULTS

The groups were comparable in age (p = 0.45), sex (p = 0.62), IQ (p = 0.57), and performance validity (p-values = 0.10-0.91). We observed no impairment on any CANTAB test in this FMS/FS sample compared to HCs, although the FMS/FS group displayed shorter reaction times on the Emotional Bias task (anger) (p = 0.01, np2 = 0.20). The groups did not differ in subjective performance ratings (p-values 0.15). Whilst CANTAB attentional set-shifting performance (total trials/errors) correlated with subjective performance ratings in HCs (p-values<0.005, rs = -0.85), these correlations were non-significant in the FMS/FS sample (p-values = 0.10-0.13, rs-values = -0.46-0.50). The FMS/FS group reported more daily cognitive complaints than HCs (p = 0.006, g = 0.92), which were associated with subjective performance ratings on CANTAB sustained attention (p = 0.001, rs = -0.74) and working memory tests (p < 0.001, rs = -0.75), and with depression (p = 0.003, rs = 0.70), and somatoform (p = 0.003, rs = 0.70) and psychological dissociation (p-values<0.005, rs-values = 0.67-0.85).

CONCLUSIONS

These results suggest a discordance between objective and subjective neurocognitive functioning in this FMS/FS sample, reflecting intact test performance alongside poorer subjective cognitive functioning. Further investigation of neurocognitive functioning in FND subgroups is necessary.

Base rates of non-credible performance in a post-secondary student sample seeking accessibility accommodations

Objective: Performance Validity Tests (PVTs) have been used to identify non-credible performance in clinical, medicolegal, forensic, and, more recently, academic settings. The inclusion of PVTs when administering psychoeducational assessments is essential given that specific accommodation such as flexible deadlines and increased writing time can provide an external incentive for students without disabilities to feign symptoms. Method: The present study used archival data to establish base rates of non-credible performance in a sample of post-secondary students (n = 1045) who underwent a comprehensive psychoeducational evaluation for the purposes of obtaining academic accommodations. In accordance with current guidelines, non-credible performance was determined by failure on two or more freestanding or embedded PVTs. Results: 9.4% of participants failed at least two of the PVTs they were administered, of which 8.5% failed two PVTs, and approximately 1% failed three PVTs. Base rates of failure for specific PVTs ranged from 25% (b Test) to 11.2% (TOVA). Conclusions: The present study found a lower base rate of non-credible performance than previously observed in comparable populations. This likely reflects the utilization of conservative criteria in detecting non-credible performance to avoid false positives. By contrast, inconsistent base rates previously found in the literature may reflect inconsistent methodologies. These results further emphasize the importance of administering multiple PVTs during psychoeducational assessments. The implications of these findings can further inform clinicians administering assessments in academic settings and aid in the appropriate utilization of PVTs in psychoeducational evaluation to determine accessibility accommodations.

Adaptation and initial validation of the Arabic version of the Word Memory Test (WMTARB)

BACKGROUND

The feigning of cognitive impairment is common in neuropsychological assessments, especially in a medicolegal setting. The Word Memory Test (WMT) is a forced-choice recognition memory performance validity test (PVT) which is widely used to detect noncredible performance. Though translated to several languages, this was not done for one of the most common languages, Arabic. The aim of the current study was to evaluate the convergent validity of the Arabic adaptation of the WMT (WMT_ARB) among Israeli Arabic speakers.

METHODS

We adapted the WMT to Arabic using the back-translation method and in accordance with relevant guidelines. We then randomly assigned healthy Arabic speaking adults (N = 63) to either a simulation or honest control condition. The participants then performed neuropsychological tests which included the WMT_ARB and the Test of Memory Malingering (TOMM), a well-validated nonverbal PVT.

RESULTS

The WMT_ARB had high split-half reliability and its measures were significantly correlated with that of the TOMM (p < .001). High concordance was found in classification of participants using the WMT_ARB and TOMM (specificity = 94.29% and sensitivity = 100% using the conventional TOMM trial 2 cutoff as gold standard). As expected, simulators' accuracy on the WMT_ARB was significantly lower than that of honest controls. None of the demographic variables significantly correlated with WMT_ARB measures.

CONCLUSION

The WMT_ARB shows initial evidence of reliability and validity, emphasizing its potential use in the large population of Arabic speakers and universality in detecting noncredible performance. The findings, however, are preliminary and mandate validation in clinical settings.

Five shades of gray: Conceptual and methodological issues around multivariate models of performance validity

OBJECTIVE

This study was designed to empirically investigate the signal detection profile of various multivariate models of performance validity tests (MV-PVTs) and explore several contested assumptions underlying validity assessment in general and MV-PVTs specifically.

METHOD

Archival data were collected from 167 patients (52.4%male; MAge = 39.7) clinicially evaluated subsequent to a TBI. Performance validity was psychometrically defined using two free-standing PVTs and five composite measures, each based on five embedded PVTs.

RESULTS

MV-PVTs had superior classification accuracy compared to univariate cutoffs. The similarity between predictor and criterion PVTs influenced signal detection profiles. False positive rates (FPR) in MV-PVTs can be effectively controlled using more stringent multivariate cutoffs. In addition to Pass and Fail, Borderline is a legitimate third outcome of performance validity assessment. Failing memory-based PVTs was associated with elevated self-reported psychiatric symptoms.

CONCLUSIONS

Concerns about elevated FPR in MV-PVTs are unsubstantiated. In fact, MV-PVTs are psychometrically superior to individual components. Instrumentation artifacts are endemic to PVTs, and represent both a threat and an opportunity during the interpretation of a given neurocognitive profile. There is no such thing as too much information in performance validity assessment. Psychometric issues should be evaluated based on empirical, not theoretical models. As the number/severity of embedded PVT failures accumulates, assessors must consider the possibility of non-credible presentation and its clinical implications to neurorehabilitation.

The Impact of Learning and Memory on Performance Validity Tests in a Mixed Clinical Pediatric Population

OBJECTIVE

This study examined the degree to which verbal and visuospatial memory abilities influence performance validity test (PVT) performance in a mixed clinical pediatric sample.

METHOD

Data from 252 consecutive clinical pediatric cases (Mage=11.23 years, SD=4.02; 61.9% male) seen for outpatient neuropsychological assessment were collected. Measures of learning and memory (e.g., The California Verbal Learning Test-Children's Version; Child and Adolescent Memory Profile [ChAMP]), performance validity (Test of Memory Malingering Trial 1 [TOMM T1]; Wechsler Intelligence Scale for Children-Fifth Edition [WISC-V] or Wechsler Adult Intelligence Scale-Fourth Edition Digit Span indices; ChAMP Overall Validity Index), and intellectual abilities (e.g., WISC-V) were included.

RESULTS

Learning/memory abilities were not significantly correlated with TOMM T1 and accounted for relatively little variance in overall TOMM T1 performance (i.e., ≤6%). Conversely, ChAMP Validity Index scores were significantly correlated with verbal and visual learning/memory abilities, and learning/memory accounted for significant variance in PVT performance (12%-26%). Verbal learning/memory performance accounted for 5%-16% of the variance across the Digit Span PVTs. No significant differences in TOMM T1 and Digit Span PVT scores emerged between verbal/visual learning/memory impairment groups. ChAMP validity scores were lower for the visual learning/memory impairment group relative to the nonimpaired group.

CONCLUSIONS

Findings highlight the utility of including PVTs as standard practice for pediatric populations, particularly when memory is a concern. Consistent with the adult literature, TOMM T1 outperformed other PVTs in its utility even among the diverse clinical sample with/without learning/memory impairment. In contrast, use of Digit Span indices appear to be best suited in the presence of visuospatial (but not verbal) learning/memory concerns. Finally, the ChAMP's embedded validity measure was most strongly impacted by learning/memory performance.

Introduction to special issue on effort testing in children and adolescents

This manuscript reviews the past history of effort testing in children and adolescents. Results in children and adults are briefly described on tests such as the Word Memory Test (WMT), the Medical Symptom Validity Test (MSVT), the Nonverbal Medical Symptom Validity Test (NV-MSVT) and Test of Memory Malingering (TOMM). Articles in the Special Issue are also briefly described.

The use of the MSVT in children and adolescents with epilepsy

Pediatric neuropsychologists are increasingly recognizing the importance of performance validity testing during evaluations. The use of such measures to detect insufficient effort is of particular importance in pediatric epilepsy evaluations, where test results are often used to guide surgical decisions and failure to detect poor task engagement can result in postsurgical cognitive decline. The present investigation assesses the utility of the Medical Symptom Validity Test (MSVT) in 104 clinically referred children and adolescents with epilepsy. Though the overall failure rate was 15.4% of the total group, children with 2nd grade or higher reading skills (a requirement of the task) passed at a very high rate (96.6%). Of the three failures, two were unequivocally deemed true positives, while the third failed due to extreme somnolence during testing. Notably, for those with ≥2nd grade reading levels, MSVT validity indices were unrelated to patient age, intellectual functioning, or age of epilepsy onset, while modest relations were seen with specific memory measures, number of epilepsy medications, and seizure frequency. Despite these associations, however, this did not result in more failures in this population of children and adolescents with substantial neurologic involvement, as pass rates exceeded 92% for those with intellectual disability, high seizure frequency, high medication burden, and even prior surgical resection of critical memory structures.

Noncredible effort on the Nonverbal-Medical Symptom Validity Test (NV-MSVT): Impact on cognitive performance in pediatric mild traumatic brain injury

This study examined failure rates on the Nonverbal-Medical Symptom Validity Test (NV-MSVT) and its impact on cognitive performance in a sample of youths with mild traumatic brain injury (mTBI). Participants were 184 children and adolescents who presented to a multidisciplinary concussion clinic for a targeted neuropsychological evaluation. Performance Validity Tests (PVTs) were a part of the standard battery, including the NV-MSVT. Twenty-eight participants (15.2%) failed the NV-MSVT, none of whom displayed a genuine memory impairment profile (GMIP). Participants who failed the NV-MSVT performed significantly worse than those who passed the NV-MSVT on measures of IQ, memory, and immediate attention/working memory. There was no significant difference between groups on processing speed, sustained attention, cognitive flexibility, or sight word reading level. Aside from a slight difference in age, NV-MSVT failure was not impacted by demographic variables (sex, race), premorbid risk factors (pre-injury ADHD, learning disabilities, psychiatric diagnoses or treatment, developmental delays, or prior special education), injury-related variables (time since injury, positive neuroimaging findings, post-traumatic amnesia, number of prior mTBIs, etc.) or post-mTBI anxiety/depression. That said, participants who failed NV-MSVT endorsed significantly more severe postconcussive symptoms. These findings support the use of the NV-MSVT in neuropsychological evaluation of children and adolescents with mTBI.

Comparison of the Word Memory Test and the Test of Memory Malingering in detecting invalid performance in neuropsychological testing

Given the prevalence of compensation seeking patients who exaggerate or fabricate their symptoms, the assessment of performance and symptom validity throughout testing is vital in neuropsychological evaluations. Two of the most commonly utilized performance validity tests (PVTs) are the Word Memory Test (WMT) and the Test of Memory Malingering (TOMM). While both have proven successful in detecting invalid performance, some studies suggest greater sensitivity in the WMT relative to the TOMM. To improve upon previous research, this study compared performance in individuals who completed both the WMT and TOMM during a neuropsychological evaluation. Participants included 268 cases from a clinical private practice consisting of primarily disability claimants. One-way multivariate analysis of variance (MANOVA) compared neuropsychological performance of participants who passed both PVTs (n = 198) versus those who failed the WMT but passed the TOMM (n = 70). Global suppression of neuropsychological scores was found for participants who failed the WMT but passed the TOMM, as well as more psychiatric symptoms reported on questionnaires, relative to those who passed both PVTs. These findings suggest that those passing the TOMM but failing the WMT demonstrated performance invalidity, which illustrates the WMT's enhanced sensitivity.

Performance validity test failure predicts suppression of neuropsychological test results in developmentally disabled children

There is increasing awareness of the need to use Performance Validity Tests (PVTs) when assessing cognitive abilities in children. Since 1996, there has been an ongoing clinical study of the use of three PVTs with a consecutive series of 1,285 developmentally disabled children. In this study, we report on the results of these PVTs in children of many diagnostic categories. Failure rates on all three tests were very low. The mean scores on the effort measures in those passing the PVTs were extremely high. Failure on each PVT was found to be associated with a significant and widespread suppression of scores across a neuropsychological battery. Failure on even one PVT significantly suppresses ability test scores.

Construct validity for the verbal associated pairs screen

The present study introduces the Verbal Associated Pairs Screen (VAPS) as a new measure for assessing performance validity in pediatric populations. This study presents initial data on psychometric properties and establishes construct validity for the VAPS in a sample of 30 adolescent healthy controls and 206 youths with traumatic brain injury (TBI: moderate/severe, N = 30; mild, N = 176). The control group's age (M = 14.93, SD = 1.8) was significantly higher than the moderate/severe TBI (M = 13.9, SD = 2.8), t(68.508) = -3.038, p = .003, and mild TBI (mTBI) groups (M = 14, SD = 2.8), t(54.147) = 2.038, p = .046. The TBI groups were administered the VAPS in accord with other established performance validity tests (PVTs) and well-established  memory tests as part of routine clinical evaluations. The healthy control group was administered the VAPS only. VAPS score distributions for the control group were negatively skewed and highly kurtotic. VAPS scores from the moderate/severe TBI and control groups were indistinguishable for Trial 2 (U = 274, p < .01) and the Delay (U = 396, p = .218). In the mTBI group, convergent and divergent validity was established with other well-validated PVTs and memory tests, respectively. ROC curve analyses identified optimal cutoff scores for the VAPS Total Score, with acceptable sensitivity (55%) and excellent specificity (100%), as well as strong detectability (AUC = .829, 95% CI: 0.731 - 0.928, p < .001). Clinical applications, limitations, and directions for future research with the VAPS are discussed.

Use of the 21-Item Test with children and adolescents 5 to 16 years of age

The goals of the present study were to: (a) collect preliminary normative data on the 21-Item Test in children and adolescents and to determine if cutoffs for biased responding in adults can be generalized to a younger population; (b) determine if Caucasian and Hispanic children perform differently on the test; and (c) ascertain the relationships of age and verbal intelligence with effort test performance. The 21-Item Test was administered to 153 children attending either a public (n = 96) or private (n = 57) school. The Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV) was also administered to the private school sample. The results indicated that the 21-Item Test has potential as a tool for identifying children who do not put forth maximal effort during formal assessment and that the adult cutoffs appear applicable to children 8 years and older. Caucasian and Hispanic participants performed similarly on the test. Forced-choice scores ≤12 were associated with age and lower levels of performance on the Verbal Comprehension Index of the WISC-IV.

Performance validity testing in children and adolescents: A descriptive study comparing direct and embedded measures

Effort testing is a standard element in adult neuropsychological assessment. Research examining performance validity tests (PVTs) has focused on adults. The purpose of this descriptive study was to examine direct and embedded PVTs in children and adolescents. The Test of Memory Malingering (TOMM), a stand-alone PVT, was compared to two embedded measures of effort: California Verbal Learning Test-Children's Version/Second Edition (CVLT-C/II) Recognition Discriminability and Reliable Digit Span (RDS). The sample consisted of 119 children and adolescents referred for outpatient assessment. Cut-off scores used for PVTs were based on previous studies. Results revealed 3/119 failures on the TOMM, 14/119 failures on the RDS, and 34/119 failures on the CVLT-C/II. There was a significant difference between failures on the TOMM and the RDS, as well as between failures on the TOMM and the CVLT-C/II and the RDS and the CVLT-C/II. The results of this study demonstrate that PVTs commonly used with adults may require modifications, including adjusted cut-off scores, to be appropriate with children and adolescents. Results of this study also suggest that relying on more than one measure likely provides the most utility.