Performance and Symptom Validity Testing in Pediatric Assessment: A Review of the Literature

Young children's performance on a design fluency task: longitudinal data on total number of designs, clustering and switching, and regression-based norms

INTRODUCTION

Design fluency (DF) tasks are commonly used to assess executive functions such as attentional control, cognitive flexibility, self-monitoring and strategy use. Next to the total number of correct designs, the standard outcome of a DF task, clustering and switching can help disentangle the processes underlying DF performance. We present the first longitudinal study of 4-8-year-old children's developmental DF trajectories.

METHOD

At initial enrollment, children (n = 228) were aged between 4.05 and 6.88 years (M = 5.18, SD = 0.59) and attended Dutch primary schools. The DF task was administered at three time points, each time point separated by approximately 1 year. Data were analyzed using mixed regression for total number of correct designs and switching, and mixed logistic regression analysis for clustering.

RESULTS

The total number of correct designs increased linearly across the three time points. Across all time points, children made very few clusters, and most clusters consisted of only 3 designs. Clustering only increased at the third assessment compared to the two previous assessments. Switching increased up to the second assessment, but not after that. The number of switches was highly correlated with the total number of correct designs at all time points (r = 0.78 to r = 0.85). These developmental trajectories were similar for all children regardless of their baseline age. Normative data are given for the total number of correct designs and switching.

CONCLUSIONS

Children as of age 4 onwards can perform a DF task. For children as young as 4-8 years old, computing clustering, and switching measures is of limited value to study cognitive processes underlying DF performance, next to the total number of correct designs. There were no sex differences on any of the DF outcomes. Level of parental education (LPE) was positively associated with the total number of correct designs and switching.

Performance and symptom validity indicators among children undergoing cognitive surveillance following treatment for craniopharyngioma

Background

Performance validity tests (PVTs) and symptom validity tests (SVTs) are essential to neuropsychological evaluations, helping ensure findings reflect true abilities or concerns. It is unclear how PVTs and SVTs perform in children who received radiotherapy for brain tumors. Accordingly, we investigated the rate of noncredible performance on validity indicators as well as associations with fatigue and lower intellectual functioning.

Methods

Embedded PVTs and SVTs were investigated in 98 patients with pediatric craniopharyngioma undergoing proton radiotherapy (PRT). The contribution of fatigue, sleepiness, and lower intellectual functioning to embedded PVT performance was examined. Further, we investigated PVTs and SVTs in relation to cognitive performance at pre-PRT baseline and change over time.

Results

SVTs on parent measures were not an area of concern. PVTs identified 0-31% of the cohort as demonstrating possible noncredible performance at baseline, with stable findings 1 year following PRT. Reliable digit span (RDS) noted the highest PVT failure rate; RDS has been criticized for false positives in pediatric populations, especially children with neurological impairment. Objective sleepiness was strongly associated with PVT failure, stressing need to consider arousal level when interpreting cognitive performance in children with craniopharyngioma. Lower intellectual functioning also needs to be considered when interpreting task engagement indices as it was strongly associated with PVT failure.

Conclusions

Embedded PVTs should be used with caution in pediatric craniopharyngioma patients who have received PRT. Future research should investigate different cut-off scores and validity indicator combinations to best differentiate noncredible performance due to task engagement versus variable arousal and/or lower intellectual functioning.

A comparison of MVP and MSVT performance among a diverse clinical pediatric sample

The Memory Validity Profile (MVP) and Medical Symptom Validity Test (MSVT) are performance validity tests (PVTs) used to identify potential noncredible test performance during psychological evaluations. This study sought to examine the agreement between MVP and MSVT pass rates, as well as to determine if there are differences in MVP pass rates when using the cutoff score in the MVP professional manual compared with the experimental cutoff score of <31. Via retrospective review of records, 106 clients at a private neuropsychological clinic who had been given the MVP and the MSVT were identified. Results indicated that only one client met the manual cutoff scores, compared to 20 clients who failed the MSVT, raising concerns regarding the sensitivity of the MVP. Utilizing the receiver operator characteristic (ROC), curve analyses indicated fair discriminability of the MVP for the 106 participants (AUC = .717) with acceptable sensitivity (.50) and specificity (.92) for an MVP total score cutoff of <31. These findings support the utility of the experimental cut score in improving the sensitivity while maintaining adequate specificity in a clinically mixed population.

Memory performance on the ChAMP in autism spectrum disorder with and without attention-deficit/hyperactivity disorder

Memory difficulties have been identified in youth with neurodevelopmental conditions including autism spectrum disorder (ASD) and attention-deficit/hyperactivity disorder (ADHD). The Child and Adolescent Memory Profile (ChAMP) is a newer memory measure with a burgeoning research base. This study seeks to explore memory performance on the Lists and Objects subtests of the ChAMP in a clinical sample of those with ASD with/without co-occurring ADHD. Participants were 146 youth referred for a neuropsychological evaluation (M age = 11.8 years; 76.03% male) diagnosed with ASD (N = 92 with ADHD, N = 54 without). Logistic regression (p = .393) indicated ChAMP performance is not predictive of whether the ASD group had co-occurring ADHD indicating there is no additive effect on memory. Compared to the ChAMP examiner's manual ASD sample, this study sample performed significantly better (p <.001) on all ChAMP measures. While the ChAMP is sensitive to memory difficulties in neurodevelopmental disorders, as indicated by the performance of the manual sample, the ASD sample of the manual may differ from other ASD samples. There were no differences between verbal and visual memory performance across the present study's sample.

Post-Surgical Cognitive Outcomes of Moyamoya Disease: A Systematic Review

BACKGROUND

Moyamoya disease (MMD) is an occlusive arteriopathy leading to stroke. Progressive if left untreated, revascularization surgery has become the mainstay of treatment. Although clinical and radiographic outcomes of MMD after intervention are well-characterized, cognitive outcomes in pediatric patients remain unclear. We aimed to examine postoperative cognitive outcomes in children with MMD, examine factors associated with cognitive changes after intervention, and define the effect of revascularization surgery on cognitive outcomes.

METHODS

A systematic review was conducted following PRISMA guidelines searching PubMed, Embase, and Scopus databases. Articles met inclusion criteria if they studied pediatric patients undergoing revascularization surgery for MMD and examined cognitive outcomes either qualitatively or quantitatively. All data extracted from included articles was examined descriptively.

RESULTS

Of 1091 resultant articles, 12 articles containing 446 patients were included. Surgery was associated with maintained or improved full scale intellectual quotient (IQ), performance IQ, perceptual IQ, memory quotient and verbal memory. However, 70% of patients had impaired cognitive function, with associated poor school performance. Improvements in cognition were associated with increased cerebral blood flow, particularly to the middle cerebral artery, due to the development of collaterals. Female sex, shorter duration from symptom onset to surgery, and surgery after age 7 were predictive of cognitive improvement. Completed ischemic stroke prior to surgery was associated with poorer cognitive outcomes.

CONCLUSIONS

Although children with MMD have improved cognitive outcomes following revascularization overall, a distinct subset experience cognitive impairment. Consideration of patient-specific and treatment-related factors is important to enable proper risk stratification and inform management approaches.

Performance and Symptom Validity in Outpatient Pediatric Neuropsychological Evaluations

This study aimed to clarify the value of using different types of validity measures in pediatric neuropsychological evaluations. We examined the relationship between performance (PVT) and symptom (SVT) validity tests as well as demographic variables and results from a screening test of learning and memory (i.e. Child and Adolescent Memory Profile [ChAMP]) in a mixed pediatric sample (n = 103). There was minimal overlap between PVT and SVT failures. Regression analyses demonstrated that PVT results, parental education, and history of special education were statistically significant predictors of ChAMP results, whereas SVT results were not.

An alternative approach to TOMM cutoff scores using a large sample of military personnel

The accuracy of neuropsychological assessments relies on participants exhibiting their true abilities during administration. The Test of Memory Malingering (TOMM) is a popular performance validity test used to determine whether an individual is providing honest answers. While the TOMM has proven to be highly sensitive to those who are deliberately exaggerating their symptoms, there is a limited explanation regarding the significance of using 45 as a cutoff score. The present study aims to further investigate this question by examining TOMM scores obtained in a large sample of active-duty military personnel (N = 859, M = 26 years, SD = 6.14, 97.31% males, 72.44% white). Results indicated that no notable discrepancies existed between the frequency of participants who scored a 45 and those who scored slightly below a 45 on the TOMM. The sensitivity and specificity of the TOMM were derived using the forced-choice recognition (FCR) scores obtained by participants on the California Verbal Learning Test, Second Edition (CVLT-II). The sensitivity for each trial of the TOMM was 0.84, 0.55, and 0.63, respectively; the specificity for each trial of the TOMM was 0.69, 0.93, and 0.92, respectively. Because sensitivity and specificity rates are both of importance in this study, balanced accuracy scores were also reported. Results suggested that various alternative cutoff scores produced a more accurate classification compared to the traditional cutoff of 45. Further analyses using Fisher's exact test also indicated that there were no significant performance differences on the FCR of the CVLT-II between individuals who received a 44 and individuals who received a 45 on the TOMM. The current study provides evidence on why the traditional cutoff may not be the most effective score. Future research should consider employing alternative methods which do not rely on a single score.

An exploratory study of the convergent validity of the Test of Effort (TOE) in adults with acquired brain injury

PRIMARY OBJECTIVE

To examine the convergent validity of the Test of Effort (TOE), a performance validity test (PVT) currently under development that employs a two-subtest (one verbal, one visual), forced-choice recognition memory format.

RESEARCH DESIGN

A descriptive, correlational design was employed to describe performance on the TOE and examine the convergent validity between the TOE and comparison measures.

METHODS AND PROCEDURES

A sample of 53 individuals with chronic acquired brain injury (ABI) were administered the TOE and three well-validated PVTs (Reliable Digit Span [RDS], Test of Memory Malingering [TOMM] and Dot Counting Test [DCT]).

MAIN OUTCOMES AND RESULTS

The TOE appeared more difficult than it actually was, suggesting adequate face validity. Medium-to-large correlations were observed between the TOE and established PVTs, suggesting good convergent validity. Provisional cutoff scores are offered based on performance of a subgroup of participants with "sufficient effort."

CONCLUSIONS

Overall, the TOE shows promise as a PVT measure for clinical use. Future studies with larger and more diverse samples are needed to more fully determine the psychometric characteristics of the TOE.

Neuropsychology consultation to identify learning disorders in children and adolescents: a proposal based on lessons learned during the COVID-19 pandemic

Learning disorders are common neurodevelopmental conditions, occurring both idiopathically and in the context of other medical conditions. They are frequently comorbid with other neurodevelopmental and psychiatric conditions. Delayed identification and treatment have been associated with significant negative psychosocial consequences. The need for pediatric neuropsychologists to efficiently screen for learning disorders is likely to increase in the months and years following the COVID-19 pandemic, which has severely disrupted access to educational services, especially for children who also face racial and economic disparities. In this paper, we describe a consultation model that can be used to screen for learning disorders and can be completed using both in-person and telemedicine visits. Implementation may result in earlier intervention for struggling children, increase access to neuropsychological services without increasing wait times for comprehensive evaluations, and provide opportunities for collaborations with other health professionals (e.g., pediatricians, therapists, psychiatrists, and neurologists).

Patterns of Performance and Symptom Validity Test Findings After Mild Traumatic Brain Injury

OBJECTIVE

The purpose of this study was to evaluate the presence of demographic, injury and neuropsychological correlates of distinct patterns of performance validity test and symptom validity test results in persons with mild traumatic brain injury (mTBI).

METHOD

One hundred and seventy-eight persons with mTBI completed the Test of Memory Malingering (TOMM; performance validity) and the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF; symptom validity) within 1-12 months postinjury. Four groups were compared: (a) pass both TOMM and MMPI-2-RF validity criteria, (b) pass TOMM and fail MMPI-2-RF, (c) fail TOMM and pass MMPI-2-RF, and (d) fail both TOMM and MMPI-2-RF.

RESULTS

Compared to Group a, participants in combined Groups b-d were more than twice as likely to be engaged in financial compensation-seeking and about four times less likely to have neuroimaging evidence of an intracranial lesion. The average performance of Group d on an independent test of verbal learning was more than 1.5 standard deviations below that of Group a. Participants in Group b were more likely to have intracranial lesions on neuroimaging than participants in Group c.

CONCLUSION

Performance and symptom validity tests provide complementary and non-redundant information in persons with mTBI. Whereas financial compensation-seeking is associated with increased risk of failure of either PVT or SVT, or both, the presence of intracranial findings on neuroimaging is associated with decreased risk of such.