A survey of neuropsychologists’ use of validity tests with children and adolescents

A comparison of MVP and MSVT performance among a diverse clinical pediatric sample

The Memory Validity Profile (MVP) and Medical Symptom Validity Test (MSVT) are performance validity tests (PVTs) used to identify potential noncredible test performance during psychological evaluations. This study sought to examine the agreement between MVP and MSVT pass rates, as well as to determine if there are differences in MVP pass rates when using the cutoff score in the MVP professional manual compared with the experimental cutoff score of <31. Via retrospective review of records, 106 clients at a private neuropsychological clinic who had been given the MVP and the MSVT were identified. Results indicated that only one client met the manual cutoff scores, compared to 20 clients who failed the MSVT, raising concerns regarding the sensitivity of the MVP. Utilizing the receiver operator characteristic (ROC), curve analyses indicated fair discriminability of the MVP for the 106 participants (AUC = .717) with acceptable sensitivity (.50) and specificity (.92) for an MVP total score cutoff of <31. These findings support the utility of the experimental cut score in improving the sensitivity while maintaining adequate specificity in a clinically mixed population.

Performance and Symptom Validity in Outpatient Pediatric Neuropsychological Evaluations

This study aimed to clarify the value of using different types of validity measures in pediatric neuropsychological evaluations. We examined the relationship between performance (PVT) and symptom (SVT) validity tests as well as demographic variables and results from a screening test of learning and memory (i.e. Child and Adolescent Memory Profile [ChAMP]) in a mixed pediatric sample (n = 103). There was minimal overlap between PVT and SVT failures. Regression analyses demonstrated that PVT results, parental education, and history of special education were statistically significant predictors of ChAMP results, whereas SVT results were not.

New Child and Adolescent Memory Profile Embedded Performance Validity Test

OBJECTIVE

It is essential to interpret performance validity tests (PVTs) that are well-established and have strong psychometrics. This study evaluated the Child and Adolescent Memory Profile (ChAMP) Validity Indicator (VI) using a pediatric sample with traumatic brain injury (TBI).

METHOD

A cross-sectional sample of N = 110 youth (mean age = 15.1 years, standard deviation [SD] = 2.4 range = 8-18) on average 32.7 weeks (SD = 40.9) post TBI (71.8% mild/concussion; 3.6% complicated mild; 24.6% moderate-to-severe) were administered the ChAMP and two stand-alone PVTs. Criterion for valid performance was scores above cutoffs on both PVTs; criterion for invalid performance was scores below cutoffs on both PVTs. Classification statistics were used to evaluate the existing ChAMP VI and establish a new VI cutoff score if needed.

RESULTS

There were no significant differences in demographics or time since injury between those deemed valid (n = 96) or invalid (n = 14), but all ChAMP scores were significantly lower in those deemed invalid. The original ChAMP VI cutoff score was highly specific (no false positives) but also highly insensitive (sensitivity [SN] = .07, specificity [SP] = 1.0). Based on area under the curve (AUC) analysis (0.94), a new cutoff score was established using the sum of scaled scores (VI-SS). A ChAMP VI-SS score of 32 or lower achieved strong SN (86%) and SP (92%). Using a 15% base rate, positive predictive value was 64% and negative predictive value was 97%.

CONCLUSIONS

The originally proposed ChAMP VI has insufficient SN in pediatric TBI. However, this study yields a promising new ChAMP VI-SS, with classification metrics that exceed any other current embedded PVT in pediatrics.

The Temporal Memory Sequence Test (TMST) in children: Validity test performance in clinically referred children

Validity evaluation is fundamental in neuropsychological assessment in adults, with increasing interest among pediatric neuropsychologists. Although some measures exist, given time constraints placed on clinicians, and children's limited sustained attention, development of less time-consuming measures is beneficial. We explored the use of the Temporal Memory Sequence Test (TMST), a new performance validity test, in clinically referred children. One minor adaptation included reading the instructions and labels to non-fluent readers. Participants were 68 consecutive clinically referred children and adolescents, aged 6-18 years, with neurological (n = 46) or behavioral (n = 22) difficulties. Applying the adult cutoff, 83.8% passed the TMST. Age, gender, and diagnosis did not differ between children passing the TMST cutoff and those who failed it. Classification accuracy calculated against three embedded measures of performance validity (Wechsler scale Digit Span, Coding, and Processing Speed Index) indicated specificity over 90% (Digit Span: 94%, Coding: 96%, Processing Speed Index: 92%) and sensitivity between 30 and 33%. For individuals without Intellectual Disability (ID), 90.9% passed the TMST, and intelligence did not predict success. Thus, the use of the TMST with the adult cutoff was supported in children without ID, offering an additional validity measure for clinically referred children.

Updating 'Stories' on social media and its relationships to contextual age and narcissism: A tale of three platforms - WhatsApp, Instagram and Facebook

This study extends the uses and gratifications research into the feature use of social media platforms by identifying the motives for updating ‘Stories’ on three social media platforms, i.e. WhatsApp, Instagram and Facebook. Using survey data from adults (N = 338), this research identified seven motivations (i.e. socially rewarding self-promotion, social sharing, social influence, disclosure, escape, entertainment, and trendy fashion) for updating Stories on WhatsApp, Instagram and Facebook. Age and gender differences were identified in the motivations for updating Stories on the three platforms. Cross-platform analysis revealed significant differences in the motivations for updating Stories across the three platforms. Social influence and disclosure motives positively predicted the intensity of updating Stories on all three platforms. Socially rewarding self-promotion predicted the intensity of updating Stories on Instagram and Facebook. Social sharing positively predicted the intensity of updating Stories on WhatsApp and Instagram, and trendy fashion emerged as a positive predictor of story updates only on Instagram. Finally, an analysis of social and psychological predictors revealed that Interpersonal interaction and Social activity positively predicted the intensity of updating Stories across the three platforms. Narcissists also exhibited high intensity of updating Stories on WhatsApp, Instagram, and Facebook.

American Academy of Clinical Neuropsychology (AACN) 2021 consensus statement on validity assessment: Update of the 2009 AACN consensus conference statement on neuropsychological assessment of effort, response bias, and malingering

Objective: Citation and download data pertaining to the 2009 AACN consensus statement on validity assessment indicated that the topic maintained high interest in subsequent years, during which key terminology evolved and relevant empirical research proliferated. With a general goal of providing current guidance to the clinical neuropsychology community regarding this important topic, the specific update goals were to: identify current key definitions of terms relevant to validity assessment; learn what experts believe should be reaffirmed from the original consensus paper, as well as new consensus points; and incorporate the latest recommendations regarding the use of validity testing, as well as current application of the term 'malingering.' Methods: In the spring of 2019, four of the original 2009 work group chairs and additional experts for each work group were impaneled. A total of 20 individuals shared ideas and writing drafts until reaching consensus on January 21, 2021. Results: Consensus was reached regarding affirmation of prior salient points that continue to garner clinical and scientific support, as well as creation of new points. The resulting consensus statement addresses definitions and differential diagnosis, performance and symptom validity assessment, and research design and statistical issues. Conclusions/Importance: In order to provide bases for diagnoses and interpretations, the current consensus is that all clinical and forensic evaluations must proactively address the degree to which results of neuropsychological and psychological testing are valid. There is a strong and continually-growing evidence-based literature on which practitioners can confidently base their judgments regarding the selection and interpretation of validity measures.

A Systematic Review and Meta-Analysis of the Utility of the Test of Memory Malingering in Pediatric Examinees

Objective

This is the first systematic review and meta-analysis of the Test of Memory Malingering (TOMM) in pediatric examinees. It adheres to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

Method

A systematic literature search was conducted using PsycINFO and PubMed, reviewing articles from January 1997 to July 2019. Books providing data on pediatric validity testing were also reviewed for references to relevant articles. Eligibility criteria included publication in a peer-reviewed journal, utilizing a pediatric sample, providing sufficient data to calculate specificity and/or sensitivity, and providing a means for evaluating validity status external to the TOMM. After selection criteria were applied, 9 articles remained for meta-analysis. Samples included clinical patients and healthy children recruited for research purposes; ages ranged from 5 to 18. Fixed and random effects models were used to calculate classification accuracy statistics.

Results

Traditional adult-derived cutoffs for Trial 2 and Retention were highly specific (0.96–0.99) in pediatric examinees for both clinical and research samples. Sensitivity was relatively strong (0.68–0.70), although only two studies reported sensitivity rates. A supplemental review of the literature corroborated these findings, revealing that traditional adult-based TOMM cutoffs are supported in most pediatric settings. However, limited research exists on the impact of very young age, extremely low cognitive functioning, and varying clinical diagnoses.

Conclusions

The TOMM, at traditional adult cutoffs, has strong specificity as a performance validity test in pediatric neuropsychological evaluations. This meta-analysis found that specificity values in children are comparable to those of adults. Areas for further research are discussed.

Using the Woodcock-Johnson IV tests of cognitive abilities to detect feigned ADHD

Research has suggested that many young adults can successfully feign ADHD, reporting clinically significant symptom levels and displaying deficits on cognitive tasks when asked to do so. Standalone performance validity tests (PVTs) have shown some success in identifying feigned ADHD, but these tests are rarely used in typical ADHD evaluation batteries. The present study attempted to develop embedded PVT indices from the Woodcock-Johnson IV Tests of Cognitive Abilities (WJ-IV). College students (N = 150) completed a battery including tasks from the WJ-IV, as well as an established standalone PVT and a rating scale measuring ADHD and related symptoms. Thirty of the students had been professionally diagnosed with ADHD; of the remaining 120 students, half were asked to perform honestly and to the best of their ability on the battery, whereas the other half were asked to try to simulate ADHD. Several processing speed and working memory scores from the WJ-IV effectively identified students feigning ADHD, detecting at least 50% of those students at score cutoffs that also maintained specificity of 90% or more, close to the efficiency of the standalone PVT. In addition, students with ADHD diagnoses generally did not show deficits on the WJ-IV. Implications for practice and future research are discussed.

The use of the MSVT in children and adolescents with epilepsy

Pediatric neuropsychologists are increasingly recognizing the importance of performance validity testing during evaluations. The use of such measures to detect insufficient effort is of particular importance in pediatric epilepsy evaluations, where test results are often used to guide surgical decisions and failure to detect poor task engagement can result in postsurgical cognitive decline. The present investigation assesses the utility of the Medical Symptom Validity Test (MSVT) in 104 clinically referred children and adolescents with epilepsy. Though the overall failure rate was 15.4% of the total group, children with 2nd grade or higher reading skills (a requirement of the task) passed at a very high rate (96.6%). Of the three failures, two were unequivocally deemed true positives, while the third failed due to extreme somnolence during testing. Notably, for those with ≥2nd grade reading levels, MSVT validity indices were unrelated to patient age, intellectual functioning, or age of epilepsy onset, while modest relations were seen with specific memory measures, number of epilepsy medications, and seizure frequency. Despite these associations, however, this did not result in more failures in this population of children and adolescents with substantial neurologic involvement, as pass rates exceeded 92% for those with intellectual disability, high seizure frequency, high medication burden, and even prior surgical resection of critical memory structures.

Noncredible effort on the Nonverbal-Medical Symptom Validity Test (NV-MSVT): Impact on cognitive performance in pediatric mild traumatic brain injury

This study examined failure rates on the Nonverbal-Medical Symptom Validity Test (NV-MSVT) and its impact on cognitive performance in a sample of youths with mild traumatic brain injury (mTBI). Participants were 184 children and adolescents who presented to a multidisciplinary concussion clinic for a targeted neuropsychological evaluation. Performance Validity Tests (PVTs) were a part of the standard battery, including the NV-MSVT. Twenty-eight participants (15.2%) failed the NV-MSVT, none of whom displayed a genuine memory impairment profile (GMIP). Participants who failed the NV-MSVT performed significantly worse than those who passed the NV-MSVT on measures of IQ, memory, and immediate attention/working memory. There was no significant difference between groups on processing speed, sustained attention, cognitive flexibility, or sight word reading level. Aside from a slight difference in age, NV-MSVT failure was not impacted by demographic variables (sex, race), premorbid risk factors (pre-injury ADHD, learning disabilities, psychiatric diagnoses or treatment, developmental delays, or prior special education), injury-related variables (time since injury, positive neuroimaging findings, post-traumatic amnesia, number of prior mTBIs, etc.) or post-mTBI anxiety/depression. That said, participants who failed NV-MSVT endorsed significantly more severe postconcussive symptoms. These findings support the use of the NV-MSVT in neuropsychological evaluation of children and adolescents with mTBI.

Assessing performance validity with the TOMM and automatized sequences task in a pediatric psychiatric inpatient setting

This study examined the performance of children consecutively admitted to an inpatient psychiatric unit on the Test of Memory Malingering (TOMM) (aged 5-12; n = 96) and Automatized Sequences Task (aged 8-12; n = 67). Eighty-three percent of children passed the TOMM Trial 2 (M raw score = 47.7, SD = 4.7) and 76% of children passed the Automatized Sequences Task total time (M = 23.1 seconds; SD = 8.2). The concordance rate between the TOMM and the total time on the Automatized Sequences Task was 73.1%. Receiver operating characteristic curves indicated that of the Automatized Sequences Task subtests, only Counting 1-20 significantly differentiated children who passed Trial 2 of the TOMM from those who did not pass Trial 2 of the TOMM (area under the curve = .756, p = .006). Performance on both PVTs was unrelated to demographic characteristics and measures of psychological and neuropsychological functioning on both the TOMM and Automatized Sequences Task. Further research is needed to determine whether the nearly 1 in 4 children (23.9%) who performed below recommended cutoffs on Automatized Sequences Task reflects genuine suboptimal effort, cognitive difficulties among these children, and/or other factors.

Rey 15 item test plus recognition trial and TOMM in a community pediatric sample

In pediatric evaluations, performance validity test (PVT) selection is often constrained by reading level, developmental appropriateness of stimuli, and administration time. The Rey 15 Item Test (FIT) addresses these constraints, and ranks among the most frequently used PVTs. Unfortunately, research indicates poor sensitivity of the FIT recall trial. Boone et al. developed a FIT recognition trial and demonstrated in an adult sample that its use increased sensitivity while maintaining high specificity. These results are promising, but, to the authors' knowledge, have only been replicated once in a pediatric sample. The present study examined the FIT plus recognition trial in a sample of 72 young athletes ages 8-16 years. All data for the present study were collected during baseline cognitive evaluations. The Test of Memory Malingering (TOMM) was used as the comparison criterion. Receiver operating characteristic curve analyses showed the addition of the recognition trial did not substantially improve sensitivity of the FIT. There was a surprising lack of concordance between TOMM and FIT scores, and, whereas the FIT correlated with multiple cognitive measures, the TOMM did not correlate with any other measures. Results suggest the FIT is not appropriate for pediatric clinical care, even with the additional recognition trial.

They're not just little adults: The utility of adult performance validity measures in a mixed clinical pediatric sample

Performance validity tests (PVTs) have become a standard part of adult neuropsychological practice; however, they are less widely used in pediatric testing. The current study aimed to obtain a better understanding of the application of PVTs within a mixed clinical pediatric sample with a wide range of diagnosis, IQ, and age. Cross-sectional data were analyzed from 130 consecutive pediatric patients evaluated as part of clinical care and diagnosed with a variety of medical/neurological, developmental, and psychiatric disorders. Patients were administered a battery of neuropsychological tests; results of intellectual functioning measures (i.e., Wechsler Intelligence Scale for Children-Fifth Edition [WISC-V] or Wechsler Adult Intelligence Scale-Fourth Edition [WAIS-IV]), and PVTs (i.e., Test of Memory Malingering [TOMM] and Digit Span [DS] subtests of the WISC-V/WAIS-IV) were analyzed to assess PVT performance across the sample as well as age- and Full-Scale IQ-related (FSIQ) effects on pass rate. Results suggested that the TOMM is an effective validity test for youth, as the TOMM adult cutoff score was also valid for children (88% pass rate on TOMM trial 1 cut-score ≥41, 71% pass rate on TOMM trial 1 cut-score ≥45). In contrast, Reliable Digit Span (RDS) was less accurate (34% failed RDS [cut-score ≤6], 54% failed RDS-r [cut-score ≤10], and 25% failed DS ACSS [cut-score ≤5]) using standard adult cutoffs. Notably, although TOMM scores were not strongly influenced by IQ, DS scores increased as IQ increased. Overall, further analysis of PVTs can champion new standards of practice through additional research establishing PVT accuracy within pediatric populations.

Detecting Feigned Cognitive Impairment With Schretlen’s Malingering Scale Vocabulary and Abstraction Test

Abstract. Schretlen’s Malingering Scale Vocabulary and Abstraction test (MSVA) differs from the majority of performance validity tests in that it focuses on the detection of feigned impairments in semantic knowledge and perceptual reasoning rather than feigned memory problems. We administered the MSVA to children ( n = 41), forensic inpatients with intellectual disability ( n = 25), forensic inpatients with psychiatric symptoms ( n = 57), and three groups of undergraduate students ( n = 30, n = 79, and n = 90, respectively), asking approximately half of each of these samples to feign impairment and the other half to respond genuinely. With cutpoints chosen so as to keep false-positive rates below 10%, detection rates of experimentally feigned cognitive impairment were high in children (90%) and inpatients with intellectual disability (100%), but low in adults without intellectual disability (46%). The rates of significantly below-chance performance were low (4%), except in children (47%) and intellectually disabled inpatients (50%). The reliability of the MSVA was excellent (Cronbach’s α = .93–.97) and the MSVA proved robust against coaching (i.e., informed attempts to evade detection while feigning). We conclude that the MSVA is not ready yet for clinical use, but that it shows sufficient promise to warrant further validation efforts.

"Must try harder." Is effort and performance validity testing a necessary part of pediatric neuropsychological assessment?

Neuropsychological assessments results have significant implications for pediatric populations, based upon the assumption that the young person has adopted an effortful approach and has engaged in assessment. There is a commonly-accepted risk to assuming the validity of neuropsychological assessment results with adults, and, therefore, performance validity testing (PVT) has become a major topic of research and investigation and has become an accepted part of routine assessment. The same approach has not been adopted in assessment with children and a paucity of studies has focused on PVT in children. We review studies that demonstrate that children are equal to adults in their ability to use deception and that clinicians cannot detect false-effort without use of validity tests. We explore how frequently such tests are used and how well they work in assessment with children, and the limits, complexities, and constraints of adapting adult tests. We advocate that adequate performance validity testing is essential in order to maximize confidence in the results and we hypothesize that assessment with pediatric populations should take into account a range of influences, such as neuro-developmental factors associated with age of the child and suitability of proposed measures according to the evidence-base.

Construct validity for the verbal associated pairs screen

The present study introduces the Verbal Associated Pairs Screen (VAPS) as a new measure for assessing performance validity in pediatric populations. This study presents initial data on psychometric properties and establishes construct validity for the VAPS in a sample of 30 adolescent healthy controls and 206 youths with traumatic brain injury (TBI: moderate/severe, N = 30; mild, N = 176). The control group's age (M = 14.93, SD = 1.8) was significantly higher than the moderate/severe TBI (M = 13.9, SD = 2.8), t(68.508) = -3.038, p = .003, and mild TBI (mTBI) groups (M = 14, SD = 2.8), t(54.147) = 2.038, p = .046. The TBI groups were administered the VAPS in accord with other established performance validity tests (PVTs) and well-established  memory tests as part of routine clinical evaluations. The healthy control group was administered the VAPS only. VAPS score distributions for the control group were negatively skewed and highly kurtotic. VAPS scores from the moderate/severe TBI and control groups were indistinguishable for Trial 2 (U = 274, p < .01) and the Delay (U = 396, p = .218). In the mTBI group, convergent and divergent validity was established with other well-validated PVTs and memory tests, respectively. ROC curve analyses identified optimal cutoff scores for the VAPS Total Score, with acceptable sensitivity (55%) and excellent specificity (100%), as well as strong detectability (AUC = .829, 95% CI: 0.731 - 0.928, p < .001). Clinical applications, limitations, and directions for future research with the VAPS are discussed.

Use of the 21-Item Test with children and adolescents 5 to 16 years of age

The goals of the present study were to: (a) collect preliminary normative data on the 21-Item Test in children and adolescents and to determine if cutoffs for biased responding in adults can be generalized to a younger population; (b) determine if Caucasian and Hispanic children perform differently on the test; and (c) ascertain the relationships of age and verbal intelligence with effort test performance. The 21-Item Test was administered to 153 children attending either a public (n = 96) or private (n = 57) school. The Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV) was also administered to the private school sample. The results indicated that the 21-Item Test has potential as a tool for identifying children who do not put forth maximal effort during formal assessment and that the adult cutoffs appear applicable to children 8 years and older. Caucasian and Hispanic participants performed similarly on the test. Forced-choice scores ≤12 were associated with age and lower levels of performance on the Verbal Comprehension Index of the WISC-IV.

The impact of pediatric neuropsychological consultation in mild traumatic brain injury: a model for providing feedback after invalid performance

OBJECTIVE

In recent years, pediatric practitioners have increasingly recognized the importance of objectively measuring performance validity during clinical assessments. Yet, no studies have examined the impact of neuropsychological consultation when invalid performance has been identified in pediatric populations and little published guidance exists for clinical management. Here we provide a conceptual model for providing feedback after noncredible performance has been detected. In a pilot study, we examine caregiver satisfaction and postconcussive symptoms following provision of this feedback for patients seen through our concussion program.

METHODS

Participants (N = 70) were 8-17-year-olds with a history of mild traumatic brain injury who underwent an abbreviated neuropsychological evaluation between 2 and 12 months post-injury. We examined postconcussive symptom reduction and caregiver satisfaction after neuropsychological evaluation between groups of patients who were determined to have provided noncredible effort (n = 9) and those for whom no validity concerns were present (n = 61).

RESULTS

We found similarly high levels of caregiver satisfaction between groups and greater reduction in self-reported symptoms after feedback was provided using the model with children with noncredible presentations compared to those with credible presentations.

CONCLUSION

The current study lends preliminary support to the idea that the identification and communication of invalid performance can be a beneficial clinical intervention that promotes high levels of caregiver satisfaction and a reduction in self-reported and caregiver-reported symptoms.