Base Rates of Performance and Symptom Validity Test Failures in Active Duty and Veteran Samples Referred for Attention-Deficit/Hyperactivity Disorder Evaluation

OBJECTIVE

The objective of this study was to determine base rates of response bias in veterans and service members (SM) referred specifically for attention-deficit/hyperactivity disorder (ADHD) evaluation.

METHOD

Observational study of various performance validity tests (PVTs) and symptom validity tests (SVTs) in a sample of SMs (n = 94) and veterans (n = 504) referred for clinical evaluation of ADHD.

RESULTS

SVT and PVT failure rates were similar between the samples, but they were lower than previous Veterans Affairs (VA) and SM studies that were not exclusive to ADHD evaluations. Invalid reporting across all SVT scales on the Minnesota Multiphasic Personality Inventory and Personality Assessment Inventory was relatively uncommon, with rates of invalid scores falling at less than 7%. In both samples, free-standing PVTs were failed at about 22%.

CONCLUSIONS

Although the base rates of PVT and SVT failures in ADHD-specific evaluations were lower than previously published data on non-ADHD-specific evaluations in veterans and SMs, the current study continues to support the inclusion of these measures.

The Impact of Adverse Childhood Experiences on Symptom and Performance Validity Tests Among a Multiracial Sample Presenting for ADHD Evaluation

OBJECTIVE

Adverse childhood experiences (ACEs) are commonly reported in individuals presenting for attention-deficit hyperactivity disorder (ADHD) evaluation. Performance validity tests (PVTs) and symptom validity tests (SVTs) are essential to ADHD evaluations in young adults, but extant research suggests that those who report ACEs may be inaccurately classified as invalid on these measures. The current study aimed to assess the degree to which ACE exposure differentiated PVT and SVT performance and ADHD symptom reporting in a multi-racial sample of adults presenting for ADHD evaluation.

METHOD

This study included 170 adults referred for outpatient neuropsychological ADHD evaluation who completed the ACE Checklist and a neurocognitive battery that included multiple PVTs and SVTs. Analysis of variance was used to examine differences in PVT and SVT performance among those with high (≥4) and low (≤3) reported ACEs.

RESULTS

Main effects of the ACE group were observed, such that high ACE group reporting demonstrated higher scores on SVTs assessing ADHD symptom over-reporting and infrequent psychiatric and somatic symptoms on the Minnesota Multiphasic Personality Inventory-2-Restructured Form. Conversely, no significant differences emerged in total PVT failures across ACE groups.

CONCLUSIONS

Those with high ACE exposure were more likely to have higher scores on SVTs assessing over-reporting and infrequent responses. In contrast, ACE exposure did not affect PVT performance. Thus, ACE exposure should be considered specifically when evaluating SVT performance in the context of ADHD evaluations, and more work is needed to understand factors that contribute to different patterns of symptom reporting as a function of ACE exposure.

Is suboptimal effort an issue? A systematic review on neuropsychological performance validity in major depressive disorder

BACKGROUND

In Major Depressive Disorder (MDD), emotion- and motivation related symptoms may affect effort during neuropsychological testing. Performance Validity Tests (PVT's) are therefore essential, but are rarely mentioned in research on cognitive functioning in MDD. We aimed to assess the proportion of MDD patients with demonstrated valid performance and determine cognitive functioning in patients with valid performance. This is the first systematic review on neuropsychological performance validity in MDD.

METHODS

Databases PubMed, PsycINFO, Embase, and Cochrane Library were searched for studies reporting on PVT results of adult MDD patients. We meta-analyzed the proportion of MDD patients with PVT scores indicative of valid performance.

RESULTS

Seven studies with a total of 409 MDD patients fulfilled inclusion criteria. Six studies reported the exact proportion of patients with PVT scores indicative of valid performance, which ranged from 60 to 100 % with a proportion estimate of 94 %. Four studies reported on cognitive functioning in MDD patients with valid performance. Two out of these studies found memory impairment in a minority of MDD patients and two out of these studies found no cognitive impairment.

LIMITATIONS

Small number of studies and small sample sizes.

CONCLUSIONS

A surprisingly small number of studies reported on PVT in MDD. About 94 % of MDD patients in studies using PVT's had valid neuropsychological test performance. Concessive information regarding cognitive functioning in MDD patients with valid performance was lacking. Neuropsychological performance validity should be taken into account since this may alter conclusions regarding cognitive functioning.

Symptom and performance validity in neuropsychological assessments of outpatients 15-30 years of age

OBJECTIVE

It has been suggested that performance validity tests (PVTs) assess the credibility of a patient's objective test performance, symptom validity tests (SVTs) assess the credibility of a patient's subjective complaints, and that PVTs and SVTs are independent measures. The aim of this study was to test this hypothesis and investigate the associations of PVTs and SVTs with cognitive tests and self-reported measures.

METHOD

Patients 15-30 years of age were consecutively recruited from a neurorehabilitation outpatient clinic. The participants completed a battery of cognitive tests and self-reporting questionnaires as well as the Test of Memory Malingering (TOMM) and the Structured Inventory of Malingered Symptomatology (SIMS).

RESULTS

Forty-eight neurological patients of various etiologies were included. The TOMM (PVT) was related to cognitive tests across cognitive domains and unrelated to self-reporting measures. The SIMS (SVT) was related to other self-reported measures of perceived stress, anxiety, and depression, though also related to cognitive test of attention and processing speed. The PVT and the SVT were unrelated and hence seemed to measure two different constructs of validity.

CONCLUSIONS

This study supports the definition of PVTs and SVTs as two independent measures, consequently that both should be applied routinely in neuropsychological assessments.

The rate of psychiatric disorders in veterans undergoing intensive EEG monitoring is associated with symptom and performance invalidity

To determine if the number of participants with psychiatric disorders increased in association with failures on symptom validity tests (SVTs) and a performance validity test (PVT) in Veterans admitted for evaluation of possible seizures.

The 254 participants were Veterans undergoing inpatient video-EEG monitoring for the diagnosis of possible seizures. DSM-IV psychiatric disorders were diagnosed with the SCID IV. Symptom exaggeration was assessed with the MMPI-2-RF and performance validity with the TOMM.

On the MMPI-2-RF, 27.6%-32.7% showed symptom exaggeration. Participants who exaggerated on the MMPI-2-RF were more often diagnosed with psychiatric disorders. The TOMM was failed by 15.4% of the sample. Participants who failed the TOMM were more often diagnosed with an Axis I disorder but not with a personality disorder. The MMPI-2-RF was invalid in more cases than the TOMM, but 7.9% of the sample generated a valid MMPI-2-RF and an invalid TOMM.

The correlational design does not allow conclusions about cause and effect. The invalid groups may have had a higher rate of psychopathology. The number of participants with psychiatric disorders increased in association with symptom exaggeration and performance invalidity. Symptom exaggeration was more frequent than performance invalidity, but the TOMM made a unique contribution to identification of invalidity. The routine clinical use of SVTs and PVTs is supported. The results also suggest the need for caution in diagnosing psychiatric disorders when there is symptom exaggeration or performance invalidity, because diagnostic validity is dependent on the accuracy of symptom reporting.

Objective Cognitive Outcomes and Subjective Emotional Sequelae in Litigating Adults with a Traumatic Brain Injury: The Impact of Performance and Symptom Validity Measures

OBJECTIVE

This study examined the relative contribution of performance and symptom validity in litigating adults with traumatic brain injury (TBI), as a function of TBI severity, and examined the relationship between self-reported emotional symptoms and cognitive tests scores while controlling for validity test performance.

METHOD

Participants underwent neuropsychological assessment between January 2012 and June 2021 in the context of compensation-seeking claims related to a TBI. All participants completed a cognitive test battery, the Personality Assessment Inventory (including symptom validity tests; SVTs), and multiple performance validity tests (PVTs). Data analyses included independent t-tests, one-way ANOVAs, correlation analyses, and hierarchical multiple regression.

RESULTS

A total of 370 participants were included. Atypical PVT and SVT performance were associated with poorer cognitive test performance and higher emotional symptom report, irrespective of TBI severity. PVTs and SVTs had an additive effect on cognitive test performance for uncomplicated mTBI, but less so for more severe TBI. The relationship between emotional symptoms and cognitive test performance diminished substantially when validity test performance was controlled, and validity test performance had a substantially larger impact than emotional symptoms on cognitive test performance.

CONCLUSION

Validity test performance has a significant impact on the neuropsychological profiles of people with TBI, irrespective of TBI severity, and plays a significant role in the relationship between emotional symptoms and cognitive test performance. Adequate validity testing should be incorporated into every neuropsychological assessment, and associations between emotional symptoms and cognitive outcomes that do not consider validity testing should be interpreted with extreme caution.

The Relationship Between Cognitive Functioning and Symptoms of Depression, Anxiety, and Post-Traumatic Stress Disorder in Adults with a Traumatic Brain Injury: a Meta-Analysis

A thorough understanding of the relationship between cognitive test performance and symptoms of depression, anxiety, or post-traumatic stress disorder (PTSD) in people with traumatic brain injury (TBI) is important given the high prevalence of these emotional symptoms following injury. It is also important to understand whether these relationships are affected by TBI severity, and the validity of test performance and symptom report. This meta-analysis was conducted to investigate whether these symptoms are associated with cognitive test performance alterations in adults with a TBI. This meta-analysis was prospectively registered on the PROSPERO International Prospective Register of Systematic Reviews website (registration number: CRD42018089194). The electronic databases Medline, PsycINFO, and CINAHL were searched for journal articles published up until May 2020. In total, 61 studies were included, which enabled calculation of pooled effect sizes for the cognitive domains of immediate memory (verbal and visual), recent memory (verbal and visual), attention, executive function, processing speed, and language. Depression had a small, negative relationship with most cognitive domains. These relationships remained, for the most part, when samples with mild TBI (mTBI)-only were analysed separately, but not for samples with more severe TBI (sTBI)-only. A similar pattern of results was found in the anxiety analysis. PTSD had a small, negative relationship with verbal memory, in samples with mTBI-only. No data were available for the PTSD analysis with sTBI samples. Moderator analyses indicated that the relationships between emotional symptoms and cognitive test performance may be impacted to some degree by exclusion of participants with atypical performance on performance validity tests (PVTs) or symptom validity tests (SVTs), however there were small study numbers and changes in effect size were not statistically significant. These findings are useful in synthesising what is currently known about the relationship between cognitive test performance and emotional symptoms in adults with TBI, demonstrating significant, albeit small, relationships between emotional symptoms and cognitive test performance in multiple domains, in non-military samples. Some of these relationships appeared to be mildly impacted by controlling for performance validity or symptom validity, however this was based on the relatively few studies using validity tests. More research including PVTs and SVTs whilst examining the relationship between emotional symptoms and cognitive outcomes is needed.

Patterns of Performance and Symptom Validity Test Findings After Mild Traumatic Brain Injury

OBJECTIVE

The purpose of this study was to evaluate the presence of demographic, injury and neuropsychological correlates of distinct patterns of performance validity test and symptom validity test results in persons with mild traumatic brain injury (mTBI).

METHOD

One hundred and seventy-eight persons with mTBI completed the Test of Memory Malingering (TOMM; performance validity) and the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF; symptom validity) within 1-12 months postinjury. Four groups were compared: (a) pass both TOMM and MMPI-2-RF validity criteria, (b) pass TOMM and fail MMPI-2-RF, (c) fail TOMM and pass MMPI-2-RF, and (d) fail both TOMM and MMPI-2-RF.

RESULTS

Compared to Group a, participants in combined Groups b-d were more than twice as likely to be engaged in financial compensation-seeking and about four times less likely to have neuroimaging evidence of an intracranial lesion. The average performance of Group d on an independent test of verbal learning was more than 1.5 standard deviations below that of Group a. Participants in Group b were more likely to have intracranial lesions on neuroimaging than participants in Group c.

CONCLUSION

Performance and symptom validity tests provide complementary and non-redundant information in persons with mTBI. Whereas financial compensation-seeking is associated with increased risk of failure of either PVT or SVT, or both, the presence of intracranial findings on neuroimaging is associated with decreased risk of such.

Non-credible symptom report in the clinical evaluation of adult ADHD: development and initial validation of a new validity index embedded in the Conners' adult ADHD rating scales

As attention-deficit/hyperactivity disorder (ADHD) is a feasible target for individuals aiming to procure stimulant medication or accommodations, there is a high clinical need for accurate assessment of adult ADHD. Proven falsifiability of commonly used diagnostic instruments is therefore of concern. The present study aimed to develop a new, ADHD-specific infrequency index to aid the detection of non-credible self-report. Disorder-specific adaptations of four detection strategies were embedded into the Conners’ Adult ADHD Rating Scales (CAARS) and tested for infrequency among credible neurotypical controls (n = 1001) and credible adults with ADHD (n = 100). The new index’ ability to detect instructed simulators (n = 242) and non-credible adults with ADHD (n = 22) was subsequently examined using ROC analyses. Applying a conservative cut-off score, the new index identified 30% of participants instructed to simulate ADHD while retaining a specificity of 98%. Items assessing supposed symptoms of ADHD proved most useful in distinguishing genuine patients with ADHD from simulators, whereas inquiries into unusual symptom combinations produced a small effect. The CAARS Infrequency Index (CII) outperformed the new infrequency index in terms of sensitivity (46%), but not overall classification accuracy as determined in ROC analyses. Neither the new infrequency index nor the CII detected non-credible adults diagnosed with ADHD with adequate accuracy. In contrast, both infrequency indices showed high classification accuracy when used to detect symptom over-report. Findings support the new indices’ utility as an adjunct measure in uncovering feigned ADHD, while underscoring the need to differentiate general over-reporting from specific forms of feigning.

Performance and Symptom Validity Assessment in Patients with Apathy and Cognitive Impairment

OBJECTIVE

Performance and symptom validity tests (PVTs and SVTs) measure the credibility of the assessment results. Cognitive impairment and apathy potentially interfere with validity test performance and may thus lead to an incorrect (i.e., false-positive) classification of the patient's scores as non-credible. The study aimed at examining the false-positive rate of three validity tests in patients with cognitive impairment and apathy.

METHODS

A cross-sectional, comparative study was performed in 56 patients with dementia, 41 patients with mild cognitive impairment, and 41 patients with Parkinson's disease. Two PVTs - the Test of Memory Malingering (TOMM) and the Dot Counting Test (DCT) - and one SVT - the Structured Inventory of Malingered Symptomatology (SIMS) - were administered. Apathy was measured with the Apathy Evaluation Scale, and severity of cognitive impairment with the Mini Mental State Examination.

RESULTS

The failure rate was 13.7% for the TOMM, 23.8% for the DCT, and 12.5% for the SIMS. Of the patients with data on all three tests (n = 105), 13.5% failed one test, 2.9% failed two tests, and none failed all three. Failing the PVTs was associated with cognitive impairment, but not with apathy. Failing the SVT was related to apathy, but not to cognitive impairment.

CONCLUSIONS

In patients with cognitive impairment or apathy, failing one validity test is not uncommon. Validity tests are differentially sensitive to cognitive impairment and apathy. However, the rule that at least two validity tests should be failed to identify non-credibility seemed to ensure a high percentage of correct classification of credibility.

Illness Perceptions Predict Cognitive Performance Validity

OBJECTIVES

The aim of this study was to investigate the relationship of psychological variables to cognitive performance validity test (PVT) results in mixed forensic and nonforensic clinical samples.

METHODS

Participants included 183 adults who underwent comprehensive neuropsychological examination. Criterion groups were formed, that is, Credible Group or Noncredible Group, based upon their performance on the Word Memory Test and other stand-alone and embedded PVT measures.

RESULTS

Multivariate logistic regression analysis identified three significant predictors of cognitive performance validity. These included two psychological constructs, for example, Cogniphobia (perception that cognitive effort will exacerbate neurological symptoms), and Symptom Identity (perception that current symptoms are the result of illness or injury), and one contextual factor (forensic). While there was no interaction between these factors, elevated scores were most often observed in the forensic sample, suggesting that these independently contributing intrinsic psychological factors are more likely to occur in a forensic environment.

CONCLUSIONS

Illness perceptions were significant predictors of cognitive performance validity particularly when they reached very elevated levels. Extreme elevations were more common among participants in the forensic sample, and potential reasons for this pattern are explored. (JINS, 2018, 24, 735-745).

Differentiating Factitious from Malingered Symptomatology: the Development of a Psychometric Approach

Psychometric symptom validity assessment is becoming increasingly part and parcel of psychological and neuropsychological assessments. An unresolved and rarely addressed issue concerns the differentiation between factitious and malingered symptom presentations: present-day symptom validity tests can assess whether an examinee presents with noncredible symptomatology, but not why an examinee does so. We explored this issue by developing the Symptom and Disposition Interview (SDI); a symptom validity test that incorporates strategies intended to gauge internal incentives associated with factitious disorder. The merits of the SDI were explored and compared to a traditional symptom validity test (the Structured Inventory of Malingered Symptomatology) in two analogue studies, each with factitious and malingering conditions (n = 24-30 per condition) and a clinical control group (n = 34, n = 40). Overall, the results were positive: The SDI was as effective in detecting feigned symptom presentations as a traditional symptom validity test and superior in differentiating factitious from malingered symptom presentations. We conclude that the SDI is not ready for clinical use, but that psychometric approaches to the assessment of factitious symptomatology, such as the SDI, appear sufficiently promising to warrant future research.

Assessment of cognitive function in patients with metastatic cancer: Are we using the right tools?

OBJECTIVE

This study aimed at analyzing the validity and reliability of the continuous reaction time (CRT) test, the finger-tapping test (FTT), the Digit Span Test (DST), the Trail Making Test - part B (TMTB), and the Mini-Mental State Examination (MMSE) in patients with metastatic cancer.

METHOD

Eighty adult patients and 81 healthy controls were assessed between July of 2010 and November of 2015. The neuropsychological tests were analyzed regarding construct/discriminant/criterion validity and reliability.

RESULTS

In terms of construct validity, it was not possible to estimate a model for the MMSE because of a skewed response distribution. For discriminant validity, patients were slower on two measures of the CRT (p = 0.00483, p = 0.00030) and FTT dominant hand (p = 0.00306). Regarding sensitivity and specificity, only the DST and TMTB seemed to predict cognitive deficit; however, the ROC curve areas were ≤ 0.73. In terms of criterion validity, there were few significant correlations between the tests and the sociodemographic and clinical variables, and for the most part were very weak. Reliability was deemed to be adequate for the TMTB, DST, and FTT.

SIGNIFICANCE OF RESULTS

The findings of the full validation analyses were not clear-cut. However, CRT test, DST, FTT, and TMTB demonstrated partial positive results, indicating that these tests have good potential for use in clinical settings and require further study.

Word Memory Test Performance Across Cognitive Domains, Psychiatric Presentations, and Mild Traumatic Brain Injury

Objective

The current study addressed two aims: (i) determine how Word Memory Test (WMT) performance relates to test performance across numerous cognitive domains and (ii) evaluate how current psychiatric disorders or mild traumatic brain injury (mTBI) history affects performance on the WMT after excluding participants with poor symptom validity.

Method

Participants were 235 Iraq and Afghanistan-era veterans (Mage = 35.5) who completed a comprehensive neuropsychological battery. Participants were divided into two groups based on WMT performance (Pass = 193, Fail = 42). Tests were grouped into cognitive domains and an average z-score was calculated for each domain.

Results

Significant differences were found between those who passed and those who failed the WMT on the memory, attention, executive function, and motor output domain z-scores. WMT failure was associated with a larger performance decrement in the memory domain than the sensation or visuospatial-construction domains. Participants with a current psychiatric diagnosis or mTBI history were significantly more likely to fail the WMT, even after removing participants with poor symptom validity.

Conclusions

Results suggest that the WMT is most appropriate for assessing validity in the domains of attention, executive function, motor output and memory, with little relationship to performance in domains of sensation or visuospatial-construction. Comprehensive cognitive batteries would benefit from inclusion of additional performance validity tests in these domains. Additionally, symptom validity did not explain higher rates of WMT failure in individuals with a current psychiatric diagnosis or mTBI history. Further research is needed to better understand how these conditions may affect WMT performance.

Poor symptom and performance validity in regularly referred Hospital outpatients: Link with standard clinical measures, and role of incentives

We investigated the frequency of symptom validity test (SVT) failure and its clinical correlates in a large, heterogeneous sample of hospital outpatients referred for psychological assessment for clinical purposes. We studied patients (N=469), who were regularly referred for assessment to the psychology departments of five hospitals. Background characteristics, including information about incentives, were obtained with a checklist completed by the clinician. As a measure of over-reporting, the Structured Inventory of Malingered Symptomatology (SIMS) was administered to all patients. The Amsterdam Short-Term Memory test (ASTM), a cognitive underperformance measure, was only administered to patients who were referred for a neuropsychological assessment. Symptom over-reporting occurred in a minority of patients, ranging from 12% to 19% in the main diagnostic patient groups. Patients with morbid obesity had a low rate of over-reporting (1%). The SIMS was positively associated with levels of self-reported psychological symptoms. Cognitive underperformance occurred in 29.3% of the neuropsychological assessments. The ASTM was negatively associated with memory test performance. We found no association between SVT failure and financial incentives. Our results support the recommendation to routinely evaluate symptom validity in clinical assessments of hospital patients. The dynamics behind invalid symptom reporting need to be further elucidated.

Cumulative false positive rates given multiple performance validity tests: commentary on Davis and Millis (2014) and Larrabee (2014)

Controversy has arisen over interpretation of performance validity tests (PVTs) when multiple PVTs are given. Some papers state that more stringent criteria are needed to judge overall performance as invalid, while others argue that concerns about the number of PVTs are overstated and that widely used criteria are appropriate. We examine theoretical models and assumptions, and analyze published data to determine the magnitude of effects implied by theory and observed in practice. Assertions advanced in the primary papers are examined for consistency with the empirical data. Existing theoretical models do not account well for the diverse empirical data, substantial empirical effects remain poorly understood, and the primary papers include assertions that are not empirically supported. The results indicate that: (a) neuropsychology lacks solid theoretical bases for estimating PVT failure rates given various combinations of PVTs, and thus needs to rely on empirical data; (b) existing empirical data fail to support the application of any uniform criteria across the broad range of scenarios involving multiple PVTs; and (c) practice should rely on empirical studies involving combinations of PVTs that have been studied together, in samples clearly appropriate to the individual case, using experimental designs germane to the questions under consideration.

Utility of the test of memory malingering (TOMM) in children ages 4-7 years with and without ADHD

There is growing consensus that assessment for non-credible performance is a necessary component of pediatric neuropsychological examination. The current study examined the utility and validity of the Test of Memory Malingering (TOMM) in children ages 4-7 years with and without Attention-deficit/Hyperactivity Disorder (ADHD); 66 children (30 controls, 36 ADHD) completed all three TOMM trials. There were no significant group differences in total score on any trial, or passing rate for Trial 2 or Retention. Four-year-olds with ADHD achieved "passing" score on Trial 1 less often than controls. Across groups, performance on Trial 2 and Retention improved with age, such that 85% of the sample achieved a passing score. Four-year-olds had greater difficulty and achieved a passing score significantly less often than children 5-7 years. Moreover, half of the 4-year-olds performed worse on Retention than Trial 2, calling into question the utility of the Retention trial at this age. Performance was associated with IQ only within the ADHD group on the Retention trial. Results suggest that the TOMM can be used with confidence in clinical groups as young as 5 years. Among 4-year-olds, performance appears dependent on severity of ADHD or disruptive behaviors, and may be associated with factors other than effort.