Portland digit recognition test-computerized: Measuring response latency improves the detection of malingering

Preliminary validation of the computerized N-Tri - A Tri-Choice naming and response bias test

The study describes the validation of a computerized adaptation of the novel Tri-Choice Naming and Response Bias Measure (N-Tri) developed to detect untruthful responding while being less susceptible to coaching than existing measures. We hypothesized that the N-Tri would have comparable sensitivity and specificity to traditional tests but would have improved accuracy for detecting coached simulators. Four-hundred volunteers were randomly assigned to one of three groups: uncoached simulators' group (n = 118), coached simulators' group (n = 136), or control group (n = 146). Both simulator groups were asked to feign concussion symptoms, but the coached group received a test-taking strategy and a description of concussion symptoms. The participants were administered the computerized version of the new measure in conjunction with computerized adaptations of two well-validated response bias tests commonly used to detect cognitive malingering, the Reliable Digit Span (RDS) and Portland Digit Recognition Test (PDRT). Our data show the new measure correlated highly with other established measures. However, the classification accuracy did not significantly increase when compared to the traditional tests. Our findings support that the N-Tri performs at a comparable level to existing forced choice measures of response bias. Nevertheless, the N-Tri could potentially improve the detection of response bias as existing tests become more recognizable by the public.

Development and Initial Validation of the Perceptual Assessment of Memory (PASSOM): A Simulator Study

OBJECTIVE

Performance validity tests (PVTs) are an integral component of neuropsychological assessment. There is a need for the development of more PVTs, especially those employing covert determinations. The aim of the present study was to provide initial validation of a new computerized PVT, the Perceptual Assessment of Memory (PASSOM).

METHOD

Participants were 58 undergraduate students randomly assigned to a simulator (SIM) or control (CON) group. All participants were provided written instructions for their role prior to testing and were administered the PASSOM as part of a brief battery of neurocognitive tests. Indices of interest included response accuracy for Trials 1 and 2, and total errors across Trials, as well as response time (RT) for Trials 1 and 2, and total RT for both Trials.

RESULTS

The SIM group produced significantly more errors than the CON group for Trials 1 and 2, and committed more total errors across trials. Significantly longer response latencies were found for the SIM group compared to the CON group for all RT indices examined. Linear regression modeling indicated excellent group classification for all indices studied, with areas under the curve ranging from 0.92 to 0.95. Sensitivity and specificity rates were good for several cut scores across all of the accuracy and RT indices, and sensitivity improved greatly by combining RT cut scores with the more traditional accuracy cut scores.

CONCLUSION

Findings demonstrate the ability of the PASSOM to distinguish individuals instructed to feign cognitive impairment from those told to perform to the best of their ability.

Performance validity assessment using response time on the Warrington Recognition Memory Test

OBJECTIVE

The present study tested the incremental utility of response time (RT) on the Warrington Recognition Memory Test - Words (RMT-W) in classifying bona fide versus feigned TBI.

METHOD

Participants were 173 adults: 55 with moderate to severe TBI, 69 healthy comparisons (HC) instructed to perform their best, and 49 healthy adults coached to simulate TBI (SIM). Participants completed a computerized version of the RMT-W in the context of a comprehensive neuropsychological battery. Groups were compared on RT indices including mean RT (overall, correct trials, incorrect trials) and variability, as well as the traditional RMT-W accuracy score.

RESULTS

Several RT indices differed significantly across groups, although RMT-W accuracy predicted group membership more strongly than any individual RT index. SIM showed longer average RT than both TBI and HC. RT variability and RT for incorrect trials distinguished SIM-HC but not SIM-TBI comparisons. In general, results for SIM-TBI comparisons were weaker than SIM-HC results. For SIM-HC comparisons, classification accuracy was excellent for all multivariable models incorporating RMT-W accuracy with one of the RT indices. For SIM-TBI comparisons, classification accuracies for multivariable models ranged from acceptable to excellent discriminability. In addition to mean RT and RT on correct trials, the ratio of RT on correct items to incorrect items showed incremental predictive value to accuracy.

CONCLUSION

Findings support the growing body of research supporting the value of combining RT with PVTs in discriminating between verified and feigned TBI. The diagnostic accuracy of the RMT-W can be improved by incorporating RT.

Response Time Measures as Supplementary Validity Indicators in Forced-Choice Recognition Memory Performance Validity Tests: A Systematic Review

Performance validity tests (PVTs) based on the forced-choice recognition memory (FCRM) paradigm are commonly used for the detection of noncredible performance. Examinees' response times (RTs) are affected by cognitive processes associated with deception and can also be gathered without lengthening the duration of the assessment. Consequently, interest in the utility of these measures as supplementary validity indicators in FCRM-PVTs has grown over the years. The current systematic review summarizes both clinical and simulation (i.e., healthy participants simulating cognitive impairment) studies of RTs in FCRM-PVTs. The findings of 25 peer-reviewed articles (n = 26 empirical studies) indicate that noncredible performance in FCRM-PVTs is associated with longer RTs. Additionally, there are indications that noncredible performance is associated with larger variability in RTs. RT measures, however, have lower discrimination capacity than conventional accuracy measures. Their utility may therefore lie in reaching decisions regarding cases with border zone accuracy scores, as well as aiding in the detection of more sophisticated examinees who are aware of the use of accuracy-based validity indicators in FCRM-PVTs. More research, however, is required before these measures are incorporated in daily practice and clinical decision-making processes.

Discriminating Fake From True Brain Injury Using Latency of Left Frontal Neural Responses During Old/New Memory Recognition

Traumatic brain injury (TBI) is a major public health concern that affects 69 million individuals each year worldwide. Neuropsychologists report that up to 40% of individuals undergoing evaluations for TBI may be malingering neurocognitive deficits for a compensatory reward. The memory recognition test of malingering detection is effective but can be coached behaviorally. There is great need to develop a novel neural based method for discriminating fake from true brain injury. Here we test the hypothesis that decision making of faking memory deficits prolongs frontal neural responses. We applied an advanced method measuring decision latency in milliseconds for discriminating true TBI from malingerers who fake brain injury. To test this hypothesis, latencies of memory-related brain potentials were compared among true patients with moderate or severe TBI, and healthy age-matched individuals who were assigned either to be honest or faking memory deficit. Scalp signals of electroencephalography (EEG) were recorded with a 32-channel cap during an Old/New memory recognition task in three age- and education-matched groups: honest (n = 12), malingering (n = 15), and brain injured (n = 14) individuals. Bilateral fractional latencies of late positive ERP at frontal sites were compared among the three groups under both studied (Old) and non-studied (New) memory recognition conditions. Results show a significant difference between the fractional latencies of the late positive component during recognition of studied items in malingerers (averaged latencies = 396 ms) and the true brain injured subjects (mean = 312 ms) in the frontal sites. Only malingers showed asymmetrical frontal activity compared to the two other groups. These new findings support the hypothesis that that additional frontal processing of malingering individuals is measurably different from those of actual patients with brain injury. In contrast to our previous reported method using difference waves of amplitudes at frontal to posterior midline sites during new items recognition (Vagnini et al., 2008), there was no significant latency difference among groups during recognition of New items. The current method using delayed left frontal neural responses during studied items reached sensitivity of 80% and specificity of 79% in detecting malingers from true brain injury.

Reaction time as an indicator of insufficient effort: Development and validation of an embedded performance validity parameter

Subnormal performance in attention tasks may result from various sources including lack of effort. In this report, the derivation and validation of a performance validity parameter for reaction time is described, using a set of malingering-indices ("Slick-criteria"), and 3 independent samples of participants (total n =893). The Slick-criteria yield an estimate of the probability of malingering based on the presence of an external incentive, evidence from neuropsychological testing, from self-report and clinical data. In study (1) a validity parameter is derived using reaction time data of a sample, composed of inpatients with recent severe brain lesions not involved in litigation and of litigants with and without brain lesion. In study (2) the validity parameter is tested in an independent sample of litigants. In study (3) the parameter is applied to an independent sample comprising cooperative and non-cooperative testees. Logistic regression analysis led to a derived validity parameter based on median reaction time and standard deviation. It performed satisfactorily in studies (2) and (3) (study 2 sensitivity=0.94, specificity=1.00; study 3 sensitivity=0.79, specificity=0.87). The findings suggest that median reaction time and standard deviation may be used as indicators of negative response bias.

Initial clinical validation of an embedded performance validity measure within the automated neuropsychological metrics (ANAM)

The measurement of effort and performance validity is essential for computerized testing where less direct supervision is needed. The clinical validation of an Automated Neuropsychological Metrics-Performance Validity Index (ANAM-PVI) was examined by converting ANAM test scores into a common metric based on their relative infrequency in an outpatient clinic sample with presumed good effort. Optimal ANAM-PVI cut-points were determined using receiver operator characteristic (ROC) curve analyses and an a priori specificity of 90%. Sensitivity/specificity was examined in available validation samples (controls, simulators, and neurorehabilitation patients). ANAM-PVI scores differed between groups with simulators scoring the highest. ROC curve analysis indicated excellent discriminability of ANAM-PVI scores ≥5 to detect simulators versus controls (area under the curve = 0.858; odds ratio for detecting suboptimal performance = 15.6), but resulted in a 27% false-positive rate in the clinical sample. When specificity in the clinical sample was set at 90%, sensitivity decreased (68%), but was consistent with other embedded effort measures. Results support the ANAM-PVI as an embedded effort measure and demonstrate the value of sample-specific cut-points in groups with cognitive impairment. Examination of different cut-points indicates that clinicians should choose sample-specific cut-points based on sensitivity and specificity rates that are most appropriate for their patient population with higher cut-points for those expected to have severe cognitive impairment (e.g., dementia or severe acquired brain injury).

New measures to detect malingered neurocognitive deficit: applying reaction time and event-related potentials

The ability of the Test of Memory Malingering (TOMM), reaction times (RTs), and event-related potentials (ERPs) to detect malingered neurocognitive deficit (MNCD) was examined in 32 normal individuals answering under honest (HON; n = 16) or malingering (MAL; n = 16) instructions as well as in 15 patients with traumatic brain injury (TBI) who answered under honest instructions. Overall, the TOMM was the most effective at classifying groups. However, new accuracy, RT, and ERP measures reached promising hit rates in the range of 71-88%. In particular, the difference in frontal versus posterior ERP obtained during an old-new task was effective at classifying MAL versus TBI (hit rate = 87%).

Malingering in the medical setting

Malingering of mental illness has been studied extensively; however, malingered medical illness has been examined much less avidly. While in theory any ailment can be fabricated or self-induced, pain--including lower back pain, cervical pain, and fibromyalgia--and cognitive deficits associated with mild head trauma or toxic exposure are feigned most frequently, especially in situations where there are financial incentives to malinger. Structured assessments have been developed to help detect both types of malingering; however, in daily practice, the physician should generally suspect malingering when there are tangible incentives and when reported symptoms do not match the physical examination or no organic basis for the physical complaints is found.

Neuropsychological Test Performance of Successful Brain Injury Simulators

This study provided an examination of the performance characteristics of successful brain injury simulators (SBIS). Coached (n = 56) and uncoached (n = 35) brain injury simulators received instructions to fake cognitive impairment; controls were asked to do their best. The Test of Memory Malingering (TOMM) was administered along with standard neuropsychological measures (e.g., Wisconsin Card Sorting Test). The TOMM identified 80% of uncoached and 60% of coached brain injury simulators. SBIS were participants from the brain injury simulation groups whose TOMM performance indicated adequate effort. A total of 32% of all brain injury simulators scored above the TOMM cutoff scores for adequate effort (the SBIS group). Significantly more coached than uncoached participants composed the SBIS group (76% vs. 24%, respectively). SBIS performed significantly worse than controls and significantly better than unsuccessful brain injury simulators on select standard neuropsychological measures. The SBIS scores were lowered compared to controls; in some instances this lowered performance was at a clinically relevant level.

Detection of feigned recognition memory impairment using the old/new effect of the event-related potential

Twenty-four undergraduate university students with no known neurological disorders completed the Recognition Memory Test (Warrington, A., 1984. Recognition Memory Test manual. Windsor, Berkshire: NFER-Nelson.) while event-related potentials (ERPs) were recorded. Twelve subjects were instructed to feign a recognition memory deficit (malingering group), while the remainder served as controls. The malingerers performed poorly on the test compared to the control group. The 'old/new effect', an ERP measure thought to reflect recognition memory processes, did not differ between the groups, indicating recognition of previously learned material in the malingering group despite poor test performance. The study also revealed a second, early, old/new effect, maximal at left frontal sites in the malingering relative to the control group, suggesting task-related processing differences between the two groups. These effects appear to be of potential value in the detection of malingering of cognitive impairment in the clinical situation.

Neuropsychological Assessment of Suspected Malingering: Research Results, Evaluation Techniques, and Further Directions of Research and Application

Possibilities for neuropsychological assessment of suspected malingering are addressed. First, various forms of deception (malingering, factitious disorders, hysteria) and their implications for clinical neuropsychology are discussed. Then, threshold models for the detection of malingering as well as specially designed assessment techniques (e. g., individual tests, deficit testing, tests specifically for malingerers, and symptom validity testing) are described. Finally, the current status of clinical methods and research strategies is summarized, and recent and further developments of assessment and research are reported.

Victoria Symptom Validity Test: efficiency for detecting feigned memory impairment and relationship to neuropsychological tests and MMPI-2 validity scales

Error scores and response times from a computer-administered, forced-choice recognition test of symptom validity were evaluated for efficiency in detecting feigned memory deficits. Participants included controls (n = 95), experimental malingerers (n = 43), compensation-seeking patients (n = 206), and patients not seeking financial compensation (n = 32). Adopting a three-level cut-score system that classified participant performance as malingered, questionable, or valid greatly improved sensitivity with relatively little impact on specificity. For error scores, convergent validity was found to be adequate and divergent validity was found to be excellent. Although response times showed promise for assisting in the detection of feigned impairment, divergent and convergent validity were weaker, suggesting somewhat less utility than error scores.

Detecting the malingering of cognitive deficits: an update

There has recently been a dramatic increase of empirical studies that investigate methods for detecting malingering of cognitive deficits. The present review focuses on a comparison of simulated and suspected malingerers in the malingering literature, and critiques the numerous approaches to the detection of malingering. The approaches that are reviewed include detection of floor effects, discrepancies of information, response bias, neuropsychological tests and batteries, symptom validity testing, and anomalous performance on memory tests. The latter approach has only recently been proposed by researchers and may show the most promise.