Attorney demands for protected psychological test information: Is access necessary for cross examination or does it lead to misinformation? An interorganizational* position paper

Objective: Some attorneys claim that to adequately cross examine neuropsychological experts, they require direct access to protected test information, rather than having test data analyzed by retained neuropsychological experts. The objective of this paper is to critically examine whether direct access to protected test materials by attorneys is indeed necessary, appropriate, and useful to the trier-of-fact. Method: Examples are provided of the types of nonscientific misinformation that occur when attorneys, who lack adequate training in testing, attempt to independently interpret neurocognitive/psychological test data. Results: Release of protected test information to attorneys introduces inaccurate information to the trier of fact, and jeopardizes future use of tests because non-psychologists are not ethically bound to protect test content. Conclusion: The public policy underlying the right of attorneys to seek possibly relevant documents should not outweigh the damage to tests and resultant misinformation that arise when protected test information is released directly to attorneys. The solution recommended by neuropsychological/psychological organizations and test publishers is to have protected psychological test information exchanged directly and only between clinical psychologist/neuropsychologist experts.

Threats to the integrity of psychological assessment: The misuse of test raw data and materials

In the practice of psychological assessment there have been warnings for decades by the American Psychological Association (APA), the National Academy of Neuropsychology (NAN), other associations, and test vendors, against the disclosure of test raw data and test materials. Psychological assessment occurs across several different practice environments, and test raw data is a particularly sensitive aspect of practice considering what it implicitly represents about a client/patient, and this concept is further developed in this paper. Many times, test materials are intellectual property protected by copyrights and user agreements. It follows that improper management of the release of test raw data and test materials threatens the scientific integrity of psychological assessment. Here the matters of test raw data, test materials, and different practice environments are addressed to highlight the challenges involved with improper releases and to offer guidance concerning good-faith efforts to preserve the integrity of psychological assessment and legal agreements. The unique demands of forensic practice are also discussed, including attorneys' needs for cross-examination and discovery, which may place psychologists (and other duly vetted evaluators) in conflict with their commitment to professional ethical codes and legal agreements. To this end, important threats to the proper use of test raw data and test materials include uninformed professionals and compromised evaluators. In this paper, the mishandling of test raw data and materials by both psychologists and other evaluators is reviewed, representative case examples, including those from the literature, are provided, pertinent case law is discussed, and practical stepwise conflict resolutions are offered.

Profile of Embedded Validity Indicators in Criminal Defendants with Verified Valid Neuropsychological Test Performance

Objective

Few studies have examined the use of embedded validity indicators (EVIs) in criminal-forensic practice settings, where judgements regarding performance validity can carry severe consequences for the individual and society. This study sought to examine how various EVIs perform in criminal defendant populations, and determine relationships between EVI scores and intrapersonal variables thought to influence performance validity.

Method

Performance on 16 empirically established EVI cutoffs were examined in a sample of 164 criminal defendants with valid performance who were referred for forensic neuropsychological evaluation. Subsequent analyses examined the relationship between EVI scores and intrapersonal variables in 83 of these defendants.

Results

Half of the EVIs (within the Wechsler Adult Intelligence Scale Digit Span Total, Conners’ Continuous Performance Test Commissions, Wechsler Memory Scale Logical Memory I and II, Controlled Oral Word Association Test, Trail Making Test Part B, and Stroop Word and Color) performed as intended in this sample. The EVIs that did not perform as intended were significantly influenced by relevant intrapersonal variables, including below-average intellectual functioning and history of moderate–severe traumatic brain injury and neurodevelopmental disorder.

Conclusions

This study identifies multiple EVIs appropriate for use in criminal-forensic settings. However, based on these findings, practitioners may wish to be selective in choosing and interpreting EVIs for forensic evaluations of criminal court defendants.

Eye tracking as a mean to detect feigned cognitive impairment in the word memory test

Eye movements showed initial promise for the detection of deception and may be harder to consciously manipulate than conventional accuracy measures. Therefore, we integrated an eye-tracker with the Word Memory Test (WMT) and tested its usefulness for the detection of feigned cognitive impairment. As part of the study, simulators (n = 44) and honest controls (n = 41) performed WMT's immediate-recognition (IR) subtest while their eye movements were recorded. In comparison to the control group, simulators spent less time gazing at relevant stimuli, spent more time gazing at irrelevant stimuli, and had a lower saccade rate. Group classification using a scale that combined the eye movement measures and the WMT's accuracy measure showed tentative promise (i.e., it enhanced classification compared to the use of the accuracy measure as the sole predictor of group membership). Overall, integration of an eye-tracker with the WMT was found to be feasible and the eye movement measures showed initial promise for the detection of feigned cognitive impairment. Moreover, eye movement measures proved useful in enhancing our understanding of strategies utilized by the simulators and the cognitive processes that affect their behavior. While the findings are clearly preliminary, we hope that they will encourage further research of these promising psychophysiological measures.

A New Index for the MMPI-2 Test for Detecting Dissimulation in Forensic Evaluations: A Pilot Study

This pilot study is the starting point of a potentially broad research project aimed at identifying new strategies for assessing malingering during forensic evaluations. The forensic group was comprised of 67 males who were seeking some sort of certification (e.g., adoption, child custody, driver's license, issuance of gun permits, etc.); the nonforensic group was comprised of 62 healthy male volunteers. Each participant was administered the MMPI-2. Statistical analyses were conducted on obtained scores of 48 MMPI-2 scales. In the first step, parametric statistics were adopted to identify the best combination of MMPI-2 scales that differentiated the two groups of participants. In the second step, frequency-based, nonparametric methods were used for diagnostic purposes.

RESULTS

A model that utilized the best three predictors ("7-Pt", "L," and "1-Hs") was developed and used to calculate the Forensic Evaluation Dissimulation Index (FEDI), which features satisfactory diagnostic accuracy (0.9), sensitivity (0.82), specificity (0.81), and likelihood ratio indices (LR+ = 4.32; LR- = 0.22).

Derivation and Cross-Validation of Cutoff Scores for Patients With Schizophrenia Spectrum Disorders on WAIS-IV Digit Span–Based Performance Validity Measures

Two studies were conducted to identify and cross-validate cutoff scores on the Wechsler Adult Intelligence Scale–Fourth Edition Digit Span–based embedded performance validity (PV) measures for individuals with schizophrenia spectrum disorders. In Study 1, normative scores were identified on Digit Span–embedded PV measures among a sample of patients ( n = 84) with schizophrenia spectrum diagnoses who had no known incentive to perform poorly and who put forth valid effort on external PV tests. Previously identified cutoff scores resulted in unacceptable false positive rates and lower cutoff scores were adopted to maintain specificity levels ≥90%. In Study 2, the revised cutoff scores were cross-validated within a sample of schizophrenia spectrum patients ( n = 96) committed as incompetent to stand trial. Performance on Digit Span PV measures was significantly related to Full Scale IQ in both studies, indicating the need to consider the intellectual functioning of examinees with psychotic spectrum disorders when interpreting scores on Digit Span PV measures.

Detecting faked psychopathology: a comparison of two tests to detect malingered psychopathology using a simulation design

Malingered psychopathology has the potential to be a costly social problem and there is a need for studies that compare the malingering detection capabilities of tests of psychopathology. This study investigated the capacity of two measures to detect simulated psychopathology. Forty-one first-year psychology students were randomly allocated to experimental groups that included malingering and control conditions. Analogue malingerers were given a financial incentive to simulate believable psychological impairment. Controls received standardised test instructions and the prize incentive, contingent on good effort. In a between-group simulation design, group differences on the Personality Assessment Inventory (PAI) and the revised Symptom Checklist-90 (SCL-90-R) were assessed. Group comparisons revealed elevation of the majority of clinical index scores among malingerers and a consistent pattern of results across tests. Analysis of the test operating characteristics of the malingering indices for these measures revealed superior detection of simulated malingering using the PAI, particularly Rogers' Discriminant Function, although classification accuracy of all malingering indexes was improved when adjusted cut-offs were used. Overall, results from this study demonstrate the vulnerability of the PAI and (SCL-90-R) to simulated psychopathology, but also the capacity of these measures to detect such performance when specific indexes are used.

Deterring malingered psychopathology: The effect of warning simulating malingerers

The utility of a warning to deter malingering on measures of personality and psychopathology was examined. Sixty-seven first year psychology students were randomly assigned to one of three conditions: unwarned malingerers, warned malingerers, and controls. Participants in the two malingering groups were given a financial incentive to simulate believable psychological impairment. Warned malingerers received an additional warning that the tests could detect malingering and that detection would result in loss of course credit. Controls received standardized test instructions. It was hypothesized that the malingering incentive would be sufficient to induce malingering, but that a deterrence theory warning would have a subsequent deterrent effect. Between-groups analyses indicated that the warning used in this study significantly altered test performance on the Personality Assessment Inventory (PAI) and revised Symptom Checklist 90 (SCL-90-R). Warned malingerers scored significantly lower (faked less) than unwarned malingerers on the majority of the psychopathology scales and frequently approximated control group performances. These results support the effectiveness of a warning to complement existing malingering detection methods.

Psychiatry's problem child: PTSD in the forensic context: (part 1)

OBJECTIVE

The aim of this paper was to consider some of the recent literature that has raised questions about the prevalence of, and problems with, post-traumatic stress disorder (PTSD) in the forensic context and whether it is a failure by experts to consider adequately Criterion A (1), which leads them into error and results in a skewed picture of the prevalence of PTSD in litigation.

METHOD

The nature of the stressor criterion for PTSD from DSM-III to DSM-IV-TR, and recent literature relating to problems with the manner in which that stressor criterion is applied in forensic contexts, are considered.

RESULTS

It becomes clear that, notwithstanding that the class of persons who can suffer PTSD after a particularly traumatic event has widened since it was included in the DSM-III, the nature of the stressor required before a diagnosis of PTSD can be made has remained fairly constant.

CONCLUSIONS

Whether it is because PTSD is commonly associated with receiving a benefit (e.g. through post-war disability benefits, victims of crime compensation or because of a desire to use it as an ideological weapon), PTSD has become a ubiquitous presence in the forensic context, notwithstanding objective (i.e. non-forensic) clinical studies which refer to its rarity. It is clear now that PTSD symptoms can be readily simulated when the simulator is seeking a benefit. It is also becoming clear that some psychiatrists are not being as rigorous in the forensic context as they might be in a clinical context. Thus, with discussion now underway regarding DSM-V, it may be timely to reconsider the utility of PTSD as a clinical condition.

Detecting Feigned Impairment with the Digit Span and Vocabulary Subtests of the Wechsler Adult Intelligence Scale—Third Edition

The WAIS-III Digit Span and Vocabulary subtests were investigated as indicators of feigned cognitive impairment. Participants included 64 undergraduates randomly assigned to control, symptom-coached, or test-coached groups. Six previously researched validity indicators were examined. We hypothesized that symptom-coached participants would perform worse relative to test-coached simulators. Analyses determined both simulator groups performed lower than controls on all indicators except Vocabulary. Symptom-coached participants, however, did not differ from test-coached participants on any indicator. Classification accuracies for these six indicators ranged from 42 to 78%. While the WAIS-III validity indicators hold some promise, they should not be employed as independent measures.

Replication of the Test of Memory Malingering (TOMM) in a Traumatic Brain Injury and Head Trauma Sample

The purpose of the current study was to replicate the clinical validation study of the Test of Memory Malingering (Tombaugh, 19961997). Fifty cases were selected from two different participant pools: medically documented traumatic brain-injury patients and compensation-seeking, mild head trauma cases. Results replicated and expanded those found by Tombaugh and indicated that employing a Trial 2 cutting score of greater than 5 errors provided adequate sensitivity to suboptimal effort while demonstrating remarkable insensitivity to cognitively impaired traumatic brain-injury cases. Use of a lower cutting score (e.g., 2 to 5 errors) is also discussed.

Known-Groups Cross-Validation of the Letter Memory Test in a Compensation-Seeking Mixed Neurologic Sample

Compensation-seeking neuropsychological evaluees were classified into Honest (HON; n = 37) or Probable Cognitive Feigning (PCF; n = 53) groups based on results from the Victoria Symptom Validity Test, the Test of Memory Malingering, and the Digit Span subtest of the Wechsler Adult Intelligence Scale--3rd ed. The groups were generally comparable on demographic, background, and injury severity characteristics, although HON TBI participants were significantly more likely to have a documented loss of consciousness, whereas PCF participants were significantly more likely to be currently on disability. PCF participants scored significantly lower on many neuropsychological test, particularly of memory, as well as higher on most MMPI-2 clinical scales. The PCF group also had significantly higher scores on multiple indices of feigning of psychiatric symptoms. Results from the Letter Memory Test (LMT) were significantly lower for the PCF group, and using the recommended cutting score, specificity was .984, whereas sensitivity was .640, suggesting adequate performance on cross-validation.

"Cognitive Impairment Associated with Toxigenic Fungal Exposure": A Critique and Critical Analysis

Recently Gordon et al. (2004) reported the results of a study relating exposure to toxigenic fungi to cognitive impairment. A number of methodological problems limit the utility of this study, including the authors' not examining a number of potential confounding variables, use of a mild traumatic brain injury group as a comparison, and problematic statistical analyses increasing the Type 1 error rate. Based on these limitations, the authors' conclusion that mold-exposed people have more cognitive and physical symptoms than normal controls is questionable at best. This critique describes some of the problems with the methodology employed and the related difficulty in drawing conclusions from their work.