1
|
Tyson BT, Shahein A, Abeare CA, Baker SD, Kent K, Roth RM, Erdodi LA. Replicating a Meta-Analysis: The Search for the Optimal Word Choice Test Cutoff Continues. Assessment 2023; 30:2476-2490. [PMID: 36752050 DOI: 10.1177/10731911221147043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/09/2023]
Abstract
This study was designed to expand on a recent meta-analysis that identified ≤42 as the optimal cutoff on the Word Choice Test (WCT). We examined the base rate of failure and the classification accuracy of various WCT cutoffs in four independent clinical samples (N = 252) against various psychometrically defined criterion groups. WCT ≤ 47 achieved acceptable combinations of specificity (.86-.89) at .49 to .54 sensitivity. Lowering the cutoff to ≤45 improved specificity (.91-.98) at a reasonable cost to sensitivity (.39-.50). Making the cutoff even more conservative (≤42) disproportionately sacrificed sensitivity (.30-.38) for specificity (.98-1.00), while still classifying 26.7% of patients with genuine and severe deficits as non-credible. Critical item (.23-.45 sensitivity at .89-1.00 specificity) and time-to-completion cutoffs (.48-.71 sensitivity at .87-.96 specificity) were effective alternative/complementary detection methods. Although WCT ≤ 45 produced the best overall classification accuracy, scores in the 43 to 47 range provide comparable objective psychometric evidence of non-credible responding. Results question the need for designating a single cutoff as "optimal," given the heterogeneity of signal detection environments in which individual assessors operate. As meta-analyses often fail to replicate, ongoing research is needed on the classification accuracy of various WCT cutoffs.
Collapse
Affiliation(s)
| | | | | | | | | | - Robert M Roth
- Dartmouth-Hitchcock Medical Center, Lebanon, NH, USA
| | | |
Collapse
|
2
|
Cutler L, Greenacre M, Abeare CA, Sirianni CD, Roth R, Erdodi LA. Multivariate models provide an effective psychometric solution to the variability in classification accuracy of D-KEFS Stroop performance validity cutoffs. Clin Neuropsychol 2023; 37:617-649. [PMID: 35946813 DOI: 10.1080/13854046.2022.2073914] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Abstract
ObjectiveThe study was designed to expand on the results of previous investigations on the D-KEFS Stroop as a performance validity test (PVT), which produced diverging conclusions. Method The classification accuracy of previously proposed validity cutoffs on the D-KEFS Stroop was computed against four different criterion PVTs in two independent samples: patients with uncomplicated mild TBI (n = 68) and disability benefit applicants (n = 49). Results Age-corrected scaled scores (ACSSs) ≤6 on individual subtests often fell short of specificity standards. Making the cutoffs more conservative improved specificity, but at a significant cost to sensitivity. In contrast, multivariate models (≥3 failures at ACSS ≤6 or ≥2 failures at ACSS ≤5 on the four subtests) produced good combinations of sensitivity (.39-.79) and specificity (.85-1.00), correctly classifying 74.6-90.6% of the sample. A novel validity scale, the D-KEFS Stroop Index correctly classified between 78.7% and 93.3% of the sample. Conclusions A multivariate approach to performance validity assessment provides a methodological safeguard against sample- and instrument-specific fluctuations in classification accuracy, strikes a reasonable balance between sensitivity and specificity, and mitigates the invalid before impaired paradox.
Collapse
Affiliation(s)
- Laura Cutler
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada
| | - Matthew Greenacre
- Schulich School of Medicine, Western University, London, Ontario, Canada
| | - Christopher A Abeare
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada
| | | | - Robert Roth
- Department of Psychiatry, Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire, USA
| | - Laszlo A Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada
| |
Collapse
|
3
|
Brooks BL, Fay-McClymont TB, MacAllister WS, Vasserman M, Mish S, Sherman EMS. New Child and Adolescent Memory Profile Embedded Performance Validity Test. ARCHIVES OF CLINICAL NEUROPSYCHOLOGY : THE OFFICIAL JOURNAL OF THE NATIONAL ACADEMY OF NEUROPSYCHOLOGISTS 2023:6972889. [PMID: 36617240 DOI: 10.1093/arclin/acac110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 12/14/2022] [Indexed: 01/09/2023]
Abstract
OBJECTIVE It is essential to interpret performance validity tests (PVTs) that are well-established and have strong psychometrics. This study evaluated the Child and Adolescent Memory Profile (ChAMP) Validity Indicator (VI) using a pediatric sample with traumatic brain injury (TBI). METHOD A cross-sectional sample of N = 110 youth (mean age = 15.1 years, standard deviation [SD] = 2.4 range = 8-18) on average 32.7 weeks (SD = 40.9) post TBI (71.8% mild/concussion; 3.6% complicated mild; 24.6% moderate-to-severe) were administered the ChAMP and two stand-alone PVTs. Criterion for valid performance was scores above cutoffs on both PVTs; criterion for invalid performance was scores below cutoffs on both PVTs. Classification statistics were used to evaluate the existing ChAMP VI and establish a new VI cutoff score if needed. RESULTS There were no significant differences in demographics or time since injury between those deemed valid (n = 96) or invalid (n = 14), but all ChAMP scores were significantly lower in those deemed invalid. The original ChAMP VI cutoff score was highly specific (no false positives) but also highly insensitive (sensitivity [SN] = .07, specificity [SP] = 1.0). Based on area under the curve (AUC) analysis (0.94), a new cutoff score was established using the sum of scaled scores (VI-SS). A ChAMP VI-SS score of 32 or lower achieved strong SN (86%) and SP (92%). Using a 15% base rate, positive predictive value was 64% and negative predictive value was 97%. CONCLUSIONS The originally proposed ChAMP VI has insufficient SN in pediatric TBI. However, this study yields a promising new ChAMP VI-SS, with classification metrics that exceed any other current embedded PVT in pediatrics.
Collapse
Affiliation(s)
- Brian L Brooks
- Neurosciences Program, Alberta Children's Hospital, Calgary, Alberta T3B 6A8, Canada.,Departments of Pediatrics, Clinical Neurosciences, and Psychology, University of Calgary, Calgary, Alberta T2N 1N4, Canada.,Child Brain and Mental Health Section, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta T2N 1N4, Canada
| | - Taryn B Fay-McClymont
- Neurosciences Program, Alberta Children's Hospital, Calgary, Alberta T3B 6A8, Canada.,Child Brain and Mental Health Section, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta T2N 1N4, Canada.,Department of Pediatrics, University of Calgary, Calgary, Alberta T2N 1N4, Canada.,Department of Psychology, University of British Columbia Okanagan, Kelowna, British Columbia V1V 1V7, Canada
| | - William S MacAllister
- Neurosciences Program, Alberta Children's Hospital, Calgary, Alberta T3B 6A8, Canada.,Child Brain and Mental Health Section, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta T2N 1N4, Canada.,Department of Pediatrics, University of Calgary, Calgary, Alberta T2N 1N4, Canada
| | - Marsha Vasserman
- Neurosciences Program, Alberta Children's Hospital, Calgary, Alberta T3B 6A8, Canada.,Child Brain and Mental Health Section, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta T2N 1N4, Canada.,Department of Pediatrics, University of Calgary, Calgary, Alberta T2N 1N4, Canada
| | - Sandra Mish
- Neurosciences Program, Alberta Children's Hospital, Calgary, Alberta T3B 6A8, Canada
| | | |
Collapse
|
4
|
Profile of Embedded Validity Indicators in Criminal Defendants with Verified Valid Neuropsychological Test Performance. Arch Clin Neuropsychol 2022; 38:513-524. [DOI: 10.1093/arclin/acac073] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Revised: 07/21/2022] [Accepted: 08/09/2022] [Indexed: 11/15/2022] Open
Abstract
Abstract
Objective
Few studies have examined the use of embedded validity indicators (EVIs) in criminal-forensic practice settings, where judgements regarding performance validity can carry severe consequences for the individual and society. This study sought to examine how various EVIs perform in criminal defendant populations, and determine relationships between EVI scores and intrapersonal variables thought to influence performance validity.
Method
Performance on 16 empirically established EVI cutoffs were examined in a sample of 164 criminal defendants with valid performance who were referred for forensic neuropsychological evaluation. Subsequent analyses examined the relationship between EVI scores and intrapersonal variables in 83 of these defendants.
Results
Half of the EVIs (within the Wechsler Adult Intelligence Scale Digit Span Total, Conners’ Continuous Performance Test Commissions, Wechsler Memory Scale Logical Memory I and II, Controlled Oral Word Association Test, Trail Making Test Part B, and Stroop Word and Color) performed as intended in this sample. The EVIs that did not perform as intended were significantly influenced by relevant intrapersonal variables, including below-average intellectual functioning and history of moderate–severe traumatic brain injury and neurodevelopmental disorder.
Conclusions
This study identifies multiple EVIs appropriate for use in criminal-forensic settings. However, based on these findings, practitioners may wish to be selective in choosing and interpreting EVIs for forensic evaluations of criminal court defendants.
Collapse
|
5
|
Smith A, Thomas J, Friedhoff C, Chin E. The Utility of the Test of Memory Malingering Trial 1 in Differentiating Neurocognitive, Emotional, and Behavioral Functioning in a Pediatric Concussion Population. Arch Clin Neuropsychol 2021; 37:322-337. [PMID: 34386811 DOI: 10.1093/arclin/acab065] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2020] [Revised: 06/08/2021] [Accepted: 07/21/2021] [Indexed: 11/15/2022] Open
Abstract
OBJECTIVE In concussion populations, suboptimal task engagement detected by performance validity tests (PVTs) has been associated with poorer neuropsychological scores and greater post-concussive symptoms (PCS). This study examined if Pass/Fail status on the Test of Memory Malingering-TOMM Trial 1-differentiated the neurocognitive, emotional, and behavioral profile of pediatric patients with concussion. METHOD This study utilized archival data from 93 patients (mean age = 14.56 and SD = 2.01) with a history of concussion who were assessed at ~5-6 weeks post-injury (mean days = 40.27 and SD = 35.41). Individuals were divided into "Pass" and "Fail" groups based on TOMM Trial 1 performance. The testing battery included ACT, CPT-II and III, HVLT-R, WJ-III and IV ACH, ImPACT, BASC-2, and BRIEF. RESULTS The overall pass rate on Trial 1 was 70% (mean = 46.04 and SD = 4.55). Findings suggested that a passing score on Trial 1 may be associated with adequate performance across the remaining two trials of the TOMM. The Fail group scored significantly lower across attention, memory, and processing speed measures when compared with the Pass group. On rating scales, significantly more concerns were endorsed with the Fail group for attention and executive functioning relative to the Pass group. Parents generally endorsed significantly more concerns for executive functioning when compared with their children's self-reported symptoms. There was a trend for the Fail group to report more PCS; however, they did not significantly differ from the Pass group for depression, anxiety, or somatization. CONCLUSIONS This study highlights the importance of utilizing PVTs when evaluating concussion recovery.
Collapse
Affiliation(s)
- Alphonso Smith
- AMITA Health Neurosciences Institute - Center for Pediatric Brain, Hoffman Estates, IL, USA
| | - Julia Thomas
- AMITA Health Neurosciences Institute - Center for Pediatric Brain, Hoffman Estates, IL, USA
| | - Claire Friedhoff
- AMITA Health Neurosciences Institute - Center for Pediatric Brain, Hoffman Estates, IL, USA
| | - Esther Chin
- AMITA Health Neurosciences Institute - Center for Pediatric Brain, Hoffman Estates, IL, USA
| |
Collapse
|
6
|
Berger C, Lev A, Braw Y, Elbaum T, Wagner M, Rassovsky Y. Detection of Feigned ADHD Using the MOXO-d-CPT. J Atten Disord 2021; 25:1032-1047. [PMID: 31364437 DOI: 10.1177/1087054719864656] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
Abstract
Objective: The objective of this study was to assess the MOXO-d-CPT utility in detecting feigned ADHD and establish cutoffs with adequate specificity and sensitivity. Method: The study had two phases. First, using a prospective design, healthy adults who simulated ADHD were compared with healthy controls and ADHD patients who performed the tasks to the best of their ability (n = 47 per group). Participants performed the MOXO-d-CPT and an established performance validity test (PVT). Second, the MOXO-d-CPT classification accuracy, employed in Phase 1, was retrospectively compared with archival data of 47 ADHD patients and age-matched healthy controls. Results: Simulators performed significantly worse on all MOXO-d-CPT indices than healthy controls and ADHD patients. Three MOXO-d-CPT indices (attention, hyperactivity, impulsivity) and a scale combining these indices showed adequate discriminative capacity. Conclusion: The MOXO-d-CPT showed promise for the detection of feigned ADHD and, pending replication, can be employed for this aim in clinical practice and ADHD research.
Collapse
Affiliation(s)
| | - Astar Lev
- Bar-Ilan University, Ramat Gan, Israel
| | | | | | | | - Yuri Rassovsky
- Bar-Ilan University, Ramat Gan, Israel.,University of California, Los Angeles, USA
| |
Collapse
|
7
|
Abeare K, Razvi P, Sirianni CD, Giromini L, Holcomb M, Cutler L, Kuzmenka P, Erdodi LA. Introducing Alternative Validity Cutoffs to Improve the Detection of Non-credible Symptom Report on the BRIEF. PSYCHOLOGICAL INJURY & LAW 2021. [DOI: 10.1007/s12207-021-09402-4] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
|
8
|
Cutler L, Abeare CA, Messa I, Holcomb M, Erdodi LA. This will only take a minute: Time cutoffs are superior to accuracy cutoffs on the forced choice recognition trial of the Hopkins Verbal Learning Test - Revised. APPLIED NEUROPSYCHOLOGY-ADULT 2021; 29:1425-1439. [PMID: 33631077 DOI: 10.1080/23279095.2021.1884555] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
OBJECTIVE This study was designed to evaluate the classification accuracy of the recently introduced forced-choice recognition trial to the Hopkins Verbal Learning Test - Revised (FCRHVLT-R) as a performance validity test (PVT) in a clinical sample. Time-to-completion (T2C) for FCRHVLT-R was also examined. METHOD Forty-three students were assigned to either the control or the experimental malingering (expMAL) condition. Archival data were collected from 52 adults clinically referred for neuropsychological assessment. Invalid performance was defined using expMAL status, two free-standing PVTs and two validity composites. RESULTS Among students, FCRHVLT-R ≤11 or T2C ≥45 seconds was specific (0.86-0.93) to invalid performance. Among patients, an FCRHVLT-R ≤11 was specific (0.94-1.00), but relatively insensitive (0.38-0.60) to non-credible responding0. T2C ≥35 s produced notably higher sensitivity (0.71-0.89), but variable specificity (0.83-0.96). The T2C achieved superior overall correct classification (81-86%) compared to the accuracy score (68-77%). The FCRHVLT-R provided incremental utility in performance validity assessment compared to previously introduced validity cutoffs on Recognition Discrimination. CONCLUSIONS Combined with T2C, the FCRHVLT-R has the potential to function as a quick, inexpensive and effective embedded PVT. The time-cutoff effectively attenuated the low ceiling of the accuracy scores, increasing sensitivity by 19%. Replication in larger and more geographically and demographically diverse samples is needed before the FCRHVLT-R can be endorsed for routine clinical application.
Collapse
Affiliation(s)
- Laura Cutler
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, ON, Canada
| | - Christopher A Abeare
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, ON, Canada
| | - Isabelle Messa
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, ON, Canada
| | | | - Laszlo A Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, ON, Canada
| |
Collapse
|
9
|
Ord AS, Miskey HM, Lad S, Richter B, Nagy K, Shura RD. Examining embedded validity indicators in Conners continuous performance test-3 (CPT-3). Clin Neuropsychol 2020; 35:1426-1441. [DOI: 10.1080/13854046.2020.1751301] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Affiliation(s)
- Anna S. Ord
- W.G. Hefner VA Medical Center, Salisbury, NC, USA
- Mid-Atlantic Mental Illness Research Education and Clinical Center, Durham, NC, USA
| | - Holly M. Miskey
- W.G. Hefner VA Medical Center, Salisbury, NC, USA
- Mid-Atlantic Mental Illness Research Education and Clinical Center, Durham, NC, USA
- Wake Forest School of Medicine, Winston-Salem, NC, USA
- Via College of Osteopathic Medicine, Blacksburg, VA, USA
| | - Sagar Lad
- W.G. Hefner VA Medical Center, Salisbury, NC, USA
- Mid-Atlantic Mental Illness Research Education and Clinical Center, Durham, NC, USA
| | - Beth Richter
- W.G. Hefner VA Medical Center, Salisbury, NC, USA
| | | | - Robert D. Shura
- W.G. Hefner VA Medical Center, Salisbury, NC, USA
- Mid-Atlantic Mental Illness Research Education and Clinical Center, Durham, NC, USA
- Wake Forest School of Medicine, Winston-Salem, NC, USA
- Via College of Osteopathic Medicine, Blacksburg, VA, USA
| |
Collapse
|
10
|
Bosworth C, Dodd JN. Noncredible effort on the Nonverbal-Medical Symptom Validity Test (NV-MSVT): Impact on cognitive performance in pediatric mild traumatic brain injury. APPLIED NEUROPSYCHOLOGY. CHILD 2020; 9:367-374. [PMID: 32223424 DOI: 10.1080/21622965.2020.1742717] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Abstract
This study examined failure rates on the Nonverbal-Medical Symptom Validity Test (NV-MSVT) and its impact on cognitive performance in a sample of youths with mild traumatic brain injury (mTBI). Participants were 184 children and adolescents who presented to a multidisciplinary concussion clinic for a targeted neuropsychological evaluation. Performance Validity Tests (PVTs) were a part of the standard battery, including the NV-MSVT. Twenty-eight participants (15.2%) failed the NV-MSVT, none of whom displayed a genuine memory impairment profile (GMIP). Participants who failed the NV-MSVT performed significantly worse than those who passed the NV-MSVT on measures of IQ, memory, and immediate attention/working memory. There was no significant difference between groups on processing speed, sustained attention, cognitive flexibility, or sight word reading level. Aside from a slight difference in age, NV-MSVT failure was not impacted by demographic variables (sex, race), premorbid risk factors (pre-injury ADHD, learning disabilities, psychiatric diagnoses or treatment, developmental delays, or prior special education), injury-related variables (time since injury, positive neuroimaging findings, post-traumatic amnesia, number of prior mTBIs, etc.) or post-mTBI anxiety/depression. That said, participants who failed NV-MSVT endorsed significantly more severe postconcussive symptoms. These findings support the use of the NV-MSVT in neuropsychological evaluation of children and adolescents with mTBI.
Collapse
Affiliation(s)
- Christopher Bosworth
- St. Louis Children's Hospital, Washington University School of Medicine, St. Louis, MO, USA
| | - Jonathan N Dodd
- Department of Neuropsychology, WellStar Medical Group-Psychological Services, Marietta, GA, USA
| |
Collapse
|
11
|
Lace JW, Grant AF, Ruppert P, Kaufman DAS, Teague CL, Lowell K, Gfeller JD. Detecting noncredible performance with the neuropsychological assessment battery, screening module: A simulation study. Clin Neuropsychol 2019; 35:572-596. [DOI: 10.1080/13854046.2019.1694703] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/07/2023]
Affiliation(s)
- John W. Lace
- Department of Psychology, Saint Louis University, St. Louis, MO, USA
| | - Alex F. Grant
- Department of Psychology, Saint Louis University, St. Louis, MO, USA
| | - Phillip Ruppert
- Department of Psychiatry and Behavioral Neuroscience, Saint Louis University, St. Louis, MO, USA
| | | | - Carson L. Teague
- Department of Psychology, Saint Louis University, St. Louis, MO, USA
| | - Kimberly Lowell
- Department of Psychology, Saint Louis University, St. Louis, MO, USA
| | | |
Collapse
|
12
|
Erdodi LA, Taylor B, Sabelli AG, Malleck M, Kirsch NL, Abeare CA. Demographically Adjusted Validity Cutoffs on the Finger Tapping Test Are Superior to Raw Score Cutoffs in Adults with TBI. PSYCHOLOGICAL INJURY & LAW 2019. [DOI: 10.1007/s12207-019-09352-y] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
|
13
|
Lichtenstein JD, Flaro L, Baldwin FS, Rai J, Erdodi LA. Further Evidence for Embedded Performance Validity Tests in Children within the Conners’ Continuous Performance Test – Second Edition. Dev Neuropsychol 2019; 44:159-171. [DOI: 10.1080/87565641.2019.1565535] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Affiliation(s)
- Jonathan D. Lichtenstein
- Department of Psychiatry, Pediatric Neuropsychology Program, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire
- Department of Pediatrics, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire
- The Dartmouth Institute for Health Policy an d Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire
| | | | - Fern S. Baldwin
- Department of Psychiatry, Pediatric Neuropsychology Program, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire
| | - Jaspreet Rai
- Department of Psychology, Neuropsychology Track, University of Windsor, Ontario, Canada
| | - Laszlo A. Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Ontario, Canada
| |
Collapse
|
14
|
Emhoff SM, Lynch JK, McCaffrey RJ. Performance and Symptom Validity Testing in Pediatric Assessment: A Review of the Literature. Dev Neuropsychol 2018; 43:671-707. [DOI: 10.1080/87565641.2018.1525612] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Affiliation(s)
- Stephanie M. Emhoff
- Department of Psychology, University at Albany, State University of New York, Albany, New York, USA
| | - Julie K. Lynch
- Department of Psychology, University at Albany, State University of New York, Albany, New York, USA
- Albany Neuropsychological Associates, Albany, New York, USA
| | - Robert J. McCaffrey
- Department of Psychology, University at Albany, State University of New York, Albany, New York, USA
- Albany Neuropsychological Associates, Albany, New York, USA
| |
Collapse
|
15
|
Erdodi LA. Aggregating validity indicators: The salience of domain specificity and the indeterminate range in multivariate models of performance validity assessment. APPLIED NEUROPSYCHOLOGY-ADULT 2017; 26:155-172. [PMID: 29111772 DOI: 10.1080/23279095.2017.1384925] [Citation(s) in RCA: 42] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
This study was designed to examine the "domain specificity" hypothesis in performance validity tests (PVTs) and the epistemological status of an "indeterminate range" when evaluating the credibility of a neuropsychological profile using a multivariate model of performance validity assessment. While previous research suggests that aggregating PVTs produces superior classification accuracy compared to individual instruments, the effect of the congruence between the criterion and predictor variable on signal detection and the issue of classifying borderline cases remain understudied. Data from a mixed clinical sample of 234 adults referred for cognitive evaluation (MAge = 46.6; MEducation = 13.5) were collected. Two validity composites were created: one based on five verbal PVTs (EI-5VER) and one based on five nonverbal PVTs (EI-5NV) and compared against several other PVTs. Overall, language-based tests of cognitive ability were more sensitive to elevations on the EI-5VER compared to visual-perceptual tests; whereas, the opposite was observed with the EI-5NV. However, the match between predictor and criterion variable had a more complex relationship with classification accuracy, suggesting the confluence of multiple factors (sensory modality, cognitive domain, testing paradigm). An "indeterminate range" of performance validity emerged that was distinctly different from both the Pass and the Fail group. Trichotomized criterion PVTs (Pass-Borderline-Fail) had a negative linear relationship with performance on tests of cognitive ability, providing further support for an "in-between" category separating the unequivocal Pass and unequivocal Fail classification range. The choice of criterion variable can influence classification accuracy in PVT research. Establishing a Borderline range between Pass and Fail more accurately reflected the distribution of scores on multiple PVTs. The traditional binary classification system imposes an artificial dichotomy on PVTs that was not fully supported by the data. Accepting "indeterminate" as a legitimate third outcome of performance validity assessment has the potential to improve the clinical utility of PVTs and defuse debates regarding "near-Passes" and "soft Fails."
Collapse
Affiliation(s)
- Laszlo A Erdodi
- a Department of Psychology , University of Windsor , Windsor , Canada
| |
Collapse
|
16
|
Detecting Feigned Attention-Deficit/Hyperactivity Disorder (ADHD): Current Methods and Future Directions. PSYCHOLOGICAL INJURY & LAW 2017. [DOI: 10.1007/s12207-017-9286-6] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
|
17
|
Erdodi LA, Lichtenstein JD. Invalid before impaired: an emerging paradox of embedded validity indicators. Clin Neuropsychol 2017; 31:1029-1046. [DOI: 10.1080/13854046.2017.1323119] [Citation(s) in RCA: 52] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Laszlo A. Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Canada
| | - Jonathan D. Lichtenstein
- Department of Psychiatry, Neuropsychology Services, Geisel School of Medicine at Dartmouth, Lebanon, NH, USA
| |
Collapse
|
18
|
Lichtenstein JD, Erdodi LA, Rai JK, Mazur-Mosiewicz A, Flaro L. Wisconsin Card Sorting Test embedded validity indicators developed for adults can be extended to children. Child Neuropsychol 2016; 24:247-260. [DOI: 10.1080/09297049.2016.1259402] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Jonathan D. Lichtenstein
- Department of Psychiatry, Neuropsychology Services, Geisel School of Medicine at Dartmouth, Lebanon, NH, USA
| | - Laszlo A. Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, ON, USA
| | - Jaspreet K. Rai
- Department of Psychology, Neuropsychology Track, University of Windsor, ON, USA
| | - Anya Mazur-Mosiewicz
- Department of Psychology, Chicago School of Professional Psychology, IL, USA
- Department of Psychiatry and Behavioral Science, Oklahoma State University, Tulsa, OK, USA
| | | |
Collapse
|
19
|
Erdodi LA, Pelletier CL, Roth RM. Elevations on select Conners’ CPT-II scales indicate noncredible responding in adults with traumatic brain injury. APPLIED NEUROPSYCHOLOGY-ADULT 2016; 25:19-28. [DOI: 10.1080/23279095.2016.1232262] [Citation(s) in RCA: 33] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Laszlo A. Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada
- Department of Psychiatry, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire, USA
| | - Chantalle L. Pelletier
- Department of Psychiatry, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire, USA
| | - Robert M. Roth
- Department of Psychiatry, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire, USA
| |
Collapse
|