1
|
Lace JW, Sanborn V, Galioto R. Standalone Performance Validity Tests May Be Differentially Related to Measures of Working Memory, Processing Speed, and Verbal Memory in Patients With Multiple Sclerosis. Assessment 2024; 31:732-744. [PMID: 37303186 DOI: 10.1177/10731911231178289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Cognitive functioning may account for minimal levels (i.e., 5%-14%) of variance of performance validity test (PVT) scores in clinical examinees. The present study extended this research twofold: (a) by determining the variance cognitive functioning explains within three distinct PVTs (b) in a sample of patients with multiple sclerosis (pwMS). Seventy-five pwMS (Mage = 48.50, 70.6% female, 80.9% White) completed the Victoria Symptom Validity Test (VSVT), Word Choice Test (WCT), Dot Counting Test (DCT), and three objective measures of working memory, processing speed, and verbal memory as part of clinical neuropsychological assessment. Regression analyses in credible groups (ns ranged from 54 to 63) indicated that cognitive functioning explained 24% to 38% of the variance in logarithmically transformed PVT variables. Variance from cognitive testing differed across PVTs: verbal memory significantly influenced both VSVT and WCT scores; working memory influenced VSVT and DCT scores; and processing speed influenced DCT scores. The WCT appeared least related to cognitive functioning of the included PVTs. Alternative plausible explanations, including the apparent domain/modality specificity hypothesis of PVTs versus the potential sensitivity of these PVTs to neurocognitive dysfunction in pwMS were discussed. Continued psychometric investigations into factors affecting performance validity, especially in multiple sclerosis, are warranted.
Collapse
Affiliation(s)
- John W Lace
- Cleveland Clinic Foundation, OH, USA
- Prevea Health, Green Bay, WI, USA
| | - Victoria Sanborn
- Kent State University, OH, USA
- VA Boston Healthcare System, Boston, MA, USA
| | - Rachel Galioto
- Cleveland Clinic Foundation, Mellen Center for Multiple Sclerosis, OH, USA
| |
Collapse
|
2
|
Tyson BT, Shahein A, Abeare CA, Baker SD, Kent K, Roth RM, Erdodi LA. Replicating a Meta-Analysis: The Search for the Optimal Word Choice Test Cutoff Continues. Assessment 2023; 30:2476-2490. [PMID: 36752050 DOI: 10.1177/10731911221147043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/09/2023]
Abstract
This study was designed to expand on a recent meta-analysis that identified ≤42 as the optimal cutoff on the Word Choice Test (WCT). We examined the base rate of failure and the classification accuracy of various WCT cutoffs in four independent clinical samples (N = 252) against various psychometrically defined criterion groups. WCT ≤ 47 achieved acceptable combinations of specificity (.86-.89) at .49 to .54 sensitivity. Lowering the cutoff to ≤45 improved specificity (.91-.98) at a reasonable cost to sensitivity (.39-.50). Making the cutoff even more conservative (≤42) disproportionately sacrificed sensitivity (.30-.38) for specificity (.98-1.00), while still classifying 26.7% of patients with genuine and severe deficits as non-credible. Critical item (.23-.45 sensitivity at .89-1.00 specificity) and time-to-completion cutoffs (.48-.71 sensitivity at .87-.96 specificity) were effective alternative/complementary detection methods. Although WCT ≤ 45 produced the best overall classification accuracy, scores in the 43 to 47 range provide comparable objective psychometric evidence of non-credible responding. Results question the need for designating a single cutoff as "optimal," given the heterogeneity of signal detection environments in which individual assessors operate. As meta-analyses often fail to replicate, ongoing research is needed on the classification accuracy of various WCT cutoffs.
Collapse
Affiliation(s)
| | | | | | | | | | - Robert M Roth
- Dartmouth-Hitchcock Medical Center, Lebanon, NH, USA
| | | |
Collapse
|
3
|
Leonhard C. Review of Statistical and Methodological Issues in the Forensic Prediction of Malingering from Validity Tests: Part I: Statistical Issues. Neuropsychol Rev 2023; 33:581-603. [PMID: 37612531 DOI: 10.1007/s11065-023-09601-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2021] [Accepted: 03/29/2023] [Indexed: 08/25/2023]
Abstract
Forensic neuropsychological examinations with determination of malingering have tremendous social, legal, and economic consequences. Thousands of studies have been published aimed at developing and validating methods to diagnose malingering in forensic settings, based largely on approximately 50 validity tests, including embedded and stand-alone performance validity tests. This is the first part of a two-part review. Part I explores three statistical issues related to the validation of validity tests as predictors of malingering, including (a) the need to report a complete set of classification accuracy statistics, (b) how to detect and handle collinearity among validity tests, and (c) how to assess the classification accuracy of algorithms for aggregating information from multiple validity tests. In the Part II companion paper, three closely related research methodological issues will be examined. Statistical issues are explored through conceptual analysis, statistical simulations, and through reanalysis of findings from prior validation studies. Findings suggest extant neuropsychological validity tests are collinear and contribute redundant information to the prediction of malingering among forensic examinees. Findings further suggest that existing diagnostic algorithms may miss diagnostic accuracy targets under most realistic conditions. The review makes several recommendations to address these concerns, including (a) reporting of full confusion table statistics with 95% confidence intervals in diagnostic trials, (b) the use of logistic regression, and (c) adoption of the consensus model on the "transparent reporting of multivariate prediction models for individual prognosis or diagnosis" (TRIPOD) in the malingering literature.
Collapse
Affiliation(s)
- Christoph Leonhard
- The Chicago School of Professional Psychology at Xavier University of Louisiana, Box 200, 1 Drexel Dr, New Orleans, LA, 70125, USA.
| |
Collapse
|
4
|
Tyson BT, Pyne SR, Crisan I, Calamia M, Holcomb M, Giromini L, Erdodi LA. Logical memory, visual reproduction, and verbal paired associates are effective embedded validity indicators in patients with traumatic brain injury. APPLIED NEUROPSYCHOLOGY. ADULT 2023:1-10. [PMID: 36881969 DOI: 10.1080/23279095.2023.2179400] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/09/2023]
Abstract
OBJECTIVE This study was design to evaluate the potential of the recognition trials for the Logical Memory (LM), Visual Reproduction (VR), and Verbal Paired Associates (VPA) subtests of the Wechsler Memory Scales-Fourth Edition (WMS-IV) to serve as embedded performance validity tests (PVTs). METHOD The classification accuracy of the three WMS-IV subtests was computed against three different criterion PVTs in a sample of 103 adults with traumatic brain injury (TBI). RESULTS The optimal cutoffs (LM ≤ 20, VR ≤ 3, VPA ≤ 36) produced good combinations of sensitivity (.33-.87) and specificity (.92-.98). An age-corrected scaled score of ≤5 on either of the free recall trials on the VPA was specific (.91-.92) and relatively sensitive (.48-.57) to psychometrically defined invalid performance. A VR I ≤ 5 or VR II ≤ 4 had comparable specificity, but lower sensitivity (.25-.42). There was no difference in failure rate as a function of TBI severity. CONCLUSIONS In addition to LM, VR, and VPA can also function as embedded PVTs. Failing validity cutoffs on these subtests signals an increased risk of non-credible presentation and is robust to genuine neurocognitive impairment. However, they should not be used in isolation to determine the validity of an overall neurocognitive profile.
Collapse
Affiliation(s)
- Brad T Tyson
- Evergreen Neuroscience Institute, Evergreen Health Medical Center, Kirkland, WA, USA
| | | | - Iulia Crisan
- Department of Psychology, West University of Timisoara, Timisoara, Romania
| | - Matthew Calamia
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA
| | | | | | - Laszlo A Erdodi
- Jefferson Neurobehavioral Group, New Orleans, LA, USA
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, ON, Canada
| |
Collapse
|
5
|
Chang F, Cerny BM, Tse PKY, Rauch AA, Khan H, Phillips MS, Fletcher NB, Resch ZJ, Ovsiew GP, Jennette KJ, Soble JR. Using the Grooved Pegboard Test as an Embedded Validity Indicator in a Mixed Neuropsychiatric Sample with Varying Cognitive Impairment: Cross-Validation Problems. Percept Mot Skills 2023; 130:770-789. [PMID: 36634223 DOI: 10.1177/00315125231151779] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
Embedded validity indicators (EVIs) derived from motor tests have received less empirical attention than those derived from tests of other neuropsychological abilities, particularly memory. Preliminary evidence suggests that the Grooved Pegboard Test (GPB) may function as an EVI, but existing studies were largely conducted using simulators and population samples without cognitive impairment. In this study we aimed to evaluate the GPB's classification accuracy as an EVI among a mixed clinical neuropsychiatric sample with and without cognitive impairment. This cross-sectional study comprised 223 patients clinically referred for neuropsychological testing. GPB raw and T-scores for both dominant and nondominant hands were examined as EVIs. A known-groups design, based on ≤1 failure on a battery of validated, independent criterion PVTs, showed that GPB performance differed significantly by validity group. Within the valid group, receiver operating characteristic curve analyses revealed that only the dominant hand raw score displayed acceptable classification accuracy for detecting invalid performance (area under curve [AUC] = .72), with an optimal cut-score of ≥106 seconds (33% sensitivity/88% specificity). All other scores had marginally lower classification accuracy (AUCs = .65-.68) for differentiating valid from invalid performers. Therefore, the GPB demonstrated limited utility as an EVI in a clinical sample containing patients with bona fide cognitive impairment.
Collapse
Affiliation(s)
- Fini Chang
- Department of Psychiatry, 12247University of Illinois College of Medicine, Chicago, Illinois, United States.,Department of Psychology, 12247University of Illinois at Chicago, Chicago, Illinois, United States
| | - Brian M Cerny
- Department of Psychiatry, 12247University of Illinois College of Medicine, Chicago, Illinois, United States.,Department of Psychology, Illinois Institute of Technology, Chicago, Illinois, United States
| | - Phoebe Ka Yin Tse
- Department of Psychiatry, 12247University of Illinois College of Medicine, Chicago, Illinois, United States.,Department of Clinical Psychology, The Chicago School of Professional Psychology, Chicago, Illinois, United States
| | - Andrew A Rauch
- Department of Psychiatry, 12247University of Illinois College of Medicine, Chicago, Illinois, United States.,Department of Psychology, Loyola University Chicago, Chicago, Illinois, United States
| | - Humza Khan
- Department of Psychiatry, 12247University of Illinois College of Medicine, Chicago, Illinois, United States.,Department of Psychology, Illinois Institute of Technology, Chicago, Illinois, United States
| | - Matthew S Phillips
- Department of Psychiatry, 12247University of Illinois College of Medicine, Chicago, Illinois, United States.,Department of Clinical Psychology, The Chicago School of Professional Psychology, Chicago, Illinois, United States
| | - Noah B Fletcher
- Department of Psychiatry, 12247University of Illinois College of Medicine, Chicago, Illinois, United States
| | - Zachary J Resch
- Department of Psychiatry, 12247University of Illinois College of Medicine, Chicago, Illinois, United States
| | - Gabriel P Ovsiew
- Department of Psychiatry, 12247University of Illinois College of Medicine, Chicago, Illinois, United States
| | - Kyle J Jennette
- Department of Psychiatry, 12247University of Illinois College of Medicine, Chicago, Illinois, United States
| | - Jason R Soble
- Department of Psychiatry, 12247University of Illinois College of Medicine, Chicago, Illinois, United States.,Department of Neurology, 12247University of Illinois College of Medicine, Chicago, Illinois, United States
| |
Collapse
|
6
|
Martin BJ, Sober JD, Millis SR, Hanks RA, Reslan S, Waldron-Perrine B. CVLT-3 response bias as an indicator of performance validity in a litigating population. Clin Neuropsychol 2023; 37:81-90. [PMID: 34689724 DOI: 10.1080/13854046.2021.1993347] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
This study examined the efficacy of CVLT-3 response bias (i.e., parametric and nonparametric response bias) indices in differentiating between a clinical sample with traumatic brain injury and a litigating sample with poor performance validity. Participants included 106 individuals, divided into two groups: clinical group with TBI (n = 56) and a litigating group who demonstrated inadequate performance validity (n = 50), as measured by failure on at least two performance validity tests. Archival CVLT-II data was rescored utilizing the CVLT-3 scoring and normative data. Receiver operator characteristic (ROC) curve analysis was used to evaluate the diagnostic discriminability of the two response bias indices. Both parametric and nonparametric bias indices showed acceptable levels of diagnostic discrimination: AUC = .791 for parametric response bias and AUC = .753 for nonparametric response bias. Parametric response bias' discrimination was statistically superior to the nonparametric responses bias' discrimination. The CVLT-3 response bias score demonstrated good sensitivity and specificity when differentiating between individuals in a clinical sample with TBI and individuals in litigation who demonstrated inadequate performance validity.
Collapse
Affiliation(s)
- Bess J Martin
- Neuropsychology, Rehabilitation Institute of Michigan, Detroit, Michigan, USA
| | - Jonathan D Sober
- Neuropsychology, Rehabilitation Institute of Michigan, Detroit, Michigan, USA
| | - Scott R Millis
- Department of Physical Medicine and Rehabilitation, Rehabilitation Institute of Michigan, Wayne State University School of Medicine, Detroit, Michigan, USA
| | - Robin A Hanks
- Department of Physical Medicine and Rehabilitation, Rehabilitation Institute of Michigan, Wayne State University School of Medicine, Detroit, Michigan, USA
| | - Summar Reslan
- Department of Physical Medicine and Rehabilitation, Rehabilitation Institute of Michigan, Wayne State University School of Medicine, Detroit, Michigan, USA
| | - Brigid Waldron-Perrine
- Department of Physical Medicine and Rehabilitation, Rehabilitation Institute of Michigan, Wayne State University School of Medicine, Detroit, Michigan, USA
| |
Collapse
|
7
|
Weigard A, Spencer RJ. Benefits and challenges of using logistic regression to assess neuropsychological performance validity: Evidence from a simulation study. Clin Neuropsychol 2023; 37:34-59. [PMID: 35006042 PMCID: PMC9273108 DOI: 10.1080/13854046.2021.2023650] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2021] [Accepted: 12/22/2021] [Indexed: 02/07/2023]
Abstract
Logistic regression (LR) is recognized as a promising method for making decisions about neuropsychological performance validity by integrating information across multiple measures. However, this method has yet to be widely adopted in clinical practice, likely because several open questions remain about its utility relative to simpler methods, its effectiveness across different clinical contexts, and its feasibility at sample sizes common in the field. The current study addresses these questions by assessing classification performance of logistic regression and alternative methods across an array of simulated data sets. We simulated scores of valid and invalid performers on 6 tests designed to mimic the psychometric and distributional properties of real performance validity measures. Out-of-sample predictive performance of LR and a commonly used alternative ("vote counting") was assessed across different base rates, validity measure properties, and sample sizes. LR improved classification accuracy by 2%-12% across simulation conditions, primarily by improving sensitivity. False positives and negatives can be further reduced when LR predictions are interpreted as continuous, rather than binary. LR made robust predictions at sample sizes feasible for neuropsychology research (N = 307) and when as few as 2 tests with good psychometric properties were used. Although training and test data sets of at least several hundred individuals may be required to develop and evaluate LR models for use in clinical practice, LR promises to be an efficient and powerful tool for improving judgements about performance validity. We offer several recommendations for model development and LR interpretation in a clinical setting.
Collapse
Affiliation(s)
| | - Robert J. Spencer
- Department of Psychiatry, University of Michigan
- VA Ann Arbor Healthcare System
| |
Collapse
|
8
|
Boress K, Gaasedelen OJ, Croghan A, Johnson MK, Caraher K, Basso MR, Whiteside DM. Validation of the Personality Assessment Inventory (PAI) scale of scales in a mixed clinical sample. Clin Neuropsychol 2022; 36:1844-1859. [PMID: 33730975 PMCID: PMC8474121 DOI: 10.1080/13854046.2021.1900400] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
Objective: This exploratory study examined the classification accuracy of three derived scales aimed at detecting cognitive response bias in neuropsychological samples. The derived scales are composed of existing scales from the Personality Assessment Inventory (PAI). A mixed clinical sample of consecutive outpatients referred for neuropsychological assessment at a large Midwestern academic medical center was utilized. Participants and Methods: Participants included 332 patients who completed study's embedded and free-standing performance validity tests (PVTs) and the PAI. PASS and FAIL groups were created based on PVT performance to evaluate the classification accuracy of the derived scales. Three new scales, Cognitive Bias Scale of Scales 1-3, (CB-SOS1-3) were derived by combining existing scales by either summing the scales together and dividing by the total number of scales summed, or by logistically deriving a variable from the contributions of several scales. Results: All of the newly derived scales significantly differentiated between PASS and FAIL groups. All of the derived SOS scales demonstrated acceptable classification accuracy (i.e. CB-SOS1 AUC = 0.72; CB-SOS2 AUC = 0.73; CB-SOS3 AUC = 0.75). Conclusions: This exploratory study demonstrates that attending to scale-level PAI data may be a promising area of research in improving prediction of PVT failure.
Collapse
Affiliation(s)
- Kaley Boress
- Department of Psychiatry, University of Iowa Hospitals and Clinics, Iowa City, IA, USA
| | | | - Anna Croghan
- Department of Psychiatry, University of Iowa Hospitals and Clinics, Iowa City, IA, USA
| | - Marcie King Johnson
- Department of Psychiatry, University of Iowa Hospitals and Clinics, Iowa City, IA, USA,Department of Psychological and Brain Sciences, University of Iowa, Iowa City, IA, USA
| | - Kristen Caraher
- Department of Psychiatry, University of Iowa Hospitals and Clinics, Iowa City, IA, USA
| | - Michael R. Basso
- Department of Psychiatry and Psychology, Mayo Clinic, Rochester, NY, USA
| | - Douglas M. Whiteside
- Department of Rehabilitation Medicine, Neuropsychology Laboratory, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
9
|
Giromini L, Young G, Sellbom M. Assessing Negative Response Bias Using Self-Report Measures: New Articles, New Issues. PSYCHOLOGICAL INJURY & LAW 2022. [DOI: 10.1007/s12207-022-09444-2] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
Abstract
AbstractIn psychological injury and related forensic evaluations, two types of tests are commonly used to assess Negative Response Bias (NRB): Symptom Validity Tests (SVTs) and Performance Validity Tests (PVTs). SVTs assess the credibility of self-reported symptoms, whereas PVTs assess the credibility of observed performance on cognitive tasks. Compared to the large and ever-growing number of published PVTs, there are still relatively few validated self-report SVTs available to professionals for assessing symptom validity. In addition, while several studies have examined how to combine and integrate the results of multiple independent PVTs, there are few studies to date that have addressed the combination and integration of information obtained from multiple self-report SVTs. The Special Issue of Psychological Injury and Law introduced in this article aims to help fill these gaps in the literature by providing readers with detailed information about the convergent and incremental validity, strengths and weaknesses, and applicability of a number of selected measures of NRB under different conditions and in different assessment contexts. Each of the articles in this Special Issue focuses on a particular self-report SVT or set of SVTs and summarizes their conditions of use, strengths, weaknesses, and possible cut scores and relative hit rates. Here, we review the psychometric properties of the 19 selected SVTs and discuss their advantages and disadvantages. In addition, we make tentative proposals for the field to consider regarding the number of SVTs to be used in an assessment, the number of SVT failures required to invalidate test results, and the issue of redundancy when selecting multiple SVTs for an assessment.
Collapse
|
10
|
Thompson JL, Beltran-Najera I, Johnson B, Morales Y, Woods SP. Evidence for neuropsychological health disparities in Black Americans with HIV disease. Clin Neuropsychol 2022; 36:388-413. [PMID: 35166174 PMCID: PMC8868032 DOI: 10.1080/13854046.2021.1947387] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
Abstract
OBJECTIVE Black Americans are at high risk for HIV disease and associated morbidity. The impact and clinical correlates of HIV-associated neurocognitive impairment among Black Americans is not fully understood. The current study uses a full factorial design to examine the independent and combined effects of race and HIV disease on neurocognitive functioning, including its associations with everyday functioning and clinical disease markers in Black and White persons with HIV (PWH). METHOD Participants included 40 Black PWH, 83 White PWH, 28 Black HIV- and 64 White HIV- individuals. Neurocognition was measured by raw sample-based z-scores from a clinical battery. Everyday functioning was assessed using self- and clinician-rated measures of cognitive symptoms and activities of daily living. HIV-associated neurocognitive disorders were also classified using demographically adjusted normative standards and the Frascati criteria. RESULTS We observed a significant three-way interaction between HIV, race, and domain on raw neurocognitive z-scores. This omnibus effect was driven by medium and large effect size decrements in processing speed and semantic memory, respectively, in Black PWH compared to other study groups. Black PWH also demonstrated higher frequencies of HIV-associated neurocognitive disorders as compared to White PWH. Unexpectedly, global neurocognitive performance was negatively related to everyday functioning impairments for White PWH, but not for Black PWH. CONCLUSIONS Systemic disadvantages for Black Americans may combine with HIV disease to compound some neurocognitive impairments in this under-served population. Prospective studies are needed to identify better ways to prevent, measure, diagnose, and manage HIV-associated neurocognitive disorders among Black Americans.
Collapse
Affiliation(s)
| | | | | | | | - Steven Paul Woods
- Corresponding author: Steven Paul Woods, Psy.D. . Address: 126 Heyne Building, Suite 239D, Houston, TX 77004-5022. Phone: 713-743-6415
| |
Collapse
|
11
|
Dunn A, Pyne S, Tyson B, Roth R, Shahein A, Erdodi L. Critical Item Analysis Enhances the Classification Accuracy of the Logical Memory Recognition Trial as a Performance Validity Indicator. Dev Neuropsychol 2021; 46:327-346. [PMID: 34525856 DOI: 10.1080/87565641.2021.1956499] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
OBJECTIVE : Replicate previous research on Logical Memory Recognition (LMRecog) and perform a critical item analysis. METHOD : Performance validity was psychometrically operationalized in a mixed clinical sample of 213 adults. Classification of the LMRecog and nine critical items (CR-9) was computed. RESULTS : LMRecog ≤20 produced a good combination of sensitivity (.30-.35) and specificity (.89-.90). CR-9 ≥5 and ≥6 had comparable classification accuracy. CR-9 ≥5 increased sensitivity by 4% over LMRecog ≤20; CR-9 ≥6 increased specificity by 6-8% over LMRecog ≤20; CR-9 ≥7 increased specificity by 8-15%. CONCLUSIONS : Critical item analysis enhances the classification accuracy of the optimal LMRecog cutoff (≤20).
Collapse
Affiliation(s)
- Alexa Dunn
- Department of Psychology, University of Windsor, Windsor, Canada
| | - Sadie Pyne
- Windsor Neuropsychology, Windsor, Canada
| | - Brad Tyson
- Neuroscience Institute, Evergreen Neuroscience Institute, EvergreenHealth Medical Center, Kirkland, USA
| | - Robert Roth
- Neuropsychology Services, Dartmouth-Hitchcock Medical Center, USA
| | - Ayman Shahein
- Department of Clinical Neurosciences, University of Calgary, Calgary, Canada
| | - Laszlo Erdodi
- Department of Psychology, University of Windsor, Windsor, Canada
| |
Collapse
|
12
|
North C, Desai R, Saunders R, Suárez-González A, Bamiou D, Costafreda SG, de Haan G, Halls G, Heutink J, O'Nions E, Utoomprurkporn N, John A, Stott J. Neuropsychological deficits in Posterior Cortical Atrophy and typical Alzheimer's disease: A meta-analytic review. Cortex 2021; 143:223-236. [PMID: 34464853 DOI: 10.1016/j.cortex.2021.07.011] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2021] [Revised: 05/13/2021] [Accepted: 07/30/2021] [Indexed: 11/27/2022]
Abstract
AIMS To identify cognitive tests that best differentiate between Posterior Cortical Atrophy (PCA) and typical Alzheimer's Disease (tAD), as well as PCA and healthy control (HC) participants. METHOD Medline, PsycInfo and Web of Science were systematically searched using terms related to PCA, tAD, and cognitive testing. Seventeen studies were identified, including 441 PCA, 391 tAD, and 284 HC participants. Standardised effect sizes of mean scores were calculated to measure performance differences on cognitive tests for PCA versus tAD and PCA versus HC groups. Meta-analyses used a random effects model. RESULTS The most discriminating cognitive tests for PCA and tAD presentations were measures of visuospatial function and verbal memory. Large, significant effect sizes were produced for all measures of visuospatial function, most notably for Rey-Osterrieth Copy (Hedges' g = -2.79), VOSP Fragmented letters (Hedges' g = -1.73), VOSP Dot Counting (Hedges' g = -1.74), and VOSP Cube Analysis (Hedges' g = -1.98). For measures of verbal memory, the RAVLT delay and Digit Span Backwards produced significant medium effects (Hedges' g = .62 and -.56, respectively). CONCLUSION Establishing a common framework for testing individuals with PCA has important implications for diagnosis and treatment, and forms a practical objective for future research. Findings from this meta-analysis suggest that measures of visuospatial function and verbal memory would form an important part of this framework.
Collapse
Affiliation(s)
- Courtney North
- ADAPT Lab, Research Department of Clinical, Educational and Health Psychology, University College London, London, UK
| | - Roopal Desai
- ADAPT Lab, Research Department of Clinical, Educational and Health Psychology, University College London, London, UK.
| | - Rob Saunders
- Centre for Outcomes Research and Effectiveness, University College London, UK
| | | | - Doris Bamiou
- UCL Ear Institute, University College London, UK
| | - Sergi G Costafreda
- ADAPT Lab, Research Department of Clinical, Educational and Health Psychology, University College London, London, UK
| | - Gera de Haan
- University of Groningen, Groningen, the Netherlands; Royal Dutch Visio, Centre of Expertise for Blind and Partially Sighted People, Huizen, the Netherlands
| | - Georgia Halls
- ADAPT Lab, Research Department of Clinical, Educational and Health Psychology, University College London, London, UK
| | - Joost Heutink
- University of Groningen, Groningen, the Netherlands; Royal Dutch Visio, Centre of Expertise for Blind and Partially Sighted People, Huizen, the Netherlands
| | - Elizabeth O'Nions
- ADAPT Lab, Research Department of Clinical, Educational and Health Psychology, University College London, London, UK
| | - Nattawan Utoomprurkporn
- UCL Ear Institute, University College London, UK; Faculty of Medicine, Chulalongkorn University, Thailand
| | - Amber John
- ADAPT Lab, Research Department of Clinical, Educational and Health Psychology, University College London, London, UK
| | - Joshua Stott
- ADAPT Lab, Research Department of Clinical, Educational and Health Psychology, University College London, London, UK
| |
Collapse
|
13
|
Erdodi LA. Five shades of gray: Conceptual and methodological issues around multivariate models of performance validity. NeuroRehabilitation 2021; 49:179-213. [PMID: 34420986 DOI: 10.3233/nre-218020] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
OBJECTIVE This study was designed to empirically investigate the signal detection profile of various multivariate models of performance validity tests (MV-PVTs) and explore several contested assumptions underlying validity assessment in general and MV-PVTs specifically. METHOD Archival data were collected from 167 patients (52.4%male; MAge = 39.7) clinicially evaluated subsequent to a TBI. Performance validity was psychometrically defined using two free-standing PVTs and five composite measures, each based on five embedded PVTs. RESULTS MV-PVTs had superior classification accuracy compared to univariate cutoffs. The similarity between predictor and criterion PVTs influenced signal detection profiles. False positive rates (FPR) in MV-PVTs can be effectively controlled using more stringent multivariate cutoffs. In addition to Pass and Fail, Borderline is a legitimate third outcome of performance validity assessment. Failing memory-based PVTs was associated with elevated self-reported psychiatric symptoms. CONCLUSIONS Concerns about elevated FPR in MV-PVTs are unsubstantiated. In fact, MV-PVTs are psychometrically superior to individual components. Instrumentation artifacts are endemic to PVTs, and represent both a threat and an opportunity during the interpretation of a given neurocognitive profile. There is no such thing as too much information in performance validity assessment. Psychometric issues should be evaluated based on empirical, not theoretical models. As the number/severity of embedded PVT failures accumulates, assessors must consider the possibility of non-credible presentation and its clinical implications to neurorehabilitation.
Collapse
|
14
|
Sabelli AG, Messa I, Giromini L, Lichtenstein JD, May N, Erdodi LA. Symptom Versus Performance Validity in Patients with Mild TBI: Independent Sources of Non-credible Responding. PSYCHOLOGICAL INJURY & LAW 2021. [DOI: 10.1007/s12207-021-09400-6] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
|
15
|
Erdodi LA, Abeare CA. Stronger Together: The Wechsler Adult Intelligence Scale-Fourth Edition as a Multivariate Performance Validity Test in Patients with Traumatic Brain Injury. Arch Clin Neuropsychol 2020; 35:188-204. [PMID: 31696203 DOI: 10.1093/arclin/acz032] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2019] [Revised: 06/18/2019] [Accepted: 06/22/2019] [Indexed: 12/17/2022] Open
Abstract
OBJECTIVE This study was designed to evaluate the classification accuracy of a multivariate model of performance validity assessment using embedded validity indicators (EVIs) within the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV). METHOD Archival data were collected from 100 adults with traumatic brain injury (TBI) consecutively referred for neuropsychological assessment in a clinical setting. The classification accuracy of previously published individual EVIs nested within the WAIS-IV and a composite measure based on six independent EVIs were evaluated against psychometrically defined non-credible performance. RESULTS Univariate validity cutoffs based on age-corrected scaled scores on Coding, Symbol Search, Digit Span, Letter-Number-Sequencing, Vocabulary minus Digit Span, and Coding minus Symbol Search were strong predictors of psychometrically defined non-credible responding. Failing ≥3 of these six EVIs at the liberal cutoff improved specificity (.91-.95) over univariate cutoffs (.78-.93). Conversely, failing ≥2 EVIs at the more conservative cutoff increased and stabilized sensitivity (.43-.67) compared to univariate cutoffs (.11-.63) while maintaining consistently high specificity (.93-.95). CONCLUSIONS In addition to being a widely used test of cognitive functioning, the WAIS-IV can also function as a measure of performance validity. Consistent with previous research, combining information from multiple EVIs enhanced the classification accuracy of individual cutoffs and provided more stable parameter estimates. If the current findings are replicated in larger, diagnostically and demographically heterogeneous samples, the WAIS-IV has the potential to become a powerful multivariate model of performance validity assessment. BRIEF SUMMARY Using a combination of multiple performance validity indicators embedded within the subtests of theWechsler Adult Intelligence Scale, the credibility of the response set can be establishedwith a high level of confidence. Multivariatemodels improve classification accuracy over individual tests. Relying on existing test data is a cost-effective approach to performance validity assessment.
Collapse
Affiliation(s)
- Laszlo A Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, ON, Canada
| | - Christopher A Abeare
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, ON, Canada
| |
Collapse
|
16
|
Kilciksiz CM, Keefe R, Benoit J, Öngür D, Torous J. Verbal memory measurement towards digital perspectives in first-episode psychosis: A review. Schizophr Res Cogn 2020; 21:100177. [PMID: 32322540 PMCID: PMC7163058 DOI: 10.1016/j.scog.2020.100177] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2020] [Revised: 03/20/2020] [Accepted: 03/21/2020] [Indexed: 12/13/2022]
Abstract
BACKGROUND Even in the early phases of psychotic spectrum illnesses such as schizophrenia, patients can experience cognitive decline or deficits prior to the onset of psychotic symptoms such as delusions and hallucinations. In this systematic review, we assessed which verbal memory assessments are most widely used in first-episode psychosis and may be applied via digital technologies (smartphone applications, etc.) for use in early detection. METHODS In November 2019, we searched for studies measuring verbal memory in first episode psychosis or schizophrenia over the past 10 years on PubMed and PsycINFO. We screened abstracts of these studies and excluded review studies. Full-texts of included studies were used to identify the verbal memory measurement tests, follow-up frequencies, and sample sizes. RESULTS We screened 233 reports and found that 120 original research studies measured verbal memory in first episode psychosis over the past 10 years. Four of these studies specified using a computer, 24 (20%) used a paper-pen format, 1(1%) used both, and 91 (76%) studies did not specify their administration tools or suggest there were offered in digital formats. Thirty-five (30%) studies had follow-up measurements of verbal memory, while 85 (70%) had only a single verbal memory measurement. DISCUSSION While many scales are commonly used to measure verbal memory in first episode psychosis, they are not often administered via digital technology. There is an emerging opportunity to administer these and other tests via digital technologies for expanding access to early detection of cognitive decline in clinical high risk and first-episode psychosis.
Collapse
Affiliation(s)
- Can Mişel Kilciksiz
- Digital Psychiatry Division, Psychosis Research Program, Beth Israel Deaconess Medical Center, Boston, MA, United States of America
- Department of Psychiatry, Harvard Medical School, Boston, MA, United States of America
| | - Richard Keefe
- Psychiatry and Behavioral Sciences, Duke University School of Medicine, Durham, NC, United States of America
| | - James Benoit
- Digital Psychiatry Division, Psychosis Research Program, Beth Israel Deaconess Medical Center, Boston, MA, United States of America
- Department of Psychiatry, Harvard Medical School, Boston, MA, United States of America
| | - Dost Öngür
- Department of Psychiatry, Harvard Medical School, Boston, MA, United States of America
- Psychotic Disorders Division, McLean Hospital, Belmont, MA, United States of America
| | - John Torous
- Digital Psychiatry Division, Psychosis Research Program, Beth Israel Deaconess Medical Center, Boston, MA, United States of America
- Department of Psychiatry, Harvard Medical School, Boston, MA, United States of America
| |
Collapse
|
17
|
Kirk JW, Baker DA, Kirk JJ, MacAllister WS. A review of performance and symptom validity testing with pediatric populations. APPLIED NEUROPSYCHOLOGY-CHILD 2020; 9:292-306. [DOI: 10.1080/21622965.2020.1750118] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Affiliation(s)
- John W. Kirk
- Kirk Neurobehavioral Health, Louisville, CO, USA
| | - David A. Baker
- Department of Physical Medicine & Rehabilitation, Children’s Hospital Colorado, University of Colorado School of Medicine, Aurora, CO, USA
| | | | - William S. MacAllister
- Alberta Children’s Hospital Research Institute and Alberta Health Sciences, Alberta, Canada
| |
Collapse
|
18
|
Optimization of Performance Validity Test (PVT) Cutoffs across Healthy and Non-Referred Clinical Research Samples. PSYCHOLOGICAL INJURY & LAW 2020. [DOI: 10.1007/s12207-020-09372-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
19
|
Thiruselvam I, Hoelzle JB. Refined Measurement of Verbal Learning and Memory: Application of Item Response Theory to California Verbal Learning Test - Second Edition (CVLT-II) Learning Trials. Arch Clin Neuropsychol 2020; 35:90-104. [PMID: 30615062 DOI: 10.1093/arclin/acy097] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2018] [Revised: 10/30/2018] [Accepted: 11/23/2018] [Indexed: 01/04/2023] Open
Abstract
OBJECTIVE This study was conducted to investigate how items contained within the California Verbal Learning Test - Second Edition (CVLT-II; Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (2000). California Verbal Learning Test (2nd ed.). San Antonio, TX: Psychological Corporation) learning trials function and to evaluate whether weighted scoring approaches might improve quantification of verbal memory. METHOD Archival data from 577 individuals (338 healthy young adults, 239 medical or psychiatric patients referred to a neuropsychology clinic) were obtained and evaluated using item response theory. RESULTS The serial position effect was evident across trials, but was most evident in Trials 1, 2, and 3. CVLT-II Trial 5 was the most effective among the five learning trials in quantifying verbal memory, although it was most effective when measuring lower memory ability levels. In contrast, CVLT-II Trial 1 items had, on average, the highest difficulty levels. Various weighted scoring approaches did not appear incrementally helpful in improving prediction of memory performance. CONCLUSION Specific items and trials differentially discriminate between examinees with low levels of memory ability; it is important to thoroughly evaluate item properties of tests used in clinical decision-making.
Collapse
Affiliation(s)
| | - James B Hoelzle
- Department of Psychology, Marquette University, Milwaukee, WI, USA
| |
Collapse
|
20
|
Moore RA, Lippa SM, Brickell TA, French LM, Lange RT. Clinical utility of WAIS-IV 'excessive decline from premorbid functioning' scores to detect invalid test performance following traumatic brain injury. Clin Neuropsychol 2019; 34:512-528. [PMID: 31608766 DOI: 10.1080/13854046.2019.1668059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
Objective: Excessive Decline from Premorbid Functioning (EDPF), an atypical discrepancy between demographically predicted and obtained Wechsler Adult Intelligence Scale-4th Edition (WAIS-IV) scores, has been recently proposed as a potential embedded performance validity test (PVT). This study examined the clinical utility of EDPF scores to detect invalid test performance following traumatic brain injury (TBI).Methods: Participants were 194 U.S. military service members who completed neuropsychological testing on average 2.4 years (SD = 4.0) following uncomplicated mild, complicated mild, moderate, severe, or penetrating TBI (Age: M = 34.0, SD = 9.9). Using TBI severity and PVT performance (i.e., PVT Pass/Fail), participants were classified into three groups: Uncomplicated Mild TBI-PVT Fail (MTBI-Fail; n = 21), Uncomplicated Mild TBI-PVT Pass (MTBI-Pass; n = 94), and Complicated Mild to Severe/Penetrating TBI-PVT Pass (CM/STBI-Pass; n = 79). Seven EDPF measures were calculated by subtracting WAIS-IV obtained index scores from the demographically predicted scores from the Test of Premorbid Functioning (TOPF). Cutoff scores to detect invalid test performance were examined for each EDPF measure separately.Results: The MTBI-Fail group had higher scores than the MTBI-Pass and CM/STBI-Pass groups on five of the seven EDPF measures (p<.05). Overall, the EDPF measure using the Processing Speed Index (EDPF-PSI) was the most useful score to detect invalid test performance. However, sensitivity was only low to moderate depending on the cutoff score used.Conclusions: These findings provide support for the use of EDPF as an embedded PVT to be considered along with other performance validity data when administering the WAIS-IV.
Collapse
Affiliation(s)
- Rich A Moore
- Neuropsychology Assessment Services, Walter Reed National Military Medical Center, Bethesda, MD, USA
| | - Sara M Lippa
- Defense and Veterans Brain Injury Center, Walter Reed National Military Medical Center, Silver Spring, MD, USA.,National Intrepid Center of Excellence, Walter Reed National Military Medical Center, Bethesda, MD, USA
| | - Tracey A Brickell
- Defense and Veterans Brain Injury Center, Walter Reed National Military Medical Center, Silver Spring, MD, USA.,National Intrepid Center of Excellence, Walter Reed National Military Medical Center, Bethesda, MD, USA.,Uniformed Services University of the Health Sciences, Bethesda, MD, USA
| | - Louis M French
- Defense and Veterans Brain Injury Center, Walter Reed National Military Medical Center, Silver Spring, MD, USA.,National Intrepid Center of Excellence, Walter Reed National Military Medical Center, Bethesda, MD, USA.,Uniformed Services University of the Health Sciences, Bethesda, MD, USA
| | - Rael T Lange
- Defense and Veterans Brain Injury Center, Walter Reed National Military Medical Center, Silver Spring, MD, USA.,National Intrepid Center of Excellence, Walter Reed National Military Medical Center, Bethesda, MD, USA.,University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
21
|
|
22
|
Rai JK, Erdodi LA. Impact of criterion measures on the classification accuracy of TOMM-1. APPLIED NEUROPSYCHOLOGY-ADULT 2019; 28:185-196. [PMID: 31187632 DOI: 10.1080/23279095.2019.1613994] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
This study was designed to examine the effect of various criterion measures on the classification accuracy of Trial 1 of the Test of Memory Malingering (TOMM-1), a free-standing performance validity test (PVT). Archival data were collected from a case sequence of 91 (M Age = 42.2 years; M Education = 12.7) patients clinically referred for neuropsychological assessment. Trials 2 and Retention of the TOMM, the Word Choice Test, and three validity composites were used as criterion PVTs. Classification accuracy varied systematically as a function of criterion PVT. TOMM-1 ≤ 43 emerged as the optimal cutoff, resulting in a wide range of sensitivity (.47-1.00), with perfect overall specificity. Failing the TOMM-1 was unrelated to age, education or gender, but was associated with elevated self-reported depression. Results support the utility of TOMM-1 as an independent, free-standing, single-trial PVT. Consistent with previous reports, the choice of criterion measure influences parameter estimates of the PVT being calibrated. The methodological implications of modality specificity to PVT research and clinical/forensic practice should be considered when evaluating cutoffs or interpreting scores in the failing range.
Collapse
Affiliation(s)
- Jaspreet K Rai
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada.,University of Windsor, Edmonton, Alberta, Canada
| | - Laszlo A Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Ontario, Canada
| |
Collapse
|
23
|
Erdodi LA, Taylor B, Sabelli AG, Malleck M, Kirsch NL, Abeare CA. Demographically Adjusted Validity Cutoffs on the Finger Tapping Test Are Superior to Raw Score Cutoffs in Adults with TBI. PSYCHOLOGICAL INJURY & LAW 2019. [DOI: 10.1007/s12207-019-09352-y] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
|
24
|
Abeare C, Sabelli A, Taylor B, Holcomb M, Dumitrescu C, Kirsch N, Erdodi L. The Importance of Demographically Adjusted Cutoffs: Age and Education Bias in Raw Score Cutoffs Within the Trail Making Test. PSYCHOLOGICAL INJURY & LAW 2019. [DOI: 10.1007/s12207-019-09353-x] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
|
25
|
Gasquoine PG, Weimer AA, Orta M. Specificity rates for non-clinical, bilingual, Mexican Americans with California Verbal Learning Test3, verbal fluency, and Rey 15-IR performance validity measures. Clin Neuropsychol 2018; 33:1445-1454. [DOI: 10.1080/13854046.2018.1530374] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
- Philip Gerard Gasquoine
- Department of Psychological Science, University of Texas Rio Grande Valley, Edinburg, TX, USA
| | - Amy A. Weimer
- School of Family & Consumer Sciences, Texas State University, San Marcos, TX, USA
| | - Marlene Orta
- Department of Psychological Science, University of Texas Rio Grande Valley, Edinburg, TX, USA
| |
Collapse
|
26
|
Aita SL, Sofko CA, Hill BD, Musso MW, Boettcher AC. Utility of the Personality Assessment Inventory in detecting feigned Attention-Deficit/Hyperactivity Disorder (ADHD): The Feigned Adult ADHD index. Arch Clin Neuropsychol 2018; 33:832-844. [PMID: 29186287 DOI: 10.1093/arclin/acx113] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2017] [Accepted: 10/31/2017] [Indexed: 11/12/2022] Open
Abstract
Objective The high potential for secondary gain among college students presenting for Attention-Deficit/Hyperactivity Disorder (ADHD) evaluations highlights the need for psychometrically sound embedded validity indicators. The purpose of this study was to develop new validity indicators specific to feigned ADHD for the Personality Assessment Inventory (PAI) and compare them to preexisting imbedded PAI validity measures. Method PAI scales that were theoretically related to feigned ADHD were evaluated. A binomial (ADHD simulators, n = 138, and genuine ADHD, n = 142) logistic regression was conducted with selected PAI scales and subscales. Classification rates were compared between the new and existing validity scales. A similar approach was used for item-level data in a second study in a subgroup of the original sample. Results The derived PAI scale-based logistic regression had a sensitivity of 54% and specificity of 92%. This algorithm accurately identified 97% of healthy controls as not feigning ADHD and correctly identified 98% of a no diagnosis group and 75% of a mood/anxiety disorders group. Classification accuracy of the new index was superior to the majority of existing PAI validity scales across groups. An item-level PAI algorithm had a sensitivity of 85% and specificity of 97% for identifying feigned ADHD. Conclusions New validity measures were compared to existing PAI validity indicators and performed better than many of them in this study. The algorithms developed in this study of ADHD simulators and genuine ADHD cases have adequate sensitivity and good specificity and appear to function differently than other PAI symptom validity scales.
Collapse
Affiliation(s)
- Stephen L Aita
- Psychology Department, University of South Alabama, Mobile, AL, USA
| | - Channing A Sofko
- Department of Psychiatry and Human Behavior, Warren Alpert Medical School of Brown University, Rhode Island Hospital, Providence, RI, USA
| | - Benjamin D Hill
- Psychology Department, University of South Alabama, Mobile, AL, USA
| | - Mandi W Musso
- Department of Graduate Medical Education, Our Lady of the Lake Medical Center, Baton Rouge, LA, USA
| | - Anneliese C Boettcher
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| |
Collapse
|
27
|
Detecting Invalid Performance in Youth with Traumatic Brain Injury Using the Child and Adolescent Memory Profile (ChAMP) Lists Subtest. JOURNAL OF PEDIATRIC NEUROPSYCHOLOGY 2018. [DOI: 10.1007/s40817-018-0056-5] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
28
|
Abstract
OBJECTIVES Obstructive sleep apnea (OSA) is associated with cognitive impairment but the relationships between specific biomarkers and neurocognitive domains remain unclear. The present study examined the influence of common health comorbidities on these relationships. Adults with suspected OSA (N=60; 53% male; M age=52 years; SD=14) underwent neuropsychological evaluation before baseline polysomnography (PSG). Apneic syndrome severity, hypoxic strain, and sleep architecture disturbance were assessed through PSG. METHODS Depression (Center for Epidemiological Studies Depression Scale, CESD), pain, and medical comorbidity (Charlson Comorbidity Index) were measured via questionnaires. Processing speed, attention, vigilance, memory, executive functioning, and motor dexterity were evaluated with cognitive testing. A winnowing approach identified 9 potential moderation models comprised of a correlated PSG variable, comorbid health factor, and cognitive performance. RESULTS Regression analyses identified one significant moderation model: average blood oxygen saturation (AVO2) and depression predicting recall memory, accounting for 31% of the performance variance, p<.001. Depression was a significant predictor of recall memory, p<.001, but AVO2 was not a significant predictor. The interaction between depression and AVO2 was significant, accounting for an additional 10% of the variance, p<.001. The relationship between low AVO2 and low recall memory performance emerged when depression severity ratings approached a previously established clinical cutoff score (CESD=16). CONCLUSIONS This study examined sleep biomarkers with specific neurocognitive functions among individuals with suspected OSA. Findings revealed that depression burden uniquely influence this pathophysiological relationship, which may aid clinical management. (JINS, 2018, 28, 864-875).
Collapse
|
29
|
Abstract
OBJECTIVES The aim of this study was to investigate the relationship of psychological variables to cognitive performance validity test (PVT) results in mixed forensic and nonforensic clinical samples. METHODS Participants included 183 adults who underwent comprehensive neuropsychological examination. Criterion groups were formed, that is, Credible Group or Noncredible Group, based upon their performance on the Word Memory Test and other stand-alone and embedded PVT measures. RESULTS Multivariate logistic regression analysis identified three significant predictors of cognitive performance validity. These included two psychological constructs, for example, Cogniphobia (perception that cognitive effort will exacerbate neurological symptoms), and Symptom Identity (perception that current symptoms are the result of illness or injury), and one contextual factor (forensic). While there was no interaction between these factors, elevated scores were most often observed in the forensic sample, suggesting that these independently contributing intrinsic psychological factors are more likely to occur in a forensic environment. CONCLUSIONS Illness perceptions were significant predictors of cognitive performance validity particularly when they reached very elevated levels. Extreme elevations were more common among participants in the forensic sample, and potential reasons for this pattern are explored. (JINS, 2018, 24, 735-745).
Collapse
|
30
|
An KY, Charles J, Ali S, Enache A, Dhuga J, Erdodi LA. Reexamining performance validity cutoffs within the Complex Ideational Material and the Boston Naming Test–Short Form using an experimental malingering paradigm. J Clin Exp Neuropsychol 2018; 41:15-25. [DOI: 10.1080/13803395.2018.1483488] [Citation(s) in RCA: 35] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Affiliation(s)
- Kelly Y. An
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| | - Jordan Charles
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| | - Sami Ali
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| | - Anca Enache
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| | - Jasmine Dhuga
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| | - Laszlo A. Erdodi
- Department of Psychology, University of Windsor, Windsor, ON, Canada
| |
Collapse
|
31
|
Whiteside DM, Caraher K, Hahn-Ketter A, Gaasedelen O, Basso MR. Classification accuracy of individual and combined executive functioning embedded performance validity measures in mild traumatic brain injury. APPLIED NEUROPSYCHOLOGY-ADULT 2018. [DOI: 10.1080/23279095.2018.1443935] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
Affiliation(s)
| | - Kristen Caraher
- Department of Psychiatry, University of Iowa, Iowa City, Iowa, USA
| | - Amanda Hahn-Ketter
- Department of Rehabilitation Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Owen Gaasedelen
- Department of Psychiatry, University of Iowa, Iowa City, Iowa, USA
| | | |
Collapse
|
32
|
White Matter Associations With Performance Validity Testing in Veterans With Mild Traumatic Brain Injury: The Utility of Biomarkers in Complicated Assessment. J Head Trauma Rehabil 2018; 31:346-59. [PMID: 26360002 DOI: 10.1097/htr.0000000000000183] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
OBJECTIVE Failure on performance validity tests (PVTs) is common in Veterans with histories of mild traumatic brain injury (mTBI), leading to questionable validity of clinical presentations. PARTICIPANTS Using diffusion tensor imaging, we investigated white matter (WM) integrity and cognition in 79 Veterans with history of mTBI who passed PVTs (n = 43; traumatic brain injury [TBI]-passed), history of mTBI who failed at least 1 PVT (n = 13; TBI-failed), and military controls (n = 23; MCs) with no history of TBI. RESULTS The TBI-failed group demonstrated significantly lower cognitive scores relative to MCs and the TBI-passed group; however, no such differences were observed between MCs and the TBI-passed group. On a global measure of WM integrity (ie, WM burden), the TBI-failed group showed more overall WM abnormalities than the other groups. However, no differences were observed between the MCs and TBI-passed group on WM burden. Interestingly, regional WM analyses revealed abnormalities in the anterior internal capsule and cingulum of both TBI subgroups relative to MCs. Moreover, compared with the TBI-passed group, the TBI-failed group demonstrated significantly decreased WM integrity in the corpus callosum. CONCLUSIONS Findings revealed that, within our sample, WM abnormalities are evident in those who fail PVTs. This study adds to the burgeoning PVT literature by suggesting that poor PVT performance does not negate the possibility of underlying WM abnormalities in military personnel with history of mTBI.
Collapse
|
33
|
Kanser RJ, Rapport LJ, Bashem JR, Hanks RA. Detecting malingering in traumatic brain injury: Combining response time with performance validity test accuracy. Clin Neuropsychol 2018; 33:90-107. [DOI: 10.1080/13854046.2018.1440006] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Affiliation(s)
- Robert J. Kanser
- Department of Psychology, Wayne State University, Detroit, MI, USA
| | - Lisa J. Rapport
- Department of Psychology, Wayne State University, Detroit, MI, USA
| | - Jesse R. Bashem
- Department of Psychology, Wayne State University, Detroit, MI, USA
| | - Robin A. Hanks
- Department of Physical Medicine and Rehabilitation, Wayne State University, Detroit, MI, USA
| |
Collapse
|
34
|
Persinger VC, Whiteside DM, Bobova L, Saigal SD, Vannucci MJ, Basso MR. Using the California Verbal Learning Test, Second Edition as an embedded performance validity measure among individuals with TBI and individuals with psychiatric disorders. Clin Neuropsychol 2017; 32:1039-1053. [DOI: 10.1080/13854046.2017.1419507] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Affiliation(s)
- Virginia C. Persinger
- Department of Neuropsychology, Methodist Rehabilitation Center, Jackson, MS, USA
- Department of Clinical Psychology, Adler University, Chicago, IL, USA
| | | | - Lyuba Bobova
- Department of Clinical Psychology, Adler University, Chicago, IL, USA
| | - Seema D. Saigal
- Department of Clinical Psychology, Adler University, Chicago, IL, USA
| | - Marla J. Vannucci
- Department of Clinical Psychology, Adler University, Chicago, IL, USA
| | | |
Collapse
|
35
|
Erdodi LA, Abeare CA, Medoff B, Seke KR, Sagar S, Kirsch NL. A Single Error Is One Too Many: The Forced Choice Recognition Trial of the CVLT-II as a Measure of Performance Validity in Adults with TBI. Arch Clin Neuropsychol 2017; 33:845-860. [DOI: 10.1093/acn/acx110] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/20/2017] [Indexed: 11/12/2022] Open
Affiliation(s)
- Laszlo A Erdodi
- Department of Psychology, University of Windsor, 168 Chrysler Hall South, Windsor, Canada ON
| | - Christopher A Abeare
- Department of Psychology, University of Windsor, 170 Chrysler Hall South, Windsor, Canada ON
| | - Brent Medoff
- The Commonwealth Medical College, 525 Pine St, Scranton, PA, USA
| | - Kristian R Seke
- University of Windsor, Brain-Cognition-Neuroscience Program, G105 Chrysler Hall North, Windsor, Canada ON
| | - Sanya Sagar
- Department of Psychology, University of Windsor, 109 Chrysler Hall North, Windsor, Canada ON
| | - Ned L Kirsch
- Department of Physical Medicine and Rehabilitation, University of Michigan, Briarwood Circle #4 Ann Arbor, MI, USA
| |
Collapse
|
36
|
Erdodi LA. Aggregating validity indicators: The salience of domain specificity and the indeterminate range in multivariate models of performance validity assessment. APPLIED NEUROPSYCHOLOGY-ADULT 2017; 26:155-172. [PMID: 29111772 DOI: 10.1080/23279095.2017.1384925] [Citation(s) in RCA: 42] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
This study was designed to examine the "domain specificity" hypothesis in performance validity tests (PVTs) and the epistemological status of an "indeterminate range" when evaluating the credibility of a neuropsychological profile using a multivariate model of performance validity assessment. While previous research suggests that aggregating PVTs produces superior classification accuracy compared to individual instruments, the effect of the congruence between the criterion and predictor variable on signal detection and the issue of classifying borderline cases remain understudied. Data from a mixed clinical sample of 234 adults referred for cognitive evaluation (MAge = 46.6; MEducation = 13.5) were collected. Two validity composites were created: one based on five verbal PVTs (EI-5VER) and one based on five nonverbal PVTs (EI-5NV) and compared against several other PVTs. Overall, language-based tests of cognitive ability were more sensitive to elevations on the EI-5VER compared to visual-perceptual tests; whereas, the opposite was observed with the EI-5NV. However, the match between predictor and criterion variable had a more complex relationship with classification accuracy, suggesting the confluence of multiple factors (sensory modality, cognitive domain, testing paradigm). An "indeterminate range" of performance validity emerged that was distinctly different from both the Pass and the Fail group. Trichotomized criterion PVTs (Pass-Borderline-Fail) had a negative linear relationship with performance on tests of cognitive ability, providing further support for an "in-between" category separating the unequivocal Pass and unequivocal Fail classification range. The choice of criterion variable can influence classification accuracy in PVT research. Establishing a Borderline range between Pass and Fail more accurately reflected the distribution of scores on multiple PVTs. The traditional binary classification system imposes an artificial dichotomy on PVTs that was not fully supported by the data. Accepting "indeterminate" as a legitimate third outcome of performance validity assessment has the potential to improve the clinical utility of PVTs and defuse debates regarding "near-Passes" and "soft Fails."
Collapse
Affiliation(s)
- Laszlo A Erdodi
- a Department of Psychology , University of Windsor , Windsor , Canada
| |
Collapse
|
37
|
Psychometric Markers of Genuine and Feigned Neurodevelopmental Disorders in the Context of Applying for Academic Accommodations. PSYCHOLOGICAL INJURY & LAW 2017. [DOI: 10.1007/s12207-017-9287-5] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
|
38
|
Erdodi LA, Lichtenstein JD. Invalid before impaired: an emerging paradox of embedded validity indicators. Clin Neuropsychol 2017; 31:1029-1046. [DOI: 10.1080/13854046.2017.1323119] [Citation(s) in RCA: 52] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Laszlo A. Erdodi
- Department of Psychology, Neuropsychology Track, University of Windsor, Windsor, Canada
| | - Jonathan D. Lichtenstein
- Department of Psychiatry, Neuropsychology Services, Geisel School of Medicine at Dartmouth, Lebanon, NH, USA
| |
Collapse
|
39
|
Erdodi LA, Tyson BT, Abeare CA, Zuccato BG, Rai JK, Seke KR, Sagar S, Roth RM. Utility of critical items within the Recognition Memory Test and Word Choice Test. APPLIED NEUROPSYCHOLOGY-ADULT 2017; 25:327-339. [DOI: 10.1080/23279095.2017.1298600] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Laszlo A. Erdodi
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
- Department of Psychiatry, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire, USA
| | - Bradley T. Tyson
- Department of Psychiatry, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire, USA
- Western Washington Medical Group, Everett, Washington, USA
| | | | - Brandon G. Zuccato
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| | - Jaspreet K. Rai
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| | - Kristian R. Seke
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| | - Sanya Sagar
- Department of Psychology, University of Windsor, Windsor, Ontario, Canada
| | - Robert M. Roth
- Department of Psychiatry, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire, USA
| |
Collapse
|
40
|
Martin PK, Hunter BP, Rach AM, Heinrichs RJ, Schroeder RW. Excessive decline from premorbid functioning: detecting performance invalidity with the WAIS-IV and demographic predictions. Clin Neuropsychol 2017; 31:829-843. [DOI: 10.1080/13854046.2017.1284265] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Phillip K. Martin
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| | - Ben P. Hunter
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| | - Amanda M. Rach
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| | - Robin J. Heinrichs
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| | - Ryan W. Schroeder
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| |
Collapse
|
41
|
Sawyer RJ, Testa SM, Dux M. Embedded performance validity tests within the Hopkins Verbal Learning Test – Revised and the Brief Visuospatial Memory Test – Revised. Clin Neuropsychol 2016; 31:207-218. [DOI: 10.1080/13854046.2016.1245787] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- R. John Sawyer
- Baltimore Veterans Affairs Medical Center, Baltimore, MD, USA
| | - S. Marc Testa
- Berman Brain and Spine Institute, Sinai Hospital, Baltimore, MD, USA
| | - Moira Dux
- Baltimore Veterans Affairs Medical Center, Baltimore, MD, USA
| |
Collapse
|
42
|
Erdodi LA, Tyson BT, Shahein AG, Lichtenstein JD, Abeare CA, Pelletier CL, Zuccato BG, Kucharski B, Roth RM. The power of timing: Adding a time-to-completion cutoff to the Word Choice Test and Recognition Memory Test improves classification accuracy. J Clin Exp Neuropsychol 2016; 39:369-383. [DOI: 10.1080/13803395.2016.1230181] [Citation(s) in RCA: 38] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
43
|
CVLT-II Forced Choice Recognition Trial as an Embedded Validity Indicator: A Systematic Review of the Evidence. J Int Neuropsychol Soc 2016; 22:851-8. [PMID: 27619108 DOI: 10.1017/s1355617716000746] [Citation(s) in RCA: 77] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
OBJECTIVES The Forced Choice Recognition (FCR) trial of the California Verbal Learning Test, 2nd edition, was designed as an embedded performance validity test (PVT). To our knowledge, this is the first systematic review of classification accuracy against reference PVTs. METHODS Results from peer-reviewed studies with FCR data published since 2002 encompassing a variety of clinical, research, and forensic samples were summarized, including 37 studies with FCR failure rates (N=7575) and 17 with concordance rates with established PVTs (N=4432). RESULTS All healthy controls scored >14 on FCR. On average, 16.9% of the entire sample scored ≤14, while 25.9% failed reference PVTs. Presence or absence of external incentives to appear impaired (as identified by researchers) resulted in different failure rates (13.6% vs. 3.5%), as did failing or passing reference PVTs (49.0% vs. 6.4%). FCR ≤14 produced an overall classification accuracy of 72%, demonstrating higher specificity (.93) than sensitivity (.50) to invalid performance. Failure rates increased with the severity of cognitive impairment. CONCLUSIONS In the absence of serious neurocognitive disorder, FCR ≤14 is highly specific, but only moderately sensitive to invalid responding. Passing FCR does not rule out a non-credible presentation, but failing FCR rules it in with high accuracy. The heterogeneity in sample characteristics and reference PVTs, as well as the quality of the criterion measure across studies, is a major limitation of this review and the basic methodology of PVT research in general. (JINS, 2016, 22, 851-858).
Collapse
|
44
|
Martin PK, Schroeder RW, Wyman-Chick KA, Hunter BP, Heinrichs RJ, Baade LE. Rates of Abnormally Low TOPF Word Reading Scores in Individuals Failing Versus Passing Performance Validity Testing. Assessment 2016; 25:640-652. [DOI: 10.1177/1073191116656796] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
The present study examined the impact of performance validity test (PVT) failure on the Test of Premorbid Functioning (TOPF) in a sample of 252 neuropsychological patients. Word reading performance differed significantly according to PVT failure status, and number of PVTs failed accounted for 7.4% of the variance in word reading performance, even after controlling for education. Furthermore, individuals failing ≥2 PVTs were twice as likely as individuals passing all PVTs (33% vs. 16%) to have abnormally low obtained word reading scores relative to demographically predicted scores when using a normative base rate of 10% to define abnormality. When compared with standardization study clinical groups, those failing ≥2 PVTs were twice as likely as patients with moderate to severe traumatic brain injury and as likely as patients with Alzheimer’s dementia to obtain abnormally low TOPF word reading scores. Findings indicate that TOPF word reading based estimates of premorbid functioning should not be interpreted in individuals invalidating cognitive testing.
Collapse
Affiliation(s)
| | | | - Kathryn A. Wyman-Chick
- University of Kansas School of Medicine, Wichita, KS, USA
- University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Ben P. Hunter
- University of Kansas School of Medicine, Wichita, KS, USA
| | | | - Lyle E. Baade
- University of Kansas School of Medicine, Wichita, KS, USA
| |
Collapse
|
45
|
Hanks RA, Jackson AM, Crisanti LK. Predictive validity of a brief outpatient neuropsychological battery in individuals 1–25 years post traumatic brain injury. Clin Neuropsychol 2016; 30:1074-86. [DOI: 10.1080/13854046.2016.1194479] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Robin A. Hanks
- Department of Physical Medicine and Rehabilitation, Wayne State University School of Medicine, Detroit, MI, USA
- Department of Rehabilitation Psychology and Neuropsychology, Rehabilitation Institute of Michigan, Detroit, MI, USA
| | | | - Lauren K. Crisanti
- Department of Rehabilitation Psychology and Neuropsychology, Rehabilitation Institute of Michigan, Detroit, MI, USA
| |
Collapse
|
46
|
The BDAE Complex Ideational Material—a Measure of Receptive Language or Performance Validity? PSYCHOLOGICAL INJURY & LAW 2016. [DOI: 10.1007/s12207-016-9254-6] [Citation(s) in RCA: 36] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
47
|
Schroeder RW, Martin PK, Odland AP. Expert beliefs and practices regarding neuropsychological validity testing. Clin Neuropsychol 2016; 30:515-35. [DOI: 10.1080/13854046.2016.1177118] [Citation(s) in RCA: 50] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Ryan W. Schroeder
- Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| | - Phillip K. Martin
- Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| | | |
Collapse
|
48
|
Erdodi L, Roth R. Low scores on BDAE Complex Ideational Material are associated with invalid performance in adults without aphasia. APPLIED NEUROPSYCHOLOGY-ADULT 2016; 24:264-274. [DOI: 10.1080/23279095.2016.1154856] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Laszlo Erdodi
- Department of Psychology, University of Windsor, Ontario, Canada
| | - Robert Roth
- Department of Psychiatry, Geisel School of Medicine at Dartmouth, Lebanon, New Hampshire, USA
| |
Collapse
|
49
|
Loring DW, Goldstein FC, Chen C, Drane DL, Lah JJ, Zhao L, Larrabee GJ. False-Positive Error Rates for Reliable Digit Span and Auditory Verbal Learning Test Performance Validity Measures in Amnestic Mild Cognitive Impairment and Early Alzheimer Disease. Arch Clin Neuropsychol 2016; 31:313-31. [PMID: 27084732 DOI: 10.1093/arclin/acw014] [Citation(s) in RCA: 47] [Impact Index Per Article: 5.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/18/2016] [Indexed: 11/14/2022] Open
Abstract
OBJECTIVE The objective is to examine failure on three embedded performance validity tests [Reliable Digit Span (RDS), Auditory Verbal Learning Test (AVLT) logistic regression, and AVLT recognition memory] in early Alzheimer disease (AD; n = 178), amnestic mild cognitive impairment (MCI; n = 365), and cognitively intact age-matched controls (n = 206). METHOD Neuropsychological tests scores were obtained from subjects participating in the Alzheimer's Disease Neuroimaging Initiative (ADNI). RESULTS RDS failure using a ≤7 RDS threshold was 60/178 (34%) for early AD, 52/365 (14%) for MCI, and 17/206 (8%) for controls. A ≤6 RDS criterion reduced this rate to 24/178 (13%) for early AD, 15/365 (4%) for MCI, and 7/206 (3%) for controls. AVLT logistic regression probability of ≥.76 yielded unacceptably high false-positive rates in both clinical groups [early AD = 149/178 (79%); MCI = 159/365 (44%)] but not cognitively intact controls (13/206, 6%). AVLT recognition criterion of ≤9/15 classified 125/178 (70%) of early AD, 155/365 (42%) of MCI, and 18/206 (9%) of control scores as invalid, which decreased to 66/178 (37%) for early AD, 46/365 (13%) for MCI, and 10/206 (5%) for controls when applying a ≤5/15 criterion. Despite high false-positive rates across individual measures and thresholds, combining RDS ≤ 6 and AVLT recognition ≤9/15 classified only 9/178 (5%) of early AD and 4/365 (1%) of MCI patients as invalid performers. CONCLUSIONS Embedded validity cutoffs derived from mixed clinical groups produce unacceptably high false-positive rates in MCI and early AD. Combining embedded PVT indicators lowers the false-positive rate.
Collapse
Affiliation(s)
- David W Loring
- Department of Neurology, Emory University School of Medicine, Atlanta, GA, USA Department of Pediatrics, Emory University School of Medicine, Atlanta, GA, USA
| | - Felicia C Goldstein
- Department of Neurology, Emory University School of Medicine, Atlanta, GA, USA
| | - Chuqing Chen
- Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, USA
| | - Daniel L Drane
- Department of Neurology, Emory University School of Medicine, Atlanta, GA, USA Department of Pediatrics, Emory University School of Medicine, Atlanta, GA, USA
| | - James J Lah
- Department of Neurology, Emory University School of Medicine, Atlanta, GA, USA
| | - Liping Zhao
- Department of Biostatistics, Rollins School of Public Health, Emory University, Atlanta, GA, USA
| | | | | |
Collapse
|
50
|
Reslan S, Axelrod BN. Evaluating the Medical Symptom Validity Test (MSVT) in a Sample of Veterans Between the Ages of 18 to 64. APPLIED NEUROPSYCHOLOGY-ADULT 2016; 24:132-139. [DOI: 10.1080/23279095.2015.1107565] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Affiliation(s)
- Summar Reslan
- Detroit Medical Center, Rehabilitation Institute of Michigan, Detroit, Michigan, USA
| | - Bradley N. Axelrod
- John D. Dingell Department of Veterans Affairs Medical Center, Detroit, Michigan, USA
| |
Collapse
|