1
|
Paulino M, Moniz M, Moura O, Rijo D, Morey L, Simões MR. Psychometric properties of the Portuguese version of the Personality Assessment Inventory: normative data and reliability. Front Psychol 2024; 15:1359793. [PMID: 38873528 PMCID: PMC11169847 DOI: 10.3389/fpsyg.2024.1359793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Accepted: 04/25/2024] [Indexed: 06/15/2024] Open
Abstract
Introduction Originally published in the United States of America in 1991, the Personality Assessment Inventory (PAI) has been translated and adapted to a growing number of countries, but Portugal had yet to study its adequacy to the Portuguese population. Methods The current study aimed to investigate the Portuguese normative data, the predictive effect of sociodemographic variables on the PAI scores, and the reliability of the Portuguese version of the PAI. Additionally, results were compared with other international versions of the PAI. The sample was comprised of 900 participants (age: M = 43.13, SD = 14.28, range = 18-75), recruited from various regions of Portugal. Results Findings showed that the Portuguese sample scored higher than the U.S. and other international versions of the PAI in most scales. Sociodemographic variables (e.g., gender, age, and educational level) were significant predictors on PAI scores. The internal consistency of the Portuguese sample revealed lower values on the validity scales, but adequate on the clinical, treatment, and interpersonal scales. Overall, the Portuguese PAI revealed adequate psychometric properties, with normative results often superior to other international versions of the inventory. Discussion It is a crucial step into the Portuguese adaptation and validation of this instrument, a measure with considerable potential in clinical, forensic, and research contexts. This adaptation may lead to the growth and development of the psychological assessment field in Portugal, and the opportunity to develop future cross-cultural studies with other international versions of the PAI.
Collapse
Affiliation(s)
- Mauro Paulino
- Center for Research in Neuropsychology and Cognitive and Behavioral Intervention (CINEICC), University of Coimbra, Coimbra, Portugal
- Psychological Assessment and Psychometrics Laboratory (PsyAssessmentLab), University of Coimbra, Coimbra, Portugal
- Universidade Europeia, Faculty of Health Sciences, Lisbon, Portugal
- Institute of Clinical and Forensic Psychology (MIND), Lisbon, Portugal
| | - Mariana Moniz
- Center for Research in Neuropsychology and Cognitive and Behavioral Intervention (CINEICC), University of Coimbra, Coimbra, Portugal
- Psychological Assessment and Psychometrics Laboratory (PsyAssessmentLab), University of Coimbra, Coimbra, Portugal
- Institute of Clinical and Forensic Psychology (MIND), Lisbon, Portugal
| | - Octávio Moura
- Center for Research in Neuropsychology and Cognitive and Behavioral Intervention (CINEICC), University of Coimbra, Coimbra, Portugal
- Psychological Assessment and Psychometrics Laboratory (PsyAssessmentLab), University of Coimbra, Coimbra, Portugal
| | - Daniel Rijo
- Center for Research in Neuropsychology and Cognitive and Behavioral Intervention (CINEICC), University of Coimbra, Coimbra, Portugal
- Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| | - Leslie Morey
- College of Arts & Sciences, Texas A&M University Central Texas, Killeen, TX, United States
| | - Mário R. Simões
- Center for Research in Neuropsychology and Cognitive and Behavioral Intervention (CINEICC), University of Coimbra, Coimbra, Portugal
- Psychological Assessment and Psychometrics Laboratory (PsyAssessmentLab), University of Coimbra, Coimbra, Portugal
- Faculty of Psychology and Educational Sciences, University of Coimbra, Coimbra, Portugal
| |
Collapse
|
2
|
Henry GK. Ability of the Wisconsin Card-Sorting Test-64 as an embedded measure to identify noncredible neurocognitive performance in mild traumatic brain injury litigants. APPLIED NEUROPSYCHOLOGY. ADULT 2024:1-7. [PMID: 38684109 DOI: 10.1080/23279095.2024.2348012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/02/2024]
Abstract
OBJECTIVE To investigate the ability of selective measures on the Wisconsin Card Sorting Test-64 (WCST-64) to predict noncredible neurocognitive dysfunction in a large sample of mild traumatic brain injury (mTBI) litigants. METHOD Participants included 114 adults who underwent a comprehensive neuropsychological examination. Criterion groups were formed based upon their performance on stand-alone measures of cognitive performance validity (PVT). RESULTS Participants failing PVTs performed worse across all WCST-64 dependent variables of interest compared to participants who passed PVTs. Receiver operating curve analysis revealed that only categories completed was a significant predictors of PVT status. Multivariate logistic regression did not add to classification accuracy. CONCLUSION Consideration of noncredible executive functioning may be warranted in mild traumatic brain injury (mTBI) litigants who complete ≤ 1 category on the WCST-64.
Collapse
|
3
|
Basso MR, Whiteside DM, Combs D. Introduction to the special issue on performance validity: what are we doing? What should we do? J Clin Exp Neuropsychol 2024; 46:1-5. [PMID: 38678395 DOI: 10.1080/13803395.2024.2347119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/30/2024]
|
4
|
Denning JH, Horner MD. The impact of race and other demographic factors on the false positive rates of five embedded Performance Validity Tests (PVTs) in a Veteran sample. J Clin Exp Neuropsychol 2024; 46:25-35. [PMID: 38353039 DOI: 10.1080/13803395.2024.2314737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Accepted: 01/11/2024] [Indexed: 05/12/2024]
Abstract
INTRODUCTION It is common to use normative adjustments based on race to maintain accuracy when interpreting cognitive test results during neuropsychological assessment. However, embedded performance validity tests (PVTs) do not adjust for these racial differences and may result in elevated rates of false positives in African American/Black (AA) samples compared to European American/White (EA) samples. METHODS Veterans without Major Neurocognitive Disorder completed an outpatient neuropsychological assessment and were deemed to be performing in a valid manner (e.g., passing both the Test of Memory Malingering Trial 1 (TOMM1) and the Medical Symptom Validity Test (MSVT), (n = 531, EA = 473, AA = 58). Five embedded PVTs were administered to all patients: WAIS-III/IV Processing Speed Index (PSI), Brief Visuospatial Memory Test-Revised: Discrimination Index (BVMT-R), TMT-A (secs), California Verbal Learning Test-II (CVLT-II) Forced Choice, and WAIS-III/IV Digit Span Scaled Score. Individual PVT false positive rates, as well as the rate of failing two or more embedded PVTs, were calculated. RESULTS Failure rates of two embedded PVTs (PSI, TMT-A), and the total number of PVTs failed, were higher in the AA sample. The PSI and TMT-A remained significantly impacted by race after accounting for age, education, sex, and presence of Mild Neurocognitive Disorder. There were PVT failure rates greater than 10% (and considered false positives) in both groups (AA: PSI, TMT-A, and BVMT-R, 12-24%; EA: BVMT-R, 17%). Failing 2 or more PVTs (AA = 9%, EA = 4%) was impacted by education and Mild Neurocognitive Disorder but not by race. CONCLUSIONS Individual (timed) PVTs showed higher false positive rates in the AA sample even after accounting for demographic factors and diagnosis of Mild Neurocognitive Disorder. Requiring failure on 2 or more embedded PVTs reduced false positive rates to acceptable levels across both groups (10% or less) and was not significantly influenced by race.
Collapse
Affiliation(s)
- John H Denning
- Mental Health Service, Ralph H. Johnson Veterans Affairs Health Care System, Charleston, SC, USA
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
| | - Michael David Horner
- Mental Health Service, Ralph H. Johnson Veterans Affairs Health Care System, Charleston, SC, USA
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
| |
Collapse
|
5
|
Henry GK. Detection of noncredible cognitive performance with Wechsler Memory Scale-IV measures in mild traumatic brain injury litigants. APPLIED NEUROPSYCHOLOGY. ADULT 2023:1-8. [PMID: 38039520 DOI: 10.1080/23279095.2023.2287139] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/03/2023]
Abstract
OBJECTIVE To investigate the operating characteristics of selective measures on the Wechsler Memory Scale-IV (WMS-IV) to predict noncredible neurocognitive dysfunction in a sample of mild traumatic brain injury (mTBI) litigants. METHOD Participants included 110 adults who underwent a comprehensive neuropsychological examination. Criterion groups were formed based upon their performance on stand-alone measures of cognitive performance validity testing (PVT). RESULTS Participants failing two stand-alone PVTs exhibited significantly lower scores across all WMS-IV dependent variables of interest compared to participants who passed both PVTs. Participants who failed one PVT were excluded. Bivariate logistic regression revealed that all six dependent variables were significant predictors of PVT status. The best prediction model consisted of three WMS-IV variables including Logical Memory Delayed Recall (LM2), Logical Memory Recognition (LMR), and Visual Reproduction Recognition (VRR). This model demonstrated an accuracy of 90.2%, 0.89 sensitivity, 0.92 specificity, and a Receiver Operating Curve (ROC) of 0.957. CONCLUSION The current empirically-derived cutscores and logit equation for the WMS-IV may be an additional consideration in analyzing database validity and noncredible performance in mTBI personal injury litigants ages 18-69.
Collapse
|
6
|
Leonhard C. Quo Vadis Forensic Neuropsychological Malingering Determinations? Reply to Drs. Bush, Faust, and Jewsbury. Neuropsychol Rev 2023; 33:653-657. [PMID: 37594691 DOI: 10.1007/s11065-023-09606-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Accepted: 05/24/2023] [Indexed: 08/19/2023]
Abstract
The thoughtful commentaries in this volume of Drs. Bush, Jewsbury, and Faust add to the impact of the two reviews in this volume of statistical and methodological issues in the forensic neuropsychological determination of malingering based on performance and symptom validity tests (PVTs and SVTs). In his commentary, Dr. Bush raises, among others, the important question of whether such malingering determinations can still be considered as meeting the legal Daubert standard which is the basis for neuropsychological expert testimony. Dr. Jewsbury focuses mostly on statistical issues and agrees with two key points of the statistical review: Positive likelihood chaining is not a mathematically tenable method to combine findings of multiple PVTs and SVTs, and the Simple Bayes method is not applicable to malingering determinations. Dr. Faust adds important narrative texture to the implications for forensic neuropsychological practice and points to a need for research into factors other than malingering that may explain PVT and SVT failures. These commentaries put into even sharper focus the serious questions raised in the reviews about the scientific basis of present practices in the forensic neuropsychological determination of malingering.
Collapse
Affiliation(s)
- Christoph Leonhard
- The Chicago School of Professional Psychology, Xavier University of Louisiana, New Orleans, USA.
| |
Collapse
|
7
|
Rubenzer SJ. Ruling out feigned crime-related amnesia? A response to Acklin (2022). BEHAVIORAL SCIENCES & THE LAW 2023; 41:207-223. [PMID: 37071040 DOI: 10.1002/bsl.2623] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Revised: 01/22/2023] [Accepted: 03/21/2023] [Indexed: 06/19/2023]
Abstract
In a recent paper, Acklin discussed a case of possible amnesia for a murder in terms of neurobiology, psychoanalysis, and personality assessment. Acklin accepted the defendant's claim of amnesia for the crime as genuine. The considerable literature that takes a skeptical view of crime-related amnesia was not cited, and the possibility of feigning or malingering was "ruled out" with a single sentence that does not withstand scrutiny. A review of the literature on feigned amnesia suggests that it may not be possible to rule out malingering even if the best available tools are used: There has been minimal investigation of most validity tests and estimates of base rates of feigned amnesia for a crime vary widely and make estimates of Negative Predictor Power highly unreliable. Although one cannot know from the information presented if Acklin's defendant legitimately experienced amnesia, feigning could not be ruled out using an interview and the test data cited by Acklin. I call for a moratorium on publication of further articles on crime specific amnesia that do not conscientiously examine other potential explanations and do not use current best practices for assessing negative response bias.
Collapse
|
8
|
Malik HB, Norman JB. Best Practices and Methodological Strategies for Addressing Generalizability in Neuropsychological Assessment. JOURNAL OF PEDIATRIC NEUROPSYCHOLOGY 2023; 9:47-63. [PMID: 37250805 PMCID: PMC10182845 DOI: 10.1007/s40817-023-00145-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Revised: 04/11/2023] [Accepted: 04/15/2023] [Indexed: 05/31/2023]
Abstract
Generalizability considerations are widely discussed and a core foundation for understanding when and why treatment effects will replicate across sample demographics. However, guidelines on assessing and reporting generalizability-related factors differ across fields and are inconsistently applied. This paper synthesizes obstacles and best practices to apply recent work on measurement and sample diversity. We present a brief history of how knowledge in psychology has been constructed, with implications for who has been historically prioritized in research. We then review how generalizability remains a contemporary threat to neuropsychological assessment and outline best practices for researchers and clinical neuropsychologists. In doing so, we provide concrete tools to evaluate whether a given assessment is generalizable across populations and assist researchers in effectively testing and reporting treatment differences across sample demographics.
Collapse
Affiliation(s)
- Hinza B. Malik
- Department of Psychology, University of North Carolina Wilmington, 601 South College Road, Wilmington, NC 28403-5612 USA
| | - Jasmine B. Norman
- Department of Psychology, University of North Carolina Wilmington, 601 South College Road, Wilmington, NC 28403-5612 USA
| |
Collapse
|
9
|
Davis JJ. Time is money: Examining the time cost and associated charges of common performance validity tests. Clin Neuropsychol 2023; 37:475-490. [PMID: 35414332 DOI: 10.1080/13854046.2022.2063190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Abstract
Objective: This study presents data on the time cost and associated charges for common performance validity tests (PVTs). It also applies an approach from cost effectiveness research to comparison of tests that incorporates cost and classification accuracy. Method: A recent test usage survey was used to identify PVTs in common use among adult neuropsychologists. Data on test administration and scoring time were aggregated. Charges per test were calculated. A cost effectiveness approach was applied to compare pairs of tests from three studies using data on test administration time and classification accuracy operationalized as improvement in posterior probability beyond base rate. Charges per unit increase in posterior probability over base rate were calculated for base rates of invalidity ranging from 10 to 40%. Results: Ten commonly used PVTs measures showed a wide range in test administration and scoring time from 1 to 3 minutes to over 40 minutes with associated charge estimates from $4 to $284. Cost effectiveness comparisons illustrated the nuance in test selection and benefit of considering cost in relation to outcome rather than prioritizing time (i.e. cost minimization) classification accuracy alone. Conclusions: Findings extend recent research efforts to fill knowledge gaps related to the cost of neuropsychological evaluation. The cost effectiveness approach warrants further study in other samples with different neuropsychological and outcome measures.
Collapse
Affiliation(s)
- Jeremy J Davis
- Department of Neurology, Glenn Biggs Institute for Alzheimer's and Neurogenerative Diseases, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
| |
Collapse
|
10
|
Henry GK. The Pain Disability Index: Effects of performance and symptom validity in mild traumatic brain injury litigants with persistent Post-Concussion pain complaints. Clin Neuropsychol 2023; 37:448-458. [PMID: 35109767 DOI: 10.1080/13854046.2022.2029576] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/28/2023]
Abstract
OBJECTIVE Objectives of the current study were to cross validate the Pain Disability Index (PDI) as a measure of symptom validity in a large sample of mild traumatic brain injury (MTBI) litigants with persistent post-concussive pain complaints, and investigate the effects of performance and symptom validity testing on PDI scores. METHODS Participants included 91 adults who underwent a comprehensive neuropsychological examination. Criterion groups were formed based upon their performance on stand-alone measures of cognitive performance validity (PVT), and the MMPI-2-RF Symptom Validity Scale (FBS-r) as a measure of symptom validity (SVT). RESULTS Participants who failed PVT and SVT scored significantly higher on the PDI compared to participants who passed both. Failing both was associated with a large effect size. Failing PVT, but passing SVT, was associated with a medium effect on PDI scores, while passing PVT, but failing SVT demonstrated a small effect. A PDI cutscore of 49 was associated with .90 specificity and .47 sensitivity. CONCLUSION The PDI demonstrates external validity as a self-report measure of symptom validity in MTBI litigants with persistent post-concussive pain complaints. A dose response relationship exists between PVT, SVT and PDI scores. Forensic examiners should include both PVT and SVT to optimize clinical decision making when evaluating MTBI litigants with complaints of pain-related disability years post-incident.
Collapse
|
11
|
Henry GK. The Continuous Visual Memory Test: Update and extension on the operating characteristics as an embedded measure of cognitive performance validity. Clin Neuropsychol 2023; 37:194-206. [PMID: 34890307 DOI: 10.1080/13854046.2021.2010807] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
Objective: To compare and update predictive models comprised of embedded measures from the Continuous Visual Memory Test (CVMT) in their ability to predict performance validity in personal injury litigants. Methods: Ninety-two personal injury litigants underwent a comprehensive neuropsychological examination. Criterion groups were formed, i.e. PVT-Pass and PVT-Fail, based upon their performance on stand-alone measures of performance validity (PVT). Independent-samples t-tests investigated group differences on dependent variables of interest while logistic regression analyses, as well as a decision tree classification procedure, were employed to identify the best predictive model. Results: The PVT-Fail group scored significantly lower on the 20-item Larrabee Index (LI), and three CVMT variables comprising the Henry-Enders Index (HEI) including Hits, Total Score, and Delayed Recall, but significantly higher on False Alarm Errors. Although the Total score was the best single predictor of PVT status, the addition of LI improved sensitivity. The best predictive model was derived via a classification and regression tree analysis which selected LI and CVMT-FA resulting in .91 specificity, .60 sensitivity, and ROC = 0.832. Conclusion: In the current study total CVMT scores < 70, and LI scores < 18 were rare for PI litigants with MTBI and not seen in litigants with moderate and severe brain injury who passed PVTs. Three predictive CVMT models were derived. When failure on one of the models is observed then concerns about the credibility of visual memory performance should be considered with particular attention to other stand-alone and embedded measures of performance validity.
Collapse
|
12
|
Weigard A, Spencer RJ. Benefits and challenges of using logistic regression to assess neuropsychological performance validity: Evidence from a simulation study. Clin Neuropsychol 2023; 37:34-59. [PMID: 35006042 PMCID: PMC9273108 DOI: 10.1080/13854046.2021.2023650] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2021] [Accepted: 12/22/2021] [Indexed: 02/07/2023]
Abstract
Logistic regression (LR) is recognized as a promising method for making decisions about neuropsychological performance validity by integrating information across multiple measures. However, this method has yet to be widely adopted in clinical practice, likely because several open questions remain about its utility relative to simpler methods, its effectiveness across different clinical contexts, and its feasibility at sample sizes common in the field. The current study addresses these questions by assessing classification performance of logistic regression and alternative methods across an array of simulated data sets. We simulated scores of valid and invalid performers on 6 tests designed to mimic the psychometric and distributional properties of real performance validity measures. Out-of-sample predictive performance of LR and a commonly used alternative ("vote counting") was assessed across different base rates, validity measure properties, and sample sizes. LR improved classification accuracy by 2%-12% across simulation conditions, primarily by improving sensitivity. False positives and negatives can be further reduced when LR predictions are interpreted as continuous, rather than binary. LR made robust predictions at sample sizes feasible for neuropsychology research (N = 307) and when as few as 2 tests with good psychometric properties were used. Although training and test data sets of at least several hundred individuals may be required to develop and evaluate LR models for use in clinical practice, LR promises to be an efficient and powerful tool for improving judgements about performance validity. We offer several recommendations for model development and LR interpretation in a clinical setting.
Collapse
Affiliation(s)
| | - Robert J. Spencer
- Department of Psychiatry, University of Michigan
- VA Ann Arbor Healthcare System
| |
Collapse
|
13
|
Robinson A, Huber M, Breaux E, Pugh E, Calamia M. Failing The b Test: The influence of cutoff scores and criterion group approaches in a sample of adults referred for psychoeducational evaluation. J Clin Exp Neuropsychol 2022; 44:619-626. [PMID: 36727266 DOI: 10.1080/13803395.2022.2153805] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
Abstract
OBJECTIVE Previous research has shown that both criterion grouping approaches and cutoff scores can impact PVT classification accuracy statistics. This study aimed to examine the influence of cutoff scores and criterion grouping approaches on The b Test, a measure designed to identify feigned impairment in visual scanning, processing speed, and letter identification. METHOD Two hundred ninety-seven adults referred for psychoeducational testing were included with the majority of individuals seeking academic accommodations (n = 215). Cutoff scores of ≥82, ≥90, and ≥120 were utilized along with two different criterion group approaches, 0 PVT failures vs. ≥2 PVT failures and 0 PVT failures versus ≥ 1 PVT failures. RESULTS Failure rates for The b Test in the overall sample ranged from 12.5% to 16.2%. Subgroup analyses in those referred specifically for ADHD revealed failure rates for The b Test ranging from 10.5% to 14.2%. ROC curves within the full sample and ADHD subsample demonstrated significant AUCs utilizing both criterion group approaches (AUC = .66 - .78). Sensitivity and specificity varied as a function of criterion group approach and cutoff score, with 0 PVT failures vs. ≥ 2 PVT failures resulting in the greatest sensitivity when maximizing specificity at ≥.90 in the full sample and ADHD sample. CONCLUSIONS The results demonstrate that criterion approaches and cutoff scores impact classification accuracy of The b Test with 0 PVT vs. ≥ 2 PVT failures demonstrating the greatest classification accuracy. Special considerations should be made with regard to clinical decision making in the context of psychoeducational evaluations given that a large portion of individuals seeking accommodations fail only one PVT. Limitations of this study are also discussed.
Collapse
Affiliation(s)
- Anthony Robinson
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA
| | - Marissa Huber
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA
| | - Eathan Breaux
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA
| | - Erika Pugh
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA
| | - Matthew Calamia
- Department of Psychology, Louisiana State University, Baton Rouge, LA, USA
| |
Collapse
|
14
|
Binder LM, Tadrous-Furnanz SK, Storzbach D, Larrabee GJ, Salinsky MC. The rate of psychiatric disorders in veterans undergoing intensive EEG monitoring is associated with symptom and performance invalidity. Clin Neuropsychol 2022; 36:2120-2134. [PMID: 34632958 DOI: 10.1080/13854046.2021.1974564] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
To determine if the number of participants with psychiatric disorders increased in association with failures on symptom validity tests (SVTs) and a performance validity test (PVT) in Veterans admitted for evaluation of possible seizures. The 254 participants were Veterans undergoing inpatient video-EEG monitoring for the diagnosis of possible seizures. DSM-IV psychiatric disorders were diagnosed with the SCID IV. Symptom exaggeration was assessed with the MMPI-2-RF and performance validity with the TOMM. On the MMPI-2-RF, 27.6%-32.7% showed symptom exaggeration. Participants who exaggerated on the MMPI-2-RF were more often diagnosed with psychiatric disorders. The TOMM was failed by 15.4% of the sample. Participants who failed the TOMM were more often diagnosed with an Axis I disorder but not with a personality disorder. The MMPI-2-RF was invalid in more cases than the TOMM, but 7.9% of the sample generated a valid MMPI-2-RF and an invalid TOMM. The correlational design does not allow conclusions about cause and effect. The invalid groups may have had a higher rate of psychopathology. The number of participants with psychiatric disorders increased in association with symptom exaggeration and performance invalidity. Symptom exaggeration was more frequent than performance invalidity, but the TOMM made a unique contribution to identification of invalidity. The routine clinical use of SVTs and PVTs is supported. The results also suggest the need for caution in diagnosing psychiatric disorders when there is symptom exaggeration or performance invalidity, because diagnostic validity is dependent on the accuracy of symptom reporting.
Collapse
Affiliation(s)
| | | | | | | | - Martin C Salinsky
- VA Healthcare System, Portland, Oregon, USA.,Oregon Health and Science University, Portland, Oregon, USA
| |
Collapse
|
15
|
Schroeder RW, Clark HA, Martin PK. Base rates of invalidity when patients undergoing routine clinical evaluations have social security disability as an external incentive. Clin Neuropsychol 2022; 36:1902-1914. [PMID: 33706657 DOI: 10.1080/13854046.2021.1895322] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
Objective: Social Security Disability is a common external incentive in neuropsychological evaluations. This study determined base rates of invalidity when patients referred for routine clinical evaluations have Social Security Disability as an external incentive. Method: Patients (n = 242) were grouped as validly or invalidly performing based on the use of multiple performance validity tests. Frequency analyses were then conducted. Results: As a whole, 46.0% of clinically referred patients with Social Security Disability as an external incentive produced invalid data. When divided by disability pursuit status, 58.6% of individuals already receiving Social Security Disability, 44.6% of individuals actively seeking Social Security Disability, and 39.3% of individuals considering seeking Social Security Disability produced invalid data. By comparison, only 8.5% of clinically referred patients without known external incentives produced invalid data. Conclusions: Beyond establishing base rates, these data indicate that the external incentive, not necessarily the evaluation setting, increases the rate of invalidity, as obtained base rates mirror those observed in independent medical examinations. In addition, this study highlights that even patients who report that they are considering but have not committed themselves to pursuing an external incentive frequently invalidate testing.
Collapse
Affiliation(s)
- Ryan W Schroeder
- Department of Psychiatry & Behavioral Sciences, University of Kansas School of Medicine - Wichita, Wichita, Kansas, USA
| | - Hilary A Clark
- Department of Psychiatry & Behavioral Sciences, University of Kansas School of Medicine - Wichita, Wichita, Kansas, USA
| | - Phillip K Martin
- Department of Psychiatry & Behavioral Sciences, University of Kansas School of Medicine - Wichita, Wichita, Kansas, USA
| |
Collapse
|
16
|
Henry GK. Response time measures on the Word Memory Test do not add incremental validity to accuracy scores in predicting noncredible neurocognitive dysfunction in mild traumatic brain injury litigants. APPLIED NEUROPSYCHOLOGY. ADULT 2022:1-7. [PMID: 36170848 DOI: 10.1080/23279095.2022.2126320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
The objective of the current study was to investigate whether response time measures on the Word Memory Test (WMT) increase predictive validity on determining noncredible neurocognitive dysfunction in a large sample of mild traumatic brain injury (MTBI) litigants. Participants included 203 adults who underwent a comprehensive neuropsychological examination. Criterion groups were formed based upon their performance on stand-alone measures of cognitive performance validity (PVT). Participants failing PVTs exhibited significantly slower response times and lower accuracy on the WMT compared to participants who passed PVTs. Response time measures did not add significant incremental validity beyond that afforded by WMT accuracy measures alone. The best predictor of PVT status was the WMT Consistency Score (CNS) which was associated with an extremely large effect size (d = 16.44), followed by Immediate Recognition (IR: d = 10.68) and Delayed Recognition (DR: d = 10.10).
Collapse
|
17
|
Ratcliffe LN, Hale AC, Gradwohl BD, Spencer RJ. Preliminary findings from reevaluating the MMPI Response Bias Scale items in veterans undergoing neuropsychological evaluation. APPLIED NEUROPSYCHOLOGY. ADULT 2022:1-8. [PMID: 35917583 DOI: 10.1080/23279095.2022.2106571] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
The Response Bias Scale (RBS) was developed to predict non-credible cognitive presentations among disability claimants without head injury. Developers used empirical keying, which is independent of apparent content, to select items from the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) item pool that distinguished between individuals passing or failing performance validity tests (PVTs). No study has examined which of these items would have psychometric value when used in clinical neuropsychological evaluations. This study reexamined items comprising RBS with reference to manifest item content, internal consistency, PVTs, and a symptom validity test (SVT) in a sample of 173 predominately White male veterans (MAGE = 50.70, MEDU = 13.73) in a VA outpatient neuropsychology clinic. Participants completed the MMPI-2 Restructured Form (MMPI-2-RF), PVTs, and an SVT. The 28-item RBS appears to contain three types of items: those that manifestly address cognitive functioning, those that are supported but do not appear to address cognitive functioning, and nine items that were unrelated to cognition and not statistically supported. The 19 empirically supported items, or RBS-19, predicted PVT and SVT failures marginally better than the RBS. Both the RBS and RBS-19 had stronger relationships with SVTs relative to PVTs. Although the removal of the nine problematic items improved the diagnostic accuracy of the scale, it still did not reach the level that is generally considered to be clinically optimal. The RBS-19 offers a measure with improved internal consistency and predictive validity compared to the RBS and warrants additional research.
Collapse
Affiliation(s)
- Lauren N Ratcliffe
- Mental Health Service, VA Ann Arbor Healthcare System, Ann Arbor, MI, USA
- Department of Psychiatry, Michigan Medicine, Ann Arbor, MI, USA
- Department of Clinical Psychology, Mercer University College of Health Professions, Atlanta, GA, USA
| | - Andrew C Hale
- Mental Health Service, VA Ann Arbor Healthcare System, Ann Arbor, MI, USA
- Department of Psychiatry, Michigan Medicine, Ann Arbor, MI, USA
| | - Brian D Gradwohl
- Mental Health Service, VA Ann Arbor Healthcare System, Ann Arbor, MI, USA
- Department of Psychiatry, Michigan Medicine, Ann Arbor, MI, USA
| | - Robert J Spencer
- Mental Health Service, VA Ann Arbor Healthcare System, Ann Arbor, MI, USA
- Department of Psychiatry, Michigan Medicine, Ann Arbor, MI, USA
| |
Collapse
|
18
|
Fox ME, King TZ. Considerations for Reliable Digit Span as a performance validity test for long-term survivors of childhood brain tumors. APPLIED NEUROPSYCHOLOGY. ADULT 2022; 29:469-477. [PMID: 32503366 DOI: 10.1080/23279095.2020.1771714] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
The Reliable Digit Span (RDS) is a performance validity test (PVT) used widely within non-clinical samples, but its utility is in question in clinical groups with cognitive impairment. To investigate, RDS scores were calculated and correlated with the Neurological Predictor Scale, an informant-reported Activities of Daily Living score, and a proxy measure of intelligence (Vocabulary) for 83 adult survivors of childhood brain tumors and 105 healthy controls. Analyses were covaried for age at examination. Participants were divided into passing and failing groups at each RDS cutoff, and ANCOVAs for each of the three variables of interest covaried for age at the examination were run. RDS was correlated with all three variables of interest in survivors but only Vocabulary in controls. At the ≤7 cutoff, passing and failing survivors demonstrated significant differences across all variables of interest, while passing and failing controls differed only on Vocabulary. Differences were also found between passing and failing survivors at lower cutoffs. RDS is related to and likely impacted by various neurological and cognitive challenges faced by brain tumor survivors. Using the standard RDS cutoff of ≤7 may result in inaccurate interpretation of valid performance in this population; therefore, the use of other PVTs is recommended.
Collapse
Affiliation(s)
| | - Tricia Z King
- Department of Psychology and the Neuroscience Institute, Georgia State University, Atlanta, GA, USA
| |
Collapse
|
19
|
Lovett BJ, Spenceley LM, Schaberg TM, Best H. Response Validity in Psychoeducational evaluations: Results from a National Survey of School Psychologists. PSYCHOLOGY IN THE SCHOOLS 2022. [DOI: 10.1002/pits.22696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Affiliation(s)
- Benjamin J. Lovett
- School Psychology Program, Teachers College Columbia University New York New York USA
| | - Laura M. Spenceley
- Counseling and Psychological Services State University of New York at Oswego Oswego New York USA
| | - Theresa M. Schaberg
- School Psychology Program, Teachers College Columbia University New York New York USA
| | - Haley Best
- Counseling and Psychological Services State University of New York at Oswego Oswego New York USA
| |
Collapse
|
20
|
Uiterwijk D, Wong D, Stargatt R, Crowe SF. Performance and symptom validity testing in neuropsychological assessments in Australia: a survey of practises and beliefs. AUSTRALIAN PSYCHOLOGIST 2021. [DOI: 10.1080/00050067.2021.1948797] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Affiliation(s)
- Daniel Uiterwijk
- School of Psychology and Public Health, La Trobe University, Victoria Australia
| | - Dana Wong
- School of Psychology and Public Health, La Trobe University, Victoria Australia
| | - Robyn Stargatt
- School of Psychology and Public Health, La Trobe University, Victoria Australia
| | - Simon F. Crowe
- School of Psychology and Public Health, La Trobe University, Victoria Australia
| |
Collapse
|
21
|
Rhoads T, Neale AC, Resch ZJ, Cohen CD, Keezer RD, Cerny BM, Jennette KJ, Ovsiew GP, Soble JR. Psychometric implications of failure on one performance validity test: a cross-validation study to inform criterion group definition. J Clin Exp Neuropsychol 2021; 43:437-448. [PMID: 34233580 DOI: 10.1080/13803395.2021.1945540] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
Introduction: Research to date has supported the use of multiple performance validity tests (PVTs) for determining validity status in clinical settings. However, the implications of including versus excluding patients failing one PVT remains a source of debate, and methodological guidelines for PVT research are lacking. This study evaluated three validity classification approaches (i.e. 0 vs. ≥2, 0-1 vs. ≥2, and 0 vs. ≥1 PVT failures) using three reference standards (i.e. criterion PVT groupings) to recommend approaches best suited to establishing validity groups in PVT research methodology.Method: A mixed clinical sample of 157 patients was administered freestanding (Medical Symptom Validity Test, Dot Counting Test, Test of Memory Malingering, Word Choice Test), and embedded PVTs (Reliable Digit Span, RAVLT Effort Score, Stroop Word Reading, BVMT-R Recognition Discrimination) during outpatient neuropsychological evaluation. Three reference standards (i.e. two freestanding and three embedded PVTs from the above list) were created. Rey 15-Item Test and RAVLT Forced Choice were used solely as outcome measures in addition to two freestanding PVTs not employed in the reference standard. Receiver operating characteristic curve analyses evaluated classification accuracy using the three validity classification approaches for each reference standard.Results: When patients failing only one PVT were excluded or classified as valid, classification accuracy ranged from acceptable to excellent. However, classification accuracy was poor to acceptable when patients failing one PVT were classified as invalid. Sensitivity/specificity across two of the validity classification approaches (0 vs. ≥2; 0-1 vs. ≥2) remained reasonably stable.Conclusions: These results reflect that both inclusion and exclusion of patients failing one PVT are acceptable approaches to PVT research methodology and the choice of method likely depends on the study rationale. However, including such patients in the invalid group yields unacceptably poor classification accuracy across a number of psychometrically robust outcome measures and therefore is not recommended.
Collapse
Affiliation(s)
- Tasha Rhoads
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Psychology, Rosalind Franklin University of Medicine and Science, North Chicago, IL, USA
| | - Alec C Neale
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Psychology, Rosalind Franklin University of Medicine and Science, North Chicago, IL, USA
| | - Zachary J Resch
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Psychology, Rosalind Franklin University of Medicine and Science, North Chicago, IL, USA
| | - Cari D Cohen
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Psychology, Rosalind Franklin University of Medicine and Science, North Chicago, IL, USA
| | - Richard D Keezer
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Psychology, Wheaton College, Wheaton, IL, USA
| | - Brian M Cerny
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Psychology, Illinois Institute of Technology, Chicago, IL, USA
| | - Kyle J Jennette
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Gabriel P Ovsiew
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Jason R Soble
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Neurology, University of Illinois College of Medicine, Chicago, IL, USA
| |
Collapse
|
22
|
Martinez KA, Sayers C, Hayes C, Martin PK, Clark CB, Schroeder RW. Normal cognitive test scores cannot be interpreted as accurate measures of ability in the context of failed performance validity testing: A symptom- and detection-coached simulation study. J Clin Exp Neuropsychol 2021; 43:301-309. [PMID: 33998369 DOI: 10.1080/13803395.2021.1926435] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
Introduction: While use of performance validity tests (PVTs) has become a standard of practice in neuropsychology, there are differing opinions regarding whether to interpret cognitive test data when standard scores fall within normal limits despite PVTs being failed. This study is the first to empirically determine whether normal cognitive test scores underrepresent functioning when PVTs are failed.Method: Participants, randomly assigned to either a simulated malingering group (n = 50) instructed to mildly suppress test performances or a best-effort/control group (n = 50), completed neuropsychological tests which included the North American Adult Reading Test (NAART), California Verbal Learning Test - 2nd Edition (CVLT-II), and Test of Memory Malingering (TOMM).Results: Groups were not significantly different in age, sex, education, or NAART predicted intellectual ability, but simulators performed significantly worse than controls on the TOMM, CVLT-II Forced Choice Recognition, and CVLT-II Short Delay Free Recall. The groups did not significantly differ on other examined CVLT-II measures. Of simulators who failed validity testing, 36% scored no worse than average and 73% scored no worse than low average on any of the examined CVLT-II indices.Conclusions: Of simulated malingerers who failed validity testing, nearly three-fourths were able to produce cognitive test scores that were within normal limits, which indicates that normal cognitive performances cannot be interpreted as accurately reflecting an individual's capabilities when obtained in the presence of validity test failure. At the same time, only 2 of 50 simulators were successful in passing validity testing while scoring within an impaired range on cognitive testing. This latter finding indicates that successfully feigning cognitive deficits is difficult when PVTs are utilized within the examination.
Collapse
Affiliation(s)
- Karen A Martinez
- Department of Psychology, Wichita State University, Wichita, KS, USA
| | - Courtney Sayers
- Department of Psychology, Wichita State University, Wichita, KS, USA
| | - Charles Hayes
- Department of Psychology, Wichita State University, Wichita, KS, USA
| | - Phillip K Martin
- Department of Psychiatry & Behavioral Sciences, University of Kansas School of Medicine - Wichita, Wichita, KS, USA
| | - C Brendan Clark
- Department of Psychology, Wichita State University, Wichita, KS, USA
| | - Ryan W Schroeder
- Department of Psychiatry & Behavioral Sciences, University of Kansas School of Medicine - Wichita, Wichita, KS, USA
| |
Collapse
|
23
|
Berger C, Lev A, Braw Y, Elbaum T, Wagner M, Rassovsky Y. Detection of Feigned ADHD Using the MOXO-d-CPT. J Atten Disord 2021; 25:1032-1047. [PMID: 31364437 DOI: 10.1177/1087054719864656] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
Abstract
Objective: The objective of this study was to assess the MOXO-d-CPT utility in detecting feigned ADHD and establish cutoffs with adequate specificity and sensitivity. Method: The study had two phases. First, using a prospective design, healthy adults who simulated ADHD were compared with healthy controls and ADHD patients who performed the tasks to the best of their ability (n = 47 per group). Participants performed the MOXO-d-CPT and an established performance validity test (PVT). Second, the MOXO-d-CPT classification accuracy, employed in Phase 1, was retrospectively compared with archival data of 47 ADHD patients and age-matched healthy controls. Results: Simulators performed significantly worse on all MOXO-d-CPT indices than healthy controls and ADHD patients. Three MOXO-d-CPT indices (attention, hyperactivity, impulsivity) and a scale combining these indices showed adequate discriminative capacity. Conclusion: The MOXO-d-CPT showed promise for the detection of feigned ADHD and, pending replication, can be employed for this aim in clinical practice and ADHD research.
Collapse
Affiliation(s)
| | - Astar Lev
- Bar-Ilan University, Ramat Gan, Israel
| | | | | | | | - Yuri Rassovsky
- Bar-Ilan University, Ramat Gan, Israel.,University of California, Los Angeles, USA
| |
Collapse
|
24
|
Donders J, Lefebre N, Goldsworthy R. Patterns of Performance and Symptom Validity Test Findings After Mild Traumatic Brain Injury. Arch Clin Neuropsychol 2021; 36:394-402. [PMID: 31732733 DOI: 10.1093/arclin/acz057] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2019] [Revised: 05/03/2019] [Accepted: 08/19/2019] [Indexed: 11/12/2022] Open
Abstract
OBJECTIVE The purpose of this study was to evaluate the presence of demographic, injury and neuropsychological correlates of distinct patterns of performance validity test and symptom validity test results in persons with mild traumatic brain injury (mTBI). METHOD One hundred and seventy-eight persons with mTBI completed the Test of Memory Malingering (TOMM; performance validity) and the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF; symptom validity) within 1-12 months postinjury. Four groups were compared: (a) pass both TOMM and MMPI-2-RF validity criteria, (b) pass TOMM and fail MMPI-2-RF, (c) fail TOMM and pass MMPI-2-RF, and (d) fail both TOMM and MMPI-2-RF. RESULTS Compared to Group a, participants in combined Groups b-d were more than twice as likely to be engaged in financial compensation-seeking and about four times less likely to have neuroimaging evidence of an intracranial lesion. The average performance of Group d on an independent test of verbal learning was more than 1.5 standard deviations below that of Group a. Participants in Group b were more likely to have intracranial lesions on neuroimaging than participants in Group c. CONCLUSION Performance and symptom validity tests provide complementary and non-redundant information in persons with mTBI. Whereas financial compensation-seeking is associated with increased risk of failure of either PVT or SVT, or both, the presence of intracranial findings on neuroimaging is associated with decreased risk of such.
Collapse
Affiliation(s)
- Jacobus Donders
- Department of Psychology, Mary Free Bed Rehabilitation Hospital, Grand Rapids, MI, USA
| | - Nathan Lefebre
- Department of Psychology, Calvin College, Grand Rapids, MI, USA
| | - Rachael Goldsworthy
- Department of Psychology, Mary Free Bed Rehabilitation Hospital, Grand Rapids, MI, USA
| |
Collapse
|
25
|
Sweet JJ, Heilbronner RL, Morgan JE, Larrabee GJ, Rohling ML, Boone KB, Kirkwood MW, Schroeder RW, Suhr JA. American Academy of Clinical Neuropsychology (AACN) 2021 consensus statement on validity assessment: Update of the 2009 AACN consensus conference statement on neuropsychological assessment of effort, response bias, and malingering. Clin Neuropsychol 2021; 35:1053-1106. [PMID: 33823750 DOI: 10.1080/13854046.2021.1896036] [Citation(s) in RCA: 154] [Impact Index Per Article: 51.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
Objective: Citation and download data pertaining to the 2009 AACN consensus statement on validity assessment indicated that the topic maintained high interest in subsequent years, during which key terminology evolved and relevant empirical research proliferated. With a general goal of providing current guidance to the clinical neuropsychology community regarding this important topic, the specific update goals were to: identify current key definitions of terms relevant to validity assessment; learn what experts believe should be reaffirmed from the original consensus paper, as well as new consensus points; and incorporate the latest recommendations regarding the use of validity testing, as well as current application of the term 'malingering.' Methods: In the spring of 2019, four of the original 2009 work group chairs and additional experts for each work group were impaneled. A total of 20 individuals shared ideas and writing drafts until reaching consensus on January 21, 2021. Results: Consensus was reached regarding affirmation of prior salient points that continue to garner clinical and scientific support, as well as creation of new points. The resulting consensus statement addresses definitions and differential diagnosis, performance and symptom validity assessment, and research design and statistical issues. Conclusions/Importance: In order to provide bases for diagnoses and interpretations, the current consensus is that all clinical and forensic evaluations must proactively address the degree to which results of neuropsychological and psychological testing are valid. There is a strong and continually-growing evidence-based literature on which practitioners can confidently base their judgments regarding the selection and interpretation of validity measures.
Collapse
Affiliation(s)
- Jerry J Sweet
- Department of Psychiatry & Behavioral Sciences, NorthShore University HealthSystem, Evanston, IL, USA
| | | | | | | | - Martin L Rohling
- Psychology Department, University of South Alabama, Mobile, AL, USA
| | - Kyle B Boone
- California School of Forensic Studies, Alliant International University, Los Angeles, CA, USA
| | - Michael W Kirkwood
- Department of Physical Medicine & Rehabilitation, University of Colorado School of Medicine and Children's Hospital Colorado, Aurora, CO, USA
| | - Ryan W Schroeder
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| | - Julie A Suhr
- Psychology Department, Ohio University, Athens, OH, USA
| | | |
Collapse
|
26
|
Victoria Symptom Validity Test: A Systematic Review and Cross-Validation Study. Neuropsychol Rev 2021; 31:331-348. [PMID: 33433828 DOI: 10.1007/s11065-021-09477-5] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Accepted: 01/03/2021] [Indexed: 12/12/2022]
Abstract
The Victoria Symptom Validity Test (VSVT) is a performance validity test (PVT) with over two decades of empirical backing, although methodological limitations within the extant literature restrict its clinical and research generalizability. Chief among these constraints includes limited consensus on the most accurate index within the VSVT and the most appropriate cut-scores within each VSVT validity index. The current systematic review synthesizes existing VSVT validation studies and provides additional cross-validation in an independent sample using a known-groups design. We completed a systematic search of the literature, identifying 17 peer-reviewed studies for synthesis (7 simulation designs, 7 differential prevalence designs, and 3 known-groups designs). The independent cross-validation sample consisted of 200 mixed clinical neuropsychiatric patients referred for outpatient neuropsychological evaluation. Across all indices, Total item accuracy produced the strongest psychometric properties at an optimal cut-score of ≤ 40 (62% sensitivity/88% specificity). However, ROC curve analyses for all VSVT indices yielded statistically significant areas under the curve (AUCs; .73-81), suggestive of moderate classification accuracy. Cut-scores derived using the independent cross-validation sample converged with some previous findings supporting cut-scores of ≤ 22 for Easy item accuracy and ≤ 40 for Total item accuracy, although divergent findings were noted for Difficult item accuracy. Overall, VSVT validity indicators have adequate diagnostic accuracy across populations, with the current study providing additional support for its use as a psychometrically sound PVT in clinical settings. However, caution is recommended among patients with certain verified clinical conditions (e.g., dementia) and those with pronounced working memory deficits due to concerns for increased risk of false positives.
Collapse
|
27
|
Sweet JJ, Klipfel KM, Nelson NW, Moberg PJ. Professional Practices, Beliefs, and Incomes of Postdoctoral Trainees: The AACN, NAN, SCN 2020 Practice and ‘Salary Survey’. Arch Clin Neuropsychol 2020; 36:1-16. [DOI: 10.1093/arclin/acaa116] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2020] [Revised: 10/07/2020] [Accepted: 11/01/2020] [Indexed: 11/12/2022] Open
Abstract
Abstract
Objective
Within a portion of the 2020 professional practice and “salary survey,” to update key information regarding neuropsychology postdoctoral trainees.
Methods
Postdoctoral trainees were contacted via a variety of membership listings, including the listserv used by the program directors of the Association of Postdoctoral Programs in Clinical Neuropsychology (APPCN). Invitations sent in multiple waves to members of numerous neuropsychological organizations via e-messages and physical postcards included the request that postdoctoral trainees participate. The survey website was opened on January 17, 2020 and closed on April 2, 2020, during which time a total of 178 postdoctoral trainees in the USA and 3 in Canada participated.
Results
Response rate was estimated to be 56.4%, which adequately represents the target sample. The modal postdoctoral trainee is a woman whose internship was American Psychological Association (APA)-accredited and whose postdoctoral training is in an APPCN program that adheres to Houston Conference training guidelines. Extensive clinical experiences in neuropsychology in the form of externship practica and during internship were reported by the majority of trainees prior to postdoctoral training. There are few differences between APPCN and non-APPCN trainees and reported training experiences. Job satisfaction is high. Salaries appear to have increased substantially in recent years. There is universal interest in pursuing board certification. Support for the empirical foundations justifying assessment of response validity is high.
Conclusions
Surveys of postdoctoral trainees continue to provide valuable perspectives regarding training background, clinical experiences, practice beliefs, and incomes of individuals who will soon launch their careers in clinical neuropsychology.
Collapse
Affiliation(s)
- Jerry J Sweet
- NorthShore University HealthSystem, Department of Psychiatry & Behavioral Sciences, Evanston, Illinois. USA
- University of Chicago, Pritzker School of Medicine, Department of Psychiatry & Behavioral Neuroscience, Chicago, Illinois. USA
| | - Kristen M Klipfel
- NorthShore University HealthSystem, Department of Psychiatry & Behavioral Sciences, Evanston, Illinois. USA
| | - Nathaniel W Nelson
- Graduate School of Professional Psychology, Morrison Family College of Health, University of St. Thomas, Minneapolis, Minnesota. USA
| | - Paul J Moberg
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania. USA
| |
Collapse
|
28
|
Chafetz MD. Deception is different: Negative validity test findings do not provide "evidence" for "good effort". Clin Neuropsychol 2020; 36:1244-1264. [PMID: 33300435 DOI: 10.1080/13854046.2020.1840633] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
OBJECTIVE The purpose of this paper is to determine whether negative validity test findings should be used in the Bayesian aggregate along with positive test findings for the determination of malingering as the condition of interest (COI). Method: Evidence-based diagnostic methods for conditions in neuropsychology and medicine were reviewed for comparison with their use in cases of malingering. Logical and Bayesian analyses of these cases were applied. A case study showed that negative validity test findings did not indicate "good effort". Results: Deception about illness is fundamentally different from other constructs/diseases in evidence-based medicine and neuropsychology. This is because deception involves a deliberate process that may involve coaching, claimant research, and/or focusing the deception on one aspect (e.g., slowness) as opposed to other neurocognitive problems (e.g., memory). Comparatively, other conditions in medicine and neuropsychology are unlikely to be manipulated by the patient. Conclusions: The assertion by Frederick (2015) and Black, Necrason, and Omasta (2016) that both positive and negative validity test findings must be used together in the aggregate does not stand up to this comparative scrutiny. The fundamental assumption by these authors that a negative test finding concerning malingering represents "good effort" is flawed; it simply represents lack of evidence of malingering, which cannot be construed as evidence of lack of malingering. We recommend that in forensic determination of malingering, negative validity test findings should not be used in a Bayesian aggregation. This conclusion is consistent with current practices in the field.
Collapse
Affiliation(s)
- Michael D Chafetz
- Independent Practice, Algiers Neurobehavioral Resource, LLC, New Orleans, LA, USA
| |
Collapse
|
29
|
Sullivan-Baca E, Naylon K, Zartman A, Ardolf B, Westhafer JG. Gender Differences in Veterans Referred for Neuropsychological Evaluation in an Outpatient Neuropsychology Consultation Service. Arch Clin Neuropsychol 2020; 35:562-575. [PMID: 32128586 DOI: 10.1093/arclin/acaa008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2019] [Revised: 01/21/2020] [Accepted: 01/25/2020] [Indexed: 12/13/2022] Open
Abstract
OBJECTIVE The number of women veterans seeking Veterans Health Administration services has substantially increased over the past decade. Neuropsychology remains an understudied area in the examination of gender differences. The present study sought to delineate similarities and differences in men and women veterans presenting for neuropsychological evaluation in terms of demographics, referral, medical conditions, effort, and outcome diagnosis. METHOD A database collected from an outpatient VA neuropsychology clinic from 2013 to 2019 was analyzed (n = 232 women, 2642 men). Additional analyses examined younger (n = 836 men, 155 women) and older (n = 1805 men, 77 women) age cohorts. RESULTS Women veterans were younger and more educated than men, whereas men had higher prevalence of vascular risk factors. Both groups were most often referred from mental health clinics and memory was the most common referral question. Although men performed worse on performance validity measures, clinicians rated women as evidencing poorer effort on a cumulative rating based on formal and embedded performance validity measures, behavioral observations, and inconsistent test patterns. Older women reported more depressive symptoms than older men and were more commonly diagnosed with depression. CONCLUSIONS This exploratory study fills a gap in the understanding of gender differences in veterans presenting for neuropsychological evaluations. Findings emphasize consideration for the intersection of gender with demographics, medical factors, effort, and psychological symptoms by VA neuropsychologists. A better understanding of relationships between gender and these factors may inform neuropsychologists' test selection, interpretation of behavioral observations, and diagnostic considerations to best treat women veterans.
Collapse
Affiliation(s)
| | - Kara Naylon
- VA North Texas Healthcare System, Dallas, TX, USA
| | | | - Barry Ardolf
- VA North Texas Healthcare System, Dallas, TX, USA
| | | |
Collapse
|
30
|
Clark HA, Martin PK, Okut H, Schroeder RW. A Systematic Review and Meta-Analysis of the Utility of the Test of Memory Malingering in Pediatric Examinees. Arch Clin Neuropsychol 2020; 35:1312-1322. [DOI: 10.1093/arclin/acaa075] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2020] [Revised: 08/22/2020] [Accepted: 08/24/2020] [Indexed: 11/12/2022] Open
Abstract
Abstract
Objective
This is the first systematic review and meta-analysis of the Test of Memory Malingering (TOMM) in pediatric examinees. It adheres to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.
Method
A systematic literature search was conducted using PsycINFO and PubMed, reviewing articles from January 1997 to July 2019. Books providing data on pediatric validity testing were also reviewed for references to relevant articles. Eligibility criteria included publication in a peer-reviewed journal, utilizing a pediatric sample, providing sufficient data to calculate specificity and/or sensitivity, and providing a means for evaluating validity status external to the TOMM. After selection criteria were applied, 9 articles remained for meta-analysis. Samples included clinical patients and healthy children recruited for research purposes; ages ranged from 5 to 18. Fixed and random effects models were used to calculate classification accuracy statistics.
Results
Traditional adult-derived cutoffs for Trial 2 and Retention were highly specific (0.96–0.99) in pediatric examinees for both clinical and research samples. Sensitivity was relatively strong (0.68–0.70), although only two studies reported sensitivity rates. A supplemental review of the literature corroborated these findings, revealing that traditional adult-based TOMM cutoffs are supported in most pediatric settings. However, limited research exists on the impact of very young age, extremely low cognitive functioning, and varying clinical diagnoses.
Conclusions
The TOMM, at traditional adult cutoffs, has strong specificity as a performance validity test in pediatric neuropsychological evaluations. This meta-analysis found that specificity values in children are comparable to those of adults. Areas for further research are discussed.
Collapse
Affiliation(s)
- Hilary A Clark
- Department of Psychiatry & Behavioral Sciences, University of Kansas School of Medicine – Wichita, Wichita, KS 67226, USA
| | - Phillip K Martin
- Department of Psychiatry & Behavioral Sciences, University of Kansas School of Medicine – Wichita, Wichita, KS 67226, USA
| | - Hayrettin Okut
- Office of Research, University of Kansas School of Medicine – Wichita, Wichita, KS 67214, USA
| | - Ryan W Schroeder
- Department of Psychiatry & Behavioral Sciences, University of Kansas School of Medicine – Wichita, Wichita, KS 67226, USA
| |
Collapse
|
31
|
Nikolai T, Cechova K, Bukacova K, Fendrych Mazancova A, Markova H, Bezdicek O, Hort J, Vyhnalek M. Delayed matching to sample task 48: assessment of malingering with simulating design. AGING NEUROPSYCHOLOGY AND COGNITION 2020; 28:797-811. [PMID: 32998629 DOI: 10.1080/13825585.2020.1826898] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
The results of neuropsychological tests may be distorted by patients who exaggerate cognitive deficits. Eighty-three patients with cognitive deficit [Amnestic Mild Cognitive Impairment (aMCI), n = 53; Alzheimer's disease (AD) dementia, n = 30], 44 healthy older adults (HA), and 30 simulators of AD (s-AD) underwent comprehensive neuropsychological assessment. Receiver Operating Characteristic (ROC) analysis revealed high specificity but low sensitivity of the Delayed Matching to Sample Task (DMS48) in differentiating s-AD from AD dementia (87 and 53%, respectively) and from aMCI (96 and 57%). The sensitivity was considerably increased by using the DMS48/Rey Auditory Verbal Learning Test (RAVLT) ratio (specificity and sensitivity 93% and 93% for AD dementia and 96% and 80% for aMCI). The DMS48 differentiates s-AD from both aMCI and AD dementia with high specificity but low sensitivity. Its predictive value greatly increased when evaluated together with the RAVLT.
Collapse
Affiliation(s)
- T Nikolai
- Department of Neurology, Neuropsychology Laboratory, 1st Faculty of Medicine and General University Hospital, Prague, Czech Republic.,International Clinical Research Center, St. Anne's University Hospital Brno, Brno, Czech Republic
| | - K Cechova
- International Clinical Research Center, St. Anne's University Hospital Brno, Brno, Czech Republic.,Department of Neurology, Memory Clinic, 2nd Faculty of Medicine, Charles University and Motol University Hospital, Prague, Czech Republic
| | - K Bukacova
- Department of Neurology, Neuropsychology Laboratory, 1st Faculty of Medicine and General University Hospital, Prague, Czech Republic
| | - A Fendrych Mazancova
- Department of Neurology, Neuropsychology Laboratory, 1st Faculty of Medicine and General University Hospital, Prague, Czech Republic.,International Clinical Research Center, St. Anne's University Hospital Brno, Brno, Czech Republic
| | - H Markova
- International Clinical Research Center, St. Anne's University Hospital Brno, Brno, Czech Republic.,Department of Neurology, Memory Clinic, 2 Faculty of Medicine, Charles University and Motol University Hospital, Prague, Czech Republic
| | - O Bezdicek
- Department of Neurology, Neuropsychology Laboratory, 1st Faculty of Medicine and General University Hospital, Prague, Czech Republic
| | - J Hort
- International Clinical Research Center, St. Anne's University Hospital Brno, Brno, Czech Republic.,Department of Neurology, Memory Clinic, 2 Faculty of Medicine, Charles University and Motol University Hospital, Prague, Czech Republic
| | - M Vyhnalek
- International Clinical Research Center, St. Anne's University Hospital Brno, Brno, Czech Republic.,Department of Neurology, Memory Clinic, 2 Faculty of Medicine, Charles University and Motol University Hospital, Prague, Czech Republic
| |
Collapse
|
32
|
Sherman EMS, Slick DJ, Iverson GL. Multidimensional Malingering Criteria for Neuropsychological Assessment: A 20-Year Update of the Malingered Neuropsychological Dysfunction Criteria. Arch Clin Neuropsychol 2020; 35:735-764. [PMID: 32377667 PMCID: PMC7452950 DOI: 10.1093/arclin/acaa019] [Citation(s) in RCA: 152] [Impact Index Per Article: 38.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2019] [Accepted: 03/12/2020] [Indexed: 11/17/2022] Open
Abstract
OBJECTIVES Empirically informed neuropsychological opinion is critical for determining whether cognitive deficits and symptoms are legitimate, particularly in settings where there are significant external incentives for successful malingering. The Slick, Sherman, and Iversion (1999) criteria for malingered neurocognitive dysfunction (MND) are considered a major milestone in the field's operationalization of neurocognitive malingering and have strongly influenced the development of malingering detection methods, including serving as the criterion of malingering in the validation of several performance validity tests (PVTs) and symptom validity tests (SVTs) (Slick, D.J., Sherman, E.M.S., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13(4), 545-561). However, the MND criteria are long overdue for revision to address advances in malingering research and to address limitations identified by experts in the field. METHOD The MND criteria were critically reviewed, updated with reference to research on malingering, and expanded to address other forms of malingering pertinent to neuropsychological evaluation such as exaggeration of self-reported somatic and psychiatric symptoms. RESULTS The new proposed criteria simplify diagnostic categories, expand and clarify external incentives, more clearly define the role of compelling inconsistencies, address issues concerning PVTs and SVTs (i.e., number administered, false positives, and redundancy), better define the role of SVTs and of marked discrepancies indicative of malingering, and most importantly, clearly define exclusionary criteria based on the last two decades of research on malingering in neuropsychology. Lastly, the new criteria provide specifiers to better describe clinical presentations for use in neuropsychological assessment. CONCLUSIONS The proposed multidimensional malingering criteria that define cognitive, somatic, and psychiatric malingering for use in neuropsychological assessment are presented.
Collapse
Affiliation(s)
| | | | - Grant L Iverson
- Department of Physical Medicine and Rehabilitation, Harvard Medical School, Boston, MA, USA
- Spaulding Rehabilitation Hospital and Spaulding Research Institute, Charlestown, MA, USA
- Home Base, A Red Sox Foundation and Massachusetts General Hospital Program, Charlestown, MA, USA
| |
Collapse
|
33
|
Henry GK, Gornbein J. The Modified Somatic Perception Questionnaire: Operating characteristics in non-pain forensic samples. APPLIED NEUROPSYCHOLOGY-ADULT 2020; 29:598-604. [PMID: 32692261 DOI: 10.1080/23279095.2020.1794868] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
The objective of the current archival study was to investigate the ability of the Modified Somatic Perception Questionnaire (MSPQ) to discriminate between noncredible and credible neurocognitive dysfunction in a large mixed non-pain forensic sample comprised of personal injury litigants and disability claimants. Participants included 149 adults who underwent comprehensive neuropsychological examination. Criterion groups were formed, i.e., Credible Group (CG), or Noncredible Group (NCG) based upon their performance on stand-alone performance validity tests (PVT) including the Word Memory Test (WMT), and/or Test of Memory Malingering (TOMM), and Victoria Symptom Validity Test (VSVT). After excluding examinees with evidence of somatization participants in the NCG scored significantly higher on the MSPQ compared to participants in the CG. Scores on the MSPQ were not only related to cognitive performance validity but also amplified by the co-existence of somatization. There were no significant effects of gender, age, or race on MSPQ scores. We conclude that the MSPQ demonstrates external validity as a self-report measure of symptom validity that is sensitive to somatic and autonomic complaints in non-pain forensic samples. Future research is needed to develop MSPQ cutscores with external validity pertinent to other non-pain populations.
Collapse
Affiliation(s)
| | - Jeffrey Gornbein
- Biostatistics, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA
| |
Collapse
|
34
|
Martin PK, Schroeder RW, Olsen DH. Performance validity in the dementia clinic: Specificity of validity tests when used individually and in aggregate across levels of cognitive impairment severity. Clin Neuropsychol 2020; 36:165-188. [DOI: 10.1080/13854046.2020.1778790] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Affiliation(s)
- Phillip K. Martin
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine – Wichita, Wichita, KS, USA
| | - Ryan W. Schroeder
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine – Wichita, Wichita, KS, USA
| | | |
Collapse
|
35
|
Comparison of Clinical Psychologist and Physician Beliefs and Practices Concerning Malingering: Results from a Mixed Methods Study. PSYCHOLOGICAL INJURY & LAW 2020. [DOI: 10.1007/s12207-020-09374-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
36
|
Martin PK, Schroeder RW. Base Rates of Invalid Test Performance Across Clinical Non-forensic Contexts and Settings. Arch Clin Neuropsychol 2020; 35:717-725. [DOI: 10.1093/arclin/acaa017] [Citation(s) in RCA: 53] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2019] [Revised: 01/22/2020] [Accepted: 02/25/2020] [Indexed: 11/14/2022] Open
Abstract
Abstract
Objective
Base rates of invalidity in forensic neuropsychological contexts are well explored and believed to approximate 40%, whereas base rates of invalidity across clinical non-forensic contexts are relatively less known.
Methods
Adult-focused neuropsychologists (n = 178) were surveyed regarding base rates of invalidity across various clinical non-forensic contexts and practice settings. Median values were calculated and compared across contexts and settings.
Results
The median estimated base rate of invalidity across clinical non-forensic evaluations was 15%. When examining specific clinical contexts and settings, base rate estimates varied from 5% to 50%. Patients with medically unexplained symptoms (50%), external incentives (25%–40%), and oppositional attitudes toward testing (37.5%) were reported to have the highest base rates of invalidity. Patients with psychiatric illness, patients evaluated for attention deficit hyperactivity disorder, and patients with a history of mild traumatic brain injury were also reported to invalidate testing at relatively high base rates (approximately 20%). Conversely, patients presenting for dementia evaluation and patients with none of the previously mentioned histories and for whom invalid testing was unanticipated were estimated to produce invalid testing in only 5% of cases. Regarding practice setting, Veterans Affairs providers reported base rates of invalidity to be nearly twice that of any other clinical settings.
Conclusions
Non-forensic clinical patients presenting with medically unexplained symptoms, external incentives, or oppositional attitudes are reported to invalidate testing at base rates similar to that of forensic examinees. The impact of context-specific base rates on the clinical evaluation of invalidity is discussed.
Collapse
Affiliation(s)
- Phillip K Martin
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| | - Ryan W Schroeder
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| |
Collapse
|
37
|
Martin PK, Schroeder RW. Feedback with patients who produce invalid testing: Professional values and reported practices. Clin Neuropsychol 2020; 35:1134-1153. [PMID: 32031048 DOI: 10.1080/13854046.2020.1722243] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
OBJECTIVE Providing feedback to clinical patients who produce invalid neuropsychological test data presents with several potential challenges. Given the limited guidance available on the topic, neuropsychologists most likely utilize approaches that idiosyncratically incorporate professional values related to both assessment and patient care to address overarching feedback goals. The current article discusses professional values believed to inform feedback decisions and presents survey results regarding how neuropsychologists would approach feedback across various clinical scenarios where testing is invalid. METHOD Participants were 209 adult-focused clinical neuropsychologists recruited via professional listservs, the majority of whom reported being board certified. Respondents were provided three case vignettes of clinical patients who produced invalid test data and were asked how they would provide feedback to each patient. Open-ended responses were coded to determine the relative frequency of feedback approaches, explicit statements, and implied goals. RESULTS Nearly all respondents (98%) indicated that feedback would include some description of the invalid findings, and most respondents indicated that they would provide explanations for the reasons of invalidity (67%) and statements regarding the impact of invalidity on test interpretation (75%). There was little agreement across respondents, however, regarding specific feedback approach. Feedback goals included to Inform, Investigate, Treat, Educate, and Correct, with the presence of these implied goals also varying across respondents. CONCLUSIONS The findings indicate that there is minimal consensus regarding feedback approaches provided to patients who produce invalid test data and underscore a need for further development and validation of specific feedback methods. The results are discussed within the context of the potentially competing professional values of evidence-based assessment, patient-doctor collaboration, and aspirational principles of beneficence and fidelity.
Collapse
Affiliation(s)
- Phillip K Martin
- Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| | - Ryan W Schroeder
- Psychiatry and Behavioral Sciences, University of Kansas School of Medicine, Wichita, KS, USA
| |
Collapse
|
38
|
Olsen DH, Schroeder RW, Martin PK. Cross-validation of the Invalid Forgetting Frequency Index (IFFI) from the Test of Memory Malingering. Arch Clin Neuropsychol 2019; 36:437-441. [DOI: 10.1093/arclin/acz064] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2019] [Revised: 10/08/2019] [Indexed: 11/14/2022] Open
Abstract
Abstract
Objective
To increase sensitivity of the Test of Memory Malingering (TOMM), adjustments have been proposed, including adding consistency indices. The Invalid Forgetting Frequency Index (IFFI) is the most recently developed consistency index. While strong classification accuracy rates were originally reported, it currently lacks cross-validation.
Method
A sample of 184 outpatients was utilized. Valid performers passed all criterion performance validity tests (PVTs) and invalid performers failed two or more PVTs. Classification accuracy statistics were calculated.
Results
AUC for the IFFI was 0.80, demonstrating adequate discrimination between valid and invalid groups. A score of 3 or more inconsistent responses resulted in sensitivity and specificity rates of 63% and 92%, respectively.
Conclusions
This is the first article to cross-validate the IFFI. In both the original IFFI study and the current study, the same cut-off was found to maintain at least 90% specificity while producing higher sensitivity rates than those achieved by traditional TOMM indices.
Collapse
Affiliation(s)
- Daniel H Olsen
- University of Kansas School of Medicine – Wichita, Department of Psychiatry and Behavioral Sciences, Wichita, Kansas, United States
| | - Ryan W Schroeder
- University of Kansas School of Medicine – Wichita, Department of Psychiatry and Behavioral Sciences, Wichita, Kansas, United States
| | - Phillip K Martin
- University of Kansas School of Medicine – Wichita, Department of Psychiatry and Behavioral Sciences, Wichita, Kansas, United States
| |
Collapse
|
39
|
Whiteside DM, Hunt I, Choate A, Caraher K, Basso MR. Stratified performance on the Test of Memory Malingering (TOMM) is associated with differential responding on the Personality Assessment Inventory (PAI). J Clin Exp Neuropsychol 2019; 42:131-141. [DOI: 10.1080/13803395.2019.1695749] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Affiliation(s)
- Douglas M. Whiteside
- Department of Rehabilitation Medicine, Neuropsychology Laboratory, University of Minnesota, Minneapolis, MN, USA
| | - Isaac Hunt
- Department of Neurology, St. Mary’s Medical Center, Essentia Health, Duluth, MN, USA
| | - Alyssa Choate
- Department of Psychiatry, University of Iowa Hospitals and Clinics, Iowa City, IA, USA
| | - Kristen Caraher
- Department of Psychiatry, University of Iowa Hospitals and Clinics, Iowa City, IA, USA
| | - Michael R. Basso
- Department of Psychiatry and Psychology, Mayo Clinic, Rochester, NY, USA
| |
Collapse
|
40
|
Martin PK, Schroeder RW, Olsen DH, Maloy H, Boettcher A, Ernst N, Okut H. A systematic review and meta-analysis of the Test of Memory Malingering in adults: Two decades of deception detection. Clin Neuropsychol 2019; 34:88-119. [DOI: 10.1080/13854046.2019.1637027] [Citation(s) in RCA: 68] [Impact Index Per Article: 13.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Phillip K. Martin
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine –Wichita, Wichita, KS, USA
| | - Ryan W. Schroeder
- Department of Psychiatry and Behavioral Sciences, University of Kansas School of Medicine –Wichita, Wichita, KS, USA
| | - Daniel H. Olsen
- University of Kansas School of Medicine – Wichita, Wichita, KS, USA
| | - Halley Maloy
- University of Kansas School of Medicine – Wichita, Wichita, KS, USA
| | | | - Nathan Ernst
- University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| | - Hayrettin Okut
- University of Kansas School of Medicine – Wichita, Wichita, KS, USA
| |
Collapse
|
41
|
McGuire C, Crawford S, Evans JJ. Effort Testing in Dementia Assessment: A Systematic Review. Arch Clin Neuropsychol 2019; 34:114-131. [PMID: 29579143 DOI: 10.1093/arclin/acy012] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2017] [Accepted: 01/30/2018] [Indexed: 11/14/2022] Open
Abstract
Objective Interpretation of neuropsychological test data is only valid when appropriate effort has been exerted. Research, however, suggests that neuropsychologists do not always formally test for effort and that this may especially be the case in the context of dementia assessment. This review systematically examined the literature that has investigated the use of both purpose-built and embedded effort-sensitive indices in dementia, mild cognitive impairment (MCI), and healthy control samples. The aim was to determine which tests of effort are most sensitive to suboptimal effort and least sensitive to the type of cognitive impairment seen in dementia. Methods A systematic search of databases was conducted to October 2017. There was no start date. Results Twenty-five studies were included for review. The studies were divided into two categories according to methodology. One category of studies (n = 5) was reviewed using a tailored methodological quality rating checklist whilst the remaining studies (n = 20) were reviewed using the Crowe Critical Appraisal Tool (CCAT). Conclusions The results of this review suggest that PVTs which take a hierarchical approach to effort testing such as the WMT, MSVT, and NV-MSVT are preferable for use with older adults who are under investigation for possible dementia. These tests go above and beyond the traditional pass/fail approach of more traditional tests of effort because they allow the examiner to analyze the examinee's profile of scores. The methodological limitations and challenges involved in this field of research are discussed.
Collapse
Affiliation(s)
- Claire McGuire
- NHS Greater Glasgow & Clyde, Older People's Psychology Service, Eastwood Health and Care Centre, Drumby Crescent, Clarkston, UK
| | - Stephanie Crawford
- NHS Greater Glasgow & Clyde, Older People's Psychology Service, Crown House, 30 King Street, Greenock, UK
| | - Jonathan J Evans
- Mental Health and Wellbeing, University of Glasgow, The Academic Centre, Gartnavel Royal Hospital, 1055 Great Western Road, Glasgow, UK
| |
Collapse
|
42
|
Marcopulos BA, Guterbock TM, Matusz EF. Survey research in neuropsychology: A systematic review. Clin Neuropsychol 2019; 34:32-55. [DOI: 10.1080/13854046.2019.1590643] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Bernice a. Marcopulos
- Department of Graduate Psychology, James Madison University, Harrisonburg, Virginia, USA
- Department of Psychiatry and Neurobehavioral Sciences, University of Virginia, Charlottesville, Virginia, USA
| | - Thomas M. Guterbock
- Center for Survey Research and Department of Sociology, University of Virginia, Charlottesville, Virginia, USA
| | - Emily F. Matusz
- Department of Graduate Psychology, James Madison University, Harrisonburg, Virginia, USA
| |
Collapse
|
43
|
Schroeder RW, Olsen DH, Martin PK. Classification accuracy rates of four TOMM validity indices when examined independently and jointly. Clin Neuropsychol 2019; 33:1373-1387. [DOI: 10.1080/13854046.2019.1619839] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Ryan W. Schroeder
- Department of Psychiatry & Behavioral Sciences, University of Kansas School of Medicine – Wichita, Wichita, KS, USA
| | - Daniel H. Olsen
- Department of Psychiatry & Behavioral Sciences, University of Kansas School of Medicine – Wichita, Wichita, KS, USA
| | - Phillip K. Martin
- Department of Psychiatry & Behavioral Sciences, University of Kansas School of Medicine – Wichita, Wichita, KS, USA
| |
Collapse
|
44
|
Larrabee GJ, Rohling ML, Meyers JE. Use of multiple performance and symptom validity measures: Determining the optimal per test cutoff for determination of invalidity, analysis of skew, and inter-test correlations in valid and invalid performance groups. Clin Neuropsychol 2019; 33:1354-1372. [DOI: 10.1080/13854046.2019.1614227] [Citation(s) in RCA: 32] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
| | - Martin L. Rohling
- Department of Psychology, University of South Alabama, Mobile, AL, USA
| | | |
Collapse
|
45
|
Jurick SM, Crocker LD, Keller AV, Hoffman SN, Bomyea J, Jacobson MW, Jak AJ. The Minnesota Multiphasic Personality Inventory-2-RF in Treatment-Seeking Veterans with History of Mild Traumatic Brain Injury. Arch Clin Neuropsychol 2019; 34:366-380. [PMID: 29850866 DOI: 10.1093/arclin/acy048] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2017] [Revised: 03/26/2018] [Accepted: 05/09/2018] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVE This study examined the Minnesota Multiphasic Personality Inventory-Second Edition-Restructured Form (MMPI-2-RF) to better understand symptom presentation in a sample of treatment-seeking Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) Veterans with self-reported history of mild traumatic brain injury (mTBI). METHOD Participants underwent a comprehensive clinical neuropsychological battery including performance and symptom validity measures and self-report measures of depressive, posttraumatic, and post-concussive symptomatology. Those with possible symptom exaggeration (SE+) on the MMPI-2-RF were compared with those without (SE-) with regard to injury, psychiatric, validity, and cognitive variables. RESULTS Between 50% and 87% of participants demonstrated possible symptom exaggeration on one or more MMPI-2-RF validity scales, and a large majority were elevated on content scales related to cognitive, somatic, and emotional complaints. The SE+ group reported higher depressive, posttraumatic, and post-concussive symptomatology, had higher scores on symptom validity measures, and performed more poorly on neuropsychological measures compared with the SE- group. There were no group differences with regard to injury variables or performance validity measures. Participants were more likely to exhibit possible symptom exaggeration on cognitive/somatic compared with traditional psychopathological validity scales. CONCLUSIONS A sizable portion of treatment-seeking OEF/OIF Veterans demonstrated possible symptom exaggeration on MMPI-2-RF validity scales, which was associated with elevated scores on self-report measures and poorer cognitive performance, but not higher rates of performance validity failure, suggesting symptom and performance validity are distinct concepts. These findings have implications for the interpretation of clinical data in the context of possible symptom exaggeration and treatment in Veterans with persistent post-concussive symptoms.
Collapse
Affiliation(s)
- S M Jurick
- Department of Psychiatry, San Diego State University/University of California San Diego Joint Doctoral Program in Clinical Psychology, San Diego, CA, USA.,Veterans Medical Research Foundation, San Diego, CA, USA
| | - L D Crocker
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA.,Center of Excellence for Stress and Mental Health, VA San Diego Healthcare System, San Diego, CA, USA
| | - A V Keller
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA
| | - S N Hoffman
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA
| | - J Bomyea
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA
| | - M W Jacobson
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA.,Department of Psychiatry, University of California San Diego, San Diego, CA, USA
| | - A J Jak
- Psychology Service, VA San Diego Healthcare System, San Diego, CA, USA.,Center of Excellence for Stress and Mental Health, VA San Diego Healthcare System, San Diego, CA, USA.,Department of Psychiatry, University of California San Diego, San Diego, CA, USA
| |
Collapse
|
46
|
Bain KM, Soble JR, Webber TA, Messerly JM, Bailey KC, Kirton JW, McCoy KJM. Cross-validation of three Advanced Clinical Solutions performance validity tests: Examining combinations of measures to maximize classification of invalid performance. APPLIED NEUROPSYCHOLOGY-ADULT 2019; 28:24-34. [PMID: 30987451 DOI: 10.1080/23279095.2019.1585352] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
Use of multiple performance validity tests (PVTs) may best identify invalid performance, though few studies have examined the utility and accuracy of combining PVTs. This study examined the following PVTs in the Advanced Clinical Solutions (ACS) package to determine their utility alone and in concert: Word Choice Test (WCT), Reliable Digit Span (RDS), and Logical Memory Recognition (LMR). Ninety-three veterans participated in clinical neuropsychological evaluations to determine presence of cognitive impairment; 25% of the performances were deemed invalid via criterion PVTs. Classification accuracy of the ACS measures was assessed via receiver operating characteristic curves, while logistic regressions determined utility of combining these PVTs. The WCT demonstrated superior classification accuracy compared to the two embedded measures of the ACS, even in veterans with cognitive impairment. The two embedded measures (even when used in concert) exhibited inadequate classification accuracy. A combined model with all three ACS PVTs similarly demonstrated little benefit of the embedded indicators over the WCT alone. Results suggest the ACS WCT has utility for detecting invalid performance in a clinical sample with likely cognitive impairment, though the embedded ACS measures (RDS and LMR) may have limited incremental utility, particularly in individuals with cognitive impairment.
Collapse
Affiliation(s)
- Kathleen M Bain
- South Texas Veterans Health Care System, San Antonio, Texas, USA
| | - Jason R Soble
- University of Illinois at Chicago College of Medicine, Chicago, Illinois, USA
| | - Troy A Webber
- Michael E. DeBakey VA Medical Center, Houston, Texas, USA
| | | | - K Chase Bailey
- University of Texas Southwestern Medical Center at Dallas, Dallas, Texas, USA
| | - Joshua W Kirton
- Fort Carson Army Medical Center, Colorado Springs, Colorado, USA
| | - Karin J M McCoy
- South Texas Veterans Health Care System, San Antonio, Texas, USA
| |
Collapse
|
47
|
Elbaum T, Golan L, Lupu T, Wagner M, Braw Y. Establishing supplementary response time validity indicators in the Word Memory Test (WMT) and directions for future research. APPLIED NEUROPSYCHOLOGY-ADULT 2019; 27:403-413. [DOI: 10.1080/23279095.2018.1555161] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
- Tomer Elbaum
- Department of Psychology, Ariel University, Ariel, Israel
- Department of Industrial Engineering & Management, Ariel University, Ariel, Israel
| | - Lior Golan
- Department of Psychology, Ariel University, Ariel, Israel
| | - Tamar Lupu
- Department of Psychology, Ariel University, Ariel, Israel
| | - Michael Wagner
- Department of Industrial Engineering & Management, Ariel University, Ariel, Israel
| | - Yoram Braw
- Department of Psychology, Ariel University, Ariel, Israel
| |
Collapse
|
48
|
Tonks J, Whitfield CK, Williams WH, Slater AM, Frampton IJ. "Must try harder." Is effort and performance validity testing a necessary part of pediatric neuropsychological assessment? APPLIED NEUROPSYCHOLOGY-CHILD 2018; 9:97-105. [PMID: 30583707 DOI: 10.1080/21622965.2018.1524766] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
Neuropsychological assessments results have significant implications for pediatric populations, based upon the assumption that the young person has adopted an effortful approach and has engaged in assessment. There is a commonly-accepted risk to assuming the validity of neuropsychological assessment results with adults, and, therefore, performance validity testing (PVT) has become a major topic of research and investigation and has become an accepted part of routine assessment. The same approach has not been adopted in assessment with children and a paucity of studies has focused on PVT in children. We review studies that demonstrate that children are equal to adults in their ability to use deception and that clinicians cannot detect false-effort without use of validity tests. We explore how frequently such tests are used and how well they work in assessment with children, and the limits, complexities, and constraints of adapting adult tests. We advocate that adequate performance validity testing is essential in order to maximize confidence in the results and we hypothesize that assessment with pediatric populations should take into account a range of influences, such as neuro-developmental factors associated with age of the child and suitability of proposed measures according to the evidence-base.
Collapse
Affiliation(s)
- James Tonks
- 1University of Exeter Medical School, Exeter, United Kingdom of Great Britain and Northern Ireland
| | - Charlotte Katie Whitfield
- 2School of Psychology, Cardiff University, Cardiff, United Kingdom of Great Britain and Northern Ireland
| | - W Huw Williams
- 3School of Psychology, Washington Singer Laboratories, University of Exeter College of Life and Environmental Sciences, Exeter, United Kingdom of Great Britain and Northern Ireland
| | - Alan M Slater
- School of Psychology, Washington Singer Laboratories, University of Exeter College of Life and Environmental Sciences, Exeter, United Kingdom of Great Britain and Northern Ireland
| | - Ian J Frampton
- School of Psychology, Washington Singer Laboratories, University of Exeter College of Life and Environmental Sciences, Exeter, United Kingdom of Great Britain and Northern Ireland
| |
Collapse
|
49
|
Poynter K, Boone KB, Ermshar A, Miora D, Cottingham M, Victor TL, Ziegler E, Zeller MA, Wright M. Wait, There’s a Baby in this Bath Water! Update on Quantitative and Qualitative Cut-Offs for Rey 15-Item Recall and Recognition. Arch Clin Neuropsychol 2018; 34:1367-1380. [DOI: 10.1093/arclin/acy087] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2018] [Revised: 10/10/2018] [Accepted: 10/17/2018] [Indexed: 11/14/2022] Open
Abstract
Abstract
Objective
Evaluate the effectiveness of Rey 15-item plus recognition data in a large neuropsychological sample.
Method
Rey 15-item plus recognition scores were compared in credible (n = 138) and noncredible (n = 353) neuropsychology referrals.
Results
Noncredible patients scored significantly worse than credible patients on all Rey 15-item plus recognition scores. When cut-offs were selected to maintain at least 89.9% specificity, cut-offs could be made more stringent, with the highest sensitivity found for recognition correct (cut-off ≤11; 62.6% sensitivity) and the combination score (recall + recognition – false positives; cut-off ≤22; 60.6% sensitivity), followed by recall correct (cut-off ≤11; 49.3% sensitivity), and recognition false positive errors (≥3; 17.9% sensitivity). A cut-off of ≥4 applied to a summed qualitative error score for the recall trial resulted in 19.4% sensitivity. Approximately 10% of credible subjects failed either recall correct or recognition correct, whereas two-thirds of noncredible patients (67.7%) showed this pattern. Thirteen percent of credible patients failed either recall correct, recognition correct, or the recall qualitative error score, whereas nearly 70% of noncredible patients failed at least one of the three. Some individual qualitative recognition errors had low false positive rates (<2%) indicating that their presence was virtually pathognomonic for noncredible performance. Older age (>50) and IQ < 80 were associated with increased false positive rates in credible patients.
Conclusions
Data on a larger sample than that available in the 2002 validation study show that Rey 15-item plus recognition cut-offs can be made more stringent, and thereby detect up to 70% of noncredible test takers, but the test should be used cautiously in older individuals and in individuals with lowered IQ.
Collapse
Affiliation(s)
- Kellie Poynter
- California School of Forensic Studies, Alliant International University, Los Angeles, CA, USA
| | - Kyle Brauer Boone
- California School of Forensic Studies, Alliant International University, Los Angeles, CA, USA
| | - Annette Ermshar
- California School of Forensic Studies, Alliant International University, Los Angeles, CA, USA
| | - Deborah Miora
- California School of Forensic Studies, Alliant International University, Los Angeles, CA, USA
| | - Maria Cottingham
- Mental Health Care Line, Veterans Administration Tennessee Valley Healthcare System, Nashville, TN, USA
| | - Tara L Victor
- California State University, Dominguez Hills, Carson, CA, USA
| | | | - Michelle A Zeller
- West Los Angeles Veterans Administration Medical Center, Los Angeles, CA, USA
| | | |
Collapse
|
50
|
Goette WF, Goette HE. A meta-analysis of the accuracy of embedded performance validity indicators from the repeatable battery for the assessment of neuropsychological status. Clin Neuropsychol 2018; 33:1044-1068. [PMID: 30472924 DOI: 10.1080/13854046.2018.1538429] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
Objective: Meta-analyze the embedded performance validity tests of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), emphasizing two topics: diagnostic accuracy and relationships with demographics. Method: Literature was identified through a review of PsycINFO, PubMed, and Google Scholar. Bivariate linear mixed models were used for diagnostic accuracy analyses. Random-effects analysis of correlations with demographic variables was also used for the Effort Index. Results: Diagnostic accuracy analyses utilized 12 studies of the Effort Index (EI; N = 1469) and 6 studies of the Effort Scale (ES; N = 854). The EI was found to have a sensitivity of .44 (95% CI: .28 - .62), specificity of .87 (95% CI: .78 - .93), and diagnostic odds ratio (DOR) of 5.41 (95% CI: 3.74 - 7.58). The ES produced a sensitivity of .67 (95% CI: .50 - .81), specificity of .72 (95% CI: .53 - .85), and DOR of 5.97 (95% CI: 1.70 - 15.30). Meta-analysis of correlations utilized 14 samples of the EI (N = 3781), but insufficient data were available for the ES. Correlations between the EI and age (r = .10, 95% CI: .05 - .16), education (r = -.11, 95% CI: -.18 to -.04), and the RBANS Total Scale (r = -.45, 95% CI: -.62 to -.24) were significant. These results appeared to be robust to publication bias. Conclusion: Results support use of the EI over the ES; however, the latter is comparatively less studied and did not have similar variability in reported cut-off scores. ES accuracy statistics were significantly predicted by sample variables, which may be related to its relatively fewer studies. Both measures should be used cautiously in the context of genuine cognitive impairment.
Collapse
Affiliation(s)
- William F Goette
- a Department of Psychology and Counseling , The University of Texas at Tyler , Tyler , Texas , USA
| | - Haley E Goette
- b Department of History , The University of Texas at Tyler , Tyler , Texas , USA
| |
Collapse
|