1
|
Finley JCA. Performance validity testing: the need for digital technology and where to go from here. Front Psychol 2024; 15:1452462. [PMID: 39193033 PMCID: PMC11347285 DOI: 10.3389/fpsyg.2024.1452462] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2024] [Accepted: 07/29/2024] [Indexed: 08/29/2024] Open
Affiliation(s)
- John-Christopher A. Finley
- Department of Psychiatry and Behavioral Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, United States
| |
Collapse
|
2
|
Dong H, Koerts J, Pijnenborg GHM, Scherbaum N, Müller BW, Fuermaier ABM. Cognitive Underperformance in a Mixed Neuropsychiatric Sample at Diagnostic Evaluation of Adult ADHD. J Clin Med 2023; 12:6926. [PMID: 37959391 PMCID: PMC10647211 DOI: 10.3390/jcm12216926] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 10/31/2023] [Accepted: 11/02/2023] [Indexed: 11/15/2023] Open
Abstract
(1) Background: The clinical assessment of attention-deficit/hyperactivity disorder (ADHD) in adulthood is known to show non-trivial base rates of noncredible performance and requires thorough validity assessment. (2) Objectives: The present study estimated base rates of noncredible performance in clinical evaluations of adult ADHD on one or more of 17 embedded validity indicators (EVIs). This study further examines the effect of the order of test administration on EVI failure rates, the association between cognitive underperformance and symptom overreporting, and the prediction of cognitive underperformance by clinical information. (3) Methods: A mixed neuropsychiatric sample (N = 464, ADHD = 227) completed a comprehensive neuropsychological assessment battery on the Vienna Test System (VTS; CFADHD). Test performance allows the computation of 17 embedded performance validity indicators (PVTs) derived from eight different neuropsychological tests. Further, all participants completed several self- and other-report symptom rating scales assessing depressive symptoms and cognitive functioning. The Conners' Adult ADHD Rating Scale and the Beck Depression Inventory-II were administered to derive embedded symptom validity measures (SVTs). (4) Results and conclusion: Noncredible performance occurs in a sizeable proportion of about 10% up to 30% of individuals throughout the entire battery. Tests for attention and concentration appear to be the most adequate and sensitive for detecting underperformance. Cognitive underperformance represents a coherent construct and seems dissociable from symptom overreporting. These results emphasize the importance of performing multiple PVTs, at different time points, and promote more accurate calculation of the positive and negative predictive values of a given validity measure for noncredible performance during clinical assessments. Future studies should further examine whether and how the present results stand in other clinical populations, by implementing rigorous reference standards of noncredible performance, characterizing those failing PVT assessments, and differentiating between underlying motivations.
Collapse
Affiliation(s)
- Hui Dong
- Department of Clinical and Developmental Neuropsychology, Faculty of Behavioral and Social Sciences, University of Groningen, 9712 TS Groningen, The Netherlands; (H.D.); (J.K.); (G.H.M.P.)
| | - Janneke Koerts
- Department of Clinical and Developmental Neuropsychology, Faculty of Behavioral and Social Sciences, University of Groningen, 9712 TS Groningen, The Netherlands; (H.D.); (J.K.); (G.H.M.P.)
| | - Gerdina H. M. Pijnenborg
- Department of Clinical and Developmental Neuropsychology, Faculty of Behavioral and Social Sciences, University of Groningen, 9712 TS Groningen, The Netherlands; (H.D.); (J.K.); (G.H.M.P.)
| | - Norbert Scherbaum
- LVR University Hospital, Department of Psychiatry and Psychotherapy, Faculty of Medicine, University of Duisburg-Essen, 45147 Essen, Germany; (N.S.); (B.W.M.)
| | - Bernhard W. Müller
- LVR University Hospital, Department of Psychiatry and Psychotherapy, Faculty of Medicine, University of Duisburg-Essen, 45147 Essen, Germany; (N.S.); (B.W.M.)
- Department of Psychology, University of Wuppertal, 42119 Wuppertal, Germany
| | - Anselm B. M. Fuermaier
- Department of Clinical and Developmental Neuropsychology, Faculty of Behavioral and Social Sciences, University of Groningen, 9712 TS Groningen, The Netherlands; (H.D.); (J.K.); (G.H.M.P.)
| |
Collapse
|
3
|
Lee Y, Lee Y, Yoo S, Shin S, Park H, Kim D. The Psychomotor Cognition Test for Measurement of Sleepiness/Fatigue on a Touch Screen . ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083744 DOI: 10.1109/embc40787.2023.10340988] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
The Psychomotor Vigilance Test (PVT) is a simple and reliable performance test that measures sustained attention, alertness level, and fatigue level. The PVT is a convenient tool that can be used in real time in situ through a mobile device without the assistance of experts and therefore can be used to improve safety and prevent accidents. However, the original PVT is vulnerable to the subject's intentional concentration on the test, and the variance range among tests is narrow; these factors limit its usefulness in classifying the level of fatigue. This study overcome these limitations and develop the Psychomotor Cognition Test (PCT) by transforming the PVT into a tool that stably classifies fatigue levels, still requiring a short period of time. In the PCT, compared to the PVT, reaction time is significantly longer, and success rate is significantly lower (both p<0.0001). Whereas reaction time and success rate of the PVT do not show a significant correlation with fatigue level, those of the PCT show significant correlations with fatigue level, respectively (p<0.001). This study suggests that the PCT can be used in real time in situ as a risk management tool for workers performing dangerous tasks and can become an even more powerful tool when combined with other physiological indicators.
Collapse
|
4
|
Lev A, Elbaum T, Berger C, Braw Y. Feigned ADHD Associated Cognitive Impairment: Utility of Integrating an Eye-tracker and the MOXO-dCPT. J Atten Disord 2022; 26:1212-1222. [PMID: 34911385 DOI: 10.1177/10870547211063643] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
OBJECTIVE The current study assessed the utility of eye-movements measures, gathered while participants performed a commercially available Continuous Performance Test (CPT), to detect feigned ADHD-associated cognitive impairment. METHOD Healthy simulators (n = 37), ADHD patients (n = 33), and healthy controls (n = 36) performed an eye-tracker integrated MOXO-dCPT and a stand-alone validity indicator. RESULTS Simulators gazed significantly longer at regions that were irrelevant for successful MOXO-dCPT performance compared to ADHD patients and healthy controls. This eye-movement measure, however, had lower sensitivity than traditional MOXO-dCPT indices. DISCUSSION Gaze direction measures, gathered while performing a CPT, show initial promise as validity indicators. Traditional CPT measures, however, are more sensitive and therefore offer a more promising path for the establishment of CPT-based validity indicators. The current study is an initial exploration of the issue and further evaluation of both theoretical and practical aspects is mandated.
Collapse
Affiliation(s)
- Astar Lev
- Bar-Ilan University, Ramat Gan, Israel
| | | | | | | |
Collapse
|
5
|
Kenton JA, Young JW. Preclinical Evaluation of Attention and Impulsivity Relevant to Determining ADHD Mechanisms and Treatments. Curr Top Behav Neurosci 2022; 57:291-320. [PMID: 35606639 DOI: 10.1007/7854_2022_340] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
People with Attention-Deficit Hyperactivity Disorder (ADHD) exhibit inattention, hyperactivity, and/or impulsivity. Symptoms of ADHD emerge in childhood and can continue throughout adulthood. Clinical assessments to diagnose ADHD can include administration of continuous performance tests (CPTs). CPTs provide an objective measure of inattention, requiring individuals to respond to targets (attention), and inhibit response to non-targets (impulsivity). When investigating the mechanisms of, and novel treatments for, ADHD it is important to measure such behavioral domains (attention and impulsivity). Some well-established preclinical tasks purport to assess attention in rodents but, unlike CPTs, do not require non-target inhibition, limiting their ADHD-relevance.Recently developed tasks recreate CPTs for rodents. The 5-Choice CPT (5C-CPT) contains non-target stimuli, enabling use of signal detection theory to evaluate performance, consistent with CPTs. The 5C-CPT has been adapted for use in humans, enabling direct cross-species comparisons of performance. A newer task, the rodent CPT (rCPT), is a touchscreen-based analog of CPTs, utilizing symbols instead of a simple stimulus array. Currently, the rCPT may be more akin to a go/no-go task, equally presenting targets/non-targets, although numerous variants exist - a strength. The 5C-CPT and rCPT emulate human CPTs and provide the most up-to-date information on ADHD-relevant studies for understanding attention/impulsivity.
Collapse
Affiliation(s)
- Johnny A Kenton
- Department of Psychiatry, University of California San Diego, La Jolla, CA, USA
| | - Jared W Young
- Department of Psychiatry, University of California San Diego, La Jolla, CA, USA.
- Research Service, VA San Diego Healthcare System, San Diego, CA, USA.
| |
Collapse
|
6
|
Pollock B, Harrison AG, Armstrong IT. What can we learn about performance validity from TOVA response profiles? J Clin Exp Neuropsychol 2021; 43:412-425. [PMID: 34088256 DOI: 10.1080/13803395.2021.1932762] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
Given the functional impairments associated with Attention Deficit/Hyperactivity Disorder (AD/HD), a valid diagnosis is important. However, particularly when carried out in adulthood, the diagnostic process can be challenging and is complicated by conclusive evidence that a proportion of individuals referred for evaluation of AD/HD exaggerate or feign their symptoms. Relatively few methods, however, exist to identify such feigning. While continuous performance tests (CPTs) may provide useful information regarding performance validity, the question remains as to whether there are consistent patterns of exaggeration demonstrated by those feigning AD/HD. Thus, this study used cluster analysis to determine whether valid and reliable performance clusters would emerge based on CPT performance. Using archival data from a university-based AD/HD screening clinic, we investigated the performance of 305 adults on the Test of Variables of Attention (TOVA). Three profiles emerged, including one cluster who demonstrated exceptionally low performance on the TOVA, exceptionally high reports of AD/HD symptomology, and higher rates of failure on symptom and performance validity tests. The implication from our analysis is that this group most likely represents individuals who were exaggerating or magnifying their difficulties. The results reaffirm previous research showing that performance profiles on a continuous performance test can be used as an indicator of credible performance.
Collapse
Affiliation(s)
- Beth Pollock
- Regional Assessment & Resource Centre, Queens University, Kingston, Ontario, Canada
| | - Allyson G Harrison
- Regional Assessment & Resource Centre, Queens University, Kingston, Ontario, Canada
| | - Irene T Armstrong
- Regional Assessment & Resource Centre, Queens University, Kingston, Ontario, Canada
| |
Collapse
|
7
|
Kosky KM, Lace JW, Austin TA, Seitz DJ, Clark B. The utility of the Wisconsin card sorting test, 64-card version to detect noncredible attention-deficit/hyperactivity disorder. APPLIED NEUROPSYCHOLOGY-ADULT 2020; 29:1231-1241. [DOI: 10.1080/23279095.2020.1864633] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Affiliation(s)
- Karen M. Kosky
- Department of Health Psychology, University of Missouri, Columbia, MO, USA
| | - John W. Lace
- Department of Neurology, Cleveland Clinic, Cleveland, OH, USA
| | - Tara A. Austin
- University of Texas at Austin Dell Medical School, Austin, TX, USA
| | - Dylan J. Seitz
- Department of Neurology, Indiana University School of Medicine, Indianapolis, IN, USA
| | - Brook Clark
- Department of Health Psychology, University of Missouri, Columbia, MO, USA
| |
Collapse
|