1
|
Baruzzo E, Terruzzi S, Feder B, Papagno C, Smirni D. Verbal and non-verbal recognition memory assessment: validation of a computerized version of the Recognition Memory Test. Neurol Sci 2024; 45:1979-1988. [PMID: 38129589 PMCID: PMC11021307 DOI: 10.1007/s10072-023-07171-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 10/30/2023] [Indexed: 12/23/2023]
Abstract
BACKGROUND The use of computerized devices for neuropsychological assessment (CNADs) as an effective alternative to the traditional pencil-and-paper modality has recently increased exponentially, both in clinical practice and research, especially due to the pandemic. However, several authors underline that the computerized modality requires the same psychometric validity as "in-presence" tests. The current study aimed at building and validating a computerized version of the verbal and non-verbal recognition memory test (RMT) for words, unknown faces and buildings. METHODS Seventy-two healthy Italian participants, with medium-high education and ability to proficiently use computerized systems, were enrolled. The sample was subdivided into six groups, one for each age decade. Twelve neurological patients with mixed aetiology, age and educational level were also recruited. Both the computerized and the paper-and-pencil versions of the RMT were administered in two separate sessions. RESULTS In healthy participants, the computerized and the paper-and-pencil versions of the RMT showed statistical equivalence for words, unknown faces and buildings. In the neurological patients, no statistical difference was found between the performance at the two versions of the RMT. A moderate-to-good inter-rater reliability between the two versions was also found in both samples. Finally, the computerized version of the RMT was perceived as acceptable by both healthy participants and neurological patients at System Usability Scale (SUS). CONCLUSION The computerized version of the RMT can be used as a reliable alternative to the traditional version.
Collapse
Affiliation(s)
- Elena Baruzzo
- Center for Mind/Brain Sciences-CIMeC, University of Trento, Rovereto, Italy.
| | - Stefano Terruzzi
- Center for Mind/Brain Sciences-CIMeC, University of Trento, Rovereto, Italy
| | - Beatrice Feder
- Center for Mind/Brain Sciences-CIMeC, University of Trento, Rovereto, Italy
| | - Costanza Papagno
- Center for Mind/Brain Sciences-CIMeC, University of Trento, Rovereto, Italy
| | - Daniela Smirni
- Department of Psychology, Educational Science and Human Movement, University of Palermo, Palermo, Italy
| |
Collapse
|
2
|
Roor JJ, Peters MJV, Dandachi-FitzGerald B, Ponds RWHM. Performance Validity Test Failure in the Clinical Population: A Systematic Review and Meta-Analysis of Prevalence Rates. Neuropsychol Rev 2024; 34:299-319. [PMID: 36872398 PMCID: PMC10920461 DOI: 10.1007/s11065-023-09582-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2022] [Accepted: 11/16/2022] [Indexed: 03/07/2023]
Abstract
Performance validity tests (PVTs) are used to measure the validity of the obtained neuropsychological test data. However, when an individual fails a PVT, the likelihood that failure truly reflects invalid performance (i.e., the positive predictive value) depends on the base rate in the context in which the assessment takes place. Therefore, accurate base rate information is needed to guide interpretation of PVT performance. This systematic review and meta-analysis examined the base rate of PVT failure in the clinical population (PROSPERO number: CRD42020164128). PubMed/MEDLINE, Web of Science, and PsychINFO were searched to identify articles published up to November 5, 2021. Main eligibility criteria were a clinical evaluation context and utilization of stand-alone and well-validated PVTs. Of the 457 articles scrutinized for eligibility, 47 were selected for systematic review and meta-analyses. Pooled base rate of PVT failure for all included studies was 16%, 95% CI [14, 19]. High heterogeneity existed among these studies (Cochran's Q = 697.97, p < .001; I2 = 91%; τ2 = 0.08). Subgroup analysis indicated that pooled PVT failure rates varied across clinical context, presence of external incentives, clinical diagnosis, and utilized PVT. Our findings can be used for calculating clinically applied statistics (i.e., positive and negative predictive values, and likelihood ratios) to increase the diagnostic accuracy of performance validity determination in clinical evaluation. Future research is necessary with more detailed recruitment procedures and sample descriptions to further improve the accuracy of the base rate of PVT failure in clinical practice.
Collapse
Affiliation(s)
- Jeroen J Roor
- Department of Medical Psychology, VieCuri Medical Center, Venlo, The Netherlands.
- School for Mental Health and Neuroscience, Maastricht University, Maastricht, The Netherlands.
| | - Maarten J V Peters
- Department of Clinical Psychological Science, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, The Netherlands
| | - Brechje Dandachi-FitzGerald
- Department of Clinical Psychological Science, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, The Netherlands
- Faculty of Psychology, Open University, Heerlen, The Netherlands
| | - Rudolf W H M Ponds
- School for Mental Health and Neuroscience, Maastricht University, Maastricht, The Netherlands
- Department of Medical Psychology, Amsterdam University Medical Centres, location VU, Amsterdam, The Netherlands
| |
Collapse
|
3
|
Rohling ML, Demakis GJ, Langhinrichsen-Rohling J. Lowered cutoffs to reduce false positives on the Word Memory Test. J Clin Exp Neuropsychol 2024; 46:67-79. [PMID: 38362939 DOI: 10.1080/13803395.2024.2314736] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2023] [Accepted: 01/11/2024] [Indexed: 02/17/2024]
Abstract
OBJECTIVE To adjust the decision criterion for the Word Memory Test (WMT, Green, 2003) to minimize the frequency of false positives. METHOD Archival data were combined into a database (n = 3,210) to examine the best cut score for the WMT. We compared results based on the original scoring rules and those based on adjusted scoring rules using a criterion based on 16 performance validity tests (PVTs) exclusive of the WMT. Cutoffs based on peer-reviewed publications and test manuals were used. The resulting PVT composite was considered the best estimate of validity status. We focused on a specificity of .90 with a false-positive rate of less than .10 across multiple samples. RESULTS Each examinee was administered the WMT, as well as on average 5.5 (SD = 2.5) other PVTs. Based on the original scoring rules of the WMT, 31.8% of examinees failed. Using a single failure on the criterion PVT (C-PVT), the base rate of failure was 45.9%. When requiring two or more failures on the C-PVT, the failure rate dropped to 22.8%. Applying a contingency analysis (i.e., X2) to the two failures model on the C-PVT measure and using the original rules for the WMT resulted in only 65.3% agreement. However, using our adjusted rules for the WMT, which consisted of relying on only the IR and DR WMT subtest scores with a cutoff of 77.5%, agreement between the adjusted and the C-PVT criterion equaled 80.8%, for an improvement of 12.1% identified. The adjustmeny resulted in a 49.2% reduction in false positives while preserving a sensitivity of 53.6%. The specificity for the new rules was 88.8%, for a false positive rate of 11.2%. CONCLUSIONS Results supported lowering of the cut score for correct responding from 82.5% to 77.5% correct. We also recommend discontinuing the use of the Consistency subtest score in the determination of WMT failure.
Collapse
|
4
|
Laera G, Hering A, Kliegel M. Assessing time-based prospective memory online: A comparison study between laboratory-based and web-based testing. Q J Exp Psychol (Hove) 2024:17470218231220578. [PMID: 38053325 DOI: 10.1177/17470218231220578] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
Prospective memory (PM, i.e., the ability to remember and perform future intentions) is assessed mainly within laboratory settings; however, in the last two decades, several studies have started testing PM online. Most part of those studies focused on event-based PM (EBPM), and only a few assessed time-based PM (TBPM), possibly because time keeping is difficult to control or standardise without experimental control. Thus, it is still unclear whether time monitoring patterns in online studies replicate typical patterns obtained in laboratory tasks. In this study, we therefore aimed to investigate whether the behavioural outcome measures obtained from the traditional TBPM paradigm in the laboratory-accuracy and time monitoring-are comparable with an online version in a sample of 101 younger adults. Results showed no significant difference in TBPM performance in the laboratory versus online setting, as well as no difference in time monitoring. However, we found that participants were somewhat faster and more accurate at the ongoing task during the laboratory assessment, but those differences were not related to holding an intention in mind. The findings suggest that, although participants seemed generally more distracted when tested remotely, online assessment yielded similar results in key temporal characteristics and behavioural performance as for the laboratory assessment. The results are discussed in terms of possible conceptual and methodological implications for online testing.
Collapse
Affiliation(s)
- Gianvito Laera
- Cognitive Aging Lab (CAL), Faculty of Psychology and Educational Sciences, University of Geneva, Geneva, Switzerland
- Centre for the Interdisciplinary Study of Gerontology and Vulnerability, University of Geneva, Geneva, Switzerland
- LIVES-Overcoming Vulnerability: Life Course Perspectives, Swiss National Centre of Competence in Research, Lausanne, Switzerland
| | - Alexandra Hering
- Cognitive Aging Lab (CAL), Faculty of Psychology and Educational Sciences, University of Geneva, Geneva, Switzerland
- Centre for the Interdisciplinary Study of Gerontology and Vulnerability, University of Geneva, Geneva, Switzerland
- Department of Developmental Psychology, Tilburg School for Social and Behavioral Sciences, Tilburg University, Tilburg, The Netherlands
| | - Matthias Kliegel
- Cognitive Aging Lab (CAL), Faculty of Psychology and Educational Sciences, University of Geneva, Geneva, Switzerland
- Centre for the Interdisciplinary Study of Gerontology and Vulnerability, University of Geneva, Geneva, Switzerland
- LIVES-Overcoming Vulnerability: Life Course Perspectives, Swiss National Centre of Competence in Research, Lausanne, Switzerland
| |
Collapse
|
5
|
Jiang L, Xu M, Xia S, Zhu J, Zhou Q, Xu L, Shi C, Wu D. Reliability and validity of the electronic version of the Hopkins verbal learning test-revised in middle-aged and elderly Chinese people. Front Aging Neurosci 2023; 15:1124731. [PMID: 37377673 PMCID: PMC10292015 DOI: 10.3389/fnagi.2023.1124731] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2023] [Accepted: 05/16/2023] [Indexed: 06/29/2023] Open
Abstract
Background The aging population is increasing, making it essential to have a standardized, convenient, and valid electronic memory test that can be accessed online for older people and caregivers. The electronic version of the Hopkins Verbal Learning Test-Revised (HVLT-R) as a test with these advantages and its reliability and validity has not yet been tested. Thus, this study examined the reliability and validity of the electronic version of the HVLT-R in middle-aged and elderly Chinese people to provide a scientific basis for its future dissemination and use. Methods We included 1,925 healthy participants aged over 40, among whom 38 were retested after 3-6 months. In addition, 65 participants completed both the pad and paper-and-pencil versions of the HVLT-R (PAP-HVLT-R). We also recruited 42 Alzheimer's disease (AD) patients, and 45 amnestic mild cognitive impairment (aMCI) patients. All participants completed the Pad-HVLT-R, the Hong Kong Brief Cognitive Test (HKBC), the Brief Visual Memory Test-Revised (BVMT-R), and the Logical Memory Test (LM). Results (1) Reliability: the Cronbach's α value was 0.94, the split-half reliability was 0.96. The test-retest correlation coefficients were moderate, ranging from 0.38 to 0.65 for direct variables and 0.16 to 0.52 for derived variables; (2) Concurrent validity: the Pad-HVLT-R showed a moderate correlation with the HKBC and BVMT-R, with correlation coefficients between total recall of 0.41 and 0.54, and between long-delayed recall of 0.42 and 0.59, respectively. It also showed a high correlation with the LM, with correlation coefficients of 0.72 for total recall and 0.62 for long-delayed recall; (3) Convergent validity: the Pad-HVLT-R was moderately correlated with the PAP version, with correlation coefficients ranging from 0.29 to 0.53 for direct variables and 0.15 to 0.43 for derived variables; (4) Discriminant capacity: the Pad-HVLT-R was effective in differentiating AD patients, as demonstrated by the ROC analysis with AUC values of 0.834 and 0.934 for total recall and long-delayed recall, respectively. Conclusion (1) The electronic version of HVLT-R has good reliability and validity in middle-aged and elderly Chinese people; (2) The electronic version of HVLT-R can be used as an effective tool to distinguish AD patients from healthy people.
Collapse
Affiliation(s)
- Lichen Jiang
- Medical Psychological Center, The Second Xiangya Hospital, Central South University, Changsha, Hunan, China
| | - Ming Xu
- Medical Psychological Center, The Second Xiangya Hospital, Central South University, Changsha, Hunan, China
| | - Shunyao Xia
- Institute of Mental Health, Peking University Sixth Hospital, Peking University, Beijing, China
- NHC Key Laboratory for Mental Health, National Clinical Research Center for Mental Disorders, Peking University Sixth Hospital, Peking University, Beijing, China
| | - Jiahui Zhu
- Institute of Mental Health, Peking University Sixth Hospital, Peking University, Beijing, China
- NHC Key Laboratory for Mental Health, National Clinical Research Center for Mental Disorders, Peking University Sixth Hospital, Peking University, Beijing, China
| | - Qi Zhou
- Institute of Mental Health, Peking University Sixth Hospital, Peking University, Beijing, China
- NHC Key Laboratory for Mental Health, National Clinical Research Center for Mental Disorders, Peking University Sixth Hospital, Peking University, Beijing, China
| | - Luoyi Xu
- Department of Psychiatry, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, Zhejiang, China
| | - Chuan Shi
- Institute of Mental Health, Peking University Sixth Hospital, Peking University, Beijing, China
- NHC Key Laboratory for Mental Health, National Clinical Research Center for Mental Disorders, Peking University Sixth Hospital, Peking University, Beijing, China
| | - Daxing Wu
- Medical Psychological Center, The Second Xiangya Hospital, Central South University, Changsha, Hunan, China
- Medical Psychological Institute, Central South University, Changsha, Hunan, China
- National Clinical Research Center for Mental Disorders, Changsha, Hunan, China
| |
Collapse
|
6
|
Martínez-Pernía D, Olavarría L, Fernández-Manjón B, Cabello V, Henríquez F, Robert P, Alvarado L, Barría S, Antivilo A, Velasquez J, Cerda M, Farías G, Torralva T, Ibáñez A, Parra MA, Gilbert S, Slachevsky A. The limitations and challenges in the assessment of executive dysfunction associated with real-world functioning: The opportunity of serious games. APPLIED NEUROPSYCHOLOGY. ADULT 2023:1-17. [PMID: 36827177 PMCID: PMC11177293 DOI: 10.1080/23279095.2023.2174438] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Abstract
Nowadays, there is a broad range of methods for detecting and evaluating executive dysfunction ranging from clinical interview to neuropsychological evaluation. Nevertheless, a critical issue of these assessments is the lack of correspondence of the neuropsychological test's results with real-world functioning. This paper proposes serious games as a new framework to improve the neuropsychological assessment of real-world functioning. We briefly discuss the contribution and limitations of current methods of evaluation of executive dysfunction (paper-and-pencil tests, naturalistic observation methods, and Information and Communications Technologies) to inform on daily life functioning. Then, we analyze what are the limitations of these methods to predict real-world performance: (1) A lack of appropriate instruments to investigate the complexity of real-world functioning, (2) the vast majority of neuropsychological tests assess well-structured tasks, and (3) measurement of behaviors are based on simplistic data collection and statistical analysis. This work shows how serious games offer an opportunity to develop more efficient tools to detect executive dysfunction in everyday life contexts. Serious games provide meaningful narrative stories and virtual or real environments that immerse the user in natural and social environments with social interactions. In those highly interactive game environments, the player needs to adapt his/her behavioral performance to novel and ill-structured tasks which are suited for collecting user interaction evidence. Serious games offer a novel opportunity to develop better tools to improve diagnosis of the executive dysfunction in everyday life contexts. However, more research is still needed to implement serious games in everyday clinical practice.
Collapse
Affiliation(s)
- David Martínez-Pernía
- Center for Social and Cognitive Neuroscience (CSCN), School of Psychology, Universidad Adolfo Ibáñez, Santiago, Chile
- Geroscience Center for Brain Health and Metabolism (GERO), Santiago, Chile
- Memory and Neuropsychiatric Center (CMYN), Memory Unit - Neurology Department, Hospital del Salvador and Faculty of Medicine, University of Chile, Santiago, Chile
| | - Loreto Olavarría
- Memory and Neuropsychiatric Center (CMYN), Memory Unit - Neurology Department, Hospital del Salvador and Faculty of Medicine, University of Chile, Santiago, Chile
| | | | - Victoria Cabello
- Neuropsychology and Clinical Neuroscience Laboratory (LANNEC), Physiopathology Department - Biomedical Science Institute, Neuroscience and East Neuroscience Departments, Faculty of Medicine, University of Chile, Santiago, Chile
| | - Fernando Henríquez
- Geroscience Center for Brain Health and Metabolism (GERO), Santiago, Chile
- Memory and Neuropsychiatric Center (CMYN), Memory Unit - Neurology Department, Hospital del Salvador and Faculty of Medicine, University of Chile, Santiago, Chile
- Neuropsychology and Clinical Neuroscience Laboratory (LANNEC), Physiopathology Department - Biomedical Science Institute, Neuroscience and East Neuroscience Departments, Faculty of Medicine, University of Chile, Santiago, Chile
- Laboratory for Cognitive and Evolutionary Neuroscience (LaNCE), Department of Psychiatry, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Philippe Robert
- Cognition Behavior Technology (CoBTeK) Lab, FRIS-Université Côte d'Azur, Nice, France
| | - Luís Alvarado
- Departamento de Psiquiatría y Salud Mental Norte, Universidad de Chile, Santiago, Chile
| | - Silvia Barría
- Departamento de Ciencias Neurologicas Oriente, Facultad de Medicina, Universidad de Chile, and Servicio de Neurología, Hospital del Salvador, Santiago, Chile
| | - Andrés Antivilo
- Departamento de Ciencias Neurologicas Oriente, Facultad de Medicina, Universidad de Chile, and Servicio de Neurología, Hospital del Salvador, Santiago, Chile
| | - Juan Velasquez
- Facultad de Ciencias Físicas y Matemáticas, Web Intelligence Center, Universidad de Chile, Santiago, Chile
- Department of Industrial Engineering, Faculty of Physical and Mathematical Sciences, Instituto Sistemas Complejos de Ingeniería (ISCI), University of Chile, Santiago, Chile
| | - Mauricio Cerda
- Integrative Biology Program, Institute of Biomedical Sciences, and Center for Medical Informatics and Telemedicine, Faculty of Medicine, and Biomedical Neuroscience Institute, Facultad de Medicina, Universidad de Chile, Santiago, Chile
| | - Gonzalo Farías
- Department of Neurology North, Faculty of Medicine, University of Chile, Santiago, Chile
- Center for advanced clinical research (CICA), Hospital Clínico Universidad de Chile, Chile
| | - Teresa Torralva
- Institute of Cognitive and Translational Neuroscience (INCYT), Instituto de Neurología Cognitiva Foundation, Favaloro University, Buenos Aires, Argentina
| | - Agustín Ibáñez
- Latin American Brain Health Institute (BrainLat), Universidad Adolfo Ibáñez, Santiago, Chile
- Cognitive Neuroscience Center (CNC), Universidad de San Andrés, National Scientific and Technical Research Council (CONICET), Buenos Aires, Argentina
- Global Brain Health Institute, University of California, San Francisco, San Francisco, CA, USA
- Trinity College Dublin (TCD), Dublin, Ireland
| | - Mario A Parra
- School of Psychological Sciences and Health, University of Strathclyde, Glasgow, UK
| | - Sam Gilbert
- Institute of Cognitive Neuroscience, University College London, London, UK
| | - Andrea Slachevsky
- Geroscience Center for Brain Health and Metabolism (GERO), Santiago, Chile
- Memory and Neuropsychiatric Center (CMYN), Memory Unit - Neurology Department, Hospital del Salvador and Faculty of Medicine, University of Chile, Santiago, Chile
- Neuropsychology and Clinical Neuroscience Laboratory (LANNEC), Physiopathology Department - Biomedical Science Institute, Neuroscience and East Neuroscience Departments, Faculty of Medicine, University of Chile, Santiago, Chile
- Department of Neurology and Psychiatry, Clínica Alemana-Universidad del Desarrollo, Santiago, Chile
| |
Collapse
|
7
|
Device-based physical activity and late-life depressive symptoms: An analysis of influential factors using share data. J Affect Disord 2023; 322:267-272. [PMID: 36375542 DOI: 10.1016/j.jad.2022.11.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Revised: 08/25/2022] [Accepted: 11/06/2022] [Indexed: 11/13/2022]
Abstract
BACKGROUND Regular physical activity is effective for preventing and managing depression; however, only a few studies have assessed physical activity using device-based measures. We aimed to examine the association between device-based total physical activity and late-life depressive symptoms and explore which factors may explain this relationship. METHODS Data from 10 European countries who participated in wave 8 of the Survey of Health, Ageing and Retirement in Europe (SHARE) were analyzed. Triaxial accelerometers (Axivity AX3; Axivity, Newcastle UK) were used to assess total physical activity. Depressive symptoms were assessed through the EURO-D scale. Possion regression models and mediation analysis were performed. RESULTS The final sample included 785 older adults (70 participants had missing data in one or more variables and were excluded from the analysis) [mean (SD): age 68.6 (8.8) years; 59.2 % female]. After adjusting for several confounders, a 10 % increase in the mean acceleration was associated with lower depressive symptoms (B = -0.0228; 95%CI = -0.0395, -0.0061). This association was partly explained by limitations in activities of daily living and quality of life. LIMITATIONS Cross-sectional design, convenience sample, bi-directionality, lack of cutoff points for classification of the Axivity AX3 placed in the thigh, into intensities. CONCLUSIONS Higher device-based total physical activity was linked with lower depressive symptoms in older European adults. Future studies employing device-based measures that allow to better capture important physical activity dimensions (i.e., intensity) will better inform about dose-response relationships.
Collapse
|
8
|
Using Technology for the Efficient and Precise Assessment of Cognitive Skills in Countries with Limited Standardized Assessment Instruments: A Report on the Case of Saudi Arabia. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12031617] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
In Saudi Arabia, the country’s progress toward appropriate and inclusive education programs for children with disabilities is still evolving. A crucial aspect of this evolution has been the development of a comprehensive assessment battery that covers a broad range of cognitive factors for the diagnosis of neurodevelopment disorders and other types of intellectual atypicalities, including giftedness. The Alif–Ya Assessment Battery consists of 47 subtests based on various theories of intelligence. Alif–Ya capitalizes on advanced technologies to enable its delivery either in person or remotely. Moreover, over half of Alif–Ya’s subtests are adaptive; items are selected for the test takers based on their previous responses. In this paper, we provide an overview of the Alif–Ya Assessment Battery, describe how it was designed to make the best use of the latest and best features of technology for the appropriate and accurate assessment of children and adolescents in the Kingdom of Saudi Arabia via remote or in-person administration, and present initial data collected with the battery.
Collapse
|
9
|
Leong V, Raheel K, Sim JY, Kacker K, Karlaftis VM, Vassiliu C, Kalaivanan K, Chen SHA, Robbins TW, Sahakian BJ, Kourtzi Z. A New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High-Quality Data: Development and Usability Study. J Med Internet Res 2022; 24:e28368. [PMID: 34989691 PMCID: PMC8778570 DOI: 10.2196/28368] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2021] [Revised: 06/25/2021] [Accepted: 07/27/2021] [Indexed: 01/06/2023] Open
Abstract
BACKGROUND The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the forefront as a promising solution for the rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. This study examines the opportunities and challenges afforded by the societal shift toward web-based testing and highlights an urgent need to establish a standard data quality assurance framework for online studies. OBJECTIVE This study aims to develop and validate a new supervised online testing methodology, remote guided testing (RGT). METHODS A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory, and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (n=41) or online using remote guided testing (n=44) and delivered using identical web-based platforms (Cambridge Neuropsychological Test Automated Battery, Inquisit, and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, and response times) and overall task performance measures. RESULTS The results indicated that, across all data quality and performance measures, RGT data was statistically-equivalent to in-person data collected in the lab (P>.40 for all comparisons). Moreover, RGT participants out-performed the lab group on measured verbal intelligence (P<.001), which could reflect test environment differences, including possible effects of mask-wearing on communication. CONCLUSIONS These data suggest that the RGT methodology could help ameliorate concerns regarding online data quality-particularly for studies involving high-risk or rare cohorts-and offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.
Collapse
Affiliation(s)
- Victoria Leong
- Psychology, School of Social Sciences, Nanyang Technological University, Singapore, Singapore
- Centre for Research and Development in Learning, Nanyang Technological University, Singapore, Singapore
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Kausar Raheel
- Psychology, School of Social Sciences, Nanyang Technological University, Singapore, Singapore
| | - Jia Yi Sim
- Psychology, School of Social Sciences, Nanyang Technological University, Singapore, Singapore
| | - Kriti Kacker
- Psychology, School of Social Sciences, Nanyang Technological University, Singapore, Singapore
| | - Vasilis M Karlaftis
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
| | - Chrysoula Vassiliu
- Faculty of Modern and Medieval Languages and Linguistics, University of Cambridge, Cambridge, United Kingdom
| | - Kastoori Kalaivanan
- Centre for Research and Development in Learning, Nanyang Technological University, Singapore, Singapore
| | - S H Annabel Chen
- Psychology, School of Social Sciences, Nanyang Technological University, Singapore, Singapore
- Centre for Research and Development in Learning, Nanyang Technological University, Singapore, Singapore
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
- National Institute of Education, Nanyang Technological University, Singapore, Singapore
| | - Trevor W Robbins
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, United Kingdom
| | - Barbara J Sahakian
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, United Kingdom
- Department of Psychiatry, University of Cambridge, Cambridge, United Kingdom
| | - Zoe Kourtzi
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
| |
Collapse
|
10
|
McWhirter L, Ritchie CW, Stone J, Carson A. Performance validity test failure in clinical populations-a systematic review. J Neurol Neurosurg Psychiatry 2020; 91:945-952. [PMID: 32651247 DOI: 10.1136/jnnp-2020-323776] [Citation(s) in RCA: 45] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/08/2020] [Revised: 06/03/2020] [Accepted: 06/07/2020] [Indexed: 12/27/2022]
Abstract
Performance validity tests (PVTs) are widely used in attempts to quantify effort and/or detect negative response bias during neuropsychological testing. However, it can be challenging to interpret the meaning of poor PVT performance in a clinical context. Compensation-seeking populations predominate in the PVT literature. We aimed to establish base rates of PVT failure in clinical populations without known external motivation to underperform. We searched MEDLINE, EMBASE and PsycINFO for studies reporting PVT failure rates in adults with defined clinical diagnoses, excluding studies of active or veteran military personnel, forensic populations or studies of participants known to be litigating or seeking disability benefits. Results were summarised by diagnostic group and implications discussed. Our review identified 69 studies, and 45 different PVTs or indices, in clinical populations with intellectual disability, degenerative brain disease, brain injury, psychiatric disorders, functional disorders and epilepsy. Various pass/fail cut-off scores were described. PVT failure was common in all clinical groups described, with failure rates for some groups and tests exceeding 25%. PVT failure is common across a range of clinical conditions, even in the absence of obvious incentive to underperform. Failure rates are no higher in functional disorders than in other clinical conditions. As PVT failure indicates invalidity of other attempted neuropsychological tests, the finding of frequent and unexpected failure in a range of clinical conditions raises important questions about the degree of objectivity afforded to neuropsychological tests in clinical practice and research.
Collapse
Affiliation(s)
- Laura McWhirter
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Craig W Ritchie
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Jon Stone
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Alan Carson
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| |
Collapse
|
11
|
Backx R, Skirrow C, Dente P, Barnett JH, Cormack FK. Comparing Web-Based and Lab-Based Cognitive Assessment Using the Cambridge Neuropsychological Test Automated Battery: A Within-Subjects Counterbalanced Study. J Med Internet Res 2020; 22:e16792. [PMID: 32749999 PMCID: PMC7435628 DOI: 10.2196/16792] [Citation(s) in RCA: 57] [Impact Index Per Article: 14.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2019] [Revised: 05/15/2020] [Accepted: 06/11/2020] [Indexed: 12/22/2022] Open
Abstract
Background Computerized assessments are already used to derive accurate and reliable measures of cognitive function. Web-based cognitive assessment could improve the accessibility and flexibility of research and clinical assessment, widen participation, and promote research recruitment while simultaneously reducing costs. However, differences in context may influence task performance. Objective This study aims to determine the comparability of an unsupervised, web-based administration of the Cambridge Neuropsychological Test Automated Battery (CANTAB) against a typical in-person lab-based assessment, using a within-subjects counterbalanced design. The study aims to test (1) reliability, quantifying the relationship between measurements across settings using correlational approaches; (2) equivalence, the extent to which test results in different settings produce similar overall results; and (3) agreement, by quantifying acceptable limits to bias and differences between measurement environments. Methods A total of 51 healthy adults (32 women and 19 men; mean age 36.8, SD 15.6 years) completed 2 testing sessions, which were completed on average 1 week apart (SD 4.5 days). Assessments included equivalent tests of emotion recognition (emotion recognition task [ERT]), visual recognition (pattern recognition memory [PRM]), episodic memory (paired associate learning [PAL]), working memory and spatial planning (spatial working memory [SWM] and one touch stockings of Cambridge), and sustained attention (rapid visual information processing [RVP]). Participants were randomly allocated to one of the two groups, either assessed in-person in the laboratory first (n=33) or with unsupervised web-based assessments on their personal computing systems first (n=18). Performance indices (errors, correct trials, and response sensitivity) and median reaction times were extracted. Intraclass and bivariate correlations examined intersetting reliability, linear mixed models and Bayesian paired sample t tests tested for equivalence, and Bland-Altman plots examined agreement. Results Intraclass correlation (ICC) coefficients ranged from ρ=0.23-0.67, with high correlations in 3 performance indices (from PAL, SWM, and RVP tasks; ρ≥0.60). High ICC values were also seen for reaction time measures from 2 tasks (PRM and ERT tasks; ρ≥0.60). However, reaction times were slower during web-based assessments, which undermined both equivalence and agreement for reaction time measures. Performance indices did not differ between assessment settings and generally showed satisfactory agreement. Conclusions Our findings support the comparability of CANTAB performance indices (errors, correct trials, and response sensitivity) in unsupervised, web-based assessments with in-person and laboratory tests. Reaction times are not as easily translatable from in-person to web-based testing, likely due to variations in computer hardware. The results underline the importance of examining more than one index to ascertain comparability, as high correlations can present in the context of systematic differences, which are a product of differences between measurement environments. Further work is now needed to examine web-based assessments in clinical populations and in larger samples to improve sensitivity for detecting subtler differences between test settings.
Collapse
Affiliation(s)
- Rosa Backx
- Cambridge Cognition Ltd, Cambridge, United Kingdom
| | - Caroline Skirrow
- Cambridge Cognition Ltd, Cambridge, United Kingdom.,School of Psychological Science, University of Bristol, Bristol, United Kingdom
| | | | - Jennifer H Barnett
- Cambridge Cognition Ltd, Cambridge, United Kingdom.,Department of Psychiatry, University of Cambridge, Cambridge, United Kingdom
| | | |
Collapse
|
12
|
Bracken MR, Mazur-Mosiewicz A, Glazek K. Trail Making Test: Comparison of paper-and-pencil and electronic versions. APPLIED NEUROPSYCHOLOGY-ADULT 2018; 26:522-532. [DOI: 10.1080/23279095.2018.1460371] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Affiliation(s)
- Magdalene R. Bracken
- Department of Psychiatry, Allegheny General Hospital, Pittsburgh, Pennsylvania, USA
| | - Anya Mazur-Mosiewicz
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University, Tulsa, Oklahoma, USA
- The Chicago School of Professional Psychology, Los Angeles, California, USA
| | - Kuba Glazek
- The Chicago School of Professional Psychology, Los Angeles, California, USA
| |
Collapse
|
13
|
Abstract
BACKGROUND Physical activity and body mass predict cognition in the elderly. However, mixed evidence suggests that obesity is associated with poorer cognition, while also protecting against cognitive decline in older age. PURPOSE We investigated whether body mass independently predicted cognition in older age and whether these associations changed over time. METHODS A latent curve structural equation modeling approach was used to analyze data from a sample of aging adults (N = 8442) split into two independent subsamples, collected over 6 years. RESULTS Lower baseline Body Mass Index (BMI) and higher physical activity independently predicted greater baseline cognition (p < 0.001). Decreases in BMI and physical activity independently predicted greater decline in the slope of cognition (p < 0.001). CONCLUSIONS Our results support the obesity paradox in cognitive aging, with lower baseline body mass predicting better cognition, but less decline over time protecting against cognitive decline. We discuss how weight loss in the elderly may serve as a useful indicator of co-occurring cognitive decline, and we discuss implications for health care professionals.
Collapse
|
14
|
Binder LM, Chafetz MD. [Formula: see text]Determination of the smoking gun of intent: significance testing of forced choice results in social security claimants. Clin Neuropsychol 2017; 32:132-144. [PMID: 28617092 DOI: 10.1080/13854046.2017.1337931] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
OBJECTIVE Significantly below-chance findings on forced choice tests have been described as revealing "the smoking gun of intent" that proved malingering. The issues of probability levels, one-tailed vs. two-tailed tests, and the combining of PVT scores on significantly below-chance findings were addressed in a previous study, with a recommendation of a probability level of .20 to test the significance of below-chance results. The purpose of the present study was to determine the rate of below-chance findings in a Social Security Disability claimant sample using the previous recommendations. METHOD We compared the frequency of below-chance results on forced choice performance validity tests (PVTs) at two levels of significance, .05 and .20, and when using significance testing on individual subtests of the PVTs compared with total scores in claimants for Social Security Disability in order to determine the rate of the expected increase. RESULTS The frequency of significant results increased with the higher level of significance for each subtest of the PVT and when combining individual test sections to increase the number of test items, with up to 20% of claimants showing significantly below-chance results at the higher p-value. CONCLUSIONS These findings are discussed in light of Social Security Administration policy, showing an impact on policy issues concerning child abuse and neglect, and the importance of using these techniques in evaluations for Social Security Disability.
Collapse
|
15
|
Bourassa K, Sbarra DA. Body mass and cognitive decline are indirectly associated via inflammation among aging adults. Brain Behav Immun 2017; 60:63-70. [PMID: 27658542 DOI: 10.1016/j.bbi.2016.09.023] [Citation(s) in RCA: 37] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/16/2016] [Revised: 08/30/2016] [Accepted: 09/19/2016] [Indexed: 11/26/2022] Open
Abstract
Inflammatory models of neurodegeneration suggest that higher circulating levels of inflammation can lead to cognitive decline. Despite established independent associations between greater body mass, increased inflammation, and cognitive decline, no prior research has explored whether markers of systemic inflammation might mediate the association between body mass and changes in cognitive functioning. To test such a model, we used two longitudinal subsamples (ns=9066; 12,561) of aging adults from the English Longitudinal Study of Ageing (ELSA) study, which included two cognitive measures components of memory and executive functioning, as well as measurements of body mass and systemic inflammation, assessed via C-reactive protein (CRP). Greater body mass was indirectly associated with declines in memory and executive functioning over 6years via relatively higher levels of CRP. Our results suggest that systemic inflammation is one biologically plausible mechanism through which differences in body mass might influence changes in cognitive functioning among aging adults.
Collapse
Affiliation(s)
- Kyle Bourassa
- Department of Psychology, University of Arizona, 1503 E. University Blvd., Bldg #68, Rm. 312., Tucson, AZ 85721-0068, United States.
| | - David A Sbarra
- Department of Psychology, University of Arizona, 1503 E. University Blvd., Bldg #68, Rm. 312., Tucson, AZ 85721-0068, United States
| |
Collapse
|
16
|
Bourassa KJ, Memel M, Woolverton C, Sbarra DA. Social participation predicts cognitive functioning in aging adults over time: comparisons with physical health, depression, and physical activity. Aging Ment Health 2017; 21:133-146. [PMID: 26327492 DOI: 10.1080/13607863.2015.1081152] [Citation(s) in RCA: 141] [Impact Index Per Article: 20.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
OBJECTIVES Several risk and protective factors are associated with changes in cognitive functioning in aging adults - including physical health, depression, physical activity, and social activities - though the findings for participation in social activities are mixed. This study investigated the longitudinal association between social participation and two domains of cognitive functioning, memory and executive function. A primary goal of our analyses was to determine whether social participation predicted cognitive functioning over-and-above physical health, depression, and physical activity in a sample with adequate power to detect unique effects. METHOD The sample included aging adults (N = 19,832) who participated in a large, multi-national study and provided data across six years; split into two random subsamples. Unique associations between the predictors of interest and cognitive functioning over time and within occasion were assessed in a latent curve growth model. RESULTS Social participation predicted both domains of cognitive functioning at each occasion, and the relative magnitude of this effect was comparable to physical health, depression, and physical activity level. In addition, social participation at the first time point predicted change in cognitive functioning over time. The substantive results in the initial sample were replicated in the second independent subsample. CONCLUSION Overall, the magnitude of the association of social participation is comparable to other well-established predictors of cognitive functioning, providing evidence that social participation plays an important role in cognitive functioning and successful aging.
Collapse
Affiliation(s)
- Kyle J Bourassa
- a Department of Psychology , University of Arizona , Tucson , AZ , USA
| | - Molly Memel
- a Department of Psychology , University of Arizona , Tucson , AZ , USA
| | - Cindy Woolverton
- a Department of Psychology , University of Arizona , Tucson , AZ , USA
| | - David A Sbarra
- a Department of Psychology , University of Arizona , Tucson , AZ , USA
| |
Collapse
|
17
|
Edwards J, Vess J, Reger G, Cernich A. The use of virtual reality in the military's assessment of service members with traumatic brain injury: recent developments and emerging opportunities. APPLIED NEUROPSYCHOLOGY-ADULT 2013; 21:220-30. [PMID: 25084846 DOI: 10.1080/09084282.2013.796554] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Abstract
Traumatic brain injury (TBI) is a common event in the current extended conflicts by American service members, with estimates that as many as 300,000 have sustained combat-related concussions during Operation Iraqi Freedom and Operation Enduring Freedom. The limited ecological validity of traditional neuropsychological assessment measures presents a challenge to effective postconcussion evaluation of service members in relation to fitness-for-duty decisions or rehabilitation needs. Virtual reality (VR) technology offers a promising opportunity to advance the field of functional assessment for TBI. This article reviews the current professional literature on VR applications for TBI assessment, with special emphasis on those that are particularly relevant to U.S. service members. VR affords several advantages for clinical use. These include assessment of complex sets of cognitive and behavioral functions rather than the isolated components assessed by traditional measures; more precise control over the standardized presentation of task stimuli and the recording of response data; and enhanced ecological validity that can lead to more useful assessment data in the applied contexts faced by the U.S. military.
Collapse
Affiliation(s)
- Joe Edwards
- a Department of Behavioral Health , TBI Clinic , Ft. Wainwright , Alaska
| | | | | | | |
Collapse
|
18
|
Williamson DJ, Holsman M, Chaytor N, Miller JW, Drane DL. Abuse, Not Financial Incentive, Predicts Non-Credible Cognitive Performance in Patients With Psychogenic Non-Epileptic Seizures. Clin Neuropsychol 2012; 26:588-98. [DOI: 10.1080/13854046.2012.670266] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
|
19
|
Young JC, Sawyer RJ, Roper BL, Baughman BC. Expansion and Re-examination of Digit Span Effort Indices on the WAIS-IV. Clin Neuropsychol 2012; 26:147-59. [DOI: 10.1080/13854046.2011.647083] [Citation(s) in RCA: 48] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/14/2022]
|