Comparison of PC and iPad administrations of the Cogstate Brief Battery in the Mayo Clinic Study of Aging: Assessing cross-modality equivalence of computerized neuropsychological tests

Mayo Normative Studies: regression-based normative data for remote self-administration of the Stricker Learning Span, Symbols Test and Mayo Test Drive Screening Battery Composite and validation in individuals with Mild Cognitive Impairment and dementia

Objective

Few normative data for unsupervised, remotely-administered computerized cognitive measures are available. We examined variables to include in normative models for Mayo Test Drive (a multi-device remote cognitive assessment platform) measures, developed normative data, and validated the norms.

Method

1240 Cognitively Unimpaired (CU) adults ages 32-100-years (96% white) from the Mayo Clinic Study of Aging and Mayo Alzheimer's Disease Research Center with Clinical Dementia Rating® of 0 were included. We converted raw scores to normalized scaled scores and derived regression-based normative data adjusting for age, age2, sex and education (base model); alternative norms are also provided (age+age2+sex; age+age2). We assessed additional terms using an a priori cut-off of 1% variance improvement above the base model. We examined low test performance rates (<-1 standard deviation) in independent validation samples (n=167 CU, n=64 mild cognitive impairment (MCI), n=14 dementia). Rates were significantly different when 95% confidence intervals (CI) did not include the expected 14.7% base rate.

Results

No model terms met the a priori cut-off beyond the base model, including device type, response input source (e.g., mouse, etc.) or session interference. Norms showed expected low performance rates in CU and greater rates of low performance in MCI and dementia in independent validation samples.

Conclusion

Typical normative models appear appropriate for remote self-administered MTD measures and are sensitive to cognitive impairment. Device type and response input source did not explain enough variance for inclusion in normative models but are important for individual-level interpretation. Future work will increase inclusion of individuals from under-represented groups.

Accuracy of Reaction Time Measurement on Automated Neuropsychological Assessment Metric UltraMobile

OBJECTIVE

This observational study examined the accuracy of simple reaction time (RT) measurements on various touchscreen tablet devices using the Automated Neuropsychological Assessment Metric (ANAM) UltraMobile test battery. The study investigated the implications of interpreting ANAM UltraMobile with laptop-based normative data by analyzing the magnitude and variability of RT accuracy across devices.

METHOD

RT accuracy on 10 different tablets was assessed using a photodetector and robotic arm to respond to stimuli at predetermined response times. The recorded RT was compared with the true RT obtained from the robotic arm to calculate the RT error.

RESULTS

ANAM UltraMobile recorded slower RTs than the laptop version. Additionally, RT error varied considerably among the 10 tablet models, suggesting psychometrically significant implications that could lead to interpretive errors when using laptop-based normative data.

CONCLUSIONS

Relative to the RT error from the laptop-based version of ANAM, tablet data from ANAM UltraMobile are significantly slower and exhibit large variability between devices. These differences may have clinically significant implications for the comparability of the two versions. The findings suggest that further research with human participants is needed to assess the equivalence of ANAM UltraMobile with its predecessor.

The Real-World Usability, Feasibility, and Performance Distributions of Deploying a Digital Toolbox of Computerized Assessments to Remotely Evaluate Brain Health: Development and Usability Study

BACKGROUND

An ongoing global challenge is managing brain health and understanding how performance changes across the lifespan.

OBJECTIVE

We developed and deployed a set of self-administrable, computerized assessments designed to measure key indexes of brain health across the visual and auditory sensory modalities. In this pilot study, we evaluated the usability, feasibility, and performance distributions of the assessments in a home-based, real-world setting without supervision.

METHODS

Potential participants were untrained users who self-registered on an existing brain training app called BrainHQ. Participants were contacted via a recruitment email and registered remotely to complete a demographics questionnaire and 29 unique assessments on their personal devices. We examined participant engagement, descriptive and psychometric properties of the assessments, associations between performance and self-reported demographic variables, cognitive profiles, and factor loadings.

RESULTS

Of the 365,782 potential participants contacted via a recruitment email, 414 (0.11%) registered, of whom 367 (88.6%) completed at least one assessment and 104 (25.1%) completed all 29 assessments. Registered participants were, on average, aged 63.6 (SD 14.8; range 13-107) years, mostly female (265/414, 64%), educated (329/414, 79.5% with a degree), and White (349/414, 84.3% White and 48/414, 11.6% people of color). A total of 72% (21/29) of the assessments showed no ceiling or floor effects or had easily modifiable score bounds to eliminate these effects. When correlating performance with self-reported demographic variables, 72% (21/29) of the assessments were sensitive to age, 72% (21/29) of the assessments were insensitive to gender, 93% (27/29) of the assessments were insensitive to race and ethnicity, and 93% (27/29) of the assessments were insensitive to education-based differences. Assessments were brief, with a mean duration of 3 (SD 1.0) minutes per task. The pattern of performance across the assessments revealed distinctive cognitive profiles and loaded onto 4 independent factors.

CONCLUSIONS

The assessments were both usable and feasible and warrant a full normative study. A digital toolbox of scalable and self-administrable assessments that can evaluate brain health at a glance (and longitudinally) may lead to novel future applications across clinical trials, diagnostics, and performance optimization.

Neural network process simulations support a distributed memory system and aid design of a novel computer adaptive digital memory test for preclinical and prodromal Alzheimer's disease

OBJECTIVE

Growing evidence supports the importance of learning as a central deficit in preclinical/prodromal Alzheimer's disease. The aims of this study were to conduct a series of neural network simulations to develop a functional understanding of a distributed, nonmodular memory system that can learn efficiently without interference. This understanding is applied to the development of a novel digital memory test.

METHOD

Simulations using traditional feed forward neural network architectures to learn simple logic problems are presented. The simulations demonstrate three limitations: (a) inefficiency, (b) an inability to learn problems consistently, and (c) catastrophic interference when given multiple problems. A new mirrored cascaded architecture is introduced to address these limitations, with support provided by a series of simulations.

RESULTS

The mirrored cascaded architecture demonstrates efficient and consistent learning relative to feed forward networks but also suffers from catastrophic interference. Addition of context values to add the capability of distinguishing features as part of learning eliminates the problem of interference in the mirrored cascaded, but not the feed forward, architectures.

CONCLUSIONS

A mirrored cascaded architecture addresses the limitations of traditional feed forward neural networks, provides support for a distributed memory system, and emphasizes the importance of context to avoid interference. These process models contributed to the design of a digital computer-adaptive word list learning test that places maximum stress on the capability to distinguish specific episodes of learning. Process simulations provide a useful method of testing models of brain function and contribute to new approaches to neuropsychological assessment. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Cognitive performance in older people after mild traumatic brain injury: Trauma effects and other risk factors

OBJECTIVE

Cognitive symptoms are common in the initial weeks after mTBI, but recovery is generally expected within three months. However, there is limited information about recovery specifically in older age cohorts. Therefore, this study investigated cognitive outcome three months after mTBI in older adults (≥ 65 years) compared to trauma and community age-matched controls and explored risk factors for outcome after traumatic injury.

METHODS

Older mTBI patients (n = 40) and older adults with mild traumatic injury but without head injury (n = 66) were compared to a noninjured community control group (n = 47). Cognitive assessment included neuropsychological and computerized tests. Group differences were compared on individual tasks and overall cognitive performances using composite scores. Regression analyses identified predictors of outcome for trauma patients and moderator analyses explored possible interactions of mTBI severity with age and cognition.

RESULTS

As well as lower performances in processing speed and memory, both trauma groups had significantly lower performance on composite neuropsychological (d = .557 and .670) and computerized tasks (d = .783 and .824) compared to noninjured controls. Age, education, and history of depression were direct predictors of cognitive performance after mild traumatic injury (with or without head injury). Further moderation analysis demonstrated that mTBI severity (Glasgow Coma Scale < 15) moderated the impact of older age on computerized assessment (β = -.138).

CONCLUSIONS

Three months after mild trauma (regardless of head injury), older people demonstrate lower cognition compared to noninjured peers. However, severity of mTBI (Glasgow Coma Scale < 15) can interact with older age to predict poorer cognitive outcomes.

A novel computer adaptive word list memory test optimized for remote assessment: Psychometric properties and associations with neurodegenerative biomarkers in older women without dementia

Introduction

This study established the psychometric properties and preliminary validity of the Stricker Learning Span (SLS), a novel computer adaptive word list memory test designed for remote assessment and optimized for smartphone use.

Methods

Women enrolled in the Mayo Clinic Specialized Center of Research Excellence (SCORE) were recruited via e-mail or phone to complete two remote cognitive testing sessions. Convergent validity was assessed through correlation with previously administered in-person neuropsychological tests (n = 96, ages 55-79) and criterion validity through associations with magnetic resonance imaging measures of neurodegeneration sensitive to Alzheimer's disease (n = 47).

Results

SLS performance significantly correlated with the Auditory Verbal Learning Test and measures of neurodegeneration (temporal meta-regions of interest and entorhinal cortical thickness, adjusting for age and education). Test-retest reliabilities across two sessions were 0.71-0.76 (two-way mixed intraclass correlation coefficients).

Discussion

The SLS is a valid and reliable self-administered memory test that shows promise for remote assessment of aging and neurodegenerative disorders.

Laptop-Administered NIH Toolbox and Cogstate Brief Battery in Community-Dwelling Black Adults: Unexpected Pattern of Cognitive Performance between MCI and Healthy Controls

OBJECTIVE

Black adults are approximately twice as likely to develop Alzheimer's disease (AD) than non-Hispanic Whites and access diagnostic services later in their illness. This dictates the need to develop assessments that are cost-effective, easily administered, and sensitive to preclinical stages of AD, such as mild cognitive impairment (MCI). Two computerized cognitive batteries, NIH Toolbox-Cognition and Cogstate Brief Battery, have been developed. However, utility of these measures for clinical characterization remains only partially determined. We sought to determine the convergent validity of these computerized measures in relation to consensus diagnosis in a sample of MCI and healthy controls (HC).

METHOD

Participants were community-dwelling Black adults who completed the neuropsychological battery and other Uniform Data Set (UDS) forms from the AD centers program for consensus diagnosis (HC = 61; MCI = 43) and the NIH Toolbox-Cognition and Cogstate batteries. Discriminant function analysis was used to determine which cognitive tests best differentiated the groups.

RESULTS

NIH Toolbox crystallized measures, Oral Reading and Picture Vocabulary, were the most sensitive in identifying MCI apart from HC. Secondarily, deficits in memory and executive subtests were also predictive. UDS neuropsychological test analyses showed the expected pattern of memory and executive functioning tests differentiating MCI from HC.

CONCLUSIONS

Contrary to expectation, NIH Toolbox crystallized abilities appeared preferentially sensitive to diagnostic group differences. This study highlights the importance of further research into the validity and clinical utility of computerized neuropsychological tests within ethnic minority populations.

Validity of Digital Assessments in Screening for HIV-Related Cognitive Impairment: a Review

PURPOSE OF REVIEW

While traditional neuropsychological tests are the gold standard in screening for HIV-related cognitive impairment, computerized neuropsychological assessment devices (CNADs) offer an alternative to these time- and resource-intensive batteries and may prove to be particularly useful for remote assessments or longitudinal monitoring. This review seeks to describe the benefits, limitations, and validity of CNADs in the evaluation of HIV-associated neurocognitive disorder (HAND).

RECENT FINDINGS

We identified eight CNADs that have undergone validity testing for cognitive impairment in the setting of HIV. Included among these are batteries that have been modeled after the traditional neuropsychological exam, as well as others that implement new technologies, such as simulated reality and daily ecological assessments in their testing. Currently, these digital batteries do not yet have the ability to supplant gold standard neuropsychological tests in screening for HAND. However, many have the potential to become effective clinical screening tools.

A Comparison of Cross-Sectional and Longitudinal Methods of Defining Objective Subtle Cognitive Decline in Preclinical Alzheimer's Disease Based on Cogstate One Card Learning Accuracy Performance

BACKGROUND

Longitudinal, but not cross-sectional, cognitive testing is one option proposed to define transitional cognitive decline for individuals on the Alzheimer's disease continuum.

OBJECTIVE

Compare diagnostic accuracy of cross-sectional subtle objective cognitive impairment (sOBJ) and longitudinal objective decline (ΔOBJ) over 30 months for identifying 1) cognitively unimpaired participants with preclinical Alzheimer's disease defined by elevated brain amyloid and tau (A+T+) and 2) incident mild cognitive impairment (MCI) based on Cogstate One Card Learning (OCL) accuracy performance.

METHODS

Mayo Clinic Study of Aging cognitively unimpaired participants aged 50 + with amyloid and tau PET scans (n = 311) comprised the biomarker-defined sample. A case-control sample of participants aged 65 + remaining cognitively unimpaired for at least 30 months included 64 who subsequently developed MCI (incident MCI cases) and 184 controls, risk-set matched by age, sex, education, and visit number. sOBJ was assessed by OCL z-scores. ΔOBJ was assessed using within subjects' standard deviation and annualized change from linear regression or linear mixed effects (LME) models. Concordance measures Area Under the ROC Curve (AUC) or C-statistic and odds ratios (OR) from conditional logistic regression models were derived. sOBJ and ΔOBJ were modeled jointly to compare methods.

RESULTS

sOBJ and ΔOBJ-LME methods differentiated A+T+ from A-T- (AUC = 0.64, 0.69) and controls from incident MCI (C-statistic = 0.59, 0.69) better than chance; other ΔOBJ methods did not. ΔOBJ-LME improved prediction of future MCI over baseline sOBJ (p = 0.003) but not over 30-month sOBJ (p = 0.09).

CONCLUSION

Longitudinal decline did not offer substantial benefit over cross-sectional assessment in detecting preclinical Alzheimer's disease or incident MCI.

Reliability and Validity of a Home-Based Self-Administered Computerized Test of Learning and Memory Using Speech Recognition

INTRODUCTION

The objective of this study is to evaluate the reliability and validity of the ReVeRe^TM word list recall test (RWLRT), which uses speech recognition, when administered remotely and unsupervised.

METHODS

Prospective cohort study. Participants included 249 cognitively intact community dwelling older adults. Measures included clinician administered neuropsychological assessments at baseline and unsupervised remotely administered tests of cognition from six time-points over six months.

RESULTS

The RWLRT showed acceptable validity. Reliability coefficients varied across time points, with poor reliability between times 1 and 2 and fair-to-good reliability across the remaining five testing sessions. Practice effects were observed with repeated administration as expected.

DISCUSSION

Unsupervised computerized tests of cognition, particularly word list learning and memory tests that use speech recognition, have significant potential for large scale early detection and long-term tracking of cognitive decline due to AD.

Diagnostic accuracy of the Cogstate Brief Battery for prevalent MCI and prodromal AD (MCI A+ T+ ) in a population-based sample

INTRODUCTION

This study evaluated the diagnostic accuracy of the Cogstate Brief Battery (CBB) for mild cognitive impairment (MCI) and prodromal Alzheimer's disease (AD) in a population-based sample.

METHODS

Participants included adults ages 50+ classified as cognitively unimpaired (CU, n = 2866) or MCI (n = 226), and a subset with amyloid (A) and tau (T) positron emission tomography who were AD biomarker negative (A-T-) or had prodromal AD (A+T+).

RESULTS

Diagnostic accuracy of the Learning/Working Memory Composite (Lrn/WM) for discriminating all CU and MCI was moderate (area under the curve [AUC] = 0.75), but improved when discriminating CU A-T- and MCI A+T+ (AUC = 0.93) and when differentiating MCI participants without AD biomarkers from those with prodromal AD (AUC = 0.86). Conventional cut-offs yielded lower than expected sensitivity for both MCI (38%) and prodromal AD (73%).

DISCUSSION

Clinical utility of the CBB for detecting MCI in a population-based sample is lower than expected. Caution is needed when using currently available CBB normative data for clinical interpretation.

Longitudinal Comparison of in Clinic and at Home Administration of the Cogstate Brief Battery and Demonstrated Practice Effects in the Mayo Clinic Study of Aging

BACKGROUND

The Cogstate Brief Battery (CBB) is a computerized cognitive assessment that can be completed in clinic or at home. Design/Objective: This retrospective study investigated whether practice effects / performance trajectories of the CBB differ by location of administration.

PARTICIPANTS/SETTING

Participants included 1439 cognitively unimpaired individuals age 50-75 at baseline participating in the Mayo Clinic Study of Aging (MCSA), a population-based study of cognitive aging. Sixty three percent of participants completed the CBB in clinic only and 37% completed CBB both in clinic and at home.

MEASUREMENTS

The CBB consists of four subtests: Detection, Identification, One Card Learning, and One Back. Linear mixed effects models were used to evaluate performance trajectories in clinic and at home.

RESULTS

Results demonstrated significant practice effects between sessions 1 to 2 for most CBB measures. Practice effects continued over subsequent testing sessions, to a lesser degree. Average practice effects/trajectories were similar for each location (home vs. clinic). One Card Learning and One Back accuracy performances were lower at home than in clinic, and this difference was large in magnitude for One Card Learning accuracy. Participants performed faster at home on Detection reaction time, although this difference was small in magnitude.

CONCLUSIONS

Results suggest the location where the CBB is completed has an important impact on performance, particularly for One Card Learning accuracy, and there are practice effects across repeated sessions that are similar regardless of where testing is completed.

Diagnostic and Prognostic Accuracy of the Cogstate Brief Battery and Auditory Verbal Learning Test in Preclinical Alzheimer's Disease and Incident Mild Cognitive Impairment: Implications for Defining Subtle Objective Cognitive Impairment

BACKGROUND

There are detectable cognitive differences in cognitively unimpaired (CU) individuals with preclinical Alzheimer's disease (AD).

OBJECTIVE

To determine whether cross-sectional performance on the Cogstate Brief Battery (CBB) and Auditory Verbal Learning Test (AVLT) could identify 1) CU participants with preclinical AD defined by neuroimaging biomarkers of amyloid and tau, and 2) incident mild cognitive impairment (MCI)/dementia.

METHOD

CU participants age 50+ were eligible if they had 1) amyloid (A) and tau (T) imaging within two years of their baseline CBB or 2) at least one follow-up visit. AUROC analyses assessed the ability of measures to differentiate groups. We explored the frequency of cross-sectional subtle objective cognitive impairment (sOBJ) defined as performance ≤-1 SD on CBB Learning/Working Memory Composite (Lrn/WM) or AVLT delayed recall using age-corrected normative data.

RESULTS

A+T+ (n = 33, mean age 79.5) and A+T- (n = 61, mean age 77.8) participants were older than A-T- participants (n = 146, mean age 66.3), and comparable on sex and education. Lrn/WM did not differentiate A + T+or A+T- from A-T- participants. AVLT differentiated both A+T+ and A+T- from A-T- participants; 45% of A+T+ and 25% of A+T- participants met sOBJ criteria. The follow-up cohort included 150 CU individuals who converted to MCI/dementia and 450 age, sex, and education matched controls. Lrn/WM and AVLT differentiated between stable and converter CU participants.

CONCLUSION

Among CU participants, AVLT helped differentiate A+T+ and A+T- from A-T- participants. The CBB did not differentiate biomarker subgroups, but showed potential for predicting incident MCI/dementia. Results inform future definitions of sOBJ.

CogEvo, a cognitive function balancer, is a sensitive and easy psychiatric test battery for age-related cognitive decline

Aim

We examined whether a newly developed computer‐aided neuropsychiatric series of test, CogEvo, is necessary and sufficient for the evaluation of cognitive function in older people.

Methods

A total of 272 participants in worthwhile life activity for the prevention of decline in mobility and cognitive function were administered tests every week at 33 locations in Fukaura‐machi, Japan. Basic profile information, a Mini‐Mental State Examination (MMSE), a CogEvo and a clock drawing test were used in the present study.

Results

Our results are summarized as: (i) the total score of the CogEvo and MMSE tests decreased significantly according to age and in age group analysis; (ii) scores from the CogEvo and MMSE tests showed a significant correlation; (iii) MMSE scores showed marked ceiling effects; (iv) analysis of cognitive domains, such as orientation, attention, memory and executive function, and spatial cognition using CogEvo showed significant age‐dependent impairment; (v) CogEvo discriminated three score groups of MMSE results with sensitivity and specificity of 70% and 60% in the <23 score group, 78% and 54% in the 24–26 score group, and 85% and 70% in the >27 score group, respectively; (vi) CogEvo memory tests reflected more detailed recall function than registration function; and (vii) CogEvo spatial cognition test results were correlated with test items of the MMSE and clock drawing tests.

Conclusions

CogEvo is an easy and potentially useful computer‐aided test battery that can be used to evaluate age‐related or pathological decline in cognitive function from middle age and in preclinical stages of dementia. Geriatr Gerontol Int 2019; ••: ••–••.

Computerized neuropsychological assessment devices in multiple sclerosis: A systematic review

Background:

The proliferation of computerized neuropsychological assessment devices (CNADs) for screening and monitoring cognitive impairment is increasing exponentially. Previous reviews of computerized tests for multiple sclerosis (MS) were primarily qualitative and did not rigorously compare CNADs on psychometric properties.

Objective:

We aimed to systematically review the literature on the use of CNADs in MS and identify test batteries and single tests with good evidence for reliability and validity.

Method:

A search of four major online databases was conducted for publications related to computerized testing and MS. Test–retest reliability and validity coefficients and effect sizes were recorded for each CNAD test, along with administration characteristics.

Results:

We identified 11 batteries and 33 individual tests from 120 peer-reviewed articles meeting the inclusion criteria. CNADs with the strongest psychometric support include the CogState Brief Battery, Cognitive Drug Research Battery, NeuroTrax, CNS-Vital Signs, and computer-based administrations of the Symbol Digit Modalities Test.

Conclusion:

We identified several CNADs that are valid to screen for MS-related cognitive impairment, or to supplement full, conventional neuropsychological assessment. The necessity of testing with a technician, and in a controlled clinic/laboratory environment, remains uncertain.