Online cognition: factors facilitating reliable online neuropsychological test results

Digital Neuropsychology beyond Computerized Cognitive Assessment: Applications of Novel Digital Technologies

Compared with other health disciplines, there is a stagnation in technological innovation in the field of clinical neuropsychology. Traditional paper-and-pencil tests have a number of shortcomings, such as low-frequency data collection and limitations in ecological validity. While computerized cognitive assessment may help overcome some of these issues, current computerized paradigms do not address the majority of these limitations. In this paper, we review recent literature on the applications of novel digital health approaches, including ecological momentary assessment, smartphone-based assessment and sensors, wearable devices, passive driving sensors, smart homes, voice biomarkers, and electronic health record mining, in neurological populations. We describe how each digital tool may be applied to neurologic care and overcome limitations of traditional neuropsychological assessment. Ethical considerations, limitations of current research, as well as our proposed future of neuropsychological practice are also discussed.

Online cognitive testing in Parkinson's disease: advantages and challenges

Parkinson's disease (PD) is primarily characterized by motor symptoms. Yet, many people with PD experience cognitive decline, which is often unnoticed by clinicians, although it may have a significant impact on quality of life. For over half a century, traditional in-person PD cognitive assessment lacked accessibility, scalability, and specificity due to its inherent limitations. In this review, we propose that novel methods of online cognitive assessment could potentially address these limitations. We first outline the challenges of traditional in-person cognitive testing in PD. We then summarize the existing literature on online cognitive testing in PD. Finally, we explore the advantages, but also the limitations, of three major processes involved in online PD cognitive testing: recruitment and sampling methods, measurement and participation, and disease monitoring and management. Taking the limitations into account, we aim to highlight the potential of online cognitive testing as a more accessible and efficient approach to cognitive testing in PD.

Test-retest reliability and reliable change index of the Philips IntelliSpace Cognition digital test battery

OBJECTIVE

This article provides the test-retest reliability and Reliable Change Indices (RCIs) of the Philips IntelliSpace Cognition (ISC) platform, which contains digitized versions of well-established neuropsychological tests.

METHOD

147 participants (ages 19 to 88) completed a digital cognitive test battery on the ISC platform or paper-pencil versions of the same test battery during two separate visits. Intraclass correlation coefficients (ICC) were calculated separately for the ISC and analog test versions to compare reliabilities between administration modalities. RCIs were calculated for the digital tests using the practice-adjusted RCI and standardized regression-based (SRB) method.

RESULTS

Test-retest reliabilities for the ISC tests ranged from moderate to excellent and were comparable to the test-retest reliabilities for the paper-pencil tests. Baseline test performance, retest interval, age, and education predicted test performance at visit 2 with baseline test performance being the strongest predictor for all outcome measures. For most outcome measures, both methods for the calculation of RCIs show agreement on whether or not a reliable change was observed.

CONCLUSIONS

RCIs for the digital tests enable clinicians to determine whether a measured change between assessments is due to real improvement or decline. Together, this contributes to the growing evidence for the clinical utility of the ISC platform.

Assessing time-based prospective memory online: A comparison study between laboratory-based and web-based testing

Prospective memory (PM, i.e., the ability to remember and perform future intentions) is assessed mainly within laboratory settings; however, in the last two decades, several studies have started testing PM online. Most part of those studies focused on event-based PM (EBPM), and only a few assessed time-based PM (TBPM), possibly because time keeping is difficult to control or standardise without experimental control. Thus, it is still unclear whether time monitoring patterns in online studies replicate typical patterns obtained in laboratory tasks. In this study, we therefore aimed to investigate whether the behavioural outcome measures obtained from the traditional TBPM paradigm in the laboratory-accuracy and time monitoring-are comparable with an online version in a sample of 101 younger adults. Results showed no significant difference in TBPM performance in the laboratory versus online setting, as well as no difference in time monitoring. However, we found that participants were somewhat faster and more accurate at the ongoing task during the laboratory assessment, but those differences were not related to holding an intention in mind. The findings suggest that, although participants seemed generally more distracted when tested remotely, online assessment yielded similar results in key temporal characteristics and behavioural performance as for the laboratory assessment. The results are discussed in terms of possible conceptual and methodological implications for online testing.

FACEmemory®, an Innovative Online Platform for Episodic Memory Pre-Screening: Findings from the First 3,000 Participants

BACKGROUND

The FACEmemory® online platform comprises a complex memory test and sociodemographic, medical, and family questions. This is the first study of a completely self-administered memory test with voice recognition, pre-tested in a memory clinic, sensitive to Alzheimer's disease, using information and communication technologies, and offered freely worldwide.

OBJECTIVE

To investigate the demographic and clinical variables associated with the total FACEmemory score, and to identify distinct patterns of memory performance on FACEmemory.

METHODS

Data from the first 3,000 subjects who completed the FACEmemory test were analyzed. Descriptive analyses were applied to demographic, FACEmemory, and medical and family variables; t-test and chi-square analyses were used to compare participants with preserved versus impaired performance on FACEmemory (cut-off = 32); multiple linear regression was used to identify variables that modulate FACEmemory performance; and machine learning techniques were applied to identify different memory patterns.

RESULTS

Participants had a mean age of 50.57 years and 13.65 years of schooling; 64.07% were women, and 82.10% reported memory complaints with worries. The group with impaired FACEmemory performance (20.40%) was older, had less schooling, and had a higher prevalence of hypertension, diabetes, dyslipidemia, and family history of neurodegenerative disease than the group with preserved performance. Age, schooling, sex, country, and completion of the medical and family history questionnaire were associated with the FACEmemory score. Finally, machine learning techniques identified four patterns of FACEmemory performance: normal, dysexecutive, storage, and completely impaired.

CONCLUSIONS

FACEmemory is a promising tool for assessing memory in people with subjective memory complaints and for raising awareness about cognitive decline in the community.

Computerized Cognitive Test Batteries for Children and Adolescents-A Scoping Review of Tools For Lab- and Web-Based Settings From 2000 to 2021

OBJECTIVE

Cognitive functioning is essential to well-being. Since cognitive difficulties are common in many disorders, their early identification is critical, notably during childhood and adolescence. This scoping review aims to provide a comprehensive literature overview of computerized cognitive test batteries (CCTB) that have been developed and used in children and adolescents over the past 22 years and to evaluate their psychometric properties.

METHOD

Among 3192 records identified from three databases (PubMed, PsycNET, and Web of Science) between 2000 and 2021, 564 peer-reviewed articles conducted in children and adolescents aged 3 to 18 years met inclusion criteria. Twenty main CCTBs were identified and further reviewed following PRISMA guidelines. Relevant study details (sample information, topic, location, setting, norms, and psychometrics) were extracted, as well as administration and instrument characteristics for the main CCTBs.

RESULTS

Findings suggest that CCTB use varies according to age, location, and topic, with eight tools accounting for 85% of studies, and the Cambridge Neuropsychological Test Automated Battery (CANTAB) being most frequently used. Few instruments were applied in web-based settings or include social cognition tasks. Only 13% of studies reported psychometric properties.

CONCLUSIONS

Over the past two decades, a high number of computerized cognitive batteries have been developed. Among these, more validation studies are needed, particularly across diverse cultural contexts. This review offers a comprehensive synthesis of CCTBs to aid both researchers and clinicians to conduct cognitive assessments in children in either a lab- or web-based setting.

Cardiff Online Cognitive Assessment in a National Sample: Cross-Sectional Web-Based Study

BACKGROUND

Psychiatric disorders are associated with cognitive impairment. We have developed a web-based, 9-task cognitive battery to measure the core domains affected in people with psychiatric disorders. To date, this assessment has been used to collect data on a clinical sample of participants with psychiatric disorders.

OBJECTIVE

The aims of this study were (1) to establish a briefer version of the battery (called the Cardiff Online Cognitive Assessment [CONCA]) that can give a valid measure of cognitive ability ("g") and (2) to collect normative data and demonstrate CONCA's application in a health population sample.

METHODS

Based on 6 criteria and data from our previous study, we selected 5 out of the original 9 tasks to include in CONCA. These included 3 core tasks that were sufficient to derive a measure of "g" and 2 optional tasks. Participants from a web-based national cohort study (HealthWise Wales) were invited to complete CONCA. Completion rates, sample characteristics, performance distributions, and associations between cognitive performance and demographic characteristics and mental health measures were examined.

RESULTS

A total of 3679 participants completed at least one CONCA task, of which 3135 completed all 3 core CONCA tasks. Performance on CONCA was associated with age (B=-0.05, SE 0.002; P<.001), device (tablet computer: B=-0.26, SE 0.05; P<.001; smartphone: B=-0.46, SE 0.05; P<.001), education (degree: B=1.68, SE 0.14; P<.001), depression symptoms (B=-0.04, SE 0.01; P<.001), and anxiety symptoms (B=-0.04, SE 0.01; P<.001).

CONCLUSIONS

CONCA provides a valid measure of "g," which can be derived using as few as 3 tasks that take no more than 15 minutes. Performance on CONCA showed associations with demographic characteristics in the expected direction and was associated with current depression and anxiety symptoms. The effect of device on cognitive performance is an important consideration for research using web-based assessments.

Evaluating cognitive impairment in the early stages of Parkinson's disease using the Cambridge brain sciences-cognitive platform

BACKGROUND

Non-motor symptoms (NMS) such as cognitive impairment are among common presentations in patients with Parkinson's disease (PD). In parallel with motor symptoms, these impediments can affect PD patients' quality of life. However, cognitive impairment has received less attention in early PD. On the other hand, the relationship between olfactory symptoms and cognitive impairment is unclear in early PD. Considering the importance of accurate and timely assessment of cognitive function in PD patients using readily available/validated tests, this study has employed the Cambridge Brain Sciences-Cognitive Platform (CBS-CP) as a computer-based tool to assess cognitive presentations in early PD patients.

METHODS

Thirty-four eligible males and females were assigned to PD and healthy controls (HCs). The cognitive performance was assessed using CBS-CP and Mini-Mental State Examination (MMSE), and olfactory function was measured through the standardized olfactory Quick Smell test (QST).

RESULTS

PD patients had poorer performance in all CBS-CP tasks, including short-term memory, attention, and reasoning domains than HCs. Meanwhile, the verbal domain task scores showed no significant difference between groups. PD MMSE results were in the normal range (mean=26.96), although there was a significant difference between the PD and HCs groups (P = 0.000). Our results revealed no correlation between cognitive impairment and olfactory function in PD patients.

CONCLUSION

Given the widely studied features of CBS-CP and its reliability across published evidence, CBS-CP appears to be a suitable measurement to evaluate cognitive impairment in early PD with normal MMSE scores. It seems cognitive and olfactory impairments are independent in early PD.

DATA AVAILABILITY STATEMENT

The datasets generated during the current study are available from the corresponding author upon reasonable request.

Computerised cognitive assessment in patients with traumatic brain injury: an observational study of feasibility and sensitivity relative to established clinical scales

Background

Online technology could potentially revolutionise how patients are cognitively assessed and monitored. However, it remains unclear whether assessments conducted remotely can match established pen-and-paper neuropsychological tests in terms of sensitivity and specificity.

Methods

This observational study aimed to optimise an online cognitive assessment for use in traumatic brain injury (TBI) clinics. The tertiary referral clinic in which this tool has been clinically implemented typically sees patients a minimum of 6 months post-injury in the chronic phase. Between March and August 2019, we conducted a cross-group, cross-device and factor analyses at the St. Mary's Hospital TBI clinic and major trauma wards at Imperial College NHS trust and St. George's Hospital in London (UK), to identify a battery of tasks that assess aspects of cognition affected by TBI. Between September 2019 and February 2020, we evaluated the online battery against standard face-to-face neuropsychological tests at the Imperial College London research centre. Canonical Correlation Analysis (CCA) determined the shared variance between the online battery and standard neuropsychological tests. Finally, between October 2020 and December 2021, the tests were integrated into a framework that automatically generates a results report where patients' performance is compared to a large normative dataset. We piloted this as a practical tool to be used under supervised and unsupervised conditions at the St. Mary's Hospital TBI clinic in London (UK).

Findings

The online assessment discriminated processing-speed, visual-attention, working-memory, and executive-function deficits in TBI. CCA identified two significant modes indicating shared variance with standard neuropsychological tests (r = 0.86, p < 0.001 and r = 0.81, p = 0.02). Sensitivity to cognitive deficits after TBI was evident in the TBI clinic setting under supervised and unsupervised conditions (F (15,555) = 3.99; p < 0.001).

Interpretation

Online cognitive assessment of TBI patients is feasible, sensitive, and efficient. When combined with normative sociodemographic models and autogenerated reports, it has the potential to transform cognitive assessment in the healthcare setting.

Funding

This work was funded by a National Institute for Health Research (NIHR) Invention for Innovation (i4i) grant awarded to DJS and AH (II-LB-0715-20006).

Teleneuropsychology: normative data for the assessment of memory in online settings

INTRODUCTION

The COVID-19 pandemic has forced significant changes in clinical practice. Psychologists and neuropsychologists had to modify their settings to assess patients' abilities, switching from an in-person modality to a remote setting by using video calling platforms. Consequently, this change brought about the need for new normative data tailored to remote settings.

AIM AND METHODS

The study aimed to develop normative data for the online assessment of neuropsychological memory tests and to compare it with the published norms obtained in standard settings. Two hundred and four healthy Italian volunteers performed three verbal memory tests through the Google Meet platform: the Digit Span (Backward and Forward), the Rey Auditory Verbal Learning, and the Verbal Paired Associated Learning Test.

RESULTS

This research provides specific norms that consider the influence of demographic characteristics. Their comparison with published norms shows a medium to high agreement between systems. The present study provides a reference for the clinical use of neuropsychological instruments to assess verbal memory in a remote setting and offers specific recommendations.

Associations between Participant Characteristics and Participant Feedback about an Unsupervised Online Cognitive Assessment in a Research Registry

BACKGROUND

This study aims to understand whether and how participant characteristics (age, gender, education, ethnocultural identity) are related to their feedback about taking a remote, unsupervised, online cognitive assessment.

METHODS

The Brain Health Registry is a public online registry which includes cognitive assessments. Multivariable ordinal regressions assessed associations between participant characteristics and feedback responses of older (55+) participants (N=11,553) regarding their Cogstate Brief Battery assessment experience.

RESULTS

Higher age, secondary education or less, Latino identity, and female gender were associated with a poorer assessment experience; higher age and a non-White identity were associated with experiencing the assessment instructions as less clear; and higher age, non-White identity, and secondary education or less were associated with rating additional human support with the assessment as more useful.

DISCUSSION

Our findings highlight the importance of improving the design and instructions of unsupervised, remote, online cognitive assessments to better suit the needs of diverse communities.

Web-Based Independent Versus Laboratory-Based Stop-Signal Task Performance: Within-Subjects Counterbalanced Comparison Study

Background

Considered a facet of behavioral impulsivity, response inhibition facilitates adaptive and goal-directed behavior. It is often assessed using the Stop-Signal Task (SST), which is presented on stand-alone computers under controlled laboratory conditions. Sample size may consequently be a function of cost or time and sample diversity constrained to those willing or able to attend the laboratory. Statistical power and generalizability of results might, in turn, be impacted. Such limitations may potentially be overcome via the implementation of web-based testing.

Objective

The aim of this study was to investigate if there were differences between variables derived from a web-based SST when it was undertaken independently—that is, outside the laboratory, on any computer, and in the absence of researchers—versus when it was performed under laboratory conditions.

Methods

We programmed a web-based SST in HTML and JavaScript and employed a counterbalanced design. A total of 166 individuals (mean age 19.72, SD 1.85, range 18-36 years; 146/166, 88% female) were recruited. Of them, 79 undertook the independent task prior to visiting the laboratory and 78 completed the independent task following their laboratory visit. The average time between SST testing was 3.72 (SD 2.86) days. Dependent samples and Bayesian paired samples t tests were used to examine differences between laboratory-based and independent SST variables. Correlational analyses were conducted on stop-signal reaction times (SSRT).

Results

After exclusions, 123 participants (mean age 19.73, SD 1.97 years) completed the SST both in the laboratory and independently. While participants were less accurate on go trials and exhibited reduced inhibitory control when undertaking the independent—compared to the laboratory-based—SST, there was a positive association between the SSRT of each condition (r=.48; P<.001; 95% CI 0.33-0.61).

Conclusions

Findings suggest a web-based SST, which participants undertake on any computer, at any location, and in the absence of the researcher, is a suitable measure of response inhibition.

Test-Retest Reliability of Remote ImPACT Administration

OBJECTIVE

To evaluate the performance and test-retest reliability obtained when administering a computerized baseline neurocognitive exam to NCAA Division I student-athletes in a controlled laboratory setting versus an uncontrolled remote location.

METHOD

A sample of 129 (female = 100) Division I student-athletes completed Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) pre-season assessments for two distinct and respective sports seasons in a controlled laboratory environment and an uncontrolled remote environment. Depending on the environment, participants were given verbal (controlled) or written (uncontrolled) guidelines for taking the test.

RESULTS

Multivariate repeated-measures ANOVA's determined that there were no within-subject differences between testing environments on ImPACT composite scores and cognitive efficiency index (CEI). The Chi-square test did not find any significant differences in impulse control or the number of invalid test scores, as determined by ImPACT, between environments. Intraclass correlations found the ImPACT subtest scores to range in test-retest reliability across testing environments, demonstrating moderate (verbal memory composite, r = 0.46; visual memory composite, r = 0.64; reaction time, r = 0.61; impulse control, r = 0.52; and CEI, r = 0.61) and good (visual motor composite, r = 0.77) test-retest reliability.

CONCLUSIONS

Results indicate that ImPACT is reliable between controlled and uncontrolled testing environments. This further suggests that ImPACT can be administered in a remote environment, pending specific adherence to testing instructions, or in the event of social distancing or isolation policies.

Web-Based Cognitive Testing in Psychiatric Research: Validation and Usability Study

Background

Cognitive impairments are features of many psychiatric disorders and affect functioning. A barrier to cognitive research on psychiatric disorders is the lack of large cross-disorder data sets. However, the collection of cognitive data can be logistically challenging and expensive. Web-based collection may be an alternative; however, little is known about who does and does not complete web-based cognitive assessments for psychiatric research.

Objective

The aims of this study are to develop a web-based cognitive battery for use in psychiatric research, validate the battery against the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Consensus Cognitive Battery, and compare the characteristics of the participants who chose to take part with those of the individuals who did not participate.

Methods

Tasks were developed by The Many Brains Project and selected to measure the domains specified by the MATRICS initiative. We undertook a cross-validation study of 65 participants with schizophrenia, bipolar disorder, depression, or no history of psychiatric disorders to compare the web-based tasks with the MATRICS Consensus Cognitive Battery. Following validation, we invited participants from 2 large ongoing genetic studies, which recruited participants with psychiatric disorders to complete the battery and evaluated the demographic and clinical characteristics of those who took part.

Results

Correlations between web-based and MATRICS tasks ranged between 0.26 and 0.73. Of the 961 participants, 887 (92.3%) completed at least one web-based task, and 644 (67%) completed all tasks, indicating adequate completion rates. Predictors of web-based participation included being female (odds ratio [OR] 1.3, 95% CI 1.07-1.58), ethnicity other than White European (OR 0.66, 95% CI 0.46-0.96), higher levels of education (OR 1.19, 95% CI 1.11-1.29), diagnosis of an eating disorder (OR 2.17, 95% CI 1.17-4) or depression and anxiety (OR 5.12, 95% CI 3.38-7.83), and absence of a diagnosis of schizophrenia (OR 0.59, 95% CI 0.35-0.94). Lower performance on the battery was associated with poorer functioning (B=−1.76, SE 0.26; P<.001).

Conclusions

Our findings offer valuable insights into the advantages and disadvantages of testing cognitive function remotely for mental health research.

A New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High-Quality Data: Development and Usability Study

Background

The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the forefront as a promising solution for the rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. This study examines the opportunities and challenges afforded by the societal shift toward web-based testing and highlights an urgent need to establish a standard data quality assurance framework for online studies.

Objective

This study aims to develop and validate a new supervised online testing methodology, remote guided testing (RGT).

Methods

A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory, and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (n=41) or online using remote guided testing (n=44) and delivered using identical web-based platforms (Cambridge Neuropsychological Test Automated Battery, Inquisit, and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, and response times) and overall task performance measures.

Results

The results indicated that, across all data quality and performance measures, RGT data was statistically-equivalent to in-person data collected in the lab (P>.40 for all comparisons). Moreover, RGT participants out-performed the lab group on measured verbal intelligence (P<.001), which could reflect test environment differences, including possible effects of mask-wearing on communication.

Conclusions

These data suggest that the RGT methodology could help ameliorate concerns regarding online data quality—particularly for studies involving high-risk or rare cohorts—and offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.

Subjective cognitive decline is associated with a higher risk of objective cognitive decline: A cross-sectional and longitudinal study

BACKGROUND

Subjective cognitive decline (SCD) is considered as an independent risk factor for objective cognitive impairment, such as dementia and mild cognitive impairment (MCI), but the mechanism is unclear.

METHODS

The current study consisted of two parts, the first of which included 1,010 older adults with SCD and 535 normal controls and was followed for 1 year. The second cross-sectional study included 94 older adults with SCD and 64 healthy controls. Unlike the first cohort, subjects in the second study underwent magnetic resonance imaging and had more detailed neuropsychological tests, such as Mini- mental State Examination (MMSE), Montreal Cognitive Assessment (MoCA), Digit Span, Auditory Verbal Learning Test (AVLT), Associative Learning Test (ALT), Verbal Fluency (VF), Wechsler's filling and Wechsler's building blocks.

RESULTS

In cohort 1, we found that SCD had a higher risk of objective cognitive impairment compared to normal controls (X² = 20.354, p = 0.002), and the results of Cox Regression analysis also suggest that SCD was a risk factor for objective cognitive decline (p < 0.001, HR = 2.608, 95%CI: 2.213-3.075). In study 2, we found that the scores of MoCA, digit span, verbal fluency, and Wechsler's filling of SCD elderly were significantly lower than those of normal controls, but the cortical thickness of the rostral middle frontal gyrus (RMFG) was significantly higher than that of normal controls (p < 0.05).

CONCLUSIONS

SCD is a cognition-related disease with multi-cognitive domain impairment, which is associated with a higher risk of objective cognitive impairment. Moreover, the increased cortical thickness of the left rostral middle frontal gyrus (RMFG) might be an important mechanism of cognitive decline in SCD.

The TestMyBrain Digital Neuropsychology Toolkit: Development and Psychometric Characteristics

INTRODUCTION

To allow continued administration of neuropsychological evaluations remotely during the pandemic, tests from the not-for-profit platform, TestMyBrain.org (TMB), were used to develop the TMB Digital Neuropsychology Toolkit (DNT). This study details the psychometric characteristics of the DNT, as well as the infrastructure and development of the DNT.

METHOD

The DNT was primarily distributed for clinical use, with (72.8%) of individuals requesting access for clinical purposes. To assess reliability and validity of the DNT, anonymous data from DNT test administrations were analyzed and compared to a large, non-clinical normative sample from TMB.

RESULTS

DNT test scores showed acceptable to very good split-half reliability (.68-.99). Factor analysis revealed three latent factors, corresponding to processing speed, working memory, and a broader general cognitive ability factor that included perceptual reasoning and episodic memory. Average test scores were slightly poorer for the DNT sample than for the TMB comparison sample, as expected given the clinical use of the DNT.

CONCLUSIONS

Initial estimates of reliability and validity of DNT tests support their use as digital measures of neuropsychological functioning. Tests within cognitive domains correlated highly with each other and demonstrated good reliability and validity. Future work will seek to validate DNT tests in specific clinical populations and determine best practices for using DNT outcome measures to assess engagement and psychological symptomatology.

Objective Measures of Cognitive Performance in Sleep Disorder Research

Neurocognitive tests offer objective and reliable assessment of patients' status and progress. However, there is no consensus on how to use neurocognitive assessment in sleep disorder research. An effective use of neurocognitive assessment must be based on standardized practices and have a firm theoretic basis. The aim of this review is to offer an overview of how different tests have been used in the field, mapping each test onto a corresponding cognitive domain and propose how to move forward with a suggested cognitive battery of tests covering all major cognitive domains.

The Geneva Space Cruiser: a fully self-administered online tool to assess prospective memory across the adult lifespan

The current study aimed to examine whether the Geneva Space Cruiser - a new online adaptation of the Cruiser - represents a valid, reliable and useful tool to assess prospective memory (PM) across the adult lifespan via fully self-administered online testing. Therefore, an adult lifespan sample of 252 adults (19-86 years old) performed the Geneva Space Cruiser in the laboratory and online, at home, and also performed a more traditional laboratory PM task. A second sample of 224 young adults (19-35 years old) participated in a test-retest online assessment of the Geneva Space Cruiser. Bayesian analyses showed that the Geneva Space Cruiser yielded similar results when administered in the laboratory versus online, both in terms of data distribution as well as of key outcome measures (i.e., PM performance and monitoring). Results further showed very good test-retest reliability and acceptable construct validity. Finally, the online tool was sensitive for detecting age-differences similar to those typically observed in laboratory studies. Together, our findings suggest that the Geneva Space Cruiser represents a rather valid, moderately to highly reliable, and generally useful tool to assess PM in online testing across wide ranges of the adult lifespan, with certain limitations for the oldest participants and for women.

Segmentation of Prefrontal Lobe Based on Improved Clustering Algorithm in Patients with Diabetes

Diabetics are prone to postoperative cognitive dysfunction (POCD). The occurrence may be related to the damage of the prefrontal lobe. In this study, the prefrontal lobe was segmented based on an improved clustering algorithm in patients with diabetes, in order to evaluate the relationship between prefrontal lobe volume and COPD. In this study, a total of 48 diabetics who underwent selective noncardiac surgery were selected. Preoperative magnetic resonance imaging (MRI) images of the patients were segmented based on the improved clustering algorithm, and their prefrontal volume was measured. The correlation between the volume of the prefrontal lobe and Z-score or blood glucose was analyzed. Qualitative analysis shows that the gray matter, white matter, and cerebrospinal fluid based on the improved clustering algorithm were easy to distinguish. Quantitative evaluation results show that the proposed segmentation algorithm can obtain the optimal Jaccard coefficient and the least average segmentation time. There was a negative correlation between the volume of the prefrontal lobe and the Z-score. The cut-off value of prefrontal lobe volume for predicting POCD was <179.8, with the high specificity. There was a negative correlation between blood glucose and volume of the prefrontal lobe. From the results, we concluded that the segmentation of the prefrontal lobe based on an improved clustering algorithm before operation may predict the occurrence of POCD in diabetics.

Initial Adaptation of the General Cognitive Assessment Battery by Cognifit™ for Bulgarian Older Adults

BACKGROUND

Online neuropsychological assessment batteries may facilitate the screening of cognitive functions in older adults and could be useful for early diagnosis and detection of cognitive impairments.

OBJECTIVE

The primary aim of this study was to assess the psychometric qualities of an online multi-domain cognitive assessment battery (General Cognitive Assessment Battery (GCAB) by Cognifit™) applied for the first time in Bulgaria.

METHODS

A total of 20 healthy older adults (6 male and 14 female, aged 60-82) completed the GCAB as well as the Mini-Mental State Examination (MMSE) and the Consortium to Establish a Registry for Alzheimer's Disease (CERAD) neuropsychological battery. Descriptive statistics were used to describe the demographic characteristics of the sample and the scores on the GCAB and the CERAD battery. The internal consistency of the GCAB was evaluated using item analysis and measured with Cronbach's alpha. The concurrent validity of the GCAB was assessed with respect to the CERAD using Spearman's r after verifying the linear relationship between the GCAB and CERAD scores.

RESULTS

The GCAB showed good concurrent validity when compared with the corresponding CERAD tests. The correlation coefficients ranged from 0.67 for working memory to 0.47 for short-term auditory memory. We found very good reliability of the GCAB, with the inter-class correlation coefficient higher than 0.8 for all cognitive domains. There were no significant correlations between MMSE and GCAB scores.

CONCLUSION

The GCAB was found to be valid for the cognitive screening of Bulgarian healthy older adults and may provide an adequate assessment of their cognitive status. The GCAB showed good concurrent validity when compared with the CERAD battery, measuring similar cognitive constructs. Further work is necessary to explore its validity and reliability.

Interindividual and intraindividual variability in amnestic mild cognitive impairment (aMCI) measured with an online cognitive assessment

INTRODUCTION

Mean cognitive performance is worse in amnestic mild cognitive impairment (aMCI) compared to control groups. However, studies on variability of cognitive performance in aMCI have yielded inconclusive results, with many differences in variability measures and samples from one study to another.

METHODS

We examined variability in aMCI using an existing older adult sample (n = 91; 51 with aMCI, 40 with normal cognition for age), measured with an online self-administered computerized cognitive assessment (Cogniciti's Brain Health Assessment). Our methodology extended past findings by using pure measures of variability (controlling for confounding effects of group performance or practice), and a clinically representative aMCI sample (reflecting the continuum of cognitive performance between normal cognition and aMCI).

RESULTS

Between-group t-tests showed significantly greater between-person variability (interindividual variability or diversity) in overall cognitive performance in aMCI than controls, although the effect size was with a small to moderate effect size, d = 0.44. No significant group differences were found in within-person variability (intraindividual variability) across cognitive tasks (dispersion) or across trials of a response time task (inconsistency), which may be because we used a sample measuring the continuum of cognitive performance. Exploratory correlation analyses showed that a worse overall score was associated with greater inter- and intraindividual variability, and that variability measures were correlated with each other, indicating people with worse cognitive performance were more variable.

DISCUSSION

The current study demonstrates that self-administered online tests can be used to remotely assess different types of variability in people at risk of Alzheimer`s. Our findings show small but significantly more interindividual differences in people with aMCI. This diversity is considered as "noise" in standard assessments of mean performance, but offers an interesting and cognitively informative "signal" in itself.

The Cognitive Online Self-Test Amsterdam (COST-A): Establishing norm scores in a community-dwelling population

BACKGROUND

Heightened public awareness about Alzheimer's disease and dementia increases the need for at-home cognitive self-testing. We offered Cognitive Online Self-Test Amsterdam (COST-A) to independent groups of cognitively normal adults and investigated the robustness of a norm-score formula and cutoff.

METHODS

Three thousand eighty-eight participants (mean age ± standard deviation = 61 ± 12 years, 70% female) completed COST-A and evaluated it. Demographically adjusted norm scores were the difference between expected COST-A scores, based on age, gender, and education, and actual scores. We applied the resulting norm-score formula to two independent cohorts.

RESULTS

Participants evaluated COST-A to be of adequate difficulty and duration. Our norm-score formula was shown to be robust: ≈8% of participants in two cognitively normal cohorts had abnormal scores. A cutoff of -1.5 standard deviations proved optimal for distinguishing normal from impaired cognition.

CONCLUSION

With robust norm scores, COST-A is a promising new tool for research and clinical practice, providing low cost and minimally invasive remote assessment of cognitive functioning.

Reliability and Validity of a Home-Based Self-Administered Computerized Test of Learning and Memory Using Speech Recognition

INTRODUCTION

The objective of this study is to evaluate the reliability and validity of the ReVeRe^TM word list recall test (RWLRT), which uses speech recognition, when administered remotely and unsupervised.

METHODS

Prospective cohort study. Participants included 249 cognitively intact community dwelling older adults. Measures included clinician administered neuropsychological assessments at baseline and unsupervised remotely administered tests of cognition from six time-points over six months.

RESULTS

The RWLRT showed acceptable validity. Reliability coefficients varied across time points, with poor reliability between times 1 and 2 and fair-to-good reliability across the remaining five testing sessions. Practice effects were observed with repeated administration as expected.

DISCUSSION

Unsupervised computerized tests of cognition, particularly word list learning and memory tests that use speech recognition, have significant potential for large scale early detection and long-term tracking of cognitive decline due to AD.

Current State of Self-Administered Brief Computerized Cognitive Assessments for Detection of Cognitive Disorders in Older Adults: A Systematic Review

Early diagnosis of cognitive disorders in older adults is a major healthcare priority with benefits to patients, families, and health systems. Rapid advances in digital technology offer potential for developing innovative diagnostic pathways to support early diagnosis. Brief self-administered computerized cognitive tools in particular hold promise for clinical implementation by minimizing demands on staff time. In this study, we conducted a systematic review of self-administered computerized cognitive assessment measures designed for the detection of cognitive impairment in older adults. Studies were identified via a systematic search of published peer-reviewed literature across major scientific databases. All studies reporting on psychometric validation of brief (≤30 minutes) self-administered computerized measures for detection of MCI and all-cause dementia in older adults were included. Seventeen studies reporting on 10 cognitive tools met inclusion criteria and were subjected to systematic review. There was substantial variability in characteristics of validation samples and reliability and validity estimates. Only 2 measures evaluated feasibility and usability in the intended clinical settings. Similar to past reviews, we found variability across measures with regard to psychometric rigor and potential for widescale applicability in clinical settings. Despite the promise that self-administered cognitive tests hold for clinical implementation, important gaps in scientific rigor in development, validation, and feasibility studies of these measures remain. Developments in technology and biomarker studies provide potential avenues for future directions on the use of digital technology in clinical care.

A new computerized assessment battery for cognition (C-ABC) to detect mild cognitive impairment and dementia around 5 min

This study aimed to develop a new computerized assessment battery for cognition (C-ABC) to detect mild cognitive impairment (MCI) and dementia. We performed C-ABC in subjects with dementia (n = 422), MCI (n = 145), and normal cognition (NC; n = 574), and analyzed by age stratum (50s, 60s, and 70–85 years). To distinguish MCI from NC, the C-ABC total combined score, which were calculated by dividing the C-ABC total score by the C-ABC required time, revealed the best area under the curves (AUC) at 0.838 and 0.735 in the 50s and 60s age groups, respectively; notably, this entire procedure took approximately 5 min. To distinguish dementia from NC and MCI, the partial items of C-ABC (items 3 + 6 combined score) revealed the best AUCs at 0.910, 0.874, and 0.882 in the 50s, 60s, and 70–85 age groups, respectively. Furthermore, the items 3 + 6 combined score established the best AUC at 0.794 in the 70–85 age group to distinguish MCI from NC; this entire procedure took around 2 min. Hence, this study suggests that C-ABC could be a useful tool for detecting dementia or MCI in a short time.

Comparing Web-Based and Lab-Based Cognitive Assessment Using the Cambridge Neuropsychological Test Automated Battery: A Within-Subjects Counterbalanced Study

Background

Computerized assessments are already used to derive accurate and reliable measures of cognitive function. Web-based cognitive assessment could improve the accessibility and flexibility of research and clinical assessment, widen participation, and promote research recruitment while simultaneously reducing costs. However, differences in context may influence task performance.

Objective

This study aims to determine the comparability of an unsupervised, web-based administration of the Cambridge Neuropsychological Test Automated Battery (CANTAB) against a typical in-person lab-based assessment, using a within-subjects counterbalanced design. The study aims to test (1) reliability, quantifying the relationship between measurements across settings using correlational approaches; (2) equivalence, the extent to which test results in different settings produce similar overall results; and (3) agreement, by quantifying acceptable limits to bias and differences between measurement environments.

Methods

A total of 51 healthy adults (32 women and 19 men; mean age 36.8, SD 15.6 years) completed 2 testing sessions, which were completed on average 1 week apart (SD 4.5 days). Assessments included equivalent tests of emotion recognition (emotion recognition task [ERT]), visual recognition (pattern recognition memory [PRM]), episodic memory (paired associate learning [PAL]), working memory and spatial planning (spatial working memory [SWM] and one touch stockings of Cambridge), and sustained attention (rapid visual information processing [RVP]). Participants were randomly allocated to one of the two groups, either assessed in-person in the laboratory first (n=33) or with unsupervised web-based assessments on their personal computing systems first (n=18). Performance indices (errors, correct trials, and response sensitivity) and median reaction times were extracted. Intraclass and bivariate correlations examined intersetting reliability, linear mixed models and Bayesian paired sample t tests tested for equivalence, and Bland-Altman plots examined agreement.

Results

Intraclass correlation (ICC) coefficients ranged from ρ=0.23-0.67, with high correlations in 3 performance indices (from PAL, SWM, and RVP tasks; ρ≥0.60). High ICC values were also seen for reaction time measures from 2 tasks (PRM and ERT tasks; ρ≥0.60). However, reaction times were slower during web-based assessments, which undermined both equivalence and agreement for reaction time measures. Performance indices did not differ between assessment settings and generally showed satisfactory agreement.

Conclusions

Our findings support the comparability of CANTAB performance indices (errors, correct trials, and response sensitivity) in unsupervised, web-based assessments with in-person and laboratory tests. Reaction times are not as easily translatable from in-person to web-based testing, likely due to variations in computer hardware. The results underline the importance of examining more than one index to ascertain comparability, as high correlations can present in the context of systematic differences, which are a product of differences between measurement environments. Further work is now needed to examine web-based assessments in clinical populations and in larger samples to improve sensitivity for detecting subtler differences between test settings.

Unsupervised assessment of cognition in the Healthy Brain Project: Implications for web-based registries of individuals at risk for Alzheimer's disease

INTRODUCTION

Web-based platforms are used increasingly to assess cognitive function in unsupervised settings. The utility of cognitive data arising from unsupervised assessments remains unclear. We examined the acceptability, usability, and validity of unsupervised cognitive testing in middle-aged adults enrolled in the Healthy Brain Project.

METHODS

A total of 1594 participants completed unsupervised assessments of the Cogstate Brief Battery. Acceptability was defined by the amount of missing data, and usability by examining error of test performance and the time taken to read task instructions and complete tests (learnability).

RESULTS

Overall, we observed high acceptability (98% complete data) and high usability (95% met criteria for low error rates and high learnability). Test validity was confirmed by observation of expected inverse relationships between performance and increasing test difficulty and age.

CONCLUSION

Consideration of test design paired with acceptability and usability criteria can provide valid indices of cognition in the unsupervised settings used to develop registries of individuals at risk for Alzheimer's disease.

Evidence of Validity for a Newly Developed Digital Cognitive Test Battery

Clinical practice still relies heavily on traditional paper-and-pencil testing to assess a patient’s cognitive functions. Digital technology has the potential to be an efficient and powerful alternative, but for many of the existing digital tests and test batteries the psychometric properties have not been properly established. We validated a newly developed digital test battery consisting of digitized versions of conventional neuropsychological tests. Two confirmatory factor analysis models were specified: a model based on traditional neuropsychological theory and expert consensus and one based on the Cattell-Horn-Carroll (CHC) taxonomy. For both models, the outcome measures of the digital tests loaded on the cognitive domains in the same way as established in the neuropsychological literature. Interestingly, no clear distinction could be made between the CHC model and traditional neuropsychological model in terms of model fit. Taken together, these findings provide preliminary evidence for the structural validity of the digital cognitive test battery.

Monitoring of Neurocognitive Function in the Care of Patients with Brain Tumors

PURPOSE OF REVIEW

A detailed characterization of the nature of neurocognitive impairment in patients with brain tumors is provided, as well as considerations for clinical practice regarding neuropsychological assessment throughout the disease course.

RECENT FINDINGS

Neurocognitive impairment is common in patients with brain tumors and may result from the tumor itself, as a consequence of treatment, including surgery, chemotherapy, and radiation, or in association with supportive care medications (e.g., anticonvulsant and pain medications). Serial surveillance of neurocognitive functioning in this population can facilitate medical decision-making and inform recommendations to improve patient daily functioning and quality of life. Neuropsychological assessment is increasingly recognized as a critical component of the multidisciplinary care of patients with brain tumors and has already had practice-changing effects. Further understanding of genetic risk factors for neurocognitive decline along with the development of novel assessment and intervention strategies may further enhance functioning and general well-being in this patient population.

Effect of physical exercise on cognitive function and brain measures after chemotherapy in patients with breast cancer (PAM study): protocol of a randomised controlled trial

INTRODUCTION

After treatment with chemotherapy, many patients with breast cancer experience cognitive problems. While limited interventions are available to improve cognitive functioning, physical exercise showed positive effects in healthy older adults and people with mild cognitive impairment. The Physical Activity and Memory study aims to investigate the effect of physical exercise on cognitive functioning and brain measures in chemotherapy-exposed patients with breast cancer with cognitive problems.

METHODS AND ANALYTICS

One hundred and eighty patients with breast cancer with cognitive problems 2-4 years after diagnosis are randomised (1:1) into an exercise intervention or a control group. The 6-month exercise intervention consists of twice a week 1-hour aerobic and strength exercises supervised by a physiotherapist and twice a week 1-hour Nordic or power walking. The control group is asked to maintain their habitual activity pattern during 6 months. The primary outcome (verbal learning) is measured at baseline and 6 months. Further measurements include online neuropsychological tests, self-reported cognitive complaints, a 3-tesla brain MRI, patient-reported outcomes (quality of life, fatigue, depression, anxiety, work performance), blood sampling and physical fitness. The MRI scans and blood sampling will be used to gain insight into underlying mechanisms. At 18 months online neuropsychological tests, self-reported cognitive complaints and patient-reported outcomes will be repeated.

ETHICS AND DISSEMINATION

Study results may impact usual care if physical exercise improves cognitive functioning for breast cancer survivors.

TRIAL REGISTRATION NUMBER

NTR6104.

Device-Embedded Cameras for Eye Tracking-Based Cognitive Assessment: Implications for Teleneuropsychology

Introduction: Widespread screening for cognitive decline is an important challenge to address as the aging population grows, but there is currently a shortage of clinical infrastructure to meet the demand for in-person evaluation. Remotely delivered assessments that utilize eye-tracking data from webcams, such as visual paired comparison (VPC) tasks, could increase access to remote, asynchronous neuropsychological screening for cognitive decline but further validation against clinical-grade eye trackers is required. Methods: To demonstrate equivalence between a novel automated scoring system for eye-tracking metrics acquired through a laptop-embedded camera and a gold-standard eye tracker, we analyzed VPC data from 18 subjects aged 50+ with normal cognitive function across three visits. The eye tracker data were scored by the manufacturer's software, and the webcam data were scored by a novel algorithm. Results: Automated scoring of webcam-based VPC data revealed strong correlations with the clinical-grade eye-tracking camera. Correlation of mean VPC performance across all time points was robust: r = 0.95 (T1 r = 0.97; T2 r = 0.88; T3 r = 0.97; p's < 0.001). Correlation of per-trial performance across time points was also robust: r = 0.88 (T1 r = 0.85; T2 r = 0.89; T3 r = 0.92; p's < 0.001). Mean differences between performance data acquired by each device were 0.00. Conclusion: These results suggest that device-embedded cameras are a valid and scalable alternative to traditional laboratory-based equipment for gaze-based tasks measuring cognitive function. The validation of this technique represents an important technical advance for the field of teleneuropsychology.