The necessary, albeit belated, transition to computerized cognitive assessment

The Real-World Usability, Feasibility, and Performance Distributions of Deploying a Digital Toolbox of Computerized Assessments to Remotely Evaluate Brain Health: Development and Usability Study

BACKGROUND

An ongoing global challenge is managing brain health and understanding how performance changes across the lifespan.

OBJECTIVE

We developed and deployed a set of self-administrable, computerized assessments designed to measure key indexes of brain health across the visual and auditory sensory modalities. In this pilot study, we evaluated the usability, feasibility, and performance distributions of the assessments in a home-based, real-world setting without supervision.

METHODS

Potential participants were untrained users who self-registered on an existing brain training app called BrainHQ. Participants were contacted via a recruitment email and registered remotely to complete a demographics questionnaire and 29 unique assessments on their personal devices. We examined participant engagement, descriptive and psychometric properties of the assessments, associations between performance and self-reported demographic variables, cognitive profiles, and factor loadings.

RESULTS

Of the 365,782 potential participants contacted via a recruitment email, 414 (0.11%) registered, of whom 367 (88.6%) completed at least one assessment and 104 (25.1%) completed all 29 assessments. Registered participants were, on average, aged 63.6 (SD 14.8; range 13-107) years, mostly female (265/414, 64%), educated (329/414, 79.5% with a degree), and White (349/414, 84.3% White and 48/414, 11.6% people of color). A total of 72% (21/29) of the assessments showed no ceiling or floor effects or had easily modifiable score bounds to eliminate these effects. When correlating performance with self-reported demographic variables, 72% (21/29) of the assessments were sensitive to age, 72% (21/29) of the assessments were insensitive to gender, 93% (27/29) of the assessments were insensitive to race and ethnicity, and 93% (27/29) of the assessments were insensitive to education-based differences. Assessments were brief, with a mean duration of 3 (SD 1.0) minutes per task. The pattern of performance across the assessments revealed distinctive cognitive profiles and loaded onto 4 independent factors.

CONCLUSIONS

The assessments were both usable and feasible and warrant a full normative study. A digital toolbox of scalable and self-administrable assessments that can evaluate brain health at a glance (and longitudinally) may lead to novel future applications across clinical trials, diagnostics, and performance optimization.

Robotic assessment of sensorimotor and cognitive deficits in patients with temporal lobe epilepsy

OBJECTIVE

Individuals with temporal lobe epilepsy (TLE) frequently demonstrate impairments in executive function, working memory, and/or declarative memory. It is recommended that screening for cognitive impairment is undertaken in all people newly diagnosed with epilepsy. However, standard neuropsychological assessments are a limited resource and thus not available to all. Our study investigated the use of robotic technology (the Kinarm robot) for cognitive screening.

METHODS

27 participants with TLE (17 left) underwent both a brief neuropsychological screening and a robotic (Kinarm) assessment. The degree of impairments and correlations between standardized scores from both approaches to assessments were analysed across different neurocognitive domains. Performance was compared between people with left and right TLE to look for laterality effects. Finally, the association between the duration of epilepsy and performance was assessed.

RESULTS

Across the 6 neurocognitive domains (attention, executive function, language, memory, motor and visuospatial) assessed by our neuropsychological screening, all showed scores that significantly correlated with Kinarm tasks assessing the same cognitive domains except language and memory that were not adequately assessed with Kinarm. Participants with right TLE performed worse on most tasks than those with left TLE, including both visuospatial (typically considered right hemisphere), and verbal memory and language tasks (typically considered left hemisphere). No correlations were found between the duration of epilepsy and either the neuropsychological screening or Kinarm assessment.

SIGNIFICANCE

Our findings suggest that Kinarm may be a useful tool in screening for neurocognitive impairment in people with TLE. Further development may facilitate an easier and more rapid screening of cognition in people with epilepsy and distinguishing patterns of cognitive impairment.

Validity and reliability of a digital solution for cognitive assessment: The Brain on Track®

Background

Cognitive assessment and the early detection of cognitive impairments have been enhanced by the use of remote digital solutions. The Brain on Track® is one of these digital solutions used in clinical practice for online screening and monitoring of cognitive functioning.

Objectives

This study aimed to explore the validity and reliability of the Brain on Track® computerized test on a tablet device in adults.

Methods

A community sample of 54 young adults, 51 middle-aged adults, and 50 older adults were invited to attend in two assessment sessions. The first session included data collection on sociodemographic data, user digital literacy, Brain on Track® on the computer and on the tablet device, and usability from the user and moderator perspective. The second session included the Montreal Cognitive Assessment Questionnaire (MoCA) and a second completion of the Brain on Track® on tablet to assess the criterion validity and test-retest reliability. Hypothesis testing was used to assess construct validity.

Results

A weak to moderate correlation was found between the Brain on Track® tablet score and the MoCA. The ICC was 0.72, 0.84, and 0.79, and Cronbach's alpha was 0.84, 0.83, and 0.89 in young adults, middle-aged adults, and older adults, respectively.

Conclusions

This study suggested that the Brain on Track® administered using a tablet device has criterion validity, particularly in middle-aged and older adults, and internal consistency and test-retest reliability in adults of any age group.

Cognitive decline monitoring through a web-based application

Cognitive decline usually begins after individuals reach maturity, which is more evident in late adulthood. Rapid and constant cognitive screenings allow early detection of cognitive decline and motivate individuals to participate in prevention interventions. Due to accelerated technological advances, cognitive screening and training are now available to the layperson using electronic devices connected to the internet. Large datasets generated by these platforms provide a unique opportunity to explore cognitive development throughout life and across multiple naturalistic environments. However, such data collection mechanisms must be validated. This study aimed to determine whether the data gathered by commercial visuospatial and phonological working memory tests (CogniFit Inc., San Francisco, USA) confirm the well-established argument that age predicts cognitive decline. Data from 3,212 participants (2,238 females) who were 45 years old or older were analyzed. A linear regression analysis explored the relationship between age and working memory while controlling for gender, sleep quality, and physical activity (variables that are known to affect working memory). We found that age negatively predicts working memory. Furthermore, there was an interaction between age and gender for visuospatial working memory, indicating that although male participants significantly outperformed females, the relationship between age and working memory differs for females and males. Our results suggest that the computerized assessment of visuospatial and phonological working memory is sensible enough to predict cognitive functions in aging. Suggestions for improving the sensitivity of self-reports are discussed. Further studies must explore the nature of gender effects on cognitive aging.