Testing a Novel Web-Based Neurocognitive Battery in the General Community: Validation and Usability Study

The Real-World Usability, Feasibility, and Performance Distributions of Deploying a Digital Toolbox of Computerized Assessments to Remotely Evaluate Brain Health: Development and Usability Study

BACKGROUND

An ongoing global challenge is managing brain health and understanding how performance changes across the lifespan.

OBJECTIVE

We developed and deployed a set of self-administrable, computerized assessments designed to measure key indexes of brain health across the visual and auditory sensory modalities. In this pilot study, we evaluated the usability, feasibility, and performance distributions of the assessments in a home-based, real-world setting without supervision.

METHODS

Potential participants were untrained users who self-registered on an existing brain training app called BrainHQ. Participants were contacted via a recruitment email and registered remotely to complete a demographics questionnaire and 29 unique assessments on their personal devices. We examined participant engagement, descriptive and psychometric properties of the assessments, associations between performance and self-reported demographic variables, cognitive profiles, and factor loadings.

RESULTS

Of the 365,782 potential participants contacted via a recruitment email, 414 (0.11%) registered, of whom 367 (88.6%) completed at least one assessment and 104 (25.1%) completed all 29 assessments. Registered participants were, on average, aged 63.6 (SD 14.8; range 13-107) years, mostly female (265/414, 64%), educated (329/414, 79.5% with a degree), and White (349/414, 84.3% White and 48/414, 11.6% people of color). A total of 72% (21/29) of the assessments showed no ceiling or floor effects or had easily modifiable score bounds to eliminate these effects. When correlating performance with self-reported demographic variables, 72% (21/29) of the assessments were sensitive to age, 72% (21/29) of the assessments were insensitive to gender, 93% (27/29) of the assessments were insensitive to race and ethnicity, and 93% (27/29) of the assessments were insensitive to education-based differences. Assessments were brief, with a mean duration of 3 (SD 1.0) minutes per task. The pattern of performance across the assessments revealed distinctive cognitive profiles and loaded onto 4 independent factors.

CONCLUSIONS

The assessments were both usable and feasible and warrant a full normative study. A digital toolbox of scalable and self-administrable assessments that can evaluate brain health at a glance (and longitudinally) may lead to novel future applications across clinical trials, diagnostics, and performance optimization.

Web-based cognitive assessment in older adults: Where do we stand?

PURPOSE OF REVIEW

The use of digital tools for remote cognitive measurement of older adults is generating increasing interest due to the numerous advantages offered for accessibility and scalability. However, these tools also pose distinctive challenges, necessitating a thorough analysis of their psychometric properties, feasibility and acceptability.

RECENT FINDINGS

In this narrative review, we present the recent literature on the use of web-based cognitive assessment to characterize cognition in older adults and to contribute to the diagnosis of age-related neurodegenerative diseases. We present and discuss three types of web-based cognitive assessments: conventional cognitive tests administered through videoconferencing; unsupervised web-based assessments conducted on a computer; and unsupervised web-based assessments performed on smartphones.

SUMMARY

There have been considerable progress documenting the properties, strengths and limitations of web-based cognitive assessments. For the three types of assessments reported here, the findings support their promising potential for older adults. However, certain aspects, such as the construct validity of these tools and the development of robust norms, remain less well documented. Nonetheless, the beneficial potential of these tools, and their current validation and feasibility data, justify their application [see Supplementary Digital Content (SDC), http://links.lww.com/CONR/A69 ].

The survey for memory, attention, and reaction time (SMART): Preliminary normative online panel data and user attitudes for a brief web-based cognitive performance measure

ObjectiveThe Survey for Memory, Attention, and Reaction Time (SMART) was recently introduced as a brief (<5 min), self-administered, web-based measure of cognitive performance in older adults. The purpose of this study was threefold: (1) to develop preliminary norms on the SMART; (2) to examine the relationship between demographic variables (i.e. age, sex, education), device type used, and SMART performance; and (3) to assess user attitudes of the SMART. Method A sample of 1,050 community-dwelling adults (M age =59.5 (15.2), M education = 16.5 (2.1), 67.1% female, 96% white) was recruited from an ongoing web-based research cohort. Participants completed the SMART, which consists of four face-valid cognitive tasks assessing visual memory, attention/processing speed, and executive functioning. SMART performance outcome metrics were subtest completion time (CT), click count, and total CT. Participants provided demographic information and completed a survey of user attitudes toward the SMART (i.e. usability, acceptability). Results Older age was the only demographic variable associated with slower SMART total CT (r = .60, p <.001). Education was not associated with SMART CT or click counts overall (p > .05). Male sex was generally associated with longer SMART CT (p < .001, partial eta squared = .14) on all sub-tests. Regarding acceptability, 97.3% indicated willingness to take the SMART again, with more than half willing to complete it on a weekly basis. Conclusion The preliminary normative data on the SMART indicates that it is a feasible and well-accepted web-based cognitive assessment tool that can be administered on multiple device platforms.