Unsupervised mobile cognitive testing for use in preclinical Alzheimer's disease

Remote and unsupervised digital memory assessments can reliably detect cognitive impairment in Alzheimer's disease

INTRODUCTION

Remote unsupervised cognitive assessments have the potential to complement and facilitate cognitive assessment in clinical and research settings.

METHODS

Here, we evaluate the usability, validity, and reliability of unsupervised remote memory assessments via mobile devices in individuals without dementia from the Swedish BioFINDER-2 study and explore their prognostic utility regarding future cognitive decline.

RESULTS

Usability was rated positively; remote memory assessments showed good construct validity with traditional neuropsychological assessments and were significantly associated with tau-positron emission tomography and downstream magnetic resonance imaging measures. Memory performance at baseline was associated with future cognitive decline and prediction of future cognitive decline was further improved by combining remote digital memory assessments with plasma p-tau217. Finally, retest reliability was moderate for a single assessment and good for an aggregate of two sessions.

DISCUSSION

Our results demonstrate that unsupervised digital memory assessments might be used for diagnosis and prognosis in Alzheimer's disease, potentially in combination with plasma biomarkers.

HIGHLIGHTS

Remote and unsupervised digital memory assessments are feasible in older adults and individuals in early stages of Alzheimer's disease. Digital memory assessments are associated with neuropsychological in-clinic assessments, tau-positron emission tomography and magnetic resonance imaging measures. Combination of digital memory assessments with plasma p-tau217 holds promise for prognosis of future cognitive decline. Future validation in further independent, larger, and more diverse cohorts is needed to inform clinical implementation.

Capturing learning curves with the multiday Boston Remote Assessment of Neurocognitive Health (BRANCH): Feasibility, reliability, and validity

OBJECTIVE

Unsupervised remote digital cognitive assessment makes frequent testing feasible and allows for measurement of learning over repeated evaluations on participants' own devices. This provides the opportunity to derive individual multiday learning curve scores over short intervals. Here, we report feasibility, reliability, and validity, of a 7-day cognitive battery from the Boston Remote Assessment for Neurocognitive Health (Multiday BRANCH), an unsupervised web-based assessment.

METHOD

Multiday BRANCH was administered remotely to 181 cognitively unimpaired older adults using their own electronic devices. For 7 consecutive days, participants completed three tests with associative memory components (Face-Name, Groceries-Prices, Digit Signs), using the same stimuli, to capture multiday learning curves for each test. We assessed the feasibility of capturing learning curves across the 7 days. Additionally, we examined the reliability and associations of learning curves with demographics, and traditional cognitive and subjective report measures.

RESULTS

Multiday BRANCH was feasible with 96% of participants completing all study assessments; there were no differences dependent on type of device used (t = 0.71, p = .48) or time of day completed (t = -0.08, p = .94). Psychometric properties of the learning curves were sound including good test-retest reliability of individuals' curves (intraclass correlation = 0.94). Learning curves were positively correlated with in-person cognitive tests and subjective report of cognitive complaints.

CONCLUSIONS

Multiday BRANCH is a feasible, reliable, and valid cognitive measure that may be useful for identifying subtle changes in learning and memory processes in older adults. In the future, we will determine whether Multiday BRANCH is predictive of the presence of preclinical Alzheimer's disease. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Design and Feasibility Analysis of a Smartphone-Based Digital Cognitive Assessment Study in the Framingham Heart Study

BACKGROUND

Smartphone-based digital technology is increasingly being recognized as a cost-effective, scalable, and noninvasive method of collecting longitudinal cognitive and behavioral data. Accordingly, a state-of-the-art 3-year longitudinal project focused on collecting multimodal digital data for early detection of cognitive impairment was developed.

METHODS AND RESULTS

A smartphone application collected 2 modalities of cognitive data, digital voice and screen-based behaviors, from the FHS (Framingham Heart Study) multigenerational Generation 2 (Gen 2) and Generation 3 (Gen 3) cohorts. To understand the feasibility of conducting a smartphone-based study, participants completed a series of questions about their smartphone and app use, as well as sensory and environmental factors that they encountered while completing the tasks on the app. Baseline data collected to date were from 537 participants (mean age=66.6 years, SD=7.0; 58.47% female). Across the younger participants from the Gen 3 cohort (n=455; mean age=60.8 years, SD=8.2; 59.12% female) and older participants from the Gen 2 cohort (n=82; mean age=74.2 years, SD=5.8; 54.88% female), an average of 76% participants agreed or strongly agreed that they felt confident about using the app, 77% on average agreed or strongly agreed that they were able to use the app on their own, and 81% on average rated the app as easy to use.

CONCLUSIONS

Based on participant ratings, the study findings are promising. At baseline, the majority of participants are able to complete the app-related tasks, follow the instructions, and encounter minimal barriers to completing the tasks independently. These data provide evidence that designing and collecting smartphone application data in an unsupervised, remote, and naturalistic setting in a large, community-based population is feasible.

Predicting Brain Amyloid Status Using the National Institute of Health Toolbox (NIHTB) for Assessment of Neurological and Behavioral Function

BACKGROUND

Amyloid-beta (Aβ) plaque is a neuropathological hallmark of Alzheimer's disease (AD). As anti-amyloid monoclonal antibodies enter the market, predicting brain amyloid status is critical to determine treatment eligibility.

OBJECTIVE

To predict brain amyloid status utilizing machine learning approaches in the Advancing Reliable Measurement in Alzheimer's Disease and Cognitive Aging (ARMADA) study.

DESIGN

ARMADA is a multisite study that implemented the National Institute of Health Toolbox for Assessment of Neurological and Behavioral Function (NIHTB) in older adults with different cognitive ability levels (normal, mild cognitive impairment, early-stage dementia of the AD type).

SETTING

Participants across various sites were involved in the ARMADA study for validating the NIHTB.

PARTICIPANTS

199 ARMADA participants had either PET or CSF information (mean age 76.3 ± 7.7, 51.3% women, 42.3% some or complete college education, 50.3% graduate education, 88.9% White, 33.2% with positive AD biomarkers).

MEASUREMENTS

We used cognition, emotion, motor, sensation scores from NIHTB, and demographics to predict amyloid status measured by PET or CSF. We applied LASSO and random forest models and used the area under the receiver operating curve (AUROC) to evaluate the ability to identify amyloid positivity.

RESULTS

The random forest model reached AUROC of 0.74 with higher specificity than sensitivity (AUROC 95% CI:0.73 - 0.76, Sensitivity 0.50, Specificity 0.88) on the held-out test set; higher than the LASSO model (0.68 (95% CI:0.68 - 0.69)). The 10 features with the highest importance from the random forest model are: picture sequence memory, cognition total composite, cognition fluid composite, list sorting working memory, words-in-noise test (hearing), pattern comparison processing speed, odor identification, 2-minutes-walk endurance, 4-meter walk gait speed, and picture vocabulary. Overall, our model revealed the validity of measurements in cognition, motor, and sensation domains, in associating with AD biomarkers.

CONCLUSION

Our results support the utilization of the NIH toolbox as an efficient and standardizable AD biomarker measurement that is better at identifying amyloid negative (i.e., high specificity) than positive cases (i.e., low sensitivity).

Adherence type impacts completion rates of frequent mobile cognitive assessments among older adults with and without cognitive impairment

Background

Prior to a diagnosis of Alzheimer's disease, many individuals experience cognitive and behavioral fluctuations that are not detected during a single session of traditional neuropsychological assessment. Mobile applications now enable high-frequency cognitive data to be collected remotely, introducing new opportunities and challenges. Emerging evidence suggests cognitively impaired older adults are capable of completing mobile assessments frequently, but no study has observed whether completion rates vary by assessment frequency or adherence type.

Methods

Thirty-three older adults were recruited from the Boston University Alzheimer's Disease Research Center (mean age = 73.5 years; 27.3% cognitively impaired; 57.6% female; 81.8% White, 18.2% Black). Participants remotely downloaded and completed the DANA Brain Vital application on their own mobile devices throughout the study. The study schedule included seventeen assessments to be completed over the course of a year. Specific periods during which assessments were expected to be completed were defined as subsegments, while segments consisted of multiple subsegments. The first segment included three subsegments to be completed within one week, the second segment included weekly subsegments and spanned three weeks, and the third and fourth segments included monthly subsegments spanning five and six months, respectively. Three distinct adherence types - subsegment adherence, segment adherence, and cumulative adherence - were examined to determine how completion rates varied depending on assessment frequency and adherence type.

Results

Adherence type significantly impacted whether the completion rates declined. When utilizing subsegment adherence, the completion rate significantly declined (p = 0.05) during the fourth segment. However, when considering completion rates from the perspective of segment adherence, a decline in completion rate was not observed. Overall adherence rates increased as adherence parameters were broadened from subsegment adherence (60.6%) to segment adherence (78.8%), to cumulative adherence (90.9%).

Conclusions

Older adults, including those with cognitive impairment, are able to complete remote cognitive assessments at a high-frequency, but may not necessarily adhere to prescribed schedules.

Intensive assessment of executive functions derived from performance in cognitive training games

Traditional neuropsychological tests accurately describe the current cognitive state but fall short to characterize cognitive change over multiple short time periods. We present an innovative approach to remote monitoring of executive functions on a monthly basis, which leverages the performance indicators from self-administered computerized cognitive training games (NUP-EXE). We evaluated the measurement properties of NUP-EXE in N = 56 individuals (59% women, 60-80 years) at increased risk of Alzheimer's disease (APOE-ϵ4 carriers with subjective cognitive decline) who completed a 12-month multimodal intervention for preventing cognitive decline. NUP-EXE presented good psychometric properties and greater sensitivity to change than traditional tests. Improvements in NUP-EXE correlated with improvements in functionality and were affected by participants' age and gender. This novel data collection methodology is expected to allow a more accurate characterization of an individual's response to a cognitive decline preventive intervention and to inform development of outcome measures for a new generation of intervention trials.

Remote Digital Technologies for the Early Detection and Monitoring of Cognitive Decline in Patients With Type 2 Diabetes: Insights From Studies of Neurodegenerative Diseases

Type 2 diabetes (T2D) is a risk factor for cognitive decline. In neurodegenerative disease research, remote digital cognitive assessments and unobtrusive sensors are gaining traction for their potential to improve early detection and monitoring of cognitive impairment. Given the high prevalence of cognitive impairments in T2D, these digital tools are highly relevant. Further research incorporating remote digital biomarkers of cognition, behavior, and motor functioning may enable comprehensive characterizations of patients with T2D and may ultimately improve clinical care and equitable access to research participation. The aim of this commentary article is to review the feasibility, validity, and limitations of using remote digital cognitive tests and unobtrusive detection methods to identify and monitor cognitive decline in neurodegenerative conditions and apply these insights to patients with T2D.

Assessment of cognitive performance in multiple sclerosis using smartphone-based training games: a feasibility study

BACKGROUND

Cognitive impairment occurs in up to 70% of people with MS (pwMS) and has a large impact on quality of life and working capacity. As part of the development of a smartphone-app (dreaMS) for monitoring MS disease activity and progression, we assessed the feasibility and acceptance of using cognitive games as assessment tools for cognitive domains.

METHODS

We integrated ten cognitive games in the dreaMS app. Participants were asked to play these games twice a week for 5 weeks. All subjects underwent a battery of established neuropsychological tests. User feedback on acceptance was obtained via a five-point Likert-scale questionnaire. We correlated game performance measures with predetermined reference tests (Spearman's rho) and analyzed differences between pwMS and Healthy Controls (rank biserial correlation).

RESULTS

We included 31 pwMS (mean age 43.4 ± 12.0 years; 68% females; median Expanded Disability Status Scale score 3.0, range 1.0-6.0) and 31 age- and sex-matched HC. All but one game showed moderate-strong correlations with their reference tests, (|r_s|= 0.34-0.77). Performance improved in both groups over the 5 weeks. Average ratings for overall impression and meaningfulness were 4.6 (range 4.2-4.9) and 4.7 (range 4.5-4.8), respectively.

CONCLUSION

Moderate-strong correlations with reference tests suggest that adaptive cognitive games may be used as measures of cognitive domains. The practice effects observed suggest that game-derived measures may capture change over time. All games were perceived as enjoyable and meaningful, features crucial for long-term adherence. Our results encourage further validation of adaptive cognitive games as monitoring tools for cognition in larger studies of longer duration.

STUDY REGISTER

ClinicalTrials.gov: NCT04413032.

Associations between Participant Characteristics and Participant Feedback about an Unsupervised Online Cognitive Assessment in a Research Registry

BACKGROUND

This study aims to understand whether and how participant characteristics (age, gender, education, ethnocultural identity) are related to their feedback about taking a remote, unsupervised, online cognitive assessment.

METHODS

The Brain Health Registry is a public online registry which includes cognitive assessments. Multivariable ordinal regressions assessed associations between participant characteristics and feedback responses of older (55+) participants (N=11,553) regarding their Cogstate Brief Battery assessment experience.

RESULTS

Higher age, secondary education or less, Latino identity, and female gender were associated with a poorer assessment experience; higher age and a non-White identity were associated with experiencing the assessment instructions as less clear; and higher age, non-White identity, and secondary education or less were associated with rating additional human support with the assessment as more useful.

DISCUSSION

Our findings highlight the importance of improving the design and instructions of unsupervised, remote, online cognitive assessments to better suit the needs of diverse communities.

Unsupervised mobile app-based cognitive testing in a population-based study of older adults born 1944

BACKGROUND

Mobile app-based tools have the potential to yield rapid, cost-effective, and sensitive measures for detecting dementia-related cognitive impairment in clinical and research settings. At the same time, there is a substantial need to validate these tools in real-life settings. The primary aim of this study was thus to evaluate the feasibility, validity, and reliability of mobile app-based tasks for assessing cognitive function in a population-based sample of older adults.

METHOD

A total of 172 non-demented (Clinical Dementia Rating 0 and 0.5) older participants (aged 76-77) completed two mobile app-based memory tasks-the Mnemonic Discrimination Task for Objects and Scenes (MDT-OS) and the long-term (24 h) delayed Object-In-Room Recall Task (ORR-LDR). To determine the validity of the tasks for measuring relevant cognitive functions in this population, we assessed relationships with conventional cognitive tests. In addition, psychometric properties, including test-retest reliability, and the participants' self-rated experience with mobile app-based cognitive tasks were assessed.

RESULT

MDT-OS and ORR-LDR were weakly-to-moderately correlated with the Preclinical Alzheimer's Cognitive Composite (PACC5) (r = 0.3-0.44, p < .001) and with several other measures of episodic memory, processing speed, and executive function. Test-retest reliability was poor-to-moderate for one single session but improved to moderate-to-good when using the average of two sessions. We observed no significant floor or ceiling effects nor effects of education or gender on task performance. Contextual factors such as distractions and screen size did not significantly affect task performance. Most participants deemed the tasks interesting, but many rated them as highly challenging. While several participants reported distractions during tasks, most could concentrate well. However, there were difficulties in completing delayed recall tasks on time in this unsupervised and remote setting.

CONCLUSION

Our study proves the feasibility of mobile app-based cognitive assessments in a community sample of older adults, demonstrating its validity in relation to conventional cognitive measures and its reliability for repeated measurements over time. To further strengthen study adherence, future studies should implement additional measures to improve task completion on time.

Validation of a Remote and Fully Automated Story Recall Task to Assess for Early Cognitive Impairment in Older Adults: Longitudinal Case-Control Observational Study

Background

Story recall is a simple and sensitive cognitive test that is commonly used to measure changes in episodic memory function in early Alzheimer disease (AD). Recent advances in digital technology and natural language processing methods make this test a candidate for automated administration and scoring. Multiple parallel test stimuli are required for higher-frequency disease monitoring.

Objective

This study aims to develop and validate a remote and fully automated story recall task, suitable for longitudinal assessment, in a population of older adults with and without mild cognitive impairment (MCI) or mild AD.

Methods

The “Amyloid Prediction in Early Stage Alzheimer’s disease” (AMYPRED) studies recruited participants in the United Kingdom (AMYPRED-UK: NCT04828122) and the United States (AMYPRED-US: NCT04928976). Participants were asked to complete optional daily self-administered assessments remotely on their smart devices over 7 to 8 days. Assessments included immediate and delayed recall of 3 stories from the Automatic Story Recall Task (ASRT), a test with multiple parallel stimuli (18 short stories and 18 long stories) balanced for key linguistic and discourse metrics. Verbal responses were recorded and securely transferred from participants’ personal devices and automatically transcribed and scored using text similarity metrics between the source text and retelling to derive a generalized match score. Group differences in adherence and task performance were examined using logistic and linear mixed models, respectively. Correlational analysis examined parallel-forms reliability of ASRTs and convergent validity with cognitive tests (Logical Memory Test and Preclinical Alzheimer’s Cognitive Composite with semantic processing). Acceptability and usability data were obtained using a remotely administered questionnaire.

Results

Of the 200 participants recruited in the AMYPRED studies, 151 (75.5%)—78 cognitively unimpaired (CU) and 73 MCI or mild AD—engaged in optional remote assessments. Adherence to daily assessment was moderate and did not decline over time but was higher in CU participants (ASRTs were completed each day by 73/106, 68.9% participants with MCI or mild AD and 78/94, 83% CU participants). Participants reported favorable task usability: infrequent technical problems, easy use of the app, and a broad interest in the tasks. Task performance improved modestly across the week and was better for immediate recall. The generalized match scores were lower in participants with MCI or mild AD (Cohen d=1.54). Parallel-forms reliability of ASRT stories was moderate to strong for immediate recall (mean rho 0.73, range 0.56-0.88) and delayed recall (mean rho=0.73, range=0.54-0.86). The ASRTs showed moderate convergent validity with established cognitive tests.

Conclusions

The unsupervised, self-administered ASRT task is sensitive to cognitive impairments in MCI and mild AD. The task showed good usability, high parallel-forms reliability, and high convergent validity with established cognitive tests. Remote, low-cost, low-burden, and automatically scored speech assessments could support diagnostic screening, health care, and treatment monitoring.

The ambulatory battery of creativity: Additional evidence for reliability and validity

Psychometrically sound instruments that assess temporal dynamics of creative abilities are limited. The Ambulatory Battery of Creativity (ABC) is designed to assess creative ideation performance multiple times in everyday life and was proven to capture the intra-individual dynamic of creative abilities reliably and validly. The present ambulatory study aimed to replicate and extend the psychometric evidence of the novel ABC. Sixty-nine participants worked on the ABC during a 5-day ambulatory assessment protocol. Each day, participants completed six randomly presented items of the verbal and the figural ABC. Matching previous psychometric analyses, the results indicated good between-person (≥0.80) and good within-person (≥0.72) reliability. Furthermore, evidence for between-person and within-person validity of the ABC was obtained. Performance in the verbal and the figural ABC were interrelated and correlated with an independent measure of creative potential. The verbal ABC was further associated with openness, self-reported creative behavior, creative activities, and creative achievements, thus providing additional evidence of construct validity, especially for the verbal ABC. Finally, the verbal and the figural ABC yielded convincing within-person validity: Longer response times and higher subjective originality ratings were associated with more original ideas. This replication and extension of the ABC’s psychometric properties indicates that it enables a reliable and valid assessment of moment-to-moment fluctuations of creative ideation abilities in everyday life, which may facilitate the investigation of exciting new research questions related to dynamic aspects of creative ability.

Exploratory Research on Key Technology of Human-Computer Interactive 2.5-Minute Fast Digital Early Warning for Mild Cognitive Impairment

Objective. As the preclinical stage of Alzheimer’s disease (AD), Mild Cognitive Impairment (MCI) is characterized by hidden onset, which is difficult to detect early. Traditional neuropsychological scales are main tools used for assessing MCI. However, due to its strong subjectivity and the influence of many factors such as subjects’ educational background, language and hearing ability, and time cost, its accuracy as the standard of early screening is low. Therefore, the purpose of this paper is to propose a new key technology of fast digital early warning for MCI based on eye movement objective data analysis. Methodology. Firstly, four exploratory indexes (test durations, correlation degree, lengths of gaze trajectory, and drift rate) of MCI early warning are determined based on the relevant literature research and semistructured expert interview; secondly, the eye movement state is captured based on the eye tracker to realize the data extraction of four exploratory indexes. On this basis, the human-computer interactive 2.5-minute fast digital early warning paradigm for MCI is designed; thirdly, the rationality of the four early warning indexes proposed in this paper and their early warning effectiveness on MCI are verified. Results. Through the small sample test of human-computer interactive 2.5 fast digital early warning paradigm for MCI conducted by 32 elderly people aged 70–90 in a medical institution in Hangzhou, the two indexes of “correlation degree” and “drift rate” with statistical differences are selected. The experiment results show that AUC of this MCI early warning paradigm is 0.824. Conclusion. The key technology of human-computer interactive 2.5 fast digital early warning for MCI proposed in this paper overcomes the limitations of the existing MCI early warning tools, such as low objectification level, high dependence on professional doctors, long test time, requiring high educational level, and so on. The experiment results show that the early warning technology, as a new generation of objective and effective digital early warning tool, can realize 2.5-minute fast and high-precision preliminary screening and early warning for MCI in the elderly.