A Comparison of Electronic and Paper Versions of the Montreal Cognitive Assessment

Psychometric Evaluation of a Tablet-Based Tool to Detect Mild Cognitive Impairment in Older Adults: Mixed Methods Study

BACKGROUND

With the rapid aging of the global population, the prevalence of mild cognitive impairment (MCI) and dementia is anticipated to surge worldwide. MCI serves as an intermediary stage between normal aging and dementia, necessitating more sensitive and effective screening tools for early identification and intervention. The BrainFx SCREEN is a novel digital tool designed to assess cognitive impairment. This study evaluated its efficacy as a screening tool for MCI in primary care settings, particularly in the context of an aging population and the growing integration of digital health solutions.

OBJECTIVE

The primary objective was to assess the validity, reliability, and applicability of the BrainFx SCREEN (hereafter, the SCREEN) for MCI screening in a primary care context. We conducted an exploratory study comparing the SCREEN with an established screening tool, the Quick Mild Cognitive Impairment (Qmci) screen.

METHODS

A concurrent mixed methods, prospective study using a quasi-experimental design was conducted with 147 participants from 5 primary care Family Health Teams (FHTs; characterized by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. Participants included health care practitioners, patients, and FHT administrative executives. Individuals aged ≥55 years with no history of MCI or diagnosis of dementia rostered in a participating FHT were eligible to participate. Participants were screened using both the SCREEN and Qmci. The study also incorporated the Geriatric Anxiety Scale-10 to assess general anxiety levels at each cognitive screening. The SCREEN's scoring was compared against that of the Qmci and the clinical judgment of health care professionals. Statistical analyses included sensitivity, specificity, internal consistency, and test-retest reliability assessments.

RESULTS

The study found that the SCREEN's longer administration time and complex scoring algorithm, which is proprietary and unavailable for independent analysis, presented challenges. Its internal consistency, indicated by a Cronbach α of 0.63, was below the acceptable threshold. The test-retest reliability also showed limitations, with moderate intraclass correlation coefficient (0.54) and inadequate κ (0.15) values. Sensitivity and specificity were consistent (63.25% and 74.07%, respectively) between cross-tabulation and discrepant analysis. In addition, the study faced limitations due to its demographic skew (96/147, 65.3% female, well-educated participants), the absence of a comprehensive gold standard for MCI diagnosis, and financial constraints limiting the inclusion of confirmatory neuropsychological testing.

CONCLUSIONS

The SCREEN, in its current form, does not meet the necessary criteria for an optimal MCI screening tool in primary care settings, primarily due to its longer administration time and lower reliability. As the number of digital health technologies increases and evolves, further testing and refinement of tools such as the SCREEN are essential to ensure their efficacy and reliability in real-world clinical settings. This study advocates for continued research in this rapidly advancing field to better serve the aging population.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/25520.

Remote assessment of cognition in Parkinson's disease and Cerebellar Ataxia: the MoCA test in English and Hebrew

There is a critical need for accessible neuropsychological testing for basic research and translational studies worldwide. Traditional in-person neuropsychological studies are inherently difficult to conduct because testing requires the recruitment and participation of individuals with neurological conditions. Consequently, studies are often based on small sample sizes, are highly time-consuming, and lack diversity. To address these challenges, in the last decade, the utilization of remote testing platforms has demonstrated promising results regarding the feasibility and efficiency of collecting patient data online. Herein, we tested the validity and generalizability of remote administration of the Montreal Cognitive Assessment (MoCA) test. We administered the MoCA to English and Hebrew speakers from three different populations: Parkinson's disease, Cerebellar Ataxia, and healthy controls via video conferencing. First, we found that the online MoCA scores do not differ from traditional in-person studies, demonstrating convergent validity. Second, the MoCA scores of both our online patient groups were lower than controls, demonstrating construct validity. Third, we did not find differences between the two language versions of the remote MoCA, supporting its generalizability to different languages and the efficiency of collecting binational data (USA and Israel). Given these results, future studies can utilize the remote MoCA, and potentially other remote neuropsychological tests to collect data more efficiently across multiple different patient populations, language versions, and nations.

A pilot examination of the validity of stylus and finger drawing on visuomotor-mediated tests on ACEmobile

INTRODUCTION

Cognitive assessments, such as the Addenbrooke's Cognitive Examination (ACE-III) and Montreal Cognitive Assessment (MoCA), have been modified for administration using tablet computers. While this offers important advantages for practice, it may also threaten the test validity. The current study sought to test whether administering visuospatial and writing tests using a tablet (finger or stylus drawing), would demonstrate equivalence to traditional pencil and paper administration on ACEmobile.

METHOD

This study recruited 26 participants with Alzheimer's disease and 23 healthy older adults. Most participants had low familiarity with using a tablet computer. Participants completed ACEmobile in its entirety, after which they repeated the infinity loops, cube, and clock drawing and sentence writing tests by drawing with a stylus and their finger onto an iPad. Performance on the drawing and writing tests using a stylus, finger, and pencil were compared.

RESULTS

Statistically significant differences were observed between the finger and pencil administration on the ACEmobile, with participants performing worse on the finger drawing trials. Differences in scores were most apparent on the sentence writing task. In contrast, no statistical differences were observed between the pencil and stylus administration.

DISCUSSION

The findings of this pilot study have important implications for clinical neuropsychology and demonstrate that administering ACEmobile drawing tests with finger drawing is invalid. However, due to the small sample size, a lack of counterbalancing and the narrow range of scores of the dependent variable, we are unable to confidently interpret the validity of stylus drawing. This is an important consideration for future research.

A novel digital clock drawing test as a screening tool for perioperative neurocognitive disorders: A feasibility study

BACKGROUND

We developed a digital clock drawing test (dCDT), an adaptation of the original pen and paper clock test, that may be advantageous over previous dCDTs in the perioperative environment. We trialed our dCDT on a tablet device in the preoperative period to determine the feasibility of administration in this setting. To assess the clinical utility of this test, we examined the relationship between the performance on the test and compared derived digital clock measures with the 4 A's Test (4AT), a delirium and cognition screening tool.

METHODS

We recruited a sample of 102 adults aged 65 years and over presenting for elective surgery in a single tertiary hospital. Participants completed the 4AT, followed by both command and copy clock conditions of the dCDT. We recorded time-based clock-drawing metrics, alongside clock replications scored using the Montreal Cognitive Assessment (MoCA) clock scoring criteria.

RESULTS

The dCDT had an acceptance rate of 99%. After controlling for demographic variables and prior tablet use, regression analyses showed higher 4AT scores were associated with greater dCDT time (seconds) for both command (β = 8.2, P = .020) and copy clocks (β = 12, P = .005) and lower MoCA-based clock scores in both command (OR = 0.19, P = .001) and copy conditions (OR = 0.14, P = .012).

CONCLUSION

The digital clock drawing test is feasible to administer and is highly acceptable to older adults in a preoperative setting. We demonstrated a significant association between both the dCDT time and clock score metrics, with the established 4AT. Our results provide convergent validity of the dCDT in the preoperative setting.

Current State of Self-Administered Brief Computerized Cognitive Assessments for Detection of Cognitive Disorders in Older Adults: A Systematic Review

Early diagnosis of cognitive disorders in older adults is a major healthcare priority with benefits to patients, families, and health systems. Rapid advances in digital technology offer potential for developing innovative diagnostic pathways to support early diagnosis. Brief self-administered computerized cognitive tools in particular hold promise for clinical implementation by minimizing demands on staff time. In this study, we conducted a systematic review of self-administered computerized cognitive assessment measures designed for the detection of cognitive impairment in older adults. Studies were identified via a systematic search of published peer-reviewed literature across major scientific databases. All studies reporting on psychometric validation of brief (≤30 minutes) self-administered computerized measures for detection of MCI and all-cause dementia in older adults were included. Seventeen studies reporting on 10 cognitive tools met inclusion criteria and were subjected to systematic review. There was substantial variability in characteristics of validation samples and reliability and validity estimates. Only 2 measures evaluated feasibility and usability in the intended clinical settings. Similar to past reviews, we found variability across measures with regard to psychometric rigor and potential for widescale applicability in clinical settings. Despite the promise that self-administered cognitive tests hold for clinical implementation, important gaps in scientific rigor in development, validation, and feasibility studies of these measures remain. Developments in technology and biomarker studies provide potential avenues for future directions on the use of digital technology in clinical care.

Focusing on Emotional and Social Intelligence Stimulation of People With Dementia by Playing a Serious Game—Proof of Concept Study

Context: Dementia is one of the top five chronic diseases, which has an overwhelming impact on patients' life quality, family, and caregivers. Currently, research relating to people with dementia (PwD) focuses on the deterioration of cognitive abilities. A more innovative approach, and the one taken by this paper, is focusing on methods to maintain and improve functionality, communication and quality of life of PwD by building on remaining capacities in the yet unexplored domain of emotional and social intelligence (ESI). The use of serious games for PwD (SG4D) aimed at building social and emotional capacity is a budding field of research.

Objectives: Proof of concept that the, low cost, easy to deploy SG4D, called “My Brain Works” (MBW), co-designed with PwD, enhances ESI, based on the Bar-On ESI model.

Methods: 27 PwD, clients at MELABEV dementia day center, participated in a mixed methods 12 weeks pilot, proof of concept study using a tablet SG4D co-designed with PwD. Quantitative performance data was collected automatically by the tablet during game sessions. In this paper we focus on the analysis of the qualitative and quantitative data related to ESI, observed by 10 different researchers, during each game session.

Results: Quantitative data revealed: both the PwD with high and low MoCA scores had similar average ESI scores. Qualitative analysis revealed that the PwD demonstrated 9 sub-components of the Bar-On ESI Model.

Conclusion: While there is no drug to stop cognitive decline associated with dementia, interventions related to ESI, on the other hand, may improve functioning and quality of life. Despite declines in cognitive abilities, our study shows that a tablet based SG4D can stimulate their ESI and evoke responses in self-awareness, empathy, social and communication capacities. Using SG4D to exercise and maintain social skills is an area that may be promising in the future and may help counter the negative effects of social isolation and loneliness. Such games, while not focusing on cognitive improvement, may also impact on cognitive functioning and help bridge the gap between caregiver and PwD. More research is needed with larger sample sizes.