Distance Assessment for Detecting Cognitive Impairment in Older Adults: A Systematic Review of Psychometric Evidence

Tele-neuropsychological multidomain assessment in Italian people with cognitive disorders: Reliability and user satisfaction

BACKGROUND

Tele-neuropsychology has already been employed in neurocognitive disorders, however, in Italy, the evidence of its psychometric quality and satisfaction is still limited.

OBJECTIVE

This study aimed to: (1) evaluate the reliability of a standardized battery of neuropsychological screening and domain-specific tests delivered at home via videoconference to a sample of Italian people with cognitive disorders, compared with traditional face-to-face administration; (2) assess the feasibility and satisfaction about remote administration.

METHODS

This crossover study enrolled patients with subjective cognitive disorder, mild neurocognitive disorder, or dementia. All participants performed a brief neuropsychological screening assessment in face-to-face or remote mode. People with milder cognitive disorders (Mini-Mental State Examination ≥ 20) also performed an extended neuropsychological battery. After 15 days, each participant repeated the same assessment in the opposite mode. Finally, participants completed a satisfaction questionnaire.

RESULTS

114 participants were initially enrolled in the study and 100 individuals performed both face-to-face and remote assessments (14 dropouts). All neuropsychological tests showed agreement between the two modes of administration, except for the Digit Span Forward (p = 0.009) and oral-Symbol Digit Modalities Test (p < 0.001). According to Lin's concordance correlation coefficient (LCCCs), tests also showed good or excellent reliability (LCCCs between 0.609 and 0.964); only the Digit Span Backward and the Stroop test showed moderate reliability (LCCCs =0.514-0.441, respectively). Among 100 participants, 65 patients answered the satisfaction questionnaire, declaring high satisfaction for the remote evaluation.

CONCLUSIONS

Tele-neuropsychology seems to work with in-home assessments among Italian patients with cognitive disorders and seems to be acceptable to them.

TRIAL REGISTRATION

ClinicalTrials.gov, Identifier: NCT06078332 (October 17, 2023).

Detecting mild cognitive impairment remotely with the modified memory impairment screen by telephone

The original Memory Impairment Screen by Telephone (MIST) was designed to identify individuals with dementia but was relatively ineffective for identification of less severe impairment observed in mild cognitive impairment (MCI). We expanded the original MIST to create a modified instrument (mMIST) with greater sensitivity to less severe memory impairment. Older men and women with subjective cognitive decline were assessed by phone with the mMIST and subsequently classified independently with MCI or non-pathological cognitive decline. Participants with MCI produced lower scores on the mMIST than did participants without MCI, 10.8 ± 2.7 vs 13.3 ± 1.3, t = 5.68, p < 0.001, and performance on the mMIST predicted performances on the California Verbal Learning Test (CVLT), Verbal Paired Associate Learning Test (VPAL), Montreal Cognitive Assessment (MoCA) total score, and MoCA memory index score, p < 0.001. Receiver operating characteristic (ROC) analyses identified the optimal cut score on the mMIST to distinguish participants with and without MCI with Sensitivity = 73.1%, Specificity = 79.1%, and AUC = 0.79. Predictive values for distinguishing the amnestic form of MCI (aMCI) from non-amnestic MCI were Sensitivity = 81.8%, Specificity = 30%, and AUC = 0.82. These findings indicate that the mMIST is a valid screening instrument for identifying MCI. It can be administered remotely at low cost and low participant burden. Also, the mMIST has potential utility for remote cross-sectional and longitudinal evaluation in research and clinical contexts. Further investigation is indicated to corroborate its utility for assessment of aging patients and research participants.

Web-based cognitive assessment in older adults: Where do we stand?

PURPOSE OF REVIEW

The use of digital tools for remote cognitive measurement of older adults is generating increasing interest due to the numerous advantages offered for accessibility and scalability. However, these tools also pose distinctive challenges, necessitating a thorough analysis of their psychometric properties, feasibility and acceptability.

RECENT FINDINGS

In this narrative review, we present the recent literature on the use of web-based cognitive assessment to characterize cognition in older adults and to contribute to the diagnosis of age-related neurodegenerative diseases. We present and discuss three types of web-based cognitive assessments: conventional cognitive tests administered through videoconferencing; unsupervised web-based assessments conducted on a computer; and unsupervised web-based assessments performed on smartphones.

SUMMARY

There have been considerable progress documenting the properties, strengths and limitations of web-based cognitive assessments. For the three types of assessments reported here, the findings support their promising potential for older adults. However, certain aspects, such as the construct validity of these tools and the development of robust norms, remain less well documented. Nonetheless, the beneficial potential of these tools, and their current validation and feasibility data, justify their application [see Supplementary Digital Content (SDC), http://links.lww.com/CONR/A69 ].

Health economic analysis of the integrated cognitive assessment tool to aid dementia diagnosis in the United Kingdom

Objectives

The aim of this study was to develop a comprehensive economic evaluation of the integrated cognitive assessment (ICA) tool compared with standard cognitive tests when used for dementia screening in primary care and for initial patient triage in memory clinics.

Methods

ICA was compared with standard of care comprising a mixture of cognitive assessment tools over a lifetime horizon and employing the UK health and social care perspective. The model combined a decision tree to capture the initial outcomes of the cognitive testing with a Markov structure that estimated long-term outcomes of people with dementia. Quality of life outcomes were quantified using quality-adjusted life years (QALYs), and the economic benefits were assessed using net monetary benefit (NMB). Both costs and QALYs were discounted at 3.5% per annum and cost-effectiveness was assessed using a threshold of £20,000 per QALY gained.

Results

ICA dominated standard cognitive assessment tools in both the primary care and memory clinic settings. Introduction of the ICA tool was estimated to result in a lifetime cost saving of approximately £123 and £226 per person in primary care and memory clinics, respectively. QALY gains associated with early diagnosis were modest (0.0016 in primary care and 0.0027 in memory clinic). The net monetary benefit (NMB) of ICA introduction was estimated at £154 in primary care and £281 in the memory clinic settings.

Conclusion

Introduction of ICA as a tool to screen primary care patients for dementia and perform initial triage in memory clinics could be cost saving to the UK public health and social care payer.

The use of the integrated cognitive assessment to improve the efficiency of primary care referrals to memory services in the accelerating dementia pathway technologies study

Background

Current primary care cognitive assessment tools are either crude or time-consuming instruments that can only detect cognitive impairment when it is well established. This leads to unnecessary or late referrals to memory services, by which time the disease may have already progressed into more severe stages. Due to the COVID-19 pandemic, some memory services have adapted to the new environment by shifting to remote assessments of patients to meet service user demand. However, the use of remote cognitive assessments has been inconsistent, and there has been little evaluation of the outcome of such a change in clinical practice. Emerging research has highlighted computerized cognitive tests, such as the Integrated Cognitive Assessment (ICA), as the leading candidates for adoption in clinical practice. This is true both during the pandemic and in the post-COVID-19 era as part of healthcare innovation.

Objectives

The Accelerating Dementias Pathways Technologies (ADePT) Study was initiated in order to address this challenge and develop a real-world evidence basis to support the adoption of ICA as an inexpensive screening tool for the detection of cognitive impairment and improving the efficiency of the dementia care pathway.

Methods

Ninety-nine patients aged 55-90 who have been referred to a memory clinic by a general practitioner (GP) were recruited. Participants completed the ICA either at home or in the clinic along with medical history and usability questionnaires. The GP referral and ICA outcome were compared with the specialist diagnosis obtained at the memory clinic.Participants were given the option to carry out a retest visit where they were again given the chance to take the ICA test either remotely or face-to-face.

Results

The primary outcome of the study compared GP referral with specialist diagnosis of mild cognitive impairment (MCI) and dementia. Of those the GP referred to memory clinics, 78% were necessary referrals, with ~22% unnecessary referrals, or patients who should have been referred to other services as they had disorders other than MCI/dementia. In the same population the ICA was able to correctly identify cognitive impairment in ~90% of patients, with approximately 9% of patients being false negatives. From the subset of unnecessary GP referrals, the ICA classified ~72% of those as not having cognitive impairment, suggesting that these unnecessary referrals may not have been made if the ICA was in use. ICA demonstrated a sensitivity of 93% for dementia and 83% for MCI, with a specificity of 80% for both conditions in detecting cognitive impairment. Additionally, the test-retest prediction agreement for the ICA was 87.5%.

Conclusion

The results from this study demonstrate the potential of the ICA as a screening tool, which can be used to support accurate referrals from primary care settings, along with the work conducted in memory clinics and in secondary care. The ICA's sensitivity and specificity in detecting cognitive impairment in MCI surpassed the overall standard of care reported in existing literature.

Preferences of NIA Alzheimer's Disease Research Center participants regarding remote assessment during the COVID-19 pandemic

Introduction

During the COVID-19 pandemic, in-person research study visits were moved to an online format using a variety of communication platforms (e.g., Webex and Zoom). Increased technology use among older adults allowed for greater insight regarding the remote research study visit format.

Methods

A survey developed by the National Alzheimer's Coordinating Center (NACC) was distributed among 12 Alzheimer's Disease Research Centers (ADRCs). The COVID-19 Technology Accessibility Survey aimed to understand preferences of older adults regarding their research study visits and how they accessed the internet.

Results

Among 12 ADRCs, 2070 responses were received (mean age: 72.8 years [standard deviation (SD) = 10.4], mean education: 16.6 years [SD = 2.6], race/ethnicity: 85% White/non-Hispanic). Among respondents, those with some form of cognitive impairment were more likely to prefer remote research study visits (mild cognitive impairment [MCI] vs. normal [odds ratio (OR) = 1.40, P = 0.02] and dementia vs. normal [OR = 1.48, P < 0.01]). Respondents with cognitive impairment were also less likely to have interest in smartphone use during at-home study visits (MCI vs. normal [OR = 0.71, P = 0.02] and dementia vs. normal [OR = 0.63, P < 0.001]). Results were similar regarding tablet use (MCI vs. normal [OR = 0.73, P = 0.04] and dementia vs. normal [OR = 0.72, P = 0.01]). Geographical location was analyzed in terms of the percentage of respondents in each region who preferred remote research study visits: West, 51%; Midwest, 34%; South, 41%; and Northeast, 57% (P < 0.0001).

Discussion

Results from the study suggest that there is a growing interest in the remote research study visit format. Further studies will allow for greater understanding and development of this research format.

The Italian telephone-based Verbal Fluency Battery (t-VFB): standardization and preliminary clinical usability evidence

Background

This study aimed at standardizing and providing preliminary evidence on the clinical usability of the Italian telephone-based Verbal Fluency Battery (t-VFB), which includes phonemic (t-PVF), semantic (t-SVF) and alternate (t-AVF) verbal fluency tasks.

Methods

Three-hundred and thirty-five Italian healthy participants (HPs; 140 males; age range = 18-96 years; education range = 4-23 years) and 27 individuals with neurodegenerative or cerebrovascular diseases were administered the t-VFB. Switch number and cluster size were computed via latent semantic analyses. HPs underwent the telephone-based Mental State Examination (MMSE) and Backward Digit Span (BDS). Construct validity, factorial structure, internal consistency, test-retest and inter-rater reliability and equivalence with the in-person Verbal Fluency tasks were assessed. Norms were derived via Equivalent Scores. Diagnostic accuracy against clinical populations was assessed.

Results

The majority of t-VFB scores correlated among each other and with the BDS, but not with the MMSE. Switch number correlated with t-PVF, t-SVF, t-AVF scores, whilst cluster size with the t-SVF and t-AVF scores only. The t-VFB was underpinned by a mono-component structure and was internally consistent (Cronbach's α = 0.91). Test-retest (ICC = 0.69-0.95) and inter-rater reliability (ICC = 0.98-1) were optimal. Each t-VFB test was statistically equivalent to its in-person version (equivalence bounds yielding a p < 0.05). Education predicted all t-VFB scores, whereas age t-SVF and t-AVF scores and sex only some t-SVF scores. Diagnostic accuracy against clinical samples was optimal (AUC = 0.81-0.86).

Discussion

The t-VFB is a valid, reliable and normed telephone-based assessment tool for language and executive functioning, equivalent to the in-person version; results show promising evidence of its diagnostic accuracy in neurological populations.

Speech Recognition in Noise Performance Measured Remotely Versus In-Laboratory From Older and Younger Listeners

PURPOSE

This study examined the performance difference between remote and in-laboratory test modalities with a speech recognition in noise task in older and younger adults.

METHOD

Four groups of participants (younger remote, younger in-laboratory, older remote, and older in-laboratory) were tested on a speech recognition in noise protocol with 72 sentences.

RESULTS

While the younger remote group performed more poorly than the younger in-laboratory group, older participants' performance was comparable between the two modality groups, particularly in the easy to moderately difficult conditions. These results persisted after controlling for demographic variables (e.g., age, gender, and education).

CONCLUSION

While these findings generally support the feasibility of remote data collection with older participants for research on speech perception, they also suggest that technological proficiency is an important factor that affects performance on remote testing in the aging population.

Italian telephone-based Mini-Mental State Examination (Itel-MMSE): item-level psychometric properties

Background

The Italian telephone-based Mini-Mental State Examination (Itel-MMSE), despite being psychometrically sound, has shown relevant ceiling effects, which may negatively impact the interpretation of its scores. In address to overcome such an issue, this study aimed at providing item-level insights on the Itel-MMSE through Item Response Theory (IRT) analyses.

Methods

Five-hundred and sixty-seven healthy Italian adults (227 males, 340 females; mean age: 51 ± 17 years, range 18–96; mean education: 13.31 ± 4.3 years). A two-parameter logistic IRT model was implemented to assess item discrimination and difficulty of the Itel-MMSE. Construct unidimensionality, statistical independence of items, and model and item fit were tested. Informativity levels were also assessed graphically.

Results

With respect to the Itel-MMSE total score, ceiling effects were found in 92.7% of participants. Unidimensionality was violated; both model and item fit were poor; a few items showed statistical dependence. Both the whole test and its items proved to be scarcely informative, especially for medium-to-high levels of ability, except for attention and spatial orientation subtests, which consistently yielded the highest discriminative capability.

Discussion

The Itel-MMSE appears to be most informative in low-performing healthy individuals. However, the present findings should not lead practitioners to aprioristically equate ceiling effects/low informativity to clinical uselessness. Items assessing attention and, to a lesser extent, spatial orientation appear to be the most informative.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40520-021-02041-4.

Older Adults' Likelihood of Participating in Home-Based Cognitive Assessment

INTRODUCTION

Timely detection of cognitive impairment among older adults has shown to lead to better health and financial outcomes but is hampered by psychological, financial, and physical barriers to participation in cognitive assessment. Home-based cognitive assessment (HBCA) could help overcome some of these barriers. This study aimed to examine older adults' likelihood of participation in HBCA and identify factors predicting this likelihood.

MATERIALS AND METHODS

A cross-sectional online survey distributed through Amazon Mechanical Turk, was used to collect data from adults aged 50 years or older residing in the USA The survey was designed to gauge attitudes toward technology and cognitive assessment and to measure psychological variables including subjective cognitive decline (SCD), depression, and anxiety. Information on income and geographic location (rural vs. suburban and urban) was also collected. Univariate and hierarchical regression analyses were conducted to examine the contributions of these variables to a composite measure of likelihood of participation in HBCA.

RESULTS

Complete data were obtained on n = 483 (age 50-79). Approximately, two-thirds of respondents described themselves as likely or very likely to participate in HBCA. In univariate analyses, younger age, higher income, higher technology assessment acceptance scores, and higher SCD burden were associated with higher likelihood of participation. Hierarchical regression revealed significant stepwise increments in explained variance: demographic variables 4.1%, technology acceptance 25.2%, assessment acceptance 15.4%, and SCD burden 1.6%. The contribution of SCD was moderated by sex and found for women but not for men.

DISCUSSION/CONCLUSION

A large proportion of adults aged 50 and above described themselves likely to participate in HBCA. Middle-aged, technology-savvy, higher income adults expressed the most positive attitudes toward this type of service. Of interest is that HBCA may be particularly acceptable among older women with SCD, a group known to be at risk of cognitive decline. Our findings support the expansion of cognitive assessment services to the home setting.

The Use of a Computerized Cognitive Assessment to Improve the Efficiency of Primary Care Referrals to Memory Services: Protocol for the Accelerating Dementia Pathway Technologies (ADePT) Study

Background

Existing primary care cognitive assessment tools are crude or time-consuming screening instruments which can only detect cognitive impairment when it is well established. Due to the COVID-19 pandemic, memory services have adapted to the new environment by moving to remote patient assessments to continue meeting service user demand. However, the remote use of cognitive assessments has been variable while there has been scant evaluation of the outcome of such a change in clinical practice. Emerging research in remote memory clinics has highlighted computerized cognitive tests, such as the Integrated Cognitive Assessment (ICA), as prominent candidates for adoption in clinical practice both during the pandemic and for post-COVID-19 implementation as part of health care innovation.

Objective

The aim of the Accelerating Dementia Pathway Technologies (ADePT) study is to develop a real-world evidence basis to support the adoption of ICA as an inexpensive screening tool for the detection of cognitive impairment to improve the efficiency of the dementia care pathway.

Methods

Patients who have been referred to a memory clinic by a general practitioner (GP) are recruited. Participants complete the ICA either at home or in the clinic along with medical history and usability questionnaires. The GP referral and ICA outcome are compared with the specialist diagnosis obtained at the memory clinic. The clinical outcomes as well as National Health Service reference costing data will be used to assess the potential health and economic benefits of the use of the ICA in the dementia diagnosis pathway.

Results

The ADePT study was funded in January 2020 by Innovate UK (Project Number 105837). As of September 2021, 86 participants have been recruited in the study, with 23 participants also completing a retest visit. Initially, the study was designed for in-person visits at the memory clinic; however, in light of the COVID-19 pandemic, the study was amended to allow remote as well as face-to-face visits. The study was also expanded from a single site to 4 sites in the United Kingdom. We expect results to be published by the second quarter of 2022.

Conclusions

The ADePT study aims to improve the efficiency of the dementia care pathway at its very beginning and supports systems integration at the intersection between primary and secondary care. The introduction of a standardized, self-administered, digital assessment tool for the timely detection of neurodegeneration as part of a decision support system that can signpost accordingly can reduce unnecessary referrals, service backlog, and assessment variability.

Trial Registration

ISRCTN 16596456; https://www.isrctn.com/ISRCTN16596456

International Registered Report Identifier (IRRID)

DERR1-10.2196/34475

Tele-neuropsychological assessment tools in Italy: a systematic review on psychometric properties and usability

Background

The current COVID-19 pandemic has abruptly catalysed a shift towards remote assessment in neuropsychological practice (tele-neuropsychology, t-NPs). Although the validity of t-NPs diagnostics is gaining recognition worldwide, little is known about its implementation in Italy. The present review by the Italian working group on tele-neuropsychology (TELA) aims at describing the availability, psychometric properties, and feasibility of t-NPs tools currently available in Italy.

Methods

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. This work was pre-registered on the Prospective Register of Systematic Reviews (PROSPERO; CRD42021239687). Observational studies reporting telephone-, videoconference- or web-based assessment of cognition/behaviour in Italian both healthy participants (HPs) and patients were included. Bias assessment was performed through ad hoc scales.

Results

Fourteen studies were included from an initial N = 895 (4 databases searched). Studies were subdivided into those focused on psychometric properties and those characterized by a predominant applied nature. The majority of studies addressed either adult/elderly HPs or neurological/internal patients. Multi-domain screening tools for cognition, behaviour, mood/anxiety and quality of life were the most represented. Findings regarding validity, reliability, sensitivity, specificity and clinical usability were reported for cognitive screenings — the telephone- and videoconference-based Mini-Mental State Examination and the Telephone Interview for Cognitive Status.

Discussion

Positive albeit preliminary evidence regarding psychometric properties and feasibility in both clinical and non-clinical populations of Italian t-NPs brief screening tools are herewith provided. Further studies exploring clinical usability of t-NPs and psychometric properties/feasibility of tests for the in-depth assessment of specific cognitive domains are necessary.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10072-021-05719-9.