Assessing the Utility of Language and Voice Biomarkers to Predict Cognitive Impairment in the Framingham Heart Study Cognitive Aging Cohort Data

Prediction of Alzheimer's disease progression within 6 years using speech: A novel approach leveraging language models

INTRODUCTION

Identification of individuals with mild cognitive impairment (MCI) who are at risk of developing Alzheimer's disease (AD) is crucial for early intervention and selection of clinical trials.

METHODS

We applied natural language processing techniques along with machine learning methods to develop a method for automated prediction of progression to AD within 6 years using speech. The study design was evaluated on the neuropsychological test interviews of n = 166 participants from the Framingham Heart Study, comprising 90 progressive MCI and 76 stable MCI cases.

RESULTS

Our best models, which used features generated from speech data, as well as age, sex, and education level, achieved an accuracy of 78.5% and a sensitivity of 81.1% to predict MCI-to-AD progression within 6 years.

DISCUSSION

The proposed method offers a fully automated procedure, providing an opportunity to develop an inexpensive, broadly accessible, and easy-to-administer screening tool for MCI-to-AD progression prediction, facilitating development of remote assessment.

HIGHLIGHTS

Voice recordings from neuropsychological exams coupled with basic demographics can lead to strong predictive models of progression to dementia from mild cognitive impairment. The study leveraged AI methods for speech recognition and processed the resulting text using language models. The developed AI-powered pipeline can lead to fully automated assessment that could enable remote and cost-effective screening and prognosis for Alzehimer's disease.

Exploring the usability of the virtual reality module LEAF CAFÉ: a qualitative think-aloud study

BACKGROUND

The global healthcare system faces increasing strain from our ageing population, primarily due to the growing prevalence of age-related health conditions such as dementia. While modern healthcare technology offers potential solutions, it frequently lacks user-friendliness for older adults. Virtual Reality (VR) has emerged as a promising tool for diagnosing cognitive impairment, offering innovative solutions where traditional methods may fall short. This study explores older adults' perspectives on the usability of a newly designed VR module for cognitive assessment.

METHODS

During a 100-min session, participants were asked to engage and complete recall and recognition tasks within the VR module (think-aloud approach) and provide feedback upon completion (semi-structured interviews). Audio materials were transcribed for analysis and recordings of the users' interactions with the module were annotated to provide additional context. These combined textual data were analysed using content coding and thematic analysis to identify themes that reflect how participants used the module's features and what features are desirable to support that process better.

RESULTS

Participants (N = 10; Mean age = 73.3, SD = 7.53, range = 65-83 years) perceived the VR module as user-friendly and endorsed its potential as a cognitive screener due to its engaging and immersive nature. Older adults highlighted three key aspects of the module: the usefulness of the platform's ability to offer a comprehensive and reliable evaluation of an individual's cognitive abilities; the need to present concise and relevant content to optimise engagement and use; and the importance of overcoming barriers to support implementation. Suggested game improvements centred on food recognition and adjusting difficulty levels. Barriers to implementation included technology challenges for older adults and concerns about the game's suitability for everyday scenarios. Participants stressed the need for reliable implementation strategies, proposing locations such as libraries and advocating for home-based screening.

CONCLUSION

Continued improvements in accessibility suggest that VR tools could help with diagnosing cognitive impairment in older adults. Using a simulated environment to assess cognitive status might fill the gap between current diagnostic methods, aiding treatment planning and early intervention. However, these findings should be approached cautiously, as more research is needed to fully grasp the potential impact of VR tools in this context.

Automated Classification of Cognitive Decline and Probable Alzheimer's Dementia Across Multiple Speech and Language Domains

BACKGROUND

Decline in language has emerged as a new potential biomarker for the early detection of Alzheimer's disease (AD). It remains unclear how sensitive language measures are across different tasks, language domains, and languages, and to what extent changes can be reliably detected in early stages such as subjective cognitive decline (SCD) and mild cognitive impairment (MCI).

METHOD

Using a scene construction task for speech elicitation in a new Spanish/Catalan speaking cohort (N = 119), we automatically extracted features across seven domains, three acoustic (spectral, cepstral, and voice quality), one prosodic, and three from text (morpholexical, semantic, and syntactic). They were forwarded to a random forest classifier to evaluate the discriminability of participants with probable AD dementia, amnestic and nonamnestic MCI, SCD, and cognitively healthy controls. Repeated-measures analyses of variance and paired-samples Wilcoxon signed-ranks test were used to assess whether and how performance differs significantly across groups and linguistic domains.

RESULTS

The performance scores of the machine learning classifier were generally satisfactorily high, with the highest scores over .9. Model performance was significantly different for linguistic domains (p < .001), and speech versus text (p = .043), with speech features outperforming textual features, and voice quality performing best. High diagnostic classification accuracies were seen even within both cognitively healthy (controls vs. SCD) and MCI (amnestic and nonamnestic) groups.

CONCLUSION

Speech-based machine learning is powerful in detecting cognitive decline and probable AD dementia across a range of different feature domains, though important differences exist between these domains as well.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.23699733.

Harnessing acoustic speech parameters to decipher amyloid status in individuals with mild cognitive impairment

Alzheimer's disease (AD) is a neurodegenerative condition characterized by a gradual decline in cognitive functions. Currently, there are no effective treatments for AD, underscoring the importance of identifying individuals in the preclinical stages of mild cognitive impairment (MCI) to enable early interventions. Among the neuropathological events associated with the onset of the disease is the accumulation of amyloid protein in the brain, which correlates with decreased levels of Aβ42 peptide in the cerebrospinal fluid (CSF). Consequently, the development of non-invasive, low-cost, and easy-to-administer proxies for detecting Aβ42 positivity in CSF becomes particularly valuable. A promising approach to achieve this is spontaneous speech analysis, which combined with machine learning (ML) techniques, has proven highly useful in AD. In this study, we examined the relationship between amyloid status in CSF and acoustic features derived from the description of the Cookie Theft picture in MCI patients from a memory clinic. The cohort consisted of fifty-two patients with MCI (mean age 73 years, 65% female, and 57% positive amyloid status). Eighty-eight acoustic parameters were extracted from voice recordings using the extended Geneva Minimalistic Acoustic Parameter Set (eGeMAPS), and several ML models were used to classify the amyloid status. Furthermore, interpretability techniques were employed to examine the influence of input variables on the determination of amyloid-positive status. The best model, based on acoustic variables, achieved an accuracy of 75% with an area under the curve (AUC) of 0.79 in the prediction of amyloid status evaluated by bootstrapping and Leave-One-Out Cross Validation (LOOCV), outperforming conventional neuropsychological tests (AUC = 0.66). Our results showed that the automated analysis of voice recordings derived from spontaneous speech tests offers valuable insights into AD biomarkers during the preclinical stages. These findings introduce novel possibilities for the use of digital biomarkers to identify subjects at high risk of developing AD.

Discrimination of mild cognitive impairment based on involuntary changes caused in voice elements

In this study, the technique associated with the capturing involuntary changes in voice elements caused by diseases is applied to diagnose them and a voice index is proposed to discriminate mild cognitive impairments. The participants in this study included 399 elderly people aged 65 years or older living in Matsumoto City, Nagano Prefecture, Japan. The participants were categorized into healthy and mild cognitive impairment groups based on clinical evaluation. It was hypothesized that as dementia progressed, task performance would become more challenging, and the effects on vocal cords and prosody would change significantly. In the study, voice samples of the participants were recorded while they were engaged in mental calculational tasks and during the reading of the results of the calculations written on paper. The change in prosody during the calculation from that during reading was expressed based on the difference in the acoustics. Principal component analysis was used to aggregate groups of voice features with similar characteristics of feature differences into several principal components. These principal components were combined with logistic regression analysis to propose a voice index to discriminate different mild cognitive impairment types. Discrimination accuracies of 90% and 65% were obtained for discriminations using the proposed index on the training and verification data (obtained from a population different from the training data), respectively. Therefore, it is suggested that the proposed index may be utilized as a means for discriminating mild cognitive impairments.

Reading and lexical-semantic retrieval tasks outperforms single task speech analysis in the screening of mild cognitive impairment and Alzheimer's disease

Age-related cognitive impairment have increased dramatically in recent years, which has risen the interes in developing screening tools for mild cognitive impairment and Alzheimer's disease. Speech analysis allows to exploit the behavioral consequences of cognitive deficits on the patient's vocal performance so that it is possible to identify pathologies affecting speech production such as dementia. Previous studies have further shown that the speech task used determines how the speech parameters are altered. We aim to combine the impairments in several speech production tasks in order to improve the accuracy of screening through speech analysis. The sample consists of 72 participants divided into three equal groups of healthy older adults, people with mild cognitive impairment, or Alzheimer's disease, matched by age and education. A complete neuropsychological assessment and two voice recordings were performed. The tasks required the participants to read a text, and complete a sentence with semantic information. A stepwise linear discriminant analysis was performed to select speech parameters with discriminative power. The discriminative functions obtained an accuracy of 83.3% in simultaneous classifications of several levels of cognitive impairment. It would therefore be a promising screening tool for dementia.

Voice biomarkers as indicators of cognitive changes in middle and later adulthood

Voice prosody measures have been linked with Alzheimer's disease (AD), but it is unclear whether they are associated with normal cognitive aging. We assessed relationships between voice measures and 10-year cognitive changes in the MIDUS national sample of middle-aged and older adults ages 42-92, with a mean age of 64.09 (standard deviation = 11.23) at the second wave. Seven cognitive tests were assessed in 2003-2004 (Wave 2) and 2013-2014 (Wave 3). Voice measures were collected at Wave 3 (N = 2585) from audio recordings of the cognitive interviews. Analyses controlled for age, education, depressive symptoms, and health. As predicted, higher jitter was associated with greater declines in episodic memory, verbal fluency, and attention switching. Lower pulse was related to greater decline in episodic memory, and fewer voice breaks were related to greater declines in episodic memory and verbal fluency, although the direction of these effects was contrary to hypotheses. Findings suggest that voice biomarkers may offer a promising approach for early detection of risk factors for cognitive impairment or AD.

Automated detection of mild cognitive impairment and dementia from voice recordings: A natural language processing approach

INTRODUCTION

Automated computational assessment of neuropsychological tests would enable widespread, cost-effective screening for dementia.

METHODS

A novel natural language processing approach is developed and validated to identify different stages of dementia based on automated transcription of digital voice recordings of subjects' neuropsychological tests conducted by the Framingham Heart Study (n = 1084). Transcribed sentences from the test were encoded into quantitative data and several models were trained and tested using these data and the participants' demographic characteristics.

RESULTS

Average area under the curve (AUC) on the held-out test data reached 92.6%, 88.0%, and 74.4% for differentiating Normal cognition from Dementia, Normal or Mild Cognitive Impairment (MCI) from Dementia, and Normal from MCI, respectively.

DISCUSSION

The proposed approach offers a fully automated identification of MCI and dementia based on a recorded neuropsychological test, providing an opportunity to develop a remote screening tool that could be adapted easily to any language.

Health Professionals’ Experience Using an Azure Voice-Bot to Examine Cognitive Impairment (WAY2AGE)

Virtual Assistants (VA) are a new groundbreaking tool for screening cognitive impairment by healthcare professionals. By providing the volume of data needed in healthcare guidance, better treatment monitoring and optimization of costs are expected. One of the first steps in the development of these items is the experience of the healthcare professionals in their use. The general goal of the current project, WAY2AGE, is to examine healthcare professionals’ experience in using an Azure voice-bot for screening cognitive impairment. In this way, back-end services, such as the ChatBot, Speech Service and databases, are provided by the cloud platform Azure (Paas) for a pilot study. Most of the underlying scripts are implemented in Python, Net, JavaScript and open software. A sample of 30 healthcare workers volunteered to participate by answering a list of question in a survey set-up, following the example provided in the previous literature. Based on the current results, WAY2AGE was evaluated very positively in several categories. The main challenge of WAY2AGE is the articulation problems of some older people, which can lead to errors in the transcription of audio to text that will be addressed in the second phase. Following an analysis of the perception of a group of thirty health professionals on its usability, potential limitations and opportunities for future research are discussed.

A Neuropsychological Perspective on Defining Cognitive Impairment in the Clinical Study of Alzheimer’s Disease: Towards a More Continuous Approach

The global fight against Alzheimer’s disease (AD) poses unique challenges for the field of neuropsychology. Along with the increased focus on early detection of AD pathophysiology, characterizing the earliest clinical stage of the disease has become a priority. We believe this is an important time for neuropsychology to consider how our approach to the characterization of cognitive impairment can be improved to detect subtle cognitive changes during early-stage AD. The present article aims to provide a critical examination of how we define and measure cognitive status in the context of aging and AD. First, we discuss pitfalls of current methods for defining cognitive impairment within the context of research shifting to earlier (pre)symptomatic disease stages. Next, we introduce a shift towards a more continuous approach for identifying early markers of cognitive decline and characterizing progression and discuss how this may be facilitated by novel assessment approaches. Finally, we summarize potential implications and challenges of characterizing cognitive status using a continuous approach.

Using Digital Tools to Advance Alzheimer's Drug Trials During a Pandemic: The EU/US CTAD Task Force

The 2020 COVID-19 pandemic has disrupted Alzheimer’s disease (AD) clinical studies worldwide. Digital technologies may help minimize disruptions by enabling remote assessment of subtle cognitive and functional changes over the course of the disease. The EU/US Clinical Trials in Alzheimer’s Disease (CTAD) Task Force met virtually in November 2020 to explore the opportunities and challenges associated with the use of digital technologies in AD clinical research. While recognizing the potential of digital tools to accelerate clinical trials, improve the engagement of diverse populations, capture clinically meaningful data, and lower costs, questions remain regarding the stability, validity, generalizability, and reproducibility of digital data. Substantial concerns also exist regarding regulatory acceptance and privacy. Nonetheless, the Task Force supported further exploration of digital technologies through collaboration and data sharing, noting the need for standardization of digital readouts. They also concluded that while it may be premature to employ remote assessments for trials of novel experimental medications, remote studies of non-invasive, multi-domain approaches may be feasible at this time.

Detection of dementia on voice recordings using deep learning: a Framingham Heart Study

BACKGROUND

Identification of reliable, affordable, and easy-to-use strategies for detection of dementia is sorely needed. Digital technologies, such as individual voice recordings, offer an attractive modality to assess cognition but methods that could automatically analyze such data are not readily available.

METHODS AND FINDINGS

We used 1264 voice recordings of neuropsychological examinations administered to participants from the Framingham Heart Study (FHS), a community-based longitudinal observational study. The recordings were 73 min in duration, on average, and contained at least two speakers (participant and examiner). Of the total voice recordings, 483 were of participants with normal cognition (NC), 451 recordings were of participants with mild cognitive impairment (MCI), and 330 were of participants with dementia (DE). We developed two deep learning models (a two-level long short-term memory (LSTM) network and a convolutional neural network (CNN)), which used the audio recordings to classify if the recording included a participant with only NC or only DE and to differentiate between recordings corresponding to those that had DE from those who did not have DE (i.e., NDE (NC+MCI)). Based on 5-fold cross-validation, the LSTM model achieved a mean (±std) area under the receiver operating characteristic curve (AUC) of 0.740 ± 0.017, mean balanced accuracy of 0.647 ± 0.027, and mean weighted F1 score of 0.596 ± 0.047 in classifying cases with DE from those with NC. The CNN model achieved a mean AUC of 0.805 ± 0.027, mean balanced accuracy of 0.743 ± 0.015, and mean weighted F1 score of 0.742 ± 0.033 in classifying cases with DE from those with NC. For the task related to the classification of participants with DE from NDE, the LSTM model achieved a mean AUC of 0.734 ± 0.014, mean balanced accuracy of 0.675 ± 0.013, and mean weighted F1 score of 0.671 ± 0.015. The CNN model achieved a mean AUC of 0.746 ± 0.021, mean balanced accuracy of 0.652 ± 0.020, and mean weighted F1 score of 0.635 ± 0.031 in classifying cases with DE from those who were NDE.

CONCLUSION

This proof-of-concept study demonstrates that automated deep learning-driven processing of audio recordings of neuropsychological testing performed on individuals recruited within a community cohort setting can facilitate dementia screening.

Detection of dementia on voice recordings using deep learning: a Framingham Heart Study

BACKGROUND

Identification of reliable, affordable, and easy-to-use strategies for detection of dementia is sorely needed. Digital technologies, such as individual voice recordings, offer an attractive modality to assess cognition but methods that could automatically analyze such data are not readily available.

METHODS AND FINDINGS

We used 1264 voice recordings of neuropsychological examinations administered to participants from the Framingham Heart Study (FHS), a community-based longitudinal observational study. The recordings were 73 min in duration, on average, and contained at least two speakers (participant and examiner). Of the total voice recordings, 483 were of participants with normal cognition (NC), 451 recordings were of participants with mild cognitive impairment (MCI), and 330 were of participants with dementia (DE). We developed two deep learning models (a two-level long short-term memory (LSTM) network and a convolutional neural network (CNN)), which used the audio recordings to classify if the recording included a participant with only NC or only DE and to differentiate between recordings corresponding to those that had DE from those who did not have DE (i.e., NDE (NC+MCI)). Based on 5-fold cross-validation, the LSTM model achieved a mean (±std) area under the receiver operating characteristic curve (AUC) of 0.740 ± 0.017, mean balanced accuracy of 0.647 ± 0.027, and mean weighted F1 score of 0.596 ± 0.047 in classifying cases with DE from those with NC. The CNN model achieved a mean AUC of 0.805 ± 0.027, mean balanced accuracy of 0.743 ± 0.015, and mean weighted F1 score of 0.742 ± 0.033 in classifying cases with DE from those with NC. For the task related to the classification of participants with DE from NDE, the LSTM model achieved a mean AUC of 0.734 ± 0.014, mean balanced accuracy of 0.675 ± 0.013, and mean weighted F1 score of 0.671 ± 0.015. The CNN model achieved a mean AUC of 0.746 ± 0.021, mean balanced accuracy of 0.652 ± 0.020, and mean weighted F1 score of 0.635 ± 0.031 in classifying cases with DE from those who were NDE.

CONCLUSION

This proof-of-concept study demonstrates that automated deep learning-driven processing of audio recordings of neuropsychological testing performed on individuals recruited within a community cohort setting can facilitate dementia screening.

Dementia risks identified by vocal features via telephone conversations: A novel machine learning prediction model

Due to difficulty in early diagnosis of Alzheimer's disease (AD) related to cost and differentiated capability, it is necessary to identify low-cost, accessible, and reliable tools for identifying AD risk in the preclinical stage. We hypothesized that cognitive ability, as expressed in the vocal features in daily conversation, is associated with AD progression. Thus, we have developed a novel machine learning prediction model to identify AD risk by using the rich voice data collected from daily conversations, and evaluated its predictive performance in comparison with a classification method based on the Japanese version of the Telephone Interview for Cognitive Status (TICS-J). We used 1,465 audio data files from 99 Healthy controls (HC) and 151 audio data files recorded from 24 AD patients derived from a dementia prevention program conducted by Hachioji City, Tokyo, between March and May 2020. After extracting vocal features from each audio file, we developed machine-learning models based on extreme gradient boosting (XGBoost), random forest (RF), and logistic regression (LR), using each audio file as one observation. We evaluated the predictive performance of the developed models by describing the receiver operating characteristic (ROC) curve, calculating the areas under the curve (AUCs), sensitivity, and specificity. Further, we conducted classifications by considering each participant as one observation, computing the average of their audio files' predictive value, and making comparisons with the predictive performance of the TICS-J based questionnaire. Of 1,616 audio files in total, 1,308 (81.0%) were randomly allocated to the training data and 308 (19.1%) to the validation data. For audio file-based prediction, the AUCs for XGboost, RF, and LR were 0.863 (95% confidence interval [CI]: 0.794-0.931), 0.882 (95% CI: 0.840-0.924), and 0.893 (95%CI: 0.832-0.954), respectively. For participant-based prediction, the AUC for XGboost, RF, LR, and TICS-J were 1.000 (95%CI: 1.000-1.000), 1.000 (95%CI: 1.000-1.000), 0.972 (95%CI: 0.918-1.000) and 0.917 (95%CI: 0.918-1.000), respectively. There was difference in predictive accuracy of XGBoost and TICS-J with almost approached significance (p = 0.065). Our novel prediction model using the vocal features of daily conversations demonstrated the potential to be useful for the AD risk assessment.

Neuropsychological test validation of speech markers of cognitive impairment in the Framingham Cognitive Aging Cohort

AIM

Although clinicians primarily diagnose dementia based on a combination of metrics such as medical history and formal neuropsychological tests, recent work using linguistic analysis of narrative speech to identify dementia has shown promising results. We aim to build upon research by Thomas JA & Burkardt HA et al. (J Alzheimers Dis. 2020;76:905-22) and Alhanai et al. (arXiv:1710.07551v1. 2020) on the Framingham Heart Study (FHS) Cognitive Aging Cohort by 1) demonstrating the predictive capability of linguistic analysis in differentiating cognitively normal from cognitively impaired participants and 2) comparing the performance of the original linguistic features with the performance of expanded features.

METHODS

Data were derived from a subset of the FHS Cognitive Aging Cohort. We analyzed a sub-selection of 98 participants, which provided 127 unique audio files and clinical observations (n = 127, female = 47%, cognitively impaired = 43%). We built on previous work which extracted original linguistic features from transcribed audio files by extracting expanded features. We used both feature sets to train logistic regression classifiers to distinguish cognitively normal from cognitively impaired participants and compared the predictive power of the original and expanded linguistic feature sets, and participants' Mini-Mental State Examination (MMSE) scores.

RESULTS

Based on the area under the receiver-operator characteristic curve (AUC) of the models, both the original (AUC = 0.882) and expanded (AUC = 0.883) feature sets outperformed MMSE (AUC = 0.870) in classifying cognitively impaired and cognitively normal participants. Although the original and expanded feature sets had similar AUC, the expanded feature set showed better positive and negative predictive value [expanded: positive predictive value (PPV) = 0.738, negative predictive value (NPV) = 0.889; original: PPV = 0.701, NPV = 0.869].

CONCLUSIONS

Linguistic analysis has been shown to be a potentially powerful tool for clinical use in classifying cognitive impairment. This study expands the work of several others, but further studies into the plausibility of speech analysis in clinical use are vital to ensure the validity of speech analysis for clinical classification of cognitive impairment.