Identifying Mild Cognitive Impairment and mild Alzheimer’s disease based on spontaneous speech using ASR and linguistic features

Automatic speech analysis for detecting cognitive decline of older adults

Background

Speech analysis has been expected to help as a screening tool for early detection of Alzheimer's disease (AD) and mild-cognitively impairment (MCI). Acoustic features and linguistic features are usually used in speech analysis. However, no studies have yet determined which type of features provides better screening effectiveness, especially in the large aging population of China.

Objective

Firstly, to compare the screening effectiveness of acoustic features, linguistic features, and their combination using the same dataset. Secondly, to develop Chinese automated diagnosis model using self-collected natural discourse data obtained from native Chinese speakers.

Methods

A total of 92 participants from communities in Shanghai, completed MoCA-B and a picture description task based on the Cookie Theft under the guidance of trained operators, and were divided into three groups including AD, MCI, and heathy control (HC) based on their MoCA-B score. Acoustic features (Pitches, Jitter, Shimmer, MFCCs, Formants) and linguistic features (part-of-speech, type-token ratio, information words, information units) are extracted. The machine algorithms used in this study included logistic regression, random forest (RF), support vector machines (SVM), Gaussian Naive Bayesian (GNB), and k-Nearest neighbor (kNN). The validation accuracies of the same ML model using acoustic features, linguistic features, and their combination were compared.

Results

The accuracy with linguistic features is generally higher than acoustic features in training. The highest accuracy to differentiate HC and AD is 80.77% achieved by SVM, based on all the features extracted from the speech data, while the highest accuracy to differentiate HC and AD or MCI is 80.43% achieved by RF, based only on linguistic features.

Conclusion

Our results suggest the utility and validity of linguistic features in the automated diagnosis of cognitive impairment, and validated the applicability of automated diagnosis for Chinese language data.

Connected Speech Fluency in Poststroke and Progressive Aphasia: A Scoping Review of Quantitative Approaches and Features

PURPOSE

Speech fluency has important diagnostic implications for individuals with poststroke aphasia (PSA) as well as primary progressive aphasia (PPA), and quantitative assessment of connected speech has emerged as a widely used approach across both etiologies. The purpose of this review was to provide a clearer picture on the range, nature, and utility of individual quantitative speech/language measures and methods used to assess connected speech fluency in PSA and PPA, and to compare approaches across etiologies.

METHOD

We conducted a scoping review of literature published between 2012 and 2022 following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews guidelines. Forty-five studies were included in the review. Literature was charted and summarized by etiology and characteristics of included patient populations and method(s) used for derivation and analysis of speech/language features. For a subset of included articles, we also charted the individual quantitative speech/language features reported and the level of significance of reported results.

RESULTS

Results showed that similar methodological approaches have been used to quantify connected speech fluency in both PSA and PPA. Two hundred nine individual speech-language features were analyzed in total, with low levels of convergence across etiology on specific features but greater agreement on the most salient features. The most useful features for differentiating fluent from nonfluent aphasia in both PSA and PPA were features related to overall speech quantity, speech rate, or grammatical competence.

CONCLUSIONS

Data from this review demonstrate the feasibility and utility of quantitative approaches to index connected speech fluency in PSA and PPA. We identified emergent trends toward automated analysis methods and data-driven approaches, which offer promising avenues for clinical translation of quantitative approaches. There is a further need for improved consensus on which subset of individual features might be most clinically useful for assessment and monitoring of fluency.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.25537237.

Detection of Mild Cognitive Impairment Through Hand Motor Function Under Digital Cognitive Test: Mixed Methods Study

BACKGROUND

Early detection of cognitive impairment or dementia is essential to reduce the incidence of severe neurodegenerative diseases. However, currently available diagnostic tools for detecting mild cognitive impairment (MCI) or dementia are time-consuming, expensive, or not widely accessible. Hence, exploring more effective methods to assist clinicians in detecting MCI is necessary.

OBJECTIVE

In this study, we aimed to explore the feasibility and efficiency of assessing MCI through movement kinetics under tablet-based "drawing and dragging" tasks.

METHODS

We iteratively designed "drawing and dragging" tasks by conducting symposiums, programming, and interviews with stakeholders (neurologists, nurses, engineers, patients with MCI, healthy older adults, and caregivers). Subsequently, stroke patterns and movement kinetics were evaluated in healthy control and MCI groups by comparing 5 categories of features related to hand motor function (ie, time, stroke, frequency, score, and sequence). Finally, user experience with the overall cognitive screening system was investigated using structured questionnaires and unstructured interviews, and their suggestions were recorded.

RESULTS

The "drawing and dragging" tasks can detect MCI effectively, with an average accuracy of 85% (SD 2%). Using statistical comparison of movement kinetics, we discovered that the time- and score-based features are the most effective among all the features. Specifically, compared with the healthy control group, the MCI group showed a significant increase in the time they took for the hand to switch from one stroke to the next, with longer drawing times, slow dragging, and lower scores. In addition, patients with MCI had poorer decision-making strategies and visual perception of drawing sequence features, as evidenced by adding auxiliary information and losing more local details in the drawing. Feedback from user experience indicates that our system is user-friendly and facilitates screening for deficits in self-perception.

CONCLUSIONS

The tablet-based MCI detection system quantitatively assesses hand motor function in older adults and further elucidates the cognitive and behavioral decline phenomenon in patients with MCI. This innovative approach serves to identify and measure digital biomarkers associated with MCI or Alzheimer dementia, enabling the monitoring of changes in patients' executive function and visual perceptual abilities as the disease advances.

Deep Learning for Alzheimer's Disease Prediction: A Comprehensive Review

Alzheimer's disease (AD) is a neurological disorder that significantly impairs cognitive function, leading to memory loss and eventually death. AD progresses through three stages: early stage, mild cognitive impairment (MCI) (middle stage), and dementia. Early diagnosis of Alzheimer's disease is crucial and can improve survival rates among patients. Traditional methods for diagnosing AD through regular checkups and manual examinations are challenging. Advances in computer-aided diagnosis systems (CADs) have led to the development of various artificial intelligence and deep learning-based methods for rapid AD detection. This survey aims to explore the different modalities, feature extraction methods, datasets, machine learning techniques, and validation methods used in AD detection. We reviewed 116 relevant papers from repositories including Elsevier (45), IEEE (25), Springer (19), Wiley (6), PLOS One (5), MDPI (3), World Scientific (3), Frontiers (3), PeerJ (2), Hindawi (2), IO Press (1), and other multiple sources (2). The review is presented in tables for ease of reference, allowing readers to quickly grasp the key findings of each study. Additionally, this review addresses the challenges in the current literature and emphasizes the importance of interpretability and explainability in understanding deep learning model predictions. The primary goal is to assess existing techniques for AD identification and highlight obstacles to guide future research.

Distinguishable features of spontaneous speech in Alzheimer's clinical syndrome and healthy controls

There is growing evidence that subtle changes in spontaneous speech may reflect early pathological changes in cognitive function. Recent work has found that lexical-semantic features of spontaneous speech predict cognitive dysfunction in individuals with mild cognitive impairment (MCI). The current study assessed whether Ostrand and Gunstad's (OG) lexical-semantic features extend to predicting cognitive status in a sample of individuals with Alzheimer's clinical syndrome (ACS) and healthy controls. Four additional (New) speech indices shown to be important in language processing research were also explored in this sample to extend prior work. Speech transcripts of the Cookie Theft Task from 81 individuals with ACS (Mage = 72.7 years, SD = 8.80, 70.4% female) and 61 healthy controls (HC) (Mage = 63.9 years, SD = 8.52, 62.3% female) from Dementia Bank were analyzed. Random forest and logistic machine learning techniques examined whether subject-level lexical-semantic features could be used to accurately discriminate those with ACS from HC. Results showed that logistic models with the New lexical-semantic features obtained good classification accuracy (78.4%), but the OG features had wider success across machine learning model types. In terms of sensitivity and specificity, the random forest model trained on the OG features was the most balanced. Findings from the current study suggest that features of spontaneous speech used to predict MCI may also distinguish between individuals with ACS and healthy controls. Future work should evaluate these lexical-semantic features in pre-clinical persons to further explore their potential to assist with early detection through speech analysis.

A novel speech analysis algorithm to detect cognitive impairment in a Spanish population

Objective

Early detection of cognitive impairment in the elderly is crucial for diagnosis and appropriate care. Brief, cost-effective cognitive screening instruments are needed to help identify individuals who require further evaluation. This study presents preliminary data on a new screening technology using automated voice recording analysis software in a Spanish population.

Method

Data were collected from 174 Spanish-speaking individuals clinically diagnosed as cognitively normal (CN, n = 87) or impaired (mild cognitive impairment [MCI], n = 63; all-cause dementia, n = 24). Participants were recorded performing four common language tasks (Animal fluency, alternating fluency [sports and fruits], phonemic "F" fluency, and Cookie Theft Description). Recordings were processed via text-transcription and digital-signal processing techniques to capture neuropsychological variables and audio characteristics. A training sample of 122 subjects with similar demographics across groups was used to develop an algorithm to detect cognitive impairment. Speech and task features were used to develop five independent machine learning (ML) models to compute scores between 0 and 1, and a final algorithm was constructed using repeated cross-validation. A socio-demographically balanced subset of 52 participants was used to test the algorithm. Analysis of covariance (ANCOVA), covarying for demographic characteristics, was used to predict logistically-transformed algorithm scores.

Results

Mean logit algorithm scores were significantly different across groups in the testing sample (p < 0.01). Comparisons of CN with impaired (MCI + dementia) and MCI groups using the final algorithm resulted in an AUC of 0.93/0.90, with overall accuracy of 88.4%/87.5%, sensitivity of 87.5/83.3, and specificity of 89.2/89.2, respectively.

Conclusion

Findings provide initial support for the utility of this automated speech analysis algorithm as a screening tool for cognitive impairment in Spanish speakers. Additional study is needed to validate this technology in larger and more diverse clinical populations.

Pausing and fluency in speech of patients with relapsing-remitting multiple sclerosis

Multiple Sclerosis (MS) causes a variety of symptoms in speech production, such as more frequent pauses and an increase in the duration of pauses in the speech. However, there is almost no data on whether the disease affects speech fluency in other ways, such as changes in the frequency of disfluencies in speech. The main question of this study is the following: if we examine speech fluency in speech tasks requiring different cognitive load, will there be a difference between patients and controls? Twenty people with relapsing-remitting MS (3 men and 17 women) and 20 age- and education-matched control speakers (4 men and 16 women) participated in the study. Speech samples were recorded with each participant in three speech tasks: 1) spontaneous narratives about their own lives, 2) narratives about their previous day, and 3) narrative recalls based on a heard text. In the speech samples, pauses and disfluencies were annotated and the duration of pauses was measured. Then, the frequency of pauses and disfluencies were calculated and the types of disfluencies were examined. The results show that there are differences in the frequency and duration of pauses between people with MS and controls. However, there were no significant differences in the frequency of disfluencies between the groups. The same types of disfluencies occurred in the same frequency in both groups. The results help to better understand the speech production processes in MS.

Natural language processing techniques for studying language in pathological ageing: A scoping review

BACKGROUND

In the past few years there has been a growing interest in the employment of verbal productions as digital biomarkers, namely objective, quantifiable behavioural data that can be collected and measured by means of digital devices, allowing for a low-cost pathology detection, classification and monitoring. Numerous research papers have been published on the automatic detection of subtle verbal alteration, starting from written texts, raw speech recordings and transcripts, and such linguistic analysis has been singled out as a cost-effective method for diagnosing dementia and other medical conditions common among elderly patients (e.g., cognitive dysfunctions associated with metabolic disorders, dysarthria).

AIMS

To provide a critical appraisal and synthesis of evidence concerning the application of natural language processing (NLP) techniques for clinical purposes in the geriatric population. In particular, we discuss the state of the art on studying language in healthy and pathological ageing, focusing on the latest research efforts to build non-intrusive language-based tools for the early identification of cognitive frailty due to dementia. We also discuss some challenges and open problems raised by this approach.

METHODS & PROCEDURES

We performed a scoping review to examine emerging evidence about this novel domain. Potentially relevant studies published up to November 2021 were identified from the databases of MEDLINE, Cochrane and Web of Science. We also browsed the proceedings of leading international conferences (e.g., ACL, COLING, Interspeech, LREC) from 2017 to 2021, and checked the reference lists of relevant studies and reviews.

MAIN CONTRIBUTION

The paper provides an introductory, but complete, overview of the application of NLP techniques for studying language disruption due to dementia. We also suggest that this technique can be fruitfully applied to other medical conditions (e.g., cognitive dysfunctions associated with dysarthria, cerebrovascular disease and mood disorders).

CONCLUSIONS & IMPLICATIONS

Despite several critical points need to be addressed by the scientific community, a growing body of empirical evidence shows that NLP techniques can represent a promising tool for studying language changes in pathological aging, with a high potential to lead a significant shift in clinical practice.

WHAT THIS PAPER ADDS

What is already known on this subject Speech and languages abilities change due to non-pathological neurocognitive ageing and neurodegenerative processes. These subtle verbal modifications can be measured through NLP techniques and used as biomarkers for screening/diagnostic purposes in the geriatric population (i.e., digital linguistic biomarkers-DLBs). What this paper adds to existing knowledge The review shows that DLBs can represent a promising clinical tool, with a high potential to spark a major shift to dementia assessment in the elderly. Some challenges and open problems are also discussed. What are the potential or actual clinical implications of this work? This methodological review represents a starting point for clinicians approaching the DLB research field for studying language in healthy and pathological ageing. It summarizes the state of the art and future research directions of this novel approach.

Pre-training and ensembling based Alzheimer's disease detection

BACKGROUND

Alzheimer's disease (AD) endangers the physical and mental health of the elderly, constituting one of the most crucial social challenges. Due to lack of effective AD intervention drugs, it is very important to diagnose AD in the early stage, especially in the Mild Cognitive Impairment (MCI) phase.

OBJECTIVE

At present, an automatic classification technology is urgently needed to assist doctors in analyzing the status of the candidate patient. The artificial intelligence enhanced Alzheimer's disease detection can reduce costs to detect Alzheimer's disease.

METHODS

In this paper, a novel pre-trained ensemble-based AD detection (PEADD) framework with three base learners (i.e., ResNet, VGG, and EfficientNet) for both the audio-based and PET (Positron Emission Tomography)-based AD detection is proposed under a unified image modality. Specifically, the effectiveness of context-enriched image modalities instead of the traditional speech modality (i.e., context-free audio matrix) for the audio-based AD detection, along with simple and efficient image denoising strategy has been inspected comprehensively. Meanwhile, the PET-based AD detection based on the denoised PET image has been described. Furthermore, different voting methods for applying an ensemble strategy (i.e., hard voting and soft voting) has been investigated in detail.

RESULTS

The results showed that the classification accuracy was 92% and 99% on the audio-based and PET-based AD datasets, respectively. Our extensive experimental results demonstrate that our PEADD outperforms the state-of-the-art methods on both audio-based and PET-based AD datasets simultaneously.

CONCLUSIONS

The network model can provide an objective basis for doctors to detect Alzheimer's Disease.

Speech changes in old age: Methodological considerations for speech-based discrimination of healthy ageing and Alzheimer's disease

BACKGROUND

Recent evidence suggests that speech substantially changes in ageing. As a complex neurophysiological process, it can accurately reflect changes in the motor and cognitive systems underpinning human speech. Since healthy ageing is not always easily discriminable from early stages of dementia based on cognitive and behavioural hallmarks, speech is explored as a preclinical biomarker of pathological itineraries in old age. A greater and more specific impairment of neuromuscular activation, as well as  a specific cognitive and linguistic impairment in dementia, unchain discriminating changes in speech. Yet, there is no consensus on such discriminatory speech parameters, neither on how they should be elicited and assessed.

AIMS

To provide a state-of-the-art on speech parameters that allow for early discrimination between healthy and pathological ageing; the aetiology of these parameters; the effect of the type of experimental stimuli on speech elicitation and the predictive power of different speech parameters; and the most promising methods for speech analysis and their clinical implications.

METHODS & PROCEDURES

A scoping review methodology is used in accordance with the PRISMA model. Following a systematic search of PubMed, PsycINFO and CINAHL, 24 studies are included and analysed in the review.

MAIN CONTRIBUTION

The results of this review yield three key questions for the clinical assessment of speech in ageing. First, acoustic and temporal parameters are more sensitive to changes in pathological ageing and, of these two, temporal variables are more affected by cognitive impairment. Second, different types of stimuli can trigger speech parameters with different degree of accuracy for the discrimination of clinical groups. Tasks with higher cognitive load are more precise in eliciting higher levels of accuracy. Finally, automatic speech analysis for the discrimination of healthy and pathological ageing should be improved for both research and clinical practice.

CONCLUSIONS & IMPLICATIONS

Speech analysis is a promising non-invasive tool for the preclinical screening of healthy and pathological ageing. The main current challenges of speech analysis in ageing are the automatization of its clinical assessment and the consideration of the speaker's cognitive background during evaluation.

WHAT THIS PAPER ADDS

What is already known on the subject Societal aging  goes hand in hand with the rising incidence of ageing-related neurodegenerations, mainly Alzheimer's disease (AD). This is particularly noteworthy in countries with longer life expectancies. Healthy ageing and early stages of AD share a set of cognitive and behavioural characteristics. Since there is no cure for dementias, developing methods for accurate discrimination of healthy ageing and early AD is currently a priority. Speech has been described as one of the most significantly impaired features in AD. Neuropathological alterations in motor and cognitive systems would underlie specific speech impairment in dementia. Since speech can be evaluated quickly, non-invasively and inexpensively, its value for the clinical assessment of ageing itineraries may be particularly high. What this paper adds to existing knowledge Theoretical and experimental advances in the assessment of speech as a marker of AD have developed rapidly over the last decade. Yet, they are not always known to clinicians. Furthermore, there is a need to provide an updated state-of-the-art on which speech features are discriminatory to AD, how they can be assessed, what kind of results they can yield, and how such results should be interpreted. This article provides an updated overview of speech profiling, methods of speech measurement and analysis, and the clinical power of speech assessment for early discrimination of AD as the most common cause of dementia. What are the potential or actual clinical implications of this work? This article provides an overview of the predictive potential of different speech parameters in relation to AD cognitive impairment. In addition, it discusses the effect that the cognitive state, the type of elicitation task and the type of assessment method may have on the results of the speech-based analysis in ageing.

Neural Computation-Based Methods for the Early Diagnosis and Prognosis of Alzheimer's Disease Not Using Neuroimaging Biomarkers: A Systematic Review

Background

The growing number of older adults in recent decades has led to more prevalent geriatric diseases, such as strokes and dementia. Therefore, Alzheimer's disease (AD), as the most common type of dementia, has become more frequent too.

Background

Objective: The goals of this work are to present state-of-the-art studies focused on the automatic diagnosis and prognosis of AD and its early stages, mainly mild cognitive impairment, and predicting how the research on this topic may change in the future.

Methods

Articles found in the existing literature needed to fulfill several selection criteria. Among others, their classification methods were based on artificial neural networks (ANNs), including deep learning, and data not from brain signals or neuroimaging techniques were used. Considering our selection criteria, 42 articles published in the last decade were finally selected.

Results

The most medically significant results are shown. Similar quantities of articles based on shallow and deep ANNs were found. Recurrent neural networks and transformers were common with speech or in longitudinal studies. Convolutional neural networks (CNNs) were popular with gait or combined with others in modular approaches. Above one third of the cross-sectional studies utilized multimodal data. Non-public datasets were frequently used in cross-sectional studies, whereas the opposite in longitudinal ones. The most popular databases were indicated, which will be helpful for future researchers in this field.

Conclusions

The introduction of CNNs in the last decade and their superb results with neuroimaging data did not negatively affect the usage of other modalities. In fact, new ones emerged.

Machine Learning Approaches for Dementia Detection Through Speech and Gait Analysis: A Systematic Literature Review

Background

Dementia is a general term for several progressive neurodegenerative disorders including Alzheimer's disease. Timely and accurate detection is crucial for early intervention. Advancements in artificial intelligence present significant potential for using machine learning to aid in early detection.

Objective

Summarize the state-of-the-art machine learning-based approaches for dementia prediction, focusing on non-invasive methods, as the burden on the patients is lower. Specifically, the analysis of gait and speech performance can offer insights into cognitive health through clinically cost-effective screening methods.

Methods

A systematic literature review was conducted following the PRISMA protocol (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). The search was performed on three electronic databases (Scopus, Web of Science, and PubMed) to identify the relevant studies published between 2017 to 2022. A total of 40 papers were selected for review.

Results

The most common machine learning methods employed were support vector machine followed by deep learning. Studies suggested the use of multimodal approaches as they can provide comprehensive and better prediction performance. Deep learning application in gait studies is still in the early stages as few studies have applied it. Moreover, including features of whole body movement contribute to better classification accuracy. Regarding speech studies, the combination of different parameters (acoustic, linguistic, cognitive testing) produced better results.

Conclusions

The review highlights the potential of machine learning, particularly non-invasive approaches, in the early prediction of dementia. The comparable prediction accuracies of manual and automatic speech analysis indicate an imminent fully automated approach for dementia detection.

Automated Classification of Cognitive Decline and Probable Alzheimer's Dementia Across Multiple Speech and Language Domains

BACKGROUND

Decline in language has emerged as a new potential biomarker for the early detection of Alzheimer's disease (AD). It remains unclear how sensitive language measures are across different tasks, language domains, and languages, and to what extent changes can be reliably detected in early stages such as subjective cognitive decline (SCD) and mild cognitive impairment (MCI).

METHOD

Using a scene construction task for speech elicitation in a new Spanish/Catalan speaking cohort (N = 119), we automatically extracted features across seven domains, three acoustic (spectral, cepstral, and voice quality), one prosodic, and three from text (morpholexical, semantic, and syntactic). They were forwarded to a random forest classifier to evaluate the discriminability of participants with probable AD dementia, amnestic and nonamnestic MCI, SCD, and cognitively healthy controls. Repeated-measures analyses of variance and paired-samples Wilcoxon signed-ranks test were used to assess whether and how performance differs significantly across groups and linguistic domains.

RESULTS

The performance scores of the machine learning classifier were generally satisfactorily high, with the highest scores over .9. Model performance was significantly different for linguistic domains (p < .001), and speech versus text (p = .043), with speech features outperforming textual features, and voice quality performing best. High diagnostic classification accuracies were seen even within both cognitively healthy (controls vs. SCD) and MCI (amnestic and nonamnestic) groups.

CONCLUSION

Speech-based machine learning is powerful in detecting cognitive decline and probable AD dementia across a range of different feature domains, though important differences exist between these domains as well.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.23699733.

A Pilot Fuzzy System with Virtual Reality for Mild Cognitive Impairment (MCI) Assessment

Mild cognitive impairment (MCI) is when brain function declines. MCI is the gray area transitioning from normal aging to the AD stage. Currently, the majority of early MCI diagnoses are processed through comprehensive neuropsychological tests. These tests may take the form of interviews, paper-and-pencil tests, or computer-based tests. There may be resistance from the subject if he/she has to undergo many screening tests simultaneously for multiple evaluation information, resulting in execution difficulty. The objectives of this study are to use 3D virtual reality to create an entertaining test scenario integrating the Mini-Cog, SPMSQ, MMSE, SLUMS, CDR, and CASI for middle-aged to older adults, furthermore, to employ fuzzy logic control (FLC) technology to develop a "MCI assessment system" for obtaining some pilot information for MCI assessment. There were 24 middle-aged to older adults aged from 50 to 65 years who participated in the evaluation experiment. The results showed that the MCI assessment system developed in this study is highly correlated with the traditional screening tests, including the Mini-Cog, SPMSQ, MMSE, SLUMS, and CASI. The assessment system can provide an integrated reference score for clinic workers in making judgments. In addition, the distribution of the System Usability Scale (SUS) evaluation scores for the MCI assessment system revealed that 87.5% were grade C (good to use) or above and 29.2% were grade B (extremely good to use) or above. The assessment system received positive feedback from the subjects.

Harnessing acoustic speech parameters to decipher amyloid status in individuals with mild cognitive impairment

Alzheimer's disease (AD) is a neurodegenerative condition characterized by a gradual decline in cognitive functions. Currently, there are no effective treatments for AD, underscoring the importance of identifying individuals in the preclinical stages of mild cognitive impairment (MCI) to enable early interventions. Among the neuropathological events associated with the onset of the disease is the accumulation of amyloid protein in the brain, which correlates with decreased levels of Aβ42 peptide in the cerebrospinal fluid (CSF). Consequently, the development of non-invasive, low-cost, and easy-to-administer proxies for detecting Aβ42 positivity in CSF becomes particularly valuable. A promising approach to achieve this is spontaneous speech analysis, which combined with machine learning (ML) techniques, has proven highly useful in AD. In this study, we examined the relationship between amyloid status in CSF and acoustic features derived from the description of the Cookie Theft picture in MCI patients from a memory clinic. The cohort consisted of fifty-two patients with MCI (mean age 73 years, 65% female, and 57% positive amyloid status). Eighty-eight acoustic parameters were extracted from voice recordings using the extended Geneva Minimalistic Acoustic Parameter Set (eGeMAPS), and several ML models were used to classify the amyloid status. Furthermore, interpretability techniques were employed to examine the influence of input variables on the determination of amyloid-positive status. The best model, based on acoustic variables, achieved an accuracy of 75% with an area under the curve (AUC) of 0.79 in the prediction of amyloid status evaluated by bootstrapping and Leave-One-Out Cross Validation (LOOCV), outperforming conventional neuropsychological tests (AUC = 0.66). Our results showed that the automated analysis of voice recordings derived from spontaneous speech tests offers valuable insights into AD biomarkers during the preclinical stages. These findings introduce novel possibilities for the use of digital biomarkers to identify subjects at high risk of developing AD.

Embracing the uncertainty in human-machine collaboration to support clinical decision-making for mental health conditions

Two significant obstacles exist preventing the widespread usage of Deep Learning (DL) models for predicting healthcare outcomes in general and mental health conditions in particular. Firstly, DL models do not quantify the uncertainty in their predictions, so clinicians are unsure of which predictions they can trust. Secondly, DL models do not triage, i.e., separate which cases could be best handled by the human or the model. This paper attempts to address these obstacles using Bayesian Deep Learning (BDL), which extends DL probabilistically and allows us to quantify the model's uncertainty, which we use to improve human-model collaboration. We implement a range of state-of-the-art DL models for Natural Language Processing and apply a range of BDL methods to these models. Taking a step closer to the real-life scenarios of human-AI collaboration, we propose a Referral Learning methodology for the models that make predictions for certain instances while referring the rest of the instances to a human expert for further assessment. The study demonstrates that models can significantly enhance their performance by seeking human assistance in cases where the model exhibits high uncertainty, which is closely linked to misclassifications. Referral Learning offers two options: (1) supporting humans in cases where the model predicts with certainty, and (2) triaging cases where the model evaluated when it had a better chance of being right than the human by evaluating human disagreement. The latter method combines model uncertainty from BDL and human disagreement from multiple annotations, resulting in improved triaging capabilities.

Extracting Phonetic Posterior-Based Features for Detecting Multiple Sclerosis From Speech

Multiple sclerosis (MS) is a chronic inflammatory disease of the central nervous system which, in addition to affecting motor and cognitive functions, may also lead to specific changes in the speech of patients. Speech production, comprehension, repetition and naming tasks, as well as structural and content changes in narratives, might indicate a limitation of executive functions. In this study we present a speech-based machine learning technique to distinguish speakers with relapsing-remitting subtype MS and healthy controls (HC). We exploit the fact that MS might cause a motor speech disorder similar to dysarthria, which, with our hypothesis, might affect the phonetic posterior estimates supplied by a Deep Neural Network acoustic model. From our experimental results, the proposed posterior posteriorgram-based feature extraction approach is useful for detecting MS: depending on the actual speech task, we obtained Equal Error Rate values as low as 13.3%, and AUC scores up to 0.891, indicating a competitive and more consistent classification performance compared to both the x-vector and the openSMILE 'ComParE functionals' attributes. Besides this discrimination performance, the interpretable nature of the phonetic posterior features might also make our method suitable for automatic MS screening or monitoring the progression of the disease. Furthermore, by examining which specific phonetic groups are the most useful for this feature extraction process, the potential utility of the proposed phonetic features could also be utilized in the speech therapy of MS patients.

Applied Machine Learning Techniques to Diagnose Voice-Affecting Conditions and Disorders: Systematic Literature Review

BACKGROUND

Normal voice production depends on the synchronized cooperation of multiple physiological systems, which makes the voice sensitive to changes. Any systematic, neurological, and aerodigestive distortion is prone to affect voice production through reduced cognitive, pulmonary, and muscular functionality. This sensitivity inspired using voice as a biomarker to examine disorders that affect the voice. Technological improvements and emerging machine learning (ML) technologies have enabled possibilities of extracting digital vocal features from the voice for automated diagnosis and monitoring systems.

OBJECTIVE

This study aims to summarize a comprehensive view of research on voice-affecting disorders that uses ML techniques for diagnosis and monitoring through voice samples where systematic conditions, nonlaryngeal aerodigestive disorders, and neurological disorders are specifically of interest.

METHODS

This systematic literature review (SLR) investigated the state of the art of voice-based diagnostic and monitoring systems with ML technologies, targeting voice-affecting disorders without direct relation to the voice box from the point of view of applied health technology. Through a comprehensive search string, studies published from 2012 to 2022 from the databases Scopus, PubMed, and Web of Science were scanned and collected for assessment. To minimize bias, retrieval of the relevant references in other studies in the field was ensured, and 2 authors assessed the collected studies. Low-quality studies were removed through a quality assessment and relevant data were extracted through summary tables for analysis. The articles were checked for similarities between author groups to prevent cumulative redundancy bias during the screening process, where only 1 article was included from the same author group.

RESULTS

In the analysis of the 145 included studies, support vector machines were the most utilized ML technique (51/145, 35.2%), with the most studied disease being Parkinson disease (PD; reported in 87/145, 60%, studies). After 2017, 16 additional voice-affecting disorders were examined, in contrast to the 3 investigated previously. Furthermore, an upsurge in the use of artificial neural network-based architectures was observed after 2017. Almost half of the included studies were published in last 2 years (2021 and 2022). A broad interest from many countries was observed. Notably, nearly one-half (n=75) of the studies relied on 10 distinct data sets, and 11/145 (7.6%) used demographic data as an input for ML models.

CONCLUSIONS

This SLR revealed considerable interest across multiple countries in using ML techniques for diagnosing and monitoring voice-affecting disorders, with PD being the most studied disorder. However, the review identified several gaps, including limited and unbalanced data set usage in studies, and a focus on diagnostic test rather than disorder-specific monitoring. Despite the limitations of being constrained by only peer-reviewed publications written in English, the SLR provides valuable insights into the current state of research on ML-based voice-affecting disorder diagnosis and monitoring and highlighting areas to address in future research.

Reading and lexical-semantic retrieval tasks outperforms single task speech analysis in the screening of mild cognitive impairment and Alzheimer's disease

Age-related cognitive impairment have increased dramatically in recent years, which has risen the interes in developing screening tools for mild cognitive impairment and Alzheimer's disease. Speech analysis allows to exploit the behavioral consequences of cognitive deficits on the patient's vocal performance so that it is possible to identify pathologies affecting speech production such as dementia. Previous studies have further shown that the speech task used determines how the speech parameters are altered. We aim to combine the impairments in several speech production tasks in order to improve the accuracy of screening through speech analysis. The sample consists of 72 participants divided into three equal groups of healthy older adults, people with mild cognitive impairment, or Alzheimer's disease, matched by age and education. A complete neuropsychological assessment and two voice recordings were performed. The tasks required the participants to read a text, and complete a sentence with semantic information. A stepwise linear discriminant analysis was performed to select speech parameters with discriminative power. The discriminative functions obtained an accuracy of 83.3% in simultaneous classifications of several levels of cognitive impairment. It would therefore be a promising screening tool for dementia.

Varied performance of picture description task as a screening tool across MCI subtypes

A picture description task is a component of Miro Health's platform for self-administration of neurobehavioral assessments. Picture description has been used as a screening tool for identification of individuals with Alzheimer's disease and mild cognitive impairment (MCI), but currently requires in-person administration and scoring by someone with access to and familiarity with a scoring rubric. The Miro Health implementation allows broader use of this assessment through self-administration and automated processing, analysis, and scoring to deliver clinically useful quantifications of the users' speech production, vocal characteristics, and language. Picture description responses were collected from 62 healthy controls (HC), and 33 participants with MCI: 18 with amnestic MCI (aMCI) and 15 with non-amnestic MCI (naMCI). Speech and language features and contrasts between pairs of features were evaluated for differences in their distributions in the participant subgroups. Picture description features were selected and combined using penalized logistic regression to form risk scores for classification of HC versus MCI as well as HC versus specific MCI subtypes. A picture-description based risk score distinguishes MCI and HC with an area under the receiver operator curve (AUROC) of 0.74. When contrasting specific subtypes of MCI and HC, the classifiers have an AUROC of 0.88 for aMCI versus HC and and AUROC of 0.61 for naMCI versus HC. Tests of association of individual features or contrasts of pairs of features with HC versus aMCI identified 20 features with p-values below 5e-3 and False Discovery Rates (FDRs) at or below 0.113, and 61 contrasts with p-values below 5e-4 and FDRs at or below 0.132. Findings suggest that performance of picture description as a screening tool for MCI detection will vary greatly by MCI subtype or by the proportion of various subtypes in an undifferentiated MCI population.

The Generalizability of Longitudinal Changes in Speech Before Alzheimer's Disease Diagnosis

BACKGROUND

Language impairment in Alzheimer's disease (AD) has been widely studied but due to limited data availability, relatively few studies have focused on the longitudinal change in language in the individuals who later develop AD. Significant differences in speech have previously been found by comparing the press conference transcripts of President Bush and President Reagan, who was later diagnosed with AD.

OBJECTIVE

In the current study, we explored whether the patterns previously established in the single AD-healthy control (HC) participant pair apply to a larger group of individuals who later receive AD diagnosis.

METHODS

We replicated previous methods on two larger corpora of longitudinal spontaneous speech samples of public figures, consisting of 10 and 9 AD-HC participant pairs. As we failed to find generalizable patterns of language change using previous methodology, we proposed alternative methods for data analysis, investigating the benefits of using different language features and their change with age, and compiling the single features into aggregate scores.

RESULTS

The single features that showed the strongest results were moving average type:token ratio (MATTR) and pronoun-related features. The aggregate scores performed better than the single features, with lexical diversity capturing a similar change in two-thirds of the participants.

CONCLUSION

Capturing universal patterns of language change prior to AD can be challenging, but the decline in lexical diversity and changes in MATTR and pronoun-related features act as promising measures that reflect the cognitive changes in many participants.

The Role of Silence in Verbal Fluency Tasks - A New Approach for the Detection of Mild Cognitive Impairment

OBJECTIVE

Most recordings of verbal fluency tasks include substantial amounts of task-irrelevant content that could provide clinically valuable information for the detection of mild cognitive impairment (MCI). We developed a method for the analysis of verbal fluency, focusing not on the task-relevant words but on the silent segments, the hesitations, and the irrelevant utterances found in the voice recordings.

METHODS

Phonemic ('k', 't', 'a') and semantic (animals, food items, actions) verbal fluency data were collected from healthy control (HC; n = 25; M_age = 67.32) and MCI (n = 25; M_age = 71.72) participants. After manual annotation of the voice samples, 10 temporal parameters were computed based on the silent and the task-irrelevant segments. Traditional fluency measures, based on word count (correct words, errors, repetitions) were also employed in order to compare the outcome of the two methods.

RESULTS

Two silence-based parameters (the number of silent pauses and the average length of silent pauses) and the average word transition time differed significantly between the two groups in the case of all three semantic fluency tasks. Subsequent receiver operating characteristic (ROC) analysis showed that these three temporal parameters had classification abilities similar to the traditional measure of counting correct words.

CONCLUSION

In our approach for verbal fluency analysis, silence-related parameters displayed classification ability similar to the most widely used traditional fluency measure. Based on these results, an automated tool using voiced-unvoiced segmentation may be developed enabling swift and cost-effective verbal fluency-based MCI screening.

Classification of mild cognitive impairment based on handwriting dynamics and qEEG

Subtle changes in fine motor control and quantitative electroencephalography (qEEG) in patients with mild cognitive impairment (MCI) are important in screening for early dementia in primary care populations. In this study, an automated, non-invasive and rapid detection protocol for mild cognitive impairment based on handwriting kinetics and quantitative EEG analysis was proposed, and a classification model based on a dual fusion of feature and decision layers was designed for clinical decision-marking. Seventy-nine volunteers (39 healthy elderly controls and 40 patients with mild cognitive impairment) were recruited for this study, and the handwritten data and the EEG signals were performed using a tablet and MUSE under four designed handwriting tasks. Sixty-eight features were extracted from the EEG and handwriting parameters of each test. Features selected from both models were fused using a late feature fusion strategy with a weighted voting strategy for decision making, and classification accuracy was compared using three different classifiers under handwritten features, EEG features and fused features respectively. The results show that the dual fusion model can further improve the classification accuracy, with the highest classification accuracy for the combined features and the best classification result of 96.3% using SVM with RBF kernel as the base classifier. In addition, this not only supports the greater significance of multimodal data for differentiating MCI, but also tests the feasibility of using the portable EEG headband as a measure of EEG in patients with cognitive impairment.

Synergy through integration of digital cognitive tests and wearable devices for mild cognitive impairment screening

Introduction

Advances in mobile computing platforms and the rapid development of wearable devices have made possible the continuous monitoring of patients with mild cognitive impairment (MCI) and their daily activities. Such rich data can reveal more subtle changes in patients' behavioral and physiological characteristics, providing new ways to detect MCI anytime, anywhere. Therefore, we aimed to investigate the feasibility and validity of digital cognitive tests and physiological sensors applied to MCI assessment.

Methods

We collected photoplethysmography (PPG), electrodermal activity (EDA) and electroencephalogram (EEG) signals from 120 participants (61 MCI patients, 59 healthy controls) during rest and cognitive testing. The features extracted from these physiological signals involved the time domain, frequency domain, time-frequency domain and statistics. Time and score features during the cognitive test are automatically recorded by the system. In addition, selected features of all modalities were classified by tenfold cross-validation using five different classifiers.

Results

The experimental results showed that the weighted soft voting strategy combining five classifiers achieved the highest classification accuracy (88.9%), precision (89.9%), recall (88.2%), and F1 score (89.0%). Compared to healthy controls, the MCI group typically took longer to recall, draw, and drag. Moreover, during cognitive testing, MCI patients showed lower heart rate variability, higher electrodermal activity values, and stronger brain activity in the alpha and beta bands.

Discussion

It was found that patients' classification performance improved when combining features from multiple modalities compared to using only tablet parameters or physiological features, indicating that our scheme could reveal MCI-related discriminative information. Furthermore, the best classification results on the digital span test across all tasks suggest that MCI patients may have deficits in attention and short-term memory that came to the fore earlier. Finally, integrating tablet cognitive tests and wearable sensors would provide a new direction for creating an easy-to-use and at-home self-check MCI screening tool.

Automatic Detection of Putative Mild Cognitive Impairment from Speech Acoustic Features in Mandarin-Speaking Elders

BACKGROUND

To date, the reliable detection of mild cognitive impairment (MCI) remains a significant challenge for clinicians. Very few studies investigated the sensitivity of acoustic features in detecting Mandarin-speaking elders at risk for MCI, defined as "putative MCI" (pMCI).

OBJECTIVE

This study sought to investigate the possibility of using automatically extracted speech acoustic features to detect elderly people with pMCI and reveal the potential acoustic markers of cognitive decline at an early stage.

METHODS

Forty-one older adults with pMCI and 41 healthy elderly controls completed four reading tasks (syllable utterance, tongue twister, diadochokinesis, and short sentence reading), from which acoustic features were extracted automatically to train machine learning classifiers. Correlation analysis was employed to evaluate the relationship between classifier predictions and participants' cognitive ability measured by Mini-Mental State Examination 2.

RESULTS

Classification results revealed that some temporal features (e.g., speech rate, utterance duration, and the number of silent pauses), spectral features (e.g., variability of F1 and F2), and energy features (e.g., SD of peak intensity and SD of intensity range) were effective predictors of pMCI. The best classification result was achieved in the Random Forest classifier (accuracy = 0.81, AUC = 0.81). Correlation analysis uncovered a strong negative correlation between participants' cognitive test scores and the probability estimates of pMCI in the Random Forest classifier, and a modest negative correlation in the Support Vector Machine classifier.

CONCLUSIONS

The automatic acoustic analysis of speech could provide a promising non-invasive way to assess and monitor the early cognitive decline in Mandarin-speaking elders.

Deep learning-based speech analysis for Alzheimer's disease detection: a literature review

BACKGROUND

Alzheimer's disease has become one of the most common neurodegenerative diseases worldwide, which seriously affects the health of the elderly. Early detection and intervention are the most effective prevention methods currently. Compared with traditional detection methods such as traditional scale tests, electroencephalograms, and magnetic resonance imaging, speech analysis is more convenient for automatic large-scale Alzheimer's disease detection and has attracted extensive attention from researchers. In particular, deep learning-based speech analysis and language processing techniques for Alzheimer's disease detection have been studied and achieved impressive results.

METHODS

To integrate the latest research progresses, hundreds of relevant papers from ACM, DBLP, IEEE, PubMed, Scopus, Web of Science electronic databases, and other sources were retrieved. We used these keywords for paper search: (Alzheimer OR dementia OR cognitive impairment) AND (speech OR voice OR audio) AND (deep learning OR neural network).

CONCLUSIONS

Fifty-two papers were finally retained after screening. We reviewed and presented the speech databases, deep learning methods, and model performances of these studies. In the end, we pointed out the mainstreams and limitations in the current studies and provided a direction for future research.

A longitudinal observational study of home-based conversations for detecting early dementia: protocol for the CUBOId TV task

INTRODUCTION

Limitations in effective dementia therapies mean that early diagnosis and monitoring are critical for disease management, but current clinical tools are impractical and/or unreliable, and disregard short-term symptom variability. Behavioural biomarkers of cognitive decline, such as speech, sleep and activity patterns, can manifest prodromal pathological changes. They can be continuously measured at home with smart sensing technologies, and permit leveraging of interpersonal interactions for optimising diagnostic and prognostic performance. Here we describe the ContinUous behavioural Biomarkers Of cognitive Impairment (CUBOId) study, which explores the feasibility of multimodal data fusion for in-home monitoring of mild cognitive impairment (MCI) and early Alzheimer's disease (AD). The report focuses on a subset of CUBOId participants who perform a novel speech task, the 'TV task', designed to track changes in ecologically valid conversations with disease progression.

METHODS AND ANALYSIS

CUBOId is a longitudinal observational study. Participants have diagnoses of MCI or AD, and controls are their live-in partners with no such diagnosis. Multimodal activity data were passively acquired from wearables and in-home fixed sensors over timespans of 8-25 months. At two time points participants completed the TV task over 5 days by recording audio of their conversations as they watched a favourite TV programme, with further testing to be completed after removal of the sensor installations. Behavioural testing is supported by neuropsychological assessment for deriving ground truths on cognitive status. Deep learning will be used to generate fused multimodal activity-speech embeddings for optimisation of diagnostic and predictive performance from speech alone.

ETHICS AND DISSEMINATION

CUBOId was approved by an NHS Research Ethics Committee (Wales REC; ref: 18/WA/0158) and is sponsored by University of Bristol. It is supported by the National Institute for Health Research Clinical Research Network West of England. Results will be reported at conferences and in peer-reviewed scientific journals.

Multimodal Fusion of Smart Home and Text-based Behavior Markers for Clinical Assessment Prediction

New modes of technology are offering unprecedented opportunities to unobtrusively collect data about people's behavior. While there are many use cases for such information, we explore its utility for predicting multiple clinical assessment scores. Because clinical assessments are typically used as screening tools for impairment and disease, such as mild cognitive impairment (MCI), automatically mapping behavioral data to assessment scores can help detect changes in health and behavior across time. In this paper, we aim to extract behavior markers from two modalities, a smart home environment and a custom digital memory notebook app, for mapping to ten clinical assessments that are relevant for monitoring MCI onset and changes in cognitive health. Smart home-based behavior markers reflect hourly, daily, and weekly activity patterns, while app-based behavior markers reflect app usage and writing content/style derived from free-form journal entries. We describe machine learning techniques for fusing these multimodal behavior markers and utilizing joint prediction. We evaluate our approach using three regression algorithms and data from 14 participants with MCI living in a smart home environment. We observed moderate to large correlations between predicted and ground-truth assessment scores, ranging from r = 0.601 to r = 0.871 for each clinical assessment.

Evaluating Web-Based Automatic Transcription for Alzheimer Speech Data: Transcript Comparison and Machine Learning Analysis

Background

Speech data for medical research can be collected noninvasively and in large volumes. Speech analysis has shown promise in diagnosing neurodegenerative disease. To effectively leverage speech data, transcription is important, as there is valuable information contained in lexical content. Manual transcription, while highly accurate, limits the potential scalability and cost savings associated with language-based screening.

Objective

To better understand the use of automatic transcription for classification of neurodegenerative disease, namely, Alzheimer disease (AD), mild cognitive impairment (MCI), or subjective memory complaints (SMC) versus healthy controls, we compared automatically generated transcripts against transcripts that went through manual correction.

Methods

We recruited individuals from a memory clinic (“patients”) with a diagnosis of mild-to-moderate AD, (n=44, 30%), MCI (n=20, 13%), SMC (n=8, 5%), as well as healthy controls (n=77, 52%) living in the community. Participants were asked to describe a standardized picture, read a paragraph, and recall a pleasant life experience. We compared transcripts generated using Google speech-to-text software to manually verified transcripts by examining transcription confidence scores, transcription error rates, and machine learning classification accuracy. For the classification tasks, logistic regression, Gaussian naive Bayes, and random forests were used.

Results

The transcription software showed higher confidence scores (P<.001) and lower error rates (P>.05) for speech from healthy controls compared with patients. Classification models using human-verified transcripts significantly (P<.001) outperformed automatically generated transcript models for both spontaneous speech tasks. This comparison showed no difference in the reading task. Manually adding pauses to transcripts had no impact on classification performance. However, manually correcting both spontaneous speech tasks led to significantly higher performances in the machine learning models.

Conclusions

We found that automatically transcribed speech data could be used to distinguish patients with a diagnosis of AD, MCI, or SMC from controls. We recommend a human verification step to improve the performance of automatic transcripts, especially for spontaneous tasks. Moreover, human verification can focus on correcting errors and adding punctuation to transcripts. However, manual addition of pauses is not needed, which can simplify the human verification step to more efficiently process large volumes of speech data.

Temporal speech parameters detect mild cognitive impairment in different languages: validation and comparison of the Speech-GAP Test® in English and Hungarian

BACKGROUND

The development of automatic speech recognition (ASR) technology allows the analysis of temporal (time-based) speech parameters characteristic of mild cognitive impairment (MCI). However, no information has been available on whether the analysis of spontaneous speech can be used with the same efficiency in different language environments.

OBJECTIVE

The main goal of this international pilot study is to address the question whether the Speech-Gap Test® (S-GAP Test®), previously tested in the Hungarian language, is appropriate for and applicable to the recognition of MCI in other languages such as English.

METHOD

After an initial screening of 88 individuals, English-speaking (n = 33) and Hungarian-speaking (n = 33) participants were classified as having MCI or as healthy controls (HC) based on Petersen's criteria. Speech of each participant was recorded via a spontaneous speech task. 15 temporal parameters were determined and calculated by means of ASR.

RESULTS

Seven temporal parameters in the English-speaking sample and 5 in the Hungarian-speaking sample showed significant differences between the MCI and the HC group. Receiver operating characteristics (ROC) analysis clearly distinguished the English-speaking MCI cases from the HC group based on speech tempo and articulation tempo with 100% sensitivity, and on three more temporal parameters with high sensitivity (85.7%). In the Hungarian-speaking sample, the ROC analysis showed similar sensitivity rates (92.3%).

CONCLUSION

The results of this study in different native-speaking populations suggest that changes in acoustic parameters detected by the S-GAP Test® might be present across different languages.

Temporal Speech Parameters Indicate Early Cognitive Decline in Elderly Patients With Type 2 Diabetes Mellitus

INTRODUCTION

The earliest signs of cognitive decline include deficits in temporal (time-based) speech characteristics. Type 2 diabetes mellitus (T2DM) patients are more prone to mild cognitive impairment (MCI). The aim of this study was to compare the temporal speech characteristics of elderly (above 50 y) T2DM patients with age-matched nondiabetic subjects.

MATERIALS AND METHODS

A total of 160 individuals were screened, 100 of whom were eligible (T2DM: n=51; nondiabetic: n=49). Participants were classified either as having healthy cognition (HC) or showing signs of MCI. Speech recordings were collected through a phone call. Based on automatic speech recognition, 15 temporal parameters were calculated.

RESULTS

The HC with T2DM group showed significantly shorter utterance length, higher duration rate of silent pause and total pause, and higher average duration of silent pause and total pause compared with the HC without T2DM group. Regarding the MCI participants, parameters were similar between the T2DM and the nondiabetic subgroups.

CONCLUSIONS

Temporal speech characteristics of T2DM patients showed early signs of altered cognitive functioning, whereas neuropsychological tests did not detect deterioration. This method is useful for identifying the T2DM patients most at risk for manifest MCI, and could serve as a remote cognitive screening tool.

Improving Alzheimer's Disease Detection for Speech Based on Feature Purification Network

Alzheimer's disease (AD) is a neurodegenerative disease involving the decline of cognitive ability with illness progresses. At present, the diagnosis of AD mainly depends on the interviews between patients and doctors, which is slow, expensive, and subjective, so it is not a better solution to recognize AD using the currently available neuropsychological examinations and clinical diagnostic criteria. A recent study has indicated the potential of language analysis for AD diagnosis. In this study, we proposed a novel feature purification network that can improve the representation learning of transformer model further. Though transformer has made great progress in generating discriminative features because of its long-distance reasoning ability, there is still room for improvement. There exist many common features that are not indicative of any specific class, and we rule out the influence of common features from traditional features extracted by transformer encoder and can get more discriminative features for classification. We apply this method to improve transformer's performance on three public dementia datasets and get improved classification results markedly. Specifically, the method on Pitt datasets gets state-of-the-art (SOTA) result.

Speech- and Language-Based Classification of Alzheimer's Disease: A Systematic Review

Background: Alzheimer’s disease (AD) has paramount importance due to its rising prevalence, the impact on the patient and society, and the related healthcare costs. However, current diagnostic techniques are not designed for frequent mass screening, delaying therapeutic intervention and worsening prognoses. To be able to detect AD at an early stage, ideally at a pre-clinical stage, speech analysis emerges as a simple low-cost non-invasive procedure. Objectives: In this work it is our objective to do a systematic review about speech-based detection and classification of Alzheimer’s Disease with the purpose of identifying the most effective algorithms and best practices. Methods: A systematic literature search was performed from Jan 2015 up to May 2020 using ScienceDirect, PubMed and DBLP. Articles were screened by title, abstract and full text as needed. A manual complementary search among the references of the included papers was also performed. Inclusion criteria and search strategies were defined a priori. Results: We were able: to identify the main resources that can support the development of decision support systems for AD, to list speech features that are correlated with the linguistic and acoustic footprint of the disease, to recognize the data models that can provide robust results and to observe the performance indicators that were reported. Discussion: A computational system with the adequate elements combination, based on the identified best-practices, can point to a whole new diagnostic approach, leading to better insights about AD symptoms and its disease patterns, creating conditions to promote a longer life span as well as an improvement in patient quality of life. The clinically relevant results that were identified can be used to establish a reference system and help to define research guidelines for future developments.

The applicability of the Beck Depression Inventory and Hamilton Depression Scale in the automatic recognition of depression based on speech signal processing

Depression is a growing problem worldwide, impacting on an increasing number of patients, and also affecting health systems and the global economy. The most common diagnostical rating scales of depression are self-reported or clinician-administered, which differ in the symptoms that they are sampling. Speech is a promising biomarker in the diagnostical assessment of depression, due to non-invasiveness and cost and time efficiency. In our study, we try to achieve a more accurate, sensitive model for determining depression based on speech processing. Regression and classification models were also developed using a machine learning method. During the research, we had access to a large speech database that includes speech samples from depressed and healthy subjects. The database contains the Beck Depression Inventory (BDI) score of each subject and the Hamilton Rating Scale for Depression (HAMD) score of 20% of the subjects. This fact provided an opportunity to compare the usefulness of BDI and HAMD for training models of automatic recognition of depression based on speech signal processing. We found that the estimated values of the acoustic model trained on BDI scores are closer to HAMD assessment than to the BDI scores, and the partial application of HAMD scores instead of BDI scores in training improves the accuracy of automatic recognition of depression.

Automatic Assessment of Loneliness in Older Adults Using Speech Analysis on Responses to Daily Life Questions

Loneliness is a perceived state of social and emotional isolation that has been associated with a wide range of adverse health effects in older adults. Automatically assessing loneliness by passively monitoring daily behaviors could potentially contribute to early detection and intervention for mitigating loneliness. Speech data has been successfully used for inferring changes in emotional states and mental health conditions, but its association with loneliness in older adults remains unexplored. In this study, we developed a tablet-based application and collected speech responses of 57 older adults to daily life questions regarding, for example, one's feelings and future travel plans. From audio data of these speech responses, we automatically extracted speech features characterizing acoustic, prosodic, and linguistic aspects, and investigated their associations with self-rated scores of the UCLA Loneliness Scale. Consequently, we found that with increasing loneliness scores, speech responses tended to have less inflections, longer pauses, reduced second formant frequencies, reduced variances of the speech spectrum, more filler words, and fewer positive words. The cross-validation results showed that regression and binary-classification models using speech features could estimate loneliness scores with an R 2 of 0.57 and detect individuals with high loneliness scores with 95.6% accuracy, respectively. Our study provides the first empirical results suggesting the possibility of using speech data that can be collected in everyday life for the automatic assessments of loneliness in older adults, which could help develop monitoring technologies for early detection and intervention for mitigating loneliness.

Data-Limited Deep Learning Methods for Mild Cognitive Impairment Classification in Alzheimer's Disease Patients

Mild Cognitive Impairment (MCI) is the stage between the declining of normal brain function and the more serious decline of dementia. Alzheimer's disease (AD) is one of the leading forms of dementia. Although MCI does not always lead to AD, an early diagnosis of MCI may be helpful in finding those with early signs of AD. The Alzheimer's Disease Neuroimaging Initiative (ADNI) has utilized magnetic resonance imaging (MRI) for the diagnosis of MCI and AD. MCI can be separated into two types: Early MCI (EMCI) and Late MCI (LMCI). Furthermore, MRI results can be separated into three views of axial, coronal and sagittal planes. In this work, we perform binary classifications between healthy people and the two types of MCI based on limited MRI images using deep learning approaches. Specifically, we implement and compare two various convolutional neural network (CNN) architectures. The MRIs of 516 patients were used in this study: 172 control normal (CN), 172 EMCI patients and 172 LMCI patients. For this data set, 50% of the images were used for training, 20% for validation, and the remaining 30% for testing. The results showed that the best classification for one model was between CN and LMCI for the coronal view with an accuracy of 79.67%. In addition, we achieved 67.85% accuracy for the second proposed model for the same classification group.

Classification of Alzheimer's Disease Leveraging Multi-task Machine Learning Analysis of Speech and Eye-Movement Data

Alzheimer's disease (AD) is a progressive neurodegenerative condition that results in impaired performance in multiple cognitive domains. Preclinical changes in eye movements and language can occur with the disease, and progress alongside worsening cognition. In this article, we present the results from a machine learning analysis of a novel multimodal dataset for AD classification. The cohort includes data from two novel tasks not previously assessed in classification models for AD (pupil fixation and description of a pleasant past experience), as well as two established tasks (picture description and paragraph reading). Our dataset includes language and eye movement data from 79 memory clinic patients with diagnoses of mild-moderate AD, mild cognitive impairment (MCI), or subjective memory complaints (SMC), and 83 older adult controls. The analysis of the individual novel tasks showed similar classification accuracy when compared to established tasks, demonstrating their discriminative ability for memory clinic patients. Fusing the multimodal data across tasks yielded the highest overall AUC of 0.83 ± 0.01, indicating that the data from novel tasks are complementary to established tasks.

A Pilot Tool of the Virtual Scenario Initial Dementia Cognitive Screening (VSIDCS) with a Cultural Exhibition for Improving the Standard Traditional Test

Dementia has become a serious global health problem for older people. In the past, primary screening for dementia was carried out by a paper test. These standard traditional tests (e.g., Montreal Cognitive Assessment (MoCA) and Mini-Mental State Examination (MMSE)) have been used for many years. In addition to paper tests, is there another way to let people have better involvement and emotions during the test procedure? With the advancement of technology, the application of virtual reality (VR) and augmented reality (AR) have changed and improved many medical technologies. However, there are few applications of VR and AR in dementia screening. The purpose of this study was to apply VR and AR to construct a pilot tool for virtual scenario initial dementia cognitive screening (VSIDCS) with a cultural exhibition, to achieve better involvement and emotions in participants. There were three operating interfaces designed for the system: a VR screening interface, cognitive board, and AR recognition interface. There were twenty-four middle-aged people (Female 10 and Male 14 between 50 and 65 years of age and with an average age of 58.7 years) selected for the test. The results of the experiments showed that VSIDCS test scores are consistent with those of the MoCA and MMSE. Additionally, VSIDCS can induce better involvement and emotions than the MoCA and MMSE. Participants showed better enthusiasm and more positive experiences during the VSIDCS test.

Identification of Mild Cognitive Impairment Among Chinese Based on Multiple Spoken Tasks

BACKGROUND

Previous studies explored the use of noninvasive biomarkers of speech and language for the detection of mild cognitive impairment (MCI). Yet, most of them employed single task which might not have adequately captured all aspects of their cognitive functions.

OBJECTIVE

The present study aimed to achieve the state-of-the-art accuracy in detecting individuals with MCI using multiple spoken tasks and uncover task-specific contributions with a tentative interpretation of features.

METHODS

Fifty patients clinically diagnosed with MCI and 60 healthy controls completed three spoken tasks (picture description, semantic fluency, and sentence repetition), from which multidimensional features were extracted to train machine learning classifiers. With a late-fusion configuration, predictions from multiple tasks were combined and correlated with the participants' cognitive ability assessed using the Montreal Cognitive Assessment (MoCA). Statistical analyses on pre-defined features were carried out to explore their association with the diagnosis.

RESULTS

The late-fusion configuration could effectively boost the final classification result (SVM: F1 = 0.95; RF: F1 = 0.96; LR: F1 = 0.93), outperforming each individual task classifier. Besides, the probability estimates of MCI were strongly correlated with the MoCA scores (SVM: -0.74; RF: -0.71; LR: -0.72).

CONCLUSION

Each single task tapped more dominantly to distinct cognitive processes and have specific contributions to the prediction of MCI. Specifically, picture description task characterized communications at the discourse level, while semantic fluency task was more specific to the controlled lexical retrieval processes. With greater demands on working memory load, sentence repetition task uncovered memory deficits through modified speech patterns in the reproduced sentences.

Comparing Pre-trained and Feature-Based Models for Prediction of Alzheimer's Disease Based on Speech

Introduction: Research related to the automatic detection of Alzheimer's disease (AD) is important, given the high prevalence of AD and the high cost of traditional diagnostic methods. Since AD significantly affects the content and acoustics of spontaneous speech, natural language processing, and machine learning provide promising techniques for reliably detecting AD. There has been a recent proliferation of classification models for AD, but these vary in the datasets used, model types and training and testing paradigms. In this study, we compare and contrast the performance of two common approaches for automatic AD detection from speech on the same, well-matched dataset, to determine the advantages of using domain knowledge vs. pre-trained transfer models. Methods: Audio recordings and corresponding manually-transcribed speech transcripts of a picture description task administered to 156 demographically matched older adults, 78 with Alzheimer's Disease (AD) and 78 cognitively intact (healthy) were classified using machine learning and natural language processing as "AD" or "non-AD." The audio was acoustically-enhanced, and post-processed to improve quality of the speech recording as well control for variation caused by recording conditions. Two approaches were used for classification of these speech samples: (1) using domain knowledge: extracting an extensive set of clinically relevant linguistic and acoustic features derived from speech and transcripts based on prior literature, and (2) using transfer-learning and leveraging large pre-trained machine learning models: using transcript-representations that are automatically derived from state-of-the-art pre-trained language models, by fine-tuning Bidirectional Encoder Representations from Transformer (BERT)-based sequence classification models. Results: We compared the utility of speech transcript representations obtained from recent natural language processing models (i.e., BERT) to more clinically-interpretable language feature-based methods. Both the feature-based approaches and fine-tuned BERT models significantly outperformed the baseline linguistic model using a small set of linguistic features, demonstrating the importance of extensive linguistic information for detecting cognitive impairments relating to AD. We observed that fine-tuned BERT models numerically outperformed feature-based approaches on the AD detection task, but the difference was not statistically significant. Our main contribution is the observation that when tested on the same, demographically balanced dataset and tested on independent, unseen data, both domain knowledge and pretrained linguistic models have good predictive performance for detecting AD based on speech. It is notable that linguistic information alone is capable of achieving comparable, and even numerically better, performance than models including both acoustic and linguistic features here. We also try to shed light on the inner workings of the more black-box natural language processing model by performing an interpretability analysis, and find that attention weights reveal interesting patterns such as higher attribution to more important information content units in the picture description task, as well as pauses and filler words. Conclusion: This approach supports the value of well-performing machine learning and linguistically-focussed processing techniques to detect AD from speech and highlights the need to compare model performance on carefully balanced datasets, using consistent same training parameters and independent test datasets in order to determine the best performing predictive model.

Ten Years of Research on Automatic Voice and Speech Analysis of People With Alzheimer's Disease and Mild Cognitive Impairment: A Systematic Review Article

Background: The field of voice and speech analysis has become increasingly popular over the last 10 years, and articles on its use in detecting neurodegenerative diseases have proliferated. Many studies have identified characteristic speech features that can be used to draw an accurate distinction between healthy aging among older people and those with mild cognitive impairment and Alzheimer's disease. Speech analysis has been singled out as a cost-effective and reliable method for detecting the presence of both conditions. In this research, a systematic review was conducted to determine these features and their diagnostic accuracy.

Methods: Peer-reviewed literature was located across multiple databases, involving studies that apply new procedures of automatic speech analysis to collect behavioral evidence of linguistic impairments along with their diagnostic accuracy on Alzheimer's disease and mild cognitive impairment. The risk of bias was assessed by using JBI and QUADAS-2 checklists.

Results: Thirty-five papers met the inclusion criteria; of these, 11 were descriptive studies that either identified voice features or explored their cognitive correlates, and the rest were diagnostic studies. Overall, the studies were of good quality and presented solid evidence of the usefulness of this technique. The distinctive acoustic and rhythmic features found are gathered. Most studies record a diagnostic accuracy over 88% for Alzheimer's and 80% for mild cognitive impairment.

Conclusion: Automatic speech analysis is a promising tool for diagnosing mild cognitive impairment and Alzheimer's disease. The reported features seem to be indicators of the cognitive changes in older people. The specific features and the cognitive changes involved could be the subject of further research.

Characterising spoken responses to an intelligent virtual agent by persons with mild cognitive impairment

The diagnosis of Mild Cognitive Impairment (MCI) characterises patients at risk of dementia and may provide an opportunity for disease-modifying interventions. Identifying persons with MCI (PwMCI) from adults of a similar age without cognitive complaints is a significant challenge. The main aims of this study were to determine whether generic speech differences were evident between PwMCI and healthy controls (HC), whether such differences were identifiable in responses to recent or remote memory questions, and to determine which speech variables showed the clearest between-group differences. This study analysed recordings of 8 PwMCI (5 females, 3 males) and 14 HC of a similar age (8 females, 6 males). Participants were recorded interacting with an intelligent virtual agent: a computer-generated talking head on a computer screen which asks pre-recorded questions when prompted by the interviewee through pressing the next key on a computer keyboard. Responses to recent and remote memory questions were analysed. Mann-Whitney U tests were used to test for statistically significant differences between PwMCI and HC on each of 12 speech variables, relating to temporal characteristics, number of words produced and pitch. It was found that compared to HC, PwMCI produce speech for less time and in shorter chunks, they pause more often and for longer, take longer to begin speaking and produce fewer words in their answers. It was also found that the PwMCI and HC were more alike when responding to remote memory questions than when responding to recent memory questions. These findings show great promise and suggest that detailed speech analysis can make an important contribution to diagnostic and stratification systems in patients with memory complaints.

Pauses for Detection of Alzheimer’s Disease

Pauses, disfluencies and language problems in Alzheimer’s disease can be naturally modeled by fine-tuning Transformer-based pre-trained language models such as BERT and ERNIE. Using this method with pause-encoded transcripts, we achieved 89.6% accuracy on the test set of the ADReSS (Alzheimer’sDementiaRecognition throughSpontaneousSpeech) Challenge. The best accuracy was obtained with ERNIE, plus an encoding of pauses. Robustness is a challenge for large models and small training sets. Ensemble over many runs of BERT/ERNIE fine-tuning reduced variance and improved accuracy. We found thatumwas used much less frequently in Alzheimer’s speech, compared touh. We discussed this interesting finding from linguistic and cognitive perspectives.

Classifying Alzheimer's Disease Using Audio and Text-Based Representations of Speech

Alzheimer's Disease (AD) is a form of dementia that affects the memory, cognition, and motor skills of patients. Extensive research has been done to develop accessible, cost-effective, and non-invasive techniques for the automatic detection of AD. Previous research has shown that speech can be used to distinguish between healthy patients and afflicted patients. In this paper, the ADReSS dataset, a dataset balanced by gender and age, was used to automatically classify AD from spontaneous speech. The performance of five classifiers, as well as a convolutional neural network and long short-term memory network, was compared when trained on audio features (i-vectors and x-vectors) and text features (word vectors, BERT embeddings, LIWC features, and CLAN features). The same audio and text features were used to train five regression models to predict the Mini-Mental State Examination score for each patient, a score that has a maximum value of 30. The top-performing classification models were the support vector machine and random forest classifiers trained on BERT embeddings, which both achieved an accuracy of 85.4% on the test set. The best-performing regression model was the gradient boosting regression model trained on BERT embeddings and CLAN features, which had a root mean squared error of 4.56 on the test set. The performance on both tasks illustrates the feasibility of using speech to classify AD and predict neuropsychological scores.