1
|
Xu F, Zheng Q, Shi J, Yan K, Wang M. Pre-training and ensembling based Alzheimer's disease detection. Technol Health Care 2024; 32:379-395. [PMID: 37545287 DOI: 10.3233/thc-230571] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/08/2023]
Abstract
BACKGROUND Alzheimer's disease (AD) endangers the physical and mental health of the elderly, constituting one of the most crucial social challenges. Due to lack of effective AD intervention drugs, it is very important to diagnose AD in the early stage, especially in the Mild Cognitive Impairment (MCI) phase. OBJECTIVE At present, an automatic classification technology is urgently needed to assist doctors in analyzing the status of the candidate patient. The artificial intelligence enhanced Alzheimer's disease detection can reduce costs to detect Alzheimer's disease. METHODS In this paper, a novel pre-trained ensemble-based AD detection (PEADD) framework with three base learners (i.e., ResNet, VGG, and EfficientNet) for both the audio-based and PET (Positron Emission Tomography)-based AD detection is proposed under a unified image modality. Specifically, the effectiveness of context-enriched image modalities instead of the traditional speech modality (i.e., context-free audio matrix) for the audio-based AD detection, along with simple and efficient image denoising strategy has been inspected comprehensively. Meanwhile, the PET-based AD detection based on the denoised PET image has been described. Furthermore, different voting methods for applying an ensemble strategy (i.e., hard voting and soft voting) has been investigated in detail. RESULTS The results showed that the classification accuracy was 92% and 99% on the audio-based and PET-based AD datasets, respectively. Our extensive experimental results demonstrate that our PEADD outperforms the state-of-the-art methods on both audio-based and PET-based AD datasets simultaneously. CONCLUSIONS The network model can provide an objective basis for doctors to detect Alzheimer's Disease.
Collapse
|
2
|
Sakharova T, Mao S, Osadchuk M. Updated Models of Alzheimer's Disease with Deep Neural Networks. J Alzheimers Dis 2024; 100:685-697. [PMID: 38905045 DOI: 10.3233/jad-240183] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/23/2024]
Abstract
Background In recent years, researchers have focused on developing precise models for the progression of Alzheimer's disease (AD) using deep neural networks. Forecasting the progression of AD through the analysis of time series data represents a promising approach. Objective The primary objective of this research is to formulate an effective methodology for forecasting the progression of AD through the integration of multi-task learning techniques and the analysis of pertinent medical data. Methods This study primarily utilized volumetric measurements obtained through magnetic resonance imaging (MRI), trajectories of cognitive assessments, and clinical status indicators. The research encompassed 150 patients diagnosed with AD who underwent examination between 2020 and 2022 in Beijing, China. A multi-task learning approach was employed to train forecasting models using MRI data, trajectories of cognitive assessments, and clinical status. Correlation analysis was conducted at various time points. Results At the baseline, a robust correlation was observed among the forecasting tasks: 0.75 for volumetric MRI measurements, 0.62 for trajectories of cognitive assessment, and 0.48 for clinical status. The implementation of a multi-task learning framework enhanced performance by 12.7% for imputing missing values and 14.8% for prediction accuracy. Conclusions The findings of our study, indicate that multi-task learning can effectively predict the progression of AD. However, it is important to note that the study's generalizability may be limited due to the restricted dataset and the specific population under examination. These conclusions represent a significant stride toward more precise diagnosis and treatment of this neurological disorder.
Collapse
Affiliation(s)
- Tatyana Sakharova
- Department of Biology and General Genetics, I.M. Sechenov First Moscow State Medical University, Moscow, Russia
| | - Siqi Mao
- Department of Statistics, Florida State University, Tallahassee, FL, USA
| | - Mikhail Osadchuk
- Department of Polyclinic Therapy, I.M. Sechenov First Moscow State Medical University, Moscow, Russia
| |
Collapse
|
3
|
Cabrera-León Y, Báez PG, Fernández-López P, Suárez-Araujo CP. Neural Computation-Based Methods for the Early Diagnosis and Prognosis of Alzheimer's Disease Not Using Neuroimaging Biomarkers: A Systematic Review. J Alzheimers Dis 2024; 98:793-823. [PMID: 38489188 PMCID: PMC11091566 DOI: 10.3233/jad-231271] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/03/2024] [Indexed: 03/17/2024]
Abstract
Background The growing number of older adults in recent decades has led to more prevalent geriatric diseases, such as strokes and dementia. Therefore, Alzheimer's disease (AD), as the most common type of dementia, has become more frequent too. Background Objective: The goals of this work are to present state-of-the-art studies focused on the automatic diagnosis and prognosis of AD and its early stages, mainly mild cognitive impairment, and predicting how the research on this topic may change in the future. Methods Articles found in the existing literature needed to fulfill several selection criteria. Among others, their classification methods were based on artificial neural networks (ANNs), including deep learning, and data not from brain signals or neuroimaging techniques were used. Considering our selection criteria, 42 articles published in the last decade were finally selected. Results The most medically significant results are shown. Similar quantities of articles based on shallow and deep ANNs were found. Recurrent neural networks and transformers were common with speech or in longitudinal studies. Convolutional neural networks (CNNs) were popular with gait or combined with others in modular approaches. Above one third of the cross-sectional studies utilized multimodal data. Non-public datasets were frequently used in cross-sectional studies, whereas the opposite in longitudinal ones. The most popular databases were indicated, which will be helpful for future researchers in this field. Conclusions The introduction of CNNs in the last decade and their superb results with neuroimaging data did not negatively affect the usage of other modalities. In fact, new ones emerged.
Collapse
Affiliation(s)
- Ylermi Cabrera-León
- Instituto Universitario de Cibernética, Empresa y Sociedad, Universidad de Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Canary Islands, Spain
| | - Patricio García Báez
- Departamento de Ingeniería Informática y de Sistemas, Escuela Superior de Ingeniería y Tecnología, Universidad de La Laguna, San Cristóbal de La Laguna, Canary Islands, Spain
| | - Pablo Fernández-López
- Instituto Universitario de Cibernética, Empresa y Sociedad, Universidad de Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Canary Islands, Spain
| | - Carmen Paz Suárez-Araujo
- Instituto Universitario de Cibernética, Empresa y Sociedad, Universidad de Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Canary Islands, Spain
| |
Collapse
|
4
|
Parsapoor M. AI-based assessments of speech and language impairments in dementia. Alzheimers Dement 2023; 19:4675-4687. [PMID: 37578167 DOI: 10.1002/alz.13395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Revised: 06/03/2023] [Accepted: 06/05/2023] [Indexed: 08/15/2023]
Abstract
Recent advancements in the artificial intelligence (AI) domain have revolutionized the early detection of cognitive impairments associated with dementia. This has motivated clinicians to use AI-powered dementia detection systems, particularly systems developed based on individuals' and patients' speech and language, for a quick and accurate identification of patients with dementia. This paper reviews articles about developing assessment tools using machine learning and deep learning algorithms trained by vocal and textual datasets.
Collapse
Affiliation(s)
- Mahboobeh Parsapoor
- Centre de Recherche Informatique de Montréal: CRIM, Montreal, Quebec, Canada
| |
Collapse
|
5
|
Zolnoori M, Zolnour A, Topaz M. ADscreen: A speech processing-based screening system for automatic identification of patients with Alzheimer's disease and related dementia. Artif Intell Med 2023; 143:102624. [PMID: 37673583 PMCID: PMC10483114 DOI: 10.1016/j.artmed.2023.102624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2022] [Revised: 06/22/2023] [Accepted: 07/08/2023] [Indexed: 09/08/2023]
Abstract
Alzheimer's disease and related dementias (ADRD) present a looming public health crisis, affecting roughly 5 million people and 11 % of older adults in the United States. Despite nationwide efforts for timely diagnosis of patients with ADRD, >50 % of them are not diagnosed and unaware of their disease. To address this challenge, we developed ADscreen, an innovative speech-processing based ADRD screening algorithm for the protective identification of patients with ADRD. ADscreen consists of five major components: (i) noise reduction for reducing background noises from the audio-recorded patient speech, (ii) modeling the patient's ability in phonetic motor planning using acoustic parameters of the patient's voice, (iii) modeling the patient's ability in semantic and syntactic levels of language organization using linguistic parameters of the patient speech, (iv) extracting vocal and semantic psycholinguistic cues from the patient speech, and (v) building and evaluating the screening algorithm. To identify important speech parameters (features) associated with ADRD, we used the Joint Mutual Information Maximization (JMIM), an effective feature selection method for high dimensional, small sample size datasets. Modeling the relationship between speech parameters and the outcome variable (presence/absence of ADRD) was conducted using three different machine learning (ML) architectures with the capability of joining informative acoustic and linguistic with contextual word embedding vectors obtained from the DistilBERT (Bidirectional Encoder Representations from Transformers). We evaluated the performance of the ADscreen on an audio-recorded patients' speech (verbal description) for the Cookie-Theft picture description task, which is publicly available in the dementia databank. The joint fusion of acoustic and linguistic parameters with contextual word embedding vectors of DistilBERT achieved F1-score = 84.64 (standard deviation [std] = ±3.58) and AUC-ROC = 92.53 (std = ±3.34) for training dataset, and F1-score = 89.55 and AUC-ROC = 93.89 for the test dataset. In summary, ADscreen has a strong potential to be integrated with clinical workflow to address the need for an ADRD screening tool so that patients with cognitive impairment can receive appropriate and timely care.
Collapse
Affiliation(s)
- Maryam Zolnoori
- Columbia University Medical Center, New York, NY, United States of America; School of Nursing, Columbia University, New York, NY, United States of America.
| | - Ali Zolnour
- School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran
| | - Maxim Topaz
- Columbia University Medical Center, New York, NY, United States of America; School of Nursing, Columbia University, New York, NY, United States of America
| |
Collapse
|
6
|
Zheng C, Bouazizi M, Ohtsuki T, Kitazawa M, Horigome T, Kishimoto T. Detecting Dementia from Face-Related Features with Automated Computational Methods. Bioengineering (Basel) 2023; 10:862. [PMID: 37508889 PMCID: PMC10376259 DOI: 10.3390/bioengineering10070862] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Revised: 07/13/2023] [Accepted: 07/14/2023] [Indexed: 07/30/2023] Open
Abstract
Alzheimer's disease (AD) is a type of dementia that is more likely to occur as people age. It currently has no known cure. As the world's population is aging quickly, early screening for AD has become increasingly important. Traditional screening methods such as brain scans or psychiatric tests are stressful and costly. The patients are likely to feel reluctant to such screenings and fail to receive timely intervention. While researchers have been exploring the use of language in dementia detection, less attention has been given to face-related features. The paper focuses on investigating how face-related features can aid in detecting dementia by exploring the PROMPT dataset that contains video data collected from patients with dementia during interviews. In this work, we extracted three types of features from the videos, including face mesh, Histogram of Oriented Gradients (HOG) features, and Action Units (AU). We trained traditional machine learning models and deep learning models on the extracted features and investigated their effectiveness in dementia detection. Our experiments show that the use of HOG features achieved the highest accuracy of 79% in dementia detection, followed by AU features with 71% accuracy, and face mesh features with 66% accuracy. Our results show that face-related features have the potential to be a crucial indicator in automated computational dementia detection.
Collapse
Affiliation(s)
- Chuheng Zheng
- Graduate School of Science and Technology, Keio University, Yokohama 223-0061, Kanagawa, Japan
| | - Mondher Bouazizi
- Faculty of Science and Technology, Keio University, Yokohama 223-0061, Kanagawa, Japan
| | - Tomoaki Ohtsuki
- Faculty of Science and Technology, Keio University, Yokohama 223-0061, Kanagawa, Japan
| | - Momoko Kitazawa
- School of Medicine, Keio University, 35 Shinanomachi, Shinjuku-ku, Tokyo 160-8582, Japan
| | - Toshiro Horigome
- School of Medicine, Keio University, 35 Shinanomachi, Shinjuku-ku, Tokyo 160-8582, Japan
| | - Taishiro Kishimoto
- School of Medicine, Keio University, 35 Shinanomachi, Shinjuku-ku, Tokyo 160-8582, Japan
| |
Collapse
|
7
|
Lau C, Zhu X, Chan WY. Automatic depression severity assessment with deep learning using parameter-efficient tuning. Front Psychiatry 2023; 14:1160291. [PMID: 37398577 PMCID: PMC10308283 DOI: 10.3389/fpsyt.2023.1160291] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Accepted: 06/01/2023] [Indexed: 07/04/2023] Open
Abstract
Introduction To assist mental health care providers with the assessment of depression, research to develop a standardized, accessible, and non-invasive technique has garnered considerable attention. Our study focuses on the application of deep learning models for automatic assessment of depression severity based on clinical interview transcriptions. Despite the recent success of deep learning, the lack of large-scale high-quality datasets is a major performance bottleneck for many mental health applications. Methods A novel approach is proposed to address the data scarcity problem for depression assessment. It leverages both pretrained large language models and parameter-efficient tuning techniques. The approach is built upon adapting a small set of tunable parameters, known as prefix vectors, to guide a pretrained model towards predicting the Patient Health Questionnaire (PHQ)-8 score of a person. Experiments were conducted on the Distress Analysis Interview Corpus - Wizard of Oz (DAIC-WOZ) benchmark dataset with 189 subjects, partitioned into training, development, and test sets. Model learning was done on the training set. Prediction performance mean and standard deviation of each model, with five randomly-initialized runs, were reported on the development set. Finally, optimized models were evaluated on the test set. Results The proposed model with prefix vectors outperformed all previously published methods, including models which utilized multiple types of data modalities, and achieved the best reported performance on the test set of DAIC-WOZ with a root mean square error of 4.67 and a mean absolute error of 3.80 on the PHQ-8 scale. Compared to conventionally fine-tuned baseline models, prefix-enhanced models were less prone to overfitting by using far fewer training parameters (<6% relatively). Discussion While transfer learning through pretrained large language models can provide a good starting point for downstream learning, prefix vectors can further adapt the pretrained models effectively to the depression assessment task by only adjusting a small number of parameters. The improvement is in part due to the fine-grain flexibility of prefix vector size in adjusting the model's learning capacity. Our results provide evidence that prefix-tuning can be a useful approach in developing tools for automatic depression assessment.
Collapse
|
8
|
Dupuy-Zini A, Audeh B, Gérardin C, Duclos C, Gagneux-Brunon A, Bousquet C. Users' Reactions to Announced Vaccines Against COVID-19 Before Marketing in France: Analysis of Twitter Posts. J Med Internet Res 2023; 25:e37237. [PMID: 36596215 PMCID: PMC10132828 DOI: 10.2196/37237] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Revised: 07/17/2022] [Accepted: 08/09/2022] [Indexed: 01/04/2023] Open
Abstract
BACKGROUND Within a few months, the COVID-19 pandemic had spread to many countries and had been a real challenge for health systems all around the world. This unprecedented crisis has led to a surge of online discussions about potential cures for the disease. Among them, vaccines have been at the heart of the debates and have faced lack of confidence before marketing in France. OBJECTIVE This study aims to identify and investigate the opinions of French Twitter users on the announced vaccines against COVID-19 through sentiment analysis. METHODS This study was conducted in 2 phases. First, we filtered a collection of tweets related to COVID-19 available on Twitter from February 2020 to August 2020 with a set of keywords associated with vaccine mistrust using word embeddings. Second, we performed sentiment analysis using deep learning to identify the characteristics of vaccine mistrust. The model was trained on a hand-labeled subset of 4548 tweets. RESULTS A set of 69 relevant keywords were identified as the semantic concept of the word "vaccin" (vaccine in French) and focused mainly on conspiracies, pharmaceutical companies, and alternative treatments. Those keywords enabled us to extract nearly 350,000 tweets in French. The sentiment analysis model achieved 0.75 accuracy. The model then predicted 16% of positive tweets, 41% of negative tweets, and 43% of neutral tweets. This allowed us to explore the semantic concepts of positive and negative tweets and to plot the trends of each sentiment. The main negative rhetoric identified from users' tweets was that vaccines are perceived as having a political purpose and that COVID-19 is a commercial argument for the pharmaceutical companies. CONCLUSIONS Twitter might be a useful tool to investigate the arguments for vaccine mistrust because it unveils political criticism contrasting with the usual concerns on adverse drug reactions. As the opposition rhetoric is more consistent and more widely spread than the positive rhetoric, we believe that this research provides effective tools to help health authorities better characterize the risk of vaccine mistrust.
Collapse
Affiliation(s)
- Alexandre Dupuy-Zini
- Laboratoire d'Informatique Médicale et d'Ingénierie des connaissances en e-Santé, LIMICS, Sorbonne Université, Université Sorbonne Paris Nord, Institut national de la santé et de la recherche médicale, INSERM, Paris, France
| | - Bissan Audeh
- Laboratoire d'Informatique Médicale et d'Ingénierie des connaissances en e-Santé, LIMICS, Sorbonne Université, Université Sorbonne Paris Nord, Institut national de la santé et de la recherche médicale, INSERM, Paris, France
| | - Christel Gérardin
- Institut Pierre Louis d'Epidémiologie et de Santé Publique, Département de médecine interne, Sorbonne Université, Paris, France
| | - Catherine Duclos
- Laboratoire d'Informatique Médicale et d'Ingénierie des connaissances en e-Santé, LIMICS, Sorbonne Université, Université Sorbonne Paris Nord, Institut national de la santé et de la recherche médicale, INSERM, Paris, France
| | - Amandine Gagneux-Brunon
- Groupe sur l'Immunité des Muqueuses et Agents Pathogènes, Centre International de Recherche en Infectiologie, University of Lyon, Saint Etienne, France
- Vaccinologie, Centre Hospitalier Universitaire de Saint-Etienne, Saint Etienne, France
| | - Cedric Bousquet
- Laboratoire d'Informatique Médicale et d'Ingénierie des connaissances en e-Santé, LIMICS, Sorbonne Université, Université Sorbonne Paris Nord, Institut national de la santé et de la recherche médicale, INSERM, Paris, France
- Service de santé publique et information médicale, Centre Hospitalier Universitaire de Saint Etienne, Saint Etienne, France
| |
Collapse
|
9
|
Detecting dementia from speech and transcripts using transformers. COMPUT SPEECH LANG 2023. [DOI: 10.1016/j.csl.2023.101485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023]
|
10
|
Yang Q, Li X, Ding X, Xu F, Ling Z. Deep learning-based speech analysis for Alzheimer's disease detection: a literature review. Alzheimers Res Ther 2022; 14:186. [PMID: 36517837 PMCID: PMC9749308 DOI: 10.1186/s13195-022-01131-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2022] [Accepted: 11/23/2022] [Indexed: 12/23/2022]
Abstract
BACKGROUND Alzheimer's disease has become one of the most common neurodegenerative diseases worldwide, which seriously affects the health of the elderly. Early detection and intervention are the most effective prevention methods currently. Compared with traditional detection methods such as traditional scale tests, electroencephalograms, and magnetic resonance imaging, speech analysis is more convenient for automatic large-scale Alzheimer's disease detection and has attracted extensive attention from researchers. In particular, deep learning-based speech analysis and language processing techniques for Alzheimer's disease detection have been studied and achieved impressive results. METHODS To integrate the latest research progresses, hundreds of relevant papers from ACM, DBLP, IEEE, PubMed, Scopus, Web of Science electronic databases, and other sources were retrieved. We used these keywords for paper search: (Alzheimer OR dementia OR cognitive impairment) AND (speech OR voice OR audio) AND (deep learning OR neural network). CONCLUSIONS Fifty-two papers were finally retained after screening. We reviewed and presented the speech databases, deep learning methods, and model performances of these studies. In the end, we pointed out the mainstreams and limitations in the current studies and provided a direction for future research.
Collapse
Affiliation(s)
- Qin Yang
- iFlytek Research, iFlytek Co.Ltd, Hefei, China
| | - Xin Li
- grid.59053.3a0000000121679639NELSLIP, University of Science and Technology of China, Hefei, China ,iFlytek Research, iFlytek Co.Ltd, Hefei, China
| | - Xinyun Ding
- iFlytek Research, iFlytek Co.Ltd, Hefei, China
| | - Feiyang Xu
- iFlytek Research, iFlytek Co.Ltd, Hefei, China
| | - Zhenhua Ling
- grid.59053.3a0000000121679639NELSLIP, University of Science and Technology of China, Hefei, China
| |
Collapse
|
11
|
Nunes M, Teles AS, Farias D, Diniz C, Bastos VH, Teixeira S. A Telemedicine Platform for Aphasia: Protocol for a Development and Usability Study. JMIR Res Protoc 2022; 11:e40603. [PMID: 36422881 PMCID: PMC9732749 DOI: 10.2196/40603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2022] [Revised: 09/02/2022] [Accepted: 10/28/2022] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND Aphasia is a central disorder of comprehension and expression of language that cannot be attributed to a peripheral sensory deficit or a peripheral motor disorder. The diagnosis and treatment of aphasia are complex. Interventions that facilitate this process can lead to an increase in the number of assisted patients and greater precision in the therapeutic choice by the health professional. OBJECTIVE This paper describes a protocol for a study that aims to implement a computer-based solution (ie, a telemedicine platform) that uses deep learning to classify vocal data from participants with aphasia and to develop serious games to treat aphasia. Additionally, this study aims to evaluate the usability and user experience of the proposed solution. METHODS Our interactive and smart platform will be developed to provide an alternative option for professionals and their patients with aphasia. We will design 2 serious games for aphasia rehabilitation and a deep learning-driven computational solution to aid diagnosis. A pilot evaluation of usability and user experience will reveal user satisfaction with platform features. RESULTS Data collection began in June 2022 and is currently ongoing. Results of system development as well as usability should be published by mid-2023. CONCLUSIONS This research will contribute to the treatment and diagnosis of aphasia by developing a telemedicine platform based on a co-design process. Therefore, this research will provide an alternative method for health care to patients with aphasia. Additionally, it will guide further studies with the same purpose. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) PRR1-10.2196/40603.
Collapse
Affiliation(s)
- Monara Nunes
- Federal University of Piauí, Regeneração, Brazil
| | | | - Daniel Farias
- Federal University of Delta do Parnaíba, Parnaíba, Brazil
| | - Claudia Diniz
- Federal University of Delta do Parnaíba, Parnaíba, Brazil
| | | | | |
Collapse
|
12
|
Fristed E, Skirrow C, Meszaros M, Lenain R, Meepegama U, Papp KV, Ropacki M, Weston J. Leveraging speech and artificial intelligence to screen for early Alzheimer's disease and amyloid beta positivity. Brain Commun 2022; 4:fcac231. [PMID: 36381988 PMCID: PMC9639797 DOI: 10.1093/braincomms/fcac231] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Revised: 06/30/2022] [Accepted: 09/13/2022] [Indexed: 08/27/2023] Open
Abstract
Early detection of Alzheimer's disease is required to identify patients suitable for disease-modifying medications and to improve access to non-pharmacological preventative interventions. Prior research shows detectable changes in speech in Alzheimer's dementia and its clinical precursors. The current study assesses whether a fully automated speech-based artificial intelligence system can detect cognitive impairment and amyloid beta positivity, which characterize early stages of Alzheimer's disease. Two hundred participants (age 54-85, mean 70.6; 114 female, 86 male) from sister studies in the UK (NCT04828122) and the USA (NCT04928976), completed the same assessments and were combined in the current analyses. Participants were recruited from prior clinical trials where amyloid beta status (97 amyloid positive, 103 amyloid negative, as established via PET or CSF test) and clinical diagnostic status was known (94 cognitively unimpaired, 106 with mild cognitive impairment or mild Alzheimer's disease). The automatic story recall task was administered during supervised in-person or telemedicine assessments, where participants were asked to recall stories immediately and after a brief delay. An artificial intelligence text-pair evaluation model produced vector-based outputs from the original story text and recorded and transcribed participant recalls, quantifying differences between them. Vector-based representations were fed into logistic regression models, trained with tournament leave-pair-out cross-validation analysis to predict amyloid beta status (primary endpoint), mild cognitive impairment and amyloid beta status in diagnostic subgroups (secondary endpoints). Predictions were assessed by the area under the receiver operating characteristic curve for the test result in comparison with reference standards (diagnostic and amyloid status). Simulation analysis evaluated two potential benefits of speech-based screening: (i) mild cognitive impairment screening in primary care compared with the Mini-Mental State Exam, and (ii) pre-screening prior to PET scanning when identifying an amyloid positive sample. Speech-based screening predicted amyloid beta positivity (area under the curve = 0.77) and mild cognitive impairment or mild Alzheimer's disease (area under the curve = 0.83) in the full sample, and predicted amyloid beta in subsamples (mild cognitive impairment or mild Alzheimer's disease: area under the curve = 0.82; cognitively unimpaired: area under the curve = 0.71). Simulation analyses indicated that in primary care, speech-based screening could modestly improve detection of mild cognitive impairment (+8.5%), while reducing false positives (-59.1%). Furthermore, speech-based amyloid pre-screening was estimated to reduce the number of PET scans required by 35.3% and 35.5% in individuals with mild cognitive impairment and cognitively unimpaired individuals, respectively. Speech-based assessment offers accessible and scalable screening for mild cognitive impairment and amyloid beta positivity.
Collapse
Affiliation(s)
| | | | | | | | | | - Kathryn V Papp
- Center for Alzheimer Research and Treatment, Department of Neurology, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, 02115, USA
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, 02114, USA
| | - Michael Ropacki
- Strategic Global Research & Development, Temecula, California, 94019, USA
| | | |
Collapse
|
13
|
Ilias L, Askounis D. Explainable Identification of Dementia from Transcripts using Transformer Networks. IEEE J Biomed Health Inform 2022; 26:4153-4164. [PMID: 35511841 DOI: 10.1109/jbhi.2022.3172479] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Alzheimers disease (AD) is the main cause of dementia which is accompanied by loss of memory and may lead to severe consequences in peoples everyday life if not diagnosed on time. Very few works have exploited transformer-based networks and despite the high accuracy achieved, little work has been done in terms of model interpretability. In addition, although Mini-Mental State Exam (MMSE) scores are inextricably linked with the identification of dementia, research works face the task of dementia identification and the task of the prediction of MMSE scores as two separate tasks. In order to address these limitations, we employ several transformer-based models, with BERT achieving the highest accuracy accounting for 87.50%. Concurrently, we propose an interpretable method to detect AD patients based on siamese networks reaching accuracy up to 83.75%. Next, we introduce two multi-task learning models, where the main task refers to the identification of dementia (binary classification), while the auxiliary one corresponds to the identification of the severity of dementia (multiclass classification). Our model obtains accuracy equal to 86.25% on the detection of AD patients in the multi-task learning setting. Finally, we present some new methods to identify the linguistic patterns used by AD patients and non-AD ones, including text statistics, vocabulary uniqueness, word usage, correlations via a detailed linguistic analysis, and explainability techniques (LIME). Findings indicate significant differences in language between AD and non-AD patients.
Collapse
|
14
|
Liu N, Luo K, Yuan Z, Chen Y. A Transfer Learning Method for Detecting Alzheimer's Disease Based on Speech and Natural Language Processing. Front Public Health 2022; 10:772592. [PMID: 35493375 PMCID: PMC9043451 DOI: 10.3389/fpubh.2022.772592] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Accepted: 02/24/2022] [Indexed: 11/13/2022] Open
Abstract
Alzheimer's disease (AD) is a neurodegenerative disease that is difficult to be detected using convenient and reliable methods. The language change in patients with AD is an important signal of their cognitive status, which potentially helps in early diagnosis. In this study, we developed a transfer learning model based on speech and natural language processing (NLP) technology for the early diagnosis of AD. The lack of large datasets limits the use of complex neural network models without feature engineering, while transfer learning can effectively solve this problem. The transfer learning model is firstly pre-trained on large text datasets to get the pre-trained language model, and then, based on such a model, an AD classification model is performed on small training sets. Concretely, a distilled bidirectional encoder representation (distilBert) embedding, combined with a logistic regression classifier, is used to distinguish AD from normal controls. The model experiment was evaluated on Alzheimer's dementia recognition through spontaneous speech datasets in 2020, including the balanced 78 healthy controls (HC) and 78 patients with AD. The accuracy of the proposed model is 0.88, which is almost equivalent to the champion score in the challenge and a considerable improvement over the baseline of 75% established by organizers of the challenge. As a result, the transfer learning method in this study improves AD prediction, which does not only reduces the need for feature engineering but also addresses the lack of sufficiently large datasets.
Collapse
Affiliation(s)
- Ning Liu
- School of Public Health, Hangzhou Normal University, Hangzhou, China
- Department of Mathematics and Computer Science, Fujian Provincial Key Laboratory of Data-Intensive Computing, Quanzhou Normal University, Quanzhou, China
| | - Kexue Luo
- Tongde Hospital of Zhejiang Province Geriatrics, Hangzhou, China
| | - Zhenming Yuan
- School of Information Science and Technology, Hangzhou Normal University, Hangzhou, China
| | - Yan Chen
- International Unresponsive Wakefulness Syndrome and Consciousness Science Institute, Hangzhou Normal University, Hangzhou, China
- *Correspondence: Yan Chen
| |
Collapse
|
15
|
Ilias L, Askounis D. Multimodal Deep Learning Models for Detecting Dementia From Speech and Transcripts. Front Aging Neurosci 2022; 14:830943. [PMID: 35370608 PMCID: PMC8969102 DOI: 10.3389/fnagi.2022.830943] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Accepted: 02/17/2022] [Indexed: 11/13/2022] Open
Abstract
Alzheimer's dementia (AD) entails negative psychological, social, and economic consequences not only for the patients but also for their families, relatives, and society in general. Despite the significance of this phenomenon and the importance for an early diagnosis, there are still limitations. Specifically, the main limitation is pertinent to the way the modalities of speech and transcripts are combined in a single neural network. Existing research works add/concatenate the image and text representations, employ majority voting approaches or average the predictions after training many textual and speech models separately. To address these limitations, in this article we present some new methods to detect AD patients and predict the Mini-Mental State Examination (MMSE) scores in an end-to-end trainable manner consisting of a combination of BERT, Vision Transformer, Co-Attention, Multimodal Shifting Gate, and a variant of the self-attention mechanism. Specifically, we convert audio to Log-Mel spectrograms, their delta, and delta-delta (acceleration values). First, we pass each transcript and image through a BERT model and Vision Transformer, respectively, adding a co-attention layer at the top, which generates image and word attention simultaneously. Secondly, we propose an architecture, which integrates multimodal information to a BERT model via a Multimodal Shifting Gate. Finally, we introduce an approach to capture both the inter- and intra-modal interactions by concatenating the textual and visual representations and utilizing a self-attention mechanism, which includes a gate model. Experiments conducted on the ADReSS Challenge dataset indicate that our introduced models demonstrate valuable advantages over existing research initiatives achieving competitive results in both the AD classification and MMSE regression task. Specifically, our best performing model attains an accuracy of 90.00% and a Root Mean Squared Error (RMSE) of 3.61 in the AD classification task and MMSE regression task, respectively, achieving a new state-of-the-art performance in the MMSE regression task.
Collapse
Affiliation(s)
- Loukas Ilias
- Decision Support Systems Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece
| | | |
Collapse
|
16
|
Zhu Y, Obyat A, Liang X, Batsis JA, Roth RM. WavBERT: Exploiting Semantic and Non-semantic Speech using Wav2vec and BERT for Dementia Detection. INTERSPEECH 2021; 2021:3790-3794. [PMID: 37063977 PMCID: PMC10102979 DOI: 10.21437/interspeech.2021-332] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/18/2023]
Abstract
In this paper, we exploit semantic and non-semantic information from patient's speech data using Wav2vec and Bidirectional Encoder Representations from Transformers (BERT) for dementia detection. We first propose a basic WavBERT model by extracting semantic information from speech data using Wav2vec, and analyzing the semantic information using BERT for dementia detection. While the basic model discards the non-semantic information, we propose extended WavBERT models that convert the output of Wav2vec to the input to BERT for preserving the non-semantic information in dementia detection. Specifically, we determine the locations and lengths of inter-word pauses using the number of blank tokens from Wav2vec where the threshold for setting the pauses is automatically generated via BERT. We further design a pre-trained embedding conversion network that converts the output embedding of Wav2vec to the input embedding of BERT, enabling the fine-tuning of WavBERT with non-semantic information. Our evaluation results using the ADReSSo dataset showed that the WavBERT models achieved the highest accuracy of 83.1% in the classification task, the lowest Root-Mean-Square Error (RMSE) score of 4.44 in the regression task, and a mean F1 of 70.91% in the progression task. We confirmed the effectiveness of WavBERT models exploiting both semantic and non-semantic speech.
Collapse
Affiliation(s)
- Youxiang Zhu
- Department of Computer Science, University of Massachusetts Boston, MA, USA
| | - Abdelrahman Obyat
- Department of Computer Science, University of Massachusetts Boston, MA, USA
| | - Xiaohui Liang
- Department of Computer Science, University of Massachusetts Boston, MA, USA
| | - John A Batsis
- School of Medicine, University of North Carolina, Chapel Hill, NC, USA
| | - Robert M Roth
- Geisel School of Medicine at Dartmouth, Lebanon, NH, USA
| |
Collapse
|