Identification of digital voice biomarkers for cognitive health

Exploring explainable AI features in the vocal biomarkers of lung disease

This review delves into the burgeoning field of explainable artificial intelligence (XAI) in the detection and analysis of lung diseases through vocal biomarkers. Lung diseases, often elusive in their early stages, pose a significant public health challenge. Recent advancements in AI have ushered in innovative methods for early detection, yet the black-box nature of many AI models limits their clinical applicability. XAI emerges as a pivotal tool, enhancing transparency and interpretability in AI-driven diagnostics. This review synthesizes current research on the application of XAI in analyzing vocal biomarkers for lung diseases, highlighting how these techniques elucidate the connections between specific vocal features and lung pathology. We critically examine the methodologies employed, the types of lung diseases studied, and the performance of various XAI models. The potential for XAI to aid in early detection, monitor disease progression, and personalize treatment strategies in pulmonary medicine is emphasized. Furthermore, this review identifies current challenges, including data heterogeneity and model generalizability, and proposes future directions for research. By offering a comprehensive analysis of explainable AI features in the context of lung disease detection, this review aims to bridge the gap between advanced computational approaches and clinical practice, paving the way for more transparent, reliable, and effective diagnostic tools.

Prediction of Alzheimer's disease progression within 6 years using speech: A novel approach leveraging language models

INTRODUCTION

Identification of individuals with mild cognitive impairment (MCI) who are at risk of developing Alzheimer's disease (AD) is crucial for early intervention and selection of clinical trials.

METHODS

We applied natural language processing techniques along with machine learning methods to develop a method for automated prediction of progression to AD within 6 years using speech. The study design was evaluated on the neuropsychological test interviews of n = 166 participants from the Framingham Heart Study, comprising 90 progressive MCI and 76 stable MCI cases.

RESULTS

Our best models, which used features generated from speech data, as well as age, sex, and education level, achieved an accuracy of 78.5% and a sensitivity of 81.1% to predict MCI-to-AD progression within 6 years.

DISCUSSION

The proposed method offers a fully automated procedure, providing an opportunity to develop an inexpensive, broadly accessible, and easy-to-administer screening tool for MCI-to-AD progression prediction, facilitating development of remote assessment.

HIGHLIGHTS

Voice recordings from neuropsychological exams coupled with basic demographics can lead to strong predictive models of progression to dementia from mild cognitive impairment. The study leveraged AI methods for speech recognition and processed the resulting text using language models. The developed AI-powered pipeline can lead to fully automated assessment that could enable remote and cost-effective screening and prognosis for Alzehimer's disease.

Digital Neuropsychology beyond Computerized Cognitive Assessment: Applications of Novel Digital Technologies

Compared with other health disciplines, there is a stagnation in technological innovation in the field of clinical neuropsychology. Traditional paper-and-pencil tests have a number of shortcomings, such as low-frequency data collection and limitations in ecological validity. While computerized cognitive assessment may help overcome some of these issues, current computerized paradigms do not address the majority of these limitations. In this paper, we review recent literature on the applications of novel digital health approaches, including ecological momentary assessment, smartphone-based assessment and sensors, wearable devices, passive driving sensors, smart homes, voice biomarkers, and electronic health record mining, in neurological populations. We describe how each digital tool may be applied to neurologic care and overcome limitations of traditional neuropsychological assessment. Ethical considerations, limitations of current research, as well as our proposed future of neuropsychological practice are also discussed.

Using voice biomarkers for frailty classification

Clinicians use the patient's voice intuitively to evaluate general health and frailty. Voice is an emerging health indicator but has been scarcely studied in the context of frailty. This study explored voice parameters as possible predictors of frailty in older adults. Fifty-three participants over 70 years old were recruited from rehabilitation wards at a tertiary medical center. Participants' frailty was assessed using Rockwood frailty index and they were classified as most-frail (n = 33, 68%) or less-frail (n = 20, 32%). Participants were recorded counting from 1 to 10 and backwards using a smartphone recording application. The following voice biomarkers were derived: peak and average volume, peak/average volume ratio, pauses' total length, and pause length standard deviation. The most-frail group had a higher peak volume/average volume ratio (p = 0.03) and greater variance in lengths of pauses between speech segments (p = 0.002). These parameters indicate greater speech irregularity in the most-frail, compared to the less-frail. The most-frail group also had a longer total duration of pauses (p = 0.02). No statistically significant difference was found in peak and average volume (p = 0.75 and 0.39). Most-frail participants' speech had different characteristics, compared to participants in the less-frail group. This is a first step to developing an AI-based frailty assessment tool that can assist in identifying our most vulnerable patients.

Clinical classification of memory and cognitive impairment with multimodal digital biomarkers

INTRODUCTION

Early detection of Alzheimer's disease and cognitive impairment is critical to improving the healthcare trajectories of aging adults, enabling early intervention and potential prevention of decline.

METHODS

To evaluate multi-modal feature sets for assessing memory and cognitive impairment, feature selection and subsequent logistic regressions were used to identify the most salient features in classifying Rey Auditory Verbal Learning Test-determined memory impairment.

RESULTS

Multimodal models incorporating graphomotor, memory, and speech and voice features provided the stronger classification performance (area under the curve = 0.83; sensitivity = 0.81, specificity = 0.80). Multimodal models were superior to all other single modality and demographics models.

DISCUSSION

The current research contributes to the prevailing multimodal profile of those with cognitive impairment, suggesting that it is associated with slower speech with a particular effect on the duration, frequency, and percentage of pauses compared to normal healthy speech.

Exploring cognitive progression subtypes in the Framingham Heart Study

INTRODUCTION

Alzheimer's disease (AD) is a heterogeneous disorder characterized by complex underlying neuropathology that is not fully understood. This study aimed to identify cognitive progression subtypes and examine their correlation with clinical outcomes.

METHODS

Participants of this study were recruited from the Framingham Heart Study. The Subtype and Stage Inference (SuStaIn) method was used to identify cognitive progression subtypes based on eight cognitive domains.

RESULTS

Three cognitive progression subtypes were identified, including verbal learning (Subtype 1), abstract reasoning (Subtype 2), and visual memory (Subtype 3). These subtypes represent different domains of cognitive decline during the progression of AD. Significant differences in age of onset among the different subtypes were also observed. A higher SuStaIn stage was significantly associated with increased mortality risk.

DISCUSSION

This study provides a characterization of AD heterogeneity in cognitive progression, emphasizing the importance of developing personalized approaches for risk stratification and intervention.

Highlights

We used the Subtype and Stage Inference (SuStaIn) method to identify three cognitive progression subtypes.Different subtypes have significant variations in age of onset.Higher stages of progression are associated with increased mortality risk.

Stakeholder perspectives on ethical and trustworthy voice AI in health care

Objective

Voice as a health biomarker using artificial intelligence (AI) is gaining momentum in research. The noninvasiveness of voice data collection through accessible technology (such as smartphones, telehealth, and ambient recordings) or within clinical contexts means voice AI may help address health disparities and promote the inclusion of marginalized communities. However, the development of AI-ready voice datasets free from bias and discrimination is a complex task. The objective of this study is to better understand the perspectives of engaged and interested stakeholders regarding ethical and trustworthy voice AI, to inform both further ethical inquiry and technology innovation.

Methods

A questionnaire was administered to voice AI experts, clinicians, scholars, patients, trainees, and policy-makers who participated at the 2023 Voice AI Symposium organized by the Bridge2AI-Voice AI Consortium. The survey used a mix of Likert scale, ranking and open-ended questions. A total of 27 stakeholders participated in the study.

Results

The main results of the study are the identification of priorities in terms of ethical issues, an initial definition of ethically sourced data for voice AI, insights into the use of synthetic voice data, and proposals for acting on the trustworthiness of voice AI. The study shows a diversity of perspectives and adds nuance to the planning and development of ethical and trustworthy voice AI.

Conclusions

This study represents the first stakeholder survey related to voice as a biomarker of health published to date. This study sheds light on the critical importance of ethics and trustworthiness in the development of voice AI technologies for health applications.

Voice analysis results in individuals with Alzheimer's disease: How do age and cognitive status affect voice parameters?

BACKGROUND

Reports of acoustic changes in the voice in individuals with Alzheimer's disease (AD) and the relationship of acoustic changes with age and cognitive status are still limited.

OBJECTIVE

This study aims to determine the changes in voice analysis results in AD, as well as the effects of age and cognitive status on voice parameters.

METHODS

The study included 47 (AD: 30; healthy: 17) women with a mean age of 76.13 years. The acoustic voice parameters mean fundamental frequency (F0), relative average perturbation (RAP), jitter percent (Jitt), shimmer percent (Shim), and noise-to-harmonic ratio were detected. The mini-mental state examination (MMSE) was utilized.

RESULTS

F0, Shim, Jitt, and RAP values were found to be statistically significantly higher in individuals with AD compared to healthy individuals. There was a significant negative correlation between MMSE and F0, Jitt, RAP and Shim, and the MMSE score had a significant negative effect on F0, Jitt, and RAP (p < .05).

CONCLUSION

Cognitive status was discovered to significantly impact the voice, with fundamental frequency and frequency and amplitude perturbations increasing as cognitive level decreases. In order to contribute to the therapy process for voice disorders, cognitive functions can be focused on in addition to voice therapy.

Applied Machine Learning Techniques to Diagnose Voice-Affecting Conditions and Disorders: Systematic Literature Review

BACKGROUND

Normal voice production depends on the synchronized cooperation of multiple physiological systems, which makes the voice sensitive to changes. Any systematic, neurological, and aerodigestive distortion is prone to affect voice production through reduced cognitive, pulmonary, and muscular functionality. This sensitivity inspired using voice as a biomarker to examine disorders that affect the voice. Technological improvements and emerging machine learning (ML) technologies have enabled possibilities of extracting digital vocal features from the voice for automated diagnosis and monitoring systems.

OBJECTIVE

This study aims to summarize a comprehensive view of research on voice-affecting disorders that uses ML techniques for diagnosis and monitoring through voice samples where systematic conditions, nonlaryngeal aerodigestive disorders, and neurological disorders are specifically of interest.

METHODS

This systematic literature review (SLR) investigated the state of the art of voice-based diagnostic and monitoring systems with ML technologies, targeting voice-affecting disorders without direct relation to the voice box from the point of view of applied health technology. Through a comprehensive search string, studies published from 2012 to 2022 from the databases Scopus, PubMed, and Web of Science were scanned and collected for assessment. To minimize bias, retrieval of the relevant references in other studies in the field was ensured, and 2 authors assessed the collected studies. Low-quality studies were removed through a quality assessment and relevant data were extracted through summary tables for analysis. The articles were checked for similarities between author groups to prevent cumulative redundancy bias during the screening process, where only 1 article was included from the same author group.

RESULTS

In the analysis of the 145 included studies, support vector machines were the most utilized ML technique (51/145, 35.2%), with the most studied disease being Parkinson disease (PD; reported in 87/145, 60%, studies). After 2017, 16 additional voice-affecting disorders were examined, in contrast to the 3 investigated previously. Furthermore, an upsurge in the use of artificial neural network-based architectures was observed after 2017. Almost half of the included studies were published in last 2 years (2021 and 2022). A broad interest from many countries was observed. Notably, nearly one-half (n=75) of the studies relied on 10 distinct data sets, and 11/145 (7.6%) used demographic data as an input for ML models.

CONCLUSIONS

This SLR revealed considerable interest across multiple countries in using ML techniques for diagnosing and monitoring voice-affecting disorders, with PD being the most studied disorder. However, the review identified several gaps, including limited and unbalanced data set usage in studies, and a focus on diagnostic test rather than disorder-specific monitoring. Despite the limitations of being constrained by only peer-reviewed publications written in English, the SLR provides valuable insights into the current state of research on ML-based voice-affecting disorder diagnosis and monitoring and highlighting areas to address in future research.

Automated detection of mild cognitive impairment and dementia from voice recordings: A natural language processing approach

INTRODUCTION

Automated computational assessment of neuropsychological tests would enable widespread, cost-effective screening for dementia.

METHODS

A novel natural language processing approach is developed and validated to identify different stages of dementia based on automated transcription of digital voice recordings of subjects' neuropsychological tests conducted by the Framingham Heart Study (n = 1084). Transcribed sentences from the test were encoded into quantitative data and several models were trained and tested using these data and the participants' demographic characteristics.

RESULTS

Average area under the curve (AUC) on the held-out test data reached 92.6%, 88.0%, and 74.4% for differentiating Normal cognition from Dementia, Normal or Mild Cognitive Impairment (MCI) from Dementia, and Normal from MCI, respectively.

DISCUSSION

The proposed approach offers a fully automated identification of MCI and dementia based on a recorded neuropsychological test, providing an opportunity to develop a remote screening tool that could be adapted easily to any language.

Multimodal Machine Learning for 10-Year Dementia Risk Prediction: The Framingham Heart Study

BACKGROUND

Early prediction of dementia risk is crucial for effective interventions. Given the known etiologic heterogeneity, machine learning methods leveraging multimodal data, such as clinical manifestations, neuroimaging biomarkers, and well-documented risk factors, could predict dementia more accurately than single modal data.

OBJECTIVE

This study aims to develop machine learning models that capitalize on neuropsychological (NP) tests, magnetic resonance imaging (MRI) measures, and clinical risk factors for 10-year dementia prediction.

METHODS

This study included participants from the Framingham Heart Study, and various data modalities such as NP tests, MRI measures, and demographic variables were collected. CatBoost was used with Optuna hyperparameter optimization to create prediction models for 10-year dementia risk using different combinations of data modalities. The contribution of each modality and feature for the prediction task was also quantified using Shapley values.

RESULTS

This study included 1,031 participants with normal cognitive status at baseline (age 75±5 years, 55.3% women), of whom 205 were diagnosed with dementia during the 10-year follow-up. The model built on three modalities demonstrated the best dementia prediction performance (AUC 0.90±0.01) compared to single modality models (AUC range: 0.82-0.84). MRI measures contributed most to dementia prediction (mean absolute Shapley value: 3.19), suggesting the necessity of multimodal inputs.

CONCLUSION

This study shows that a multimodal machine learning framework had a superior performance for 10-year dementia risk prediction. The model can be used to increase vigilance for cognitive deterioration and select high-risk individuals for early intervention and risk management.

Artificial Intelligence-Enabled End-To-End Detection and Assessment of Alzheimer's Disease Using Voice

There is currently no simple, widely available screening method for Alzheimer's disease (AD), partly because the diagnosis of AD is complex and typically involves expensive and sometimes invasive tests not commonly available outside highly specialized clinical settings. Here, we developed an artificial intelligence (AI)-powered end-to-end system to detect AD and predict its severity directly from voice recordings. At the core of our system is the pre-trained data2vec model, the first high-performance self-supervised algorithm that works for speech, vision, and text. Our model was internally evaluated on the ADReSSo (Alzheimer's Dementia Recognition through Spontaneous Speech only) dataset containing voice recordings of subjects describing the Cookie Theft picture, and externally validated on a test dataset from DementiaBank. The AI model can detect AD with average area under the curve (AUC) of 0.846 and 0.835 on held-out and external test set, respectively. The model was well-calibrated (Hosmer-Lemeshow goodness-of-fit p-value = 0.9616). Moreover, the model can reliably predict the subject's cognitive testing score solely based on raw voice recordings. Our study demonstrates the feasibility of using the AI-powered end-to-end model for early AD diagnosis and severity prediction directly based on voice, showing its potential for screening Alzheimer's disease in a community setting.

Association Between Acoustic Features and Neuropsychological Test Performance in the Framingham Heart Study: Observational Study

BACKGROUND

Human voice has increasingly been recognized as an effective indicator for the detection of cognitive disorders. However, the association of acoustic features with specific cognitive functions and mild cognitive impairment (MCI) has yet to be evaluated in a large community-based population.

OBJECTIVE

This study aimed to investigate the association between acoustic features and neuropsychological (NP) tests across multiple cognitive domains and evaluate the added predictive power of acoustic composite scores for the classification of MCI.

METHODS

This study included participants without dementia from the Framingham Heart Study, a large community-based cohort with longitudinal surveillance for incident dementia. For each participant, 65 low-level acoustic descriptors were derived from voice recordings of NP test administration. The associations between individual acoustic descriptors and 18 NP tests were assessed with linear mixed-effect models adjusted for age, sex, and education. Acoustic composite scores were then built by combining acoustic features significantly associated with NP tests. The added prediction power of acoustic composite scores for prevalent and incident MCI was also evaluated.

RESULTS

The study included 7874 voice recordings from 4950 participants (age: mean 62, SD 14 years; 4336/7874, 55.07% women), of whom 453 were diagnosed with MCI. In all, 8 NP tests were associated with more than 15 acoustic features after adjusting for multiple testing. Additionally, 4 of the acoustic composite scores were significantly associated with prevalent MCI and 7 were associated with incident MCI. The acoustic composite scores can increase the area under the curve of the baseline model for MCI prediction from 0.712 to 0.755.

CONCLUSIONS

Multiple acoustic features are significantly associated with NP test performance and MCI, which can potentially be used as digital biomarkers for early cognitive impairment monitoring.

Voice biomarkers as indicators of cognitive changes in middle and later adulthood

Voice prosody measures have been linked with Alzheimer's disease (AD), but it is unclear whether they are associated with normal cognitive aging. We assessed relationships between voice measures and 10-year cognitive changes in the MIDUS national sample of middle-aged and older adults ages 42-92, with a mean age of 64.09 (standard deviation = 11.23) at the second wave. Seven cognitive tests were assessed in 2003-2004 (Wave 2) and 2013-2014 (Wave 3). Voice measures were collected at Wave 3 (N = 2585) from audio recordings of the cognitive interviews. Analyses controlled for age, education, depressive symptoms, and health. As predicted, higher jitter was associated with greater declines in episodic memory, verbal fluency, and attention switching. Lower pulse was related to greater decline in episodic memory, and fewer voice breaks were related to greater declines in episodic memory and verbal fluency, although the direction of these effects was contrary to hypotheses. Findings suggest that voice biomarkers may offer a promising approach for early detection of risk factors for cognitive impairment or AD.

Validation of the Remote Automated ki:e Speech Biomarker for Cognition in Mild Cognitive Impairment: Verification and Validation following DiME V3 Framework

INTRODUCTION

Progressive cognitive decline is the cardinal behavioral symptom in most dementia-causing diseases such as Alzheimer's disease. While most well-established measures for cognition might not fit tomorrow's decentralized remote clinical trials, digital cognitive assessments will gain importance. We present the evaluation of a novel digital speech biomarker for cognition (SB-C) following the Digital Medicine Society's V3 framework: verification, analytical validation, and clinical validation.

METHODS

Evaluation was done in two independent clinical samples: the Dutch DeepSpA (N = 69 subjective cognitive impairment [SCI], N = 52 mild cognitive impairment [MCI], and N = 13 dementia) and the Scottish SPeAk datasets (N = 25, healthy controls). For validation, two anchor scores were used: the Mini-Mental State Examination (MMSE) and the Clinical Dementia Rating (CDR) scale.

RESULTS

Verification: The SB-C could be reliably extracted for both languages using an automatic speech processing pipeline. Analytical Validation: In both languages, the SB-C was strongly correlated with MMSE scores. Clinical Validation: The SB-C significantly differed between clinical groups (including MCI and dementia), was strongly correlated with the CDR, and could track the clinically meaningful decline.

CONCLUSION

Our results suggest that the ki:e SB-C is an objective, scalable, and reliable indicator of cognitive decline, fit for purpose as a remote assessment in clinical early dementia trials.

The Boston Process Approach and Digital Neuropsychological Assessment: Past Research and Future Directions

Neuropsychological assessment using the Boston Process Approach (BPA) suggests that an analysis of the strategy or the process by which tasks and neuropsychological tests are completed, and the errors made during test completion convey much information regarding underlying brain and cognition and are as important as overall summary scores. Research over the last several decades employing an analysis of process and errors has been able to dissociate between dementia patients diagnosed with Alzheimer's disease, vascular dementia associated with MRI-determined white matter alterations, and Parkinson's disease; and between mild cognitive impairment subtypes. Nonetheless, BPA methods can be labor intensive to deploy. However, the recent availability of digital platforms for neuropsychological test administration and scoring now enables reliable, rapid, and objective data collection. Further, digital technology can quantify highly nuanced data previously unobtainable to define neurocognitive constructs with high accuracy. In this paper, a brief review of the BPA is provided. Studies that demonstrate how digital technology translates BPA into specific neurocognitive constructs using the Clock Drawing Test, Backward Digit Span Test, and a Digital Pointing Span Test are described. Implications for using data driven artificial intelligence-supported analytic approaches enabling the creation of more sensitive and specific detection/diagnostic algorithms for putative neurodegenerative illness are also discussed.

Using Digital Tools to Advance Alzheimer's Drug Trials During a Pandemic: The EU/US CTAD Task Force

The 2020 COVID-19 pandemic has disrupted Alzheimer’s disease (AD) clinical studies worldwide. Digital technologies may help minimize disruptions by enabling remote assessment of subtle cognitive and functional changes over the course of the disease. The EU/US Clinical Trials in Alzheimer’s Disease (CTAD) Task Force met virtually in November 2020 to explore the opportunities and challenges associated with the use of digital technologies in AD clinical research. While recognizing the potential of digital tools to accelerate clinical trials, improve the engagement of diverse populations, capture clinically meaningful data, and lower costs, questions remain regarding the stability, validity, generalizability, and reproducibility of digital data. Substantial concerns also exist regarding regulatory acceptance and privacy. Nonetheless, the Task Force supported further exploration of digital technologies through collaboration and data sharing, noting the need for standardization of digital readouts. They also concluded that while it may be premature to employ remote assessments for trials of novel experimental medications, remote studies of non-invasive, multi-domain approaches may be feasible at this time.

Neuropsychological test validation of speech markers of cognitive impairment in the Framingham Cognitive Aging Cohort

AIM

Although clinicians primarily diagnose dementia based on a combination of metrics such as medical history and formal neuropsychological tests, recent work using linguistic analysis of narrative speech to identify dementia has shown promising results. We aim to build upon research by Thomas JA & Burkardt HA et al. (J Alzheimers Dis. 2020;76:905-22) and Alhanai et al. (arXiv:1710.07551v1. 2020) on the Framingham Heart Study (FHS) Cognitive Aging Cohort by 1) demonstrating the predictive capability of linguistic analysis in differentiating cognitively normal from cognitively impaired participants and 2) comparing the performance of the original linguistic features with the performance of expanded features.

METHODS

Data were derived from a subset of the FHS Cognitive Aging Cohort. We analyzed a sub-selection of 98 participants, which provided 127 unique audio files and clinical observations (n = 127, female = 47%, cognitively impaired = 43%). We built on previous work which extracted original linguistic features from transcribed audio files by extracting expanded features. We used both feature sets to train logistic regression classifiers to distinguish cognitively normal from cognitively impaired participants and compared the predictive power of the original and expanded linguistic feature sets, and participants' Mini-Mental State Examination (MMSE) scores.

RESULTS

Based on the area under the receiver-operator characteristic curve (AUC) of the models, both the original (AUC = 0.882) and expanded (AUC = 0.883) feature sets outperformed MMSE (AUC = 0.870) in classifying cognitively impaired and cognitively normal participants. Although the original and expanded feature sets had similar AUC, the expanded feature set showed better positive and negative predictive value [expanded: positive predictive value (PPV) = 0.738, negative predictive value (NPV) = 0.889; original: PPV = 0.701, NPV = 0.869].

CONCLUSIONS

Linguistic analysis has been shown to be a potentially powerful tool for clinical use in classifying cognitive impairment. This study expands the work of several others, but further studies into the plausibility of speech analysis in clinical use are vital to ensure the validity of speech analysis for clinical classification of cognitive impairment.