Talk2Me: Automated linguistic data collection for personal assessment

A speech-based prognostic model for dysarthria progression in ALS

Objective: We demonstrated that it was possible to predict ALS patients' degree of future speech impairment based on past data. We used longitudinal data from two ALS studies where participants recorded their speech on a daily or weekly basis and provided ALSFRS-R speech subscores on a weekly or quarterly basis (quarter-annually). Methods: Using their speech recordings, we measured articulatory precision (a measure of the crispness of pronunciation) using an algorithm that analyzed the acoustic signal of each phoneme in the words produced. First, we established the analytical and clinical validity of the measure of articulatory precision, showing that the measure correlated with perceptual ratings of articulatory precision (r = .9). Second, using articulatory precision from speech samples from each participant collected over a 45-90 day model calibration period, we showed it was possible to predict articulatory precision 30-90 days after the last day of the model calibration period. Finally, we showed that the predicted articulatory precision scores mapped onto ALSFRS-R speech subscores. Results: the mean absolute error was as low as 4% for articulatory precision and 14% for ALSFRS-R speech subscores relative to the total range of their respective scales. Conclusion: Our results demonstrated that a subject-specific prognostic model for speech predicts future articulatory precision and ALSFRS-R speech values accurately.

Exploring the Use of Natural Language Processing for Objective Assessment of Disorganized Speech in Schizophrenia

Objective

Measurement-based care tools in psychiatry are useful for symptom monitoring and detecting response to treatment, but methods for quick and objective measurement are lacking especially for acute psychosis. The aim of this study was to explore potential language markers, detected by natural language processing (NLP) methods, as a means to objectively measure the severity of psychotic symptoms of schizophrenia in an acute clinical setting.

Methods

Twenty-two speech samples were collected from seven participants who were hospitalized for schizophrenia, and their symptoms were evaluated over time with SAPS/SANS and TLC scales. Linguistic features were extracted from the speech data using machine learning techniques. Spearman's correlation was performed to examine the relationship between linguistic features and symptoms. Various machine learning models were evaluated by cross-validation methods for their ability to predict symptom severity using the linguistic markers.

Results

Reduced lexical richness and syntactic complexity were characteristic of negative symptoms, while lower content density and more repetitions in speech were predictors of positive symptoms. Machine learning models predicted severity of alogia, illogicality, poverty of speech, social inattentiveness, and TLC scores with up to 82% accuracy. Additionally, speech incoherence was quantifiable through language markers derived from NLP methods.

Conclusions

These preliminary findings suggest that NLP may be useful in identifying clinically relevant language markers of schizophrenia, which can enhance objectivity in symptom monitoring during hospitalization. Further work is needed to replicate these findings in a larger data set and explore methods for feasible implementation in practice.

Performance of machine learning algorithms for dementia assessment: impacts of language tasks, recording media, and modalities

OBJECTIVES

Automatic speech and language assessment methods (SLAMs) can help clinicians assess speech and language impairments associated with dementia in older adults. The basis of any automatic SLAMs is a machine learning (ML) classifier that is trained on participants' speech and language. However, language tasks, recording media, and modalities impact the performance of ML classifiers. Thus, this research has focused on evaluating the effects of the above-mentioned factors on the performance of ML classifiers that can be used for dementia assessment.

METHODOLOGY

Our methodology includes the following steps: (1) Collecting speech and language datasets from patients and healthy controls; (2) Using feature engineering methods which include feature extraction methods to extract linguistic and acoustic features and feature selection methods to select most informative features; (3) Training different ML classifiers; and (4) Evaluating the performance of ML classifiers to investigate the impacts of language tasks, recording media, and modalities on dementia assessment.

RESULTS

Our results show that (1) the ML classifiers trained with the picture description language task perform better than the classifiers trained with the story recall language task; (2) the data obtained from phone-based recordings improves the performance of ML classifiers compared to data obtained from web-based recordings; and (3) the ML classifiers trained with acoustic features perform better than the classifiers trained with linguistic features.

CONCLUSION

This research demonstrates that we can improve the performance of automatic SLAMs as dementia assessment methods if we: (1) Use the picture description task to obtain participants' speech; (2) Collect participants' voices via phone-based recordings; and (3) Train ML classifiers using only acoustic features. Our proposed methodology will help future researchers to investigate the impacts of different factors on the performance of ML classifiers for assessing dementia.

Island Study Linking Aging and Neurodegenerative Disease (ISLAND) Targeting Dementia Risk Reduction: Protocol for a Prospective Web-Based Cohort Study

Background

Up to 40% of incident dementia is considered attributable to behavioral and lifestyle factors. Given the current lack of medical treatments and the projected increase in dementia prevalence, a focus on prevention through risk reduction is needed.

Objective

We aim to increase dementia risk knowledge and promote changes in dementia risk behaviors at individual and population levels.

Methods

The Island Study Linking Aging and Neurodegenerative Disease (ISLAND) is a long-term prospective, web-based cohort study with nested interventions that will be conducted over a 10-year period. Target participants (n=10,000) reside in Tasmania and are aged 50 years or over. Survey data on knowledge, attitudes, and behaviors related to modifiable dementia risk factors will be collected annually. After each survey wave, participants will be provided with a personalized dementia risk profile containing guidelines for reducing risk across 9 behavioral and lifestyle domains and with opportunities to engage in educational and behavioral interventions targeting risk reduction. Survey data will be modeled longitudinally with intervention engagement indices, cognitive function indices, and blood-based biomarkers, to measure change in risk over time.

Results

In the initial 12 months (October 2019 to October 2020), 6410 participants have provided baseline data. The study is ongoing.

Conclusions

Recruitment targets are feasible and efforts are ongoing to achieve a representative sample. Findings will inform future public health dementia risk reduction initiatives by showing whether, when, and how dementia risk can be lowered through educational and behavioral interventions, delivered in an uncontrolled real-world context.

International Registered Report Identifier (IRRID)

DERR1-10.2196/34688

Automated semantic relevance as an indicator of cognitive decline: Out-of-sample validation on a large-scale longitudinal dataset

We developed and evaluated an automatically extracted measure of cognition (semantic relevance) using automated and manual transcripts of audio recordings from healthy and cognitively impaired participants describing the Cookie Theft picture from the Boston Diagnostic Aphasia Examination. We describe the rationale and metric validation. We developed the measure on one dataset and evaluated it on a large database (>2000 samples) by comparing accuracy against a manually calculated metric and evaluating its clinical relevance. The fully automated measure was accurate (r = .84), had moderate to good reliability (intra-class correlation = .73), correlated with Mini-Mental State Examination and improved the fit in the context of other automatic language features (r = .65), and longitudinally declined with age and level of cognitive impairment. This study demonstrates the use of a rigorous analytical and clinical framework for validating automatic measures of speech, and applied it to a measure that is accurate and clinically relevant.

Automated analysis of propositional idea density in older adults

Previous research suggests oral and written language can act as barometers of an individual's cognitive function, potentially providing a screening tool for the earliest stages of Alzheimer's disease (AD) and other forms of dementia. Idea density is a measure of the rate at which ideas, or elementary predications, are expressed and may provide an ideal measure for early detection of deficits in language. Previous research has shown that when no restrictions are set on the topic of the idea, a decrease in propositional idea density (PID) is associated with an increased risk of developing AD. However, this has been limited by moderate sample sizes and manual transcribing. Technological advancement has enabled the automated calculation of PID from tools such as the Computerized Propositional Idea Density Rater (CPIDR). We delivered an online autobiographical writing task to older adult Australians from ISLAND (Island Study Linking Ageing and Neurodegenerative Disease). Linear regression models were fitted in R. We analysed text files (range 10-1180 words) using CPIDRv5 provided by 3316 (n = 853 males [25.7%], n = 2463 females [74.3%]) ISLAND participants. Over 358,957 words written in 3316 written autobiographical responses were analysed. Mean PID was higher in females (53.5 [±3.69]) than males (52.6 [±4.50]). Both advancing age and being male were significantly associated with a decrease in PID (p < .001). Automated methods of language analysis hold great promise for the early detection of subtle deficits in language capacity. Although our effect sizes were small, PID may be a sensitive measure of deficits in language in ageing individuals and is able to be collected at scale using online methods of data capture.

Detection of Mild Cognitive Impairment Through Natural Language and Touchscreen Typing Processing

Mild cognitive impairment (MCI), an identified prodromal stage of Alzheimer's Disease (AD), often evades detection in the early stages of the condition, when existing diagnostic methods are employed in the clinical setting. From an alternative perspective, smartphone interaction behavioral data, unobtrusively acquired in a non-clinical setting, can assist the screening and monitoring of MCI and its symptoms' progression. In this vein, the diagnostic ability of digital biomarkers, drawn from Fine Motor Impairment (FMI)- and Spontaneous Written Speech (SWS)-related data analysis, are examined here. In particular, keystroke dynamics derived from touchscreen typing activities, using Convolutional Neural Networks, along with linguistic features of SWS through Natural Language Processing (NLP), were used to distinguish amongst MCI patients and healthy controls (HC). Analytically, three indices of FMI (rigidity, bradykinesia and alternate finger tapping) and nine NLP features, related with lexical richness, grammatical, syntactical complexity, and word deficits, formed the feature space. The proposed approach was tested on two demographically matched groups of 11 MCI patients and 12 HC, having undergone the same neuropsychological tests, producing 4,930 typing sessions and 78 short texts, within 6 months, for analysis. A cascaded-classifier scheme was realized under three different feature combinations and validated via a Leave-One-Subject-Out cross-validation scheme. The acquired results have shown: (a) keystroke features with a k-NN classifier achieved an Area Under Curve (AUC) of 0.78 [95% confidence interval (CI):0.68-0.88; specificity/sensitivity (SP/SE): 0.64/0.92], (b) NLP features with a Logistic regression classifier achieved an AUC of 0.76 (95% CI: 0.65-0.85; SP/SE: 0.80/0.71), and (c) an ensemble model with the fusion of keystroke and NLP features resulted in AUC of 0.75 (95% CI:0.63-0.86; SP/SE 0.90/0.60). The current findings indicate the potentiality of new digital biomarkers to capture early stages of cognitive decline, providing a highly specific remote screening tool in-the-wild.

Multimodal Capture of Patient Behaviour for Improved Detection of Early Dementia: Clinical Feasibility and Preliminary Results

Non-invasive automatic screening for Alzheimer’s disease has the potential to improve diagnostic accuracy while lowering healthcare costs. Previous research has shown that patterns in speech, language, gaze, and drawing can help detect early signs of cognitive decline. In this paper, we describe a highly multimodal system for unobtrusively capturing data during real clinical interviews conducted as part of cognitive assessments for Alzheimer’s disease. The system uses nine different sensor devices (smartphones, a tablet, an eye tracker, a microphone array, and a wristband) to record interaction data during a specialist’s first clinical interview with a patient, and is currently in use at Karolinska University Hospital in Stockholm, Sweden. Furthermore, complementary information in the form of brain imaging, psychological tests, speech therapist assessment, and clinical meta-data is also available for each patient. We detail our data-collection and analysis procedure and present preliminary findings that relate measures extracted from the multimodal recordings to clinical assessments and established biomarkers, based on data from 25 patients gathered thus far. Our findings demonstrate feasibility for our proposed methodology and indicate that the collected data can be used to improve clinical assessments of early dementia.

Words describing feelings about death: A comparison of sentiment for self and others and changes over time

Understanding public attitudes towards death is needed to inform health policies to foster community death awareness and preparedness. Linguistic sentiment analysis of how people describe their feelings about death can add to knowledge gained from traditional self-reports. This study provided the first description of emotive attitudes expressed towards death utilising textual sentiment analysis for the dimensions of valence, arousal and dominance. A linguistic lexicon of sentiment norms was applied to activities conducted in an online course for the general-public designed to generate discussion about death. We analysed the sentiment of words people chose to describe feelings about death, for themselves, for perceptions of the feelings of ‘others’, and for longitudinal changes over the time-period of exposure to a course about death (n = 1491). The results demonstrated that sadness pervades affective responses to death, and that inevitability, peace, and fear were also frequent reactions. However, words chosen to represent perceptions of others’ feelings towards death suggested that participants perceived others as feeling more negative about death than they do themselves. Analysis of valence, arousal and dominance dimensions of sentiment pre-to-post course participation demonstrated that participants chose significantly happier (more positive) valence words, less arousing (calmer) words, and more dominant (in-control) words to express their feelings about death by the course end. This suggests that the course may have been helpful in participants becoming more emotionally accepting in their feelings and attitude towards death. Furthermore, the change over time appeared greater for younger participants, who showed more increase in the dominance (power/control) and pleasantness (valence) in words chosen at course completion. Sentiment analysis of words to describe death usefully extended our understanding of community death attitudes and emotions. Future application of sentiment analysis to other related areas of health policy interest such as attitudes towards Advance Care Planning and palliative care may prove fruitful.

Repeatability of Commonly Used Speech and Language Features for Clinical Applications

Introduction

Changes in speech have the potential to provide important information on the diagnosis and progression of various neurological diseases. Many researchers have relied on open-source speech features to develop algorithms for measuring speech changes in clinical populations as they are convenient and easy to use. However, the repeatability of open-source features in the context of neurological diseases has not been studied.

Methods

We used a longitudinal sample of healthy controls, individuals with amyotrophic lateral sclerosis, and individuals with suspected frontotemporal dementia, and we evaluated the repeatability of acoustic and language features separately on these 3 data sets.

Results

Repeatability was evaluated using intraclass correlation (ICC) and the within-subjects coefficient of variation (WSCV). In 3 sets of tasks, the median ICC were between 0.02 and 0.55, and the median WSCV were between 29 and 79%.

Conclusion

Our results demonstrate that the repeatability of speech features extracted using open-source tool kits is low. Researchers should exercise caution when developing digital health models with open-source speech features. We provide a detailed summary of feature-by-feature repeatability results (ICC, WSCV, SE of measurement, limits of agreement for WSCV, and minimal detectable change) in the online supplementary material so that researchers may incorporate repeatability information into the models they develop.

A Conversational Robot for Older Adults with Alzheimer’s Disease

Amid the rising cost of Alzheimer’s disease (AD), assistive health technologies can reduce care-giving burden by aiding in assessment, monitoring, and therapy. This article presents a pilot study testing the feasibility and effect of a conversational robot in a cognitive assessment task with older adults with AD. We examine the robot interactions through dialogue and miscommunication analysis, linguistic feature analysis, and the use of a qualitative analysis, in which we report key themes that were prevalent throughout the study. While conversations were typically better with human conversation partners (being longer, with greater engagement and less misunderstanding), we found that the robot was generally well liked by participants and that it was able to capture their interest in dialogue. Miscommunication due to issues of understanding and intelligibility did not seem to deter participants from their experience. Furthermore, in automatically extracting linguistic features, we examine how non-acoustic aspects of language change across participants with varying degrees of cognitive impairment, highlighting the robot’s potential as a monitoring tool. This pilot study is an exploration of how conversational robots can be used to support individuals with AD.