1
|
Oh C, Morris R, Wang X, Raskin MS. Analysis of emotional prosody as a tool for differential diagnosis of cognitive impairments: a pilot research. Front Psychol 2023; 14:1129406. [PMID: 37425151 PMCID: PMC10327638 DOI: 10.3389/fpsyg.2023.1129406] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Accepted: 05/26/2023] [Indexed: 07/11/2023] Open
Abstract
Introduction This pilot research was designed to investigate if prosodic features from running spontaneous speech could differentiate dementia of the Alzheimer's type (DAT), vascular dementia (VaD), mild cognitive impairment (MCI), and healthy cognition. The study included acoustic measurements of prosodic features (Study 1) and listeners' perception of emotional prosody differences (Study 2). Methods For Study 1, prerecorded speech samples describing the Cookie Theft picture from 10 individuals with DAT, 5 with VaD, 9 with MCI, and 10 neurologically healthy controls (NHC) were obtained from the DementiaBank. The descriptive narratives by each participant were separated into utterances. These utterances were measured on 22 acoustic features via the Praat software and analyzed statistically using the principal component analysis (PCA), regression, and Mahalanobis distance measures. Results The analyses on acoustic data revealed a set of five factors and four salient features (i.e., pitch, amplitude, rate, and syllable) that discriminate the four groups. For Study 2, a group of 28 listeners served as judges of emotions expressed by the speakers. After a set of training and practice sessions, they were instructed to indicate the emotions they heard. Regression measures were used to analyze the perceptual data. The perceptual data indicated that the factor underlying pitch measures had the greatest strength for the listeners to separate the groups. Discussion The present pilot work showed that using acoustic measures of prosodic features may be a functional method for differentiating among DAT, VaD, MCI, and NHC. Future studies with data collected under a controlled environment using better stimuli are warranted.
Collapse
Affiliation(s)
- Chorong Oh
- School of Rehabilitation and Communication Sciences, Ohio University, Athens, OH, United States
| | - Richard Morris
- School of Communication Science and Disorders, Florida State University, Tallahassee, FL, United States
| | - Xianhui Wang
- School of Medicine, University of California Irvine, Irvine, CA, United States
| | - Morgan S. Raskin
- School of Communication Science and Disorders, Florida State University, Tallahassee, FL, United States
| |
Collapse
|
2
|
Mavragani A, Kimura D, Kosugi A, Shinkawa K, Takase T, Kobayashi M, Yamada Y, Nemoto M, Watanabe R, Ota M, Higashi S, Nemoto K, Arai T, Nishimura M. Screening of Mild Cognitive Impairment Through Conversations With Humanoid Robots: Exploratory Pilot Study. JMIR Form Res 2023; 7:e42792. [PMID: 36637896 PMCID: PMC9883738 DOI: 10.2196/42792] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2022] [Revised: 11/23/2022] [Accepted: 12/01/2022] [Indexed: 12/04/2022] Open
Abstract
BACKGROUND The rising number of patients with dementia has become a serious social problem worldwide. To help detect dementia at an early stage, many studies have been conducted to detect signs of cognitive decline by prosodic and acoustic features. However, many of these methods are not suitable for everyday use as they focus on cognitive function or conversational speech during the examinations. In contrast, conversational humanoid robots are expected to be used in the care of older people to help reduce the work of care and monitoring through interaction. OBJECTIVE This study focuses on early detection of mild cognitive impairment (MCI) through conversations between patients and humanoid robots without a specific examination, such as neuropsychological examination. METHODS This was an exploratory study involving patients with MCI and cognitively normal (CN) older people. We collected the conversation data during neuropsychological examination (Mini-Mental State Examination [MMSE]) and everyday conversation between a humanoid robot and 94 participants (n=47, 50%, patients with MCI and n=47, 50%, CN older people). We extracted 17 types of prosodic and acoustic features, such as the duration of response time and jitter, from these conversations. We conducted a statistical significance test for each feature to clarify the speech features that are useful when classifying people into CN people and patients with MCI. Furthermore, we conducted an automatic classification experiment using a support vector machine (SVM) to verify whether it is possible to automatically classify these 2 groups by the features identified in the statistical significance test. RESULTS We obtained significant differences in 5 (29%) of 17 types of features obtained from the MMSE conversational speech. The duration of response time, the duration of silent periods, and the proportion of silent periods showed a significant difference (P<.001) and met the reference value r=0.1 (small) of the effect size. Additionally, filler periods (P<.01) and the proportion of fillers (P=.02) showed a significant difference; however, these did not meet the reference value of the effect size. In contrast, we obtained significant differences in 16 (94%) of 17 types of features obtained from the everyday conversations with the humanoid robot. The duration of response time, the duration of speech periods, jitter (local, relative average perturbation [rap], 5-point period perturbation quotient [ppq5], difference of difference of periods [ddp]), shimmer (local, amplitude perturbation quotient [apq]3, apq5, apq11, average absolute differences between the amplitudes of consecutive periods [dda]), and F0cov (coefficient of variation of the fundamental frequency) showed a significant difference (P<.001). In addition, the duration of response time, the duration of silent periods, the filler period, and the proportion of fillers showed significant differences (P<.05). However, only jitter (local) met the reference value r=0.1 (small) of the effect size. In the automatic classification experiment for the classification of participants into CN and MCI groups, the results showed 66.0% accuracy in the MMSE conversational speech and 68.1% accuracy in everyday conversations with the humanoid robot. CONCLUSIONS This study shows the possibility of early and simple screening for patients with MCI using prosodic and acoustic features from everyday conversations with a humanoid robot with the same level of accuracy as the MMSE.
Collapse
Affiliation(s)
| | | | | | | | - Toshiro Takase
- Healthcare and Life Science, IBM Consulting, IBM Japan, Ltd, Tokyo, Japan
| | | | | | - Miyuki Nemoto
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Ryohei Watanabe
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Miho Ota
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Shinji Higashi
- Department of Psychiatry, Ibaraki Medical Center, Tokyo Medical University, Ibaraki, Japan
| | - Kiyotaka Nemoto
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Tetsuaki Arai
- Department of Psychiatry, Division of Clinical Medicine, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan
| | - Masafumi Nishimura
- Department of Informatics, Graduate School of Intergraded Science and Technology, Shizuoka University, Hamamatsu, Japan
| |
Collapse
|
3
|
Changes in Speech Range Profile Are Associated with Cognitive Impairment. Dement Neurocogn Disord 2021; 20:89-98. [PMID: 34795772 PMCID: PMC8585535 DOI: 10.12779/dnd.2021.20.4.89] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2021] [Revised: 10/19/2021] [Accepted: 10/20/2021] [Indexed: 12/02/2022] Open
Abstract
Background and Purpose The aim of this study was to describe the variations in the speech range profile (SRP) of patients affected by cognitive decline. Methods We collected the data of patients managed for suspected voice and speech disorders, and suspected cognitive impairment. Patients underwent an Ear Nose and Throat evaluation and Mini-Mental State Examination (MMSE). To obtain SRP, we asked the patients to read 18 sentences twice, at their most comfortable pitch and loudness as they would do in daily conversation, and recorded their voice on to computer software. Results The study included 61 patients. The relationship between the MMSE score and SRP parameters was established. Increased severity of the MMSE score resulted in a statistically significant reduction in the average values of the semitones to the phonetogram, and the medium and maximum sound pressure levels (p<0.001). The maximum predictivity of MMSE was based on the highly significant values of semitones (p<0.001) and the maximum sound pressure levels (p=0.010). Conclusions The differences in SRP between the various groups were analyzed. Specifically, the SRP value decreased with increasing severity of cognitive decline. SRP was useful in highlighting the relationship between all cognitive declines tested and speech.
Collapse
|
4
|
Walker G, Morris LA, Christensen H, Mirheidari B, Reuber M, Blackburn DJ. Characterising spoken responses to an intelligent virtual agent by persons with mild cognitive impairment. CLINICAL LINGUISTICS & PHONETICS 2021; 35:237-252. [PMID: 32552087 DOI: 10.1080/02699206.2020.1777586] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/29/2020] [Revised: 05/27/2020] [Accepted: 05/31/2020] [Indexed: 06/11/2023]
Abstract
The diagnosis of Mild Cognitive Impairment (MCI) characterises patients at risk of dementia and may provide an opportunity for disease-modifying interventions. Identifying persons with MCI (PwMCI) from adults of a similar age without cognitive complaints is a significant challenge. The main aims of this study were to determine whether generic speech differences were evident between PwMCI and healthy controls (HC), whether such differences were identifiable in responses to recent or remote memory questions, and to determine which speech variables showed the clearest between-group differences. This study analysed recordings of 8 PwMCI (5 females, 3 males) and 14 HC of a similar age (8 females, 6 males). Participants were recorded interacting with an intelligent virtual agent: a computer-generated talking head on a computer screen which asks pre-recorded questions when prompted by the interviewee through pressing the next key on a computer keyboard. Responses to recent and remote memory questions were analysed. Mann-Whitney U tests were used to test for statistically significant differences between PwMCI and HC on each of 12 speech variables, relating to temporal characteristics, number of words produced and pitch. It was found that compared to HC, PwMCI produce speech for less time and in shorter chunks, they pause more often and for longer, take longer to begin speaking and produce fewer words in their answers. It was also found that the PwMCI and HC were more alike when responding to remote memory questions than when responding to recent memory questions. These findings show great promise and suggest that detailed speech analysis can make an important contribution to diagnostic and stratification systems in patients with memory complaints.
Collapse
Affiliation(s)
- Gareth Walker
- School of English, University of Sheffield , Sheffield, UK
| | - Lee-Anne Morris
- Sheffield Institute for Translational Neuroscience (SITraN), University of Sheffield , Sheffield, UK
| | - Heidi Christensen
- Department of Computer Science, University of Sheffield , Sheffield, UK
| | - Bahman Mirheidari
- Department of Computer Science, University of Sheffield , Sheffield, UK
| | - Markus Reuber
- Academic Neurology Unit, Royal Hallamshire Hospital, University of Sheffield , Sheffield, UK
| | - Daniel J Blackburn
- Sheffield Institute for Translational Neuroscience (SITraN), University of Sheffield , Sheffield, UK
| |
Collapse
|