Detection of clinical depression in adolescents' speech during family interactions

Deconstructing demographic bias in speech-based machine learning models for digital health

Introduction

Machine learning (ML) algorithms have been heralded as promising solutions to the realization of assistive systems in digital healthcare, due to their ability to detect fine-grain patterns that are not easily perceived by humans. Yet, ML algorithms have also been critiqued for treating individuals differently based on their demography, thus propagating existing disparities. This paper explores gender and race bias in speech-based ML algorithms that detect behavioral and mental health outcomes.

Methods

This paper examines potential sources of bias in the data used to train the ML, encompassing acoustic features extracted from speech signals and associated labels, as well as in the ML decisions. The paper further examines approaches to reduce existing bias via using the features that are the least informative of one's demographic information as the ML input, and transforming the feature space in an adversarial manner to diminish the evidence of the demographic information while retaining information about the focal behavioral and mental health state.

Results

Results are presented in two domains, the first pertaining to gender and race bias when estimating levels of anxiety, and the second pertaining to gender bias in detecting depression. Findings indicate the presence of statistically significant differences in both acoustic features and labels among demographic groups, as well as differential ML performance among groups. The statistically significant differences present in the label space are partially preserved in the ML decisions. Although variations in ML performance across demographic groups were noted, results are mixed regarding the models' ability to accurately estimate healthcare outcomes for the sensitive groups.

Discussion

These findings underscore the necessity for careful and thoughtful design in developing ML models that are capable of maintaining crucial aspects of the data and perform effectively across all populations in digital healthcare applications.

Effects of Age-Dependent Hormonal Changes and Estrogen Supplementation on Voice in Girls with Anorexia Nervosa-Preliminary Report

INTRODUCTION

Human development includes lots of physical and emotional changes. The human voice depends on age. Voice production is a complex physiological and acoustic phenomenon that depends on many factors such as structure, hormone level, degree of fatigue or nutrition and hydration of the body, systemic diseases, and emotional state. All these factors can be present in anorexia nervosa (AN), such as excessive weight loss, generated hydro-electrolytic changes, nutritional deficiencies, hormonal disturbances in the function of the hypothalamic-pituitary-adrenal axis, the hypothalamic-pituitary-thyroid axis, the hypothalamic-pituitary-ovarian axis, and emotional distress. The prevalence of AN ranges between 0.3% and 3%, and it is the third most common chronic disease affecting adolescent girls. However, voice changes related to AN have not been fully investigated.

OBJECTIVE

The purpose of this study was to evaluate the impact of AN on age-related changes in the voice of adolescent women-before and after puberty, particularly through acoustic analysis. An additional objective was to evaluate estrogen substitution in female patients with AN in order to investigate their effect on voice condition.

MATERIALS AND METHODS

126 girls diagnosed with AN (15.32 ± 2.12 years, range 12-19, BMI = 14.38 ± 1.67), were assessed for the condition of the voice such as perceptual voice evaluation on the GRBAS scale, maximal phonation time (MPT), laryngoscopy, with special attention to voice acoustic analysis-Multi-Dimensional Voice Program (MDVP). The control group (B) included 93 girls without eating disturbances (aged 12-19, mean age 15.52 ± 2.40, BMI = 21.50 ± 1.54). Perceptual voice assessment, aerodynamic test MPT, and acoustic parameters were analyzed in age groups (≤16 years and >16 years). The human vocal tract is sensitive to sex hormones, so the analysis was carried out in the group up to the age of 16 and above 16 to check possible effects.

RESULTS

GRBAS scale was higher in girls with AN compared to the control group for breathiness (B) (P = 0.0002) and asthenia (A) (P < 0.05). The median GRBAS scale for the older group of anorexic women was the highest (2.0). The mean MPT for group A was significantly lower (15.40 ± 3.51 seconds). Comparing age subgroups there was a prolongation of MPT in the healthy group (in groups ≤16 years and >16 years respectively 21.13 seconds versus 25.40 seconds) and a shortening in the anorectic group (≤16 years versus >16 years: 17.06 seconds versus 14.17 seconds). There was no difference between groups A and C up to 16 years of age, but above 16 years of age appeared (14.17 seconds versus 25.40 seconds). Acoustic analysis revealed lower F0 values in group A and C in older subgroups (215,85 Hz versus 236,01 Hz-statistically significant), as well as between subgroups both groups (A: 251,38 Hz versus 215,85 Hz; C: 248,20 Hz versus 236,01 Hz). A narrowing of the vocal range in girls over 16 years in group A was observed. There were no statistically significant differences in F0 between subgroups ≤16 years in groups A and C (251.38 Hz versus 248.20 Hz). The ENT study found that more than half of the girls (54.55%) over the age of 16 who took hormone supplementation manifested laryngeal structure that was normal for their age, there was no effect of hormone supplementation on any of the MDVP parameters between the drug-taking and non-drug-taking groups.

CONCLUSIONS

The acoustic results of the voice in MDVP measurements in adolescent women with AN are not within the normal range and do not mimic the normal developmental changes of the voice. The most important acoustic characteristics of the voice are changes in the fundamental frequency F0 and the range of the voice tended to be more severe in anorectic women >16 years of age and to increase with age, indicating a possible cumulative effect of malnutrition-related disorders as well as hormonal dysfunctions. MDVP can be considered a simple, non-invasive method of assessing the voice organ in AN. MPT differentiated the study groups well: statistically significant differences were noted both between the groups, as well as between age groups. There was no significant effect of oral hormone supplementation on any parameters of the voice. In conclusion, body mass and fat volume in AN may be related to voice production/physiology, affecting voice quality, voice acoustic parameters, voice aerodynamics, and phonatory range in an age-dependent manner. Future studies are needed to assess the long-term efficacy of estrogen treatment in AN.

Stressed Speech Emotion Recognition Using Teager Energy and Spectral Feature Fusion with Feature Optimization

The objective of speech emotion recognition (SER) is to enhance man-machine interface. It can also be used to cover the physiological state of a person in critical situations. In recent time, speech emotion recognition also finds its operations in medicine and forensics. A new feature extraction technique using Teager energy operator (TEO) is proposed for the detection of stressed emotions as Teager energy-autocorrelation envelope (TEO-Auto-Env). TEO is basically designed for increasing the energies of the stressed speech signals whose energies are reduced during the speech production process and hence used in this analysis. A stressed speech emotion recognition (SSER) system is developed using TEO-Auto-Env and spectral feature combination for detecting the emotions. The spectral features considered are Mel-frequency cepstral coefficients (MFCC), linear prediction cepstral coefficients (LPCC), and relative spectra-perceptual linear prediction (RASTA-PLP). EMO-DB (German), EMOVO (Italian), IITKGP (Telugu), and EMA (English) databases are used in this analysis. The classification of the emotions is carried out using the k-nearest neighborhood (k-NN) classifier for gender-dependent (GD) and speaker-independent (SI) cases. The proposed SSER system provides improved accuracy compared to the existing ones. Average recall is used for performance evaluation. The highest classification accuracy is achieved using the feature combination of TEO-Auto-Env, MFCC, and LPCC features with 91.4% (SI), 91.4% (GD-male), and 93.1%(GD-female) for EMO-DB; 68.5% (SI), 68.5% (GD-male), and 74.6% (GD-female) for EMOVO; 90.6%(SI), 91% (GD-male), and 92.3% (GD-female) for EMA; and 95.1% (GD-female) for IITKGP female database.

Fast and accurate assessment of depression based on voice acoustic features: a cross-sectional and longitudinal study

Background

Depression is a widespread mental disorder that affects a significant portion of the population. However, the assessment of depression is often subjective, relying on standard questions or interviews. Acoustic features have been suggested as a reliable and objective alternative for depression assessment. Therefore, in this study, we aim to identify and explore voice acoustic features that can effectively and rapidly predict the severity of depression, as well as investigate the potential correlation between specific treatment options and voice acoustic features.

Methods

We utilized voice acoustic features correlated with depression scores to train a prediction model based on artificial neural network. Leave-one-out cross-validation was performed to evaluate the performance of the model. We also conducted a longitudinal study to analyze the correlation between the improvement of depression and changes in voice acoustic features after an Internet-based cognitive-behavioral therapy (ICBT) program consisting of 12 sessions.

Results

Our study showed that the neural network model trained based on the 30 voice acoustic features significantly correlated with HAMD scores can accurately predict the severity of depression with an absolute mean error of 3.137 and a correlation coefficient of 0.684. Furthermore, four out of the 30 features significantly decreased after ICBT, indicating their potential correlation with specific treatment options and significant improvement in depression (p < 0.05).

Conclusion

Voice acoustic features can effectively and rapidly predict the severity of depression, providing a low-cost and efficient method for screening patients with depression on a large scale. Our study also identified potential acoustic features that may be significantly related to specific treatment options for depression.

Multidimensional voiceprint feature assessment system for identifying the depression in children and adolescents: a diagnostic test

Objective

We designed a diagnostic test to evaluate the effectiveness and accuracy of a multidimensional voiceprint feature diagnostic assessment (MVFDA) system vs. the 24-item Hamilton Rating Scale for Depression (HAMD-24) for adjunctive diagnosis of children and adolescents with major depressive disorder (MDD).

Methods

This study included 55 children aged 6-16 years who were clinically diagnosed with MDD according to the DSM-5 and analyzed by professional physicians, and 55 healthy children (typically developing). Each subject completed a voice recording and was scored on the HAMD-24 scale by a trained rater. We calculated the validity indices, including sensitivity, specificity, Youden's index, likelihood ratio, and other indices including predictive value, diagnostic odds ratio, diagnostic accuracy, and area under the curve (AUC), to assess the effectiveness of the MVFDA system in addition to the HAMD-24.

Results

The sensitivity (92.73 vs. 76.36%) and the specificity (90.91 vs. 85.45%) of the MVFDA system are significantly higher than those of the HAMD-24. The AUC of the MVFDA system is also higher than that of the HAMD-24. There is a statistically significant difference between the groups (p < 0.05), and both of them have high diagnostic accuracy. In addition, the diagnostic efficacy of the MVFDA system is higher than that of HAMD-24 in terms of the Youden index, diagnostic accuracy, likelihood ratio, diagnostic odds ratio, and predictive value.

Conclusion

The MVFDA has performed well in clinical diagnostic trials for the identification of MDD in children and adolescents by capturing objective sound features. Compared with the scale assessment method, the MVFDA system could be further promoted in clinical practice due to its advantages of simple operation, objective rating, and high diagnostic efficiency.

Estimating Depressive Symptom Class from Voice

Voice-based depression detection methods have been studied worldwide as an objective and easy method to detect depression. Conventional studies estimate the presence or severity of depression. However, an estimation of symptoms is a necessary technique not only to treat depression, but also to relieve patients' distress. Hence, we studied a method for clustering symptoms from HAM-D scores of depressed patients and by estimating patients in different symptom groups based on acoustic features of their speech. We could separate different symptom groups with an accuracy of 79%. The results suggest that voice from speech can estimate the symptoms associated with depression.

A speech corpus of Quechua Collao for automatic dimensional emotion recognition

Automatic speech emotion recognition is an important research topic for human-computer interaction and affective computing. Over ten million people speak the Quechua language throughout South America, and one of the most known variants is the Quechua Collao one. However, this language can be considered a low resource for machine emotion recognition, creating a barrier for Quechua speakers who want to use this technology. Therefore, the contribution of this work is a 15 hours speech corpus in Quechua Collao, which is made publicly available to the research community. The corpus was created from a set of words and sentences explicitly collected for this task, divided into nine categorical emotions: happy, sad, bored, fear, sleepy, calm, excited, angry, and neutral. The annotation was performed on a 5-value discrete scale according to 3 dimensions: valence, arousal, and dominance. To demonstrate the usefulness of the corpus, we have performed speech emotion recognition using machine learning methods and neural networks.

Exploration of Despair Eccentricities Based on Scale Metrics with Feature Sampling Using a Deep Learning Algorithm

The majority of people in the modern biosphere struggle with depression as a result of the coronavirus pandemic's impact, which has adversely impacted mental health without warning. Even though the majority of individuals are still protected, it is crucial to check for post-corona virus symptoms if someone is feeling a little lethargic. In order to identify the post-coronavirus symptoms and attacks that are present in the human body, the recommended approach is included. When a harmful virus spreads inside a human body, the post-diagnosis symptoms are considerably more dangerous, and if they are not recognised at an early stage, the risks will be increased. Additionally, if the post-symptoms are severe and go untreated, it might harm one's mental health. In order to prevent someone from succumbing to depression, the technology of audio prediction is employed to recognise all the symptoms and potentially dangerous signs. Different choral characters are used to combine machine-learning algorithms to determine each person's mental state. Design considerations are made for a separate device that detects audio attribute outputs in order to evaluate the effectiveness of the suggested technique; compared to the previous method, the performance metric is substantially better by roughly 67%.

Enhanced depression detection from speech using Quantum Whale Optimization Algorithm for feature selection

There is an urgent need to detect depression using a non-intrusive approach that is reliable and accurate. In this paper, a simple and efficient unimodal depression detection approach based on speech is proposed, which is non-invasive, cost-effective and computationally inexpensive. A set of spectral, temporal and spectro-temporal features is derived from the speech signal of healthy and depressed subjects. To select a minimal subset of the relevant and non-redundant speech features to detect depression, a two-phase approach based on the nature-inspired wrapper-based feature selection Quantum-based Whale Optimization Algorithm (QWOA) is proposed. Experiments are performed on the publicly available Distress Analysis Interview Corpus Wizard-of-Oz (DAICWOZ) dataset and compared with three established univariate filtering techniques for feature selection and four well-known evolutionary algorithms. The proposed model outperforms all the univariate filter feature selection techniques and the evolutionary algorithms. It has low computational complexity in comparison to traditional wrapper-based evolutionary methods. The performance of the proposed approach is superior in comparison to existing unimodal and multimodal automated depression detection models. The combination of spectral, temporal and spectro-temporal speech features gave the best result with the LDA classifier. The performance achieved with the proposed approach, in terms of F1-score for the depressed class and the non-depressed class and error is 0.846, 0.932 and 0.094 respectively. Statistical tests demonstrate that the acoustic features selected using the proposed approach are non-redundant and discriminatory. Statistical tests also establish that the performance of the proposed approach is significantly better than that of the traditional wrapper-based evolutionary methods.

A Model of Normality Inspired Deep Learning Framework for Depression Relapse Prediction Using Audiovisual Data

BACKGROUND

Depression (Major Depressive Disorder) is one of the most common mental illnesses. According to the World Health Organization, more than 300 million people in the world are affected. A first depressive episode can be solved by a spontaneous remission within 6 to 12 months. It has been shown that depression affects speech production and facial expressions. Although numerous studies are proposed in the literature for depression recognition using audiovisual cues, depression relapse using audiovisual cues has not been studied in the literature.

METHOD

In this paper, we propose a deep learning-based approach for depression recognition and depression relapse prediction using audiovisual data. For more versatility and reusability, the proposed approach is based on a Model of Normality inspired framework where we define depression relapse by the closeness of the audiovisual patterns of a subject after a symptom-free period to the audiovisual patterns of depressed subjects. A model of Normality is an anomaly detection distance-based approach that computes a distance of normality between the deep audiovisual encoding of a test sample and a learned representation from audiovisual encodings of anomaly-free data.

RESULTS

The proposed approach shows a very promising results with an accuracy of 87.4% and a F1-score of 82.3% for relapse/depression prediction using a Leave-One-Subject-Out training strategy on the DAIC-Woz dataset.

CONCLUSION

The proposed model of normality-based framework is accurate in detecting depression and in predicting depression relapse. A prospective monitoring system is proposed for assisting depressed patients. The proposed framework is easily extensible and others modalities will be integrated in future works.

Diagnosis of COVID-19 via acoustic analysis and artificial intelligence by monitoring breath sounds on smartphones

Scientific evidence shows that acoustic analysis could be an indicator for diagnosing COVID-19. From analyzing recorded breath sounds on smartphones, it is discovered that patients with COVID-19 have different patterns in both the time domain and frequency domain. These patterns are used in this paper to diagnose the infection of COVID-19. Statistics of the sound signals, analysis in the frequency domain, and Mel-Frequency Cepstral Coefficients (MFCCs) are then calculated and applied in two classifiers, k-Nearest Neighbors (kNN) and Convolutional Neural Network (CNN), to diagnose whether a user is contracted with COVID-19 or not. Test results show that, amazingly, an accuracy of over 97% could be achieved with a CNN classifier and more than 85% on kNN with optimized features. Optimization methods for selecting the best features and using various metrics to evaluate the performance are also demonstrated in this paper. Owing to the high accuracy of the CNN model, the CNN model was implemented in an Android app to diagnose COVID-19 with a probability to indicate the confidence level. The initial medical test shows a similar test result between the method proposed in this paper and the lateral flow method, which indicates that the proposed method is feasible and effective. Because of the use of breath sound and tested on the smartphone, this method could be used by everybody regardless of the availability of other medical resources, which could be a powerful tool for society to diagnose COVID-19.

Acoustic features of voice in adults suffering from depression

In order to examine the differences in people suffering from depression (EG, N=18) compared to the healthy controls (CG1, N=24) and people with the diagnosed psychogenic voice disorder (CG2, N=9), nine acoustic features of voice were assessed among the total of 51 participants using the MDVP software programme ("Kay Elemetrics" Corp., model 4300). Nine acoustic parameters were analysed on the basis of the sustained phonation of the vowel /a/. The results revealed that the mean values of all acoustic parameters differed in the EG compared to both the CG1 and CG2 as follows: the parameters which indicate frequency variability (Jitt, PPQ), amplitude variability (Shim, vAm, APQ) and noise and tremor parameters (NHR, VTI) were higher; only the parameters of fundamental frequency (F0) and soft index phonation (SPI) were lower (F0 compared to CG1, and SPI compared to CG1 and CG2). Only the PPQ parameter was not significant. vAm and APQ had the highest discriminant value for depression. The acoustic features of voice, analysed in this study with regard to the sustained phonation of a vowel, were different and discriminant in the EG compared to CG1 and CG2. In voice analysis, the parameters vAm and APQ could potentially be the markers indicative of depression. The results of this research point to the importance of the voice, that is, its acoustic indicators, in recognizing depression. Important parameters that could help create a programme for the automatic recognition of depression are those from the domain of voice intensity variation.

The applicability of the Beck Depression Inventory and Hamilton Depression Scale in the automatic recognition of depression based on speech signal processing

Depression is a growing problem worldwide, impacting on an increasing number of patients, and also affecting health systems and the global economy. The most common diagnostical rating scales of depression are self-reported or clinician-administered, which differ in the symptoms that they are sampling. Speech is a promising biomarker in the diagnostical assessment of depression, due to non-invasiveness and cost and time efficiency. In our study, we try to achieve a more accurate, sensitive model for determining depression based on speech processing. Regression and classification models were also developed using a machine learning method. During the research, we had access to a large speech database that includes speech samples from depressed and healthy subjects. The database contains the Beck Depression Inventory (BDI) score of each subject and the Hamilton Rating Scale for Depression (HAMD) score of 20% of the subjects. This fact provided an opportunity to compare the usefulness of BDI and HAMD for training models of automatic recognition of depression based on speech signal processing. We found that the estimated values of the acoustic model trained on BDI scores are closer to HAMD assessment than to the BDI scores, and the partial application of HAMD scores instead of BDI scores in training improves the accuracy of automatic recognition of depression.

Automatic Assessment of Loneliness in Older Adults Using Speech Analysis on Responses to Daily Life Questions

Loneliness is a perceived state of social and emotional isolation that has been associated with a wide range of adverse health effects in older adults. Automatically assessing loneliness by passively monitoring daily behaviors could potentially contribute to early detection and intervention for mitigating loneliness. Speech data has been successfully used for inferring changes in emotional states and mental health conditions, but its association with loneliness in older adults remains unexplored. In this study, we developed a tablet-based application and collected speech responses of 57 older adults to daily life questions regarding, for example, one's feelings and future travel plans. From audio data of these speech responses, we automatically extracted speech features characterizing acoustic, prosodic, and linguistic aspects, and investigated their associations with self-rated scores of the UCLA Loneliness Scale. Consequently, we found that with increasing loneliness scores, speech responses tended to have less inflections, longer pauses, reduced second formant frequencies, reduced variances of the speech spectrum, more filler words, and fewer positive words. The cross-validation results showed that regression and binary-classification models using speech features could estimate loneliness scores with an R 2 of 0.57 and detect individuals with high loneliness scores with 95.6% accuracy, respectively. Our study provides the first empirical results suggesting the possibility of using speech data that can be collected in everyday life for the automatic assessments of loneliness in older adults, which could help develop monitoring technologies for early detection and intervention for mitigating loneliness.

Classification of Depression and Other Psychiatric Conditions Using Speech Features Extracted from a Thai Psychiatric and Verbal Screening Test

Depression is a common and serious mental illness which negatively affects daily functioning. To prevent the progression of the illness into severe or long-term consequences, early diagnosis is crucial. We developed an automated speech feature analysis application for depression and other psychiatric disorders derived from a developed Thai psychiatric and verbal screening test. The screening test includes Thai's version of Patient Health Questionnaire-9 (PHQ-9) and Hamilton Depression Rating Scale (HAM-D), and 32 additional emotion-induced questions. Case-control study was conducted on speech features from 66 participants. Twenty seven of those had depression (DP), 12 had other psychiatric disorders (OP), and 27 were normal controls (NC). The five-fold cross-validation from 6 settings of 5 classifiers with the combination of PHQ-9 and HAM-D scores, and speech features were examined. Results showed highest performance from the multilayer perceptron (MLP) classifier which yielded 83.33% sensitivity, 91.67% specificity, and 83.33% accuracy, where negative-emotional questions were most effective in classification. The automated speech feature analysis showed promising results for screening patients with depression or other psychiatric disorders. The current application is accessible through smartphone, making it a feasible and intuitive setup for low-resource countries such as Thailand.

End-to-end multimodal clinical depression recognition using deep neural networks: A comparative analysis

BACKGROUND AND OBJECTIVE

Major Depressive Disorder is a highly prevalent and disabling mental health condition. Numerous studies explored multimodal fusion systems combining visual, audio, and textual features via deep learning architectures for clinical depression recognition. Yet, no comparative analysis for multimodal depression analysis has been proposed in the literature.

METHODS

In this paper, an up-to-date literature overview of multimodal depression recognition is presented and an extensive comparative analysis of different deep learning architectures for depression recognition is performed. First, audio features based Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) are studied. Then, early-level and model-level fusion of deep audio features with visual and textual features through LSTM and CNN architectures are investigated.

RESULTS

The performance of the proposed architectures using an hold-out strategy on the DAIC-WOZ dataset (80% training, 10% validation, 10% test split) for binary and severity levels of depression recognition is tested. Using this strategy, a set of experiments have been performed and they have demonstrated: (1) LSTM-based audio features perform slightly better than CNN ones with an accuracy of 66.25% versus 65.60% for binary depression classes. (2) the model level fusion of deep audio and visual features using LSTM network performed the best with an accuracy of 77.16%, a precision of 53% for the depressed class, and a precision of 83% for the non-depressed class. The given network obtained a normalized Root Mean Square Error (RMSE) of 0.15 for depression severity level prediction. Using a Leave-One-Subject-Out strategy, this network achieved an accuracy of 95.38% for binary depression detection, and a normalized RMSE of 0.1476 for depression severity level prediction. Our best-performing architecture outperforms all state-of-the-art approaches on DAIC-WOZ dataset.

CONCLUSIONS

The obtained results show that the proposed LSTM-based surpass the proposed CNN-based architectures allowing to learn temporal dynamics representations of multimodal features. Furthermore, model-level fusion of audio and visual features using an LSTM network leads to the best performance. Our best-performing architecture successfully detects depression using a speech segment of less than 8 seconds, and an average prediction computation time of less than 6ms; making it suitable for real-world clinical applications.

A Mobile Sensing App to Monitor Youth Mental Health: Observational Pilot Study

Background

Internalizing disorders are the most common psychiatric problems observed among youth in Canada. Sadly, youth with internalizing disorders often avoid seeking clinical help and rarely receive adequate treatment. Current methods of assessing internalizing disorders usually rely on subjective symptom ratings, but internalizing symptoms are frequently underreported, which creates a barrier to the accurate assessment of these symptoms in youth. Therefore, novel assessment tools that use objective data need to be developed to meet the highest standards of reliability, feasibility, scalability, and affordability. Mobile sensing technologies, which unobtrusively record aspects of youth behaviors in their daily lives with the potential to make inferences about their mental health states, offer a possible method of addressing this assessment barrier.

Objective

This study aims to explore whether passively collected smartphone sensor data can be used to predict internalizing symptoms among youth in Canada.

Methods

In this study, the youth participants (N=122) completed self-report assessments of symptoms of anxiety, depression, and attention-deficit hyperactivity disorder. Next, the participants installed an app, which passively collected data about their mobility, screen time, sleep, and social interactions over 2 weeks. Then, we tested whether these passive sensor data could be used to predict internalizing symptoms among these youth participants.

Results

More severe depressive symptoms correlated with more time spent stationary (r=0.293; P=.003), less mobility (r=0.271; P=.006), higher light intensity during the night (r=0.227; P=.02), and fewer outgoing calls (r=−0.244; P=.03). In contrast, more severe anxiety symptoms correlated with less time spent stationary (r=−0.249; P=.01) and greater mobility (r=0.234; P=.02). In addition, youths with higher anxiety scores spent more time on the screen (r=0.203; P=.049). Finally, adding passively collected smartphone sensor data to the prediction models of internalizing symptoms significantly improved their fit.

Conclusions

Passively collected smartphone sensor data provide a useful way to monitor internalizing symptoms among youth. Although the results replicated findings from adult populations, to ensure clinical utility, they still need to be replicated in larger samples of youth. The work also highlights intervention opportunities via mobile technology to reduce the burden of internalizing symptoms early on.

Measuring neuropsychiatric symptoms in patients with early cognitive decline using speech analysis

BACKGROUND

Certain neuropsychiatric symptoms (NPS), namely apathy, depression, and anxiety demonstrated great value in predicting dementia progression, representing eventually an opportunity window for timely diagnosis and treatment. However, sensitive and objective markers of these symptoms are still missing. Therefore, the present study aims to investigate the association between automatically extracted speech features and NPS in patients with mild neurocognitive disorders.

METHODS

Speech of 141 patients aged 65 or older with neurocognitive disorder was recorded while performing two short narrative speech tasks. NPS were assessed by the neuropsychiatric inventory. Paralinguistic markers relating to prosodic, formant, source, and temporal qualities of speech were automatically extracted, correlated with NPS. Machine learning experiments were carried out to validate the diagnostic power of extracted markers.

RESULTS

Different speech variables are associated with specific NPS; apathy correlates with temporal aspects, and anxiety with voice quality-and this was mostly consistent between male and female after correction for cognitive impairment. Machine learning regressors are able to extract information from speech features and perform above baseline in predicting anxiety, apathy, and depression scores.

CONCLUSIONS

Different NPS seem to be characterized by distinct speech features, which are easily extractable automatically from short vocal tasks. These findings support the use of speech analysis for detecting subtypes of NPS in patients with cognitive impairment. This could have great implications for the design of future clinical trials as this cost-effective method could allow more continuous and even remote monitoring of symptoms.

Screening major depressive disorder using vocal acoustic features in the elderly by sex

BACKGROUND

Vocal acoustic features are potential biomarkers of elderly depression. Previous automated diagnostic tests for depression have employed unstandardized speech samples, and few studies have considered differences in voice reactivity. We aimed to develop a voice-based screening test for depression measuring vocal acoustic features of elderly Koreans while they read a series of mood-inducing sentences (MIS).

METHODS

In this case-control study, we recruited 61 individuals with major depressive disorder and 143 healthy controls (mean age [SD]: 72 [6]; female, 70%) from the community-dwelling elderly population. Participants were asked to read MIS and their variation pattern of acoustic features represented by the correlation distance between two MIS were analyzed as input features using the univariate feature selection technique and subsequently classified by AdaBoost.

RESULTS

Acoustic features showing significant discriminatory performances were spectral and energy-related features for males (sensitivity 0.95, specificity 0.88, and accuracy 0.86) and prosody-related features for females (sensitivity 0.73, specificity 0.86, and accuracy 0.77). The correlation distance between negative and positive MIS was significantly shorter in the depressed group than in the healthy control (F = 18.574, P < 0.001).

LIMITATIONS

Small sample size and relatively homogenous clinical profile of depression could limit the generalizability.

CONCLUSIONS

While reading MIS, spectral and energy-related acoustic features for males and prosody-related features for females are good discriminators for major depressive disorder. These features may be used as biomarkers of depression in the elderly.

Measuring Stress in Health Professionals Over the Phone Using Automatic Speech Analysis During the COVID-19 Pandemic: Observational Pilot Study

Background

During the COVID-19 pandemic, health professionals have been directly confronted with the suffering of patients and their families. By making them main actors in the management of this health crisis, they have been exposed to various psychosocial risks (stress, trauma, fatigue, etc). Paradoxically, stress-related symptoms are often underreported in this vulnerable population but are potentially detectable through passive monitoring of changes in speech behavior.

Objective

This study aims to investigate the use of rapid and remote measures of stress levels in health professionals working during the COVID-19 outbreak. This was done through the analysis of participants’ speech behavior during a short phone call conversation and, in particular, via positive, negative, and neutral storytelling tasks.

Methods

Speech samples from 89 health care professionals were collected over the phone during positive, negative, and neutral storytelling tasks; various voice features were extracted and compared with classical stress measures via standard questionnaires. Additionally, a regression analysis was performed.

Results

Certain speech characteristics correlated with stress levels in both genders; mainly, spectral (ie, formant) features, such as the mel-frequency cepstral coefficient, and prosodic characteristics, such as the fundamental frequency, appeared to be sensitive to stress. Overall, for both male and female participants, using vocal features from the positive tasks for regression yielded the most accurate prediction results of stress scores (mean absolute error 5.31).

Conclusions

Automatic speech analysis could help with early detection of subtle signs of stress in vulnerable populations over the phone. By combining the use of this technology with timely intervention strategies, it could contribute to the prevention of burnout and the development of comorbidities, such as depression or anxiety.

What You Say or How You Say It? Depression Detection Through Joint Modeling of Linguistic and Acoustic Aspects of Speech

Depression is one of the most common mental health issues. (It affects more than 4% of the world’s population, according to recent estimates.) This article shows that the joint analysis of linguistic and acoustic aspects of speech allows one to discriminate between depressed and nondepressed speakers with an accuracy above 80%. The approach used in the work is based on networks designed for sequence modeling (bidirectional Long-Short Term Memory networks) and multimodal analysis methodologies (late fusion, joint representation and gated multimodal units). The experiments were performed over a corpus of 59 interviews (roughly 4 hours of material) involving 29 individuals diagnosed with depression and 30 control participants. In addition to an accuracy of 80%, the results show that multimodal approaches perform better than unimodal ones owing to people’s tendency to manifest their condition through one modality only, a source of diversity across unimodal approaches. In addition, the experiments show that it is possible to measure the “confidence” of the approach and automatically identify a subset of the test data in which the performance is above a predefined threshold. It is possible to effectively detect depression by using unobtrusive and inexpensive technologies based on the automatic analysis of speech and language.

Speech Technology for Healthcare: Opportunities, Challenges, and State of the Art

Speech technology is not appropriately explored even though modern advances in speech technology-especially those driven by deep learning (DL) technology-offer unprecedented opportunities for transforming the healthcare industry. In this paper, we have focused on the enormous potential of speech technology for revolutionising the healthcare domain. More specifically, we review the state-of-the-art approaches in automatic speech recognition (ASR), speech synthesis or text to speech (TTS), and health detection and monitoring using speech signals. We also present a comprehensive overview of various challenges hindering the growth of speech-based services in healthcare. To make speech-based healthcare solutions more prevalent, we discuss open issues and suggest some possible research directions aimed at fully leveraging the advantages of other technologies for making speech-based healthcare solutions more effective.

Evaluation of Speech-Based Digital Biomarkers: Review and Recommendations

Speech represents a promising novel biomarker by providing a window into brain health, as shown by its disruption in various neurological and psychiatric diseases. As with many novel digital biomarkers, however, rigorous evaluation is currently lacking and is required for these measures to be used effectively and safely. This paper outlines and provides examples from the literature of evaluation steps for speech-based digital biomarkers, based on the recent V3 framework (Goldsack et al., 2020). The V3 framework describes 3 components of evaluation for digital biomarkers: verification, analytical validation, and clinical validation. Verification includes assessing the quality of speech recordings and comparing the effects of hardware and recording conditions on the integrity of the recordings. Analytical validation includes checking the accuracy and reliability of data processing and computed measures, including understanding test-retest reliability, demographic variability, and comparing measures to reference standards. Clinical validity involves verifying the correspondence of a measure to clinical outcomes which can include diagnosis, disease progression, or response to treatment. For each of these sections, we provide recommendations for the types of evaluation necessary for speech-based biomarkers and review published examples. The examples in this paper focus on speech-based biomarkers, but they can be used as a template for digital biomarker development more generally.

Mobile and wearable technology for monitoring depressive symptoms in children and adolescents: A scoping review

BACKGROUND

There has been rapid growth of mobile and wearable tools that may help to overcome challenges in the diagnosis and prediction of Major Depressive Disorder in children and adolescents, tasks that rely on clinical reporting that is inherently based on retrospective recall of symptoms and associated features. This article reviews more objective ways of measuring and monitoring mood within this population.

METHODS

A scoping review of peer-reviewed studies examined published research that employs mobile and wearable tools to characterize depression in children and/or adolescents. Our search strategy included the following terms: (1) monitoring or prediction (2) depression (3) mobile apps or wearables and (4) children and youth (including adolescents), and was applied to five databases.

RESULTS

Our search produced 829 citations (2008- Feb 2019), of which 30 (journal articles, conference papers and abstracts) were included in the analysis, and 2 reviews included in our discussion. The majority of the evidence involved smartphone apps, with very few studies using actigraphy. Mobile and wearables captured a variety of data including unobtrusive passive analytics, movement and light data, plus physical and mental health data, including depressive symptom monitoring. Most studies also examined feasibility.

LIMITATIONS

This review was limited to published research in the English language. The review criteria excluded any apps that were mainly treatment focused, therefore there was not much of a focus on clinical outcomes.

CONCLUSIONS

This scoping review yielded a variety of studies with heterogeneous populations, research methods and study objectives, which limited our ability to address our research objectives cohesively. Certain mobile technologies, however, have demonstrated feasibility for tracking depression that could inform models for predicting relapse.

Cell-Coupled Long Short-Term Memory With L -Skip Fusion Mechanism for Mood Disorder Detection Through Elicited Audiovisual Features

In early stages, patients with bipolar disorder are often diagnosed as having unipolar depression in mood disorder diagnosis. Because the long-term monitoring is limited by the delayed detection of mood disorder, an accurate and one-time diagnosis is desirable to avoid delay in appropriate treatment due to misdiagnosis. In this paper, an elicitation-based approach is proposed for realizing a one-time diagnosis by using responses elicited from patients by having them watch six emotion-eliciting videos. After watching each video clip, the conversations, including patient facial expressions and speech responses, between the participant and the clinician conducting the interview were recorded. Next, the hierarchical spectral clustering algorithm was employed to adapt the facial expression and speech response features by using the extended Cohn-Kanade and eNTERFACE databases. A denoizing autoencoder was further applied to extract the bottleneck features of the adapted data. Then, the facial and speech bottleneck features were input into support vector machines to obtain speech emotion profiles (EPs) and the modulation spectrum (MS) of the facial action unit sequence for each elicited response. Finally, a cell-coupled long short-term memory (LSTM) network with an L -skip fusion mechanism was proposed to model the temporal information of all elicited responses and to loosely fuse the EPs and the MS for conducting mood disorder detection. The experimental results revealed that the cell-coupled LSTM with the L -skip fusion mechanism has promising advantages and efficacy for mood disorder detection.

Multimodal Depression Detection: Fusion of Electroencephalography and Paralinguistic Behaviors Using a Novel Strategy for Classifier Ensemble

Currently, depression has become a common mental disorder and one of the main causes of disability worldwide. Due to the difference in depressive symptoms evoked by individual differences, how to design comprehensive and effective depression detection methods has become an urgent demand. This study explored from physiological and behavioral perspectives simultaneously and fused pervasive electroencephalography (EEG) and vocal signals to make the detection of depression more objective, effective and convenient. After extraction of several effective features for these two types of signals, we trained six representational classifiers on each modality, then denoted diversity and correlation of decisions from different classifiers using co-decision tensor and combined these decisions into the ultimate classification result with multi-agent strategy. Experimental results on 170 (81 depressed patients and 89 normal controls) subjects showed that the proposed multi-modal depression detection strategy is superior to the single-modal classifiers or other typical late fusion strategies in accuracy, f1-score and sensitivity. This work indicates that late fusion of pervasive physiological and behavioral signals is promising for depression detection and the multi-agent strategy can take advantage of diversity and correlation of different classifiers effectively to gain a better final decision.

Re-examining the robustness of voice features in predicting depression: Compared with baseline of confounders

A large proportion of Depression Disorder patients do not receive an effective diagnosis, which makes it necessary to find a more objective assessment to facilitate a more rapid and accurate diagnosis of depression. Speech data is easy to acquire clinically, its association with depression has been studied, although the actual predictive effect of voice features has not been examined. Thus, we do not have a general understanding of the extent to which voice features contribute to the identification of depression. In this study, we investigated the significance of the association between voice features and depression using binary logistic regression, and the actual classification effect of voice features on depression was re-examined through classification modeling. Nearly 1000 Chinese females participated in this study. Several different datasets was included as test set. We found that 4 voice features (PC1, PC6, PC17, PC24, P<0.05, corrected) made significant contribution to depression, and that the contribution effect of the voice features alone reached 35.65% (Nagelkerke's R²). In classification modeling, voice data based model has consistently higher predicting accuracy(F-measure) than the baseline model of demographic data when tested on different datasets, even across different emotion context. F-measure of voice features alone reached 81%, consistent with existing data. These results demonstrate that voice features are effective in predicting depression and indicate that more sophisticated models based on voice features can be built to help in clinical diagnosis.

Depression Detection Through Speech Analysis : A Survey

Depression is a common and serious medical illness which affects the way how we think, feel and act. Although harmless in its initial stages, it can cause serious problems if detected at a later stage. Due to advancements in technology, it is now possible to detect signs of depression. Different implementation of machine learning algorithms has been worked upon to detect factors causing depression. It is found that speech of a person is dramatically affected and various vocal features are used to classify depression.

Identification of Affective State Change in Adults With Aphasia Using Speech Acoustics

Purpose

The current study aimed to identify objective acoustic measures related to affective state change in the speech of adults with post-stroke aphasia.

Method

The speech of 20 post-stroke adults with aphasia was recorded during picture description and administration of the Western Aphasia Battery-Revised (Kertesz, 2006). In addition, participants completed the Self-Assessment Manikin (Bradley & Lang, 1994) and the Stress Scale (Tobii Dynavox, 1981-2016) before and after the language tasks. Speech from each participant was used to detect a change in affective state test scores between the beginning and ending speech.

Results

Machine learning revealed moderate success in classifying depression, minimal success in predicting depression and stress numeric scores, and minimal success in classifying changes in affective state class between the beginning and ending speech.

Conclusions

The results suggest the existence of objectively measurable aspects of speech that may be used to identify changes in acute affect from adults with aphasia. This work is exploratory and hypothesis-generating; more work will be needed to make conclusive claims. Further work in this area could lead to automated tools to assist clinicians with their diagnoses of stress, depression, and other forms of affect in adults with aphasia.

Detecting Depression Using an Ensemble Logistic Regression Model Based on Multiple Speech Features

Early intervention for depression is very important to ease the disease burden, but current diagnostic methods are still limited. This study investigated automatic depressed speech classification in a sample of 170 native Chinese subjects (85 healthy controls and 85 depressed patients). The classification performances of prosodic, spectral, and glottal speech features were analyzed in recognition of depression. We proposed an ensemble logistic regression model for detecting depression (ELRDD) in speech. The logistic regression, which was superior in recognition of depression, was selected as the base classifier. This ensemble model extracted many speech features from different aspects and ensured diversity of the base classifier. ELRDD provided better classification results than the other compared classifiers. A technique for identifying depression based on ELRDD, ELRDD-E, was here suggested and tested. It offered encouraging outcomes, revealing a high accuracy level of 75.00% for females and 81.82% for males, as well as an advantageous sensitivity/specificity ratio of 79.25%/70.59% for females and 78.13%/85.29% for males.

The Effortless Assessment of Risk States (EARS) Tool: An Interpersonal Approach to Mobile Sensing

BACKGROUND

To predict and prevent mental health crises, we must develop new approaches that can provide a dramatic advance in the effectiveness, timeliness, and scalability of our interventions. However, current methods of predicting mental health crises (eg, clinical monitoring, screening) usually fail on most, if not all, of these criteria. Luckily for us, 77% of Americans carry with them an unprecedented opportunity to detect risk states and provide precise life-saving interventions. Smartphones present an opportunity to empower individuals to leverage the data they generate through their normal phone use to predict and prevent mental health crises.

OBJECTIVE

To facilitate the collection of high-quality, passive mobile sensing data, we built the Effortless Assessment of Risk States (EARS) tool to enable the generation of predictive machine learning algorithms to solve previously intractable problems and identify risk states before they become crises.

METHODS

The EARS tool captures multiple indices of a person's social and affective behavior via their naturalistic use of a smartphone. Although other mobile data collection tools exist, the EARS tool places a unique emphasis on capturing the content as well as the form of social communication on the phone. Signals collected include facial expressions, acoustic vocal quality, natural language use, physical activity, music choice, and geographical location. Critically, the EARS tool collects these data passively, with almost no burden on the user. We programmed the EARS tool in Java for the Android mobile platform. In building the EARS tool, we concentrated on two main considerations: (1) privacy and encryption and (2) phone use impact.

RESULTS

In a pilot study (N=24), participants tolerated the EARS tool well, reporting minimal burden. None of the participants who completed the study reported needing to use the provided battery packs. Current testing on a range of phones indicated that the tool consumed approximately 15% of the battery over a 16-hour period. Installation of the EARS tool caused minimal change in the user interface and user experience. Once installation is completed, the only difference the user notices is the custom keyboard.

CONCLUSIONS

The EARS tool offers an innovative approach to passive mobile sensing by emphasizing the centrality of a person's social life to their well-being. We built the EARS tool to power cutting-edge research, with the ultimate goal of leveraging individual big data to empower people and enhance mental health.

Detecting Manic State of Bipolar Disorder Based on Support Vector Machine and Gaussian Mixture Model Using Spontaneous Speech

OBJECTIVE

This study was aimed to compare the accuracy of Support Vector Machine (SVM) and Gaussian Mixture Model (GMM) in the detection of manic state of bipolar disorders (BD) of single patients and multiple patients.

METHODS

21 hospitalized BD patients (14 females, average age 34.5±15.3) were recruited after admission. Spontaneous speech was collected through a preloaded smartphone. Firstly, speech features [pitch, formants, mel-frequency cepstrum coefficients (MFCC), linear prediction cepstral coefficient (LPCC), gamma-tone frequency cepstral coefficients (GFCC) etc.] were preprocessed and extracted. Then, speech features were selected using the features of between-class variance and within-class variance. The manic state of patients was then detected by SVM and GMM methods.

RESULTS

LPCC demonstrated the best discrimination efficiency. The accuracy of manic state detection for single patients was much better using SVM method than GMM method. The detection accuracy for multiple patients was higher using GMM method than SVM method.

CONCLUSION

SVM provided an appropriate tool for detecting manic state for single patients, whereas GMM worked better for multiple patients' manic state detection. Both of them could help doctors and patients for better diagnosis and mood state monitoring in different situations.

A Weakly Supervised Learning Framework for Detecting Social Anxiety and Depression

Although social anxiety and depression are common, they are often underdiagnosed and undertreated, in part due to difficulties identifying and accessing individuals in need of services. Current assessments rely on client self-report and clinician judgment, which are vulnerable to social desirability and other subjective biases. Identifying objective, nonburdensome markers of these mental health problems, such as features of speech, could help advance assessment, prevention, and treatment approaches. Prior research examining speech detection methods has focused on fully supervised learning approaches employing strongly labeled data. However, strong labeling of individuals high in symptoms or state affect in speech audio data is impractical, in part because it is not possible to identify with high confidence which regions of a long speech indicate the person's symptoms or affective state. We propose a weakly supervised learning framework for detecting social anxiety and depression from long audio clips. Specifically, we present a novel feature modeling technique named NN2Vec that identifies and exploits the inherent relationship between speakers' vocal states and symptoms/affective states. Detecting speakers high in social anxiety or depression symptoms using NN2Vec features achieves F-1 scores 17% and 13% higher than those of the best available baselines. In addition, we present a new multiple instance learning adaptation of a BLSTM classifier, named BLSTM-MIL. Our novel framework of using NN2Vec features with the BLSTM-MIL classifier achieves F-1 scores of 90.1% and 85.44% in detecting speakers high in social anxiety and depression symptoms.

Word Repetition in Separate Conversations for Detecting Dementia:A Preliminary Evaluation on Data of Regular Monitoring Service

For detecting early signs of dementia, monitoring technology has been actively investigated due to the low diagnostic coverage as well as the requirement for early intervention. Although language features have been used for detecting the language dysfunctions resulting from dementia in neuropsychological tests, features that can be extracted by regular conversations remain unexplored. Here, we propose a feature to characterize the atypical repetition of words on different days which is observed in patients with dementia. We tested it on data obtained from a daily monitoring service for eight elderly people, including two who had been diagnosed with dementia. We found that our feature outperformed the existing linguistic features used in previous studies, such as vocabulary richness and repetitiveness, in terms of effect size and AUC score. The results suggest that the use of our proposed feature holds promise for improving detection performance in everyday situations such as regular monitoring.

Analysis on speech signal features of manic patients

Given the lack of effective biological markers for early diagnosis of bipolar mania, and the tendency for voice fluctuation during transition between mood states, this study aimed to investigate the speech features of manic patients to identify a potential set of biomarkers for diagnosis of bipolar mania. 30 manic patients and 30 healthy controls were recruited and their corresponding speech features were collected during natural dialogue using the Automatic Voice Collecting System. Bech-Rafaelsdn Mania Rating Scale (BRMS) and Clinical impression rating scale (CGI) were used to assess illness. The speech features were compared between two groups: mood group (mania vs remission) and bipolar group (manic patients vs healthy individuals). We found that the characteristic speech signals differed between mood groups and bipolar groups. The fourth formant (F4) and Linear Prediction Coefficient (LPC) (P < .05) were significantly differed when patients transmitted from manic to remission state. The first formant (F1), the second formant (F2), and LPC (P < .05) also played key roles in distinguishing between patients and healthy individuals. In addition, there was a significantly correlation between LPC and BRMS, indicating that LPC may play an important role in diagnosis of bipolar mania. In this study we traced speech features of bipolar mania during natural dialogue (conversation), which is an accessible approach in clinic practice. Such specific indicators may respectively serve as promising biomarkers for benefiting the diagnosis and clinical therapeutic evaluation of bipolar mania.

Major depressive disorder discrimination using vocal acoustic features

BACKGROUND

The voice carries various information produced by vibrations of the vocal cords and the vocal tract. Though many studies have reported a relationship between vocal acoustic features and depression, including mel-frequency cepstrum coefficients (MFCCs) which applied to speech recognition, there have been few studies in which acoustic features allowed discrimination of patients with depressive disorder. Vocal acoustic features as biomarker of depression could make differential diagnosis of patients with depressive state. In order to achieve differential diagnosis of depression, in this preliminary study, we examined whether vocal acoustic features could allow discrimination between depressive patients and healthy controls.

METHODS

Subjects were 36 patients who met the criteria for major depressive disorder and 36 healthy controls with no current or past psychiatric disorders. Voices of reading out digits before and after verbal fluency task were recorded. Voices were analyzed using OpenSMILE. The extracted acoustic features, including MFCCs, were used for group comparison and discriminant analysis between patients and controls.

RESULTS

The second dimension of MFCC (MFCC 2) was significantly different between groups and allowed the discrimination between patients and controls with a sensitivity of 77.8% and a specificity of 86.1%. The difference in MFCC 2 between the two groups reflected an energy difference of frequency around 2000-3000Hz.

CONCLUSIONS

The MFCC 2 was significantly different between depressive patients and controls. This feature could be a useful biomarker to detect major depressive disorder.

LIMITATIONS

Sample size was relatively small. Psychotropics could have a confounding effect on voice.

Voice quality in patients suffering from bipolar disease

People suffering from bipolar disease are more and more common. Such pathology can severely affect patients' lifestyle by wide, and sometimes extreme, mood swings. Biosignals can be very useful to understand this disease. Specifically, speech-related features have been seen to vary in depressed people with respect to healthy subjects. Usually prosodic, spectral and energy-related features are studied. Some further information, instead, can be provided studying voice quality. According to Laver's model, voice quality is sensitive and depends on both anatomic/physiologic issues and long-term muscular adjustments of the larynx or the supraglottal vocal tract. A pilot study on both bipolar patients and healthy control subjects, performed by means of the Long-Term Average Spectrum (LTAS) is presented. The effects on LTAS estimation of a F0-correction procedure are discussed. Pairwise statistical comparisons between subjects in euthymic and depressed states and euthymic and hypomanic states were performed. Significant differences were found in some frequency intervals in both cases. The F0-correction procedure modified the values of the significant frequency intervals in the euthymic/depressed comparison, that also was characterized by a change of F0. Noticeably, no statistically significant differences were found in control subjects acquired in the same mood state. Though the number of subjects is small, the results are encouraging given their coherence across patients and the lack of differences in the control group. Finally, this work suggests that particular vocal settings might be involved in different mood states.

The development of the Athens Emotional States Inventory (AESI): collection, validation and automatic processing of emotionally loaded sentences

OBJECTIVES

The development of ecologically valid procedures for collecting reliable and unbiased emotional data towards computer interfaces with social and affective intelligence targeting patients with mental disorders.

METHODS

Following its development, presented with, the Athens Emotional States Inventory (AESI) proposes the design, recording and validation of an audiovisual database for five emotional states: anger, fear, joy, sadness and neutral. The items of the AESI consist of sentences each having content indicative of the corresponding emotion. Emotional content was assessed through a survey of 40 young participants with a questionnaire following the Latin square design. The emotional sentences that were correctly identified by 85% of the participants were recorded in a soundproof room with microphones and cameras. A preliminary validation of AESI is performed through automatic emotion recognition experiments from speech.

RESULTS

The resulting database contains 696 recorded utterances in Greek language by 20 native speakers and has a total duration of approximately 28 min. Speech classification results yield accuracy up to 75.15% for automatically recognizing the emotions in AESI.

CONCLUSIONS

These results indicate the usefulness of our approach for collecting emotional data with reliable content, balanced across classes and with reduced environmental variability.

Smartphone Application for the Analysis of Prosodic Features in Running Speech with a Focus on Bipolar Disorders: System Performance Evaluation and Case Study

Bipolar disorder is one of the most common mood disorders characterized by large and invalidating mood swings. Several projects focus on the development of decision support systems that monitor and advise patients, as well as clinicians. Voice monitoring and speech signal analysis can be exploited to reach this goal. In this study, an Android application was designed for analyzing running speech using a smartphone device. The application can record audio samples and estimate speech fundamental frequency, F0, and its changes. F0-related features are estimated locally on the smartphone, with some advantages with respect to remote processing approaches in terms of privacy protection and reduced upload costs. The raw features can be sent to a central server and further processed. The quality of the audio recordings, algorithm reliability and performance of the overall system were evaluated in terms of voiced segment detection and features estimation. The results demonstrate that mean F0 from each voiced segment can be reliably estimated, thus describing prosodic features across the speech sample. Instead, features related to F0 variability within each voiced segment performed poorly. A case study performed on a bipolar patient is presented.

Exploratory Study of Depressed Adolescents' Life Narratives

Objective: The aim of this study was to explore the life stories of depressive adolescents and compare them with non-clinical adolescents' life stories. Methods: For this purpose, we compared 20 life stories of hospitalized adolescents suffering from major depressive episode with 40 life stories of adolescents attending school divided into two groups: 20 non-depressed and 20 depressed adolescents. Results: Results showed that life stories differed as a function of psychopathology. Depressed hospitalized adolescents spoke about their disease and defined themselves by their depression. The depressed adolescents in school concentrated on schooling and school achievements, while the non-depressed group defined themselves by their family, friends and inclusion in a peer group. Conclusion: These analyses allowed us to highlight specific themes mentioned by each of the three groups of adolescents. Although life stories are personal and unique, analysis of such stories allows us to better understand the daily reality of depressive adolescents and the relationships between the life events they experience, daily stressors, depression and how they construct their personal history.