Effectiveness of a Voice-Based Mental Health Evaluation System for Mobile Devices: Prospective Study

Use of voice features from smartphones for monitoring depressive disorders: Scoping review

Object

This review evaluates the use of smartphone-based voice data for predicting and monitoring depression.

Methods

A scoping review was conducted, examining 14 studies from Medline, Scopus, and Web of Science (2010-2023) on voice data collection methods and the use of voice features for minitoring depression.

Results

Voice data, especially prosodic features like fundamental frequency and pitch, show promise for predicting depression, though their sole predictive power requires further validation. Integrating voice with multimodal sensor data has been shown to improve accuracy significantly.

Conclusion

Smartphone-based voice monitoring offers a promising, noninvasive, and cost-effective approach to depression management. The integration of machine learning with sensor data could significantly enhance mental health monitoring, necessitating further research and longitudinal studies for validation.

Severity Classification Using Dynamic Time Warping-Based Voice Biomarkers for Patients With COVID-19: Feasibility Cross-Sectional Study

Background

In Japan, individuals with mild COVID-19 illness previously required to be monitored in designated areas and were hospitalized only if their condition worsened to moderate illness or worse. Daily monitoring using a pulse oximeter was a crucial indicator for hospitalization. However, a drastic increase in the number of patients resulted in a shortage of pulse oximeters for monitoring. Therefore, an alternative and cost-effective method for monitoring patients with mild illness was required. Previous studies have shown that voice biomarkers for Parkinson disease or Alzheimer disease are useful for classifying or monitoring symptoms; thus, we tried to adapt voice biomarkers for classifying the severity of COVID-19 using a dynamic time warping (DTW) algorithm where voice wavelets can be treated as 2D features; the differences between wavelet features are calculated as scores.

Objective

This feasibility study aimed to test whether DTW-based indices can generate voice biomarkers for a binary classification model using COVID-19 patients' voices to distinguish moderate illness from mild illness at a significant level.

Methods

We conducted a cross-sectional study using voice samples of COVID-19 patients. Three kinds of long vowels were processed into 10-cycle waveforms with standardized power and time axes. The DTW-based indices were generated by all pairs of waveforms and tested with the Mann-Whitney U test (α<.01) and verified with a linear discrimination analysis and confusion matrix to determine which indices were better for binary classification of disease severity. A binary classification model was generated based on a generalized linear model (GLM) using the most promising indices as predictors. The receiver operating characteristic curve/area under the curve (ROC/AUC) validated the model performance, and the confusion matrix calculated the model accuracy.

Results

Participants in this study (n=295) were infected with COVID-19 between June 2021 and March 2022, were aged 20 years or older, and recuperated in Kanagawa prefecture. Voice samples (n=110) were selected from the participants' attribution matrix based on age group, sex, time of infection, and whether they had mild illness (n=61) or moderate illness (n=49). The DTW-based variance indices were found to be significant (P<.001, except for 1 of 6 indices), with a balanced accuracy in the range between 79% and 88.6% for the /a/, /e/, and /u/ vowel sounds. The GLM achieved a high balance accuracy of 86.3% (for /a/), 80.2% (for /e/), and 88% (for /u/) and ROC/AUC of 94.8% (95% CI 90.6%-94.8%) for /a/, 86.5% (95% CI 79.8%-86.5%) for /e/, and 95.6% (95% CI 92.1%-95.6%) for /u/.

Conclusions

The proposed model can be a voice biomarker for an alternative and cost-effective method of monitoring the progress of COVID-19 patients in care.

Distinguish the Severity of Illness Associated with Novel Coronavirus (COVID-19) Infection via Sustained Vowel Speech Features

The authors are currently conducting research on methods to estimate psychiatric and neurological disorders from a voice by focusing on the features of speech. It is empirically known that numerous psychosomatic symptoms appear in voice biomarkers; in this study, we examined the effectiveness of distinguishing changes in the symptoms associated with novel coronavirus infection using speech features. Multiple speech features were extracted from the voice recordings, and, as a countermeasure against overfitting, we selected features using statistical analysis and feature selection methods utilizing pseudo data and built and verified machine learning algorithm models using LightGBM. Applying 5-fold cross-validation, and using three types of sustained vowel sounds of /Ah/, /Eh/, and /Uh/, we achieved a high performance (accuracy and AUC) of over 88% in distinguishing "asymptomatic or mild illness (symptoms)" and "moderate illness 1 (symptoms)". Accordingly, the results suggest that the proposed index using voice (speech features) can likely be used in distinguishing the symptoms associated with novel coronavirus infection.

The ME-BYO index: A development and validation project of a novel comprehensive health index

Quantifying health status and identifying modifiable factors are essential for effective and individualized prevention of age-related conditions and for promoting health during aging. The ME-BYO concept from Kanagawa Prefecture, one of Japan's largest prefectures, can be used to establish a healthy aging society. In disease etiology, ME-BYO considers the status of an individual's body and mind as changing continuously from healthy to sick instead of being a dichotomy between the two. ME-BYO conceptualizes the entire process of this change. The ME-BYO index was developed in 2019 to comprehensively and numerically measure and visualize an individual's current health status and future disease risk by quantifying data on the four domains of metabolic function, locomotor function, cognitive function, and mental resilience. The ME-BYO index has been implemented in the personal health management application "My ME-BYO." However, scientific validation of this index and the development of a practical application using healthcare data remain to be completed. In 2020, our research team started a project to refine the ME-BYO index using data from the Kanagawa ME-BYO prospective cohort study, which is a large population-based genomic cohort study. This project will scientifically evaluate the ME-BYO index and develop a practical application for promoting healthy aging.

Evaluation of the validity and reliability of the 10-meter walk test using a smartphone application among Japanese older adults

Objective

Maintaining or improving regular walking speed can help extend healthy life expectancy and prevent frailty. The evaluation of walking speed can help individuals meet their daily exercise goals; therefore, it may be beneficial as a public health policy for residents to measure and evaluate their walking speed easily. This study aimed to verify the validity and reliability of a smartphone application for the 10-m walk test, measured in the general population.

Methods

The study participants were men (n = 20) and women (n = 20) aged 65–85 years. The 10-m walk tests were performed at the usual walking speed, using the stopwatch function of a newly developed smartphone application. A total of three 10-m walk tests were performed simultaneously with the study participants and professional fitness instructors to evaluate the criterion-related validity and the test-retest reliability.

Results

A strong positive correlation was found in the criterion-related validity by the study participants and professional staff for the average of the three trials {r = 0.961 [95% confidence interval (CI) = 0.927, 0.979]}. The results revealed good reliability, with an intraclass correlation coefficient of 0.712 (95% CI = 0.571, 0.823).

Conclusion

The smartphone application walking speed measurement method can be widely used by the general public and is useful for health promotion.

Detection of Major Depressive Disorder Based on a Combination of Voice Features: An Exploratory Approach

In general, it is common knowledge that people's feelings are reflected in their voice and facial expressions. This research work focuses on developing techniques for diagnosing depression based on acoustic properties of the voice. In this study, we developed a composite index of vocal acoustic properties that can be used for depression detection. Voice recordings were collected from patients undergoing outpatient treatment for major depressive disorder at a hospital or clinic following a physician's diagnosis. Numerous features were extracted from the collected audio data using openSMILE software. Furthermore, qualitatively similar features were combined using principal component analysis. The resulting components were incorporated as parameters in a logistic regression based classifier, which achieved a diagnostic accuracy of ~90% on the training set and ~80% on the test set. Lastly, the proposed metric could serve as a new measure for evaluation of major depressive disorder.

Checking the validity and reliability of the Japanese version of the Mini-Cog using a smartphone application

Objective

Cognitive decline is an important and well-documented health problem. The Mini-Cog, a simple cognitive function test, is recommended as a potential early cognitive screening tool. Kanagawa Prefecture, one of the largest prefectures in Japan, developed this self-testing application on a smartphone to enable a large number of residents to assess their cognitive function. This study aimed to verify the validity and reliability of the Mini-Cog.

Results

Twenty men and 20 women aged 65–85 years were enrolled. Criterion-related validity of the method tested by professional staff was found to have an acceptable positive association. The test–retest reliability was lower than the clinically expected intraclass correlation coefficient value because of the inclusion of learning and order effects. If the Mini-Cog score of this application is low, the system is equipped with a function that advises the users on preventing cognitive decline, directing them to the appropriate services, and recommending visits to a medical institution. Therefore, the system can be used continuously as a tool for health behaviors and promotions.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13104-022-06101-4.

Performance Evaluation of a Voice-Based Depression Assessment System Considering the Number and Type of Input Utterances

It is empirically known that mood changes affect facial expressions and voices. In this study, the authors have focused on the voice to develop a method for estimating depression in individuals from their voices. A short input voice is ideal for applying the proposed method to a wide range of applications. Therefore, we evaluated this method using multiple input utterances while assuming a unit utterance input. The experimental results revealed that depressive states could be estimated with sufficient accuracy using the smallest number of utterances when positive utterances were included in three to four input utterances.

Depressive Mood Assessment Method Based on Emotion Level Derived from Voice: Comparison of Voice Features of Individuals with Major Depressive Disorders and Healthy Controls

Background: In many developed countries, mood disorders have become problematic, and the economic loss due to treatment costs and interference with work is immeasurable. Therefore, a simple technique to determine individuals’ depressive state and stress level is desired. Methods: We developed a method to assess specific the psychological issues of individuals with major depressive disorders using emotional components contained in their voice. We propose two indices: vitality, a short-term index, and mental activity, a long-term index capturing trends in vitality. To evaluate our method, we used the voices of healthy individuals (n = 14) and patients with major depression (n = 30). The patients were also assessed by specialists using the Hamilton Rating Scale for Depression (HAM-D). Results: A significant negative correlation existed between the vitality extracted from the voices and HAM-D scores (r = −0.33, p < 0.05). Furthermore, we could discriminate the voice data of healthy individuals and patients with depression with a high accuracy using the vitality indicator (p = 0.0085, area under the curve of the receiver operating characteristic curve = 0.76).