Automated detection and classification of basic shapes of newborn cry melody

A Narrative Review on Different Novel Machine Learning Techniques for Detecting Pathologies in Infants From Born Baby Cries

This paper reviews the research work on the analysis and classification of pathological infant cries in the last 50 years. The literature review mainly covers the need and role of early clinical diagnosis, pathologies detected from cry samples, challenges in pathological cry signal data acquisition, signal processing techniques, and signal classifiers. The signal processing techniques include preprocessing, feature extraction from domains, such as time, spectral, time-frequency, prosodic, wavelet, etc, and feature selection for selecting dominant features. Literature covers traditional machine learning classifiers, such as Bayesian networks, decision trees, K-nearest neighbor, support vector machine, Gaussian mixture model, etc, and recently added neural network models, such as convolutional neural networks, regression neural networks, probabilistic neural networks, graph neural networks, etc. Significant experimental results of pathological cry identification and classification are listed for comparison. Finally, it suggests future research in the direction of database preparation, feature analysis and extraction, neural network classifiers to provide a non-invasive and robust automatic infant cry analysis model.

Multi-modal analysis of infant cry types characterization: Acoustics, body language and brain signals

BACKGROUND

Infant crying is the first attempt babies use to communicate during their initial months of life. A misunderstanding of the cry message can compromise infant care and future neurodevelopmental process.

METHODS

An exploratory study collecting multimodal data (i.e., crying, electroencephalography (EEG), near-infrared spectroscopy (NIRS), facial expressions, and body movements) from 38 healthy full-term newborns was conducted. Cry types were defined based on different conditions (i.e., hunger, sleepiness, fussiness, need to burp, and distress). Statistical analysis, Machine Learning (ML), and Deep Learning (DL) techniques were used to identify relevant features for cry type classification and to evaluate a robust DL algorithm named Acoustic MultiStage Interpreter (AMSI).

RESULTS

Significant differences were found across cry types based on acoustics, EEG, NIRS, facial expressions, and body movements. Acoustics and body language were identified as the most relevant ML features to support the cause of crying. The DL AMSI algorithm achieved an accuracy rate of 92%.

CONCLUSIONS

This study set a precedent for cry analysis research by highlighting the complexity of newborn cry expression and strengthening the potential use of infant cry analysis as an objective, reliable, accessible, and non-invasive tool for cry interpretation, improving the infant-parent relationship and ensuring family well-being.

How can cry acoustics associate newborns' distress levels with neurophysiological and behavioral signals?

Introduction

Even though infant crying is a common phenomenon in humans' early life, it is still a challenge for researchers to properly understand it as a reflection of complex neurophysiological functions. Our study aims to determine the association between neonatal cry acoustics with neurophysiological signals and behavioral features according to different cry distress levels of newborns.

Methods

Multimodal data from 25 healthy term newborns were collected simultaneously recording infant cry vocalizations, electroencephalography (EEG), near-infrared spectroscopy (NIRS) and videos of facial expressions and body movements. Statistical analysis was conducted on this dataset to identify correlations among variables during three different infant conditions (i.e., resting, cry, and distress). A Deep Learning (DL) algorithm was used to objectively and automatically evaluate the level of cry distress in infants.

Results

We found correlations between most of the features extracted from the signals depending on the infant's arousal state, among them: fundamental frequency (F0), brain activity (delta, theta, and alpha frequency bands), cerebral and body oxygenation, heart rate, facial tension, and body rigidity. Additionally, these associations reinforce that what is occurring at an acoustic level can be characterized by behavioral and neurophysiological patterns. Finally, the DL audio model developed was able to classify the different levels of distress achieving 93% accuracy.

Conclusion

Our findings strengthen the potential of crying as a biomarker evidencing the physical, emotional and health status of the infant becoming a crucial tool for caregivers and clinicians.

A Machine Learning Approach to Classify Biomedical Acoustic Features for Baby Cries

Communication is imperative for living beings for exchanging information. But for newborns, the only way of communicating with the world is through crying, and it is the only medium through which caregivers can know about the needs of their children. Timely addressing baby cries is very important so that the child is relieved at the earliest. It has been a challenge, especially for new parents. The literature says newborn babies use The Dustan Baby Language to communicate. According to this language, there are five words to understand a baby's needs, which are "Neh" (hungry), "Eh" (burp is needed), "Owh/Oah" (fatigue), "Eair/Eargghh" (cramps), "Heh" (feel hot or wet, physical discomfort). This research aims to develop a model for recognizing baby cries and distinguishing between different kinds of baby cries. Here we more broadly focus on whether the infant is in pain due to hunger or discomfort. The study proposes a comparative approach using four classification models: random forest, support vector machine, logistic regression, and decision tree. These algorithms learn from the spectral features: chroma_stft, spectral_centroid, bandwidth, spectral_rolloff, mel-frequency cepstral coefficients, linear predictive coding, res, zero_crossing_rate extracted from the infant cry. The support vector machine model outperforms other classifiers for correctly classifying infant cries.

Machine Learning Assessment of Spasmodic Dysphonia Based on Acoustical and Perceptual Parameters

Adductor spasmodic dysphonia is a type of adult-onset focal dystonia characterized by involuntary spasms of laryngeal muscles. This paper applied machine learning techniques for the severity assessment of spasmodic dysphonia. To this aim, 7 perceptual indices and 48 acoustical parameters were estimated from the Italian word /a’jwɔle/ emitted by 28 female patients, manually segmented from a standardized sentence and used as features in two classification experiments. Subjects were divided into three severity classes (mild, moderate, severe) on the basis of the G (grade) score of the GRB scale. The first aim was that of finding relationships between perceptual and objective measures with the Local Interpretable Model-Agnostic Explanations method. Then, the development of a diagnostic tool for adductor spasmodic dysphonia severity assessment was investigated. Reliable relationships between G; R (Roughness); B (Breathiness); Spasmodicity; and the acoustical parameters: voiced percentage, F2 median, and F1 median were found. After data scaling, Bayesian hyperparameter optimization, and leave-one-out cross-validation, a k-nearest neighbors model provided 89% accuracy in distinguishing patients among the three severity classes. The proposed methods highlighted the best acoustical parameters that could be used jointly with GRB indices to support the perceptual evaluation of spasmodic dysphonia and provide a tool to help severity assessment of spasmodic dysphonia.

Minor second intervals: A shared signature for infant cries and sadness in music

In Western music and in music of other cultures, minor chords, modes and intervals evoke sadness. It has been proposed that this emotional interpretation of melodic intervals (the distance between two pitches, expressed in semitones) is common to music and vocal expressions. Here, we asked expert musicians to transcribe into music scores spontaneous vocalizations of pre-verbal infants to test the hypothesis that melodic intervals that evoke sadness in music (i.e., minor 2nd) are more represented in cry compared to neutral utterances. Results showed that the unison, major 2nd, minor 2nd, major 3rd, minor 3rd, perfect 4th and perfect 5th are all represented in infant vocalizations. However, minor 2nd outnumbered all other intervals in cry vocalizations, but not in neutral babbling. These findings suggest that the association between minor intervals and sadness may develop in humans because a critically relevant social cue (infant cry) contains a statistical regularity: the association between minor 2nd and negative emotional valence.

Extraction of Premature Newborns' Spontaneous Cries in the Real Context of Neonatal Intensive Care Units

Cry analysis is an important tool to evaluate the development of preterm infants. However, the context of Neonatal Intensive Care Units is challenging, since a wide variety of sounds can occur (e.g., alarms and adult voices). In this paper, a method to extract cries is proposed. It is based on an initial segmentation between silence and sound events, followed by feature extraction on the resulting audio segments and a cry and non-cry classification. A database of 198 cry events coming from 21 newborns and 439 non-cry events was created. Then, a set of features—including Mel-Frequency Cepstral Coefficients—issued from principal component analysis, was computed to describe each audio segment. For the first time in cry analysis, noise was handled using harmonic plus noise analysis. Several machine learning models have been compared. The K-Nearest Neighbours approach showed the best results with a precision of 92.9%. To test the approach in a monitoring application, 412 h of recordings were automatically processed. The cries automatically selected were replayed and a precision of 92.2% was obtained. The impact of errors on the fundamental frequency characterisation was also studied. Results show that despite a difficult context, automatic cry extraction for non-invasive monitoring of vocal development of preterm infants is achievable.

Machine Learning-Based Cry Diagnostic System for Identifying Septic Newborns

BACKGROUND AND OBJECTIVE

Processing the newborns' cry audio signal (CAS) provides valuable information about the newborns' condition. This information can be used to diagnose the disease. This article analyzes the CASs of newborns under two months old using machine learning approaches to develop an automatic diagnostic system for identifying septic infants from healthy ones. Septic infants have not been studied in this context.

METHODOLOGY

The proposed features include Mel frequency cepstral coefficients and the prosodic features of tilt, rhythm, and intensity. The performance of each feature set was evaluated using a collection of classifiers, including Support Vector Machine (SVM), decision tree, and discriminant analysis. We also examined the majority voting method for improving the classification results and feature manipulation and multiple classifier framework, which has not previously been reported in the literature on developing an automatic diagnostic system based on the infant's CAS. We tested our methodology on two datasets of expiration and inspiration episodes of newborns' CASs.

RESULTS AND CONCLUSION

The framework of the concatenation of all feature sets using quadratic SVM resulted in the best F-score with 86% for the expiration dataset. Furthermore, the framework of tilt feature set with quadratic discriminant with 83.90% resulted in the best F-score for inspiration. We found out that septic infants cry differently than healthy infants through these experiments. Thus, our proposed method can be used as a noninvasive tool for identifying septic infants from healthy ones only based on their CAS.

Identification of Diseases in Newborns Using Advanced Acoustic Features of Cry Signals

Our challenge in the current study is to extend research on the cries of newborns for the early diagnosis of different pathologies. This paper proposes a recognition system for healthy and pathological cries using a probabilistic neural network classifier. Two different kinds of features have been used to characterize newborn cry signals: 1) acoustic features such as fundamental frequency glide (F0glide) and resonance frequencies dysregulation (RFsdys); 2) conventional features such as mel-frequency cestrum coefficients. This paper describes the automatic estimation of the proposed characteristics and the performance evaluation of these features in identifying pathological cries. The adopted methods for F0glides and RFsdys estimation are based on the derived function of the F0 contour and the jump "J" of the RFs between two subsequent tunings, respectively. The database used contains 3250 cry samples of full-term and preterm newborns, and includes healthy and pathologic cries. The obtained results indicate the important association between the quantified features and some studied pathologies, and also an improvement in the identification of pathologic cries. The best result obtained is 88.71% for the correct identification of health status of preterm newborns, and 82% for the correct identification of full-term infants with a specific disease. We conclude that using the proposed characteristics improves the diagnosis of pathologies in newborns. Moreover, the method applied in the estimation of these characteristics allows us to extend this study to other uninvestigated pathologies.

Video and audio processing in paediatrics: a review

OBJECTIVE

Video and sound acquisition and processing technologies have seen great improvements in recent decades, with many applications in the biomedical area. The aim of this paper is to review the overall state of the art of advances within these topics in paediatrics and to evaluate their potential application for monitoring in the neonatal intensive care unit (NICU).

APPROACH

For this purpose, more than 150 papers dealing with video and audio processing were reviewed. For both topics, clinical applications are described according to the considered cohorts-full-term newborns, infants and toddlers or preterm newborns. Then, processing methods are presented, in terms of data acquisition, feature extraction and characterization.

MAIN RESULTS

The paper first focuses on the exploitation of video recordings; these began to be automatically processed in the 2000s and we show that they have mainly been used to characterize infant motion. Other applications, including respiration and heart rate estimation and facial analysis, are also presented. Audio processing is then reviewed, with a focus on the analysis of crying. The first studies in this field focused on induced-pain cries and the newest ones deal with spontaneous cries; the analyses are mainly based on frequency features. Then, some papers dealing with non-cry signals are also discussed.

SIGNIFICANCE

Finally, we show that even if recent improvements in digital video and signal processing allow for increased automation of processing, the context of the NICU makes a fully automated analysis of long recordings problematic. A few proposals for overcoming some of the limitations are given.