A Deep Neural Network Based Glottal Flow Model for Predicting Fluid-Structure Interactions during Voice Production

An Improved Glottal Flow Model Based on Seq2Seq LSTM for Simulation of Vocal Fold Vibration

OBJECTIVES

An improved data-driven glottal flow model for fluid-structure interaction (FSI) simulation of the vocal fold vibration is proposed in this paper. This model aims to improve the prediction performance of the previously developed deep neural network (DNN) based empirical flow model (EFM)1 on accuracy and efficiency.

METHODS

A Seq2Seq long short-term memory (LSTM) network is employed in the present model to infer the flow rate and pressure distribution from the subglottal pressure and cross-section area distribution of the glottis. The training data is collected from the generalized glottal shape library generated in Zhang et al.1 RESULTS AND CONCLUSIONS: Compared to the EFM, the present model not only discards the time-consuming optimization process, but also drastically reduces the errors, therefore the prediction performance can be greatly improved. The present model is evaluated by coupling with a solid dynamics solver for FSI simulation, and the results demonstrate a great improvement on accuracy and efficiency.

Deep Learning for Neuromuscular Control of Vocal Source for Voice Production

A computational neuromuscular control system that generates lung pressure and three intrinsic laryngeal muscle activations (cricothyroid, thyroarytenoid, and lateral cricoarytenoid) to control the vocal source was developed. In the current study, LeTalker, a biophysical computational model of the vocal system was used as the physical plant. In the LeTalker, a three-mass vocal fold model was used to simulate self-sustained vocal fold oscillation. A constant/ǝ/vowel was used for the vocal tract shape. The trachea was modeled after MRI measurements. The neuromuscular control system generates control parameters to achieve four acoustic targets (fundamental frequency, sound pressure level, normalized spectral centroid, and signal-to-noise ratio) and four somatosensory targets (vocal fold length, and longitudinal fiber stress in the three vocal fold layers). The deep-learning-based control system comprises one acoustic feedforward controller and two feedback (acoustic and somatosensory) controllers. Fifty thousand steady speech signals were generated using the LeTalker for training the control system. The results demonstrated that the control system was able to generate the lung pressure and the three muscle activations such that the four acoustic and four somatosensory targets were reached with high accuracy. After training, the motor command corrections from the feedback controllers were minimal compared to the feedforward controller except for thyroarytenoid muscle activation.

A Deep Learning-Based Generalized Empirical Flow Model of Glottal Flow During Normal Phonation

This paper proposes a deep learning-based generalized empirical flow model (EFM) that can provide a fast and accurate prediction of the glottal flow during normal phonation. The approach is based on the assumption that the vibration of the vocal folds can be represented by a universal kinematics equation (UKE), which is used to generate a glottal shape library. For each shape in the library, the ground truth values of the flow rate and pressure distribution are obtained from the high-fidelity Navier-Stokes (N-S) solution. A fully connected deep neural network (DNN) is then trained to build the empirical mapping between the shapes and the flow rate and pressure distributions. The obtained DNN-based EFM is coupled with a finite element method (FEM)-based solid dynamics solver for fluid-structure-interaction (FSI) simulation of phonation. The EFM is evaluated by comparing the N-S solutions in both static glottal shapes and FSI simulations. The results demonstrate a good prediction performance in accuracy and efficiency.

Biomechanical Models to Represent Vocal Physiology: A Systematic Review

Biomechanical modeling allows obtaining information on physical phenomena that cannot be directly observed. This study aims to review models that represent voice production. A systematic review of the literature was conducted using PubMed/Medline, SCOPUS, and IEEE Xplore databases. To select the papers, we used the protocol PRISMA Statement. A total of 53 publications were included in this review. This article considers a taxonomic classification of models found in the literature. We propose four categories in the taxonomy: (1) Models representing the Source (Vocal folds); (2) Models representing the Filter (Vocal Tract); (3) Models representing the Source - Filter Interaction; and (4) Models representing the Airflow - Source Interaction. We include a bibliographic analysis with the evolution of the publications per category. We provide an analysis of the number as well of publications in journals per year. Moreover, we present an analysis of the term occurrence and its frequency of usage, as found in the literature. In each category, different types of vocal production models are mentioned and analyzed. The models account for the analysis of evidence about aerodynamic, biomechanical, and acoustic phenomena and their correlation with the physiological processes involved in the production of the human voice. This review gives an insight into the state of the art related to the mathematical modeling of voice production, analyzed from the viewpoint of vocal physiology.

A one-dimensional flow model enhanced by machine learning for simulation of vocal fold vibration

A one-dimensional (1D) unsteady and viscous flow model that is derived from the momentum and mass conservation equations is described, and to enhance this physics-based model, a machine learning approach is used to determine the unknown modeling parameters. Specifically, an idealized larynx model is constructed and ten cases of three-dimensional (3D) fluid-structure interaction (FSI) simulations are performed. The flow data are then extracted to train the 1D flow model using a sparse identification approach for nonlinear dynamical systems. As a result of training, we obtain the analytical expressions for the entrance effect and pressure loss in the glottis, which are then incorporated in the flow model to conveniently handle different glottal shapes due to vocal fold vibration. We apply the enhanced 1D flow model in the FSI simulation of both idealized vocal fold geometries and subject-specific anatomical geometries reconstructed from the magnetic resonance imaging images of rabbits' larynges. The 1D flow model is evaluated in both of these setups and shown to have robust performance. Therefore, it provides a fast simulation tool that is superior to the previous 1D models.

Two-Phase Flow Simulations Using 1D Centerline-Based C- and U-Shaped Pipe Meshes

This study aims to investigate the pressure changes, bubble dynamics, and flow physics inside the U- and C-shaped pipes with four different gravitational directions. The simulation is performed using a 1D centerline-based mesh generation technique along with a two-fluid model in the open-source software, OpenFOAM v.6. The continuity and momentum equations of the two-fluid model are discretized using the pressure-implicit method for the pressure-linked equation algorithm. The static and hydrostatic pressures in the two-phase flow were consistent with those of single-phase flow. The dynamic pressure in the two-phase flow was strongly influenced by the effect of the buoyancy force. In particular, if the direction of buoyancy force is the same as the flow direction, the dynamic pressure of the air phase increases, and that of the water phase decreases to satisfy the law of conservation of mass. Dean flows are observed on the transverse plane of the curve regions in both C-shaped and U-shaped pipes. The turbulent kinetic energy is stronger in a two-phase flow than in a single-phase flow. Using the 1D centerline-based mesh generation technique, we demonstrate the changes in pressure and the turbulent kinetic energy of the single- and two-phase flows, which could be observed in curve pipes.

A deep learning application to approximate the geometric orifice and coaptation areas of the polymeric heart valves under time - varying transvalvular pressure

Machine learning and deep learning frameworks have been presented as a substitute for lengthy computational analysis, such as finite element analysis, computational fluid dynamics, and fluid-structure interaction. In this study, our objective was to apply a deep learning framework to predict the geometric orifice (GOA) and the coaptation areas (CA) of the polymeric heart valves under the time-varying transvalvular pressure. 377 different valve geometries were generated by changing the control coordinates of the attachment and the belly curve. The GOA and the CA values were obtained at the maximum and the minimum transvalvular pressure, respectively. The results showed that the applied framework can accurately predict the GOA and the CA despite being trained with a relatively smaller data set. The presented framework can reduce the required time of the lengthy FE frameworks.