Vibration parameter extraction from endoscopic image series of the vocal folds

Neural network-based estimation of biomechanical vocal fold parameters

Vocal fold (VF) vibrations are the primary source of human phonation. High-speed video (HSV) endoscopy enables the computation of descriptive VF parameters for assessment of physiological properties of laryngeal dynamics, i.e., the vibration of the VFs. However, underlying biomechanical factors responsible for physiological and disordered VF vibrations cannot be accessed. In contrast, physically based numerical VF models reveal insights into the organ's oscillations, which remain inaccessible through endoscopy. To estimate biomechanical properties, previous research has fitted subglottal pressure-driven mass-spring-damper systems, as inverse problem to the HSV-recorded VF trajectories, by global optimization of the numerical model. A neural network trained on the numerical model may be used as a substitute for computationally expensive optimization, yielding a fast evaluating surrogate of the biomechanical inverse problem. This paper proposes a convolutional recurrent neural network (CRNN)-based architecture trained on regression of a physiological-based biomechanical six-mass model (6 MM). To compare with previous research, the underlying biomechanical factor "subglottal pressure" prediction was tested against 288 HSV ex vivo porcine recordings. The contributions of this work are two-fold: first, the presented CRNN with the 6 MM handles multiple trajectories along the VFs, which allows for investigations on local changes in VF characteristics. Second, the network was trained to reproduce further important biomechanical model parameters like VF mass and stiffness on synthetic data. Unlike in a previous work, the network in this study is therefore an entire surrogate of the inverse problem, which allowed for explicit computation of the fitted model using our approach. The presented approach achieves a best-case mean absolute error (MAE) of 133 Pa (13.9%) in subglottal pressure prediction with 76.6% correlation on experimental data and a re-estimated fundamental frequency MAE of 15.9 Hz (9.9%). In-detail training analysis revealed subglottal pressure as the most learnable parameter. With the physiological-based model design and advances in fast parameter prediction, this work is a next step in biomechanical VF model fitting and the estimation of laryngeal kinematics.

Voice Feature Selection to Improve Performance of Machine Learning Models for Voice Production Inversion

OBJECTIVE

Estimation of physiological control parameters of the vocal system from the produced voice outcome has important applications in clinical management of voice disorders . Previously we developed a simulation-based neural network for estimation of vocal fold geometry, mechanical properties, and subglottal pressure from voice outcome features that characterize the acoustics of the produced voice. The goals of this study are to (1) explore the possibility of improving the estimation accuracy of physiological control parameters by including voice outcome features characterizing vocal fold vibration; and (2) identify voice feature sets that optimize both estimation accuracy and robustness to measurement noise.

METHODS

Feedforward neural networks are trained to solve the inversion problem of estimating the physiological control parameters of a three-dimensional body-cover vocal fold model from different sets of voice outcome features that characterize the simulated voice acoustics, glottal flow, and vocal fold vibration. A sensitivity analysis is then performed to evaluate the contribution of individual voice features to the overall performance of the neural networks in estimating the physiologic control parameters.

RESULTS AND CONCLUSIONS

While including voice outcome features characterizing vocal fold vibration increases estimation accuracy, it also reduces the network's robustness to measurement noise, due to high sensitivity of network performance to voice outcome features measuring the absolute amplitudes of the glottal flow and area waveforms, which are also difficult to measure accurately in practical applications. By excluding such glottal flow-based features and replacing glottal area-based features by their normalized counterparts, we are able to significantly improve both estimation accuracy and robustness to noise. We further show that similar estimation accuracy and robustness can be achieved with an even smaller set of voice outcome features by excluding features of small sensitivity.

Long-term performance assessment of fully automatic biomedical glottis segmentation at the point of care

Deep Learning has a large impact on medical image analysis and lately has been adopted for clinical use at the point of care. However, there is only a small number of reports of long-term studies that show the performance of deep neural networks (DNNs) in such an environment. In this study, we measured the long-term performance of a clinically optimized DNN for laryngeal glottis segmentation. We have collected the video footage for two years from an AI-powered laryngeal high-speed videoendoscopy imaging system and found that the footage image quality is stable across time. Next, we determined the DNN segmentation performance on lossy and lossless compressed data revealing that only 9% of recordings contain segmentation artifacts. We found that lossy and lossless compression is on par for glottis segmentation, however, lossless compression provides significantly superior image quality. Lastly, we employed continual learning strategies to continuously incorporate new data into the DNN to remove the aforementioned segmentation artifacts. With modest manual intervention, we were able to largely alleviate these segmentation artifacts by up to 81%. We believe that our suggested deep learning-enhanced laryngeal imaging platform consistently provides clinically sound results, and together with our proposed continual learning scheme will have a long-lasting impact on the future of laryngeal imaging.

The influence of flow model selection on finite element model parameter estimation using Bayesian inference

Recently, Bayesian estimation coupled with finite element modeling has been demonstrated as a viable tool for estimating vocal fold material properties from kinematic information obtained via high-speed video recordings. In this article, the sensitivity of the parameter estimations to the employed fluid model is explored by considering Bernoulli and one-dimensional viscous fluid flow models. Simulation results indicate that prescribing an ad hoc separation location for the Bernoulli flow model can lead to large estimate biases, whereas including the separation location as an estimated parameter leads to results comparable to that of the viscous fluid flow model.

Method for Horizontal Calibration of Laser-Projection Transnasal Fiberoptic High-Speed Videoendoscopy

OBJECTIVE

Calibrated horizontal measurements (e.g., mm) from endoscopic procedures could be utilized for advancement of evidence-based practice and personalized medicine. However, the size of an object in endoscopic images is not readily calibrated and depends on multiple factors, including the distance between the endoscope and the target surface. Additionally, acquired images may have significant non-linear distortion that would further complicate calibrated measurements. This study used a recently developed in-vivo laser-projection fiberoptic laryngoscope and proposes a method for calibrated spatial measurements.

METHOD

A set of circular grids were recorded at multiple working distances. A statistical model was trained that would map from pixel length of the object, the working distance, and the spatial location of the target object into its mm length.

RESULT

A detailed analysis of the performance of the proposed method is presented. The analyses have shown that the accuracy of the proposed method does not depend on the working distance and length of the target object. The estimated average magnitude of error was 0.27 mm, which is three times lower than the existing alternative.

CONCLUSION

The presented method can achieve sub-millimeter accuracy in horizontal measurement.

SIGNIFICANCE

Evidence-based practice and personalized medicine could significantly benefit from the proposed method. Implications of the findings for other endoscopic procedures are also discussed.

Rethinking glottal midline detection

A healthy voice is crucial for verbal communication and hence in daily as well as professional life. The basis for a healthy voice are the sound producing vocal folds in the larynx. A hallmark of healthy vocal fold oscillation is the symmetric motion of the left and right vocal fold. Clinically, videoendoscopy is applied to assess the symmetry of the oscillation and evaluated subjectively. High-speed videoendoscopy, an emerging method that allows quantification of the vocal fold oscillation, is more commonly employed in research due to the amount of data and the complex, semi-automatic analysis. In this study, we provide a comprehensive evaluation of methods that detect fully automatically the glottal midline. We used a biophysical model to simulate different vocal fold oscillations, extended the openly available BAGLS dataset using manual annotations, utilized both, simulations and annotated endoscopic images, to train deep neural networks at different stages of the analysis workflow, and compared these to established computer vision algorithms. We found that classical computer vision perform well on detecting the glottal midline in glottis segmentation data, but are outperformed by deep neural networks on this task. We further suggest GlottisNet, a multi-task neural architecture featuring the simultaneous prediction of both, the opening between the vocal folds and the symmetry axis, leading to a huge step forward towards clinical applicability of quantitative, deep learning-assisted laryngeal endoscopy, by fully automating segmentation and midline detection.

BAGLS, a multihospital Benchmark for Automatic Glottis Segmentation

Laryngeal videoendoscopy is one of the main tools in clinical examinations for voice disorders and voice research. Using high-speed videoendoscopy, it is possible to fully capture the vocal fold oscillations, however, processing the recordings typically involves a time-consuming segmentation of the glottal area by trained experts. Even though automatic methods have been proposed and the task is particularly suited for deep learning methods, there are no public datasets and benchmarks available to compare methods and to allow training of generalizing deep learning models. In an international collaboration of researchers from seven institutions from the EU and USA, we have created BAGLS, a large, multihospital dataset of 59,250 high-speed videoendoscopy frames with individually annotated segmentation masks. The frames are based on 640 recordings of healthy and disordered subjects that were recorded with varying technical equipment by numerous clinicians. The BAGLS dataset will allow an objective comparison of glottis segmentation methods and will enable interested researchers to train their own models and compare their methods.

Estimation of vocal fold physiology from voice acoustics using machine learning

The goal of this study is to estimate vocal fold geometry, stiffness, position, and subglottal pressure from voice acoustics, toward clinical and other voice technology applications. Unlike previous voice inversion research that often uses lumped-element models of phonation, this study explores the feasibility of voice inversion using data generated from a three-dimensional voice production model. Neural networks are trained to estimate vocal fold properties and subglottal pressure from voice features extracted from the simulation data. Results show reasonably good estimation accuracy, particularly for vocal fold properties with a consistent global effect on voice production, and reasonable agreement with excised human larynx experiment.

Laryngeal Image Processing of Vocal Folds Motion

This review provides a comprehensive compilation, from a digital image processing point of view of the most important techniques currently developed to characterize and quantify the vibration behaviour of the vocal folds, along with a detailed description of the laryngeal image modalities currently used in the clinic. The review presents an overview of the most significant glottal-gap segmentation and facilitative playbacks techniques used in the literature for the mentioned purpose, and shows the drawbacks and challenges that still remain unsolved to develop robust vocal folds vibration function analysis tools based on digital image processing.

A Deep Neural Network Based Glottal Flow Model for Predicting Fluid-Structure Interactions during Voice Production

This paper proposes a machine-learning based reduced-order model that can provide fast and accurate prediction of the glottal flow during voice production. The model is based on the Bernoulli equation with a viscous loss term predicted by a deep neural network (DNN) model. The training data of the DNN model is a Navier-Stokes (N-S) equation-based three-dimensional simulation of glottal flows in various glottal shapes generated by a synthetic shape function, which can be obtained by superimposing the instantaneous modal displacements during vibration on the prephonatory geometry of the glottal shape. The input parameters of the DNN model are the geometric and flow parameters extracted from discretized cross sections of the glottal shapes and the output target is the corresponding flow resistance coefficient. With this trained DNN-Bernoulli model, the flow resistance coefficient as well as the flow rate and pressure distribution in any given glottal shape generated by the synthetic shape function can be predicted. The model is further coupled with a finite-element method based solid dynamics solver for simulating fluid-structure interactions (FSI). The prediction performance of the model for both static shape and FSI simulations is evaluated by comparing the solutions to those obtained by the Bernoulli and N-S model. The model shows a good prediction performance in accuracy and efficiency, suggesting a promise for future clinical use.

The effect of high-speed videoendoscopy configuration on reduced-order model parameter estimates by Bayesian inference

Bayesian inference has been previously demonstrated as a viable inverse analysis tool for estimating subject-specific reduced-order model parameters and uncertainties. However, previous studies have relied upon simulated glottal area waveforms with superimposed random noise as the measurement. In practice, high-speed videoendoscopy is used to measure glottal area, which introduces practical imaging effects not captured in simulated data, such as viewing angle, frame rate, and camera resolution. Herein, high-speed videos of the vocal folds were approximated by recording the trajectories of physical vocal fold models controlled by a symmetric body-cover model. Twenty videos were recorded, varying subglottal pressure, cricothyroid activation, and viewing angle, with frame rate and video resolution varied by digital video manipulation. Bayesian inference was used to estimate subglottal pressure and cricothyroid activation from glottal area waveforms extracted from the videos. The resulting estimates show off-axis viewing of 10° can lead to a 10% bias in the estimated subglottal pressure. A viewing model is introduced such that viewing angle can be included as an estimated parameter, which alleviates estimate bias. Frame rate and pixel resolution were found to primarily affect uncertainty of parameter estimates up to a limit where spatial and temporal resolutions were too poor to resolve the glottal area. Since many high-speed cameras have the ability to sacrifice spatial for temporal resolution, the findings herein suggest that Bayesian inference studies employing high-speed video should increase temporal resolutions at the expense of spatial resolution for reduced estimate uncertainties.

The effect of high-speed videoendoscopy configuration on reduced-order model parameter estimates by Bayesian inference

Bayesian inference has been previously demonstrated as a viable inverse analysis tool for estimating subject-specific reduced-order model parameters and uncertainties. However, previous studies have relied upon simulated glottal area waveforms with superimposed random noise as the measurement. In practice, high-speed videoendoscopy is used to measure glottal area, which introduces practical imaging effects not captured in simulated data, such as viewing angle, frame rate, and camera resolution. Herein, high-speed videos of the vocal folds were approximated by recording the trajectories of physical vocal fold models controlled by a symmetric body-cover model. Twenty videos were recorded, varying subglottal pressure, cricothyroid activation, and viewing angle, with frame rate and video resolution varied by digital video manipulation. Bayesian inference was used to estimate subglottal pressure and cricothyroid activation from glottal area waveforms extracted from the videos. The resulting estimates show off-axis viewing of 10° can lead to a 10% bias in the estimated subglottal pressure. A viewing model is introduced such that viewing angle can be included as an estimated parameter, which alleviates estimate bias. Frame rate and pixel resolution were found to primarily affect uncertainty of parameter estimates up to a limit where spatial and temporal resolutions were too poor to resolve the glottal area. Since many high-speed cameras have the ability to sacrifice spatial for temporal resolution, the findings herein suggest that Bayesian inference studies employing high-speed video should increase temporal resolutions at the expense of spatial resolution for reduced estimate uncertainties.

Bayesian Inference of Vocal Fold Material Properties from Glottal Area Waveforms Using a 2D Finite Element Model

Bayesian estimation has been previously demonstrated as a viable method for developing subject-specific vocal fold models from observations of the glottal area waveform. These prior efforts, however, have been restricted to lumped-element fitting models and synthetic observation data. The indirect relationship between the lumped-element parameters and physical tissue properties renders extracting the latter from the former difficult. Herein we propose a finite element fitting model, which treats the vocal folds as a viscoelastic deformable body comprised of three layers. Using the glottal area waveforms generated by self-oscillating silicone vocal folds we directly estimate the elastic moduli, density, and other material properties of the silicone folds using a Bayesian importance sampling approach. Estimated material properties agree with the “ground truth” experimental values to within 3% for most parameters. By considering cases with varying subglottal pressure and medial compression we demonstrate that the finite element model coupled with Bayesian estimation is sufficiently sensitive to distinguish between experimental configurations. Additional information not available experimentally, namely, contact pressures, are extracted from the developed finite element models. The contact pressures are found to increase with medial compression and subglottal pressure, in agreement with expectation.

The relationship between biomechanics of pharyngoesophageal segment and tracheoesophageal phonation

This study examined the relationship between biomechanical features of the pharyngoesophageal (PE) segment, acoustic characteristics of tracheoesophageal (TE) phonation, and patients’ satisfaction with TE phonation. Fifteen patients using TE phonation after total laryngectomy completed the Voice Symptom Scale (VoiSS) and underwent acoustic voice analysis for cepstral peak prominence (CPP) and relative intensity. High resolution manometry (HRM) combined with videofluoroscopy was used to evaluate PE segment pressure and calculate the pressure gradient (ΔP), which was the pressure difference between the upper oesophagus and a point two centimetres above the vibrating PE segment. The upper oesophageal sphincter (UOS) minimal diameters were measured by Endolumenal Functional Lumen Imaging Probe (EndoFLIP). HRM detected rapid pressure changes at the level of the 4th – 6th cervical vertebra. CPP, relative intensity, and ΔP were significant predictors of satisfactory TE phonation. ΔP was a significant predictor of CPP and intensity. Minimal UOS diameter was a significant predictor of relative intensity of TE phonation. In two patients with unsuccessful TE phonation, endoscopic dilatation subsequently restored TE phonation. These findings suggest that sufficient ΔP and large UOS diameter are required for satisfactory TE phonation. Endoscopic dilatation increasing UOS diameter may provide a new approach to treat unsuccessful TE phonation.

Estimating Vocal Fold Contact Pressure from Raw Laryngeal High-Speed Videoendoscopy Using a Hertz Contact Model

The development of trauma-induced lesions of the vocal folds (VFs) has been linked to a high collision pressure on the VF surface. However, there are no direct methods for the clinical assessment of VF collision, thus limiting the objective assessment of these disorders. In this study, we develop a video processing technique to directly quantify the mechanical impact of the VFs using solely laryngeal kinematic data. The technique is based on an edge tracking framework that estimates the kinematic sequence of each VF edge with a Kalman filter approach and a Hertzian impact model to predict the contact force during the collision. The proposed formulation overcomes several limitations of prior efforts since it uses a more relevant VF contact geometry, it does not require calibrated physical dimensions, it is normalized by the tissue properties, and it applies a correction factor for using a superior view only. The proposed approach is validated against numerical models, silicone vocal fold models, and prior studies. A case study with high-speed videoendoscopy recordings provides initial insights between the sound pressure level and contact pressure. Thus, the proposed method has a high potential in clinical practice and could also be adapted to operate with laryngeal stroboscopic systems.

A method to directly estimate the contact pressure of the vocal folds using uncalibrated laryngeal kinematic data is presented. The approach is promising in enhancing the objective assessment of vocal function in clinical settings, especially for studying same-subject variations.

Neurophysiological Muscle Activation Scheme for Controlling Vocal Fold Models

A physiologically-based scheme that incorporates inherent neurological fluctuations in the activation of intrinsic laryngeal muscles into a lumped-element vocal fold model is proposed. Herein, muscles are activated through a combination of neural firing rate and recruitment of additional motor units, both of which have stochastic components. The mathematical framework and underlying physiological assumptions are described, and the effects of the fluctuations are tested via a parametric analysis using a body-cover model of the vocal folds for steady-state sustained vowels. The inherent muscle activation fluctuations have a bandwidth that varies with the firing rate, yielding both low and high-frequency components. When applying the proposed fluctuation scheme to the voice production model, changes in the dynamics of the system can be observed, ranging from fluctuations in the fundamental frequency to unstable behavior near bifurcation regions. The resulting coefficient of variation of the model parameters is not uniform with muscle activation. The stochastic components of muscle activation influence both the fine structure variability and the ability to achieve a target value for pitch control. These components can have a significant impact on the vocal fold parameters, as well as the outputs of the voice production model. Good agreement was found when contrasting the proposed scheme with prior experimental studies accounting for variability in vocal fold posturing and spectral characteristics of the muscle activation signal. The proposed scheme constitutes a novel and physiologically-based approach for controlling lumped-element models for normal voice production and can be extended to explore neuropathological conditions.

Laryngeal Pressure Estimation With a Recurrent Neural Network

Quantifying the physical parameters of voice production is essential for understanding the process of phonation and can aid in voice research and diagnosis. As an alternative to invasive measurements, they can be estimated by formulating an inverse problem using a numerical forward model. However, high-fidelity numerical models are often computationally too expensive for this. This paper presents a novel approach to train a long short-term memory network to estimate the subglottal pressure in the larynx at massively reduced computational cost using solely synthetic training data. We train the network on synthetic data from a numerical two-mass model and validate it on experimental data from 288 high-speed ex vivo video recordings of porcine vocal folds from a previous study. The training requires significantly fewer model evaluations compared with the previous optimization approach. On the test set, we maintain a comparable performance of 21.2% versus previous 17.7% mean absolute percentage error in estimating the subglottal pressure. The evaluation of one sample requires a vanishingly small amount of computation time. The presented approach is able to maintain estimation accuracy of the subglottal pressure at significantly reduced computational cost. The methodology is likely transferable to estimate other parameters and training with other numerical models. This improvement should allow the adoption of more sophisticated, high-fidelity numerical models of the larynx. The vast speedup is a critical step to enable a future clinical application and knowledge of parameters such as the subglottal pressure will aid in diagnosis and treatment selection.

Pathological Voice Source Analysis System Using a Flow Waveform-Matched Biomechanical Model

Voice production occurs through vocal cord and vibration coupled to glottal airflow. Vocal cord lesions affect the vocal system and lead to voice disorders. In this paper, a pathological voice source analysis system is designed. This study integrates nonlinear dynamics with an optimized asymmetric two-mass model to explore nonlinear characteristics of vocal cord vibration, and changes in acoustic parameters, such as fundamental frequency, caused by distinct subglottal pressure and varying degrees of vocal cord paralysis are analyzed. Various samples of sustained vowel /a/ of normal and pathological voices were extracted from MEEI (Massachusetts Eye and Ear Infirmary) database. A fitting procedure combining genetic particle swarm optimization and a quasi-Newton method was developed to optimize the biomechanical model parameters and match the targeted voice source. Experimental results validate the applicability of the proposed model to reproduce vocal cord vibration with high accuracy, and show that paralyzed vocal cord increases the model coupling stiffness.

Biomechanical simulation of vocal fold dynamics in adults based on laryngeal high-speed videoendoscopy

MOTIVATION

Human voice is generated in the larynx by the two oscillating vocal folds. Owing to the limited space and accessibility of the larynx, endoscopic investigation of the actual phonatory process in detail is challenging. Hence the biomechanics of the human phonatory process are still not yet fully understood. Therefore, we adapt a mathematical model of the vocal folds towards vocal fold oscillations to quantify gender and age related differences expressed by computed biomechanical model parameters.

METHODS

The vocal fold dynamics are visualized by laryngeal high-speed videoendoscopy (4000 fps). A total of 33 healthy young subjects (16 females, 17 males) and 11 elderly subjects (5 females, 6 males) were recorded. A numerical two-mass model is adapted to the recorded vocal fold oscillations by varying model masses, stiffness and subglottal pressure. For adapting the model towards the recorded vocal fold dynamics, three different optimization algorithms (Nelder-Mead, Particle Swarm Optimization and Simulated Bee Colony) in combination with three cost functions were considered for applicability. Gender differences and age-related kinematic differences reflected by the model parameters were analyzed.

RESULTS AND CONCLUSION

The biomechanical model in combination with numerical optimization techniques allowed phonatory behavior to be simulated and laryngeal parameters involved to be quantified. All three optimization algorithms showed promising results. However, only one cost function seems to be suitable for this optimization task. The gained model parameters reflect the phonatory biomechanics for men and women well and show quantitative age- and gender-specific differences. The model parameters for younger females and males showed lower subglottal pressures, lower stiffness and higher masses than the corresponding elderly groups. Females exhibited higher subglottal pressures, smaller oscillation masses and larger stiffness than the corresponding similar aged male groups. Optimizing numerical models towards vocal fold oscillations is useful to identify underlying laryngeal components controlling the phonatory process.

Modeling the Pathophysiology of Phonotraumatic Vocal Hyperfunction With a Triangular Glottal Model of the Vocal Folds

PURPOSE

Our goal was to test prevailing assumptions about the underlying biomechanical and aeroacoustic mechanisms associated with phonotraumatic lesions of the vocal folds using a numerical lumped-element model of voice production.

METHOD

A numerical model with a triangular glottis, posterior glottal opening, and arytenoid posturing is proposed. Normal voice is altered by introducing various prephonatory configurations. Potential compensatory mechanisms (increased subglottal pressure, muscle activation, and supraglottal constriction) are adjusted to restore an acoustic target output through a control loop that mimics a simplified version of auditory feedback.

RESULTS

The degree of incomplete glottal closure in both the membranous and posterior portions of the folds consistently leads to a reduction in sound pressure level, fundamental frequency, harmonic richness, and harmonics-to-noise ratio. The compensatory mechanisms lead to significantly increased vocal-fold collision forces, maximum flow-declination rate, and amplitude of unsteady flow, without significantly altering the acoustic output.

CONCLUSION

Modeling provided potentially important insights into the pathophysiology of phonotraumatic vocal hyperfunction by demonstrating that compensatory mechanisms can counteract deterioration in the voice acoustic signal due to incomplete glottal closure, but this also leads to high vocal-fold collision forces (reflected in aerodynamic measures), which significantly increases the risk of developing phonotrauma.

An optical flow-based state-space model of the vocal folds

High-speed movies of the vocal fold vibration are valuable data to reveal vocal fold features for voice pathology diagnosis. This work presents a suitable Bayesian model and a purely theoretical discussion for further development of a framework for continuum biomechanical features estimation. A linear and Gaussian nonstationary state-space model is proposed and thoroughly discussed. The evolution model is based on a self-sustained three-dimensional finite element model of the vocal folds, and the observation model involves a dense optical flow algorithm. The results show that the method is able to capture different deformation patterns between the computed optical flow and the finite element deformation, controlled by the choice of the model tissue parameters.

An extended Kalman filter approach to non-stationary Bayesian estimation of reduced-order vocal fold model parameters

The Bayesian framework for parameter inference provides a basis from which subject-specific reduced-order vocal fold models can be generated. Previously, it has been shown that a particle filter technique is capable of producing estimates and associated credibility intervals of time-varying reduced-order vocal fold model parameters. However, the particle filter approach is difficult to implement and has a high computational cost, which can be barriers to clinical adoption. This work presents an alternative estimation strategy based upon Kalman filtering aimed at reducing the computational cost of subject-specific model development. The robustness of this approach to Gaussian and non-Gaussian noise is discussed. The extended Kalman filter (EKF) approach is found to perform very well in comparison with the particle filter technique at dramatically lower computational cost. Based upon the test cases explored, the EKF is comparable in terms of accuracy to the particle filter technique when greater than 6000 particles are employed; if less particles are employed, the EKF actually performs better. For comparable levels of accuracy, the solution time is reduced by 2 orders of magnitude when employing the EKF. By virtue of the approximations used in the EKF, however, the credibility intervals tend to be slightly underpredicted.

Evaluation of the combined use of narrow band imaging and high-speed imaging to discriminate laryngeal lesions

BACKGROUND AND OBJECTIVE

Laryngeal lesions are usually investigated by microlaryngoscopy, biopsy, and histopathology. This study aimed to evaluate the combined use of Narrow Band Imaging (NBI) and High-Speed Imaging (HSI) in the differentiation of glottic lesions in awake patients.

STUDY DESIGN

Prospective diagnostic study.

MATERIALS AND METHODS

Thirty-six awake patients with 41 glottic lesions were investigated with both NBI and HSI, and the suspected diagnoses were compared to the histopathological results of tissue biopsies taken during subsequent microlaryngoscopies. Of the 41 lesions, 28 were primary lesions and 13 recurrent lesions after previous laryngeal pathologies.

RESULTS

Sensitivity, specificity, positive predictive value, and negative predictive value in the differentiation between benign/premalignant and malignant lesions with both NBI and HSI accounted to 100.0%, 79.4%, 50.0%, and 100.0%. Sensitivities and specificities were 100.0% and 85.7% for HSI alone, and 100.0% and 79.4% for NBI alone. Regarding only primary lesions the results were generally better with sensitivities and specificities of 100% and 81% for NBI, 100% and 84.2% for HSI and 100% and 85.7% for the combination of both methods, respectively.

CONCLUSION

NBI and HSI both seem to be promising adjunct tools in the differentiation of various laryngeal lesions in awake patients with high sensitivities. Specificities, however, were moderate but could be increased when using NBI and HSI in combination in a subgroup of patients with only primary lesions. Although both methods still have limitations they might ameliorate the evaluation of suspicious laryngeal lesions in the future and could possibly spare patients from repeated invasive tissue biopsies. Lasers Surg. Med. 49:609-618, 2017. © 2017 Wiley Periodicals, Inc.

Mechanics of human voice production and control

As the primary means of communication, voice plays an important role in daily life. Voice also conveys personal information such as social status, personal traits, and the emotional state of the speaker. Mechanically, voice production involves complex fluid-structure interaction within the glottis and its control by laryngeal muscle activation. An important goal of voice research is to establish a causal theory linking voice physiology and biomechanics to how speakers use and control voice to communicate meaning and personal information. Establishing such a causal theory has important implications for clinical voice management, voice training, and many speech technology applications. This paper provides a review of voice physiology and biomechanics, the physics of vocal fold vibration and sound production, and laryngeal muscular control of the fundamental frequency of voice, vocal intensity, and voice quality. Current efforts to develop mechanical and computational models of voice production are also critically reviewed. Finally, issues and future challenges in developing a causal theory of voice production and perception are discussed.

Videokymography in Voice Disorders: What to Look For?

Objectives:

Kymographic imaging through videokymography has been recognized as a convenient, novel way to display laryngeal behavior, yet little systematic research has been done to map the relevant features displayed in such images. Here we have aimed at specification of these features to enable systematic visual characterization and categorization of vocal fold vibratory patterns in voice disorders.

Methods:

A cross-sectional, descriptive design was used. We selected 45 subjects and extracted 100 videokymographic images from the archive of more than 7,000 videokymographic examinations of subjects with a wide range of voice disorders. The images showed a large variety of vocal fold vibratory behaviors during sustained phonations. We visually identified the prominent features that distinguished the vibration patterns across the images.

Results:

We divided the findings into 10 feature categories. They included refined traditional features (eg, mucosal waves), as well as additional features that are obscured in strobolaryngoscopy (eg, different types of irregularities, left-right frequency differences, shapes of lateral and medial peaks, cycle aberrations).

Conclusions:

The variations in the identified features reveal different behavioral origins of voice disorders. The findings open new possibilities for objective documentation and for monitoring vocal fold behavior in clinical practice through kymographic imaging.

Non-stationary Bayesian estimation of parameters from a body cover model of the vocal folds

The evolution of reduced-order vocal fold models into clinically useful tools for subject-specific diagnosis and treatment hinges upon successfully and accurately representing an individual patient in the modeling framework. This, in turn, requires inference of model parameters from clinical measurements in order to tune a model to the given individual. Bayesian analysis is a powerful tool for estimating model parameter probabilities based upon a set of observed data. In this work, a Bayesian particle filter sampling technique capable of estimating time-varying model parameters, as occur in complex vocal gestures, is introduced. The technique is compared with time-invariant Bayesian estimation and least squares methods for determining both stationary and non-stationary parameters. The current technique accurately estimates the time-varying unknown model parameter and maintains tight credibility bounds. The credibility bounds are particularly relevant from a clinical perspective, as they provide insight into the confidence a clinician should have in the model predictions.

Voice pathology classification based on High-Speed Videoendoscopy

This work presents a method for automatical and objective classification of patients with healthy and pathological vocal fold vibration impairments using High-Speed Videoendoscopy of the larynx. We used an image segmentation and extraction of a novel set of numerical parameters describing the spatio-temporal dynamics of vocal folds to classification according to the normal and pathological cases and achieved 73,3% cross-validation classification accuracy. This approach is promising to develop an automatic diagnosis tool of voice disorders.

A generalized procedure for analyzing sustained and dynamic vocal fold vibrations from laryngeal high-speed videos using phonovibrograms

OBJECTIVE

This work presents a computer-based approach to analyze the two-dimensional vocal fold dynamics of endoscopic high-speed videos, and constitutes an extension and generalization of a previously proposed wavelet-based procedure. While most approaches aim for analyzing sustained phonation conditions, the proposed method allows for a clinically adequate analysis of both dynamic as well as sustained phonation paradigms.

MATERIALS AND METHODS

The analysis procedure is based on a spatio-temporal visualization technique, the phonovibrogram, that facilitates the documentation of the visible laryngeal dynamics. From the phonovibrogram, a low-dimensional set of features is computed using a principle component analysis strategy that quantifies the type of vibration patterns, irregularity, lateral symmetry and synchronicity, as a function of time. Two different test bench data sets are used to validate the approach: (I) 150 healthy and pathologic subjects examined during sustained phonation. (II) 20 healthy and pathologic subjects that were examined twice: during sustained phonation and a glissando from a low to a higher fundamental frequency. In order to assess the discriminative power of the extracted features, a Support Vector Machine is trained to distinguish between physiologic and pathologic vibrations. The results for sustained phonation sequences are compared to the previous approach. Finally, the classification performance of the stationary analyzing procedure is compared to the transient analysis of the glissando maneuver.

RESULTS

For the first test bench the proposed procedure outperformed the previous approach (proposed feature set: accuracy: 91.3%, sensitivity: 80%, specificity: 97%, previous approach: accuracy: 89.3%, sensitivity: 76%, specificity: 96%). Comparing the classification performance of the second test bench further corroborates that analyzing transient paradigms provides clear additional diagnostic value (glissando maneuver: accuracy: 90%, sensitivity: 100%, specificity: 80%, sustained phonation: accuracy: 75%, sensitivity: 80%, specificity: 70%).

CONCLUSIONS

The incorporation of parameters describing the temporal evolvement of vocal fold vibration clearly improves the automatic identification of pathologic vibration patterns. Furthermore, incorporating a dynamic phonation paradigm provides additional valuable information about the underlying laryngeal dynamics that cannot be derived from sustained conditions. The proposed generalized approach provides a better overall classification performance than the previous approach, and hence constitutes a new advantageous tool for an improved clinical diagnosis of voice disorders.

Development of a time-dependent numerical model for the assessment of non-stationary pharyngoesophageal tissue vibrations after total laryngectomy

Laryngeal cancer due to, e.g., extensive smoking and/or alcohol consumption can necessitate the excision of the entire larynx. After such a total laryngectomy, the voice generating structures are lost and with that the quality of life of the concerning patients is drastically reduced. However, the vibrations of the remaining tissue in the so called pharyngoesophageal (PE) segment can be applied as alternative sound generator. Tissue, scar, and geometric aspects of the PE-segment determine the postoperative substitute voice characteristic, being highly important for the future live of the patient. So far, PE-dynamics are simulated by a biomechanical model which is restricted to stationary vibrations, i.e., variations in pitch and amplitude cannot be handled. In order to investigate the dynamical range of PE-vibrations, knowledge about the temporal processes during substitute voice production is of crucial interest. Thus, time-dependent model parameters are suggested in order to quantify non-stationary PE-vibrations and drawing conclusions on the temporal characteristics of tissue stiffness, oscillating mass, pressure, and geometric distributions within the PE-segment. To adapt the numerical model to the PE-vibrations, an automatic, block-based optimization procedure is applied, comprising a combined global and local optimization approach. The suggested optimization procedure is validated with 75 synthetic data sets, simulating non-stationary oscillations of differently shaped PE-segments. The application to four high-speed recordings is shown and discussed. The correlation between model and PE-dynamics is ≥ 97%.

Quantifying spatiotemporal properties of vocal fold dynamics based on a multiscale analysis of phonovibrograms

In order to objectively assess the laryngeal vibratory behavior, endoscopic high-speed cameras capture several thousand frames per second of the vocal folds during phonation. However, judging all inherent clinically relevant features is a challenging task and requires well-founded expert knowledge. In this study, an automated wavelet-based analysis of laryngeal high-speed videos based on phonovibrograms is presented. The phonovibrogram is an image representation of the spatiotemporal pattern of vocal fold vibration and constitutes the basis for a computer-based analysis of laryngeal dynamics. The features extracted from the wavelet transform are shown to be closely related to a basic set of video-based measurements categorized by the European Laryngological Society for a subjective assessment of pathologic voices. The wavelet-based analysis further offers information about irregularity and lateral asymmetry and asynchrony. It is demonstrated in healthy and pathologic subjects as well as for a surgical group that was examined before and after the removal of a vocal fold polyp. The features were found to not only classify glottal closure characteristics but also quantify the impact of pathologies on the vibratory behavior. The interpretability and the discriminative power of the proposed feature set show promising relevance for a computer-assisted diagnosis and classification of voice disorders.

Characterizing vibratory kinematics in children and adults with high-speed digital imaging

PURPOSE The aim of this study is to quantify and identify characteristic vibratory motion in typically developing prepubertal children and young adults using high-speed digital imaging. METHOD The vibrations of the vocal folds were recorded from 27 children (ages 5-9 years) and 35 adults (ages 21-45 years), with high speed at 4,000 frames per second for sustained phonation. Kinematic features of amplitude periodicity, time periodicity, phase asymmetry, spatial symmetry, and glottal gap index were analyzed from the glottal area waveform across mean and standard deviation (i.e., intercycle variability) for each measure. RESULTS Children exhibited lower mean amplitude periodicity compared to men and women and lower time periodicity compared to men. Children and women exhibited greater variability in amplitude periodicity, time periodicity, phase asymmetry, and glottal gap index compared to men. Women had lower mean values of amplitude periodicity and time periodicity compared to men. CONCLUSION Children differed both spatially but more temporally in vocal fold motion, suggesting the need for the development of children-specific kinematic norms. Results suggest more uncontrolled vibratory motion in children, reflecting changes in the vocal fold layered structure and aero-acoustic source mechanisms.

Measuring body layer vibration of vocal folds by high-frame-rate ultrasound synchronized with a modified electroglottograph

The body-cover concept suggests that the vibration of body layer is an indispensable component of vocal fold vibration. To quantify this vibration, a synchronized system composed of a high-frame-rate ultrasound and a modified electroglottograph (EGG) was employed in this paper to simultaneously image the body layer vibration and record the vocal fold vibration phase information during natural phonations. After data acquisition, the displacements of in vivo body layer vibrations were measured from the ultrasonic radio frequency data, and the temporal reconstruction method was used to enhance the measurement accuracy. Results showed that the modified EGG, the waveform and characteristic points of which were identical to the conventional EGG, resolved the position conflict between the ultrasound transducer and EGG electrodes. The location and range of the vibrating body layer in the estimated displacement image were more clear and discernible than in the ultrasonic B-mode image. Quantitative analysis for vibration features of the body layer demonstrated that the body layer moved as a unit in the superior-inferior direction during the phonation of normal chest registers.

A wavelet-based approach for a continuous analysis of phonovibrograms

Recently, endoscopic high-speed laryngoscopy has been established for commercial use and constitutes a state-of-the-art technique to examine vocal fold dynamics. Despite overcoming many limitations of commonly applied stroboscopy it has not gained widespread clinical application, yet. A major drawback is a missing methodology of extracting valuable features to support visual assessment or computer-aided diagnosis. In this paper a compact and descriptive feature set is presented. The feature extraction routines are based on two-dimensional color graphs called phonovibrograms (PVG). These graphs contain the full spatio-temporal pattern of vocal fold dynamics and are therefore suited to derive features that comprehensively describe the vibration pattern of vocal folds. Within our approach, clinically relevant features such as glottal closure type, symmetry and periodicity are quantified in a set of 10 descriptive features. The suitability for classification tasks is shown using a clinical data set comprising 50 healthy and 50 paralytic subjects. A classification accuracy of 93.2% has been achieved.

Three-dimensional biomechanical properties of human vocal folds: parameter optimization of a numerical model to match in vitro dynamics

The human voice signal originates from the vibrations of the two vocal folds within the larynx. The interactions of several intrinsic laryngeal muscles adduct and shape the vocal folds to facilitate vibration in response to airflow. Three-dimensional vocal fold dynamics are extracted from in vitro hemilarynx experiments and fitted by a numerical three-dimensional-multi-mass-model (3DM) using an optimization procedure. In this work, the 3DM dynamics are optimized over 24 experimental data sets to estimate biomechanical vocal fold properties during phonation. Accuracy of the optimization is verified by low normalized error (0.13 ± 0.02), high correlation (83% ± 2%), and reproducible subglottal pressure values. The optimized, 3DM parameters yielded biomechanical variations in tissue properties along the vocal fold surface, including variations in both the local mass and stiffness of vocal folds. That is, both mass and stiffness increased along the superior-to-inferior direction. These variations were statistically analyzed under different experimental conditions (e.g., an increase in tension as a function of vocal fold elongation and an increase in stiffness and a decrease in mass as a function of glottal airflow). The study showed that physiologically relevant vocal fold tissue properties, which cannot be directly measured during in vivo human phonation, can be captured using this 3D-modeling technique.

Computation of physiological human vocal fold parameters by mathematical optimization of a biomechanical model

With the use of an endoscopic, high-speed camera, vocal fold dynamics may be observed clinically during phonation. However, observation and subjective judgment alone may be insufficient for clinical diagnosis and documentation of improved vocal function, especially when the laryngeal disease lacks any clear morphological presentation. In this study, biomechanical parameters of the vocal folds are computed by adjusting the corresponding parameters of a three-dimensional model until the dynamics of both systems are similar. First, a mathematical optimization method is presented. Next, model parameters (such as pressure, tension and masses) are adjusted to reproduce vocal fold dynamics, and the deduced parameters are physiologically interpreted. Various combinations of global and local optimization techniques are attempted. Evaluation of the optimization procedure is performed using 50 synthetically generated data sets. The results show sufficient reliability, including 0.07 normalized error, 96% correlation, and 91% accuracy. The technique is also demonstrated on data from human hemilarynx experiments, in which a low normalized error (0.16) and high correlation (84%) values were achieved. In the future, this technique may be applied to clinical high-speed images, yielding objective measures with which to document improved vocal function of patients with voice disorders.

Measuring body-cover vibration of vocal folds based on high frame rate ultrasonic imaging and high-speed video

Vibration of vocal folds is a body-cover layered vibration pattern due to the two-layer tissue structures of vocal folds. A method based on a synchronal imaging system is proposed in order to image and measure the body-cover vibration pattern of vocal folds. This imaging system contains two parts: high-frame-rate ultrasonic imaging part and high-speed video part, which can synchronously image the vibration of the body and cover layers a thigh speed. Then, image analysis methods are applied to measure the body-cover vibration of vocal folds from both recorded image sequences. We analyze characteristics of body-layer vibration based on the measurements from designed experiments. Moreover, these results meet simulations of a body-cover model.

Substitute voice production: quantification of PE segment vibrations using a biomechanical model

After total larynx excision due to laryngeal cancer, the tracheoesophageal substitute tissue vibrations at the intersection between the pharynx and the esophagus [pharyngoesophageal segment (PE segment)] serve as voice generator. The quality of the substitute voice significantly depends on the vibratory characteristics of the PE segment. For improving voice rehabilitation, the relationship between the PE dynamics and the resulting substitute voice quality is a matter of particular interest. Precondition for a comprehensive analysis of this relationship is an objective quantification of the PE vibrations. For quantification purposes, a method is proposed, which is based on the reproduction of the tissue vibrations by means of a biomechanical model of the PE segment. An optimization procedure for an automatic determination of appropriate model parameters is suggested to adapt the model dynamics to tissue movements extracted from high-speed (HS) videos. The applicability of the optimization procedure is evaluated with ten synthetic data sets. A mean error of 8.2% for the determination of previously defined model parameters was achieved as well as an overall stability of 7.1%. The application of the model to six HS recordings presented a mean correlation of the vibration patterns of 82%.

Optical measurements of vocal fold tensile properties: implications for phonatory mechanics

In voice research, in vitro tensile stretch experiments of vocal fold tissues are commonly employed to determine the tissue biomechanical properties. In the standard stretch-release protocol, tissue deformation is computed from displacements applied to sutures inserted through the thyroid and arytenoid cartilages, with the cartilages assumed to be rigid. Here, a non-contact optical method was employed to determine the actual tissue deformation of vocal fold lamina propria specimens from three excised human larynges in uniaxial tensile tests. Specimen deformation was found to consist not only of deformation of the tissue itself, but also deformation of the cartilages, as well as suture alignment and tightening. Stress-stretch curves of a representative load cycle were characterized by an incompressible Ogden model. The initial longitudinal elastic modulus was found to be considerably higher if determined based on optical displacement measurements than typical values reported in the literature. The present findings could change the understanding of the mechanics underlying vocal fold vibration. Given the high longitudinal elastic modulus the lamina propria appeared to demonstrate a substantial level of anisotropy. Consequently, transverse shear could play a significant role in vocal fold vibration, and fundamental frequencies of phonation should be predicted by beam theories accounting for such effects.

Theoretical modeling and experimental high-speed imaging of elongated vocal folds

In this paper, the role of vocal fold elongation in governing glottal movement dynamics was theoretically and experimentally investigated. A theoretical model was first proposed to incorporate vocal fold elongation into the two-mass model. This model predicted the direct and nondirect components of the glottal time series as a function of vocal fold elongation. Furthermore, high-speed digital imaging was applied in excised larynx experiments to visualize vocal fold vibrations with variable vocal fold elongation from -10% to 50% and subglottal pressures of 18- and 24-cm H(2)O. Comparison between theoretical model simulations and experimental observations showed good agreement. A relative maximum was seen in the nondirect component of glottal area, suggesting that an optimal elongation could maximize the vocal fold vibratory power. However, sufficiently large vocal fold elongations caused the nondirect component to approach zero and the direct component to approach a constant. These results showed that vocal fold elongation plays an important role in governing the dynamics of glottal area movement and validated the applicability of the proposed theoretical model and high-speed imaging to investigate laryngeal activity.

Biomechanical modeling of the three-dimensional aspects of human vocal fold dynamics

Human voice originates from the three-dimensional (3D) oscillations of the vocal folds. In previous studies, biomechanical properties of vocal fold tissues have been predicted by optimizing the parameters of simple two-mass-models to fit its dynamics to the high-speed imaging data from the clinic. However, only lateral and longitudinal displacements of the vocal folds were considered. To extend previous studies, a 3D mass-spring, cover-model is developed, which predicts the 3D vibrations of the entire medial surface of the vocal fold. The model consists of five mass planes arranged in vertical direction. Each plane contains five longitudinal, mass-spring, coupled oscillators. Feasibility of the model is assessed using a large body of dynamical data previously obtained from excised human larynx experiments, in vivo canine larynx experiments, physical models, and numerical models. Typical model output was found to be similar to existing findings. The resulting model enables visualization of the 3D dynamics of the human vocal folds during phonation for both symmetric and asymmetric vibrations.

Advances in laryngeal imaging

Imaging and image analysis became an important issue in laryngeal diagnostics. Various techniques, such as videostroboscopy, videokymography, digital kymography, or ultrasonography are available and are used in research and clinical practice. This paper reviews recent advances in imaging for laryngeal diagnostics.

[Basic research on vocal fold dynamics: three-dimensional vibration analysis of human and canine larynges]

BACKGROUND

The understanding of normal and pathological vocal fold dynamics is the basis for a pathophysiological motivated voice therapy. Crucial vocal fold dynamics concerning voice production occur at the medial part of the vocal fold which is seen as the most critical region of mucosal wave propagation. Due to the limited size of the larynx the possibilities of laryngeal imaging by endoscopic techniques are limited.

MATERIAL AND METHODS

This work describes an experimental set-up that enables quantification of the entire medial and superior vocal fold surface using excised human and in vivo canine larynges.

RESULTS

The data obtained enable analysis of vocal fold deflections, velocities, and mucosal wave propagation. The reciprocal dependencies can be examined and different areas of vocal fold dynamics located. The vertical components obscured in clinical endoscopy can be visualized. This is not negligible.

CONCLUSIONS

In particular it is shown that the vertical deflection, which cannot be observed by clinical examination, plays an important part in the dynamics and therefore cannot be omitted for therapeutic procedures. The theoretically assumed entrainment and influence of the two main vibration modes enabling normal phonation is confirmed.

[Phonovibrogram: vocal fold dynamics integrated within a single image]

BACKGROUND

Stroboscopy is widely used and is quite adequate for the examination of normal voices, but with increasing hoarseness its suitability declines, even when it is supplemented by video recordings and image evaluation. Real-time procedures such as videokymography or high-speed (HS) video imaging are more suitable methods of observing the movements of the vocal folds in such cases. A drawback of any video recording is the later time-consuming offline replay of the films in slow motion and our restricted pattern recognition for motion and other time-dependent processes.

METHODS

The phonovibrogram (PVG) is an image-processing algorithm that extracts the vocal fold motions of a whole laryngoscopic HS video film and automatically compresses them into a single image.

RESULTS

Simple patterns that vary from person to person are revealed by PVG; these can be categorized by means of simple geometric forms, which a human observer can more easily recognize and interpret than dynamic motion patterns. The PVG computation is described in detail and an extensive guide to interpretation is given, illustrated by reference to theoretical and real examples.

CONCLUSION

In clinical conditions, HS laryngoscopic video recording is useful only in association with automatic image processing. The PVG procedure is a promising approach and tests should be performed with a view to further clinical validation.