Spatiotemporal classification of vocal fold dynamics by a multimass model comprising time-dependent parameters

A Generative Method for a Laryngeal Biosignal

The Glottal Area Waveform (GAW) is an important component in quantitative clinical voice assessment, providing valuable insights into vocal fold function. In this study, we introduce a novel method employing Variational Autoencoders (VAEs) to generate synthetic GAWs. Our approach enables the creation of synthetic GAWs that closely replicate real-world data, offering a versatile tool for researchers and clinicians. We elucidate the process of manipulating the VAE latent space using the Glottal Opening Vector (GlOVe). The GlOVe allows precise control over the synthetic closure and opening of the vocal folds. By utilizing the GlOVe, we generate synthetic laryngeal biosignals. These biosignals accurately reflect vocal fold behavior, allowing for the emulation of realistic glottal opening changes. This manipulation extends to the introduction of arbitrary oscillations in the vocal folds, closely resembling real vocal fold oscillations. The range of factor coefficient values enables the generation of diverse biosignals with varying frequencies and amplitudes. Our results demonstrate that this approach yields highly accurate laryngeal biosignals, with the Normalized Mean Absolute Error values for various frequencies ranging from 9.6 ⋅ 10-3 to 1.20 ⋅ 10-2 for different experimented frequencies, alongside a remarkable training effectiveness, reflected in reductions of up to approximately 89.52% in key loss components. This proposed method may have implications for downstream speech synthesis and phonetics research, offering the potential for advanced and natural-sounding speech technologies.

Correlating Perceptual Voice Quality in Adductor Spasmodic Dysphonia With Computer Vision Assessment of Glottal Geometry Dynamics

PURPOSE

This study examined the relationship between voice quality and glottal geometry dynamics in patients with adductor spasmodic dysphonia (ADSD).

METHOD

An objective computer vision and machine learning system was developed to extract glottal geometry dynamics from nasolaryngoscopic video recordings for 78 patients with ADSD. General regression models were used to examine the relationship between overall voice quality and 15 variables that capture glottal geometry dynamics derived from the computer vision system. Two experts in ADSD independently rated voice quality for two separate voice tasks for every patient, yielding four different voice quality rating models.

RESULTS

All four of the regression models exhibited positive correlations with clinical assessments of voice quality (R ²s = .30-.34, Spearman rho = .55-.61, all with p < .001). Seven to 10 variables were included in each model. There was high overlap in the variables included between the four models, and the sign of the correlation with voice quality was consistent for each variable across all four regression models.

CONCLUSION

We found specific glottal geometry dynamics that correspond to voice quality in ADSD.

Interdependencies between acoustic and high-speed videoendoscopy parameters

In voice research, uncovering relations between the oscillating vocal folds, being the sound source of phonation, and the resulting perceived acoustic signal are of great interest. This is especially the case in the context of voice disorders, such as functional dysphonia (FD). We investigated 250 high-speed videoendoscopy (HSV) recordings with simultaneously recorded acoustic signals (124 healthy females, 60 FD females, 44 healthy males, 22 FD males). 35 glottal area waveform (GAW) parameters and 14 acoustic parameters were calculated for each recording. Linear and non-linear relations between GAW and acoustic parameters were investigated using Pearson correlation coefficients (PCC) and distance correlation coefficients (DCC). Further, norm values for parameters obtained from 250 ms long sustained phonation data (vowel /i/) were provided. 26 PCCs in females (5.3%) and 8 in males (1.6%) were found to be statistically significant (|corr.| ≥ 0.3). Only minor differences were found between PCCs and DCCs, indicating presence of weak non-linear dependencies between parameters. Fundamental frequency was involved in the majority of all relevant PCCs between GAW and acoustic parameters (19 in females and 7 in males). The most distinct difference between correlations in females and males was found for the parameter Period Variability Index. The study shows only weak relations between investigated acoustic and GAW-parameters. This indicates that the reduction of the complex 3D glottal dynamics to the 1D-GAW may erase laryngeal dynamic characteristics that are reflected within the acoustic signal. Hence, other GAW parameters, 2D-, 3D-laryngeal dynamics and vocal tract parameters should be further investigated towards potential correlations to the acoustic signal.

Rethinking glottal midline detection

A healthy voice is crucial for verbal communication and hence in daily as well as professional life. The basis for a healthy voice are the sound producing vocal folds in the larynx. A hallmark of healthy vocal fold oscillation is the symmetric motion of the left and right vocal fold. Clinically, videoendoscopy is applied to assess the symmetry of the oscillation and evaluated subjectively. High-speed videoendoscopy, an emerging method that allows quantification of the vocal fold oscillation, is more commonly employed in research due to the amount of data and the complex, semi-automatic analysis. In this study, we provide a comprehensive evaluation of methods that detect fully automatically the glottal midline. We used a biophysical model to simulate different vocal fold oscillations, extended the openly available BAGLS dataset using manual annotations, utilized both, simulations and annotated endoscopic images, to train deep neural networks at different stages of the analysis workflow, and compared these to established computer vision algorithms. We found that classical computer vision perform well on detecting the glottal midline in glottis segmentation data, but are outperformed by deep neural networks on this task. We further suggest GlottisNet, a multi-task neural architecture featuring the simultaneous prediction of both, the opening between the vocal folds and the symmetry axis, leading to a huge step forward towards clinical applicability of quantitative, deep learning-assisted laryngeal endoscopy, by fully automating segmentation and midline detection.

The effect of high-speed videoendoscopy configuration on reduced-order model parameter estimates by Bayesian inference

Bayesian inference has been previously demonstrated as a viable inverse analysis tool for estimating subject-specific reduced-order model parameters and uncertainties. However, previous studies have relied upon simulated glottal area waveforms with superimposed random noise as the measurement. In practice, high-speed videoendoscopy is used to measure glottal area, which introduces practical imaging effects not captured in simulated data, such as viewing angle, frame rate, and camera resolution. Herein, high-speed videos of the vocal folds were approximated by recording the trajectories of physical vocal fold models controlled by a symmetric body-cover model. Twenty videos were recorded, varying subglottal pressure, cricothyroid activation, and viewing angle, with frame rate and video resolution varied by digital video manipulation. Bayesian inference was used to estimate subglottal pressure and cricothyroid activation from glottal area waveforms extracted from the videos. The resulting estimates show off-axis viewing of 10° can lead to a 10% bias in the estimated subglottal pressure. A viewing model is introduced such that viewing angle can be included as an estimated parameter, which alleviates estimate bias. Frame rate and pixel resolution were found to primarily affect uncertainty of parameter estimates up to a limit where spatial and temporal resolutions were too poor to resolve the glottal area. Since many high-speed cameras have the ability to sacrifice spatial for temporal resolution, the findings herein suggest that Bayesian inference studies employing high-speed video should increase temporal resolutions at the expense of spatial resolution for reduced estimate uncertainties.

The effect of high-speed videoendoscopy configuration on reduced-order model parameter estimates by Bayesian inference

Bayesian inference has been previously demonstrated as a viable inverse analysis tool for estimating subject-specific reduced-order model parameters and uncertainties. However, previous studies have relied upon simulated glottal area waveforms with superimposed random noise as the measurement. In practice, high-speed videoendoscopy is used to measure glottal area, which introduces practical imaging effects not captured in simulated data, such as viewing angle, frame rate, and camera resolution. Herein, high-speed videos of the vocal folds were approximated by recording the trajectories of physical vocal fold models controlled by a symmetric body-cover model. Twenty videos were recorded, varying subglottal pressure, cricothyroid activation, and viewing angle, with frame rate and video resolution varied by digital video manipulation. Bayesian inference was used to estimate subglottal pressure and cricothyroid activation from glottal area waveforms extracted from the videos. The resulting estimates show off-axis viewing of 10° can lead to a 10% bias in the estimated subglottal pressure. A viewing model is introduced such that viewing angle can be included as an estimated parameter, which alleviates estimate bias. Frame rate and pixel resolution were found to primarily affect uncertainty of parameter estimates up to a limit where spatial and temporal resolutions were too poor to resolve the glottal area. Since many high-speed cameras have the ability to sacrifice spatial for temporal resolution, the findings herein suggest that Bayesian inference studies employing high-speed video should increase temporal resolutions at the expense of spatial resolution for reduced estimate uncertainties.

Biomechanical simulation of vocal fold dynamics in adults based on laryngeal high-speed videoendoscopy

MOTIVATION

Human voice is generated in the larynx by the two oscillating vocal folds. Owing to the limited space and accessibility of the larynx, endoscopic investigation of the actual phonatory process in detail is challenging. Hence the biomechanics of the human phonatory process are still not yet fully understood. Therefore, we adapt a mathematical model of the vocal folds towards vocal fold oscillations to quantify gender and age related differences expressed by computed biomechanical model parameters.

METHODS

The vocal fold dynamics are visualized by laryngeal high-speed videoendoscopy (4000 fps). A total of 33 healthy young subjects (16 females, 17 males) and 11 elderly subjects (5 females, 6 males) were recorded. A numerical two-mass model is adapted to the recorded vocal fold oscillations by varying model masses, stiffness and subglottal pressure. For adapting the model towards the recorded vocal fold dynamics, three different optimization algorithms (Nelder-Mead, Particle Swarm Optimization and Simulated Bee Colony) in combination with three cost functions were considered for applicability. Gender differences and age-related kinematic differences reflected by the model parameters were analyzed.

RESULTS AND CONCLUSION

The biomechanical model in combination with numerical optimization techniques allowed phonatory behavior to be simulated and laryngeal parameters involved to be quantified. All three optimization algorithms showed promising results. However, only one cost function seems to be suitable for this optimization task. The gained model parameters reflect the phonatory biomechanics for men and women well and show quantitative age- and gender-specific differences. The model parameters for younger females and males showed lower subglottal pressures, lower stiffness and higher masses than the corresponding elderly groups. Females exhibited higher subglottal pressures, smaller oscillation masses and larger stiffness than the corresponding similar aged male groups. Optimizing numerical models towards vocal fold oscillations is useful to identify underlying laryngeal components controlling the phonatory process.

An extended Kalman filter approach to non-stationary Bayesian estimation of reduced-order vocal fold model parameters

The Bayesian framework for parameter inference provides a basis from which subject-specific reduced-order vocal fold models can be generated. Previously, it has been shown that a particle filter technique is capable of producing estimates and associated credibility intervals of time-varying reduced-order vocal fold model parameters. However, the particle filter approach is difficult to implement and has a high computational cost, which can be barriers to clinical adoption. This work presents an alternative estimation strategy based upon Kalman filtering aimed at reducing the computational cost of subject-specific model development. The robustness of this approach to Gaussian and non-Gaussian noise is discussed. The extended Kalman filter (EKF) approach is found to perform very well in comparison with the particle filter technique at dramatically lower computational cost. Based upon the test cases explored, the EKF is comparable in terms of accuracy to the particle filter technique when greater than 6000 particles are employed; if less particles are employed, the EKF actually performs better. For comparable levels of accuracy, the solution time is reduced by 2 orders of magnitude when employing the EKF. By virtue of the approximations used in the EKF, however, the credibility intervals tend to be slightly underpredicted.

Non-stationary Bayesian estimation of parameters from a body cover model of the vocal folds

The evolution of reduced-order vocal fold models into clinically useful tools for subject-specific diagnosis and treatment hinges upon successfully and accurately representing an individual patient in the modeling framework. This, in turn, requires inference of model parameters from clinical measurements in order to tune a model to the given individual. Bayesian analysis is a powerful tool for estimating model parameter probabilities based upon a set of observed data. In this work, a Bayesian particle filter sampling technique capable of estimating time-varying model parameters, as occur in complex vocal gestures, is introduced. The technique is compared with time-invariant Bayesian estimation and least squares methods for determining both stationary and non-stationary parameters. The current technique accurately estimates the time-varying unknown model parameter and maintains tight credibility bounds. The credibility bounds are particularly relevant from a clinical perspective, as they provide insight into the confidence a clinician should have in the model predictions.

A generalized procedure for analyzing sustained and dynamic vocal fold vibrations from laryngeal high-speed videos using phonovibrograms

OBJECTIVE

This work presents a computer-based approach to analyze the two-dimensional vocal fold dynamics of endoscopic high-speed videos, and constitutes an extension and generalization of a previously proposed wavelet-based procedure. While most approaches aim for analyzing sustained phonation conditions, the proposed method allows for a clinically adequate analysis of both dynamic as well as sustained phonation paradigms.

MATERIALS AND METHODS

The analysis procedure is based on a spatio-temporal visualization technique, the phonovibrogram, that facilitates the documentation of the visible laryngeal dynamics. From the phonovibrogram, a low-dimensional set of features is computed using a principle component analysis strategy that quantifies the type of vibration patterns, irregularity, lateral symmetry and synchronicity, as a function of time. Two different test bench data sets are used to validate the approach: (I) 150 healthy and pathologic subjects examined during sustained phonation. (II) 20 healthy and pathologic subjects that were examined twice: during sustained phonation and a glissando from a low to a higher fundamental frequency. In order to assess the discriminative power of the extracted features, a Support Vector Machine is trained to distinguish between physiologic and pathologic vibrations. The results for sustained phonation sequences are compared to the previous approach. Finally, the classification performance of the stationary analyzing procedure is compared to the transient analysis of the glissando maneuver.

RESULTS

For the first test bench the proposed procedure outperformed the previous approach (proposed feature set: accuracy: 91.3%, sensitivity: 80%, specificity: 97%, previous approach: accuracy: 89.3%, sensitivity: 76%, specificity: 96%). Comparing the classification performance of the second test bench further corroborates that analyzing transient paradigms provides clear additional diagnostic value (glissando maneuver: accuracy: 90%, sensitivity: 100%, specificity: 80%, sustained phonation: accuracy: 75%, sensitivity: 80%, specificity: 70%).

CONCLUSIONS

The incorporation of parameters describing the temporal evolvement of vocal fold vibration clearly improves the automatic identification of pathologic vibration patterns. Furthermore, incorporating a dynamic phonation paradigm provides additional valuable information about the underlying laryngeal dynamics that cannot be derived from sustained conditions. The proposed generalized approach provides a better overall classification performance than the previous approach, and hence constitutes a new advantageous tool for an improved clinical diagnosis of voice disorders.

Entrance loss coefficients and exit coefficients for a physical model of the glottis with convergent angles

Pressure distributions were obtained for 5°, 10°, and 20° convergent angles with a static physical model (M5) of the glottis. Measurements were made for minimal glottal diameters from d = 0.005-0.32 cm with a range of transglottal pressures of interest for phonation. Entrance loss coefficients were calculated at the glottal entrance for each minimal diameter and transglottal pressure to measure how far the flows in this region deviate from Bernoulli flow. Exit coefficients were also calculated to determine the presence and magnitude of pressure recovery near the glottal exit. The entrance loss coefficients for the three convergent angles vary from values near 2.3-3.4 for d = 0.005 cm to values near 0.6 for d = 0.32 cm. These coefficients extend the tables of entrance loss and exit coefficients obtained for the uniform glottis according to Fulcher, Scherer, and Powell [J. Acoust. Soc. Am. 129, 1548-1553 (2011)].

Development of a time-dependent numerical model for the assessment of non-stationary pharyngoesophageal tissue vibrations after total laryngectomy

Laryngeal cancer due to, e.g., extensive smoking and/or alcohol consumption can necessitate the excision of the entire larynx. After such a total laryngectomy, the voice generating structures are lost and with that the quality of life of the concerning patients is drastically reduced. However, the vibrations of the remaining tissue in the so called pharyngoesophageal (PE) segment can be applied as alternative sound generator. Tissue, scar, and geometric aspects of the PE-segment determine the postoperative substitute voice characteristic, being highly important for the future live of the patient. So far, PE-dynamics are simulated by a biomechanical model which is restricted to stationary vibrations, i.e., variations in pitch and amplitude cannot be handled. In order to investigate the dynamical range of PE-vibrations, knowledge about the temporal processes during substitute voice production is of crucial interest. Thus, time-dependent model parameters are suggested in order to quantify non-stationary PE-vibrations and drawing conclusions on the temporal characteristics of tissue stiffness, oscillating mass, pressure, and geometric distributions within the PE-segment. To adapt the numerical model to the PE-vibrations, an automatic, block-based optimization procedure is applied, comprising a combined global and local optimization approach. The suggested optimization procedure is validated with 75 synthetic data sets, simulating non-stationary oscillations of differently shaped PE-segments. The application to four high-speed recordings is shown and discussed. The correlation between model and PE-dynamics is ≥ 97%.

Characterizing vibratory kinematics in children and adults with high-speed digital imaging

PURPOSE The aim of this study is to quantify and identify characteristic vibratory motion in typically developing prepubertal children and young adults using high-speed digital imaging. METHOD The vibrations of the vocal folds were recorded from 27 children (ages 5-9 years) and 35 adults (ages 21-45 years), with high speed at 4,000 frames per second for sustained phonation. Kinematic features of amplitude periodicity, time periodicity, phase asymmetry, spatial symmetry, and glottal gap index were analyzed from the glottal area waveform across mean and standard deviation (i.e., intercycle variability) for each measure. RESULTS Children exhibited lower mean amplitude periodicity compared to men and women and lower time periodicity compared to men. Children and women exhibited greater variability in amplitude periodicity, time periodicity, phase asymmetry, and glottal gap index compared to men. Women had lower mean values of amplitude periodicity and time periodicity compared to men. CONCLUSION Children differed both spatially but more temporally in vocal fold motion, suggesting the need for the development of children-specific kinematic norms. Results suggest more uncontrolled vibratory motion in children, reflecting changes in the vocal fold layered structure and aero-acoustic source mechanisms.

Quantitative analysis of organic vocal fold pathologies in females by high-speed endoscopy

OBJECTIVES/HYPOTHESIS

Quantitative analysis of endoscopic high-speed video recordings of vocal fold vibrations has been growing in importance in recent years. The videos have mainly been analyzed using subjective evaluation, but this is examiner dependent, and the results show inadequate interobserver agreement. The aims of this study were therefore to identify appropriate objective parameters for analyzing high-speed recordings to differentiate healthy voice production from organic disorders.

STUDY DESIGN

METHODS

A total of 152 females were examined, divided into 77 healthy and 75 with four different pathological conditions: laryngeal epithelial thickening, Reinke edema, vocal fold polyps, and vocal fold cysts. Vocal fold vibrations were recorded with a high-speed camera (4,000 Hz, 256 × 256 pixels) during sustained phonation. Parameters computed from the glottal area waveform (GAW) and from phonovibrogram (PVG) were analyzed. Multiparametric linear discriminant analysis was performed to classify pathological conditions versus the healthy group.

RESULTS

Twenty of 44 parameters were identified that are capable of distinguishing between the individual types of pathology. PVG parameters showed better performance than GAW parameters. Parameters representing vibrational periodicity via standard deviation showed better performance than absolute parameters. In addition, linear discriminant analysis achieved reliable differentiation between healthy and pathological vocal fold vibrations: 72% for the five-class problem (all groups separately) and 88% for the two-class problem (healthy vs. all pathologies taken as one class).

CONCLUSIONS

The study succeeded in defining objective parameters for analyzing endoscopic high-speed videos and suggesting first parameters for differentiation between healthy dynamics and dynamics of organic pathologies.

Computation of physiological human vocal fold parameters by mathematical optimization of a biomechanical model

With the use of an endoscopic, high-speed camera, vocal fold dynamics may be observed clinically during phonation. However, observation and subjective judgment alone may be insufficient for clinical diagnosis and documentation of improved vocal function, especially when the laryngeal disease lacks any clear morphological presentation. In this study, biomechanical parameters of the vocal folds are computed by adjusting the corresponding parameters of a three-dimensional model until the dynamics of both systems are similar. First, a mathematical optimization method is presented. Next, model parameters (such as pressure, tension and masses) are adjusted to reproduce vocal fold dynamics, and the deduced parameters are physiologically interpreted. Various combinations of global and local optimization techniques are attempted. Evaluation of the optimization procedure is performed using 50 synthetically generated data sets. The results show sufficient reliability, including 0.07 normalized error, 96% correlation, and 91% accuracy. The technique is also demonstrated on data from human hemilarynx experiments, in which a low normalized error (0.16) and high correlation (84%) values were achieved. In the future, this technique may be applied to clinical high-speed images, yielding objective measures with which to document improved vocal function of patients with voice disorders.

Pressure distributions in a static physical model of the uniform glottis: entrance and exit coefficients

Pressure distributions for the uniform glottis were obtained with a static physical model (M5). Glottal diameters of d=0.005, 0.0075, 0.01, 0.02, 0.04, 0.08, 0.16, and 0.32 cm were used with a range of phonatory transglottal pressures. At each pressure and diameter, entrance loss and exit coefficients were determined. In general, both coefficients decreased in value as the transglottal pressure or the diameter increased. Entrance loss coefficients ranged from 0.69 to 17.6. Use of these coefficients with the measured flow rates in straightforward equations accurately reproduced the pressure distributions within the glottis and along the inferior vocal fold surface.

Theoretical modeling and experimental high-speed imaging of elongated vocal folds

In this paper, the role of vocal fold elongation in governing glottal movement dynamics was theoretically and experimentally investigated. A theoretical model was first proposed to incorporate vocal fold elongation into the two-mass model. This model predicted the direct and nondirect components of the glottal time series as a function of vocal fold elongation. Furthermore, high-speed digital imaging was applied in excised larynx experiments to visualize vocal fold vibrations with variable vocal fold elongation from -10% to 50% and subglottal pressures of 18- and 24-cm H(2)O. Comparison between theoretical model simulations and experimental observations showed good agreement. A relative maximum was seen in the nondirect component of glottal area, suggesting that an optimal elongation could maximize the vocal fold vibratory power. However, sufficiently large vocal fold elongations caused the nondirect component to approach zero and the direct component to approach a constant. These results showed that vocal fold elongation plays an important role in governing the dynamics of glottal area movement and validated the applicability of the proposed theoretical model and high-speed imaging to investigate laryngeal activity.

Biomechanical modeling of the three-dimensional aspects of human vocal fold dynamics

Human voice originates from the three-dimensional (3D) oscillations of the vocal folds. In previous studies, biomechanical properties of vocal fold tissues have been predicted by optimizing the parameters of simple two-mass-models to fit its dynamics to the high-speed imaging data from the clinic. However, only lateral and longitudinal displacements of the vocal folds were considered. To extend previous studies, a 3D mass-spring, cover-model is developed, which predicts the 3D vibrations of the entire medial surface of the vocal fold. The model consists of five mass planes arranged in vertical direction. Each plane contains five longitudinal, mass-spring, coupled oscillators. Feasibility of the model is assessed using a large body of dynamical data previously obtained from excised human larynx experiments, in vivo canine larynx experiments, physical models, and numerical models. Typical model output was found to be similar to existing findings. The resulting model enables visualization of the 3D dynamics of the human vocal folds during phonation for both symmetric and asymmetric vibrations.

Typing vocal fold vibratory patterns in excised larynx experiments via digital kymography

OBJECTIVES

Signal typing is central to the understanding of vocal fold vibratory patterns. Digital kymography (DKG) allows the direct observation of vocal fold vibratory patterns, and therefore, using DKG for vibratory signal typing may provide a useful complement to traditional signal typing techniques.

METHODS

Video data collected from 20 larynges excised from mongrel dogs were observed with DKG in order to find examples of type 1 (nearly periodic), type 2 (subharmonic), and type 3 (aperiodic) vibratory patterns. The time series, frequency spectra, and correlation dimensions were calculated for each signal type.

RESULTS

The type 1 pattern showed a periodic time series of glottal edges and a discrete frequency spectrum. The type 2 vibratory pattern displayed a time series of alternating high- and low-amplitude waves and a frequency spectrum that included a subharmonic (F0/2) frequency component. Regular and symmetric vibratory patterns were observed in the type 1 and type 2 patterns. The type 3 vibratory pattern was characterized by an aperiodic time series of glottal edges, a broadband frequency spectrum, and irregular and asymmetric vibratory patterns. The correlation dimension estimates increased from type 1 to type 2 to type 3.

CONCLUSIONS

Imaging with DKG demonstrated an ability to assign a signal type to various laryngeal vibrations. Signal typing techniques utilizing direct observation of the vocal folds could be useful in determining valid methods for the analysis of vocal fold vibrations.