Irregular vocal-fold vibration--high-speed observation and modeling

Deriving Vocal Fold Oscillation Information from Recorded Voice Signals Using Models of Phonation

During phonation, the vocal folds exhibit a self-sustained oscillatory motion, which is influenced by the physical properties of the speaker's vocal folds and driven by the balance of bio-mechanical and aerodynamic forces across the glottis. Subtle changes in the speaker's physical state can affect voice production and alter these oscillatory patterns. Measuring these can be valuable in developing computational tools that analyze voice to infer the speaker's state. Traditionally, vocal fold oscillations (VFOs) are measured directly using physical devices in clinical settings. In this paper, we propose a novel analysis-by-synthesis approach that allows us to infer the VFOs directly from recorded speech signals on an individualized, speaker-by-speaker basis. The approach, called the ADLES-VFT algorithm, is proposed in the context of a joint model that combines a phonation model (with a glottal flow waveform as the output) and a vocal tract acoustic wave propagation model such that the output of the joint model is an estimated waveform. The ADLES-VFT algorithm is a forward-backward algorithm which minimizes the error between the recorded waveform and the output of this joint model to estimate its parameters. Once estimated, these parameter values are used in conjunction with a phonation model to obtain its solutions. Since the parameters correlate with the physical properties of the vocal folds of the speaker, model solutions obtained using them represent the individualized VFOs for each speaker. The approach is flexible and can be applied to various phonation models. In addition to presenting the methodology, we show how the VFOs can be quantified from a dynamical systems perspective for classification purposes. Mathematical derivations are provided in an appendix for better readability.

Investigation of Vocal Bifurcations and Voice Patterns Induced by Asymmetry of Pathological Vocal Folds

PURPOSE

Vocal fold asymmetry creates irregular entrainments and modulations in voice, which may lead to rough perceptual quality. The presence of asymmetry can also cause mid-phonation bifurcations where a small change in the phonatory system causes a drastic change in vibration pattern, resulting in transitions in and out of rough voice. This study surveys sustained phonation recordings of speakers with the diagnoses of vocal fold polyp or unilateral vocal fold paralysis to investigate the resulting voice patterns.

METHOD

This retrospective study observed 71 sustained phonation recordings from 48 patients. Segments with distinctive signal patterns were identified within each recording with narrowband spectrogram and computer-assisted analysis of spectral peaks.

RESULTS

Phonation segmentation yielded 240 segments across all the recordings. Five voice patterns were recognized: (regularly or irregularly) entrained, modulated, uncoupled, unstable, and pulsed. Thirty-six patients (75%) exhibited irregular patterns. No single irregular pattern lasted for the entire phonation and was always accompanied by at least one mid-phonation bifurcation. Durations of the irregular segments (M = 0.4 s) were significantly shorter than the segments with the regular pattern (M = 1.4 s).

CONCLUSIONS

The results suggest that vocal fold pathology frequently introduces dynamic vibratory patterns that affect both the acoustic signals and perceptions. Due to these abnormalities, it is important for clinical voice assessment protocols, both perceptual and acoustic, to account for these possible bifurcations, irregular signal patterns, and their tendencies.

Comparison of one-dimensional and three-dimensional glottal flow models in left-right asymmetric vocal fold conditions

While the glottal flow is often simplified as one-dimensional (1D) in computational models of phonation to reduce computational costs, the 1D flow model has not been validated in left-right asymmetric vocal fold conditions, as often occur in both normal and pathological voice production. In this study, we performed three-dimensional (3D) and 1D flow simulations coupled to a two-mass model of adult male vocal folds and compared voice production at different degrees of left-right stiffness asymmetry. The flow and acoustic fields in 3D were obtained by solving the compressible Navier-Stokes equations using the volume penalization method with the moving vocal fold wall as an immersed boundary. Despite differences in the predicted flow pressure on vocal fold surface between the 1D and 3D flow models, the results showed reasonable agreement in vocal fold vibration patterns and selected voice outcome measures between the 1D and 3D models for the range of left-right asymmetric conditions investigated. This indicates that vocal fold properties play a larger role than the glottal flow in determining the overall pattern of vocal fold vibration and the produced voice, and the 1D flow simplification is sufficient in modeling phonation, at least for the simplified glottal geometry of this study.

Investigating blunt force trauma to the larynx: The role of inferior-superior vocal fold displacement on phonation

Blunt force trauma to the larynx, which may result from motor vehicle collisions, sports activities, etc., can cause significant damage, often leading to displaced fractures of the laryngeal cartilages, thereby disrupting vocal function. Current surgical interventions primarily focus on airway restoration to stabilize the patient, with restoration of vocal function usually being a secondary consideration. Due to laryngeal fracture, asymmetric vertical misalignment of the left or right vocal fold (VF) in the inferior-superior direction often occurs. This affects VF closure and can lead to a weak, breathy voice requiring increased vocal effort. It is unclear, however, how much vertical VF misalignment can be tolerated before voice quality degrades significantly. To address this need, the influence of inferior-superior VF displacement on phonation is investigated in 1.0mm increments using synthetic, self-oscillating VF models in a physiologically-representative facility. Acoustic (SPL, frequency, H1-H2, jitter, and shimmer), kinematic (amplitude and phase differences), and aerodynamic parameters (flow rate and subglottal pressure) are investigated as a function of inferior-superior vertical displacement. Significant findings include that once the inferior-superior medial length of the VF is surpassed, sustained phonation degrades precipitously, becoming severely pathological. If laryngeal reconstruction approaches can ensure VF contact is maintained during phonation (i.e., vertical displacement doesn't surpass VF medial length), improved vocal outcomes are expected.

Pre- and post-operative high-speed videolaryngoscopy in unilateral vocal cord paralysis following autologous fat augmentation

PURPOSE

To compare high-speed videolaryngoscopy (HSV) parameters such as open quotient (OQ), amplitude symmetry index (ASI), phase symmetry index (PSI), and frequency symmetry index (FSI), of the unilateral vocal cord paralysis (UVCP) patients pre and post (after 6 months) autologous fat augmentation.

MATERIALS AND METHODS

This retrospective study evaluated all age and gender patients with UVCP that underwent autologous fat augmentation from July 2016 to July 2019. The OQ, ASI, PSI, and FSI were calculated from the HSV recordings by using the montage and fast Fourier transform point analysis. The pre-and post-operative means were compared using a paired student t-test, with a p-value less than 0.05 considered significant.

RESULT

A total of 37 patients, age 41.2 ± 11.3 years (21 to 67 years), 59.4% females and 40.6% males, were included in the study. The average duration of symptom onset was 2.3 ± 0.87 months. The post-operative mean values of OQ, ASI, PSI, and FSI following the fat augmentation were significantly improved compared to the pre-operative mean values with p-values <0.0001, 0.0018, 0.0011, and 0.0006, respectively.

CONCLUSION

There was a significant improvement in the OQ, ASI, PSI, and FSI in UVCP patients after 6 months of autologous fat augmentation, signifying an enhanced vibratory function. The ability of HSV to measure the minute details of vocal cord vibration by providing quantitative measurements has also been highlighted. The need for future prospective research with an increased sample size and longer duration of follow up is recommended.

Rethinking glottal midline detection

A healthy voice is crucial for verbal communication and hence in daily as well as professional life. The basis for a healthy voice are the sound producing vocal folds in the larynx. A hallmark of healthy vocal fold oscillation is the symmetric motion of the left and right vocal fold. Clinically, videoendoscopy is applied to assess the symmetry of the oscillation and evaluated subjectively. High-speed videoendoscopy, an emerging method that allows quantification of the vocal fold oscillation, is more commonly employed in research due to the amount of data and the complex, semi-automatic analysis. In this study, we provide a comprehensive evaluation of methods that detect fully automatically the glottal midline. We used a biophysical model to simulate different vocal fold oscillations, extended the openly available BAGLS dataset using manual annotations, utilized both, simulations and annotated endoscopic images, to train deep neural networks at different stages of the analysis workflow, and compared these to established computer vision algorithms. We found that classical computer vision perform well on detecting the glottal midline in glottis segmentation data, but are outperformed by deep neural networks on this task. We further suggest GlottisNet, a multi-task neural architecture featuring the simultaneous prediction of both, the opening between the vocal folds and the symmetry axis, leading to a huge step forward towards clinical applicability of quantitative, deep learning-assisted laryngeal endoscopy, by fully automating segmentation and midline detection.

Quantifying the auto-oscillation complexity following water spraying with interest for phonation

Human voiced sound production or phonation is the result of a fluid-structure instability in the larynx leading to vocal folds auto-oscillation. In this paper, the effect of surface hydration following water spraying (0 up to 5 ml) on an ongoing auto-oscillation is studied experimentally using different mechanical deformable vocal folds replicas. The complexity of the oscillation is quantified on the upstream pressure by a phase space recurrence and complexity analysis. It is shown that: (1) the ratio of the degree of determinism to the recurrence rate of the phase space states γ and (2) estimated correlation dimension D_{2} are suitable parameters to grasp the effect of hydration on the oscillation pattern. The oscillation regime after hydration can either remain deterministic or approach a chaotic regime depending on initial conditions prior to water spraying, such as elasticity, glottal aperture, as well as oscillation complexity.

Laryngeal Pressure Estimation With a Recurrent Neural Network

Quantifying the physical parameters of voice production is essential for understanding the process of phonation and can aid in voice research and diagnosis. As an alternative to invasive measurements, they can be estimated by formulating an inverse problem using a numerical forward model. However, high-fidelity numerical models are often computationally too expensive for this. This paper presents a novel approach to train a long short-term memory network to estimate the subglottal pressure in the larynx at massively reduced computational cost using solely synthetic training data. We train the network on synthetic data from a numerical two-mass model and validate it on experimental data from 288 high-speed ex vivo video recordings of porcine vocal folds from a previous study. The training requires significantly fewer model evaluations compared with the previous optimization approach. On the test set, we maintain a comparable performance of 21.2% versus previous 17.7% mean absolute percentage error in estimating the subglottal pressure. The evaluation of one sample requires a vanishingly small amount of computation time. The presented approach is able to maintain estimation accuracy of the subglottal pressure at significantly reduced computational cost. The methodology is likely transferable to estimate other parameters and training with other numerical models. This improvement should allow the adoption of more sophisticated, high-fidelity numerical models of the larynx. The vast speedup is a critical step to enable a future clinical application and knowledge of parameters such as the subglottal pressure will aid in diagnosis and treatment selection.

Biomechanical simulation of vocal fold dynamics in adults based on laryngeal high-speed videoendoscopy

MOTIVATION

Human voice is generated in the larynx by the two oscillating vocal folds. Owing to the limited space and accessibility of the larynx, endoscopic investigation of the actual phonatory process in detail is challenging. Hence the biomechanics of the human phonatory process are still not yet fully understood. Therefore, we adapt a mathematical model of the vocal folds towards vocal fold oscillations to quantify gender and age related differences expressed by computed biomechanical model parameters.

METHODS

The vocal fold dynamics are visualized by laryngeal high-speed videoendoscopy (4000 fps). A total of 33 healthy young subjects (16 females, 17 males) and 11 elderly subjects (5 females, 6 males) were recorded. A numerical two-mass model is adapted to the recorded vocal fold oscillations by varying model masses, stiffness and subglottal pressure. For adapting the model towards the recorded vocal fold dynamics, three different optimization algorithms (Nelder-Mead, Particle Swarm Optimization and Simulated Bee Colony) in combination with three cost functions were considered for applicability. Gender differences and age-related kinematic differences reflected by the model parameters were analyzed.

RESULTS AND CONCLUSION

The biomechanical model in combination with numerical optimization techniques allowed phonatory behavior to be simulated and laryngeal parameters involved to be quantified. All three optimization algorithms showed promising results. However, only one cost function seems to be suitable for this optimization task. The gained model parameters reflect the phonatory biomechanics for men and women well and show quantitative age- and gender-specific differences. The model parameters for younger females and males showed lower subglottal pressures, lower stiffness and higher masses than the corresponding elderly groups. Females exhibited higher subglottal pressures, smaller oscillation masses and larger stiffness than the corresponding similar aged male groups. Optimizing numerical models towards vocal fold oscillations is useful to identify underlying laryngeal components controlling the phonatory process.

Effect of level difference between left and right vocal folds on phonation: Physical experiment and theoretical study

As an alternative factor to produce asymmetry between left and right vocal folds, the present study focuses on level difference, which is defined as the distance between the upper surfaces of the bilateral vocal folds in the inferior-superior direction. Physical models of the vocal folds were utilized to study the effect of the level difference on the phonation threshold pressure. A vocal tract model was also attached to the vocal fold model. For two types of different models, experiments revealed that the phonation threshold pressure tended to increase as the level difference was extended. Based upon a small amplitude approximation of the vocal fold oscillations, a theoretical formula was derived for the phonation threshold pressure. This theory agrees with the experiments, especially when the phase difference between the left and right vocal folds is not extensive. Furthermore, an asymmetric two-mass model was simulated with a level difference to validate the experiments as well as the theory. The primary conclusion is that the level difference has a potential effect on voice production especially for patients with an extended level of vertical difference in the vocal folds, which might be taken into account for the diagnosis of voice disorders.

Vocal development in a Waddington landscape

Vocal development is the adaptive coordination of the vocal apparatus, muscles, the nervous system, and social interaction. Here, we use a quantitative framework based on optimal control theory and Waddington’s landscape metaphor to provide an integrated view of this process. With a biomechanical model of the marmoset monkey vocal apparatus and behavioral developmental data, we show that only the combination of the developing vocal tract, vocal apparatus muscles and nervous system can fully account for the patterns of vocal development. Together, these elements influence the shape of the monkeys’ vocal developmental landscape, tilting, rotating or shifting it in different ways. We can thus use this framework to make quantitative predictions regarding how interfering factors or experimental perturbations can change the landscape within a species, or to explain comparative differences in vocal development across species

DOI:http://dx.doi.org/10.7554/eLife.20782.001

As infants develop they learn new behaviors and refine existing ones. For example, human infants progress from crying to babbling to producing speech-like sounds. A complex sequence of changes in muscles, the nervous system and in patterns of interactions with other individuals all contribute to these emerging behaviors.

Despite this complexity, most studies of vocal development have only considered single factors in isolation. A study of speech development, for example, might examine how changes in the brain enable infants to imitate sounds. However, that same study will probably ignore how changes in the structure of the vocal cords, or in the behavior of the parents, also promote imitation.

Young marmoset monkeys, like human infants, gradually develop from producing immature cries to adult-like calls. Teramoto, Takahashi et al. built a computational model of this process and compared the model to data from real animals. The first version of the model focused solely on how the marmosets’ vocal cords grow, and did not fully reproduce how adult-like calls emerge in real marmosets. Teramoto, Takahashi et al. therefore added factors to the model that simulate improvements in muscle control, learning in the nervous system and in the behavior of other animals. These findings show that, to reflect how adult-like calls emerge in real marmosets, the model needs to include all of these factors.

The model developed by Teramoto, Takahashi et al. may also provide insights into why vocal learning and some other behaviors emerge in some species and not others. It may also be used to predict the consequences of disrupting individual processes in young animals at particular points in time and how such disruptions shape the way an animal develops on its way to adulthood.

DOI:http://dx.doi.org/10.7554/eLife.20782.002

Visualizing the Vibration of Laryngeal Tissue during Phonation Using Ultrafast Plane Wave Ultrasonography

Ultrafast plane wave ultrasonography is employed in this study to visualize the vibration of the larynx and quantify the vibration phase as well as the vibration amplitude of the laryngeal tissue. Ultrasonic images were obtained at 5000 to 10,000 frames/s in the coronal plane at the level of the glottis. Although the image quality degraded when the imaging mode was switched from conventional ultrasonography to ultrafast plane wave ultrasonography, certain anatomic structures such as the vocal folds, as well as the sub- and supraglottic structures, including the false vocal folds, can be identified in the ultrafast plane wave ultrasonic image. The periodic vibration of the vocal fold edge could be visualized in the recorded image sequence during phonation. Furthermore, a motion estimation method was used to quantify the displacement of laryngeal tissue from hundreds of frames of ultrasonic data acquired. Vibratory displacement waveforms of the sub- and supraglottic structures were successfully obtained at a high level of ultrasonic signal correlation. Moreover, statistically significant differences in vibration pattern between the sub- and supraglottic structures were found. Variation of vibration amplitude along the subglottic mucosal surface is significantly smaller than that along the supraglottic mucosal surface. Phase delay of vibration along the subglottic mucosal surface is significantly smaller than that along the supraglottic mucosal surface.

Synthetic multi-line kymographic analysis: A spatiotemporal data reduction technique for high-speed videoendoscopy

High-speed videoendoscopy (HSV) enables observation of the true vibratory behavior of the vocal folds. To quantify the vocal fold vibration captured by the HSV, lateral movement features (e.g., glottal width and vocal fold edge displacements) have been extracted as functions of time. The most common analysis method is to extract the features on a lateral strip used to form digital kymogram. The weakness of this method is that it can only capture the vibrational behavior local to the strip location. While the multi-line kymographic approach has been utilized to capture the spatial diversity, the observation points are either fixed or manually positioned. Behaviors of pathological vocal folds, especially those with lesions, are expected to be spatially diverse and also diverse among speakers, making fixed observation points ineffective. This paper proposes a technique to synthesize kymographic waveforms from full spatiotemporal HSV feature data to extract distinctive behaviors automatically. Each synthesized waveform represents a non-overlapping section of the glottis, where vocal folds are locally behaving homogeneously. The efficacy of the algorithm is demonstrated with four HSV recordings (three pathological) and discussed, including mitigation of the known drawbacks.

Voice pathology classification based on High-Speed Videoendoscopy

This work presents a method for automatical and objective classification of patients with healthy and pathological vocal fold vibration impairments using High-Speed Videoendoscopy of the larynx. We used an image segmentation and extraction of a novel set of numerical parameters describing the spatio-temporal dynamics of vocal folds to classification according to the normal and pathological cases and achieved 73,3% cross-validation classification accuracy. This approach is promising to develop an automatic diagnosis tool of voice disorders.

Aerodynamic measures of glottal function: what extra can they tell us and how do they guide management?

PURPOSE OF REVIEW

This article will define the major advances in laryngeal aerodynamics research from recent evidence-based literature.

RECENT FINDINGS

Recently published research focuses on new applications of aerodynamic parameters to improve patient diagnosis and outcomes, as well as further elucidating the mechanisms of phonation using computational modeling and excised larynges.

SUMMARY

Although there is an extensive amount of research on improving the diagnosis and treatment of voice disorders using aerodynamics, the majority of recent literature lacks any conclusive evidence on new methods for use in the clinic; further research in these is needed. The best practices for resonance tube phonation in water and semi-occluded voice therapy are being investigated, as is the exact mechanism by which glottal airflow interacts with vocal folds to produce phonation. It is recommended that clinicians evaluate patients with Parkinson's disease on the basis of airflow declination and lung volume expended per syllable to avoid dependence on an acoustic signal. In addition, advances in modeling laryngeal disorders and structures will contribute to future research into treatments and diagnosis. Now that the groundwork has been laid, it is crucial to begin evaluating such techniques in patient populations.

Self-entrainment of the right and left vocal fold oscillators

This article presents an analysis of entrained oscillations of the right and left vocal folds in the presence of asymmetries. A simple one-mass model is proposed for each vocal fold. A stiffness asymmetry and open glottis oscillations are considered first, and regions of oscillation are determined by a stability analysis and an averaging technique. The results show that the subglottal threshold pressure for 1:1 entrainment increases with the asymmetry. Within that region, both folds oscillate with the same amplitude and with the lax fold delayed in time with regard to the tense fold. At large asymmetries, a region involving several different phase entrainments or toroidal regimes at constant threshold pressure appears. The effect of vocal fold collisions and asymmetry in the damping coefficients of the oscillators are explored next by means of numerical analyses. It is shown that the damping asymmetry expands the 1:1 entrainment region at low subglottal pressures across the whole asymmetry range. In the expanded region, the oscillator with the lowest natural frequency is dominant and the other oscillator has a large phase advance and small amplitude. The theoretical results are finally compared with data collected from a mechanical replica of the vocal folds.

Computational study of effects of tension imbalance on phonation in a three-dimensional tubular larynx model

OBJECTIVES

The present study explores the use of a continuum-based computational model to investigate the effect of left-right tension imbalance on vocal fold (VF) vibrations and glottal aerodynamics, as well as its implication on phonation. The study allows us to gain new insights into the underlying physical mechanism of irregularities induced by VF tension imbalance associated with unilateral cricothyroid muscle paralysis.

METHODS

A three-dimensional simulation of glottal flow and VF dynamics in a tubular laryngeal model with tension imbalance was conducted by using a coupled flow-structure interaction computational model. Tension imbalance was modeled by reducing by 20% the Young's modulus of one of the VFs, while holding VF length constant. Effects of tension imbalance on vibratory characteristic of the VFs and on the time-varying properties of glottal airflow as well as the aerodynamic energy transfer are comprehensively analyzed.

RESULTS AND CONCLUSIONS

The analysis demonstrates that the continuum-based biomechanical model can provide a good description of phonatory dynamics in tension imbalance conditions. It is found that although 20% tension imbalance does not have noticeable effects on the fundamental frequency, it does lead to a larger glottal flow leakage and asymmetric vibrations of the two VFs. A detailed analysis of the energy transfer suggests that the majority of the energy is consumed by the lateral motion of the VFs and the net energy transferred to the softer fold is less than the one transferred to the normal fold.

Vocal fold augmentation with injectable polycaprolactone microspheres/pluronic F127 hydrogel: long-term in vivo study for the treatment of glottal insufficiency

There is increasing demand for reconstruction of glottal insufficiency. Several injection materials have been examined for this purpose, but all had limitations, such as poor long-term durability, migration from the injection site, inflammation, granuloma formation, and interference with vocal fold vibration due to viscoelastic mismatch. Here, we developed a novel injection material, consisting of polycaprolactone (PCL) microspheres, which exhibits better viscoelasticity than conventional materials, and Pluronic F127 carrier, which decreases the migration of the injection materials. The material was injected into rabbits with glottal insufficiency and compared with the FDA-approved injection material, calcium hydroxylapatite (CaHA). Endoscopic and histological examinations indicated that PCL/Pluronic F127 remained at the injection site with no inflammatory response or granuloma formation, whereas CaHA leaked out and migrated from the injection site. Therefore, vocal fold augmentation was almost completely retained during the 12-month follow-up period in this study. Moreover, induced phonation and high-speed recording of vocal fold vibration showed decreased vocal fold gap area in the PCL/Pluronic F127 group. Our newly developed injection material, PCL/Pluronic F127, permits efficient augmentation of paralyzed vocal fold without complications, a concept that can be applied clinically, as demonstrated by the successful long-term follow-up.

Vocal fold vibration measurements using laser Doppler vibrometry

The objective of this study was to measure the velocity of the superior surface of human vocal folds during phonation using laser Doppler vibrometry (LDV). A custom-made endoscopic laser beam deflection unit was designed and fabricated. An in vivo clinical experimental procedure was developed to simultaneously collect LDV velocity and video from videolaryngoscopy. The velocity along the direction of the laser beam, i.e., the inferior-superior direction, was captured. The velocity was synchronous with electroglottograph and sound level meter data. The vibration energy of the vocal folds was determined to be significant up to a frequency of 3 kHz. Three characteristic vibrational waveforms were identified which may indicate bifurcations between vibrational modes of the mucosal wave. No relationship was found between the velocity amplitude and phonation frequency or sound pressure level. A correlation was found between the peak-to-peak displacement amplitude and phonation frequency. A sparse map of the velocity amplitudes on the vocal fold surface was obtained.

Computational modeling of phonatory dynamics in a tubular three-dimensional model of the human larynx

Simulation of the phonatory flow-structure interaction has been conducted in a three-dimensional, tubular shaped laryngeal model that has been designed with a high level of realism with respect to the human laryngeal anatomy. A non-linear spring-based contact force model is also implemented for the purpose of representing contact in more general conditions, especially those associated with three-dimensional modeling of phonation in the presence of vocal fold pathologies. The model is used to study the effects of a moderate (20%) vocal-fold tension imbalance on the phonatory dynamics. The characteristic features of phonation for normal as well as tension-imbalanced vocal folds, such as glottal waveform, glottal jet evolution, mucosal wave-type vocal-fold motion, modal entrainment, and asymmetric glottal jet deflection have been discussed in detail and compared to established data. It is found that while a moderate level of tension asymmetry does not change the vibratory dynamics significantly, it can potentially lead to measurable deterioration in voice quality.

Nonlinear dynamic-based analysis of severe dysphonia in patients with vocal fold scar and sulcus vocalis

OBJECTIVE

The primary goal of this study was to evaluate a nonlinear dynamic approach to the acoustic analysis of dysphonia associated with vocal fold scar and sulcus vocalis.

STUDY DESIGN

Case-control study.

METHODS

Acoustic voice samples from scar/sulcus patients and age-/sex-matched controls were analyzed using correlation dimension (D2) and phase plots, time-domain based perturbation indices (jitter, shimmer, signal-to-noise ratio [SNR]), and an auditory-perceptual rating scheme. Signal typing was performed to identify samples with bifurcations and aperiodicity.

RESULTS

Type 2 and 3 acoustic signals were highly represented in the scar/sulcus patient group. When data were analyzed irrespective of signal type, all perceptual and acoustic indices successfully distinguished scar/sulcus patients from controls. Removal of type 2 and 3 signals eliminated the previously identified differences between experimental groups for all acoustic indices except D2. The strongest perceptual-acoustic correlation in our data set was observed for SNR and the weakest correlation was observed for D2.

CONCLUSIONS

These findings suggest that D2 is inferior to time-domain based perturbation measures for the analysis of dysphonia associated with scar/sulcus; however, time-domain based algorithms are inherently susceptible to inflation under highly aperiodic (ie, type 2 and 3) signal conditions. Auditory-perceptual analysis, unhindered by signal aperiodicity, is therefore a robust strategy for distinguishing scar/sulcus patient voices from normal voices. Future acoustic analysis research in this area should consider alternative (e.g., frequency- and quefrency-domain based) measures alongside additional nonlinear approaches.

Global modeling of complex data series using the term-ranking approach and its application to voice synthesis

A term-ranking approach is proposed to globally model the underlying dynamics of a chaotic series. The basic idea of this approach is to rank candidate bases before they are used to construct the global model. The ranked bases are involved in the global model one by one in a sequence from high to low until the best model is found. Simulations show that the model obtained by the term-ranking approach has a much longer prediction time, but fewer coefficients, than the widely used standard model. The proposed approach is also successfully applied to coding and synthesis of chaoslike voice data, showing promise for its use with truly noisy experimental data.

Coded strobing photography: compressive sensing of high speed periodic videos

We show that, via temporal modulation, one can observe and capture a high-speed periodic video well beyond the abilities of a low-frame-rate camera. By strobing the exposure with unique sequences within the integration time of each frame, we take coded projections of dynamic events. From a sequence of such frames, we reconstruct a high-speed video of the high-frequency periodic process. Strobing is used in entertainment, medical imaging, and industrial inspection to generate lower beat frequencies. But this is limited to scenes with a detectable single dominant frequency and requires high-intensity lighting. In this paper, we address the problem of sub-Nyquist sampling of periodic signals and show designs to capture and reconstruct such signals. The key result is that for such signals, the Nyquist rate constraint can be imposed on the strobe rate rather than the sensor rate. The technique is based on intentional aliasing of the frequency components of the periodic signal while the reconstruction algorithm exploits recent advances in sparse representations and compressive sensing. We exploit the sparsity of periodic signals in the Fourier domain to develop reconstruction algorithms that are inspired by compressive sensing.

Automatic diagnosis of vocal fold paresis by employing phonovibrogram features and machine learning methods

The clinical diagnosis of voice disorders is based on examination of the rapidly moving vocal folds during phonation (f0: 80-300Hz) with state-of-the-art endoscopic high-speed cameras. Commonly, analysis is performed in a subjective and time-consuming manner via slow-motion video playback and exhibits low inter- and intra-rater reliability. In this study an objective method to overcome this drawback is presented being based on Phonovibrography, a novel image analysis technique. For a collective of 45 normophonic and paralytic voices the laryngeal dynamics were captured by specialized Phonovibrogram features and analyzed with different machine learning algorithms. Classification accuracies reached 93% for 2-class and 73% for 3-class discrimination. The results were validated by subjective expert ratings given the same diagnostic criteria. The automatic Phonovibrogram analysis approach exceeded the experienced raters' classifications by 9%. The presented method holds a lot of potential for providing reliable vocal fold diagnosis support in the future.

Collagen injection for correcting vocal fold asymmetry: high-speed imaging

OBJECTIVES

We hypothesized that high-speed digital imaging with videokymographic and laryngotopographic analysis would provide a quantitative method to evaluate the effect of collagen injection for the correction of asymmetric and irregular vocal fold vibration in unilateral vocal fold paralysis.

METHODS

Videokymographic and laryngotopographic analysis was performed for high-speed digital recordings of vocal fold vibration for visualizing the glottal vibratory patterns, and for quantifying the frequency of vibration of each vocal fold, respectively, including comparisons between the paralyzed and normal vocal folds before and after surgery. This included prospective observations of 11 subjects with unilateral vocal fold paralysis (4 male, 7 female; mean +/- SD age, 67.1 +/- 12.0 years) using high-speed digital image analysis before and after collagen injection.

RESULTS

Analysis of the laryngotopographs revealed 2 distinct frequencies of vibration for the paralyzed and contralateral vocal folds for 8 of the 11 subjects before surgery. After collagen injection, the vibration frequencies became identical, despite asymmetric vibration amplitudes. Asymmetric vibration amplitudes were also observed in the other 3 subjects before surgery, but the amplitudes became symmetric after collagen injection, despite a persistent phase shift.

CONCLUSIONS

Asymmetric vibration in vocal fold paralysis was exemplified by differences in vibration frequency and amplitude between the vocal folds. The present study showed that after collagen injection, these aspects of vibratory patterns improved toward symmetry. This surgical procedure could improve the functional symmetry of the larynx for phonation.

Arytenoid Adduction for Correcting Vocal Fold Asymmetry: High-Speed Imaging

Objectives

We hypothesized that high-speed digital imaging provides a quantitative method to evaluate the effect of arytenoid adduction for the correction of asymmetric and irregular vocal fold vibration in unilateral vocal fold paralysis.

Methods

Six subjects with unilateral vocal fold paralysis participated in the study (4 male, 2 female; mean [±SD] age, 52.5 ± 21.3 years). Videokymographic and laryngotopographic methods for image analysis were performed for highspeed recordings of vocal fold vibration for visualizing the glottal vibratory patterns, and for quantifying the frequency of vibration of each vocal fold, respectively. Comparisons of the paralyzed and the normal vocal folds were made before and after arytenoid adduction.

Results

Analysis of the laryngotopographs revealed 2 distinct frequencies of vibration for the paralyzed and the contralateral vocal folds for all subjects before surgery. After arytenoid adduction, the vibration frequencies became identical or nearly identical in all subjects.

Conclusions

Asymmetric vibration in vocal fold paralysis was exemplified by differences in vibration frequency between the vocal folds. The present data showed that after arytenoid adduction the vibration frequencies and the vibratory patterns of the contralateral vocal folds approached symmetry. This surgical procedure could improve the functional symmetry of the larynx for phonation.

Biomechanical modeling of register transitions and the role of vocal tract resonators

Biomechanical modeling and bifurcation theory are applied to study phonation onset and register transition. A four-mass body-cover model with a smooth geometry is introduced to reproduce characteristic features of chest and falsetto registers. Sub- and supraglottal resonances are modeled using a wave-reflection model. Simulations for increasing and decreasing subglottal pressure reveal that the phonation onset exhibits amplitude jumps and hysteresis referring to a subcritical Hopf bifurcation. The onset pressure is reduced due to vocal tract resonances. Hysteresis is observed also for the voice breaks at the chest-falsetto transition. Varying the length of the subglottal resonator has only minor effects on this register transition. Contrarily, supraglottal resonances have a strong effect on the pitch, at which the chest-falsetto transition is found. Experiment of glissando singing shows that the supraglottis has indeed an influence on the register transition.

Biomechanical modeling of the three-dimensional aspects of human vocal fold dynamics

Human voice originates from the three-dimensional (3D) oscillations of the vocal folds. In previous studies, biomechanical properties of vocal fold tissues have been predicted by optimizing the parameters of simple two-mass-models to fit its dynamics to the high-speed imaging data from the clinic. However, only lateral and longitudinal displacements of the vocal folds were considered. To extend previous studies, a 3D mass-spring, cover-model is developed, which predicts the 3D vibrations of the entire medial surface of the vocal fold. The model consists of five mass planes arranged in vertical direction. Each plane contains five longitudinal, mass-spring, coupled oscillators. Feasibility of the model is assessed using a large body of dynamical data previously obtained from excised human larynx experiments, in vivo canine larynx experiments, physical models, and numerical models. Typical model output was found to be similar to existing findings. The resulting model enables visualization of the 3D dynamics of the human vocal folds during phonation for both symmetric and asymmetric vibrations.

Advances in laryngeal imaging

Imaging and image analysis became an important issue in laryngeal diagnostics. Various techniques, such as videostroboscopy, videokymography, digital kymography, or ultrasonography are available and are used in research and clinical practice. This paper reviews recent advances in imaging for laryngeal diagnostics.

Amplitude and frequency modulation control of sound production in a mechanical model of the avian syrinx

Birdsong has developed into one of the important models for motor control of learned behaviour and shows many parallels with speech acquisition in humans. However, there are several experimental limitations to studying the vocal organ - the syrinx - in vivo. The multidisciplinary approach of combining experimental data and mathematical modelling has greatly improved the understanding of neural control and peripheral motor dynamics of sound generation in birds. Here, we present a simple mechanical model of the syrinx that facilitates detailed study of vibrations and sound production. Our model resembles the 'starling resistor', a collapsible tube model, and consists of a tube with a single membrane in its casing, suspended in an external pressure chamber and driven by various pressure patterns. With this design, we can separately control 'bronchial' pressure and tension in the oscillating membrane and generate a wide variety of 'syllables' with simple sweeps of the control parameters. We show that the membrane exhibits high frequency, self-sustained oscillations in the audio range (>600 Hz fundamental frequency) using laser Doppler vibrometry, and systematically explore the conditions for sound production of the model in its control space. The fundamental frequency of the sound increases with tension in three membranes with different stiffness and mass. The lower-bound fundamental frequency increases with membrane mass. The membrane vibrations are strongly coupled to the resonance properties of the distal tube, most likely because of its reflective properties to sound waves. Our model is a gross simplification of the complex morphology found in birds, and more closely resembles mathematical models of the syrinx. Our results confirm several assumptions underlying existing mathematical models in a complex geometry.

Effects of mucosal loading on vocal fold vibration

A chain model was proposed in this study to examine the effects of mucosal loading on vocal fold vibration. Mucosal loading was defined as the loading caused by the interaction between the vocal folds and the surrounding tissue. In the proposed model, the vocal folds and the surrounding tissue were represented by a series of oscillators connected by a coupling spring. The lumped masses, springs, and dampers of the oscillators modeled the tissue properties of mass, stiffness, and viscosity, respectively. The coupling spring exemplified the tissue interactions. By numerically solving this chain model, the effects of mucosal loading on the phonation threshold pressure, phonation instability pressure, and energy distribution in a voice production system were studied. It was found that when mucosal loading is small, phonation threshold pressure increases with the damping constant R(r), the mass constant R(m), and the coupling constant R(mu) of mucosal loading but decreases with the stiffness constant R(k). Phonation instability pressure is also related to mucosal loading. It was found that phonation instability pressure increases with the coupling constant R(mu) but decreases with the stiffness constant R(k) of mucosal loading. Therefore, it was concluded that mucosal loading directly affects voice production.

Phase response curves elucidating the dynamics of coupled oscillators

Phase response curves (PRCs) are widely used in circadian clocks, neuroscience, and heart physiology. They quantify the response of an oscillator to pulse-like perturbations. Phase response curves provide valuable information on the properties of oscillators and their synchronization. This chapter discusses biological self-sustained oscillators (circadian clock, physiological rhythms, etc.) in the context of nonlinear dynamics theory. Coupled oscillators can synchronize with different frequency ratios, can generate toroidal dynamics (superposition of independent frequencies), and may lead to deterministic chaos. These nonlinear phenomena can be analyzed with the aid of a phase transition curve, which is intimately related to the phase response curve. For illustration purposes, this chapter discusses a model of circadian oscillations based on a delayed negative feedback. In a second part, the chapter provides a step-by-step recipe to measure phase response curves. It discusses specifications of this recipe for circadian rhythms, heart rhythms, neuronal spikes, central pattern generators, and insect communication. Finally, it stresses the predictive power of measured phase response curves. PRCs can be used to quantify the coupling strength of oscillations, to classify oscillator types, and to predict the complex dynamics of periodically driven oscillations.

Two-voice complexity from a single side of the syrinx in northern mockingbird Mimus polyglottos vocalizations

The diverse vocal signals of songbirds are produced by highly coordinated motor patterns of syringeal and respiratory muscles. These muscles control separate sound generators on the right and left side of the duplex vocal organ, the syrinx. Whereas most song is under active neural control, there has been a growing interest in a different class of nonlinear vocalizations consisting of frequency jumps, subharmonics, biphonation and deterministic chaos that are also present in the vocal repertoires of many vertebrates, including many birds. These nonlinear phenomena may not require active neural control, depending instead on the intrinsic nonlinear dynamics of the oscillators housed within each side of the syrinx. This study investigates the occurrence of these phenomena in the vocalizations of intact northern mockingbirds Mimus polyglottos. By monitoring respiratory pressure and airflow on each side of the syrinx, we provide the first analysis of the contribution made by each side of the syrinx to the production of nonlinear phenomena and are able to reliably discriminate two-voice vocalizations from potentially similar appearing, unilaterally produced, nonlinear events. We present the first evidence of syringeal lateralization of nonlinear dynamics during bilaterally produced chaotic calls. The occurrence of unilateral nonlinear events was not consistently correlated with fluctuations in air sac pressure or the rate of syringeal airflow. Our data support previous hypotheses for mechanical and acoustic coupling between the two sides of the syrinx. These results help lay a foundation upon which to understand the communicative functions of nonlinear phenomena.

Theoretical simulation and experimental validation of inverse quasi-one-dimensional steady and unsteady glottal flow models

In physical modeling of phonation, the pressure drop along the glottal constriction is classically assessed with the glottal geometry and the subglottal pressure as known input parameters. Application of physical modeling to study phonation abnormalities and pathologies requires input parameters related to in vivo measurable quantities commonly corresponding to the physical model output parameters. Therefore, the current research presents the inversion of some popular simplified flow models in order to estimate the subglottal pressure, the glottal constriction area, or the separation coefficient inherent to the simplified flow modeling for steady and unsteady flow conditions. The inverse models are firstly validated against direct simulations and secondly against in vitro measurements performed for different configurations of rigid vocal fold replicas mounted in a suitable experimental setup. The influence of the pressure corrections related to viscosity and flow unsteadiness on the flow modeling is quantified. The inversion of one-dimensional glottal flow models including the major viscous effects can predict the main flow quantities with respect to the in vitro measurements. However, the inverse model accuracy is strongly dependent on the pertinence of the direct flow modeling. The choice of the separation coefficient is preponderant to obtain pressure predictions relevant to the experimental data.

Spatio-temporal quantification of vocal fold vibrations using high-speed videoendoscopy and a biomechanical model

Pathologic changes within the organic constitution of vocal folds or a functional impairment of the larynx may result in disturbed or even irregular vocal fold vibrations. The consequences are perturbations of the acoustic speech signal which are perceived as a hoarse voice. By means of appropriate image processing techniques, the vocal fold dynamics are extracted from digital high-speed videos. This study addresses the approach to obtain a parametric description of the spatio-temporal characteristics of the vocal fold oscillations for the aim of classification. For this purpose a biomechanical vocal fold model is introduced. An automatic optimization procedure is developed for fitting the model dynamics to the observed vocal fold oscillations. Thus, the resulting parameter values represent a specific vibration pattern and serve as an objective quantification measure. Performance and reliability of the optimization procedure are validated with synthetically generated data sets. The high-speed videos of two normal voice subjects and six patients suffering from different voice disorders are processed. The resulting model parameters represent a rough approximation of physiological parameters along the entire vocal folds.

Spatiotemporal classification of vocal fold dynamics by a multimass model comprising time-dependent parameters

A model-based approach is proposed to objectively measure and classify vocal fold vibrations by left-right asymmetries along the anterior-posterior direction, especially in the case of nonstationary phonation. For this purpose, vocal fold dynamics are recorded in real time with a digital high-speed camera during phonation of sustained vowels as well as pitch raises. The dynamics of a multimass model with time-dependent parameters are matched to vocal fold vibrations extracted at dorsal, medial, and ventral positions by an automatic optimization procedure. The block-based optimization accounts for nonstationary vibrations and compares the vocal fold and model dynamics by wavelet coefficients. The optimization is verified with synthetically generated data sets and is applied to 40 clinical high-speed recordings comprising normal and pathological voice subjects. The resulting model parameters allow an intuitive visual assessment of vocal fold instabilities within an asymmetry diagram and are applicable to an objective quantification of asymmetries.

Phonovibrography: mapping high-speed movies of vocal fold vibrations into 2-D diagrams for visualizing and analyzing the underlying laryngeal dynamics

Endoscopic high-speed laryngoscopy in combination with image analysis strategies is the most promising approach to investigate the interrelation between vocal fold vibrations and voice disorders. So far, due to the lack of an objective and standardized analysis procedure a unique characterization of vocal fold vibrations has not been achieved yet. We present a visualization and analysis strategy which transforms the segmented edges of vibrating vocal folds into a single 2-D image, denoted Phonovibrogram (PVG). Within a PVG the individual type of vocal fold vibration becomes uniquely characterized by specific geometric patterns. The PVG geometries give an intuitive access on the type and degree of the laryngeal asymmetry and can be quantified using an image segmentation approach. The PVG analysis was applied to 14 representative recordings derived from a high-speed database comprising normal and pathological voices. We demonstrate that PVGs are capable to differentiate and quantify different types of normal and pathological vocal fold vibrations. The objective and precise quantification of the PVG geometry may have the potential to realize a novel classification of vocal fold vibrations.

Calibration of laryngeal endoscopic high-speed image sequences by an automated detection of parallel laser line projections

High-speed laryngeal endoscopic systems record vocal fold vibrations during phonation in real-time. For a quantitative analysis of vocal fold dynamics a metrical scale is required to get absolute laryngeal dimensions of the recorded image sequence. For the clinical use there is no automated and stable calibration procedure up to now. A calibration method is presented that consists of a laser projection device and the corresponding image processing for the automated detection of the laser calibration marks. The laser projection device is clipped to the endoscope and projects two parallel laser lines with a known distance to each other as calibration information onto the vocal folds. Image processing methods automatically identify the pixels belonging to the projected laser lines in the image data. The line detection bases on a Radon transform approach and is a two-stage process, which successively uses temporal and spatial characteristics of the projected laser lines in the high-speed image sequence. The robustness and the applicability are demonstrated with clinical endoscopic image sequences. The combination of the laser projection device and the image processing enables the calibration of laryngeal endoscopic images within the vocal fold plane and thus provides quantitative metrical data of vocal fold dynamics.

Comparison of biomechanical modeling of register transitions and voice instabilities with excised larynx experiments

Voice instabilities were studied using excised human larynx experiments and biomechanical modeling. With a controlled elongation of the vocal folds, the experiments showed registers with chest-like and falsetto-like vibrations. Observed instabilities included abrupt jumps between the two registers exhibiting hysteresis, aphonic episodes, subharmonics, and chaos near the register transitions. In order to model these phenomena, a three-mass model was constructed by adding a third mass on top of the simplified two-mass model. Simulation studies showed that the three-mass model can vibrate in both chest-like and falsetto-like patterns. Variation of tension parameters which mimic activities of laryngeal muscles could induce transitions between both registers. For reduced prephonatory areas and damping constants, extended coexistence of chest and falsetto registers was found, in agreement with experimental data. Subharmonics and deterministic chaos were observed close to transitions between the registers. It is concluded that the abrupt changes between chest and falsetto registers can be understood as shifts in dominance of eigenmodes of the vocal folds.

Extracting Physiologically Relevant Parameters of Vocal Folds From High-Speed Video Image Series

In this paper, a new method is proposed to extract the physiologically relevant parameters of the vocal fold mathematic model including masses, spring constants and damper constants from high-speed video (HSV) image series. This method uses a genetic algorithm to optimize the model parameters until the model and the realistic vocal folds have similar dynamic behavior. Numerical experiments theoretically test the validity of the proposed parameter estimation method. Then the validated method is applied to extract the physiologically relevant parameters from the glottal area series measured by HSV in an excised larynx model. With the estimated parameters, the vocal fold model accurately describes the vibration of the observed vocal folds. Further studies show that the proposed parameter estimation method can successfully detect the increase of longitudinal tension due to the vocal fold elongation from the glottal area signal. These results imply the potential clinical application of this method in inspecting the tissue properties of vocal fold.

Clinically evaluated procedure for the reconstruction of vocal fold vibrations from endoscopic digital high-speed videos

Investigation of voice disorders requires the examination of vocal fold vibrations. State of the art is the recording of endoscopic high-speed movies which capture vocal fold vibrations in real-time. It enables investigating the interrelation between disturbances of vocal fold vibrations and voice disorders. However, the lack of clinical studies and of a standardized procedure to reconstruct vocal fold vibrations from high-speed videos constrain the clinical acceptance of the high-speed technique. An image processing approach is presented that extracts the vibrating vocal fold edges from digital high-speed movies. The initial segmentation is principally based on a seeded region-growing algorithm. Even in movies with low image quality the algorithm segments successfully the glottal area by an introduced two-dimensional threshold matrix. Following segmentation, the vocal fold edges are reconstructed from the computed time-varying glottal area. The performance of the procedure was objectively evaluated within a study comprising 372 high-speed recordings. The accuracy of vocal fold reconstruction exceeds manual segmentation results obtained by clinical experts. The algorithm reaches an information flow-rate of up to 98 images per second. The robustness and high accuracy of the procedure makes it suitable for the application in clinical routine. It enables an objective and highly accurate description of vocal fold vibrations which is essential to realize extensive clinical studies which focus on the classification of voice disorders.