Intersegmenter Variability in High-Speed Laryngoscopy-Based Glottal Area Waveform Measures

Deep Learning-Based Detection of Glottis Segmentation Failures

Medical image segmentation is crucial for clinical applications, but challenges persist due to noise and variability. In particular, accurate glottis segmentation from high-speed videos is vital for voice research and diagnostics. Manual searching for failed segmentations is labor-intensive, prompting interest in automated methods. This paper proposes the first deep learning approach for detecting faulty glottis segmentations. For this purpose, faulty segmentations are generated by applying both a poorly performing neural network and perturbation procedures to three public datasets. Heavy data augmentations are added to the input until the neural network's performance decreases to the desired mean intersection over union (IoU). Likewise, the perturbation procedure involves a series of image transformations to the original ground truth segmentations in a randomized manner. These data are then used to train a ResNet18 neural network with custom loss functions to predict the IoU scores of faulty segmentations. This value is then thresholded with a fixed IoU of 0.6 for classification, thereby achieving 88.27% classification accuracy with 91.54% specificity. Experimental results demonstrate the effectiveness of the presented approach. Contributions include: (i) a knowledge-driven perturbation procedure, (ii) a deep learning framework for scoring and detecting faulty glottis segmentations, and (iii) an evaluation of custom loss functions.

An Investigation of Acoustic Back-Coupling in Human Phonation on a Synthetic Larynx Model

In the human phonation process, acoustic standing waves in the vocal tract can influence the fluid flow through the glottis as well as vocal fold oscillation. To investigate the amount of acoustic back-coupling, the supraglottal flow field has been recorded via high-speed particle image velocimetry (PIV) in a synthetic larynx model for several configurations with different vocal tract lengths. Based on the obtained velocity fields, acoustic source terms were computed. Additionally, the sound radiation into the far field was recorded via microphone measurements and the vocal fold oscillation via high-speed camera recordings. The PIV measurements revealed that near a vocal tract resonance frequency fR, the vocal fold oscillation frequency fo (and therefore also the flow field's fundamental frequency) jumps onto fR. This is accompanied by a substantial relative increase in aeroacoustic sound generation efficiency. Furthermore, the measurements show that fo-fR-coupling increases vocal efficiency, signal-to-noise ratio, harmonics-to-noise ratio and cepstral peak prominence. At the same time, the glottal volume flow needed for stable vocal fold oscillation decreases strongly. All of this results in an improved voice quality and phonation efficiency so that a person phonating with fo-fR-coupling can phonate longer and with better voice quality.

Effect of Ligament Fibers on Dynamics of Synthetic, Self-Oscillating Vocal Folds in a Biomimetic Larynx Model

Synthetic silicone larynx models are essential for understanding the biomechanics of physiological and pathological vocal fold vibrations. The aim of this study is to investigate the effects of artificial ligament fibers on vocal fold vibrations in a synthetic larynx model, which is capable of replicating physiological laryngeal functions such as elongation, abduction, and adduction. A multi-layer silicone model with different mechanical properties for the musculus vocalis and the lamina propria consisting of ligament and mucosa was used. Ligament fibers of various diameters and break resistances were cast into the vocal folds and tested at different tension levels. An electromechanical setup was developed to mimic laryngeal physiology. The measurements included high-speed video recordings of vocal fold vibrations, subglottal pressure and acoustic. For the evaluation of the vibration characteristics, all measured values were evaluated and compared with parameters from ex and in vivo studies. The fundamental frequency of the synthetic larynx model was found to be approximately 200-520 Hz depending on integrated fiber types and tension levels. This range of the fundamental frequency corresponds to the reproduction of a female normal and singing voice range. The investigated voice parameters from vocal fold vibration, acoustics, and subglottal pressure were within normal value ranges from ex and in vivo studies. The integration of ligament fibers leads to an increase in the fundamental frequency with increasing airflow, while the tensioning of the ligament fibers remains constant. In addition, a tension increase in the fibers also generates a rise in the fundamental frequency delivering the physiological expectation of the dynamic behavior of vocal folds.

Extent and Effect of Covering Laryngeal Structures with Synthetic Laryngeal Mucus via Two Different Administration Techniques

OBJECTIVE

The first goal of this study was to investigate the coverage of laryngeal structures using two potential administration techniques for synthetic mucus: inhalation and lozenge ingestion. As a second research question, the study investigated the potential effects of these techniques on standardized voice assessment parameters.

METHODS

Fluorescein was added to throat lozenges and to an inhalation solution to visualize the coverage of laryngeal structures through blue light imaging. The study included 70 vocally healthy subjects. Fifty subjects underwent administration via lozenge ingestion and 20 subjects performed the inhalation process. For the first research question, the recordings from the blue light imaging system were categorized to compare the extent of coverage on individual laryngeal structures objectively. Secondly, a standardized voice evaluation protocol was performed before and after each administration to determine any measurable effects of typical voice parameters.

RESULTS

The administration via inhalation demonstrated complete coverage of all laryngeal structures, including the vocal folds, ventricular folds, and arytenoid cartilages, as visualized by the fluorescent dye. In contrast, the application of the lozenge predominantly covered the pharynx and laryngeal surface toward the aryepiglottic fold, but not the inferior structures. All in all, the comparison before and after administration showed no clear effect, although a minor deterioration of the acoustic signal was noted in the shimmer and cepstral peak prominence after the inhalation.

CONCLUSIONS

Our findings indicate that the inhalation process is a more effective technique for covering deeper laryngeal structures such as the vocal folds and ventricular folds with synthetic mucus. This knowledge enables further in vivo studies on the role of laryngeal mucus in phonation in general, and how it can be substituted or supplemented for patients with reduced glandular activity as well as for heavy voice users.

Mechanical Parameters Based on High-Speed Videoendoscopy of the Vocal Folds in Patients With Ectodermal Dysplasia

OBJECTIVE

Patients suffering from ectodermal dysplasia (ED), which is an inherited disorder in the development of the ectodermal structures, have a significantly reduced expression of teeth, hair, sweat glands, and salivary glands in the respiratory tract including the larynx. Previous studies within the framework of the present project showed a significantly reduced saliva production and an impairment of the acoustic outcome in ED patients compared to the control group. However, until now, no statistically significant difference between EDs and controls could be found regarding vocal fold dynamics in the high-speed videoendoscopy (HSV) recordings using representative parameters on closure, symmetry, and periodicity. The aim of this study is to examine the role of tissue characteristics by means of objective mechanical parameters derived from HSV recordings.

METHODS

This study includes 28 ED patients and 42 controls (no ED, healthy voice). The vocal fold oscillations were recorded by high-speed videoendoscopy (HSV@4kHz). Based on the dynamical measures of the glottal area waveform (GAW), objective glottal dynamic parameters associated with tissue properties like flexibility and stiffness were computed.

RESULTS

The present evaluation displays a significant difference between male ED patients and male controls concerning the HSV-based mechanical parameters indicating reduced stiffness and increased deformability for the vocal folds of male ED patients. In contrast to strongly amplitude-dependent parameters, the primarily velocity-based parameters showed no statistically significant deviation.

CONCLUSIONS

The presented data provides the first promising indication toward the underlying causes on the laryngeal level leading to the voice conspicuities in ED patients. The significant difference concerning the mechanical parameters suggests a different composition of the extracellular matrix of the tissue of the vocal folds of ED patients compared to controls.

Synthetic mucus for an ex vivo phonation setup: Creation, application, and effect on excised porcine larynges

Laryngeal mucus hydrates and lubricates the deformable tissue of the vocal folds and acts as a boundary layer with the airflow from the lungs. However, the effects of the mucus' viscoelasticity on phonation remain widely unknown and mucus has not yet been established in experimental procedures of voice research. In this study, four synthetic mucus samples were created on the basis of xanthan with focus on physiological frequency-dependent viscoelastic properties, which cover viscosities and elasticities over 2 orders of magnitude. An established ex vivo experimental setup was expanded by a reproducible and controllable application method of synthetic mucus. The application method and the suitability of the synthetic mucus samples were successfully verified by fluorescence evidence on the vocal folds even after oscillation experiments. Subsequently, the impact of mucus viscoelasticity on the oscillatory dynamics of the vocal folds, the subglottal pressure, and acoustic signal was investigated with 24 porcine larynges (2304 datasets). Despite the large differences of viscoelasticity, the phonatory characteristics remained stable with only minor statistically significant differences. Overall, this study increased the level of realism in the experimental setup for replication of the phonatory process enabling further research on pathological mucus and exploration of therapeutic options.

Re-Training of Convolutional Neural Networks for Glottis Segmentation in Endoscopic High-Speed Videos

Endoscopic high-speed video (HSV) systems for visualization and assessment of vocal fold dynamics in the larynx are diverse and technically advancing. To consider resulting "concepts shifts" for neural network (NN)-based image processing, re-training of already trained and used NNs is necessary to allow for sufficiently accurate image processing for new recording modalities. We propose and discuss several re-training approaches for convolutional neural networks (CNN) being used for HSV image segmentation. Our baseline CNN was trained on the BAGLS data set (58,750 images). The new BAGLS-RT data set consists of additional 21,050 images from previously unused HSV systems, light sources, and different spatial resolutions. Results showed that increasing data diversity by means of preprocessing already improves the segmentation accuracy (mIoU + 6.35%). Subsequent re-training further increases segmentation performance (mIoU + 2.81%). For re-training, finetuning with dynamic knowledge distillation showed the most promising results. Data variety for training and additional re-training is a helpful tool to boost HSV image segmentation quality. However, when performing re-training, the phenomenon of catastrophic forgetting should be kept in mind, i.e., adaption to new data while forgetting already learned knowledge.

Laryngeal and Acoustic Analysis of Chest and Head Registers Extended Across a Three-Octave Range: A Case Study

Voice registers are assumed to be related to different laryngeal adjustments, but objective evidence has been insufficient. While chest register is usually associated with the lower pitch range, and head register with the higher pitch range, here we investigated a professional singer who claimed an ability to produce both these registers at every pitch, throughout her entire singing range. The singer performed separated phonations alternating between the two registers (further called chest-like and head-like) at all pitches from C3 (131 Hz) to C6 (1047 Hz). We monitored the vocal fold vibrations using high-speed video endoscopy and electroglottography. The microphone sound was recorded and used for blind listening tests performed by the three authors (insiders) and by six "naive" participants (outsiders). The outsiders correctly identified the registers in 64% of the cases, and the insiders in 89% of the cases. Objective analysis revealed larger closed quotient and vertical phase differences for the chest-like register within the lower range below G4 (<392 Hz), and also a larger closed quotient at the membranous glottis within the higher range above Bb4 (>466 Hz), but not between Ab4-A4 (415-440 Hz). The normalized amplitude quotient was consistently lower in the chest-like register throughout the entire range. The results indicate that that the singer employed subtle laryngeal control mechanisms for the chest-like and head-like phonations on top of the traditionally recognized low-pitched chest and high-pitched head register phenomena. Across all pitches, the chest-like register was produced with more rapid glottal closure that was usually, but not necessarily, accompanied also by stronger adduction of membranous glottis. These register changes were not always easily perceivable by listeners, however.

Comparative analysis of high-speed videolaryngoscopy images and sound data simultaneously acquired from rigid and flexible laryngoscope: a pilot study

High-Speed Videoendoscopy (HSV) is becoming a robust tool for the assessment of vocal fold vibration in laboratory investigation and clinical practice. We describe the first successful application of flexible High Speed Videoendoscopy with innovative laser light source conducted in clinical settings. The acquired image and simultaneously recorded audio data are compared to the results obtained by means of a rigid endoscope. We demonstrated that the HSV recordings with fiber-optic laryngoscope have enabled obtaining consistently bright, color images suitable for parametrization of vocal fold oscillation similarly as in the case of the HSV data obtained from a rigid laryngoscope. The comparison of period and amplitude perturbation parameters calculated on the basis of image and audio data acquired from flexible and rigid HSV recording objectively confirm that flexible High-Speed Videoendoscopy is a more suitable method for examination of natural phonation. The HSV-based measures generated from this kymographic analysis are arguably a superior representation of the vocal fold vibrations than the acoustic analysis because their quantification is independent of the vocal tract influences. This experimental study has several implications for further research in the field of HSV application in clinical assessment of glottal pathologies nature and its effect on vocal folds vibrations.

OpenHSV: an open platform for laryngeal high-speed videoendoscopy

High-speed videoendoscopy is an important tool to study laryngeal dynamics, to quantify vocal fold oscillations, to diagnose voice impairments at laryngeal level and to monitor treatment progress. However, there is a significant lack of an open source, expandable research tool that features latest hardware and data analysis. In this work, we propose an open research platform termed OpenHSV that is based on state-of-the-art, commercially available equipment and features a fully automatic data analysis pipeline. A publicly available, user-friendly graphical user interface implemented in Python is used to interface the hardware. Video and audio data are recorded in synchrony and are subsequently fully automatically analyzed. Video segmentation of the glottal area is performed using efficient deep neural networks to derive glottal area waveform and glottal midline. Established quantitative, clinically relevant video and audio parameters were implemented and computed. In a preliminary clinical study, we recorded video and audio data from 28 healthy subjects. Analyzing these data in terms of image quality and derived quantitative parameters, we show the applicability, performance and usefulness of OpenHSV. Therefore, OpenHSV provides a valid, standardized access to high-speed videoendoscopy data acquisition and analysis for voice scientists, highlighting its use as a valuable research tool in understanding voice physiology. We envision that OpenHSV serves as basis for the next generation of clinical HSV systems.

Are source-filter interactions detectable in classical singing during vowel glides?

In recent studies, it has been assumed that vocal tract formants (F_n) and the voice source could interact. However, there are only few studies analyzing this assumption in vivo. Here, the vowel transition /i/-/a/-/u/-/i/ of 12 professional classical singers (6 females, 6 males) when phonating on the pitch D4 [fundamental frequency (ƒ_o) ca. 294 Hz] were analyzed using transnasal high speed videoendoscopy (20.000 fps), electroglottography (EGG), and audio recordings. F_n data were calculated using a cepstral method. Source-filter interaction candidates (SFICs) were determined by (a) algorithmic detection of major intersections of F_n/nƒ_o and (b) perceptual assessment of the EGG signal. Although the open quotient showed some increase for the /i-a/ and /u-i/ transitions, there were no clear effects at the expected F_n/nƒ_o intersections. In contrast, ƒ_o adjustments and changes in the phonovibrogram occurred at perceptually derived SFICs, suggesting level-two interactions. In some cases, these were constituted by intersections between higher nƒ_o and F_n. The presented data partially corroborates that vowel transitions may result in level-two interactions also in professional singers. However, the lack of systematically detectable effects suggests either the absence of a strong interaction or existence of confounding factors, which may potentially counterbalance the level-two-interactions.

Fluid-structure-acoustic interactions in an ex vivo porcine phonation model

In the clinic, many diagnostic and therapeutic procedures focus on the oscillation patterns of the vocal folds (VF). Dynamic characteristics of the VFs, such as symmetry, periodicity, and full glottal closure, are considered essential features for healthy phonation. However, the relevance of these individual factors in the complex interaction between the airflow, laryngeal structures, and the resulting acoustics has not yet been quantified. Sustained phonation was induced in nine excised porcine larynges without vocal tract (supraglottal structures had been removed above the ventricular folds). The multimodal setup was designed to simultaneously control and monitor key aspects of phonation in the three essential parts of the larynx. More specifically, measurements will comprise (1) the subglottal pressure signal, (2) high-speed recordings in the glottal plane, and (3) the acoustic signal in the supraglottal region. The automated setup regulates glottal airflow, asymmetric arytenoid adduction, and the pre-phonatory glottal gap. Statistical analysis revealed a beneficial influence of VF periodicity and glottal closure on the signal quality of the subglottal pressure and the supraglottal acoustics, whereas VF symmetry only had a negligible influence. Strong correlations were found between the subglottal and supraglottal signal quality, with significant improvement of the acoustic quality for high levels of periodicity and glottal closure.

Rethinking glottal midline detection

A healthy voice is crucial for verbal communication and hence in daily as well as professional life. The basis for a healthy voice are the sound producing vocal folds in the larynx. A hallmark of healthy vocal fold oscillation is the symmetric motion of the left and right vocal fold. Clinically, videoendoscopy is applied to assess the symmetry of the oscillation and evaluated subjectively. High-speed videoendoscopy, an emerging method that allows quantification of the vocal fold oscillation, is more commonly employed in research due to the amount of data and the complex, semi-automatic analysis. In this study, we provide a comprehensive evaluation of methods that detect fully automatically the glottal midline. We used a biophysical model to simulate different vocal fold oscillations, extended the openly available BAGLS dataset using manual annotations, utilized both, simulations and annotated endoscopic images, to train deep neural networks at different stages of the analysis workflow, and compared these to established computer vision algorithms. We found that classical computer vision perform well on detecting the glottal midline in glottis segmentation data, but are outperformed by deep neural networks on this task. We further suggest GlottisNet, a multi-task neural architecture featuring the simultaneous prediction of both, the opening between the vocal folds and the symmetry axis, leading to a huge step forward towards clinical applicability of quantitative, deep learning-assisted laryngeal endoscopy, by fully automating segmentation and midline detection.

BAGLS, a multihospital Benchmark for Automatic Glottis Segmentation

Laryngeal videoendoscopy is one of the main tools in clinical examinations for voice disorders and voice research. Using high-speed videoendoscopy, it is possible to fully capture the vocal fold oscillations, however, processing the recordings typically involves a time-consuming segmentation of the glottal area by trained experts. Even though automatic methods have been proposed and the task is particularly suited for deep learning methods, there are no public datasets and benchmarks available to compare methods and to allow training of generalizing deep learning models. In an international collaboration of researchers from seven institutions from the EU and USA, we have created BAGLS, a large, multihospital dataset of 59,250 high-speed videoendoscopy frames with individually annotated segmentation masks. The frames are based on 640 recordings of healthy and disordered subjects that were recorded with varying technical equipment by numerous clinicians. The BAGLS dataset will allow an objective comparison of glottis segmentation methods and will enable interested researchers to train their own models and compare their methods.