1
|
Paderno A, Bedi N, Rau A, Holsinger CF. Computer Vision and Videomics in Otolaryngology-Head and Neck Surgery: Bridging the Gap Between Clinical Needs and the Promise of Artificial Intelligence. Otolaryngol Clin North Am 2024; 57:703-718. [PMID: 38981809 DOI: 10.1016/j.otc.2024.05.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/11/2024]
Abstract
This article discusses the role of computer vision in otolaryngology, particularly through endoscopy and surgery. It covers recent applications of artificial intelligence (AI) in nonradiologic imaging within otolaryngology, noting the benefits and challenges, such as improving diagnostic accuracy and optimizing therapeutic outcomes, while also pointing out the necessity for enhanced data curation and standardized research methodologies to advance clinical applications. Technical aspects are also covered, providing a detailed view of the progression from manual feature extraction to more complex AI models, including convolutional neural networks and vision transformers and their potential application in clinical settings.
Collapse
Affiliation(s)
- Alberto Paderno
- IRCCS Humanitas Research Hospital, via Manzoni 56, Rozzano, Milan 20089, Italy; Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, Pieve Emanuele, Milan 20072, Italy.
| | - Nikita Bedi
- Division of Head and Neck Surgery, Department of Otolaryngology, Stanford University, Palo Alto, CA, USA
| | - Anita Rau
- Department of Biomedical Data Science, Stanford University, Palo Alto, CA, USA
| | | |
Collapse
|
2
|
Maskeliūnas R, Damaševičius R, Kulikajevas A, Pribuišis K, Uloza V. Alaryngeal Speech Enhancement for Noisy Environments Using a Pareto Denoising Gated LSTM. J Voice 2024:S0892-1997(24)00228-5. [PMID: 39107213 DOI: 10.1016/j.jvoice.2024.07.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2024] [Revised: 07/13/2024] [Accepted: 07/15/2024] [Indexed: 08/09/2024]
Abstract
Loss of the larynx significantly alters natural voice production, requiring alternative communication modalities and rehabilitation methods to restore speech intelligibility and improve the quality of life of affected individuals. This paper explores advances in alaryngeal speech enhancement to improve signal quality and reduce background noise, focusing on individuals who have undergone laryngectomy. In this study, speech samples were obtained from 23 Lithuanian males who had undergone laryngectomy with secondary implantation of the tracheoesophageal prosthesis (TEP). Pareto-optimized gated long short-term memory was trained on tracheoesophageal speech data to recognize complex temporal connections and contextual information in speech signals. The system was able to distinguish between actual speech and various forms of noise and artifacts, resulting in a 25% drop in the mean signal-to-noise ratio compared to other approaches. According to acoustic analysis, the system significantly decreased the number of unvoiced frames (proportion of voiced frames) from 40% to 10% while maintaining stable proportions of voiced frames (proportion of voiced speech frames) and average voicing evidence (average voice evidence in voiced frames), indicating the accuracy of the approach in selectively attenuating noise and undesired speech artifacts while preserving important speech information.
Collapse
Affiliation(s)
- Rytis Maskeliūnas
- Centre of Real Time Computer Systems, Kaunas University of Technology, Kaunas, Lithuania.
| | - Robertas Damaševičius
- Centre of Real Time Computer Systems, Kaunas University of Technology, Kaunas, Lithuania
| | - Audrius Kulikajevas
- Centre of Real Time Computer Systems, Kaunas University of Technology, Kaunas, Lithuania
| | - Kipras Pribuišis
- Department of Otolaryngology, Lithuanian University of Health Sciences, Kaunas, Lithuania
| | - Virgilijus Uloza
- Department of Otolaryngology, Lithuanian University of Health Sciences, Kaunas, Lithuania
| |
Collapse
|
3
|
Nobel SMN, Swapno SMMR, Islam MR, Safran M, Alfarhood S, Mridha MF. A machine learning approach for vocal fold segmentation and disorder classification based on ensemble method. Sci Rep 2024; 14:14435. [PMID: 38910146 DOI: 10.1038/s41598-024-64987-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2024] [Accepted: 06/14/2024] [Indexed: 06/25/2024] Open
Abstract
In the healthcare domain, the essential task is to understand and classify diseases affecting the vocal folds (VFs). The accurate identification of VF disease is the key issue in this domain. Integrating VF segmentation and disease classification into a single system is challenging but important for precise diagnostics. Our study addresses this challenge by combining VF illness categorization and VF segmentation into a single integrated system. We utilized two effective ensemble machine learning methods: ensemble EfficientNetV2L-LGBM and ensemble UNet-BiGRU. We utilized the EfficientNetV2L-LGBM model for classification, achieving a training accuracy of 98.88%, validation accuracy of 97.73%, and test accuracy of 97.88%. These exceptional outcomes highlight the system's ability to classify different VF illnesses precisely. In addition, we utilized the UNet-BiGRU model for segmentation, which attained a training accuracy of 92.55%, a validation accuracy of 89.87%, and a significant test accuracy of 91.47%. In the segmentation task, we examined some methods to improve our ability to divide data into segments, resulting in a testing accuracy score of 91.99% and an Intersection over Union (IOU) of 87.46%. These measures demonstrate skill of the model in accurately defining and separating VF. Our system's classification and segmentation results confirm its capacity to effectively identify and segment VF disorders, representing a significant advancement in enhancing diagnostic accuracy and healthcare in this specialized field. This study emphasizes the potential of machine learning to transform the medical field's capacity to categorize VF and segment VF, providing clinicians with a vital instrument to mitigate the profound impact of the condition. Implementing this innovative approach is expected to enhance medical procedures and provide a sense of optimism to those globally affected by VF disease.
Collapse
Affiliation(s)
- S M Nuruzzaman Nobel
- Department of Computer Science and Engineering, Bangladesh University of Business and Technology, Dhaka, 1216, Bangladesh
| | - S M Masfequier Rahman Swapno
- Department of Computer Science and Engineering, Bangladesh University of Business and Technology, Dhaka, 1216, Bangladesh
| | - Md Rajibul Islam
- Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong, China
| | - Mejdl Safran
- Department of Computer Science, College of Computer and Information Sciences, King Saud University, P. O. Box 51178, 11543, Riyadh, Saudi Arabia.
| | - Sultan Alfarhood
- Department of Computer Science, College of Computer and Information Sciences, King Saud University, P. O. Box 51178, 11543, Riyadh, Saudi Arabia
| | - M F Mridha
- Department of Computer Science, American International University-Bangladesh, Dhaka, 1229, Bangladesh
| |
Collapse
|
4
|
Ferrari M, Mularoni F, Taboni S, Crosetti E, Pessina C, Carobbio ALC, Montalto N, Marchi F, Vural A, Paderno A, Caprioli S, Gaudioso P, Fermi M, Rigoni F, Saccardo T, Contro G, Ruaro A, Lo Manto A, Varago C, Baldovin M, Bandolin L, Filauro M, Sampieri C, Missale F, Ioppi A, Carta F, Ramanzin M, Ravanelli M, Maiolo V, Bertotto I, Del Bon F, Lancini D, Mariani C, Marrosu V, Tatti M, Cağlı S, Yüce I, Gündoğ M, Dogan S, Anile G, Gottardi C, Busato F, Vallin A, Gennarini F, Bossi P, Ghi MG, Lionello M, Zanoletti E, Marioni G, Maroldi R, Mattioli F, Puxeddu R, Bertolin A, Presutti L, Piazza C, Succo G, Peretti G, Nicolai P. How reliable is assessment of true vocal cord-arytenoid unit mobility in patients affected by laryngeal cancer? a multi-institutional study on 366 patients from the ARYFIX collaborative group. Oral Oncol 2024; 152:106744. [PMID: 38520756 DOI: 10.1016/j.oraloncology.2024.106744] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Revised: 03/01/2024] [Accepted: 03/05/2024] [Indexed: 03/25/2024]
Abstract
PURPOSE In clinical practice the assessment of the "vocal cord-arytenoid unit" (VCAU) mobility is crucial in the staging, prognosis, and choice of treatment of laryngeal squamous cell carcinoma (LSCC). The aim of the present study was to measure repeatability and reliability of clinical assessment of VCAU mobility and radiologic analysis of posterior laryngeal extension. METHODS In this multi-institutional retrospective study, patients with LSCC-induced impairment of VCAU mobility who received curative treatment were included; pre-treatment endoscopy and contrast-enhanced imaging were collected and evaluated by raters. According to their evaluations, concordance, number of assigned categories, and inter- and intra-rater agreement were calculated. RESULTS Twenty-two otorhinolaryngologists evaluated 366 videolaryngoscopies (total evaluations: 2170) and 6 radiologists evaluated 237 imaging studies (total evaluations: 477). The concordance of clinical rating was excellent in only 22.7% of cases. Overall, inter- and intra-rater agreement was weak. Supraglottic cancers and transoral endoscopy were associated with the lowest inter-observer reliability values. Radiologic inter-rater agreement was low and did not vary with imaging technique. Intra-rater reliability of radiologic evaluation was optimal. CONCLUSIONS The current methods to assess VCAU mobility and posterior extension of LSCC are flawed by weak inter-observer agreement and reliability. Radiologic evaluation was characterized by very high intra-rater agreement, but weak inter-observer reliability. The relevance of VCAU mobility assessment in laryngeal oncology should be re-weighted. Patients affected by LSCC requiring imaging should be referred to dedicated radiologists with experience in head and neck oncology.
Collapse
Affiliation(s)
- M Ferrari
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy.
| | - F Mularoni
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| | - S Taboni
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| | - E Crosetti
- Otorhinolaryngology Department - Head Neck Cancer Unit, San Giovanni Bosco Hospital, Turin, Italy
| | - C Pessina
- Radiology Unit, Sant'Antonio Hospital, University of Padova, Padova, Italy
| | - A L C Carobbio
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| | - N Montalto
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| | - F Marchi
- Unit of Otorhinolaryngology, Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Largo Rosanna Benzi 10, 16121 Genoa, Italy; Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genova, Genoa, Italy
| | - A Vural
- Otorhinolaryngology Unit, Istanbul University - Cerrahpasa, Cerrahpasa Medical Faculty, Istanbul, Turkey
| | - A Paderno
- IRCCS Humanitas Research Hospital, via Manzoni 56, 20089 Rozzano, Milan, Italy
| | - S Caprioli
- Radiology Unit, San Martino Hospital, Genoa, Italy
| | - P Gaudioso
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| | - M Fermi
- Otorhinolaryngology Unit, Azienda Ospedaliera Universitaria di Bologna IRCCS, Bologna, Italy; Department of Surgical and Medical Sciences, University of Bologna, Bologna, Italy
| | - F Rigoni
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| | - T Saccardo
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| | - G Contro
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| | - A Ruaro
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| | - A Lo Manto
- Otorhinolaryngology Unit, Infermi Hospital, Rimini, Italy
| | - C Varago
- Otorhinolaryngology Unit, Vittorio Veneto Hospital, Treviso, Italy
| | - M Baldovin
- Otorhinolaryngology Unit, San Martino di Belluno Hospital, Belluno, Italy
| | - L Bandolin
- Otorhinolaryngology Unit, Hospital of Santorso, Vicenza, Italy
| | - M Filauro
- Unit of Otorhinolaryngology, Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Largo Rosanna Benzi 10, 16121 Genoa, Italy; Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genova, Genoa, Italy
| | - C Sampieri
- Department of Experimental Medicine (DIMES), University of Genoa, Italy; Unit of Head and Neck Tumors, Hospital Clínic, Barcelona, Spain
| | - F Missale
- Department of Otorhinolaryngology, Head and Neck Surgery, Maastricht University Medical Center, 6202 AZ Maastricht, the Netherlands
| | - A Ioppi
- Department of Otorhinolaryngology-Head and Neck Surgery, "S. Chiara" Hospital, Azienda Provinciale per i Servizi Sanitari (APSS), Trento, Italy
| | - F Carta
- Otorhinolaryngology Unit, Azienda Ospedaliero-Universitaria di Cagliari, Cagliari, Italy
| | - M Ramanzin
- Radiology Unit, Hospital of Vicenza, Vicenza, Italy
| | - M Ravanelli
- Radiology Unit, ASST Spedali Civili di Brescia, University of Brescia, Brescia, Italy
| | - V Maiolo
- Pediatric and Adult Cardiothoracic and Vascular, Oncohematologic and Emergency Radiology Unit (IRCCS AOUBO), University of Bologna, Italy
| | - I Bertotto
- Radiology Unit, IRCCS Istituto di Candiolo, Turin, Italy
| | - F Del Bon
- Otorhinolaryngology Unit, ASST Spedali Civili di Brescia, University of Brescia, Brescia, Italy
| | - D Lancini
- Otorhinolaryngology Unit, ASST Spedali Civili di Brescia, University of Brescia, Brescia, Italy
| | - C Mariani
- Otorhinolaryngology Unit, Azienda Ospedaliero-Universitaria di Cagliari, Cagliari, Italy
| | - V Marrosu
- Otorhinolaryngology Unit, Azienda Ospedaliero-Universitaria di Cagliari, Cagliari, Italy
| | - M Tatti
- Otorhinolaryngology Unit, Azienda Ospedaliero-Universitaria di Cagliari, Cagliari, Italy
| | - S Cağlı
- Otorhinolaryngology Unit, Erciyes University, Kayseri, Turkey
| | - I Yüce
- Otorhinolaryngology Unit, Erciyes University, Kayseri, Turkey
| | - M Gündoğ
- Department of Radiation Oncology, Erciyes University, Kayseri, Turkey
| | - S Dogan
- Department of Radiology, Erciyes University, Kayseri, Turkey
| | - G Anile
- Unit of Medical Oncology 2, "Istituto Oncologico Veneto", Padova, Italy
| | - C Gottardi
- Unit of Medical Oncology 2, "Istituto Oncologico Veneto", Padova, Italy
| | - F Busato
- Unit of Radiation Oncology, Policlinico Abano, Padova, Italy
| | - A Vallin
- Unit of Otorhinolaryngology, Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Largo Rosanna Benzi 10, 16121 Genoa, Italy; Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genova, Genoa, Italy
| | - F Gennarini
- Otorhinolaryngology Unit, ASST Spedali Civili di Brescia, University of Brescia, Brescia, Italy
| | - P Bossi
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, 20072 Pieve Emanuele, Milan, Italy; IRCCS Humanitas Research Hospital, via Manzoni 56, 20089 Rozzano, Milan, Italy
| | - M G Ghi
- Unit of Medical Oncology 2, "Istituto Oncologico Veneto", Padova, Italy
| | - M Lionello
- Otorhinolaryngology Unit, Vittorio Veneto Hospital, Treviso, Italy
| | - E Zanoletti
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| | - G Marioni
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| | - R Maroldi
- Radiology Unit, ASST Spedali Civili di Brescia, University of Brescia, Brescia, Italy
| | - F Mattioli
- Otorhinolaryngology Unit, University of Modena, Modena, Italy
| | - R Puxeddu
- Otorhinolaryngology Unit, Azienda Ospedaliero-Universitaria di Cagliari, Cagliari, Italy; King's College Hospital London, Dubai, United Arab Emirates
| | - A Bertolin
- Otorhinolaryngology Unit, Vittorio Veneto Hospital, Treviso, Italy
| | - L Presutti
- Otorhinolaryngology Unit, Azienda Ospedaliera Universitaria di Bologna IRCCS, Bologna, Italy; Department of Surgical and Medical Sciences, University of Bologna, Bologna, Italy
| | - C Piazza
- Otorhinolaryngology Unit, ASST Spedali Civili di Brescia, University of Brescia, Brescia, Italy
| | - G Succo
- Otorhinolaryngology Department - Head Neck Cancer Unit, San Giovanni Bosco Hospital, Turin, Italy; Oncology Department, University of Turin, Turin, Italy
| | - G Peretti
- Unit of Otorhinolaryngology, Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Largo Rosanna Benzi 10, 16121 Genoa, Italy; Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genova, Genoa, Italy
| | - P Nicolai
- Otorhinolaryngology and Head and Neck Unit, Department of Neuroscience, Azienda Ospedale Università di Padova, Padova, Italy
| |
Collapse
|
5
|
Dadras AA, Aichinger P. Deep Learning-Based Detection of Glottis Segmentation Failures. Bioengineering (Basel) 2024; 11:443. [PMID: 38790311 PMCID: PMC11118004 DOI: 10.3390/bioengineering11050443] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2024] [Revised: 04/23/2024] [Accepted: 04/26/2024] [Indexed: 05/26/2024] Open
Abstract
Medical image segmentation is crucial for clinical applications, but challenges persist due to noise and variability. In particular, accurate glottis segmentation from high-speed videos is vital for voice research and diagnostics. Manual searching for failed segmentations is labor-intensive, prompting interest in automated methods. This paper proposes the first deep learning approach for detecting faulty glottis segmentations. For this purpose, faulty segmentations are generated by applying both a poorly performing neural network and perturbation procedures to three public datasets. Heavy data augmentations are added to the input until the neural network's performance decreases to the desired mean intersection over union (IoU). Likewise, the perturbation procedure involves a series of image transformations to the original ground truth segmentations in a randomized manner. These data are then used to train a ResNet18 neural network with custom loss functions to predict the IoU scores of faulty segmentations. This value is then thresholded with a fixed IoU of 0.6 for classification, thereby achieving 88.27% classification accuracy with 91.54% specificity. Experimental results demonstrate the effectiveness of the presented approach. Contributions include: (i) a knowledge-driven perturbation procedure, (ii) a deep learning framework for scoring and detecting faulty glottis segmentations, and (iii) an evaluation of custom loss functions.
Collapse
Affiliation(s)
| | - Philipp Aichinger
- Speech and Hearing Science Lab, Division of Phoniatrics-Logopedics, Department of Otorhinolaryngology, Medical University of Vienna, Währinger Gürtel 18-20, 1090 Vienna, Austria;
| |
Collapse
|
6
|
Darvish M, Kist AM. A Generative Method for a Laryngeal Biosignal. J Voice 2024:S0892-1997(24)00019-5. [PMID: 38395653 DOI: 10.1016/j.jvoice.2024.01.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Revised: 01/26/2024] [Accepted: 01/26/2024] [Indexed: 02/25/2024]
Abstract
The Glottal Area Waveform (GAW) is an important component in quantitative clinical voice assessment, providing valuable insights into vocal fold function. In this study, we introduce a novel method employing Variational Autoencoders (VAEs) to generate synthetic GAWs. Our approach enables the creation of synthetic GAWs that closely replicate real-world data, offering a versatile tool for researchers and clinicians. We elucidate the process of manipulating the VAE latent space using the Glottal Opening Vector (GlOVe). The GlOVe allows precise control over the synthetic closure and opening of the vocal folds. By utilizing the GlOVe, we generate synthetic laryngeal biosignals. These biosignals accurately reflect vocal fold behavior, allowing for the emulation of realistic glottal opening changes. This manipulation extends to the introduction of arbitrary oscillations in the vocal folds, closely resembling real vocal fold oscillations. The range of factor coefficient values enables the generation of diverse biosignals with varying frequencies and amplitudes. Our results demonstrate that this approach yields highly accurate laryngeal biosignals, with the Normalized Mean Absolute Error values for various frequencies ranging from 9.6 ⋅ 10-3 to 1.20 ⋅ 10-2 for different experimented frequencies, alongside a remarkable training effectiveness, reflected in reductions of up to approximately 89.52% in key loss components. This proposed method may have implications for downstream speech synthesis and phonetics research, offering the potential for advanced and natural-sounding speech technologies.
Collapse
Affiliation(s)
- Mahdi Darvish
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Andreas M Kist
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.
| |
Collapse
|
7
|
Malinowski J, Pietruszewska W, Kowalczyk M, Niebudek-Bogusz E. Value of high-speed videoendoscopy as an auxiliary tool in differentiation of benign and malignant unilateral vocal lesions. J Cancer Res Clin Oncol 2024; 150:10. [PMID: 38216796 PMCID: PMC10786956 DOI: 10.1007/s00432-023-05543-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 12/13/2023] [Indexed: 01/14/2024]
Abstract
PURPOSE The study aimed to assess the relevance of objective vibratory parameters derived from high-speed videolaryngoscopy (HSV) as a supporting tool, to assist clinicians in establishing the initial diagnosis of benign and malignant glottal organic lesions. METHODS The HSV examinations were conducted in 175 subjects: 50 normophonic, 85 subjects with benign vocal fold lesions, and 40 with early glottic cancer; organic lesions were confirmed by histopathologic examination. The parameters, derived from HSV kymography: amplitude, symmetry, and glottal dynamic characteristics, were compared statistically between the groups with the following ROC analysis. RESULTS Among 14 calculated parameters, 10 differed significantly between the groups. Four of them, the average resultant amplitude of the involved vocal fold (AmpInvolvedAvg), average amplitude asymmetry for the whole glottis and its middle third part (AmplAsymAvg; AmplAsymAvg_2/3), and absolute average phase difference (AbsPhaseDiffAvg), showed significant differences between benign and malignant lesions. Amplitude values were decreasing, while asymmetry and phase difference values were increasing with the risk of malignancy. In ROC analysis, the highest AUC was observed for AmpAsymAvg (0.719; p < 0.0001), and next in order was AmpInvolvedAvg (0.70; p = 0.0002). CONCLUSION The golden standard in the diagnosis of organic lesions of glottis remains clinical examination with videolaryngoscopy, confirmed by histopathological examination. Our results showed that measurements of amplitude, asymmetry, and phase of vibrations in malignant vocal fold masses deteriorate significantly in comparison to benign vocal lesions. High-speed videolaryngoscopy could aid their preliminary differentiation noninvasively before histopathological examination; however, further research on larger groups is needed.
Collapse
Affiliation(s)
- Jakub Malinowski
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, Lodz, Poland.
| | - Wioletta Pietruszewska
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, Lodz, Poland
| | - Magdalena Kowalczyk
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, Lodz, Poland
| | - Ewa Niebudek-Bogusz
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, Lodz, Poland
| |
Collapse
|
8
|
Pennington-FitzGerald W, Joshi A, Honzel E, Hernandez-Morato I, Pitman MJ, Moayedi Y. Development and Application of Automated Vocal Fold Tracking Software in a Rat Surgical Model. Laryngoscope 2024; 134:340-346. [PMID: 37543969 DOI: 10.1002/lary.30930] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Revised: 06/21/2023] [Accepted: 07/15/2023] [Indexed: 08/08/2023]
Abstract
OBJECTIVE The rat is a widely used model for studying vocal fold (VF) function after recurrent laryngeal nerve injury, but common techniques for evaluating rat VF motion remain subjective and imprecise. To address this, we developed a software package, called RatVocalTracker1.0 (RVT1.0), to quantify VF motion and tested it on rats with iatrogenic unilateral vocal fold paralysis (VFP). METHODS A deep neural network was trained to identify the positions of the VFs and arytenoid cartilages (ACs) in transoral laryngoscope videos of the rat glottis. Software was developed to estimate glottic midline, VF displacement, VF velocity, and AC angle. The software was applied to laryngoscope videos of adult rats before and after right recurrent and superior laryngeal nerve transection (N = 15; 6M, 9F). All software calculated metrics were compared before and after injury and validated against manually calculated metrics. RESULTS RVT1.0 accurately tracked and quantified VF displacement, VF velocity, and AC angle. Significant differences were found before and after surgery for all RVT1.0 calculated metrics. There was strong agreement between programmatically and manually calculated measures. Automated analysis was also more efficient than nearly all manual methods. CONCLUSION This approach provides fast, accurate assessment of VF motion in rats with minimal labor and allows for quantitative comparison of lateral differences in movement. Through this novel analysis method, we can differentiate healthy movement from unilateral VFP. RVT1.0 is open-source and will be a valuable tool for researchers using the rat model for laryngology research. LEVEL OF EVIDENCE NA Laryngoscope, 134:340-346, 2024.
Collapse
Affiliation(s)
| | - Abhinav Joshi
- The Center for Voice and Swallowing, Department of Otolaryngology-Head & Neck Surgery, Columbia University Irving Medical Center, New York, New York, U.S.A
| | - Emily Honzel
- College of Physicians and Surgeons, Columbia University, New York, New York, U.S.A
| | - Ignacio Hernandez-Morato
- The Center for Voice and Swallowing, Department of Otolaryngology-Head & Neck Surgery, Columbia University Irving Medical Center, New York, New York, U.S.A
| | - Michael J Pitman
- The Center for Voice and Swallowing, Department of Otolaryngology-Head & Neck Surgery, Columbia University Irving Medical Center, New York, New York, U.S.A
| | - Yalda Moayedi
- The Center for Voice and Swallowing, Department of Otolaryngology-Head & Neck Surgery, Columbia University Irving Medical Center, New York, New York, U.S.A
- Department of Neurology, Columbia University, New York, New York, U.S.A
| |
Collapse
|
9
|
DiMatteo AL, Neubauer J, Lott DG, Zacharias S, Murry T, Chen W. High Speed Video Observations of Vocal Fold Kinematics While Playing a Clarinet. J Voice 2024; 38:197-203. [PMID: 34521588 DOI: 10.1016/j.jvoice.2021.07.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2021] [Revised: 07/27/2021] [Accepted: 07/28/2021] [Indexed: 10/20/2022]
Abstract
INTRODUCTION In this pilot study, flexible high-speed video (HSV) technology with synchronized audio is used to visualize vocal fold behavior in a wind instrumentalist. Specifically, this study aims to contribute to describing the vocal fold behavior of a professional clarinet musician playing varying tones and melodies. METHOD Vocal folds of a healthy 26-year-old professional clarinet musician were recorded utilizing a HSV camera coupled to a flexible endoscope, which allowed the synchronous recording of audio with vocal fold movement at the onset, playing, and offset of playing. Two raters experienced with analyzing vocal folds described vocal fold motion of each sample. Samples were processed through a software model to determine the vocal fold movement relative to their position at rest. Digital kymograms (DKGs) were additionally obtained to visualize vocal fold micromovements throughout each sample. RESULTS At the onset of playing, the raters observed the vocal folds moving to a paramedian position. Vocal fold adduction varied according to the task performed and ranged from 58% to 77% of the original resting glottis width. The calculated changes in glottis width and DKGs were consistent with the descriptions by the raters. CONCLUSION This study demonstrates the utility of flexible HSV in observing vocal fold motion before, during, and after events other than sustained phonation. The incomplete adducted vocal fold postures observed while playing the clarinet not only differ from phonation but also differ depending on task. These various postures may contribute to voice fatigue in wind instrumentalists or to various disorders experienced during their careers.
Collapse
Affiliation(s)
| | | | | | | | - Thomas Murry
- Drexel University College of Medicine, Philadelphia, Pennsylvania; Loma Linda Health University, Loma Linda, California.
| | - Wenli Chen
- Private Practice, West Palm Beach, Florida
| |
Collapse
|
10
|
Yousef AM, Deliyski DD, Zayernouri M, Zacharias SRC, Naghibolhosseini M. Deep Learning-Based Analysis of Glottal Attack and Offset Times in Adductor Laryngeal Dystonia. J Voice 2023:S0892-1997(23)00319-3. [PMID: 37977969 PMCID: PMC11093885 DOI: 10.1016/j.jvoice.2023.10.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 10/05/2023] [Accepted: 10/06/2023] [Indexed: 11/19/2023]
Abstract
OBJECTIVE Diagnosis of adductor laryngeal dystonia (AdLD) is challenging as it mimics voice features of other voice disorders. This could lead to misdiagnosis (or delayed diagnosis) and ineffective treatments of AdLD. This paper develops automated measurements of glottal attack time (GAT) and glottal offset time (GOT) from high-speed videoendoscopy (HSV) in connected speech as objective measures that can potentially facilitate the diagnosis of this disorder in the future. METHODS HSV data were recorded from vocally normal adults and patients with AdLD during the reading of the "Rainbow Passage" and six CAPE-V (Consensus Auditory-Perceptual Evaluation of Voice) sentences. A deep learning framework was designed and trained to segment the glottal area and detect the vocal fold edges in the HSV dataset. This automated framework allowed us to automatically measure and quantify the GATs and GOTs for the participants. Accordingly, a comparison was held between the obtained measurements among vocally normal speakers and those with AdLD. RESULTS The automated framework was successfully developed and able to accurately segment the glottal area/edges. The precise automated measurements of GAT and GOT revealed minor, nonsignificant differences compared to the results of manual analysis-showing a strong correlation between the measures by the automated and manual methods. The results showed significant differences in the GAT values between the vocally normal subjects and AdLD patients, with larger variability in both the GAT and GOT measures in the AdLD group. CONCLUSIONS The developed automated approach for GAT and GOT measurement can be valuable in clinical practice. These quantitative measurements can be used as meaningful biomarkers of the impaired vocal function in AdLD and help its differential diagnosis in the future.
Collapse
Affiliation(s)
- Ahmed M Yousef
- Department of Communicative Sciences and Disorders, Michigan State University, East Lansing, Michigan
| | - Dimitar D Deliyski
- Department of Communicative Sciences and Disorders, Michigan State University, East Lansing, Michigan
| | - Mohsen Zayernouri
- Departments of Mechanical Engineering & Statistics and Probability, Michigan State University, East Lansing, Michigan
| | - Stephanie R C Zacharias
- Head and Neck Regenerative Medicine Program, Mayo Clinic, Scottsdale, Arizona; Department of Otolaryngology-Head and Neck Surgery, Mayo Clinic, Phoenix, Arizona
| | - Maryam Naghibolhosseini
- Department of Communicative Sciences and Disorders, Michigan State University, East Lansing, Michigan.
| |
Collapse
|
11
|
DeVore EK, Adamian N, Jowett N, Wang T, Song P, Franco R, Naunheim MR. Predictive Outcomes of Deep Learning Measurement of the Anterior Glottic Angle in Bilateral Vocal Fold Immobility. Laryngoscope 2023; 133:2285-2291. [PMID: 36326102 DOI: 10.1002/lary.30473] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2022] [Revised: 10/19/2022] [Accepted: 10/21/2022] [Indexed: 11/05/2022]
Abstract
OBJECTIVE (1) To compare maximum glottic opening angle (anterior glottic angle, AGA) in patients with bilateral vocal fold immobility (BVFI), unilateral vocal fold immobility (UVFI) and normal larynges (NL), and (2) to correlate maximum AGA with patient-reported outcome measures. METHODS Patients wisth BVFI, UVFI, and NL were retrospectively studied. An open-source deep learning-based computer vision tool for vocal fold tracking was used to analyze videolaryngoscopy. Minimum and maximum AGA were calculated and correlated with three patient-reported outcomes measures. RESULTS Two hundred and fourteen patients were included. Mean maximum AGA was 29.91° (14.40° SD), 42.59° (12.37° SD), and 57.08° (11.14° SD) in BVFI (N = 70), UVFI (N = 70), and NL (N = 72) groups, respectively (p < 0.001). Patients requiring operative airway intervention for BVFI had an average maximum AGA of 24.94° (10.66° SD), statistically different from those not requiring intervention (p = 0.0001). There was moderate negative correlation between Dyspnea Index scores and AGA (Spearman r = -0.345, p = 0.0003). Maximum AGA demonstrated high discriminatory ability for BVFI diagnosis (AUC 0.92, 95% CI 0.81-0.97, p < 0.001) and moderate ability to predict need for operative airway intervention (AUC 0.77, 95% CI 0.64-0.89, p < 0.001). CONCLUSIONS A computer vision tool for quantitative assessment of the AGA from videolaryngoscopy demonstrated ability to discriminate between patients with BVFI, UVFI, and normal controls and predict need for operative airway intervention. This tool may be useful for assessment of other neurological laryngeal conditions and may help guide decision-making in laryngeal surgery. LEVEL OF EVIDENCE III Laryngoscope, 133:2285-2291, 2023.
Collapse
Affiliation(s)
- Elliana Kirsh DeVore
- Department of Otolaryngology, Harvard Medical School, Boston, Massachusetts, USA
- Department of Otolaryngology, Massachusetts Eye and Ear Infirmary, Boston, Massachusetts, USA
| | - Nat Adamian
- Department of Otolaryngology, Harvard Medical School, Boston, Massachusetts, USA
- Department of Otolaryngology, Massachusetts Eye and Ear Infirmary, Boston, Massachusetts, USA
| | - Nate Jowett
- Department of Otolaryngology, Harvard Medical School, Boston, Massachusetts, USA
- Department of Otolaryngology, Massachusetts Eye and Ear Infirmary, Boston, Massachusetts, USA
| | - Tiffany Wang
- Department of Otolaryngology, Harvard Medical School, Boston, Massachusetts, USA
- Department of Otolaryngology, Massachusetts Eye and Ear Infirmary, Boston, Massachusetts, USA
| | - Phillip Song
- Department of Otolaryngology, Harvard Medical School, Boston, Massachusetts, USA
- Department of Otolaryngology, Massachusetts Eye and Ear Infirmary, Boston, Massachusetts, USA
| | - Ramon Franco
- Department of Otolaryngology, Harvard Medical School, Boston, Massachusetts, USA
- Department of Otolaryngology, Massachusetts Eye and Ear Infirmary, Boston, Massachusetts, USA
| | - Matthew Roberts Naunheim
- Department of Otolaryngology, Harvard Medical School, Boston, Massachusetts, USA
- Department of Otolaryngology, Massachusetts Eye and Ear Infirmary, Boston, Massachusetts, USA
| |
Collapse
|
12
|
Paderno A, Villani FP, Fior M, Berretti G, Gennarini F, Zigliani G, Ulaj E, Montenegro C, Sordi A, Sampieri C, Peretti G, Moccia S, Piazza C. Instance segmentation of upper aerodigestive tract cancer: site-specific outcomes. ACTA OTORHINOLARYNGOLOGICA ITALICA : ORGANO UFFICIALE DELLA SOCIETA ITALIANA DI OTORINOLARINGOLOGIA E CHIRURGIA CERVICO-FACCIALE 2023; 43:283-290. [PMID: 37488992 PMCID: PMC10366566 DOI: 10.14639/0392-100x-n2336] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Accepted: 01/08/2023] [Indexed: 07/26/2023]
Abstract
Objective To achieve instance segmentation of upper aerodigestive tract (UADT) neoplasms using a deep learning (DL) algorithm, and to identify differences in its diagnostic performance in three different sites: larynx/hypopharynx, oral cavity and oropharynx. Methods A total of 1034 endoscopic images from 323 patients were examined under narrow band imaging (NBI). The Mask R-CNN algorithm was used for the analysis. The dataset split was: 935 training, 48 validation and 51 testing images. Dice Similarity Coefficient (Dsc) was the main outcome measure. Results Instance segmentation was effective in 76.5% of images. The mean Dsc was 0.90 ± 0.05. The algorithm correctly predicted 77.8%, 86.7% and 55.5% of lesions in the larynx/hypopharynx, oral cavity, and oropharynx, respectively. The mean Dsc was 0.90 ± 0.05 for the larynx/hypopharynx, 0.60 ± 0.26 for the oral cavity, and 0.81 ± 0.30 for the oropharynx. The analysis showed inferior diagnostic results in the oral cavity compared with the larynx/hypopharynx (p < 0.001). Conclusions The study confirms the feasibility of instance segmentation of UADT using DL algorithms and shows inferior diagnostic results in the oral cavity compared with other anatomic areas.
Collapse
Affiliation(s)
- Alberto Paderno
- Unit of Otorhinolaryngology, Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | | | - Milena Fior
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Giulia Berretti
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Francesca Gennarini
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Gabriele Zigliani
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Emanuela Ulaj
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Claudia Montenegro
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Alessandra Sordi
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Claudio Sampieri
- Unit of Otorhinolaryngology, Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
| | - Giorgio Peretti
- Unit of Otorhinolaryngology, Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
| | - Sara Moccia
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Cesare Piazza
- Unit of Otorhinolaryngology, Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| |
Collapse
|
13
|
Malinowski J, Pietruszewska W, Stawiski K, Kowalczyk M, Barańska M, Rycerz A, Niebudek-Bogusz E. High-Speed Videoendoscopy Enhances the Objective Assessment of Glottic Organic Lesions: A Case-Control Study with Multivariable Data-Mining Model Development. Cancers (Basel) 2023; 15:3716. [PMID: 37509377 PMCID: PMC10378075 DOI: 10.3390/cancers15143716] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2023] [Revised: 07/13/2023] [Accepted: 07/19/2023] [Indexed: 07/30/2023] Open
Abstract
The aim of the study was to utilize a quantitative assessment of the vibratory characteristics of vocal folds in diagnosing benign and malignant lesions of the glottis using high-speed videolaryngoscopy (HSV). METHODS Case-control study including 100 patients with unilateral vocal fold lesions in comparison to 38 normophonic subjects. Quantitative assessment with the determination of vocal fold oscillation parameters was performed based on HSV kymography. Machine-learning predictive models were developed and validated. RESULTS All calculated parameters differed significantly between healthy subjects and patients with organic lesions. The first predictive model distinguishing any organic lesion patients from healthy subjects reached an area under the curve (AUC) equal to 0.983 and presented with 89.3% accuracy, 97.0% sensitivity, and 71.4% specificity on the testing set. The second model identifying malignancy among organic lesions reached an AUC equal to 0.85 and presented with 80.6% accuracy, 100% sensitivity, and 71.1% specificity on the training set. Important predictive factors for the models were frequency perturbation measures. CONCLUSIONS The standard protocol for distinguishing between benign and malignant lesions continues to be clinical evaluation by an experienced ENT specialist and confirmed by histopathological examination. Our findings did suggest that advanced machine learning models, which consider the complex interactions present in HSV data, could potentially indicate a heightened risk of malignancy. Therefore, this technology could prove pivotal in aiding in early cancer detection, thereby emphasizing the need for further investigation and validation.
Collapse
Affiliation(s)
- Jakub Malinowski
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, 90-419 Lodz, Poland
| | - Wioletta Pietruszewska
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, 90-419 Lodz, Poland
| | - Konrad Stawiski
- Department of Radiation Oncology, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA 02115, USA
- Department of Biostatistics and Translational Medicine, Medical University of Lodz, 90-419 Lodz, Poland
| | - Magdalena Kowalczyk
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, 90-419 Lodz, Poland
| | - Magda Barańska
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, 90-419 Lodz, Poland
| | - Aleksander Rycerz
- Department of Biostatistics and Translational Medicine, Medical University of Lodz, 90-419 Lodz, Poland
| | - Ewa Niebudek-Bogusz
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, 90-419 Lodz, Poland
| |
Collapse
|
14
|
Tran BA, Dao TTP, Dung HDQ, Van NB, Ha CC, Pham NH, Nguyen TCHTNC, Nguyen TC, Pham MK, Tran MK, Tran TM, Tran MT. Support of deep learning to classify vocal fold images in flexible laryngoscopy. Am J Otolaryngol 2023; 44:103800. [PMID: 36905912 DOI: 10.1016/j.amjoto.2023.103800] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2022] [Accepted: 02/19/2023] [Indexed: 02/26/2023]
Abstract
PURPOSE To collect a dataset with adequate laryngoscopy images and identify the appearance of vocal folds and their lesions in flexible laryngoscopy images by objective deep learning models. METHODS We adopted a number of novel deep learning models to train and classify 4549 flexible laryngoscopy images as no vocal fold, normal vocal folds, and abnormal vocal folds. This could help these models recognize vocal folds and their lesions within these images. Ultimately, we made a comparison between the results of the state-of-the-art deep learning models, and another comparison of the results between the computer-aided classification system and ENT doctors. RESULTS This study exhibited the performance of the deep learning models by evaluating laryngoscopy images collected from 876 patients. The efficiency of the Xception model was higher and steadier than almost the rest of the models. The accuracy of no vocal fold, normal vocal folds, and vocal fold abnormalities on this model were 98.90 %, 97.36 %, and 96.26 %, respectively. Compared to our ENT doctors, the Xception model produced better results than a junior doctor and was near an expert. CONCLUSION Our results show that current deep learning models can classify vocal fold images well and effectively assist physicians in vocal fold identification and classification of normal or abnormal vocal folds.
Collapse
Affiliation(s)
- Bich Anh Tran
- Otorhinolaryngology Department, Cho Ray Hospital, Ho Chi Minh City, Viet Nam.
| | - Thao Thi Phuong Dao
- University of Science, VNUHCM, Ho Chi Minh City, Viet Nam; John von Neumann Institute, VNUHCM, Ho Chi Minh City, Viet Nam; Vietnam National University, Ho Chi Minh City, Viet Nam; Department of Otolaryngology, Thong Nhat Hospital, Ho Chi Minh City, Viet Nam.
| | - Ho Dang Quy Dung
- Department of Endoscopy, Cho Ray Hospital, Ho Chi Minh City, Viet Nam.
| | - Ngoc Boi Van
- Department of Otolaryngology, Vinmec Central Park International Hospital, Ho Chi Minh City, Viet Nam.
| | - Chanh Cong Ha
- Department of Otolaryngology, 7A Military Hospital, Ho Chi Minh City, Viet Nam.
| | - Nam Hoang Pham
- Otorhinolaryngology Department, Cho Ray Hospital, Ho Chi Minh City, Viet Nam.
| | | | - Tan-Cong Nguyen
- University of Science, VNUHCM, Ho Chi Minh City, Viet Nam; University of Social Sciences and Humanities, VNUHCM, Ho Chi Minh City, Vietnam; Vietnam National University, Ho Chi Minh City, Viet Nam.
| | - Minh-Khoi Pham
- University of Science, VNUHCM, Ho Chi Minh City, Viet Nam; Vietnam National University, Ho Chi Minh City, Viet Nam.
| | - Mai-Khiem Tran
- University of Science, VNUHCM, Ho Chi Minh City, Viet Nam; John von Neumann Institute, VNUHCM, Ho Chi Minh City, Viet Nam; Vietnam National University, Ho Chi Minh City, Viet Nam.
| | - Truong Minh Tran
- Otorhinolaryngology Department, Cho Ray Hospital, Ho Chi Minh City, Viet Nam.
| | - Minh-Triet Tran
- University of Science, VNUHCM, Ho Chi Minh City, Viet Nam; John von Neumann Institute, VNUHCM, Ho Chi Minh City, Viet Nam; Vietnam National University, Ho Chi Minh City, Viet Nam.
| |
Collapse
|
15
|
Kruse E, Döllinger M, Schützenberger A, Kist AM. GlottisNetV2: Temporal Glottal Midline Detection Using Deep Convolutional Neural Networks. IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE 2023; 11:137-144. [PMID: 36816097 PMCID: PMC9933989 DOI: 10.1109/jtehm.2023.3237859] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Revised: 11/27/2022] [Accepted: 01/04/2023] [Indexed: 11/26/2023]
Abstract
High-speed videoendoscopy is a major tool for quantitative laryngology. Glottis segmentation and glottal midline detection are crucial for computing vocal fold-specific, quantitative parameters. However, fully automated solutions show limited clinical applicability. Especially unbiased glottal midline detection remains a challenging problem. We developed a multitask deep neural network for glottis segmentation and glottal midline detection. We used techniques from pose estimation to estimate the anterior and posterior points in endoscopy images. Neural networks were set up in TensorFlow/Keras and trained and evaluated with the BAGLS dataset. We found that a dual decoder deep neural network termed GlottisNetV2 outperforms the previously proposed GlottisNet in terms of MAPE on the test dataset (1.85% to 6.3%) while converging faster. Using various hyperparameter tunings, we allow fast and directed training. Using temporal variant data on an additional data set designed for this task, we can improve the median prediction accuracy from 2.1% to 1.76% when using 12 consecutive frames and additional temporal filtering. We found that temporal glottal midline detection using a dual decoder architecture together with keypoint estimation allows accurate midline prediction. We show that our proposed architecture allows stable and reliable glottal midline predictions ready for clinical use and analysis of symmetry measures.
Collapse
Affiliation(s)
- Elina Kruse
- Department Artificial Intelligence in Biomedical EngineeringFriedrich-Alexander-University Erlangen–Nürnberg (FAU)91052ErlangenGermany
| | - Michael Döllinger
- Division of Phoniatrics and Pediatric AudiologyDepartment of Otorhinolaryngology, Head and Neck SurgeryUniversity Hospital Erlangen, Friedrich-Alexander-University Erlangen–Nürnberg (FAU)91054ErlangenGermany
| | - Anne Schützenberger
- Division of Phoniatrics and Pediatric AudiologyDepartment of Otorhinolaryngology, Head and Neck SurgeryUniversity Hospital Erlangen, Friedrich-Alexander-University Erlangen–Nürnberg (FAU)91054ErlangenGermany
| | - Andreas M. Kist
- Department Artificial Intelligence in Biomedical EngineeringFriedrich-Alexander-University Erlangen–Nürnberg (FAU)91052ErlangenGermany
| |
Collapse
|
16
|
Pedersen M, Larsen CF, Madsen B, Eeg M. Localization and quantification of glottal gaps on deep learning segmentation of vocal folds. Sci Rep 2023; 13:878. [PMID: 36650265 PMCID: PMC9845318 DOI: 10.1038/s41598-023-27980-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Accepted: 01/11/2023] [Indexed: 01/19/2023] Open
Abstract
The entire glottis has mostly been the focus in the tracking of the vocal folds, both manually and automatically. From a treatment point of view, the various regions of the glottis are of specific interest. The aim of the study was to test if it was possible to supplement an existing convolutional neural network (CNN) with post-network calculations for the localization and quantification of posterior glottal gaps during phonation, usable for vocal fold function analysis of e.g. laryngopharyngeal reflux findings. 30 subjects/videos with insufficient closure in the rear glottal area and 20 normal subjects/videos were selected from our database, recorded with a commercial high-speed video setup (HSV with 4000 frames per second), and segmented with an open-source CNN for validating voice function. We made post-network calculations to localize and quantify the 10% and 50% distance lines from the rear part of the glottis. The results showed a significant difference using the algorithm at the 10% line distance between the two groups of p < 0.0001 and no difference at 50%. These novel results show that it is possible to use post-network calculations on CNNs for the localization and quantification of posterior glottal gaps.
Collapse
|
17
|
Konradi J, Zajber M, Betz U, Drees P, Gerken A, Meine H. AI-Based Detection of Aspiration for Video-Endoscopy with Visual Aids in Meaningful Frames to Interpret the Model Outcome. SENSORS (BASEL, SWITZERLAND) 2022; 22:9468. [PMID: 36502169 PMCID: PMC9736280 DOI: 10.3390/s22239468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Revised: 11/29/2022] [Accepted: 11/30/2022] [Indexed: 06/17/2023]
Abstract
Disorders of swallowing often lead to pneumonia when material enters the airways (aspiration). Flexible Endoscopic Evaluation of Swallowing (FEES) plays a key role in the diagnostics of aspiration but is prone to human errors. An AI-based tool could facilitate this process. Recent non-endoscopic/non-radiologic attempts to detect aspiration using machine-learning approaches have led to unsatisfying accuracy and show black-box characteristics. Hence, for clinical users it is difficult to trust in these model decisions. Our aim is to introduce an explainable artificial intelligence (XAI) approach to detect aspiration in FEES. Our approach is to teach the AI about the relevant anatomical structures, such as the vocal cords and the glottis, based on 92 annotated FEES videos. Simultaneously, it is trained to detect boluses that pass the glottis and become aspirated. During testing, the AI successfully recognized the glottis and the vocal cords but could not yet achieve satisfying aspiration detection quality. While detection performance must be optimized, our architecture results in a final model that explains its assessment by locating meaningful frames with relevant aspiration events and by highlighting suspected boluses. In contrast to comparable AI tools, our framework is verifiable and interpretable and, therefore, accountable for clinical users.
Collapse
Affiliation(s)
- Jürgen Konradi
- Institute of Physical Therapy, Prevention and Rehabilitation, University Medical Center of the Johannes Gutenberg-University Mainz, 55131 Mainz, Germany
| | - Milla Zajber
- Department for Health Care & Nursing, Catholic University of Applied Sciences, 55122 Mainz, Germany
| | - Ulrich Betz
- Institute of Physical Therapy, Prevention and Rehabilitation, University Medical Center of the Johannes Gutenberg-University Mainz, 55131 Mainz, Germany
| | - Philipp Drees
- Department of Orthopedics and Trauma Surgery, University Medical Center of the Johannes Gutenberg-University Mainz, 55131 Mainz, Germany
| | - Annika Gerken
- Fraunhofer Institute for Digital Medicine MEVIS, 28359 Bremen, Germany
| | - Hans Meine
- Fraunhofer Institute for Digital Medicine MEVIS, 28359 Bremen, Germany
| |
Collapse
|
18
|
Fast JF, Oltmann A, Spindeldreier S, Ptok M. Computational Analysis of the Droplet-Stimulated Laryngeal Adductor Reflex in High-Speed Sequences. Laryngoscope 2022; 132:2412-2419. [PMID: 35133015 DOI: 10.1002/lary.30041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Revised: 12/28/2021] [Accepted: 01/23/2022] [Indexed: 12/16/2022]
Abstract
OBJECTIVES/HYPOTHESIS The laryngeal adductor reflex (LAR) is an important protective mechanism of the airways. Its physiology is still not completely understood. The available methods for LAR evaluation offer limited reproducibility and/or rely on subjective interpretation. A new approach, termed Microdroplet Impulse Testing of the LAR (MIT-LAR), was recently introduced. Here, the LAR is elicited by a droplet and a laryngoscopic high-speed recording is acquired simultaneously. In the present work, image-processing algorithms for autonomous MIT-LAR sequence analysis were developed. This allowed the automated approximation of kinematic LAR parameters in humans. STUDY DESIGN Development and testing of computational methods. METHODS Computational image processing enabled the autonomous estimation of the glottal area, the glottal angle, and the vocal fold edge distance in MIT-LAR sequences. A suitable analytical representation of these glottal parameters allowed the extraction of seven relevant LAR parameters. The obtained values were compared to the literature. RESULTS A generalized logistic function showed the highest average goodness of fit among four different analytical approaches for each of the glottal parameters. Autonomous sequence analysis yielded bilateral LAR response latencies of (229 ± 116) ms and (182 ± 60) ms for cases of complete and incomplete glottal closure, respectively. The initial/average/maximum angular vocal fold adduction velocity was estimated at (157 ± 115) °s-1 /(891 ± 516) °s-1 /(929 ± 583) °s-1 and (88 ± 53) °s-1 /(421 ± 221) °s-1 /(520 ± 238) °s-1 for complete and incomplete glottal closure, respectively. CONCLUSION The automated extraction of LAR parameters from laryngoscopic high-speed sequences can potentially increase the objectiveness of optical LAR characterization and reduce the associated workload. The proposed methods may thus be helpful for future research on this vital reflex. LEVEL OF EVIDENCE NA Laryngoscope, 132:2412-2419, 2022.
Collapse
Affiliation(s)
- Jacob Friedemann Fast
- Department of Phoniatrics and Pediatric Audiology, Hannover Medical School, Hanover, Germany.,Institute of Mechatronic Systems, Leibniz Universität Hannover, Hanover, Germany
| | - Andra Oltmann
- Institute of Mechatronic Systems, Leibniz Universität Hannover, Hanover, Germany.,Department of Modeling and Simulation, Fraunhofer Research Institution for Individualized and Cell-Based Medical Engineering, Lübeck, Germany
| | - Svenja Spindeldreier
- Institute of Mechatronic Systems, Leibniz Universität Hannover, Hanover, Germany
| | - Martin Ptok
- Department of Phoniatrics and Pediatric Audiology, Hannover Medical School, Hanover, Germany
| |
Collapse
|
19
|
Peterson QA, Fei T, Sy LE, Froeschke LL, Mendelsohn AH, Berke GS, Peterson DA. Correlating Perceptual Voice Quality in Adductor Spasmodic Dysphonia With Computer Vision Assessment of Glottal Geometry Dynamics. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2022; 65:3695-3708. [PMID: 36130065 PMCID: PMC9927624 DOI: 10.1044/2022_jslhr-22-00053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
PURPOSE This study examined the relationship between voice quality and glottal geometry dynamics in patients with adductor spasmodic dysphonia (ADSD). METHOD An objective computer vision and machine learning system was developed to extract glottal geometry dynamics from nasolaryngoscopic video recordings for 78 patients with ADSD. General regression models were used to examine the relationship between overall voice quality and 15 variables that capture glottal geometry dynamics derived from the computer vision system. Two experts in ADSD independently rated voice quality for two separate voice tasks for every patient, yielding four different voice quality rating models. RESULTS All four of the regression models exhibited positive correlations with clinical assessments of voice quality (R 2s = .30-.34, Spearman rho = .55-.61, all with p < .001). Seven to 10 variables were included in each model. There was high overlap in the variables included between the four models, and the sign of the correlation with voice quality was consistent for each variable across all four regression models. CONCLUSION We found specific glottal geometry dynamics that correspond to voice quality in ADSD.
Collapse
Affiliation(s)
- Quinn A. Peterson
- Department of Computer Science and Software Engineering, California Polytechnic State University, San Luis Obispo
| | - Teng Fei
- Department of Cognitive Science, University of California, San Diego, La Jolla
| | - Lauren E. Sy
- Department of Cognitive Science, University of California, San Diego, La Jolla
| | | | - Abie H. Mendelsohn
- Department of Head and Neck Surgery, David Geffen School of Medicine, University of California, Los Angeles
| | - Gerald S. Berke
- Department of Head and Neck Surgery, David Geffen School of Medicine, University of California, Los Angeles
| | - David A. Peterson
- Institute for Neural Computation, University of California, San Diego, La Jolla
| |
Collapse
|
20
|
Sakthivel S, Prabhu V. Optimal Deep Learning-Based Vocal Fold Disorder Detection and Classification Model on High-Speed Video Endoscopy. JOURNAL OF HEALTHCARE ENGINEERING 2022; 2022:4248938. [PMID: 36353680 PMCID: PMC9640237 DOI: 10.1155/2022/4248938] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/23/2022] [Revised: 09/04/2022] [Accepted: 09/21/2022] [Indexed: 08/08/2023]
Abstract
The use of high-speed video-endoscopy (HSV) in the study of phonatory processes linked to speech needs the precise identification of vocal fold boundaries at the time of vibration. The HSV is a unique laryngeal imaging technology that captures intracycle vocal fold vibrations at a higher frame rate without the need for auditory inputs. The HSV is also effective in identifying the vibrational characteristics of the vocal folds with an increased temporal resolution during retained phonation and flowing speech. Clinically significant vocal fold vibratory characteristics in running speech can be retrieved by creating automated algorithms for extracting HSV-based vocal fold vibration data. The best deep learning-based diagnosis and categorization of vocal fold abnormalities is due to the usage of HSV (ODL-VFDDC). The suggested ODL-VFDDC technique starts with temporal segmentation and motion correction to identify vocalized regions from the HSV recording and gathers the position of movable vocal folds across frames. The attributes gathered are fed into the deep belief network (DBN) model. Furthermore, the agricultural fertility algorithm (AFA) is used to optimize the hyperparameter tuning of the DBN model, which improves classification results. In terms of vocal fold disorder classification, the testing results demonstrated that the ODL-VFDDC technique beats the other existing methodologies. The farmland fertility algorithm (FFA) is then used to accurately determine the glottal limits of vibrating vocal folds. The suggested method has successfully tracked the speech fold boundaries across frames with minimum processing cost and high resilience to picture noise. This method gives a way to look at how the vocal folds move during a connected speech that is completely done by itself.
Collapse
Affiliation(s)
- S. Sakthivel
- Department of Computer Science and Engineering, Vel Tech High Tech Dr. Rangarajan Dr. Sakunthala Engineering College, Avadi, Chennai, India
| | - V. Prabhu
- Department of Electronics and Communication Engineering, Vel Tech Multi Tech Dr. Rangarajan Dr. Sakunthala Engineering College, Chennai, India
| |
Collapse
|
21
|
Döllinger M, Schraut T, Henrich LA, Chhetri D, Echternach M, Johnson AM, Kunduk M, Maryn Y, Patel RR, Samlan R, Semmler M, Schützenberger A. Re-Training of Convolutional Neural Networks for Glottis Segmentation in Endoscopic High-Speed Videos. APPLIED SCIENCES (BASEL, SWITZERLAND) 2022; 12:9791. [PMID: 37583544 PMCID: PMC10427138 DOI: 10.3390/app12199791] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 08/17/2023]
Abstract
Endoscopic high-speed video (HSV) systems for visualization and assessment of vocal fold dynamics in the larynx are diverse and technically advancing. To consider resulting "concepts shifts" for neural network (NN)-based image processing, re-training of already trained and used NNs is necessary to allow for sufficiently accurate image processing for new recording modalities. We propose and discuss several re-training approaches for convolutional neural networks (CNN) being used for HSV image segmentation. Our baseline CNN was trained on the BAGLS data set (58,750 images). The new BAGLS-RT data set consists of additional 21,050 images from previously unused HSV systems, light sources, and different spatial resolutions. Results showed that increasing data diversity by means of preprocessing already improves the segmentation accuracy (mIoU + 6.35%). Subsequent re-training further increases segmentation performance (mIoU + 2.81%). For re-training, finetuning with dynamic knowledge distillation showed the most promising results. Data variety for training and additional re-training is a helpful tool to boost HSV image segmentation quality. However, when performing re-training, the phenomenon of catastrophic forgetting should be kept in mind, i.e., adaption to new data while forgetting already learned knowledge.
Collapse
Affiliation(s)
- Michael Döllinger
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhino-laryngology Head & Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, 91054 Erlangen, Germany
| | - Tobias Schraut
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhino-laryngology Head & Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, 91054 Erlangen, Germany
| | - Lea A. Henrich
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhino-laryngology Head & Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, 91054 Erlangen, Germany
| | - Dinesh Chhetri
- Department of Head and Neck Surgery, David Geffen School of Medicine at the University of California, Los Angeles, Los Angeles, CA 90095, USA
| | - Matthias Echternach
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Munich University Hospital (LMU), 80331 Munich, Germany
| | - Aaron M. Johnson
- NYU Voice Center, Department of Otolaryngology–Head and Neck Surgery, New York University, Grossman School of Medicine, New York, NY 10001, USA
| | - Melda Kunduk
- Department of Communication Sciences and Disorders, Louisiana State University, Baton Rouge, LA 70801, USA
| | - Youri Maryn
- Department of Speech, Language and Hearing Sciences, University of Ghent, 9000 Ghent, Belgium
| | - Rita R. Patel
- Department of Speech, Language and Hearing Sciences, Indiana University, Bloomington, IA 47401, USA
| | - Robin Samlan
- Department of Speech, Language, & Hearing Sciences, University of Arizona, Tucson, AZ 85641, USA
| | - Marion Semmler
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhino-laryngology Head & Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, 91054 Erlangen, Germany
| | - Anne Schützenberger
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhino-laryngology Head & Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, 91054 Erlangen, Germany
| |
Collapse
|
22
|
Kaluza J, Niebudek-Bogusz E, Malinowski J, Strumillo P, Pietruszewska W. Assessment of Vocal Fold Stiffness by Means of High-Speed Videolaryngoscopy with Laryngotopography in Prediction of Early Glottic Malignancy: Preliminary Report. Cancers (Basel) 2022; 14:4697. [PMID: 36230618 PMCID: PMC9563419 DOI: 10.3390/cancers14194697] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Revised: 09/07/2022] [Accepted: 09/19/2022] [Indexed: 11/16/2022] Open
Abstract
One of the most important challenges in laryngological practice is the early diagnosis of laryngeal cancer. Detection of non-vibrating areas affected by neoplastic lesions of the vocal folds can be crucial in the recognition of early cancerogenous infiltration. Glottal pathologies associated with abnormal vibration patterns of the vocal folds can be detected and quantified using High-speed Videolaryngoscopy (HSV), also in subjects with severe voice disorders, and analyzed with the aid of computer image processing procedures. We present a method that enables the assessment of vocal fold pathologies with the use of HSV. The calculated laryngotopographic (LTG) maps of the vocal folds based on HSV allowed for a detailed characterization of vibration patterns and abnormalities in different regions of the vocal folds. We verified our methods with HSV recordings from 31 subjects with a normophonic voice and benign and malignant vocal fold lesions. We proposed the novel Stiffness Asymmetry Index (SAI) to differentiate between early glottis cancer (SAI = 0.65 ± 0.18) and benign vocal fold masses (SAI = 0.16 ± 0.13). Our results showed that these glottal pathologies might be noninvasively distinguished prior to histopathological examination. However, this needs to be confirmed by further research on larger groups of benign and malignant laryngeal lesions.
Collapse
Affiliation(s)
- Justyna Kaluza
- Institute of Electronics, Lodz University of Technology, 90-924 Lodz, Poland
| | - Ewa Niebudek-Bogusz
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, 90-001 Lodz, Poland
| | - Jakub Malinowski
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, 90-001 Lodz, Poland
| | - Pawel Strumillo
- Institute of Electronics, Lodz University of Technology, 90-924 Lodz, Poland
| | - Wioletta Pietruszewska
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, 90-001 Lodz, Poland
| |
Collapse
|
23
|
Yousef AM, Deliyski DD, Zacharias SRC, Naghibolhosseini M. Deep-Learning-Based Representation of Vocal Fold Dynamics in Adductor Spasmodic Dysphonia during Connected Speech in High-Speed Videoendoscopy. J Voice 2022:S0892-1997(22)00263-6. [PMID: 36154973 PMCID: PMC10030376 DOI: 10.1016/j.jvoice.2022.08.022] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Revised: 08/14/2022] [Accepted: 08/17/2022] [Indexed: 11/28/2022]
Abstract
OBJECTIVE Adductor spasmodic dysphonia (AdSD) is a neurogenic dystonia, which causes spasms of the laryngeal muscles. This disorder mainly affects production of connected speech. To understand how AdSD affects vocal fold (VF) movements and hence, the speech signal, it is necessary to study VF kinematics during the running speech. This paper introduces an automated method for analysis of VF vibrations in AdSD using laryngeal high-speed videoendoscopy (HSV) in running speech. METHODS A monochrome HSV system was used to obtain video recordings from vocally normal individuals and AdSD patients during production of the six CAPE-V sentences and the "Rainbow Passage." A deep neural network was designed based on the UNet architecture. The network was developed for glottal area segmentation in HSV data providing a tool for quantitative analysis of VF vibrations in both norm and AdSD. The network was trained and validated using the manually labeled HSV frames. After training the network, the segmentation quality was quantitatively evaluated against visual analysis results of a test dataset including segregated HSV frames and a short sequence of VF vibrations in consecutive frames. RESULTS The developed convolutional network was successfully trained and demonstrated an accurate segmentation on the testing dataset with a mean Intersection over Union (IoU) of 0.81 and a mean Boundary-F1 score of 0.93. Moreover, the visual assessment of the automated technique showed an accurate detection of the glottal edges/area in the HSV data even with challenging image quality and excessive laryngeal maneuvers of AdSD patients during the running speech. CONCLUSION The introduced automated approach provides an accurate representation of the glottal edges/area during connected speech in HSV data for norm and AdSD patients. This method facilitates the development of HSV-based measures to quantify VF dynamics in AdSD. Using HSV to automatically analyze VF vibrations in AdSD can allow for understanding AdSD vocal mechanisms and characteristics.
Collapse
Affiliation(s)
- Ahmed M Yousef
- Department of Communicative Sciences and Disorders, Michigan State University, East Lansing, Michigan
| | - Dimitar D Deliyski
- Department of Communicative Sciences and Disorders, Michigan State University, East Lansing, Michigan
| | - Stephanie R C Zacharias
- Head and Neck Regenerative Medicine Program, Mayo Clinic, Scottsdale, Arizona; Department of Otolaryngology-Head and Neck Surgery, Mayo Clinic, Phoenix, Arizona
| | - Maryam Naghibolhosseini
- Department of Communicative Sciences and Disorders, Michigan State University, East Lansing, Michigan.
| |
Collapse
|
24
|
Long-term performance assessment of fully automatic biomedical glottis segmentation at the point of care. PLoS One 2022; 17:e0266989. [PMID: 36129922 PMCID: PMC9491538 DOI: 10.1371/journal.pone.0266989] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 07/25/2022] [Indexed: 12/04/2022] Open
Abstract
Deep Learning has a large impact on medical image analysis and lately has been adopted for clinical use at the point of care. However, there is only a small number of reports of long-term studies that show the performance of deep neural networks (DNNs) in such an environment. In this study, we measured the long-term performance of a clinically optimized DNN for laryngeal glottis segmentation. We have collected the video footage for two years from an AI-powered laryngeal high-speed videoendoscopy imaging system and found that the footage image quality is stable across time. Next, we determined the DNN segmentation performance on lossy and lossless compressed data revealing that only 9% of recordings contain segmentation artifacts. We found that lossy and lossless compression is on par for glottis segmentation, however, lossless compression provides significantly superior image quality. Lastly, we employed continual learning strategies to continuously incorporate new data into the DNN to remove the aforementioned segmentation artifacts. With modest manual intervention, we were able to largely alleviate these segmentation artifacts by up to 81%. We believe that our suggested deep learning-enhanced laryngeal imaging platform consistently provides clinically sound results, and together with our proposed continual learning scheme will have a long-lasting impact on the future of laryngeal imaging.
Collapse
|
25
|
Paderno A, Gennarini F, Sordi A, Montenegro C, Lancini D, Villani FP, Moccia S, Piazza C. Artificial intelligence in clinical endoscopy: Insights in the field of videomics. Front Surg 2022; 9:933297. [PMID: 36171813 PMCID: PMC9510389 DOI: 10.3389/fsurg.2022.933297] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2022] [Accepted: 08/22/2022] [Indexed: 11/13/2022] Open
Abstract
Artificial intelligence is being increasingly seen as a useful tool in medicine. Specifically, these technologies have the objective to extract insights from complex datasets that cannot easily be analyzed by conventional statistical methods. While promising results have been obtained for various -omics datasets, radiological images, and histopathologic slides, analysis of videoendoscopic frames still represents a major challenge. In this context, videomics represents a burgeoning field wherein several methods of computer vision are systematically used to organize unstructured data from frames obtained during diagnostic videoendoscopy. Recent studies have focused on five broad tasks with increasing complexity: quality assessment of endoscopic images, classification of pathologic and nonpathologic frames, detection of lesions inside frames, segmentation of pathologic lesions, and in-depth characterization of neoplastic lesions. Herein, we present a broad overview of the field, with a focus on conceptual key points and future perspectives.
Collapse
Affiliation(s)
- Alberto Paderno
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
- Correspondence: Alberto Paderno
| | - Francesca Gennarini
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
| | - Alessandra Sordi
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
| | - Claudia Montenegro
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
| | - Davide Lancini
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
| | - Francesca Pia Villani
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, Pisa, Italy
| | - Sara Moccia
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, Pisa, Italy
| | - Cesare Piazza
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
| |
Collapse
|
26
|
Zita A, Novozámský A, Zitová B, Šorel M, Herbst CT, Vydrová J, Švec JG. Videokymogram Analyzer Tool: Human–computer comparison. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103878] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
27
|
Kist AM, Breininger K, Dörrich M, Dürr S, Schützenberger A, Semmler M. A single latent channel is sufficient for biomedical glottis segmentation. Sci Rep 2022; 12:14292. [PMID: 35995933 PMCID: PMC9395348 DOI: 10.1038/s41598-022-17764-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Accepted: 07/30/2022] [Indexed: 11/23/2022] Open
Abstract
Glottis segmentation is a crucial step to quantify endoscopic footage in laryngeal high-speed videoendoscopy. Recent advances in deep neural networks for glottis segmentation allow for a fully automatic workflow. However, exact knowledge of integral parts of these deep segmentation networks remains unknown, and understanding the inner workings is crucial for acceptance in clinical practice. Here, we show that a single latent channel as a bottleneck layer is sufficient for glottal area segmentation using systematic ablations. We further demonstrate that the latent space is an abstraction of the glottal area segmentation relying on three spatially defined pixel subtypes allowing for a transparent interpretation. We further provide evidence that the latent space is highly correlated with the glottal area waveform, can be encoded with four bits, and decoded using lean decoders while maintaining a high reconstruction accuracy. Our findings suggest that glottis segmentation is a task that can be highly optimized to gain very efficient and explainable deep neural networks, important for application in the clinic. In the future, we believe that online deep learning-assisted monitoring is a game-changer in laryngeal examinations.
Collapse
Affiliation(s)
- Andreas M Kist
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-University Erlangen-Nürnberg, 91052, Erlangen, Germany.
| | - Katharina Breininger
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-University Erlangen-Nürnberg, 91052, Erlangen, Germany
| | - Marion Dörrich
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-University Erlangen-Nürnberg, 91052, Erlangen, Germany
| | - Stephan Dürr
- Division of Phoniatrics and Pediatric Audiology at the Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, 91054, Erlangen, Germany
| | - Anne Schützenberger
- Division of Phoniatrics and Pediatric Audiology at the Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, 91054, Erlangen, Germany
| | - Marion Semmler
- Division of Phoniatrics and Pediatric Audiology at the Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, 91054, Erlangen, Germany
| |
Collapse
|
28
|
Azam MA, Sampieri C, Ioppi A, Benzi P, Giordano GG, De Vecchi M, Campagnari V, Li S, Guastini L, Paderno A, Moccia S, Piazza C, Mattos LS, Peretti G. Videomics of the Upper Aero-Digestive Tract Cancer: Deep Learning Applied to White Light and Narrow Band Imaging for Automatic Segmentation of Endoscopic Images. Front Oncol 2022; 12:900451. [PMID: 35719939 PMCID: PMC9198427 DOI: 10.3389/fonc.2022.900451] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2022] [Accepted: 04/26/2022] [Indexed: 12/13/2022] Open
Abstract
Introduction Narrow Band Imaging (NBI) is an endoscopic visualization technique useful for upper aero-digestive tract (UADT) cancer detection and margins evaluation. However, NBI analysis is strongly operator-dependent and requires high expertise, thus limiting its wider implementation. Recently, artificial intelligence (AI) has demonstrated potential for applications in UADT videoendoscopy. Among AI methods, deep learning algorithms, and especially convolutional neural networks (CNNs), are particularly suitable for delineating cancers on videoendoscopy. This study is aimed to develop a CNN for automatic semantic segmentation of UADT cancer on endoscopic images. Materials and Methods A dataset of white light and NBI videoframes of laryngeal squamous cell carcinoma (LSCC) was collected and manually annotated. A novel DL segmentation model (SegMENT) was designed. SegMENT relies on DeepLabV3+ CNN architecture, modified using Xception as a backbone and incorporating ensemble features from other CNNs. The performance of SegMENT was compared to state-of-the-art CNNs (UNet, ResUNet, and DeepLabv3). SegMENT was then validated on two external datasets of NBI images of oropharyngeal (OPSCC) and oral cavity SCC (OSCC) obtained from a previously published study. The impact of in-domain transfer learning through an ensemble technique was evaluated on the external datasets. Results 219 LSCC patients were retrospectively included in the study. A total of 683 videoframes composed the LSCC dataset, while the external validation cohorts of OPSCC and OCSCC contained 116 and 102 images. On the LSCC dataset, SegMENT outperformed the other DL models, obtaining the following median values: 0.68 intersection over union (IoU), 0.81 dice similarity coefficient (DSC), 0.95 recall, 0.78 precision, 0.97 accuracy. For the OCSCC and OPSCC datasets, results were superior compared to previously published data: the median performance metrics were, respectively, improved as follows: DSC=10.3% and 11.9%, recall=15.0% and 5.1%, precision=17.0% and 14.7%, accuracy=4.1% and 10.3%. Conclusion SegMENT achieved promising performances, showing that automatic tumor segmentation in endoscopic images is feasible even within the highly heterogeneous and complex UADT environment. SegMENT outperformed the previously published results on the external validation cohorts. The model demonstrated potential for improved detection of early tumors, more precise biopsies, and better selection of resection margins.
Collapse
Affiliation(s)
- Muhammad Adeel Azam
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Claudio Sampieri
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Alessandro Ioppi
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Pietro Benzi
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Giorgio Gregory Giordano
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Marta De Vecchi
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Valentina Campagnari
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Shunlei Li
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Luca Guastini
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Alberto Paderno
- Unit of Otorhinolaryngology - Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy.,Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, Brescia, Italy
| | - Sara Moccia
- The BioRobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Cesare Piazza
- Unit of Otorhinolaryngology - Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy.,Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, Brescia, Italy
| | - Leonardo S Mattos
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Giorgio Peretti
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| |
Collapse
|
29
|
Yousef AM, Deliyski DD, Zacharias SRC, de Alarcon A, Orlikoff RF, Naghibolhosseini M. A Deep Learning Approach for Quantifying Vocal Fold Dynamics During Connected Speech Using Laryngeal High-Speed Videoendoscopy. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2022; 65:2098-2113. [PMID: 35605603 PMCID: PMC9567340 DOI: 10.1044/2022_jslhr-21-00540] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Revised: 01/30/2022] [Accepted: 02/28/2022] [Indexed: 06/15/2023]
Abstract
PURPOSE Voice disorders are best assessed by examining vocal fold dynamics in connected speech. This can be achieved using flexible laryngeal high-speed videoendoscopy (HSV), which enables us to study vocal fold mechanics with high temporal details. Analysis of vocal fold vibration using HSV requires accurate segmentation of the vocal fold edges. This article presents an automated deep-learning scheme to segment the glottal area in HSV from which the glottal edges are derived during connected speech. METHOD Using a custom-built HSV system, data were obtained from a vocally healthy participant reciting the "Rainbow Passage." A deep neural network was designed for glottal area segmentation in the HSV data. A recently introduced hybrid approach by the authors was utilized as an automated labeling tool to train the network on a set of HSV frames, where the glottis region was automatically annotated during vocal fold vibrations. The network was then tested against manually segmented frames using different metrics, intersection over union (IoU), and Boundary F1 (BF) score, and its performance was assessed on various phonatory events on the HSV sequence. RESULTS The designed network was successfully trained using the hybrid approach, without the need for manual labeling, and tested on the manually labeled data. The performance metrics showed a mean IoU of 0.82 and a mean BF score of 0.96. In addition, the evaluation assessment of the network's performance demonstrated an accurate segmentation of the glottal edges/area even during complex nonstationary phonatory events and when vocal folds were not vibrating, thus overcoming the limitations of the previous hybrid approach that could only be applied to the vibrating vocal folds. CONCLUSIONS The introduced automated scheme guarantees accurate glottis representation in challenging color HSV data with lower image quality and excessive laryngeal maneuvers during all instances of connected speech. This facilitates the future development of HSV-based measures to assess the running vibratory characteristics of the vocal folds in speakers with and without voice disorder. SUPPLEMENTAL MATERIAL https://doi.org/10.23641/asha.19798864.
Collapse
Affiliation(s)
- Ahmed M. Yousef
- Department of Communicative Sciences and Disorders, Michigan State University, East Lansing
| | - Dimitar D. Deliyski
- Department of Communicative Sciences and Disorders, Michigan State University, East Lansing
| | - Stephanie R. C. Zacharias
- Head and Neck Regenerative Medicine Program, Mayo Clinic, Scottsdale, AZ
- Department of Otolaryngology—Head and Neck Surgery, Mayo Clinic, Phoenix, AZ
| | - Alessandro de Alarcon
- Division of Pediatric Otolaryngology, Cincinnati Children's Hospital Medical Center, OH
- Department of Otolaryngology—Head and Neck Surgery, University of Cincinnati, OH
| | - Robert F. Orlikoff
- College of Allied Health Sciences, East Carolina University, Greenville, NC
| | | |
Collapse
|
30
|
Pedersen M, Jønsson AO, Larsen CF. Genetics and voice production in childhood and adolescence - a review. Int J Pediatr Adolesc Med 2022; 9:73-77. [PMID: 35663784 PMCID: PMC9152557 DOI: 10.1016/j.ijpam.2021.02.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2020] [Accepted: 02/14/2021] [Indexed: 12/05/2022]
Abstract
Adolescence is a challenging time of change in voicing, normally and in pathology. An increased focus on voice production in relation to genetics can expand our knowledge of the onset of puberty and voice change. Our aim with this review was to connect research of genetics to voice production in adolescence. We need further understanding of the developmental background of voice in childhood and adolescence, because many genetic multi handicaps include voice production. Genetic development related to voice production was the focus in a search made by the Royal English Society of Medicine, with only a few results. We supplemented with references to genetic studies of adults and animals as well as adjacent areas of voice production. The genetic development of voice production is steered from the hypothalamus probably related to growth hormone. The genetic voice production in adults form the basis for understanding development. Some research results were found related to the pubertal steps. The findings are important in the future, using advanced voice analysis and artificial intelligence methods in patients with Multi handicaps.
Collapse
Affiliation(s)
- Mette Pedersen
- Medical Centre, Østergade 18, 1, 1100, Copenhagen, Denmark
| | - Anders Overgård Jønsson
- University of Copenhagen, Faculty of Health and Medical Sciences, Blegdamsvej 3B, 2200, Copenhagen, Denmark
| | | |
Collapse
|
31
|
Wang YY, Hamad AS, Palaniappan K, Lever TE, Bunyak F. LARNet-STC: Spatio-temporal orthogonal region selection network for laryngeal closure detection in endoscopy videos. Comput Biol Med 2022; 144:105339. [PMID: 35263687 PMCID: PMC8995389 DOI: 10.1016/j.compbiomed.2022.105339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2021] [Revised: 02/18/2022] [Accepted: 02/19/2022] [Indexed: 11/28/2022]
Abstract
The vocal folds (VFs) are a pair of muscles in the larynx that play a critical role in breathing, swallowing, and speaking. VF function can be adversely affected by various medical conditions including head or neck injuries, stroke, tumor, and neurological disorders. In this paper, we propose a deep learning system for automated detection of laryngeal adductor reflex (LAR) events in laryngeal endoscopy videos to enable objective, quantitative analysis of VF function. The proposed deep learning system incorporates our novel orthogonal region selection network and temporal context. This network learns to directly map its input to a VF open/close state without first segmenting or tracking the VF region. This one-step approach drastically reduces manual annotation needs from labor-intensive segmentation masks or VF motion tracks to frame-level class labels. The proposed spatio-temporal network with an orthogonal region selection subnetwork allows integration of local image features, global image features, and VF state information in time for robust LAR event detection. The proposed network is evaluated against several network variations that incorporate temporal context and is shown to lead to better performance. The experimental results show promising performance for automated, objective, and quantitative analysis of LAR events from laryngeal endoscopy videos with over 90% and 99% F1 scores for LAR and non-LAR frames respectively.
Collapse
Affiliation(s)
- Yang Yang Wang
- Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, 65211, Missouri, USA
| | - Ali S Hamad
- Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, 65211, Missouri, USA
| | - Kannappan Palaniappan
- Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, 65211, Missouri, USA
| | - Teresa E Lever
- Department of Otolaryngology - Head and Neck Surgery, University of Missouri, Columbia, 65211, Missouri, USA
| | - Filiz Bunyak
- Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, 65211, Missouri, USA.
| |
Collapse
|
32
|
Kopczynski B, Niebudek-Bogusz E, Pietruszewska W, Strumillo P. Segmentation of Glottal Images from High-Speed Videoendoscopy Optimized by Synchronous Acoustic Recordings. SENSORS (BASEL, SWITZERLAND) 2022; 22:1751. [PMID: 35270897 PMCID: PMC8915112 DOI: 10.3390/s22051751] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/30/2021] [Revised: 02/12/2022] [Accepted: 02/15/2022] [Indexed: 05/17/2023]
Abstract
Laryngeal high-speed videoendoscopy (LHSV) is an imaging technique offering novel visualization quality of the vibratory activity of the vocal folds. However, in most image analysis methods, the interaction of the medical personnel and access to ground truth annotations are required to achieve accurate detection of vocal folds edges. In our fully automatic method, we combine video and acoustic data that are synchronously recorded during the laryngeal endoscopy. We show that the image segmentation algorithm of the glottal area can be optimized by matching the Fourier spectra of the pre-processed video and the spectra of the acoustic recording during the phonation of sustained vowel /i:/. We verify our method on a set of LHSV recordings taken from subjects with normophonic voice and patients with voice disorders due to glottal insufficiency. We show that the computed geometric indices of the glottal area make it possible to discriminate between normal and pathologic voices. The median of the Open Quotient and Minimal Relative Glottal Area values for healthy subjects were 0.69 and 0.06, respectively, while for dysphonic subjects were 1 and 0.35, respectively. We also validate these results using independent phoniatrician experts.
Collapse
Affiliation(s)
- Bartosz Kopczynski
- Institute of Electronics, Lodz University of Technology, 90-924 Lodz, Poland;
| | - Ewa Niebudek-Bogusz
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, 90-001 Lodz, Poland; (E.N.-B.); (W.P.)
| | - Wioletta Pietruszewska
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, 90-001 Lodz, Poland; (E.N.-B.); (W.P.)
| | - Pawel Strumillo
- Institute of Electronics, Lodz University of Technology, 90-924 Lodz, Poland;
| |
Collapse
|
33
|
Ishac D, Matta S, Bin S, Aziz H, Karam E, Abche A, Nassar G. Objective Assessment of Covid-19 Severity Affecting the Vocal and Respiratory System Using a Wearable, Autonomous Sound Collar. Cell Mol Bioeng 2021; 15:67-86. [PMID: 34777597 PMCID: PMC8570400 DOI: 10.1007/s12195-021-00712-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Accepted: 10/11/2021] [Indexed: 11/13/2022] Open
Abstract
Introduction Since the outbreak began in January 2020, Covid-19 has affected more than 161 million people worldwide and resulted in about 3.3 million deaths. Despite efforts to detect human infection with the virus as early as possible, the confirmatory test still requires the analysis of sputum or blood with estimated results available within approximately 30 minutes; this may potentially be followed by clinical referral if the patient shows signs of aggravated pneumonia. This work aims to implement a soft collar as a sound device dedicated to the objective evaluation of the pathophysiological state resulting from dysphonia of laryngeal origin or respiratory failure of inflammatory origin, in particular caused by Covid-19. Methods In this study, we exploit the vibrations of waves generated by the vocal and respiratory system of 30 people. A biocompatible acoustic sensor embedded in a soft collar around the neck collects these waves. The collar is also equipped with thermal sensors and a cross-data analysis module in both the temporal and frequency domains (STFT). The optimal coupling conditions and the electrical and dimensional characteristics of the sensors were defined based on a mathematical approach using a matrix formalism. Results The characteristics of the signals in the time domain combined with the quantities obtained from the STFT offer multidimensional information and a decision support tool for determining a pathophysiological state representative of the symptoms explored. The device, tested on 30 people, was able to differentiate patients with mild symptoms from those who had developed acute signs of respiratory failure on a severity scale of 1 to 10. Conclusion With the health constraints imposed by the effects of Covid-19, the heavy organization to be implemented resulting from the flow of diagnostics, tests and clinical management, it was urgent to develop innovative and safe biomedical technologies. This passive listening technique will contribute to the non-invasive assessment and dynamic observation of lesions. Moreover, it merits further examination to provide support for medical operators to improve clinical management. Supplementary Information The online version contains supplementary material available at 10.1007/s12195-021-00712-w.
Collapse
Affiliation(s)
- D Ishac
- Electrical Engineering Department, University of Balamand (UOB), Balamand, Lebanon
| | - S Matta
- Electrical Engineering Department, University of Balamand (UOB), Balamand, Lebanon
| | - S Bin
- College of Physics, University of Qingdao, Qingdao, China
| | - H Aziz
- Department of Pulmonary Pathology, Sahlgrenska University Hospital, Göteborg, Sweden
| | - E Karam
- Electrical Engineering Department, University of Balamand (UOB), Balamand, Lebanon
| | - A Abche
- Electrical Engineering Department, University of Balamand (UOB), Balamand, Lebanon
| | - G Nassar
- IEMN - CNRS UMR 8520-INSA (HdF)-Lille academic, Lille, France
| |
Collapse
|
34
|
Malinowski J, Niebudek-Bogusz E, Just M, Morawska J, Racino A, Hoffman J, Barańska M, Kowalczyk MM, Pietruszewska W. Laryngeal High-Speed Videoendoscopy with Laser Illumination: A Preliminary Report. Otolaryngol Pol 2021; 75:1-10. [PMID: 35175220 DOI: 10.5604/01.3001.0015.2575] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
<br><b>Introduction:</b> Advances in computer image analysis have enabled the use of new functional imaging methods in the diagnosis of laryngeal diseases. Particularly interesting techniques of dynamic laryngeal imaging involve High Speed Videoendoscopy (HSV). This still-developed technique allows to overcome the limitations of laryngovideostroboscopy (LVS) and a more detailed analysis of the glottal function based on the image of the actual vibrations of the vocal folds. It also enables the determination of objective coefficients parameterizing phonatory vibrations of the vocal folds.</br> <br><b>Aim:</b> The aim of this pilot study was to evaluate the use of a high-speed videoendoscopy set with laser illumination for the diagnosis of glottic pathology in ENT practice.</br> <br><b>Material and methods:</b> The study included 40 patients who underwent LVS followed by HSV. The modern HSV examination kit - Advanced Larynx Imager System (ALIS), used for the first time in a clinical setting in Poland, is characterized by significantly improved, compared to the previously used high-speed cameras, operational parameters - a light head, the possibility of continuous lighting operation without excessive heating of the head tip, registration of the image in full color scale. Thanks to such modernization, the safety and course of the examination do not differ from laryngoscopy conducted with commonly used recorders. The device owes some of these improvements to a laser illuminator which was used for the first time as the main light source in a high-speed camera. In the study, two cases were selected to present the results of HSV and the analysis of the generated kymograms - a woman with no glottic pathology and a man with a polyp of the right vocal fold. In the first case, the HSV examination compared with the LVS revealed a discrete glottis functional disorder in the form of a tendency to hyperphonation. The patient with an organic lesion had a clearly visible irregularity of vocal fold vibrations, which also allowed to trace mucosal wave disturbances related to its reflection from the pathological structure of the glottis and the formation of a return wave, both on the fold affected by the lesion and, to a lesser extent, contralaterally. The glottic dysfunctions observed in the studied patients were confirmed in the generated kymograms and the graphs of the glottal width waveform (GWW), as well as in the parameters calculated on their basis, assessing the frequency and amplitude of phonatory vibrations.</br> <br><b>Conclusions:</b> The use of high-speed videoendoscopy allows for a much more accurate assessment of the phonatory function of the glottis than in laryngovideostroboscopy. The presented HSV system allows for obtaining high quality kinematic images of the larynx, color fidelity, and contrast. The use of this technology in laryngological practice enables precise structural and functional assessment of the glottis and detection of discrete phonation disorders that elude the techniques used so far.</br>.
Collapse
Affiliation(s)
- Jakub Malinowski
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, Poland
| | - Ewa Niebudek-Bogusz
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, Poland
| | - Marcin Just
- Diagnova Technologies, Wroclaw Technology Park, Wroclaw, Poland
| | - Joanna Morawska
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, Poland
| | - Anna Racino
- Diagnova Technologies, Wroclaw Technology Park, Wroclaw, Poland
| | - Joanna Hoffman
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, Poland
| | - Magda Barańska
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, Poland
| | | | - Wioletta Pietruszewska
- Department of Otolaryngology, Head and Neck Oncology, Medical University of Lodz, Poland
| |
Collapse
|
35
|
Using deep learning to identify the recurrent laryngeal nerve during thyroidectomy. Sci Rep 2021; 11:14306. [PMID: 34253767 PMCID: PMC8275665 DOI: 10.1038/s41598-021-93202-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2021] [Accepted: 06/22/2021] [Indexed: 11/16/2022] Open
Abstract
Surgeons must visually distinguish soft-tissues, such as nerves, from surrounding anatomy to prevent complications and optimize patient outcomes. An accurate nerve segmentation and analysis tool could provide useful insight for surgical decision-making. Here, we present an end-to-end, automatic deep learning computer vision algorithm to segment and measure nerves. Unlike traditional medical imaging, our unconstrained setup with accessible handheld digital cameras, along with the unstructured open surgery scene, makes this task uniquely challenging. We investigate one common procedure, thyroidectomy, during which surgeons must avoid damaging the recurrent laryngeal nerve (RLN), which is responsible for human speech. We evaluate our segmentation algorithm on a diverse dataset across varied and challenging settings of operating room image capture, and show strong segmentation performance in the optimal image capture condition. This work lays the foundation for future research in real-time tissue discrimination and integration of accessible, intelligent tools into open surgery to provide actionable insights.
Collapse
|
36
|
Kist AM, Dürr S, Schützenberger A, Döllinger M. OpenHSV: an open platform for laryngeal high-speed videoendoscopy. Sci Rep 2021; 11:13760. [PMID: 34215788 PMCID: PMC8253769 DOI: 10.1038/s41598-021-93149-0] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2020] [Accepted: 06/03/2021] [Indexed: 11/22/2022] Open
Abstract
High-speed videoendoscopy is an important tool to study laryngeal dynamics, to quantify vocal fold oscillations, to diagnose voice impairments at laryngeal level and to monitor treatment progress. However, there is a significant lack of an open source, expandable research tool that features latest hardware and data analysis. In this work, we propose an open research platform termed OpenHSV that is based on state-of-the-art, commercially available equipment and features a fully automatic data analysis pipeline. A publicly available, user-friendly graphical user interface implemented in Python is used to interface the hardware. Video and audio data are recorded in synchrony and are subsequently fully automatically analyzed. Video segmentation of the glottal area is performed using efficient deep neural networks to derive glottal area waveform and glottal midline. Established quantitative, clinically relevant video and audio parameters were implemented and computed. In a preliminary clinical study, we recorded video and audio data from 28 healthy subjects. Analyzing these data in terms of image quality and derived quantitative parameters, we show the applicability, performance and usefulness of OpenHSV. Therefore, OpenHSV provides a valid, standardized access to high-speed videoendoscopy data acquisition and analysis for voice scientists, highlighting its use as a valuable research tool in understanding voice physiology. We envision that OpenHSV serves as basis for the next generation of clinical HSV systems.
Collapse
Affiliation(s)
- Andreas M Kist
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, Waldstr. 1, 91054, Erlangen, Germany. .,Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-University Erlangen-Nürnberg, Henkestr. 91, 91054, Erlangen, Germany.
| | - Stephan Dürr
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, Waldstr. 1, 91054, Erlangen, Germany
| | - Anne Schützenberger
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, Waldstr. 1, 91054, Erlangen, Germany
| | - Michael Döllinger
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, Waldstr. 1, 91054, Erlangen, Germany
| |
Collapse
|
37
|
Kist AM, Gómez P, Dubrovskiy D, Schlegel P, Kunduk M, Echternach M, Patel R, Semmler M, Bohr C, Dürr S, Schützenberger A, Döllinger M. A Deep Learning Enhanced Novel Software Tool for Laryngeal Dynamics Analysis. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2021; 64:1889-1903. [PMID: 34000199 DOI: 10.1044/2021_jslhr-20-00498] [Citation(s) in RCA: 43] [Impact Index Per Article: 14.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Purpose High-speed videoendoscopy (HSV) is an emerging, but barely used, endoscopy technique in the clinic to assess and diagnose voice disorders because of the lack of dedicated software to analyze the data. HSV allows to quantify the vocal fold oscillations by segmenting the glottal area. This challenging task has been tackled by various studies; however, the proposed approaches are mostly limited and not suitable for daily clinical routine. Method We developed a user-friendly software in C# that allows the editing, motion correction, segmentation, and quantitative analysis of HSV data. We further provide pretrained deep neural networks for fully automatic glottis segmentation. Results We freely provide our software Glottis Analysis Tools (GAT). Using GAT, we provide a general threshold-based region growing platform that enables the user to analyze data from various sources, such as in vivo recordings, ex vivo recordings, and high-speed footage of artificial vocal folds. Additionally, especially for in vivo recordings, we provide three robust neural networks at various speed and quality settings to allow a fully automatic glottis segmentation needed for application by untrained personnel. GAT further evaluates video and audio data in parallel and is able to extract various features from the video data, among others the glottal area waveform, that is, the changing glottal area over time. In total, GAT provides 79 unique quantitative analysis parameters for video- and audio-based signals. Many of these parameters have already been shown to reflect voice disorders, highlighting the clinical importance and usefulness of the GAT software. Conclusion GAT is a unique tool to process HSV and audio data to determine quantitative, clinically relevant parameters for research, diagnosis, and treatment of laryngeal disorders. Supplemental Material https://doi.org/10.23641/asha.14575533.
Collapse
Affiliation(s)
- Andreas M Kist
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology-Head & Neck Surgery, University Hospital Erlangen, Germany
| | - Pablo Gómez
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology-Head & Neck Surgery, University Hospital Erlangen, Germany
| | - Denis Dubrovskiy
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology-Head & Neck Surgery, University Hospital Erlangen, Germany
| | - Patrick Schlegel
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology-Head & Neck Surgery, University Hospital Erlangen, Germany
| | - Melda Kunduk
- Department of Communication Sciences and Disorders, Louisiana State University, Baton Rouge
| | - Matthias Echternach
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Munich University Hospital (LMU), Germany
| | - Rita Patel
- Department of Speech, Language and Hearing Sciences, College of Arts and Sciences, Indiana University, Bloomington
| | - Marion Semmler
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology-Head & Neck Surgery, University Hospital Erlangen, Germany
| | - Christopher Bohr
- Klinik und Poliklinik für Hals-Nasen-Ohren-Heilkunde Universitätsklinikum Regensburg, Germany
| | - Stephan Dürr
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology-Head & Neck Surgery, University Hospital Erlangen, Germany
| | - Anne Schützenberger
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology-Head & Neck Surgery, University Hospital Erlangen, Germany
| | - Michael Döllinger
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology-Head & Neck Surgery, University Hospital Erlangen, Germany
| |
Collapse
|
38
|
Kiser KJ, Barman A, Stieb S, Fuller CD, Giancardo L. Novel Autosegmentation Spatial Similarity Metrics Capture the Time Required to Correct Segmentations Better Than Traditional Metrics in a Thoracic Cavity Segmentation Workflow. J Digit Imaging 2021; 34:541-553. [PMID: 34027588 PMCID: PMC8329111 DOI: 10.1007/s10278-021-00460-3] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2020] [Revised: 03/28/2021] [Accepted: 05/04/2021] [Indexed: 12/20/2022] Open
Abstract
Automated segmentation templates can save clinicians time compared to de novo segmentation but may still take substantial time to review and correct. It has not been thoroughly investigated which automated segmentation-corrected segmentation similarity metrics best predict clinician correction time. Bilateral thoracic cavity volumes in 329 CT scans were segmented by a UNet-inspired deep learning segmentation tool and subsequently corrected by a fourth-year medical student. Eight spatial similarity metrics were calculated between the automated and corrected segmentations and associated with correction times using Spearman’s rank correlation coefficients. Nine clinical variables were also associated with metrics and correction times using Spearman’s rank correlation coefficients or Mann–Whitney U tests. The added path length, false negative path length, and surface Dice similarity coefficient correlated better with correction time than traditional metrics, including the popular volumetric Dice similarity coefficient (respectively ρ = 0.69, ρ = 0.65, ρ = − 0.48 versus ρ = − 0.25; correlation p values < 0.001). Clinical variables poorly represented in the autosegmentation tool’s training data were often associated with decreased accuracy but not necessarily with prolonged correction time. Metrics used to develop and evaluate autosegmentation tools should correlate with clinical time saved. To our knowledge, this is only the second investigation of which metrics correlate with time saved. Validation of our findings is indicated in other anatomic sites and clinical workflows. Novel spatial similarity metrics may be preferable to traditional metrics for developing and evaluating autosegmentation tools that are intended to save clinicians time.
Collapse
Affiliation(s)
- Kendall J. Kiser
- Department of Radiation Oncology, Washington University School of Medicine in St. Louis, St. Louis, MO USA
| | - Arko Barman
- Center for Precision Health, UT Health School of Biomedical Informatics, Houston, TX USA
| | - Sonja Stieb
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Clifton D. Fuller
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Luca Giancardo
- Center for Precision Health, UT Health School of Biomedical Informatics, Houston, TX USA
| |
Collapse
|
39
|
Cho WK, Lee YJ, Joo HA, Jeong IS, Choi Y, Nam SY, Kim SY, Choi SH. Diagnostic Accuracies of Laryngeal Diseases Using a Convolutional Neural Network-Based Image Classification System. Laryngoscope 2021; 131:2558-2566. [PMID: 34000069 DOI: 10.1002/lary.29595] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2020] [Revised: 04/17/2021] [Accepted: 04/21/2021] [Indexed: 12/21/2022]
Abstract
OBJECTIVES/HYPOTHESIS There may be an interobserver variation in the diagnosis of laryngeal disease based on laryngoscopic images according to clinical experience. Therefore, this study is aimed to perform computer-assisted diagnosis for common laryngeal diseases using deep learning-based disease classification models. STUDY DESIGN Experimental study with retrospective data METHODS: A total of 4106 images (cysts, nodules, polyps, leukoplakia, papillomas, Reinke's edema, granulomas, palsies, and normal cases) were analyzed. After equal distribution of diseases into ninefolds, stratified eightfold cross-validation was performed for training, validation process and remaining onefold was used as a test dataset. A trained model was applied to test sets, and model performance was assessed for precision (positive predictive value), recall (sensitivity), accuracy, F1 score, precision-recall (PR) curve, and PR-area under the receiver operating characteristic curve (PR-AUC). Outcomes were compared to those of visual assessments by four trainees. RESULTS The trained deep neural networks (DNNs) outperformed trainees' visual assessments in discriminating cysts, granulomas, nodules, normal cases, palsies, papillomas, and polyps according to the PR-AUC and F1 score. The lowest F1 score and PR-AUC of DNNs were estimated for Reinke's edema (0.720, 0.800) and nodules (0.730, 0.780) but were comparable to the mean of the two trainees' F1 score with the best performances (0.765 and 0.675, respectively). In discriminating papillomas, the F1 score was much higher for DNNs (0.870) than for trainees (0.685). Overall, DNNs outperformed all trainees (micro-average PR-AUC = 0.95; macro-average PR-AUC = 0.91). CONCLUSIONS DNN technology could be applied to laryngoscopy to supplement clinical assessment of examiners by providing additional diagnostic clues and having a role as a reference of diagnosis. LEVEL OF EVIDENCE 3 Laryngoscope, 2021.
Collapse
Affiliation(s)
- Won Ki Cho
- Department of Otorhinolaryngology-Head and Neck Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Yeong Ju Lee
- Department of Otorhinolaryngology-Head and Neck Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Hye Ah Joo
- Department of Otorhinolaryngology-Head and Neck Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - In Seong Jeong
- Department of Otorhinolaryngology-Head and Neck Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Yeonjoo Choi
- Department of Otorhinolaryngology-Head and Neck Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Soon Yuhl Nam
- Department of Otorhinolaryngology-Head and Neck Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Sang Yoon Kim
- Department of Otorhinolaryngology-Head and Neck Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Seung-Ho Choi
- Department of Otorhinolaryngology-Head and Neck Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| |
Collapse
|
40
|
Yousef AM, Deliyski DD, Zacharias SRC, de Alarcon A, Orlikoff RF, Naghibolhosseini M. A Hybrid Machine-Learning-Based Method for Analytic Representation of the Vocal Fold Edges during Connected Speech. APPLIED SCIENCES-BASEL 2021; 11. [PMID: 33717604 PMCID: PMC7954580 DOI: 10.3390/app11031179] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Investigating the phonatory processes in connected speech from high-speed videoendoscopy (HSV) demands the accurate detection of the vocal fold edges during vibration. The present paper proposes a new spatio-temporal technique to automatically segment vocal fold edges in HSV data during running speech. The HSV data were recorded from a vocally normal adult during a reading of the “Rainbow Passage.” The introduced technique was based on an unsupervised machine-learning (ML) approach combined with an active contour modeling (ACM) technique (also known as a hybrid approach). The hybrid method was implemented to capture the edges of vocal folds on different HSV kymograms, extracted at various cross-sections of vocal folds during vibration. The k-means clustering method, an ML approach, was first applied to cluster the kymograms to identify the clustered glottal area and consequently provided an initialized contour for the ACM. The ACM algorithm was then used to precisely detect the glottal edges of the vibrating vocal folds. The developed algorithm was able to accurately track the vocal fold edges across frames with low computational cost and high robustness against image noise. This algorithm offers a fully automated tool for analyzing the vibratory features of vocal folds in connected speech.
Collapse
Affiliation(s)
- Ahmed M. Yousef
- Department of Communicative Sciences and Disorders, Michigan State University, East Lansing, MI 48824, USA
| | - Dimitar D. Deliyski
- Department of Communicative Sciences and Disorders, Michigan State University, East Lansing, MI 48824, USA
| | - Stephanie R. C. Zacharias
- Head and Neck Regenerative Medicine Program, Mayo Clinic, Scottsdale, AZ 85259, and Department of Otolaryngology-Head and Neck Surgery, Mayo Clinic, Phoenix, AZ 85054, USA
| | - Alessandro de Alarcon
- Division of Pediatric Otolaryngology, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH 45229, and Department of Otolaryngology—Head and Neck Surgery, University of Cincinnati College of Medicine, Cincinnati, OH 45267, USA
| | - Robert F. Orlikoff
- College of Allied Health Sciences, East Carolina University, Greenville, NC 27834, USA
| | - Maryam Naghibolhosseini
- Department of Communicative Sciences and Disorders, Michigan State University, East Lansing, MI 48824, USA
- Correspondence: ; Tel.: +1-517-884-2256
| |
Collapse
|
41
|
Abstract
A healthy voice is crucial for verbal communication and hence in daily as well as professional life. The basis for a healthy voice are the sound producing vocal folds in the larynx. A hallmark of healthy vocal fold oscillation is the symmetric motion of the left and right vocal fold. Clinically, videoendoscopy is applied to assess the symmetry of the oscillation and evaluated subjectively. High-speed videoendoscopy, an emerging method that allows quantification of the vocal fold oscillation, is more commonly employed in research due to the amount of data and the complex, semi-automatic analysis. In this study, we provide a comprehensive evaluation of methods that detect fully automatically the glottal midline. We used a biophysical model to simulate different vocal fold oscillations, extended the openly available BAGLS dataset using manual annotations, utilized both, simulations and annotated endoscopic images, to train deep neural networks at different stages of the analysis workflow, and compared these to established computer vision algorithms. We found that classical computer vision perform well on detecting the glottal midline in glottis segmentation data, but are outperformed by deep neural networks on this task. We further suggest GlottisNet, a multi-task neural architecture featuring the simultaneous prediction of both, the opening between the vocal folds and the symmetry axis, leading to a huge step forward towards clinical applicability of quantitative, deep learning-assisted laryngeal endoscopy, by fully automating segmentation and midline detection.
Collapse
|
42
|
Belagali V, Rao M V A, Gopikishore P, Krishnamurthy R, Ghosh PK. Two step convolutional neural network for automatic glottis localization and segmentation in stroboscopic videos. BIOMEDICAL OPTICS EXPRESS 2020; 11:4695-4713. [PMID: 32923072 PMCID: PMC7449707 DOI: 10.1364/boe.396252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/28/2020] [Revised: 07/16/2020] [Accepted: 07/16/2020] [Indexed: 06/11/2023]
Abstract
Precise analysis of the vocal fold vibratory pattern in a stroboscopic video plays a key role in the evaluation of voice disorders. Automatic glottis segmentation is one of the preliminary steps in such analysis. In this work, it is divided into two subproblems namely, glottis localization and glottis segmentation. A two step convolutional neural network (CNN) approach is proposed for the automatic glottis segmentation. Data augmentation is carried out using two techniques : 1) Blind rotation (WB), 2) Rotation with respect to glottis orientation (WO). The dataset used in this study contains stroboscopic videos of 18 subjects with Sulcus vocalis, in which the glottis region is annotated by three speech language pathologists (SLPs). The proposed two step CNN approach achieves an average localization accuracy of 90.08% and a mean dice score of 0.65.
Collapse
Affiliation(s)
- Varun Belagali
- Computer Science and Engineering, RV College of Engineering, Bangalore 560059, India
| | - Achuth Rao M V
- Electrical Engineering, Indian Institute of Science, Bangalore 560012, India
| | | | - Rahul Krishnamurthy
- Department of Audiology and Speech Language Pathology, Kasturba Medical College, Mangalore, Manipal Academy of Higher Education, Manipal, India
| | | |
Collapse
|
43
|
Gómez P, Kist AM, Schlegel P, Berry DA, Chhetri DK, Dürr S, Echternach M, Johnson AM, Kniesburges S, Kunduk M, Maryn Y, Schützenberger A, Verguts M, Döllinger M. BAGLS, a multihospital Benchmark for Automatic Glottis Segmentation. Sci Data 2020; 7:186. [PMID: 32561845 PMCID: PMC7305104 DOI: 10.1038/s41597-020-0526-3] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2019] [Accepted: 05/15/2020] [Indexed: 02/06/2023] Open
Abstract
Laryngeal videoendoscopy is one of the main tools in clinical examinations for voice disorders and voice research. Using high-speed videoendoscopy, it is possible to fully capture the vocal fold oscillations, however, processing the recordings typically involves a time-consuming segmentation of the glottal area by trained experts. Even though automatic methods have been proposed and the task is particularly suited for deep learning methods, there are no public datasets and benchmarks available to compare methods and to allow training of generalizing deep learning models. In an international collaboration of researchers from seven institutions from the EU and USA, we have created BAGLS, a large, multihospital dataset of 59,250 high-speed videoendoscopy frames with individually annotated segmentation masks. The frames are based on 640 recordings of healthy and disordered subjects that were recorded with varying technical equipment by numerous clinicians. The BAGLS dataset will allow an objective comparison of glottis segmentation methods and will enable interested researchers to train their own models and compare their methods.
Collapse
Affiliation(s)
- Pablo Gómez
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander University Erlangen-Nürnberg, Waldstraße 1, 91054, Erlangen, Germany.
| | - Andreas M Kist
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander University Erlangen-Nürnberg, Waldstraße 1, 91054, Erlangen, Germany.
| | - Patrick Schlegel
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander University Erlangen-Nürnberg, Waldstraße 1, 91054, Erlangen, Germany
| | - David A Berry
- Department of Head and Neck Surgery, David Geffen School of Medicine at the University of California, Los Angeles, Los Angeles, California, USA
| | - Dinesh K Chhetri
- Department of Head and Neck Surgery, David Geffen School of Medicine at the University of California, Los Angeles, Los Angeles, California, USA
| | - Stephan Dürr
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander University Erlangen-Nürnberg, Waldstraße 1, 91054, Erlangen, Germany
| | - Matthias Echternach
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Munich University Hospital (LMU), Munich, Germany
| | - Aaron M Johnson
- NYU Voice Center, Department of Otolaryngology - Head and Neck Surgery, New York University School of Medicine, New York, New York, USA
| | - Stefan Kniesburges
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander University Erlangen-Nürnberg, Waldstraße 1, 91054, Erlangen, Germany
| | - Melda Kunduk
- Department of Communication Sciences and Disorders, Louisiana State University, Baton Rouge, Louisiana, USA
| | - Youri Maryn
- European Institute for ORL-HNS, Department of Otorhinolaryngology and Head & Neck Surgery, Sint-Augustinus GZA, Wilrijk, Belgium
- Department of Speech, Language and Hearing sciences, University of Ghent, Ghent, Belgium
- Faculty of Education, Health and Social Work, University College Ghent, Ghent, Belgium
- Faculty of Psychology and Educational Sciences, School of Logopedics, Université Catholique de Louvain, Louvain-la-Neuve, Belgium
- Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium
| | - Anne Schützenberger
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander University Erlangen-Nürnberg, Waldstraße 1, 91054, Erlangen, Germany
| | - Monique Verguts
- European Institute for ORL-HNS, Department of Otorhinolaryngology and Head & Neck Surgery, Sint-Augustinus GZA, Wilrijk, Belgium
- Department of Otorhinolaryngology and Voice Disorders, Diest General Hospital, Diest, Belgium
| | - Michael Döllinger
- Division of Phoniatrics and Pediatric Audiology, Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Erlangen, Friedrich-Alexander University Erlangen-Nürnberg, Waldstraße 1, 91054, Erlangen, Germany
| |
Collapse
|