1
|
Paderno A, Bedi N, Rau A, Holsinger CF. Computer Vision and Videomics in Otolaryngology-Head and Neck Surgery: Bridging the Gap Between Clinical Needs and the Promise of Artificial Intelligence. Otolaryngol Clin North Am 2024:S0030-6665(24)00074-4. [PMID: 38981809 DOI: 10.1016/j.otc.2024.05.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/11/2024]
Abstract
This article discusses the role of computer vision in otolaryngology, particularly through endoscopy and surgery. It covers recent applications of artificial intelligence (AI) in nonradiologic imaging within otolaryngology, noting the benefits and challenges, such as improving diagnostic accuracy and optimizing therapeutic outcomes, while also pointing out the necessity for enhanced data curation and standardized research methodologies to advance clinical applications. Technical aspects are also covered, providing a detailed view of the progression from manual feature extraction to more complex AI models, including convolutional neural networks and vision transformers and their potential application in clinical settings.
Collapse
Affiliation(s)
- Alberto Paderno
- IRCCS Humanitas Research Hospital, via Manzoni 56, Rozzano, Milan 20089, Italy; Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, Pieve Emanuele, Milan 20072, Italy.
| | - Nikita Bedi
- Division of Head and Neck Surgery, Department of Otolaryngology, Stanford University, Palo Alto, CA, USA
| | - Anita Rau
- Department of Biomedical Data Science, Stanford University, Palo Alto, CA, USA
| | | |
Collapse
|
2
|
Oliver J, Alapati R, Lee J, Bur A. Artificial Intelligence in Head and Neck Surgery. Otolaryngol Clin North Am 2024:S0030-6665(24)00070-7. [PMID: 38910064 DOI: 10.1016/j.otc.2024.05.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/25/2024]
Abstract
This article explores artificial intelligence's (AI's) role in otolaryngology for head and neck cancer diagnosis and management. It highlights AI's potential in pattern recognition for early cancer detection, prognostication, and treatment planning, primarily through image analysis using clinical, endoscopic, and histopathologic images. Radiomics is also discussed at length, as well as the many ways that radiologic image analysis can be utilized, including for diagnosis, lymph node metastasis prediction, and evaluation of treatment response. The study highlights AI's promise and limitations, underlining the need for clinician-data scientist collaboration to enhance head and neck cancer care.
Collapse
Affiliation(s)
- Jamie Oliver
- Department of Otolaryngology-Head and Neck Surgery, University of Kansas School of Medicine, 3901 Rainbow Boulevard M.S. 3010, Kansas City, KS, USA
| | - Rahul Alapati
- Department of Otolaryngology-Head and Neck Surgery, University of Kansas School of Medicine, 3901 Rainbow Boulevard M.S. 3010, Kansas City, KS, USA
| | - Jason Lee
- Department of Otolaryngology-Head and Neck Surgery, University of Kansas School of Medicine, 3901 Rainbow Boulevard M.S. 3010, Kansas City, KS, USA
| | - Andrés Bur
- Department of Otolaryngology-Head and Neck Surgery, University of Kansas School of Medicine, 3901 Rainbow Boulevard M.S. 3010, Kansas City, KS, USA.
| |
Collapse
|
3
|
Paderno A, Rau A, Bedi N, Bossi P, Mercante G, Piazza C, Holsinger FC. Computer Vision Foundation Models in Endoscopy: Proof of Concept in Oropharyngeal Cancer. Laryngoscope 2024. [PMID: 38850247 DOI: 10.1002/lary.31534] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2024] [Revised: 04/15/2024] [Accepted: 05/06/2024] [Indexed: 06/10/2024]
Abstract
OBJECTIVES To evaluate the performance of vision transformer-derived image embeddings for distinguishing between normal and neoplastic tissues in the oropharynx and to investigate the potential of computer vision (CV) foundation models in medical imaging. METHODS Computational study using endoscopic frames with a focus on the application of a self-supervised vision transformer model (DINOv2) for tissue classification. High-definition endoscopic images were used to extract image patches that were then normalized and processed using the DINOv2 model to obtain embeddings. These embeddings served as input for a standard support vector machine (SVM) to classify the tissues as neoplastic or normal. The model's discriminative performance was validated using an 80-20 train-validation split. RESULTS From 38 endoscopic NBI videos, 327 image patches were analyzed. The classification results in the validation cohort demonstrated high accuracy (92%) and precision (89%), with a perfect recall (100%) and an F1-score of 94%. The receiver operating characteristic (ROC) curve yielded an area under the curve (AUC) of 0.96. CONCLUSION The use of large vision model-derived embeddings effectively differentiated between neoplastic and normal oropharyngeal tissues. This study supports the feasibility of employing CV foundation models like DINOv2 in the endoscopic evaluation of mucosal lesions, potentially augmenting diagnostic precision in Otorhinolaryngology. LEVEL OF EVIDENCE 4 Laryngoscope, 2024.
Collapse
Affiliation(s)
- Alberto Paderno
- Otorhinolaryngology Unit, IRCCS Humanitas Research Hospital, Milan, Italy
- Department of Biomedical Sciences, Humanitas University, Milan, Italy
| | - Anita Rau
- Department of Biomedical Data Science, Stanford University, Palo Alto, California, U.S.A
| | - Nikita Bedi
- Division of Head and Neck Surgery, Department of Otolaryngology, Stanford University, Palo Alto, California, U.S.A
| | - Paolo Bossi
- Department of Biomedical Sciences, Humanitas University, Milan, Italy
- Oncology Unit, IRCCS Humanitas Research Hospital, Milan, Italy
| | - Giuseppe Mercante
- Otorhinolaryngology Unit, IRCCS Humanitas Research Hospital, Milan, Italy
- Department of Biomedical Sciences, Humanitas University, Milan, Italy
| | - Cesare Piazza
- Unit of Otorhinolaryngology - Head and Neck Surgery, ASST Spedali Civili, Department of Surgical and Medical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Floyd Christopher Holsinger
- Division of Head and Neck Surgery, Department of Otolaryngology, Stanford University, Palo Alto, California, U.S.A
| |
Collapse
|
4
|
Tie CW, Li DY, Zhu JQ, Wang ML, Wang JH, Chen BH, Li Y, Zhang S, Liu L, Guo L, Yang L, Yang LQ, Wei J, Jiang F, Zhao ZQ, Wang GQ, Zhang W, Zhang QM, Ni XG. Multi-Instance Learning for Vocal Fold Leukoplakia Diagnosis Using White Light and Narrow-Band Imaging: A Multicenter Study. Laryngoscope 2024. [PMID: 38801129 DOI: 10.1002/lary.31537] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Revised: 05/01/2024] [Accepted: 05/09/2024] [Indexed: 05/29/2024]
Abstract
OBJECTIVES Vocal fold leukoplakia (VFL) is a precancerous lesion of laryngeal cancer, and its endoscopic diagnosis poses challenges. We aim to develop an artificial intelligence (AI) model using white light imaging (WLI) and narrow-band imaging (NBI) to distinguish benign from malignant VFL. METHODS A total of 7057 images from 426 patients were used for model development and internal validation. Additionally, 1617 images from two other hospitals were used for model external validation. Modeling learning based on WLI and NBI modalities was conducted using deep learning combined with a multi-instance learning approach (MIL). Furthermore, 50 prospectively collected videos were used to evaluate real-time model performance. A human-machine comparison involving 100 patients and 12 laryngologists assessed the real-world effectiveness of the model. RESULTS The model achieved the highest area under the receiver operating characteristic curve (AUC) values of 0.868 and 0.884 in the internal and external validation sets, respectively. AUC in the video validation set was 0.825 (95% CI: 0.704-0.946). In the human-machine comparison, AI significantly improved AUC and accuracy for all laryngologists (p < 0.05). With the assistance of AI, the diagnostic abilities and consistency of all laryngologists improved. CONCLUSIONS Our multicenter study developed an effective AI model using MIL and fusion of WLI and NBI images for VFL diagnosis, particularly aiding junior laryngologists. However, further optimization and validation are necessary to fully assess its potential impact in clinical settings. LEVEL OF EVIDENCE 3 Laryngoscope, 2024.
Collapse
Affiliation(s)
- Cheng-Wei Tie
- Department of Endoscopy, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - De-Yang Li
- The First Affiliated Hospital of Harbin Medical University, Harbin, China
| | - Ji-Qing Zhu
- Department of Endoscopy, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mei-Ling Wang
- Department of Endoscopy, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital & Shenzhen Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shenzhen, China
| | - Jian-Hui Wang
- Department of Endoscopy, Shanxi Province Cancer Hospital/Shanxi Hospital Affiliated to Cancer Hospital, Chinese Academy of Medical Sciences/Cancer Hospital Affiliated to Shanxi Medical University, Taiyuan, China
| | - Bing-Hong Chen
- Department of Endoscopy, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital & Shenzhen Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shenzhen, China
| | - Ying Li
- Department of Endoscopy, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital & Shenzhen Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shenzhen, China
| | - Sen Zhang
- Department of Otolaryngology Head and Neck Surgery, The First Hospital, Shanxi Medical University, Taiyuan, China
| | - Lin Liu
- Department of Otolaryngology Head and Neck Surgery, Dalian Friendship Hospital, Dalian, China
| | - Li Guo
- Department of Otolaryngology Head and Neck Surgery, The First Affiliated Hospital, College of Clinical Medicine of Henan University of Science and Technology, Luoyang, China
| | - Long Yang
- Department of Otolaryngology, The Second People's Hospital of Baoshan City, Baoshan, China
| | - Li-Qun Yang
- Department of Otolaryngology, The Second People's Hospital of Baoshan City, Baoshan, China
| | - Jiao Wei
- Department of Otolaryngology, Qujing Second People's Hospital of Yunnan Province, Qujing, China
| | - Feng Jiang
- Department of Otolaryngology, Kunming First People's Hospital, Kunming, China
| | - Zhi-Qiang Zhao
- Department of Otolaryngology, Baoshan People's Hospital, Baoshan, China
| | - Gui-Qi Wang
- Department of Endoscopy, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Wei Zhang
- Department of Endoscopy, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital & Shenzhen Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Shenzhen, China
| | - Quan-Mao Zhang
- Department of Endoscopy, Shanxi Province Cancer Hospital/Shanxi Hospital Affiliated to Cancer Hospital, Chinese Academy of Medical Sciences/Cancer Hospital Affiliated to Shanxi Medical University, Taiyuan, China
| | - Xiao-Guang Ni
- Department of Endoscopy, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
5
|
Kim YE, Serpedin A, Periyakoil P, German D, Rameau A. Sociodemographic reporting in videomics research: a review of practices in otolaryngology - head and neck surgery. Eur Arch Otorhinolaryngol 2024:10.1007/s00405-024-08659-0. [PMID: 38704768 DOI: 10.1007/s00405-024-08659-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2024] [Accepted: 04/02/2024] [Indexed: 05/07/2024]
Abstract
OBJECTIVE To assess reporting practices of sociodemographic data in Upper Aerodigestive Tract (UAT) videomics research in Otolaryngology-Head and Neck Surgery (OHNS). STUDY DESIGN Narrative review. METHODS Four online research databases were searched for peer-reviewed articles on videomics and UAT endoscopy in OHNS, published since January 1, 2017. Title and abstract search, followed by a full-text screening was performed. Dataset audit criteria were determined by the MINIMAR reporting standards for patient demographic characteristics, in addition to gender and author affiliations. RESULTS Of the 57 studies that were included, 37% reported any sociodemographic information on their dataset. Among these studies, all reported age, most reported sex (86%), two (10%) reported race, and one (5%) reported ethnicity and socioeconomic status. No studies reported gender. Most studies (84%) included at least one female author, and more than half of the studies (53%) had female first/senior authors, with no significant differences in the rate of sociodemographic reporting in studies with and without female authors (any female author: p = 0.2664; first/senior female author: p > 0.9999). Most studies based in the US reported at least one sociodemographic variable (79%), compared to those in Europe (24%) and in Asia (20%) (p = 0.0012). The rates of sociodemographic reporting in journals of different categories were as follows: clinical OHNS: 44%, clinical non-OHNS: 40%, technical: 42%, interdisciplinary: 10%. CONCLUSIONS There is prevalent underreporting of sociodemographic information in OHNS videomics research utilizing UAT endoscopy. Routine reporting of sociodemographic information should be implemented for AI-based research to help minimize algorithmic biases that have been previously demonstrated. LEVEL OF EVIDENCE: 4
Collapse
Affiliation(s)
- Yeo Eun Kim
- Department of Otolaryngology-Head and Neck Surgery, Weill Cornell Medicine, Sean Parker Institute for the Voice, 240 East 59th St, New York, NY, 10022, USA
| | - Aisha Serpedin
- Department of Otolaryngology-Head and Neck Surgery, Weill Cornell Medicine, Sean Parker Institute for the Voice, 240 East 59th St, New York, NY, 10022, USA
| | - Preethi Periyakoil
- Department of Otolaryngology-Head and Neck Surgery, Weill Cornell Medicine, Sean Parker Institute for the Voice, 240 East 59th St, New York, NY, 10022, USA
| | - Daniel German
- Department of Otolaryngology-Head and Neck Surgery, Weill Cornell Medicine, Sean Parker Institute for the Voice, 240 East 59th St, New York, NY, 10022, USA
| | - Anaïs Rameau
- Department of Otolaryngology-Head and Neck Surgery, Weill Cornell Medicine, Sean Parker Institute for the Voice, 240 East 59th St, New York, NY, 10022, USA.
| |
Collapse
|
6
|
Bensoussan Y, Elemento O, Rameau A. Voice as an AI Biomarker of Health-Introducing Audiomics. JAMA Otolaryngol Head Neck Surg 2024; 150:283-284. [PMID: 38386315 DOI: 10.1001/jamaoto.2023.4807] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/23/2024]
Abstract
This Viewpoint discusses the need to create standards for audiomics to identify unique audio biomarkers of health and disease—now possible because of more efficient voice data analysis available through the use of artificial intelligence (AI)—and to improve patient care.
Collapse
Affiliation(s)
- Yaël Bensoussan
- USF Health Voice Center, Department of Otolaryngology-Head & Neck Surgery, University of South Florida Health Morsani College of Medicine, Tampa
| | - Olivier Elemento
- Englander Institute for Precision Medicine, Weill Cornell Medicine, New York, New York
| | - Anaïs Rameau
- Sean Parker Institute for the Voice, Department of Otolaryngology-Head and Neck Surgery, Weill Cornell Medicine, New York, New York
| |
Collapse
|
7
|
Ishikawa Y, Sugino T, Okubo K, Nakajima Y. Detecting the location of lung cancer on thoracoscopic images using deep convolutional neural networks. Surg Today 2023; 53:1380-1387. [PMID: 37354240 DOI: 10.1007/s00595-023-02708-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2022] [Accepted: 04/03/2023] [Indexed: 06/26/2023]
Abstract
OBJECTIVES The prevalence of minimally invasive surgeries has increased the need for tumor detection using thoracoscopic images during lung cancer surgery. We conducted this study to analyze the efficacy of a deep convolutional neural network (DCNN) for tumor detection using recorded thoracoscopic images of pulmonary surfaces. MATERIALS AND METHODS We collected 644 intraoperative thoracoscopic images of changes in pulmonary appearance from 427 patients with lung cancer between 2012 and 2021. The lesion areas on the thoracoscopic images were detected by bounding boxes using an advanced version of YOLO, a well-known DCNN for object detection. The DCNN model was trained and evaluated by a 15-fold cross-validation scheme. Each predicted bounding box was considered successful detection when it overlapped more than 50% of the lesion areas annotated by board-certified surgeons. RESULTS AND CONCLUSIONS Precision, recall, and F1-measured values of 91.9%, 90.5%, and 91.1%, respectively, were obtained. The presence of lymphatic vessel invasion was associated with successful detection (p = 0.045). The presence of pathological pleural invasion also showed a tendency toward successful detection (p = 0.081). The proposed DCNN-based algorithm yielded an accuracy of more than 90% tumor detection. These algorithms will help surgeons detect lung cancer displayed on a screen automatically.
Collapse
Affiliation(s)
- Yuya Ishikawa
- Department of Thoracic Surgery, Tokyo Medical and Dental University, Tokyo, Japan
| | - Takaaki Sugino
- Department of Biomedical Information, Institute of Biomaterials and Bioengineering, Tokyo Medical and Dental University, 2-3-10, Surugadai, Chiyoda-ku, Tokyo, 101-0062, Japan
| | - Kenichi Okubo
- Department of Thoracic Surgery, Tokyo Medical and Dental University, Tokyo, Japan
| | - Yoshikazu Nakajima
- Department of Biomedical Information, Institute of Biomaterials and Bioengineering, Tokyo Medical and Dental University, 2-3-10, Surugadai, Chiyoda-ku, Tokyo, 101-0062, Japan.
| |
Collapse
|
8
|
Paderno A, Villani FP, Sordi A, Montenegro C, Moccia S. Deep learning in endoscopy: the importance of standardisation. ACTA OTORHINOLARYNGOLOGICA ITALICA : ORGANO UFFICIALE DELLA SOCIETA ITALIANA DI OTORINOLARINGOLOGIA E CHIRURGIA CERVICO-FACCIALE 2023; 43:430-432. [PMID: 37814976 PMCID: PMC10773540 DOI: 10.14639/0392-100x-n2580] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/26/2023] [Accepted: 04/03/2023] [Indexed: 10/11/2023]
Affiliation(s)
- Alberto Paderno
- Unit of Otorhinolaryngology - Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | | | - Alessandra Sordi
- Unit of Otorhinolaryngology - Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Claudia Montenegro
- Unit of Otorhinolaryngology - Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Sara Moccia
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, Pisa, Italy
| |
Collapse
|
9
|
Tsilivigkos C, Athanasopoulos M, Micco RD, Giotakis A, Mastronikolis NS, Mulita F, Verras GI, Maroulis I, Giotakis E. Deep Learning Techniques and Imaging in Otorhinolaryngology-A State-of-the-Art Review. J Clin Med 2023; 12:6973. [PMID: 38002588 PMCID: PMC10672270 DOI: 10.3390/jcm12226973] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2023] [Revised: 11/02/2023] [Accepted: 11/06/2023] [Indexed: 11/26/2023] Open
Abstract
Over the last decades, the field of medicine has witnessed significant progress in artificial intelligence (AI), the Internet of Medical Things (IoMT), and deep learning (DL) systems. Otorhinolaryngology, and imaging in its various subspecialties, has not remained untouched by this transformative trend. As the medical landscape evolves, the integration of these technologies becomes imperative in augmenting patient care, fostering innovation, and actively participating in the ever-evolving synergy between computer vision techniques in otorhinolaryngology and AI. To that end, we conducted a thorough search on MEDLINE for papers published until June 2023, utilizing the keywords 'otorhinolaryngology', 'imaging', 'computer vision', 'artificial intelligence', and 'deep learning', and at the same time conducted manual searching in the references section of the articles included in our manuscript. Our search culminated in the retrieval of 121 related articles, which were subsequently subdivided into the following categories: imaging in head and neck, otology, and rhinology. Our objective is to provide a comprehensive introduction to this burgeoning field, tailored for both experienced specialists and aspiring residents in the domain of deep learning algorithms in imaging techniques in otorhinolaryngology.
Collapse
Affiliation(s)
- Christos Tsilivigkos
- 1st Department of Otolaryngology, National and Kapodistrian University of Athens, Hippocrateion Hospital, 115 27 Athens, Greece; (A.G.); (E.G.)
| | - Michail Athanasopoulos
- Department of Otolaryngology, University Hospital of Patras, 265 04 Patras, Greece; (M.A.); (N.S.M.)
| | - Riccardo di Micco
- Department of Otolaryngology and Head and Neck Surgery, Medical School of Hannover, 30625 Hannover, Germany;
| | - Aris Giotakis
- 1st Department of Otolaryngology, National and Kapodistrian University of Athens, Hippocrateion Hospital, 115 27 Athens, Greece; (A.G.); (E.G.)
| | - Nicholas S. Mastronikolis
- Department of Otolaryngology, University Hospital of Patras, 265 04 Patras, Greece; (M.A.); (N.S.M.)
| | - Francesk Mulita
- Department of Surgery, University Hospital of Patras, 265 04 Patras, Greece; (G.-I.V.); (I.M.)
| | - Georgios-Ioannis Verras
- Department of Surgery, University Hospital of Patras, 265 04 Patras, Greece; (G.-I.V.); (I.M.)
| | - Ioannis Maroulis
- Department of Surgery, University Hospital of Patras, 265 04 Patras, Greece; (G.-I.V.); (I.M.)
| | - Evangelos Giotakis
- 1st Department of Otolaryngology, National and Kapodistrian University of Athens, Hippocrateion Hospital, 115 27 Athens, Greece; (A.G.); (E.G.)
| |
Collapse
|
10
|
Esmaeili N, Davaris N, Boese A, Illanes A, Navab N, Friebe M, Arens C. Contact Endoscopy - Narrow Band Imaging (CE-NBI) data set for laryngeal lesion assessment. Sci Data 2023; 10:733. [PMID: 37865668 PMCID: PMC10590430 DOI: 10.1038/s41597-023-02629-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Accepted: 10/11/2023] [Indexed: 10/23/2023] Open
Abstract
The endoscopic examination of subepithelial vascular patterns within the vocal fold is crucial for clinicians seeking to distinguish between benign lesions and laryngeal cancer. Among innovative techniques, Contact Endoscopy combined with Narrow Band Imaging (CE-NBI) offers real-time visualization of these vascular structures. Despite the advent of CE-NBI, concerns have arisen regarding the subjective interpretation of its images. As a result, several computer-based solutions have been developed to address this issue. This study introduces the CE-NBI data set, the first publicly accessible data set that features enhanced and magnified visualizations of subepithelial blood vessels within the vocal fold. This data set encompasses 11144 images from 210 adult patients with pathological vocal fold conditions, where CE-NBI images are annotated using three distinct label categories. The data set has proven invaluable for numerous clinical assessments geared toward diagnosing laryngeal cancer using Optical Biopsy. Furthermore, given its versatility for various image analysis tasks, we have devised and implemented diverse image classification scenarios using Machine Learning (ML) approaches to address critical clinical challenges in assessing laryngeal lesions.
Collapse
Affiliation(s)
- Nazila Esmaeili
- Department of Otorhinolaryngology, Head and Neck Surgery, Justus Liebig University of Giessen, 35392, Giessen, Germany.
- Chair for Computer Aided Medical Procedures and Augmented Reality, Technical University of Munich, 85748, Munich, Germany.
- SURAG Medical GmbH, 04103, Leipzig, Germany.
| | - Nikolaos Davaris
- Department of Otorhinolaryngology, Head and Neck Surgery, Giessen University Hospital, 35392, Giessen, Germany
- Department of Otorhinolaryngology, Head and Neck Surgery, Magdeburg University Hospital, 39120, Magdeburg, Germany
| | - Axel Boese
- INKA-Innovation Laboratory for Image Guided Therapy, Medical Faculty, Otto-von-Guericke University Magdeburg, 39120, Magdeburg, Germany
| | | | - Nassir Navab
- Chair for Computer Aided Medical Procedures and Augmented Reality, Technical University of Munich, 85748, Munich, Germany
| | - Michael Friebe
- INKA-Innovation Laboratory for Image Guided Therapy, Medical Faculty, Otto-von-Guericke University Magdeburg, 39120, Magdeburg, Germany
- Department of Biocybernetics and Biomedical Engineering, AGH University Kraków, 30-059, Kraków, Poland
- CIBE - Center for Innovation, Business Development & Entrepreneurship, FOM University of Applied Sciences, 45141, Essen, Germany
| | - Christoph Arens
- Department of Otorhinolaryngology, Head and Neck Surgery, Giessen University Hospital, 35392, Giessen, Germany
| |
Collapse
|
11
|
Sampieri C, Baldini C, Azam MA, Moccia S, Mattos LS, Vilaseca I, Peretti G, Ioppi A. Artificial Intelligence for Upper Aerodigestive Tract Endoscopy and Laryngoscopy: A Guide for Physicians and State-of-the-Art Review. Otolaryngol Head Neck Surg 2023; 169:811-829. [PMID: 37051892 DOI: 10.1002/ohn.343] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Revised: 03/03/2023] [Accepted: 03/23/2023] [Indexed: 04/14/2023]
Abstract
OBJECTIVE The endoscopic and laryngoscopic examination is paramount for laryngeal, oropharyngeal, nasopharyngeal, nasal, and oral cavity benign lesions and cancer evaluation. Nevertheless, upper aerodigestive tract (UADT) endoscopy is intrinsically operator-dependent and lacks objective quality standards. At present, there has been an increased interest in artificial intelligence (AI) applications in this area to support physicians during the examination, thus enhancing diagnostic performances. The relative novelty of this research field poses a challenge both for the reviewers and readers as clinicians often lack a specific technical background. DATA SOURCES Four bibliographic databases were searched: PubMed, EMBASE, Cochrane, and Google Scholar. REVIEW METHODS A structured review of the current literature (up to September 2022) was performed. Search terms related to topics of AI, machine learning (ML), and deep learning (DL) in UADT endoscopy and laryngoscopy were identified and queried by 3 independent reviewers. Citations of selected studies were also evaluated to ensure comprehensiveness. CONCLUSIONS Forty-one studies were included in the review. AI and computer vision techniques were used to achieve 3 fundamental tasks in this field: classification, detection, and segmentation. All papers were summarized and reviewed. IMPLICATIONS FOR PRACTICE This article comprehensively reviews the latest developments in the application of ML and DL in UADT endoscopy and laryngoscopy, as well as their future clinical implications. The technical basis of AI is also explained, providing guidance for nonexpert readers to allow critical appraisal of the evaluation metrics and the most relevant quality requirements.
Collapse
Affiliation(s)
- Claudio Sampieri
- Department of Experimental Medicine (DIMES), University of Genoa, Genoa, Italy
- Functional Unit of Head and Neck Tumors, Hospital Clínic, Barcelona, Spain
- Otorhinolaryngology Department, Hospital Clínic, Barcelona, Spain
| | - Chiara Baldini
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy
- Dipartimento di Informatica, Bioingegneria, Robotica e Ingegneria dei Sistemi (DIBRIS), University of Genoa, Genoa, Italy
| | - Muhammad Adeel Azam
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy
- Dipartimento di Informatica, Bioingegneria, Robotica e Ingegneria dei Sistemi (DIBRIS), University of Genoa, Genoa, Italy
| | - Sara Moccia
- Department of Excellence in Robotics and AI, The BioRobotics Institute, Pisa, Italy
| | - Leonardo S Mattos
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Isabel Vilaseca
- Functional Unit of Head and Neck Tumors, Hospital Clínic, Barcelona, Spain
- Otorhinolaryngology Department, Hospital Clínic, Barcelona, Spain
- Head Neck Clínic, Agència de Gestió d'Ajuts Universitaris i de Recerca, Barcelona, Catalunya, Spain
- Surgery and Medical-Surgical Specialties Department, Faculty of Medicine and Health Sciences, Universitat de Barcelona, Barcelona, Spain
- Translational Genomics and Target Therapies in Solid Tumors Group, Faculty of Medicine, Institut d́Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
- University of Barcelona, Barcelona, Spain
| | - Giorgio Peretti
- Unit of Otorhinolaryngology-Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
- Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Alessandro Ioppi
- Unit of Otorhinolaryngology-Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
- Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| |
Collapse
|
12
|
Zhong NN, Wang HQ, Huang XY, Li ZZ, Cao LM, Huo FY, Liu B, Bu LL. Enhancing head and neck tumor management with artificial intelligence: Integration and perspectives. Semin Cancer Biol 2023; 95:52-74. [PMID: 37473825 DOI: 10.1016/j.semcancer.2023.07.002] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Revised: 07/11/2023] [Accepted: 07/15/2023] [Indexed: 07/22/2023]
Abstract
Head and neck tumors (HNTs) constitute a multifaceted ensemble of pathologies that primarily involve regions such as the oral cavity, pharynx, and nasal cavity. The intricate anatomical structure of these regions poses considerable challenges to efficacious treatment strategies. Despite the availability of myriad treatment modalities, the overall therapeutic efficacy for HNTs continues to remain subdued. In recent years, the deployment of artificial intelligence (AI) in healthcare practices has garnered noteworthy attention. AI modalities, inclusive of machine learning (ML), neural networks (NNs), and deep learning (DL), when amalgamated into the holistic management of HNTs, promise to augment the precision, safety, and efficacy of treatment regimens. The integration of AI within HNT management is intricately intertwined with domains such as medical imaging, bioinformatics, and medical robotics. This article intends to scrutinize the cutting-edge advancements and prospective applications of AI in the realm of HNTs, elucidating AI's indispensable role in prevention, diagnosis, treatment, prognostication, research, and inter-sectoral integration. The overarching objective is to stimulate scholarly discourse and invigorate insights among medical practitioners and researchers to propel further exploration, thereby facilitating superior therapeutic alternatives for patients.
Collapse
Affiliation(s)
- Nian-Nian Zhong
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Han-Qi Wang
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Xin-Yue Huang
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Zi-Zhan Li
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Lei-Ming Cao
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Fang-Yi Huo
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China
| | - Bing Liu
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China; Department of Oral & Maxillofacial - Head Neck Oncology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China.
| | - Lin-Lin Bu
- State Key Laboratory of Oral & Maxillofacial Reconstruction and Regeneration, Key Laboratory of Oral Biomedicine Ministry of Education, Hubei Key Laboratory of Stomatology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China; Department of Oral & Maxillofacial - Head Neck Oncology, School & Hospital of Stomatology, Wuhan University, Wuhan 430079, China.
| |
Collapse
|
13
|
Hong Y, Yu Q, Mo F, Yin M, Xu C, Zhu S, Lin J, Xu G, Gao J, Liu L, Wang Y. Deep learning to predict esophageal variceal bleeding based on endoscopic images. J Int Med Res 2023; 51:3000605231200371. [PMID: 37818651 PMCID: PMC10566287 DOI: 10.1177/03000605231200371] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Accepted: 08/24/2023] [Indexed: 10/12/2023] Open
Abstract
OBJECTIVE Esophageal varix (EV) bleeding is a particularly serious complications of cirrhosis. Prediction of EV bleeding requires extensive endoscopy experience; it remains unreliable and inefficient. This retrospective cohort study evaluated the feasibility of using deep learning (DL) to predict the 12-month risk of EV bleeding based on endoscopic images. METHODS Six DL models were trained to perform binary classification of endoscopic images of EV bleeding. The models were subsequently validated using an external test dataset, then compared with classifications performed by two endoscopists. RESULTS In the validation dataset, EfficientNet had the highest accuracy (0.910), followed by ConvMixer (0.898) and Xception (0.875). In the test dataset, EfficientNet maintained the highest accuracy (0.893), which was better than the endoscopists (0.800 and 0.763). Notably, one endoscopist displayed higher recall (0.905), compared with EfficientNet (0.870). When their predictions were assisted by artificial intelligence, the accuracies of the two endoscopists increased by 17.3% and 19.0%. Moreover, statistical agreement among the models was dependent on model architecture. CONCLUSIONS This study demonstrated the feasibility of using DL to predict the 12-month risk of EV bleeding based on endoscopic images. The findings suggest that artificial intelligence-aided diagnosis will be a useful addition to cirrhosis management.
Collapse
Affiliation(s)
- Yu Hong
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Qianqian Yu
- Department of Oncology, Jintan Affiliated Hospital of Jiangsu University, Jintan, China
| | - Feng Mo
- Department of General Surgery, Jintan Affiliated Hospital of Jiangsu University, Jintan, China
| | - Minyue Yin
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Chang Xu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Shiqi Zhu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Jiaxi Lin
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Guoting Xu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Jingwen Gao
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Lu Liu
- Department of Gastroenterology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Yu Wang
- Department of General Surgery, Jintan Affiliated Hospital of Jiangsu University, Jintan, China
| |
Collapse
|
14
|
Kim GH, Hwang YJ, Lee H, Sung ES, Nam KW. Convolutional neural network-based vocal cord tumor classification technique for home-based self-prescreening purpose. Biomed Eng Online 2023; 22:81. [PMID: 37596652 PMCID: PMC10439563 DOI: 10.1186/s12938-023-01139-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Accepted: 07/20/2023] [Indexed: 08/20/2023] Open
Abstract
BACKGROUND In this study, we proposed a deep learning technique that can simultaneously detect suspicious positions of benign vocal cord tumors in laparoscopic images and classify the types of tumors into cysts, granulomas, leukoplakia, nodules and polyps. This technique is useful for simplified home-based self-prescreening purposes to detect the generation of tumors around the vocal cord early in the benign stage. RESULTS We implemented four convolutional neural network (CNN) models (two Mask R-CNNs, Yolo V4, and a single-shot detector) that were trained, validated and tested using 2183 laryngoscopic images. The experimental results demonstrated that among the four applied models, Yolo V4 showed the highest F1-score for all tumor types (0.7664, cyst; 0.9875, granuloma; 0.8214, leukoplakia; 0.8119, nodule; and 0.8271, polyp). The model with the lowest false-negative rate was different for each tumor type (Yolo V4 for cysts/granulomas and Mask R-CNN for leukoplakia/nodules/polyps). In addition, the embedded-operated Yolo V4 model showed an approximately equivalent F1-score (0.8529) to that of the computer-operated Yolo-4 model (0.8683). CONCLUSIONS Based on these results, we conclude that the proposed deep-learning-based home screening techniques have the potential to aid in the early detection of tumors around the vocal cord and can improve the long-term survival of patients with vocal cord tumors.
Collapse
Affiliation(s)
- Gun Ho Kim
- Medical Research Institute, Pusan National University, Yangsan, Korea
- Department of Biomedical Engineering, Pusan National University Yangsan Hospital, Yangsan, Korea
| | - Young Jun Hwang
- Department of Biomedical Engineering, School of Medicine, Pusan National University, 49, Busandaehak-Ro, Mulgeum-Eup, Yangsan, 50629 Korea
| | - Hongje Lee
- Department of Nuclear Medicine, Dongnam Institute of Radiological & Medical Sciences, Busan, Korea
| | - Eui-Suk Sung
- Department of Otolaryngology-Head and Neck Surgery, Pusan National University Yangsan Hospital, Yangsan, Korea
- Department of Otolaryngology-Head and Neck Surgery, School of Medicine, Pusan National University, Yangsan, Korea
- Research Institute for Convergence of Biomedical Science and Technology, Pusan National University Yangsan Hospital, Yangsan, Korea
| | - Kyoung Won Nam
- Department of Biomedical Engineering, Pusan National University Yangsan Hospital, Yangsan, Korea
- Department of Biomedical Engineering, School of Medicine, Pusan National University, 49, Busandaehak-Ro, Mulgeum-Eup, Yangsan, 50629 Korea
- Research Institute for Convergence of Biomedical Science and Technology, Pusan National University Yangsan Hospital, Yangsan, Korea
| |
Collapse
|
15
|
Paderno A, Villani FP, Fior M, Berretti G, Gennarini F, Zigliani G, Ulaj E, Montenegro C, Sordi A, Sampieri C, Peretti G, Moccia S, Piazza C. Instance segmentation of upper aerodigestive tract cancer: site-specific outcomes. ACTA OTORHINOLARYNGOLOGICA ITALICA : ORGANO UFFICIALE DELLA SOCIETA ITALIANA DI OTORINOLARINGOLOGIA E CHIRURGIA CERVICO-FACCIALE 2023; 43:283-290. [PMID: 37488992 PMCID: PMC10366566 DOI: 10.14639/0392-100x-n2336] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Accepted: 01/08/2023] [Indexed: 07/26/2023]
Abstract
Objective To achieve instance segmentation of upper aerodigestive tract (UADT) neoplasms using a deep learning (DL) algorithm, and to identify differences in its diagnostic performance in three different sites: larynx/hypopharynx, oral cavity and oropharynx. Methods A total of 1034 endoscopic images from 323 patients were examined under narrow band imaging (NBI). The Mask R-CNN algorithm was used for the analysis. The dataset split was: 935 training, 48 validation and 51 testing images. Dice Similarity Coefficient (Dsc) was the main outcome measure. Results Instance segmentation was effective in 76.5% of images. The mean Dsc was 0.90 ± 0.05. The algorithm correctly predicted 77.8%, 86.7% and 55.5% of lesions in the larynx/hypopharynx, oral cavity, and oropharynx, respectively. The mean Dsc was 0.90 ± 0.05 for the larynx/hypopharynx, 0.60 ± 0.26 for the oral cavity, and 0.81 ± 0.30 for the oropharynx. The analysis showed inferior diagnostic results in the oral cavity compared with the larynx/hypopharynx (p < 0.001). Conclusions The study confirms the feasibility of instance segmentation of UADT using DL algorithms and shows inferior diagnostic results in the oral cavity compared with other anatomic areas.
Collapse
Affiliation(s)
- Alberto Paderno
- Unit of Otorhinolaryngology, Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | | | - Milena Fior
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Giulia Berretti
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Francesca Gennarini
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Gabriele Zigliani
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Emanuela Ulaj
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Claudia Montenegro
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Alessandra Sordi
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| | - Claudio Sampieri
- Unit of Otorhinolaryngology, Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
| | - Giorgio Peretti
- Unit of Otorhinolaryngology, Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
| | - Sara Moccia
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Cesare Piazza
- Unit of Otorhinolaryngology, Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, School of Medicine, Brescia, Italy
| |
Collapse
|
16
|
A Novel Framework of Manifold Learning Cascade-Clustering for the Informative Frame Selection. Diagnostics (Basel) 2023; 13:diagnostics13061151. [PMID: 36980459 PMCID: PMC10047422 DOI: 10.3390/diagnostics13061151] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 03/05/2023] [Accepted: 03/10/2023] [Indexed: 03/19/2023] Open
Abstract
Narrow band imaging is an established non-invasive tool used for the early detection of laryngeal cancer in surveillance examinations. Most images produced from the examination are useless, such as blurred, specular reflection, and underexposed. Removing the uninformative frames is vital to improve detection accuracy and speed up computer-aided diagnosis. It often takes a lot of time for the physician to manually inspect the informative frames. This issue is commonly addressed by a classifier with task-specific categories of the uninformative frames. However, the definition of the uninformative categories is ambiguous, and tedious labeling still cannot be avoided. Here, we show that a novel unsupervised scheme is comparable to the current benchmarks on the dataset of NBI-InfFrames. We extract feature embedding using a vanilla neural network (VGG16) and introduce a new dimensionality reduction method called UMAP that distinguishes the feature embedding in the lower-dimensional space. Along with the proposed automatic cluster labeling algorithm and cost function in Bayesian optimization, the proposed method coupled with UMAP achieves state-of-the-art performance. It outperforms the baseline by 12% absolute. The overall median recall of the proposed method is currently the highest, 96%. Our results demonstrate the effectiveness of the proposed scheme and the robustness of detecting the informative frames. It also suggests the patterns embedded in the data help develop flexible algorithms that do not require manual labeling.
Collapse
|
17
|
Bensoussan Y, Vanstrum EB, Johns MM, Rameau A. Artificial Intelligence and Laryngeal Cancer: From Screening to Prognosis: A State of the Art Review. Otolaryngol Head Neck Surg 2023; 168:319-329. [PMID: 35787073 DOI: 10.1177/01945998221110839] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Accepted: 06/13/2022] [Indexed: 11/16/2022]
Abstract
OBJECTIVE This state of the art review aims to examine contemporary advances in applications of artificial intelligence (AI) to the screening, detection, management, and prognostication of laryngeal cancer (LC). DATA SOURCES Four bibliographic databases were searched: PubMed, EMBASE, Cochrane, and IEEE. REVIEW METHODS A structured review of the current literature (up to January 2022) was performed. Search terms related to topics of AI in LC were identified and queried by 2 independent reviewers. Citations of selected studies and review articles were also evaluated to ensure comprehensiveness. CONCLUSIONS AI applications in LC have encompassed a variety of data modalities, including radiomics, genomics, acoustics, clinical data, and videomics, to support screening, diagnosis, therapeutic decision making, and prognosis. However, most studies remain at the proof-of-concept level, as AI algorithms are trained on single-institution databases with limited data sets and a single data modality. IMPLICATIONS FOR PRACTICE AI algorithms in LC will need to be trained on large multi-institutional data sets and integrate multimodal data for optimal performance and clinical utility from screening to prognosis. Out of the data types reviewed, genomics has the most potential to provide generalizable models thanks to available large multi-institutional open access genomic data sets. Voice acoustic data represent an inexpensive and accurate biomarker, which is easy and noninvasive to capture, offering a unique opportunity for screening and monitoring of LA, especially in low-resource settings.
Collapse
Affiliation(s)
- Yael Bensoussan
- Department of Otolaryngology-Head and Neck Surgery, University of South Florida, Tampa, Florida, USA
| | - Erik B Vanstrum
- Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Michael M Johns
- Department of Otolaryngology-Head and Neck Surgery, University of Southern California, Los Angeles, California, USA
| | - Anaïs Rameau
- Department of Otolaryngology-Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medical College, New York, New York, USA
| |
Collapse
|
18
|
Paderno A, Gennarini F, Sordi A, Montenegro C, Lancini D, Villani FP, Moccia S, Piazza C. Artificial intelligence in clinical endoscopy: Insights in the field of videomics. Front Surg 2022; 9:933297. [PMID: 36171813 PMCID: PMC9510389 DOI: 10.3389/fsurg.2022.933297] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2022] [Accepted: 08/22/2022] [Indexed: 11/13/2022] Open
Abstract
Artificial intelligence is being increasingly seen as a useful tool in medicine. Specifically, these technologies have the objective to extract insights from complex datasets that cannot easily be analyzed by conventional statistical methods. While promising results have been obtained for various -omics datasets, radiological images, and histopathologic slides, analysis of videoendoscopic frames still represents a major challenge. In this context, videomics represents a burgeoning field wherein several methods of computer vision are systematically used to organize unstructured data from frames obtained during diagnostic videoendoscopy. Recent studies have focused on five broad tasks with increasing complexity: quality assessment of endoscopic images, classification of pathologic and nonpathologic frames, detection of lesions inside frames, segmentation of pathologic lesions, and in-depth characterization of neoplastic lesions. Herein, we present a broad overview of the field, with a focus on conceptual key points and future perspectives.
Collapse
Affiliation(s)
- Alberto Paderno
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
- Correspondence: Alberto Paderno
| | - Francesca Gennarini
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
| | - Alessandra Sordi
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
| | - Claudia Montenegro
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
| | - Davide Lancini
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
| | - Francesca Pia Villani
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, Pisa, Italy
| | - Sara Moccia
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, Pisa, Italy
| | - Cesare Piazza
- Unit of Otorhinolaryngology—Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
| |
Collapse
|
19
|
Ali H, Sharif M, Yasmin M, Rehmani MH. A shallow extraction of texture features for classification of abnormal video endoscopy frames. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
20
|
Azam MA, Sampieri C, Ioppi A, Benzi P, Giordano GG, De Vecchi M, Campagnari V, Li S, Guastini L, Paderno A, Moccia S, Piazza C, Mattos LS, Peretti G. Videomics of the Upper Aero-Digestive Tract Cancer: Deep Learning Applied to White Light and Narrow Band Imaging for Automatic Segmentation of Endoscopic Images. Front Oncol 2022; 12:900451. [PMID: 35719939 PMCID: PMC9198427 DOI: 10.3389/fonc.2022.900451] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2022] [Accepted: 04/26/2022] [Indexed: 12/13/2022] Open
Abstract
Introduction Narrow Band Imaging (NBI) is an endoscopic visualization technique useful for upper aero-digestive tract (UADT) cancer detection and margins evaluation. However, NBI analysis is strongly operator-dependent and requires high expertise, thus limiting its wider implementation. Recently, artificial intelligence (AI) has demonstrated potential for applications in UADT videoendoscopy. Among AI methods, deep learning algorithms, and especially convolutional neural networks (CNNs), are particularly suitable for delineating cancers on videoendoscopy. This study is aimed to develop a CNN for automatic semantic segmentation of UADT cancer on endoscopic images. Materials and Methods A dataset of white light and NBI videoframes of laryngeal squamous cell carcinoma (LSCC) was collected and manually annotated. A novel DL segmentation model (SegMENT) was designed. SegMENT relies on DeepLabV3+ CNN architecture, modified using Xception as a backbone and incorporating ensemble features from other CNNs. The performance of SegMENT was compared to state-of-the-art CNNs (UNet, ResUNet, and DeepLabv3). SegMENT was then validated on two external datasets of NBI images of oropharyngeal (OPSCC) and oral cavity SCC (OSCC) obtained from a previously published study. The impact of in-domain transfer learning through an ensemble technique was evaluated on the external datasets. Results 219 LSCC patients were retrospectively included in the study. A total of 683 videoframes composed the LSCC dataset, while the external validation cohorts of OPSCC and OCSCC contained 116 and 102 images. On the LSCC dataset, SegMENT outperformed the other DL models, obtaining the following median values: 0.68 intersection over union (IoU), 0.81 dice similarity coefficient (DSC), 0.95 recall, 0.78 precision, 0.97 accuracy. For the OCSCC and OPSCC datasets, results were superior compared to previously published data: the median performance metrics were, respectively, improved as follows: DSC=10.3% and 11.9%, recall=15.0% and 5.1%, precision=17.0% and 14.7%, accuracy=4.1% and 10.3%. Conclusion SegMENT achieved promising performances, showing that automatic tumor segmentation in endoscopic images is feasible even within the highly heterogeneous and complex UADT environment. SegMENT outperformed the previously published results on the external validation cohorts. The model demonstrated potential for improved detection of early tumors, more precise biopsies, and better selection of resection margins.
Collapse
Affiliation(s)
- Muhammad Adeel Azam
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Claudio Sampieri
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Alessandro Ioppi
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Pietro Benzi
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Giorgio Gregory Giordano
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Marta De Vecchi
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Valentina Campagnari
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Shunlei Li
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Luca Guastini
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Alberto Paderno
- Unit of Otorhinolaryngology - Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy.,Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, Brescia, Italy
| | - Sara Moccia
- The BioRobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Cesare Piazza
- Unit of Otorhinolaryngology - Head and Neck Surgery, ASST Spedali Civili of Brescia, Brescia, Italy.,Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, University of Brescia, Brescia, Italy
| | - Leonardo S Mattos
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Giorgio Peretti
- Unit of Otorhinolaryngology - Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.,Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| |
Collapse
|
21
|
Saidak Z, Piazza C. Editorial: Oral Oncology: From Precise Surgery to Precision Medicine and Surgery. FRONTIERS IN ORAL HEALTH 2022; 3:913172. [PMID: 35571978 PMCID: PMC9096239 DOI: 10.3389/froh.2022.913172] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2022] [Accepted: 04/12/2022] [Indexed: 11/20/2022] Open
Affiliation(s)
- Zuzana Saidak
- UR7516 CHIMERE, Université de Picardie Jules Verne, Amiens, France
- Centre de Biologie Humaine, Centre Hospitalier Universitaire (CHU) Amiens, Amiens, France
- *Correspondence: Zuzana Saidak
| | - Cesare Piazza
- Department of Otorhinolaryngology, Head and Neck Surgery, University of Brescia, Brescia, Italy
- Cesare Piazza
| |
Collapse
|
22
|
Is the exoscope ready to replace the operative microscope in transoral surgery? Curr Opin Otolaryngol Head Neck Surg 2022; 30:79-86. [PMID: 35131988 DOI: 10.1097/moo.0000000000000794] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
PURPOSE OF REVIEW Exoscopes are external digital devices that provide enhanced and magnified visualization of the surgical field. They usually have dedicated digital controls and a more compact mechanical structure than operative microscopes and current robotic surgical systems. This technology has significant potential in otolaryngology - head and neck surgery, especially concerning the field of transoral approaches. We herein analysed the overall technical characteristics of currently available exoscopic systems and contextualized their advantages and drawbacks in the setting of transoral surgery. RECENT FINDINGS The actual advantages of exoscopy are still indeterminate, as it has only been applied to limited surgical series. However, its specific properties are herein compared with conventional transoral microsurgery and transoral robotic surgery, discussing the available literature on such a topic, filtered on the basis of the authors' experience and its possible future evolutions. Finally, a summary of current experiences in the field of three-dimensional (3D) transoral exoscopic surgery is presented, highlighting differences compared with standard approaches. SUMMARY 3D-exoscopic transoral surgery will possibly play an essential role in future management of early laryngeal and oropharyngeal lesions, significantly shifting the paradigms of this type of procedures.
Collapse
|
23
|
Paderno A, Bossi P, Piazza C. Editorial: Advances in the Multidisciplinary Management of Oral Cancer. Front Oncol 2021; 11:817756. [PMID: 34976848 PMCID: PMC8714657 DOI: 10.3389/fonc.2021.817756] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Accepted: 11/29/2021] [Indexed: 12/24/2022] Open
Affiliation(s)
- Alberto Paderno
- Unit of Otorhinolaryngology – Head and Neck Surgery, Azienda Socio Sanitaria Territoriale (ASST) Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
- *Correspondence: Alberto Paderno,
| | - Paolo Bossi
- Medical Oncology, Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health University of Brescia, Azienda Socio Sanitaria Territoriale (ASST) Spedali Civili, Brescia, Italy
| | - Cesare Piazza
- Unit of Otorhinolaryngology – Head and Neck Surgery, Azienda Socio Sanitaria Territoriale (ASST) Spedali Civili of Brescia, Brescia, Italy
- Department of Medical and Surgical Specialties, Radiological Sciences, and Public Health, School of Medicine, University of Brescia, Brescia, Italy
| |
Collapse
|
24
|
Mattos LS, Acemoglu A, Geraldes A, Laborai A, Schoob A, Tamadazte B, Davies B, Wacogne B, Pieralli C, Barbalata C, Caldwell DG, Kundrat D, Pardo D, Grant E, Mora F, Barresi G, Peretti G, Ortiz J, Rabenorosoa K, Tavernier L, Pazart L, Fichera L, Guastini L, Kahrs LA, Rakotondrabe M, Andreff N, Deshpande N, Gaiffe O, Renevier R, Moccia S, Lescano S, Ortmaier T, Penza V. μRALP and Beyond: Micro-Technologies and Systems for Robot-Assisted Endoscopic Laser Microsurgery. Front Robot AI 2021; 8:664655. [PMID: 34568434 PMCID: PMC8455830 DOI: 10.3389/frobt.2021.664655] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2021] [Accepted: 07/14/2021] [Indexed: 01/05/2023] Open
Abstract
Laser microsurgery is the current gold standard surgical technique for the treatment of selected diseases in delicate organs such as the larynx. However, the operations require large surgical expertise and dexterity, and face significant limitations imposed by available technology, such as the requirement for direct line of sight to the surgical field, restricted access, and direct manual control of the surgical instruments. To change this status quo, the European project μRALP pioneered research towards a complete redesign of current laser microsurgery systems, focusing on the development of robotic micro-technologies to enable endoscopic operations. This has fostered awareness and interest in this field, which presents a unique set of needs, requirements and constraints, leading to research and technological developments beyond μRALP and its research consortium. This paper reviews the achievements and key contributions of such research, providing an overview of the current state of the art in robot-assisted endoscopic laser microsurgery. The primary target application considered is phonomicrosurgery, which is a representative use case involving highly challenging microsurgical techniques for the treatment of glottic diseases. The paper starts by presenting the motivations and rationale for endoscopic laser microsurgery, which leads to the introduction of robotics as an enabling technology for improved surgical field accessibility, visualization and management. Then, research goals, achievements, and current state of different technologies that can build-up to an effective robotic system for endoscopic laser microsurgery are presented. This includes research in micro-robotic laser steering, flexible robotic endoscopes, augmented imaging, assistive surgeon-robot interfaces, and cognitive surgical systems. Innovations in each of these areas are shown to provide sizable progress towards more precise, safer and higher quality endoscopic laser microsurgeries. Yet, major impact is really expected from the full integration of such individual contributions into a complete clinical surgical robotic system, as illustrated in the end of this paper with a description of preliminary cadaver trials conducted with the integrated μRALP system. Overall, the contribution of this paper lays in outlining the current state of the art and open challenges in the area of robot-assisted endoscopic laser microsurgery, which has important clinical applications even beyond laryngology.
Collapse
Affiliation(s)
| | | | | | - Andrea Laborai
- Department of Otorhinolaryngology, Guglielmo da Saliceto Hospital, Piacenza, Italy
| | | | - Brahim Tamadazte
- Institut des Systèmes Intelligents et de Robotique, Sorbonne Université, CNRS, Paris, France
| | | | - Bruno Wacogne
- FEMTO-ST Institute, Univ. Bourgogne Franche-Comte, CNRS, Besançon, France.,Centre Hospitalier Régional Universitaire, Besançon, France
| | - Christian Pieralli
- FEMTO-ST Institute, Univ. Bourgogne Franche-Comte, CNRS, Besançon, France
| | - Corina Barbalata
- Mechanical and Industrial Engineering Department, Louisiana State University, Baton Rouge, LA, United States
| | | | | | - Diego Pardo
- Istituto Italiano di Tecnologia, Genoa, Italy
| | - Edward Grant
- Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, NC, United States
| | - Francesco Mora
- Clinica Otorinolaringoiatrica, IRCCS Policlinico San Martino, Genoa, Italy.,Dipartimento di Scienze Chirurgiche e Diagnostiche Integrate, Università Degli Studi di Genova, Genoa, Italy
| | | | - Giorgio Peretti
- Clinica Otorinolaringoiatrica, IRCCS Policlinico San Martino, Genoa, Italy.,Dipartimento di Scienze Chirurgiche e Diagnostiche Integrate, Università Degli Studi di Genova, Genoa, Italy
| | - Jesùs Ortiz
- Istituto Italiano di Tecnologia, Genoa, Italy
| | - Kanty Rabenorosoa
- FEMTO-ST Institute, Univ. Bourgogne Franche-Comte, CNRS, Besançon, France
| | | | - Lionel Pazart
- Centre Hospitalier Régional Universitaire, Besançon, France
| | - Loris Fichera
- Department of Robotics Engineering, Worcester Polytechnic Institute, Worcester, MA, United States
| | - Luca Guastini
- Clinica Otorinolaringoiatrica, IRCCS Policlinico San Martino, Genoa, Italy.,Dipartimento di Scienze Chirurgiche e Diagnostiche Integrate, Università Degli Studi di Genova, Genoa, Italy
| | - Lüder A Kahrs
- Department of Mathematical and Computational Sciences, University of Toronto, Mississauga, ON, Canada
| | - Micky Rakotondrabe
- National School of Engineering in Tarbes, University of Toulouse, Tarbes, France
| | - Nicolas Andreff
- FEMTO-ST Institute, Univ. Bourgogne Franche-Comte, CNRS, Besançon, France
| | | | - Olivier Gaiffe
- Centre Hospitalier Régional Universitaire, Besançon, France
| | - Rupert Renevier
- FEMTO-ST Institute, Univ. Bourgogne Franche-Comte, CNRS, Besançon, France
| | - Sara Moccia
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Sergio Lescano
- FEMTO-ST Institute, Univ. Bourgogne Franche-Comte, CNRS, Besançon, France
| | - Tobias Ortmaier
- Institute of Mechatronic Systems, Leibniz Universität Hannover, Garbsen, Germany
| | | |
Collapse
|
25
|
Yao P, Usman M, Chen YH, German A, Andreadis K, Mages K, Rameau A. Applications of Artificial Intelligence to Office Laryngoscopy: A Scoping Review. Laryngoscope 2021; 132:1993-2016. [PMID: 34582043 DOI: 10.1002/lary.29886] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2021] [Revised: 09/15/2021] [Accepted: 09/17/2021] [Indexed: 01/16/2023]
Abstract
OBJECTIVES/HYPOTHESIS This scoping review aims to provide a broad overview of the applications of artificial intelligence (AI) to office laryngoscopy to identify gaps in knowledge and guide future research. STUDY DESIGN Scoping Review. METHODS Searches for studies on AI and office laryngoscopy were conducted in five databases. Title and abstract and then full-text screening were performed. Primary research studies published in English of any date were included. Studies were summarized by: AI applications, targeted conditions, imaging modalities, author affiliations, and dataset characteristics. RESULTS Studies focused on vocal fold vibration analysis (43%), lesion recognition (24%), and vocal fold movement determination (19%). The most frequently automated tasks were recognition of vocal fold nodules (19%), polyp (14%), paralysis (11%), paresis (8%), and cyst (7%). Imaging modalities included high-speed laryngeal videos (45%), stroboscopy (29%), and narrow band imaging endoscopy (7%). The body of literature was primarily authored by science, technology, engineering, and math (STEM) specialists (76%) with only 30 studies (31%) involving co-authorship by STEM specialists and otolaryngologists. Datasets were mostly from single institution (84%) and most commonly originated from Germany (23%), USA (16%), Spain (9%), Italy (8%), and China (8%). Demographic information was only reported in 39 studies (40%), with age and sex being the most commonly reported, whereas race/ethnicity and gender were not reported in any studies. CONCLUSION More interdisciplinary collaboration between STEM and otolaryngology research teams improved demographic reporting especially of race and ethnicity to ensure broad representation, and larger and more geographically diverse datasets will be crucial to future research on AI in office laryngoscopy. LEVEL OF EVIDENCE N/A Laryngoscope, 2021.
Collapse
Affiliation(s)
- Peter Yao
- Department of Otolaryngology-Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medicine, New York, New York, U.S.A
| | - Moon Usman
- Department of Otolaryngology-Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medicine, New York, New York, U.S.A
| | - Yu H Chen
- Department of Otolaryngology-Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medicine, New York, New York, U.S.A
| | - Alexander German
- Department of Otolaryngology-Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medicine, New York, New York, U.S.A
| | - Katerina Andreadis
- Department of Otolaryngology-Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medicine, New York, New York, U.S.A
| | - Keith Mages
- Department of Otolaryngology-Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medicine, New York, New York, U.S.A
| | - Anaïs Rameau
- Department of Otolaryngology-Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medicine, New York, New York, U.S.A
| |
Collapse
|
26
|
Using deep learning to identify the recurrent laryngeal nerve during thyroidectomy. Sci Rep 2021; 11:14306. [PMID: 34253767 PMCID: PMC8275665 DOI: 10.1038/s41598-021-93202-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2021] [Accepted: 06/22/2021] [Indexed: 11/16/2022] Open
Abstract
Surgeons must visually distinguish soft-tissues, such as nerves, from surrounding anatomy to prevent complications and optimize patient outcomes. An accurate nerve segmentation and analysis tool could provide useful insight for surgical decision-making. Here, we present an end-to-end, automatic deep learning computer vision algorithm to segment and measure nerves. Unlike traditional medical imaging, our unconstrained setup with accessible handheld digital cameras, along with the unstructured open surgery scene, makes this task uniquely challenging. We investigate one common procedure, thyroidectomy, during which surgeons must avoid damaging the recurrent laryngeal nerve (RLN), which is responsible for human speech. We evaluate our segmentation algorithm on a diverse dataset across varied and challenging settings of operating room image capture, and show strong segmentation performance in the optimal image capture condition. This work lays the foundation for future research in real-time tissue discrimination and integration of accessible, intelligent tools into open surgery to provide actionable insights.
Collapse
|
27
|
Paderno A, Piazza C, Del Bon F, Lancini D, Tanagli S, Deganello A, Peretti G, De Momi E, Patrini I, Ruperti M, Mattos LS, Moccia S. Deep Learning for Automatic Segmentation of Oral and Oropharyngeal Cancer Using Narrow Band Imaging: Preliminary Experience in a Clinical Perspective. Front Oncol 2021; 11:626602. [PMID: 33842330 PMCID: PMC8024583 DOI: 10.3389/fonc.2021.626602] [Citation(s) in RCA: 27] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2020] [Accepted: 03/08/2021] [Indexed: 01/22/2023] Open
Abstract
Introduction Fully convoluted neural networks (FCNN) applied to video-analysis are of particular interest in the field of head and neck oncology, given that endoscopic examination is a crucial step in diagnosis, staging, and follow-up of patients affected by upper aero-digestive tract cancers. The aim of this study was to test FCNN-based methods for semantic segmentation of squamous cell carcinoma (SCC) of the oral cavity (OC) and oropharynx (OP). Materials and Methods Two datasets were retrieved from the institutional registry of a tertiary academic hospital analyzing 34 and 45 NBI endoscopic videos of OC and OP lesions, respectively. The dataset referring to the OC was composed of 110 frames, while 116 frames composed the OP dataset. Three FCNNs (U-Net, U-Net 3, and ResNet) were investigated to segment the neoplastic images. FCNNs performance was evaluated for each tested network and compared to the gold standard, represented by the manual annotation performed by expert clinicians. Results For FCNN-based segmentation of the OC dataset, the best results in terms of Dice Similarity Coefficient (Dsc) were achieved by ResNet with 5(×2) blocks and 16 filters, with a median value of 0.6559. In FCNN-based segmentation for the OP dataset, the best results in terms of Dsc were achieved by ResNet with 4(×2) blocks and 16 filters, with a median value of 0.7603. All tested FCNNs presented very high values of variance, leading to very low values of minima for all metrics evaluated. Conclusions FCNNs have promising potential in the analysis and segmentation of OC and OP video-endoscopic images. All tested FCNN architectures demonstrated satisfying outcomes in terms of diagnostic accuracy. The inference time of the processing networks were particularly short, ranging between 14 and 115 ms, thus showing the possibility for real-time application.
Collapse
Affiliation(s)
- Alberto Paderno
- Department of Otorhinolaryngology-Head and Neck Surgery, ASST-Spedali Civili of Brescia, University of Brescia, Brescia, Italy
| | - Cesare Piazza
- Department of Otorhinolaryngology-Head and Neck Surgery, ASST-Spedali Civili of Brescia, University of Brescia, Brescia, Italy
| | - Francesca Del Bon
- Department of Otorhinolaryngology-Head and Neck Surgery, ASST-Spedali Civili of Brescia, University of Brescia, Brescia, Italy
| | - Davide Lancini
- Department of Otorhinolaryngology-Head and Neck Surgery, ASST-Spedali Civili of Brescia, University of Brescia, Brescia, Italy
| | - Stefano Tanagli
- Department of Otorhinolaryngology-Head and Neck Surgery, ASST-Spedali Civili of Brescia, University of Brescia, Brescia, Italy
| | - Alberto Deganello
- Department of Otorhinolaryngology-Head and Neck Surgery, ASST-Spedali Civili of Brescia, University of Brescia, Brescia, Italy
| | - Giorgio Peretti
- Department of Otorhinolaryngology-Head and Neck Surgery, IRCCS San Martino Hospital, University of Genoa, Genoa, Italy
| | - Elena De Momi
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy
| | - Ilaria Patrini
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy
| | - Michela Ruperti
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy
| | - Leonardo S Mattos
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Sara Moccia
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy.,The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy.,Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| |
Collapse
|