A Web-Based Deep Learning Model for Automated Diagnosis of Otoscopic Images

Deep Learning Techniques and Imaging in Otorhinolaryngology-A State-of-the-Art Review

Over the last decades, the field of medicine has witnessed significant progress in artificial intelligence (AI), the Internet of Medical Things (IoMT), and deep learning (DL) systems. Otorhinolaryngology, and imaging in its various subspecialties, has not remained untouched by this transformative trend. As the medical landscape evolves, the integration of these technologies becomes imperative in augmenting patient care, fostering innovation, and actively participating in the ever-evolving synergy between computer vision techniques in otorhinolaryngology and AI. To that end, we conducted a thorough search on MEDLINE for papers published until June 2023, utilizing the keywords 'otorhinolaryngology', 'imaging', 'computer vision', 'artificial intelligence', and 'deep learning', and at the same time conducted manual searching in the references section of the articles included in our manuscript. Our search culminated in the retrieval of 121 related articles, which were subsequently subdivided into the following categories: imaging in head and neck, otology, and rhinology. Our objective is to provide a comprehensive introduction to this burgeoning field, tailored for both experienced specialists and aspiring residents in the domain of deep learning algorithms in imaging techniques in otorhinolaryngology.

ISOM 2023 research Panel 4 - Diagnostics and microbiology of otitis media

OBJECTIVES

To identify and review key research advances from the literature published between 2019 and 2023 on the diagnosis and microbiology of otitis media (OM) including acute otitis media (AOM), recurrent AOM (rAOM), otitis media with effusion (OME), chronic suppurative otitis media (CSOM) and AOM complications (mastoiditis).

DATA SOURCES

PubMed database of the National Library of Medicine.

REVIEW METHODS

All relevant original articles published in Medline in English between July 2019 and February 2023 were identified. Studies that were reviews, case studies, relating to OM complications (other than mastoiditis), and studies focusing on guideline adherence, and consensus statements were excluded. Members of the panel drafted the report based on these search results.

MAIN FINDINGS

For the diagnosis section, 2294 unique records screened, 55 were eligible for inclusion. For the microbiology section 705 unique records were screened and 137 articles were eligible for inclusion. The main themes that arose in OM diagnosis were the need to incorporate multiple modalities including video-otoscopy, tympanometry, telemedicine and artificial intelligence for accurate diagnoses in all diagnostic settings. Further to this, was the use of new, cheap, readily available tools which may improve access in rural and lowmiddle income (LMIC) settings. For OM aetiology, PCR remains the most sensitive method for detecting middle ear pathogens with microbiome analysis still largely restricted to research use. The global pandemic response reduced rates of OM in children, but post-pandemic shifts should be monitored.

IMPLICATION FOR PRACTICE AND FUTURE RESEARCH

Cheap, easy to use multi-technique assessments combined with artificial intelligence and/or telemedicine should be integrated into future practice to improve diagnosis and treatment pathways in OM diagnosis. Longitudinal studies investigating the in-vivo process of OM development, timings and in-depth interactions between the triad of bacteria, viruses and the host immune response are still required. Standardized methods of collection and analysis for microbiome studies to enable inter-study comparisons are required. There is a need to target underlying biofilms if going to effectively prevent rAOM and OME and possibly enhance ventilation tube retention.

Image-Based Artificial Intelligence Technology for Diagnosing Middle Ear Diseases: A Systematic Review

Otolaryngological diagnoses, such as otitis media, are traditionally performed using endoscopy, wherein diagnostic accuracy can be subjective and vary among clinicians. The integration of objective tools, like artificial intelligence (AI), could potentially improve the diagnostic process by minimizing the influence of subjective biases and variability. We systematically reviewed the AI techniques using medical imaging in otolaryngology. Relevant studies related to AI-assisted otitis media diagnosis were extracted from five databases: Google Scholar, PubMed, Medline, Embase, and IEEE Xplore, without date restrictions. Publications that did not relate to AI and otitis media diagnosis or did not utilize medical imaging were excluded. Of the 32identified studies, 26 used tympanic membrane images for classification, achieving an average diagnosis accuracy of 86% (range: 48.7-99.16%). Another three studies employed both segmentation and classification techniques, reporting an average diagnosis accuracy of 90.8% (range: 88.06-93.9%). These findings suggest that AI technologies hold promise for improving otitis media diagnosis, offering benefits for telemedicine and primary care settings due to their high diagnostic accuracy. However, to ensure patient safety and optimal outcomes, further improvements in diagnostic performance are necessary.

Diagnosis, Treatment, and Management of Otitis Media with Artificial Intelligence

A common infectious disease, otitis media (OM) has a low rate of early diagnosis, which significantly increases the difficulty of treating the disease and the likelihood of serious complications developing including hearing loss, speech impairment, and even intracranial infection. Several areas of healthcare have shown great promise in the application of artificial intelligence (AI) systems, such as the accurate detection of diseases, the automated interpretation of images, and the prediction of patient outcomes. Several articles have reported some machine learning (ML) algorithms such as ResNet, InceptionV3 and Unet, were applied to the diagnosis of OM successfully. The use of these techniques in the OM is still in its infancy, but their potential is enormous. We present in this review important concepts related to ML and AI, describe how these technologies are currently being applied to diagnosing, treating, and managing OM, and discuss the challenges associated with developing AI-assisted OM technologies in the future.

Use of artificial intelligence for the diagnosis of cholesteatoma

Objectives

Accurate diagnosis of cholesteatomas is crucial. However, cholesteatomas can easily be missed in routine otoscopic exams. Convolutional neural networks (CNNs) have performed well in medical image classification, so we evaluated their use for detecting cholesteatomas in otoscopic images.

Study Design

Design and evaluation of artificial intelligence driven workflow for cholesteatoma diagnosis.

Methods

Otoscopic images collected from the faculty practice of the senior author were deidentified and labeled by the senior author as cholesteatoma, abnormal non-cholesteatoma, or normal. An image classification workflow was developed to automatically differentiate cholesteatomas from other possible tympanic membrane appearances. Eight pretrained CNNs were trained on our otoscopic images, then tested on a withheld subset of images to evaluate their final performance. CNN intermediate activations were also extracted to visualize important image features.

Results

A total of 834 otoscopic images were collected, further categorized into 197 cholesteatoma, 457 abnormal non-cholesteatoma, and 180 normal. Final trained CNNs demonstrated strong performance, achieving accuracies of 83.8%-98.5% for differentiating cholesteatoma from normal, 75.6%-90.1% for differentiating cholesteatoma from abnormal non-cholesteatoma, and 87.0%-90.4% for differentiating cholesteatoma from non-cholesteatoma (abnormal non-cholesteatoma + normal). DenseNet201 (100% sensitivity, 97.1% specificity), NASNetLarge (100% sensitivity, 88.2% specificity), and MobileNetV2 (94.1% sensitivity, 100% specificity) were among the best performing CNNs in distinguishing cholesteatoma versus normal. Visualization of intermediate activations showed robust detection of relevant image features by the CNNs.

Conclusion

While further refinement and more training images are needed to improve performance, artificial intelligence-driven analysis of otoscopic images shows great promise as a diagnostic tool for detecting cholesteatomas.

Level of Evidence

3.

Front-end deep learning web apps development and deployment: a review

Machine learning and deep learning models are commonly developed using programming languages such as Python, C++, or R and deployed as web apps delivered from a back-end server or as mobile apps installed from an app store. However, recently front-end technologies and JavaScript libraries, such as TensorFlow.js, have been introduced to make machine learning more accessible to researchers and end-users. Using JavaScript, TensorFlow.js can define, train, and run new or existing, pre-trained machine learning models entirely in the browser from the client-side, which improves the user experience through interaction while preserving privacy. Deep learning models deployed on front-end browsers must be small, have fast inference, and ideally be interactive in real-time. Therefore, the emphasis on development and deployment is different. This paper aims to review the development and deployment of these deep-learning web apps to raise awareness of the recent advancements and encourage more researchers to take advantage of this technology for their own work. First, the rationale behind the deployment stack (front-end, JavaScript, and TensorFlow.js) is discussed. Then, the development approach for obtaining deep learning models that are optimized and suitable for front-end deployment is then described. The article also provides current web applications divided into seven categories to show deep learning potential on the front end. These include web apps for deep learning playground, pose detection and gesture tracking, music and art creation, expression detection and facial recognition, video segmentation, image and signal analysis, healthcare diagnosis, recognition, and identification.

Feasibility of the Machine Learning Network to Diagnose Tympanic Membrane Lesions without Coding Experience

A machine learning platform operated without coding knowledge (Teachable machine^®) has been introduced. The aims of the present study were to assess the performance of the Teachable machine^® for diagnosing tympanic membrane lesions. A total of 3024 tympanic membrane images were used to train and validate the diagnostic performance of the network. Tympanic membrane images were labeled as normal, otitis media with effusion (OME), chronic otitis media (COM), and cholesteatoma. According to the complexity of the categorization, Level I refers to normal versus abnormal tympanic membrane; Level II was defined as normal, OME, or COM + cholesteatoma; and Level III distinguishes between all four pathologies. In addition, eighty representative test images were used to assess the performance. Teachable machine^® automatically creates a classification network and presents diagnostic performance when images are uploaded. The mean accuracy of the Teachable machine^® for classifying tympanic membranes as normal or abnormal (Level I) was 90.1%. For Level II, the mean accuracy was 89.0% and for Level III it was 86.2%. The overall accuracy of the classification of the 80 representative tympanic membrane images was 78.75%, and the hit rates for normal, OME, COM, and cholesteatoma were 95.0%, 70.0%, 90.0%, and 60.0%, respectively. Teachable machine^® could successfully generate the diagnostic network for classifying tympanic membrane.

Automated classification of otitis media with OCT: augmenting pediatric image datasets with gold-standard animal model data

Otitis media (OM) is an extremely common disease that affects children worldwide. Optical coherence tomography (OCT) has emerged as a noninvasive diagnostic tool for OM, which can detect the presence and quantify the properties of middle ear fluid and biofilms. Here, the use of OCT data from the chinchilla, the gold-standard OM model for the human disease, is used to supplement a human image database to produce diagnostically relevant conclusions in a machine learning model. Statistical analysis shows the datatypes are compatible, with a blended-species model reaching ∼95% accuracy and F1 score, maintaining performance while additional human data is collected.

Artificial intelligence to classify ear disease from otoscopy: A systematic review and meta-analysis

Objectives

To summarise the accuracy of artificial intelligence (AI) computer vision algorithms to classify ear disease from otoscopy.

Design

Systematic review and meta‐analysis.

Methods

Using the PRISMA guidelines, nine online databases were searched for articles that used AI computer vision algorithms developed from various methods (convolutional neural networks, artificial neural networks, support vector machines, decision trees and k‐nearest neighbours) to classify otoscopic images. Diagnostic classes of interest: normal tympanic membrane, acute otitis media (AOM), otitis media with effusion (OME), chronic otitis media (COM) with or without perforation, cholesteatoma and canal obstruction.

Main outcome measures

Accuracy to correctly classify otoscopic images compared to otolaryngologists (ground truth). The Quality Assessment of Diagnostic Accuracy Studies Version 2 tool was used to assess the quality of methodology and risk of bias.

Results

Thirty‐nine articles were included. Algorithms achieved 90.7% (95%CI: 90.1–91.3%) accuracy to difference between normal or abnormal otoscopy images in 14 studies. The most common multiclassification algorithm (3 or more diagnostic classes) achieved 97.6% (95%CI: 97.3–97.9%) accuracy to differentiate between normal, AOM and OME in three studies. AI algorithms outperformed human assessors to classify otoscopy images achieving 93.4% (95%CI: 90.5–96.4%) versus 73.2% (95%CI: 67.9–78.5%) accuracy in three studies. Convolutional neural networks achieved the highest accuracy compared to other classification methods.

Conclusion

AI can classify ear disease from otoscopy. A concerted effort is required to establish a comprehensive and reliable otoscopy database for algorithm training. An AI‐supported otoscopy system may assist health care workers, trainees and primary care practitioners with less otology experience identify ear disease.