Automated classification platform for the identification of otitis media using optical coherence tomography

Multi-modal deep learning for joint prediction of otitis media and diagnostic difficulty

Objectives

In this study, we propose a diagnostic model for automatic detection of otitis media based on combined input of otoscopy images and wideband tympanometry measurements.

Methods

We present a neural network-based model for the joint prediction of otitis media and diagnostic difficulty. We use the subclassifications acute otitis media and otitis media with effusion. The proposed approach is based on deep metric learning, and we compare this with the performance of a standard multi-task network.

Results

The proposed deep metric approach shows good performance on both tasks, and we show that the multi-modal input increases the performance for both classification and difficulty estimation compared to the models trained on the modalities separately. An accuracy of 86.5% is achieved for the classification task, and a Kendall rank correlation coefficient of 0.45 is achieved for difficulty estimation, corresponding to a correct ranking of 72.6% of the cases.

Conclusion

This study demonstrates the strengths of a multi-modal diagnostic tool using both otoscopy images and wideband tympanometry measurements for the diagnosis of otitis media. Furthermore, we show that deep metric learning improves the performance of the models.

Label-Free Optical Technologies for Middle-Ear Diseases

Medical applications of optical technology have increased tremendously in recent decades. Label-free techniques have the unique advantage of investigating biological samples in vivo without introducing exogenous agents. This is especially beneficial for a rapid clinical translation as it reduces the need for toxicity studies and regulatory approval for exogenous labels. Emerging applications have utilized label-free optical technology for screening, diagnosis, and surgical guidance. Advancements in detection technology and rapid improvements in artificial intelligence have expedited the clinical implementation of some optical technologies. Among numerous biomedical application areas, middle-ear disease is a unique space where label-free technology has great potential. The middle ear has a unique anatomical location that can be accessed through a dark channel, the external auditory canal; it can be sampled through a tympanic membrane of approximately 100 microns in thickness. The tympanic membrane is the only membrane in the body that is surrounded by air on both sides, under normal conditions. Despite these favorable characteristics, current examination modalities for middle-ear space utilize century-old technology such as white-light otoscopy. This paper reviews existing label-free imaging technologies and their current progress in visualizing middle-ear diseases. We discuss potential opportunities, barriers, and practical considerations when transitioning label-free technology to clinical applications.

Texture-based speciation of otitis media-related bacterial biofilms from optical coherence tomography images using supervised classification

Otitis media (OM) is primarily a bacterial middle-ear infection prevalent among children worldwide. In recurrent and/or chronic OM cases, antibiotic-resistant bacterial biofilms can develop in the middle ear. A biofilm related to OM typically contains one or multiple bacterial strains, the most common include Haemophilus influenzae, Streptococcus pneumoniae, Moraxella catarrhalis, Pseudomonas aeruginosa, and Staphylococcus aureus. Optical coherence tomography (OCT) has been used clinically to visualize the presence of bacterial biofilms in the middle ear. This study used OCT to compare microstructural image texture features from primary bacterial biofilms in vitro and in vivo. The proposed method applied supervised machine-learning-based frameworks (SVM, random forest (RF), and XGBoost) to classify and speciate multiclass bacterial biofilms from the texture features extracted from OCT B-Scan images obtained from in vitro cultures and from clinically-obtained in vivo images from human subjects. Our findings show that optimized SVM-RBF and XGBoost classifiers can help distinguish bacterial biofilms by incorporating clinical knowledge into classification decisions. Furthermore, both classifiers achieved more than 95% of AUC (area under receiver operating curve), detecting each biofilm class. These results demonstrate the potential for differentiating OM-causing bacterial biofilms through texture analysis of OCT images and a machine-learning framework, which could provide additional clinically relevant data during real-time in vivo characterization of ear infections.

Image-Based Artificial Intelligence Technology for Diagnosing Middle Ear Diseases: A Systematic Review

Otolaryngological diagnoses, such as otitis media, are traditionally performed using endoscopy, wherein diagnostic accuracy can be subjective and vary among clinicians. The integration of objective tools, like artificial intelligence (AI), could potentially improve the diagnostic process by minimizing the influence of subjective biases and variability. We systematically reviewed the AI techniques using medical imaging in otolaryngology. Relevant studies related to AI-assisted otitis media diagnosis were extracted from five databases: Google Scholar, PubMed, Medline, Embase, and IEEE Xplore, without date restrictions. Publications that did not relate to AI and otitis media diagnosis or did not utilize medical imaging were excluded. Of the 32identified studies, 26 used tympanic membrane images for classification, achieving an average diagnosis accuracy of 86% (range: 48.7-99.16%). Another three studies employed both segmentation and classification techniques, reporting an average diagnosis accuracy of 90.8% (range: 88.06-93.9%). These findings suggest that AI technologies hold promise for improving otitis media diagnosis, offering benefits for telemedicine and primary care settings due to their high diagnostic accuracy. However, to ensure patient safety and optimal outcomes, further improvements in diagnostic performance are necessary.

Diagnosis, Treatment, and Management of Otitis Media with Artificial Intelligence

A common infectious disease, otitis media (OM) has a low rate of early diagnosis, which significantly increases the difficulty of treating the disease and the likelihood of serious complications developing including hearing loss, speech impairment, and even intracranial infection. Several areas of healthcare have shown great promise in the application of artificial intelligence (AI) systems, such as the accurate detection of diseases, the automated interpretation of images, and the prediction of patient outcomes. Several articles have reported some machine learning (ML) algorithms such as ResNet, InceptionV3 and Unet, were applied to the diagnosis of OM successfully. The use of these techniques in the OM is still in its infancy, but their potential is enormous. We present in this review important concepts related to ML and AI, describe how these technologies are currently being applied to diagnosing, treating, and managing OM, and discuss the challenges associated with developing AI-assisted OM technologies in the future.

Advances in Artificial Intelligence to Diagnose Otitis Media: State of the Art Review

OBJECTIVE

Otitis media (OM) is a model disease for developing, validating, and implementing artificial intelligence (AI) techniques. We aim to review the state of the art applications of AI used to diagnose OM in pediatric and adult populations.

DATA SOURCES

Several comprehensive databases were searched to identify all articles that applied AI technologies to diagnose OM.

REVIEW METHODS

Relevant articles from January 2010 through May 2021 were identified by title and abstract. Articles were excluded if they did not discuss AI in conjunction with diagnosing OM. References of included studies and relevant review articles were cross-referenced to identify any additional studies.

CONCLUSION

Title and abstract screening resulted in full-text retrieval of 40 articles that met initial screening parameters. Of this total, secondary review articles (n = 7) and commentary-based articles (n = 2) were removed, as were articles that did not specifically discuss AI and OM diagnosis (n = 5), leaving 25 articles for review. Applications of AI technologies specific to diagnosing OM included machine learning and natural language processing (n = 23) and prototype approaches (n = 2).

IMPLICATIONS FOR PRACTICE

This review emphasizes the utility of AI techniques to automate and aid in diagnosing OM. Although these techniques are still in the development and testing stages, AI has the potential to improve the practice of otolaryngologists and primary care clinicians by increasing the efficiency and accuracy of diagnoses.

Performing tympanometry using smartphones

Background

Tympanometry is used as part of a battery of tests for screening of middle ear function and may help diagnose middle ear disorders, but remains available only on expensive test equipment.

Methods

We report a low-cost smartphone-based tympanometer system that consists of a lightweight and portable attachment to vary air pressure in the ear and measure middle ear function. The smartphone displays a tympanogram and reports peak acoustic admittance in realtime. Our programmable and open-source system operates at 226 Hz and was tested on 50 pediatric patient ears in an audiology clinic in parallel with a commercial tympanometer.

Results

Our study shows an average agreement of 86 ± 2% between the 100 tympanograms produced by the smartphone and commercial device when five pediatric audiologists classified them into five classes based on the Liden and Jerger classification.

Conclusion

Given the accessibility and prevalence of budget smartphones in developing countries, our open-source tool may help provide timely and affordable screening of middle ear disorders.

Tympanometry is a test used to evaluate the health of the middle ear, which is involved in hearing, and helps in diagnosing middle ear disorders. However, the test currently requires expensive equipment and is limited in availability in resource-constrained settings. We present a low-cost smartphone-based tympanometry system. The device is able to change the air pressure of the ear canal and measure ear drum mobility. Our system, for which the software and hardware are openly available, is tested in 50 ears from children attending an audiology clinic. A panel of pediatric audiologists classified tympanometry measurements from our device and a commercial tympanometer with good agreement. Given the increasing availability of smartphones in developing countries, our system has the potential to make screening of middle ear disorders more accessible.

Chan et al. develop an open-source smartphone-based tympanometer to evaluate middle ear function. The authors test their device in a group of paediatric patients at an audiology clinic and report a high level of agreement in audiologists’ classification of tympanograms produced using this device compared with a commercial device.

"Human vs Machine" Validation of a Deep Learning Algorithm for Pediatric Middle Ear Infection Diagnosis

OBJECTIVE

We compared the diagnostic performance of human clinicians with that of a neural network algorithm developed using a library of tympanic membrane images derived from children taken to the operating room with the intent of performing myringotomy and possible tube placement for recurrent acute otitis media (AOM) or otitis media with effusion (OME).

STUDY DESIGN

Retrospective cohort study.

SETTING

Tertiary academic medical center from 2018 to 2021.

METHODS

A training set of 639 images of tympanic membranes representing normal, OME, and AOM was used to train a neural network as well as a proprietary commercial image classifier from Google. Model diagnostic prediction performance in differentiating normal vs nonpurulent vs purulent effusion was scored based on classification accuracy. A web-based survey was developed to test human clinicians' diagnostic accuracy on a novel image set, and this was compared head to head against our model.

RESULTS

Our model achieved a mean prediction accuracy of 80.8% (95% CI, 77.0%-84.6%). The Google model achieved a prediction accuracy of 85.4%. In a validation survey of 39 clinicians analyzing a sample of 22 endoscopic ear images, the average diagnostic accuracy was 65.0%. On the same data set, our model achieved an accuracy of 95.5%.

CONCLUSION

Our model outperformed certain groups of human clinicians in assessing images of tympanic membranes for effusions in children. Reduced diagnostic error rates using machine learning models may have implications in reducing rates of misdiagnosis, potentially leading to fewer missed diagnoses, unnecessary antibiotic prescriptions, and surgical procedures.

Automated classification of otitis media with OCT: augmenting pediatric image datasets with gold-standard animal model data

Otitis media (OM) is an extremely common disease that affects children worldwide. Optical coherence tomography (OCT) has emerged as a noninvasive diagnostic tool for OM, which can detect the presence and quantify the properties of middle ear fluid and biofilms. Here, the use of OCT data from the chinchilla, the gold-standard OM model for the human disease, is used to supplement a human image database to produce diagnostically relevant conclusions in a machine learning model. Statistical analysis shows the datatypes are compatible, with a blended-species model reaching ∼95% accuracy and F1 score, maintaining performance while additional human data is collected.

A Machine Learning Approach to Screen for Otitis Media Using Digital Otoscope Images Labelled by an Expert Panel

Background: Otitis media includes several common inflammatory conditions of the middle ear that can have severe complications if left untreated. Correctly identifying otitis media can be difficult and a screening system supported by machine learning would be valuable for this prevalent disease. This study investigated the performance of a convolutional neural network in screening for otitis media using digital otoscopic images labelled by an expert panel. Methods: Five experienced otologists diagnosed 347 tympanic membrane images captured with a digital otoscope. Images with a majority expert diagnosis (n = 273) were categorized into three screening groups Normal, Pathological and Wax, and the same images were used for training and testing of the convolutional neural network. Expert panel diagnoses were compared to the convolutional neural network classification. Different approaches to the convolutional neural network were tested to identify the best performing model. Results: Overall accuracy of the convolutional neural network was above 0.9 in all except one approach. Sensitivity to finding ears with wax or pathology was above 93% in all cases and specificity was 100%. Adding more images to train the convolutional neural network had no positive impact on the results. Modifications such as normalization of datasets and image augmentation enhanced the performance in some instances. Conclusions: A machine learning approach could be used on digital otoscopic images to accurately screen for otitis media.

Multimodal Handheld Probe for Characterizing Otitis Media - Integrating Raman Spectroscopy and Optical Coherence Tomography

Otitis media (OM) is a common disease of the middle ear, affecting 80% of children before the age of three. The otoscope, a simple illuminated magnifier, is the standard clinical diagnostic tool to observe the middle ear. However, it has limited contrast to detect signs of infection, such as clearly identifying and characterizing middle ear fluid or biofilms that accumulate within the middle ear. Likewise, invasive sampling of every subject is not clinically indicated nor practical. Thus, collecting accurate noninvasive diagnostic factors is vital for clinicians to deliver a precise diagnosis and effective treatment regimen. To address this need, a combined benchtop Raman spectroscopy (RS) and optical coherence tomography (OCT) system was developed. Together, RS-OCT can non-invasively interrogate the structural and biochemical signatures of the middle ear under normal and infected conditions.In this paper, in vivo RS scans from pediatric clinical human subjects presenting with OM were evaluated in parallel with RS-OCT data of physiologically relevant in vitro ear models. Component-level characterization of a healthy tympanic membrane and malleus bone, as well as OM-related middle ear fluid, identified the optimal position within the ear for RS-OCT data collection. To address the design challenges in developing a system specific to clinical use, a prototype non-contact multimodal handheld probe was built and successfully tested in vitro. Design criteria have been developed to successfully address imaging constraints imposed by physiological characteristics of the ear and optical safety limits. Here, we present the pathway for translation of RS-OCT for non-invasive detection of OM.

Shortwave infrared otoscopy for diagnosis of middle ear effusions: a machine-learning-based approach

Otitis media, a common disease marked by the presence of fluid within the middle ear space, imparts a significant global health and economic burden. Identifying an effusion through the tympanic membrane is critical to diagnostic success but remains challenging due to the inherent limitations of visible light otoscopy and user interpretation. Here we describe a powerful diagnostic approach to otitis media utilizing advancements in otoscopy and machine learning. We developed an otoscope that visualizes middle ear structures and fluid in the shortwave infrared region, holding several advantages over traditional approaches. Images were captured in vivo and then processed by a novel machine learning based algorithm. The model predicts the presence of effusions with greater accuracy than current techniques, offering specificity and sensitivity over 90%. This platform has the potential to reduce costs and resources associated with otitis media, especially as improvements are made in shortwave imaging and machine learning.

Handheld Briefcase Optical Coherence Tomography with Real-Time Machine Learning Classifier for Middle Ear Infections

A middle ear infection is a prevalent inflammatory disease most common in the pediatric population, and its financial burden remains substantial. Current diagnostic methods are highly subjective, relying on visual cues gathered by an otoscope. To address this shortcoming, optical coherence tomography (OCT) has been integrated into a handheld imaging probe. This system can non-invasively and quantitatively assess middle ear effusions and identify the presence of bacterial biofilms in the middle ear cavity during ear infections. Furthermore, the complete OCT system is housed in a standard briefcase to maximize its portability as a diagnostic device. Nonetheless, interpreting OCT images of the middle ear more often requires expertise in OCT as well as middle ear infections, making it difficult for an untrained user to operate the system as an accurate stand-alone diagnostic tool in clinical settings. Here, we present a briefcase OCT system implemented with a real-time machine learning platform for middle ear infections. A random forest-based classifier can categorize images based on the presence of middle ear effusions and biofilms. This study demonstrates that our briefcase OCT system coupled with machine learning can provide user-invariant classification results of middle ear conditions, which may greatly improve the utility of this technology for the diagnosis and management of middle ear infections.

Machine Learning for Accurate Intraoperative Pediatric Middle Ear Effusion Diagnosis

OBJECTIVES

Misdiagnosis of acute and chronic otitis media in children can result in significant consequences from either undertreatment or overtreatment. Our objective was to develop and train an artificial intelligence algorithm to accurately predict the presence of middle ear effusion in pediatric patients presenting to the operating room for myringotomy and tube placement.

METHODS

We trained a neural network to classify images as " normal" (no effusion) or "abnormal" (effusion present) using tympanic membrane images from children taken to the operating room with the intent of performing myringotomy and possible tube placement for recurrent acute otitis media or otitis media with effusion. Model performance was tested on held-out cases and fivefold cross-validation.

RESULTS

The mean training time for the neural network model was 76.0 (SD ± 0.01) seconds. Our model approach achieved a mean image classification accuracy of 83.8% (95% confidence interval [CI]: 82.7-84.8). In support of this classification accuracy, the model produced an area under the receiver operating characteristic curve performance of 0.93 (95% CI: 0.91-0.94) and F1-score of 0.80 (95% CI: 0.77-0.82).

CONCLUSIONS

Artificial intelligence-assisted diagnosis of acute or chronic otitis media in children may generate value for patients, families, and the health care system by improving point-of-care diagnostic accuracy. With a small training data set composed of intraoperative images obtained at time of tympanostomy tube insertion, our neural network was accurate in predicting the presence of a middle ear effusion in pediatric ear cases. This diagnostic accuracy performance is considerably higher than human-expert otoscopy-based diagnostic performance reported in previous studies.

In vivo dynamic characterization of the human tympanic membrane using pneumatic optical coherence tomography

Decreased mobility of the human eardrum, the tympanic membrane (TM), is an essential indicator of a prevalent middle ear infection. The current diagnostic method to assess TM mobility is via pneumatic otoscopy, which provides subjective and qualitative information of subtle motion. In this study, a handheld spectral-domain pneumatic optical coherence tomography system was developed to simultaneously measure the displacement of the TM, air pressure inputs applied to a sealed ear canal, and to perform digital pneumatic otoscopy. A novel approach based on quantitative parameters is presented to characterize spatial and temporal variations of the dynamic TM motion. Furthermore, the TM motions of normal middle ears are compared with those of ears with middle ear infections. The capability of noninvasively measuring the rapid motion of the TM is beneficial to understand the complex dynamics of the human TM, and can ultimately lead to improved diagnosis and management of middle ear infections.

Longitudinal optical coherence tomography to visualize the in vivo response of middle ear biofilms to antibiotic therapy

Studying the impact of antibiotic treatment on otitis media (OM), the leading cause of primary care office visits during childhood, is critical to develop appropriate treatment strategies. Tracking dynamic middle ear conditions during antibiotic treatment is not readily applicable in patients, due to the limited diagnostic techniques available to detect the smaller amount and variation of middle ear effusion (MEE) and middle ear bacterial biofilm, responsible for chronic and recurrent OM. To overcome these challenges, a handheld optical coherence tomography (OCT) system has been developed to monitor in vivo response of biofilms and MEEs in the OM-induced chinchilla model, the standard model for human OM. As a result, the formation of MEE as well as biofilm adherent to the tympanic membrane (TM) was longitudinally assessed as OM developed. Various types of MEEs and biofilms in the chinchilla model were identified, which showed comparable features as those in humans. Furthermore, the effect of antibiotics on the biofilm as well as the amount and type of MEEs was investigated with low-dose and high-dose treatment (ceftriaxone). The capability of OCT to non-invasively track and examine middle ear conditions is highly beneficial for therapeutic OM studies and will lead to improved management of OM in patients.

Advances in Optical Detection of Human-Associated Pathogenic Bacteria

Bacterial infection is a global burden that results in numerous hospital visits and deaths annually. The rise of multi-drug resistant bacteria has dramatically increased this burden. Therefore, there is a clinical need to detect and identify bacteria rapidly and accurately in their native state or a culture-free environment. Current diagnostic techniques lack speed and effectiveness in detecting bacteria that are culture-negative, as well as options for in vivo detection. The optical detection of bacteria offers the potential to overcome these obstacles by providing various platforms that can detect bacteria rapidly, with minimum sample preparation, and, in some cases, culture-free directly from patient fluids or even in vivo. These modalities include infrared, Raman, and fluorescence spectroscopy, along with optical coherence tomography, interference, polarization, and laser speckle. However, these techniques are not without their own set of limitations. This review summarizes the strengths and weaknesses of utilizing each of these optical tools for rapid bacteria detection and identification.

Optical coherence tomography: current and future clinical applications in otology

PURPOSE OF REVIEW

This article reviews literature on the use of optical coherence tomography (OCT) in otology and provides the reader with a timely update on its current clinical and research applications. The discussion focuses on the principles of OCT, the use of the technology for the diagnosis of middle ear disease and for the delineation of in-vivo cochlear microarchitecture and function.

RECENT FINDINGS

Recent advances in OCT include the measurement of structural and vibratory properties of the tympanic membrane, ossicles and inner ear in healthy and diseased states. Accurate, noninvasive diagnosis of middle ear disease, such as otosclerosis and acute otitis media using OCT, has been validated in clinical studies, whereas inner ear OCT imaging remains at the preclinical stage. The development of recent microscopic, otoscopic and endoscopic systems to address clinical and research problems is reviewed.

SUMMARY

OCT is a real-time, noninvasive, nonionizing, point-of-care imaging modality capable of imaging ear structures in vivo. Although current clinical systems are mainly focused on middle ear imaging, OCT has also been shown to have the ability to identify inner ear disease, an exciting possibility that will become increasingly relevant with the advent of targeted inner ear therapies.

A novel 3D printing PCL/GelMA scaffold containing USPIO for MRI-guided bile duct repair

Making artificial bile ducts in vitro for repairing and replacing diseased bile ducts is an important concept in tissue engineering. This study printed a tubular composite scaffold using polycaprolactone (PCL) through the current 3D printing method. It served as a matrix for the organoid cells of the bile duct to proliferation, migration, and differentiation. The PCL scaffold full of bile duct-like organ cells can achieve the effect of bionics, replacing the original bile duct to perform its proper function. In order to enrich the performance of the tubular scaffold, hydrogels were also used in this study. Applying a layer of gelatin methacryloyl (GelMA) hydrogel with an appropriate thickness on the outer layer of the PCL scaffold not only protects and supports the scaffold, but also improves the biocompatibility of the printed bile duct. In addition, ultrasmall superparamagnetic iron oxide (USPIO) nanoparticles dispersed in GelMA served as the contrast agent to monitor the repair of the lesion site and the degradation of the bile duct in real time by magnetic resonance imaging (MRI). In this study, a tubular composite scaffold that could reconstruct bile duct function and possess a real-time MRI imaging property was constructed by 3D printing. After 13 days of the co-culture of bone marrow derived stem cells (BMSCs), the survival rate of the BMSCs was greater than 95%, and the coverage of the BMSCs was as high as 90%. At the same time, the compression modulus of the stent could reach 17.41 kPa and the Young's modulus could reach 5.03 kPa. Thus, the mechanical properties of it can meet the needs of human implantation. USPIO can achieve MRI imaging in situ and nondestructively monitor the degradation of the stent in the body. In summary, PCL/GelMA/USPIO bile duct scaffolds are beneficial to the proliferation of cells on the scaffolds and can be used to construct biologically active artificial bile ducts.