A hybrid feature-based segmentation and classification system for the computer aided self-diagnosis of otitis media

Digital Otoscopy With Computer-Aided Composite Image Generation: Impact on the Correct Diagnosis, Confidence, and Time

OBJECTIVE

This study investigated the comparative performance of ear, nose, and throat (ENT) physicians in correctly detecting ear abnormalities when reviewing digital otoscopy imaging using 3 different visualization methods, including computer-assisted composite images called "SelectStitch," single video frame "Still" images, and video clips. The study also explored clinicians' diagnostic confidence levels and the time to make a diagnosis.

STUDY DESIGN

Clinician diagnostic reader study.

SETTING

Online diagnostic survey of ENT physicians.

METHODS

Nine ENT physicians reviewed digital otoscopy examinations from 86 ears with various diagnoses (normal, perforation, retraction, middle ear effusion, tympanosclerosis). Otoscopy examinations used artificial-intelligence (AI)-based computer-aided composite image generation from a video clip (SelectStitch), manually selected best still frame from a video clip (Still), or the entire video clip. Statistical analyses included comparisons of ability to detect correct diagnosis, confidence levels, and diagnosis times.

RESULTS

The ENT physicians' ability to detect ear abnormalities (33.2%-68.7%) varied depending on the pathologies. SelectStitch and Still images were not statistically different in detecting abnormalities (P > .50), but both were different from Video (P < .01). However, the performance improvement observed with Videos came at the cost of significantly longer time to determining the diagnosis. The level of confidence in the diagnosis was positively associated with correct diagnoses, but varied by particular pathology.

CONCLUSION

This study explores the potential of computer-assisted techniques like SelectStitch in enhancing otoscopic diagnoses and time-saving, which could benefit telemedicine settings. Comparable performance between computer-generated and manually selected images suggests the potential of AI algorithms for otoscopy applications.

Artificial Intelligence in Otology, Rhinology, and Laryngology: A Narrative Review of Its Current and Evolving Picture

With technological advancements, artificial intelligence (AI) has progressed to become a ubiquitous part of human life. Its aspects in otorhinolaryngology are varied and are continuously evolving. Currently, AI has applications in hearing aids, imaging technologies, interpretation of auditory brain stem systems, and many more in otology. In rhinology, AI is seen to impact navigation, robotic surgeries, and the determination of various anomalies. Detection of voice pathologies and imaging are some areas of laryngology where AI is being used. This review gives an outlook on the diverse elements, applications, and advancements of AI in otorhinolaryngology. The various subfields of AI including machine learning, neural networks, and deep learning are also discussed. Clinical integration of AI and otorhinolaryngology has immense potential to revolutionize the healthcare system and improve the standards of patient care. The current applications of AI and its future scopes in developing this field are highlighted in this review.

A deep learning approach to the diagnosis of atelectasis and attic retraction pocket in otitis media with effusion using otoscopic images

BACKGROUND

This study aimed to develop and validate a deep learning (DL) model to identify atelectasis and attic retraction pocket in cases of otitis media with effusion (OME) using multi-center otoscopic images.

METHOD

A total of 6393 OME otoscopic images from three centers were used to develop and validate a DL model for detecting atelectasis and attic retraction pocket. A threefold random cross-validation procedure was adopted to divide the dataset into training validation sets on a patient level. A team of otologists was assigned to diagnose and characterize atelectasis and attic retraction pocket in otoscopic images. Receiver operating characteristic (ROC) curves, including area under the ROC curve (AUC), accuracy, sensitivity, and specificity were used to assess the performance of the DL model. Class Activation Mapping (CAM) illustrated the discriminative regions in the otoscopic images.

RESULTS

Among all OME otoscopic images, 3564 (55.74%) were identified with attic retraction pocket, and 2460 (38.48%) with atelectasis. The diagnostic DL model of attic retraction pocket and atelectasis achieved a threefold cross-validation accuracy of 89% and 79%, AUC of 0.89 and 0.87, a sensitivity of 0.93 and 0.71, and a specificity of 0.62 and 0.84, respectively. Larger and deeper cases of atelectasis and attic retraction pocket showed greater weight, based on the red color depicted in the heat map of CAM.

CONCLUSION

The DL algorithm could be employed to identify atelectasis and attic retraction pocket in otoscopic images of OME, and as a tool to assist in the accurate diagnosis of OME.

Efficient and accurate diagnosis of otomycosis using an ensemble deep-learning model

Otomycosis accounts for over 15% of cases of external otitis worldwide. It is common in humid regions and Chinese cultures with ear-cleaning custom. Aspergillus and Candida are the major pathogens causing long-term infection. Early endoscopic and microbiological examinations, performed by otologists and microbiologists, respectively, are important for the appropriate medical treatment of otomycosis. The deep-learning model is a novel automatic diagnostic program that provides quick and accurate diagnoses using a large database of images acquired in clinical settings. The aim of the present study was to introduce a machine-learning model to accurately and quickly diagnose otomycosis caused by Aspergillus and Candida. We propose a computer-aided decision-making system based on a deep-learning model comprising two subsystems: Java web application and image classification. The web application subsystem provides a user-friendly webpage to collect consulted images and display the calculation results. The image classification subsystem mainly trained neural network models for end-to-end data inference. The end user uploads a few images obtained with the ear endoscope, and the system returns the classification results to the user in the form of category probability values. To accurately diagnose otomycosis, we used otoendoscopic images and fungal culture secretion. Fungal fluorescence, culture, and DNA sequencing were performed to confirm the pathogens Aspergillus or Candida spp. In addition, impacted cerumen, external otitis, and normal external auditory canal endoscopic images were retained for reference. We merged these four types of images into an otoendoscopic image gallery. To achieve better accuracy and generalization abilities after model-training, we selected 2,182 of approximately 4,000 ear endoscopic images as training samples and 475 as validation samples. After selecting the deep neural network models, we tested the ResNet, SENet, and EfficientNet neural network models with different numbers of layers. Considering the accuracy and operation speed, we finally chose the EfficientNetB6 model, and the probability values of the four categories of otomycosis, impacted cerumen, external otitis, and normal cases were outputted. After multiple model training iterations, the average accuracy of the overall validation sample reached 92.42%. The results suggest that the system could be used as a reference for general practitioners to obtain more accurate diagnoses of otomycosis.

Radiomics of high-resolution computed tomography for the differentiation between cholesteatoma and middle ear inflammation: effects of post-reconstruction methods in a dual-center study

Objectives

To evaluate the performance of radiomic features extracted from high-resolution computed tomography (HRCT) for the differentiation between cholesteatoma and middle ear inflammation (MEI), and to investigate the impact of post-reconstruction harmonization and data resampling.

Methods

One hundred patients were included in this retrospective dual-center study: 48 with histology-proven cholesteatoma (center A: 23; center B: 25) and 52 with MEI (A: 27; B: 25). Radiomic features (co-occurrence and run-length matrix, absolute gradient, autoregressive model, Haar wavelet transform) were extracted from manually defined 2D-ROIs. The ten best features for lesion differentiation were selected using probability of error and average correlation coefficients. A multi-layer perceptron feed-forward artificial neural network (MLP-ANN) was used for radiomics-based classification, with histopathology serving as the reference standard (70% of cases for training, 30% for validation). The analysis was performed five times each on (a) unmodified data and on data that were (b) resampled to the same matrix size, and (c) corrected for acquisition protocol differences using ComBat harmonization.

Results

Using unmodified data, the MLP-ANN classification yielded an overall median area under the receiver operating characteristic curve (AUC) of 0.78 (0.72–0.84). Using original data from center A and resampled data from center B, an overall median AUC of 0.88 (0.82–0.99) was yielded, while using ComBat harmonized data, an overall median AUC of 0.89 (0.79–0.92) was revealed.

Conclusion

Radiomic features extracted from HRCT differentiate between cholesteatoma and MEI. When using multi-centric data obtained with differences in CT acquisition parameters, data resampling and ComBat post-reconstruction harmonization clearly improve radiomics-based lesion classification.

Key Points

• Unenhanced high-resolution CT coupled with radiomics analysis may be useful for the differentiation between cholesteatoma and middle ear inflammation.

• Pooling of data extracted from inhomogeneous CT datasets does not appear meaningful without further post-processing.

• When using multi-centric CT data obtained with differences in acquisition parameters, post-reconstruction harmonization and data resampling clearly improve radiomics-based soft-tissue differentiation.

OtoMatch: Content-based eardrum image retrieval using deep learning

Acute infections of the middle ear are the most commonly treated childhood diseases. Because complications affect children's language learning and cognitive processes, it is essential to diagnose these diseases in a timely and accurate manner. The prevailing literature suggests that it is difficult to accurately diagnose these infections, even for experienced ear, nose, and throat (ENT) physicians. Advanced care practitioners (e.g., nurse practitioners, physician assistants) serve as first-line providers in many primary care settings and may benefit from additional guidance to appropriately determine the diagnosis and treatment of ear diseases. For this purpose, we designed a content-based image retrieval (CBIR) system (called OtoMatch) for normal, middle ear effusion, and tympanostomy tube conditions, operating on eardrum images captured with a digital otoscope. We present a method that enables the conversion of any convolutional neural network (trained for classification) into an image retrieval model. As a proof of concept, we converted a pre-trained deep learning model into an image retrieval system. We accomplished this by changing the fully connected layers into lookup tables. A database of 454 labeled eardrum images (179 normal, 179 effusion, and 96 tube cases) was used to train and test the system. On a 10-fold cross validation, the proposed method resulted in an average accuracy of 80.58% (SD 5.37%), and maximum F1 score of 0.90 while retrieving the most similar image from the database. These are promising results for the first study to demonstrate the feasibility of developing a CBIR system for eardrum images using the newly proposed methodology.

Computer-aided diagnosis of external and middle ear conditions: A machine learning approach

In medicine, a misdiagnosis or the absence of specialists can affect the patient’s health, leading to unnecessary tests and increasing the costs of healthcare. In particular, the lack of specialists in otolaryngology in third world countries forces patients to seek medical attention from general practitioners, whom might not have enough training and experience for making correct diagnosis in this field. To tackle this problem, we propose and test a computer-aided system based on machine learning models and image processing techniques for otoscopic examination, as a support for a more accurate diagnosis of ear conditions at primary care before specialist referral; in particular, for myringosclerosis, earwax plug, and chronic otitis media. To characterize the tympanic membrane and ear canal for each condition, we implemented three different feature extraction methods: color coherence vector, discrete cosine transform, and filter bank. We also considered three machine learning algorithms: support vector machine (SVM), k-nearest neighbor (k-NN) and decision trees to develop the ear condition predictor model. To conduct the research, our database included 160 images as testing set and 720 images as training and validation sets of 180 patients. We repeatedly trained the learning models using the training dataset and evaluated them using the validation dataset to thus obtain the best feature extraction method and learning model that produce the highest validation accuracy. The results showed that the SVM and k-NN presented the best performance followed by decision trees model. Finally, we performed a classification stage –i.e., diagnosis– using testing data, where the SVM model achieved an average classification accuracy of 93.9%, average sensitivity of 87.8%, average specificity of 95.9%, and average positive predictive value of 87.7%. The results show that this system might be used for general practitioners as a reference to make better decisions in the ear pathologies diagnosis.

Transfer representation learning for medical image analysis

There are two major challenges to overcome when developing a classifier to perform automatic disease diagnosis. First, the amount of labeled medical data is typically very limited, and a classifier cannot be effectively trained to attain high disease-detection accuracy. Second, medical domain knowledge is required to identify representative features in data for detecting a target disease. Most computer scientists and statisticians do not have such domain knowledge. In this work, we show that employing transfer learning can remedy both problems. We use Otitis Media (OM) to conduct our case study. Instead of using domain knowledge to extract features from labeled OM images, we construct features based on a dataset entirely OM-irrelevant. More specifically, we first learn a codebook in an unsupervised way from 15 million images collected from ImageNet. The codebook gives us what the encoders consider being the fundamental elements of those 15 million images. We then encode OM images using the codebook and obtain a weighting vector for each OM image. Using the resulting weighting vectors as the feature vectors of the OM images, we employ a traditional supervised learning algorithm to train an OM classifier. The achieved detection accuracy is 88.5% (89.63% in sensitivity and 86.9% in specificity), markedly higher than all previous attempts, which relied on domain experts to help extract features.