Smartphone-based artificial intelligence using a transfer learning algorithm for the detection and diagnosis of middle ear diseases: A retrospective deep learning study

Artificial intelligence-assisted grading for tear trough deformity

BACKGROUND

Various classification systems for tear trough deformity (TTD) have been published; however, their complexity can pose challenges in clinical use, especially for less experienced surgeons. It is believed that artificial intelligence (AI) technology can address some of these challenges by reducing inadvertent errors and improving the accuracy of medical practice. In this study, we aimed to establish a reliable and precise digital image grading model for TTD using smartphone-based photography enhanced using AI deep learning technology. This model is designed to aid and guide surgeons, particularly those who are less experienced or from younger generations, during clinical examinations and in making decisions regarding further surgical interventions.

MATERIALS AND METHODS

A total of 504 patients and 983 photos were included in the study. We adopted the Barton's grading system for TTD. All photos were taken using the same smartphone and processed and analyzed using the medical AI assistant (MAIA™) software. The photos were then randomly divided into two groups to establish training and testing models.

RESULTS

The confusion matrix for the training model demonstrated a sensitivity of 56%, specificity of 87.3%, F1 score of 0.57, and an area under the curve (AUROC) of 0.85. For the testing group, the sensitivity was 49.3%, specificity was 85%, F1 score was 0.49, and AUROC was 0.83. Representative heatmaps were also generated.

CONCLUSION

Our study is the first to demonstrate that tear trough deformities can be easily categorized using a built-in smartphone camera in conjunction with an AI deep learning program. This approach can reduce errors during clinical patient evaluations, particularly for less experienced practitioners.

Development and validation of a smartphone-based deep-learning-enabled system to detect middle-ear conditions in otoscopic images

Middle-ear conditions are common causes of primary care visits, hearing impairment, and inappropriate antibiotic use. Deep learning (DL) may assist clinicians in interpreting otoscopic images. This study included patients over 5 years old from an ambulatory ENT practice in Strasbourg, France, between 2013 and 2020. Digital otoscopic images were obtained using a smartphone-attached otoscope (Smart Scope, Karl Storz, Germany) and labeled by a senior ENT specialist across 11 diagnostic classes (reference standard). An Inception-v2 DL model was trained using 41,664 otoscopic images, and its diagnostic accuracy was evaluated by calculating class-specific estimates of sensitivity and specificity. The model was then incorporated into a smartphone app called i-Nside. The DL model was evaluated on a validation set of 3,962 images and a held-out test set comprising 326 images. On the validation set, all class-specific estimates of sensitivity and specificity exceeded 98%. On the test set, the DL model achieved a sensitivity of 99.0% (95% confidence interval: 94.5-100) and a specificity of 95.2% (91.5-97.6) for the binary classification of normal vs. abnormal images; wax plugs were detected with a sensitivity of 100% (94.6-100) and specificity of 97.7% (95.0-99.1); other class-specific estimates of sensitivity and specificity ranged from 33.3% to 92.3% and 96.0% to 100%, respectively. We present an end-to-end DL-enabled system able to achieve expert-level diagnostic accuracy for identifying normal tympanic aspects and wax plugs within digital otoscopic images. However, the system's performance varied for other middle-ear conditions. Further prospective validation is necessary before wider clinical deployment.

Smartphone-Based Artificial Intelligence for the Detection and Diagnosis of Pediatric Diseases: A Comprehensive Review

In recent years, the use of smartphones and other wireless technology in medical care has developed rapidly. However, in some cases, especially for pediatric medical problems, the reliability of information accessed by mobile health technology remains debatable. The main aim of this paper is to evaluate the relevance of smartphone applications in the detection and diagnosis of pediatric medical conditions for which the greatest number of applications have been developed. This is the case of smartphone applications developed for the diagnosis of acute otitis media, otitis media with effusion, hearing impairment, obesity, amblyopia, and vision screening. In some cases, the information given by these applications has significantly improved the diagnostic ability of physicians. However, distinguishing between applications that can be effective and those that may lead to mistakes can be very difficult. This highlights the importance of a careful application selection before including smartphone-based artificial intelligence in everyday clinical practice.

Feasibility of Multimodal Artificial Intelligence Using GPT-4 Vision for the Classification of Middle Ear Disease: Qualitative Study and Validation

BACKGROUND

The integration of artificial intelligence (AI), particularly deep learning models, has transformed the landscape of medical technology, especially in the field of diagnosis using imaging and physiological data. In otolaryngology, AI has shown promise in image classification for middle ear diseases. However, existing models often lack patient-specific data and clinical context, limiting their universal applicability. The emergence of GPT-4 Vision (GPT-4V) has enabled a multimodal diagnostic approach, integrating language processing with image analysis.

OBJECTIVE

In this study, we investigated the effectiveness of GPT-4V in diagnosing middle ear diseases by integrating patient-specific data with otoscopic images of the tympanic membrane.

METHODS

The design of this study was divided into two phases: (1) establishing a model with appropriate prompts and (2) validating the ability of the optimal prompt model to classify images. In total, 305 otoscopic images of 4 middle ear diseases (acute otitis media, middle ear cholesteatoma, chronic otitis media, and otitis media with effusion) were obtained from patients who visited Shinshu University or Jichi Medical University between April 2010 and December 2023. The optimized GPT-4V settings were established using prompts and patients' data, and the model created with the optimal prompt was used to verify the diagnostic accuracy of GPT-4V on 190 images. To compare the diagnostic accuracy of GPT-4V with that of physicians, 30 clinicians completed a web-based questionnaire consisting of 190 images.

RESULTS

The multimodal AI approach achieved an accuracy of 82.1%, which is superior to that of certified pediatricians at 70.6%, but trailing behind that of otolaryngologists at more than 95%. The model's disease-specific accuracy rates were 89.2% for acute otitis media, 76.5% for chronic otitis media, 79.3% for middle ear cholesteatoma, and 85.7% for otitis media with effusion, which highlights the need for disease-specific optimization. Comparisons with physicians revealed promising results, suggesting the potential of GPT-4V to augment clinical decision-making.

CONCLUSIONS

Despite its advantages, challenges such as data privacy and ethical considerations must be addressed. Overall, this study underscores the potential of multimodal AI for enhancing diagnostic accuracy and improving patient care in otolaryngology. Further research is warranted to optimize and validate this approach in diverse clinical settings.

Smartphone-based device for point-of-care diagnostics of pulmonary inflammation using convolutional neural networks (CNNs)

In pulmonary inflammation diseases, like COVID-19, lung involvement and inflammation determine the treatment regime. Respiratory inflammation is typically arisen due to the cytokine storm and the leakage of the vessels for immune cells recruitment. Currently, such a situation is detected by the clinical judgment of a specialist or precisely by a chest CT scan. However, the lack of accessibility to the CT machines in many poor medical centers as well as its expensive service, demands more accessible methods for fast and cheap detection of lung inflammation. Here, we have introduced a novel method for tracing the inflammation and lung involvement in patients with pulmonary inflammation, such as COVID-19, by a simple electrolyte detection in their sputum samples. The presence of the electrolyte in the sputum sample results in the fern-like structures after air-drying. These fern patterns are different in the CT positive and negative cases that are detected by an AI application on a smartphone and using a low-cost and portable mini-microscope. Evaluating 160 patient-derived sputum sample images, this method demonstrated an interesting accuracy of 95%, as confirmed by CT-scan results. This finding suggests that the method has the potential to serve as a promising and reliable approach for recognizing lung inflammatory diseases, such as COVID-19.

Optimizing age-related hearing risk predictions: an advanced machine learning integration with HHIE-S

OBJECTIVES

The elderly are disproportionately affected by age-related hearing loss (ARHL). Despite being a well-known tool for ARHL evaluation, the Hearing Handicap Inventory for the Elderly Screening version (HHIE-S) has only traditionally been used for direct screening using self-reported outcomes. This work uses a novel integration of machine learning approaches to improve the predicted accuracy of the HHIE-S tool for ARHL in older adults.

METHODS

We employed a dataset that was gathered between 2016 and 2018 and included 1,526 senior citizens from several Taipei City Hospital branches. 80% of the data were used for training (n = 1220) and 20% were used for testing (n = 356). XGBoost, Gradient Boosting, and LightGBM were among the machine learning models that were only used and assessed on the training set. In order to prevent data leakage and overfitting, the Light Gradient Boosting Machine (LGBM) model-which had the greatest AUC of 0.83 (95% CI 0.81-0.85)-was then only used on the holdout testing data.

RESULTS

On the testing set, the LGBM model showed a strong AUC of 0.82 (95% CI 0.79-0.86), far outperforming conventional techniques. Notably, several HHIE-S items and age were found to be significant characteristics. In contrast to traditional HHIE research, which concentrates on the psychological effects of hearing loss, this study combines cutting-edge machine learning techniques-specifically, the LGBM classifier-with the HHIE-S tool. The incorporation of SHAP values enhances the interpretability of the model's predictions and provides a more comprehensive comprehension of the significance of various aspects.

CONCLUSIONS

Our methodology highlights the great potential that arises from combining machine learning with validated hearing evaluation instruments such as the HHIE-S. Healthcare practitioners can anticipate ARHL more accurately thanks to this integration, which makes it easier to intervene quickly and precisely.

ISOM 2023 research Panel 4 - Diagnostics and microbiology of otitis media

OBJECTIVES

To identify and review key research advances from the literature published between 2019 and 2023 on the diagnosis and microbiology of otitis media (OM) including acute otitis media (AOM), recurrent AOM (rAOM), otitis media with effusion (OME), chronic suppurative otitis media (CSOM) and AOM complications (mastoiditis).

DATA SOURCES

PubMed database of the National Library of Medicine.

REVIEW METHODS

All relevant original articles published in Medline in English between July 2019 and February 2023 were identified. Studies that were reviews, case studies, relating to OM complications (other than mastoiditis), and studies focusing on guideline adherence, and consensus statements were excluded. Members of the panel drafted the report based on these search results.

MAIN FINDINGS

For the diagnosis section, 2294 unique records screened, 55 were eligible for inclusion. For the microbiology section 705 unique records were screened and 137 articles were eligible for inclusion. The main themes that arose in OM diagnosis were the need to incorporate multiple modalities including video-otoscopy, tympanometry, telemedicine and artificial intelligence for accurate diagnoses in all diagnostic settings. Further to this, was the use of new, cheap, readily available tools which may improve access in rural and lowmiddle income (LMIC) settings. For OM aetiology, PCR remains the most sensitive method for detecting middle ear pathogens with microbiome analysis still largely restricted to research use. The global pandemic response reduced rates of OM in children, but post-pandemic shifts should be monitored.

IMPLICATION FOR PRACTICE AND FUTURE RESEARCH

Cheap, easy to use multi-technique assessments combined with artificial intelligence and/or telemedicine should be integrated into future practice to improve diagnosis and treatment pathways in OM diagnosis. Longitudinal studies investigating the in-vivo process of OM development, timings and in-depth interactions between the triad of bacteria, viruses and the host immune response are still required. Standardized methods of collection and analysis for microbiome studies to enable inter-study comparisons are required. There is a need to target underlying biofilms if going to effectively prevent rAOM and OME and possibly enhance ventilation tube retention.

Applications of Artificial Intelligence in Temporal Bone Imaging: Advances and Future Challenges

The applications of artificial intelligence (AI) in temporal bone (TB) imaging have gained significant attention in recent years, revolutionizing the field of otolaryngology and radiology. Accurate interpretation of imaging features of TB conditions plays a crucial role in diagnosing and treating a range of ear-related pathologies, including middle and inner ear diseases, otosclerosis, and vestibular schwannomas. According to multiple clinical studies published in the literature, AI-powered algorithms have demonstrated exceptional proficiency in interpreting imaging findings, not only saving time for physicians but also enhancing diagnostic accuracy by reducing human error. Although several challenges remain in routinely relying on AI applications, the collaboration between AI and healthcare professionals holds the key to better patient outcomes and significantly improved patient care. This overview delivers a comprehensive update on the advances of AI in the field of TB imaging, summarizes recent evidence provided by clinical studies, and discusses future insights and challenges in the widespread integration of AI in clinical practice.

Diagnosis, Treatment, and Management of Otitis Media with Artificial Intelligence

A common infectious disease, otitis media (OM) has a low rate of early diagnosis, which significantly increases the difficulty of treating the disease and the likelihood of serious complications developing including hearing loss, speech impairment, and even intracranial infection. Several areas of healthcare have shown great promise in the application of artificial intelligence (AI) systems, such as the accurate detection of diseases, the automated interpretation of images, and the prediction of patient outcomes. Several articles have reported some machine learning (ML) algorithms such as ResNet, InceptionV3 and Unet, were applied to the diagnosis of OM successfully. The use of these techniques in the OM is still in its infancy, but their potential is enormous. We present in this review important concepts related to ML and AI, describe how these technologies are currently being applied to diagnosing, treating, and managing OM, and discuss the challenges associated with developing AI-assisted OM technologies in the future.

Use of artificial intelligence for the diagnosis of cholesteatoma

Objectives

Accurate diagnosis of cholesteatomas is crucial. However, cholesteatomas can easily be missed in routine otoscopic exams. Convolutional neural networks (CNNs) have performed well in medical image classification, so we evaluated their use for detecting cholesteatomas in otoscopic images.

Study Design

Design and evaluation of artificial intelligence driven workflow for cholesteatoma diagnosis.

Methods

Otoscopic images collected from the faculty practice of the senior author were deidentified and labeled by the senior author as cholesteatoma, abnormal non-cholesteatoma, or normal. An image classification workflow was developed to automatically differentiate cholesteatomas from other possible tympanic membrane appearances. Eight pretrained CNNs were trained on our otoscopic images, then tested on a withheld subset of images to evaluate their final performance. CNN intermediate activations were also extracted to visualize important image features.

Results

A total of 834 otoscopic images were collected, further categorized into 197 cholesteatoma, 457 abnormal non-cholesteatoma, and 180 normal. Final trained CNNs demonstrated strong performance, achieving accuracies of 83.8%-98.5% for differentiating cholesteatoma from normal, 75.6%-90.1% for differentiating cholesteatoma from abnormal non-cholesteatoma, and 87.0%-90.4% for differentiating cholesteatoma from non-cholesteatoma (abnormal non-cholesteatoma + normal). DenseNet201 (100% sensitivity, 97.1% specificity), NASNetLarge (100% sensitivity, 88.2% specificity), and MobileNetV2 (94.1% sensitivity, 100% specificity) were among the best performing CNNs in distinguishing cholesteatoma versus normal. Visualization of intermediate activations showed robust detection of relevant image features by the CNNs.

Conclusion

While further refinement and more training images are needed to improve performance, artificial intelligence-driven analysis of otoscopic images shows great promise as a diagnostic tool for detecting cholesteatomas.

Level of Evidence

3.