Automated multi-class classification for prediction of tympanic membrane changes with deep learning models

Feasibility of Multimodal Artificial Intelligence Using GPT-4 Vision for the Classification of Middle Ear Disease: Qualitative Study and Validation

BACKGROUND

The integration of artificial intelligence (AI), particularly deep learning models, has transformed the landscape of medical technology, especially in the field of diagnosis using imaging and physiological data. In otolaryngology, AI has shown promise in image classification for middle ear diseases. However, existing models often lack patient-specific data and clinical context, limiting their universal applicability. The emergence of GPT-4 Vision (GPT-4V) has enabled a multimodal diagnostic approach, integrating language processing with image analysis.

OBJECTIVE

In this study, we investigated the effectiveness of GPT-4V in diagnosing middle ear diseases by integrating patient-specific data with otoscopic images of the tympanic membrane.

METHODS

The design of this study was divided into two phases: (1) establishing a model with appropriate prompts and (2) validating the ability of the optimal prompt model to classify images. In total, 305 otoscopic images of 4 middle ear diseases (acute otitis media, middle ear cholesteatoma, chronic otitis media, and otitis media with effusion) were obtained from patients who visited Shinshu University or Jichi Medical University between April 2010 and December 2023. The optimized GPT-4V settings were established using prompts and patients' data, and the model created with the optimal prompt was used to verify the diagnostic accuracy of GPT-4V on 190 images. To compare the diagnostic accuracy of GPT-4V with that of physicians, 30 clinicians completed a web-based questionnaire consisting of 190 images.

RESULTS

The multimodal AI approach achieved an accuracy of 82.1%, which is superior to that of certified pediatricians at 70.6%, but trailing behind that of otolaryngologists at more than 95%. The model's disease-specific accuracy rates were 89.2% for acute otitis media, 76.5% for chronic otitis media, 79.3% for middle ear cholesteatoma, and 85.7% for otitis media with effusion, which highlights the need for disease-specific optimization. Comparisons with physicians revealed promising results, suggesting the potential of GPT-4V to augment clinical decision-making.

CONCLUSIONS

Despite its advantages, challenges such as data privacy and ethical considerations must be addressed. Overall, this study underscores the potential of multimodal AI for enhancing diagnostic accuracy and improving patient care in otolaryngology. Further research is warranted to optimize and validate this approach in diverse clinical settings.

Accurate Segmentation and Tracking of Chorda Tympani in Endoscopic Middle Ear Surgery with Artificial Intelligence

Objective: We introduce a novel endoscopic middle ear surgery dataset specifically designed for evaluating deep learning (DL)-based semantic segmentation of chorda tympani. Methods: We curated a dataset comprising 8240 images from 25 patients, divided into a training set (20%, 1648 images), validation set (5%, 412 images), and test set (75%, 6180 images). We employed data enhancement techniques to expand the picture size of the training and validation sets by 5 times (training set: 8240 images, verification set: 2060 images). Subsequently, we employed a multistage transfer learning training method to establish, train, and validate various convolutional neural networks. Results: On the validation set of 2060 labeled images, our proposed network achieved good results, with the U-net exhibiting the highest effectiveness (mIOU = 0.8737, mPA = 0.9263). Furthermore, when applied to the test dataset of 6180 raw images and contrasted with the prediction of otologists, the overall performance of the U-net was excellent (accuracy = 0.911, precision = 0.9823, sensitivity = 0.8777, specificity = 0.9714). Conclusions: Our findings demonstrate that DL can be successfully employed for automatic segmentation of chorda tympani in endoscopic middle ear surgery, yielding high-performance results. This study validates the potential feasibility of future intelligent navigation technologies to assist in endoscopic middle ear surgery.

ISOM 2023 research Panel 4 - Diagnostics and microbiology of otitis media

OBJECTIVES

To identify and review key research advances from the literature published between 2019 and 2023 on the diagnosis and microbiology of otitis media (OM) including acute otitis media (AOM), recurrent AOM (rAOM), otitis media with effusion (OME), chronic suppurative otitis media (CSOM) and AOM complications (mastoiditis).

DATA SOURCES

PubMed database of the National Library of Medicine.

REVIEW METHODS

All relevant original articles published in Medline in English between July 2019 and February 2023 were identified. Studies that were reviews, case studies, relating to OM complications (other than mastoiditis), and studies focusing on guideline adherence, and consensus statements were excluded. Members of the panel drafted the report based on these search results.

MAIN FINDINGS

For the diagnosis section, 2294 unique records screened, 55 were eligible for inclusion. For the microbiology section 705 unique records were screened and 137 articles were eligible for inclusion. The main themes that arose in OM diagnosis were the need to incorporate multiple modalities including video-otoscopy, tympanometry, telemedicine and artificial intelligence for accurate diagnoses in all diagnostic settings. Further to this, was the use of new, cheap, readily available tools which may improve access in rural and lowmiddle income (LMIC) settings. For OM aetiology, PCR remains the most sensitive method for detecting middle ear pathogens with microbiome analysis still largely restricted to research use. The global pandemic response reduced rates of OM in children, but post-pandemic shifts should be monitored.

IMPLICATION FOR PRACTICE AND FUTURE RESEARCH

Cheap, easy to use multi-technique assessments combined with artificial intelligence and/or telemedicine should be integrated into future practice to improve diagnosis and treatment pathways in OM diagnosis. Longitudinal studies investigating the in-vivo process of OM development, timings and in-depth interactions between the triad of bacteria, viruses and the host immune response are still required. Standardized methods of collection and analysis for microbiome studies to enable inter-study comparisons are required. There is a need to target underlying biofilms if going to effectively prevent rAOM and OME and possibly enhance ventilation tube retention.

Diagnosis, Treatment, and Management of Otitis Media with Artificial Intelligence

A common infectious disease, otitis media (OM) has a low rate of early diagnosis, which significantly increases the difficulty of treating the disease and the likelihood of serious complications developing including hearing loss, speech impairment, and even intracranial infection. Several areas of healthcare have shown great promise in the application of artificial intelligence (AI) systems, such as the accurate detection of diseases, the automated interpretation of images, and the prediction of patient outcomes. Several articles have reported some machine learning (ML) algorithms such as ResNet, InceptionV3 and Unet, were applied to the diagnosis of OM successfully. The use of these techniques in the OM is still in its infancy, but their potential is enormous. We present in this review important concepts related to ML and AI, describe how these technologies are currently being applied to diagnosing, treating, and managing OM, and discuss the challenges associated with developing AI-assisted OM technologies in the future.