Deep reinforcement learning and convolutional autoencoders for anomaly detection of congenital inner ear malformations in clinical CT images

Detection of abnormalities within the inner ear is a challenging task even for experienced clinicians. In this study, we propose an automated method for automatic abnormality detection to provide support for the diagnosis and clinical management of various otological disorders. We propose a framework for inner ear abnormality detection based on deep reinforcement learning for landmark detection which is trained uniquely in normative data. In our approach, we derive two abnormality measurements: Dimage and Uimage. The first measurement, Dimage, is based on the variability of the predicted configuration of a well-defined set of landmarks in a subspace formed by the point distribution model of the location of those landmarks in normative data. We create this subspace using Procrustes shape alignment and Principal Component Analysis projection. The second measurement, Uimage, represents the degree of hesitation of the agents when approaching the final location of the landmarks and is based on the distribution of the predicted Q-values of the model for the last ten states. Finally, we unify these measurements in a combined anomaly measurement called Cimage. We compare our method's performance with a 3D convolutional autoencoder technique for abnormality detection using the patch-based mean squared error between the original and the generated image as a basis for classifying abnormal versus normal anatomies. We compare both approaches and show that our method, based on deep reinforcement learning, shows better detection performance for abnormal anatomies on both an artificial and a real clinical CT dataset of various inner ear malformations with an increase of 11.2% of the area under the ROC curve. Our method also shows more robustness against the heterogeneous quality of the images in our dataset.

Multi-modal deep learning for joint prediction of otitis media and diagnostic difficulty

Objectives

In this study, we propose a diagnostic model for automatic detection of otitis media based on combined input of otoscopy images and wideband tympanometry measurements.

Methods

We present a neural network-based model for the joint prediction of otitis media and diagnostic difficulty. We use the subclassifications acute otitis media and otitis media with effusion. The proposed approach is based on deep metric learning, and we compare this with the performance of a standard multi-task network.

Results

The proposed deep metric approach shows good performance on both tasks, and we show that the multi-modal input increases the performance for both classification and difficulty estimation compared to the models trained on the modalities separately. An accuracy of 86.5% is achieved for the classification task, and a Kendall rank correlation coefficient of 0.45 is achieved for difficulty estimation, corresponding to a correct ranking of 72.6% of the cases.

Conclusion

This study demonstrates the strengths of a multi-modal diagnostic tool using both otoscopy images and wideband tympanometry measurements for the diagnosis of otitis media. Furthermore, we show that deep metric learning improves the performance of the models.

Inter-rater reliability of the diagnosis of otitis media based on otoscopic images and wideband tympanometry measurements

OBJECTIVES

This study aims to investigate the inter-rater reliability and agreement of the diagnosis of otitis media with effusion, acute otitis media, and no effusion cases based on an otoscopy image and in some cases an additional wideband tympanometry measurement of the patient.

METHODS

1409 cases were examined and diagnosed by an otolaryngologist in the clinic, and otoscopy examination and wideband tympanometry (WBT) measurement were conducted. Afterwards, four otolaryngologists (Ear, Nose, and Throat doctors, ENTs), who did not perform the acute examination of the patients, evaluated the otoscopy images and WBT measurements results for diagnosis (acute otitis media, otitis media with effusion, or no effusion). They also specified their diagnostic certainty for each case, and reported whether they used the image, wideband tympanometry, or both, for diagnosis.

RESULTS

All four ENTs agreed on the diagnosis in 57% of the cases, with a pairwise agreement of 74%, and a Light's Kappa of 0.58. There are, however, large differences in agreement and certainty between the three diagnoses. Acute otitis media yields the highest agreement (77% between all four ENTs) and certainty (0.90), while no effusion shows much lower agreement and certainty (34% and 0.58, respectively). There is a positive correlation between certainty and agreement between the ENTs across all cases, and both certainty and agreement increase for cases where a WBT measurement is shown in addition to the otoscopy image.

CONCLUSIONS

The inter-rater reliability between four ENTs was high when diagnosing acute otitis media and lower when diagnosing otitis media with effusion. However, WBT can add valuable information to get closer to the ground-truth diagnosis without myringotomy. Furthermore, the diagnostic certainty increases when the WBT is examined together with the otoscopy image.

Deep metric learning for otitis media classification

In this study, we propose an automatic diagnostic algorithm for detecting otitis media based on otoscopy images of the tympanic membrane. A total of 1336 images were assessed by a medical specialist into three diagnostic groups: acute otitis media, otitis media with effusion, and no effusion. To provide proper treatment and care and limit the use of unnecessary antibiotics, it is crucial to correctly detect tympanic membrane abnormalities, and to distinguish between acute otitis media and otitis media with effusion. The proposed approach for this classification task is based on deep metric learning, and this study compares the performance of different distance-based metric loss functions. Contrastive loss, triplet loss and multi-class N-pair loss are employed, and compared with the performance of standard cross-entropy and class-weighted cross-entropy classification networks. Triplet loss achieves high precision on a highly imbalanced data set, and the deep metric methods provide useful insight into the decision making of a neural network. The results are comparable to the best clinical experts and paves the way for more accurate and operator-independent diagnosis of otitis media.