Improving Uncertainty Estimation With Semi-Supervised Deep Learning for COVID-19 Detection Using Chest X-Ray Images

Trustworthy clinical AI solutions: A unified review of uncertainty quantification in Deep Learning models for medical image analysis

The full acceptance of Deep Learning (DL) models in the clinical field is rather low with respect to the quantity of high-performing solutions reported in the literature. End users are particularly reluctant to rely on the opaque predictions of DL models. Uncertainty quantification methods have been proposed in the literature as a potential solution, to reduce the black-box effect of DL models and increase the interpretability and the acceptability of the result by the final user. In this review, we propose an overview of the existing methods to quantify uncertainty associated with DL predictions. We focus on applications to medical image analysis, which present specific challenges due to the high dimensionality of images and their variable quality, as well as constraints associated with real-world clinical routine. Moreover, we discuss the concept of structural uncertainty, a corpus of methods to facilitate the alignment of segmentation uncertainty estimates with clinical attention. We then discuss the evaluation protocols to validate the relevance of uncertainty estimates. Finally, we highlight the open challenges for uncertainty quantification in the medical field.

COVID-19 Detection using adopted convolutional neural networks and high-performance computing

The COVID 19 pandemic is highly contagious disease is wreaking havoc on people's health and well-being around the world. Radiological imaging with chest radiography is one among the key screening procedure. This disease contaminates the respiratory system and impacts the alveoli, which are small air sacs in the lungs. Several artificial intelligence (AI)-based method to detect COVID-19 have been introduced. The recognition of disease patients using features and variation in chest radiography images was demonstrated using this model. In proposed paper presents a model, a deep convolutional neural network (CNN) with ResNet50 configuration, that really is freely-available and accessible to the common people for detecting this infection from chest radiography scans. The introduced model is capable of recognizing coronavirus diseases from CT scan images that identifies the real time condition of covid-19 patients. Furthermore, the database is capable of tracking detected patients and maintaining their database for increasing accuracy of the training model. The proposed model gives approximately 97% accuracy in determining the above-mentioned results related to covid-19 disease by employing the combination of adopted-CNN and ResNet50 algorithms.

Data augmentation based semi-supervised method to improve COVID-19 CT classification

The Coronavirus (COVID-19) outbreak of December 2019 has become a serious threat to people around the world, creating a health crisis that infected millions of lives, as well as destroying the global economy. Early detection and diagnosis are essential to prevent further transmission. The detection of COVID-19 computed tomography images is one of the important approaches to rapid diagnosis. Many different branches of deep learning methods have played an important role in this area, including transfer learning, contrastive learning, ensemble strategy, etc. However, these works require a large number of samples of expensive manual labels, so in order to save costs, scholars adopted semi-supervised learning that applies only a few labels to classify COVID-19 CT images. Nevertheless, the existing semi-supervised methods focus primarily on class imbalance and pseudo-label filtering rather than on pseudo-label generation. Accordingly, in this paper, we organized a semi-supervised classification framework based on data augmentation to classify the CT images of COVID-19. We revised the classic teacher-student framework and introduced the popular data augmentation method Mixup, which widened the distribution of high confidence to improve the accuracy of selected pseudo-labels and ultimately obtain a model with better performance. For the COVID-CT dataset, our method makes precision, F1 score, accuracy and specificity 21.04%, 12.95%, 17.13% and 38.29% higher than average values for other methods respectively, For the SARS-COV-2 dataset, these increases were 8.40%, 7.59%, 9.35% and 12.80% respectively. For the Harvard Dataverse dataset, growth was 17.64%, 18.89%, 19.81% and 20.20% respectively. The codes are available at https://github.com/YutingBai99/COVID-19-SSL.

Artificial Intelligence-Assisted Chest X-ray for the Diagnosis of COVID-19: A Systematic Review and Meta-Analysis

Because it is an accessible and routine image test, medical personnel commonly use a chest X-ray for COVID-19 infections. Artificial intelligence (AI) is now widely applied to improve the precision of routine image tests. Hence, we investigated the clinical merit of the chest X-ray to detect COVID-19 when assisted by AI. We used PubMed, Cochrane Library, MedRxiv, ArXiv, and Embase to search for relevant research published between 1 January 2020 and 30 May 2022. We collected essays that dissected AI-based measures used for patients diagnosed with COVID-19 and excluded research lacking measurements using relevant parameters (i.e., sensitivity, specificity, and area under curve). Two independent researchers summarized the information, and discords were eliminated by consensus. A random effects model was used to calculate the pooled sensitivities and specificities. The sensitivity of the included research studies was enhanced by eliminating research with possible heterogeneity. A summary receiver operating characteristic curve (SROC) was generated to investigate the diagnostic value for detecting COVID-19 patients. Nine studies were recruited in this analysis, including 39,603 subjects. The pooled sensitivity and specificity were estimated as 0.9472 (p = 0.0338, 95% CI 0.9009-0.9959) and 0.9610 (p < 0.0001, 95% CI 0.9428-0.9795), respectively. The area under the SROC was 0.98 (95% CI 0.94-1.00). The heterogeneity of diagnostic odds ratio was presented in the recruited studies (I² = 36.212, p = 0.129). The AI-assisted chest X-ray scan for COVID-19 detection offered excellent diagnostic potential and broader application.

Bilateral adaptive graph convolutional network on CT based Covid-19 diagnosis with uncertainty-aware consensus-assisted multiple instance learning

Coronavirus disease (COVID-19) has caused a worldwide pandemic, putting millions of people's health and lives in jeopardy. Detecting infected patients early on chest computed tomography (CT) is critical in combating COVID-19. Harnessing uncertainty-aware consensus-assisted multiple instance learning (UC-MIL), we propose to diagnose COVID-19 using a new bilateral adaptive graph-based (BA-GCN) model that can use both 2D and 3D discriminative information in 3D CT volumes with arbitrary number of slices. Given the importance of lung segmentation for this task, we have created the largest manual annotation dataset so far with 7,768 slices from COVID-19 patients, and have used it to train a 2D segmentation model to segment the lungs from individual slices and mask the lungs as the regions of interest for the subsequent analyses. We then used the UC-MIL model to estimate the uncertainty of each prediction and the consensus between multiple predictions on each CT slice to automatically select a fixed number of CT slices with reliable predictions for the subsequent model reasoning. Finally, we adaptively constructed a BA-GCN with vertices from different granularity levels (2D and 3D) to aggregate multi-level features for the final diagnosis with the benefits of the graph convolution network's superiority to tackle cross-granularity relationships. Experimental results on three largest COVID-19 CT datasets demonstrated that our model can produce reliable and accurate COVID-19 predictions using CT volumes with any number of slices, which outperforms existing approaches in terms of learning and generalisation ability. To promote reproducible research, we have made the datasets, including the manual annotations and cleaned CT dataset, as well as the implementation code, available at https://doi.org/10.5281/zenodo.6361963.

Uncertain-CAM: Uncertainty-Based Ensemble Machine Voting for Improved COVID-19 CXR Classification and Explainability

The ongoing coronavirus disease 2019 (COVID-19) pandemic has had a significant impact on patients and healthcare systems across the world. Distinguishing non-COVID-19 patients from COVID-19 patients at the lowest possible cost and in the earliest stages of the disease is a major issue. Additionally, the implementation of explainable deep learning decisions is another issue, especially in critical fields such as medicine. The study presents a method to train deep learning models and apply an uncertainty-based ensemble voting policy to achieve 99% accuracy in classifying COVID-19 chest X-rays from normal and pneumonia-related infections. We further present a training scheme that integrates the cyclic cosine annealing approach with cross-validation and uncertainty quantification that is measured using prediction interval coverage probability (PICP) as final ensemble voting weights. We also propose the Uncertain-CAM technique, which improves deep learning explainability and provides a more reliable COVID-19 classification system. We introduce a new image processing technique to measure the explainability based on ground-truth, and we compared it with the widely adopted Grad-CAM method.

Uncertainty Estimation in Medical Image Classification: Systematic Review

Background

Deep neural networks are showing impressive results in different medical image classification tasks. However, for real-world applications, there is a need to estimate the network’s uncertainty together with its prediction.

Objective

In this review, we investigate in what form uncertainty estimation has been applied to the task of medical image classification. We also investigate which metrics are used to describe the effectiveness of the applied uncertainty estimation

Methods

Google Scholar, PubMed, IEEE Xplore, and ScienceDirect were screened for peer-reviewed studies, published between 2016 and 2021, that deal with uncertainty estimation in medical image classification. The search terms “uncertainty,” “uncertainty estimation,” “network calibration,” and “out-of-distribution detection” were used in combination with the terms “medical images,” “medical image analysis,” and “medical image classification.”

Results

A total of 22 papers were chosen for detailed analysis through the systematic review process. This paper provides a table for a systematic comparison of the included works with respect to the applied method for estimating the uncertainty.

Conclusions

The applied methods for estimating uncertainties are diverse, but the sampling-based methods Monte-Carlo Dropout and Deep Ensembles are used most frequently. We concluded that future works can investigate the benefits of uncertainty estimation in collaborative settings of artificial intelligence systems and human experts.

International Registered Report Identifier (IRRID)

RR2-10.2196/11936

Image enhancement techniques on deep learning approaches for automated diagnosis of COVID-19 features using CXR images

The outbreak of novel coronavirus (COVID-19) disease has infected more than 135.6 million people globally. For its early diagnosis, researchers consider chest X-ray examinations as a standard screening technique in addition to RT-PCR test. Majority of research work till date focused only on application of deep learning approaches that is relevant but lacking in better pre-processing of CXR images. Towards this direction, this study aims to explore cumulative effects of image denoising and enhancement approaches on the performance of deep learning approaches. Regarding pre-processing, suitable methods for X-ray images, Histogram equalization, CLAHE and gamma correction have been tested individually and along with adaptive median filter, median filter, total variation filter and gaussian denoising filters. Proposed study compared eleven combinations in exploration of most coherent approach in greedy manner. For more robust analysis, we compared ten CNN architectures for performance evaluation with and without enhancement approaches. These models are InceptionV3, InceptionResNetV2, MobileNet, MobileNetV2, Vgg19, NASNetMobile, ResNet101, DenseNet121, DenseNet169, DenseNet201. These models are trained in 4-way (COVID-19 pneumonia vs Viral vs Bacterial pneumonia vs Normal) and 3-way classification scenario (COVID-19 vs Pneumonia vs Normal) on two benchmark datasets. The proposed methodology determines with TVF + Gamma, models achieve higher classification accuracy and sensitivity. In 4-way classification MobileNet with TVF + Gamma achieves top accuracy of 93.25% with 1.91% improvement in accuracy score, COVID-19 sensitivity of 98.72% and F1-score of 92.14%. In 3-way classification our DenseNet201 with TVF + Gamma gains accuracy of 91.10% with improvement of 1.47%, COVID-19 sensitivity of 100% and F1-score of 91.09%. Proposed study concludes that deep learning modes with gamma correction and TVF + Gamma has superior performance compared to state-of-the-art models. This not only minimizes overlapping between COVID-19 and virus pneumonia but advantageous in time required to converge best possible results.

COVID-19 Detection in CT/X-ray Imagery Using Vision Transformers

The steady spread of the 2019 Coronavirus disease has brought about human and economic losses, imposing a new lifestyle across the world. On this point, medical imaging tests such as computed tomography (CT) and X-ray have demonstrated a sound screening potential. Deep learning methodologies have evidenced superior image analysis capabilities with respect to prior handcrafted counterparts. In this paper, we propose a novel deep learning framework for Coronavirus detection using CT and X-ray images. In particular, a Vision Transformer architecture is adopted as a backbone in the proposed network, in which a Siamese encoder is utilized. The latter is composed of two branches: one for processing the original image and another for processing an augmented view of the original image. The input images are divided into patches and fed through the encoder. The proposed framework is evaluated on public CT and X-ray datasets. The proposed system confirms its superiority over state-of-the-art methods on CT and X-ray data in terms of accuracy, precision, recall, specificity, and F1 score. Furthermore, the proposed system also exhibits good robustness when a small portion of training data is allocated.

Objective evaluation of deep uncertainty predictions for COVID-19 detection

Deep neural networks (DNNs) have been widely applied for detecting COVID-19 in medical images. Existing studies mainly apply transfer learning and other data representation strategies to generate accurate point estimates. The generalization power of these networks is always questionable due to being developed using small datasets and failing to report their predictive confidence. Quantifying uncertainties associated with DNN predictions is a prerequisite for their trusted deployment in medical settings. Here we apply and evaluate three uncertainty quantification techniques for COVID-19 detection using chest X-Ray (CXR) images. The novel concept of uncertainty confusion matrix is proposed and new performance metrics for the objective evaluation of uncertainty estimates are introduced. Through comprehensive experiments, it is shown that networks pertained on CXR images outperform networks pretrained on natural image datasets such as ImageNet. Qualitatively and quantitatively evaluations also reveal that the predictive uncertainty estimates are statistically higher for erroneous predictions than correct predictions. Accordingly, uncertainty quantification methods are capable of flagging risky predictions with high uncertainty estimates. We also observe that ensemble methods more reliably capture uncertainties during the inference. DNN-based solutions for COVID-19 detection have been mainly proposed without any principled mechanism for risk mitigation. Previous studies have mainly focused on on generating single-valued predictions using pretrained DNNs. In this paper, we comprehensively apply and comparatively evaluate three uncertainty quantification techniques for COVID-19 detection using chest X-Ray images. The novel concept of uncertainty confusion matrix is proposed and new performance metrics for the objective evaluation of uncertainty estimates are introduced for the first time. Using these new uncertainty performance metrics, we quantitatively demonstrate when we could trust DNN predictions for COVID-19 detection from chest X-rays. It is important to note the proposed novel uncertainty evaluation metrics are generic and could be applied for evaluation of probabilistic forecasts in all classification problems.

Uncertainty-aware convolutional neural network for COVID-19 X-ray images classification

Deep learning (DL) has shown great success in the field of medical image analysis. In the wake of the current pandemic situation of SARS-CoV-2, a few pioneering works based on DL have made significant progress in automated screening of COVID-19 disease from the chest X-ray (CXR) images. But these DL models have no inherent way of expressing uncertainty associated with the model's prediction, which is very important in medical image analysis. Therefore, in this paper, we develop an uncertainty-aware convolutional neural network model, named UA-ConvNet, for the automated detection of COVID-19 disease from CXR images, with an estimation of associated uncertainty in the model's predictions. The proposed approach utilizes the EfficientNet-B3 model and Monte Carlo (MC) dropout, where an EfficientNet-B3 model has been fine-tuned on the CXR images. During inference, MC dropout has been applied for M forward passes to obtain the posterior predictive distribution. After that mean and entropy have been calculated on the obtained predictive distribution to get the mean prediction and model uncertainty. The proposed method is evaluated on the three different datasets of chest X-ray images, namely the COVID19CXr, X-ray image, and Kaggle datasets. The proposed UA-ConvNet model achieves a G-mean of 98.02% (with a Confidence Interval (CI) of 97.99-98.07) and sensitivity of 98.15% for the multi-class classification task on the COVID19CXr dataset. For binary classification, the proposed model achieves a G-mean of 99.16% (with a CI of 98.81-99.19) and a sensitivity of 99.30% on the X-ray Image dataset. Our proposed approach shows its superiority over the existing methods for diagnosing the COVID-19 cases from the CXR images.

Application of artificial intelligence in diagnosing COVID-19 disease symptoms on chest X-rays: A systematic review

This systematic review focuses on using artificial intelligence (AI) to detect COVID-19 infection with the help of X-ray images. Methodology: In January 2022, the authors searched PubMed, Embase and Scopus using specific medical subject headings terms and filters. All articles were independently reviewed by two reviewers. All conflicts resulting from a misunderstanding were resolved by a third independent researcher. After assessing abstracts and article usefulness, eliminating repetitions and applying inclusion and exclusion criteria, six studies were found to be qualified for this study. Results: The findings from individual studies differed due to the various approaches of the authors. Sensitivity was 72.59%-100%, specificity was 79%-99.9%, precision was 74.74%-98.7%, accuracy was 76.18%-99.81%, and the area under the curve was 95.24%-97.7%. Conclusion: AI computational models used to assess chest X-rays in the process of diagnosing COVID-19 should achieve sufficiently high sensitivity and specificity. Their results and performance should be repeatable to make them dependable for clinicians. Moreover, these additional diagnostic tools should be more affordable and faster than the currently available procedures. The performance and calculations of AI-based systems should take clinical data into account.

CoWarriorNet: A Novel Deep-Learning Framework for CoVID-19 Detection from Chest X-Ray Images

Even after scavenging the existence of mankind for the past year, the wrath of CoVID-19 is yet to die down. Countries like India are still getting haunted by the devastating conundrum, with coronavirus ripping through its citizens in the concurrent second wave. The surge of cases has prompted rapid intervention, with medical authorities pushing it to the limit to curve a roadblock to its aggressive growth. But, even after effortless work, human intervention remains slow and insufficient. Furthermore, relevant testing methodologies have shown weakness while detecting threats, with the recent growth of post-Covid complexities, thereby leaving a painful mark. This as such created a major requirement for technological advancements, which can cater to the mass. The growth of computational prowess in the past decade made the field of Deep Learning a major contributor in curving out algorithms to solve this. Adding to the excellent foundation of Deep Learning, this paper, proposes a novel CoWarriorNet model for rapid detection of CoVID-19, via chest X-ray images, which adds in an extra layer of precision and confirmation in the detection of cases in both pre-Covid and post-Covid conditions. The proposed classification model curves out an excellent accuracy of 97.8%, with the major eye-candy being the sensitivity rate of 0.99 when detecting CoVID-19 cases. This model introduces a new concept of Alpha Trimmed Average Pooling, which along with the novel architecture adds a subtle touch to its high efficiency, thereby giving a much-needed solution to the medical experts. The two-mouthed architecture provides the added benefit of a confidence score, deducing human aid in case of discrepancy.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00354-021-00143-1.

Machine Learning for Health: Algorithm Auditing & Quality Control

Developers proposing new machine learning for health (ML4H) tools often pledge to match or even surpass the performance of existing tools, yet the reality is usually more complicated. Reliable deployment of ML4H to the real world is challenging as examples from diabetic retinopathy or Covid-19 screening show. We envision an integrated framework of algorithm auditing and quality control that provides a path towards the effective and reliable application of ML systems in healthcare. In this editorial, we give a summary of ongoing work towards that vision and announce a call for participation to the special issue  Machine Learning for Health: Algorithm Auditing & Quality Control in this journal to advance the practice of ML4H auditing.

The Applications of Artificial Intelligence in Chest Imaging of COVID-19 Patients: A Literature Review

Diagnostic imaging is regarded as fundamental in the clinical work-up of patients with a suspected or confirmed COVID-19 infection. Recent progress has been made in diagnostic imaging with the integration of artificial intelligence (AI) and machine learning (ML) algorisms leading to an increase in the accuracy of exam interpretation and to the extraction of prognostic information useful in the decision-making process. Considering the ever expanding imaging data generated amid this pandemic, COVID-19 has catalyzed the rapid expansion in the application of AI to combat disease. In this context, many recent studies have explored the role of AI in each of the presumed applications for COVID-19 infection chest imaging, suggesting that implementing AI applications for chest imaging can be a great asset for fast and precise disease screening, identification and characterization. However, various biases should be overcome in the development of further ML-based algorithms to give them sufficient robustness and reproducibility for their integration into clinical practice. As a result, in this literature review, we will focus on the application of AI in chest imaging, in particular, deep learning, radiomics and advanced imaging as quantitative CT.