Objective evaluation of deep uncertainty predictions for COVID-19 detection

A review of uncertainty quantification in medical image analysis: Probabilistic and non-probabilistic methods

The comprehensive integration of machine learning healthcare models within clinical practice remains suboptimal, notwithstanding the proliferation of high-performing solutions reported in the literature. A predominant factor hindering widespread adoption pertains to an insufficiency of evidence affirming the reliability of the aforementioned models. Recently, uncertainty quantification methods have been proposed as a potential solution to quantify the reliability of machine learning models and thus increase the interpretability and acceptability of the results. In this review, we offer a comprehensive overview of the prevailing methods proposed to quantify the uncertainty inherent in machine learning models developed for various medical image tasks. Contrary to earlier reviews that exclusively focused on probabilistic methods, this review also explores non-probabilistic approaches, thereby furnishing a more holistic survey of research pertaining to uncertainty quantification for machine learning models. Analysis of medical images with the summary and discussion on medical applications and the corresponding uncertainty evaluation protocols are presented, which focus on the specific challenges of uncertainty in medical image analysis. We also highlight some potential future research work at the end. Generally, this review aims to allow researchers from both clinical and technical backgrounds to gain a quick and yet in-depth understanding of the research in uncertainty quantification for medical image analysis machine learning models.

Uncertainty quantification in multi-class segmentation: Comparison between Bayesian and non-Bayesian approaches in a clinical perspective

BACKGROUND

Automatic segmentation techniques based on Convolutional Neural Networks (CNNs) are widely adopted to automatically identify any structure of interest from a medical image, as they are not time consuming and not subject to high intra- and inter-operator variability. However, the adoption of these approaches in clinical practice is slowed down by some factors, such as the difficulty in providing an accurate quantification of their uncertainty.

PURPOSE

This work aims to evaluate the uncertainty quantification provided by two Bayesian and two non-Bayesian approaches for a multi-class segmentation problem, and to compare the risk propensity among these approaches, considering CT images of patients affected by renal cancer (RC).

METHODS

Four uncertainty quantification approaches were implemented in this work, based on a benchmark CNN currently employed in medical image segmentation: two Bayesian CNNs with different regularizations (Dropout and DropConnect), named BDR and BDC, an ensemble method (Ens) and a test-time augmentation (TTA) method. They were compared in terms of segmentation accuracy, using the Dice score, uncertainty quantification, using the ratio of correct-certain pixels (RCC) and incorrect-uncertain pixels (RIU), and with respect to inter-observer variability in manual segmentation. They were trained with the Kidney and Kidney Tumor Segmentation Challenge launched in 2021 (Kits21), for which multi-class segmentations of kidney, RC, and cyst on 300 CT volumes are available. Moreover, they were tested considering this and other two public renal CT datasets.

RESULTS

Accuracy results achieved large differences across the structures of interest for all approaches, with an average Dice score of 0.92, 0.58, and 0.21 for kidney, tumor, and cyst, respectively. In terms of uncertainties, TTA provided the highest uncertainty, followed by Ens and BDC, whereas BDR provided the lowest, and minimized the number of incorrect certain pixels worse than the other approaches. Again, large differences were seen across the three structures in terms of RCC and RIU. These metrics were associated with different risk propensity, as BDR was the most risk-taking approach, able to provide higher accuracy in its prediction, but failing to assign uncertainty on incorrect segmentation in every case. The other three approaches were more conservative, providing large uncertainty regions, with the drawback of giving alert also on correct areas. Finally, the analysis of the inter-observer segmentation variability showed a significant variation among the four approaches on the external dataset, with BDR reporting the lowest agreement (Dice = 0.82), and TTA obtaining the highest score (Dice = 0.94).

CONCLUSIONS

Our outcomes highlight the importance of quantifying the segmentation uncertainty and that decision-makers can choose the approach most in line with the risk propensity degree required by the application and their policy.

An Optimized Uncertainty-Aware Training Framework for Neural Networks

Uncertainty quantification (UQ) for predictions generated by neural networks (NNs) is of vital importance in safety-critical applications. An ideal model is supposed to generate low uncertainty for correct predictions and high uncertainty for incorrect predictions. The main focus of state-of-the-art training algorithms is to optimize the NN parameters to improve the accuracy-related metrics. Training based on uncertainty metrics has been fully ignored or overlooked in the literature. This article introduces a novel uncertainty-aware training algorithm for classification tasks. A novel predictive uncertainty estimate-based objective function is defined and optimized using the stochastic gradient descent method. This new multiobjective loss function covers both accuracy and uncertainty accuracy (UA) simultaneously during training. The performance of the proposed training framework is compared from different aspects with other UQ techniques for different benchmarks. The obtained results demonstrate the effectiveness of the proposed framework for developing the NN models capable of generating reliable uncertainty estimates.

Trustworthy clinical AI solutions: A unified review of uncertainty quantification in Deep Learning models for medical image analysis

The full acceptance of Deep Learning (DL) models in the clinical field is rather low with respect to the quantity of high-performing solutions reported in the literature. End users are particularly reluctant to rely on the opaque predictions of DL models. Uncertainty quantification methods have been proposed in the literature as a potential solution, to reduce the black-box effect of DL models and increase the interpretability and the acceptability of the result by the final user. In this review, we propose an overview of the existing methods to quantify uncertainty associated with DL predictions. We focus on applications to medical image analysis, which present specific challenges due to the high dimensionality of images and their variable quality, as well as constraints associated with real-world clinical routine. Moreover, we discuss the concept of structural uncertainty, a corpus of methods to facilitate the alignment of segmentation uncertainty estimates with clinical attention. We then discuss the evaluation protocols to validate the relevance of uncertainty estimates. Finally, we highlight the open challenges for uncertainty quantification in the medical field.

Uncertainty-aware image classification on 3D CT lung

Early detection is crucial for lung cancer to prolong the patient's survival. Existing model architectures used in such systems have shown promising results. However, they lack reliability and robustness in their predictions and the models are typically evaluated on a single dataset, making them overconfident when a new class is present. With the existence of uncertainty, uncertain images can be referred to medical experts for a second opinion. Thus, we propose an uncertainty-aware framework that includes three phases: data preprocessing and model selection and evaluation, uncertainty quantification (UQ), and uncertainty measurement and data referral for the classification of benign and malignant nodules using 3D CT images. To quantify the uncertainty, we employed three approaches; Monte Carlo Dropout (MCD), Deep Ensemble (DE), and Ensemble Monte Carlo Dropout (EMCD). We evaluated eight different deep learning models consisting of ResNet, DenseNet, and the Inception network family, all of which achieved average F1 scores above 0.832, and the highest average value of 0.845 was obtained using InceptionResNetV2. Furthermore, incorporating the UQ demonstrated significant improvement in the overall model performance. Upon evaluation of the uncertainty estimate, MCD outperforms the other UQ models except for the metric, URecall, where DE and EMCD excel, implying that they are better at identifying incorrect predictions with higher uncertainty levels, which is vital in the medical field. Finally, we show that using a threshold for data referral can greatly improve the performance further, increasing the accuracy up to 0.959.

Uncovering the effects of model initialization on deep model generalization: A study with adult and pediatric chest X-ray images

Model initialization techniques are vital for improving the performance and reliability of deep learning models in medical computer vision applications. While much literature exists on non-medical images, the impacts on medical images, particularly chest X-rays (CXRs) are less understood. Addressing this gap, our study explores three deep model initialization techniques: Cold-start, Warm-start, and Shrink and Perturb start, focusing on adult and pediatric populations. We specifically focus on scenarios with periodically arriving data for training, thereby embracing the real-world scenarios of ongoing data influx and the need for model updates. We evaluate these models for generalizability against external adult and pediatric CXR datasets. We also propose novel ensemble methods: F-score-weighted Sequential Least-Squares Quadratic Programming (F-SLSQP) and Attention-Guided Ensembles with Learnable Fuzzy Softmax to aggregate weight parameters from multiple models to capitalize on their collective knowledge and complementary representations. We perform statistical significance tests with 95% confidence intervals and p-values to analyze model performance. Our evaluations indicate models initialized with ImageNet-pretrained weights demonstrate superior generalizability over randomly initialized counterparts, contradicting some findings for non-medical images. Notably, ImageNet-pretrained models exhibit consistent performance during internal and external testing across different training scenarios. Weight-level ensembles of these models show significantly higher recall (p<0.05) during testing compared to individual models. Thus, our study accentuates the benefits of ImageNet-pretrained weight initialization, especially when used with weight-level ensembles, for creating robust and generalizable deep learning solutions.

Body composition predicts hypertension using machine learning methods: a cohort study

We used machine learning methods to investigate if body composition indices predict hypertension. Data from a cohort study was used, and 4663 records were included (2156 were male, 1099 with hypertension, with the age range of 35-70 years old). Body composition analysis was done using bioelectrical impedance analysis (BIA); weight, basal metabolic rate, total and regional fat percentage (FATP), and total and regional fat-free mass (FFM) were measured. We used machine learning methods such as Support Vector Classifier, Decision Tree, Stochastic Gradient Descend Classifier, Logistic Regression, Gaussian Naïve Bayes, K-Nearest Neighbor, Multi-Layer Perceptron, Random Forest, Gradient Boosting, Histogram-based Gradient Boosting, Bagging, Extra Tree, Ada Boost, Voting, and Stacking to classify the investigated cases and find the most relevant features to hypertension. FATP, AFFM, BMR, FFM, TRFFM, AFATP, LFATP, and older age were the top features in hypertension prediction. Arm FFM, basal metabolic rate, total FFM, Trunk FFM, leg FFM, and male gender were inversely associated with hypertension, but total FATP, arm FATP, leg FATP, older age, trunk FATP, and female gender were directly associated with hypertension. AutoMLP, stacking and voting methods had the best performance for hypertension prediction achieving an accuracy rate of 90%, 84% and 83%, respectively. By using machine learning methods, we found that BIA-derived body composition indices predict hypertension with acceptable accuracy.

Identification of Clinical Features Associated with Mortality in COVID-19 Patients

Understanding clinical features and risk factors associated with COVID-19 mortality is needed to early identify critically ill patients, initiate treatments and prevent mortality. A retrospective study on COVID-19 patients referred to a tertiary hospital in Iran between March and November 2020 was conducted. COVID-19-related mortality and its association with clinical features including headache, chest pain, symptoms on computerized tomography (CT), hospitalization, time to infection, history of neurological disorders, having a single or multiple risk factors, fever, myalgia, dizziness, seizure, abdominal pain, nausea, vomiting, diarrhoea and anorexia were investigated. Based on the investigation outcome, decision tree and dimension reduction algorithms were used to identify the aforementioned risk factors. Of the 3008 patients (mean age 59.3 ± 18.7 years, 44% women) with COVID-19, 373 died. There was a significant association between COVID-19 mortality and old age, headache, chest pain, low respiratory rate, oxygen saturation < 93%, need for a mechanical ventilator, having symptoms on CT, hospitalization, time to infection, neurological disorders, cardiovascular diseases and having a risk factor or multiple risk factors. In contrast, there was no significant association between mortality and gender, fever, myalgia, dizziness, seizure, abdominal pain, nausea, vomiting, diarrhoea and anorexia. Our results might help identify early symptoms related to COVID-19 and better manage patients according to the extracted decision tree. The proposed ML models identified a number of clinical features and risk factors associated with mortality in COVID-19 patients. These models if implemented in a clinical setting might help to early identify patients needing medical attention and care. However, more studies are needed to confirm these findings.

Evaluating deep learning predictions for COVID-19 from X-ray images using leave-one-out predictive densities

Early detection of the COVID-19 virus is an important task for controlling the spread of the pandemic. Imaging techniques such as chest X-ray are relatively inexpensive and accessible, but its interpretation requires expert knowledge to evaluate the disease severity. Several approaches for automatic COVID-19 detection using deep learning techniques have been proposed. While most approaches show high accuracy on the COVID-19 detection task, there is not enough evidence on external evaluation for this technique. Furthermore, data scarcity and sampling biases make difficult to properly evaluate model predictions. In this paper, we propose stochastic gradient Langevin dynamics (SGLD) to take into account the model uncertainty. Four different deep learning architectures are trained using SGLD and compared to their baselines using stochastic gradient descent. The model uncertainties are also evaluated according to their convergence properties and the leave-one-out predictive densities. The proposed approach is able to reduce overconfidence of the baseline estimators while also retaining predictive accuracy for the best-performing cases.

Data augmentation based semi-supervised method to improve COVID-19 CT classification

The Coronavirus (COVID-19) outbreak of December 2019 has become a serious threat to people around the world, creating a health crisis that infected millions of lives, as well as destroying the global economy. Early detection and diagnosis are essential to prevent further transmission. The detection of COVID-19 computed tomography images is one of the important approaches to rapid diagnosis. Many different branches of deep learning methods have played an important role in this area, including transfer learning, contrastive learning, ensemble strategy, etc. However, these works require a large number of samples of expensive manual labels, so in order to save costs, scholars adopted semi-supervised learning that applies only a few labels to classify COVID-19 CT images. Nevertheless, the existing semi-supervised methods focus primarily on class imbalance and pseudo-label filtering rather than on pseudo-label generation. Accordingly, in this paper, we organized a semi-supervised classification framework based on data augmentation to classify the CT images of COVID-19. We revised the classic teacher-student framework and introduced the popular data augmentation method Mixup, which widened the distribution of high confidence to improve the accuracy of selected pseudo-labels and ultimately obtain a model with better performance. For the COVID-CT dataset, our method makes precision, F1 score, accuracy and specificity 21.04%, 12.95%, 17.13% and 38.29% higher than average values for other methods respectively, For the SARS-COV-2 dataset, these increases were 8.40%, 7.59%, 9.35% and 12.80% respectively. For the Harvard Dataverse dataset, growth was 17.64%, 18.89%, 19.81% and 20.20% respectively. The codes are available at https://github.com/YutingBai99/COVID-19-SSL.

Uncertainty quantification in DenseNet model using myocardial infarction ECG signals

BACKGROUND AND OBJECTIVE

Myocardial infarction (MI) is a life-threatening condition diagnosed acutely on the electrocardiogram (ECG). Several errors, such as noise, can impair the prediction of automated ECG diagnosis. Therefore, quantification and communication of model uncertainty are essential for reliable MI diagnosis.

METHODS

A Dirichlet DenseNet model that could analyze out-of-distribution data and detect misclassification of MI and normal ECG signals was developed. The DenseNet model was first trained with the pre-processed MI ECG signals (from the best lead V6) acquired from the Physikalisch-Technische Bundesanstalt (PTB) database, using the reverse Kullback-Leibler (KL) divergence loss. The model was then tested with newly synthesized ECG signals with added em and ma noise samples. Predictive entropy was used as an uncertainty measure to determine the misclassification of normal and MI signals. Model performance was evaluated using four uncertainty metrics: uncertainty sensitivity (UNSE), uncertainty specificity (UNSP), uncertainty accuracy (UNAC), and uncertainty precision (UNPR); the classification threshold was set at 0.3.

RESULTS

The UNSE of the DenseNet model was low but increased over the studied decremental noise range (-6 to 24 dB), indicating that the model grew more confident in classifying the signals as they got less noisy. The model became more certain in its predictions from SNR values of 12 dB and 18 dB onwards, yielding UNAC values of 80% and 82.4% for em and ma noise signals, respectively. UNSP and UNPR values were close to 100% for em and ma noise signals, indicating that the model was self-aware of what it knew and didn't.

CONCLUSION

Through this work, it has been established that the model is reliable as it was able to convey when it was not confident in the diagnostic information it was presenting. Thus, the model is trustworthy and can be used in healthcare applications, such as the emergency diagnosis of MI on ECGs.

Breast Cancer Dataset, Classification and Detection Using Deep Learning

Incorporating scientific research into clinical practice via clinical informatics, which includes genomics, proteomics, bioinformatics, and biostatistics, improves patients' treatment. Computational pathology is a growing subspecialty with the potential to integrate whole slide images, multi-omics data, and health informatics. Pathology and laboratory medicine are critical to diagnosing cancer. This work will review existing computational and digital pathology methods for breast cancer diagnosis with a special focus on deep learning. The paper starts by reviewing public datasets related to breast cancer diagnosis. Additionally, existing deep learning methods for breast cancer diagnosis are reviewed. The publicly available code repositories are introduced as well. The paper is closed by highlighting challenges and future works for deep learning-based diagnosis.

Automatic diagnosis of severity of COVID-19 patients using an ensemble of transfer learning models with convolutional neural networks in CT images

Introduction: Quantification of lung involvement in COVID-19 using chest Computed tomography (CT) scan can help physicians to evaluate the progression of the disease or treatment response. This paper presents an automatic deep transfer learning ensemble based on pre-trained convolutional neural networks (CNNs) to determine the severity of COVID -19 as normal, mild, moderate, and severe based on the images of the lungs CT.

Material and methods: In this study, two different deep transfer learning strategies were used. In the first procedure, features were extracted from fifteen pre-trained CNNs architectures and then fed into a support vector machine (SVM) classifier. In the second procedure, the pre-trained CNNs were fine-tuned using the chest CT images, and then features were extracted for the purpose of classification by the softmax layer. Finally, an ensemble method was developed based on majority voting of the deep learning outputs to increase the performance of the recognition on each of the two strategies. A dataset of CT scans was collected and then labeled as normal (314), mild (262), moderate (72), and severe (35) for COVID-19 by the consensus of two highly qualified radiologists.

Results: The ensemble of five deep transfer learning outputs named EfficientNetB3, EfficientNetB4, InceptionV3, NasNetMobile, and ResNext50 in the second strategy has better results than the first strategy and also the individual deep transfer learning models in diagnosing the severity of COVID-19 with 85% accuracy.

Conclusions: Our proposed study is well suited for quantifying lung involvement of COVID-19 and can help physicians to monitor the progression of the disease.

LiteCovidNet: A lightweight deep neural network model for detection of COVID-19 using X-ray images

The syndrome called COVID-19 which was firstly spread in Wuhan, China has already been declared a globally "Pandemic." To stymie the further spread of the virus at an early stage, detection needs to be done. Artificial Intelligence-based deep learning models have gained much popularity in the detection of many diseases within the confines of biomedical sciences. In this paper, a deep neural network-based "LiteCovidNet" model is proposed that detects COVID-19 cases as the binary class (COVID-19, Normal) and the multi-class (COVID-19, Normal, Pneumonia) bifurcated based on chest X-ray images of the infected persons. An accuracy of 100% and 98.82% is achieved for binary and multi-class classification respectively which is competitive performance as compared to the other recent related studies. Hence, our methodology can be used by health professionals to validate the detection of COVID-19 infected patients at an early stage with convenient cost and better accuracy.

COVID-19 chest X-ray detection through blending ensemble of CNN snapshots

The novel COVID-19 pandemic, has effectively turned out to be one of the deadliest events in modern history, with unprecedented loss of human life, major economic and financial setbacks and has set the entire world back quite a few decades. However, detection of the COVID-19 virus has become increasingly difficult due to the mutating nature of the virus, and the rise in asymptomatic cases. To counteract this and contribute to the research efforts for a more accurate screening of COVID-19, we have planned this work. Here, we have proposed an ensemble methodology for deep learning models to solve the task of COVID-19 detection from chest X-rays (CXRs) to assist Computer-Aided Detection (CADe) for medical practitioners. We leverage the strategy of transfer learning for Convolutional Neural Networks (CNNs), widely adopted in recent literature, and further propose an efficient ensemble network for their combination. The DenseNet-201 architecture has been trained only once to generate multiple snapshots, offering diverse information about the extracted features from CXRs. We follow the strategy of decision-level fusion to combine the decision scores using the blending algorithm through a Random Forest (RF) meta-learner. Experimental results confirm the efficacy of the proposed ensemble method, as shown through impressive results upon two open access COVID-19 CXR datasets — the largest COVID-X dataset, as well as a smaller scale dataset. On the large COVID-X dataset, the proposed model has achieved an accuracy score of 94.55% and on the smaller dataset by Chowdhury et al., the proposed model has achieved a 98.13% accuracy score.

Prediction and forecasting of worldwide corona virus (COVID-19) outbreak using time series and machine learning

How will the newly discovered coronavirus (COVID-19) affect the world and what will be its global impact? For answering this question, we will require a prediction of overall recoveries and fatalities, as well as a reliable prognosis of coronavirus cases. Predicting, however, requires an ample total of past data related to it. On any particular day, the prediction is unclear since events in the future rarely repeat themselves the way that they did in the past. Furthermore, forecasts and predictions are determined by the absolute interests, accuracy of the data, and prophesied variables. In addition, psychological factors play an enormous role in how people perceive and react to the danger from the disease and therefore the fear that it is going to affect them personally. This research paper advances an unbiased method for predicting the increase of the COVID-19 employing a simple, but powerful method to do so. Assumed that the data are accurate and reliable which the longer term will still follow an equivalent disease pattern, our projections intimate with a large association. Within the COVID-19 cases were documented, in contingency, there is a steady increase. The hazards are far away from symmetric, as underestimating a pandemic's spread and failing to do enough to prevent it is far a lot worse than overspending and being too cautious when it will not be needed. This paper illustrates the timeline of a live forecasting study with huge implied implications for devising and decision-making and gives unbiased predictions on COVID-19 confirmed cases, recovered cases, deaths, and ongoing cases are shown on a continental map using data science and machine learning (ML) approaches. Utilizing these ML-based techniques, the proposed system predicts the accurate COVID-19 cases and gives better performance.

Machine learning applications for COVID-19 outbreak management

Recently, the COVID-19 epidemic has resulted in millions of deaths and has impacted practically every area of human life. Several machine learning (ML) approaches are employed in the medical field in many applications, including detecting and monitoring patients, notably in COVID-19 management. Different medical imaging systems, such as computed tomography (CT) and X-ray, offer ML an excellent platform for combating the pandemic. Because of this need, a significant quantity of study has been carried out; thus, in this work, we employed a systematic literature review (SLR) to cover all aspects of outcomes from related papers. Imaging methods, survival analysis, forecasting, economic and geographical issues, monitoring methods, medication development, and hybrid apps are the seven key uses of applications employed in the COVID-19 pandemic. Conventional neural networks (CNNs), long short-term memory networks (LSTM), recurrent neural networks (RNNs), generative adversarial networks (GANs), autoencoders, random forest, and other ML techniques are frequently used in such scenarios. Next, cutting-edge applications related to ML techniques for pandemic medical issues are discussed. Various problems and challenges linked with ML applications for this pandemic were reviewed. It is expected that additional research will be conducted in the upcoming to limit the spread and catastrophe management. According to the data, most papers are evaluated mainly on characteristics such as flexibility and accuracy, while other factors such as safety are overlooked. Also, Keras was the most often used library in the research studied, accounting for 24.4 percent of the time. Furthermore, medical imaging systems are employed for diagnostic reasons in 20.4 percent of applications.

Uncertainty Quantification in Segmenting Tuberculosis-Consistent Findings in Frontal Chest X-rays

Deep learning (DL) methods have demonstrated superior performance in medical image segmentation tasks. However, selecting a loss function that conforms to the data characteristics is critical for optimal performance. Further, the direct use of traditional DL models does not provide a measure of uncertainty in predictions. Even high-quality automated predictions for medical diagnostic applications demand uncertainty quantification to gain user trust. In this study, we aim to investigate the benefits of (i) selecting an appropriate loss function and (ii) quantifying uncertainty in predictions using a VGG16-based-U-Net model with the Monto-Carlo (MCD) Dropout method for segmenting Tuberculosis (TB)-consistent findings in frontal chest X-rays (CXRs). We determine an optimal uncertainty threshold based on several uncertainty-related metrics. This threshold is used to select and refer highly uncertain cases to an expert. Experimental results demonstrate that (i) the model trained with a modified Focal Tversky loss function delivered superior segmentation performance (mean average precision (mAP): 0.5710, 95% confidence interval (CI): (0.4021,0.7399)), (ii) the model with 30 MC forward passes during inference further improved and stabilized performance (mAP: 0.5721, 95% CI: (0.4032,0.7410), and (iii) an uncertainty threshold of 0.7 is observed to be optimal to refer highly uncertain cases.

Automated Multi-View Multi-Modal Assessment of COVID-19 Patients Using Reciprocal Attention and Biomedical Transform

Automated severity assessment of coronavirus disease 2019 (COVID-19) patients can help rationally allocate medical resources and improve patients' survival rates. The existing methods conduct severity assessment tasks mainly on a unitary modal and single view, which is appropriate to exclude potential interactive information. To tackle the problem, in this paper, we propose a multi-view multi-modal model to automatically assess the severity of COVID-19 patients based on deep learning. The proposed model receives multi-view ultrasound images and biomedical indices of patients and generates comprehensive features for assessment tasks. Also, we propose a reciprocal attention module to acquire the underlying interactions between multi-view ultrasound data. Moreover, we propose biomedical transform module to integrate biomedical data with ultrasound data to produce multi-modal features. The proposed model is trained and tested on compound datasets, and it yields 92.75% for accuracy and 80.95% for recall, which is the best performance compared to other state-of-the-art methods. Further ablation experiments and discussions conformably indicate the feasibility and advancement of the proposed model.

Improving Uncertainty Estimation With Semi-Supervised Deep Learning for COVID-19 Detection Using Chest X-Ray Images

In this work we implement a COVID-19 infection detection system based on chest X-ray images with uncertainty estimation. Uncertainty estimation is vital for safe usage of computer aided diagnosis tools in medical applications. Model estimations with high uncertainty should be carefully analyzed by a trained radiologist. We aim to improve uncertainty estimations using unlabelled data through the MixMatch semi-supervised framework. We test popular uncertainty estimation approaches, comprising Softmax scores, Monte-Carlo dropout and deterministic uncertainty quantification. To compare the reliability of the uncertainty estimates, we propose the usage of the Jensen-Shannon distance between the uncertainty distributions of correct and incorrect estimations. This metric is statistically relevant, unlike most previously used metrics, which often ignore the distribution of the uncertainty estimations. Our test results show a significant improvement in uncertainty estimates when using unlabelled data. The best results are obtained with the use of the Monte Carlo dropout method.

Handling of uncertainty in medical data using machine learning and probability theory techniques: a review of 30 years (1991-2020)

Understanding the data and reaching accurate conclusions are of paramount importance in the present era of big data. Machine learning and probability theory methods have been widely used for this purpose in various fields. One critically important yet less explored aspect is capturing and analyzing uncertainties in the data and model. Proper quantification of uncertainty helps to provide valuable information to obtain accurate diagnosis. This paper reviewed related studies conducted in the last 30 years (from 1991 to 2020) in handling uncertainties in medical data using probability theory and machine learning techniques. Medical data is more prone to uncertainty due to the presence of noise in the data. So, it is very important to have clean medical data without any noise to get accurate diagnosis. The sources of noise in the medical data need to be known to address this issue. Based on the medical data obtained by the physician, diagnosis of disease, and treatment plan are prescribed. Hence, the uncertainty is growing in healthcare and there is limited knowledge to address these problems. Our findings indicate that there are few challenges to be addressed in handling the uncertainty in medical raw data and new models. In this work, we have summarized various methods employed to overcome this problem. Nowadays, various novel deep learning techniques have been proposed to deal with such uncertainties and improve the performance in decision making.