Optimal uncertainty-guided neural network training

Aleatory-aware deep uncertainty quantification for transfer learning

The user does not have any idea about the credibility of outcomes from deep neural networks (DNN) when uncertainty quantification (UQ) is not employed. However, current Deep UQ classification models capture mostly epistemic uncertainty. Therefore, this paper aims to propose an aleatory-aware Deep UQ method for classification problems. First, we train DNNs through transfer learning and collect numeric output posteriors for all training samples instead of logical outputs. Then we determine the probability of happening a certain class from K-nearest output posteriors of the same DNN in training samples. We name this probability as opacity score, as the paper focuses on the detection of opacity on X-ray images. This score reflects the level of aleatory on the sample. When the NN is certain on the classification of the sample, the probability of happening a class becomes much higher than the probabilities of others. Probabilities for different classes become close to each other for a highly uncertain classification outcome. To capture the epistemic uncertainty, we train multiple DNNs with different random initializations, model selection, and augmentations to observe the effect of these training parameters on prediction and uncertainty. To reduce execution time, we first obtain features from the pre-trained NN. Then we apply features to the ensemble of fully connected layers to get the distribution of opacity score during the test. We also train several ResNet and DenseNet DNNs to observe the effect of model selection on prediction and uncertainty. The paper also demonstrates a patient referral framework based on the proposed uncertainty quantification. The scripts of the proposed method are available at the following link: https://github.com/dipuk0506/Aleatory-aware-UQ.

A novel deep neuroevolution-based image classification method to diagnose coronavirus disease (COVID-19)

COVID-19 has had a detrimental impact on normal activities, public safety, and the global financial system. To identify the presence of this disease within communities and to commence the management of infected patients early, positive cases should be diagnosed as quickly as possible. New results from X-ray imaging indicate that images provide key information about COVID-19. Advanced deep-learning (DL) models can be applied to X-ray radiological images to accurately diagnose this disease and to mitigate the effects of a shortage of skilled medical personnel in rural areas. However, the performance of DL models strongly depends on the methodology used to design their architectures. Therefore, deep neuroevolution (DNE) techniques are introduced to automatically design DL architectures accurately. In this paper, a new paradigm is proposed for the automated diagnosis of COVID-19 from chest X-ray images using a novel two-stage improved DNE Algorithm. The proposed DNE framework is evaluated on a real-world dataset and the results demonstrate that it provides the highest classification performance in terms of different evaluation metrics.

Towards a mathematical framework to inform neural network modelling via polynomial regression

Even when neural networks are widely used in a large number of applications, they are still considered as black boxes and present some difficulties for dimensioning or evaluating their prediction error. This has led to an increasing interest in the overlapping area between neural networks and more traditional statistical methods, which can help overcome those problems. In this article, a mathematical framework relating neural networks and polynomial regression is explored by building an explicit expression for the coefficients of a polynomial regression from the weights of a given neural network, using a Taylor expansion approach. This is achieved for single hidden layer neural networks in regression problems. The validity of the proposed method depends on different factors like the distribution of the synaptic potentials or the chosen activation function. The performance of this method is empirically tested via simulation of synthetic data generated from polynomials to train neural networks with different structures and hyperparameters, showing that almost identical predictions can be obtained when certain conditions are met. Lastly, when learning from polynomial generated data, the proposed method produces polynomials that approximate correctly the data locally.