On the robustness of deep learning-based lung-nodule classification for CT images with respect to image noise

Robustness evaluation of deep neural networks for endoscopic image analysis: Insights and strategies

Computer-aided detection and diagnosis systems (CADe/CADx) in endoscopy are commonly trained using high-quality imagery, which is not representative for the heterogeneous input typically encountered in clinical practice. In endoscopy, the image quality heavily relies on both the skills and experience of the endoscopist and the specifications of the system used for screening. Factors such as poor illumination, motion blur, and specific post-processing settings can significantly alter the quality and general appearance of these images. This so-called domain gap between the data used for developing the system and the data it encounters after deployment, and the impact it has on the performance of deep neural networks (DNNs) supportive endoscopic CAD systems remains largely unexplored. As many of such systems, for e.g. polyp detection, are already being rolled out in clinical practice, this poses severe patient risks in particularly community hospitals, where both the imaging equipment and experience are subject to considerable variation. Therefore, this study aims to evaluate the impact of this domain gap on the clinical performance of CADe/CADx for various endoscopic applications. For this, we leverage two publicly available data sets (KVASIR-SEG and GIANA) and two in-house data sets. We investigate the performance of commonly-used DNN architectures under synthetic, clinically calibrated image degradations and on a prospectively collected dataset including 342 endoscopic images of lower subjective quality. Additionally, we assess the influence of DNN architecture and complexity, data augmentation, and pretraining techniques for improved robustness. The results reveal a considerable decline in performance of 11.6% (±1.5) as compared to the reference, within the clinically calibrated boundaries of image degradations. Nevertheless, employing more advanced DNN architectures and self-supervised in-domain pre-training effectively mitigate this drop to 7.7% (±2.03). Additionally, these enhancements yield the highest performance on the manually collected test set including images with lower subjective quality. By comprehensively assessing the robustness of popular DNN architectures and training strategies across multiple datasets, this study provides valuable insights into their performance and limitations for endoscopic applications. The findings highlight the importance of including robustness evaluation when developing DNNs for endoscopy applications and propose strategies to mitigate performance loss.

An Interpretable Three-Dimensional Artificial Intelligence Model for Computer-Aided Diagnosis of Lung Nodules in Computed Tomography Images

Lung cancer is typically classified into small-cell carcinoma and non-small-cell carcinoma. Non-small-cell carcinoma accounts for approximately 85% of all lung cancers. Low-dose chest computed tomography (CT) can quickly and non-invasively diagnose lung cancer. In the era of deep learning, an artificial intelligence (AI) computer-aided diagnosis system can be developed for the automatic recognition of CT images of patients, creating a new form of intelligent medical service. For many years, lung cancer has been the leading cause of cancer-related deaths in Taiwan, with smoking and air pollution increasing the likelihood of developing the disease. The incidence of lung adenocarcinoma in never-smoking women has also increased significantly in recent years, resulting in an important public health problem. Early detection of lung cancer and prompt treatment can help reduce the mortality rate of patients with lung cancer. In this study, an improved 3D interpretable hierarchical semantic convolutional neural network named HSNet was developed and validated for the automatic diagnosis of lung cancer based on a collection of lung nodule images. The interpretable AI model proposed in this study, with different training strategies and adjustment of model parameters, such as cyclic learning rate and random weight averaging, demonstrated better diagnostic performance than the previous literature, with results of a four-fold cross-validation procedure showing calcification: 0.9873 ± 0.006, margin: 0.9207 ± 0.009, subtlety: 0.9026 ± 0.014, texture: 0.9685 ± 0.006, sphericity: 0.8652 ± 0.021, and malignancy: 0.9685 ± 0.006.

Noise-robustness test for ultrasound breast nodule neural network models as medical devices

Background

Deep learning technology has been widely applied to medical image analysis. But due to the limitations of its own imaging principle, ultrasound image has the disadvantages of low resolution and high Speckle Noise density, which not only hinder the diagnosis of patients' conditions but also affect the extraction of ultrasound image features by computer technology.

Objective

In this study, we investigate the robustness of deep convolutional neural network (CNN) for classification, segmentation, and target detection of breast ultrasound image through random Salt & Pepper Noise and Gaussian Noise.

Methods

We trained and validated 9 CNN architectures in 8617 breast ultrasound images, but tested the models with noisy test set. Then, we trained and validated 9 CNN architectures with different levels of noise in these breast ultrasound images, and tested the models with noisy test set. Diseases of each breast ultrasound image in our dataset were annotated and voted by three sonographers based on their malignancy suspiciousness. we use evaluation indexes to evaluate the robustness of the neural network algorithm respectively.

Results

There is a moderate to high impact (The accuracy of the model decreased by about 5%-40%) on model accuracy when Salt and Pepper Noise, Speckle Noise, or Gaussian Noise is introduced to the images respectively. Consequently, DenseNet, UNet++ and Yolov5 were selected as the most robust model based on the selected index. When any two of these three kinds of noise are introduced into the image at the same time, the accuracy of the model will be greatly affected.

Conclusions

Our experimental results reveal new insights: The variation trend of accuracy with the noise level in Each network used for classification tasks and object detection tasks has some unique characteristics. This finding provides us with a method to reveal the black-box architecture of computer-aided diagnosis (CAD) systems. On the other hand, the purpose of this study is to explore the impact of adding noise directly to the image on the performance of neural networks, which is different from the existing articles on robustness in the field of medical image processing. Consequently, it provides a new way to evaluate the robustness of CAD systems in the future.

A Series-Based Deep Learning Approach to Lung Nodule Image Classification

Although many studies have shown that deep learning approaches yield better results than traditional methods based on manual features, CADs methods still have several limitations. These are due to the diversity in imaging modalities and clinical pathologies. This diversity creates difficulties because of variation and similarities between classes. In this context, the new approach from our study is a hybrid method that performs classifications using both medical image analysis and radial scanning series features. Hence, the areas of interest obtained from images are subjected to a radial scan, with their centers as poles, in order to obtain series. A U-shape convolutional neural network model is then used for the 4D data classification problem. We therefore present a novel approach to the classification of 4D data obtained from lung nodule images. With radial scanning, the eigenvalue of nodule images is captured, and a powerful classification is performed. According to our results, an accuracy of 92.84% was obtained and much more efficient classification scores resulted as compared to recent classifiers.

Image Recognition of Pediatric Pneumonia Based on Fusion of Texture Features and Depth Features

Pneumonia is one of the diseases that seriously endangers human health, and it is also the leading cause of death of children under the age of five in China. The most commonly used imaging examination method for radiologists is mainly based on chest X-ray images. Still, imaging errors often result during imaging examinations due to objective factors such as visual fatigue and lack of experience. Therefore, this paper proposes a feature fusion model, FC-VGG, based on the fusion of texture features (local binary pattern LBP and directional gradient histogram HOG) and depth features. The model improves model performance by adding detailed information in texture features to the convolutional neural network while making the model more suitable for clinical use. We input the X-ray image with texture features into the modified VGG16 model, C-VGG, and then add the Add fusion method to C-VGG for feature fusion so that FC-VGG is obtained, so FC-VGG has texture features detailed information and abstract information of deep features. Through experiments, our model has achieved 92.19% accuracy in recognizing children's pneumonia images, 93.44% average precision, 92.19% average recall, and 92.81% average F1 coefficient, and the model performance exceeds existing deep learning models and traditional feature recognition algorithms.

Improving small animal cone beam CT resolution by mitigating x-ray focal spot induced blurring via deconvolution

Objective. Modern preclinical small animal radiation platforms utilize cone beam computerized tomography (CBCT) for image guidance and experiment planning purposes. The resolution of CBCT images is of particular importance for visualizing fine animal anatomical structures. One major cause of spatial resolution reduction is the finite size of the x-ray focal spot. In this work, we proposed a simple method to measure x-ray focal spot intensity map and a CBCT image domain deblurring model to mitigate the effect of focal spot-induced image blurring. Approach. We measured a projection image of a tungsten ball bearing using the flat panel detector of the CBCT platform. We built a forward blurring model of the projection image and derived the spot intensity map by deconvolving the measured projection image. Based on the measured spot intensity map, we derived a CBCT image domain blurring model for images reconstructed by the filtered backprojection algorithm. Based on this model, we computed image domain blurring kernel and improved the CBCT image resolution by deconvolving the CBCT image. Main results. We successfully measured the x-ray focal spot intensity map. The spot size characterized by full width at half maximum was ∼0.75 × 0.55 mm2 at 40 kVp. We computed image domain convolution kernels caused by the x-ray focal spot. A simulation study on noiseless projections was performed to evaluate the spatial resolution improvement exclusively by the focal spot kernel, and the modulation transfer function (MTF) at 50% was increased from 1.40 to 1.65 mm−1 for in-plane images and 1.05–1.32 mm−1 for cross-plane images. Experimental studies on a CT insert phantom and a plastinated mouse phantom demonstrated improved spatial resolution after image domain deconvolution, as indicated by visually improved resolution of fine structures. MTF at 50% was improved from 1.00 to 1.12 mm−1 for in-plane direction and from 0.72 to 0.84 mm−1 for cross-plane direction. Significance. The proposed method to mitigate blurring caused by finite x-ray spot size and improve CBCT image resolution is simple and effective.

[Performance of low-dose CT image reconstruction for detecting intracerebral hemorrhage: selection of dose, algorithms and their combinations]

OBJECTIVE

To investigate the performance of different low-dose CT image reconstruction algorithms for detecting intracerebral hemorrhage.

METHODS

Low-dose CT imaging simulation was performed on CT images of intracerebral hemorrhage at 30%, 25% and 20% of normal dose level (defined as 100% dose). Seven algorithms were tested to reconstruct low-dose CT images for noise suppression, including filtered back projection algorithm (FBP), penalized weighted least squares-total variation (PWLS-TV), non-local mean filter (NLM), block matching 3D (BM3D), residual encoding-decoding convolutional neural network (REDCNN), the FBP convolutional neural network (FBPConvNet) and image restoration iterative residual convolutional network (IRLNet). A deep learning-based model (CNN-LSTM) was used to detect intracerebral hemorrhage on normal dose CT images and low-dose CT images reconstructed using the 7 algorithms. The performance of different reconstruction algorithms for detecting intracerebral hemorrhage was evaluated by comparing the results between normal dose CT images and low-dose CT images.

RESULTS

At different dose levels, the low-dose CT images reconstructed by FBP had accuracies of detecting intracerebral hemorrhage of 82.21%, 74.61% and 65.55% at 30%, 25% and 20% dose levels, respectively. At the same dose level (30% dose), the images reconstructed by FBP, PWLS-TV, NLM, BM3D, REDCNN, FBPConvNet and IRLNet algorithms had accuracies for detecting intracerebral hemorrhage of 82.21%, 86.80%, 89.37%, 81.43%, 90.05%, 90.72% and 93.51%, respectively. The images reconstructed by IRLNet at 30%, 25% and 20% dose levels had accuracies for detecting intracerebral hemorrhage of 93.51%, 93.51% and 93.06%, respectively.

CONCLUSION

The performance of reconstructed low-dose CT images for detecting intracerebral hemorrhage is significantly affected by both dose and reconstruction algorithms. In clinical practice, choosing appropriate dose level and reconstruction algorithm can greatly reduce the radiation dose and ensure the detection performance of CT imaging for intracerebral hemorrhage.

Improving robustness of a deep learning-based lung-nodule classification model of CT images with respect to image noise

Objective. Robustness is an important aspect to consider, when developing methods for medical image analysis. This study investigated robustness properties of deep neural networks (DNNs) for a lung nodule classification problem based on CT images and proposed a solution to improve robustness.Approach. We firstly constructed a class of four DNNs with different widths, each predicting an output label (benign or malignant) for an input CT image cube containing a lung nodule. These networks were trained to achieve Area Under the Curve of 0.891-0.914 on a testing dataset. We then added to the input CT image cubes noise signals generated randomly using a realistic CT image noise model based on a noise power spectrum at 100 mAs, and monitored the DNNs output change. We definedSAR₅(%) to quantify the robustness of the trained DNN model, indicating that for 5% of CT image cubes, the noise can change the prediction results with a chance of at leastSAR₅(%). To understand robustness, we viewed the information processing pipeline by the DNN as a two-step process, with the first step using all but the last layers to extract representations of the input CT image cubes in a latent space, and the second step employing the last fully-connected layer as a linear classifier to determine the position of the sample representations relative to a decision plane. To improve robustness, we proposed to retrain the last layer of the DNN with a Supporting Vector Machine (SVM) hinge loss function to enforce the desired position of the decision plane.Main results.SAR₅ranged in 47.0%-62.0% in different DNNs. The unrobustness behavior may be ascribed to the unfavorable placement of the decision plane in the latent representation space, which made some samples be perturbed to across the decision plane and hence susceptible to noise. The DNN-SVM model improved robustness over the DNN model and reducedSAR₅by 8.8%-21.0%.Significance. This study provided insights about the potential reason for the unrobustness behavior of DNNs and the proposed DNN-SVM model improved model robustness.

3D axial-attention for lung nodule classification

PURPOSE

In recent years, Non-Local-based methods have been successfully applied to lung nodule classification. However, these methods offer 2D attention or limited 3D attention to low-resolution feature maps. Moreover, they still depend on a convenient local filter such as convolution as full 3D attention is expensive to compute and requires a big dataset, which might not be available.

METHODS

We propose to use 3D Axial-Attention, which requires a fraction of the computing power of a regular Non-Local network (i.e., self-attention). Unlike a regular Non-Local network, the 3D Axial-Attention network applies the attention operation to each axis separately. Additionally, we solve the invariant position problem of the Non-Local network by proposing to add 3D positional encoding to shared embeddings.

RESULTS

We validated the proposed method on 442 benign nodules and 406 malignant nodules, extracted from the public LIDC-IDRI dataset by following a rigorous experimental setup using only nodules annotated by at least three radiologists. Our results show that the 3D Axial-Attention model achieves state-of-the-art performance on all evaluation metrics, including AUC and Accuracy.

CONCLUSIONS

The proposed model provides full 3D attention, whereby every element (i.e., pixel) in the 3D volume space attends to every other element in the nodule effectively. Thus, the 3D Axial-Attention network can be used in all layers without the need for local filters. The experimental results show the importance of full 3D attention for classifying lung nodules.