Evaluation of a projection-domain lung nodule insertion technique in thoracic computed tomography

Image quality evaluation in deep-learning-based CT noise reduction using virtual imaging trial methods: Contrast-dependent spatial resolution

BACKGROUND

Deep-learning-based image reconstruction and noise reduction methods (DLIR) have been increasingly deployed in clinical CT. Accurate image quality assessment of these methods is challenging as the performance measured using physical phantoms may not represent the true performance of DLIR in patients since DLIR is trained mostly on patient images.

PURPOSE

In this work, we aim to develop a patient-data-based virtual imaging trial framework and, as a first application, use it to measure the spatial resolution properties of a DLIR method.

METHODS

The patient-data-based virtual imaging trial framework consists of five steps: (1) insertion of lesions into projection domain data using the acquisition geometry of the patient exam to simulate different lesion characteristics; (2) insertion of noise into projection domain data using a realistic photon statistical model of the CT system to simulate different dose levels; (3) creation of DLIR-processed images from projection or image data; (4) creation of ensembles of DLIR-processed patient images from a large number of noise and lesion realizations; and (5) evaluation of image quality using ensemble DLIR images. This framework was applied to measure the spatial resolution of a ResNet based deep convolutional neural network (DCNN) trained on patient images. Lesions in a cylindrical shape and different contrast levels (-500, -100, -50, -20, -10 HU) were inserted to the lower right lobe of the liver in a patient case. Multiple dose levels were simulated (50%, 25%, 12.5%). Each lesion and dose condition had 600 noise realizations. Multiple reconstruction and denoising methods were used on all the noise realizations, including the original filtered-backprojection (FBP), iterative reconstruction (IR), and the DCNN method with three different strength setting (DCNN-weak, DCNN-medium, and DCNN-strong). Mean lesion signal was calculated by performing ensemble averaging of all the noise realizations for each lesion and dose condition and then subtracting the lesion-present images from the lesion absent images. Modulation transfer functions (MTFs) both in-plane and along the z-axis were calculated based on the mean lesion signals. The standard deviations of MTFs at each condition were estimated with bootstrapping: randomly sampling (with replacement) all the DLIR/FBP/IR images from the ensemble data (600 samples) at each condition. The impact of varying lesion contrast, dose levels, and denoising strengths were evaluated. Statistical analysis with paired t-test was used to compare the z-axis and in-plane spatial resolution of five algorithms for five different contrasts and three dose levels.

RESULTS

The in-plane and z-axis spatial resolution degradation of DCNN becomes more severe as the contrast or radiation dose decreased, or DCNN denoising strength increased. In comparison with FBP, a 59.5% and 4.1% reduction of in-plane and z-axis MTF (in terms of spatial frequencies at 50% MTF), respectively, was observed at low contrast (-10 HU) for DCNN with the highest denoising strength at 25% routine dose level. When the dose level reduces from 50% to 12.5% of routine dose, the in-plane and z-axis MTFs reduces from 92.1% to 76.3%, and from 98.9% to 95.5%, respectively, at contrast of -100 HU, using FBP as the reference. For most conditions of contrasts and dose levels, significant differences were found among the five algorithms, with the following relationship in both in-plane and cross-plane spatial resolution: FBP > DCNN-Weak > IR > DCNN-Medium > DCNN-Strong. The spatial resolution difference among algorithms decreases at higher contrast or dose levels.

CONCLUSIONS

A patient-data-based virtual imaging trial framework was developed and applied to measuring the spatial resolution properties of a DCNN noise reduction method at different contrast and dose levels using real patient data. As with other non-linear image reconstruction and post-processing techniques, the evaluated DCNN method degraded the in-plane and z-axis spatial resolution at lower contrast levels, lower radiation dose, and higher denoising strength.

Observer-study-based approaches to quantitatively evaluate the realism of synthetic medical images

Objective.Synthetic images generated by simulation studies have a well-recognized role in developing and evaluating imaging systems and methods. However, for clinically relevant development and evaluation, the synthetic images must be clinically realistic and, ideally, have the same distribution as that of clinical images. Thus, mechanisms that can quantitatively evaluate this clinical realism and, ideally, the similarity in distributions of the real and synthetic images, are much needed.Approach.We investigated two observer-study-based approaches to quantitatively evaluate the clinical realism of synthetic images. In the first approach, we presented a theoretical formalism for the use of an ideal-observer study to quantitatively evaluate the similarity in distributions between the real and synthetic images. This theoretical formalism provides a direct relationship between the area under the receiver operating characteristic curve, AUC, for an ideal observer and the distributions of real and synthetic images. The second approach is based on the use of expert-human-observer studies to quantitatively evaluate the realism of synthetic images. In this approach, we developed a web-based software to conduct two-alternative forced-choice (2-AFC) experiments with expert human observers. The usability of this software was evaluated by conducting a system usability scale (SUS) survey with seven expert human readers and five observer-study designers. Further, we demonstrated the application of this software to evaluate a stochastic and physics-based image-synthesis technique for oncologic positron emission tomography (PET). In this evaluation, the 2-AFC study with our software was performed by six expert human readers, who were highly experienced in reading PET scans, with years of expertise ranging from 7 to 40 years (median: 12 years, average: 20.4 years).Main results.In the ideal-observer-study-based approach, we theoretically demonstrated that the AUC for an ideal observer can be expressed, to an excellent approximation, by the Bhattacharyya distance between the distributions of the real and synthetic images. This relationship shows that a decrease in the ideal-observer AUC indicates a decrease in the distance between the two image distributions. Moreover, a lower bound of ideal-observer AUC = 0.5 implies that the distributions of synthetic and real images exactly match. For the expert-human-observer-study-based approach, our software for performing the 2-AFC experiments is available athttps://apps.mir.wustl.edu/twoafc. Results from the SUS survey demonstrate that the web application is very user friendly and accessible. As a secondary finding, evaluation of a stochastic and physics-based PET image-synthesis technique using our software showed that expert human readers had limited ability to distinguish the real images from the synthetic images.Significance.This work addresses the important need for mechanisms to quantitatively evaluate the clinical realism of synthetic images. The mathematical treatment in this paper shows that quantifying the similarity in the distribution of real and synthetic images is theoretically possible by using an ideal-observer-study-based approach. Our developed software provides a platform for designing and performing 2-AFC experiments with human observers in a highly accessible, efficient, and secure manner. Additionally, our results on the evaluation of the stochastic and physics-based image-synthesis technique motivate the application of this technique to develop and evaluate a wide array of PET imaging methods.

A Bayesian approach to tissue-fraction estimation for oncological PET segmentation

Tumor segmentation in oncological PET is challenging, a major reason being the partial-volume effects (PVEs) that arise due to low system resolution and finite voxel size. The latter results in tissue-fraction effects (TFEs), i.e. voxels contain a mixture of tissue classes. Conventional segmentation methods are typically designed to assign each image voxel as belonging to a certain tissue class. Thus, these methods are inherently limited in modeling TFEs. To address the challenge of accounting for PVEs, and in particular, TFEs, we propose a Bayesian approach to tissue-fraction estimation for oncological PET segmentation. Specifically, this Bayesian approach estimates the posterior mean of the fractional volume that the tumor occupies within each image voxel. The proposed method, implemented using a deep-learning-based technique, was first evaluated using clinically realistic 2D simulation studies with known ground truth, in the context of segmenting the primary tumor in PET images of patients with lung cancer. The evaluation studies demonstrated that the method accurately estimated the tumor-fraction areas and significantly outperformed widely used conventional PET segmentation methods, including a U-net-based method, on the task of segmenting the tumor. In addition, the proposed method was relatively insensitive to PVEs and yielded reliable tumor segmentation for different clinical-scanner configurations. The method was then evaluated using clinical images of patients with stage IIB/III non-small cell lung cancer from ACRIN 6668/RTOG 0235 multi-center clinical trial. Here, the results showed that the proposed method significantly outperformed all other considered methods and yielded accurate tumor segmentation on patient images with Dice similarity coefficient (DSC) of 0.82 (95% CI: 0.78, 0.86). In particular, the method accurately segmented relatively small tumors, yielding a high DSC of 0.77 for the smallest segmented cross-section of 1.30 cm². Overall, this study demonstrates the efficacy of the proposed method to accurately segment tumors in PET images.

A physics-guided modular deep-learning based automated framework for tumor segmentation in PET

An important need exists for reliable positron emission tomography (PET) tumor-segmentation methods for tasks such as PET-based radiation-therapy planning and reliable quantification of volumetric and radiomic features. To address this need, we propose an automated physics-guided deep-learning-based three-module framework to segment PET images on a per-slice basis. The framework is designed to help address the challenges of limited spatial resolution and lack of clinical training data with known ground-truth tumor boundaries in PET. The first module generates PET images containing highly realistic tumors with known ground-truth using a new stochastic and physics-based approach, addressing lack of training data. The second module trains a modified U-net using these images, helping it learn the tumor-segmentation task. The third module fine-tunes this network using a small-sized clinical dataset with radiologist-defined delineations as surrogate ground-truth, helping the framework learn features potentially missed in simulated tumors. The framework was evaluated in the context of segmenting primary tumors in 18F-fluorodeoxyglucose (FDG)-PET images of patients with lung cancer. The framework's accuracy, generalizability to different scanners, sensitivity to partial volume effects (PVEs) and efficacy in reducing the number of training images were quantitatively evaluated using Dice similarity coefficient (DSC) and several other metrics. The framework yielded reliable performance in both simulated (DSC: 0.87 (95% confidence interval (CI): 0.86, 0.88)) and patient images (DSC: 0.73 (95% CI: 0.71, 0.76)), outperformed several widely used semi-automated approaches, accurately segmented relatively small tumors (smallest segmented cross-section was 1.83 cm2), generalized across five PET scanners (DSC: 0.74 (95% CI: 0.71, 0.76)), was relatively unaffected by PVEs, and required low training data (training with data from even 30 patients yielded DSC of 0.70 (95% CI: 0.68, 0.71)). In conclusion, the proposed automated physics-guided deep-learning-based PET-segmentation framework yielded reliable performance in delineating tumors in FDG-PET images of patients with lung cancer.