Comparison between human and model observer performance in low-contrast detection tasks in CT images: application to images reconstructed with filtered back projection and iterative algorithms

Automatic chest computed tomography image noise quantification using deep learning

PURPOSE

This study aimed to develop a deep learning (DL) method for noise quantification for clinical chest computed tomography (CT) images without the need for repeated scanning or homogeneous tissue regions.

METHODS

A comprehensive phantom CT dataset (three dose levels, six reconstruction methods, amounting to 9240 slices) was acquired and used to train a convolutional neural network (CNN) to output an estimate of local image noise standard deviations (SD) from a single CT scan input. The CNN model consisting of seven convolutional layers was trained on the phantom image dataset representing a range of scan parameters and was tested with phantom images acquired in a variety of different scan conditions, as well as publicly available chest CT images to produce clinical noise SD maps.

RESULTS

Noise SD maps predicted by the CNN agreed well with the ground truth both visually and numerically in the phantom dataset (errors of < 5 HU for most scan parameter combinations). In addition, the noise SD estimates obtained from clinical chest CT images were similar to running-average based reference estimates in areas without prominent tissue interfaces.

CONCLUSIONS

Predicting local noise magnitudes without the need for repeated scans is feasible using DL. Our implementation trained with phantom data was successfully applied to open-source clinical data with heterogeneous tissue borders and textures. We suggest that automatic DL noise mapping from clinical patient images could be used as a tool for objective CT image quality estimation and protocol optimization.

Dose-efficiency quantification of computed tomography systems using a model-observer

BACKGROUND

Recent advances in computed tomography (CT) technology have considerably improved the quality of CT images and reduced radiation exposure in patients. At present, however, there is no generally accepted figure of merit (FOM) for comparing the dose efficiencies of CT systems.

PURPOSE

(i) To establish an FOM that characterizes the quality of CT images in relation to the radiation dose by means of a mathematical model observer and (ii) to evaluate the new FOM on different CT systems and image reconstruction algorithms.

METHODS

Images of a homogeneous phantom with four low-contrast inserts were acquired using three different CT systems at three dose levels and a representative protocol for CT imaging of low-contrast objects in the abdomen. The images were reconstructed using filtered-back projection and iterative algorithms. A channelized hotelling observer with difference-of-Gaussian channels was applied to compute the detectability (d ' $d^{\prime}$ ). This was done for each insert and each of the considered imaging conditions from square regions of interest (ROIs) that were (semi-)automatically centered on the inserts. The estimated detectabilities (d ' $d^{\prime}$ ) were averaged in the first step over the three dose levels (⟨ d ' ⟩ $\langle {d^{\prime}} \rangle $ ), and subsequently over the four contrast inserts (⟨ d ' ⟩ w ${\langle {d^{\prime}} \rangle _{\rm{w}}}$ ). All calculation steps included a dedicated assessment of the related uncertainties following accepted metrological guidelines.

RESULTS

The determined detectabilities (d ' $d^{\prime}$ ) varied considerably with the contrast and diameter of the four inserts, as well as with the radiation doses and reconstruction algorithms used for image generation (d ' $d^{\prime}\;$ = 1.3-5.5). Thus, the specification of a single detectability as an FOM is not well suited for comprehensively characterizing the dose efficiency of a CT system. A more comprehensive and robust characterization was provided by the averaged detectabilities⟨ d ' ⟩ $\langle {d^{\prime}} \rangle $ and, in particular,⟨ d ' ⟩ w ${\langle {d^{\prime}} \rangle _{\rm{w}}}$ . Our analysis reveals that the model observer analysis is very sensitive to the exact position of the ROIs.

CONCLUSIONS

The presented automatable software approach yielded with the weighted detectability⟨ d ' ⟩ w ${\langle {d^{\prime}} \rangle _{\rm{w}}}$ an objective FOM to benchmark different CT systems and reconstruction algorithms in a robust and reliable manner. An essential advantage of the proposed model-observer approach is that uncertainties in the FOM can be provided, which is an indispensable prerequisite for type testing.

Automatic head computed tomography image noise quantification with deep learning

PURPOSE

Computed tomography (CT) image noise is usually determined by standard deviation (SD) of pixel values from uniform image regions. This study investigates how deep learning (DL) could be applied in head CT image noise estimation.

METHODS

Two approaches were investigated for noise image estimation of a single acquisition image: direct noise image estimation using supervised DnCNN convolutional neural network (CNN) architecture, and subtraction of a denoised image estimated with denoising UNet-CNN experimented with supervised and unsupervised noise2noise training approaches. Noise was assessed with local SD maps using 3D- and 2D-CNN architectures. Anthropomorphic phantom CT image dataset (N = 9 scans, 3 repetitions) was used for DL-model comparisons. Mean square error (MSE) and mean absolute percentage errors (MAPE) of SD values were determined using the SD values of subtraction images as ground truth. Open-source clinical head CT low-dose dataset (N_train = 37, N_test = 10 subjects) were used to demonstrate DL applicability in noise estimation from manually labeled uniform regions and in automated noise and contrast assessment.

RESULTS

The direct SD estimation using 3D-CNN was the most accurate assessment method when comparing in phantom dataset (MAPE = 15.5%, MSE = 6.3HU). Unsupervised noise2noise approach provided only slightly inferior results (MAPE = 20.2%, MSE = 13.7HU). 2DCNN and unsupervised UNet models provided the smallest MSE on clinical labeled uniform regions.

CONCLUSIONS

DL-based clinical image assessment is feasible and provides acceptable accuracy as compared to true image noise. Noise2noise approach may be feasible in clinical use where no ground truth data is available. Noise estimation combined with tissue segmentation may enable more comprehensive image quality characterization.

Deep-learning model observer for a low-contrast hepatic metastases localization task in computed tomography

PURPOSE

Conventional model observers (MO) in CT are often limited to a uniform background or varying background that is random and can be modeled in an analytical form. It is unclear if these conventional MOs can be readily generalized to predict human observer performance in clinical CT tasks that involve realistic anatomical background. Deep-learning-based model observers (DL-MO) have recently been developed, but have not been validated for challenging low contrast diagnostic tasks in abdominal CT. We consequently sought to validate a DL-MO for a low-contrast hepatic metastases localization task.

METHODS

We adapted our recently developed DL-MO framework for the liver metastases localization task. Our previously-validated projection-domain lesion-/noise-insertion techniques were used to synthesize realistic positive and low-dose abdominal CT exams, using the archived patient projection data. Ten experimental conditions were generated, which involved different lesion sizes/contrasts, radiation dose levels, and image reconstruction types. Each condition included 100 trials generated from a patient cohort of 7 cases. Each trial was presented as liver image patches (160×160×5 voxels). The DL-MO performance was calculated for each condition and was compared with human observer performance, which was obtained by three sub-specialized radiologists in an observer study. The performance of DL-MO and radiologists was gauged by the area under localization receiver-operating-characteristic curves. The generalization performance of the DL-MO was estimated with the repeated twofold cross-validation method over the same set of trials used in the human observer study. A multi-slice Channelized Hoteling Observers (CHO) was compared with the DL-MO across the same experimental conditions.

RESULTS

The performance of DL-MO was highly correlated to that of radiologists (Pearson's correlation coefficient: 0.987; 95% CI: [0.942, 0.997]). The performance level of DL-MO was comparable to that of the grouped radiologists, that is, the mean performance difference was -3.3%. The CHO performance was poorer than the grouped radiologist performance, before internal noise could be added. The correlation between CHO and radiologists was weaker (Pearson's correlation coefficient: 0.812, and 95% CI: [0.378, 0.955]), and the corresponding performance bias (-29.5%) was statistically significant.

CONCLUSION

The presented study demonstrated the potential of using the DL-MO for image quality assessment in patient abdominal CT tasks.

Toward a comparison and an optimization of CT protocols using new metrics of dose and image quality part I: prediction of human observers using a model observer for detection and discrimination tasks in low-dose CT images in various scanning conditions

In the context of reducing the patient dose coming from CT scanner examinations without penalizing the diagnosis, the assessment of both patient dose and image quality (IQ) with relevant metrics is crucial. The present study represents the first stage in a larger work, aiming to compare and optimize CT protocols using dose and IQ new metrics. We proposed here to evaluate the capacity of the Non-PreWhitening matched filter with an eye (NPWE) model observer to be a robust and accurate estimation of IQ. We focused our work on two types of clinical tasks: a low contrast detection task and a discrimination task. We designed a torso-shaped phantom, including Plastic Water^®slabs with cylindrical inserts of different diameters, sections and compositions. We led a human observer study with 13 human observers on images acquired in multiple irradiation and reconstruction scanning conditions (voltage, pitch, slice thickness, noise level of the reconstruction algorithm, energy level in dual-energy mode and dose), to evaluate the behavior of the model observer compared to the human responses faced to changing conditions. The model observer presented the same trends as the human observers with generally better results. We rescaled the NPWE model on the human responses by scanning conditions (kVp, pitch, slice thickness) to obtain the best agreement between both observer types, estimated using the Bland-Altman method. The impact of some scanning parameters was estimated using the correct answer rate given by the rescaled NPWE model, for both tasks and each insert size. In particular, the comparison between the dual-energy mode at 74 keV and the single-energy mode at 120 kVp showed that, if the 120 kVp voltage provided better results for the smallest insert at the lower doses for both tasks, their responses were equivalent in many cases.

Development of a method to create uniform phantoms for task-based assessment of CT image quality

Purpose

To develop a customized method to produce uniform phantoms for task‐based assessment of CT image quality.

Methods

Contrasts between polymethyl methacrylate (PMMA) and fructose solutions of different concentrations (240, 250, 260, 280, 290, 300, 310, 320, 330, and 340 mg/mL) were calculated. A phantom was produced by laser cutting PMMA slabs to the shape of a patient’s neck. An opening of 10 mm diameter was cut into the left parapharyngeal space. An angioplasty balloon was inserted and filled with the fructose solutions to simulate low‐contrast lesions. The phantom was scanned with six tube currents. Images were reconstructed with filtered back projection (FBP) and adaptive iterative dose reduction 3D (AIDR 3D). Calculated and measured contrasts were compared. The phantom was evaluated in a detectability experiment using images with 4 and 20 HU lesion contrast.

Results

Low‐contrast lesions of 4, 9, 11, 13, 18, 20, 24, 30, 35, and 37 HU contrast were simulated. Calculated and measured contrasts correlated excellently (r = 0.998; 95% confidence interval: 0.991 to 1). The mean ± SD difference was 0.41 ± 2.32 HU (P < 0.0001). Detection accuracy and reader confidence were 62.9 ± 18.2% and 1.58 ± 0.68 for 4 HU lesion contrast and 99.6 ± 1.3% and 4.27 ± 0.92 for 20 HU lesion contrast (P < 0.0001), confirming that the method produced lesions at the threshold of detectability.

Conclusion

A cost‐effective and flexible approach was developed to create uniform phantoms with low‐contrast signals. The method should facilitate access to customized phantoms for task‐based image quality assessment.

RDI[Formula: see text]a regression detectability index for quality assurance in: x-ray imaging

Novel iterative image reconstruction methods can help reduce the required radiation dose in x-ray diagnostics such as computed tomography (CT), while maintaining sufficient image quality. Since some of the established image quality measures are not appropriate for reliably judging the quality of images derived by iterative methods, alternative approaches such as task-specific quality assessment would be highly desirable for acceptance or constancy testing. Task-based image quality methods are also closer to tasks performed by the radiologists, such as lesion detection. However, this approach is usually hampered by a huge workload, since hundreds of images are usually required for its application. It is demonstrated that the proposed approach works reliably on the basis of significantly fewer images, and that it correlates well with results obtained from human observers.

Evaluation of low-contrast detectability for iterative reconstruction in pediatric abdominal computed tomography: a phantom study

BACKGROUND

Iterative reconstruction is offered by all vendors to achieve similar or better CT image quality at lower doses than images reconstructed with filtered back-projection.

OBJECTIVE

The purpose of this study was to investigate the dose-reduction potential for pediatric abdominal CT imaging when using either a commercially available hybrid or a commercially available model-based iterative reconstruction algorithm from a single manufacturer.

MATERIALS AND METHODS

A phantom containing four low-contrast inserts and a uniform background with total attenuation equivalent to the abdomen of an average 8-year-old child was imaged on a CT scanner (IQon; Philips Healthcare, Cleveland, OH). We reconstructed images using both hybrid iterative reconstruction (iDose⁴) and model-based iterative reconstruction (Iterative Model Reconstruction). The four low-contrast inserts had circular cross-section with diameters of 3 mm, 5 mm, 7 mm and 10 mm and contrasts of 14 Hounsfield units (HU), 7 HU, 5 HU and 3 HU, respectively. Helical scans with identical kilovoltage (kV), pitch, rotation time, and collimation were repeated on the phantom at volume CT dose index (CTDI_vol) of 2.0 milligrays (mGy), 3.0 mGy, 4.5 mGy and 6.0 mGy. We measured the contrast-to-noise ratio (CNR) in each rod across scans. Additionally, we collected sub-images containing each rod and sub-images containing the background and used them in two-alternative forced choice observer experiments with four observers (two radiologists and two physicists). We calculated the dose-reduction potential of both iterative reconstruction algorithms relative to a scan performed at 6 mGy and reconstructed with filtered back-projection.

RESULTS

We calculated dose-reduction potential by either matching average equal observer performance in the two-alternative forced choice experiments or matching CNR. When matching CNR, the dose-reduction potential was 34% to 45% for hybrid iterative reconstruction and 89% to 95% for model-based iterative reconstruction. When matching average observer performance, the dose-reduction potential was 9% to 30% for hybrid iterative reconstruction and 57% to 74% for model-based iterative reconstruction. The range in dose-reduction potential depended on the rod size and contrast level.

CONCLUSION

Observer performance in this phantom study indicates that the dose-reduction potential indicated by an observer study is less than that of CNR; extrapolating the results to clinical studies suggests that the dose-reduction potential would also be less.

Development of a computational tool for estimating computed tomography dose parameters

BACKGROUND

Computed Tomographic (CT) imaging procedures have been reported as the main source of radiation in diagnostic procedures compared to other modalities. To provide the optimal quality of CT images at the minimum radiation risk to the patient, periodic inspections and calibration tests for CT equipment are required. These tests involve a series of measurements that are time consuming and may require specific skills and highly-trained personnel.

OBJECTIVE

This study aims to develop a new computational tool to estimate the dose of CT radiation outputs and assist in the calibration of CT scanners. It may also provide an educational resource by which radiological practitioners can learn the influence of technique factors on both patient radiation dose and the produced image quality.

METHODS

The computational tool was developed using MATLAB in order to estimate the CT radiation dose parameters for different technique factors. The CT radiation dose parameters were estimated from the calibrated energy spectrum of the x-ray tube for a CT scanner.

RESULTS

The estimated dose parameters and the measured values utilising an Adult CT Head Dose Phantom showed linear correlations for different tube voltages (80 kVp, 100 kVp, 120 kVp, and 140 kVp), with R2 nearly equal to 1 (0.99). The maximum differences between the estimated and measured CTDIvol were under 5 %. For 80 kVp and low tube currents (50 mA, 100 mA), the maximum differences were under 10%.

CONCLUSIONS

The prototyped computational model provides a tool for the simulation of a machine-specific spectrum and CT dose parameters using a single dose measurement.

Evaluation of Adaptive Statistical Iterative Reconstruction-V Reconstruction Algorithm vs Filtered Back Projection in the Detection of Hypodense Liver Lesions: Reader Performance and Preferences

OBJECTIVE

The aim of the study was to evaluate diagnostic accuracy and readers' experience in the detection of focal liver lesions on computed tomography with Adaptive Statistical Iterative Reconstruction-V (ASIR-V) reconstruction compared with filtered back projection (FBP) scans.

METHODS

Fifty-five patients with liver lesions had FBP and ASIR-V scans. Two radiologists independently reviewed both sets of computed tomography scans, identifying and characterizing liver lesions.

RESULTS

Adaptive Statistical Iterative Reconstruction-V scans had a reduction in dose length product (P < 0.0001) with no difference in image contrast (P = 0.1805); image noise was less for the ASIR-V scans (P < 0.0001) and contrast-to-noise ratio was better for ASIR-V (P = 0.0002). Both readers found more hypodense liver lesions on the FBP (P = 0.01) scans. Multiple subjective imaging scores were significantly less for the ASIR-V scans for both readers.

CONCLUSIONS

Although ASIR-V scans were objectively better, our readers performed worse in lesion detection on them, suggesting a need for better education/experience with this technology during implementation.

A simple method for low-contrast detectability, image quality and dose optimisation with CT iterative reconstruction algorithms and model observers

BACKGROUND

The aim of this work was to evaluate detection of low-contrast objects and image quality in computed tomography (CT) phantom images acquired at different tube loadings (i.e. mAs) and reconstructed with different algorithms, in order to find appropriate settings to reduce the dose to the patient without any image detriment.

METHODS

Images of supraslice low-contrast objects of a CT phantom were acquired using different mAs values. Images were reconstructed using filtered back projection (FBP), hybrid and iterative model-based methods. Image quality parameters were evaluated in terms of modulation transfer function; noise, and uniformity using two software resources. For the definition of low-contrast detectability, studies based on both human (i.e. four-alternative forced-choice test) and model observers were performed across the various images.

RESULTS

Compared to FBP, image quality parameters were improved by using iterative reconstruction (IR) algorithms. In particular, IR model-based methods provided a 60% noise reduction and a 70% dose reduction, preserving image quality and low-contrast detectability for human radiological evaluation. According to the model observer, the diameters of the minimum detectable detail were around 2 mm (up to 100 mAs). Below 100 mAs, the model observer was unable to provide a result.

CONCLUSION

IR methods improve CT protocol quality, providing a potential dose reduction while maintaining a good image detectability. Model observer can in principle be useful to assist human performance in CT low-contrast detection tasks and in dose optimisation.

Quality control in cone-beam computed tomography (CBCT) EFOMP-ESTRO-IAEA protocol (summary report)

The aim of the guideline presented in this article is to unify the test parameters for image quality evaluation and radiation output in all types of cone-beam computed tomography (CBCT) systems. The applications of CBCT spread over dental and interventional radiology, guided surgery and radiotherapy. The chosen tests provide the means to objectively evaluate the performance and monitor the constancy of the imaging chain. Experience from all involved associations has been collected to achieve a consensus that is rigorous and helpful for the practice. The guideline recommends to assess image quality in terms of uniformity, geometrical precision, voxel density values (or Hounsfield units where available), noise, low contrast resolution and spatial resolution measurements. These tests usually require the use of a phantom and evaluation software. Radiation output can be determined with a kerma-area product meter attached to the tube case. Alternatively, a solid state dosimeter attached to the flat panel and a simple geometric relationship can be used to calculate the dose to the isocentre. Summary tables including action levels and recommended frequencies for each test, as well as relevant references, are provided. If the radiation output or image quality deviates from expected values, or exceeds documented action levels for a given system, a more in depth system analysis (using conventional tests) and corrective maintenance work may be required.

Construction of realistic phantoms from patient images and a commercial three-dimensional printer

The purpose of this study was to use three-dimensional (3-D) printing techniques to construct liver and brain phantoms having realistic pathologies, anatomic structures, and heterogeneous backgrounds. Patient liver and head computed tomography (CT) images were segmented into tissue, vessels, liver lesion, white and gray matter, and cerebrospinal fluid (CSF). Stereolithography files of each object were created and imported into a commercial 3-D printer. Printing materials were assigned to each object after test scans, which showed that the printing materials had CT numbers ranging from 70 to 121 HU at 120 kV. Printed phantoms were scanned on a CT scanner and images were evaluated. CT images of the liver phantom had measured CT numbers of 77.8 and 96.6 HU for the lesion and background, and 137.5 to 428.4 HU for the vessels channels, which were filled with iodine solutions. The difference in CT numbers between lesions and background (18.8 HU) was representative of the low-contrast values needed for optimization tasks. The liver phantom background was evaluated with Haralick features and showed similar texture between patient and phantom images. CT images of the brain phantom had CT numbers of 125, 134, and 108 HU for white matter, gray matter, and CSF, respectively. The CT number differences were similar to those in patient images.