Anthropomorphic model observer performance in three-dimensional detection task for low-contrast computed tomography

Accurate and efficient measurement of channelized Hotelling observer-based low-contrast detectability on the ACR CT accreditation phantom

BACKGROUND

Current CT quality control (QC) for low-contrast detectability relies on visual inspection and measurement of contrast-to-noise ratio (CNR). However, CNR numbers become unreliable when it comes to nonlinear methods, such as iterative reconstruction (IR) and deep-learning-based techniques. Image quality metrics using channelized Hotelling observer (CHO) have been validated to be well correlated with human observer performance on phantom-based and patient-based tasks, but it has not been widely used in routine CT QC mainly because the CHO calculation typically requires a large number of repeated scans in order to provide accurate and precise estimate of index of detectability (d').

PURPOSE

The main goal of this work is to optimize channel filters and other CHO parameters and accurately estimate the low-contrast detectability with minimum number of repeated scans for the widely used American College of Radiology (ACR) CT accreditation phantom so that it can become practically feasible for routine CT QC tests.

METHODS

To provide a converged d' value, an ACR phantom was repeatedly scanned 100 times at three dose levels (24, 12, and 6 mGy). Images were reconstructed with two kernels (FBP Br44 and IR Br44-3). d' as a function of number of repeated scans was determined for different number of background regions of interest (ROIs), different number of low-contrast objects, different number of slices per each object, and different channel filter options. A reference d' was established using the optimized CHO setting, and the bias of d' was quantified using the d' calculated from all 100 repeated scans. The variation of d' at each condition was estimated using a resampling method combining random subsampling among 100 repeated scans and bootstrapping of the ensembles of signal and background ROIs.

RESULTS

Optimized parameters in CHO calculation were determined: two background ROIs per object, four objects per low-contrast object size, nine non-overlapping slices per object, and a 4-channel Gabor filter. The bias and uncertainty were estimated at different numbers of repeated scans using these parameters. When only one single scan was used in the CHO calculation, the bias of d' was below 6.2% and the uncertainty 15.6-19.6% for the 6, 5, and 4 mm objects, while with three repeated scans the bias was below 2.0% and uncertainty 8.7-10.9% for the three object sizes.

CONCLUSION

With optimized parameter settings in CHO, efficient and accurate measurement of low-contrast detectability on the commonly used ACR phantom becomes feasible, which could potentially lead to adoption of CHO-based low-contrast evaluation in routine QC tests.

Novel phantom for performance evaluation of contrast-enhanced 3D rotational angiography

PURPOSE

This technical note presents an in-house phantom with a specially designed contrast-object module constructed to address the need for three-dimensional rotational angiography (3DRA) testing.

METHODS

The initial part of the study was a brief evaluation on the commercially available phantom used for 3DRA and computed tomography angiography (CTA) to confirm the need for a special phantom for 3D angiography. Once confirmed, an in-house phantom was constructed. The novel phantom was tested to evaluate the basic image performance metrics, i.e., unsharpness (MTF) and noise characterization (NPS), as well as to show its capability for vessel contrast visibility study.

RESULTS

The low contrast objects in the commercially available tools dedicated for CT is found to yield significantly lower signal difference to noise ratio (SDNR) when used for 3DRA, therefore deemed inadequate for 3DRA contrast evaluation. The constructed in-house phantom demonstrates a capability to serve for basic imaging performance check (MTF, NPS, and low contrast evaluation) for 3DRA and CTA. With higher and potentially adjustable visibility of contrast objects as artificial vessels, the in-house phantom also makes more clinically relevant tests, e.g., human- or model observer study and task-based optimization, possible.

CONCLUSION

The novel phantom with special contrast object module shows higher visibility in 3DRA compared to the currently available commercial phantom and, therefore, is recommended for use in 3D angiography.

Medical image quality metrics for foveated model observers

Purpose: A recently proposed model observer mimics the foveated nature of the human visual system by processing the entire image with varying spatial detail, executing eye movements, and scrolling through slices. The model can predict how human search performance changes with signal type and modality (2D versus 3D), yet its implementation is computationally expensive and time-consuming. Here, we evaluate various image quality metrics using extensions of the classic index of detectability expression and assess foveated model observers for search tasks. Approach: We evaluated foveated extensions of a channelized Hotelling and nonprewhitening matched filter model with an eye filter. The proposed methods involve calculating a model index of detectability ( d ' ) for each retinal eccentricity and combining these with a weighting function into a single detectability metric. We assessed different versions of the weighting function that varied in the required measurements of the human observers' search (no measurements, eye movement patterns, size of the image, and median search times). Results: We show that the index of detectability across eccentricities weighted using the eye movement patterns of observers best predicted human performance in 2D versus 3D search performance for a small microcalcification-like signal and a larger mass-like. The metric with a weighting function based on median search times was the second best predicting human results. Conclusions: The findings provide a set of model observer tools to evaluate image quality in the early stages of imaging system evaluation or design without implementing the more computationally complex foveated search model.

Foveated Model Observers for Visual Search in 3D Medical Images

Model observers have a long history of success in predicting human observer performance in clinically-relevant detection tasks. New 3D image modalities provide more signal information but vastly increase the search space to be scrutinized. Here, we compared standard linear model observers (ideal observers, non-pre-whitening matched filter with eye filter, and various versions of Channelized Hotelling models) to human performance searching in 3D 1/f2.8 filtered noise images and assessed its relationship to the more traditional location known exactly detection tasks and 2D search. We investigated two different signal types that vary in their detectability away from the point of fixation (visual periphery). We show that the influence of 3D search on human performance interacts with the signal's detectability in the visual periphery. Detection performance for signals difficult to detect in the visual periphery deteriorates greatly in 3D search but not in 3D location known exactly and 2D search. Standard model observers do not predict the interaction between 3D search and signal type. A proposed extension of the Channelized Hotelling model (foveated search model) that processes the image with reduced spatial detail away from the point of fixation, explores the image through eye movements, and scrolls across slices can successfully predict the interaction observed in humans and also the types of errors in 3D search. Together, the findings highlight the need for foveated model observers for image quality evaluation with 3D search.

A Web-Based Software Platform for Efficient and Quantitative CT Image Quality Assessment and Protocol Optimization

Channelized Hotelling observer (CHO), which has been shown to be well correlated with human observer performance in many clinical CT tasks, has a great potential to become the method of choice for objective image quality assessment. However, the use of CHO in clinical CT is still quite limited, mainly due to its complexity in measurement and calculation in practice, and the lack of access to an efficient and validated software tool for most clinical users. In this work, a web-based software platform for CT image quality assessment and protocol optimization (CTPro) was introduced. A validated CHO tool, along with other common image quality assessment tools, was made readily accessible through this web platform for clinical users and researchers without the need of installing additional software. An example of its application to evaluation of convolutional-neural-network (CNN)-based denoising was demonstrated.

A deep learning- and partial least square regression-based model observer for a low-contrast lesion detection task in CT

PURPOSE

This work aims to develop a new framework of image quality assessment using deep learning-based model observer (DL-MO) and to validate it in a low-contrast lesion detection task that involves CT images with patient anatomical background.

METHODS

The DL-MO was developed using the transfer learning strategy to incorporate a pretrained deep convolutional neural network (CNN), a partial least square regression discriminant analysis (PLS-DA) model and an internal noise component. The CNN was previously trained to achieve the state-of-the-art classification accuracy over a natural image database. The earlier layers of the CNN were used as a deep feature extractor, with the assumption that similarity exists between the CNN and the human visual system. The PLSR model was used to further engineer the deep feature for the lesion detection task in CT images. The internal noise component was incorporated to model the inefficiency and variability of human observer (HO) performance, and to generate the ultimate DL-MO test statistics. Seven abdominal CT exams were retrospectively collected from the same type of CT scanners. To compare DL-MO with HO, 12 experimental conditions with varying lesion size, lesion contrast, radiation dose, and reconstruction types were generated, each condition with 154 trials. CT images of a real liver metastatic lesion were numerically modified to generate lesion models with four lesion sizes (5, 7, 9, and 11 mm) and three contrast levels (15, 20, and 25 HU). The lesions were inserted into patient liver images using a projection-based method. A validated noise insertion tool was used to synthesize CT exams with 50% and 25% of routine radiation dose level. CT images were reconstructed using the weighted filtered back projection algorithm and an iterative reconstruction algorithm. Four medical physicists performed a two-alternative forced choice (2AFC) detection task (with multislice scrolling viewing mode) on patient images across the 12 experimental conditions. DL-MO was operated on the same datasets. Statistical analyses were performed to evaluate the correlation and agreement between DL-MO and HO.

RESULTS

A statistically significant positive correlation was observed between DL-MO and HO for the 2AFC low-contrast detection task that involves patient liver background. The corresponding Pearson product moment correlation coefficient was 0.986 [95% confidence interval (0.950, 0.996)]. Bland-Altman agreement analysis did not indicate statistically significant differences.

CONCLUSIONS

The proposed DL-MO is highly correlated with HO in a low-contrast detection task that involves realistic patient liver background. This study demonstrated the potential of the proposed DL-MO to assess image quality directly based on patient images in realistic, clinically relevant CT tasks.

Evaluation of human observer performance on lesion detectability in single-slice and multislice dedicated breast cone beam CT images with breast anatomical background

PURPOSE

We evaluate the lesion detectability using human and model observer studies in single-slice and multislice cone beam computed tomography (CBCT) images with a breast anatomical background. The purposes of this work are (a) to compare human observer detectability between single-slice and multislice images for different signal sizes and noise structures, (b) to investigate the effect of different multislice viewing modes (i.e., sequential and simultaneous) on the detectability by a human observer, and (c) to predict the detectability by a human observer in single-slice and multislice images using single-slice channelized Hotelling observer (ssCHO) and multislice CHO (msCHO), respectively.

METHODS

Breast anatomical background is modeled using a power law spectrum of mammograms and the lesion is modeled with a spherical signal. We conduct signal-known-exactly and background-known-statistically detection tasks on transverse and longitudinal images reconstructed using the Feldkamp-Davis-Kress algorithm with Hanning and Ram-Lak weighted ramp filters. The human observer study is conducted on three different viewing modes: single-slice, and sequential and simultaneous multislice. To predict the detectability by a human observer, we use ssCHO and msCHO with anthropomorphic channels (i.e., dense difference-of-Gaussian (D-DOG) and Gabor channels) and internal noise.

RESULTS

The detectability by a human observer increases for multislice images compared to single-slice images. For multislice images, the sequential viewing mode yields higher detectability than the simultaneous viewing mode. However, the relative rank of detectability by a human observer for different signal sizes, image planes, and reconstruction filters is not much different between the viewing modes. Detectability by CHO with internal noise shows good correlation with that of the human observer for all viewing modes.

CONCLUSIONS

Detectability by a human observer in CBCT images with breast anatomical background is affected by the image viewing mode, and the effect of the viewing mode depends on the signal size and noise structure. D-DOG and Gabor CHO with internal noise predict the detectability by a human observer well for both the single-slice and multislice image viewing modes.

Human and model observer performance for lesion detection in breast cone beam CT images with the FDK reconstruction

We investigate the detectability of breast cone beam computed tomography images using human and model observers and the variations of exponent, β, of the inverse power-law spectrum for various reconstruction filters and interpolation methods in the Feldkamp-Davis-Kress (FDK) reconstruction. Using computer simulation, a breast volume with a 50% volume glandular fraction and a 2mm diameter lesion are generated and projection data are acquired. In the FDK reconstruction, projection data are apodized using one of three reconstruction filters; Hanning, Shepp-Logan, or Ram-Lak, and back-projection is performed with and without Fourier interpolation. We conduct signal-known-exactly and background-known-statistically detection tasks. Detectability is evaluated by human observers and their performance is compared with anthropomorphic model observers (a non-prewhitening observer with eye filter (NPWE) and a channelized Hotelling observer with either Gabor channels or dense difference-of-Gaussian channels). Our results show that the NPWE observer with a peak frequency of 7cyc/degree attains the best correlation with human observers for the various reconstruction filters and interpolation methods. We also discover that breast images with smaller β do not yield higher detectability in the presence of quantum noise.

Correlation between a 2D channelized Hotelling observer and human observers in a low-contrast detection task with multislice reading in CT

PURPOSE

Model observers have been successfully developed and used to assess the quality of static 2D CT images. However, radiologists typically read images by paging through multiple 2D slices (i.e., multislice reading). The purpose of this study was to correlate human and model observer performance in a low-contrast detection task performed using both 2D and multislice reading, and to determine if the 2D model observer still correlate well with human observer performance in multislice reading.

METHODS

A phantom containing 18 low-contrast spheres (6 sizes × 3 contrast levels) was scanned on a 192-slice CT scanner at five dose levels (CTDI_vol = 27, 13.5, 6.8, 3.4, and 1.7 mGy), each repeated 100 times. Images were reconstructed using both filtered-backprojection (FBP) and an iterative reconstruction (IR) method (ADMIRE, Siemens). A 3D volume of interest (VOI) around each sphere was extracted and placed side-by-side with a signal-absent VOI to create a 2-alternative forced choice (2AFC) trial. Sixteen 2AFC studies were generated, each with 100 trials, to evaluate the impact of radiation dose, lesion size and contrast, and reconstruction methods on object detection. In total, 1600 trials were presented to both model and human observers. Three medical physicists acted as human observers and were allowed to page through the 3D volumes to make a decision for each 2AFC trial. The human observer performance was compared with the performance of a multislice channelized Hotelling observer (CHO_MS), which integrates multislice image data, and with the performance of previously validated CHO, which operates on static 2D images (CHO_2D). For comparison, the same 16 2AFC studies were also performed in a 2D viewing mode by the human observers and compared with the multislice viewing performance and the two CHO models.

RESULTS

Human observer performance was well correlated with the CHO_2D performance in the 2D viewing mode [Pearson product-moment correlation coefficient R = 0.972, 95% confidence interval (CI): 0.919 to 0.990] and with the CHO_MS performance in the multislice viewing mode (R = 0.952, 95% CI: 0.865 to 0.984). The CHO_2D performance, calculated from the 2D viewing mode, also had a strong correlation with human observer performance in the multislice viewing mode (R = 0.957, 95% CI: 879 to 0.985). Human observer performance varied between the multislice and 2D modes. One reader performed better in the multislice mode (P = 0.013); whereas the other two readers showed no significant difference between the two viewing modes (P = 0.057 and P = 0.38).

CONCLUSIONS

A 2D CHO model is highly correlated with human observer performance in detecting spherical low contrast objects in multislice viewing of CT images. This finding provides some evidence for the use of a simpler, 2D CHO to assess image quality in clinically relevant CT tasks where multislice viewing is used.

Comparison of low-contrast detectability between two CT reconstruction algorithms using voxel-based 3D printed textured phantoms

PURPOSE

To use novel voxel-based 3D printed textured phantoms in order to compare low-contrast detectability between two reconstruction algorithms, FBP (filtered-backprojection) and SAFIRE (sinogram affirmed iterative reconstruction) and determine what impact background texture (i.e., anatomical noise) has on estimating the dose reduction potential of SAFIRE.

METHODS

Liver volumes were segmented from 23 abdominal CT cases. The volumes were characterized in terms of texture features from gray-level co-occurrence and run-length matrices. Using a 3D clustered lumpy background (CLB) model, a fitting technique based on a genetic optimization algorithm was used to find CLB textures that were reflective of the liver textures, accounting for CT system factors of spatial blurring and noise. With the modeled background texture as a guide, four cylindrical phantoms (Textures A-C and uniform, 165 mm in diameter, and 30 mm height) were designed, each containing 20 low-contrast spherical signals (6 mm diameter at nominal contrast levels of ∼3.2, 5.2, 7.2, 10, and 14 HU with four repeats per signal). The phantoms were voxelized and input into a commercial multimaterial 3D printer (Object Connex 350), with custom software for voxel-based printing (using principles of digital dithering). Images of the textured phantoms and a corresponding uniform phantom were acquired at six radiation dose levels (SOMATOM Flash, Siemens Healthcare) and observer model detection performance (detectability index of a multislice channelized Hotelling observer) was estimated for each condition (5 contrasts × 6 doses × 2 reconstructions × 4 backgrounds = 240 total conditions). A multivariate generalized regression analysis was performed (linear terms, no interactions, random error term, log link function) to assess whether dose, reconstruction algorithm, signal contrast, and background type have statistically significant effects on detectability. Also, fitted curves of detectability (averaged across contrast levels) as a function of dose were constructed for each reconstruction algorithm and background texture. FBP and SAFIRE were compared for each background type to determine the improvement in detectability at a given dose, and the reduced dose at which SAFIRE had equivalent performance compared to FBP at 100% dose.

RESULTS

Detectability increased with increasing radiation dose (P = 2.7 × 10^-59) and contrast level (P = 2.2 × 10^-86) and was higher in the uniform phantom compared to the textured phantoms (P = 6.9 × 10^-51). Overall, SAFIRE had higher d' compared to FBP (P = 0.02). The estimated dose reduction potential of SAFIRE was found to be 8%, 10%, 27%, and 8% for Texture-A, Texture-B, Texture-C and uniform phantoms.

CONCLUSIONS

In all background types, detectability was higher with SAFIRE compared to FBP. However, the relative improvement observed from SAFIRE was highly dependent on the complexity of the background texture. Iterative algorithms such as SAFIRE should be assessed in the most realistic context possible.

Correlation between human detection accuracy and observer model-based image quality metrics in computed tomography

The purpose of this study was to compare computed tomography (CT) low-contrast detectability from human readers with observer model-based surrogates of image quality. A phantom with a range of low-contrast signals (five contrasts, three sizes) was imaged on a state-of-the-art CT scanner (Siemens' force). Images were reconstructed using filtered back projection and advanced modeled iterative reconstruction and were assessed by 11 readers using a two alternative forced choice method. Concurrently, contrast-to-noise ratio (CNR), area-weighted CNR (CNRA), and observer model-based metrics were estimated, including nonprewhitening (NPW) matched filter, NPW with eye filter (NPWE), NPW with internal noise, NPW with an eye filter and internal noise (NPWEi), channelized Hotelling observer (CHO), and CHO with internal noise (CHOi). The correlation coefficients (Pearson and Spearman), linear discriminator error, [Formula: see text], and magnitude of confidence intervals, [Formula: see text], were used to determine correlation, proper characterization of the reconstruction algorithms, and model precision, respectively. Pearson (Spearman) correlation was 0.36 (0.33), 0.83 (0.84), 0.84 (0.86), 0.86 (0.88), 0.86 (0.91), 0.88 (0.90), 0.85 (0.89), and 0.87 (0.84), [Formula: see text] was 0.25, 0.15, 0.2, 0.25, 0.3, 0.25, 0.4, and 0.45, and [Formula: see text] was [Formula: see text], [Formula: see text], [Formula: see text], [Formula: see text], [Formula: see text], [Formula: see text], [Formula: see text], and [Formula: see text] for CNR, CNRA, NPW, NPWE, NPWi, NPWEi, CHO, and CHOi, respectively.