Accurate and efficient measurement of channelized Hotelling observer-based low-contrast detectability on the ACR CT accreditation phantom

Low-contrast detectability of photon-counting-detector CT at different scan modes and image types in comparison with energy-integrating-detector CT

Purpose

We aim to compare the low-contrast detectability of a clinical whole-body photon-counting-detector (PCD)-CT at different scan modes and image types with an energy-integrating-detector (EID)-CT.

Approach

We used a channelized Hotelling observer (CHO) previously optimized for quality control purposes. An American College of Radiology CT accreditation phantom was scanned on both PCD-CT and EID-CT with 10 phantom positionings. For PCD-CT, images were generated using two scan modes, standard resolution (SR) and ultra-high-resolution (UHR); two image types, virtual monochromatic images at 70 keV and low-energy threshold (T3D); both filtered-back-projection (FBP) and iterative reconstruction (IR) reconstruction methods; and three reconstruction kernels. For each positioning, three repeated scans were acquired for each scan mode, image type, and CTDIvol of 6, 12, and 24 mGy. For EID-CT, images acquired from scans (10 positionings × 3 repeats × 3 doses) were reconstructed using the closest counterpart FBP and IR kernels. CHO was applied to calculate the index of detectability (d ' ) on both scanners.

Results

With the smooth Br44 kernel, the d ' of UHR was mostly comparable with that of the SR mode (difference: -11.4% to 8.3%, p = 0.020 to 0.956), and the T3D images had a higher d ' (difference: 0.7% to 25.6%) than 70 keV images on PCD-CT. Compared with the EID-CT, UHR-T3D of PCD-CT had non-inferior d ' (difference: -2.7% to 12.9%) with IR and non-superior d ' (difference: 0.8% to 11.2%) with FBP using the Br44 kernel. PCD-CT produced higher d ' than EID-CT by 61.8% to 247.1% with the sharper reconstruction kernels.

Conclusions

The comparison between PCD-CT and EID-CT was significantly influenced by the reconstruction method and kernel. With a smooth kernel that is typically used in low-contrast detection tasks, the PCD-CT demonstrated low-contrast detectability that was comparable to EID-CT with IR and showed no superiority when using FBP. With the use of sharper kernels, the PCD-CT significantly outperformed EID-CT in low-contrast detectability.

3D printed phantom with 12 000 submillimeter lesions to improve efficiency in CT detectability assessment

BACKGROUND

The detectability performance of a CT scanner is difficult to precisely quantify when nonlinearities are present in reconstruction. An efficient detectability assessment method that is sensitive to small effects of dose and scanner settings is desirable. We previously proposed a method using a search challenge instrument: a phantom is embedded with hundreds of lesions at random locations, and a model observer is used to detect lesions. Preliminary tests in simulation and a prototype showed promising results.

PURPOSE

In this work, we fabricated a full-size search challenge phantom with design updates, including changes to lesion size, contrast, and number, and studied our implementation by comparing the lesion detectability from a nonprewhitening (NPW) model observer between different reconstructions at different exposure levels, and by estimating the instrument sensitivity to detect changes in dose.

METHODS

Designed to fit into QRM anthropomorphic phantoms, our search challenge phantom is a cylindrical insert 10 cm wide and 4 cm thick, embedded with 12 000 lesions (nominal width of 0.6 mm, height of 0.8 mm, and contrast of -350 HU), and was fabricated using PixelPrint, a 3D printing technique. The insert was scanned alone at a high dose to assess printing accuracy. To evaluate lesion detectability, the insert was placed in a QRM thorax phantom and scanned from 50 to 625 mAs with increments of 25 mAs, once per exposure level, and the average of all exposure levels was used as high-dose reference. Scans were reconstructed with three different settings: filtered-backprojection (FBP) with Br40 and Br59, and Sinogram Affirmed Iterative Reconstruction (SAFIRE) with strength level 5 and Br59 kernel. An NPW model observer was used to search for lesions, and detection performance of different settings were compared using area under the exponential transform of free response ROC curve (AUC). Using propagation of uncertainty, the sensitivity to changes in dose was estimated by the percent change in exposure due to one standard deviation of AUC, measured from 5 repeat scans at 100, 200, 300, and 400 mAs.

RESULTS

The printed insert lesions had an average position error of 0.20 mm compared to printing reference. As the exposure level increases from 50 mAs to 625 mAs, the lesion detectability AUCs increase from 0.38 to 0.92, 0.42 to 0.98, and 0.41 to 0.97 for FBP Br40, FBP Br59, and SAFIRE Br59, respectively, with a lower rate of increase at higher exposure level. FBP Br59 performed best with AUC 0.01 higher than SAFIRE Br59 on average and 0.07 higher than FBP Br40 (all P < 0.001). The standard deviation of AUC was less than 0.006, and the sensitivity to detect changes in mAs was within 2% for FBP Br59.

CONCLUSIONS

Our 3D-printed search challenge phantom with 12 000 submillimeter lesions, together with an NPW model observer, provide an efficient CT detectability assessment method that is sensitive to subtle effects in reconstruction and is sensitive to small changes in dose.

Channelized hotelling observer-based low-contrast detectability on the ACR CT accreditation phantom: Part II. Repeatability study

BACKGROUND

Objective and quantitative evaluation for low-contrast detectability that correlates with human observer performance is lacking for routine CT quality control testing. Channelized Hotelling observer (CHO) is considered a strong candidate to fill the need but has long been deemed impractical to implement due to its requirement of a large number of repeated scans in order to provide accurate and precise estimates of index of detectability (d'). In our previous work, we optimized a CHO model observer on the American College of Radiology (ACR) CT accreditation phantom and achieved accurate measurement of d' with only 1-3 repeat scans.

PURPOSE

In this work, we aim to validate the repeatability of the proposed CHO-based low-contrast evaluation on four scanner models using the ACR CT accreditation phantom.

METHODS

The repeatability test was performed on four different scanners from two major CT manufacturers: Siemens Force and Alpha; Canon Prism and Prime SP. An ACR CT phantom was scanned 10 times, each time after repositioning of the phantom. For each repositioning, 3 repeated scans were acquired at 24, 12, and 6 mGy on all four scanner models. CHO was applied at the measured dose levels for different low-contrast object sizes (4-6 mm). The CHO was also applied to images created using deep learning-based reconstructions on Canon Prism and to four different scan/reconstruction modes on the Siemens Alpha, a photon-counting-detector (PCD)-CT. The repeatability was evaluated by the probability that a measurement would fall within the ±15% tolerance (P<15% ).

RESULTS

With the CHO setting optimized for the ACR phantom and the use of 3 repeated scans and 9 non-overlapping slices per scan, the CHO measurement could provide high repeatability with P<15% of 98.8%-99.9% at 12 mGy with IR reconstruction on all four scanners. On scanner A, P<15% were 91.5%-99.9% at the three dose levels and for all three object sizes while the numbers were 93.6%-99.998% on scanner B. P<15% were 96.5%-97.2% for the two deep learning reconstructions and 97.0%-99.97% for the four scan/reconstruction modes on the PCD-CT.

CONCLUSION

The CHO provided highly repeatable measurements with over 95% probability that a CHO measurement would lie within the ±15% tolerance for most of the dose levels and object sizes on the ACR phantom. The repeatability was maintained when the CHO was applied to images created with a commercial deep learning-based reconstruction and various scan/reconstruction modes on a PCD-CT. This study demonstrates that practical implementation of CHO for routine quality control and performance evaluation is feasible.

Automated Web-based Software for CT Quality Control Testing of Low-contrast Detectability using Model Observers

The Channelized Hotelling observer (CHO) is well correlated with human observer performance in many CT detection/classification tasks but has not been widely adopted in routine CT quality control and performance evaluation, mainly because of the lack of an easily available, efficient, and validated software tool. We developed a highly automated solution - CT image quality evaluation and Protocol Optimization (CTPro), a web-based software platform that includes CHO and other traditional image quality assessment tools such as modulation transfer function and noise power spectrum. This tool can allow easy access to the CHO for both the research and clinical community and enable efficient, accurate image quality evaluation without the need of installing additional software. Its application was demonstrated by comparing the low-contrast detectability on a clinical photon-counting-detector (PCD)-CT with a traditional energy-integrating-detector (EID)-CT, which showed UHR-T3D had 6.2% higher d' than EID-CT with IR (p = 0.047) and 4.1% lower d' without IR (p = 0.122).