Automated Quantitative Analysis of American College of Radiology PET Phantom Images

Phantom and methodology for comparison of small lesion detectability in PET

PURPOSE

The main goal of this work is to describe a phantom design, data acquisition and data analysis methodology enabling comparison of small lesion detectability between PET imaging systems and reconstruction algorithms. Several methods are currently available to characterize intrinsic and image quality performance, but none focus exclusively on small lesion detectability.

METHODS

We previously developed a small-lesion detection phantom and described initial results using a head-size phantom. Unlike most fillable nuclear medicine phantoms, this phantom offers a semi-realistic heterogenous background and wall-less contrast features. In this work, the methodology is extended to include (a) the use of both head- and body-sized phantoms and (b) a multi-scan data collection and analysis method. We present an example use case of the phantom and detection estimation methodology, comparing the small-lesion detection performance across four commercial PET/CT systems.

RESULTS

Repeat acquisitions of the phantom enabled estimation of model observer performance and surrogates of detectability. As anticipated, estimated detectability increased with the square root of system sensitivity and TOF offered marked improvement in detectability, especially for the body sized object. The proposed approach characterizing detectability at different times during the decay of the phantom enabled comparison of small lesion detectability at matched activity concentrations (and scan durations) across different scanners.

CONCLUSION

The proposed approach offers a reproducible tool for evaluating relative tradeoffs of system performance on small lesion detectability.

A review of harmonization strategies for quantitative PET

PET can reveal in vivo biological processes at the molecular level. PET-derived quantitative values have been used as a surrogate marker for clinical decision-making in numerous clinical studies and trials. However, quantitative values in PET are variable depending on technical, biological, and physical factors. The variability may have a significant impact on a study outcome. Appropriate scanner calibration and quality control, standardization of imaging protocols, and any necessary harmonization strategies are essential to make use of PET as a biomarker with low bias and variability. This review summarizes benefits, limitations, and remaining challenges for harmonization of quantitative PET, including whole-body PET in oncology, brain PET in neurology, PET/MR, and non-¹⁸F PET imaging. This review is expected to facilitate harmonization of quantitative PET and to promote the contribution of PET-derived biomarkers to research and development in medicine.

Comparison of protocols with respiratory-gated (4D) motion compensation in PET/CT: open-source package for quantification of phantom image quality

BACKGROUND

Patient's breathing affects the quality of chest images acquired with positron emission tomography/computed tomography (PET/CT) studies. Movement correction is required to optimize PET quantification in clinical settings. We present a reproducible methodology to compare the impact of different movement compensation protocols on PET image quality. Static phantom images were set as reference values, and recovery coefficients (RCs) were calculated from motion compensated images for the phantoms in respiratory movement. Image quality was evaluated in terms of: (1) volume accuracy (VA) with the NEMA phantom; (2) concentration accuracy (CA) by six refillable inserts within the electron density CIRS phantom; and (3) spatial resolution (R) with the Jaszczak phantom. Three different respiratory patterns were applied to the phantoms. We developed an open-source package to automatically analyze VA, CA and R. We compared 10 different movement compensation protocols available in the Philips Gemini TF-64 PET/CT (4-, 6-, 8- and 10-time bins, 20%-, 30%-, 40%-window width in Inhale and Exhale).

RESULTS

The homemade package provided RC values for VA, CA and R of 102 PET images in less than 5 min. Results of the comparison of the 10 different protocols demonstrated the feasibility of the proposed method for quantifying the variations observed qualitatively. Overall, prospective protocols showed better motion compensation than retrospective. The best performance was obtained for the protocol Exhale 30% (0.3 s after maximum Exhale position and window width of 30%) with RC[Formula: see text], RC[Formula: see text] and RC[Formula: see text]. Among retrospective protocols, 8 Phase protocol showed the best performance.

CONCLUSION

We provided an open-source package able to automatically evaluate the impact of motion compensation methods on PET image quality. A setup based on commonly available experimental phantoms is recommended. Its application for the comparison of 10 time-based approaches showed that Exhale 30% protocol had the best performance. The proposed framework is not specific to the phantoms and protocols presented on this study.

Advanced quantitative evaluation of PET systems using the ACR phantom and NiftyPET software

PURPOSE

A novel phantom-imaging platform, a set of software tools, for automated and high-precision imaging of the American College of Radiology (ACR) positron emission tomography (PET) phantom for PET/magnetic resonance (PET/MR) and PET/computed tomography (PET/CT) systems is proposed.

METHODS

The key feature of this platform is the vector graphics design that facilitates the automated measurement of the knife-edge response function and hence image resolution, using composite volume of interest templates in a 0.5 mm resolution grid applied to all inserts of the phantom. Furthermore, the proposed platform enables the generation of an accurate μ $\mu$ -map for PET/MR systems with a robust alignment based on two-stage image registration using specifically designed PET templates. The proposed platform is based on the open-source NiftyPET software package used to generate multiple list-mode data bootstrap realizations and image reconstructions to determine the precision of the two-stage registration and any image-derived statistics. For all the analyses, iterative image reconstruction was employed with and without modeled shift-invariant point spread function and with varying iterations of the ordered subsets expectation maximization (OSEM) algorithm. The impact of the activity outside the field of view (FOV) was assessed using two acquisitions of 30 min each, with and without the activity outside the FOV.

RESULTS

The utility of the platform has been demonstrated by providing a standard and an advanced phantom analysis including the estimation of spatial resolution using all cylindrical inserts. In the imaging planes close to the edge of the axial FOV, we observed deterioration in the quantitative accuracy, reduced resolution (FWHM increased by 1-2 mm), reduced contrast, and background uniformity due to the activity outside the FOV. Although it slows convergence, the PSF reconstruction had a positive impact on resolution and contrast recovery, but the degree of improvement depended on the regions. The uncertainty analysis based on bootstrap resampling of raw PET data indicated high precision of the two-stage registration.

CONCLUSIONS

We demonstrated that phantom imaging using the proposed methodology with the metric of spatial resolution and multiple bootstrap realizations may be helpful in more accurate evaluation of PET systems as well as in facilitating fine tuning for optimal imaging parameters in PET/MR and PET/CT clinical research studies.

Comparison and validation of automated scoring of SPECT phantom cold rod visibility

PURPOSE

Evaluation of phantom image quality is an integral component of the quality assurance of SPECT systems. This evaluation often is done by visual assessment of the resolution of known structures of a specified size, such as arrays of cold rods in a warm background. Although this method is rapid and convenient, it is qualitative and is subject to inter- and intraobserver variability. Thus an automated quantitative analysis would be preferable. Several metrics of cold rod visibility have been developed, although their suitability for SPECT quality assurance depends on how well they correspond to visual scoring by experienced observers.

METHODS

Various metrics of cold rod visibility, derived from either texture analysis or template-based analysis, were investigated. The texture analysis methods measured the normalized gray-level co-occurrence matrix (GLCM) energy ("Energy%") and entropy ("Entropy%") of each region and an associated combination of the two ("EnergyEntropy%"). One template-based method measured the rods-to-background contrast ("Contrast") and an associated visibility index (Contrast × area = "Contrast Visibility"). Another template-based method performed binary classification (BC) of the rods and background to compute the area under curve (AUC) of its receiver operating characteristics (ROC) curve ("BC-AUC") and the corresponding signal-to-noise ratio ("BC-SNR"). All these metrics were computed for 90 SPECT acquisitions of the standard American College of Radiology ("Jaszczak") phantom. Cold rod visibility was scored independently by two experienced nuclear medicine physicists on both dichotomous and 5-point scales. Scoring was performed twice by each observer to evaluate variability.

RESULTS

Interobserver agreement (Cohen's kappa statistic) was 0.78, and intraobserver reproducibility was 0.86 and 0.88, respectively, for each observer. Mean and median scores differed significantly between observers. Accuracy of each metric was assessed according to AUC of ROC analysis with respect to mean dichotomous score. The binary classification metrics had the highest accuracy (BC-AUC = 0.995, BC-SNR = 0.994), above that of the texture analysis metrics (Entropy% = 0.992, Energy% = 0.988, EnergyEntropy% = 0.992) and conventional template analysis (Contrast = 0.984, Contrast Visibility = 0.989). The metrics were similar in terms of rank correlation to mean visibility score. BC-AUC correlated linearly with mean visibility score (R²  = 0.95) and consistently performed among the highest of the metrics vs rod diameter and count level.

CONCLUSIONS

Automated quantitative analysis of SPECT phantom cold rods correlated well with visual scoring. The metrics based on binary classification performed particularly well for this task, across the range of rod diameters and count levels. The suboptimal interobserver agreement highlights the importance of developing automated algorithms for evaluating scanner performance.