Influence of cold walls on PET image quantification and volume segmentation: a phantom study

Optimization of reconstruction in quantitative brain PET images: Benefits from PSF modeling and correction of edge artifacts

BACKGROUND

Modern PET reconstruction algorithms incorporate point-spread-function (PSF) correction to mitigate partial volume effect. However, PSF correction can introduce edge artifacts that lead to quantification errors. Consequently, current international guidelines advise against using PSF correction in brain PET reconstruction.

PURPOSE

We aimed to investigate PSF-induced quantification errors in recent digital PET systems and identify conditions that mitigate them. This study utilized brain PET imaging with alginate-based realistic phantoms, simulating lesion-to-background activity ratios of 10:1 and 2:1, with eleven reconstruction parameter sets.

METHODS

Phantoms were prepared using a commercial anthropomorphic head phantom and two homemade inserts. Each insert contained a homogeneous 18F-FDG alginate background with hot spheres of varying diameter (3, 4, 6, 8, 10, 12, and 15 mm). PET imaging was conducted on a digital PET-CT system Biograph Vision 600 (Siemens), with a 10 min scan duration. Imaging was performed with and without PSF correction, with 2, 4, 6, 12, 18, or 24 iterations in reconstruction, and with or without additional Gaussian postfiltering. We assessed the recovery coefficient (RC), contrast recovery coefficient (CRC), variability, and CRC-to-variability ratios for each sphere size and reconstruction parameter set.

RESULTS

PSF-corrected images of the 10:1 spheres exhibited a nonmonotonic CRC-to-sphere diameter relationship due to edge artifacts overshoot in the 10 mm-diameter sphere. In contrast, PSF images of the 2:1 spheres showed a monotonically increasing relationship. Non-PSF images of both phantoms showed an expected monotonically increasing CRC-to-sphere diameter relationship but with lower CRC values compared to PSF images. The nonmonotonic relationship observed with 10:1 spheres was mitigated by applying a 3-mm FWHM Gaussian postfiltering. For both phantoms, reconstructions with 6 iterations, PSF correction, and additional 3-mm FWHM Gaussian postfiltering demonstrated the highest CRC-to-variability ratios.

CONCLUSIONS

Our findings indicate that Gaussian postfiltering suppresses PSF artifacts. This parameter set corrected the nonmonotonic CRC-to-sphere diameter relationship and improved the CRC-to-variability ratio compared to non-PSF reconstructions. Therefore, to enhance lesion detectability without compromising quantification accuracy, PSF correction coupled with Gaussian postfiltering should be used in brain PET.

The magnitude of the partial volume effect in SPECT imaging of the kidneys: a phantom study

Background

Single-photon emission computed tomography (SPECT) can cause an over- or underestimation of tissue activity concentration due to limitations in spatial resolution compared to the structures under study. This is commonly referred to as partial volume effect (PVE). Ideally, the PVE should be controlled for and corrected. One such correction method involves determining recovery coefficients (RC) from phantom measurements. In the literature, several studies applying simplified geometries are available. In this study, we aimed to determine kidney PVE for realistic kidney geometries. Furthermore, we proposed a new surrogate metric for predicting the extent of PVE in kidneys.

Material and methods

Based on patients’ CT data, we manufactured fillable phantoms using a 3D-printer. Nine cortex-only and ten whole-parenchyma phantoms were obtained, and one ellipsoidal phantom for comparison. To measure PVE, we placed the phantoms in a torso phantom and filled them with a specified activity concentration. The phantoms’ RCs were determined from fully quantitative SPECT/CT acquisitions at three different target-to-background ratios (TBRs). Additionally, the surface area-to-volume (SA:V) ratio was determined for all phantoms and correlated with RCs.

Results

For SPECT reconstructions with 36 iterations, average RC ± one standard deviation at a 10-to-1 TBR was 76.3 ± 1.5% and 48.4 ± 8.3% for whole-parenchyma and cortex-only phantoms, respectively. The RC for the ellipsoidal phantom was 85.4%. The RC for whole-parenchyma was significantly higher than for cortex-only phantoms (p < 0.01). The RC variance was significantly higher for cortex-only phantoms (p < 0.01). A highly significant correlation of the SA:V ratio and RC was found for all phantoms. (R² of linear regression was between 0.96 and 0.98.)

Conclusion

Changes in the specific shape of the kidneys cause large changes in PVE magnitude. Therefore, RCs derived from more simple phantoms are most likely insufficient to correct the PVE in patient images. Furthermore, one should account for the fact that intra-renal activity distribution significantly influences the extent of PVE. Additionally, we found that the SA:V ratio excellently models kidney RCs; potentially, this approach could also be applied to other geometries and represents an alternative to full imaging process simulations to determine the extent of PVE.

Design and evaluation of a modular multimodality imaging phantom to simulate heterogeneous uptake and enhancement patterns for radiomic quantification in hybrid imaging; a feasibility study

Background

Accuracy and precision assessment in radiomic features is important for the determination of their potential to characterize cancer lesions. In this regard, simulation of different imaging conditions using specialized phantoms is increasingly being investigated. In this study, the design and evaluation of a modular multimodality imaging phantom to simulate heterogeneous uptake and enhancement patterns for radiomics quantification in hybrid imaging is presented.

Methods

A modular multimodality imaging phantom was constructed that could simulate different patterns of heterogeneous uptake and enhancement patterns in positron emission tomography (PET), single‐photon emission computed tomography (SPECT), computed tomography (CT), and magnetic resonance (MR) imaging. The phantom was designed to be used as an insert in the standard NEMA‐NU2 IEC body phantom casing. The entire phantom insert is composed of three segments, each containing three separately fillable compartments. The fillable compartments between segments had different sizes in order to simulate heterogeneous patterns at different spatial scales. The compartments were separately filled with different ratios of ^99mTc‐pertechnetate, ¹⁸F‐fluorodeoxyglucose ([¹⁸F]FDG), iodine‐ and gadolinium‐based contrast agents for SPECT, PET, CT, and T₁‐weighted MR imaging respectively. Image acquisition was performed using standard oncological protocols on all modalities and repeated five times for repeatability assessment. A total of 93 radiomic features were calculated. Variability was assessed by determining the coefficient of quartile variation (CQV) of the features. Comparison of feature repeatability at different modalities and spatial scales was performed using Kruskal‐Wallis‐, Mann‐Whitney U‐, one‐way ANOVA‐ and independent t‐tests.

Results

Heterogeneous uptake and enhancement could be simulated on all four imaging modalities. Radiomic features in SPECT were significantly less stable than in all other modalities. Features in PET were significantly less stable than in MR and CT. A total of 20 features, particularly in the gray‐level co‐occurrence matrix (GLCM) and gray‐level run‐length matrix (GLRLM) class, were found to be relatively stable in all four modalities for all three spatial scales of heterogeneous patterns (with CQV < 10%).

Conclusion

The phantom was suitable for simulating heterogeneous uptake and enhancement patterns in [¹⁸F]FDG‐PET, ^99mTc‐SPECT, CT, and T₁‐weighted MR images. The results of this work indicate that the phantom might be useful for the further development and optimization of imaging protocols for radiomic quantification in hybrid imaging modalities.

Evaluation of an Automatic Classification Algorithm Using Convolutional Neural Networks in Oncological Positron Emission Tomography

Introduction: Our aim was to evaluate the performance in clinical research and in clinical routine of a research prototype, called positron emission tomography (PET) Assisted Reporting System (PARS) (Siemens Healthineers) and based on a convolutional neural network (CNN), which is designed to detect suspected cancer sites in fluorine-18 fluorodeoxyglucose (¹⁸F-FDG) PET/computed tomography (CT).

Method: We retrospectively studied two cohorts of patients. The first cohort consisted of research-based patients who underwent PET scans as part of the initial workup for diffuse large B-cell lymphoma (DLBCL). The second cohort consisted of patients who underwent PET scans as part of the evaluation of miscellaneous cancers in clinical routine. In both cohorts, we assessed the correlation between manually and automatically segmented total metabolic tumor volumes (TMTVs), and the overlap between both segmentations (Dice score). For the research cohort, we also compared the prognostic value for progression-free survival (PFS) and overall survival (OS) of manually and automatically obtained TMTVs.

Results: For the first cohort (research cohort), data from 119 patients were retrospectively analyzed. The median Dice score between automatic and manual segmentations was 0.65. The intraclass correlation coefficient between automatically and manually obtained TMTVs was 0.68. Both TMTV results were predictive of PFS (hazard ratio: 2.1 and 3.3 for automatically based and manually based TMTVs, respectively) and OS (hazard ratio: 2.4 and 3.1 for automatically based and manually based TMTVs, respectively). For the second cohort (routine cohort), data from 430 patients were retrospectively analyzed. The median Dice score between automatic and manual segmentations was 0.48. The intraclass correlation coefficient between automatically and manually obtained TMTVs was 0.61.

Conclusion: The TMTVs determined for the research cohort remain predictive of total and PFS for DLBCL. However, the segmentations and TMTVs determined automatically by the algorithm need to be verified and, sometimes, corrected to be similar to the manual segmentation.

Impact of scanner harmonization on PET-based treatment response assessment in metastatic melanoma

Patients with metastatic melanoma often receive ¹⁸F-FDG PET/CT scans on different scanners throughout their monitoring period. In this study, we quantified the impact of scanner harmonization on longitudinal changes in PET standardized uptake values using various harmonization and normalization methods, including an anthropomorphic PET phantom. Twenty metastatic melanoma patients received at least two FDG PET/CT scans, each on two different scanners with an average of 4 months (range: 2-8) between. Scans from a General Electric (GE) Discovery 710 PET CT^-1 were harmonized to the GE Discovery VCT using image reconstruction settings matching recovery coefficients in an anthropomorphic phantom with bone equivalent inserts and wall-less synthetic lesions. In patient images, SUV_max was measured for each melanoma lesion and time-point. Lesions were classified as progressing, stable, or responding based on pre-defined threshold of ±30% change in SUV_max. For comparison, harmonization was also performed using simpler methods, including harmonization using a NEMA phantom, post-reconstruction filtering, reference region normalization of SUV_max, and use of SUV_peak instead of SUV_max. In the 20 patients, 90 lesions across two time-points were available for treatment response assessment. Treatment response classification changed in 47% (42/90) of cases after harmonization with anthropomorphic phantom. Before harmonization, 37% (33/90) of the lesions were classified as stable (changing less than 30% between two time-points), while the fraction of stable lesions increased to 58% (52/90) after harmonization. Harmonization with the NEMA phantom agreed with harmonization with the anthropomorphic phantom in 91% (82/90) of cases. Post-reconstruction filtering agreed with anthropomorphic phantom-based harmonization in 83% (75/90) cases. The utilization of reference regions for normalization or SUV_peak was unable to correct for changes as identified by the anthropomorphic phantom-based harmonization. Overall, PET scanner harmonization has a major impact on individual lesion treatment response classification in metastatic melanoma patients. Harmonization using the NEMA phantom yielded similar results to harmonization using anthropomorphic phantom, while the only acceptable post-reconstruction technique was post-reconstruction filtering. Phantom-based harmonization is therefore strongly recommended when comparing lesion uptake across time-points when the images have been acquired on different PET scanners.

FDG PET/CT for Assessment of Immune Therapy: Opportunities and Understanding Pitfalls

Immune checkpoint blockade has demonstrated the ability to modulate the immune system to produce durable responses in a wide range of cancers and has significantly impacted the standard of care. However, many cancer patients still do not respond to immune checkpoint blockade or have a limited duration of antitumor responses. Moreover, immune-related adverse events caused by immune checkpoint blockade can be severe and debilitating for some patients, limiting continuation of therapy and resulting in severe autoimmune conditions. Standard-of-care conventional anatomic imaging modalities and tumor response criteria have limitations to adequately assess tumor responses, especially early in the course of therapy, for risk-adapted clinical management to inform care of patients treated with immunotherapy. Molecular imaging with position emission tomography (PET) provides a noninvasive functional biomarker of tumor response, and of immune activation, for patients on immune-based therapies to help address these needs. 18F-FDG (FDG) PET/CT is readily available clinically and a number of studies have evaluated the use of this agent for assessment of prognosis, treatment response and immune activation for patients treated with immune checkpoint blockade. In this review paper, we discuss the current oncologic applications and imaging needs of cancer immunotherapy, recent studies applying FDG PET/CT for tumor response assessment, and evaluation of immune-related adverse events for improving clinical management. We largely focus on metastatic melanoma; however, we generalize where applicable to immunotherapy in other tumor types. We also briefly discuss PET imaging and quantitation as well as emerging non-FDG PET imaging radiotracers for cancer immunotherapy imaging.

Machine-learned target volume delineation of 18F-FDG PET images after one cycle of induction chemotherapy

Biological tumour volume (GTV_PET) delineation on ¹⁸F-FDG PET acquired during induction chemotherapy (ICT) is challenging due to the reduced metabolic uptake and volume of the GTV_PET. Automatic segmentation algorithms applied to ¹⁸F-FDG PET (PET-AS) imaging have been used for GTV_PET delineation on ¹⁸F-FDG PET imaging acquired before ICT. However, their role has not been investigated in ¹⁸F-FDG PET imaging acquired after ICT. In this study we investigate PET-AS techniques, including ATLAAS a machine learned method, for accurate delineation of the GTV_PET after ICT. Twenty patients were enrolled onto a prospective phase I study (FiGaRO). PET/CT imaging was acquired at baseline and 3 weeks following 1 cycle of induction chemotherapy. The GTV_PET was manually delineated by a nuclear medicine physician and clinical oncologist. The resulting GTV_PET was used as the reference contour. The ATLAAS original statistical model was expanded to include images of reduced metabolic activity and the ATLAAS algorithm was re-trained on the new reference dataset. Estimated GTV_PET contours were derived using sixteen PET-AS methods and compared to the GTV_PET using the Dice Similarity Coefficient (DSC). The mean DSC for ATLAAS, 60% Peak Thresholding (PT60), Adaptive Thresholding (AT) and Watershed Thresholding (WT) was 0.72, 0.61, 0.63 and 0.60 respectively. The GTV_PET generated by ATLAAS compared favourably with manually delineated volumes and in comparison, to other PET-AS methods, was more accurate for GTV_PET delineation after ICT. ATLAAS would be a feasible method to reduce inter-observer variability in multi-centre trials.

A smart and operator independent system to delineate tumours in Positron Emission Tomography scans

Positron Emission Tomography (PET) imaging has an enormous potential to improve radiation therapy treatment planning offering complementary functional information with respect to other anatomical imaging approaches. The aim of this study is to develop an operator independent, reliable, and clinically feasible system for biological tumour volume delineation from PET images. Under this design hypothesis, we combine several known approaches in an original way to deploy a system with a high level of automation. The proposed system automatically identifies the optimal region of interest around the tumour and performs a slice-by-slice marching local active contour segmentation. It automatically stops when a "cancer-free" slice is identified. User intervention is limited at drawing an initial rough contour around the cancer region. By design, the algorithm performs the segmentation minimizing any dependence from the initial input, so that the final result is extremely repeatable. To assess the performances under different conditions, our system is evaluated on a dataset comprising five synthetic experiments and fifty oncological lesions located in different anatomical regions (i.e. lung, head and neck, and brain) using PET studies with 18F-fluoro-2-deoxy-d-glucose and 11C-labeled Methionine radio-tracers. Results on synthetic lesions demonstrate enhanced performances when compared against the most common PET segmentation methods. In clinical cases, the proposed system produces accurate segmentations (average dice similarity coefficient: 85.36 ± 2.94%, 85.98 ± 3.40%, 88.02 ± 2.75% in the lung, head and neck, and brain district, respectively) with high agreement with the gold standard (determination coefficient R² = 0.98). We believe that the proposed system could be efficiently used in the everyday clinical routine as a medical decision tool, and to provide the clinicians with additional information, derived from PET, which can be of use in radiation therapy, treatment, and planning.

The first MICCAI challenge on PET tumor segmentation

INTRODUCTION

Automatic functional volume segmentation in PET images is a challenge that has been addressed using a large array of methods. A major limitation for the field has been the lack of a benchmark dataset that would allow direct comparison of the results in the various publications. In the present work, we describe a comparison of recent methods on a large dataset following recommendations by the American Association of Physicists in Medicine (AAPM) task group (TG) 211, which was carried out within a MICCAI (Medical Image Computing and Computer Assisted Intervention) challenge.

MATERIALS AND METHODS

Organization and funding was provided by France Life Imaging (FLI). A dataset of 176 images combining simulated, phantom and clinical images was assembled. A website allowed the participants to register and download training data (n = 19). Challengers then submitted encapsulated pipelines on an online platform that autonomously ran the algorithms on the testing data (n = 157) and evaluated the results. The methods were ranked according to the arithmetic mean of sensitivity and positive predictive value.

RESULTS

Sixteen teams registered but only four provided manuscripts and pipeline(s) for a total of 10 methods. In addition, results using two thresholds and the Fuzzy Locally Adaptive Bayesian (FLAB) were generated. All competing methods except one performed with median accuracy above 0.8. The method with the highest score was the convolutional neural network-based segmentation, which significantly outperformed 9 out of 12 of the other methods, but not the improved K-Means, Gaussian Model Mixture and Fuzzy C-Means methods.

CONCLUSION

The most rigorous comparative study of PET segmentation algorithms to date was carried out using a dataset that is the largest used in such studies so far. The hierarchy amongst the methods in terms of accuracy did not depend strongly on the subset of datasets or the metrics (or combination of metrics). All the methods submitted by the challengers except one demonstrated good performance with median accuracy scores above 0.8.

Minimum lesion detectability as a measure of PET system performance

BACKGROUND

A phantom in combination with an imaging protocol was developed to measure the limit of small lesion detection on different PET systems. Seven small spheres with inner diameters ranging from 3.95 up to 15.43 mm were imaged in a Jaszczak ECT Phantom, in air, in a cold background, and with sphere to background contrast ratios of 15:1 down to 1.88:1. The imaging times varied from 1 to 16 min. The imaging protocol was performed on the Gemini TF and Vereos by Philips, the mCT and HRRT by Siemens, and the Discovery 710 by General Electric. For each scanning condition, the images were reconstructed with image voxel sizes of 1 to 4 mm cubic voxels. The reconstruction method used for each system was the one recommended by the manufacture to achieve best small image lesion detection results. A human observer study was performed to determine the smallest observable sphere for each scanning condition.

RESULTS

All systems were able to image the smallest sphere of 3.95 mm inner diameter at the 15 to 1 signal to background ratio when imaged for 16 min. For a typical whole body per bed position scan time of 2 to 4 min, the smallest imaged sphere varied between 4.95 and 6.23 mm at the 15:1 contrast ratio and 12.43 and 15.43 mm at a contrast ratio of 1.88:1. In general, all systems were consistent with the Rose criteria when determining lesion detectability.

CONCLUSIONS

Besides demonstrating that the current state of the art clinical PET/CT systems have the same lesion detection ability, the study demonstrates how sensitive scan time can be to detecting small lesions which have a relatively small contrast uptake in the range of just 2:1. This should help guide imaging protocols to use longer scan times over regions of the subject in which small lesions are suspect.

Automated model-based quantitative analysis of phantoms with spherical inserts in FDG PET scans

PURPOSE

Quality control plays an increasingly important role in quantitative PET imaging and is typically performed using phantoms. The purpose of this work was to develop and validate a fully automated analysis method for two common PET/CT quality assurance phantoms: the NEMA NU-2 IQ and SNMMI/CTN oncology phantom. The algorithm was designed to only utilize the PET scan to enable the analysis of phantoms with thin-walled inserts.

METHODS

We introduce a model-based method for automated analysis of phantoms with spherical inserts. Models are first constructed for each type of phantom to be analyzed. A robust insert detection algorithm uses the model to locate all inserts inside the phantom. First, candidates for inserts are detected using a scale-space detection approach. Second, candidates are given an initial label using a score-based optimization algorithm. Third, a robust model fitting step aligns the phantom model to the initial labeling and fixes incorrect labels. Finally, the detected insert locations are refined and measurements are taken for each insert and several background regions. In addition, an approach for automated selection of NEMA and CTN phantom models is presented. The method was evaluated on a diverse set of 15 NEMA and 20 CTN phantom PET/CT scans. NEMA phantoms were filled with radioactive tracer solution at 9.7:1 activity ratio over background, and CTN phantoms were filled with 4:1 and 2:1 activity ratio over background. For quantitative evaluation, an independent reference standard was generated by two experts using PET/CT scans of the phantoms. In addition, the automated approach was compared against manual analysis, which represents the current clinical standard approach, of the PET phantom scans by four experts.

RESULTS

The automated analysis method successfully detected and measured all inserts in all test phantom scans. It is a deterministic algorithm (zero variability), and the insert detection RMS error (i.e., bias) was 0.97, 1.12, and 1.48 mm for phantom activity ratios 9.7:1, 4:1, and 2:1, respectively. For all phantoms and at all contrast ratios, the average RMS error was found to be significantly lower for the proposed automated method compared to the manual analysis of the phantom scans. The uptake measurements produced by the automated method showed high correlation with the independent reference standard (R² ≥ 0.9987). In addition, the average computing time for the automated method was 30.6 s and was found to be significantly lower (P ≪ 0.001) compared to manual analysis (mean: 247.8 s).

CONCLUSIONS

The proposed automated approach was found to have less error when measured against the independent reference than the manual approach. It can be easily adapted to other phantoms with spherical inserts. In addition, it eliminates inter- and intraoperator variability in PET phantom analysis and is significantly more time efficient, and therefore, represents a promising approach to facilitate and simplify PET standardization and harmonization efforts.

Toward a standard for the evaluation of PET-Auto-Segmentation methods following the recommendations of AAPM task group No. 211: Requirements and implementation

Purpose

The aim of this paper is to define the requirements and describe the design and implementation of a standard benchmark tool for evaluation and validation of PET‐auto‐segmentation (PET‐AS) algorithms. This work follows the recommendations of Task Group 211 (TG211) appointed by the American Association of Physicists in Medicine (AAPM).

Methods

The recommendations published in the AAPM TG211 report were used to derive a set of required features and to guide the design and structure of a benchmarking software tool. These items included the selection of appropriate representative data and reference contours obtained from established approaches and the description of available metrics. The benchmark was designed in a way that it could be extendable by inclusion of bespoke segmentation methods, while maintaining its main purpose of being a standard testing platform for newly developed PET‐AS methods. An example of implementation of the proposed framework, named PETASset, was built. In this work, a selection of PET‐AS methods representing common approaches to PET image segmentation was evaluated within PETASset for the purpose of testing and demonstrating the capabilities of the software as a benchmark platform.

Results

A selection of clinical, physical, and simulated phantom data, including “best estimates” reference contours from macroscopic specimens, simulation template, and CT scans was built into the PETASset application database. Specific metrics such as Dice Similarity Coefficient (DSC), Positive Predictive Value (PPV), and Sensitivity (S), were included to allow the user to compare the results of any given PET‐AS algorithm to the reference contours. In addition, a tool to generate structured reports on the evaluation of the performance of PET‐AS algorithms against the reference contours was built. The variation of the metric agreement values with the reference contours across the PET‐AS methods evaluated for demonstration were between 0.51 and 0.83, 0.44 and 0.86, and 0.61 and 1.00 for DSC, PPV, and the S metric, respectively. Examples of agreement limits were provided to show how the software could be used to evaluate a new algorithm against the existing state‐of‐the art.

Conclusions

PETASset provides a platform that allows standardizing the evaluation and comparison of different PET‐AS methods on a wide range of PET datasets. The developed platform will be available to users willing to evaluate their PET‐AS methods and contribute with more evaluation datasets.

Validation of phantom-based harmonization for patient harmonization

PURPOSE

To improve the precision of multicenter clinical trials, several efforts are underway to determine scanner-specific parameters for harmonization using standardized phantom measurements. The goal of this study was to test the correspondence between quantification in phantom and patient images and validate the use of phantoms for harmonization of patient images.

METHODS

The National Electrical Manufacturers' Association image quality phantom with hot spheres was scanned on two time-of-flight PET scanners. Whole-body [18 F]-fluorodeoxyglucose (FDG)-PET scans were acquired of subjects on the same systems. List-mode events from spheres (diam.: 10-28 mm) measured in air on each scanner were embedded into the phantom and subject list-mode data from each scanner to create lesions with known uptake with respect to the local background in the phantom and each subject's liver and lung regions, as a proxy to characterize true lesion quantification. Images were analyzed using the contrast recovery coefficient (CRC) typically used in phantom studies and serving as a surrogate for the standardized uptake value used clinically. Postreconstruction filtering (resolution recovery and Gaussian smoothing) was applied to determine if the effect on the phantom images translates equivalently to subject images. Three postfiltering strategies were selected to harmonize the CRCmean or CRCmax values between the two scanners based on the phantom measurements and then applied to the subject images.

RESULTS

Both the average CRCmean and CRCmax values for lesions embedded in the lung and liver in four subjects (BMI range 25-38) agreed to within 5% with the CRC values for lesions embedded in the phantom for all lesion sizes. In addition, the relative changes in CRCmean and CRCmax resulting from the application of the postfilters on the subject and phantom images were consistent within measurement uncertainty. Further, the root mean squared percent difference (RMSpd ) between CRC values on the two scanners calculated over the three sphere sizes was significantly reduced in the subjects using postfiltering strategies chosen to harmonize CRCmean or CRCmax based on phantom measurements: RMSpd of the CRCmean values in subjects was reduced from 36% to < 8% after harmonizing CRCmean , while RMSpd for CRCmax was reduced from ~33% to < 6% after harmonizing CRCmax with a different strategy. However, with this strategy designed to harmonize CRCmax , the RMSpd for CRCmean only improved to ~14% in subjects.

CONCLUSIONS

The consistency of the CRC measurements between the phantom and subject data demonstrates that harmonization strategies defined with phantom studies track well to patient images. However, quantitative agreement between different scanners as represented by the RMSpd depends on the metric chosen for harmonization.

Classification and evaluation strategies of auto-segmentation approaches for PET: Report of AAPM task group No. 211

PURPOSE

The purpose of this educational report is to provide an overview of the present state-of-the-art PET auto-segmentation (PET-AS) algorithms and their respective validation, with an emphasis on providing the user with help in understanding the challenges and pitfalls associated with selecting and implementing a PET-AS algorithm for a particular application.

APPROACH

A brief description of the different types of PET-AS algorithms is provided using a classification based on method complexity and type. The advantages and the limitations of the current PET-AS algorithms are highlighted based on current publications and existing comparison studies. A review of the available image datasets and contour evaluation metrics in terms of their applicability for establishing a standardized evaluation of PET-AS algorithms is provided. The performance requirements for the algorithms and their dependence on the application, the radiotracer used and the evaluation criteria are described and discussed. Finally, a procedure for algorithm acceptance and implementation, as well as the complementary role of manual and auto-segmentation are addressed.

FINDINGS

A large number of PET-AS algorithms have been developed within the last 20 years. Many of the proposed algorithms are based on either fixed or adaptively selected thresholds. More recently, numerous papers have proposed the use of more advanced image analysis paradigms to perform semi-automated delineation of the PET images. However, the level of algorithm validation is variable and for most published algorithms is either insufficient or inconsistent which prevents recommending a single algorithm. This is compounded by the fact that realistic image configurations with low signal-to-noise ratios (SNR) and heterogeneous tracer distributions have rarely been used. Large variations in the evaluation methods used in the literature point to the need for a standardized evaluation protocol.

CONCLUSIONS

Available comparison studies suggest that PET-AS algorithms relying on advanced image analysis paradigms provide generally more accurate segmentation than approaches based on PET activity thresholds, particularly for realistic configurations. However, this may not be the case for simple shape lesions in situations with a narrower range of parameters, where simpler methods may also perform well. Recent algorithms which employ some type of consensus or automatic selection between several PET-AS methods have potential to overcome the limitations of the individual methods when appropriately trained. In either case, accuracy evaluation is required for each different PET scanner and scanning and image reconstruction protocol. For the simpler, less robust approaches, adaptation to scanning conditions, tumor type, and tumor location by optimization of parameters is necessary. The results from the method evaluation stage can be used to estimate the contouring uncertainty. All PET-AS contours should be critically verified by a physician. A standard test, i.e., a benchmark dedicated to evaluating both existing and future PET-AS algorithms needs to be designed, to aid clinicians in evaluating and selecting PET-AS algorithms and to establish performance limits for their acceptance for clinical use. The initial steps toward designing and building such a standard are undertaken by the task group members.

Multi-site quality and variability analysis of 3D FDG PET segmentations based on phantom and clinical image data

PURPOSE

Radiomics utilizes a large number of image-derived features for quantifying tumor characteristics that can in turn be correlated with response and prognosis. Unfortunately, extraction and analysis of such image-based features is subject to measurement variability and bias. The challenge for radiomics is particularly acute in Positron Emission Tomography (PET) where limited resolution, a high noise component related to the limited stochastic nature of the raw data, and the wide variety of reconstruction options confound quantitative feature metrics. Extracted feature quality is also affected by tumor segmentation methods used to define regions over which to calculate features, making it challenging to produce consistent radiomics analysis results across multiple institutions that use different segmentation algorithms in their PET image analysis. Understanding each element contributing to these inconsistencies in quantitative image feature and metric generation is paramount for ultimate utilization of these methods in multi-institutional trials and clinical oncology decision making.

METHODS

To assess segmentation quality and consistency at the multi-institutional level, we conducted a study of seven institutional members of the National Cancer Institute Quantitative Imaging Network. For the study, members were asked to segment a common set of phantom PET scans acquired over a range of imaging conditions as well as a second set of head and neck cancer (HNC) PET scans. Segmentations were generated at each institution using their preferred approach. In addition, participants were asked to repeat segmentations with a time interval between initial and repeat segmentation. This procedure resulted in overall 806 phantom insert and 641 lesion segmentations. Subsequently, the volume was computed from the segmentations and compared to the corresponding reference volume by means of statistical analysis.

RESULTS

On the two test sets (phantom and HNC PET scans), the performance of the seven segmentation approaches was as follows. On the phantom test set, the mean relative volume errors ranged from 29.9 to 87.8% of the ground truth reference volumes, and the repeat difference for each institution ranged between -36.4 to 39.9%. On the HNC test set, the mean relative volume error ranged between -50.5 to 701.5%, and the repeat difference for each institution ranged between -37.7 to 31.5%. In addition, performance measures per phantom insert/lesion size categories are given in the paper. On phantom data, regression analysis resulted in coefficient of variation (CV) components of 42.5% for scanners, 26.8% for institutional approaches, 21.1% for repeated segmentations, 14.3% for relative contrasts, 5.3% for count statistics (acquisition times), and 0.0% for repeated scans. Analysis showed that the CV components for approaches and repeated segmentations were significantly larger on the HNC test set with increases by 112.7% and 102.4%, respectively.

CONCLUSION

Analysis results underline the importance of PET scanner reconstruction harmonization and imaging protocol standardization for quantification of lesion volumes. In addition, to enable a distributed multi-site analysis of FDG PET images, harmonization of analysis approaches and operator training in combination with highly automated segmentation methods seems to be advisable. Future work will focus on quantifying the impact of segmentation variation on radiomics system performance.

Technical Note: Development of a 3D printed subresolution sandwich phantom for validation of brain SPECT analysis

PURPOSE

To make an adaptable, head shaped radionuclide phantom to simulate molecular imaging of the brain using clinical acquisition and reconstruction protocols. This will allow the characterization and correction of scanner characteristics, and improve the accuracy of clinical image analysis, including the application of databases of normal subjects.

METHODS

A fused deposition modeling 3D printer was used to create a head shaped phantom made up of transaxial slabs, derived from a simulated MRI dataset. The attenuation of the printed polylactide (PLA), measured by means of the Hounsfield unit on CT scanning, was set to match that of the brain by adjusting the proportion of plastic filament and air (fill ratio). Transmission measurements were made to verify the attenuation of the printed slabs. The radionuclide distribution within the phantom was created by adding (99m)Tc pertechnetate to the ink cartridge of a paper printer and printing images of gray and white matter anatomy, segmented from the same MRI data. The complete subresolution sandwich phantom was assembled from alternate 3D printed slabs and radioactive paper sheets, and then imaged on a dual headed gamma camera to simulate an HMPAO SPECT scan.

RESULTS

Reconstructions of phantom scans successfully used automated ellipse fitting to apply attenuation correction. This removed the variability inherent in manual application of attenuation correction and registration inherent in existing cylindrical phantom designs. The resulting images were assessed visually and by count profiles and found to be similar to those from an existing elliptical PMMA phantom.

CONCLUSIONS

The authors have demonstrated the ability to create physically realistic HMPAO SPECT simulations using a novel head-shaped 3D printed subresolution sandwich method phantom. The phantom can be used to validate all neurological SPECT imaging applications. A simple modification of the phantom design to use thinner slabs would make it suitable for use in PET.

A Dirichlet process mixture model for automatic (18)F-FDG PET image segmentation: Validation study on phantoms and on lung and esophageal lesions

PURPOSE

The aim of this study was to implement a Dirichlet process mixture (DPM) model for automatic tumor edge identification on (18)F-fluorodeoxyglucose positron emission tomography ((18)F-FDG PET) images by optimizing the parameters on which the algorithm depends, to validate it experimentally, and to test its robustness.

METHODS

The DPM model belongs to the class of the Bayesian nonparametric models and uses the Dirichlet process prior for flexible nonparametric mixture modeling, without any preliminary choice of the number of mixture components. The DPM algorithm implemented in the statistical software package R was used in this work. The contouring accuracy was evaluated on several image data sets: on an IEC phantom (spherical inserts with diameter in the range 10-37 mm) acquired by a Philips Gemini Big Bore PET-CT scanner, using 9 different target-to-background ratios (TBRs) from 2.5 to 70; on a digital phantom simulating spherical/uniform lesions and tumors, irregular in shape and activity; and on 20 clinical cases (10 lung and 10 esophageal cancer patients). The influence of the DPM parameters on contour generation was studied in two steps. In the first one, only the IEC spheres having diameters of 22 and 37 mm and a sphere of the digital phantom (41.6 mm diameter) were studied by varying the main parameters until the diameter of the spheres was obtained within 0.2% of the true value. In the second step, the results obtained for this training set were applied to the entire data set to determine DPM based volumes of all available lesions. These volumes were compared to those obtained by applying already known algorithms (Gaussian mixture model and gradient-based) and to true values, when available.

RESULTS

Only one parameter was found able to significantly influence segmentation accuracy (ANOVA test). This parameter was linearly connected to the uptake variance of the tested region of interest (ROI). In the first step of the study, a calibration curve was determined to automatically generate the optimal parameter from the variance of the ROI. This "calibration curve" was then applied to contour the whole data set. The accuracy (mean discrepancy between DPM model-based contours and reference contours) of volume estimation was below (1 ± 7)% on the whole data set (1 SD). The overlap between true and automatically segmented contours, measured by the Dice similarity coefficient, was 0.93 with a SD of 0.03.

CONCLUSIONS

The proposed DPM model was able to accurately reproduce known volumes of FDG concentration, with high overlap between segmented and true volumes. For all the analyzed inserts of the IEC phantom, the algorithm proved to be robust to variations in radius and in TBR. The main advantage of this algorithm was that no setting of DPM parameters was required in advance, since the proper setting of the only parameter that could significantly influence the segmentation results was automatically related to the uptake variance of the chosen ROI. Furthermore, the algorithm did not need any preliminary choice of the optimum number of classes to describe the ROIs within PET images and no assumption about the shape of the lesion and the uptake heterogeneity of the tracer was required.

Head and neck target delineation using a novel PET automatic segmentation algorithm

PURPOSE

To evaluate the feasibility and impact of using a novel advanced PET auto-segmentation method in Head and Neck (H&N) radiotherapy treatment (RT) planning.

METHODS

ATLAAS, Automatic decision Tree-based Learning Algorithm for Advanced Segmentation, previously developed and validated on pre-clinical data, was applied to ¹⁸F-FDG-PET/CT scans of 20 H&N patients undergoing Intensity Modulated Radiation Therapy. Primary Gross Tumour Volumes (GTVs) manually delineated on CT/MRI scans (GTVp_CT/MRI), together with ATLAAS-generated contours (GTVp_ATLAAS) were used to derive the RT planning GTV (GTVp_final). ATLAAS outlines were compared to CT/MRI and final GTVs qualitatively and quantitatively using a conformity metric.

RESULTS

The ATLAAS contours were found to be reliable and useful. The volume of GTVp_ATLAAS was smaller than GTVp_CT/MRI in 70% of the cases, with an average conformity index of 0.70. The information provided by ATLAAS was used to grow the GTVp_CT/MRI in 10 cases (up to 10.6mL) and to shrink the GTVp_CT/MRI in 7 cases (up to 12.3mL). ATLAAS provided complementary information to CT/MRI and GTVp_ATLAAS contributed to up to 33% of the final GTV volume across the patient cohort.

CONCLUSIONS

ATLAAS can deliver operator independent PET segmentation to augment clinical outlining using CT and MRI and could have utility in future clinical studies.

Point-spread function reconstructed PET images of sub-centimeter lesions are not quantitative

Background

PET image reconstruction methods include modeling of resolution degrading phenomena, often referred to as point-spread function (PSF) reconstruction. The aim of this study was to develop a clinically relevant phantom and characterize the reproducibility and accuracy of high-resolution PSF reconstructed images of small lesions, which is a prerequisite for using PET in the prediction and evaluation of responses to treatment.

Sets of small homogeneous ¹⁸F-spheres (range 3–12 mm diameter, relevant for small lesions and lymph nodes) were suspended and covered by a ¹¹C-silicone, which provided a scattering medium and a varying sphere-to-background ratio. Repeated measurements were made on PET/CT scanners from two vendors using a wide range of reconstruction parameters. Recovery coefficients (RCs) were measured for clinically used volume-of-interest definitions.

Results

For non-PSF images, RCs were reproducible and fell monotonically as the sphere diameter decreased, which is the expected behavior. PSF images converged slower and had artifacts: RCs did not fall monotonically as sphere diameters decreased but had a maximum RC for sphere sizes around 8 mm, RCs could be greater than 1, and RCs were less reproducible. To some degree, post-reconstruction filters could suppress PSF artifacts.

Conclusions

High-resolution PSF images of small lesions showed artifacts that could lead to serious misinterpretations when used for monitoring treatment response. Thus, it could be safer to use non-PSF reconstruction for quantitative purposes unless PSF reconstruction parameters are optimized for the specific task.

Electronic supplementary material

The online version of this article (doi:10.1186/s40658-016-0169-9) contains supplementary material, which is available to authorized users.

SPEQTACLE: An automated generalized fuzzy C-means algorithm for tumor delineation in PET

PURPOSE

Accurate tumor delineation in positron emission tomography (PET) images is crucial in oncology. Although recent methods achieved good results, there is still room for improvement regarding tumors with complex shapes, low signal-to-noise ratio, and high levels of uptake heterogeneity.

METHODS

The authors developed and evaluated an original clustering-based method called spatial positron emission quantification of tumor-Automatic Lp-norm estimation (SPEQTACLE), based on the fuzzy C-means (FCM) algorithm with a generalization exploiting a Hilbertian norm to more accurately account for the fuzzy and non-Gaussian distributions of PET images. An automatic and reproducible estimation scheme of the norm on an image-by-image basis was developed. Robustness was assessed by studying the consistency of results obtained on multiple acquisitions of the NEMA phantom on three different scanners with varying acquisition parameters. Accuracy was evaluated using classification errors (CEs) on simulated and clinical images. SPEQTACLE was compared to another FCM implementation, fuzzy local information C-means (FLICM) and fuzzy locally adaptive Bayesian (FLAB).

RESULTS

SPEQTACLE demonstrated a level of robustness similar to FLAB (variability of 14% ± 9% vs 14% ± 7%, p = 0.15) and higher than FLICM (45% ± 18%, p < 0.0001), and improved accuracy with lower CE (14% ± 11%) over both FLICM (29% ± 29%) and FLAB (22% ± 20%) on simulated images. Improvement was significant for the more challenging cases with CE of 17% ± 11% for SPEQTACLE vs 28% ± 22% for FLAB (p = 0.009) and 40% ± 35% for FLICM (p < 0.0001). For the clinical cases, SPEQTACLE outperformed FLAB and FLICM (15% ± 6% vs 37% ± 14% and 30% ± 17%, p < 0.004).

CONCLUSIONS

SPEQTACLE benefitted from the fully automatic estimation of the norm on a case-by-case basis. This promising approach will be extended to multimodal images and multiclass estimation in future developments.

PETSTEP: Generation of synthetic PET lesions for fast evaluation of segmentation methods

Purpose

This work describes PETSTEP (PET Simulator of Tracers via Emission Projection): a faster and more accessible alternative to Monte Carlo (MC) simulation generating realistic PET images, for studies assessing image features and segmentation techniques.

Methods

PETSTEP was implemented within Matlab as open source software. It allows generating three-dimensional PET images from PET/CT data or synthetic CT and PET maps, with user-drawn lesions and user-set acquisition and reconstruction parameters. PETSTEP was used to reproduce images of the NEMA body phantom acquired on a GE Discovery 690 PET/CT scanner, and simulated with MC for the GE Discovery LS scanner, and to generate realistic Head and Neck scans. Finally the sensitivity (S) and Positive Predictive Value (PPV) of three automatic segmentation methods were compared when applied to the scanner-acquired and PETSTEP-simulated NEMA images.

Results

PETSTEP produced 3D phantom and clinical images within 4 and 6 min respectively on a single core 2.7 GHz computer. PETSTEP images of the NEMA phantom had mean intensities within 2% of the scanner-acquired image for both background and largest insert, and 16% larger background Full Width at Half Maximum. Similar results were obtained when comparing PETSTEP images to MC simulated data. The S and PPV obtained with simulated phantom images were statistically significantly lower than for the original images, but led to the same conclusions with respect to the evaluated segmentation methods.

Conclusions

PETSTEP allows fast simulation of synthetic images reproducing scanner-acquired PET data and shows great promise for the evaluation of PET segmentation methods.

A novel phantom technique for evaluating the performance of PET auto-segmentation methods in delineating heterogeneous and irregular lesions

BACKGROUND

Positron Emission Tomography (PET)-based automatic segmentation (PET-AS) methods can improve tumour delineation for radiotherapy treatment planning, particularly for Head and Neck (H&N) cancer. Thorough validation of PET-AS on relevant data is currently needed. Printed subresolution sandwich (SS) phantoms allow modelling heterogeneous and irregular tracer uptake, while providing reference uptake data. This work aimed to demonstrate the usefulness of the printed SS phantom technique in recreating complex realistic H&N radiotracer uptake for evaluating several PET-AS methods.

METHODS

Ten SS phantoms were built from printouts representing 2mm-spaced slices of modelled H&N uptake, printed using black ink mixed with 18F-fluorodeoxyglucose, and stacked between 2mm thick plastic sheets. Spherical lesions were modelled for two contrasted uptake levels, and irregular and spheroidal tumours were modelled for homogeneous, and heterogeneous uptake including necrotic patterns. The PET scans acquired were segmented with ten custom PET-AS methods: adaptive iterative thresholding (AT), region growing, clustering applied to 2 to 8 clusters, and watershed transform-based segmentation. The difference between the resulting contours and the ground truth from the image template was evaluated using the Dice Similarity Coefficient (DSC), Sensitivity and Positive Predictive value.

RESULTS

Realistic H&N images were obtained within 90 min of preparation. The sensitivity of binary PET-AS and clustering using small numbers of clusters dropped for highly heterogeneous spheres. The accuracy of PET-AS methods dropped between 4% and 68% for irregular lesions compared to spheres of the same volume. For each geometry and uptake modelled with the SS phantoms, we report the number of clusters resulting in optimal segmentation. Radioisotope distributions representing necrotic uptakes proved most challenging for most methods. Two PET-AS methods did not include the necrotic region in the segmented volume.

CONCLUSIONS

Printed SS phantoms allowed identifying advantages and drawbacks of the different methods, determining the most robust PET-AS for the segmentation of heterogeneities and complex geometries, and quantifying differences across methods in the delineation of necrotic lesions. The printed SS phantom technique provides key advantages in the development and evaluation of PET segmentation methods and has a future in the field of radioisotope imaging.

Impact of point spread function modelling and time of flight on FDG uptake measurements in lung lesions using alternative filtering strategies

BACKGROUND

The use of maximum standardised uptake value (SUVmax) is commonplace in oncology positron emission tomography (PET). Point spread function (PSF) modelling and time-of-flight (TOF) reconstructions have a significant impact on SUVmax, presenting a challenge for centres with defined protocols for lesion classification based on SUVmax thresholds. This has perhaps led to the slow adoption of these reconstructions. This work evaluated the impact of PSF and/or TOF reconstructions on SUVmax, SUVpeak and total lesion glycolysis (TLG) under two different schemes of post-filtering.

METHODS

Post-filters to match voxel variance or SUVmax were determined using a NEMA NU-2 phantom. Images from 68 consecutive lung cancer patients were reconstructed with the standard iterative algorithm along with TOF; PSF modelling - Siemens HD·PET (HD); and combined PSF modelling and TOF - Siemens ultraHD·PET (UHD) with the two post-filter sets. SUVmax, SUVpeak, TLG and signal-to-noise ratio of tumour relative to liver (SNR(T-L)) were measured in 74 lesions for each reconstruction. Relative differences in uptake measures were calculated, and the clinical impact of any changes was assessed using published guidelines and local practice.

RESULTS

When matching voxel variance, SUVmax increased substantially (mean increase +32% and +49% for HD and UHD, respectively), potentially impacting outcome in the majority of patients. Increases in SUVpeak were less notable (mean increase +17% and +23% for HD and UHD, respectively). Increases with TOF alone were far less for both measures. Mean changes to TLG were <10% for all algorithms for either set of post-filters. SNR(T-L) were greater than ordered subset expectation maximisation (OSEM) in all reconstructions using both post-filtering sets.

CONCLUSIONS

Matching image voxel variance with PSF and/or TOF reconstructions, particularly with PSF modelling and in small lesions, resulted in considerable increases in SUVmax, inhibiting the use of defined protocols for lesion classification based on SUVmax. However, reduced partial volume effects may increase lesion detectability. Matching SUVmax in phantoms translated well to patient studies for PSF reconstruction but less well with TOF, where a small positive bias was observed in patient images. Matching SUVmax significantly reduced voxel variance and potential variability of uptake measures. Finally, TLG may be less sensitive to reconstruction methods compared with either SUVmax or SUVpeak.

Cold wall effect eliminating method to determine the contrast recovery coefficient for small animal PET scanners using the NEMA NU-4 image quality phantom

The contrast recovery coefficients (CRC) were evaluated for five different small animal PET scanners: GE Explore Vista, Genisys4, MiniPET-2, nanoScan PC and Siemens Inveon. The NEMA NU-4 2008 performance test with the suggested image quality phantom (NU4IQ) does not allow the determination of the CRC values for the hot regions in the phantom. This drawback of NU4IQ phantom motivated us to develop a new method for this purpose. The method includes special acquisition and reconstruction protocols using the original phantom, and results in an artificially merged image enabling the evaluation of CRC values. An advantageous feature of this method is that it stops the cold wall effect from distorting the CRC calculation. Our suggested protocol results in a set of CRC values contributing to the characterization of small animal PET scanners. GATE simulations were also performed to validate the new method and verify the evaluated CRC values. We also demonstrated that the numerical values of this parameter depend on the actual object contrast of the hot region(s) and this mainly comes from the spillover effect. This effect was also studied while analysing the background activity level around the hot rods. We revealed that the calculated background mean values depended on the target contrast in a scanner specific manner. Performing the artificially merged imaging procedure and additional simulations using the micro hollow sphere (MHS) phantom geometry, we also proved that the inactive wall around the hot spheres can have a remarkable impact on the calculated CRC. In conclusion, we have shown that the proposed artificial merging procedure and the commonly used NU4IQ phantom prescribed by the NEMA NU-4 can easily deliver reliable CRC data otherwise unavailable for the NU4IQ phantom in the conventional protocol or the MHS phantom.

Use of wall-less ¹⁸F-doped gelatin phantoms for improved volume delineation and quantification in PET/CT

Positron emission tomography (PET) with (18)F-FDG is a valuable tool for staging, planning treatment, and evaluating the treatment response for many different types of tumours. The correct volume estimation is of utmost importance in these situations. To date, the most common types of phantoms used in volume quantification in PET utilize fillable, hollow spheres placed in a circular or elliptical cylinder made of polymethyl methacrylate. However, the presence of a non-radioactive sphere wall between the hotspot and the background activity in images of this type of phantom could cause inaccuracies. To investigate the influence of the non-active walls, we developed a phantom without non-active sphere walls for volume delineation and quantification in PET. Three sizes of gelatin hotspots were moulded and placed in a Jaszczak phantom together with hollow plastic spheres of the same sizes containing the same activity concentration. (18)F PET measurements were made with zero background activity and with tumour-to-background ratios of 12.5, 10, 7.5, and 5. The background-corrected volume reproducing threshold, Tvol, was calculated for both the gelatin and the plastic spheres. It was experimentally verified that the apparent background dependence of Tvol, i.e., a decreasing Tvol with increasing background fraction, was not present for wall-less spheres; the opposite results were seen in plastic, hollow spheres in commercially-available phantoms. For the types of phantoms commonly used in activity quantification, the estimation of Tvol using fillable, hollow, plastic spheres with non-active walls would lead to an overestimate of the tumour volume, especially for small volumes in a high activity background.