Generic and robust method for automatic segmentation of PET images using an active contour model

Challenges and limitations in applying radiomics to PET imaging: Possible opportunities and avenues for research

Radiomics, the high-throughput extraction of quantitative imaging features from medical images, holds immense potential for advancing precision medicine in oncology and beyond. While radiomics applied to positron emission tomography (PET) imaging offers unique insights into tumor biology and treatment response, it is imperative to elucidate the challenges and constraints inherent in this domain to facilitate their translation into clinical practice. This review examines the challenges and limitations of applying radiomics to PET imaging, synthesizing findings from the last five years (2019-2023) and highlights the significance of addressing these challenges to realize the full clinical potential of radiomics in oncology and molecular imaging. A comprehensive search was conducted across multiple electronic databases, including PubMed, Scopus, and Web of Science, using keywords relevant to radiomics issues in PET imaging. Only studies published in peer-reviewed journals were eligible for inclusion in this review. Although many studies have highlighted the potential of radiomics in predicting treatment response, assessing tumor heterogeneity, enabling risk stratification, and personalized therapy selection, various challenges regarding the practical implementation of the proposed models still need to be addressed. This review illustrates the challenges and limitations of radiomics in PET imaging across various cancer types, encompassing both phantom and clinical investigations. The analyzed studies highlight the importance of reproducible segmentation methods, standardized pre-processing and post-processing methodologies, and the need to create large multicenter studies registered in a centralized database to promote the continuous validation and clinical integration of radiomics into PET imaging.

Does consensus contour improve robustness and accuracy in 18F-FDG PET radiomic features?

PURPOSE

The purpose of our study is to validate the robustness and accuracy of consensus contour in 2-deoxy-2-[18 F]fluoro-D-glucose (18 F-FDG) PET radiomic features.

METHODS

225 nasopharyngeal carcinoma (NPC) and 13 extended cardio-torso (XCAT) simulated data were enrolled. All segmentation were performed with four segmentation methods under two different initial masks, respectively. Consensus contour (ConSeg) was then developed using the majority vote rule. 107 radiomic features were extracted by Pyradiomics based on segmentation and the intraclass correlation coefficient (ICC) was calculated for each feature between masks or among segmentation, respectively. In XCAT ICC between segmentation and simulated ground truth were also calculated to access the accuracy.

RESULTS

ICC varied with the dataset, segmentation method, initial mask and feature type. ConSeg presented higher ICC for radiomic features in robustness tests and similar ICC in accuracy tests, compared with the average of four segmentation results. Higher ICC were also generally observed in irregular initial masks compared with rectangular masks in both robustness and accuracy tests. Furthermore, 19 features (17.76%) had ICC ≥ 0.75 in both robustness and accuracy tests for any of the segmentation methods or initial masks. The dataset was observed to have a large impact on the correlation relationships between radiomic features, but not the segmentation method or initial mask.

CONCLUSIONS

The consensus contour combined with irregular initial mask could improve the robustness and accuracy in radiomic analysis to some extent. The correlation relationships between radiomic features and feature clusters largely depended on the dataset, but not segmentation method or initial mask.

Comparing the performance of a deep learning-based lung gross tumour volume segmentation algorithm before and after transfer learning in a new hospital

Objectives

Radiation therapy for lung cancer requires a gross tumour volume (GTV) to be carefully outlined by a skilled radiation oncologist (RO) to accurately pinpoint high radiation dose to a malignant mass while simultaneously minimizing radiation damage to adjacent normal tissues. This is manually intensive and tedious however, it is feasible to train a deep learning (DL) neural network that could assist ROs to delineate the GTV. However, DL trained on large openly accessible data sets might not perform well when applied to a superficially similar task but in a different clinical setting. In this work, we tested the performance of DL automatic lung GTV segmentation model trained on open-access Dutch data when used on Indian patients from a large public tertiary hospital, and hypothesized that generic DL performance could be improved for a specific local clinical context, by means of modest transfer-learning on a small representative local subset.

Methods

X-ray computed tomography (CT) series in a public data set called "NSCLC-Radiomics" from The Cancer Imaging Archive was first used to train a DL-based lung GTV segmentation model (Model 1). Its performance was assessed using a different open access data set (Interobserver1) of Dutch subjects plus a private Indian data set from a local tertiary hospital (Test Set 2). Another Indian data set (Retrain Set 1) was used to fine-tune the former DL model using a transfer learning method. The Indian data sets were taken from CT of a hybrid scanner based in nuclear medicine, but the GTV was drawn by skilled Indian ROs. The final (after fine-tuning) model (Model 2) was then re-evaluated in "Interobserver1" and "Test Set 2." Dice similarity coefficient (DSC), precision, and recall were used as geometric segmentation performance metrics.

Results

Model 1 trained exclusively on Dutch scans showed a significant fall in performance when tested on "Test Set 2." However, the DSC of Model 2 recovered by 14 percentage points when evaluated in the same test set. Precision and recall showed a similar rebound of performance after transfer learning, in spite of using a comparatively small sample size. The performance of both models, before and after the fine-tuning, did not significantly change the segmentation performance in "Interobserver1."

Conclusions

A large public open-access data set was used to train a generic DL model for lung GTV segmentation, but this did not perform well initially in the Indian clinical context. Using transfer learning methods, it was feasible to efficiently and easily fine-tune the generic model using only a small number of local examples from the Indian hospital. This led to a recovery of some of the geometric segmentation performance, but the tuning did not appear to affect the performance of the model in another open-access data set.

Advances in knowledge

Caution is needed when using models trained on large volumes of international data in a local clinical setting, even when that training data set is of good quality. Minor differences in scan acquisition and clinician delineation preferences may result in an apparent drop in performance. However, DL models have the advantage of being efficiently "adapted" from a generic to a locally specific context, with only a small amount of fine-tuning by means of transfer learning on a small local institutional data set.

Does consensus contours improve robustness and accuracy on [Formula: see text]F-FDG PET imaging tumor delineation?

PURPOSE

The aim of this study is to explore the robustness and accuracy of consensus contours with 225 nasopharyngeal carcinoma (NPC) clinical cases and 13 extended cardio-torso simulated lung tumors (XCAT) based on 2-deoxy-2-[[Formula: see text]F]fluoro-D-glucose ([Formula: see text]F-FDG) PET imaging.

METHODS

Primary tumor segmentation was performed with two different initial masks on 225 NPC [Formula: see text]F-FDG PET datasets and 13 XCAT simulations using methods of automatic segmentation with active contour, affinity propagation (AP), contrast-oriented thresholding (ST), and 41% maximum tumor value (41MAX), respectively. Consensus contours (ConSeg) were subsequently generated based on the majority vote rule. The metabolically active tumor volume (MATV), relative volume error (RE), Dice similarity coefficient (DSC) and their respective test-retest (TRT) metrics between different masks were adopted to analyze the results quantitatively. The nonparametric Friedman and post hoc Wilcoxon tests with Bonferroni adjustment for multiple comparisons were performed with [Formula: see text] 0.05 considered to be significant.

RESULTS

AP presented the highest variability for MATV in different masks, and ConSeg presented much better TRT performances in MATV compared with AP, and slightly poorer TRT in MATV compared with ST or 41MAXin most cases. Similar trends were also found in RE and DSC with the simulated data. The average of four segmentation results (AveSeg) showed better or comparable results in accuracy for most cases with respect to ConSeg. AP, AveSeg and ConSeg presented better RE and DSC in irregular masks as compared with rectangle masks. Additionally, all methods underestimated the tumour boundaries in relation to the ground truth for XCAT including respiratory motion.

CONCLUSIONS

The consensus method could be a robust approach to alleviate segmentation variabilities, but did not seem to improve the accuracy of segmentation results on average. Irregular initial masks might be at least in some cases attributable to mitigate the segmentation variability as well.

Fully Automated Gross Tumor Volume Delineation From PET in Head and Neck Cancer Using Deep Learning Algorithms

PURPOSE

The availability of automated, accurate, and robust gross tumor volume (GTV) segmentation algorithms is critical for the management of head and neck cancer (HNC) patients. In this work, we evaluated 3 state-of-the-art deep learning algorithms combined with 8 different loss functions for PET image segmentation using a comprehensive training set and evaluated its performance on an external validation set of HNC patients.

PATIENTS AND METHODS

18F-FDG PET/CT images of 470 patients presenting with HNC on which manually defined GTVs serving as standard of reference were used for training (340 patients), evaluation (30 patients), and testing (100 patients from different centers) of these algorithms. PET image intensity was converted to SUVs and normalized in the range (0-1) using the SUVmax of the whole data set. PET images were cropped to 12 × 12 × 12 cm3 subvolumes using isotropic voxel spacing of 3 × 3 × 3 mm3 containing the whole tumor and neighboring background including lymph nodes. We used different approaches for data augmentation, including rotation (-15 degrees, +15 degrees), scaling (-20%, 20%), random flipping (3 axes), and elastic deformation (sigma = 1 and proportion to deform = 0.7) to increase the number of training sets. Three state-of-the-art networks, including Dense-VNet, NN-UNet, and Res-Net, with 8 different loss functions, including Dice, generalized Wasserstein Dice loss, Dice plus XEnt loss, generalized Dice loss, cross-entropy, sensitivity-specificity, and Tversky, were used. Overall, 28 different networks were built. Standard image segmentation metrics, including Dice similarity, image-derived PET metrics, first-order, and shape radiomic features, were used for performance assessment of these algorithms.

RESULTS

The best results in terms of Dice coefficient (mean ± SD) were achieved by cross-entropy for Res-Net (0.86 ± 0.05; 95% confidence interval [CI], 0.85-0.87), Dense-VNet (0.85 ± 0.058; 95% CI, 0.84-0.86), and Dice plus XEnt for NN-UNet (0.87 ± 0.05; 95% CI, 0.86-0.88). The difference between the 3 networks was not statistically significant (P > 0.05). The percent relative error (RE%) of SUVmax quantification was less than 5% in networks with a Dice coefficient more than 0.84, whereas a lower RE% (0.41%) was achieved by Res-Net with cross-entropy loss. For maximum 3-dimensional diameter and sphericity shape features, all networks achieved a RE ≤ 5% and ≤10%, respectively, reflecting a small variability.

CONCLUSIONS

Deep learning algorithms exhibited promising performance for automated GTV delineation on HNC PET images. Different loss functions performed competitively when using different networks and cross-entropy for Res-Net, Dense-VNet, and Dice plus XEnt for NN-UNet emerged as reliable networks for GTV delineation. Caution should be exercised for clinical deployment owing to the occurrence of outliers in deep learning-based algorithms.

Development of a new fully three-dimensional methodology for tumours delineation in functional images

Delineation of tumours in Positron Emission Tomography (PET) plays a crucial role in accurate diagnosis and radiotherapy treatment planning. In this context, it is of outmost importance to devise efficient and operator-independent segmentation algorithms capable of reconstructing the tumour three-dimensional (3D) shape. In previous work, we proposed a system for 3D tumour delineation on PET data (expressed in terms of Standardized Uptake Value - SUV), based on a two-step approach. Step 1 identified the slice enclosing the maximum SUV and generated a rough contour surrounding it. Such contour was then used to initialize step 2, where the 3D shape of the tumour was obtained by separately segmenting 2D PET slices, leveraging the slice-by-slice marching approach. Additionally, we combined active contours and machine learning components to improve performance. Despite its success, the slice marching approach poses unnecessary limitations that are naturally removed by performing the segmentation directly in 3D. In this paper, we migrate our system into 3D. In particular, the segmentation in step 2 is now performed by evolving an active surface directly in the 3D space. The key points of such an advancement are that it performs the shape reconstruction on the whole stack of slices simultaneously, naturally leveraging cross-slice information that could not be exploited before. Additionally, it does not require any specific stopping condition, as the active surface naturally reaches a stable topology once convergence is achieved. Performance of this fully 3D approach is evaluated on the same dataset discussed in our previous work, which comprises fifty PET scans of lung, head and neck, and brain tumours. The results have confirmed that a benefit is indeed achieved in practice for all investigated anatomical districts, both quantitatively, through a set of commonly used quality indicators (dice similarity coefficient >87.66%, Hausdorff distance < 1.48 voxel and Mahalanobis distance < 0.82 voxel), and qualitatively in terms of Likert score (>3 in 54% of the tumours).

Deep Learning Improved Clinical Target Volume Contouring Quality and Efficiency for Postoperative Radiation Therapy in Non-small Cell Lung Cancer

Purpose: To investigate whether a deep learning-assisted contour (DLAC) could provide greater accuracy, inter-observer consistency, and efficiency compared with a manual contour (MC) of the clinical target volume (CTV) for non-small cell lung cancer (NSCLC) receiving postoperative radiotherapy (PORT).

Materials and Methods: A deep dilated residual network was used to achieve the effective automatic contour of the CTV. Eleven junior physicians contoured CTVs on 19 patients by using both MC and DLAC methods independently. Compared with the ground truth, the accuracy of the contour was evaluated by using the Dice coefficient and mean distance to agreement (MDTA). The coefficient of variation (CV) and standard distance deviation (SDD) were rendered to measure the inter-observer variability or consistency. The time consumed for each of the two contouring methods was also compared.

Results: A total of 418 CTV sets were generated. DLAC improved contour accuracy when compared with MC and was associated with a larger Dice coefficient (mean ± SD: 0.75 ± 0.06 vs. 0.72 ± 0.07, p < 0.001) and smaller MDTA (mean ± SD: 2.97 ± 0.91 mm vs. 3.07 ± 0.98 mm, p < 0.001). The DLAC was also associated with decreased inter-observer variability, with a smaller CV (mean ± SD: 0.129 ± 0.040 vs. 0.183 ± 0.043, p < 0.001) and SDD (mean ± SD: 0.47 ± 0.22 mm vs. 0.72 ± 0.41 mm, p < 0.001). In addition, a value of 35% of time saving was provided by the DLAC (median: 14.81 min vs. 9.59 min, p < 0.001).

Conclusions: Compared with MC, the DLAC is a promising strategy to obtain superior accuracy, consistency, and efficiency for the PORT-CTV in NSCLC.

A segmentation method combining probability map and boundary based on multiple fully convolutional networks and repetitive training

Cell nuclei image segmentation technology can help researchers observe each cell's stress response to drug treatment. However, it is still a challenge to accurately segment the adherent cell nuclei. At present, image segmentation based on a fully convolutional network (FCN) is attracting researchers' attention. We propose a multiple FCN architecture and repetitive training (M-FCN-RT) method to learn features of cell nucleus images. In M-FCN-RT, the multiple FCN (M-FCN) architecture is composed of several single FCNs (S-FCNs) with the same structure, and each FCN is used to learn the specific features of image datasets. In this paper, the M-FCN contains three FCNs; FCN_1-2, FCN₃ and FCN_B. FCN_1-2 and FCN₃ are respectively used to learn the spatial features of cell nuclei for generating probability maps to indicate nucleus regions of an image; FCN_B (boundary FCN) is used to learn the edge features of cell nuclei for generating the nucleus boundary. For the training of each FCN, we propose a repetitive training (RT) method to improve the classification accuracy of the model. To segment cell nuclei, we finally propose an algorithm combining the probability map and boundary (PMB) to segment the adherent nuclei. This paper uses a public opening nucleus image dataset to train, verify and evaluate the proposed M-FCN-RT and PMB methods. Our M-FCN-RT method achieves a high Dice similarity coefficient (DSC) of 92.11%, 95.64% and 87.99% on the three types of sub-datasets respectively for probability maps. In addition, segmentation experimental results show the PMB method is more effective and efficient compared with other methods.

Impact of Tissue Classification in MRI-Guided Attenuation Correction on Whole-Body Patlak PET/MRI

PURPOSE

The aim of this work is to investigate the impact of tissue classification in magnetic resonance imaging (MRI)-guided positron emission tomography (PET) attenuation correction (AC) for whole-body (WB) Patlak net uptake rate constant (K_i) imaging in PET/MRI studies.

PROCEDURES

WB dynamic PET/CT data were acquired for 14 patients. The CT images were utilized to generate attenuation maps (μ-map_CTAC) of continuous attenuation coefficient values (A_coeff). The μ-map_CTAC were then segmented into four tissue classes (μ-map_4-classes), namely background (air), lung, fat, and soft tissue, where a predefined A_coeff was assigned to each class. To assess the impact of bone for AC, the bones in the μ-map_CTAC were then assigned a predefined soft tissue A_coeff (0.1 cm^-1) to produce an AC μ-map without bones (μ-map_no-bones). Thereafter, both WB static SUV and dynamic PET images were reconstructed using μ-map_CTAC, μ-map_4-classes, and μ-map_no-bones (PET_CTAC, PET_4-classes, and PET_no-bones), respectively. WB indirect and direct parametric K_i images were generated using Patlak graphical analysis. Malignant lesions were delineated on PET images with an automatic segmentation method that uses an active contour model (MASAC). Then, the quantitative metrics of the metabolically active tumor volume (MATV), target-to-background (TBR), contrast-to-noise ratio (CNR), peak region-of-interest (ROI_peak), maximum region-of-interest (ROI_max), mean region-of-interest (ROI_mean), and metabolic volume product (MVP) were analyzed. The Wilcoxon test was conducted to assess the difference between PET_4-classes and PET_no-bones against PET_CTAC for all images. The same test was also adopted to compare the differences between SUV, indirect K_i, and direct K_i images for each evaluated AC method.

RESULTS

No significant differences in MATV, TBR, and CNR were observed between PET_4-classes and PET_CTAC for either SUV or K_i images. PET_4-classes significantly overestimated ROI_peak, ROI_max, ROI_mean, as well as MVP scores compared with PET_CTAC in both SUV and K_i images. SUV images exhibited the highest median relative errors for PET_4-classes with respect to PET_CTAC (RE_4-classes): 6.91 %, 6.55 %, 5.90 %, and 6.56 % for ROI_peak, ROI_max, ROI_mean, and MVP, respectively. On the contrary, K_i images showed slightly reduced RE_4-classes (indirect 5.52 %, 5.95 %, 4.43 %, and 5.70 %, direct 6.61 %, 6.33 %, 5.53 %, and 4.96 %) for ROI_peak, ROI_max, ROI_mean, and MVP, respectively. A higher TBR was observed on indirect and direct K_i images relative to SUV, while direct K_i images demonstrated the highest CNR.

CONCLUSIONS

Four-tissue class AC may impact SUV and K_i parameter estimation but only to a limited extent, thereby suggesting that WB Patlak K_i imaging for dynamic WB PET/MRI studies is feasible. Patlak K_i imaging can enhance TBR, thereby facilitating lesion segmentation and quantification. However, patient-specific A_coeff for each tissue class should be used when possible to address the high inter-patient variability of A_coeff distributions.

Variability and Repeatability of Quantitative Uptake Metrics in 18F-FDG PET/CT of Non-Small Cell Lung Cancer: Impact of Segmentation Method, Uptake Interval, and Reconstruction Protocol

There is increased interest in various new quantitative uptake metrics beyond SUV in oncologic PET/CT studies. The purpose of this study was to investigate the variability and test-retest ratio (TRT) of metabolically active tumor volume (MATV) measurements and several other new quantitative metrics in non-small cell lung cancer using ¹⁸F-FDG PET/CT with different segmentation methods, user interactions, uptake intervals, and reconstruction protocols. Methods: Ten patients with advanced non-small cell lung cancer received 2 series of 2 whole-body ¹⁸F-FDG PET/CT scans at 60 min after injection and at 90 min after injection. PET data were reconstructed with 4 different protocols. Eight segmentation methods were applied to delineate lesions with and without a tumor mask. MATV, SUV_max, SUV_mean, total lesion glycolysis, and intralesional heterogeneity features were derived. Variability and repeatability were evaluated using a generalized-estimating-equation statistical model with Bonferroni adjustment for multiple comparisons. The statistical model, including interaction between uptake interval and reconstruction protocol, was applied individually to the data obtained from each segmentation method. Results: Without masking, none of the segmentation methods could delineate all lesions correctly. MATV was affected by both uptake interval and reconstruction settings for most segmentation methods. Similar observations were obtained for the uptake metrics SUV_max, SUV_mean, total lesion glycolysis, homogeneity, entropy, and zone percentage. No effect of uptake interval was observed on TRT metrics, whereas the reconstruction protocol affected the TRT of SUV_max Overall, segmentation methods showing poor quantitative performance in one condition showed better performance in other (combined) conditions. For some metrics, a clear statistical interaction was found between the segmentation method and both uptake interval and reconstruction protocol. Conclusion: All segmentation results need to be reviewed critically. MATV and other quantitative uptake metrics, as well as their TRT, depend on segmentation method, uptake interval, and reconstruction protocol. To obtain quantitative reliable metrics, with good TRT performance, the optimal segmentation method depends on local imaging procedure, the PET/CT system, or reconstruction protocol. Rigid harmonization of imaging procedure and PET/CT performance will be helpful in mitigating this variability.

A smart and operator independent system to delineate tumours in Positron Emission Tomography scans

Positron Emission Tomography (PET) imaging has an enormous potential to improve radiation therapy treatment planning offering complementary functional information with respect to other anatomical imaging approaches. The aim of this study is to develop an operator independent, reliable, and clinically feasible system for biological tumour volume delineation from PET images. Under this design hypothesis, we combine several known approaches in an original way to deploy a system with a high level of automation. The proposed system automatically identifies the optimal region of interest around the tumour and performs a slice-by-slice marching local active contour segmentation. It automatically stops when a "cancer-free" slice is identified. User intervention is limited at drawing an initial rough contour around the cancer region. By design, the algorithm performs the segmentation minimizing any dependence from the initial input, so that the final result is extremely repeatable. To assess the performances under different conditions, our system is evaluated on a dataset comprising five synthetic experiments and fifty oncological lesions located in different anatomical regions (i.e. lung, head and neck, and brain) using PET studies with 18F-fluoro-2-deoxy-d-glucose and 11C-labeled Methionine radio-tracers. Results on synthetic lesions demonstrate enhanced performances when compared against the most common PET segmentation methods. In clinical cases, the proposed system produces accurate segmentations (average dice similarity coefficient: 85.36 ± 2.94%, 85.98 ± 3.40%, 88.02 ± 2.75% in the lung, head and neck, and brain district, respectively) with high agreement with the gold standard (determination coefficient R² = 0.98). We believe that the proposed system could be efficiently used in the everyday clinical routine as a medical decision tool, and to provide the clinicians with additional information, derived from PET, which can be of use in radiation therapy, treatment, and planning.

Comparison of different automated lesion delineation methods for metabolic tumor volume of 18F-FDG PET/CT in patients with stage I lung adenocarcinoma

The aim of the study was to investigate the suitable segmentation method in small, low uptake and heterogeneous nodules of stage I lung adenocarcinoma.133 stage I lung adenocarcinoma patients with F-FDG PET/CT scans were enrolled in this retrospective study. All lesions were divided into different groups according to nodule density, nodule size, and the maximum standard uptake value (SUVmax) level. Four different PET segmentation methods were performed, including percentage threshold of SUVmax (T42% and T42% × RC), gradient-based threshold (adaptive iterative algorithm, AT-AIA), and background-related threshold (adaptive thresholding at 40% SUVmax, AT40%) approaches. The MTVs were evaluated and compared with CT volume (CTV). Percentage volume error (%VE) compared to CTV was calculated and the correlations between MTVs and CTV were analyzed.AT-AIA had the highest accuracy in large, high uptake, and solid nodules (72.5%, 72.4%, and 65.6%, respectively). AT40% had the highest accuracy in small, low uptake and nonsolid nodules (56.6%, 56.1%, and 62.6%, respectively). In part-solid nodules, the accuracy of AT-AIA (60.0%) and AT40% (56.7%) were higher than that of T42% and T42% × RC. The MTV of AT-AIA was in excellent correlation with the CTV in solid nodules (R = 0.831, P < .001) and in high uptake nodules (R = 0.830, P < .001). The MTV of AT40% was in good correlation with the CTV in nonsolid nodules (R = 0.686, P = .003) and in part-solid nodules (R = 0.731, P < .001).AT40% showed best performance in small, low uptake, nonsolid and part-solid lesions. AT-AIA was suitable for large, high uptake, and solid lesions.