Stability of Multi-Parametric Prostate MRI Radiomic Features to Variations in Segmentation

A Critical Analysis of the Robustness of Radiomics to Variations in Segmentation Methods in 18F-PSMA-1007 PET Images of Patients Affected by Prostate Cancer

BACKGROUND

Radiomics shows promising results in supporting the clinical decision process, and much effort has been put into its standardization, thus leading to the Imaging Biomarker Standardization Initiative (IBSI), that established how radiomics features should be computed. However, radiomics still lacks standardization and many factors, such as segmentation methods, limit study reproducibility and robustness.

AIM

We investigated the impact that three different segmentation methods (manual, thresholding and region growing) have on radiomics features extracted from 18F-PSMA-1007 Positron Emission Tomography (PET) images of 78 patients (43 Low Risk, 35 High Risk). Segmentation was repeated for each patient, thus leading to three datasets of segmentations. Then, feature extraction was performed for each dataset, and 1781 features (107 original, 930 Laplacian of Gaussian (LoG) features, 744 wavelet features) were extracted. Feature robustness and reproducibility were assessed through the intra class correlation coefficient (ICC) to measure agreement between the three segmentation methods. To assess the impact that the three methods had on machine learning models, feature selection was performed through a hybrid descriptive-inferential method, and selected features were given as input to three classifiers, K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Linear Discriminant Analysis (LDA), Random Forest (RF), AdaBoost and Neural Networks (NN), whose performance in discriminating between low-risk and high-risk patients have been validated through 30 times repeated five-fold cross validation.

CONCLUSIONS

Our study showed that segmentation methods influence radiomics features and that Shape features were the least reproducible (average ICC: 0.27), while GLCM features the most reproducible. Moreover, feature reproducibility changed depending on segmentation type, resulting in 51.18% of LoG features exhibiting excellent reproducibility (range average ICC: 0.68-0.87) and 47.85% of wavelet features exhibiting poor reproducibility that varied between wavelet sub-bands (range average ICC: 0.34-0.80) and resulted in the LLL band showing the highest average ICC (0.80). Finally, model performance showed that region growing led to the highest accuracy (74.49%), improved sensitivity (84.38%) and AUC (79.20%) in contrast with manual segmentation.

Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging

We aimed to compare the performance and interobserver agreement of radiologists manually segmenting images or those assisted by automatic segmentation. We further aimed to reduce interobserver variability and improve the consistency of radiomics features. This retrospective study included 327 patients diagnosed with prostate cancer from September 2016 to June 2018; images from 228 patients were used for automatic segmentation construction, and images from the remaining 99 were used for testing. First, four radiologists with varying experience levels retrospectively segmented 99 axial prostate images manually using T2-weighted fat-suppressed magnetic resonance imaging. Automatic segmentation was performed after 2 weeks. The Pyradiomics software package v3.1.0 was used to extract the texture features. The Dice coefficient and intraclass correlation coefficient (ICC) were used to evaluate segmentation performance and the interobserver consistency of prostate radiomics. The Wilcoxon rank sum test was used to compare the paired samples, with the significance level set at p < 0.05. The Dice coefficient was used to accurately measure the spatial overlap of manually delineated images. In all the 99 prostate segmentation result columns, the manual and automatic segmentation results of the senior group were significantly better than those of the junior group (p < 0.05). Automatic segmentation was more consistent than manual segmentation (p < 0.05), and the average ICC reached >0.85. The automatic segmentation annotation performance of junior radiologists was similar to that of senior radiologists performing manual segmentation. The ICC of radiomics features increased to excellent consistency (0.925 [0.888~0.950]). Automatic segmentation annotation provided better results than manual segmentation by radiologists. Our findings indicate that automatic segmentation annotation helps reduce variability in the perception and interpretation between radiologists with different experience levels and ensures the stability of radiomics features.