1
|
Pasini G, Russo G, Mantarro C, Bini F, Richiusa S, Morgante L, Comelli A, Russo GI, Sabini MG, Cosentino S, Marinozzi F, Ippolito M, Stefano A. A Critical Analysis of the Robustness of Radiomics to Variations in Segmentation Methods in 18F-PSMA-1007 PET Images of Patients Affected by Prostate Cancer. Diagnostics (Basel) 2023; 13:3640. [PMID: 38132224 PMCID: PMC10743045 DOI: 10.3390/diagnostics13243640] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Revised: 11/29/2023] [Accepted: 12/08/2023] [Indexed: 12/23/2023] Open
Abstract
BACKGROUND Radiomics shows promising results in supporting the clinical decision process, and much effort has been put into its standardization, thus leading to the Imaging Biomarker Standardization Initiative (IBSI), that established how radiomics features should be computed. However, radiomics still lacks standardization and many factors, such as segmentation methods, limit study reproducibility and robustness. AIM We investigated the impact that three different segmentation methods (manual, thresholding and region growing) have on radiomics features extracted from 18F-PSMA-1007 Positron Emission Tomography (PET) images of 78 patients (43 Low Risk, 35 High Risk). Segmentation was repeated for each patient, thus leading to three datasets of segmentations. Then, feature extraction was performed for each dataset, and 1781 features (107 original, 930 Laplacian of Gaussian (LoG) features, 744 wavelet features) were extracted. Feature robustness and reproducibility were assessed through the intra class correlation coefficient (ICC) to measure agreement between the three segmentation methods. To assess the impact that the three methods had on machine learning models, feature selection was performed through a hybrid descriptive-inferential method, and selected features were given as input to three classifiers, K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Linear Discriminant Analysis (LDA), Random Forest (RF), AdaBoost and Neural Networks (NN), whose performance in discriminating between low-risk and high-risk patients have been validated through 30 times repeated five-fold cross validation. CONCLUSIONS Our study showed that segmentation methods influence radiomics features and that Shape features were the least reproducible (average ICC: 0.27), while GLCM features the most reproducible. Moreover, feature reproducibility changed depending on segmentation type, resulting in 51.18% of LoG features exhibiting excellent reproducibility (range average ICC: 0.68-0.87) and 47.85% of wavelet features exhibiting poor reproducibility that varied between wavelet sub-bands (range average ICC: 0.34-0.80) and resulted in the LLL band showing the highest average ICC (0.80). Finally, model performance showed that region growing led to the highest accuracy (74.49%), improved sensitivity (84.38%) and AUC (79.20%) in contrast with manual segmentation.
Collapse
Affiliation(s)
- Giovanni Pasini
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Eudossiana 18, 00184 Rome, Italy; (G.P.); (L.M.); (F.M.)
- Institute of Molecular Bioimaging and Physiology, National Research Council (IBFM-CNR), Contrada, Pietrapollastra-Pisciotto, 90015 Cefalù, Italy; (G.R.); (S.R.); (A.C.); (A.S.)
| | - Giorgio Russo
- Institute of Molecular Bioimaging and Physiology, National Research Council (IBFM-CNR), Contrada, Pietrapollastra-Pisciotto, 90015 Cefalù, Italy; (G.R.); (S.R.); (A.C.); (A.S.)
- National Laboratory of South, National Institute for Nuclear Physics (LNS-INFN), 95125 Catania, Italy
| | - Cristina Mantarro
- Nuclear Medicine Department, Cannizzaro Hospital, 95125 Catania, Italy; (C.M.); (S.C.); (M.I.)
| | - Fabiano Bini
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Eudossiana 18, 00184 Rome, Italy; (G.P.); (L.M.); (F.M.)
| | - Selene Richiusa
- Institute of Molecular Bioimaging and Physiology, National Research Council (IBFM-CNR), Contrada, Pietrapollastra-Pisciotto, 90015 Cefalù, Italy; (G.R.); (S.R.); (A.C.); (A.S.)
| | - Lucrezia Morgante
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Eudossiana 18, 00184 Rome, Italy; (G.P.); (L.M.); (F.M.)
| | - Albert Comelli
- Institute of Molecular Bioimaging and Physiology, National Research Council (IBFM-CNR), Contrada, Pietrapollastra-Pisciotto, 90015 Cefalù, Italy; (G.R.); (S.R.); (A.C.); (A.S.)
- Ri.MED Foundation, Via Bandiera 11, 90133 Palermo, Italy
| | - Giorgio Ivan Russo
- Department of Surgery, Urology Section, University of Catania, 95125 Catania, Italy;
| | | | - Sebastiano Cosentino
- Nuclear Medicine Department, Cannizzaro Hospital, 95125 Catania, Italy; (C.M.); (S.C.); (M.I.)
| | - Franco Marinozzi
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Eudossiana 18, 00184 Rome, Italy; (G.P.); (L.M.); (F.M.)
| | - Massimo Ippolito
- Nuclear Medicine Department, Cannizzaro Hospital, 95125 Catania, Italy; (C.M.); (S.C.); (M.I.)
| | - Alessandro Stefano
- Institute of Molecular Bioimaging and Physiology, National Research Council (IBFM-CNR), Contrada, Pietrapollastra-Pisciotto, 90015 Cefalù, Italy; (G.R.); (S.R.); (A.C.); (A.S.)
- National Laboratory of South, National Institute for Nuclear Physics (LNS-INFN), 95125 Catania, Italy
| |
Collapse
|
2
|
Jin L, Ma Z, Li H, Gao F, Gao P, Yang N, Li D, Li M, Geng D. Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging. Bioengineering (Basel) 2023; 10:1340. [PMID: 38135930 PMCID: PMC10740636 DOI: 10.3390/bioengineering10121340] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Revised: 11/10/2023] [Accepted: 11/17/2023] [Indexed: 12/24/2023] Open
Abstract
We aimed to compare the performance and interobserver agreement of radiologists manually segmenting images or those assisted by automatic segmentation. We further aimed to reduce interobserver variability and improve the consistency of radiomics features. This retrospective study included 327 patients diagnosed with prostate cancer from September 2016 to June 2018; images from 228 patients were used for automatic segmentation construction, and images from the remaining 99 were used for testing. First, four radiologists with varying experience levels retrospectively segmented 99 axial prostate images manually using T2-weighted fat-suppressed magnetic resonance imaging. Automatic segmentation was performed after 2 weeks. The Pyradiomics software package v3.1.0 was used to extract the texture features. The Dice coefficient and intraclass correlation coefficient (ICC) were used to evaluate segmentation performance and the interobserver consistency of prostate radiomics. The Wilcoxon rank sum test was used to compare the paired samples, with the significance level set at p < 0.05. The Dice coefficient was used to accurately measure the spatial overlap of manually delineated images. In all the 99 prostate segmentation result columns, the manual and automatic segmentation results of the senior group were significantly better than those of the junior group (p < 0.05). Automatic segmentation was more consistent than manual segmentation (p < 0.05), and the average ICC reached >0.85. The automatic segmentation annotation performance of junior radiologists was similar to that of senior radiologists performing manual segmentation. The ICC of radiomics features increased to excellent consistency (0.925 [0.888~0.950]). Automatic segmentation annotation provided better results than manual segmentation by radiologists. Our findings indicate that automatic segmentation annotation helps reduce variability in the perception and interpretation between radiologists with different experience levels and ensures the stability of radiomics features.
Collapse
Affiliation(s)
- Liang Jin
- Radiology Department, Huashan Hospital, Affiliated with Fudan University, Shanghai 200040, China; (L.J.); (H.L.)
- Radiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, China; (Z.M.); (F.G.); (P.G.); (N.Y.); (D.L.)
| | - Zhuangxuan Ma
- Radiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, China; (Z.M.); (F.G.); (P.G.); (N.Y.); (D.L.)
| | - Haiqing Li
- Radiology Department, Huashan Hospital, Affiliated with Fudan University, Shanghai 200040, China; (L.J.); (H.L.)
| | - Feng Gao
- Radiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, China; (Z.M.); (F.G.); (P.G.); (N.Y.); (D.L.)
| | - Pan Gao
- Radiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, China; (Z.M.); (F.G.); (P.G.); (N.Y.); (D.L.)
| | - Nan Yang
- Radiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, China; (Z.M.); (F.G.); (P.G.); (N.Y.); (D.L.)
| | - Dechun Li
- Radiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, China; (Z.M.); (F.G.); (P.G.); (N.Y.); (D.L.)
| | - Ming Li
- Radiology Department, Huadong Hospital, Affiliated with Fudan University, Shanghai 200040, China; (Z.M.); (F.G.); (P.G.); (N.Y.); (D.L.)
- Institute of Functional and Molecular Medical Imaging, Shanghai 200040, China
| | - Daoying Geng
- Radiology Department, Huashan Hospital, Affiliated with Fudan University, Shanghai 200040, China; (L.J.); (H.L.)
- Institute of Functional and Molecular Medical Imaging, Shanghai 200040, China
| |
Collapse
|