Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists

Deep Learning Features Can Improve Radiomics-Based Prostate Cancer Aggressiveness Prediction

PURPOSE

Emerging evidence suggests that the use of artificial intelligence can assist in the timely detection and optimization of therapeutic approach in patients with prostate cancer. The conventional perspective on radiomics encompassing segmentation and the extraction of radiomic features considers it as an independent and sequential process. However, it is not necessary to adhere to this viewpoint. In this study, we show that besides generating masks from which radiomic features can be extracted, prostate segmentation and reconstruction models provide valuable information in their feature space, which can improve the quality of radiomic signatures models for disease aggressiveness classification.

MATERIALS AND METHODS

We perform 2,244 experiments with deep learning features extracted from 13 different models trained using different anatomic zones and characterize how modeling decisions, such as deep feature aggregation and dimensionality reduction, affect performance.

RESULTS

While models using deep features from full gland and radiomic features consistently lead to improved disease aggressiveness prediction performance, others are detrimental. Our results suggest that the use of deep features can be beneficial, but an appropriate and comprehensive assessment is necessary to ensure that their inclusion does not harm predictive performance.

CONCLUSION

The study findings reveal that incorporating deep features derived from autoencoder models trained to reconstruct the full prostate gland (both zonal models show worse performance than radiomics only models), combined with radiomic features, often lead to a statistically significant increase in model performance for disease aggressiveness classification. Additionally, the results also demonstrate that the choice of feature selection is key to achieving good performance, with principal component analysis (PCA) and PCA + relief being the best approaches and that there is no clear difference between the three proposed latent representation extraction techniques.

A noninvasive method for predicting clinically significant prostate cancer using magnetic resonance imaging combined with PRKY promoter methylation level: a machine learning study

BACKGROUND

Traditional process for clinically significant prostate cancer (csPCA) diagnosis relies on invasive biopsy and may bring pain and complications. Radiomic features of magnetic resonance imaging MRI and methylation of the PRKY promoter were found to be associated with prostate cancer.

METHODS

Fifty-four Patients who underwent prostate biopsy or photoselective vaporization of the prostate (PVP) from 2022 to 2023 were selected for this study, and their clinical data, blood samples and MRI images were obtained before the operation. Methylation level of two PRKY promoter sites, cg05618150 and cg05163709, were tested through bisulfite sequencing PCR (BSP). The PI-RADS score of each patient was estimated and the region of interest (ROI) was delineated by 2 experienced radiologists. After being extracted by a plug-in of 3D-slicer, radiomic features were selected through LASSCO regression and t-test. Selected radiomic features, methylation levels and clinical data were used for model construction through the random forest (RF) algorithm, and the predictive efficiency was analyzed by the area under the receiver operation characteristic (ROC) curve (AUC).

RESULTS

Methylation level of the site, cg05618150, was observed to be associated with prostate cancer, for which the AUC was 0.74. The AUC of T2WI in csPCA prediction was 0.84, which was higher than that of the apparent diffusion coefficient ADC (AUC = 0.81). The model combined with T2WI and clinical data reached an AUC of 0.94. The AUC of the T2WI-clinic-methylation-combined model was 0.97, which was greater than that of the model combined with the PI-RADS score, clinical data and PRKY promoter methylation levels (AUC = 0.86).

CONCLUSIONS

The model combining with radiomic features, clinical data and PRKY promoter methylation levels based on machine learning had high predictive efficiency in csPCA diagnosis.

Analysis of domain shift in whole prostate gland, zonal and lesions segmentation and detection, using multicentric retrospective data

Despite being one of the most prevalent forms of cancer, prostate cancer (PCa) shows a significantly high survival rate, provided there is timely detection and treatment. Computational methods can help make this detection process considerably faster and more robust. However, some modern machine-learning approaches require accurate segmentation of the prostate gland and the index lesion. Since performing manual segmentations is a very time-consuming task, and highly prone to inter-observer variability, there is a need to develop robust semi-automatic segmentation models. In this work, we leverage the large and highly diverse ProstateNet dataset, which includes 638 whole gland and 461 lesion segmentation masks, from 3 different scanner manufacturers provided by 14 institutions, in addition to other 3 independent public datasets, to train accurate and robust segmentation models for the whole prostate gland, zones and lesions. We show that models trained on large amounts of diverse data are better at generalizing to data from other institutions and obtained with other manufacturers, outperforming models trained on single-institution single-manufacturer datasets in all segmentation tasks. Furthermore, we show that lesion segmentation models trained on ProstateNet can be reliably used as lesion detection models.

Robustness of magnetic resonance imaging and positron emission tomography radiomic features in prostate cancer: Impact on recurrence prediction after radiation therapy

Background and purpose

Radiomic features from MRI and PET are an emerging tool with potential to improve prostate cancer outcomes. However, feature robustness due to image segmentation variations is currently unknown. Therefore, this study aimed to evaluate the robustness of radiomic features with segmentation variations and their impact on predicting biochemical recurrence (BCR).

Materials and methods

Multi-scanner, pre-radiation therapy imaging from 142 patients with localised prostate cancer was used. Imaging included T2-weighted (T2), apparent diffusion coefficient (ADC) MRI, and prostate-specific membrane antigen (PSMA)-PET. The prostate gland and intraprostatic tumours were manually and automatically segmented, and differences were quantified using Dice Coefficient (DC). Radiomic features including shape, first-order, and texture features were extracted for each segmentation from original and filtered images. Intraclass Correlation Coefficient (ICC) and Mean Absolute Percentage Difference (MAPD) were used to assess feature robustness. Random forest (RF) models were developed for each segmentation using robust features to predict BCR.

Results

Prostate gland segmentations were more consistent (mean DC = 0.78) than tumour segmentations (mean DC = 0.46). 112 (3.6 %) radiomic features demonstrated 'excellent' robustness (ICC > 0.9 and MAPD < 1 %), and 480 features (15.4 %) demonstrated 'good' robustness (ICC > 0.75 and MAPD < 5 %). PET imaging provided more features with excellent robustness than T2 and ADC. RF models showed strong predictive power for BCR with a mean area under the receiver-operator-characteristics curve (AUC) of 0.89 (range 0.85-0.93).

Conclusion

When using radiomic features for predictive modelling, segmentation variability should be considered. To develop BCR predictive models, radiomic features from the entire prostate gland are preferable over tumour segmentation-based features.

Inter-Rater Variability of Prostate Lesion Segmentation on Multiparametric Prostate MRI

INTRODUCTION

External radiotherapy is a major treatment for localized prostate cancer (PCa). Dose escalation to the whole prostate gland increases biochemical relapse-free survival but also acute and late toxicities. Dose escalation to the dominant index lesion (DIL) only is of growing interest. It requires a robust delineation of the DIL. In this context, we aimed to evaluate the inter-observer variability of DIL delineation.

MATERIAL AND METHODS

Two junior radiologists and a senior radiation oncologist delineated DILs on 64 mpMRIs of patients with histologically confirmed PCa. For each mpMRI and each reader, eight individual DIL segmentations were delineated. These delineations were blindly performed from one another and resulted from the individual analysis of the T2, apparent diffusion coefficient (ADC), b2000, and dynamic contrast enhanced (DCE) sequences, as well as the analysis of combined sequences (T2ADC, T2ADCb2000, T2ADCDCE, and T2ADCb2000DCE). Delineation variability was assessed using the DICE coefficient, Jaccard index, Hausdorff distance measure, and mean distance to agreement.

RESULTS

T2, ADC, T2ADC, b2000, T2 + ADC + b2000, T2 + ADC + DCE, and T2 + ADC + b2000 + DCE sequences obtained DICE coefficients of 0.51, 0.50, 0.54, 0.52, 0.54, 0.55, 0.53, respectively, which are significantly higher than the perfusion sequence alone (0.35, p < 0.001). The analysis of other similarity metrics lead to similar results. The tumor volume and PI-RADS classification were positively correlated with the DICE scores.

CONCLUSION

Our study showed that the contours of prostatic lesions were more reproducible on certain sequences but confirmed the great variability of prostatic contours with a maximum DICE coefficient calculated at 0.55 (joint analysis of T2, ADC, and perfusion sequences).

The role of AI in prostate MRI quality and interpretation: Opportunities and challenges

Prostate MRI plays an important role in imaging the prostate gland and surrounding tissues, particularly in the diagnosis and management of prostate cancer. With the widespread adoption of multiparametric magnetic resonance imaging in recent years, the concerns surrounding the variability of imaging quality have garnered increased attention. Several factors contribute to the inconsistency of image quality, such as acquisition parameters, scanner differences and interobserver variabilities. While efforts have been made to standardize image acquisition and interpretation via the development of systems, such as PI-RADS and PI-QUAL, the scoring systems still depend on the subjective experience and acumen of humans. Artificial intelligence (AI) has been increasingly used in many applications, including medical imaging, due to its ability to automate tasks and lower human error rates. These advantages have the potential to standardize the tasks of image interpretation and quality control of prostate MRI. Despite its potential, thorough validation is required before the implementation of AI in clinical practice. In this article, we explore the opportunities and challenges of AI, with a focus on the interpretation and quality of prostate MRI.

Navigating the intersection of 3D printing, software regulation and quality control for point-of-care manufacturing of personalized anatomical models

3D printing technology has become increasingly popular in healthcare settings, with applications of 3D printed anatomical models ranging from diagnostics and surgical planning to patient education. However, as the use of 3D printed anatomical models becomes more widespread, there is a growing need for regulation and quality control to ensure their accuracy and safety. This literature review examines the current state of 3D printing in hospitals and FDA regulation process for software intended for use in producing 3D printed models and provides for the first time a comprehensive list of approved software platforms alongside the 3D printers that have been validated with each for producing 3D printed anatomical models. The process for verification and validation of these 3D printed products, as well as the potential for inaccuracy in these models, is discussed, including methods for testing accuracy, limits, and standards for accuracy testing. This article emphasizes the importance of regulation and quality control in the use of 3D printing technology in healthcare, the need for clear guidelines and standards for both the software and the printed products to ensure the safety and accuracy of 3D printed anatomical models, and the opportunity to expand the library of regulated 3D printers.

Index lesion contouring on prostate MRI for targeted MRI/US fusion biopsy - Evaluation of mismatch between radiologists and urologists

PURPOSE

Mistargeting of focal lesions due to inaccurate segmentations can lead to false-negative findings on MRI-guided targeted biopsies. The purpose of this retrospective study was to examine inter-reader agreement of prostate index lesion segmentations from actual biopsy data between urologists and radiologists.

METHOD

Consecutive patients undergoing transperineal MRI-targeted prostate biopsy for PI-RADS 3-5 lesions between January 2020 and December 2021 were included. Agreement between segmentations on T2w-images between urologists and radiologists was assessed with Dice similarity coefficient (DSC) and 95 % Hausdorff distance (95 % HD). Differences in similarity scores were compared using Wilcoxon test. Differences depending on lesion features (size, zonal location, PI-RADS scores, lesion distinctness) were tested with Mann-Whitney U test. Correlation with prostate signal-intensity homogeneity score (PSHS) and lesion size was tested with Spearman's rank correlation.

RESULTS

Ninety-three patients (mean age 64.9 ± 7.1y, median serum PSA 6.5 [4.33-10.00]) were included. Mean similarity scores were statistically significantly lower between urologists and radiologists compared to radiologists only (DSC 0.41 ± 0.24 vs. 0.59 ± 0.23, p < 0.01; 95 %HD 6.38 ± 5.45 mm vs. 4.47 ± 4.12 mm, p < 0.01). There was a moderate and strong positive correlation between DSC scores and lesion size for segmentations from urologists and radiologists (ρ = 0.331, p = 0.002) and radiologists only (ρ = 0.501, p < 0.001). Similarity scores were worse in lesions ≤ 10 mm while other lesion features did not significantly influence similarity scores.

CONCLUSION

There is significant mismatch of prostate index lesion segmentations between urologists and radiologists. Segmentation agreement positively correlates with lesion size. PI-RADS scores, zonal location, lesion distinctness, and PSHS show no significant impact on segmentation agreement. These findings could underpin benefits of perilesional biopsies.

Radiomics-based approaches outperform visual analysis for differentiating lipoma from atypical lipomatous tumors: a review

BACKGROUND

Differentiating atypical lipomatous tumors (ALTs) and well-differentiated liposarcomas (WDLs) from benign lipomatous lesions is important for guiding clinical management, though conventional visual analysis of these lesions is challenging due to overlap of imaging features. Radiomics-based approaches may serve as a promising alternative and/or supplementary diagnostic approach to conventional imaging.

PURPOSE

The purpose of this study is to review the practice of radiomics-based imaging and systematically evaluate the literature available for studies evaluating radiomics applied to differentiating ALTs/WDLs from benign lipomas.

REVIEW

A background review of the radiomic workflow is provided, outlining the steps of image acquisition, segmentation, feature extraction, and model development. Subsequently, a systematic review of MEDLINE, EMBASE, Scopus, the Cochrane Library, and the grey literature was performed from inception to June 2022 to identify size studies using radiomics for differentiating ALTs/WDLs from benign lipomas. Radiomic models were shown to outperform conventional analysis in all but one model with a sensitivity ranging from 68 to 100% and a specificity ranging from 84 to 100%. However, current approaches rely on user input and no studies used a fully automated method for segmentation, contributing to interobserver variability and decreasing time efficiency.

CONCLUSION

Radiomic models may show improved performance for differentiating ALTs/WDLs from benign lipomas compared to conventional analysis. However, considerable variability between radiomic approaches exists and future studies evaluating a standardized radiomic model with a multi-institutional study design and preferably fully automated segmentation software are needed before clinical application can be more broadly considered.

Machine learning-based detection of aberrant deep learning segmentations of target and organs at risk for prostate radiotherapy using a secondary segmentation algorithm

Objective.The output of a deep learning (DL) auto-segmentation application should be reviewed, corrected if needed and approved before being used clinically. This verification procedure is labour-intensive, time-consuming and user-dependent, which potentially leads to significant errors with impact on the overall treatment quality. Additionally, when the time needed to correct auto-segmentations approaches the time to delineate target and organs at risk from scratch, the usability of the DL model can be questioned. Therefore, an automated quality assurance framework was developed with the aim to detect in advance aberrant auto-segmentations.Approach. Five organs (prostate, bladder, anorectum, femoral head left and right) were auto-delineated on CT acquisitions for 48 prostate patients by an in-house trained primary DL model. An experienced radiation oncologist assessed the correctness of the model output and categorised the auto-segmentations into two classes whether minor or major adaptations were needed. Subsequently, an independent, secondary DL model was implemented to delineate the same structures as the primary model. Quantitative comparison metrics were calculated using both models' segmentations and used as input features for a machine learning classification model to predict the output quality of the primary model.Main results. For every organ, the approach of independent validation by the secondary model was able to detect primary auto-segmentations that needed major adaptation with high sensitivity (recall = 1) based on the calculated quantitative metrics. The surface DSC and APL were found to be the most indicated parameters in comparison to standard quantitative metrics for the time needed to adapt auto-segmentations.Significance. This proposed method includes a proof of concept for the use of an independent DL segmentation model in combination with a ML classifier to improve time saving during QA of auto-segmentations. The integration of such system into current automatic segmentation pipelines can increase the efficiency of the radiotherapy contouring workflow.

Explainable AI for CNN-based prostate tumor segmentation in multi-parametric MRI correlated to whole mount histopathology

Automatic prostate tumor segmentation is often unable to identify the lesion even if multi-parametric MRI data is used as input, and the segmentation output is difficult to verify due to the lack of clinically established ground truth images. In this work we use an explainable deep learning model to interpret the predictions of a convolutional neural network (CNN) for prostate tumor segmentation. The CNN uses a U-Net architecture which was trained on multi-parametric MRI data from 122 patients to automatically segment the prostate gland and prostate tumor lesions. In addition, co-registered ground truth data from whole mount histopathology images were available in 15 patients that were used as a test set during CNN testing. To be able to interpret the segmentation results of the CNN, heat maps were generated using the Gradient Weighted Class Activation Map (Grad-CAM) method. The CNN achieved a mean Dice Sorensen Coefficient 0.62 and 0.31 for the prostate gland and the tumor lesions -with the radiologist drawn ground truth and 0.32 with whole-mount histology ground truth for tumor lesions. Dice Sorensen Coefficient between CNN predictions and manual segmentations from MRI and histology data were not significantly different. In the prostate the Grad-CAM heat maps could differentiate between tumor and healthy prostate tissue, which indicates that the image information in the tumor was essential for the CNN segmentation.

Evaluation of auto-segmentation accuracy of cloud-based artificial intelligence and atlas-based models

Background

Contour delineation, a crucial process in radiation oncology, is time-consuming and inaccurate due to inter-observer variation has been a critical issue in this process. An atlas-based automatic segmentation was developed to improve the delineation efficiency and reduce inter-observer variation. Additionally, automated segmentation using artificial intelligence (AI) has recently become available. In this study, auto-segmentations by atlas- and AI-based models for Organs at Risk (OAR) in patients with prostate and head and neck cancer were performed and delineation accuracies were evaluated.

Methods

Twenty-one patients with prostate cancer and 30 patients with head and neck cancer were evaluated. MIM Maestro was used to apply the atlas-based segmentation. MIM Contour ProtégéAI was used to apply the AI-based segmentation. Three similarity indices, the Dice similarity coefficient (DSC), Hausdorff distance (HD), and mean distance to agreement (MDA), were evaluated and compared with manual delineations. In addition, radiation oncologists visually evaluated the delineation accuracies.

Results

Among patients with prostate cancer, the AI-based model demonstrated higher accuracy than the atlas-based on DSC, HD, and MDA for the bladder and rectum. Upon visual evaluation, some errors were observed in the atlas-based delineations when the boundary between the small bowel or the seminal vesicle and the bladder was unclear. For patients with head and neck cancer, no significant differences were observed between the two models for almost all OARs, except small delineations such as the optic chiasm and optic nerve. The DSC tended to be lower when the HD and the MDA were smaller in small volume delineations.

Conclusions

In terms of efficiency, the processing time for head and neck cancers was much shorter than manual delineation. While quantitative evaluation with AI-based segmentation was significantly more accurate than atlas-based for prostate cancer, there was no significant difference for head and neck cancer. According to the results of visual evaluation, less necessity of manual correction in AI-based segmentation indicates that the segmentation efficiency of AI-based model is higher than that of atlas-based model. The effectiveness of the AI-based model can be expected to improve the segmentation efficiency and to significantly shorten the delineation time.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13014-021-01896-1.

Comparison of Multi-Parametric MRI of the Prostate to 3D Prostate Computer Aided Designs and 3D-Printed Prostate Models for Pre-Operative Planning of Radical Prostatectomies: A Pilot Study

OBJECTIVE

To evaluate the use of 3D computed aided designs and 3D-printed models as pre-operative planning tools for urologists, in addition to radiologist interpreted mp-MRIS, prior to radical prostatectomy procedures.

METHODS

Ten patients with biopsy-positive lesions detected on mp-MRI were retrospectively selected. Radiologists identified lesion locations using a Prostate Imaging-Reporting and Data System (PI-RADS) map and segmented the prostate, lesion(s), and surrounding anatomy to create 3D-CADs and 3D-printed models for each patient. 6 uro-oncologists randomly reviewed three modalities (mp-MRI, 3D-CAD, and 3D-printed models) for each patient and identified lesion locations which were graded for accuracy against the radiologists' answers. Questionnaires assessed decision confidence, ease-of-interpretation, and usefulness for preoperative planning for each modality.

RESULTS

Using 3D-CADs and 3D-printed models compared to mp-MRI, urologists were 2.4x and 2.8x more accurate at identifying the lesion(s), 2.7x and 3.2x faster, 1.6x and 1.63x more confident, and reported it was 1.6x and 1.7x easier to interpret. 3D-CADs and 3D-printed models were reported significantly more useful for overall pre-operative planning, identifying lesion location(s), determining degree of nerve sparing, obtaining negative margins, and patient counseling. Sub-analysis showed 3D-printed models demonstrated significant improvements in ease-of-interpretation, speed, usefulness for obtaining negative margins, and patient counseling compared to 3D-CADs.

CONCLUSION

3D-CADs and 3D-printed models are useful adjuncts to mp-MRI in providing urologists with more practical, accurate, and efficient pre-operative planning.

Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists

Background

There is increasing research in using segmentation of prostate cancer to create a digital 3D model from magnetic resonance imaging (MRI) scans for purposes of education or surgical planning. However, the variation in segmentation of prostate cancer among users and potential inaccuracy has not been studied.

Methods

Four consultant radiologists, four consultant urologists, four urology trainees, and four nonclinician segmentation scientists were asked to segment a single slice of a lateral T3 prostate tumor on MRI (“Prostate 1”), an anterior zone prostate tumor MRI (“Prostate 2”), and a kidney tumor computed tomography (CT) scan (“Kidney”). Time taken and self‐rated subjective accuracy out of a maximum score of 10 were recorded. Root mean square error, Dice coefficient, Matthews correlation coefficient, Jaccard index, specificity, and sensitivity were calculated using the radiologists as the ground truth.

Results

There was high variance among the radiologists in segmentation of Prostate 1 and 2 tumors with mean Dice coefficients of 0.81 and 0.58, respectively, compared to 0.96 for the kidney tumor. Urologists and urology trainees had similar accuracy, while nonclinicians had the lowest accuracy scores for Prostate 1 and 2 tumors (0.60 and 0.47) but similar for kidney tumor (0.95). Mean sensitivity in Prostate 1 (0.63) and Prostate 2 (0.61) was lower than specificity (0.92 and 0.93) suggesting under‐segmentation of tumors in the non‐radiologist groups. Participants spent less time on the kidney tumor segmentation and self‐rated accuracy was higher than both prostate tumors.

Conclusion

Segmentation of prostate cancers is more difficult than other anatomy such as kidney tumors. Less experienced participants appear to under‐segment models and underestimate the size of prostate tumors. Segmentation of prostate cancer is highly variable even among radiologists, and 3D modeling for clinical use must be performed with caution. Further work to develop a methodology to maximize segmentation accuracy is needed.