Deep learning algorithm performs similarly to radiologists in the assessment of prostate volume on MRI

An online national quality assessment survey of prostate MRI reading: interreader variability in prostate volume measurement and PI-RADS classification

Background

High-quality assessment of prostate MRI is fundamental in both clinical practice and screening. There is a lack of national level data on variability in prostate volume measurement and PI-RADS assessment. Methods of quality assurance need to be developed.

Methods

All Swedish radiology departments were invited to participate in an external quality assurance of prostate MRI reading. Ten prostate MRI cases were selected by an expert panel to reflect common findings. Readers measured whole gland volume (ellipsoid formula method) and assigned a PI-RADS score in a web-based PACS with full clinical functionality. Expert consensus was used as reference standard. Descriptive statistics were used to show the distribution of volume measurements and PSA density. Reader agreement was assessed using percentages and kappa scores. A feedback document was sent to all participants upon completion of the quality assurance program.

Results

Forty-three radiologists representing 17 departments read at least 7 out of 10 cases. The median difference in prostate volume assessment compared to the reference volume for the 10 cases ranged from -23 mL to + 6 mL. Per case agreement ranged from 33 % to 86 % for the assigned PI-RADS score and from 35 % to 98 % for PI-RADS 1-3 versus PI-RADS 4-5. Interreader agreement was moderate with a median kappa score of 0.53 (IQR 0.48-0.62).

Conclusion

This online model for national quality assurance programs was feasible. Rather large per-case reader variations in prostate volume assessment and PI-RADS scoring were shown. To reduce variability in clinical practice, systematic interreader comparisons should be encouraged.

The application of radiomics in cancer imaging with a focus on lung cancer, renal cell carcinoma, gastrointestinal cancer, and head and neck cancer: A systematic review

PURPOSE

To study the application of radiomics in cancer imaging with a focus on lung cancer, renal cell carcinoma, gastrointestinal cancer, and head and neck cancer.

METHODS

Different electronic databases were considered. Articles published in the last five years were analyzed (January 2019 and December 2023). Papers were selected by two investigators with over 15 years of experience in Radiomics analysis in cancer imaging. The methodological quality of each radiomics study was performed using the Radiomic Quality Score (RQS) by two different readers in consensus and then by a third operator to solve disagreements between the two readers.

RESULTS

19 articles are included in the review. Among the analyzed studies, only one study achieved an RQS of 18 reporting multivariable analyzes with also non-radiomics features and using the validation phase considering two datasets from two distinct institutes and open science and data domain.

CONCLUSION

This informative review has brought attention to the increasingly consolidated potential of Radiomics, although there are still several aspects to be evaluated before the transition to routine clinical practice. There are several challenges to address, including the need for standardization at all stages of the workflow and the potential for cross-site validation using heterogeneous real-world datasets. It will be necessary to establish and promote an imaging data acquisition protocol, conduct multicenter prospective quality control studies, add scanner differences and vendor-dependent characteristics; to collect images of individuals at additional time points, to report calibration statistics.

Impact of uncertainty quantification through conformal prediction on volume assessment from deep learning-based MRI prostate segmentation

OBJECTIVES

To estimate the uncertainty of a deep learning (DL)-based prostate segmentation algorithm through conformal prediction (CP) and to assess its effect on the calculation of the prostate volume (PV) in patients at risk of prostate cancer (PC).

METHODS

Three-hundred seventy-seven multi-center 3-Tesla axial T2-weighted exams from biopsied males (66.64  ±  7.47 years) at risk of PC were retrospectively included in the study. Assessment of PV based on PI-RADS 2.1 ellipsoid formula (PV r e f ) was available for included patients. Prostate segmentations were obtained from a DL model and used to calculate the PV (PV D L ). CP was applied at a confidence level of 85% to flag unreliable pixel segmentations of the DL model. Subsequently, the PV (PV C P ) was calculated when disregarding uncertain pixel segmentations. Agreement betweenPV D L andPV C P was evaluated against the reference standardPV r e f . Intraclass correlation coefficient (ICC) and Bland-Altman plots were used to assess the agreement. The relative volume difference (RVD) was used to evaluate the PV calculation accuracy, and the Wilcoxon Signed-Rank Test was used to assess statistical differences. A p-value < 0.05 was considered statistically significant.

RESULTS

Conformal prediction significantly reduced RVD when compared to the DL algorithm (RVD = - 2.81  ±  8.85 and RVD = -8.01  ±  11.50).PV C P showed a significantly larger agreement thanPV D L when using the reference standardPV r e f (mean difference (95% limits of agreement)PV C P : 1.27 mL (- 13.64; 16.17 mL)PV D L : 6.07 mL (- 14.29; 26.42 mL)), with an excellent ICC (PV C P : 0.97 (95% CI: 0.97 to 0.98)).

CONCLUSION

Uncertainty quantification through CP increases the accuracy and reliability of DL-based PV assessment in patients at risk of PC.

CRITICAL RELEVANCE STATEMENT

Conformal prediction can flag uncertain pixel predictions of DL-based prostate MRI segmentation at a desired confidence level, increasing the reliability and safety of prostate volume assessment in patients at risk of prostate cancer.

KEY POINTS

Conformal prediction can flag uncertain pixel predictions of prostate segmentations at a user-defined confidence level. Deep learning with conformal prediction shows high accuracy in prostate volumetric assessment. Agreement between automatic and ellipsoid-derived volume was significantly larger with conformal prediction.

Reduction of false positives using zone-specific prostate-specific antigen density for prostate MRI-based biopsy decision strategies

OBJECTIVES

To develop and test zone-specific prostate-specific antigen density (sPSAD) combined with PI-RADS to guide prostate biopsy decision strategies (BDS).

METHODS

This retrospective study included consecutive patients, who underwent prostate MRI and biopsy (01/2012-10/2018). The whole gland and transition zone (TZ) were segmented at MRI using a retrained deep learning system (DLS; nnU-Net) to calculate PSAD and sPSAD, respectively. Additionally, sPSAD and PI-RADS were combined in a BDS, and diagnostic performances to detect Grade Group ≥ 2 (GG ≥ 2) prostate cancer were compared. Patient-based cancer detection using sPSAD was assessed by bootstrapping with 1000 repetitions and reported as area under the curve (AUC). Clinical utility of the BDS was tested in the hold-out test set using decision curve analysis. Statistics included nonparametric DeLong test for AUCs and Fisher-Yates test for remaining performance metrics.

RESULTS

A total of 1604 patients aged 67 (interquartile range, 61-73) with 48% GG ≥ 2 prevalence (774/1604) were evaluated. By employing DLS-based prostate and TZ volumes (DICE coefficients of 0.89 (95% confidence interval, 0.80-0.97) and 0.84 (0.70-0.99)), GG ≥ 2 detection using PSAD was inferior to sPSAD (AUC, 0.71 (0.68-0.74)/0.73 (0.70-0.76); p < 0.001). Combining PI-RADS with sPSAD, GG ≥ 2 detection specificity doubled from 18% (10-20%) to 43% (30-44%; p < 0.001) with similar sensitivity (93% (89-96%)/97% (94-99%); p = 0.052), when biopsies were taken in PI-RADS 4-5 and 3 only if sPSAD was ≥ 0.42 ng/mL/cc as compared to all PI-RADS 3-5 cases. Additionally, using the sPSAD-based BDS, false positives were reduced by 25% (123 (104-142)/165 (146-185); p < 0.001).

CONCLUSION

Using sPSAD to guide biopsy decisions in PI-RADS 3 lesions can reduce false positives at MRI while maintaining high sensitivity for GG ≥ 2 cancers.

CLINICAL RELEVANCE STATEMENT

Transition zone-specific prostate-specific antigen density can improve the accuracy of prostate cancer detection compared to MRI assessments alone, by lowering false-positive cases without significantly missing men with ISUP GG ≥ 2 cancers.

KEY POINTS

• Prostate biopsy decision strategies using PI-RADS at MRI are limited by a substantial proportion of false positives, not yielding grade group ≥ 2 prostate cancer. • PI-RADS combined with transition zone (TZ)-specific prostate-specific antigen density (PSAD) decreased the number of unproductive biopsies by 25% compared to PI-RADS only. • TZ-specific PSAD also improved the specificity of MRI-directed biopsies by 9% compared to the whole gland PSAD, while showing identical sensitivity.

Comparison of data fusion strategies for automated prostate lesion detection using mpMRI correlated with whole mount histology

BACKGROUND

In this work, we compare input level, feature level and decision level data fusion techniques for automatic detection of clinically significant prostate lesions (csPCa).

METHODS

Multiple deep learning CNN architectures were developed using the Unet as the baseline. The CNNs use both multiparametric MRI images (T2W, ADC, and High b-value) and quantitative clinical data (prostate specific antigen (PSA), PSA density (PSAD), prostate gland volume & gross tumor volume (GTV)), and only mp-MRI images (n = 118), as input. In addition, co-registered ground truth data from whole mount histopathology images (n = 22) were used as a test set for evaluation.

RESULTS

The CNNs achieved for early/intermediate / late level fusion a precision of 0.41/0.51/0.61, recall value of 0.18/0.22/0.25, an average precision of 0.13 / 0.19 / 0.27, and F scores of 0.55/0.67/ 0.76. Dice Sorensen Coefficient (DSC) was used to evaluate the influence of combining mpMRI with parametric clinical data for the detection of csPCa. We compared the DSC between the predictions of CNN's trained with mpMRI and parametric clinical and the CNN's trained with only mpMRI images as input with the ground truth. We obtained a DSC of data 0.30/0.34/0.36 and 0.26/0.33/0.34 respectively. Additionally, we evaluated the influence of each mpMRI input channel for the task of csPCa detection and obtained a DSC of 0.14 / 0.25 / 0.28.

CONCLUSION

The results show that the decision level fusion network performs better for the task of prostate lesion detection. Combining mpMRI data with quantitative clinical data does not show significant differences between these networks (p = 0.26/0.62/0.85). The results show that CNNs trained with all mpMRI data outperform CNNs with less input channels which is consistent with current clinical protocols where the same input is used for PI-RADS lesion scoring.

TRIAL REGISTRATION

The trial was registered retrospectively at the German Register for Clinical Studies (DRKS) under proposal number Nr. 476/14 & 476/19.

An Informative Review of Radiomics Studies on Cancer Imaging: The Main Findings, Challenges and Limitations of the Methodologies

The aim of this informative review was to investigate the application of radiomics in cancer imaging and to summarize the results of recent studies to support oncological imaging with particular attention to breast cancer, rectal cancer and primitive and secondary liver cancer. This review also aims to provide the main findings, challenges and limitations of the current methodologies. Clinical studies published in the last four years (2019-2022) were included in this review. Among the 19 studies analyzed, none assessed the differences between scanners and vendor-dependent characteristics, collected images of individuals at additional points in time, performed calibration statistics, represented a prospective study performed and registered in a study database, conducted a cost-effectiveness analysis, reported on the cost-effectiveness of the clinical application, or performed multivariable analysis with also non-radiomics features. Seven studies reached a high radiomic quality score (RQS), and seventeen earned additional points by using validation steps considering two datasets from two distinct institutes and open science and data domains (radiomics features calculated on a set of representative ROIs are open source). The potential of radiomics is increasingly establishing itself, even if there are still several aspects to be evaluated before the passage of radiomics into routine clinical practice. There are several challenges, including the need for standardization across all stages of the workflow and the potential for cross-site validation using real-world heterogeneous datasets. Moreover, multiple centers and prospective radiomics studies with more samples that add inter-scanner differences and vendor-dependent characteristics will be needed in the future, as well as the collecting of images of individuals at additional time points, the reporting of calibration statistics and the performing of prospective studies registered in a study database.

Deep learning performance on MRI prostate gland segmentation: evaluation of two commercially available algorithms compared with an expert radiologist

Purpose

Accurate whole-gland prostate segmentation is crucial for successful ultrasound-MRI fusion biopsy, focal cancer treatment, and radiation therapy techniques. Commercially available artificial intelligence (AI) models, using deep learning algorithms (DLAs) for prostate gland segmentation, are rapidly increasing in numbers. Typically, their performance in a true clinical context is scarcely examined or published. We used a heterogenous clinical MRI dataset in this study aiming to contribute to validation of AI-models.

Approach

We included 123 patients in this retrospective multicenter (7 hospitals), multiscanner (8 scanners, 2 vendors, 1.5T and 3T) study comparing prostate contour assessment by 2 commercially available Food and Drug Association (FDA)-cleared and CE-marked algorithms (DLA1 and DLA2) using an expert radiologist's manual contours as a reference standard (RSexp) in this clinical heterogeneous MRI dataset. No in-house training of the DLAs was performed before testing. Several methods for comparing segmentation overlap were used, the Dice similarity coefficient (DSC) being the most important.

Results

The DSC mean and standard deviation for DLA1 versus the radiologist reference standard (RSexp) was 0.90 ± 0.05 and for DLA2 versus RSexp it was 0.89 ± 0.04 . A paired t -test to compare the DSC for DLA1 and DLA2 showed no statistically significant difference (p = 0.8 ).

Conclusions

Two commercially available DL algorithms (FDA-cleared and CE-marked) can perform accurate whole-gland prostate segmentation on a par with expert radiologist manual planimetry on a real-world clinical dataset. Implementing AI models in the clinical routine may free up time that can be better invested in complex work tasks, adding more patient value.

Three-Dimensional Measurement of the Uterus on Magnetic Resonance Images: Development and Performance Analysis of an Automated Deep-Learning Tool

Uterus measurements are useful for assessing both the treatment and follow-ups of gynaecological patients. The aim of our study was to develop a deep learning (DL) tool for fully automated measurement of the three-dimensional size of the uterus on magnetic resonance imaging (MRI). In this single-centre retrospective study, 900 cases were included to train, validate, and test a VGG-16/VGG-11 convolutional neural network (CNN). The ground truth was manual measurement. The performance of the model was evaluated using the objective key point similarity (OKS), the mean difference in millimetres, and coefficient of determination R2. The OKS of our model was 0.92 (validation) and 0.96 (test). The average deviation and R2 coefficient between the AI measurements and the manual ones were, respectively, 3.9 mm and 0.93 for two-point length, 3.7 mm and 0.94 for three-point length, 2.6 mm and 0.93 for width, 4.2 mm and 0.75 for thickness. The inter-radiologist variability was 1.4 mm. A three-dimensional automated measurement was obtained in 1.6 s. In conclusion, our model was able to locate the uterus on MRIs and place measurement points on it to obtain its three-dimensional measurement with a very good correlation compared to manual measurements.