Artificial Intelligence (AI) for Automated Cancer Detection on Prostate MRI: Opportunities and Ongoing Challenges, From the AJR Special Series on AI Applications

Part II: Effect of different evaluation methods to the application of a computer-aided prostate MRI detection/diagnosis (CADe/CADx) device on reader performance

INTRODUCTION

The construction and results of a multiple-reader multiple-case prostate MRI study are described and reported to illustrate recommendations for how to standardize artificial intelligence (AI) prostate studies per the review constituting Part I1.

METHODS

Our previously reported approach was applied to review and report an IRB approved, HIPAA compliant multiple-reader multiple-case clinical study of 150 bi-parametric prostate MRI studies across 9 readers, measuring physician performance both with and without the use of the recently FDA cleared CADe/CADx software ProstatID.

RESULTS

Unassisted reader AUC values ranged from 0.418 - 0.759, with AI assisted AUC values ranging from 0.507 - 0.787. This represented a statistically significant AUC improvement of 0.045 (α = 0.05). A free-response ROC (FROC) analysis similarly demonstrated a statistically significant increase in θ from 0.405 to 0.453 (α = 0.05). The standalone performance of ProstatID performed across all prostate tissues demonstrated an AUC of 0.929, while the standalone lesion level performance of ProstatID at all biopsied locations achieved an AUC of 0.710.

CONCLUSION

This study applies and illustrates suggested reporting and standardization methods for prostate AI studies that will make it easier to understand, evaluate and compare between AI studies. Providing radiologists with the ProstatID CADe/CADx software significantly increased diagnostic performance as assessed by both ROC and free-response ROC metrics. Such algorithms have the potential to improve radiologist performance in the detection and localization of clinically significant prostate cancer.

Deep Learning-Based T2-Weighted MR Image Quality Assessment and Its Impact on Prostate Cancer Detection Rates

BACKGROUND

Image quality evaluation of prostate MRI is important for successful implementation of MRI into localized prostate cancer diagnosis.

PURPOSE

To examine the impact of image quality on prostate cancer detection using an in-house previously developed artificial intelligence (AI) algorithm.

STUDY TYPE

Retrospective.

SUBJECTS

615 consecutive patients (median age 67 [interquartile range [IQR]: 61-71] years) with elevated serum PSA (median PSA 6.6 [IQR: 4.6-9.8] ng/mL) prior to prostate biopsy.

FIELD STRENGTH/SEQUENCE

3.0T/T2-weighted turbo-spin-echo MRI, high b-value echo-planar diffusion-weighted imaging, and gradient recalled echo dynamic contrast-enhanced.

ASSESSMENTS

Scans were prospectively evaluated during clinical readout using PI-RADSv2.1 by one genitourinary radiologist with 17 years of experience. For each patient, T2-weighted images (T2WIs) were classified as high-quality or low-quality based on evaluation of both general distortions (eg, motion, distortion, noise, and aliasing) and perceptual distortions (eg, obscured delineation of prostatic capsule, prostatic zones, and excess rectal gas) by a previously developed in-house AI algorithm. Patients with PI-RADS category 1 underwent 12-core ultrasound-guided systematic biopsy while those with PI-RADS category 2-5 underwent combined systematic and targeted biopsies. Patient-level cancer detection rates (CDRs) were calculated for clinically significant prostate cancer (csPCa, International Society of Urological Pathology Grade Group ≥2) by each biopsy method and compared between high- and low-quality images in each PI-RADS category.

STATISTICAL TESTS

Fisher's exact test. Bootstrap 95% confidence intervals (CI). A P value <0.05 was considered statistically significant.

RESULTS

385 (63%) T2WIs were classified as high-quality and 230 (37%) as low-quality by AI. Targeted biopsy with high-quality T2WIs resulted in significantly higher clinically significant CDR than low-quality images for PI-RADS category 4 lesions (52% [95% CI: 43-61] vs. 32% [95% CI: 22-42]). For combined biopsy, there was no significant difference in patient-level CDRs for PI-RADS 4 between high- and low-quality T2WIs (56% [95% CI: 47-64] vs. 44% [95% CI: 34-55]; P = 0.09).

DATA CONCLUSION

Higher quality T2WIs were associated with better targeted biopsy clinically significant cancer detection performance for PI-RADS 4 lesions. Combined biopsy might be needed when T2WI is lower quality.

LEVEL OF EVIDENCE

2 TECHNICAL EFFICACY: Stage 1.

Automated Detection and Grading of Extraprostatic Extension of Prostate Cancer at MRI via Cascaded Deep Learning and Random Forest Classification

RATIONALE AND OBJECTIVES

Extraprostatic extension (EPE) is well established as a significant predictor of prostate cancer aggression and recurrence. Accurate EPE assessment prior to radical prostatectomy can impact surgical approach. We aimed to utilize a deep learning-based AI workflow for automated EPE grading from prostate T2W MRI, ADC map, and High B DWI.

MATERIAL AND METHODS

An expert genitourinary radiologist conducted prospective clinical assessments of MRI scans for 634 patients and assigned risk for EPE using a grading technique. The training set and held-out independent test set consisted of 507 patients and 127 patients, respectively. Existing deep-learning AI models for prostate organ and lesion segmentation were leveraged to extract area and distance features for random forest classification models. Model performance was evaluated using balanced accuracy, ROC AUCs for each EPE grade, as well as sensitivity, specificity, and accuracy compared to EPE on histopathology.

RESULTS

A balanced accuracy score of .390 ± 0.078 was achieved using a lesion detection probability threshold of 0.45 and distance features. Using the test set, ROC AUCs for AI-assigned EPE grades 0-3 were 0.70, 0.65, 0.68, and 0.55 respectively. When using EPE≥ 1 as the threshold for positive EPE, the model achieved a sensitivity of 0.67, specificity of 0.73, and accuracy of 0.72 compared to radiologist sensitivity of 0.81, specificity of 0.62, and accuracy of 0.66 using histopathology as the ground truth.

CONCLUSION

Our AI workflow for assigning imaging-based EPE grades achieves an accuracy for predicting histologic EPE approaching that of physicians. This automated workflow has the potential to enhance physician decision-making for assessing the risk of EPE in patients undergoing treatment for prostate cancer due to its consistency and automation.

Contemporary Approach to Prostate Imaging and Data Reporting System Score 3 Lesions

The aim of this article is to review the technical and clinical considerations encountered with PI-RADS 3 lesions, which are equivocal for clinically significant Prostate Cancer (csPCa) with detection rates ranging between 10% and 35%. The number of PI-RADS 3 lesions reported vary according to several factors including MRI quality and radiologist training/expertise among the most influential. PI-RADS v.2.1 updated definitions for scores 2 and 3 in the PZ and scores 1 and 2 in the TZ is reviewed. The role of DWI role is highlighted in the assessment of the TZ with the possibility of upgrading score 2 lesions to score 3 based on DWI score. Given the increased utilization for prostate MRI, biparametric MRI can be considered as an alternative for low-risk patients where there is a need to rule out csPCa acknowledging this technique may increase the number of indeterminate cases going for biopsies. Management of patients with equivocal lesions at mpMRI and factors influencing biopsy decision process remain as an unmet need and additional studies using molecular/imaging markers as well as artificial intelligence tools are needed to further address their role in proper patient selection for biopsy.

The Present and Future of Artificial Intelligence in Urological Cancer

Artificial intelligence has drawn more and more attention for both research and application in the field of medicine. It has considerable potential for urological cancer detection, therapy, and prognosis prediction due to its ability to choose features in data to complete a particular task autonomously. Although the clinical application of AI is still immature and faces drawbacks such as insufficient data and a lack of prospective clinical trials, AI will play an essential role in individualization and the whole management of cancers as research progresses. In this review, we summarize the applications and studies of AI in major urological cancers, including tumor diagnosis, treatment, and prognosis prediction. Moreover, we discuss the current challenges and future applications of AI.

What benefit can be obtained from magnetic resonance imaging diagnosis with artificial intelligence in prostate cancer compared with clinical assessments?

The present study aimed to explore the potential of artificial intelligence (AI) methodology based on magnetic resonance (MR) images to aid in the management of prostate cancer (PCa). To this end, we reviewed and summarized the studies comparing the diagnostic and predictive performance for PCa between AI and common clinical assessment methods based on MR images and/or clinical characteristics, thereby investigating whether AI methods are generally superior to common clinical assessment methods for the diagnosis and prediction fields of PCa. First, we found that, in the included studies of the present study, AI methods were generally equal to or better than the clinical assessment methods for the risk assessment of PCa, such as risk stratification of prostate lesions and the prediction of therapeutic outcomes or PCa progression. In particular, for the diagnosis of clinically significant PCa, the AI methods achieved a higher summary receiver operator characteristic curve (SROC-AUC) than that of the clinical assessment methods (0.87 vs. 0.82). For the prediction of adverse pathology, the AI methods also achieved a higher SROC-AUC than that of the clinical assessment methods (0.86 vs. 0.75). Second, as revealed by the radiomics quality score (RQS), the studies included in the present study presented a relatively high total average RQS of 15.2 (11.0-20.0). Further, the scores of the individual RQS elements implied that the AI models in these studies were constructed with relatively perfect and standard radiomics processes, but the exact generalizability and clinical practicality of the AI models should be further validated using higher levels of evidence, such as prospective studies and open-testing datasets.

Artificial Intelligence in CT and MR Imaging for Oncological Applications

Cancer care increasingly relies on imaging for patient management. The two most common cross-sectional imaging modalities in oncology are computed tomography (CT) and magnetic resonance imaging (MRI), which provide high-resolution anatomic and physiological imaging. Herewith is a summary of recent applications of rapidly advancing artificial intelligence (AI) in CT and MRI oncological imaging that addresses the benefits and challenges of the resultant opportunities with examples. Major challenges remain, such as how best to integrate AI developments into clinical radiology practice, the vigorous assessment of quantitative CT and MR imaging data accuracy, and reliability for clinical utility and research integrity in oncology. Such challenges necessitate an evaluation of the robustness of imaging biomarkers to be included in AI developments, a culture of data sharing, and the cooperation of knowledgeable academics with vendor scientists and companies operating in radiology and oncology fields. Herein, we will illustrate a few challenges and solutions of these efforts using novel methods for synthesizing different contrast modality images, auto-segmentation, and image reconstruction with examples from lung CT as well as abdome, pelvis, and head and neck MRI. The imaging community must embrace the need for quantitative CT and MRI metrics beyond lesion size measurement. AI methods for the extraction and longitudinal tracking of imaging metrics from registered lesions and understanding the tumor environment will be invaluable for interpreting disease status and treatment efficacy. This is an exciting time to work together to move the imaging field forward with narrow AI-specific tasks. New AI developments using CT and MRI datasets will be used to improve the personalized management of cancer patients.

Biparametric prostate MRI: impact of a deep learning-based software and of quantitative ADC values on the inter-reader agreement of experienced and inexperienced readers

Objective

To investigate the impact of an artificial intelligence (AI) software and quantitative ADC (qADC) on the inter-reader agreement, diagnostic performance, and reporting times of prostate biparametric MRI (bpMRI) for experienced and inexperienced readers.

Materials and methods

A total of 170 multiparametric MRI (mpMRI) of patients with suspicion of prostate cancer (PCa) were retrospectively reviewed by one experienced and one inexperienced reader three times, following a wash-out period. First, only the bpMRI sequences, including T2-weighted imaging (T2WI), diffusion-weighted imaging (DWI) sequences, and apparent diffusion coefficient (ADC) maps, were used. Then, bpMRI and quantitative ADC values were used. Lastly, bpMRI and the AI software were used. Inter-reader agreement between the two readers and between each reader and the mpMRI original reports was calculated. Detection rates and reporting times were calculated for each group.

Results

Inter-reader agreement with respect to mpMRI was moderate for bpMRI, Quantib, and qADC for both the inexperienced (weighted k of 0.42, 0.45, and 0.41, respectively) and the experienced radiologists (weighted k of 0.44, 0.46, and 0.42, respectively). Detection rate of PCa was similar between the inexperienced (0.24, 0.26, and 0.23) and the experienced reader (0.26, 0.27 and 0.27), for bpMRI, Quantib, and qADC, respectively. Reporting times were lower for Quantib (8.23, 7.11, and 9.87 min for the inexperienced reader and 5.62, 5.07, and 6.21 min for the experienced reader, for bpMRI, Quantib, and qADC, respectively).

Conclusions

AI and qADC did not have a significant impact on the diagnostic performance of both readers. The use of Quantib was associated with lower reporting times.