Deep learning based classification of solid lipid-poor contrast enhancing renal masses using contrast enhanced CT

Lesion-aware cross-phase attention network for renal tumor subtype classification on multi-phase CT scans

Multi-phase computed tomography (CT) has been widely used for the preoperative diagnosis of kidney cancer due to its non-invasive nature and ability to characterize renal lesions. However, since enhancement patterns of renal lesions across CT phases are different even for the same lesion type, the visual assessment by radiologists suffers from inter-observer variability in clinical practice. Although deep learning-based approaches have been recently explored for differential diagnosis of kidney cancer, they do not explicitly model the relationships between CT phases in the network design, limiting the diagnostic performance. In this paper, we propose a novel lesion-aware cross-phase attention network (LACPANet) that can effectively capture temporal dependencies of renal lesions across CT phases to accurately classify the lesions into five major pathological subtypes from time-series multi-phase CT images. We introduce a 3D inter-phase lesion-aware attention mechanism to learn effective 3D lesion features that are used to estimate attention weights describing the inter-phase relations of the enhancement patterns. We also present a multi-scale attention scheme to capture and aggregate temporal patterns of lesion features at different spatial scales for further improvement. Extensive experiments on multi-phase CT scans of kidney cancer patients from the collected dataset demonstrate that our LACPANet outperforms state-of-the-art approaches in diagnostic accuracy.

Update on Renal Cell Carcinoma Diagnosis with Novel Imaging Approaches

This review highlights recent advances in renal cell carcinoma (RCC) imaging. It begins with dual-energy computed tomography (DECT), which has demonstrated a high diagnostic accuracy in the evaluation of renal masses. Several studies have suggested the potential benefits of iodine quantification, particularly for distinguishing low-attenuation, true enhancing solid masses from hyperdense cysts. By determining whether or not a renal mass is present, DECT could avoid the need for additional imaging studies, thereby reducing healthcare costs. DECT can also provide virtual unenhanced images, helping to reduce radiation exposure. The review then provides an update focusing on the advantages of multiparametric magnetic resonance (MR) imaging performance in the histological subtyping of RCC and in the differentiation of benign from malignant renal masses. A proposed standardized stepwise reading of images helps to identify clear cell RCC and papillary RCC with a high accuracy. Contrast-enhanced ultrasound may represent a promising diagnostic tool for the characterization of solid and cystic renal masses. Several combined pharmaceutical imaging strategies using both sestamibi and PSMA offer new opportunities in the diagnosis and staging of RCC, but their role in risk stratification needs to be evaluated. Although radiomics and tumor texture analysis are hampered by poor reproducibility and need standardization, they show promise in identifying new biomarkers for predicting tumor histology, clinical outcomes, overall survival, and the response to therapy. They have a wide range of potential applications but are still in the research phase. Artificial intelligence (AI) has shown encouraging results in tumor classification, grade, and prognosis. It is expected to play an important role in assessing the treatment response and advancing personalized medicine. The review then focuses on recently updated algorithms and guidelines. The Bosniak classification version 2019 incorporates MRI, precisely defines previously vague imaging terms, and allows a greater proportion of masses to be placed in lower-risk classes. Recent studies have reported an improved specificity of the higher-risk categories and better inter-reader agreement. The clear cell likelihood score, which adds standardization to the characterization of solid renal masses on MRI, has been validated in recent studies with high interobserver agreement. Finally, the review discusses the key imaging implications of the 2017 AUA guidelines for renal masses and localized renal cancer.

Radiomics and Artificial Intelligence: Renal Cell Carcinoma

There is a clinical need for accurate diagnosis and prognostication of kidney cancer using imaging. Radiomics and deep learning methods applied to imaging have shown promise in tasks such as tumor segmentation, classification, staging, and grading, as well as assessment of preoperative scores and correlation with tumor biomarkers. Artificial intelligence is also expected to play a significant role in advancing personalized medicine for the treatment of renal cell carcinoma.

Convolutional neural networks for the differentiation between benign and malignant renal tumors with a multicenter international computed tomography dataset

OBJECTIVES

To use convolutional neural networks (CNNs) for the differentiation between benign and malignant renal tumors using contrast-enhanced CT images of a multi-institutional, multi-vendor, and multicenter CT dataset.

METHODS

A total of 264 histologically confirmed renal tumors were included, from US and Swedish centers. Images were augmented and divided randomly 70%:30% for algorithm training and testing. Three CNNs (InceptionV3, Inception-ResNetV2, VGG-16) were pretrained with transfer learning and fine-tuned with our dataset to distinguish between malignant and benign tumors. The ensemble consensus decision of the three networks was also recorded. Performance of each network was assessed with receiver operating characteristics (ROC) curves and their area under the curve (AUC-ROC). Saliency maps were created to demonstrate the attention of the highest performing CNN.

RESULTS

Inception-ResNetV2 achieved the highest AUC of 0.918 (95% CI 0.873-0.963), whereas VGG-16 achieved an AUC of 0.813 (95% CI 0.752-0.874). InceptionV3 and ensemble achieved the same performance with an AUC of 0.894 (95% CI 0.844-0.943). Saliency maps indicated that Inception-ResNetV2 decisions are based on the characteristics of the tumor while in most tumors considering the characteristics of the interface between the tumor and the surrounding renal parenchyma.

CONCLUSION

Deep learning based on a diverse multicenter international dataset can enable accurate differentiation between benign and malignant renal tumors.

CRITICAL RELEVANCE STATEMENT

Convolutional neural networks trained on a diverse CT dataset can accurately differentiate between benign and malignant renal tumors.

KEY POINTS

• Differentiation between benign and malignant tumors based on CT is extremely challenging. • Inception-ResNetV2 trained on a diverse dataset achieved excellent differentiation between tumor types. • Deep learning can be used to distinguish between benign and malignant renal tumors.

Creation of synthetic contrast-enhanced computed tomography images using deep neural networks to screen for renal cell carcinoma

In this study, we elucidate if synthetic contrast enhanced computed tomography images created from plain computed tomography images using deep neural networks could be used for screening, clinical diagnosis, and postoperative follow-up of small-diameter renal tumors. This retrospective, multicenter study included 155 patients (artificial intelligence training cohort [n = 99], validation cohort [n = 56]) who underwent surgery for small-diameter (≤40 mm) renal tumors, with the pathological diagnosis of renal cell carcinoma, during 2010-2020. We created a learned deep neural networks using pix2pix. We examined the quality of the synthetic enhanced computed tomography images created using this deep neural networks and compared them with real enhanced computed tomography images using the zero-mean normalized cross-correlation parameter. We assessed concordance rates between real and synthetic images and diagnoses according to 10 urologists by creating a receiver operating characteristic curve and calculating the area under the curve. The synthetic computed tomography images were highly concordant with the real computed tomography images, regardless of the existence or morphology of the renal tumor. Regarding the concordance rate, a greater area under the curve was obtained with synthetic computed tomography (area under the curve = 0.892) than with only computed tomography (area under the curve = 0.720; p < 0.001). In conclusions, this study is the first to use deep neural networks to create a high-quality synthetic computed tomography image that was highly concordant with a real computed tomography image. Our synthetic computed tomography images could be used for urological diagnoses and clinical screening.

Deep learning techniques for imaging diagnosis of renal cell carcinoma: current and emerging trends

This study summarizes the latest achievements, challenges, and future research directions in deep learning technologies for the diagnosis of renal cell carcinoma (RCC). This is the first review of deep learning in RCC applications. This review aims to show that deep learning technologies hold great promise in the field of RCC diagnosis, and we look forward to more research results to meet us for the mutual benefit of renal cell carcinoma patients. Medical imaging plays an important role in the early detection of renal cell carcinoma (RCC), as well as in the monitoring and evaluation of RCC during treatment. The most commonly used technologies such as contrast enhanced computed tomography (CECT), ultrasound and magnetic resonance imaging (MRI) are now digitalized, allowing deep learning to be applied to them. Deep learning is one of the fastest growing fields in the direction of medical imaging, with rapidly emerging applications that have changed the traditional medical treatment paradigm. With the help of deep learning-based medical imaging tools, clinicians can diagnose and evaluate renal tumors more accurately and quickly. This paper describes the application of deep learning-based imaging techniques in RCC assessment and provides a comprehensive review.

Artificial intelligence-based non-invasive tumor segmentation, grade stratification and prognosis prediction for clear-cell renal-cell carcinoma

Due to the complicated histopathological characteristics of clear-cell renal-cell carcinoma (ccRCC), non-invasive prognosis before operative treatment is crucial in selecting the appropriate treatment. A total of 126 345 computerized tomography (CT) images from four independent patient cohorts were included for analysis in this study. We propose a V Bottleneck multi-resolution and focus-organ network (VB-MrFo-Net) using a cascade framework for deep learning analysis. The VB-MrFo-Net achieved better performance than VB-Net in tumor segmentation, with a Dice score of 0.87. The nuclear-grade prediction model performed best in the logistic regression classifier, with area under curve values from 0.782 to 0.746. Survival analysis revealed that our prediction model could significantly distinguish patients with high survival risk, with a hazard ratio (HR) of 2.49 [95% confidence interval (CI): 1.13-5.45, P = 0.023] in the General cohort. Excellent performance had also been verified in the Cancer Genome Atlas cohort, the Clinical Proteomic Tumor Analysis Consortium cohort, and the Kidney Tumor Segmentation Challenge cohort, with HRs of 2.77 (95%CI: 1.58-4.84, P = 0.0019), 3.83 (95%CI: 1.22-11.96, P = 0.029), and 2.80 (95%CI: 1.05-7.47, P = 0.025), respectively. In conclusion, we propose a novel VB-MrFo-Net for the renal tumor segmentation and automatic diagnosis of ccRCC. The risk stratification model could accurately distinguish patients with high tumor grade and high survival risk based on non-invasive CT images before surgical treatments, which could provide practical advice for deciding treatment options.

Artificial intelligence as a diagnostic aid in cross-sectional radiological imaging of surgical pathology in the abdominopelvic cavity: a systematic review

OBJECTIVES

There is emerging use of artificial intelligence (AI) models to aid diagnostic imaging. This review examined and critically appraised the application of AI models to identify surgical pathology from radiological images of the abdominopelvic cavity, to identify current limitations and inform future research.

DESIGN

Systematic review.

DATA SOURCES

Systematic database searches (Medline, EMBASE, Cochrane Central Register of Controlled Trials) were performed. Date limitations (January 2012 to July 2021) were applied.

ELIGIBILITY CRITERIA

Primary research studies were considered for eligibility using the PIRT (participants, index test(s), reference standard and target condition) framework. Only publications in the English language were eligible for inclusion in the review.

DATA EXTRACTION AND SYNTHESIS

Study characteristics, descriptions of AI models and outcomes assessing diagnostic performance were extracted by independent reviewers. A narrative synthesis was performed in accordance with the Synthesis Without Meta-analysis guidelines. Risk of bias was assessed (Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2)).

RESULTS

Fifteen retrospective studies were included. Studies were diverse in surgical specialty, the intention of the AI applications and the models used. AI training and test sets comprised a median of 130 (range: 5-2440) and 37 (range: 10-1045) patients, respectively. Diagnostic performance of models varied (range: 70%-95% sensitivity, 53%-98% specificity). Only four studies compared the AI model with human performance. Reporting of studies was unstandardised and often lacking in detail. Most studies (n=14) were judged as having overall high risk of bias with concerns regarding applicability.

CONCLUSIONS

AI application in this field is diverse. Adherence to reporting guidelines is warranted. With finite healthcare resources, future endeavours may benefit from targeting areas where radiological expertise is in high demand to provide greater efficiency in clinical care. Translation to clinical practice and adoption of a multidisciplinary approach should be of high priority.

PROSPERO REGISTRATION NUMBER

CRD42021237249.

Recognizing pathology of renal tumor from macroscopic cross-section image by deep learning

OBJECTIVES

This study aims to develop and evaluate the deep learning-based classification model for recognizing the pathology of renal tumor from macroscopic cross-section image.

METHODS

A total of 467 pathology-confirmed patients who received radical nephrectomy or partial nephrectomy were retrospectively enrolled. The experiment of distinguishing malignant and benign renal tumor are conducted followed by performing the multi-subtypes classification models for recognizing four subtypes of benign tumor and four subtypes of malignant tumors, respectively. The classification models used the same backbone networks which are based on the convolutional neural network (CNN), including EfficientNet-B4, ResNet-18, and VGG-16. The performance of the classification models was evaluated by area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and accuracy. Besides, we performed the quantitative comparison among these CNN models.

RESULTS

For the model to differentiate the malignant tumor from the benign tumor, three CNN models all obtained relatively satisfactory performance and the highest AUC was achieved by the ResNet-18 model (AUC = 0.9226). There is not statistically significance between EfficientNet-B4 and ResNet-18 architectures and both of them are significantly statistically better than the VGG-16 model. The micro-averaged AUC, macro-averaged sensitivity, macro-averaged specificity, and micro-averaged accuracy for the VGG-16 model to distinguish the malignant tumor subtypes achieved 0.9398, 0.5774, 0.8660, and 0.7917, respectively. The performance of the EfficientNet-B4 is not better than that of VGG-16 in terms of micro-averaged AUC except for other metrics. For the models to recognize the benign tumor subtypes, the EfficientNet-B4 ranked the best performance, but had no significantly statistical difference with other two models with respect to micro-averaged AUC.

CONCLUSIONS

The classification results were relatively satisfactory, which showed the potential for clinical application when analyzing the renal tumor macroscopic cross-section images. Automatically distinguishing the malignant tumor from benign tumor and identifying the subtypes pathology of renal tumor could make the patient-management process more efficient.

AI-Driven Robust Kidney and Renal Mass Segmentation and Classification on 3D CT Images

Early intervention in kidney cancer helps to improve survival rates. Abdominal computed tomography (CT) is often used to diagnose renal masses. In clinical practice, the manual segmentation and quantification of organs and tumors are expensive and time-consuming. Artificial intelligence (AI) has shown a significant advantage in assisting cancer diagnosis. To reduce the workload of manual segmentation and avoid unnecessary biopsies or surgeries, in this paper, we propose a novel end-to-end AI-driven automatic kidney and renal mass diagnosis framework to identify the abnormal areas of the kidney and diagnose the histological subtypes of renal cell carcinoma (RCC). The proposed framework first segments the kidney and renal mass regions by a 3D deep learning architecture (Res-UNet), followed by a dual-path classification network utilizing local and global features for the subtype prediction of the most common RCCs: clear cell, chromophobe, oncocytoma, papillary, and other RCC subtypes. To improve the robustness of the proposed framework on the dataset collected from various institutions, a weakly supervised learning schema is proposed to leverage the domain gap between various vendors via very few CT slice annotations. Our proposed diagnosis system can accurately segment the kidney and renal mass regions and predict tumor subtypes, outperforming existing methods on the KiTs19 dataset. Furthermore, cross-dataset validation results demonstrate the robustness of datasets collected from different institutions trained via the weakly supervised learning schema.

Advances in Imaging-Based Biomarkers in Renal Cell Carcinoma: A Critical Analysis of the Current Literature

Cross-sectional imaging is the standard diagnostic tool to determine underlying biology in renal masses, which is crucial for subsequent treatment. Currently, standard CT imaging is limited in its ability to differentiate benign from malignant disease. Therefore, various modalities have been investigated to identify imaging-based parameters to improve the noninvasive diagnosis of renal masses and renal cell carcinoma (RCC) subtypes. MRI was reported to predict grading of RCC and to identify RCC subtypes, and has been shown in a small cohort to predict the response to targeted therapy. Dynamic imaging is promising for the staging and diagnosis of RCC. PET/CT radiotracers, such as ¹⁸F-fluorodeoxyglucose (FDG), ¹²⁴I-cG250, radiolabeled prostate-specific membrane antigen (PSMA), and ¹¹C-acetate, have been reported to improve the identification of histology, grading, detection of metastasis, and assessment of response to systemic therapy, and to predict oncological outcomes. Moreover, ⁹⁹Tc-sestamibi and SPECT scans have shown promising results in distinguishing low-grade RCC from benign lesions. Radiomics has been used to further characterize renal masses based on semantic and textural analyses. In preliminary studies, integrated machine learning algorithms using radiomics proved to be more accurate in distinguishing benign from malignant renal masses compared to radiologists' interpretations. Radiomics and radiogenomics are used to complement risk classification models to predict oncological outcomes. Imaging-based biomarkers hold strong potential in RCC, but require standardization and external validation before integration into clinical routines.

Deep Learning for Image Analysis in Kidney Care

Analysis of medical images, such as radiological or tissue specimens, is an indispensable part of medical diagnostics. Conventionally done manually, the process may sometimes be time-consuming and prone to interobserver variability. Image classification and segmentation by deep learning strategies, predominantly convolutional neural networks, may provide a significant advance in the diagnostic process. In renal medicine, most evidence has been generated around the radiological assessment of renal abnormalities and histological analysis of renal biopsy specimens' segmentation. In this article, the basic principles of image analysis by convolutional neural networks, brief descriptions of convolutional neural networks, and their system architecture for image analysis are discussed, in combination with examples regarding their use in image analysis in nephrology.

Novel Imaging Methods for Renal Mass Characterization: A Collaborative Review

CONTEXT

The incidental detection of localized renal masses has been rising steadily, but a significant proportion of these tumors are benign or indolent and, in most cases, do not require treatment. At the present time, a majority of patients with an incidentally detected renal tumor undergo treatment for the presumption of cancer, leading to a significant number of unnecessary surgical interventions that can result in complications including loss of renal function. Thus, there exists a clinical need for improved tools to aid in the pretreatment characterization of renal tumors to inform patient management.

OBJECTIVE

To systematically review the evidence on noninvasive, imaging-based tools for solid renal mass characterization.

EVIDENCE ACQUISITION

The MEDLINE database was systematically searched for relevant studies on novel imaging techniques and interpretative tools for the characterization of solid renal masses, published in the past 10 yr.

EVIDENCE SYNTHESIS

Over the past decade, several novel imaging tools have offered promise for the improved characterization of indeterminate renal masses. Technologies of particular note include multiparametric magnetic resonance imaging of the kidney, molecular imaging with targeted radiopharmaceutical agents, and use of radiomics as well as artificial intelligence to enhance the interpretation of imaging studies. Among these, 99mTc-sestamibi single photon emission computed tomography/computed tomography (CT) for the identification of benign renal oncocytomas and hybrid oncocytic chromophobe tumors, and positron emission tomography/CT imaging with radiolabeled girentuximab for the identification of clear cell renal cell carcinoma, are likely to be closest to implementation in clinical practice.

CONCLUSIONS

A number of novel imaging tools stand poised to aid in the noninvasive characterization of indeterminate renal masses. In the future, these tools may aid in patient management by providing a comprehensive virtual biopsy, complete with information on tumor histology, underlying molecular abnormalities, and ultimately disease prognosis.

PATIENT SUMMARY

Not all renal tumors require treatment, as a significant proportion are either benign or have limited metastatic potential. Several innovative imaging tools have shown promise for their ability to improve the characterization of renal tumors and provide guidance in terms of patient management.

A Comprehensive Computer-Assisted Diagnosis System for Early Assessment of Renal Cancer Tumors

Renal cell carcinoma (RCC) is the most common and a highly aggressive type of malignant renal tumor. In this manuscript, we aim to identify and integrate the optimal discriminating morphological, textural, and functional features that best describe the malignancy status of a given renal tumor. The integrated discriminating features may lead to the development of a novel comprehensive renal cancer computer-assisted diagnosis (RC-CAD) system with the ability to discriminate between benign and malignant renal tumors and specify the malignancy subtypes for optimal medical management. Informed consent was obtained from a total of 140 biopsy-proven patients to participate in the study (male = 72 and female = 68, age range = 15 to 87 years). There were 70 patients who had RCC (40 clear cell RCC (ccRCC), 30 nonclear cell RCC (nccRCC)), while the other 70 had benign angiomyolipoma tumors. Contrast-enhanced computed tomography (CE-CT) images were acquired, and renal tumors were segmented for all patients to allow the extraction of discriminating imaging features. The RC-CAD system incorporates the following major steps: (i) applying a new parametric spherical harmonic technique to estimate the morphological features, (ii) modeling a novel angular invariant gray-level co-occurrence matrix to estimate the textural features, and (iii) constructing wash-in/wash-out slopes to estimate the functional features by quantifying enhancement variations across different CE-CT phases. These features were subsequently combined and processed using a two-stage multilayer perceptron artificial neural network (MLP-ANN) classifier to classify the renal tumor as benign or malignant and identify the malignancy subtype as well. Using the combined features and a leave-one-subject-out cross-validation approach, the developed RC-CAD system achieved a sensitivity of 95.3%±2.0%, a specificity of 99.9%±0.4%, and Dice similarity coefficient of 0.98±0.01 in differentiating malignant from benign tumors, as well as an overall accuracy of 89.6%±5.0% in discriminating ccRCC from nccRCC. The diagnostic abilities of the developed RC-CAD system were further validated using a randomly stratified 10-fold cross-validation approach. The obtained results using the proposed MLP-ANN classification model outperformed other machine learning classifiers (e.g., support vector machine, random forests, relational functional gradient boosting, etc.). Hence, integrating morphological, textural, and functional features enhances the diagnostic performance, making the proposal a reliable noninvasive diagnostic tool for renal tumors.

Kidney cancer management 3.0: can artificial intelligence make us better?

PURPOSE OF REVIEW

Artificial intelligence holds tremendous potential for disrupting clinical medicine. Here we review the current role of artificial intelligence in the kidney cancer space.

RECENT FINDINGS

Machine learning and deep learning algorithms have been developed using information extracted from radiomic, histopathologic, and genomic datasets of patients with renal masses.

SUMMARY

Although artificial intelligence applications in medicine are still in their infancy, they already hold immediate promise to improve accuracy of renal mass characterization, grade, and prognostication. As algorithms become more robust and generalizable, artificial intelligence is poised to significantly disrupt kidney cancer care.

Deep learning for end-to-end kidney cancer diagnosis on multi-phase abdominal computed tomography

In 2020, it is estimated that 73,750 kidney cancer cases were diagnosed, and 14,830 people died from cancer in the United States. Preoperative multi-phase abdominal computed tomography (CT) is often used for detecting lesions and classifying histologic subtypes of renal tumor to avoid unnecessary biopsy or surgery. However, there exists inter-observer variability due to subtle differences in the imaging features of tumor subtypes, which makes decisions on treatment challenging. While deep learning has been recently applied to the automated diagnosis of renal tumor, classification of a wide range of subtype classes has not been sufficiently studied yet. In this paper, we propose an end-to-end deep learning model for the differential diagnosis of five major histologic subtypes of renal tumors including both benign and malignant tumors on multi-phase CT. Our model is a unified framework to simultaneously identify lesions and classify subtypes for the diagnosis without manual intervention. We trained and tested the model using CT data from 308 patients who underwent nephrectomy for renal tumors. The model achieved an area under the curve (AUC) of 0.889, and outperformed radiologists for most subtypes. We further validated the model on an independent dataset of 184 patients from The Cancer Imaging Archive (TCIA). The AUC for this dataset was 0.855, and the model performed comparably to the radiologists. These results indicate that our model can achieve similar or better diagnostic performance than radiologists in differentiating a wide range of renal tumors on multi-phase CT.