Urinary Stone Detection on CT Images Using Deep Convolutional Neural Networks: Evaluation of Model Performance and Generalization

From Diagnosis to Precision Surgery: The Transformative Role of Artificial Intelligence in Urologic Imaging

The multidisciplinary nature of artificial intelligence (AI) has allowed for rapid growth of its application in medical imaging. Artificial intelligence algorithms can augment various imaging modalities, such as X-rays, CT, and MRI, to improve image quality and generate high-resolution three-dimensional images. AI reconstruction of three-dimensional models of patient anatomy from CT or MRI scans can better enable urologists to visualize structures and accurately plan surgical approaches. AI can also be optimized to create virtual reality simulations of surgical procedures based on patient-specific data, giving urologists more hands-on experience and preparation. Recent development of artificial intelligence modalities, such as TeraRecon and Ceevra, offer rapid and efficient medical imaging analyses aimed at enhancing the provision of urologic care, notably for intraoperative guidance during robot-assisted radical prostatectomy (RARP) and partial nephrectomy.

Elevating healthcare through artificial intelligence: analyzing the abdominal emergencies data set (TR_ABDOMEN_RAD_EMERGENCY) at TEKNOFEST-2022

OBJECTIVES

The artificial intelligence competition in healthcare at TEKNOFEST-2022 provided a platform to address the complex multi-class classification challenge of abdominal emergencies using computer vision techniques. This manuscript aimed to comprehensively present the methodologies for data preparation, annotation procedures, and rigorous evaluation metrics. Moreover, it was conducted to introduce a meticulously curated abdominal emergencies data set to the researchers.

METHODS

The data set underwent a comprehensive central screening procedure employing diverse algorithms extracted from the e-Nabız (Pulse) and National Teleradiology System of the Republic of Türkiye, Ministry of Health. Full anonymization of the data set was conducted. Subsequently, the data set was annotated by a group of ten experienced radiologists. The evaluation process was executed by calculating F1 scores, which were derived from the intersection over union values between the predicted bounding boxes and the corresponding ground truth (GT) bounding boxes. The establishment of baseline performance metrics involved computing the average of the highest five F1 scores.

RESULTS

Observations indicated a progressive decline in F1 scores as the threshold value increased. Furthermore, it could be deduced that class 6 (abdominal aortic aneurysm/dissection) was relatively straightforward to detect compared to other classes, with class 5 (acute diverticulitis) presenting the most formidable challenge. It is noteworthy, however, that if all achieved outcomes for all classes were considered with a threshold of 0.5, the data set's complexity and associated challenges became pronounced.

CONCLUSION

This data set's significance lies in its pioneering provision of labels and GT-boxes for six classes, fostering opportunities for researchers.

CLINICAL RELEVANCE STATEMENT

The prompt identification and timely intervention in cases of emergent medical conditions hold paramount significance. The handling of patients' care can be augmented, while the potential for errors is minimized, particularly amidst high caseload scenarios, through the application of AI.

KEY POINTS

• The data set used in artificial intelligence competition in healthcare (TEKNOFEST-2022) provides a 6-class data set of abdominal CT images consisting of a great variety of abdominal emergencies. • This data set is compiled from the National Teleradiology System data repository of emergency radiology departments of 459 hospitals. • Radiological data on abdominal emergencies is scarce in literature and this annotated competition data set can be a valuable resource for further studies and new AI models.

Can Artificial Intelligence Accurately Detect Urinary Stones? A Systematic Review

Objectives: To perform a systematic review on artificial intelligence (AI) performances to detect urinary stones. Methods: A PROSPERO-registered (CRD473152) systematic search of Scopus, Web of Science, Embase, and PubMed databases was performed to identify original research articles pertaining to AI stone detection or measurement, using search terms ("automatic" OR "machine learning" OR "convolutional neural network" OR "artificial intelligence" OR "detection" AND "stone volume"). Risk-of-bias (RoB) assessment was performed according to the Cochrane RoB tool, the Joanna Briggs Institute Checklist for nonrandomized studies, and the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Results: Twelve studies were selected for the final review, including three multicenter and nine single-center retrospective studies. Eleven studies completed at least 50% of the CLAIM checkpoints and only one presented a high RoB. All included studies aimed to detect kidney (5/12, 42%), ureter (2/12, 16%), or urinary (5/12, 42%) stones on noncontrast computed tomography (NCCT), but 42% intended to automate measurement. Stone distinction from vascular calcification interested two studies. All studies used AI machine learning network training and internal validation, but a single one provided an external validation. Trained networks achieved stone detection, with sensitivity, specificity, and accuracy rates ranging from 58.7% to 100%, 68.5% to 100%, and 63% to 99.95%, respectively. Detection Dice score ranged from 83% to 97%. A high correlation between manual and automated stone volume (r = 0.95) was noted. Differentiate distal ureteral stones and phleboliths seemed feasible. Conclusions: AI processes can achieve automated urinary stone detection from NCCT. Further studies should provide urinary stone detection coupled with phlebolith distinction and an external validation, and include anatomical abnormalities and urologic foreign bodies (ureteral stent and nephrostomy tubes) cases.

Applications of Artificial Intelligence in Acute Abdominal Imaging

Artificial intelligence (AI) is a rapidly growing field with significant implications for radiology. Acute abdominal pain is a common clinical presentation that can range from benign conditions to life-threatening emergencies. The critical nature of these situations renders emergent abdominal imaging an ideal candidate for AI applications. CT, radiographs, and ultrasound are the most common modalities for imaging evaluation of these patients. For each modality, numerous studies have assessed the performance of AI models for detecting common pathologies, such as appendicitis, bowel obstruction, and cholecystitis. The capabilities of these models range from simple classification to detailed severity assessment. This narrative review explores the evolution, trends, and challenges in AI applications for evaluating acute abdominal pathologies. We review implementations of AI for non-traumatic and traumatic abdominal pathologies, with discussion of potential clinical impact, challenges, and future directions for the technology.

Automatic Urinary Stone Detection System for Abdominal Non-Enhanced CT Images Reduces the Burden on Radiologists

To develop a fully automatic urinary stone detection system (kidney, ureter, and bladder) and to test it in a real clinical environment. The local institutional review board approved this retrospective single-center study that used non-enhanced abdominopelvic CT scans from patients admitted urology (uPatients) and emergency (ePatients). The uPatients were randomly divided into training and validation sets in a ratio of 3:1. We designed a cascade urinary stone map location-feature pyramid networks (USm-FPNs) and innovatively proposed a ureter distance heatmap method to estimate the ureter position on non-enhanced CT to further reduce the false positives. The performances of the system were compared using the free-response receiver operating characteristic curve and the precision-recall curve. This study included 811 uPatients and 356 ePatients. At stone level, the cascade detector USm-FPNs has the mean of false positives per scan (mFP) 1.88 with the sensitivity 0.977 in validation set, and mFP was further reduced to 1.18 with the sensitivity 0.977 after combining the ureter distance heatmap. At patient level, the sensitivity and precision were as high as 0.995 and 0.990 in validation set, respectively. In a real clinical set of ePatients (27.5% of patients contain stones), the mFP was 1.31 with as high as sensitivity 0.977, and the diagnostic time reduced by > 20% with the system help. A fully automatic detection system for entire urinary stones on non-enhanced CT scans was proposed and reduces obviously the burden on junior radiologists without compromising sensitivity in real emergency data.

Exploring deep learning radiomics for classifying osteoporotic vertebral fractures in X-ray images

Purpose

To develop and validate a deep learning radiomics (DLR) model that uses X-ray images to predict the classification of osteoporotic vertebral fractures (OVFs).

Material and methods

The study encompassed a cohort of 942 patients, involving examinations of 1076 vertebrae through X-ray, CT, and MRI across three distinct hospitals. The OVFs were categorized as class 0, 1, or 2 based on the Assessment System of Thoracolumbar Osteoporotic Fracture. The dataset was divided randomly into four distinct subsets: a training set comprising 712 samples, an internal validation set with 178 samples, an external validation set containing 111 samples, and a prospective validation set consisting of 75 samples. The ResNet-50 architectural model was used to implement deep transfer learning (DTL), undergoing -pre-training separately on the RadImageNet and ImageNet datasets. Features from DTL and radiomics were extracted and integrated using X-ray images. The optimal fusion feature model was identified through least absolute shrinkage and selection operator logistic regression. Evaluation of the predictive capabilities for OVFs classification involved eight machine learning models, assessed through receiver operating characteristic curves employing the "One-vs-Rest" strategy. The Delong test was applied to compare the predictive performance of the superior RadImageNet model against the ImageNet model.

Results

Following pre-training separately on RadImageNet and ImageNet datasets, feature selection and fusion yielded 17 and 12 fusion features, respectively. Logistic regression emerged as the optimal machine learning algorithm for both DLR models. Across the training set, internal validation set, external validation set, and prospective validation set, the macro-average Area Under the Curve (AUC) based on the RadImageNet dataset surpassed those based on the ImageNet dataset, with statistically significant differences observed (P<0.05). Utilizing the binary "One-vs-Rest" strategy, the model based on the RadImageNet dataset demonstrated superior efficacy in predicting Class 0, achieving an AUC of 0.969 and accuracy of 0.863. Predicting Class 1 yielded an AUC of 0.945 and accuracy of 0.875, while for Class 2, the AUC and accuracy were 0.809 and 0.692, respectively.

Conclusion

The DLR model, based on the RadImageNet dataset, outperformed the ImageNet model in predicting the classification of OVFs, with generalizability confirmed in the prospective validation set.

Surgical Artificial Intelligence: Endourology

Endourology is ripe with information that includes patient factors, laboratory tests, outcomes, and visual data, which is becoming increasingly complex to assess. Artificial intelligence (AI) has the potential to explore and define these relationships; however, humans might not be involved in the input, analysis, or even determining the methods of analysis. Herein, the authors present the current state of AI in endourology and highlight the need for urologists to share their proposed AI solutions for reproducibility outside of their institutions and prepare themselves to properly critique this new technology.

StoneNet: An Efficient Lightweight Model Based on Depthwise Separable Convolutions for Kidney Stone Detection from CT Images

Kidney stone disease is one of the most common and serious health problems in much of the world, leading to many hospitalizations with severe pain. Detecting small stones is difficult and time-consuming, so an early diagnosis of kidney disease is needed to prevent the loss of kidney failure. Recent advances in artificial intelligence (AI) found to be very successful in the diagnosis of various diseases in the biomedical field. However, existing models using deep networks have several problems, such as high computational cost, long training time, and huge parameters. Providing a low-cost solution for diagnosing kidney stones in a medical decision support system is of paramount importance. Therefore, in this study, we propose "StoneNet", a lightweight and high-performance model for the detection of kidney stones based on MobileNet using depthwise separable convolution. The proposed model includes a combination of global average pooling (GAP), batch normalization, dropout layer, and dense layers. Our study shows that using GAP instead of flattening layers greatly improves the robustness of the model by significantly reducing the parameters. The developed model is benchmarked against four pre-trained models as well as the state-of-the-art heavy model. The results show that the proposed model can achieve the highest accuracy of 97.98%, and only requires training and testing time of 996.88 s and 14.62 s. Several parameters, such as different batch sizes and optimizers, were considered to validate the proposed model. The proposed model is computationally faster and provides optimal performance than other considered models. Experiments on a large kidney dataset of 1799 CT images show that StoneNet has superior performance in terms of higher accuracy and lower complexity. The proposed model can assist the radiologist in faster diagnosis of kidney stones and has great potential for deployment in real-time applications.

Emerging Trends in Artificial Intelligence-Based Urological Imaging Technologies and Practical Applications

The integration of artificial intelligence (AI) into medical imaging has notably expanded its significance within urology. AI applications offer a broad spectrum of utilities in this domain, ranging from precise diagnosis achieved through image segmentation and anomaly detection to improved procedural assistance in biopsies and surgical interventions. Although challenges persist concerning data security, transparency, and integration into existing clinical workflows, extensive research has been conducted on AI-assisted imaging technologies while recognizing their potential to reshape urological practices. This review paper outlines current AI techniques employed for image analysis to offer an overview of the latest technological trends and applications in the field of urology.

Improved Detection of Urolithiasis Using High-Resolution Computed Tomography Images by a Vision Transformer Model

PURPOSE

Urinary stones cause lateral abdominal pain and are a prevalent condition among younger age groups. The diagnosis typically involves assessing symptoms, conducting physical examinations, performing urine tests, and utilizing radiological imaging. Artificial intelligence models have demonstrated remarkable capabilities in detecting stones. However, due to insufficient datasets, the performance of these models has not reached a level suitable for practical application. Consequently, this study introduces a vision transformer (ViT)-based pipeline for detecting urinary stones, using computed tomography images with augmentation.

METHODS

The super-resolution convolutional neural network (SRCNN) model was employed to enhance the resolution of a given dataset, followed by data augmentation using CycleGAN. Subsequently, the ViT model facilitated the detection and classification of urinary tract stones. The model's performance was evaluated using accuracy, precision, and recall as metrics.

RESULTS

The deep learning model based on ViT showed superior performance compared to other existing models. Furthermore, the performance increased with the size of the backbone model.

CONCLUSION

The study proposes a way to utilize medical data to improve the diagnosis of urinary tract stones. SRCNN was used for data preprocessing to enhance resolution, while CycleGAN was utilized for data augmentation. The ViT model was utilized for stone detection, and its performance was validated through metrics such as accuracy, sensitivity, specificity, and the F1 score. It is anticipated that this research will aid in the early diagnosis and treatment of urinary tract stones, thereby improving the efficiency of medical personnel.

A deep learning model for automated kidney calculi detection on non-contrast computed tomography scans in dogs

Nephrolithiasis is one of the most common urinary disorders in dogs. Although a majority of kidney calculi are non-obstructive and are likely to be asymptomatic, they can lead to parenchymal loss and obstruction as they progress. Thus, early diagnosis of kidney calculi is important for patient monitoring and better prognosis. However, detecting kidney calculi and monitoring changes in the sizes of the calculi from computed tomography (CT) images is time-consuming for clinicians. This study, in a first of its kind, aims to develop a deep learning model for automatic kidney calculi detection using pre-contrast CT images of dogs. A total of 34,655 transverseimage slices obtained from 76 dogs with kidney calculi were used to develop the deep learning model. Because of the differences in kidney location and calculi sizes in dogs compared to humans, several processing methods were used. The first stage of the models, based on the Attention U-Net (AttUNet), was designed to detect the kidney for the coarse feature map. Five different models-AttUNet, UTNet, TransUNet, SwinUNet, and RBCANet-were used in the second stage to detect the calculi in the kidneys, and the performance of the models was evaluated. Compared with a previously developed model, all the models developed in this study yielded better dice similarity coefficients (DSCs) for the automatic segmentation of the kidney. To detect kidney calculi, RBCANet and SwinUNet yielded the best DSC, which was 0.74. In conclusion, the deep learning model developed in this study can be useful for the automated detection of kidney calculi.

Hybrid morphological-convolutional neural networks for computer-aided diagnosis

Training deep Convolutional Neural Networks (CNNs) presents challenges in terms of memory requirements and computational resources, often resulting in issues such as model overfitting and lack of generalization. These challenges can only be mitigated by using an excessive number of training images. However, medical image datasets commonly suffer from data scarcity due to the complexities involved in their acquisition, preparation, and curation. To address this issue, we propose a compact and hybrid machine learning architecture based on the Morphological and Convolutional Neural Network (MCNN), followed by a Random Forest classifier. Unlike deep CNN architectures, the MCNN was specifically designed to achieve effective performance with medical image datasets limited to a few hundred samples. It incorporates various morphological operations into a single layer and uses independent neural networks to extract information from each signal channel. The final classification is obtained by utilizing a Random Forest classifier on the outputs of the last neural network layer. We compare the classification performance of our proposed method with three popular deep CNN architectures (ResNet-18, ShuffleNet-V2, and MobileNet-V2) using two training approaches: full training and transfer learning. The evaluation was conducted on two distinct medical image datasets: the ISIC dataset for melanoma classification and the ORIGA dataset for glaucoma classification. Results demonstrate that the MCNN method exhibits reliable performance in melanoma classification, achieving an AUC of 0.94 (95% CI: 0.91 to 0.97), outperforming the popular CNN architectures. For the glaucoma dataset, the MCNN achieved an AUC of 0.65 (95% CI: 0.53 to 0.74), which was similar to the performance of the popular CNN architectures. This study contributes to the understanding of mathematical morphology in shallow neural networks for medical image classification and highlights the potential of hybrid architectures in effectively learning from medical image datasets that are limited by a small number of case samples.

Design and Validation of a Deep Learning Model for Renal Stone Detection and Segmentation on Kidney-Ureter-Bladder Images

Kidney-ureter-bladder (KUB) imaging is used as a frontline investigation for patients with suspected renal stones. In this study, we designed a computer-aided diagnostic system for KUB imaging to assist clinicians in accurately diagnosing urinary tract stones. The image dataset used for training and testing the model comprised 485 images provided by Kaohsiung Chang Gung Memorial Hospital. The proposed system was divided into two subsystems, 1 and 2. Subsystem 1 used Inception-ResNetV2 to train a deep learning model on preprocessed KUB images to verify the improvement in diagnostic accuracy with image preprocessing. Subsystem 2 trained an image segmentation model using the ResNet hybrid, U-net, to accurately identify the contours of renal stones. The performance was evaluated using a confusion matrix for the classification model. We conclude that the model can assist clinicians in accurately diagnosing renal stones via KUB imaging. Therefore, the proposed system can assist doctors in diagnosis, reduce patients' waiting time for CT scans, and minimize the radiation dose absorbed by the body.

Transforming urinary stone disease management by artificial intelligence-based methods: A comprehensive review

Objective

To provide a comprehensive review on the existing research and evidence regarding artificial intelligence (AI) applications in the assessment and management of urinary stone disease.

Methods

A comprehensive literature review was performed using PubMed, Scopus, and Google Scholar databases to identify publications about innovative concepts or supporting applications of AI in the improvement of every medical procedure relating to stone disease. The terms ''endourology'', ''artificial intelligence'', ''machine learning'', and ''urolithiasis'' were used for searching eligible reports, while review articles, articles referring to automated procedures without AI application, and editorial comments were excluded from the final set of publications. The search was conducted from January 2000 to September 2023 and included manuscripts in the English language.

Results

A total of 69 studies were identified. The main subjects were related to the detection of urinary stones, the prediction of the outcome of conservative or operative management, the optimization of operative procedures, and the elucidation of the relation of urinary stone chemistry with various factors.

Conclusion

AI represents a useful tool that provides urologists with numerous amenities, which explains the fact that it has gained ground in the pursuit of stone disease management perfection. The effectiveness of diagnosis and therapy can be increased by using it as an alternative or adjunct to the already existing data. However, little is known concerning the potential of this vast field. Electronic patient records, containing big data, offer AI the opportunity to develop and analyze more precise and efficient diagnostic and treatment algorithms. Nevertheless, the existing applications are not generalizable in real-life practice, and high-quality studies are needed to establish the integration of AI in the management of urinary stone disease.

A multi-class deep learning model for early lung cancer and chronic kidney disease detection using computed tomography images

Lung cancer is a fatal disease caused by an abnormal proliferation of cells in the lungs. Similarly, chronic kidney disorders affect people worldwide and can lead to renal failure and impaired kidney function. Cyst development, kidney stones, and tumors are frequent diseases impairing kidney function. Since these conditions are generally asymptomatic, early, and accurate identification of lung cancer and renal conditions is necessary to prevent serious complications. Artificial Intelligence plays a vital role in the early detection of lethal diseases. In this paper, we proposed a modified Xception deep neural network-based computer-aided diagnosis model, consisting of transfer learning based image net weights of Xception model and a fine-tuned network for automatic lung and kidney computed tomography multi-class image classification. The proposed model obtained 99.39% accuracy, 99.33% precision, 98% recall, and 98.67% F1-score for lung cancer multi-class classification. Whereas, it attained 100% accuracy, F1 score, recall and precision for kidney disease multi-class classification. Also, the proposed modified Xception model outperformed the original Xception model and the existing methods. Hence, it can serve as a support tool to the radiologists and nephrologists for early detection of lung cancer and chronic kidney disease, respectively.

Deep Learning Model for Computer-Aided Diagnosis of Urolithiasis Detection from Kidney-Ureter-Bladder Images

Kidney-ureter-bladder (KUB) imaging is a radiological examination with a low cost, low radiation, and convenience. Although emergency room clinicians can arrange KUB images easily as a first-line examination for patients with suspicious urolithiasis, interpreting the KUB images correctly is difficult for inexperienced clinicians. Obtaining a formal radiology report immediately after a KUB imaging examination can also be challenging. Recently, artificial-intelligence-based computer-aided diagnosis (CAD) systems have been developed to help clinicians who are not experts make correct diagnoses for further treatment more effectively. Therefore, in this study, we proposed a CAD system for KUB imaging based on a deep learning model designed to help first-line emergency room clinicians diagnose urolithiasis accurately. A total of 355 KUB images were retrospectively collected from 104 patients who were diagnosed with urolithiasis at Kaohsiung Chang Gung Memorial Hospital. Then, we trained a deep learning model with a ResNet architecture to classify KUB images in terms of the presence or absence of kidney stones with this dataset of pre-processed images. Finally, we tuned the parameters and tested the model experimentally. The results show that the accuracy, sensitivity, specificity, and F1-measure of the model were 0.977, 0.953, 1, and 0.976 on the validation set and 0.982, 0.964, 1, and 0.982 on the testing set, respectively. Moreover, the results demonstrate that the proposed model performed well compared to the existing CNN-based methods and was able to detect urolithiasis in KUB images successfully. We expect the proposed approach to help emergency room clinicians make accurate diagnoses and reduce unnecessary radiation exposure from computed tomography (CT) scans, along with the associated medical costs.

Theranostic roles of machine learning in clinical management of kidney stone disease

Kidney stone disease (KSD) is a common illness caused by deposition of solid minerals formed inside the kidney. The disease prevalence varies, based on sociodemographic, lifestyle, dietary, genetic, gender, age, environmental and climatic factors, but has been continuously increasing worldwide. KSD is a highly recurrent disease, and the recurrence rate is about 11% within two years after the stone removal. Recently, machine learning has been widely used for KSD detection, stone type prediction, determination of appropriate treatment modality and prediction of therapeutic outcome. This review provides a brief overview of KSD and discusses how machine learning can be applied to diagnostics, therapeutics and prognostics in clinical management of KSD for better therapeutic outcome.

All-star approach to a small medical imaging dataset: combined deep, transfer, and classical machine learning approaches for the determination of radial head fractures

BACKGROUND

Radial head fractures are often evaluated in emergency departments and can easily be missed. Automated or semi-automated detection methods that help physicians may be valuable regarding the high miss rate.

PURPOSE

To evaluate the accuracy of combined deep, transfer, and classical machine learning approaches on a small dataset for determination of radial head fractures.

MATERIAL AND METHODS

A total of 48 patients with radial head fracture and 56 patients without fracture on elbow radiographs were retrospectively evaluated. The input images were obtained by cropping anteroposterior elbow radiographs around a center-point on the radial head. For fracture determination, an algorithm based on feature extraction using distinct prototypes of pretrained networks (VGG16, ResNet50, InceptionV3, MobileNetV2) representing four different approaches was developed. Reduction of feature space dimensions, feeding the most relevant features, and development of ensemble of classifiers were utilized.

RESULTS

The algorithm with the best performance consisted of preprocessing the input, computation of global maximum and global mean outputs of four distinct pretrained networks, dimensionality reduction by applying univariate and ensemble feature selectors, and applying Support Vector Machines and Random Forest classifiers to the transformed and reduced dataset. A maximum accuracy of 90% with MobileNetV2 pretrained features was reached for fracture determination with a small sample size.

CONCLUSION

Radial head fractures can be determined with a combined approach and limitations of the small sample size can be overcome by utilizing pretrained deep networks with classical machine learning methods.

RadImageNet: An Open Radiologic Deep Learning Research Dataset for Effective Transfer Learning

Purpose

To demonstrate the value of pretraining with millions of radiologic images compared with ImageNet photographic images on downstream medical applications when using transfer learning.

Materials and Methods

This retrospective study included patients who underwent a radiologic study between 2005 and 2020 at an outpatient imaging facility. Key images and associated labels from the studies were retrospectively extracted from the original study interpretation. These images were used for RadImageNet model training with random weight initiation. The RadImageNet models were compared with ImageNet models using the area under the receiver operating characteristic curve (AUC) for eight classification tasks and using Dice scores for two segmentation problems.

Results

The RadImageNet database consists of 1.35 million annotated medical images in 131 872 patients who underwent CT, MRI, and US for musculoskeletal, neurologic, oncologic, gastrointestinal, endocrine, abdominal, and pulmonary pathologic conditions. For transfer learning tasks on small datasets—thyroid nodules (US), breast masses (US), anterior cruciate ligament injuries (MRI), and meniscal tears (MRI)—the RadImageNet models demonstrated a significant advantage (P < .001) to ImageNet models (9.4%, 4.0%, 4.8%, and 4.5% AUC improvements, respectively). For larger datasets—pneumonia (chest radiography), COVID-19 (CT), SARS-CoV-2 (CT), and intracranial hemorrhage (CT)—the RadImageNet models also illustrated improved AUC (P < .001) by 1.9%, 6.1%, 1.7%, and 0.9%, respectively. Additionally, lesion localizations of the RadImageNet models were improved by 64.6% and 16.4% on thyroid and breast US datasets, respectively.

Conclusion

RadImageNet pretrained models demonstrated better interpretability compared with ImageNet models, especially for smaller radiologic datasets.

Keywords: CT, MR Imaging, US, Head/Neck, Thorax, Brain/Brain Stem, Evidence-based Medicine, Computer Applications–General (Informatics)

Supplemental material is available for this article.

Published under a CC BY 4.0 license.

See also the commentary by Cadrin-Chênevert in this issue.

Deep Segmentation Networks for Segmenting Kidneys and Detecting Kidney Stones in Unenhanced Abdominal CT Images

Recent breakthroughs of deep learning algorithms in medical imaging, automated detection, and segmentation techniques for renal (kidney) in abdominal computed tomography (CT) images have been limited. Radiomics and machine learning analyses of renal diseases rely on the automatic segmentation of kidneys in CT images. Inspired by this, our primary aim is to utilize deep semantic segmentation learning models with a proposed training scheme to achieve precise and accurate segmentation outcomes. Moreover, this work aims to provide the community with an open-source, unenhanced abdominal CT dataset for training and testing the deep learning segmentation networks to segment kidneys and detect kidney stones. Five variations of deep segmentation networks are trained and tested both dependently (based on the proposed training scheme) and independently. Upon comparison, the models trained with the proposed training scheme enable the highly accurate 2D and 3D segmentation of kidneys and kidney stones. We believe this work is a fundamental step toward AI-driven diagnostic strategies, which can be an essential component of personalized patient care and improved decision-making in treating kidney diseases.

Vision transformer and explainable transfer learning models for auto detection of kidney cyst, stone and tumor from CT-radiography

Renal failure, a public health concern, and the scarcity of nephrologists around the globe have necessitated the development of an AI-based system to auto-diagnose kidney diseases. This research deals with the three major renal diseases categories: kidney stones, cysts, and tumors, and gathered and annotated a total of 12,446 CT whole abdomen and urogram images in order to construct an AI-based kidney diseases diagnostic system and contribute to the AI community's research scope e.g., modeling digital-twin of renal functions. The collected images were exposed to exploratory data analysis, which revealed that the images from all of the classes had the same type of mean color distribution. Furthermore, six machine learning models were built, three of which are based on the state-of-the-art variants of the Vision transformers EANet, CCT, and Swin transformers, while the other three are based on well-known deep learning models Resnet, VGG16, and Inception v3, which were adjusted in the last layers. While the VGG16 and CCT models performed admirably, the swin transformer outperformed all of them in terms of accuracy, with an accuracy of 99.30 percent. The F1 score and precision and recall comparison reveal that the Swin transformer outperforms all other models and that it is the quickest to train. The study also revealed the blackbox of the VGG16, Resnet50, and Inception models, demonstrating that VGG16 is superior than Resnet50 and Inceptionv3 in terms of monitoring the necessary anatomy abnormalities. We believe that the superior accuracy of our Swin transformer-based model and the VGG16-based model can both be useful in diagnosing kidney tumors, cysts, and stones.

Ureteral calculi lithotripsy for single ureteral calculi: can DNN-assisted model help preoperatively predict risk factors for sepsis?

OBJECTIVES

To explore the utility of radiomics and deep learning model in assessing the risk factors for sepsis after flexible ureteroscopy lithotripsy (FURL) or percutaneous nephrolithotomy (PCNL) in patients with ureteral calculi.

METHODS

This retrospective analysis included 847 patients with treatment-naive proximal ureteral calculi who received FURL or PCNL. All participants were preoperatively conducted non-contrast computed tomography scans, and relevant clinical information was meanwhile collected. After propensity score matching, the radiomics model was established to predict the onset of sepsis. A deep learning model was also adapted to further improve the prediction accuracy. Performance of these trained models was verified in another independent external validation set including 40 cases of ureteral calculi patients.

RESULTS

The overall incidence of sepsis after FURL or PCNL was 5.9%. The least absolute shrinkage and selection operator (LASSO) regression analysis revealed 26 predictive variables, with an overall AUC of 0.881 (95% CI, 0.813-0.931) and an AUC of 0.783 (95% CI, 0.766-0.801) in external validation cohort. Judicious adaption of a deep neural network (DNN) model to our dataset improved the AUC to 0.920 (95% CI, 0.906-0.933) in the internal validation. To eliminate the overfitting, external validation was carried out for DNN model (AUC = 0.874 (95% CI, 0.858-0.891)).

CONCLUSIONS

The DNN was more effective than the LASSO model in revealing risk factors for sepsis after FURL or PCNL in single ureteral calculi patients, and females are more susceptible to sepsis than males. Deep learning models have the potential to act as gatekeepers to facilitate patient stratification.

KEY POINTS

• Both the least absolute shrinkage and selection operator (LASSO) and deep neural network (DNN) models were shown to be effective in sepsis prediction. • The DNN model achieved superior prediction capability, with an AUC of 0.920 (95% CI, 0.906-0.933). • DNN-assisted model has potential to serve as a gatekeeper to facilitate patient stratification.

A deep learning system for automated kidney stone detection and volumetric segmentation on noncontrast CT scans

PURPOSE

Early detection and size quantification of renal calculi are important for optimizing treatment and preventing severe kidney stone disease. Prior work has shown that volumetric measurements of kidney stones are more informative and reproducible than linear measurements. Deep learning-based systems that use abdominal noncontrast computed tomography (CT) scans may assist in detection and reduce workload by removing the need for manual stone volume measurement. Prior to this work, no such system had been developed for use on noisy low-dose CT or tested on a large-scale external dataset.

METHODS

We used a dataset of 91 CT colonography (CTC) scans with manually marked kidney stones combined with 89 CTC scans without kidney stones. To compare with a prior work half the data was used for training and half for testing. A set of CTC scans from 6185 patients from a separate institution with patient-level labels were used as an external validation set. A 3D U-Net model was employed to segment the kidneys, followed by gradient-based anisotropic denoising, thresholding, and region growing. A 13 layer convolutional neural network classifier was then applied to distinguish kidney stones from false positive regions.

RESULTS

The system achieved a sensitivity of 0.86 at 0.5 false positives per scan on a challenging test set of low-dose CT with many small stones, an improvement over an earlier work that obtained a sensitivity of 0.52. The stone volume measurements correlated well with manual measurements (r 2 = 0.95 $r^2 = 0.95$ ). For patient-level classification, the system achieved an area under the receiver-operating characteristic of 0.95 on an external validation set (sensitivity = 0.88, specificity = 0.91 at the Youden point). A common cause of false positives were small atherosclerotic plaques in the renal sinus that simulated kidney stones.

CONCLUSIONS

Our deep-learning-based system showed improvements over a previously developed system that did not use deep learning, with even higher performance on an external validation set.

Artificial intelligence in endourology: emerging technology for individualized care

PURPOSE OF REVIEW

Artificial intelligence in medicine has allowed for efficient processing of large datasets to perform cognitive tasks that facilitate clinical decision-making, and it is an emerging area of research. This review aims to highlight the most pertinent and recent research in artificial intelligence in endourology, where it has been used to optimize stone diagnosis, support decision-making regarding management, predict stone recurrence, and provide new tools for bioinformatics research within endourology.

RECENT FINDINGS

Artificial neural networks (ANN) and machine learning approaches have demonstrated high accuracy in predicting stone diagnoses, stone composition, and outcomes of spontaneous stone passage, shockwave lithotripsy (SWL), or percutaneous nephrolithotomy (PCNL); some of these models outperform more traditional predictive models and existing nomograms. In addition, these approaches have been used to predict stone recurrence, quality of life scores, and provide novel methods of mining the electronic medical record for research.

SUMMARY

Artificial intelligence can be used to enhance existing approaches to stone diagnosis, management, and prevention to provide a more individualized approach to endourologic care. Moreover, it may support an emerging area of bioinformatics research within endourology. However, despite high accuracy, many of the published algorithms lack external validity and require further study before they are more widely adopted.

Clinical Artificial Intelligence Applications in Radiology: Chest and Abdomen

Organ segmentation, chest radiograph classification, and lung and liver nodule detections are some of the popular artificial intelligence (AI) tasks in chest and abdominal radiology due to the wide availability of public datasets. AI algorithms have achieved performance comparable to humans in less time for several organ segmentation tasks, and some lesion detection and classification tasks. This article introduces the current published articles of AI applied to chest and abdominal radiology, including organ segmentation, lesion detection, classification, and predicting prognosis.

The Ascent of Artificial Intelligence in Endourology: a Systematic Review Over the Last 2 Decades

Purpose of Review

To highlight and review the application of artificial intelligence (AI) in kidney stone disease (KSD) for diagnostics, predicting procedural outcomes, stone passage, and recurrence rates. The systematic review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) checklist.

Recent Findings

This review discusses the newer advancements in AI-driven management strategies, which holds great promise to provide an essential step for personalized patient care and improved decision making. AI has been used in all areas of KSD including diagnosis, for predicting treatment suitability and success, basic science, quality of life (QOL), and recurrence of stone disease. However, it is still a research-based tool and is not used universally in clinical practice. This could be due to a lack of data infrastructure needed to train the algorithms, wider applicability in all groups of patients, complexity of its use and cost involved with it.

Summary

The constantly evolving literature and future research should focus more on QOL and the cost of KSD treatment and develop evidence-based AI algorithms that can be used universally, to guide urologists in the management of stone disease.

Artificial intelligence in stone disease

PURPOSE OF REVIEW

Artificial intelligence (AI) is the ability of a machine, or computer, to simulate intelligent behavior. In medicine, the use of large datasets enables a computer to learn how to perform cognitive tasks, thereby facilitating medical decision-making. This review aims to describe advancements in AI in stone disease to improve diagnostic accuracy in determining stone composition, to predict outcomes of surgical procedures or watchful waiting and ultimately to optimize treatment choices for patients.

RECENT FINDINGS

AI algorithms show high accuracy in different realms including stone detection and in the prediction of surgical outcomes. There are machine learning algorithms for outcomes after percutaneous nephrolithotomy, extracorporeal shockwave lithotripsy, and for ureteral stone passage. Some of these algorithms show better predictive capabilities compared to existing scoring systems and nomograms.

SUMMARY

The use of AI can facilitate the development of diagnostic and treatment algorithms in patients with stone disease. Although the generalizability and external validity of these algorithms remain uncertain, the development of highly accurate AI-based tools may enable the urologist to provide more customized patient care and superior outcomes.

Differentiation of distal ureteral stones and pelvic phleboliths using a convolutional neural network

The objectives were to develop and validate a Convolutional Neural Network (CNN) using local features for differentiating distal ureteral stones from pelvic phleboliths, compare the CNN method with a semi-quantitative method and with radiologists' assessments and to evaluate whether the assessment of a calcification and its local surroundings is sufficient for discriminating ureteral stones from pelvic phleboliths in non-contrast-enhanced CT (NECT). We retrospectively included 341 consecutive patients with acute renal colic and a ureteral stone on NECT showing either a distal ureteral stone, a phlebolith or both. A 2.5-dimensional CNN (2.5D-CNN) model was used, where perpendicular axial, coronal and sagittal images through each calcification were used as input data for the CNN. The CNN was trained on 384 calcifications, and evaluated on an unseen dataset of 50 stones and 50 phleboliths. The CNN was compared to the assessment by seven radiologists who reviewed a local 5 × 5 × 5 cm image stack surrounding each calcification, and to a semi-quantitative method using cut-off values based on the attenuation and volume of the calcifications. The CNN differentiated stones and phleboliths with a sensitivity, specificity and accuracy of 94%, 90% and 92% and an AUC of 0.95. This was similar to a majority vote accuracy of 93% and significantly higher (p = 0.03) than the mean radiologist accuracy of 86%. The semi-quantitative method accuracy was 49%. In conclusion, the CNN differentiated ureteral stones from phleboliths with higher accuracy than the mean of seven radiologists' assessments using local features. However, more than local features are needed to reach optimal discrimination.

Multi-Reader-Multi-Split Annotation of Emphysema in Computed Tomography

Emphysema is visible on computed tomography (CT) as low-density lesions representing the destruction of the pulmonary alveoli. To train a machine learning model on the emphysema extent in CT images, labeled image data is needed. The provision of these labels requires trained readers, who are a limited resource. The purpose of the study was to test the reading time, inter-observer reliability and validity of the multi-reader-multi-split method for acquiring CT image labels from radiologists. The approximately 500 slices of each stack of lung CT images were split into 1-cm chunks, with 17 thin axial slices per chunk. The chunks were randomly distributed to 26 readers, radiologists and radiology residents. Each chunk was given a quick score concerning emphysema type and severity in the left and right lung separately. A cohort of 102 subjects, with varying degrees of visible emphysema in the lung CT images, was selected from the SCAPIS pilot, performed in 2012 in Gothenburg, Sweden. In total, the readers created 9050 labels for 2881 chunks. Image labels were compared with regional annotations already provided at the SCAPIS pilot inclusion. The median reading time per chunk was 15 s. The inter-observer Krippendorff's alpha was 0.40 and 0.53 for emphysema type and score, respectively, and higher in the apical part than in the basal part of the lungs. The multi-split emphysema scores were generally consistent with regional annotations. In conclusion, the multi-reader-multi-split method provided reasonably valid image labels, with an estimation of the inter-observer reliability.

Preparing Medical Imaging Data for Machine Learning

Artificial intelligence (AI) continues to garner substantial interest in medical imaging. The potential applications are vast and include the entirety of the medical imaging life cycle from image creation to diagnosis to outcome prediction. The chief obstacles to development and clinical implementation of AI algorithms include availability of sufficiently large, curated, and representative training data that includes expert labeling (eg, annotations). Current supervised AI methods require a curation process for data to optimally train, validate, and test algorithms. Currently, most research groups and industry have limited data access based on small sample sizes from small geographic areas. In addition, the preparation of data is a costly and time-intensive process, the results of which are algorithms with limited utility and poor generalization. In this article, the authors describe fundamental steps for preparing medical imaging data in AI algorithm development, explain current limitations to data curation, and explore new approaches to address the problem of data availability.