A Dynamic Graph Cuts Method with Integrated Multiple Feature Maps for Segmenting Kidneys in 2D Ultrasound Images

A novel cross-modal data augmentation method based on contrastive unpaired translation network for kidney segmentation in ultrasound imaging

BACKGROUND

Kidney ultrasound (US) image segmentation is one of the key steps in computer-aided diagnosis and treatment planning of kidney diseases. Recently, deep learning (DL) technology has demonstrated promising prospects in automatic kidney US segmentation. However, due to the poor quality, particularly the weak boundaries in kidney US imaging, obtaining accurate annotations for DL-based segmentation methods remain a challenging and time-consuming task. This issue can hinder the application of data-hungry deep learning methods.

PURPOSE

In this paper, we explore a novel cross-modal data augmentation method aimed at enhancing the performance of DL-based segmentation networks on the limited labeled kidney US dataset.

METHODS

In particular, we adopt a novel method based on contrastive unpaired translation network (CUT) to obtain simulated labeled kidney US images at a low cost from labeled abdomen computed tomography (CT) data and unlabeled kidney US images. To effectively improve the segmentation network performance, we propose an instance-weighting training strategy that simultaneously captures useful information from both the simulated and real labeled kidney US images. We trained our generative networks on a dataset comprising 4418 labeled CT slices and 4594 unlabeled US images. For segmentation network, we used a dataset consisting of 4594 simulated and 100 real kidney US images for training, 20 images for validation, and 169 real images for testing. We compared the performance of our method to several state-of-the-art approaches using the Wilcoxon signed-rank test, and applied the Bonferroni method for multiple comparison correction.

RESULTS

The experimental results show that we can synthesize accurate labeled kidney US images with a Fréchet inception distance of 52.52. Moreover, the proposed method achieves a segmentation accuracy of 0.9360 ± 0.0398 for U-Net on normal kidney US images, and 0.7719 ± 0.2449 on the abnormal dataset, as measured by the dice similarity coefficient. When compared to other training strategies, the proposed method demonstrated statistically significant superiority, with all p-values being less than 0.01.

CONCLUSIONS

The proposed method can effectively improve the accuracy and generalization ability of kidney US image segmentation models with limited annotated training data.

Cross-modal Transfer Learning Based on an Improved CycleGAN Model for Accurate Kidney Segmentation in Ultrasound Images

OBJECTIVE

Deep-learning algorithms have been widely applied in the field of automatic kidney ultrasound (US) image segmentation. However, obtaining a large number of accurate kidney labels clinically is very difficult and time-consuming. To solve this problem, we have proposed an efficient cross-modal transfer learning method to improve the performance of the segmentation network on a limited labeled kidney US dataset.

METHODS

We aim to implement an improved image-to-image translation network called Seg-CycleGAN to generate accurate annotated kidney US data from labeled abdomen computed tomography images. The Seg-CycleGAN framework primarily consists of two structures: (i) a standard CycleGAN network to visually simulate kidney US from a publicly available labeled abdomen computed tomography dataset; (ii) and a segmentation network to ensure accurate kidney anatomical structures in US images. Based on the large number of simulated kidney US images and small number of real annotated kidney US images, we then aimed to employ a fine-tuning strategy to obtain better segmentation results.

RESULTS

To validate the effectiveness of the proposed method, we tested this method on both normal and abnormal kidney US images. The experimental results showed that the proposed method achieved a segmentation accuracy of 0.8548 in dice similarity coefficient on all testing datasets and 0.7622 on the abnormal testing dataset.

CONCLUSIONS

Compared with existing data augmentation and transfer learning methods, the proposed method improved the accuracy and generalization of the kidney US image segmentation network on a limited number of training datasets. It therefore has the potential to significantly reduce annotation costs in clinical settings.

URI-CADS: A Fully Automated Computer-Aided Diagnosis System for Ultrasound Renal Imaging

Ultrasound is a widespread imaging modality, with special application in medical fields such as nephrology. However, automated approaches for ultrasound renal interpretation still pose some challenges: (1) the need for manual supervision by experts at various stages of the system, which prevents its adoption in primary healthcare, and (2) their limited considered taxonomy (e.g., reduced number of pathologies), which makes them unsuitable for training practitioners and providing support to experts. This paper proposes a fully automated computer-aided diagnosis system for ultrasound renal imaging addressing both of these challenges. Our system is based in a multi-task architecture, which is implemented by a three-branched convolutional neural network and is capable of segmenting the kidney and detecting global and local pathologies with no need of human interaction during diagnosis. The integration of different image perspectives at distinct granularities enhanced the proposed diagnosis. We employ a large (1985 images) and demanding ultrasound renal imaging database, publicly released with the system and annotated on the basis of an exhaustive taxonomy of two global and nine local pathologies (including cysts, lithiasis, hydronephrosis, angiomyolipoma), establishing a benchmark for ultrasound renal interpretation. Experiments show that our proposed method outperforms several state-of-the-art methods in both segmentation and diagnosis tasks and leverages the combination of global and local image information to improve the diagnosis. Our results, with a 87.41% of AUC in healthy-pathological diagnosis and 81.90% in multi-pathological diagnosis, support the use of our system as a helpful tool in the healthcare system.

Fully automated kidney image biomarker prediction in ultrasound scans using Fast-Unet+

Any kidney dimension and volume variation can be a remarkable indicator of kidney disorders. Precise kidney segmentation in standard planes plays an undeniable role in predicting kidney size and volume. On the other hand, ultrasound is the modality of choice in diagnostic procedures. This paper proposes a convolutional neural network with nested layers, namely Fast-Unet++, promoting the Fast and accurate Unet model. First, the model was trained and evaluated for segmenting sagittal and axial images of the kidney. Then, the predicted masks were used to estimate the kidney image biomarkers, including its volume and dimensions (length, width, thickness, and parenchymal thickness). Finally, the proposed model was tested on a publicly available dataset with various shapes and compared with the related networks. Moreover, the network was evaluated using a set of patients who had undergone ultrasound and computed tomography. The dice metric, Jaccard coefficient, and mean absolute distance were used to evaluate the segmentation step. 0.97, 0.94, and 3.23 mm for the sagittal frame, and 0.95, 0.9, and 3.87 mm for the axial frame were achieved. The kidney dimensions and volume were evaluated using accuracy, the area under the curve, sensitivity, specificity, precision, and F1.

Novel Solution for Using Neural Networks for Kidney Boundary Extraction in 2D Ultrasound Data

Background and Objective: Kidney ultrasound (US) imaging is a significant imaging modality for evaluating kidney health and is essential for diagnosis, treatment, surgical intervention planning, and follow-up assessments. Kidney US image segmentation consists of extracting useful objects or regions from the total image, which helps determine tissue organization and improve diagnosis. Thus, obtaining accurate kidney segmentation data is an important first step for precisely diagnosing kidney diseases. However, manual delineation of the kidney in US images is complex and tedious in clinical practice. To overcome these challenges, we developed a novel automatic method for US kidney segmentation. Methods: Our method comprises two cascaded steps for US kidney segmentation. The first step utilizes a coarse segmentation procedure based on a deep fusion learning network to roughly segment each input US kidney image. The second step utilizes a refinement procedure to fine-tune the result of the first step by combining an automatic searching polygon tracking method with a machine learning network. In the machine learning network, a suitable and explainable mathematical formula for kidney contours is denoted by basic parameters. Results: Our method is assessed using 1380 trans-abdominal US kidney images obtained from 115 patients. Based on comprehensive comparisons of different noise levels, our method achieves accurate and robust results for kidney segmentation. We use ablation experiments to assess the significance of each component of the method. Compared with state-of-the-art methods, the evaluation metrics of our method are significantly higher. The Dice similarity coefficient (DSC) of our method is 94.6 ± 3.4%, which is higher than those of recent deep learning and hybrid algorithms (89.4 ± 7.1% and 93.7 ± 3.8%, respectively). Conclusions: We develop a coarse-to-refined architecture for the accurate segmentation of US kidney images. It is important to precisely extract kidney contour features because segmentation errors can cause under-dosing of the target or over-dosing of neighboring normal tissues during US-guided brachytherapy. Hence, our method can be used to increase the rigor of kidney US segmentation.

Current progress in artificial intelligence-assisted medical image analysis for chronic kidney disease: A literature review

Chronic kidney disease (CKD) causes irreversible damage to kidney structure and function. Arising from various etiologies, risk factors for CKD include hypertension and diabetes. With a progressively increasing global prevalence, CKD is an important public health problem worldwide. Medical imaging has become an important diagnostic tool for CKD through the non-invasive identification of macroscopic renal structural abnormalities. Artificial intelligence (AI)-assisted medical imaging techniques aid clinicians in the analysis of characteristics that cannot be easily discriminated by the naked eye, providing valuable information for the identification and management of CKD. Recent studies have demonstrated the effectiveness of AI-assisted medical image analysis as a clinical support tool using radiomics- and deep learning-based AI algorithms for improving the early detection, pathological assessment, and prognostic evaluation of various forms of CKD, including autosomal dominant polycystic kidney disease. Herein, we provide an overview of the potential roles of AI-assisted medical image analysis for the diagnosis and management of CKD.

Classifying Kidney Disease in a Vervet Model Using Spatially Encoded Contrast-Enhanced Ultrasound Perfusion Parameters

Early stages of diabetic kidney disease (DKD) are difficult to diagnose in patients with type 2 diabetes. This work was aimed at identifying contrast-enhanced ultrasound (CEUS) perfusion parameters, a microcirculatory biomarker indicative of early DKD progression. CEUS kidney flash-replenishment data were acquired in control, insulin resistant and diabetic vervet monkeys (N = 16). By use of a mono-exponential model, time-intensity curve parameters related to blood volume (A), velocity (β) and flow rate (perfusion index [PI]) were extracted from 10 concentric kidney layers to study spatial perfusion patterns that could serve as strong indicators of disease. Mean squared error (MSE) was used to assess model performance. Features calculated from the perfusion parameters were inputs for the linear regression models to determine which features could distinguish between cohorts. The mono-exponential model performed well, with average MSEs (±standard deviation) of 0.0254 (±0.0210), 0.0321 (±0.0242) and 0.0287 (±0.0130) for the control, insulin resistant and diabetic cohorts, respectively. Perfusion index features, with blood pressure, were the best classifiers between cohorts (p < 0.05). CEUS has the potential to detect early microvascular changes, providing insight into disease-related structural changes in the kidney. The sensitivity of this technique should be explored further by assessing various stages of DKD.

A novel convolutional neural network for kidney ultrasound images segmentation

BACKGROUND AND OBJECTIVE

Ultrasound imaging has been widely used in the screening of kidney diseases. The localization and segmentation of the kidneys in ultrasound images are helpful for the clinical diagnosis of diseases. However, it is a challenging task to segment the kidney accurately from ultrasound images due to the interference of various factors.

METHODS

In this paper, a novel multi-scale and deep-supervised CNN architecture is proposed to segment the kidney. The architecture consists of an encoder, a pyramid pooling module and a decoder. In the encoder, we design a multi-scale input pyramid with parallel branches to capture features at different scales. In the decoder, a multi-output supervision module is developed. The introduction of the multi-output supervision module enables the network to learn to predict more precise segmentation results scale-by-scale. In addition, we construct a kidney ultrasound dataset, which contains of 400 images and 400 labels.

RESULTS

To highlight effectiveness of the proposed approach, we use six quantitative indicators to compare with several state-of-the-art methods on the same kidney ultrasound dataset. The results of our method on the six indicators of accuracy, dice, jaccard, precision, recall and ASSD are 98.86%, 95.86%, 92.18%, 96.38%, 95.47% and 0.3510, respectively.

CONCLUSIONS

The analysis of evaluation indicators and segmentation results shows that our method achieves the best performance in kidney ultrasound image segmentation.

Automated measurement of total kidney volume from 3D ultrasound images of patients affected by polycystic kidney disease and comparison to MR measurements

PURPOSE

Total kidney volume (TKV) is the most important imaging biomarker for quantifying the severity of autosomal-dominant polycystic kidney disease (ADPKD). 3D ultrasound (US) can accurately measure kidney volume compared to 2D US; however, manual segmentation is tedious and requires expert annotators. We investigated a deep learning-based approach for automated segmentation of TKV from 3D US in ADPKD patients.

METHOD

We used axially acquired 3D US-kidney images in 22 ADPKD patients where each patient and each kidney were scanned three times, resulting in 132 scans that were manually segmented. We trained a convolutional neural network to segment the whole kidney and measure TKV. All patients were subsequently imaged with MRI for measurement comparison.

RESULTS

Our method automatically segmented polycystic kidneys in 3D US images obtaining an average Dice coefficient of 0.80 on the test dataset. The kidney volume measurement compared with linear regression coefficient and bias from human tracing were R² = 0.81, and - 4.42%, and between AI and reference standard were R² = 0.93, and - 4.12%, respectively. MRI and US measured kidney volumes had R² = 0.84 and a bias of 7.47%.

CONCLUSION

This is the first study applying deep learning to 3D US in ADPKD. Our method shows promising performance for auto-segmentation of kidneys using 3D US to measure TKV, close to human tracing and MRI measurement. This imaging and analysis method may be useful in a number of settings, including pediatric imaging, clinical studies, and longitudinal tracking of patient disease progression.

MBANet: Multi-branch aware network for kidney ultrasound images segmentation

Due to the influence of kidney morphology, heterogeneous structure and image quality, segmenting kidney in ultrasound images is challenging. To alleviate this challenge, we proposed a novel deep neural network architecture, namely Multi-branch Aware Network (MBANet), to segment kidney accurately and robustly. MBANet mainly consists of multi-scale feature pyramid (MSFP), multi-branch encoders (MBE) and master decoder. The design of MSFP can make the network more accessible to different kinds of class details at different scales. The information exchange between MBE can reduce the loss of feature information and improve the segmentation accuracy of the network. In addition, we designed a multi-scale fusion block (MFBlock) in the MBE to further extract and fuse more refined multi-scale image information. In order to further improve the robustness of MBANet, this paper also designed a step-by-step training mechanism. We validated the proposed approach and compared to several state-of-the-art approaches on the same kidney ultrasound datasets using six quantitative metrics. The results of our method on the six indicators of pixel accuracy (PA), intersection over union (IoU), precision, recall, specificity and F1-score (F1) are 98.83%, 92.38%, 97.10%, 95.03%, 99.46% and 0.9601, respectively. Compared with the comparison method, the average values on the six indicators are improved by about 2%. The evaluation results and segmentation results demonstrate that the proposed approach achieves the best overall performance on kidney ultrasound images segmentation.

Advanced Kidney Volume Measurement Method Using Ultrasonography with Artificial Intelligence-Based Hybrid Learning in Children

In this study, we aimed to develop a new automated method for kidney volume measurement in children using ultrasonography (US) with image pre-processing and hybrid learning and to formulate an equation to calculate the expected kidney volume. The volumes of 282 kidneys (141 subjects, <19 years old) with normal function and structure were measured using US. The volumes of 58 kidneys in 29 subjects who underwent US and computed tomography (CT) were determined by image segmentation and compared to those calculated by the conventional ellipsoidal method and CT using intraclass correlation coefficients (ICCs). An expected kidney volume equation was developed using multivariate regression analysis. Manual image segmentation was automated using hybrid learning to calculate the kidney volume. The ICCs for volume determined by image segmentation and ellipsoidal method were significantly different, while that for volume calculated by hybrid learning was significantly higher than that for ellipsoidal method. Volume determined by image segmentation was significantly correlated with weight, body surface area, and height. Expected kidney volume was calculated as (2.22 × weight (kg) + 0.252 × height (cm) + 5.138). This method will be valuable in establishing an age-matched normal kidney growth chart through the accumulation and analysis of large-scale data.

Multi-Parametric Fusion of 3D Power Doppler Ultrasound for Fetal Kidney Segmentation Using Fully Convolutional Neural Networks

Kidney development is key to the long-term health of the fetus. Renal volume and vascularity assessed by 3D ultrasound (3D-US) are known markers of wellbeing, however, a lack of real-time image segmentation solutions preclude these measures being used in a busy clinical environment. In this work, we aimed to automate kidney segmentation using fully convolutional neural networks (fCNNs). We used multi-parametric input fusion incorporating 3D B-Mode and power Doppler (PD) volumes, aiming to improve segmentation accuracy. Three different fusion strategies and their performance were assessed versus a single input (B-Mode) network. Early input-level fusion provided the best segmentation accuracy with an average Dice similarity coefficient (DSC) of 0.81 and Hausdorff distance (HD) of 8.96 mm, an improvement of 0.06 DSC and reduction of 1.43 mm HD compared to our baseline network. Compared to manual segmentation for all models, repeatability was assessed by intra-class correlation coefficients (ICC) indicating good to excellent reproducibility (ICC 0.93). The framework was extended to support multiple graphics processing units (GPUs) to better handle volumetric data, dense fCNN models, batch normalization and complex fusion networks. This work and available source code provides a framework to increase the parameter space of encoder-decoder style fCNNs across multiple GPUs and shows that application of multi-parametric 3D-US in fCNN training improves segmentation accuracy.

Exploration of a Framework for the Identification of Chronic Kidney Disease Based on 2D Ultrasound Images: A Survey

BACKGROUND

Chronic kidney disease (CKD) is a fatal disease that ultimately results in kidney failure. The primary threat is the aetiology of CKD. Over the years, researchers have proposed various techniques and methods to detect and diagnose the disease. The conventional method of detecting CKD is the determination of the estimated glomerular filtration rate by measuring creatinine levels in blood or urine. Conventional methods for the detection and classification of CKD are tedious; therefore, several researchers have suggested various alternative methods. Recently, the research community has shown keen interest in developing methods for the early detection of this disease using imaging modalities such as ultrasound, magnetic resonance imaging, and computed tomography.

DISCUSSION

The study aimed to conduct a systematic review of various existing techniques for the detection and classification of different stages of CKD using 2D ultrasound imaging of the kidney. The review was confined to 2D ultrasound images alone, considering the feasibility of implementation even in underdeveloped countries because 2D ultrasound scans are more cost effective than other modalities. The techniques and experimentation in each work were thoroughly studied and discussed in this review.

CONCLUSION

This review displayed the cutting-age research, challenges, and possibilities of further research and development in the detection and classification of CKD.

Automatic kidney segmentation in ultrasound images using subsequent boundary distance regression and pixelwise classification networks

It remains challenging to automatically segment kidneys in clinical ultrasound (US) images due to the kidneys' varied shapes and image intensity distributions, although semi-automatic methods have achieved promising performance. In this study, we propose subsequent boundary distance regression and pixel classification networks to segment the kidneys automatically. Particularly, we first use deep neural networks pre-trained for classification of natural images to extract high-level image features from US images. These features are used as input to learn kidney boundary distance maps using a boundary distance regression network and the predicted boundary distance maps are classified as kidney pixels or non-kidney pixels using a pixelwise classification network in an end-to-end learning fashion. We also adopted a data-augmentation method based on kidney shape registration to generate enriched training data from a small number of US images with manually segmented kidney labels. Experimental results have demonstrated that our method could automatically segment the kidney with promising performance, significantly better than deep learning-based pixel classification networks.

FULLY-AUTOMATIC SEGMENTATION OF KIDNEYS IN CLINICAL ULTRASOUND IMAGES USING A BOUNDARY DISTANCE REGRESSION NETWORK

It remains challenging to automatically segment kidneys in clinical ultrasound images due to the kidneys' varied shapes and image intensity distributions, although semi-automatic methods have achieved promising performance. In this study, we developed a novel boundary distance regression deep neural network to segment the kidneys, informed by the fact that the kidney boundaries are relatively consistent across images in terms of their appearance. Particularly, we first use deep neural networks pre-trained for classification of natural images to extract high-level image features from ultrasound images, then these feature maps are used as input to learn kidney boundary distance maps using a boundary distance regression network, and finally the predicted boundary distance maps are classified as kidney pixels or non-kidney pixels using a pixel classification network in an end-to-end learning fashion. Experimental results have demonstrated that our method could effectively improve the performance of automatic kidney segmentation, significantly better than deep learning based pixel classification networks.

Computer-aided diagnosis of congenital abnormalities of the kidney and urinary tract in children based on ultrasound imaging data by integrating texture image features and deep transfer learning image features

INTRODUCTION

Anatomic characteristics of kidneys derived from ultrasound images are potential biomarkers of children with congenital abnormalities of the kidney and urinary tract (CAKUT), but current methods are limited by the lack of automated processes that accurately classify diseased and normal kidneys.

OBJECTIVE

The objective of the study was to evaluate the diagnostic performance of deep transfer learning techniques to classify kidneys of normal children and those with CAKUT.

STUDY DESIGN

A transfer learning method was developed to extract features of kidneys from ultrasound images obtained during routine clinical care of 50 children with CAKUT and 50 controls. To classify diseased and normal kidneys, support vector machine classifiers were built on the extracted features using (1) transfer learning imaging features from a pretrained deep learning model, (2) conventional imaging features, and (3) their combination. These classifiers were compared, and their diagnosis performance was measured using area under the receiver operating characteristic curve (AUC), accuracy, specificity, and sensitivity.

RESULTS

The AUC for classifiers built on the combination features were 0.92, 0.88, and 0.92 for discriminating the left, right, and bilateral abnormal kidney scans from controls with classification rates of 84%, 81%, and 87%; specificity of 84%, 74%, and 88%; and sensitivity of 85%, 88%, and 86%, respectively. These classifiers performed better than classifiers built on either the transfer learning features or the conventional features alone (p < 0.001).

DISCUSSION

The present study validated transfer learning techniques for imaging feature extraction of ultrasound images to build classifiers for distinguishing children with CAKUT from controls. The experiments have demonstrated that the classifiers built on the transfer learning features and conventional image features could distinguish abnormal kidney images from controls with AUCs greater than 0.88, indicating that classification of ultrasound kidney scans has a great potential to aid kidney disease diagnosis. A limitation of the present study is the moderate number of patients that contributed data to the transfer learning approach.

CONCLUSIONS

The combination of transfer learning and conventional imaging features yielded the best classification performance for distinguishing children with CAKUT from controls based on ultrasound images of kidneys.

TRANSFER LEARNING FOR DIAGNOSIS OF CONGENITAL ABNORMALITIES OF THE KIDNEY AND URINARY TRACT IN CHILDREN BASED ON ULTRASOUND IMAGING DATA

Classification of ultrasound (US) kidney images for diagnosis of congenital abnormalities of the kidney and urinary tract (CAKUT) in children is a challenging task. It is desirable to improve existing pattern classification models that are built upon conventional image features. In this study, we propose a transfer learning-based method to extract imaging features from US kidney images in order to improve the CAKUT diagnosis in children. Particularly, a pre-trained deep learning model (imagenet-caffe-alex) is adopted for transfer learning-based feature extraction from 3-channel feature maps computed from US images, including original images, gradient features, and distanced transform features. Support vector machine classifiers are then built upon different sets of features, including the transfer learning features, conventional imaging features, and their combination. Experimental results have demonstrated that the combination of transfer learning features and conventional imaging features yielded the best classification performance for distinguishing CAKUT patients from normal controls based on their US kidney images.