Dense biased networks with deep priori anatomy and hard region adaptation: Semi-supervised learning for fine renal artery segmentation

Deep learning system for malignancy risk prediction in cystic renal lesions: a multicenter study

OBJECTIVES

To develop an interactive, non-invasive artificial intelligence (AI) system for malignancy risk prediction in cystic renal lesions (CRLs).

METHODS

In this retrospective, multicenter diagnostic study, we evaluated 715 patients. An interactive geodesic-based 3D segmentation model was created for CRLs segmentation. A CRLs classification model was developed using spatial encoder temporal decoder (SETD) architecture. The classification model combines a 3D-ResNet50 network for extracting spatial features and a gated recurrent unit (GRU) network for decoding temporal features from multi-phase CT images. We assessed the segmentation model using sensitivity (SEN), specificity (SPE), intersection over union (IOU), and dice similarity (Dice) metrics. The classification model's performance was evaluated using the area under the receiver operator characteristic curve (AUC), accuracy score (ACC), and decision curve analysis (DCA).

RESULTS

From 2012 to 2023, we included 477 CRLs (median age, 57 [IQR: 48-65]; 173 men) in the training cohort, 226 CRLs (median age, 60 [IQR: 52-69]; 77 men) in the validation cohort, and 239 CRLs (median age, 59 [IQR: 53-69]; 95 men) in the testing cohort (external validation cohort 1, cohort 2, and cohort 3). The segmentation model and SETD classifier exhibited excellent performance in both validation (AUC = 0.973, ACC = 0.916, Dice = 0.847, IOU = 0.743, SEN = 0.840, SPE = 1.000) and testing datasets (AUC = 0.998, ACC = 0.988, Dice = 0.861, IOU = 0.762, SEN = 0.876, SPE = 1.000).

CONCLUSION

The AI system demonstrated excellent benign-malignant discriminatory ability across both validation and testing datasets and illustrated improved clinical decision-making utility.

CRITICAL RELEVANCE STATEMENT

In this era when incidental CRLs are prevalent, this interactive, non-invasive AI system will facilitate accurate diagnosis of CRLs, reducing excessive follow-up and overtreatment.

KEY POINTS

The rising prevalence of CRLs necessitates better malignancy prediction strategies. The AI system demonstrated excellent diagnostic performance in identifying malignant CRL. The AI system illustrated improved clinical decision-making utility.

Imaging-based deep learning in kidney diseases: recent progress and future prospects

Kidney diseases result from various causes, which can generally be divided into neoplastic and non-neoplastic diseases. Deep learning based on medical imaging is an established methodology for further data mining and an evolving field of expertise, which provides the possibility for precise management of kidney diseases. Recently, imaging-based deep learning has been widely applied to many clinical scenarios of kidney diseases including organ segmentation, lesion detection, differential diagnosis, surgical planning, and prognosis prediction, which can provide support for disease diagnosis and management. In this review, we will introduce the basic methodology of imaging-based deep learning and its recent clinical applications in neoplastic and non-neoplastic kidney diseases. Additionally, we further discuss its current challenges and future prospects and conclude that achieving data balance, addressing heterogeneity, and managing data size remain challenges for imaging-based deep learning. Meanwhile, the interpretability of algorithms, ethical risks, and barriers of bias assessment are also issues that require consideration in future development. We hope to provide urologists, nephrologists, and radiologists with clear ideas about imaging-based deep learning and reveal its great potential in clinical practice.Critical relevance statement The wide clinical applications of imaging-based deep learning in kidney diseases can help doctors to diagnose, treat, and manage patients with neoplastic or non-neoplastic renal diseases.Key points• Imaging-based deep learning is widely applied to neoplastic and non-neoplastic renal diseases.• Imaging-based deep learning improves the accuracy of the delineation, diagnosis, and evaluation of kidney diseases.• The small dataset, various lesion sizes, and so on are still challenges for deep learning.

Learning with limited annotations: A survey on deep semi-supervised learning for medical image segmentation

Medical image segmentation is a fundamental and critical step in many image-guided clinical approaches. Recent success of deep learning-based segmentation methods usually relies on a large amount of labeled data, which is particularly difficult and costly to obtain, especially in the medical imaging domain where only experts can provide reliable and accurate annotations. Semi-supervised learning has emerged as an appealing strategy and been widely applied to medical image segmentation tasks to train deep models with limited annotations. In this paper, we present a comprehensive review of recently proposed semi-supervised learning methods for medical image segmentation and summarize both the technical novelties and empirical results. Furthermore, we analyze and discuss the limitations and several unsolved problems of existing approaches. We hope this review can inspire the research community to explore solutions to this challenge and further advance the field of medical image segmentation.

Segment anything model for medical images?

The Segment Anything Model (SAM) is the first foundation model for general image segmentation. It has achieved impressive results on various natural image segmentation tasks. However, medical image segmentation (MIS) is more challenging because of the complex modalities, fine anatomical structures, uncertain and complex object boundaries, and wide-range object scales. To fully validate SAM's performance on medical data, we collected and sorted 53 open-source datasets and built a large medical segmentation dataset with 18 modalities, 84 objects, 125 object-modality paired targets, 1050K 2D images, and 6033K masks. We comprehensively analyzed different models and strategies on the so-called COSMOS 1050K dataset. Our findings mainly include the following: (1) SAM showed remarkable performance in some specific objects but was unstable, imperfect, or even totally failed in other situations. (2) SAM with the large ViT-H showed better overall performance than that with the small ViT-B. (3) SAM performed better with manual hints, especially box, than the Everything mode. (4) SAM could help human annotation with high labeling quality and less time. (5) SAM was sensitive to the randomness in the center point and tight box prompts, and may suffer from a serious performance drop. (6) SAM performed better than interactive methods with one or a few points, but will be outpaced as the number of points increases. (7) SAM's performance correlated to different factors, including boundary complexity, intensity differences, etc. (8) Finetuning the SAM on specific medical tasks could improve its average DICE performance by 4.39% and 6.68% for ViT-B and ViT-H, respectively. Codes and models are available at: https://github.com/yuhoo0302/Segment-Anything-Model-for-Medical-Images. We hope that this comprehensive report can help researchers explore the potential of SAM applications in MIS, and guide how to appropriately use and develop SAM.

Learning Better Registration to Learn Better Few-Shot Medical Image Segmentation: Authenticity, Diversity, and Robustness

In this work, we address the task of few-shot medical image segmentation (MIS) with a novel proposed framework based on the learning registration to learn segmentation (LRLS) paradigm. To cope with the limitations of lack of authenticity, diversity, and robustness in the existing LRLS frameworks, we propose the better registration better segmentation (BRBS) framework with three main contributions that are experimentally shown to have substantial practical merit. First, we improve the authenticity in the registration-based generation program and propose the knowledge consistency constraint strategy that constrains the registration network to learn according to the domain knowledge. It brings the semantic-aligned and topology-preserved registration, thus allowing the generation program to output new data with great space and style authenticity. Second, we deeply studied the diversity of the generation process and propose the space-style sampling program, which introduces the modeling of the transformation path of style and space change between few atlases and numerous unlabeled images into the generation program. Therefore, the sampling on the transformation paths provides much more diverse space and style features to the generated data effectively improving the diversity. Third, we first highlight the robustness in the learning of segmentation in the LRLS paradigm and propose the mix misalignment regularization, which simulates the misalignment distortion and constrains the network to reduce the fitting degree of misaligned regions. Therefore, it builds regularization for these regions improving the robustness of segmentation learning. Without any bells and whistles, our approach achieves a new state-of-the-art performance in few-shot MIS on two challenging tasks that outperform the existing LRLS-based few-shot methods. We believe that this novel and effective framework will provide a powerful few-shot benchmark for the field of medical image and efficiently reduce the costs of medical image research. All of our code will be made publicly available online.

Tubular structures segmentation of pediatric abdominal-visceral ceCT images with renal tumors: Assessment, comparison and improvement

Renal tubular structures, such as ureters, arteries and veins, are very important for building a complete digital 3D anatomical model of a patient. However, they can be challenging to segment from ceCT images due to their elongated shape, diameter variation and intra- and inter-patient contrast heterogeneity. This task is even more difficult in pediatric and pathological subjects, due to high inter-subject anatomical variations, potential presence of tumors, small volume of these structures compared to the surrounding, and small available labeled datasets. Given the limited literature on methods dedicated to children, and in order to find inspirational approaches, a complete assessment of state-of-the-art methods for the segmentation of renal tubular structures on ceCT images on adults is presented. Then, these methods are tested and compared on a private pediatric and pathological dataset of 79 abdominal-visceral ceCT images with arteriovenous phase acquisitions. To the best of our knowledge, both assessment and comparison in this specific case are novel. Eventually, we also propose a new loss function which leverages for the first time the use of vesselness functions on the predicted segmentation. We show that the combination of this loss function with state-of-the-art methods improves the topological coherence of the segmented tubular structures.2.

Semi-TMS: an efficient regularization-oriented triple-teacher semi-supervised medical image segmentation model

Objective. Although convolutional neural networks (CNN) and Transformers have performed well in many medical image segmentation tasks, they rely on large amounts of labeled data for training. The annotation of medical image data is expensive and time-consuming, so it is common to use semi-supervised learning methods that use a small amount of labeled data and a large amount of unlabeled data to improve the performance of medical imaging segmentation.Approach. This work aims to enhance the segmentation performance of medical images using a triple-teacher cross-learning semi-supervised medical image segmentation with shape perception and multi-scale consistency regularization. To effectively leverage the information from unlabeled data, we design a multi-scale semi-supervised method for three-teacher cross-learning based on shape perception, called Semi-TMS. The three teacher models engage in cross-learning with each other, where Teacher A and Teacher C utilize a CNN architecture, while Teacher B employs a transformer model. The cross-learning module consisting of Teacher A and Teacher C captures local and global information, generates pseudo-labels, and performs cross-learning using prediction results. Multi-scale consistency regularization is applied separately to the CNN and Transformer to improve accuracy. Furthermore, the low uncertainty output probabilities from Teacher A or Teacher C are utilized as input to Teacher B, enhancing the utilization of prior knowledge and overall segmentation robustness.Main results. Experimental evaluations on two public datasets demonstrate that the proposed method outperforms some existing semi-segmentation models, implicitly capturing shape information and effectively improving the utilization and accuracy of unlabeled data through multi-scale consistency.Significance. With the widespread utilization of medical imaging in clinical diagnosis, our method is expected to be a potential auxiliary tool, assisting clinicians and medical researchers in their diagnoses.

MDM-U-Net: A novel network for renal cancer structure segmentation

Accurate segmentation of the renal cancer structure, including the kidney, renal tumors, veins, and arteries, has great clinical significance, which can assist clinicians in diagnosing and treating renal cancer. For accurate segmentation of the renal cancer structure in contrast-enhanced computed tomography (CT) images, we proposed a novel encoder-decoder structure segmentation network named MDM-U-Net comprising a multi-scale anisotropic convolution block, dual activation attention block, and multi-scale deep supervision mechanism. The multi-scale anisotropic convolution block was used to improve the feature extraction ability of the network, the dual activation attention block as a channel-wise mechanism was used to guide the network to exploit important information, and the multi-scale deep supervision mechanism was used to supervise the layers of the decoder part for improving segmentation performance. In this study, we developed a feasible and generalizable MDM-U-Net model for renal cancer structure segmentation, trained the model from the public KiPA22 dataset, and tested it on the KiPA22 dataset and an in-house dataset. For the KiPA22 dataset, our method ranked first in renal cancer structure segmentation, achieving state-of-the-art (SOTA) performance in terms of 6 of 12 evaluation metrics (3 metrics per structure). For the in-house dataset, our method achieves SOTA performance in terms of 9 of 12 evaluation metrics (3 metrics per structure), demonstrating its superiority and generalization ability over the compared networks in renal structure segmentation from contrast-enhanced CT scans.

Deformable medical image registration with global-local transformation network and region similarity constraint

Deformable medical image registration can achieve fast and accurate alignment between two images, enabling medical professionals to analyze images of different subjects in a unified anatomical space. As such, it plays an important role in many medical image studies. Current deep learning (DL)-based approaches for image registration directly learn spatial transformation from one image to another, relying on a convolutional neural network and ground truth or similarity metrics. However, these methods only use a global similarity energy function to evaluate the similarity of a pair of images, which ignores the similarity of regions of interest (ROIs) within the images. This can limit the accuracy of the image registration and affect the analysis of specific ROIs. Additionally, DL-based methods often estimate global spatial transformations of images directly, without considering local spatial transformations of ROIs within the images. To address this issue, we propose a novel global-local transformation network with a region similarity constraint that maximizes the similarity of ROIs within the images and estimates both global and local spatial transformations simultaneously. Experiments conducted on four public 3D MRI datasets demonstrate that the proposed method achieves the highest registration performance in terms of accuracy and generalization compared to other state-of-the-art methods.

MBD-Net: Multi-Branch Dilated Convolutional Network With Cyst Discriminator for Renal Multi-Structure Segmentation

In surgery-based renal cancer treatment, one of the most essential tasks is the three-dimensional (3D) kidney parsing on computed tomography angiography (CTA) images. In this paper, we propose an end-to-end convolutional neural network-based framework to segment multiple renal structures, including kidneys, kidney tumors, arteries, and veins from arterial-phase CT images. Our method consists of two collaborative modules: First, we propose an encoding-decoding network, named Multi-Branch Dilated Convolutional Network (MBD-Net), consisting of residual, hybrid dilated convolutional, and reduced-dimensional convolutional structures, which improves the feature extraction ability with relatively fewer network parameters. Given that renal tumors and cysts have confusing geometric structures, we also design the Cyst Discriminator to effectively distinguish tumors from cysts without labeling information via gray-scale curves and radiographic features. We have quantitatively evaluated our approach on a publicly available dataset from MICCAI 2022 Kidney Parsing for Renal Cancer Treatment Challenge (KiPA2022), with mean Dice similarity coefficient (DSC) as 96.18%, 90.99%, 88.66% and 80.35% for the kidneys, kidney tumors, arteries, and veins respectively, winning the stable and top performance in the challenge.Clinical relevance-The proposed CNN-Based framework can automatically segment 3D kidneys, renal tumors, arteries, and veins for kidney parsing techniques, benefiting surgery-based renal cancer treatment.

Deep Learning for Head and Neck CT Angiography: Stenosis and Plaque Classification

Background Studies have rarely investigated stenosis detection from head and neck CT angiography scans because accurate interpretation is time consuming and labor intensive. Purpose To develop an automated convolutional neural network-based method for accurate stenosis detection and plaque classification in head and neck CT angiography images and compare its performance with that of radiologists. Materials and Methods A deep learning (DL) algorithm was constructed and trained with use of head and neck CT angiography images that were collected retrospectively from four tertiary hospitals between March 2020 and July 2021. CT scans were partitioned into training, validation, and independent test sets at a ratio of 7:2:1. An independent test set of CT angiography scans was collected prospectively between October 2021 and December 2021 in one of the four tertiary centers. Stenosis grade categories were as follows: mild stenosis (<50%), moderate stenosis (50%-69%), severe stenosis (70%-99%), and occlusion (100%). The stenosis diagnosis and plaque classification of the algorithm were compared with the ground truth of consensus by two radiologists (with more than 10 years of experience). The performance of the models was analyzed in terms of accuracy, sensitivity, specificity, and areas under the receiver operating characteristic curve. Results There were 3266 patients (mean age ± SD, 62 years ± 12; 2096 men) evaluated. The consistency between radiologists and the DL-assisted algorithm on plaque classification was 85.6% (320 of 374 cases [95% CI: 83.2, 88.6]) on a per-vessel basis. Moreover, the artificial intelligence model assisted in visual assessment, such as increasing confidence in the degree of stenosis. This reduced the time needed for diagnosis and report writing of radiologists from 28.8 minutes ± 5.6 to 12.4 minutes ± 2.0 (P < .001). Conclusion A deep learning algorithm for head and neck CT angiography interpretation accurately determined vessel stenosis and plaque classification and had equivalent diagnostic performance when compared with experienced radiologists. © RSNA, 2023 Supplemental material is available for this article.

Boundary-Sensitive Loss Function With Location Constraint for Hard Region Segmentation

In computer-aided diagnosis and treatment planning, accurate segmentation of medical images plays an essential role, especially for some hard regions including boundaries, small objects and background interference. However, existing segmentation loss functions including distribution-, region- and boundary-based losses cannot achieve satisfactory performances on these hard regions. In this paper, a boundary-sensitive loss function with location constraint is proposed for hard region segmentation in medical images, which provides three advantages: i) our Boundary-Sensitive loss (BS-loss) can automatically pay more attention to the hard-to-segment boundaries (e.g., thin structures and blurred boundaries), thus obtaining finer object boundaries; ii) BS-loss also can adjust its attention to small objects during training to segment them more accurately; and iii) our location constraint can alleviate the negative impact of the background interference, through the distribution matching of pixels between prediction and Ground Truth (GT) along each axis. By resorting to the proposed BS-loss and location constraint, the hard regions in both foreground and background are considered. Experimental results on three public datasets demonstrate the superiority of our method. Specifically, compared to the second-best method tested in this study, our method improves performance on hard regions in terms of Dice similarity coefficient (DSC) and 95% Hausdorff distance (95%HD) of up to 4.17% and 73% respectively. In addition, it also achieves the best overall segmentation performance. Hence, we can conclude that our method can accurately segment these hard regions and improve the overall segmentation performance in medical images.

FYU-Net: A Cascading Segmentation Network for Kidney Tumor Medical Imaging

Automated segmentation of renal tumors is essential for the diagnostic evaluation of kidney cancer. However, renal tumor volume is generally small compared with the volume of the kidney and is irregularly distributed; moreover, the location and shape of renal tumors are highly variable, making the segmentation task extremely challenging. To solve the aforementioned problems, a cascaded segmentation model (FYU-Net) for computed tomography (CT) images is proposed in this paper to achieve automatic kidney tumor segmentation. The proposed model involves two main steps. In the first step, a fast scan of the kidney CT data is performed using a localization network to find slices containing tumors, and coarse segmentation is performed simultaneously. In the second step, a segmentation framework embedded with the feature pyramid network module is employed to finely segment kidney tumors. By building a feature pyramid structure, targets of different sizes are distributed to be detected on different feature layers to extract richer feature information. In addition, the top-down structure allows the information of the higher-level feature maps to be transferred to the lower-level feature maps, enhancing the semantic information of the lower-level feature maps. Comparative experiments were conducted on the Kidney PArsing Challenge 2022 public dataset; the average Jaccard coefficient and average Dice coefficient of tumor structure segmentation were more than 70.73% and more than 82.85%, respectively. The results demonstrate the effectiveness of the proposed model for kidney tumor segmentation.

A novel 3D deep learning model to automatically demonstrate renal artery segmentation and its validation in nephron-sparing surgery

Purpose

Nephron-sparing surgery (NSS) is a mainstream treatment for localized renal tumors. Segmental renal artery clamping (SRAC) is commonly used in NSS. Automatic and precise segmentations of renal artery trees are required to improve the workflow of SRAC in NSS. In this study, we developed a tridimensional kidney perfusion (TKP) model based on deep learning technique to automatically demonstrate renal artery segmentation, and verified the precision and feasibility during laparoscopic partial nephrectomy (PN).

Methods

The TKP model was established based on convolutional neural network (CNN), and the precision was validated in porcine models. From April 2018 to January 2020, TKP model was applied in laparoscopic PN in 131 patients with T1a tumors. Demographics, perioperative variables, and data from the TKP models were assessed. Indocyanine green (ICG) with near-infrared fluorescence (NIRF) imaging was applied after clamping and dice coefficient was used to evaluate the precision of the model.

Results

The precision of the TKP model was validated in porcine models with the mean dice coefficient of 0.82. Laparoscopic PN was successfully performed in all cases with segmental renal artery clamping (SRAC) under TKP model’s guidance. The mean operation time was 100.8 min; the median estimated blood loss was 110 ml. The ischemic regions recorded in NIRF imaging were highly consistent with the perfusion regions in the TKP models (mean dice coefficient = 0.81). Multivariate analysis revealed that the feeding lobar artery number was strongly correlated with tumor size and contact surface area; the supplying segmental arteries number correlated with tumor size.

Conclusions

Using the CNN technique, the TKP model is developed to automatically present the renal artery trees and precisely delineate the perfusion regions of different segmental arteries. The guidance of the TKP model is feasible and effective in nephron-sparing surgery.

Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency

The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.

Semi-Supervised Deep Transfer Learning for Benign-Malignant Diagnosis of Pulmonary Nodules in Chest CT Images

Lung cancer is the leading cause of cancer deaths worldwide. Accurately diagnosing the malignancy of suspected lung nodules is of paramount clinical importance. However, to date, the pathologically-proven lung nodule dataset is largely limited and is highly imbalanced in benign and malignant distributions. In this study, we proposed a Semi-supervised Deep Transfer Learning (SDTL) framework for benign-malignant pulmonary nodule diagnosis. First, we utilize a transfer learning strategy by adopting a pre-trained classification network that is used to differentiate pulmonary nodules from nodule-like tissues. Second, since the size of samples with pathological-proven is small, an iterated feature-matching-based semi-supervised method is proposed to take advantage of a large available dataset with no pathological results. Specifically, a similarity metric function is adopted in the network semantic representation space for gradually including a small subset of samples with no pathological results to iteratively optimize the classification network. In this study, a total of 3,038 pulmonary nodules (from 2,853 subjects) with pathologically-proven benign or malignant labels and 14,735 unlabeled nodules (from 4,391 subjects) were retrospectively collected. Experimental results demonstrate that our proposed SDTL framework achieves superior diagnosis performance, with accuracy = 88.3%, AUC = 91.0% in the main dataset, and accuracy = 74.5%, AUC = 79.5% in the independent testing dataset. Furthermore, ablation study shows that the use of transfer learning provides 2% accuracy improvement, and the use of semi-supervised learning further contributes 2.9% accuracy improvement. Results implicate that our proposed classification network could provide an effective diagnostic tool for suspected lung nodules, and might have a promising application in clinical practice.

Co-optimization Learning Network for MRI Segmentation of Ischemic Penumbra Tissues

Convolutional neural networks (CNNs) have brought hope for the medical image auxiliary diagnosis. However, the shortfall of labeled medical image data is the bottleneck that limits the performance improvement of supervised CNN methods. In addition, annotating a large number of labeled medical image data is often expensive and time-consuming. In this study, we propose a co-optimization learning network (COL-Net) for Magnetic Resonance Imaging (MRI) segmentation of ischemic penumbra tissues. COL-Net base on the limited labeled samples and consists of an unsupervised reconstruction network (R), a supervised segmentation network (S), and a transfer block (T). The reconstruction network extracts the robust features from reconstructing pseudo unlabeled samples, which is the auxiliary branch of the segmentation network. The segmentation network is used to segment the target lesions under the limited labeled samples and the auxiliary of the reconstruction network. The transfer block is used to co-optimization the feature maps between the bottlenecks of the reconstruction network and segmentation network. We propose a mix loss function to optimize COL-Net. COL-Net is verified on the public ischemic penumbra segmentation challenge (SPES) with two dozen labeled samples. Results demonstrate that COL-Net has high predictive accuracy and generalization with the Dice coefficient of 0.79. The extended experiment also shows COL-Net outperforms most supervised segmentation methods. COL-Net is a meaningful attempt to alleviate the limited labeled sample problem in medical image segmentation.

Multi-level semantic adaptation for few-shot segmentation on cardiac image sequences

Obtaining manual labels is time-consuming and labor-intensive on cardiac image sequences. Few-shot segmentation can utilize limited labels to learn new tasks. However, it suffers from two challenges: spatial-temporal distribution bias and long-term information bias. These challenges derive from the impact of the time dimension on cardiac image sequences, resulting in serious over-adaptation. In this paper, we propose the multi-level semantic adaptation (MSA) for few-shot segmentation on cardiac image sequences. The MSA addresses the two biases by exploring the domain adaptation and the weight adaptation on the semantic features in multiple levels, including sequence-level, frame-level, and pixel-level. First, the MSA proposes the dual-level feature adjustment for domain adaptation in spatial and temporal directions. This adjustment explicitly aligns the frame-level feature and the sequence-level feature to improve the model adaptation on diverse modalities. Second, the MSA explores the hierarchical attention metric for weight adaptation in the frame-level feature and the pixel-level feature. This metric focuses on the similar frame and the target region to promote the model discrimination on the border features. The extensive experiments demonstrate that our MSA is effective in few-shot segmentation on cardiac image sequences with three modalities, i.e. MR, CT, and Echo (e.g. the average Dice is 0.9243), as well as superior to the ten state-of-the-art methods.

Few-shot Learning for Deformable Medical Image Registration with Perception-Correspondence Decoupling and Reverse Teaching

Deformable medical image registration estimates corresponding deformation to align the regions of interest (ROIs) of two images to a same spatial coordinate system. However, recent unsupervised registration models only have correspondence ability without perception, making misalignment on blurred anatomies and distortion on task-unconcerned backgrounds. Label-constrained (LC) registration models embed the perception ability via labels, but the lack of texture constraints in labels and the expensive labeling costs causes distortion internal ROIs and overfitted perception. We propose the first few-shot deformable medical image registration framework, Perception-Correspondence Registration (PC-Reg), which embeds perception ability to registration models only with few labels, thus greatly improving registration accuracy and reducing distortion. 1) We propose the Perception-Correspondence Decoupling which decouples the perception and correspondence actions of registration to two CNNs. Therefore, independent optimizations and feature representations are available avoiding interference of the correspondence due to the lack of texture constraints. 2) For few-shot learning, we propose Reverse Teaching which aligns labeled and unlabeled images to each other to provide supervision information to the structure and style knowledge in unlabeled images, thus generating additional training data. Therefore, these data will reversely teach our perception CNN more style and structure knowledge, improving its generalization ability. Our experiments on three datasets with only five labels demonstrate that our PC-Reg has competitive registration accuracy and effective distortion-reducing ability. Compared with LC-VoxelMorph(lambda=1), we achieve the 12.5%, 6.3% and 1.0% Reg-DSC improvements on three datasets, revealing our framework with great potential in clinical application.

Meta grayscale adaptive network for 3D integrated renal structures segmentation

Three-dimensional (3D) integrated renal structures (IRS) segmentation targets segmenting the kidneys, renal tumors, arteries, and veins in one inference. Clinicians will benefit from the 3D IRS visual model for accurate preoperative planning and intraoperative guidance of laparoscopic partial nephrectomy (LPN). However, no success has been reported in 3D IRS segmentation due to the inherent challenges in grayscale distribution: low contrast caused by the narrow task-dependent distribution range of regions of interest (ROIs), and the networks representation preferences caused by the distribution variation inter-images. In this paper, we propose the Meta Greyscale Adaptive Network (MGANet), the first deep learning framework to simultaneously segment the kidney, renal tumors, arteries and veins on CTA images in one inference. It makes innovations in two collaborate aspects: 1) The Grayscale Interest Search (GIS) adaptively focuses segmentation networks on task-dependent grayscale distributions via scaling the window width and center with two cross-correlated coefficients for the first time, thus learning the fine-grained representation for fine segmentation. 2) The Meta Grayscale Adaptive (MGA) learning makes an image-level meta-learning strategy. It represents diverse robust features from multiple distributions, perceives the distribution characteristic, and generates the model parameters to fuse features dynamically according to image's distribution, thus adapting the grayscale distribution variation. This study enrolls 123 patients and the average Dice coefficients of the renal structures are up to 87.9%. Fine selection of the task-dependent grayscale distribution ranges and personalized fusion of multiple representations on different distributions will lead to better 3D IRS segmentation quality. Extensive experiments with promising results on renal structures reveal powerful segmentation accuracy and great clinical significance in renal cancer treatment.

Learning-based algorithms for vessel tracking: A review

Developing efficient vessel-tracking algorithms is crucial for imaging-based diagnosis and treatment of vascular diseases. Vessel tracking aims to solve recognition problems such as key (seed) point detection, centerline extraction, and vascular segmentation. Extensive image-processing techniques have been developed to overcome the problems of vessel tracking that are mainly attributed to the complex morphologies of vessels and image characteristics of angiography. This paper presents a literature review on vessel-tracking methods, focusing on machine-learning-based methods. First, the conventional machine-learning-based algorithms are reviewed, and then, a general survey of deep-learning-based frameworks is provided. On the basis of the reviewed methods, the evaluation issues are introduced. The paper is concluded with discussions about the remaining exigencies and future research.