Knowledge distillation on individual vertebrae segmentation exploiting 3D U-Net

Recent advances in medical imaging have highlighted the critical development of algorithms for individual vertebral segmentation on computed tomography (CT) scans. Essential for diagnostic accuracy and treatment planning in orthopaedics, neurosurgery and oncology, these algorithms face challenges in clinical implementation, including integration into healthcare systems. Consequently, our focus lies in exploring the application of knowledge distillation (KD) methods to train shallower networks capable of efficiently segmenting vertebrae in CT scans. This approach aims to reduce segmentation time, enhance suitability for emergency cases, and optimize computational and memory resource efficiency. Building upon prior research in the field, a two-step segmentation approach was employed. Firstly, the spine's location was determined by predicting a heatmap, indicating the probability of each voxel belonging to the spine. Subsequently, an iterative segmentation of vertebrae was performed from the top to the bottom of the CT volume over the located spine, using a memory instance to record the already segmented vertebrae. KD methods were implemented by training a teacher network with performance similar to that found in the literature, and this knowledge was distilled to a shallower network (student). Two KD methods were applied: (1) using the soft outputs of both networks and (2) matching logits. Two publicly available datasets, comprising 319 CT scans from 300 patients and a total of 611 cervical, 2387 thoracic, and 1507 lumbar vertebrae, were used. To ensure dataset balance and robustness, effective data augmentation methods were applied, including cleaning the memory instance to replicate the first vertebra segmentation. The teacher network achieved an average Dice similarity coefficient (DSC) of 88.22% and a Hausdorff distance (HD) of 7.71 mm, showcasing performance similar to other approaches in the literature. Through knowledge distillation from the teacher network, the student network's performance improved, with an average DSC increasing from 75.78% to 84.70% and an HD decreasing from 15.17 mm to 8.08 mm. Compared to other methods, our teacher network exhibited up to 99.09% fewer parameters, 90.02% faster inference time, 88.46% shorter total segmentation time, and 89.36% less associated carbon (CO2) emission rate. Regarding our student network, it featured 75.00% fewer parameters than our teacher, resulting in a 36.15% reduction in inference time, a 33.33% decrease in total segmentation time, and a 42.96% reduction in CO2 emissions. This study marks the first exploration of applying KD to the problem of individual vertebrae segmentation in CT, demonstrating the feasibility of achieving comparable performance to existing methods using smaller neural networks.

Active Learning in Brain Tumor Segmentation with Uncertainty Sampling and Annotation Redundancy Restriction

Deep learning models have demonstrated great potential in medical imaging but are limited by the expensive, large volume of annotations required. To address this, we compared different active learning strategies by training models on subsets of the most informative images using real-world clinical datasets for brain tumor segmentation and proposing a framework that minimizes the data needed while maintaining performance. Then, 638 multi-institutional brain tumor magnetic resonance imaging scans were used to train three-dimensional U-net models and compare active learning strategies. Uncertainty estimation techniques including Bayesian estimation with dropout, bootstrapping, and margins sampling were compared to random query. Strategies to avoid annotating similar images were also considered. We determined the minimum data necessary to achieve performance equivalent to the model trained on the full dataset (α = 0.05). Bayesian approximation with dropout at training and testing showed results equivalent to that of the full data model (target) with around 30% of the training data needed by random query to achieve target performance (p = 0.018). Annotation redundancy restriction techniques can reduce the training data needed by random query to achieve target performance by 20%. We investigated various active learning strategies to minimize the annotation burden for three-dimensional brain tumor segmentation. Dropout uncertainty estimation achieved target performance with the least annotated data.

Geometric evaluations of CT and MRI based deep learning segmentation for brain OARs in radiotherapy

Objective.Deep-learning auto-contouring (DL-AC) promises standardisation of organ-at-risk (OAR) contouring, enhancing quality and improving efficiency in radiotherapy. No commercial models exist for OAR contouring based on brain magnetic resonance imaging (MRI). We trained and evaluated computed tomography (CT) and MRI OAR autosegmentation models in RayStation. To ascertain clinical usability, we investigated the geometric impact of contour editing before training on model quality.Approach.Retrospective glioma cases were randomly selected for training (n= 32, 47) and validation (n= 9, 10) for MRI and CT, respectively. Clinical contours were edited using international consensus (gold standard) based on MRI and CT. MRI models were trained (i) using the original clinical contours based on planning CT and rigidly registered T1-weighted gadolinium-enhanced MRI (MRIu), (ii) as (i), further edited based on CT anatomy, to meet international consensus guidelines (MRIeCT), and (iii) as (i), further edited based on MRI anatomy (MRIeMRI). CT models were trained using: (iv) original clinical contours (CTu) and (v) clinical contours edited based on CT anatomy (CTeCT). Auto-contours were geometrically compared to gold standard validation contours (CTeCT or MRIeMRI) using Dice Similarity Coefficient, sensitivity, and mean distance to agreement. Models' performances were compared using paired Student's t-testing.Main results.The edited autosegmentation models successfully generated more segmentations than the unedited models. Paired t-testing showed editing pituitary, orbits, optic nerves, lenses, and optic chiasm on MRI before training significantly improved at least one geometry metric. MRI-based DL-AC performed worse than CT-based in delineating the lacrimal gland, whereas the CT-based performed worse in delineating the optic chiasm. No significant differences were found between the CTeCT and CTu except for optic chiasm.Significance.T1w-MRI DL-AC could segment all brain OARs except the lacrimal glands, which cannot be easily visualized on T1w-MRI. Editing contours on MRI before model training improved geometric performance. MRI DL-AC in RT may improve consistency, quality and efficiency but requires careful editing of training contours.

Voxel-Based Internal Dosimetry for 177Lu-Labeled Radiopharmaceutical Therapy Using Deep Residual Learning

Purpose

In this study, we propose a deep learning (DL)-based voxel-based dosimetry method in which dose maps acquired using the multiple voxel S-value (VSV) approach were used for residual learning.

Methods

Twenty-two SPECT/CT datasets from seven patients who underwent 177Lu-DOTATATE treatment were used in this study. The dose maps generated from Monte Carlo (MC) simulations were used as the reference approach and target images for network training. The multiple VSV approach was used for residual learning and compared with dose maps generated from deep learning. The conventional 3D U-Net network was modified for residual learning. The absorbed doses in the organs were calculated as the mass-weighted average of the volume of interest (VOI).

Results

The DL approach provided a slightly more accurate estimation than the multiple-VSV approach, but the results were not statistically significant. The single-VSV approach yielded a relatively inaccurate estimation. No significant difference was noted between the multiple VSV and DL approach on the dose maps. However, this difference was prominent in the error maps. The multiple VSV and DL approach showed a similar correlation. In contrast, the multiple VSV approach underestimated doses in the low-dose range, but it accounted for the underestimation when the DL approach was applied.

Conclusion

Dose estimation using the deep learning-based approach was approximately equal to that in the MC simulation. Accordingly, the proposed deep learning network is useful for accurate and fast dosimetry after radiation therapy using 177Lu-labeled radiopharmaceuticals.

BIRGU Net: deformable brain magnetic resonance image registration using gyral-net map and 3D Res-Unet

Deformable image registration is a fundamental procedure in medical imaging. Recently, deep learning-based deformable image registrations have achieved fast registration by learning the spatial correspondence from image pairs. However, it remains challenging in brain image registration due to the structural complexity of individual brains and the lack of ground truth for anatomical correspondences between the brain image pairs. This work devotes to achieving an end-to-end unsupervised brain deformable image registration method using the gyral-net map and 3D Res-Unet (BIRGU Net). Firstly, the gyral-net map was introduced to represent the 3D global cortex complex information of the brain image since it was considered as one of the anatomical landmarks, which can help to extract the salient structural feature of individual brains for registration. Secondly, the variant of 3D U-net architecture involving dual residual strategies was designed to map the image into the deformation field effectively and to prevent the gradient from vanishing as well. Finally, double regularized terms were imposed on the deformation field to guide the network training for leveraging the smoothness and the topology preservation of the deformation field. The registration procedure was trained in an unsupervised manner, which addressed the lack of ground truth for anatomical correspondences between the brain image pairs. The experimental results on four public data sets demonstrate that the extracted gyral-net can be an auxiliary feature for registration and the proposed network with the designed strategies can improve the registration performance since the Dice similarity coefficient (DSC) and normalized mutual information (NMI) are higher and the time consumption is comparable than the state-of-the-art. The code is available at https://github.com/mynameiswode/BIRGU-Net .

SynBOLD-DisCo: Synthetic BOLD images for distortion correction of fMRI without additional calibration scans

The blood oxygen level dependent (BOLD) signal from functional magnetic resonance imaging (fMRI) is a noninvasive technique that has been widely used in research to study brain function. However, fMRI suffers from susceptibility-induced off resonance fields which may cause geometric distortions and mismatches with anatomical images. State-of-the-art correction methods require acquiring reverse phase encoded images or additional field maps to enable distortion correction. However, not all imaging protocols include these additional scans and thus cannot take advantage of these susceptibility correction capabilities. As such, in this study we aim to enable state-of-the-art distortion correction with FSL's topup algorithm of historical and/or limited fMRI data that include only a structural image and single phase encoded fMRI. To do this, we use 3D U-net models to synthesize undistorted fMRI BOLD contrast images from the structural image and use this undistorted synthetic image as an anatomical target for distortion correction with topup. We evaluate the efficacy of this approach, named SynBOLD-DisCo (synthetic BOLD images for distortion correction), and show that BOLD images corrected using our approach are geometrically more similar to structural images than the distorted BOLD data and are practically equivalent to state-of-the-art correction methods which require reverse phase encoded data. Future directions include additional validation studies, integration with other preprocessing operations, retraining with broader pathologies, and investigating the effects of spin echo versus gradient echo images for training and distortion correction. In summary, we demonstrate SynBOLD-DisCo corrects distortion of fMRI when reverse phase encoding scans or field maps are not available.

3D deep convolutional neural network segmentation model for precipitate and porosity identification in synchrotron X-ray tomograms

New developments at synchrotron beamlines and the ongoing upgrades of synchrotron facilities allow the possibility to study complex structures with a much better spatial and temporal resolution than ever before. However, the downside is that the data collected are also significantly larger (more than several terabytes) than ever before, and post-processing and analyzing these data is very challenging to perform manually. This issue can be solved by employing automated methods such as machine learning, which show significantly improved performance in data processing and image segmentation than manual methods. In this work, a 3D U-net deep convolutional neural network (DCNN) model with four layers and base-8 characteristic features has been developed to segment precipitates and porosities in synchrotron transmission X-ray micrograms. Transmission X-ray microscopy experiments were conducted on micropillars prepared from additively manufactured 316L steel to evaluate precipitate information. After training the 3D U-net DCNN model, it was used on unseen data and the prediction was compared with manual segmentation. A good agreement was found between both segmentations. An ablation study was performed and revealed that the proposed model showed better statistics than other models with lower numbers of layers and/or characteristic features. The proposed model is able to segment several hundreds of gigabytes of data in a few minutes and could be applied to other materials and tomography techniques. The code and the fitted weights are made available with this paper for any interested researcher to use for their needs (https://github.com/manasvupadhyay/erc-gamma-3D-DCNN).

Brain Tumor Segmentation Using an Ensemble of 3D U-Nets and Overall Survival Prediction Using Radiomic Features

Accurate segmentation of different sub-regions of gliomas such as peritumoral edema, necrotic core, enhancing, and non-enhancing tumor core from multimodal MRI scans has important clinical relevance in diagnosis, prognosis and treatment of brain tumors. However, due to the highly heterogeneous appearance and shape of these tumors, segmentation of the sub-regions is challenging. Recent developments using deep learning models has proved its effectiveness in various semantic and medical image segmentation tasks, many of which are based on the U-Net network structure with symmetric encoding and decoding paths for end-to-end segmentation due to its high efficiency and good performance. In brain tumor segmentation, the 3D nature of multimodal MRI poses challenges such as memory and computation limitations and class imbalance when directly adopting the U-Net structure. In this study we aim to develop a deep learning model using a 3D U-Net with adaptations in the training and testing strategies, network structures, and model parameters for brain tumor segmentation. Furthermore, instead of picking one best model, an ensemble of multiple models trained with different hyper-parameters are used to reduce random errors from each model and yield improved performance. Preliminary results demonstrate the effectiveness of this method and achieved the 9th place in the very competitive 2018 Multimodal Brain Tumor Segmentation (BraTS) challenge. In addition, to emphasize the clinical value of the developed segmentation method, a linear model based on the radiomics features extracted from segmentation and other clinical features are developed to predict patient overall survival. Evaluation of these innovations shows high prediction accuracy in both low-grade glioma and glioblastoma patients, which achieved the 1st place in the 2018 BraTS challenge.

Fully Automated Lung Lobe Segmentation in Volumetric Chest CT with 3D U-Net: Validation with Intra- and Extra-Datasets

Lung lobe segmentation in chest CT has been used for the analysis of lung functions and surgical planning. However, accurate lobe segmentation is difficult as 80% of patients have incomplete and/or fake fissures. Furthermore, lung diseases such as chronic obstructive pulmonary disease (COPD) can increase the difficulty of differentiating the lobar fissures. Lobar fissures have similar intensities to those of the vessels and airway wall, which could lead to segmentation error in automated segmentation. In this study, a fully automated lung lobe segmentation method with 3D U-Net was developed and validated with internal and external datasets. The volumetric chest CT scans of 196 normal and mild-to-moderate COPD patients from three centers were obtained. Each scan was segmented using a conventional image processing method and manually corrected by an expert thoracic radiologist to create gold standards. The lobe regions in the CT images were then segmented using a 3D U-Net architecture with a deep convolutional neural network (CNN) using separate training, validation, and test datasets. In addition, 40 independent external CT images were used to evaluate the model. The segmentation results for both the conventional and deep learning methods were compared quantitatively to the gold standards using four accuracy metrics including the Dice similarity coefficient (DSC), Jaccard similarity coefficient (JSC), mean surface distance (MSD), and Hausdorff surface distance (HSD). In internal validation, the segmentation method achieved high accuracy for the DSC, JSC, MSD, and HSD (0.97 ± 0.02, 0.94 ± 0.03, 0.69 ± 0.36, and 17.12 ± 11.07, respectively). In external validation, high accuracy was also obtained for the DSC, JSC, MSD, and HSD (0.96 ± 0.02, 0.92 ± 0.04, 1.31 ± 0.56, and 27.89 ± 7.50, respectively). This method took 6.49 ± 1.19 s and 8.61 ± 1.08 s for lobe segmentation of the left and right lungs, respectively. Although various automatic lung lobe segmentation methods have been developed, it is difficult to develop a robust segmentation method. However, the deep learning-based 3D U-Net method showed reasonable segmentation accuracy and computational time. In addition, this method could be adapted and applied to severe lung diseases in a clinical workflow.

Fully-automatic segmentation of coronary artery using growing algorithm

Currently, cardiac computed tomography angiography (CTA) is widely applied to coronary artery disease diagnosis. Automatic segmentation of coronary artery has played an important role in coronary artery disease diagnosis. In this study, we propose and test a fully automatic coronary artery segmentation method that does not require any human-computer interaction. The proposed method uses a growing strategy and contains three main parts namely, (1) the initial seed detection that automatically detects the root points of the left and right coronary arteries where the ascending aorta meets the coronary arteries, (2) the growing strategy that searches for the neighborhood blocks to decide the existence of coronary arteries with an improved convolutional neural network, and (3) the iterative termination condition that decides whether the growing iteration finishes. The proposed framework is validated using a dataset containing 32 cardiac CTA volumes from different patients for training and testing. Experimental results show that the proposed method obtained a Dice loss ranged from 0.70 to 0.83, which indicates that the new method outperforms the traditional methods such as level set.