CU-Net: Cascaded U-Net with Loss Weighted Sampling for Brain Tumor Segmentation

Fine-Grained Permeable Surface Mapping through Parallel U-Net

Permeable surface mapping, which mainly is the identification of surface materials that will percolate, is essential for various environmental and civil engineering applications, such as urban planning, stormwater management, and groundwater modeling. Traditionally, this task involves labor-intensive manual classification, but deep learning offers an efficient alternative. Although several studies have tackled aerial image segmentation, the challenges in permeable surface mapping arid environments remain largely unexplored because of the difficulties in distinguishing pixel values of the input data and due to the unbalanced distribution of its classes. To address these issues, this research introduces a novel approach using a parallel U-Net model for the fine-grained semantic segmentation of permeable surfaces. The process involves binary classification to distinguish between entirely and partially permeable surfaces, followed by fine-grained classification into four distinct permeability levels. Results show that this novel method enhances accuracy, particularly when working with small, unbalanced datasets dominated by a single category. Furthermore, the proposed model is capable of generalizing across different geographical domains. Domain adaptation is explored to transfer knowledge from one location to another, addressing the challenges posed by varying environmental characteristics. Experiments demonstrate that the parallel U-Net model outperforms the baseline methods when applied across domains. To support this research and inspire future research, a novel permeable surface dataset is introduced, with pixel-wise fine-grained labeling for five distinct permeable surface classes. In summary, in this work, we offer a novel solution to permeable surface mapping, extend the boundaries of arid environment mapping, introduce a large-scale permeable surface dataset, and explore cross-area applications of the proposed model. The three contributions are enhancing the efficiency and accuracy of permeable surface mapping while progressing in this field.

Fully-automated, CT-only GTV contouring for palliative head and neck radiotherapy

Planning for palliative radiotherapy is performed without the advantage of MR or PET imaging in many clinics. Here, we investigated CT-only GTV delineation for palliative treatment of head and neck cancer. Two multi-institutional datasets of palliative-intent treatment plans were retrospectively acquired: a set of 102 non-contrast-enhanced CTs and a set of 96 contrast-enhanced CTs. The nnU-Net auto-segmentation network was chosen for its strength in medical image segmentation, and five approaches separately trained: (1) heuristic-cropped, non-contrast images with a single GTV channel, (2) cropping around a manually-placed point in the tumor center for non-contrast images with a single GTV channel, (3) contrast-enhanced images with a single GTV channel, (4) contrast-enhanced images with separate primary and nodal GTV channels, and (5) contrast-enhanced images along with synthetic MR images with separate primary and nodal GTV channels. Median Dice similarity coefficient ranged from 0.6 to 0.7, surface Dice from 0.30 to 0.56, and 95th Hausdorff distance from 14.7 to 19.7 mm across the five approaches. Only surface Dice exhibited statistically-significant difference across these five approaches using a two-tailed Wilcoxon Rank-Sum test (p ≤ 0.05). Our CT-only results met or exceeded published values for head and neck GTV autocontouring using multi-modality images. However, significant edits would be necessary before clinical use in palliative radiotherapy.

Retina-TransNet: A Gradient-Guided Few-Shot Retinal Vessel Segmentation Net

Due to the high labor cost of physicians, it is difficult to collect a rich amount of manually-labeled medical images for developing learning-based computer-aided diagnosis (CADx) systems or segmentation algorithms. To tackle this issue, we reshape the image segmentation task as an image-to-image (I2I) translation problem and propose a retinal vascular segmentation network, which can achieve good cross-domain generalizability even with a small amount of training data. We devise primarily two components to facilitate this I2I-based segmentation method. The first is the constraints provided by the proposed gradient-vector-flow (GVF) loss, and, the second is a two-stage Unet (2Unet) generator with a skip connection. This configuration makes 2Unet's first-stage play a role similar to conventional Unet, but forces 2Unet's second stage to learn to be a refinement module. Extensive experiments show that by re-casting retinal vessel segmentation as an image-to-image translation problem, our I2I translator-based segmentation subnetwork achieves better cross-domain generalizability than existing segmentation methods. Our model, trained on one dataset, e.g., DRIVE, can produce segmentation results stably on datasets of other domains, e.g., CHASE-DB1, STARE, HRF, and DIARETDB1, even in low-shot circumstances.

AI-based MRI auto-segmentation of brain tumor in rodents, a multicenter study

Automatic segmentation of rodent brain tumor on magnetic resonance imaging (MRI) may facilitate biomedical research. The current study aims to prove the feasibility for automatic segmentation by artificial intelligence (AI), and practicability of AI-assisted segmentation. MRI images, including T2WI, T1WI and CE-T1WI, of brain tumor from 57 WAG/Rij rats in KU Leuven and 46 mice from the cancer imaging archive (TCIA) were collected. A 3D U-Net architecture was adopted for segmentation of tumor bearing brain and brain tumor. After training, these models were tested with both datasets after Gaussian noise addition. Reduction of inter-observer disparity by AI-assisted segmentation was also evaluated. The AI model segmented tumor-bearing brain well for both Leuven and TCIA datasets, with Dice similarity coefficients (DSCs) of 0.87 and 0.85 respectively. After noise addition, the performance remained unchanged when the signal-noise ratio (SNR) was higher than two or eight, respectively. For the segmentation of tumor lesions, AI-based model yielded DSCs of 0.70 and 0.61 for Leuven and TCIA datasets respectively. Similarly, the performance is uncompromised when the SNR was over two and eight respectively. AI-assisted segmentation could significantly reduce the inter-observer disparities and segmentation time in both rats and mice. Both AI models for segmenting brain or tumor lesions could improve inter-observer agreement and therefore contributed to the standardization of the following biomedical studies.

Automatic segmentation of colon, small intestine, and duodenum based on scale attention network

PURPOSE

Automatic segmentation of colon, small intestine, and duodenum is a challenging task because of the great variability in the scale of the target organs. Multi-scale features are the key to alleviating this problem. Previous works focused on extracting discriminative multi-scale features through a hierarchical structure. Instead, the purpose of this work is to exploit these powerful multi-scale features more efficiently.

METHODS

A Scale Attention Module (SAM) was proposed to recalibrate multi-scale features by explicitly modeling their importance score adaptively. The SAM was introduced into the segmentation model to construct the Scale Attention Network (SANet). The multi-scale features extracted from the encoder were first re-extracted to obtain more specific multi-scale features. Then the SAM was applied to recalibrate the features. Specifically, for the feature of each scale, a summation of Global Average Pooling and Global Max Pooling was used to create scale-wise feature representations. According to the representations, a lightweight network was used to generate the importance score of each scale. The features were recalibrated based on the scores, and a simple pixel-by-pixel summation was used to fuse the multi-scale features. The fused multi-scale feature was fed into a segmentation head to complete the task.

RESULTS

The models were evaluated using fivefold cross-validation on 70 upper abdominal computed tomography scans of patients in a volume manner. The results showed that SANet could effectively alleviate the scale-variability problem and achieve better performance compared with UNet, Attention UNet, UNet++, Deeplabv3p, and CascadedUNet. The Dice similarity coefficients (DSCs) of colon, small intestine, and duodenum were (84.06 ± 3.66)%, (76.79 ± 5.12)%, and (61.68 ± 4.32)%, respectively. The HD95 were (7.51 ± 2.45) mm, (11.08 ± 2.45) mm, and (12.21 ± 1.95) mm, respectively. The values of relative volume difference were (3.4 ± 0.8)%, (11.6 ± 11.81)%, and (6.2 ± 3.71)%, respectively. The values of center-of-mass distance were 7.85 ± 2.82, 9.89 ± 2.70, and 9.94 ± 1.58, respectively. Compared with other attention modules and multi-scale feature exploitation approaches, SAM could obtain a 0.83-2.71 points improvement in terms of DSC with a comparable or even less number of parameters. The extensive experiments confirmed the effectiveness of SAM.

CONCLUSIONS

The SANet can efficiently exploit multi-scale features to alleviate the scale-variability problem and improve the segmentation performance on colon, small intestine, and duodenum of the upper abdomen.

CSM-Net: Automatic joint segmentation of intima-media complex and lumen in carotid artery ultrasound images

The intima-media thickness (IMT) is an effective biomarker for atherosclerosis, which is commonly measured by ultrasound technique. However, the intima-media complex (IMC) segmentation for the IMT is challenging due to confused IMC boundaries and various noises. In this paper, we propose a flexible method CSM-Net for the joint segmentation of IMC and Lumen in carotid ultrasound images. Firstly, the cascaded dilated convolutions combined with the squeeze-excitation module are introduced for exploiting more contextual features on the highest-level layer of the encoder. Furthermore, a triple spatial attention module is utilized for emphasizing serviceable features on each decoder layer. Besides, a multi-scale weighted hybrid loss function is employed to resolve the class-imbalance issues. The experiments are conducted on a private dataset of 100 images for IMC and Lumen segmentation, as well as on two public datasets of 1600 images for IMC segmentation. For the private dataset, our method obtain the IMC Dice, Lumen Dice, Precision, Recall, and F1 score of 0.814 ± 0.061, 0.941 ± 0.024, 0.911 ± 0.044, 0.916 ± 0.039, and 0.913 ± 0.027, respectively. For the public datasets, we obtain the IMC Dice, Precision, Recall, and F1 score of 0.885 ± 0.067, 0.885 ± 0.070, 0.894 ± 0.089, and 0.885 ± 0.067, respectively. The results demonstrate that the proposed method precedes some cutting-edge methods, and the ablation experiments show the validity of each module. The proposed method may be useful for the IMC segmentation of carotid ultrasound images in the clinic. Our code is publicly available at https://github.com/yuanyc798/US-IMC-code.

Cascaded Deep Convolutional Neural Networks as Improved Methods of Preprocessing Raman Spectroscopy Data

Machine learning has had a significant impact on the value of spectroscopic characterization tools, particularly in biomedical applications, due to its ability to detect latent patterns within complex spectral data. However, it often requires extensive data preprocessing, including baseline correction and denoising, which can lead to an unintentional bias during classification. To address this, we developed two deep learning methods capable of fully preprocessing raw Raman spectroscopy data without any human input. First, cascaded deep convolutional neural networks (CNN) based on either ResNet or U-Net architectures were trained on randomly generated spectra with augmented defects. Then, they were tested using simulated Raman spectra, surface-enhanced Raman spectroscopy (SERS) imaging of chemical species, low resolution Raman spectra of human bladder cancer tissue, and finally, classification of SERS spectra from human placental extracellular vesicles (EVs). Both approaches resulted in faster training and complete spectral preprocessing in a single step, with more speed, defect tolerance, and classification accuracy compared to conventional methods. These findings indicate that cascaded CNN preprocessing is ideal for biomedical Raman spectroscopy applications in which large numbers of heterogeneous spectra with diverse defects need to be automatically, rapidly, and reproducibly preprocessed.

3D U-Net Improves Automatic Brain Extraction for Isotropic Rat Brain Magnetic Resonance Imaging Data

Brain extraction is a critical pre-processing step in brain magnetic resonance imaging (MRI) analytical pipelines. In rodents, this is often achieved by manually editing brain masks slice-by-slice, a time-consuming task where workloads increase with higher spatial resolution datasets. We recently demonstrated successful automatic brain extraction via a deep-learning-based framework, U-Net, using 2D convolutions. However, such an approach cannot make use of the rich 3D spatial-context information from volumetric MRI data. In this study, we advanced our previously proposed U-Net architecture by replacing all 2D operations with their 3D counterparts and created a 3D U-Net framework. We trained and validated our model using a recently released CAMRI rat brain database acquired at isotropic spatial resolution, including T2-weighted turbo-spin-echo structural MRI and T2*-weighted echo-planar-imaging functional MRI. The performance of our 3D U-Net model was compared with existing rodent brain extraction tools, including Rapid Automatic Tissue Segmentation, Pulse-Coupled Neural Network, SHape descriptor selected External Regions after Morphologically filtering, and our previously proposed 2D U-Net model. 3D U-Net demonstrated superior performance in Dice, Jaccard, center-of-mass distance, Hausdorff distance, and sensitivity. Additionally, we demonstrated the reliability of 3D U-Net under various noise levels, evaluated the optimal training sample sizes, and disseminated all source codes publicly, with a hope that this approach will benefit rodent MRI research community. Significant Methodological Contribution: We proposed a deep-learning-based framework to automatically identify the rodent brain boundaries in MRI. With a fully 3D convolutional network model, 3D U-Net, our proposed method demonstrated improved performance compared to current automatic brain extraction methods, as shown in several qualitative metrics (Dice, Jaccard, PPV, SEN, and Hausdorff). We trust that this tool will avoid human bias and streamline pre-processing steps during 3D high resolution rodent brain MRI data analysis. The software developed herein has been disseminated freely to the community.

Technical Note: A cascade 3D U-Net for dose prediction in radiotherapy

PURPOSE

Although large datasets are available, to learn a robust dose prediction model from a limited dataset still remains challenging. This work employed cascaded deep learning models and advanced training strategies with a limited dataset to precisely predict three-dimensional (3D) dose distribution.

METHODS

A Cascade 3D (C3D) model is developed based on the cascade mechanism and 3D U-Net network units. During model training, data augmentations are used to improve the generalization ability of the prediction model. A knowledge distillation technique is employed to further improve the capability of model learning. The C3D network was evaluated using the OpenKBP challenge dataset and competed with those models proposed by more than 40 teams globally. Additionally, it was compared with five existing cutting-edge dose prediction models. The performance of these prediction models was evaluated by voxel-based mean absolute error (MAE) and clinical-related dosimetric metrics. The code and models are publicly available online (https://github.com/LSL000UD/RTDosePrediction).

RESULTS

The MAE of a single C3D model without test-time augmentation is 2.50 Gy (3.57% related to prescription dose) for nonzero dose area, which outperforms the other five dose prediction models by about 0.1 Gy-1.7 Gy. The C3D model won both dose and DVH streams of AAPM 2020 OpenKBP challenge with dose score of 2.31 and DVH score of 1.55.

CONCLUSIONS

The Cascading U-Nets is an ideal solution for 3D dose prediction from a limited dataset. The proper data preprocessing, data augmentation, and optimization procedure are more important than architectural modifications of deep learning network.

Spatial-channel relation learning for brain tumor segmentation

PURPOSE

Recently, research on brain tumor segmentation has made great progress. However, ambiguous patterns in magnetic resonance imaging data and linear fusion omitting semantic gaps between features in different branches remain challenging. We need to design a mechanism to fully utilize the similarity within the spatial space and channel space and the correlation between these two spaces to improve the result of volumetric segmentation.

METHODS

We propose a revised cascade structure network. In each subnetwork, a context exploitation module is introduced between the encoder and decoder, in which the dual attention mechanism is adopted to learn the information within the spatial space and channel space, and space interaction learning is employed to model the relation between the spatial and channel spaces.

RESULTS

Extensive experiments on the BraTS19 dataset have evaluated that our approach improves the dice coefficient (DC) by a margin of 2.1, 2.0, and 1.4 for whole tumor (WT), tumor core (TC), and enhancing tumor (ET), respectively, obtaining results competitive with the state-of-art approaches working on brain tumor segmentation.

CONCLUSIONS

Context exploitation in the embedding feature spaces, including intraspace relations and interspace relations, can effectively model dependency in semantic features and alleviate the semantic gap in multimodel data. Our approach is also robust to variations in different modality.