Semi or fully automatic tooth segmentation in CBCT images: a review

Cone beam computed tomography (CBCT) is widely employed in modern dentistry, and tooth segmentation constitutes an integral part of the digital workflow based on these imaging data. Previous methodologies rely heavily on manual segmentation and are time-consuming and labor-intensive in clinical practice. Recently, with advancements in computer vision technology, scholars have conducted in-depth research, proposing various fast and accurate tooth segmentation methods. In this review, we review 55 articles in this field and discuss the effectiveness, advantages, and disadvantages of each approach. In addition to simple classification and discussion, this review aims to reveal how tooth segmentation methods can be improved by the application and refinement of existing image segmentation algorithms to solve problems such as irregular morphology and fuzzy boundaries of teeth. It is assumed that with the optimization of these methods, manual operation will be reduced, and greater accuracy and robustness in tooth segmentation will be achieved. Finally, we highlight the challenges that still exist in this field and provide prospects for future directions.

MD-UNet: a medical image segmentation network based on mixed depthwise convolution

In the process of cancer diagnosis and treatment, accurate extraction of the lesion area helps the doctor to judge the condition. Currently, medical image segmentation algorithms based on UNet have been verified to be able to play an important role in clinical diagnosis. However, these methods still have the following drawbacks in extracting the region of interest (ROI): (1) ignoring the intra-class variability of medical images. (2) Failure to obtain effective feature redundancy. To address these problems, a U-shaped medical image segmentation network based on a Mixed depthwise convolution residual module (MDRM), called MD-UNet, is proposed in this paper. In MD-UNet, the MDRM built with a Mixed depthwise convolution attention block (MDAB) captures both local and global dependencies in the image to mitigate the effects of intra-class differences. MDAB captures valid redundant features and further captures global features of the input data. At the same time, the lightweight MDAB senses changes in the receptive field and generates multiple feature mappings. Compared with UNeXt on the ISIC2018 dataset, the MD-UNet segmentation accuracy Dice and IoU are improved by 1.33% and 1.91%, respectively. The code is available at https://github.com/Cloud-Liu/MD-UNet .

Segmentation method of magnetic resonance imaging brain tumor images based on improved UNet network

Background

Glioma is a primary malignant craniocerebral tumor commonly found in the central nervous system. According to research, preoperative diagnosis of glioma and a full understanding of its imaging features are very significant. Still, the traditional segmentation methods of image dispensation and machine wisdom are not acceptable in glioma segmentation. This analysis explores the potential of magnetic resonance imaging (MRI) brain tumor images as an effective segmentation method of glioma.

Methods

This study used 200 MRI images from the affiliated hospital and applied the 2-dimensional residual block UNet (2DResUNet). Features were extracted from input images using a 2×2 kernel size (64-kernel) 1-step 2D convolution (Conv) layer. The 2DDenseUNet model implemented in this study incorporates a ResBlock mechanism within the UNet architecture, as well as a Gaussian noise layer for data augmentation at the input stage, and a pooling layer for replacing the conventional 2D convolutional layers. Finally, the performance of the proposed protocol and its effective measures in glioma segmentation were verified.

Results

The outcomes of the 5-fold cross-validation evaluation show that the proposed 2DResUNet and 2DDenseUNet structure has a high sensitivity despite the slightly lower evaluation result on the Dice score. At the same time, compared with other models used in the experiment, the DM-DA-UNet model proposed in this paper was significantly improved in various indicators, increasing the reliability of the model and providing a reference and basis for the accurate formulation of clinical treatment strategies. The method used in this study showed stronger feature extraction ability than the UNet model. In addition, our findings demonstrated that using generalized die harm and prejudiced cross entropy as loss functions in the training process effectively alleviated the class imbalance of glioma data and effectively segmented glioma.

Conclusions

The method based on the improved UNet network has obvious advantages in the MRI brain tumor portrait segmentation procedure. The result showed that we developed a 2D residual block UNet, which can improve the incorporation of glioma segmentation into the clinical process.

Medical image segmentation network based on multi-scale frequency domain filter

With the development of deep learning, medical image segmentation in computer-aided diagnosis has become a research hotspot. Recently, UNet and its variants have become the most powerful medical image segmentation methods. However, these methods suffer from (1) insufficient sensing field and insufficient depth; (2) computational nonlinearity and redundancy of channel features; and (3) ignoring the interrelationships among feature channels. These problems lead to poor network segmentation performance and weak generalization ability. Therefore, first of all, we propose an effective replacement scheme of UNet base block, Double residual depthwise atrous convolution (DRDAC) block, to effectively improve the deficiency of receptive field and depth. Secondly, a new linear module, the Multi-scale frequency domain filter (MFDF), is designed to capture global information from the frequency domain. The high order multi-scale relationship is extracted by combining the depthwise atrous separable convolution with the frequency domain filter. Finally, a channel attention called Axial selection channel attention (ASCA) is redesigned to enhance the network's ability to model feature channel interrelationships. Further, we design a novel frequency domain medical image segmentation baseline method FDFUNet based on the above modules. We conduct extensive experiments on five publicly available medical image datasets and demonstrate that the present method has stronger segmentation performance as well as generalization ability compared to other state-of-the-art baseline methods.

Automatic thoracic aorta calcium quantification using deep learning in non-contrast ECG-gated CT images

Thoracic aorta calcium (TAC) can be assessed from cardiac computed tomography (CT) studies to improve cardiovascular risk prediction. The aim of this study was to develop a fully automatic system to detect TAC and to evaluate its performance for classifying the patients into four TAC risk categories. The method started by segmenting the thoracic aorta, combining three UNets trained with axial, sagittal and coronal CT images. Afterwards, the surrounding lesion candidates were classified using three combined convolutional neural networks (CNNs) trained with orthogonal patches. Image datasets included 1190 non-enhanced ECG-gated cardiac CT studies from a cohort of cardiovascular patients (age 57 ± 9 years, 80% men, 65% TAC > 0). In the test set (N = 119), the combination of UNets was able to successfully segment the thoracic aorta with a mean volume difference of 0.3 ± 11.7 ml (<6%) and a median Dice coefficient of 0.947. The combined CNNs accurately classified the lesion candidates and 87% of the patients (N = 104) were accurately placed in their corresponding risk categories (Kappa = 0.826, ICC = 0.9915). TAC measurement can be estimated automatically from cardiac CT images using UNets to isolate the thoracic aorta and CNNs to classify calcified lesions.

Re-UNet: a novel multi-scale reverse U-shape network architecture for low-dose CT image reconstruction

In recent years, the growing awareness of public health has brought attention to low-dose computed tomography (LDCT) scans. However, the CT image generated in this way contains a lot of noise or artifacts, which make increasing researchers to investigate methods to enhance image quality. The advancement of deep learning technology has provided researchers with novel approaches to enhance the quality of LDCT images. In the past, numerous studies based on convolutional neural networks (CNN) have yielded remarkable results in LDCT image reconstruction. Nonetheless, they all tend to continue to design new networks based on the fixed network architecture of UNet shape, which also leads to more and more complex networks. In this paper, we proposed a novel network model with a reverse U-shape architecture for the noise reduction in the LDCT image reconstruction task. In the model, we further designed a novel multi-scale feature extractor and edge enhancement module that yields a positive impact on CT images to exhibit strong structural characteristics. Evaluated on a public dataset, the experimental results demonstrate that the proposed model outperforms the compared algorithms based on traditional U-shaped architecture in terms of preserving texture details and reducing noise, as demonstrated by achieving the highest PSNR, SSIM and RMSE value. This study may shed light on the reverse U-shaped network architecture for CT image reconstruction, and could investigate the potential on other medical image processing.

A new method: Characterize and quantify biofilm wrinkles by UNet and Sholl Analysis

The wrinkles on the biofilm contain a lot of information about biofilm growth, so it is essential to characterize and quantify these wrinkles from the original microscopic images to discover more rules governing the biofilm morphology evolution. However, the existing methods to extract the wrinkles are time-consuming, error-prone, and require manual calibration. We propose a new system: using a deep learning method - UNet to identify the biofilm wrinkles in the original experimental images, which can achieve fast and accurate extraction of wrinkles on biofilms. Combining the result of UNet and medical neuron analysis method - Sholl Analysis, we can easily characterize and quantity the B. subtilis biofilm wrinkles. We proposed new characterization parameters such as wrinkle density, wrinkle length, and wrinkle projection area, which can precisely partition the biofilm surface wrinkles into different regions from the biofilm center to the edge, different regions correspond to different growth stages. Our system can be applied to study biofilms growing in different kinds of environments and to study the biofilm growth mechanisms.

ETUNet:Exploring efficient transformer enhanced UNet for 3D brain tumor segmentation

Medical image segmentation is a crucial topic in medical image processing. Accurately segmenting brain tumor regions from multimodal MRI scans is essential for clinical diagnosis and survival prediction. However, similar intensity distributions, variable tumor shapes, and fuzzy boundaries pose severe challenges for brain tumor segmentation. Traditional segmentation networks based on UNet struggle to establish explicit long-range dependencies from the feature space due to the limitations of the CNN receptive field. This is particularly crucial for dense prediction tasks such as brain tumor segmentation. Recent works have incorporated the powerful global modeling capability of Transformer into UNet to achieve more precise segmentation results. Nevertheless, these methods encounter some issues: (1) the global information is often modeled by simply stacking Transformer layers for a specific module, resulting in high computational complexity and underutilization of the potential of the UNet architecture; (2) the rich boundary information of tumor subregions in multi-scale features is often overlooked. Motivated by these challenges, we propose an advanced fusion of Transformer with UNet by reexamining the core three parts (encoder, bottleneck, and skip connections). Firstly, we introduce a CNN-Transformer module in the encoder to replace the traditional CNN module, enabling the capture of deep spatial dependencies from input images. To address high-level semantic information, we incorporate a computationally efficient spatial-channel attention layer in the bottleneck for global interaction, highlighting important semantic features from the encoder path output. For irregular lesions, we fuse the multi-scale features from the encoder output and the decoder features in the skip connections by calculating cross-attention. This adaptive querying of valuable information from multi-scale features enhances the boundary localization ability of the decoder path and suppresses redundant features with low correlation. Compared to existing methods, our model further enhances the learning capacity of the overall UNet architecture while maintaining low computational complexity. Experimental results on the BraTS2018 and BraTS2020 datasets for brain tumor segmentation tasks demonstrate that our model achieves comparable or superior results compared to recent CNN or Transformer-based models. The average DSC and HD95 on the two datasets are 0.854, 6.688, and 0.862, 5.455 respectively. At the same time, our model achieves optimal segmentation of Enhancing tumors, showcasing the effectiveness of our method. Our code will be made publicly available at https://github.com/wzhangck/ETUnet.

WATUNet: a deep neural network for segmentation of volumetric sweep imaging ultrasound

Limited access to breast cancer diagnosis globally leads to delayed treatment. Ultrasound, an effective yet underutilized method, requires specialized training for sonographers, which hinders its widespread use. Volume sweep imaging (VSI) is an innovative approach that enables untrained operators to capture high-quality ultrasound images. Combined with deep learning, like convolutional neural networks, it can potentially transform breast cancer diagnosis, enhancing accuracy, saving time and costs, and improving patient outcomes. The widely used UNet architecture, known for medical image segmentation, has limitations, such as vanishing gradients and a lack of multi-scale feature extraction and selective region attention. In this study, we present a novel segmentation model known as Wavelet_Attention_UNet (WATUNet). In this model, we incorporate wavelet gates and attention gates between the encoder and decoder instead of a simple connection to overcome the limitations mentioned, thereby improving model performance. Two datasets are utilized for the analysis: the public 'Breast Ultrasound Images' dataset of 780 images and a private VSI dataset of 3818 images, captured at the University of Rochester by the authors. Both datasets contained segmented lesions categorized into three types: no mass, benign mass, and malignant mass. Our segmentation results show superior performance compared to other deep networks. The proposed algorithm attained a Dice coefficient of 0.94 and an F1 score of 0.94 on the VSI dataset and scored 0.93 and 0.94 on the public dataset, respectively. Moreover, our model significantly outperformed other models in McNemar's test with false discovery rate correction on a 381-image VSI set. The experimental findings demonstrate that the proposed WATUNet model achieves precise segmentation of breast lesions in both standard-of-care and VSI images, surpassing state-of-the-art models. Hence, the model holds considerable promise for assisting in lesion identification, an essential step in the clinical diagnosis of breast lesions.

Deep learning-based automated kidney and cyst segmentation of autosomal dominant polycystic kidney disease using single vs. multi-institutional data

PURPOSE

This study aimed to investigate if a deep learning model trained with a single institution's data has comparable accuracy to that trained with multi-institutional data for segmenting kidney and cyst regions in magnetic resonance (MR) images of patients affected by autosomal dominant polycystic kidney disease (ADPKD).

METHODS

We used TensorFlow with a Keras custom UNet on 2D slices of 756 MRI images of kidneys with ADPKD obtained from four institutions in the Consortium for Radiologic Imaging Studies of Polycystic Kidney Disease (CRISP) study. The ground truth was determined via a manual plus global thresholding method. Five models were trained with 80 % of all institutional data (n = 604) and each institutional data (n = 232, 172, 148, or 52), respectively, and validated with 10 % and tested on an unseen 10 % of the data. The model's performance was evaluated using the Dice Similarity Coefficient (DSC).

RESULTS

The DSCs by the model trained with all institutional data ranged from 0.92 to 0.95 for kidney image segmentation, only 1-2 % higher than those by the models trained with single institutional data (0.90-0.93).In cyst segmentation, however, the DSCs by the model trained with all institutional data ranged from 0.83 to 0.89, which were 2-20 % higher than those by the models trained with single institutional data (0.66-0.86).

CONCLUSION

The UNet performance, when trained with a single institutional dataset, exhibited similar accuracy to the model trained on a multi-institutional dataset. Segmentation accuracy increases with models trained on larger sample sizes, especially in more complex cyst segmentation.

Automated wound care by employing a reliable U-Net architecture combined with ResNet feature encoders for monitoring chronic wounds

Quality of life is greatly affected by chronic wounds. It requires more intensive care than acute wounds. Schedule follow-up appointments with their doctor to track healing. Good wound treatment promotes healing and fewer problems. Wound care requires precise and reliable wound measurement to optimize patient treatment and outcomes according to evidence-based best practices. Images are used to objectively assess wound state by quantifying key healing parameters. Nevertheless, the robust segmentation of wound images is complex because of the high diversity of wound types and imaging conditions. This study proposes and evaluates a novel hybrid model developed for wound segmentation in medical images. The model combines advanced deep learning techniques with traditional image processing methods to improve the accuracy and reliability of wound segmentation. The main objective is to overcome the limitations of existing segmentation methods (UNet) by leveraging the combined advantages of both paradigms. In our investigation, we introduced a hybrid model architecture, wherein a ResNet34 is utilized as the encoder, and a UNet is employed as the decoder. The combination of ResNet34's deep representation learning and UNet's efficient feature extraction yields notable benefits. The architectural design successfully integrated high-level and low-level features, enabling the generation of segmentation maps with high precision and accuracy. Following the implementation of our model to the actual data, we were able to determine the following values for the Intersection over Union (IOU), Dice score, and accuracy: 0.973, 0.986, and 0.9736, respectively. According to the achieved results, the proposed method is more precise and accurate than the current state-of-the-art.

Enhancing COVID-19 CT Image Segmentation: A Comparative Study of Attention and Recurrence in UNet Models

Imaging plays a key role in the clinical management of Coronavirus disease 2019 (COVID-19) as the imaging findings reflect the pathological process in the lungs. The visual analysis of High-Resolution Computed Tomography of the chest allows for the differentiation of parenchymal abnormalities of COVID-19, which are crucial to be detected and quantified in order to obtain an accurate disease stratification and prognosis. However, visual assessment and quantification represent a time-consuming task for radiologists. In this regard, tools for semi-automatic segmentation, such as those based on Convolutional Neural Networks, can facilitate the detection of pathological lesions by delineating their contour. In this work, we compared four state-of-the-art Convolutional Neural Networks based on the encoder-decoder paradigm for the binary segmentation of COVID-19 infections after training and testing them on 90 HRCT volumetric scans of patients diagnosed with COVID-19 collected from the database of the Pisa University Hospital. More precisely, we started from a basic model, the well-known UNet, then we added an attention mechanism to obtain an Attention-UNet, and finally we employed a recurrence paradigm to create a Recurrent-Residual UNet (R2-UNet). In the latter case, we also added attention gates to the decoding path of an R2-UNet, thus designing an R2-Attention UNet so as to make the feature representation and accumulation more effective. We compared them to gain understanding of both the cognitive mechanism that can lead a neural model to the best performance for this task and the good compromise between the amount of data, time, and computational resources required. We set up a five-fold cross-validation and assessed the strengths and limitations of these models by evaluating the performances in terms of Dice score, Precision, and Recall defined both on 2D images and on the entire 3D volume. From the results of the analysis, it can be concluded that Attention-UNet outperforms the other models by achieving the best performance of 81.93%, in terms of 2D Dice score, on the test set. Additionally, we conducted statistical analysis to assess the performance differences among the models. Our findings suggest that integrating the recurrence mechanism within the UNet architecture leads to a decline in the model's effectiveness for our particular application.

Combined Edge Loss UNet for Optimized Segmentation in Total Knee Arthroplasty Preoperative Planning

Bone segmentation and 3D reconstruction are crucial for total knee arthroplasty (TKA) surgical planning with Personalized Surgical Instruments (PSIs). Traditional semi-automatic approaches are time-consuming and operator-dependent, although they provide reliable outcomes. Moreover, the recent expansion of artificial intelligence (AI) tools towards various medical domains is transforming modern healthcare. Accordingly, this study introduces an automated AI-based pipeline to replace the current operator-based tibia and femur 3D reconstruction procedure enhancing TKA preoperative planning. Leveraging an 822 CT image dataset, a novel patch-based method and an improved segmentation label generation algorithm were coupled to a Combined Edge Loss UNet (CEL-UNet), a novel CNN architecture featuring an additional decoding branch to boost the bone boundary segmentation. Root Mean Squared Errors and Hausdorff distances compared the predicted surfaces to the reference bones showing median and interquartile values of 0.26 (0.19-0.36) mm and 0.24 (0.18-0.32) mm, and of 1.06 (0.73-2.15) mm and 1.43 (0.82-2.86) mm for the tibia and femur, respectively, outperforming previous results of our group, state-of-the-art, and UNet models. A feasibility analysis for a PSI-based surgical plan revealed sub-millimetric distance errors and sub-angular alignment uncertainties in the PSI contact areas and the two cutting planes. Finally, operational environment testing underscored the pipeline's efficiency. More than half of the processed cases complied with the PSI prototyping requirements, reducing the overall time from 35 min to 13.1 s, while the remaining ones underwent a manual refinement step to achieve such PSI requirements, performing the procedure four to eleven times faster than the manufacturer standards. To conclude, this research advocates the need for real-world applicability and optimization of AI solutions in orthopedic surgical practice.

Deep Learning Paradigm and Its Bias for Coronary Artery Wall Segmentation in Intravascular Ultrasound Scans: A Closer Look

BACKGROUND AND MOTIVATION

Coronary artery disease (CAD) has the highest mortality rate; therefore, its diagnosis is vital. Intravascular ultrasound (IVUS) is a high-resolution imaging solution that can image coronary arteries, but the diagnosis software via wall segmentation and quantification has been evolving. In this study, a deep learning (DL) paradigm was explored along with its bias.

METHODS

Using a PRISMA model, 145 best UNet-based and non-UNet-based methods for wall segmentation were selected and analyzed for their characteristics and scientific and clinical validation. This study computed the coronary wall thickness by estimating the inner and outer borders of the coronary artery IVUS cross-sectional scans. Further, the review explored the bias in the DL system for the first time when it comes to wall segmentation in IVUS scans. Three bias methods, namely (i) ranking, (ii) radial, and (iii) regional area, were applied and compared using a Venn diagram. Finally, the study presented explainable AI (XAI) paradigms in the DL framework.

FINDINGS AND CONCLUSIONS

UNet provides a powerful paradigm for the segmentation of coronary walls in IVUS scans due to its ability to extract automated features at different scales in encoders, reconstruct the segmented image using decoders, and embed the variants in skip connections. Most of the research was hampered by a lack of motivation for XAI and pruned AI (PAI) models. None of the UNet models met the criteria for bias-free design. For clinical assessment and settings, it is necessary to move from a paper-to-practice approach.

A Novel Deep Learning Model for Medical Image Segmentation with Convolutional Neural Network and Transformer

Accurate segmentation of medical images is essential for clinical decision-making, and deep learning techniques have shown remarkable results in this area. However, existing segmentation models that combine transformer and convolutional neural networks often use skip connections in U-shaped networks, which may limit their ability to capture contextual information in medical images. To address this limitation, we propose a coordinated mobile and residual transformer UNet (MRC-TransUNet) that combines the strengths of transformer and UNet architectures. Our approach uses a lightweight MR-ViT to address the semantic gap and a reciprocal attention module to compensate for the potential loss of details. To better explore long-range contextual information, we use skip connections only in the first layer and add MR-ViT and RPA modules in the subsequent downsampling layers. In our study, we evaluated the effectiveness of our proposed method on three different medical image segmentation datasets, namely, breast, brain, and lung. Our proposed method outperformed state-of-the-art methods in terms of various evaluation metrics, including the Dice coefficient and Hausdorff distance. These results demonstrate that our proposed method can significantly improve the accuracy of medical image segmentation and has the potential for clinical applications. Illustration of the proposed MRC-TransUNet. For the input medical images, we first subject them to an intrinsic downsampling operation and then replace the original jump connection structure using MR-ViT. The output feature representations at different scales are fused by the RPA module. Finally, an upsampling operation is performed to fuse the features to restore them to the same resolution as the input image.

Deep learning-assisted IoMT framework for cerebral microbleed detection

The Internet of Things (IoT), big data, and artificial intelligence (AI) are all key technologies that influence the formation and implementation of digital medical services. Building Internet of Medical Things (IoMT) systems that combine advanced sensors with AI-powered insights is critical for intelligent medical systems. This paper presents an IoMT framework for brain magnetic resonance imaging (MRI) analysis to lessen the unavoidable diagnosis and therapy faults that occur in human clinical settings for the accurate detection of cerebral microbleeds (CMBs). The problems in accurate CMB detection include that CMBs are tiny dots 5-10 mm in diameter; they are similar to healthy tissues and are exceedingly difficult to identify, necessitating specialist guidance in remote and underdeveloped medical centers. Secondly, in the existing studies, computer-aided diagnostic (CAD) systems are designed for accurate CMB detection, however, their proposed approaches consist of two stages. Potential candidate CMBs from the complete MRI image are selected in the first stage and then passed to the phase of false-positive reduction. These pre-and post-processing steps make it difficult to build a completely automated CAD system for CMB that can produce results without human intervention. Hence, as a key goal of this work, an end-to-end enhanced UNet-based model for effective CMB detection and segmentation for IoMT devices is proposed. The proposed system requires no pre-processing or post-processing steps for CMB segmentation, and no existing research localizes each CMB pixel from the complete MRI image input. The findings indicate that the suggested method outperforms in detecting CMBs in the presence of contrast variations and similarities with other normal tissues and yields a good dice score of 0.70, an accuracy of 99 %, as well as a false-positive rate of 0.002 %. © 2017 Elsevier Inc. All rights reserved.

Learning to deep learning: statistics and a paradigm test in selecting a UNet architecture to enhance MRI

OBJECTIVE

This study aims to assess the statistical significance of training parameters in 240 dense UNets (DUNets) used for enhancing low Signal-to-Noise Ratio (SNR) and undersampled MRI in various acquisition protocols. The objective is to determine the validity of differences between different DUNet configurations and their impact on image quality metrics.

MATERIALS AND METHODS

To achieve this, we trained all DUNets using the same learning rate and number of epochs, with variations in 5 acquisition protocols, 24 loss function weightings, and 2 ground truths. We calculated evaluation metrics for two metric regions of interest (ROI). We employed both Analysis of Variance (ANOVA) and Mixed Effects Model (MEM) to assess the statistical significance of the independent parameters, aiming to compare their efficacy in revealing differences and interactions among fixed parameters.

RESULTS

ANOVA analysis showed that, except for the acquisition protocol, fixed variables were statistically insignificant. In contrast, MEM analysis revealed that all fixed parameters and their interactions held statistical significance. This emphasizes the need for advanced statistical analysis in comparative studies, where MEM can uncover finer distinctions often overlooked by ANOVA.

DISCUSSION

These findings highlight the importance of utilizing appropriate statistical analysis when comparing different deep learning models. Additionally, the surprising effectiveness of the UNet architecture in enhancing various acquisition protocols underscores the potential for developing improved methods for characterizing and training deep learning models. This study serves as a stepping stone toward enhancing the transparency and comparability of deep learning techniques for medical imaging applications.

Improved UNet with Attention for Medical Image Segmentation

Medical image segmentation is crucial for medical image processing and the development of computer-aided diagnostics. In recent years, deep Convolutional Neural Networks (CNNs) have been widely adopted for medical image segmentation and have achieved significant success. UNet, which is based on CNNs, is the mainstream method used for medical image segmentation. However, its performance suffers owing to its inability to capture long-range dependencies. Transformers were initially designed for Natural Language Processing (NLP), and sequence-to-sequence applications have demonstrated the ability to capture long-range dependencies. However, their abilities to acquire local information are limited. Hybrid architectures of CNNs and Transformer, such as TransUNet, have been proposed to benefit from Transformer's long-range dependencies and CNNs' low-level details. Nevertheless, automatic medical image segmentation remains a challenging task due to factors such as blurred boundaries, the low-contrast tissue environment, and in the context of ultrasound, issues like speckle noise and attenuation. In this paper, we propose a new model that combines the strengths of both CNNs and Transformer, with network architectural improvements designed to enrich the feature representation captured by the skip connections and the decoder. To this end, we devised a new attention module called Three-Level Attention (TLA). This module is composed of an Attention Gate (AG), channel attention, and spatial normalization mechanism. The AG preserves structural information, whereas channel attention helps to model the interdependencies between channels. Spatial normalization employs the spatial coefficient of the Transformer to improve spatial attention akin to TransNorm. To further improve the skip connection and reduce the semantic gap, skip connections between the encoder and decoder were redesigned in a manner similar to that of the UNet++ dense connection. Moreover, deep supervision using a side-output channel was introduced, analogous to BASNet, which was originally used for saliency predictions. Two datasets from different modalities, a CT scan dataset and an ultrasound dataset, were used to evaluate the proposed UNet architecture. The experimental results showed that our model consistently improved the prediction performance of the UNet across different datasets.

Automatic Identification of Glomerular in Whole-Slide Images Using a Modified UNet Model

Glomeruli are interconnected capillaries in the renal cortex that are responsible for blood filtration. Damage to these glomeruli often signifies the presence of kidney disorders like glomerulonephritis and glomerulosclerosis, which can ultimately lead to chronic kidney disease and kidney failure. The timely detection of such conditions is essential for effective treatment. This paper proposes a modified UNet model to accurately detect glomeruli in whole-slide images of kidney tissue. The UNet model was modified by changing the number of filters and feature map dimensions from the first to the last layer to enhance the model's capacity for feature extraction. Moreover, the depth of the UNet model was also improved by adding one more convolution block to both the encoder and decoder sections. The dataset used in the study comprised 20 large whole-side images. Due to their large size, the images were cropped into 512 × 512-pixel patches, resulting in a dataset comprising 50,486 images. The proposed model performed well, with 95.7% accuracy, 97.2% precision, 96.4% recall, and 96.7% F1-score. These results demonstrate the proposed model's superior performance compared to the original UNet model, the UNet model with EfficientNetb3, and the current state-of-the-art. Based on these experimental findings, it has been determined that the proposed model accurately identifies glomeruli in extracted kidney patches.

Generating synthetic images from cone beam computed tomography using self-attention residual UNet for head and neck radiotherapy

Background and purpose

Accurate CT numbers in Cone Beam CT (CBCT) are crucial for precise dose calculations in adaptive radiotherapy (ART). This study aimed to generate synthetic CT (sCT) from CBCT using deep learning (DL) models in head and neck (HN) radiotherapy.

Materials and methods

A novel DL model, the 'self-attention-residual-UNet' (ResUNet), was developed for accurate sCT generation. ResUNet incorporates a self-attention mechanism in its long skip connections to enhance information transfer between the encoder and decoder. Data from 93 HN patients, each with planning CT (pCT) and first-day CBCT images were used. Model performance was evaluated using two DL approaches (non-adversarial and adversarial training) and two model types (2D axial only vs. 2.5D axial, sagittal, and coronal). ResUNet was compared with the traditional UNet through image quality assessment (Mean Absolute Error (MAE), Peak-Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM)) and dose calculation accuracy evaluation (DVH deviation and gamma evaluation (1 %/1mm)).

Results

Image similarity evaluation results for the 2.5D-ResUNet and 2.5D-UNet models were: MAE: 46±7 HU vs. 51±9 HU, PSNR: 66.6±2.0 dB vs. 65.8±1.8 dB, and SSIM: 0.81±0.04 vs. 0.79±0.05. There were no significant differences in dose calculation accuracy between DL models. Both models demonstrated DVH deviation below 0.5 % and a gamma-pass-rate (1 %/1mm) exceeding 97 %.

Conclusions

ResUNet enhanced CT number accuracy and image quality of sCT and outperformed UNet in sCT generation from CBCT. This method holds promise for generating precise sCT for HN ART.

A New Semantic Segmentation Framework Based on UNet

This paper discusses a semantic segmentation framework and shows its application in agricultural intelligence, such as providing environmental awareness for agricultural robots to work autonomously and efficiently. We propose an ensemble framework based on the bagging strategy and the UNet network, using RGB and HSV color spaces. We evaluated the framework on our self-built dataset (Maize) and a public dataset (Sugar Beets). Then, we compared it with UNet-based methods (single RGB and single HSV), DeepLab V3+, and SegNet. Experimental results show that our ensemble framework can synthesize the advantages of each color space and obtain the best IoUs (0.8276 and 0.6972) on the datasets (Maize and Sugar Beets), respectively. In addition, including our framework, the UNet-based methods have faster speed and a smaller parameter space than DeepLab V3+ and SegNet, which are more suitable for deployment in resource-constrained environments such as mobile robots.

A multi-channel UNet framework based on SNMF-DCNN for robust heart-lung-sound separation

Cardiopulmonary and cardiovascular diseases are fatal factors that threaten human health and cause many deaths worldwide each year, so it is essential to screen cardiopulmonary disease more accurately and efficiently. Auscultation is a non-invasive method for physicians' perception of the disease. The Heart Sounds (HS) and Lung Sounds (LS) recorded by an electronic stethoscope consist of acoustic information that is helpful in the diagnosis of pulmonary conditions. Still, inter-interference between HS and LS presented in both the time and frequency domains blocks diagnostic efficiency. This paper proposes a blind source separation （BSS）strategy that first classifies Heart-Lung-Sound (HLS) according to its LS features and then separates it into HS and LS. Sparse Non-negative Matrix Factorization (SNMF) is employed to extract the LS features in HLS, then proposed a network constructed by Dilated Convolutional Neural Network (DCNN) to classify HLS into five types by the magnitude features of LS. Finally, Multi-Channel UNet (MCUNet) separation model is utilized for each category of HLS. This paper is the first to propose the HLS classification method SNMF-DCNN and apply UNet to the cardiopulmonary sound separation domain. Compared with other state-of-the-art methods, the proposed framework in this paper has higher separation quality and robustness.

GCNfold: A novel lightweight model with valid extractors for RNA secondary structure prediction

RNA secondary structure is essential for predicting the tertiary structure and understanding RNA function. Recent research tends to stack numerous modules to design large deep-learning models. This can increase the accuracy to more than 70%, as well as significant training costs and prediction efficiency. We proposed a model with three feature extractors called GCNfold. Structure Extractor utilizes a three-layer Graph Convolutional Network (GCN) to mine the structural information of RNA, such as stems, hairpin, and internal loops. Structure and Sequence Fusion embeds structural information into sequences with Transformer Encoders. Long-distance Dependency Extractor captures long-range pairwise relationships by UNet. The experiments indicate that GCNfold has a small number of parameters, a fast inference speed, and a high accuracy among all models with over 80% accuracy. Additionally, GCNfold-Small takes only 90ms to infer an RNA secondary structure and can achieve close to 90% accuracy on average. The GCNfold code is available on Github https://github.com/EnbinYang/GCNfold.

A Residual UNet Denoising Network Based on Multi-Scale Feature Extraction and Attention-Guided Filter

In order to obtain high-quality images, it is very important to remove noise effectively and retain image details reasonably. In this paper, we propose a residual UNet denoising network that adds the attention-guided filter and multi-scale feature extraction blocks. We design a multi-scale feature extraction block as the input block to expand the receiving domain and extract more useful features. We also develop the attention-guided filter block to hold the edge information. Further, we use the global residual network strategy to model residual noise instead of directly modeling clean images. Experimental results show our proposed network performs favorably against several state-of-the-art models. Our proposed model can not only suppress the noise more effectively, but also improve the sharpness of the image.

Skin lesion image segmentation based on lightweight multi-scale U-shaped network

UNet, and more recently medical image segmentation methods, utilize many parameters and computational quantities to achieve higher performance. However, due to the increasing demand for real-time medical image segmentation tasks, it is important to trade between accuracy rates and computational complexity. To this end, we propose a lightweight multi-scale U-shaped network (LMUNet), a multi-scale inverted residual and an asymmetric atrous spatial pyramid pooling-based network for skin lesion image segmentation. We test LMUNet on multiple medical image segmentation datasets, which show that it reduces the number of parameters by 67X and decreases the computational complexity by 48X while obtaining better performance over the partial lightweight networks.

Enhancing Prediction of Brain Tumor Classification Using Images and Numerical Data Features

Brain tumors, along with other diseases that harm the neurological system, are a significant contributor to global mortality. Early diagnosis plays a crucial role in effectively treating brain tumors. To distinguish individuals with tumors from those without, this study employs a combination of images and data-based features. In the initial phase, the image dataset is enhanced, followed by the application of a UNet transfer-learning-based model to accurately classify patients as either having tumors or being normal. In the second phase, this research utilizes 13 features in conjunction with a voting classifier. The voting classifier incorporates features extracted from deep convolutional layers and combines stochastic gradient descent with logistic regression to achieve better classification results. The reported accuracy score of 0.99 achieved by both proposed models shows its superior performance. Also, comparing results with other supervised learning algorithms and state-of-the-art models validates its performance.

Enhancing Breast Ultrasound Segmentation through Fine-tuning and Optimization Techniques: Sharp Attention UNet

Segmentation of breast ultrasound images is a crucial and challenging task in computer-aided diagnosis systems. Accurately segmenting masses in benign and malignant cases and identifying regions with no mass is a primary objective in breast ultrasound image segmentation. Deep learning (DL) has emerged as a powerful tool in medical image segmentation, revolutionizing how medical professionals analyze and interpret complex imaging data. The UNet architecture is a highly regarded and widely used DL model in medical image segmentation. Its distinctive architectural design and exceptional performance have made it a popular choice among researchers in the medical image segmentation field. With the increase in data and model complexity, optimization and fine-tuning models play a vital and more challenging role than before. This paper presents a comparative study evaluating the effect of image preprocessing and different optimization techniques and the importance of fine-tuning different UNet segmentation models for breast ultrasound images. Optimization and fine-tuning techniques have been applied to enhance the performance of UNet, Sharp UNet, and Attention UNet. Building upon this progress, we designed a novel approach by combining Sharp UNet and Attention UNet, known as Sharp Attention UNet. Our analysis yielded the following quantitative evaluation metrics for the Sharp Attention UNet: the dice coefficient, specificity, sensitivity, and F1 score obtained values of 0.9283, 0.9936, 0.9426, and 0.9412, respectively. In addition, McNemar's statistical test was applied to assess significant differences between the approaches. Across a number of measures, our proposed model outperforms the earlier designed models and points towards improved breast lesion segmentation algorithms.

Performance Analysis of Segmentation and Classification of CT-Scanned Ovarian Tumours Using U-Net and Deep Convolutional Neural Networks

Difficulty in detecting tumours in early stages is the major cause of mortalities in patients, despite the advancements in treatment and research regarding ovarian cancer. Deep learning algorithms were applied to serve the purpose as a diagnostic tool and applied to CT scan images of the ovarian region. The images went through a series of pre-processing techniques and, further, the tumour was segmented using the UNet model. The instances were then classified into two categories-benign and malignant tumours. Classification was performed using deep learning models like CNN, ResNet, DenseNet, Inception-ResNet, VGG16 and Xception, along with machine learning models such as Random Forest, Gradient Boosting, AdaBoosting and XGBoosting. DenseNet 121 emerges as the best model on this dataset after applying optimization on the machine learning models by obtaining an accuracy of 95.7%. The current work demonstrates the comparison of multiple CNN architectures with common machine learning algorithms, with and without optimization techniques applied.

Dairy Cow Mastitis Detection by Thermal Infrared Images Based on CLE-UNet

Thermal infrared technology is utilized for detecting mastitis in cows owing to its non-invasive and efficient characteristics. However, the presence of surrounding regions and obstacles can impede accurate temperature measurement, thereby compromising the effectiveness of dairy mastitis detection. To address these problems, we proposed the CLE-UNet (Centroid Loss Ellipticization UNet) semantic segmentation algorithm. The algorithm consists of three main parts. Firstly, we introduced the efficient channel attention (ECA) mechanism in the feature extraction layer of UNet to improve the segmentation accuracy by focusing on more useful channel features. Secondly, we proposed a new centroid loss function to facilitate the network's output to be closer to the position of the real label during the training process. Finally, we used a cow's eye ellipse fitting operation based on the similarity between the shape of the cow's eye and the ellipse. The results indicated that the CLE-UNet model obtained a mean intersection over union (MIoU) of 89.32% and an average segmentation speed of 0.049 s per frame. Compared to somatic cell count (SCC), this method achieved an accuracy, sensitivity, and F1 value of 86.67%, 82.35%, and 87.5%, respectively, for detecting mastitis in dairy cows. In conclusion, the innovative use of the CLE-UNet algorithm has significantly improved the segmentation accuracy and has proven to be an effective tool for accurately detecting cow mastitis.

LcmUNet: A Lightweight Network Combining CNN and MLP for Real-Time Medical Image Segmentation

In recent years, UNet and its improved variants have become the main methods for medical image segmentation. Although these models have achieved excellent results in segmentation accuracy, their large number of network parameters and high computational complexity make it difficult to achieve medical image segmentation in real-time therapy and diagnosis rapidly. To address this problem, we introduce a lightweight medical image segmentation network (LcmUNet) based on CNN and MLP. We designed LcmUNet's structure in terms of model performance, parameters, and computational complexity. The first three layers are convolutional layers, and the last two layers are MLP layers. In the convolution part, we propose an LDA module that combines asymmetric convolution, depth-wise separable convolution, and an attention mechanism to reduce the number of network parameters while maintaining a strong feature-extraction capability. In the MLP part, we propose an LMLP module that helps enhance contextual information while focusing on local information and improves segmentation accuracy while maintaining high inference speed. This network also covers skip connections between the encoder and decoder at various levels. Our network achieves real-time segmentation results accurately in extensive experiments. With only 1.49 million model parameters and without pre-training, LcmUNet demonstrated impressive performance on different datasets. On the ISIC2018 dataset, it achieved an IoU of 85.19%, 92.07% recall, and 92.99% precision. On the BUSI dataset, it achieved an IoU of 63.99%, 79.96% recall, and 76.69% precision. Lastly, on the Kvasir-SEG dataset, LcmUNet achieved an IoU of 81.89%, 88.93% recall, and 91.79% precision.

CNN-based automatic segmentations and radiomics feature reliability on contrast-enhanced ultrasound images for renal tumors

Objective

To investigate the feasibility and efficiency of automatic segmentation of contrast-enhanced ultrasound (CEUS) images in renal tumors by convolutional neural network (CNN) based models and their further application in radiomic analysis.

Materials and methods

From 94 pathologically confirmed renal tumor cases, 3355 CEUS images were extracted and randomly divided into training set (3020 images) and test set (335 images). According to the histological subtypes of renal cell carcinoma, the test set was further split into clear cell renal cell carcinoma (ccRCC) set (225 images), renal angiomyolipoma (AML) set (77 images) and set of other subtypes (33 images). Manual segmentation was the gold standard and serves as ground truth. Seven CNN-based models including DeepLabV3+, UNet, UNet++, UNet3+, SegNet, MultilResUNet and Attention UNet were used for automatic segmentation. Python 3.7.0 and Pyradiomics package 3.0.1 were used for radiomic feature extraction. Performance of all approaches was evaluated by the metrics of mean intersection over union (mIOU), dice similarity coefficient (DSC), precision, and recall. Reliability and reproducibility of radiomics features were evaluated by the Pearson coefficient and the intraclass correlation coefficient (ICC).

Results

All seven CNN-based models achieved good performance with the mIOU, DSC, precision and recall ranging between 81.97%-93.04%, 78.67%-92.70%, 93.92%-97.56%, and 85.29%-95.17%, respectively. The average Pearson coefficients ranged from 0.81 to 0.95, and the average ICCs ranged from 0.77 to 0.92. The UNet++ model showed the best performance with the mIOU, DSC, precision and recall of 93.04%, 92.70%, 97.43% and 95.17%, respectively. For ccRCC, AML and other subtypes, the reliability and reproducibility of radiomic analysis derived from automatically segmented CEUS images were excellent, with the average Pearson coefficients of 0.95, 0.96 and 0.96, and the average ICCs for different subtypes were 0.91, 0.93 and 0.94, respectively.

Conclusion

This retrospective single-center study showed that the CNN-based models had good performance on automatic segmentation of CEUS images for renal tumors, especially the UNet++ model. The radiomics features extracted from automatically segmented CEUS images were feasible and reliable, and further validation by multi-center research is necessary.

Accurate segmentation algorithm of acoustic neuroma in the cerebellopontine angle based on ACP-TransUNet

Acoustic neuroma is one of the most common tumors in the cerebellopontine angle area. Patients with acoustic neuroma have clinical manifestations of the cerebellopontine angle occupying syndrome, such as tinnitus, hearing impairment and even hearing loss. Acoustic neuromas often grow in the internal auditory canal. Neurosurgeons need to observe the lesion contour with the help of MRI images, which not only takes a lot of time, but also is easily affected by subjective factors. Therefore, the automatic and accurate segmentation of acoustic neuroma in cerebellopontine angle on MRI is of great significance for surgical treatment and expected rehabilitation. In this paper, an automatic segmentation method based on Transformer is proposed, using TransUNet as the core model. As some acoustic neuromas are irregular in shape and grow into the internal auditory canal, larger receptive fields are thus needed to synthesize the features. Therefore, we added Atrous Spatial Pyramid Pooling to CNN, which can obtain a larger receptive field without losing too much resolution. Since acoustic neuromas often occur in the cerebellopontine angle area with relatively fixed position, we combined channel attention with pixel attention in the up-sampling stage so as to make our model automatically learn different weights by adding the attention mechanism. In addition, we collected 300 MRI sequence nuclear resonance images of patients with acoustic neuromas in Tianjin Huanhu hospital for training and verification. The ablation experimental results show that the proposed method is reasonable and effective. The comparative experimental results show that the Dice and Hausdorff 95 metrics of the proposed method reach 95.74% and 1.9476 mm respectively, indicating that it is not only superior to the classical models such as UNet, PANet, PSPNet, UNet++, and DeepLabv3, but also show better performance than the newly-proposed SOTA (state-of-the-art) models such as CCNet, MANet, BiseNetv2, Swin-Unet, MedT, TransUNet, and UCTransNet.

MAS-UNet: a U-shaped network for prostate segmentation

Prostate cancer is a common disease that seriously endangers the health of middle-aged and elderly men. MRI images are the gold standard for assessing the health status of the prostate region. Segmentation of the prostate region is of great significance for the diagnosis of prostate cancer. In the past, some methods have been used to segment the prostate region, but segmentation accuracy still has room for improvement. This study has proposed a new image segmentation model based on Attention UNet. The model improves Attention UNet by using GN instead of BN, adding dropout to prevent overfitting, introducing the ASPP module, adding channel attention to the attention gate module, and using different channels to output segmentation results of different prostate regions. Finally, we conducted comparative experiments using five existing UNet-based models, and used the dice coefficient as the metric to evaluate the segmentation result. The proposed model achieves dice scores of 0.807 and 0.907 in the transition region and the peripheral region, respectively. The experimental results show that the proposed model is better than other UNet-based models.

Machine-learning model to delineate sub-surface agricultural drainage from satellite imagery

Knowing sub-surface drainage (tile-drain) extent is integral to understanding how landscapes respond to precipitation events and subsequent days of drying, as well as how soil characteristics and land management influence stream response. Consequently, a time-series of tile-drain extent would inform one aspect of land management that complicates our ability to explain streamflow and water-quality as a function of climate variability or conservation management. We trained a UNet machine-learning model, a convolutional neural network designed to highlight objects of interest within an image, to delineate tile-drain networks in panchromatic satellite imagery without additional data on soils, topography, or historical tile-drain extent. This was done by training the model to match the accuracy of human experts manually tracing the surface representation of tile drains in satellite imagery. Our approach began with a library of images that were used to train and quantify the accuracy of the model, with model performance tested on imagery from two areas that were not used to train the model. Satellite imagery included acquisition dates from 2008-2020. Training imagery were from agricultural areas within the U.S. Great Lakes basin. Validation imagery were from the upper Maumee River, tributary to western Lake Erie, and an Indiana, Ohio-River headwater tributary. Our analysis of the satellite imagery paired with meteorological and soil data found that during spring, a combination of relatively high solar radiation, intermediate soil-water content and bare fields enabled the best model performance. Each area of interest was heavily tile-drained, where better understanding the movement of water, nutrients, and sediment from fields to downstream water bodies is key to managing harmful algal blooms and hypoxia. The trained UNet model successfully identified tile drains visible in the validation imagery with an accuracy of 93-96% and balanced accuracy of 52-54%, similar to performance for training data (95% and 63%, respectively). Model performance will benefit from ongoing contributions to the training library. This article is protected by copyright. All rights reserved.

Single-Channel Blind Image Separation Based on Transformer-Guided GAN

Blind source separation (BSS) has been a great challenge in the field of signal processing due to the unknown distribution of the source signal and the mixing matrix. Traditional methods based on statistics and information theory use prior information such as source distribution independence, non-Gaussianity, sparsity, etc. to solve this problem. Generative adversarial networks (GANs) learn source distributions through games without being constrained by statistical properties. However, the current blind image separation methods based on GANs ignores the reconstruction of the structure and details of the separated image, resulting in residual interference source information in the generated results. This paper proposes a Transformer-guided GAN guided by an attention mechanism. Through the adversarial training of the generator and the discriminator, U-shaped Network (UNet) is used to fuse the convolutional layer features to reconstruct the structure of the separated image, and Transformer is used to calculate the position attention and guide the detailed information. We validate our method with quantitative experiments, showing that it outperforms previous blind image separation algorithms in terms of PSNR and SSIM.

Sk-Conv and SPP-based UNet for lesion segmentation of coronary optical coherence tomography

BACKGROUND

Coronary artery disease (CAD) manifests with a blockage the coronary arteries, usually due to plaque buildup, and has a serious impact on the human life. Atherosclerotic plaques, including fibrous plaques, lipid plaques, and calcified plaques can lead to occurrence of CAD. Optical coherence tomography (OCT) is employed in the clinical practice as it clearly provides a detailed display of the lesion plaques, thereby assessing the patient's condition. Analyzing the OCT images manually is a very tedious and time-consuming task for the clinicians. Therefore, automatic segmentation of the coronary OCT images is necessary.

OBJECTIVE

In view of the good utility of Unet network in the segmentation of medical images, the present study proposed the development of a Unet network based on Sk-Conv and spatial pyramid pooling modules to segment the coronary OCT images.

METHODS

In order to extract multi-scale features, these two modules were added at the bottom of UNet. Meanwhile, ablation experiments are designed to verify each module is effective.

RESULTS

After testing, our model achieves 0.8935 on f1 score and 0.7497 on mIOU. Compared to the current advanced models, our model performs better.

CONCLUSION

Our model achieves good results on OCT sequences.

Neural Networks Application for Accurate Retina Vessel Segmentation from OCT Fundus Reconstruction

The use of neural networks for retinal vessel segmentation has gained significant attention in recent years. Most of the research related to the segmentation of retinal blood vessels is based on fundus images. In this study, we examine five neural network architectures to accurately segment vessels in fundus images reconstructed from 3D OCT scan data. OCT-based fundus reconstructions are of much lower quality compared to color fundus photographs due to noise and lower and disproportionate resolutions. The fundus image reconstruction process was performed based on the segmentation of the retinal layers in B-scans. Three reconstruction variants were proposed, which were then used in the process of detecting blood vessels using neural networks. We evaluated performance using a custom dataset of 24 3D OCT scans (with manual annotations performed by an ophthalmologist) using 6-fold cross-validation and demonstrated segmentation accuracy up to 98%. Our results indicate that the use of neural networks is a promising approach to segmenting the retinal vessel from a properly reconstructed fundus.

C2BA-UNet: A context-coordination multi-atlas boundary-aware UNet-like method for PET/CT images based tumor segmentation

Tumor segmentation is a necessary step in clinical processing that can help doctors diagnose tumors and plan surgical treatments. Since tumors are usually small, the locations and appearances vary substantially across individuals, and the contrast between tumors and adjacent normal tissues is low, tumor segmentation is still a challenging task. Although convolutional neural networks (CNNs) have achieved good results in tumor segmentation, the information about tumor boundaries has been rarely explored. To solve the problem, this paper proposes a new method for automatic tumor segmentation in PET/CT images based on context-coordination and boundary-aware, termed as C2BA-UNet. We employ a UNet-like backbone network and replace the encoder with EfficientNet-B0 for efficiency. To acquire potential tumor boundaries, we propose a new multi-atlas boundary-aware (MABA) module based on gradient atlas, uncertainty atlas, and level set atlas, that focuses on uncertain regions between tumors and adjacent tissues. Furthermore, we propose a new context coordination module (CCM) to combine multi-scale context information with attention mechanism to optimize skip connection in high-level layers. To validate the superiority of our method, we conduct experiments on a publicly available soft tissue sarcoma (STS) dataset and a lymphoma dataset, and the results show our method is competitive with other comparison methods.

RAD-UNet: a Residual, Attention-Based, Dense UNet for CT Sparse Reconstruction

To suppress the streak artifacts in images reconstructed from sparse-view projections in computed tomography (CT), a residual, attention-based, dense UNet (RAD-UNet) deep network is proposed to achieve accurate sparse reconstruction. The filtered back projection (FBP) algorithm is used to reconstruct the CT image with streak artifacts from sparse-view projections. Then, the image is processed by the RAD-UNet to suppress streak artifacts and obtain high-quality CT image. Those images with streak artifacts are used as the input of the RAD-UNet, and the output-label images are the corresponding high-quality images. Through training via the large-scale training data, the RAD-UNet can obtain the capability of suppressing streak artifacts. This network combines residual connection, attention mechanism, dense connection and perceptual loss. This network can improve the nonlinear fitting capability and the performance of suppressing streak artifacts. The experimental results show that the RAD-UNet can improve the reconstruction accuracy compared with three existing representative deep networks. It may not only suppress streak artifacts but also better preserve image details. The proposed networks may be readily applied to other image processing tasks including image denoising, image deblurring, and image super-resolution.

TDD-UNet:Transformer with double decoder UNet for COVID-19 lesions segmentation

The outbreak of new coronary pneumonia has brought severe health risks to the world. Detection of COVID-19 based on the UNet network has attracted widespread attention in medical image segmentation. However, the traditional UNet model is challenging to capture the long-range dependence of the image due to the limitations of the convolution kernel with a fixed receptive field. The Transformer Encoder overcomes the long-range dependence problem. However, the Transformer-based segmentation approach cannot effectively capture the fine-grained details. We propose a transformer with a double decoder UNet for COVID-19 lesions segmentation to address this challenge, TDD-UNet. We introduce the multi-head self-attention of the Transformer to the UNet encoding layer to extract global context information. The dual decoder structure is used to improve the result of foreground segmentation by predicting the background and applying deep supervision. We performed quantitative analysis and comparison for our proposed method on four public datasets with different modalities, including CT and CXR, to demonstrate its effectiveness and generality in segmenting COVID-19 lesions. We also performed ablation studies on the COVID-19-CT-505 dataset to verify the effectiveness of the key components of our proposed model. The proposed TDD-UNet also achieves higher Dice and Jaccard mean scores and the lowest standard deviation compared to competitors. Our proposed method achieves better segmentation results than other state-of-the-art methods.

PSR-Nets: Deep neural networks with prior shift regularization for PET/CT based automatic, accurate, and calibrated whole-body lymphoma segmentation

Lymphoma is a type of lymphatic tissue originated cancer. Automatic and accurate lymphoma segmentation is critical for its diagnosis and prognosis yet challenging due to the severely class-imbalanced problem. Generally, deep neural networks trained with class-observation-frequency based re-weighting loss functions are used to address this problem. However, the majority class can be under-weighted by them, due to the existence of data overlap. Besides, they are more mis-calibrated. To resolve these, we propose a neural network with prior-shift regularization (PSR-Net), which comprises a UNet-like backbone with re-weighting loss functions, and a prior-shift regularization (PSR) module including a prior-shift layer (PSL), a regularizer generation layer (RGL), and an expected prediction confidence updating layer (EPCUL). We first propose a trainable expected prediction confidence (EPC) for each class. Periodically, PSL shifts a prior training dataset to a more informative dataset based on EPCs; RGL presents a generalized informative-voxel-aware (GIVA) loss with EPCs and calculates it on the informative dataset for model finetuning in back-propagation; and EPCUL updates EPCs to refresh PSL and RRL in next forward-propagation. PSR-Net is trained in a two- stage manner. The backbone is first trained with re-weighting loss functions, then we reload the best saved model for the backbone and continue to train it with the weighted sum of the re-weighting loss functions, the GIVA regularizer and the L2 loss function of EPCs for regularization fine-tuning. Extensive experiments are performed based on PET/CT volumes with advanced stage lymphomas. Our PSR-Net achieves 95.12% sensitivity and 87.18% Dice coefficient, demonstrating the effectiveness of PSR-Net, when compared to the baselines and the state-of-the-arts.

Autodelineation of organ at risk in head and neck cancer radiotherapy using artificial intelligence

Aim

The aim of this study is to check the practical feasibility of artificial intelligence for day-to-day operations and how it generalizes when the data have considerable interobserver variability.

Background

Automated delineation of organ at risk (OAR) using a deep learning model is reasonably accurate. This will considerably reduce the medical professional time in manually contouring the OAR and also reduce the interobserver variation among radiation oncologists. It allows for quick radiation planning which helps in adaptive radiotherapy planning.

Materials and Methods

Head and neck (HN) computed tomography (CT) scan data of 113 patients were used in this study. CT scan was done as per the institute protocol. Each patient had about 100-300 slices in Dicom format. A total number of 19,240 images were used as the data set. The OARs were delineated by the radiation oncologist in the contouring system. Of the 113 patient records, 13 records were kept aside as test dataset and the remaining 100 records were used for training the UNet 2D model. The study was performed on the spinal cord and left and right parotids as OARs on HN CT images. The model performance was quantified using the Dice similarity coefficient (DSC) score.

Results

The trained model is used to predict three OARs, spinal cord and left and right parotids. The DSC score of 84% and above could be achieved using the UNet 2D Convolutional Neural Network.

Conclusion

This study showed that the accuracy of predicted organs was within acceptable DSC scores, even when the underlying dataset has significant interobserver variability.

Application of Deep Learning to IVC Filter Detection from CT Scans

IVC filters (IVCF) perform an important function in select patients that have venous blood clots. However, they are usually intended to be temporary, and significant delay in removal can have negative health consequences for the patient. Currently, all Interventional Radiology (IR) practices are tasked with tracking patients in whom IVCF are placed. Due to their small size and location deep within the abdomen it is common for patients to forget that they have an IVCF. Therefore, there is a significant delay for a new healthcare provider to become aware of the presence of a filter. Patients may have an abdominopelvic CT scan for many reasons and, fortunately, IVCF are clearly visible on these scans. In this research a deep learning model capable of segmenting IVCF from CT scan slices along the axial plane is developed. The model achieved a Dice score of 0.82 for training over 372 CT scan slices. The segmentation model is then integrated with a prediction algorithm capable of flagging an entire CT scan as having IVCF. The prediction algorithm utilizing the segmentation model achieved a 92.22% accuracy at detecting IVCF in the scans.

Land Use Land Cover Labeling of GLOBE Images Using a Deep Learning Fusion Model

Most of the land use land cover classification methods presented in the literature have been conducted using satellite remote sensing images. High-resolution aerial imagery is now being used for land cover classification. The Global Learning and Observations to Benefit, the Environment land cover image database, is created by citizen scientists worldwide who use their handheld cameras to take a set of six images per land cover site. These images have clutter due to man-made objects, and the pixel uncertainties result in incorrect labels. The problem of accurate labeling of these land cover images is addressed. An integrated architecture that combines Unet and DeepLabV3 for initial segmentation, followed by a weighted fusion model that combines the segmentation labels, is presented. The land cover images with labels are used for training the deep learning models. The fusion model combines the labels of five images taken from the north, south, east, west, and down directions to assign a unique label to the image sets. 2916 GLOBE images have been labeled with land cover classes using the integrated model with minimal human-in-the-loop annotation. The validation step shows that our architecture of labeling the images results in 90.97% label accuracy. Our fusion model can be used for labeling large databases of land cover classes from RGB images.

Segmentation-Based Classification Deep Learning Model Embedded with Explainable AI for COVID-19 Detection in Chest X-ray Scans

Background and Motivation: COVID-19 has resulted in a massive loss of life during the last two years. The current imaging-based diagnostic methods for COVID-19 detection in multiclass pneumonia-type chest X-rays are not so successful in clinical practice due to high error rates. Our hypothesis states that if we can have a segmentation-based classification error rate <5%, typically adopted for 510 (K) regulatory purposes, the diagnostic system can be adapted in clinical settings. Method: This study proposes 16 types of segmentation-based classification deep learning-based systems for automatic, rapid, and precise detection of COVID-19. The two deep learning-based segmentation networks, namely UNet and UNet+, along with eight classification models, namely VGG16, VGG19, Xception, InceptionV3, Densenet201, NASNetMobile, Resnet50, and MobileNet, were applied to select the best-suited combination of networks. Using the cross-entropy loss function, the system performance was evaluated by Dice, Jaccard, area-under-the-curve (AUC), and receiver operating characteristics (ROC) and validated using Grad-CAM in explainable AI framework. Results: The best performing segmentation model was UNet, which exhibited the accuracy, loss, Dice, Jaccard, and AUC of 96.35%, 0.15%, 94.88%, 90.38%, and 0.99 (p-value <0.0001), respectively. The best performing segmentation-based classification model was UNet+Xception, which exhibited the accuracy, precision, recall, F1-score, and AUC of 97.45%, 97.46%, 97.45%, 97.43%, and 0.998 (p-value <0.0001), respectively. Our system outperformed existing methods for segmentation-based classification models. The mean improvement of the UNet+Xception system over all the remaining studies was 8.27%. Conclusion: The segmentation-based classification is a viable option as the hypothesis (error rate <5%) holds true and is thus adaptable in clinical practice.

Evaluation of a 2D UNet-Based Attenuation Correction Methodology for PET/MR Brain Studies

Deep learning (DL) strategies applied to magnetic resonance (MR) images in positron emission tomography (PET)/MR can provide synthetic attenuation correction (AC) maps, and consequently PET images, more accurate than segmentation or atlas-registration strategies. As first objective, we aim to investigate the best MR image to be used and the best point of the AC pipeline to insert the synthetic map in. Sixteen patients underwent a 18F-fluorodeoxyglucose (FDG) PET/computed tomography (CT) and a PET/MR brain study in the same day. PET/CT images were reconstructed with attenuation maps obtained: (1) from CT (reference), (2) from MR with an atlas-based and a segmentation-based method and (3) with a 2D UNet trained on MR image/attenuation map pairs. As for MR, T1-weighted and Zero Time Echo (ZTE) images were considered; as for attenuation maps, CTs and 511 keV low-resolution attenuation maps were assessed. As second objective, we assessed the ability of DL strategies to provide proper AC maps in presence of cranial anatomy alterations due to surgery. Three 11C-methionine (METH) PET/MR studies were considered. PET images were reconstructed with attenuation maps obtained: (1) from diagnostic coregistered CT (reference), (2) from MR with an atlas-based and a segmentation-based method and (3) with 2D UNets trained on the sixteen FDG anatomically normal patients. Only UNets taking ZTE images in input were considered. FDG and METH PET images were quantitatively evaluated. As for anatomically normal FDG patients, UNet AC models generally provide an uptake estimate with lower bias than atlas-based or segmentation-based methods. The intersubject average bias on images corrected with UNet AC maps is always smaller than 1.5%, except for AC maps generated on too coarse grids. The intersubject bias variability is the lowest (always lower than 2%) for UNet AC maps coming from ZTE images, larger for other methods. UNet models working on MR ZTE images and generating synthetic CT or 511 keV low-resolution attenuation maps therefore provide the best results in terms of both accuracy and variability. As for METH anatomically altered patients, DL properly reconstructs anatomical alterations. Quantitative results on PET images confirm those found on anatomically normal FDG patients.

Multiresolution Aggregation Transformer UNet Based on Multiscale Input and Coordinate Attention for Medical Image Segmentation

The latest medical image segmentation methods uses UNet and transformer structures with great success. Multiscale feature fusion is one of the important factors affecting the accuracy of medical image segmentation. Existing transformer-based UNet methods do not comprehensively explore multiscale feature fusion, and there is still much room for improvement. In this paper, we propose a novel multiresolution aggregation transformer UNet (MRA-TUNet) based on multiscale input and coordinate attention for medical image segmentation. It realizes multiresolution aggregation from the following two aspects: (1) On the input side, a multiresolution aggregation module is used to fuse the input image information of different resolutions, which enhances the input features of the network. (2) On the output side, an output feature selection module is used to fuse the output information of different scales to better extract coarse-grained information and fine-grained information. We try to introduce a coordinate attention structure for the first time to further improve the segmentation performance. We compare with state-of-the-art medical image segmentation methods on the automated cardiac diagnosis challenge and the 2018 atrial segmentation challenge. Our method achieved average dice score of 0.911 for right ventricle (RV), 0.890 for myocardium (Myo), 0.961 for left ventricle (LV), and 0.923 for left atrium (LA). The experimental results on two datasets show that our method outperforms eight state-of-the-art medical image segmentation methods in dice score, precision, and recall.

Brain tumor magnetic resonance image segmentation by a multiscale contextual attention module combined with a deep residual UNet (MCA-ResUNet)

Background and Objective. Automatic segmentation of MRI brain tumor area is a key step in the diagnosis and treatment of brain tumor. In recent years, the improved network based on UNet encoding and decoding structure has been widely used in brain tumor segmentation. However, due to continuous convolution and pooling operations, some spatial context information in existing networks will be discontinuous or even missing. It will affect the segmentation accuracy of the model. Therefore, the method proposed in this paper is to alleviate the lack of spatial context information and improve the accuracy of the model.Approach. This paper proposes a context attention module (multiscale contextual attention) to capture and filter out high-level features with spatial context information, which solves the problem of context information loss in feature extraction. The channel attention mechanism is introduced into the decoding structure to realize the fusion of high-level features and low-level features. The standard convolution block in the encoding and decoding structure is replaced by the pre-activated residual block to optimize the network training and improve the network performance.Results. This paper uses two public data sets (BraTs 2017 and BraTs 2019) to evaluate and verify the proposed method. Experimental results show that the proposed method can effectively alleviate the lack of spatial context information, and the segmentation performance is better than other existing methods.Significance. The method improves the segmentation performance of the model. It will assist doctors in making accurate diagnosis and provide reference basis for tumor resection. As a result, the proposed method will reduce the operation risk of patients and the postoperative recurrence rate.

A Novel Elastomeric UNet for Medical Image Segmentation

Medical image segmentation is of important support for clinical medical applications. As most of the current medical image segmentation models are limited in the U-shaped structure, to some extent the deep convolutional neural network (CNN) structure design is hard to be accomplished. The design in this study mimics the way the wave is elastomeric propagating, extending the structure from both the horizontal and spatial dimensions for realizing the Elastomeric UNet (EUNet) structure. The EUNet can be divided into two types: horizontal EUNet and spatial EUNet, based on the propagation direction. The advantages of this design are threefold. First, the training structure can be deepened effectively. Second, the independence brought by each branch (a U-shaped design) makes the flexible design redundancy available. Finally, a horizontal and vertical series-parallel structure helps on feature accumulation and recursion. Researchers can adjust the design according to the requirements to achieve better segmentation performance for the independent structural design. The proposed networks were evaluated on two datasets: a self-built dataset (multi-photon microscopy, MPM) and publicly benchmark retinal datasets (DRIVE). The results of experiments demonstrated that the performance of EUNet outperformed the UNet and its variants.

A Pipelined Tracer-Aware Approach for Lesion Segmentation in Breast DCE-MRI

The recent spread of Deep Learning (DL) in medical imaging is pushing researchers to explore its suitability for lesion segmentation in Dynamic Contrast-Enhanced Magnetic-Resonance Imaging (DCE-MRI), a complementary imaging procedure increasingly used in breast-cancer analysis. Despite some promising proposed solutions, we argue that a "naive" use of DL may have limited effectiveness as the presence of a contrast agent results in the acquisition of multimodal 4D images requiring thorough processing before training a DL model. We thus propose a pipelined approach where each stage is intended to deal with or to leverage a peculiar characteristic of breast DCE-MRI data: the use of a breast-masking pre-processing to remove non-breast tissues; the use of Three-Time-Points (3TP) slices to effectively highlight contrast agent time course; the application of a motion-correction technique to deal with patient involuntary movements; the leverage of a modified U-Net architecture tailored on the problem; and the introduction of a new "Eras/Epochs" training strategy to handle the unbalanced dataset while performing a strong data augmentation. We compared our pipelined solution against some literature works. The results show that our approach outperforms the competitors by a large margin (+9.13% over our previous solution) while also showing a higher generalization ability.