Tumor attention networks: Better feature selection, better tumor segmentation

Multi-scale object equalization learning network for intracerebral hemorrhage region segmentation

Segmentation and the subsequent quantitative assessment of the target object in computed tomography (CT) images provide valuable information for the analysis of intracerebral hemorrhage (ICH) pathology. However, most existing methods lack a reasonable strategy to explore the discriminative semantics of multi-scale ICH regions, making it difficult to address the challenge of complex morphology in clinical data. In this paper, we propose a novel multi-scale object equalization learning network (MOEL-Net) for accurate ICH region segmentation. Specifically, we first introduce a shallow feature extraction module (SFEM) for obtaining shallow semantic representations to maintain sufficient and effective detailed location information. Then, a deep feature extraction module (DFEM) is leveraged to extract the deep semantic information of the ICH region from the combination of SFEM and original image features. To further achieve equalization learning in different scales of ICH regions, we introduce a multi-level semantic feature equalization fusion module (MSFEFM), which explores the equalized fusion features of the described objects with the assistance of shallow and deep semantic information provided by SFEM and DFEM. Driven by the above three designs, MOEL-Net shows a solid capacity to capture more discriminative features in various ICH region segmentation. To promote the research of clinical automatic ICH region segmentation, we collect two datasets, VMICH and FRICH (divided into Test A and Test B) for evaluation. Experimental results show that the proposed model achieves the Dice scores of 88.28%, 90.92%, and 90.95% on the VMICH, FRICH Test A, and Test B, respectively, which outperform fourteen competing methods.

Reconstruct incomplete relation for incomplete modality brain tumor segmentation

Different brain tumor magnetic resonance imaging (MRI) modalities provide diverse tumor-specific information. Previous works have enhanced brain tumor segmentation performance by integrating multiple MRI modalities. However, multi-modal MRI data are often unavailable in clinical practice. An incomplete modality leads to missing tumor-specific information, which degrades the performance of existing models. Various strategies have been proposed to transfer knowledge from a full modality network (teacher) to an incomplete modality one (student) to address this issue. However, they neglect the fact that brain tumor segmentation is a structural prediction problem that requires voxel semantic relations. In this paper, we propose a Reconstruct Incomplete Relation Network (RIRN) that transfers voxel semantic relational knowledge from the teacher to the student. Specifically, we propose two types of voxel relations to incorporate structural knowledge: Class-relative relations (CRR) and Class-agnostic relations (CAR). The CRR groups voxels into different tumor regions and constructs a relation between them. The CAR builds a global relation between all voxel features, complementing the local inter-region relation. Moreover, we use adversarial learning to align the holistic structural prediction between the teacher and the student. Extensive experimentation on both the BraTS 2018 and BraTS 2020 datasets establishes that our method outperforms all state-of-the-art approaches.

Medical image segmentation network based on multi-scale frequency domain filter

With the development of deep learning, medical image segmentation in computer-aided diagnosis has become a research hotspot. Recently, UNet and its variants have become the most powerful medical image segmentation methods. However, these methods suffer from (1) insufficient sensing field and insufficient depth; (2) computational nonlinearity and redundancy of channel features; and (3) ignoring the interrelationships among feature channels. These problems lead to poor network segmentation performance and weak generalization ability. Therefore, first of all, we propose an effective replacement scheme of UNet base block, Double residual depthwise atrous convolution (DRDAC) block, to effectively improve the deficiency of receptive field and depth. Secondly, a new linear module, the Multi-scale frequency domain filter (MFDF), is designed to capture global information from the frequency domain. The high order multi-scale relationship is extracted by combining the depthwise atrous separable convolution with the frequency domain filter. Finally, a channel attention called Axial selection channel attention (ASCA) is redesigned to enhance the network's ability to model feature channel interrelationships. Further, we design a novel frequency domain medical image segmentation baseline method FDFUNet based on the above modules. We conduct extensive experiments on five publicly available medical image datasets and demonstrate that the present method has stronger segmentation performance as well as generalization ability compared to other state-of-the-art baseline methods.

Deep causal learning for pancreatic cancer segmentation in CT sequences

Segmenting the irregular pancreas and inconspicuous tumor simultaneously is an essential but challenging step in diagnosing pancreatic cancer. Current deep-learning (DL) methods usually segment the pancreas or tumor independently using mixed image features, which are disrupted by surrounding complex and low-contrast background tissues. Here, we proposed a deep causal learning framework named CausegNet for pancreas and tumor co-segmentation in 3D CT sequences. Specifically, a causality-aware module and a counterfactual loss are employed to enhance the DL network's comprehension of the anatomical causal relationship between the foreground elements (pancreas and tumor) and the background. By integrating causality into CausegNet, the network focuses solely on extracting intrinsic foreground causal features while effectively learning the potential causality between the pancreas and the tumor. Then based on the extracted causal features, CausegNet applies a counterfactual inference to significantly reduce the background interference and sequentially search for pancreas and tumor from the foreground. Consequently, our approach can handle deformable pancreas and obscure tumors, resulting in superior co-segmentation performance in both public and real clinical datasets, achieving the highest pancreas/tumor Dice coefficients of 86.67%/84.28%. The visualized features and anti-noise experiments further demonstrate the causal interpretability and stability of our method. Furthermore, our approach improves the accuracy and sensitivity of downstream pancreatic cancer risk assessment task by 12.50% and 50.00%, respectively, compared to experienced clinicians, indicating promising clinical applications.

A Multi-center Dental Panoramic Radiography Image Dataset for Impacted Teeth, Periodontitis, and Dental Caries: Benchmarking Segmentation and Classification Tasks

Panoramic radiography imaging plays a crucial role in the diagnostic process of dental diseases. However, current artificial intelligence research datasets for panoramic radiography dental image processing are often limited to single-center and single-task scenarios, making it difficult to generalize their results. To address this, we present a multi-center, multi-task labeled dataset. In this study, our dataset comprises three datasets obtained from different hospitals. The first set has 4940 panoramic radiography images and corresponding labels from the Stemmatological Hospital of the General Hospital of Ningxia Medical University. The second set includes 716 panoramic radiography images and labels from the People's Hospital of Yinchuan City, Ningxia. The third dataset contains 880 panoramic radiography images and labels from a hospital in Shenzhen, Guangdong Province. This comprehensive dataset encompasses three types of dental diseases: impacted teeth, periodontitis, and dental caries. Specifically, it comprises 2555 images related to impacted teeth, 2735 images related to periodontitis, and 1246 images related to dental caries. In order to evaluate the performance of the dataset, we conducted benchmark tests for segmentation and classification tasks on our dataset. The results show that the presented dataset could be effectively used for benchmarking segmentation and classification tasks critical to the diagnosis of dental diseases. To request our multi-center dataset, please visit the address: https://github.com/qinxin99/qinxini .

Boundary uncertainty aware network for automated polyp segmentation

Recently, leveraging deep neural networks for automated colorectal polyp segmentation has emerged as a hot topic due to the favored advantages in evading the limitations of visual inspection, e.g., overwork and subjectivity. However, most existing methods do not pay enough attention to the uncertain areas of colonoscopy images and often provide unsatisfactory segmentation performance. In this paper, we propose a novel boundary uncertainty aware network (BUNet) for precise and robust colorectal polyp segmentation. Specifically, considering that polyps vary greatly in size and shape, we first adopt a pyramid vision transformer encoder to learn multi-scale feature representations. Then, a simple yet effective boundary exploration module (BEM) is proposed to explore boundary cues from the low-level features. To make the network focus on the ambiguous area where the prediction score is biased to neither the foreground nor the background, we further introduce a boundary uncertainty aware module (BUM) that explores error-prone regions from the high-level features with the assistance of boundary cues provided by the BEM. Through the top-down hybrid deep supervision, our BUNet implements coarse-to-fine polyp segmentation and finally localizes polyp regions precisely. Extensive experiments on five public datasets show that BUNet is superior to thirteen competing methods in terms of both effectiveness and generalization ability.

A three-path network with multi-scale selective feature fusion, edge-inspiring and edge-guiding for liver tumor segmentation

Automatic liver tumor segmentation is one of the most important tasks in computer-aided diagnosis and treatment. Deep learning techniques have gained increasing popularity for medical image segmentation in recent years. However, due to the various shapes, sizes, and obscure boundaries of tumors, it is still difficult to automatically extract tumor regions from CT images. Based on the complementarity of edge detection and region segmentation, a three-path structure with multi-scale selective feature fusion (MSFF) module, multi-channel feature fusion (MFF) module, edge-inspiring (EI) module, and edge-guiding (EG) module is proposed in this paper. The MSFF module includes the process of generation, fusion, and selection of multi-scale features, which can adaptively correct the response weights in multiple branches to filter redundant information. The MFF module integrates richer hierarchical features to capture targets at different scales. The EI module aggregates high-level semantic information at different levels to obtain fine edge semantics, which is injected into the EG module for representation learning of segmentation features. Experiments on the LiTs2017 dataset show that our proposed method achieves a Dice index of 85.55% and a Jaccard index of 81.11%, which are higher than what can be obtained by the current state-of-the-art methods. Cross-dataset validation experiments conducted on 3Dircadb and Clinical datasets show the generalization and robustness of the proposed method by achieving dice indices of 80.14% and 81.68%, respectively.

Context fusion network with multi-scale-aware skip connection and twin-split attention for liver tumor segmentation

Manually annotating liver tumor contours is a time-consuming and labor-intensive task for clinicians. Therefore, automated segmentation is urgently needed in clinical diagnosis. However, automatic segmentation methods face certain challenges due to heterogeneity, fuzzy boundaries, and irregularity of tumor tissue. In this paper, a novel deep learning-based approach with multi-scale-aware (MSA) module and twin-split attention (TSA) module is proposed for tumor segmentation. The MSA module can bridge the semantic gap and reduce the loss of detailed information. The TSA module can recalibrate the channel response of the feature map. Eventually, we can count tumors based on the segmentation results from a 3D perspective for cancer grading. Extensive experiments conducted on the LiTS2017 dataset show the effectiveness of the proposed method by achieving a Dice index of 85.97% and a Jaccard index of 81.56% over the state of the art. In addition, the proposed method also achieved a Dice index of 83.67% and a Jaccard index of 80.11% in 3Dircadb dataset verification, which further reflects its robustness and generalization ability.

MANet: a multi-attention network for automatic liver tumor segmentation in computed tomography (CT) imaging

Automatic liver tumor segmentation is a paramount important application for liver tumor diagnosis and treatment planning. However, it has become a highly challenging task due to the heterogeneity of the tumor shape and intensity variation. Automatic liver tumor segmentation is capable to establish the diagnostic standard to provide relevant radiological information to all levels of expertise. Recently, deep convolutional neural networks have demonstrated superiority in feature extraction and learning in medical image segmentation. However, multi-layer dense feature stacks make the model quite inconsistent in imitating visual attention and awareness of radiological expertise for tumor recognition and segmentation task. To bridge that visual attention capability, attention mechanisms have developed for better feature selection. In this paper, we propose a novel network named Multi Attention Network (MANet) as a fusion of attention mechanisms to learn highlighting important features while suppressing irrelevant features for the tumor segmentation task. The proposed deep learning network has followed U-Net as the basic architecture. Moreover, residual mechanism is implemented in the encoder. Convolutional block attention module has split into channel attention and spatial attention modules to implement in encoder and decoder of the proposed architecture. The attention mechanism in Attention U-Net is integrated to extract low-level features to combine with high-level ones. The developed deep learning architecture is trained and evaluated on the publicly available MICCAI 2017 Liver Tumor Segmentation dataset and 3DIRCADb dataset under various evaluation metrics. MANet demonstrated promising results compared to state-of-the-art methods with comparatively small parameter overhead.

Attention-guided multi-scale learning network for automatic prostate and tumor segmentation on MRI

BACKGROUND AND OBJECTIVE

Image-guided clinical diagnosis can be achieved by automatically and accurately segmenting prostate and prostatic cancer in male pelvic magnetic resonance imaging (MRI) images. For accurate tumor removal, the location, number, and size of prostate cancer are crucial, especially in surgical patients. The morphological differences between the prostate and tumor regions are small, the size of the tumor is uncertain, the boundary between the tumor and surrounding tissue is blurred, and the classification that separates the normal region from the tumor is uneven. Therefore, segmenting prostate and tumor on MRI images is challenging.

METHODS

This study offers a new prostate and prostatic cancer segmentation network based on double branch attention driven multi-scale learning for MRI. To begin, the dual branch structure provides two input images with different scales for feature coding, as well as a multi-scale attention module that collects details from different scales. The features of the double branch structure are then entered into the built feature fusion module to get more complete context information. Finally, to give a more precise learning representation, each stage is built using a deep supervision mechanism.

RESULTS

The results of our proposed network's prostate and tumor segmentation on a variety of male pelvic MRI data sets show that it outperforms existing techniques. For prostate and prostatic cancer MRI segmentation, the dice similarity coefficient (DSC) values were 91.65% and 84.39%, respectively.

CONCLUSIONS

Our method maintains high correlation and consistency between automatic segmentation results and expert manual segmentation results. Accurate automatic segmentation of prostate and prostate cancer has important clinical significance.

Trustworthy Medical Image Segmentation with improved performance for in-distribution samples

Despite the enormous achievements of Deep Learning (DL) based models, their non-transparent nature led to restricted applicability and distrusted predictions. Such predictions emerge from erroneous In-Distribution (ID) and Out-Of-Distribution (OOD) samples, which results in disastrous effects in the medical domain, specifically in Medical Image Segmentation (MIS). To mitigate such effects, several existing works accomplish OOD sample detection; however, the trustworthiness issues from ID samples still require thorough investigation. To this end, a novel method TrustMIS (Trustworthy Medical Image Segmentation) is proposed in this paper, which provides the trustworthiness and improved performance of ID samples for DL-based MIS models. TrustMIS works in three folds: IT (Investigating Trustworthiness), INT (Improving Non-Trustworthy prediction) and CSO (Classifier Switching Operation). Initially, the IT method investigates the trustworthiness of MIS by leveraging similar characteristics and consistency analysis of input and its variants. Subsequently, the INT method employs the IT method to improve the performance of the MIS model. It leverages the observation that an input providing erroneous segmentation can provide correct segmentation with rotated input. Eventually, the CSO method employs the INT method to scrutinise several MIS models and selects the model that delivers the most trustworthy prediction. The experiments conducted on publicly available datasets using well-known MIS models reveal that TrustMIS has successfully provided a trustworthiness measure, outperformed the existing methods, and improved the performance of state-of-the-art MIS models. Our implementation is available at https://github.com/SnehaShukla937/TrustMIS.

Vessel Delineation Using U-Net: A Sparse Labeled Deep Learning Approach for Semantic Segmentation of Histological Images

Semantic segmentation is an important imaging analysis method enabling the identification of tissue structures. Histological image segmentation is particularly challenging, having large structural information while providing only limited training data. Additionally, labeling these structures to generate training data is time consuming. Here, we demonstrate the feasibility of a semantic segmentation using U-Net with a novel sparse labeling technique. The basic U-Net architecture was extended by attention gates, residual and recurrent links, and dropout regularization. To overcome the high class imbalance, which is intrinsic to histological data, under- and oversampling and data augmentation were used. In an ablation study, various architectures were evaluated, and the best performing model was identified. This model contains attention gates, residual links, and a dropout regularization of 0.125. The segmented images show accurate delineations of the vascular structures (with a precision of 0.9088 and an AUC-ROC score of 0.9717), and the segmentation algorithm is robust to images containing staining variations and damaged tissue. These results demonstrate the feasibility of sparse labeling in combination with the modified U-Net architecture.

Accurate tumor segmentation and treatment outcome prediction with DeepTOP

BACKGROUND

Accurate outcome prediction prior to treatment can facilitate trial design and clinical decision making to achieve better treatment outcome.

METHOD

We developed the DeepTOP tool with deep learning approach for region-of-interest segmentation and clinical outcome prediction using magnetic resonance imaging (MRI). DeepTOP was constructed with an automatic pipeline from tumor segmentation to outcome prediction. In DeepTOP, the segmentation model used U-Net with a codec structure, and the prediction model was built with a three-layer convolutional neural network. In addition, the weight distribution algorithm was developed and applied in the prediction model to optimize the performance of DeepTOP.

RESULTS

A total of 1889 MRI slices from 99 patients in the phase III multicenter randomized clinical trial (NCT01211210) on neoadjuvant treatment for rectal cancer was used to train and validate DeepTOP. We systematically optimized and validated DeepTOP with multiple devised pipelines in the clinical trial, demonstrating a better performance than other competitive algorithms in accurate tumor segmentation (Dice coefficient: 0.79; IoU: 0.75; slice-specific sensitivity: 0.98) and predicting pathological complete response to chemo/radiotherapy (accuracy: 0.789; specificity: 0.725; and sensitivity: 0.812). DeepTOP is a deep learning tool that could avoid manual labeling and feature extraction and realize automatic tumor segmentation and treatment outcome prediction by using the original MRI images.

CONCLUSION

DeepTOP is open to provide a tractable framework for the development of other segmentation and predicting tools in clinical settings. DeepTOP-based tumor assessment can provide a reference for clinical decision making and facilitate imaging marker-driven trial design.

Multi-scale multi-reception attention network for bone age assessment in X-ray images

Bone age assessment plays a significant role in estimating bone maturity. However, radiograph/X-ray images of hand bones contain a large amount of redundant information. Some detection or segmentation based methods have recently been proposed to solve this issue. These network structures are often of high complexity and might require extra annotations, which make them less applicable in practice. In this paper, we present a Multi-scale Multi-reception Attention Net (MMANet), which combines a novel Multi-scale Multi-reception Complement Attention (MMCA) network and a graph attention module with a ResNet backbone to enhance the feature representation of key regions and suppress the influence of background regions to achieve significant performance improvement. Experimental results show our MMANet is able to accurately detect key regions and achieves 3.88 mean absolute error (MAE) on the RSNA 2017 Paediatric Bone Age Challenge dataset. Our method, without explicit modelling of anatomical information, outperforms the current state-of-the-art method (MAE=3.91) by 0.03 (months) which requires extra annotations. Code is available at https://github.com/yzc1122333/BoneAgeAss.

Teeth U-Net: A segmentation model of dental panoramic X-ray images for context semantics and contrast enhancement

BACKGROUND AND OBJECTIVE

It is very significant in orthodontics and restorative dentistry that the teeth are segmented from dental panoramic X-ray images. Nevertheless, there are some problems in panoramic X-ray images of teeth, such as blurred interdental boundaries, low contrast between teeth and alveolar bone.

METHODS

In this paper, The Teeth U-Net model is proposed in this paper to resolve these problems. This paper makes the following contributions: Firstly, a Squeeze-Excitation Module is utilized in the encoder and the decoder. And proposing a dense skip connection between encoder and decoder to reduce the semantic gap. Secondly, due to the irregular shape of the teeth and the low contrast of the dental panoramic X-ray images. A Multi-scale Aggregation attention Block (MAB) in the bottleneck layer is designed to resolve this problem, which can effectively extract teeth shape features and fuse multi-scale features adaptively. Thirdly, in order to capture dental feature information in a larger field of perception, this paper designs a Dilated Hybrid self-Attentive Block (DHAB) at the bottleneck layer. This module effectively suppresses the task-irrelevant background region information without increasing the network parameters. Finally, the effectiveness of the algorithm is validated using a clinical dental panoramic X-ray image datasets.

RESULTS

The results of the three comparison experiments are shown that Accuracy, Precision, Recall, Dice, Volumetric Overlap Error and Relative Volume Difference for dental panoramic X-ray teeth segmentation are 98.53%, 95.62%, 94.51%, 94.28%, 88.92% and 95.97% by the proposed model respectively.

CONCLUSION

The proposed modules complement each other in processing every detail of the dental panoramic X-ray images, which can effectively improve the efficiency of preoperative preparation and postoperative evaluation, and promote the application of dental panoramic X-ray in medical image segmentation. There are more accuracy about Teeth U-Net than others model in dental panoramic X-ray teeth segmentation. That is very important to clinical doctors to cure in orthodontics and restorative dentistry.

A survey on deep learning applied to medical images: from simple artificial neural networks to generative models

Deep learning techniques, in particular generative models, have taken on great importance in medical image analysis. This paper surveys fundamental deep learning concepts related to medical image generation. It provides concise overviews of studies which use some of the latest state-of-the-art models from last years applied to medical images of different injured body areas or organs that have a disease associated with (e.g., brain tumor and COVID-19 lungs pneumonia). The motivation for this study is to offer a comprehensive overview of artificial neural networks (NNs) and deep generative models in medical imaging, so more groups and authors that are not familiar with deep learning take into consideration its use in medicine works. We review the use of generative models, such as generative adversarial networks and variational autoencoders, as techniques to achieve semantic segmentation, data augmentation, and better classification algorithms, among other purposes. In addition, a collection of widely used public medical datasets containing magnetic resonance (MR) images, computed tomography (CT) scans, and common pictures is presented. Finally, we feature a summary of the current state of generative models in medical image including key features, current challenges, and future research paths.

MDA-Unet: A Multi-Scale Dilated Attention U-Net for Medical Image Segmentation

The advanced development of deep learning methods has recently made significant improvements in medical image segmentation. Encoder–decoder networks, such as U-Net, have addressed some of the challenges in medical image segmentation with an outstanding performance, which has promoted them to be the most dominating deep learning architecture in this domain. Despite their outstanding performance, we argue that they still lack some aspects. First, there is incompatibility in U-Net’s skip connection between the encoder and decoder features due to the semantic gap between low-processed encoder features and highly processed decoder features, which adversely affects the final prediction. Second, it lacks capturing multi-scale context information and ignores the contribution of all semantic information through the segmentation process. Therefore, we propose a model named MDA-Unet, a novel multi-scale deep learning segmentation model. MDA-Unet improves upon U-Net and enhances its performance in segmenting medical images with variability in the shape and size of the region of interest. The model is integrated with a multi-scale spatial attention module, where spatial attention maps are derived from a hybrid hierarchical dilated convolution module that captures multi-scale context information. To ease the training process and reduce the gradient vanishing problem, residual blocks are deployed instead of the basic U-net blocks. Through a channel attention mechanism, the high-level decoder features are used to guide the low-level encoder features to promote the selection of meaningful context information, thus ensuring effective fusion. We evaluated our model on 2 different datasets: a lung dataset of 2628 axial CT images and an echocardiographic dataset of 2000 images, each with its own challenges. Our model has achieved a significant gain in performance with a slight increase in the number of trainable parameters in comparison with the basic U-Net model, providing a dice score of 98.3% on the lung dataset and 96.7% on the echocardiographic dataset, where the basic U-Net has achieved 94.2% on the lung dataset and 93.9% on the echocardiographic dataset.

Application of Improved U-Net Convolutional Neural Network for Automatic Quantification of the Foveal Avascular Zone in Diabetic Macular Ischemia

OBJECTIVES

The foveal avascular zone (FAZ) is a biomarker for quantifying diabetic macular ischemia (DMI), to automate the identification and quantification of the FAZ in DMI, using an improved U-Net convolutional neural network (CNN) and to establish a CNN model based on optical coherence tomography angiography (OCTA) images for the same purpose.

METHODS

The FAZ boundaries on the full-thickness retina of 6 × 6 mm en face OCTA images of DMI and normal eyes were manually marked. Seventy percent of OCTA images were used as the training set, and ten percent of these images were used as the validation set to train the improved U-Net CNN with two attention modules. Finally, twenty percent of the OCTA images were used as the test set to evaluate the accuracy of this model relative to that of the baseline U-Net model. This model was then applied to the public data set sFAZ to compare its effectiveness with existing models at identifying and quantifying the FAZ area.

RESULTS

This study included 110 OCTA images. The Dice score of the FAZ area predicted by the proposed method was 0.949, the Jaccard index was 0.912, and the area correlation coefficient was 0.996. The corresponding values for the baseline U-Net were 0.940, 0.898, and 0.995, respectively, and those based on the description data set sFAZ were 0.983, 0.968, and 0.950, respectively, which were better than those previously reported based on this data set.

CONCLUSIONS

The improved U-Net CNN was more accurate at automatically measuring the FAZ area on the OCTA images than the traditional CNN. The present model may measure the DMI index more accurately, thereby assisting in the diagnosis and prognosis of retinal vascular diseases such as diabetic retinopathy.