SAR-U-Net: Squeeze-and-excitation block and atrous spatial pyramid pooling based residual U-Net for automatic liver segmentation in Computed Tomography

A multi-task framework for breast cancer segmentation and classification in ultrasound imaging

BACKGROUND

Ultrasound (US) is a medical imaging modality that plays a crucial role in the early detection of breast cancer. The emergence of numerous deep learning systems has offered promising avenues for the segmentation and classification of breast cancer tumors in US images. However, challenges such as the absence of data standardization, the exclusion of non-tumor images during training, and the narrow view of single-task methodologies have hindered the practical applicability of these systems, often resulting in biased outcomes. This study aims to explore the potential of multi-task systems in enhancing the detection of breast cancer lesions.

METHODS

To address these limitations, our research introduces an end-to-end multi-task framework designed to leverage the inherent correlations between breast cancer lesion classification and segmentation tasks. Additionally, a comprehensive analysis of a widely utilized public breast cancer ultrasound dataset named BUSI was carried out, identifying its irregularities and devising an algorithm tailored for detecting duplicated images in it.

RESULTS

Experiments are conducted utilizing the curated dataset to minimize potential biases in outcomes. Our multi-task framework exhibits superior performance in breast cancer respecting single-task approaches, achieving improvements close to 15% in segmentation and classification. Moreover, a comparative analysis against the state-of-the-art reveals statistically significant enhancements across both tasks.

CONCLUSION

The experimental findings underscore the efficacy of multi-task techniques, showcasing better generalization capabilities when considering all image types: benign, malignant, and non-tumor images. Consequently, our methodology represents an advance towards more general architectures with real clinical applications in the breast cancer field.

Contour-constrained branch U-Net for accurate left ventricular segmentation in echocardiography

Using echocardiography to assess the left ventricular function is one of the most crucial cardiac examinations in clinical diagnosis, and LV segmentation plays a particularly vital role in medical image processing as many important clinical diagnostic parameters are derived from the segmentation results, such as ejection function. However, echocardiography typically has a lower resolution and contains a significant amount of noise and motion artifacts, making it a challenge to accurate segmentation, especially in the region of the cardiac chamber boundary, which significantly restricts the accurate calculation of subsequent clinical parameters. In this paper, our goal is to achieve accurate LV segmentation through a simplified approach by introducing a branch sub-network into the decoder of the traditional U-Net. Specifically, we employed the LV contour features to supervise the branch decoding process and used a cross attention module to facilitate the interaction relationship between the branch and the original decoding process, thereby improving the segmentation performance in the region LV boundaries. In the experiments, the proposed branch U-Net (BU-Net) demonstrated superior performance on CAMUS and EchoNet-dynamic public echocardiography segmentation datasets in comparison to state-of-the-art segmentation models, without the need for complex residual connections or transformer-based architectures. Our codes are publicly available at Anonymous Github https://anonymous.4open.science/r/Anoymous_two-BFF2/ .

Wound Segmentation with U-Net Using a Dual Attention Mechanism and Transfer Learning

Accurate wound segmentation is crucial for the precise diagnosis and treatment of various skin conditions through image analysis. In this paper, we introduce a novel dual attention U-Net model designed for precise wound segmentation. Our proposed architecture integrates two widely used deep learning models, VGG16 and U-Net, incorporating dual attention mechanisms to focus on relevant regions within the wound area. Initially trained on diabetic foot ulcer images, we fine-tuned the model to acute and chronic wound images and conducted a comprehensive comparison with other state-of-the-art models. The results highlight the superior performance of our proposed dual attention model, achieving a Dice coefficient and IoU of 94.1% and 89.3%, respectively, on the test set. This underscores the robustness of our method and its capacity to generalize effectively to new data.

Liver segmentation network based on detail enhancement and multi-scale feature fusion

Due to the low contrast of abdominal CT (Computer Tomography) images and the similar color and shape of the liver to other organs such as the spleen, stomach, and kidneys, liver segmentation presents significant challenges. Additionally, 2D CT images obtained from different angles (such as sagittal, coronal, and transverse planes) increase the diversity of liver morphology and the complexity of segmentation. To address these issues, this paper proposes a Detail Enhanced Convolution (DE Conv) to improve liver feature learning and thereby enhance liver segmentation performance. Furthermore, to enable the model to better learn liver features at different scales, a Multi-Scale Feature Fusion module (MSFF) is added to the skip connections in the model. The MSFF module enhances the capture of global features, thus improving the accuracy of the liver segmentation model. Through the aforementioned research, this paper proposes a liver segmentation network based on detail enhancement and multi-scale feature fusion (DEMF-Net). We conducted extensive experiments on the LiTS17 dataset, and the results demonstrate that the DEMF-Net model achieved significant improvements across various evaluation metrics. Therefore, the proposed DEMF-Net model can achieve precise liver segmentation.

A Comparative Study of Decoders for Liver and Tumor Segmentation Using a Self-ONN-Based Cascaded Framework

Background/Objectives: Accurate liver and tumor detection and segmentation are crucial in diagnosis of early-stage liver malignancies. As opposed to manual interpretation, which is a difficult and time-consuming process, accurate tumor detection using a computer-aided diagnosis system can save both time and human efforts. Methods: We propose a cascaded encoder-decoder technique based on self-organized neural networks, which is a recent variant of operational neural networks (ONNs), for accurate segmentation and identification of liver tumors. The first encoder-decoder CNN segments the liver. For generating the liver region of interest, the segmented liver mask is placed over the input computed tomography (CT) image and then fed to the second Self-ONN model for tumor segmentation. For further investigation the other three distinct encoder-decoder architectures U-Net, feature pyramid networks (FPNs), and U-Net++, have also been investigated by altering the backbone at the encoders utilizing ResNet and DenseNet variants for transfer learning. Results: For the liver segmentation task, Self-ONN with a ResNet18 backbone has achieved a dice similarity coefficient score of 98.182% and an intersection over union of 97.436%. Tumor segmentation with Self-ONN with the DenseNet201 encoder resulted in an outstanding DSC of 92.836% and IoU of 91.748%. Conclusions: The suggested method is capable of precisely locating liver tumors of various sizes and shapes, including tiny infection patches that were said to be challenging to find in earlier research.

Medical Image Segmentation Review: The Success of U-Net

Automatic medical image segmentation is a crucial topic in the medical domain and successively a critical counterpart in the computer-aided diagnosis paradigm. U-Net is the most widespread image segmentation architecture due to its flexibility, optimized modular design, and success in all medical image modalities. Over the years, the U-Net model has received tremendous attention from academic and industrial researchers who have extended it to address the scale and complexity created by medical tasks. These extensions are commonly related to enhancing the U-Net's backbone, bottleneck, or skip connections, or including representation learning, or combining it with a Transformer architecture, or even addressing probabilistic prediction of the segmentation map. Having a compendium of different previously proposed U-Net variants makes it easier for machine learning researchers to identify relevant research questions and understand the challenges of the biological tasks that challenge the model. In this work, we discuss the practical aspects of the U-Net model and organize each variant model into a taxonomy. Moreover, to measure the performance of these strategies in a clinical application, we propose fair evaluations of some unique and famous designs on well-known datasets. Furthermore, we provide a comprehensive implementation library with trained models. In addition, for ease of future studies, we created an online list of U-Net papers with their possible official implementation.

Semantic Segmentation of CT Liver Structures: A Systematic Review of Recent Trends and Bibliometric Analysis : Neural Network-based Methods for Liver Semantic Segmentation

The use of artificial intelligence (AI) in the segmentation of liver structures in medical images has become a popular research focus in the past half-decade. The performance of AI tools in screening for this task may vary widely and has been tested in the literature in various datasets. However, no scientometric report has provided a systematic overview of this scientific area. This article presents a systematic and bibliometric review of recent advances in neuronal network modeling approaches, mainly of deep learning, to outline the multiple research directions of the field in terms of algorithmic features. Therefore, a detailed systematic review of the most relevant publications addressing fully automatic semantic segmenting liver structures in Computed Tomography (CT) images in terms of algorithm modeling objective, performance benchmark, and model complexity is provided. The review suggests that fully automatic hybrid 2D and 3D networks are the top performers in the semantic segmentation of the liver. In the case of liver tumor and vasculature segmentation, fully automatic generative approaches perform best. However, the reported performance benchmark indicates that there is still much to be improved in segmenting such small structures in high-resolution abdominal CT scans.

Lightweight Hotspot Detection Model Fusing SE and ECA Mechanisms

In this paper, we propose a lightweight lithography machine learning-based hotspot detection model that integrates the Squeeze-and-Excitation (SE) attention mechanism and the Efficient Channel Attention (ECA) mechanism. These mechanisms can adaptively adjust channel weights, significantly enhancing the model's ability to extract relevant features of hotspots and non-hotspots through cross-channel interaction without dimensionality reduction. Our model extracts feature vectors through seven convolutional layers and four pooling layers, followed by three fully connected layers that map to the output, thereby simplifying the CNN network structure. Experimental results on our collected layout dataset and the ICCAD 2012 layout dataset demonstrate that our model is more lightweight. By evaluating overall accuracy, recall, and runtime, the comprehensive performance of our model is shown to exceed that of ConvNeXt, Swin transformer, and ResNet 50.

Predicting Task Activation Maps from Resting-State Functional Connectivity using Deep Learning

Recent work has shown that deep learning is a powerful tool for predicting brain activation patterns evoked through various tasks using resting state features. We replicate and improve upon this recent work to introduce two models, BrainSERF and BrainSurfGCN, that perform at least as well as the state-of-the-art while greatly reducing memory and computational footprints. Our performance analysis observed that low predictability was associated with a possible lack of task engagement derived from behavioral performance. Furthermore, a deficiency in model performance was also observed for closely matched task contrasts, likely due to high individual variability confirmed by low test-retest reliability. Overall, we successfully replicate recently developed deep learning architecture and provide scalable models for further research.

Comprehensive review of deep learning in orthopaedics: Applications, challenges, trustworthiness, and fusion

Deep learning (DL) in orthopaedics has gained significant attention in recent years. Previous studies have shown that DL can be applied to a wide variety of orthopaedic tasks, including fracture detection, bone tumour diagnosis, implant recognition, and evaluation of osteoarthritis severity. The utilisation of DL is expected to increase, owing to its ability to present accurate diagnoses more efficiently than traditional methods in many scenarios. This reduces the time and cost of diagnosis for patients and orthopaedic surgeons. To our knowledge, no exclusive study has comprehensively reviewed all aspects of DL currently used in orthopaedic practice. This review addresses this knowledge gap using articles from Science Direct, Scopus, IEEE Xplore, and Web of Science between 2017 and 2023. The authors begin with the motivation for using DL in orthopaedics, including its ability to enhance diagnosis and treatment planning. The review then covers various applications of DL in orthopaedics, including fracture detection, detection of supraspinatus tears using MRI, osteoarthritis, prediction of types of arthroplasty implants, bone age assessment, and detection of joint-specific soft tissue disease. We also examine the challenges for implementing DL in orthopaedics, including the scarcity of data to train DL and the lack of interpretability, as well as possible solutions to these common pitfalls. Our work highlights the requirements to achieve trustworthiness in the outcomes generated by DL, including the need for accuracy, explainability, and fairness in the DL models. We pay particular attention to fusion techniques as one of the ways to increase trustworthiness, which have also been used to address the common multimodality in orthopaedics. Finally, we have reviewed the approval requirements set forth by the US Food and Drug Administration to enable the use of DL applications. As such, we aim to have this review function as a guide for researchers to develop a reliable DL application for orthopaedic tasks from scratch for use in the market.

DAC-Net: A light-weight U-shaped network based efficient convolution and attention for thyroid nodule segmentation

Recently, there has been a focused effort to improve the efficiency of thyroid nodule segmentation algorithms. This endeavor has resulted in the development of increasingly complex modules, such as the Transformer, leading to models with a higher number of parameters and computing requirements. Sophisticated models have difficulties in being implemented in clinical medicine platforms because of limited resources. DAC-Net is a Lightweight U-shaped network created to achieve high performance in segmenting thyroid nodules. Our method consists of three main components: DWSE, which combines depthwise convolution and squeeze-excitation block to enhance feature extraction and connections between samples; ADA, which includes Split Atrous and Dual Attention to extract global and local feature information from various viewpoints; and CSSC, which involves channel- scale and spatial-scale connections. This module enables the fusing of multi-stage features at global and local levels, producing feature maps at different channel and geographical scales, delivering a streamlined integration of multi-scale information. Combining these three components in our U- shaped design allows us to achieve competitive performance while also decreasing the number of parameters and computing complexity. Several experiments were conducted on the DDTI and TN3K datasets. The experimental results demonstrate that our model outperforms state-of-the-art thyroid nodule segmentation architectures in terms of segmentation performance. Our model not only reduces the number of parameters and computing expenses by 73x and 56x, respectively, but also exceeds TransUNet in segmentation performance. The source code is accessible at https://github.com/Phil-y/DAC-Net.

Deep learning with uncertainty estimation for automatic tumor segmentation in PET/CT of head and neck cancers: impact of model complexity, image processing and augmentation

Objective.Target volumes for radiotherapy are usually contoured manually, which can be time-consuming and prone to inter- and intra-observer variability. Automatic contouring by convolutional neural networks (CNN) can be fast and consistent but may produce unrealistic contours or miss relevant structures. We evaluate approaches for increasing the quality and assessing the uncertainty of CNN-generated contours of head and neck cancers with PET/CT as input.Approach.Two patient cohorts with head and neck squamous cell carcinoma and baseline18F-fluorodeoxyglucose positron emission tomography and computed tomography images (FDG-PET/CT) were collected retrospectively from two centers. The union of manual contours of the gross primary tumor and involved nodes was used to train CNN models for generating automatic contours. The impact of image preprocessing, image augmentation, transfer learning and CNN complexity, architecture, and dimension (2D or 3D) on model performance and generalizability across centers was evaluated. A Monte Carlo dropout technique was used to quantify and visualize the uncertainty of the automatic contours.Main results. CNN models provided contours with good overlap with the manually contoured ground truth (median Dice Similarity Coefficient: 0.75-0.77), consistent with reported inter-observer variations and previous auto-contouring studies. Image augmentation and model dimension, rather than model complexity, architecture, or advanced image preprocessing, had the largest impact on model performance and cross-center generalizability. Transfer learning on a limited number of patients from a separate center increased model generalizability without decreasing model performance on the original training cohort. High model uncertainty was associated with false positive and false negative voxels as well as low Dice coefficients.Significance.High quality automatic contours can be obtained using deep learning architectures that are not overly complex. Uncertainty estimation of the predicted contours shows potential for highlighting regions of the contour requiring manual revision or flagging segmentations requiring manual inspection and intervention.

G-MBRMD: Lightweight liver segmentation model based on guided teaching with multi-head boundary reconstruction mapping distillation

BACKGROUND AND OBJECTIVES

Liver segmentation is pivotal for the quantitative analysis of liver cancer. Although current deep learning methods have garnered remarkable achievements for medical image segmentation, they come with high computational costs, significantly limiting their practical application in the medical field. Therefore, the development of an efficient and lightweight liver segmentation model becomes particularly important.

METHODS

In our paper, we propose a real-time, lightweight liver segmentation model named G-MBRMD. Specifically, we employ a Transformer-based complex model as the teacher and a convolution-based lightweight model as the student. By introducing proposed multi-head mapping and boundary reconstruction strategies during the knowledge distillation process, Our method effectively guides the student model to gradually comprehend and master the global boundary processing capabilities of the complex teacher model, significantly enhancing the student model's segmentation performance without adding any computational complexity.

RESULTS

On the LITS dataset, we conducted rigorous comparative and ablation experiments, four key metrics were used for evaluation, including model size, inference speed, Dice coefficient, and HD95. Compared to other methods, our proposed model achieved an average Dice coefficient of 90.14±16.78%, with only 0.6 MB memory and 0.095 s inference speed for a single image on a standard CPU. Importantly, this approach improved the average Dice coefficient of the baseline student model by 1.64% without increasing computational complexity.

CONCLUSION

The results demonstrate that our method successfully realizes the unification of segmentation precision and lightness, and greatly enhances its potential for widespread application in practical settings.

ResTransUnet: An effective network combined with Transformer and U-Net for liver segmentation in CT scans

Liver segmentation is a fundamental prerequisite for the diagnosis and surgical planning of hepatocellular carcinoma. Traditionally, the liver contour is drawn manually by radiologists using a slice-by-slice method. However, this process is time-consuming and error-prone, depending on the radiologist's experience. In this paper, we propose a new end-to-end automatic liver segmentation framework, named ResTransUNet, which exploits the transformer's ability to capture global context for remote interactions and spatial relationships, as well as the excellent performance of the original U-Net architecture. The main contribution of this paper lies in proposing a novel fusion network that combines Unet and Transformer architectures. In the encoding structure, a dual-path approach is utilized, where features are extracted separately using both convolutional neural networks (CNNs) and Transformer networks. Additionally, an effective feature enhancement unit is designed to transfer the global features extracted by the Transformer network to the CNN for feature enhancement. This model aims to address the drawbacks of traditional Unet-based methods, such as feature loss during encoding and poor capture of global features. Moreover, it avoids the disadvantages of pure Transformer models, which suffer from large parameter sizes and high computational complexity. The experimental results on the LiTS2017 dataset demonstrate remarkable performance for our proposed model, with Dice coefficients, volumetric overlap error (VOE), and relative volume difference (RVD) values for liver segmentation reaching 0.9535, 0.0804, and -0.0007, respectively. Furthermore, to further validate the model's generalization capability, we conducted tests on the 3Dircadb, Chaos, and Sliver07 datasets. The experimental results demonstrate that the proposed method outperforms other closely related models with higher liver segmentation accuracy. In addition, significant improvements can be achieved by applying our method when handling liver segmentation with small and discontinuous liver regions, as well as blurred liver boundaries. The code is available at the website: https://github.com/Jouiry/ResTransUNet.

Multi-view attribute learning and context relationship encoding enhanced segmentation of lung tumors from CT images

Graph convolutional neural networks (GCN) have shown the promise in medical image segmentation due to the flexibility of representing diverse range of image regions using graph nodes and propagating knowledge via graph edges. However, existing methods did not fully exploit the various attributes of image nodes and the context relationship among their attributes. We propose a new segmentation method with multi-similarity view enhancement and node attribute context learning (MNSeg). First, multiple views were formed by measuring the similarities among the image nodes, and MNSeg has a GCN based multi-view image node attribute learning (MAL) module to integrate various node attributes learnt from multiple similarity views. Each similarity view contains the specific similarities among all the image nodes, and it was integrated with the node attributes from all the channels to form the enhanced attributes of image nodes. Second, the context relationships among the attributes of image nodes are formulated by a transformer-based context relationship encoding (CRE) strategy to propagate these relationships across all the image nodes. During the transformer-based learning, the relationships were estimated based on the self-attention on all the image nodes, and then they were encoded into the learned node features. Finally, we design an attention at attribute category level (ACA) to discriminate and fuse the learnt diverse information from MAL, CRE, and the original node attributes. ACA identifies the more informative attribute categories by adaptively learn their importance. We validate the performance of MNSeg on a public lung tumor CT dataset and an in-house non-small cell lung cancer (NSCLC) dataset collected from the hospital. The segmentation results show that MNSeg outperformed the compared segmentation methods in terms of spatial overlap and the shape similarities. The ablation studies demonstrated the effectiveness of MAL, CRE, and ACA. The generalization ability of MNSeg was proved by the consistent improved segmentation performances using different 3D segmentation backbones.

PLU-Net: Extraction of multiscale feature fusion

In recent years, deep learning algorithms have achieved remarkable results in medical image segmentation. These networks with an enormous number of parameters often encounter challenges in handling image boundaries and details, which may result in suboptimal segmentation results. To solve the problem, we develop atrous spatial pyramid pooling (ASPP) and combine it with the squeeze-and-excitation block (SE block), as well as present the PS module, which employs a broader and multiscale receptive field at the network's bottom to obtain more detailed semantic information. We also propose the local guided block (LG block) and also its combination with the SE block to form the LS block, which can obtain more abundant local features in the feature map, so that more edge information can be retained in each down sampling process, thereby improving the performance of boundary segmentation. We propose PLU-Net and integrate our PS module and LS block into U-Net. We put our PLU-Net to the test on three benchmark datasets, and the results show that by fewer parameters and FLOPs, it outperforms on medical semantic segmentation tasks.

Semantic segmentation for tooth cracks using improved DeepLabv3+ model

Objective

Accurate and prompt detection of cracked teeth plays a critical role for human oral health. The aim of this paper is to evaluate the performance of a tooth crack segmentation model (namely, FDB-DeepLabv3+) on optical microscopic images.

Method

The FDB-DeepLabv3+ model proposed here improves feature learning by replacing the backbone with ResNet50. Feature pyramid network (FPN) is introduced to fuse muti-level features. Densely linked atrous spatial pyramid pooling (Dense ASPP) is applied to achieve denser pixel sampling and wider receptive field. Bottleneck attention module (BAM) is embedded to enhance local feature extraction.

Results

Through testing on a self-made hidden cracked tooth dataset, the proposed method outperforms four classical networks (FCN, U-Net, SegNet, DeepLabv3+) on segmentation results in terms of mean pixel accuracy (MPA) and mean intersection over union (MIoU). The network achieves an increase of 11.41% in MPA and 12.14% in MIoU compared to DeepLabv3+. Ablation experiments shows that all the modifications are beneficial.

Conclusion

An improved network is designed for segmenting tooth surface cracks with good overall performance and robustness, which may hold significant potential in computer-aided diagnosis of cracked teeth.

HIMS-Net: Horizontal-vertical interaction and multiple side-outputs network for cyst segmentation in jaw images

Jaw cysts are mainly caused by abnormal tooth development, chronic oral inflammation, or jaw damage, which may lead to facial swelling, deformity, tooth loss, and other symptoms. Due to the diversity and complexity of cyst images, deep-learning algorithms still face many difficulties and challenges. In response to these problems, we present a horizontal-vertical interaction and multiple side-outputs network for cyst segmentation in jaw images. First, the horizontal-vertical interaction mechanism facilitates complex communication paths in the vertical and horizontal dimensions, and it has the ability to capture a wide range of context dependencies. Second, the feature-fused unit is introduced to adjust the network's receptive field, which enhances the ability of acquiring multi-scale context information. Third, the multiple side-outputs strategy intelligently combines feature maps to generate more accurate and detailed change maps. Finally, experiments were carried out on the self-established jaw cyst dataset and compared with different specialist physicians to evaluate its clinical usability. The research results indicate that the Matthews correlation coefficient (Mcc), Dice, and Jaccard of HIMS-Net were 93.61, 93.66 and 88.10% respectively, which may contribute to rapid and accurate diagnosis in clinical practice.

Automated anatomical labeling of the intracranial arteries via deep learning in computed tomography angiography

Background and purpose: Anatomical labeling of the cerebral vasculature is a crucial topic in determining the morphological nature and characterizing the vital variations of vessels, yet precise labeling of the intracranial arteries is time-consuming and challenging, given anatomical structural variability and surging imaging data. We present a U-Net-based deep learning (DL) model to automatically label detailed anatomical segments in computed tomography angiography (CTA) for the first time. The trained DL algorithm was further tested on a clinically relevant set for the localization of intracranial aneurysms (IAs). Methods: 457 examinations with varying degrees of arterial stenosis were used to train, validate, and test the model, aiming to automatically label 42 segments of the intracranial arteries [e.g., 7 segments of the internal carotid artery (ICA)]. Evaluation metrics included Dice similarity coefficient (DSC), mean surface distance (MSD), and Hausdorff distance (HD). Additionally, 96 examinations containing at least one IA were enrolled to assess the model's potential in enhancing clinicians' precision in IA localization. A total of 5 clinicians with different experience levels participated as readers in the clinical experiment and identified the precise location of IA without and with algorithm assistance, where there was a washout period of 14 days between two interpretations. The diagnostic accuracy, time, and mean interrater agreement (Fleiss' Kappa) were calculated to assess the differences in clinical performance of clinicians. Results: The proposed model exhibited notable labeling performance on 42 segments that included 7 anatomical segments of ICA, with the mean DSC of 0.88, MSD of 0.82 mm and HD of 6.59 mm. Furthermore, the model demonstrated superior labeling performance in healthy subjects compared to patients with stenosis (DSC: 0.91 vs. 0.89, p < 0.05; HD: 4.75 vs. 6.19, p < 0.05). Concurrently, clinicians with model predictions achieved significant improvements when interpreting the precise location of IA. The clinicians' mean accuracy increased by 0.04 (p = 0.003), mean time to diagnosis reduced by 9.76 s (p < 0.001), and mean interrater agreement (Fleiss' Kappa) increased by 0.07 (p = 0.029). Conclusion: Our model stands proficient for labeling intracranial arteries using the largest CTA dataset. Crucially, it demonstrates clinical utility, helping prioritize the patients with high risks and ease clinical workload.

PDC-Net: parallel dilated convolutional network with channel attention mechanism for pituitary adenoma segmentation

Accurate segmentation of the medical image is the basis and premise of intelligent diagnosis and treatment, which has a wide range of clinical application value. However, the robustness and effectiveness of medical image segmentation algorithms remains a challenging subject due to the unbalanced categories, blurred boundaries, highly variable anatomical structures and lack of training samples. For this reason, we present a parallel dilated convolutional network (PDC-Net) to address the pituitary adenoma segmentation in magnetic resonance imaging images. Firstly, the standard convolution block in U-Net is replaced by a basic convolution operation and a parallel dilated convolutional module (PDCM), to extract the multi-level feature information of different dilations. Furthermore, the channel attention mechanism (CAM) is integrated to enhance the ability of the network to distinguish between lesions and non-lesions in pituitary adenoma. Then, we introduce residual connections at each layer of the encoder-decoder, which can solve the problem of gradient disappearance and network performance degradation caused by network deepening. Finally, we employ the dice loss to deal with the class imbalance problem in samples. By testing on the self-established patient dataset from Quzhou People's Hospital, the experiment achieves 90.92% of Sensitivity, 99.68% of Specificity, 88.45% of Dice value and 79.43% of Intersection over Union (IoU).

Dynamically aggregating MLPs and CNNs for skin lesion segmentation with geometry regularization

BACKGROUND AND OBJECTIVE

Melanoma is a highly malignant skin tumor. Accurate segmentation of skin lesions from dermoscopy images is pivotal for computer-aided diagnosis of melanoma. However, blurred lesion boundaries, variable lesion shapes, and other interference factors pose a challenge in this regard.

METHODS

This work proposes a novel framework called CFF-Net (Cross Feature Fusion Network) for supervised skin lesion segmentation. The encoder of the network includes dual branches, where the CNNs branch aims to extract rich local features while MLPs branch is used to establish both the global-spatial-dependencies and global-channel-dependencies for precise delineation of skin lesions. Besides, a feature-interaction module between two branches is designed for strengthening the feature representation by allowing dynamic exchange of spatial and channel information, so as to retain more spatial details and inhibit irrelevant noise. Moreover, an auxiliary prediction task is introduced to learn the global geometric information, highlighting the boundary of the skin lesion.

RESULTS

Comprehensive experiments using four publicly available skin lesion datasets (i.e., ISIC 2018, ISIC 2017, ISIC 2016, and PH2) indicated that CFF-Net outperformed the state-of-the-art models. In particular, CFF-Net greatly increased the average Jaccard Index score from 79.71% to 81.86% in ISIC 2018, from 78.03% to 80.21% in ISIC 2017, from 82.58% to 85.38% in ISIC 2016, and from 84.18% to 89.71% in PH2 compared with U-Net. Ablation studies demonstrated the effectiveness of each proposed component. Cross-validation experiments in ISIC 2018 and PH2 datasets verified the generalizability of CFF-Net under different skin lesion data distributions. Finally, comparison experiments using three public datasets demonstrated the superior performance of our model.

CONCLUSION

The proposed CFF-Net performed well in four public skin lesion datasets, especially for challenging cases with blurred edges of skin lesions and low contrast between skin lesions and background. CFF-Net can be employed for other segmentation tasks with better prediction and more accurate delineation of boundaries.

TDS-U-Net: Automatic liver and tumor separate segmentation of CT volumes using attention gates1

Considering the heterogeneity, diffusive shape, and complex background of tumors, automatic segmentation of hepatic lesions in computed tomography (CT) images has been considered a challenging task. The performance of existing methods remains subject to segmentation uncertainties, especially in tumor boundary regions. The pixel information in these regions will be affected by both sides, thereby exposing the segmentation results to missing marks. To this end, a new network architecture named Two Direction Segmentation U-Net (TDS-U-Net) is hereby designed based on the classic Attention U-Net to tackle this problem. As the most important blocks of the Attention U-Net network, attention gates (AGs) focus on the target structures of different shapes and sizes. In the last layer of TDS-U-Net, two dichotomous convolutional networks are applied to obtain the segmentation maps of the liver and the tumor respectively. Superimposing two segmented maps to obtain the final image addresses the above problems. The entire structure has been verified on two widely accepted public CT datasets, LiTS17 and KiTS19. Compared with the state of the art, this method exhibits superior performance and excellent shape extractions with high detection sensitivity, perfectly demonstrating its effectiveness in medical image segmentation.

mfeeU-Net: A multi-scale feature extraction and enhancement U-Net for automatic liver segmentation from CT Images

Medical image segmentation of the liver is an important prerequisite for clinical diagnosis and evaluation of liver cancer. For automatic liver segmentation from Computed Tomography (CT) images, we proposed a Multi-scale Feature Extraction and Enhancement U-Net (mfeeU-Net), incorporating Res2Net blocks, Squeeze-and-Excitation (SE) blocks, and Edge Attention (EA) blocks. The Res2Net blocks which are conducive to extracting multi-scale features of the liver were used as the backbone of the encoder, while the SE blocks were also added to the encoder to enhance channel information. The EA blocks were introduced to skip connections between the encoder and the decoder, to facilitate the detection of blurred liver edges where the intensities of nearby organs are close to the liver. The proposed mfeeU-Net was trained and evaluated using a publicly available CT dataset of LiTS2017. The average dice similarity coefficient, intersection-over-union ratio, and sensitivity of the mfeeU-Net for liver segmentation were 95.32%, 91.67%, and 95.53%, respectively, and all these metrics were better than those of U-Net, Res-U-Net, and Attention U-Net. The experimental results demonstrate that the mfeeU-Net can compete with and even outperform recently proposed convolutional neural networks and effectively overcome challenges, such as discontinuous liver regions and fuzzy liver boundaries.

MRI brain tumor segmentation using residual Spatial Pyramid Pooling-powered 3D U-Net

Brain tumor diagnosis has been a lengthy process, and automation of a process such as brain tumor segmentation speeds up the timeline. U-Nets have been a commonly used solution for semantic segmentation, and it uses a downsampling-upsampling approach to segment tumors. U-Nets rely on residual connections to pass information during upsampling; however, an upsampling block only receives information from one downsampling block. This restricts the context and scope of an upsampling block. In this paper, we propose SPP-U-Net where the residual connections are replaced with a combination of Spatial Pyramid Pooling (SPP) and Attention blocks. Here, SPP provides information from various downsampling blocks, which will increase the scope of reconstruction while attention provides the necessary context by incorporating local characteristics with their corresponding global dependencies. Existing literature uses heavy approaches such as the usage of nested and dense skip connections and transformers. These approaches increase the training parameters within the model which therefore increase the training time and complexity of the model. The proposed approach on the other hand attains comparable results to existing literature without changing the number of trainable parameters over larger dimensions such as 160 × 192 × 192. All in all, the proposed model scores an average dice score of 0.883 and a Hausdorff distance of 7.84 on Brats 2021 cross validation.

APESTNet with Mask R-CNN for Liver Tumor Segmentation and Classification

Diagnosis and treatment of hepatocellular carcinoma or metastases rely heavily on accurate segmentation and classification of liver tumours. However, due to the liver tumor's hazy borders and wide range of possible shapes, sizes, and positions, accurate and automatic tumour segmentation and classification remains a difficult challenge. With the advancement of computing, new models in artificial intelligence have evolved. Following its success in Natural language processing (NLP), the transformer paradigm has been adopted by the computer vision (CV) community of the NLP. While there are already accepted approaches to classifying the liver, especially in clinical settings, there is room for advancement in terms of their precision. This paper makes an effort to apply a novel model for segmenting and classifying liver tumours built on deep learning. In order to accomplish this, the created model follows a three-stage procedure consisting of (a) pre-processing, (b) liver segmentation, and (c) classification. In the first phase, the collected Computed Tomography (CT) images undergo three stages of pre-processing, including contrast improvement via histogram equalization and noise reduction via the median filter. Next, an enhanced mask region-based convolutional neural networks (Mask R-CNN) model is used to separate the liver from the CT abdominal image. To prevent overfitting, the segmented picture is fed onto an Enhanced Swin Transformer Network with Adversarial Propagation (APESTNet). The experimental results prove the superior performance of the proposed perfect on a wide variety of CT images, as well as its efficiency and low sensitivity to noise.

GCS-YOLOV4-Tiny: A lightweight group convolution network for multi-stage fruit detection

Fruits require different planting techniques at different growth stages. Traditionally, the maturity stage of fruit is judged visually, which is time-consuming and labor-intensive. Fruits differ in size and color, and sometimes leaves or branches occult some of fruits, limiting automatic detection of growth stages in a real environment. Based on YOLOV4-Tiny, this study proposes a GCS-YOLOV4-Tiny model by (1) adding squeeze and excitation (SE) and the spatial pyramid pooling (SPP) modules to improve the accuracy of the model and (2) using the group convolution to reduce the size of the model and finally achieve faster detection speed. The proposed GCS-YOLOV4-Tiny model was executed on three public fruit datasets. Results have shown that GCS-YOLOV4-Tiny has favorable performance on mAP, Recall, F1-Score and Average IoU on Mango YOLO and Rpi-Tomato datasets. In addition, with the smallest model size of 20.70 MB, the mAP, Recall, F1-score, Precision and Average IoU of GCS-YOLOV4-Tiny achieve 93.42 ± 0.44, 91.00 ± 1.87, 90.80 ± 2.59, 90.80 ± 2.77 and 76.94 ± 1.35%, respectively, on F. margarita dataset. The detection results outperform the state-of-the-art YOLOV4-Tiny model with a 17.45% increase in mAP and a 13.80% increase in F1-score. The proposed model provides an effective and efficient performance to detect different growth stages of fruits and can be extended for different fruits and crops for object or disease detections.

Quantitative analysis of artificial intelligence on liver cancer: A bibliometric analysis

Objective

To provide the current research progress, hotspots, and emerging trends for AI in liver cancer, we have compiled a relative comprehensive and quantitative report on the research of liver disease using artificial intelligence by employing bibliometrics in this study.

Methods

In this study, the Web of Science Core Collection (WoSCC) database was used to perform systematic searches using keywords and a manual screening strategy, VOSviewer was used to analyze the degree of cooperation between countries/regions and institutions, as well as the co-occurrence of cooperation between authors and cited authors. Citespace was applied to generate a dual map to analyze the relationship of citing journals and citied journals and conduct a strong citation bursts ranking analysis of references. Online SRplot was used for in-depth keyword analysis and Microsoft Excel 2019 was used to collect the targeted variables from retrieved articles.

Results

1724 papers were collected in this study, including 1547 original articles and 177 reviews. The study of AI in liver cancer mostly began from 2003 and has developed rapidly from 2017. China has the largest number of publications, and the United States has the highest H-index and total citation counts. The top three most productive institutions are the League of European Research Universities, Sun Yat Sen University, and Zhejiang University. Jasjit S. Suri and Frontiers in Oncology are the most published author and journal, respectively. Keyword analysis showed that in addition to the research on liver cancer, research on liver cirrhosis, fatty liver disease, and liver fibrosis were also common. Computed tomography was the most used diagnostic tool, followed by ultrasound and magnetic resonance imaging. The diagnosis and differential diagnosis of liver cancer are currently the most widely adopted research goals, and comprehensive analyses of multi-type data and postoperative analysis of patients with advanced liver cancer are rare. The use of convolutional neural networks is the main technical method used in studies of AI on liver cancer.

Conclusion

AI has undergone rapid development and has a wide application in the diagnosis and treatment of liver diseases, especially in China. Imaging is an indispensable tool in this filed. Mmulti-type data fusion analysis and development of multimodal treatment plans for liver cancer could become the major trend of future research in AI in liver cancer.

GCHA-Net: Global context and hybrid attention network for automatic liver segmentation

Liver segmentation is a critical step in liver cancer diagnosis and surgical planning. The U-Net's architecture is one of the most efficient deep networks for medical image segmentation. However, the continuous downsampling operators in U-Net causes the loss of spatial information. To solve these problems, we propose a global context and hybrid attention network, called GCHA-Net, to adaptive capture the structural and detailed features. To capture the global features, a global attention module (GAM) is designed to model the channel and positional dimensions of the interdependencies. To capture the local features, a feature aggregation module (FAM) is designed, where a local attention module (LAM) is proposed to capture the spatial information. LAM can make our model focus on the local liver regions and suppress irrelevant information. The experimental results on the dataset LiTS2017 show that the dice per case (DPC) value and dice global (DG) value of liver were 96.5% and 96.9%, respectively. Compared with the state-of-the-art models, our model has superior performance in liver segmentation. Meanwhile, we test the experiment results on the 3Dircadb dataset, and it shows our model can obtain the highest accuracy compared with the closely related models. From these results, it can been seen that the proposed model can effectively capture the global context information and build the correlation between different convolutional layers. The code is available at the website: https://github.com/HuaxiangLiu/GCAU-Net.

Convolutional bi-directional learning and spatial enhanced attentions for lung tumor segmentation

BACKGROUND AND OBJECTIVE

Accurate lung tumor segmentation from computed tomography (CT) is complex due to variations in tumor sizes, shapes, patterns and growing locations. Learning semantic and spatial relations between different feature channels, image regions and positions is critical yet challenging.

METHODS

We propose a new segmentation method, PRCS, by learning and integrating multi-channel contextual relations, and spatial and position dependencies across image regions. Firstly, to extract contextual relationships between different deep image feature tensor channels, we propose a new convolutional bi-directional gated recurrent unit based module for forward and backward learning. Secondly, a novel cross-channel region-level attention mechanism is proposed to discriminate the contributions of different local regions and associated features in the global learning process. Finally, spatial and position dependencies are formulated by a new position-enhanced self-attention mechanism. The new attention can measure the diverse contributions of other positions to a target position and obtain an enhanced adaptive feature vector for the target position.

RESULTS

Our model outperformed seven state-of-the-art segmentation methods on both public and in-house lung tumor datasets in terms of spatial overlapping and shape similarity. Ablation study results proved the effectiveness of three technical innovations and generalization ability on different 3D CNN segmentation backbones.

CONCLUSION

The proposed model enhanced the learning and propagation of contextual, spatial and position relations in 3D volumes, improving lung tumours' segmentation performance with large variations and indistinct boundaries. PRCS provides an effective automated approach to support precision diagnosis and treatment planning of lung cancer.

Deep supervision and atrous inception-based U-Net combining CRF for automatic liver segmentation from CT

Due to low contrast and the blurred boundary between liver tissue and neighboring organs sharing similar intensity values, the problem of liver segmentation from CT images has not yet achieved satisfactory performance and remains a challenge. To alleviate these problems, we introduce deep supervision (DS) and atrous inception (AI) technologies with conditional random field (CRF) and propose three major improvements that are experimentally shown to have substantive and practical value. First, we replace the encoder's standard convolution with the residual block. Residual blocks can increase the depth of the network. Second, we provide an AI module to connect the encoder and decoder. AI allows us to obtain multi-scale features. Third, we incorporate the DS mechanism into the decoder. This helps to make full use of information of the shallow layers. In addition, we employ the Tversky loss function to balance the segmented and non-segmented regions and perform further refinement with a dense CRF. Finally, we extensively validate the proposed method on three public databases: LiTS17, 3DIRCADb, and SLiver07. Compared to the state-of-the-art methods, the proposed method achieved increased segmentation accuracy for the livers with low contrast and the fuzzy boundary between liver tissue and neighboring organs and is, therefore, more suited for automatic segmentation of these livers.

Decision Support System for Liver Lesion Segmentation Based on Advanced Convolutional Neural Network Architectures

Given its essential role in body functions, liver cancer is the third most common cause of death from cancer, despite being the sixth most common type of cancer worldwide. Following advancements in medicine and image processing, medical image segmentation methods are receiving a great deal of attention. As a novelty, the paper proposes an intelligent decision system for segmenting liver and hepatic tumors by integrating four efficient neural networks (ResNet152, ResNeXt101, DenseNet201, and InceptionV3). Images from computed tomography for training, validation, and testing were taken from the public LiTS17 database and preprocessed to better highlight liver tissue and tumors. Global segmentation is done by separately training individual classifiers and the global system of merging individual decisions. For the aforementioned application, classification neural networks have been modified for semantic segmentation. After segmentation based on the neural network system, the images were postprocessed to eliminate artifacts. The segmentation results obtained by the system were better, from the point of view of the Dice coefficient, than those obtained by the individual networks, and comparable with those reported in recent works.

Improved U-Net Remote Sensing Classification Algorithm Fusing Attention and Multiscale Features

The selection and representation of classification features in remote sensing image play crucial roles in image classification accuracy. To effectively improve the features classification accuracy, an improved U-Net remote sensing classification algorithm fusing attention and multiscale features is proposed in this paper, called spatial attention-atrous spatial pyramid pooling U-Net (SA-UNet). This framework connects atrous spatial pyramid pooling (ASPP) with the convolutional units of the encoder of the original U-Net in the form of residuals. The ASPP module expands the receptive field, integrates multiscale features in the network, and enhances the ability to express shallow features. Through the fusion residual module, shallow and deep features are deeply fused, and the characteristics of shallow and deep features are further used. The spatial attention mechanism is used to combine spatial with semantic information so that the decoder can recover more spatial information. In this study, the crop distribution in central Guangxi province was analyzed, and experiments were conducted based on Landsat 8 multispectral remote sensing images. The experimental results showed that the improved algorithm increases the classification accuracy, with the accuracy increasing from 93.33% to 96.25%, The segmentation accuracy of sugarcane, rice, and other land increased from 96.42%, 63.37%, and 88.43% to 98.01%, 83.21%, and 95.71%, respectively. The agricultural planting area results obtained by the proposed algorithm can be used as input data for regional ecological models, which is conducive to the development of accurate and real-time crop growth change models.

Residual based attention-Unet combing DAC and RMP modules for automatic liver tumor segmentation in CT

Purpose: Due to the complex distribution of liver tumors in the abdomen, the accuracy of liver tumor segmentation cannot meet the needs of clinical assistance yet. This paper aims to propose a new end-to-end network to improve the segmentation accuracy of liver tumors from CT. Method: We proposed a hybrid network, leveraging the residual block, the context encoder (CE), and the Attention-Unet, called ResCEAttUnet. The CE comprises a dense atrous convolution (DAC) module and a residual multi-kernel pooling (RMP) module. The DAC module ensures the network derives high-level semantic information and minimizes detailed information loss. The RMP module improves the ability of the network to extract multi-scale features. Moreover, a hybrid loss function based on cross-entropy and Tversky loss function is employed to distribute the weights of the two-loss parts through training iterations. Results: We evaluated the proposed method in LiTS17 and 3DIRCADb databases. It significantly improved the segmentation accuracy compared to state-of-the-art methods. Conclusions: Experimental results demonstrate the satisfying effects of the proposed method through both quantitative and qualitative analyses, thus proving a promising tool in liver tumor segmentation.

Improved U-Net Remote Sensing Classification Algorithm Based on Multi-Feature Fusion Perception

The selection and representation of remote sensing image classification features play crucial roles in image classification accuracy. To effectively improve the classification accuracy of features, an improved U-Net network framework based on multi-feature fusion perception is proposed in this paper. This framework adds the channel attention module (CAM-UNet) to the original U-Net framework and cascades the shallow features with the deep semantic features, replaces the classification layer in the original U-Net network with a support vector machine, and finally uses the majority voting game theory algorithm to fuse the multifeature classification results and obtain the final classification results. This study used the forest distribution in Xingbin District, Laibin City, Guangxi Zhuang Autonomous Region as the research object, which is based on Landsat 8 multispectral remote sensing images, and, by combining spectral features, spatial features, and advanced semantic features, overcame the influence of the reduction in spatial resolution that occurs with the deepening of the network on the classification results. The experimental results showed that the improved algorithm can improve classification accuracy. Before the improvement, the overall segmentation accuracy and segmentation accuracy of the forestland increased from 90.50% to 92.82% and from 95.66% to 97.16%, respectively. The forest cover results obtained by the algorithm proposed in this paper can be used as input data for regional ecological models, which is conducive to the development of accurate and real-time vegetation growth change models.

Developing an explainable deep learning boundary correction method by incorporating cascaded x-Dim models to improve segmentation defects in liver CT images

Deep learning methods achieved remarkable results in medical image analysis tasks but it has not yet been widely used by medical professionals. One of the main reasons for this restricted usage is the uncertainty of the reasons that influence the decision of the model. Explainable AI methods have been developed to improve the transparency, interpretability, and explainability of the black-box AI methods. The result of an explainable segmentation method will be more trusted by experts. In this study, we designed an explainable deep correction method by incorporating cascaded 1D and 2D models to refine the output of other models and provide reliable yet accurate results. We implemented a 2-step loop with a 1D local boundary validation model in the first step, and a 2D image patch segmentation model in the second step, to refine incorrect segmented regions slice-by-slice. The proposed method improved the result of the CNN segmentation models and achieved state-of-the-art results on 3D liver segmentation with the average dice coefficient of 98.27 on the Sliver07 dataset.

Liver segmentation from computed tomography images using cascade deep learning

BACKGROUND

Liver segmentation is a fundamental step in the treatment planning and diagnosis of liver cancer. However, manual segmentation of liver is time-consuming because of the large slice quantity and subjectiveness associated with the specialist's experience, which can lead to segmentation errors. Thus, the segmentation process can be automated using computational methods for better time efficiency and accuracy. However, automatic liver segmentation is a challenging task, as the liver can vary in shape, ill-defined borders, and lesions, which affect its appearance. We aim to propose an automatic method for liver segmentation using computed tomography (CT) images.

METHODS

The proposed method, based on deep convolutional neural network models and image processing techniques, comprise of four main steps: (1) image preprocessing, (2) initial segmentation, (3) reconstruction, and (4) final segmentation.

RESULTS

We evaluated the proposed method using 131 CT images from the LiTS image base. An average sensitivity of 95.45%, an average specificity of 99.86%, an average Dice coefficient of 95.64%, an average volumetric overlap error (VOE) of 8.28%, an average relative volume difference (RVD) of -0.41%, and an average Hausdorff distance (HD) of 26.60 mm were achieved.

CONCLUSIONS

This study demonstrates that liver segmentation, even when lesions are present in CT images, can be efficiently performed using a cascade approach and including a reconstruction step based on deep convolutional neural networks.