Stacked dilated convolutions and asymmetric architecture for U-Net-based medical image segmentation

Label-Decoupled Medical Image Segmentation With Spatial-Channel Graph Convolution and Dual Attention Enhancement

Deep learning-based methods have been widely used in medical image segmentation recently. However, existing works are usually difficult to simultaneously capture global long-range information from images and topological correlations among feature maps. Further, medical images often suffer from blurred target edges. Accordingly, this paper proposes a novel medical image segmentation framework named a label-decoupled network with spatial-channel graph convolution and dual attention enhancement mechanism (LADENet for short). It constructs learnable adjacency matrices and utilizes graph convolutions to effectively capture global long-range information on spatial locations and topological dependencies between different channels in an image. Then a label-decoupled strategy based on distance transformation is introduced to decouple an original segmentation label into a body label and an edge label for supervising the body branch and edge branch. Again, a dual attention enhancement mechanism, designing a body attention block in the body branch and an edge attention block in the edge branch, is built to promote the learning ability of spatial region and boundary features. Besides, a feature interactor is devised to fully consider the information interaction between the body and edge branches to improve segmentation performance. Experiments on benchmark datasets reveal the superiority of LADENet compared to state-of-the-art approaches.

An enhanced real-time human pose estimation method based on modified YOLOv8 framework

The objective of human pose estimation (HPE) derived from deep learning aims to accurately estimate and predict the human body posture in images or videos via the utilization of deep neural networks. However, the accuracy of real-time HPE tasks is still to be improved due to factors such as partial occlusion of body parts and limited receptive field of the model. To alleviate the accuracy loss caused by these issues, this paper proposes a real-time HPE model called CCAM - Person based on the YOLOv8 framework. Specifically, we have improved the backbone and neck of the YOLOv8x-pose real-time HPE model to alleviate the feature loss and receptive field constraints. Secondly, we introduce the context coordinate attention module (CCAM) to augment the model's focus on salient features, reduce background noise interference, alleviate key point regression failure caused by limb occlusion, and improve the accuracy of pose estimation. Our approach attains competitive results on multiple metrics of two open-source datasets, MS COCO 2017 and CrowdPose. Compared with the baseline model YOLOv8x-pose, CCAM-Person improves the average precision by 2.8% and 3.5% on the two datasets, respectively.

Dynamic parametric MRI and deep learning: Unveiling renal pathophysiology through accurate kidney size quantification

Renal pathologies often manifest as alterations in kidney size, providing a valuable avenue for employing dynamic parametric MRI as a means to derive kidney size measurements for the diagnosis, treatment, and monitoring of renal disease. Furthermore, this approach holds significant potential in supporting MRI data-driven preclinical investigations into the intricate mechanisms underlying renal pathophysiology. The integration of deep learning algorithms is crucial in achieving rapid and precise segmentation of the kidney from temporally resolved parametric MRI, facilitating the use of kidney size as a meaningful (pre)clinical biomarker for renal disease. To explore this potential, we employed dynamic parametric T2 mapping of the kidney in rats in conjunction with a custom-tailored deep dilated U-Net (DDU-Net) architecture. The architecture was trained, validated, and tested on manually segmented ground truth kidney data, with benchmarking against an analytical segmentation model and a self-configuring no new U-Net. Subsequently, we applied our approach to in vivo longitudinal MRI data, incorporating interventions that emulate clinically relevant scenarios in rats. Our approach achieved high performance metrics, including a Dice coefficient of 0.98, coefficient of determination of 0.92, and a mean absolute percentage error of 1.1% compared with ground truth. The DDU-Net enabled automated and accurate quantification of acute changes in kidney size, such as aortic occlusion (-8% ± 1%), venous occlusion (5% ± 1%), furosemide administration (2% ± 1%), hypoxemia (-2% ± 1%), and contrast agent-induced acute kidney injury (11% ± 1%). This approach can potentially be instrumental for the development of dynamic parametric MRI-based tools for kidney disorders, offering unparalleled insights into renal pathophysiology.

A multi-modal vision-language pipeline strategy for contour quality assurance and adaptive optimization

Objective.Accurate delineation of organs-at-risk (OARs) is a critical step in radiotherapy. The deep learning generated segmentations usually need to be reviewed and corrected by oncologists manually, which is time-consuming and operator-dependent. Therefore, an automated quality assurance (QA) and adaptive optimization correction strategy was proposed to identify and optimize 'incorrect' auto-segmentations.Approach.A total of 586 CT images and labels from nine institutions were used. The OARs included the brainstem, parotid, and mandible. The deep learning generated contours were compared with the manual ground truth delineations. In this study, we proposed a novel contour quality assurance and adaptive optimization (CQA-AO) strategy, which consists of the following three main components: (1) the contour QA module classified the deep learning generated contours as either accepted or unaccepted; (2) the unacceptable contour categories analysis module provided the potential error reasons (five unacceptable category) and locations (attention heatmaps); (3) the adaptive correction of unacceptable contours module integrate vision-language representations and utilize convex optimization algorithms to achieve adaptive correction of 'incorrect' contours.Main results. In the contour QA tasks, the sensitivity (accuracy, precision) of CQA-AO strategy reached 0.940 (0.945, 0.948), 0.962 (0.937, 0.913), and 0.967 (0.962, 0.957) for brainstem, parotid and mandible, respectively. The unacceptable contour category analysis, the(FI,AccI,Fmicro,Fmacro)of CQA-AO strategy reached (0.901, 0.763, 0.862, 0.822), (0.855, 0.737, 0.837, 0.784), and (0.907, 0.762, 0.858, 0.821) for brainstem, parotid and mandible, respectively. After adaptive optimization correction, the DSC values of brainstem, parotid and mandible have been improved by 9.4%, 25.9%, and 13.5%, and Hausdorff distance values decreased by 62%, 70.6%, and 81.6%, respectively.Significance. The proposed CQA-AO strategy, which combines QA of contour and adaptive optimization correction for OARs contouring, demonstrated superior performance compare to conventional methods. This method can be implemented in the clinical contouring procedures and improve the efficiency of delineating and reviewing workflow.

ParaCM-PNet: A CNN-tokenized MLP combined parallel dual pyramid network for prostate and prostate cancer segmentation in MRI

The precise prostate gland and prostate cancer (PCa) segmentations enable the fusion of magnetic resonance imaging (MRI) and ultrasound imaging (US) to guide robotic prostate biopsy systems. This precise segmentation, applied to preoperative MRI images, is crucial for accurate image registration and automatic localization of the biopsy target. Nevertheless, describing local prostate lesions in MRI remains a challenging and time-consuming task, even for experienced physicians. Therefore, this research work develops a parallel dual-pyramid network that combines convolutional neural networks (CNN) and tokenized multi-layer perceptron (MLP) for automatic segmentation of the prostate gland and clinically significant PCa (csPCa) in MRI. The proposed network consists of two stages. The first stage focuses on prostate segmentation, while the second stage uses a prior partition from a previous stage to detect the cancerous regions. Both stages share a similar network architecture, combining CNN and tokenized MLP as the feature extraction backbone to creating a pyramid-structured network for feature encoding and decoding. By employing CNN layers of different scales, the network generates scale-aware local semantic features, which are integrated into feature maps and inputted into an MLP layer from a global perspective. This facilitates the complementarity between local and global information, capturing richer semantic features. Additionally, the network incorporates an interactive hybrid attention module to enhance the perception of the target area. Experimental results demonstrate the superiority of the proposed network over other state-of-the-art image segmentation methods for segmenting the prostate gland and csPCa tissue in MRI images.

CFANet: Context fusing attentional network for preoperative CT image segmentation in robotic surgery

Accurate segmentation of CT images is crucial for clinical diagnosis and preoperative evaluation of robotic surgery, but challenges arise from fuzzy boundaries and small-sized targets. In response, a novel 2D segmentation network named Context Fusing Attentional Network (CFANet) is proposed. CFANet incorporates three key modules to address these challenges, namely pyramid fusing module (PFM), parallel dilated convolution module (PDCM) and scale attention module (SAM). Integration of these modules into the encoder-decoder structure enables effective utilization of multi-level and multi-scale features. Compared with advanced segmentation method, the Dice score improved by 2.14% on the dataset of liver tumor. This improvement is expected to have a positive impact on the preoperative evaluation of robotic surgery and to support clinical diagnosis, especially in early tumor detection.

HIMS-Net: Horizontal-vertical interaction and multiple side-outputs network for cyst segmentation in jaw images

Jaw cysts are mainly caused by abnormal tooth development, chronic oral inflammation, or jaw damage, which may lead to facial swelling, deformity, tooth loss, and other symptoms. Due to the diversity and complexity of cyst images, deep-learning algorithms still face many difficulties and challenges. In response to these problems, we present a horizontal-vertical interaction and multiple side-outputs network for cyst segmentation in jaw images. First, the horizontal-vertical interaction mechanism facilitates complex communication paths in the vertical and horizontal dimensions, and it has the ability to capture a wide range of context dependencies. Second, the feature-fused unit is introduced to adjust the network's receptive field, which enhances the ability of acquiring multi-scale context information. Third, the multiple side-outputs strategy intelligently combines feature maps to generate more accurate and detailed change maps. Finally, experiments were carried out on the self-established jaw cyst dataset and compared with different specialist physicians to evaluate its clinical usability. The research results indicate that the Matthews correlation coefficient (Mcc), Dice, and Jaccard of HIMS-Net were 93.61, 93.66 and 88.10% respectively, which may contribute to rapid and accurate diagnosis in clinical practice.

Accelerating the identification of the allergenic potential of plant proteins using a stacked ensemble-learning framework

Plant-allergenic proteins (PAPs) have the potential to induce allergic reactions in certain individuals. While these proteins are generally innocuous for the majority of people, they can elicit an immune response in those with particular sensitivities. Thus, screening and prioritizing the allergenic potential of plant proteins is indispensable for the development of diagnostic tools, therapeutic interventions or medications to treat allergic reactions. However, investigating the allergenic potential of plant proteins based on experimental methods is costly and labour-intensive. Therefore, we develop StackPAP, a three-layer stacking ensemble framework for accurate large-scale identification of PAPs. In StackPAP, at the first layer, we conducted a comprehensive analysis of an extensive set of feature descriptors. Subsequently, we selected and fused five potential sequence-based feature descriptors, including amphiphilic pseudo-amino acid composition, dipeptide deviation from expected mean, amino acid composition, pseudo amino acid composition and dipeptide composition. Additionally, we applied an efficient genetic algorithm (GA-SAR) to determine informative feature sets. In the second layer, 12 powerful machine learning (ML) methods, in combination with all the informative feature sets, were employed to construct a pool of base classifiers. Finally, 13 potential base classifiers were selected using the GA-SAR method and combined to develop the final meta-classifier. Our experimental results revealed the promising prediction performance of StackPAP, with an accuracy, Matthew's correlation coefficient and AUC of 0.984, 0.969 and 0.993, respectively, as judged by the independent test dataset. In conclusion, both cross-validation and independent test results indicated the superior performance of StackPAP compared with several ML-based classifiers. To accelerate the identification of the allergenicity of plant proteins, we developed a user-friendly web server for StackPAP (https://pmlabqsar.pythonanywhere.com/StackPAP). We anticipate that StackPAP will be an efficient and useful tool for rapidly screening PAPs from a vast number of plant proteins.Communicated by Ramaswamy H. Sarma.

A modified U-net with graph representation for dose prediction in esophageal cancer radiotherapy plans

The manual design of esophageal cancer radiotherapy plan is time-consuming and labor-intensive. Automatic planning (AP) is prevalent nowadays to increase physicists' work efficiency. Because of the intuitiveness of dose distribution in AP evaluation, obtaining reasonable dose prediction provides effective guarantees to generate a satisfactory AP. Existing fully convolutional network-based methods for predicting dose distribution in esophageal cancer radiotherapy plans often capture features in a limited receptive field. Additionally, the correlations between voxel pairs are often ignored. This work modifies the U-net architecture and exploits graph convolution to capture long-range information for dose prediction in esophageal cancer plans. Meanwhile, attention mechanism gets correlations between planning target volume (PTV) and organs at risk, and adaptively learns their feature weights. Finally, a novel loss function that considers features between voxel pairs is used to highlight the predictions. 152 subjects with prescription doses of 50 Gy or 60 Gy are collected in this study. The mean absolute error and standard deviation of conformity index, homogeneity index, and max dose for PTV achieved by the proposed method are 0.036 ± 0.030, 0.036 ± 0.027, and 0.930 ± 1.162, respectively, which outperform other state-of-the-art models. The superior performance demonstrates that our proposed method has great potential for AP generation.

Uncertainty-guided cross learning via CNN and transformer for semi-supervised honeycomb lung lesion segmentation

Objective. Deep learning networks such as convolutional neural networks (CNN) and Transformer have shown excellent performance on the task of medical image segmentation, however, the usual problem with medical images is the lack of large-scale, high-quality pixel-level annotations, which is a very time-consuming and laborious task, and its further leads to compromised the performance of medical image segmentation under limited annotation conditions.Approach. In this paper, we propose a new semi-supervised learning method, uncertainty-guided cross learning, which uses a limited number of annotated samples along with a large number of unlabeled images to train the network. Specifically, we use two networks with different learning paradigms, CNN and Transformer, for cross learning, and use the prediction of one of them as a pseudo label to supervise the other, so that they can learn from each other, fully extract the local and global features of the images, and combine explicit and implicit consistency regularization constraints with pseudo label methods. On the other hand, we use epistemic uncertainty as a guiding message to encourage the model to learn high-certainty pixel information in high-confidence regions, and minimize the impact of erroneous pseudo labels on the overall learning process to improve the performance of semi-supervised segmentation methods.Main results. We conducted honeycomb lung lesion segmentation experiments using a honeycomb lung CT image dataset, and designed several sets of comparison experiments and ablation experiments to validate the effectiveness of our method. The final experimental results show that the Dice coefficient of our proposed method reaches 88.49% on the test set, and our method achieves state-of-the-art performance in honeycomb lung lesion segmentation compared to other semi-supervised learning methods.Significance. Our proposed method can effectively improve the accuracy of segmentation of honeycomb lung lesions, which provides an important reference for physicians in the diagnosis and treatment of this disease.

BézierSeg: Parametric Shape Representation for Fast Object Segmentation in Medical Images

Background: Delineating the lesion area is an important task in image-based diagnosis. Pixel-wise classification is a popular approach to segmenting the region of interest. However, at fuzzy boundaries, such methods usually result in glitches, discontinuity or disconnection, inconsistent with the fact that lesions are solid and smooth. Methods: To overcome these problems and to provide an efficient, accurate, robust and concise solution that simplifies the whole segmentation pipeline in AI-assisted applications, we propose the BézierSeg model which outputs Bézier curves encompassing the region of interest. Results: Directly modeling the contour with analytic equations ensures that the segmentation is connected and continuous, and that the boundary is smooth. In addition, it offers sub-pixel accuracy. Without loss of precision, the Bézier contour can be resampled and overlaid with images of any resolution. Moreover, clinicians can conveniently adjust the curve’s control points to refine the result. Conclusions: Our experiments show that the proposed method runs in real time and achieves accuracy competitive with pixel-wise segmentation models.

SIL-Net: A Semi-Isotropic L-shaped network for dermoscopic image segmentation

BACKGROUND

Dermoscopic image segmentation using deep learning algorithms is a critical technology for skin cancer detection and therapy. Specifically, this technology is a spatially equivariant task and relies heavily on Convolutional Neural Networks (CNNs), which lost more effective features during cascading down-sampling or up-sampling. Recently, vision isotropic architecture has emerged to eliminate cascade procedures in CNNs as well as demonstrates superior performance. Nevertheless, it cannot be used for the segmentation task directly. Based on these discoveries, this research intends to explore an efficient architecture which not only preserves the advantages of the isotropic architecture but is also suitable for clinical dermoscopic diagnosis.

METHODS

In this work, we introduce a novel Semi-Isotropic L-shaped network (SIL-Net) for dermoscopic image segmentation. First, we propose a Patch Embedding Weak Correlation (PEWC) module to address the issue of no interaction between adjacent patches during the standard Patch Embedding process. Second, a plug-and-play and zero-parameter Residual Spatial Mirror Information (RSMI) path is proposed to supplement effective features during up-sampling and optimize the lesion boundaries. Third, to further reconstruct deep features and get refined lesion regions, a Depth Separable Transpose Convolution (DSTC) based up-sampling module is designed.

RESULTS

The proposed architecture obtains state-of-the-art performance on dermoscopy benchmark datasets ISIC-2017, ISIC-2018 and PH2. Respectively, the Dice coefficient (DICE) of above datasets achieves 89.63%, 93.47%, and 95.11%, where the Mean Intersection over Union (MIoU) are 82.02%, 88.21%, and 90.81%. Furthermore, the robustness and generalizability of our method has been demonstrated through additional experiments on standard intestinal polyp datasets (CVC-ClinicDB and Kvasir-SEG).

CONCLUSION

Our findings demonstrate that SIL-Net not only has great potential for precise segmentation of the lesion region but also exhibits stronger generalizability and robustness, indicating that it meets the requirements for clinical diagnosis. Notably, our method shows state-of-the-art performance on all five datasets, which highlights the effectiveness of the semi-isotropic design mechanism.