Automatic segmentation of cardiac magnetic resonance images based on multi-input fusion network

CardSegNet: An adaptive hybrid CNN-vision transformer model for heart region segmentation in cardiac MRI

Cardiovascular MRI (CMRI) is a non-invasive imaging technique adopted for assessing the blood circulatory system's structure and function. Precise image segmentation is required to measure cardiac parameters and diagnose abnormalities through CMRI data. Because of anatomical heterogeneity and image variations, cardiac image segmentation is a challenging task. Quantification of cardiac parameters requires high-performance segmentation of the left ventricle (LV), right ventricle (RV), and left ventricle myocardium from the background. The first proposed solution here is to manually segment the regions, which is a time-consuming and error-prone procedure. In this context, many semi- or fully automatic solutions have been proposed recently, among which deep learning-based methods have revealed high performance in segmenting regions in CMRI data. In this study, a self-adaptive multi attention (SMA) module is introduced to adaptively leverage multiple attention mechanisms for better segmentation. The convolutional-based position and channel attention mechanisms with a patch tokenization-based vision transformer (ViT)-based attention mechanism in a hybrid and end-to-end manner are integrated into the SMA. The CNN- and ViT-based attentions mine the short- and long-range dependencies for more precise segmentation. The SMA module is applied in an encoder-decoder structure with a ResNet50 backbone named CardSegNet. Furthermore, a deep supervision method with multi-loss functions is introduced to the CardSegNet optimizer to reduce overfitting and enhance the model's performance. The proposed model is validated on the ACDC2017 (n=100), M&Ms (n=321), and a local dataset (n=22) using the 10-fold cross-validation method with promising segmentation results, demonstrating its outperformance versus its counterparts.

MS-TCNet: An effective Transformer-CNN combined network using multi-scale feature learning for 3D medical image segmentation

Medical image segmentation is a fundamental research problem in the field of medical image processing. Recently, the Transformer have achieved highly competitive performance in computer vision. Therefore, many methods combining Transformer with convolutional neural networks (CNNs) have emerged for segmenting medical images. However, these methods cannot effectively capture the multi-scale features in medical images, even though texture and contextual information embedded in the multi-scale features are extremely beneficial for segmentation. To alleviate this limitation, we propose a novel Transformer-CNN combined network using multi-scale feature learning for three-dimensional (3D) medical image segmentation, which is called MS-TCNet. The proposed model utilizes a shunted Transformer and CNN to construct an encoder and pyramid decoder, allowing six different scale levels of feature learning. It captures multi-scale features with refinement at each scale level. Additionally, we propose a novel lightweight multi-scale feature fusion (MSFF) module that can fully fuse the different-scale semantic features generated by the pyramid decoder for each segmentation class, resulting in a more accurate segmentation output. We conducted experiments on three widely used 3D medical image segmentation datasets. The experimental results indicated that our method outperformed state-of-the-art medical image segmentation methods, suggesting its effectiveness, robustness, and superiority. Meanwhile, our model has a smaller number of parameters and lower computational complexity than conventional 3D segmentation networks. The results confirmed that the model is capable of effective multi-scale feature learning and that the learned multi-scale features are useful for improving segmentation performance. We open-sourced our code, which can be found at https://github.com/AustinYuAo/MS-TCNet.

Construction of multi-scale feature fusion segmentation model of MRI knee images based on dual attention mechanism weighted aggregation

BACKGROUND

Early diagnosis of knee osteoarthritis is an important area of research in the field of clinical medicine. Due to the complexity in the MRI imaging sequences and the diverse structure of cartilage, there are many challenges in the segmentation of knee bone and cartilage. Relevant studies have conducted semantic fusion processing through splicing or summing forms, which results in reduced resolution and the accumulation of redundant information.

OBJECTIVE

This study was envisaged to construct an MRI image segmentation model to improve the diagnostic efficiency and accuracy of different grade knee osteoarthritis by adopting the Dual Attention and Multi-scale Feature Fusion Segmentation network (DA-MFFSnet).

METHODS

The feature information of different scales was fused through the Multi-scale Attention Downsample module to extract more accurate feature information, and the Global Attention Upsample module weighted lower-level feature information to reduce the loss of key information.

RESULTS

The collected MRI knee images were screened and labeled, and the study results showed that the segmentation effect of DA-MFFSNet model was closer to that of the manually labeled images. The mean intersection over union, the dice similarity coefficient and the volumetric overlap error was 92.74%, 91.08% and 7.44%, respectively, and the accuracy of the differential diagnosis of knee osteoarthritis was 84.42%.

CONCLUSIONS

The model exhibited better stability and classification effect. Our results indicated that the Dual Attention and Multi-scale Feature Fusion Segmentation model can improve the segmentation effect of MRI knee images in mild and medium knee osteoarthritis, thereby offering an important clinical value and improving the accuracy of the clinical diagnosis.

MESTrans: Multi-scale embedding spatial transformer for medical image segmentation

BACKGROUND AND OBJECTIVE

Transformers profiting from global information modeling derived from the self-attention mechanism have recently achieved remarkable performance in computer vision. In this study, a novel transformer-based medical image segmentation network called the multi-scale embedding spatial transformer (MESTrans) was proposed for medical image segmentation.

METHODS

First, a dataset called COVID-DS36 was created from 4369 computed tomography (CT) images of 36 patients from a partner hospital, of which 18 had COVID-19 and 18 did not. Subsequently, a novel medical image segmentation network was proposed, which introduced a self-attention mechanism to improve the inherent limitation of convolutional neural networks (CNNs) and was capable of adaptively extracting discriminative information in both global and local content. Specifically, based on U-Net, a multi-scale embedding block (MEB) and multi-layer spatial attention transformer (SATrans) structure were designed, which can dynamically adjust the receptive field in accordance with the input content. The spatial relationship between multi-level and multi-scale image patches was modeled, and the global context information was captured effectively. To make the network concentrate on the salient feature region, a feature fusion module (FFM) was established, which performed global learning and soft selection between shallow and deep features, adaptively combining the encoder and decoder features. Four datasets comprising CT images, magnetic resonance (MR) images, and H&E-stained slide images were used to assess the performance of the proposed network.

RESULTS

Experiments were performed using four different types of medical image datasets. For the COVID-DS36 dataset, our method achieved a Dice similarity coefficient (DSC) of 81.23%. For the GlaS dataset, 89.95% DSC and 82.39% intersection over union (IoU) were obtained. On the Synapse dataset, the average DSC was 77.48% and the average Hausdorff distance (HD) was 31.69 mm. For the I2CVB dataset, 92.3% DSC and 85.8% IoU were obtained.

CONCLUSIONS

The experimental results demonstrate that the proposed model has an excellent generalization ability and outperforms other state-of-the-art methods. It is expected to be a potent tool to assist clinicians in auxiliary diagnosis and to promote the development of medical intelligence technology.

Deep learning-based image segmentation model using an MRI-based convolutional neural network for physiological evaluation of the heart

Background and Objective: Cardiovascular disease is a high-fatality health issue. Accurate measurement of cardiovascular function depends on precise segmentation of physiological structure and accurate evaluation of functional parameters. Structural segmentation of heart images and calculation of the volume of different ventricular activity cycles form the basis for quantitative analysis of physiological function and can provide the necessary support for clinical physiological diagnosis, as well as the analysis of various cardiac diseases. Therefore, it is important to develop an efficient heart segmentation algorithm. Methods: A total of 275 nuclear magnetic resonance imaging (MRI) heart scans were collected, analyzed, and preprocessed from Huaqiao University Affiliated Strait Hospital, and the data were used in our improved deep learning model, which was designed based on the U-net network. The training set included 80% of the images, and the remaining 20% was the test set. Based on five time phases from end-diastole (ED) to end-systole (ES), the segmentation findings showed that it is possible to achieve improved segmentation accuracy and computational complexity by segmenting the left ventricle (LV), right ventricle (RV), and myocardium (myo). Results: We improved the Dice index of the LV to 0.965 and 0.921, and the Hausdorff index decreased to 5.4 and 6.9 in the ED and ES phases, respectively; RV Dice increased to 0.938 and 0.860, and the Hausdorff index decreased to 11.7 and 12.6 in the ED and ES, respectively; myo Dice increased to 0.889 and 0.901, and the Hausdorff index decreased to 8.3 and 9.2 in the ED and ES, respectively. Conclusion: The model obtained in the final experiment provided more accurate segmentation of the left and right ventricles, as well as the myocardium, from cardiac MRI. The data from this model facilitate the prediction of cardiovascular disease in real-time, thereby providing potential clinical utility.

Transformer-based multilevel region and edge aggregation network for magnetic resonance image segmentation

To improve the quality of magnetic resonance (MR) image edge segmentation, some researchers applied additional edge labels to train the network to extract edge information and aggregate it with region information. They have made significant progress. However, due to the intrinsic locality of convolution operations, the convolution neural network-based region and edge aggregation has limitations in modeling long-range information. To solve this problem, we proposed a novel transformer-based multilevel region and edge aggregation network for MR image segmentation. To the best of our knowledge, this is the first literature on transformer-based region and edge aggregation. We first extract multilevel region and edge features using a dual-branch module. Then, the region and edge features at different levels are inferred and aggregated through multiple transformer-based inference modules to form multilevel complementary features. Finally, the attention feature selection module aggregates these complementary features with the corresponding level region and edge features to decode the region and edge features. We evaluated our method on a public MR dataset: Medical image computation and computer-assisted intervention atrial segmentation challenge (ASC). Meanwhile, the private MR dataset considered infrapatellar fat pad (IPFP). Our method achieved a dice score of 93.2% for ASC and 91.9% for IPFP. Compared with other 2D segmentation methods, our method improved a dice score by 0.6% for ASC and 3.0% for IPFP.

High-resolution computed tomography diagnosis of pneumoconiosis complicated with pulmonary tuberculosis based on cascading deep supervision U-Net

OBJECTIVE

Pulmonary tuberculosis can promote pneumoconiosis deterioration, leading to higher mortality. This study aims to explore the diagnostic value of the cascading deep supervision U-Net (CSNet) model in pneumoconiosis complicated with pulmonary tuberculosis.

METHODS

A total of 162 patients with pneumoconiosis treated in our hospital were collected as the research objects. Patients were randomly divided into a training set (n = 113) and a test set (n = 49) in proportion (7:3). Based on the high-resolution computed tomography (HRCT), the traditional U-Net, supervision U-Net (SNet), and CSNet prediction models were constructed. Dice similarity coefficients, precision, recall, volumetric overlap error, and relative volume difference were used to evaluate the segmentation model. The area under the receiver operating characteristic curve (AUC) value represents the prediction efficiency of the model.

RESULTS

There were no statistically significant differences in gender, age, number of positive patients, and dust contact time between patients in the training set and test set (P > 0.05). The segmentation results of CSNet are better than the traditional U-Net model and the SNet model. The AUC value of the CSNet model was 0.947 (95% CI: 0.900∼0.994), which was higher than the traditional U-Net model.

CONCLUSION

The CSNet based on chest HRCT proposed in this study is superior to the traditional U-Net segmentation method in segmenting pneumoconiosis complicated with pulmonary tuberculosis. It has good prediction efficiency and can provide more clinical diagnostic value.

Multiresolution Mutual Assistance Network for Cardiac Magnetic Resonance Images Segmentation

The automatic segmentation of cardiac magnetic resonance (MR) images is the basis for the diagnosis of cardiac-related diseases. However, the segmentation of cardiac MR images is a challenging task due to the inhomogeneity of MR images intensity distribution and the unclear boundaries between adjacent tissues. In this paper, we propose a novel multiresolution mutual assistance network (MMA-Net) for cardiac MR images segmentation. It is mainly composed of multibranch input module, multiresolution mutual assistance module, and multilabel deep supervision. First, the multibranch input module helps the network to extract local and global features more pertinently. Then, the multiresolution mutual assistance module implements multiresolution feature interaction and progressively improves semantic features to more completely express the information of the tissue. Finally, the multilabel deep supervision is proposed to generate the final segmentation map. We compare with state-of-the-art medical image segmentation methods on the medical image computing and computer-assisted intervention (MICCAI) automated cardiac diagnosis challenge datasets and the MICCAI atrial segmentation challenge datasets. The mean dice scores of our method in the left atrium, right ventricle, myocardium, and left ventricle are 0.919, 0.920, 0.881, and 0.960, respectively. The analysis of evaluation indicators and segmentation results shows that our method achieves the best performance in cardiac magnetic resonance images segmentation.

Classification of lungs infected COVID-19 images based on inception-ResNet

OBJECTIVE

Nowadays, COVID-19 is spreading rapidly worldwide, and seriously threatening lives . From the perspective of security and economy, the effective control of COVID-19 has a profound impact on the entire society. An effective strategy is to diagnose earlier to prevent the spread of the disease and prompt treatment of severe cases to improve the chance of survival.

METHODS

The method of this paper is as follows: Firstly, the collected data set is processed by chest film image processing, and the bone removal process is carried out in the rib subtraction module. Then, the set preprocessing method performed histogram equalization, sharpening, and other preprocessing operations on the chest film. Finally, shallow and high-level feature mapping through the backbone network extracts the processed chest radiographs. We implement the self-attention mechanism in Inception-Resnet, perform the standard classification, and identify chest radiograph diseases through the classifier to realize the auxiliary COVID-19 diagnosis process at the medical level, all in an effort to further enhance the classification performance of the convolutional neural network. Numerous computer simulations demonstrate that the Inception-Resnet convolutional neural network performs CT image categorization and enhancement with greater efficiency and flexibility than conventional segmentation techniques.

RESULTS

The experimental COVID-19 CT dataset obtained in this paper is the new data for CT scans and medical imaging of normal, early COVID-19 patients and severe COVID-19 patients from Jinyintan hospital. The experiment plots the relationship between model accuracy, model loss and epoch, using ACC, TPR, SPE, F1 score and G-mean to measure the image maps of patients with and without the disease. Statistical measurement values are obtained by Inception-Resnet are 88.23%, 83.45%, 89.72%, 95.53% and 88.74%. The experimental results show that Inception-Resnet plays a more effective role than other image classification methods in evaluation indicators, and the method has higher robustness, accuracy and intuitiveness.

CONCLUSION

With CT images in the clinical diagnosis of COVID-19 images being widely used and the number of applied samples continuously increasing, the method in this paper is expected to become an additional diagnostic tool that can effectively improve the diagnostic accuracy of clinical COVID-19 images.

Image Recognition of Pediatric Pneumonia Based on Fusion of Texture Features and Depth Features

Pneumonia is one of the diseases that seriously endangers human health, and it is also the leading cause of death of children under the age of five in China. The most commonly used imaging examination method for radiologists is mainly based on chest X-ray images. Still, imaging errors often result during imaging examinations due to objective factors such as visual fatigue and lack of experience. Therefore, this paper proposes a feature fusion model, FC-VGG, based on the fusion of texture features (local binary pattern LBP and directional gradient histogram HOG) and depth features. The model improves model performance by adding detailed information in texture features to the convolutional neural network while making the model more suitable for clinical use. We input the X-ray image with texture features into the modified VGG16 model, C-VGG, and then add the Add fusion method to C-VGG for feature fusion so that FC-VGG is obtained, so FC-VGG has texture features detailed information and abstract information of deep features. Through experiments, our model has achieved 92.19% accuracy in recognizing children's pneumonia images, 93.44% average precision, 92.19% average recall, and 92.81% average F1 coefficient, and the model performance exceeds existing deep learning models and traditional feature recognition algorithms.

Auxiliary Pneumonia Classification Algorithm Based on Pruning Compression

Pneumonia infection is the leading cause of death in young children. The commonly used pneumonia detection method is that doctors diagnose through chest X-ray, and external factors easily interfere with the results. Assisting doctors in diagnosing pneumonia in patients based on deep learning methods can effectively eliminate similar problems. However, the complex network structure and redundant parameters of deep neural networks and the limited storage and computing resources of clinical medical hardware devices make it difficult for this method to use widely in clinical practice. Therefore, this paper studies a lightweight pneumonia classification network, CPGResNet50 (ResNet50 with custom channel pruning and ghost methods), based on ResNet50 pruning and compression to better meet the application requirements of clinical pneumonia auxiliary diagnosis with high precision and low memory. First, based on the hierarchical channel pruning method, the channel after the convolutional layer in the bottleneck part of the backbone network layer is used as the pruning object, and the pruning operation is performed after its normalization to obtain a network model with a high compression ratio. Second, the pruned convolutional layers are decomposed into original convolutions and cheap convolutions using the optimized convolution method. The feature maps generated by the two convolution parts are combined as the input to the next convolutional layer. Further, we conducted many experiments using pneumonia X-ray medical image data. The results show that the proposed method reduces the number of parameters of the ResNet50 network model from 23.7 M to 3.455 M when the pruning rate is 90%, a reduction is more than 85%, FIOPs dropped from 4.12G to 523.09 M, and the speed increased by more than 85%. The model training accuracy error remained within 1%. Therefore, the proposed method has a good performance in the auxiliary diagnosis of pneumonia and obtained good experimental results.

Deep Neural Network for Cardiac Magnetic Resonance Image Segmentation

The analysis and interpretation of cardiac magnetic resonance (CMR) images are often time-consuming. The automated segmentation of cardiac structures can reduce the time required for image analysis. Spatial similarities between different CMR image types were leveraged to jointly segment multiple sequences using a segmentation model termed a multi-image type UNet (MI-UNet). This model was developed from 72 exams (46% female, mean age 63 ± 11 years) performed on patients with hypertrophic cardiomyopathy. The MI-UNet for steady-state free precession (SSFP) images achieved a superior Dice similarity coefficient (DSC) of 0.92 ± 0.06 compared to 0.87 ± 0.08 for a single-image type UNet (p < 0.001). The MI-UNet for late gadolinium enhancement (LGE) images also had a superior DSC of 0.86 ± 0.11 compared to 0.78 ± 0.11 for a single-image type UNet (p = 0.001). The difference across image types was most evident for the left ventricular myocardium in SSFP images and for both the left ventricular cavity and the left ventricular myocardium in LGE images. For the right ventricle, there were no differences in DCS when comparing the MI-UNet with single-image type UNets. The joint segmentation of multiple image types increases segmentation accuracy for CMR images of the left ventricle compared to single-image models. In clinical practice, the MI-UNet model may expedite the analysis and interpretation of CMR images of multiple types.

Classification and Segmentation Algorithm in Benign and Malignant Pulmonary Nodules under Different CT Reconstruction

Methods

The imaging data of 55 patients with chest CT plain scan in the Xuancheng People's Hospital were collected retrospectively. The data of each patient included lung window reconstruction, mediastinum reconstruction, and bone window reconstruction. The depth neural network and 3D convolution neural network were used to construct the model and train the classification and segmentation algorithm. The pathological results were the gold standard for benign and malignant pulmonary nodules. The classification and segmentation algorithms under three CT reconstruction algorithms were compared and analyzed by analysis of variance.

Results

Under the three CT reconstruction algorithms, the classification accuracy of pulmonary nodule density types was 98.2%, 96.4%, and 94.5%, respectively. The Dice coefficients of all nodule segmentation were 80.32% ± 5.91%, 79.83% ± 6.12%, and 80.17% ± 5.89%, respectively. The diagnostic accuracy between benign and malignant pulmonary nodules under different reconstruction algorithms was 98.2%, 96.4%, and 94.5%, respectively. There was no significant difference in the classification accuracy, Dice coefficients, and diagnostic accuracy of pulmonary nodules under three different reconstruction algorithms (all P > 0.05).

Conclusion

The depth neural network algorithm combined with 3D convolution neural network has a good efficiency in identifying benign and malignant pulmonary nodules under different CT reconstruction classification and segmentation algorithms.

Image Enhancement Model Based on Deep Learning Applied to the Ureteroscopic Diagnosis of Ureteral Stones during Pregnancy

OBJECTIVE

To explore the image enhancement model based on deep learning on the effect of ureteroscopy with double J tube placement and drainage on ureteral stones during pregnancy. We compare the clinical effect of ureteroscopy with double J tube placement on pregnancy complicated with ureteral stones and use medical imaging to diagnose the patient's condition and design a treatment plan.

METHODS

The image enhancement model is constructed using deep learning and implemented for quality improvement in terms of image clarity. In the way, the relationship of the media transmittance and the image with blurring artifacts was established, and the model can estimate the ureteral stone predicted map of each region. Firstly, we proposed the evolution-based detail enhancement method. Then, the feature extraction network is used to capture blurring artifact-related features. Finally, the regression subnetwork is used to predict the media transmittance in the local area. Eighty pregnant patients with ureteral calculi treated in our hospital were selected as the research object and were divided into a test group and a control group according to the random number table method, 40 cases in each group. The test group underwent ureteroscopy double J tube placement, and the control group underwent ureteroscopy lithotripsy. Combined with the ultrasound scan results of the patients before and after the operation, the operation time, time to get out of bed, and hospitalization time of the two groups of patients were compared. The operation success rate and the incidence of complications within 1 month after surgery were counted in the two groups of patients.

RESULTS

We are able to improve the quality of the images prior to medical diagnosis. The total effective rate of the observation group was 100.0%, which is higher than that of the control group (90.0%). The difference between the two groups was statistically significant (P < 0.05). The adverse reaction rate in the observation group was 5.0%, which was lower than 17.5% in the control group. The difference between the two groups was statistically significant (P < 0.05). The comparison results are then prepared.

CONCLUSIONS

The image enhancement model based on deep learning is able to improve medical diagnosis which can assist radiologists to better locate the ureteral stones. Based on our method, double J tube placement under ureteroscopy has a significant effect on the treatment of ureteral stones during pregnancy, and it has good safety and is worthy of widespread application.

Segmentation of Prefrontal Lobe Based on Improved Clustering Algorithm in Patients with Diabetes

Diabetics are prone to postoperative cognitive dysfunction (POCD). The occurrence may be related to the damage of the prefrontal lobe. In this study, the prefrontal lobe was segmented based on an improved clustering algorithm in patients with diabetes, in order to evaluate the relationship between prefrontal lobe volume and COPD. In this study, a total of 48 diabetics who underwent selective noncardiac surgery were selected. Preoperative magnetic resonance imaging (MRI) images of the patients were segmented based on the improved clustering algorithm, and their prefrontal volume was measured. The correlation between the volume of the prefrontal lobe and Z-score or blood glucose was analyzed. Qualitative analysis shows that the gray matter, white matter, and cerebrospinal fluid based on the improved clustering algorithm were easy to distinguish. Quantitative evaluation results show that the proposed segmentation algorithm can obtain the optimal Jaccard coefficient and the least average segmentation time. There was a negative correlation between the volume of the prefrontal lobe and the Z-score. The cut-off value of prefrontal lobe volume for predicting POCD was <179.8, with the high specificity. There was a negative correlation between blood glucose and volume of the prefrontal lobe. From the results, we concluded that the segmentation of the prefrontal lobe based on an improved clustering algorithm before operation may predict the occurrence of POCD in diabetics.

A deep learning based automatic segmentation approach for anatomical structures in intensity modulation radiotherapy

OBJECTIVE

To evaluate the automatic segmentation approach for organ at risk (OARs) and compare the parameters of dose volume histogram (DVH) in radiotherapy.

METHODOLOGY

Thirty-three patients were selected to contour OARs using automatic segmentation approach which based on U-Net, applying them to a number of the nasopharyngeal carcinoma (NPC), breast, and rectal cancer respectively. The automatic contours were transferred to the Pinnacle System to evaluate contour accuracy and compare the DVH parameters.

RESULTS

The time for manual contour was 56.5 ± 9, 23.12 ± 4.23 and 45.23 ± 2.39min for the OARs of NPC, breast and rectal cancer, and for automatic contour was 1.5 ± 0.23, 1.45 ± 0.78 and 1.8 ± 0.56 min. Automatic contours of Eye with the best Dice-similarity coefficients (DSC) of 0.907 ± 0.02 while with the poorest DSC of 0.459 ± 0.112 of Spinal Cord for NPC; And Lung with the best DSC of 0.944 ± 0.03 while with the poorest DSC of 0.709 ± 0.1 of Spinal Cord for breast; And Bladder with the best DSC of 0.91 ± 0.04 while with the poorest DSC of 0.43 ± 0.1 of Femoral heads for rectal cancer. The contours of Spinal Cord in H & N had poor results due to the division of the medulla oblongata. The contours of Femoral head, which different from what we expect, also due to manual contour result in poor DSC.

CONCLUSION

The automatic contour approach based deep learning method with sufficient accuracy for research purposes. However, the value of DSC does not fully reflect the accuracy of dose distribution, but can cause dose changes due to the changes in the OARs volume and DSC from the data. Considering the significantly time-saving and good performance in partial OARs, the automatic contouring also plays a supervisory role.