GAT TransPruning: progressive channel pruning strategy combining graph attention network and transformer

Recently, large-scale artificial intelligence models with billions of parameters have achieved good results in experiments, but their practical deployment on edge computing platforms is often subject to many constraints because of their resource requirements. These models require powerful computing platforms with a high memory capacity to store and process the numerous parameters and activations, which makes it challenging to deploy these large-scale models directly. Therefore, model compression techniques are crucial role in making these models more practical and accessible. In this article, a progressive channel pruning strategy combining graph attention network and transformer, namely GAT TransPruning, is proposed, which uses the graph attention networks (GAT) and the attention of transformer mechanism to determine the channel-to-channel relationship in large networks. This approach ensures that the network maintains its critical functional connections and optimizes the trade-off between model size and performance. In this study, VGG-16, VGG-19, ResNet-18, ResNet-34, and ResNet-50 are used as large-scale network models with the CIFAR-10 and CIFAR-100 datasets for verification and quantitative analysis of the proposed progressive channel pruning strategy. The experimental results reveal that the accuracy rate only drops by 6.58% when the channel pruning rate is 89% for VGG-19/CIFAR-100. In addition, the lightweight model inference speed is 9.10 times faster than that of the original large model. In comparison with the traditional channel pruning schemes, the proposed progressive channel pruning strategy based on the GAT and Transformer cannot only cut out the insignificant weight channels and effectively reduce the model size, but also ensure that the performance drop rate of its lightweight model is still the smallest even under high pruning ratio.

SM-CycleGAN: crop image data enhancement method based on self-attention mechanism CycleGAN

Crop disease detection and crop baking stage judgement require large image data to improve accuracy. However, the existing crop disease image datasets have high asymmetry, and the poor baking environment leads to image acquisition difficulties and colour distortion. Therefore, we explore the potential of the self-attention mechanism on crop image datasets and propose an innovative crop image data-enhancement method for recurrent generative adversarial networks (GANs) fused with the self-attention mechanism to significantly enhance the perception and information capture capabilities of recurrent GANs. By introducing the self-attention mechanism module, the cycle-consistent GAN (CycleGAN) is more adept at capturing the internal correlations and dependencies of image data, thus more effectively capturing the critical information among image data. Furthermore, we propose a new enhanced loss function for crop image data to optimise the model performance and meet specific task requirements. We further investigate crop image data enhancement in different contexts to validate the performance and stability of the model. The experimental results show that, the peak signal-to-noise ratio of the SM-CycleGAN for tobacco images and tea leaf disease images are improved by 2.13% and 3.55%, and the structural similarity index measure is improved by 1.16% and 2.48% compared to CycleGAN, respectively.

Mutual Correlation Network for few-shot learning

Most metric-based Few-Shot Learning (FSL) methods focus on learning good embeddings of images. However, these methods either lack the ability to explore the cross-correlation (i.e., correlated information) between image pairs or explore limited consensus among the correlation map constrained by the limited receptive field of CNN. We propose a Mutual Correlation Network (MCNet) to explore global consensus among the correlation map by using the self-attention mechanism which has a global receptive field. Our MCNet contains two core modules: (1) a multi-level embedding module that generates multi-level embeddings for an image pair which capture hierarchical semantics, and (2) a mutual correlation module that refines correlation map of two embeddings and generates more robust relational embeddings. Extensive experiments show that our MCNet achieves competitive results on four widely-used few-shot classification benchmarks miniImageNet, tieredImageNet, CUB-200-2011, and CIFAR-FS. Code is available at https://github.com/DRGreat/MCNet.

MGDHGS: Gene-bridged metabolite-disease relationships prediction via GraphSAGE and self-attention mechanism

Metabolites represent the underlying information of biological systems. Revealing the links between metabolites and diseases can facilitate the development of targeted drugs. Traditional biological experiments can be used to validate the relationships of metabolite-disease, but these methods are time-consuming and labor-intensive. In contrast, the prevailing computational methods have improved efficiency but primarily rely on the metabolite-disease interactions, overlooking the impact of other biological components. To remedy the problem, we present a novel computational framework (MGDHGS) based on metabolite-gene-disease heterogeneous network to forecast potential associations. Specifically, we initially integrate data from multiple sources to construct metabolite-gene-disease heterogeneous network that includes known associations and computationally-derived similarities. Then, the GraphSAGE is harnessed to learn the low dimensional neighborhood representation in the heterogeneous network and self-attention mechanism is applied to effectively capture the connectivity patterns, which contributions to combine with nodes intrinsic and extrinsic features. Finally, the ultimate relationships probability scores are predicted by linear regression based on the these characteristics. The five-fold cross-validation showcases impressive AUC (0.9734) and PR (0.9718) for MGDHGS compared with five state-of-the-art methods, and the case studies validate that the metabolite-disease associations predicted by MGDHGS can be substantiated through pertinent biological experiments. The findings of this study show great potential contribution in the development of targeted drugs as well as offering solid support for our understanding of the complex interactions between metabolites, genes and diseases.

ResDeepSurv: A Survival Model for Deep Neural Networks Based on Residual Blocks and Self-attention Mechanism

Survival analysis, as a widely used method for analyzing and predicting the timing of event occurrence, plays a crucial role in the medicine field. Medical professionals utilize survival models to gain insight into the effects of patient covariates on the disease, and the correlation with the effectiveness of different treatment strategies. This knowledge is essential for the development of treatment plans and the enhancement of treatment approaches. Conventional survival models, such as the Cox proportional hazards model, require a significant amount of feature engineering or prior knowledge to facilitate personalized modeling. To address these limitations, we propose a novel residual-based self-attention deep neural network for survival modeling, called ResDeepSurv, which combines the benefits of neural networks and the Cox proportional hazards regression model. The model proposed in our study simulates the distribution of survival time and the correlation between covariates and outcomes, but does not impose strict assumptions on the basic distribution of survival data. This approach effectively accounts for both linear and nonlinear risk functions in survival data analysis. The performance of our model in analyzing survival data with various risk functions is on par with or even superior to that of other existing survival analysis methods. Furthermore, we validate the superior performance of our model in comparison to currently existing methods by evaluating multiple publicly available clinical datasets. Through this study, we prove the effectiveness of our proposed model in survival analysis, providing a promising alternative to traditional approaches. The application of deep learning techniques and the ability to capture complex relationships between covariates and survival outcomes without relying on extensive feature engineering make our model a valuable tool for personalized medicine and decision-making in clinical practice.

i5mC-DCGA: an improved hybrid network framework based on the CBAM attention mechanism for identifying promoter 5mC sites

BACKGROUND

5-Methylcytosine (5mC) plays a very important role in gene stability, transcription, and development. Therefore, accurate identification of the 5mC site is of key importance in genetic and pathological studies. However, traditional experimental methods for identifying 5mC sites are time-consuming and costly, so there is an urgent need to develop computational methods to automatically detect and identify these 5mC sites.

RESULTS

Deep learning methods have shown great potential in the field of 5mC sites, so we developed a deep learning combinatorial model called i5mC-DCGA. The model innovatively uses the Convolutional Block Attention Module (CBAM) to improve the Dense Convolutional Network (DenseNet), which is improved to extract advanced local feature information. Subsequently, we combined a Bidirectional Gated Recurrent Unit (BiGRU) and a Self-Attention mechanism to extract global feature information. Our model can learn feature representations of abstract and complex from simple sequence coding, while having the ability to solve the sample imbalance problem in benchmark datasets. The experimental results show that the i5mC-DCGA model achieves 97.02%, 96.52%, 96.58% and 85.58% in sensitivity (Sn), specificity (Sp), accuracy (Acc) and matthews correlation coefficient (MCC), respectively.

CONCLUSIONS

The i5mC-DCGA model outperforms other existing prediction tools in predicting 5mC sites, and it is currently the most representative promoter 5mC site prediction tool. The benchmark dataset and source code for the i5mC-DCGA model can be found in https://github.com/leirufeng/i5mC-DCGA .

Boundary attention with multi-task consistency constraints for semi-supervised 2D echocardiography segmentation

The 2D echocardiography semantic automatic segmentation technique is important in clinical applications for cardiac function assessment and diagnosis of cardiac diseases. However, automatic segmentation of 2D echocardiograms also faces the problems of loss of image boundary information, loss of image localization information, and limitations in data acquisition and annotation. To address these issues, this paper proposes a semi-supervised echocardiography segmentation method. It consists of two models: (1) a boundary attention transformer net (BATNet) and (2) a multi-task level semi-supervised model with consistency constraints on boundary features (semi-BATNet). BATNet is able to capture the location and spatial information of the input feature maps by using the self-attention mechanism. The multi-task level semi-supervised model with boundary feature consistency constraints (semi-BATNet) encourages consistent predictions of boundary features at different scales from the student and teacher networks to calculate the multi-scale consistency loss for unlabeled data. The proposed semi-BATNet was extensively evaluated on the dataset of cardiac acquisitions for multi-structure ultrasound segmentation (CAMUS) and self-collected echocardiography dataset from the First Affiliated Hospital of Chongqing Medical University. Experimental results on the CAMUS dataset showed that when only 25% of the images are labeled, the proposed method greatly improved the segmentation performance by utilizing unlabeled images, and it also outperformed five state-of-the-art semi-supervised segmentation methods. Moreover, when only 50% of the images labeled, semi-BATNet achieved the Dice coefficient values of 0.936, the Jaccard similarity of 0.881 on self-collected echocardiography dataset. Semi-BATNet can complete a more accurate segmentation of cardiac structures in 2D echocardiograms, indicating that it has the potential to accurately and efficiently assist cardiologists.

LSA-ac4C: A hybrid neural network incorporating double-layer LSTM and self-attention mechanism for the prediction of N4-acetylcytidine sites in human mRNA

N4-acetylcytidine (ac4C) is a vital constituent of the epitranscriptome and plays a crucial role in the regulation of mRNA expression. Numerous studies have established correlations between ac4C and the incidence, progression and prognosis of various cancers. Therefore, accurately predicting ac4C sites is an important step towards comprehending the biological functions of this modification and devising effective therapeutic interventions. Wet experiments are primary methods for studying ac4C, but computational methods have emerged as a promising supplement due to their cost-effectiveness and shorter research cycles. However, current models still have inherent limitations in terms of predictive performance and generalization ability. Here, we utilized automated machine learning technology to establish a reliable baseline and constructed a deep hybrid neural network, LSA-ac4C, which combines double-layer Long Short-Term Memory (LSTM) and self-attention mechanism for accurate ac4C sites prediction. Benchmarking comparisons demonstrate that LSA-ac4C exhibits superior performance compared to the current state-of-the-art method, with ACC, MCC and AUROC improving by 2.89 %, 5.96 % and 1.53 %, respectively, on an independent test set. Overall, LSA-ac4C serves as a powerful tool for predicting ac4C sites in human mRNA, thus benefiting research on RNA modification. For the convenience of the research community, a web server has been established at http://tubic.org/ac4C.

HViT: Hybrid vision inspired transformer for the assessment of carotid artery plaque by addressing the cross-modality domain adaptation problem in MRI

BACKGROUND

Medical image classification is crucial for accurate and efficient diagnosis, and deep learning frameworks have shown significant potential in this area. When a general learning deep model is directly deployed to a new dataset with heterogeneous features, the effect of domain shifts is usually ignored, which degrades the performance of deep learning models and leads to inaccurate predictions.

PURPOSE

This study aims to propose a framework that utilized the cross-modality domain adaptation and accurately diagnose and classify MRI scans and domain knowledge into stable and vulnerable plaque categories by a modified Vision Transformer (ViT) model for the classification of MRI scans and transformer model for domain knowledge classification.

METHODS

This study proposes a Hybrid Vision Inspired Transformer (HViT) framework that employs a convolutional layer for image pre-processing and normalization and a 3D convolutional layer to enable ViT to classify 3D images. Our proposed HViT framework introduces a slim design with a multi-branch network and channel attention, improving patch embedding extraction and information learning. Auxiliary losses target shallow features, linking them with deeper ones, enhancing information gain, and model generalization. Furthermore, replacing the MLP Head with RNN enables better backpropagation for improved performance. Moreover, we utilized a modified transformer model with LSTM positional encoding and Golve word vector to classify domain knowledge. By using ensemble learning techniques, specifically stacking ensemble learning with hard and soft prediction, we combine the predictive power of both models to address the cross-modality domain adaptation problem and improve overall performance.

RESULTS

The proposed framework achieved an accuracy of 94.32% for carotid artery plaque classification into stable and vulnerable plaque by addressing the cross-modality domain adaptation problem and improving overall performance.

CONCLUSION

The model was further evaluated using an independent dataset acquired from different hardware protocols. The results demonstrate that the proposed deep learning model significantly improves the generalization ability across different MRI scans acquired from different hardware protocols without requiring additional calibration data.

Feature hypergraph representation learning on spatial-temporal correlations for EEG emotion recognition

Electroencephalogram(EEG) becomes popular in emotion recognition for its capability of selectively reflecting the real emotional states. Existing graph-based methods have made primary progress in representing pairwise spatial relationships, but leaving higher-order relationships among EEG channels and higher-order relationships inside EEG series. Constructing a hypergraph is a general way of representing higher-order relations. In this paper, we propose a spatial-temporal hypergraph convolutional network(STHGCN) to capture higher-order relationships that existed in EEG recordings. STHGCN is a two-block hypergraph convolutional network, in which feature hypergraphs are constructed over the spectrum, space, and time domains, to explore spatial and temporal correlations under specific emotional states, namely the correlations of EEG channels and the dynamic relationships of temporal stamps. What's more, a self-attention mechanism is combined with the hypergraph convolutional network to initialize and update the relationships of EEG series. The experimental results demonstrate that constructed feature hypergraphs can effectively capture the correlations among valuable EEG channels and the correlations inside valuable EEG series, leading to the best emotion recognition accuracy among the graph methods. In addition, compared with other competitive methods, the proposed method achieves state-of-art results on SEED and SEED-IV datasets.

End to end vision transformer architecture for brain stroke assessment based on multi-slice classification and localization using computed tomography

BACKGROUND

Brain stroke is a leading cause of disability and death worldwide, and early diagnosis and treatment are critical to improving patient outcomes. Current stroke diagnosis methods are subjective and prone to errors, as radiologists rely on manual selection of the most important CT slice. This highlights the need for more accurate and reliable automated brain stroke diagnosis and localization methods to improve patient outcomes.

PURPOSE

In this study, we aimed to enhance the vision transformer architecture for the multi-slice classification of CT scans of each patient into three categories, including Normal, Infarction, Hemorrhage, and patient-wise stroke localization, based on end-to-end vision transformer architecture. This framework can provide an automated, objective, and consistent approach to stroke diagnosis and localization, enabling personalized treatment plans based on the location and extent of the stroke.

METHODS

We modified the Vision Transformer (ViT) in combination with neural network layers for the multi-slice classification of brain CT scans of each patient into normal, infarction, and hemorrhage classes. For stroke localization, we used the ViT architecture and convolutional neural network layers to detect stroke and localize it by bounding boxes for infarction and hemorrhage regions in a patient-wise manner based on multi slices.

RESULTS

Our proposed framework achieved an overall accuracy of 87.51% in classifying brain CT scan slices and showed high precision in localizing the stroke patient-wise. Our results demonstrate the potential of our method for accurate and reliable stroke diagnosis and localization.

CONCLUSION

Our study enhanced ViT architecture for automated stroke diagnosis and localization using brain CT scans, which could have significant implications for stroke management and treatment. The use of deep learning algorithms can provide a more objective and consistent approach to stroke diagnosis and potentially enable personalized treatment plans based on the location and extent of the stroke. Further studies are needed to validate our method on larger and more diverse datasets and to explore its clinical utility in real-world settings.

BiGRUD-SA: Protein S-sulfenylation sites prediction based on BiGRU and self-attention

S-sulfenylation is a vital post-translational modification (PTM) of proteins, which is an intermediate in other redox reactions and has implications for signal transduction and protein function regulation. However, there are many restrictions on the experimental identification of S-sulfenylation sites. Therefore, predicting S-sulfoylation sites by computational methods is fundamental to studying protein function and related biological mechanisms. In this paper, we propose a method named BiGRUD-SA based on bi-directional gated recurrent unit (BiGRU) and self-attention mechanism to predict protein S-sulfenylation sites. We first use AAC, BLOSUM62, AAindex, EAAC and GAAC to extract features, and do feature fusion to obtain original feature space. Next, we use SMOTE-Tomek method to handle data imbalance. Then, we input the processed data to the BiGRU and use self-attention mechanism to do further feature extraction. Finally, we input the data obtained to the deep neural networks (DNN) to identify S-sulfenylation sites. The accuracies of training set and independent test set are 96.66% and 95.91% respectively, which indicates that our method is conducive to identifying S-sulfenylation sites. Furthermore, we use a data set of S-sulfenylation sites in Arabidopsis thaliana to effectively verify the generalization ability of BiGRUD-SA method, and obtain better prediction results.

BERT-PAGG: a Chinese relationship extraction model fusing PAGG and entity location information

Relationship extraction is one of the important tasks of constructing knowledge graph. In recent years, many scholars have introduced external information other than entities into relationship extraction models, which perform better than traditional relationship extraction methods. However, they ignore the importance of the relative position between entities. Considering the relative position between entity pairs and the influence of sentence level information on the performance of relationship extraction model, this article proposes a BERT-PAGG relationship extraction model. The model introduces the location information of entities, and combines the local features extracted by PAGG module with the entity vector representation output by BERT. Specifically, BERT-PAGG integrates entity location information into local features through segmented convolution neural network, uses attention mechanism to capture more effective semantic features, and finally regulates the transmission of information flow through gating mechanism. Experimental results on two open Chinese relation extraction datasets show that the proposed method achieves the best results compared with other models. At the same time, ablation experiments show that PAGG module can effectively use external information, and the introduction of this module makes the Macro-F1 value of the model increase by at least 2.82%.

[Electrocardiogram signal classification based on fusion method of residual network and self-attention mechanism]

In the diagnosis of cardiovascular diseases, the analysis of electrocardiogram (ECG) signals has always played a crucial role. At present, how to effectively identify abnormal heart beats by algorithms is still a difficult task in the field of ECG signal analysis. Based on this, a classification model that automatically identifies abnormal heartbeats based on deep residual network (ResNet) and self-attention mechanism was proposed. Firstly, this paper designed an 18-layer convolutional neural network (CNN) based on the residual structure, which helped model fully extract the local features. Then, the bi-directional gated recurrent unit (BiGRU) was used to explore the temporal correlation for further obtaining the temporal features. Finally, the self-attention mechanism was built to weight important information and enhance model's ability to extract important features, which helped model achieve higher classification accuracy. In addition, in order to mitigate the interference on classification performance due to data imbalance, the study utilized multiple approaches for data augmentation. The experimental data in this study came from the arrhythmia database constructed by MIT and Beth Israel Hospital (MIT-BIH), and the final results showed that the proposed model achieved an overall accuracy of 98.33% on the original dataset and 99.12% on the optimized dataset, which demonstrated that the proposed model can achieve good performance in ECG signal classification, and possessed potential value for application to portable ECG detection devices.

A Self-attention Graph Convolutional Network for Precision Multi-tumor Early Diagnostics with DNA Methylation Data

DNA methylation-based precision tumor early diagnostics is emerging as state-of-the-art technology that could capture early cancer signs 3 ~ 5 years in advance, even for clinically homogenous groups. Presently, the sensitivity of early detection for many tumors is ~ 30%, which needs significant improvement. Nevertheless, based on the genome-wide DNA methylation data, one could comprehensively characterize tumors' entire molecular genetic landscape and their subtle differences. Therefore, novel high-performance methods must be modeled by considering unbiased information using excessively available DNA methylation data. To fill this gap, we have designed a computational model involving a self-attention graph convolutional network and multi-class classification support vector machine to identify the 11 most common cancers using DNA methylation data. The self-attention graph convolutional network automatically learns key methylation sites in a data-driven way. Then, multi-tumor early diagnostics is realized by training a multi-class classification support vector machine based on the selected methylation sites. We evaluated our model's performance through several data sets of experiments, and our results demonstrate the effectiveness of the selected key methylation sites, which are highly relevant for blood diagnosis. The pipeline of the self-attention graph convolutional network based computational framework.

MSTS-Net: malignancy evolution prediction of pulmonary nodules from longitudinal CT images via multi-task spatial-temporal self-attention network

PURPOSE

Longitudinal CT images contain the law of lesion growth and evolution over time. Therefore, our purpose is to explore the growth and evolution law of pulmonary lesions in the time dimension to improve the performance of predicting the malignant evolution of pulmonary nodules.

METHODS

In this paper, we propose a Multi-task Spatial-Temporal Self-attention network (MSTS-Net) to predict the malignancy growth trend of pulmonary nodules from different periods. More specifically, the model achieves lesion segmentation task and lesion prediction task by sharing the same encoder. Segmentation task boosts the performance of the prediction task. In addition, a Static Context Spatial Self-attention Module and a Dynamic Adaptive Temporal Self-Attention Module are introduced to capture both static spatial coherence patterns between consecutive slices of lesions in the same period and temporal dynamics across different time points.

RESULTS

We repeatedly evaluated the proposed method on the National Lung Screening Trial dataset and the Shanxi Cancer Hospital dataset. The final experimental results show that our MSTS-Net has an area under the ROC curve score of 0.919.

CONCLUSION

In the computer-aided prediction of the malignant evolution of pulmonary nodules, combining the characteristics of the temporal dimension of pulmonary nodules with CT data can effectively improve the accuracy of prediction. The MSTS-Net we developed has high predictive value and broad prospects for clinical application.

A meta-framework for multi-label active learning based on deep reinforcement learning

Multi-label Active Learning (MLAL) is an effective method to improve the performance of the classifier on multi-label problems with less annotation effort by allowing the learning system to actively select high-quality examples (example-label pairs) for labeling. Existing MLAL algorithms mainly focus on designing reasonable algorithms to evaluate the potential values (as previously mentioned quality) of the unlabeled data. These manually designed methods may show totally different results on various types of datasets due to the defect of the methods or the particularity of the datasets. In this paper, instead of manually designing an evaluation method, we propose a deep reinforcement learning (DRL) model to explore a general evaluation method on several seen datasets and eventually apply it to unseen datasets based on a meta framework. In addition, a self-attention mechanism along with a reward function is integrated into the DRL structure to address the label correlation and data imbalanced problems in MLAL. Comprehensive experiments show that our proposed DRL-based MLAL method is able to produce comparable results as compared with other methods reported in the literature.

[Study on the method of polysomnography sleep stage staging based on attention mechanism and bidirectional gate recurrent unit]

Polysomnography (PSG) monitoring is an important method for clinical diagnosis of diseases such as insomnia, apnea and so on. In order to solve the problem of time-consuming and energy-consuming sleep stage staging of sleep disorder patients using manual frame-by-frame visual judgment PSG, this study proposed a deep learning algorithm model combining convolutional neural networks (CNN) and bidirectional gate recurrent neural networks (Bi GRU). A dynamic sparse self-attention mechanism was designed to solve the problem that gated recurrent neural networks (GRU) is difficult to obtain accurate vector representation of long-distance information. This study collected 143 overnight PSG data of patients from Shanghai Mental Health Center with sleep disorders, which were combined with 153 overnight PSG data of patients from the open-source dataset, and selected 9 electrophysiological channel signals including 6 electroencephalogram (EEG) signal channels, 2 electrooculogram (EOG) signal channels and a single mandibular electromyogram (EMG) signal channel. These data were used for model training, testing and evaluation. After cross validation, the accuracy was (84.0±2.0)%, and Cohen's kappa value was 0.77±0.50. It showed better performance than the Cohen's kappa value of physician score of 0.75±0.11. The experimental results show that the algorithm model in this paper has a high staging effect in different populations and is widely applicable. It is of great significance to assist clinicians in rapid and large-scale PSG sleep automatic staging.

Multi-head self-attention mechanism enabled individualized hemoglobin prediction and treatment recommendation systems in anemia management for hemodialysis patients

Anemia is a critical complication in hemodialysis patients, but the response to erythropoietin-stimulating agents (ESA) treatment varies from patient to patient and is not linear across different time points. The aim of this study was to develop deep learning algorithms for individualized anemia management. We retrospectively collected 36,677 data points from 623 hemodialysis patients, including clinical data, laboratory values, hemoglobin levels, and previous ESA doses. To reduce the computational complexity associated with recurrent neural networks (RNN) in processing time-series data, we developed neural networks based on multi-head self-attention mechanisms in an efficient and effective hemoglobin prediction model. Our proposed model achieved a more accurate hemoglobin prediction than the state-of-the-art RNN model, as shown by the smaller mean absolute error (MAE) of hemoglobin (0.451 vs. 0.593 g/dL, p = 0.014). In ESA (including darbepoetin and epoetin) dose recommendation, the simulation results by our model revealed a higher rate of achieved hemoglobin targets (physician prescription vs. model: 86.3 % vs. 92.7 %, p < 0.001), a lower rate of hemoglobin levels below 10 g/dL (13.7 % vs. 7.3 %, p < 0.001) and smaller change in hemoglobin levels (0.6 g/dL vs. 0.4 g/dL, p < 0.001) in all patients. Our model holds great potential for individualized anemia management as a computerized clinical decision support system for hemodialysis patients. Further external validation with other datasets and prospective clinical utility studies are warranted.

An automatic segmentation method with self-attention mechanism on left ventricle in gated PET/CT myocardial perfusion imaging

OBJECTIVES

We aimed to propose an automatic segmentation method for left ventricular (LV) from 16 electrocardiogram (ECG) -gated ¹³N-NH₃ PET/CT myocardial perfusion imaging (MPI) to improve the performance of LV function assessment.

METHODS

Ninety-six cases with confirmed or suspected obstructive coronary artery disease (CAD) were enrolled in this research. The LV myocardial contours were delineated by physicians as ground truth. We developed an automatic segmentation method, which introduces the self-attention mechanism into 3D U-Net to capture global information of images so as to achieve fine segmentation of LV. Three cross-validation tests were performed on each gate (64 vs. 32 for training vs. validation). The effectiveness was validated by quantitative metrics (modified hausdorff distance, MHD; dice ratio, DR; 3D MHD) as well as cardiac functional parameters (end-systolic volume, ESV; end-diastolic volume, EDV; ejection fraction, EF). Furthermore, the feasibility of the proposed method was also evaluated by intra- and inter-observers with DR and 3D-MHD.

RESULTS

Compared with backbone network, the proposed approach improved the average DR from 0.905 ± 0.0193 to 0.9202 ± 0.0164, and decreased the average 3D MHD from 0.4611 ± 0.0349 to 0.4304 ± 0.0339. The average relative error of LV volume between proposed method and ground truth is 1.09±3.66%, and the correlation coefficient is 0.992 ± 0.007 (P < 0.001). The EDV, ESV, EF deduced from the proposed approach were highly correlated with ground truth (r ≥ 0.864, P < 0.001), and the correlation with commercial software is fair (r ≥ 0.871, P < 0.001). DR and 3D MHD of contours and myocardium from two observers are higher than 0.899 and less than 0.5194.

CONCLUSION

The proposed approach is highly feasible for automatic segmentation of the LV cavity and myocardium, with potential to benefit the precision of LV function assessment.

GSAML-DTA: An interpretable drug-target binding affinity prediction model based on graph neural networks with self-attention mechanism and mutual information

Identifying drug-target affinity (DTA) has great practical importance in the process of designing efficacious drugs for known diseases. Recently, numerous deep learning-based computational methods have been developed to predict drug-target affinity and achieved impressive performance. However, most of them construct the molecule (drug or target) encoder without considering the weights of features of each node (atom or residue). Besides, they generally combine drug and target representations directly, which may contain irrelevant-task information. In this study, we develop GSAML-DTA, an interpretable deep learning framework for DTA prediction. GSAML-DTA integrates a self-attention mechanism and graph neural networks (GNNs) to build representations of drugs and target proteins from the structural information. In addition, mutual information is introduced to filter out redundant information and retain relevant information in the combined representations of drugs and targets. Extensive experimental results demonstrate that GSAML-DTA outperforms state-of-the-art methods for DTA prediction on two benchmark datasets. Furthermore, GSAML-DTA has the interpretation ability to analyze binding atoms and residues, which may be conducive to chemical biology studies from data. Overall, GSAML-DTA can serve as a powerful and interpretable tool suitable for DTA modelling.

[Cascaded multi-level medical image registration method based on transformer]

In deep learning-based image registration, the deformable region with complex anatomical structures is an important factor affecting the accuracy of network registration. However, it is difficult for existing methods to pay attention to complex anatomical regions of images. At the same time, the receptive field of the convolutional neural network is limited by the size of its convolution kernel, and it is difficult to learn the relationship between the voxels with far spatial location, making it difficult to deal with the large region deformation problem. Aiming at the above two problems, this paper proposes a cascaded multi-level registration network model based on transformer, and equipped it with a difficult deformable region perceptron based on mean square error. The difficult deformation perceptron uses sliding window and floating window techniques to retrieve the registered images, obtain the difficult deformation coefficient of each voxel, and identify the regions with the worst registration effect. In this study, the cascaded multi-level registration network model adopts the difficult deformation perceptron for hierarchical connection, and the self-attention mechanism is used to extract global features in the basic registration network to optimize the registration results of different scales. The experimental results show that the method proposed in this paper can perform progressive registration of complex deformation regions, thereby optimizing the registration results of brain medical images, which has a good auxiliary effect on the clinical diagnosis of doctors.

DeepPSE: Prediction of polypharmacy side effects by fusing deep representation of drug pairs and attention mechanism

Polypharmacy (multiple use of drugs) is an effective strategy for combating complex or co-existing diseases. However, a major consequence of polypharmacy is a higher risk of adverse side effects due to drug-drug interactions, which are rare and observed in relatively small clinical testing. Thus, identification of polypharmacy side effects remains challenging. Here, we propose a deep learning-based method, DeepPSE, to predict polypharmacy side effects in an end-to-end way. DeepPSE is composed of two main modules. First, multiple types of neural networks are constructed and fused to learn the deep representation of a drug pair. Second, the encoder block of transformer that includes self-attention mechanism is built to get latent features, which are further fed into the fully connected layer to predict polypharmacy side effects of drug pairs. Further, DeepPSE is compared with five baseline or state-of-the-art methods on a benchmark dataset of 964 types of polypharmacy side effects across 63473 drug pairs. Experimental results demonstrate that DeepPSE achieves better performance than that of all five methods. The source codes and data are available at https://github.com/ShenggengLin/DeepPSE.

Diagnosis of schizophrenia with functional connectome data: a graph-based convolutional neural network approach

Previous deep learning methods have not captured graph or network representations of brain structural or functional connectome data. To address this, we developed the BrainNet-Global Covariance Pooling-Attention Convolutional Neural Network (BrainNet-GA CNN) by incorporating BrainNetCNN and global covariance pooling into the self-attention mechanism. Resting-state functional magnetic resonance imaging data were obtained from 171 patients with schizophrenia spectrum disorders (SSDs) and 161 healthy controls (HCs). We conducted an ablation analysis of the proposed BrainNet-GA CNN and quantitative performance comparisons with competing methods using the nested tenfold cross validation strategy. The performance of our model was compared with competing methods. Discriminative connections were visualized using the gradient-based explanation method and compared with the results obtained using functional connectivity analysis. The BrainNet-GA CNN showed an accuracy of 83.13%, outperforming other competing methods. Among the top 10 discriminative connections, some were associated with the default mode network and auditory network. Interestingly, these regions were also significant in the functional connectivity analysis. Our findings suggest that the proposed BrainNet-GA CNN can classify patients with SSDs and HCs with higher accuracy than other models. Visualization of salient regions provides important clinical information. These results highlight the potential use of the BrainNet-GA CNN in the diagnosis of schizophrenia.

Super-resolution of Pneumocystis carinii pneumonia CT via self-attention GAN

BACKGROUND AND OBJECTIVE

Computed tomography (CT) examination plays an important role in screening suspected and confirmed patients in pneumocystis carinii pneumonia (PCP), and the efficient acquisition of high-quality medical CT images is essential for the clinical application of computer-aided diagnosis technology. Therefore, improving the resolution of CT images of pneumonia is a very important task.

METHODS

Aiming at the problem of how to recover the texture details of the reconstructed PCP CT super-resolution image, we propose the image super-resolution reconstruction model based on self-attention generation adversarial network (SAGAN). In the SAGAN algorithm, a generator based on self-attention mechanism and residual module is used to transform a low-resolution image into a super-resolution image. A discriminator based on depth convolution network tries to distinguish the difference between the reconstructed super-resolution image and the real super-resolution image. In terms of loss function construction, on the one hand, the Charbonnier content loss function is used to improve the accuracy of image reconstruction, and on the other hand, the feature value before activation of the pre-trained VGGNet is used to calculate the perceptual loss to achieve accurate texture detail reconstruction of super-resolution images.

RESULTS

Experimental results show that our SAGAN algorithm is superior to other state-of-the-art algorithms in both peak signal-to-noise ratio (PSNR) and structural similarity score (SSIM). Specifically, our SAGAN method can obtain 31.94 dB which is 1.53 dB better than SRGAN on Set5 dataset for 4 enlargements.

CONCLUSION

Our SAGAN method can reconstruct more realistic PCP CT images with clear texture, which can help experts diagnose the condition of PCP.

An optimal ablation time prediction model based on minimizing the relapse risk

OBJECTIVE

Percutaneous microwave ablation is an essential and safe method for the treatment of liver cancer. As one therapeutic dose, ablation time is crucial to the treatment effect determined by the physicians. However, due to the different experiences of physicians and the significant individual differences of patients, the final treatment effect is also different, which makes it difficult for the ablation time recorded in the electronic health records (EHRs) to follow the same pattern. To solve this problem, we propose a data mining method based on historical treatment data recorded in EHR, which uses a robust relapse risk as strong supervision to correct the ablation time. The prediction results of this method are closer to the situation of patients without relapse, which can provide physicians with reference.

METHODS

In the proposed method, we introduce the optimization method to iteratively minimize the postoperative relapse risk and utilize gradient propagation between the risk and ablation time during iteration to correct the latter. We also apply a self-attention mechanism to find the global dependencies between each feature in EHR to improve the final prediction performance of the model.

RESULTS

Comparative experimental results show that compared with other baseline model, the proposed model achieves better performance on R-square, MAE, and MSE metric. The results of ablation experiments show that the integration of label correction and self-attention mechanism can improve the model performance.

CONCLUSIONS

We using relapse risk as strong supervision related to the ablation time can effectively correct the deviation of the ablation time as weak supervision. The self-attention mechanism in the proposed model can significantly improve the prediction performance.

Densely connected attention network for diagnosing COVID-19 based on chest CT

BACKGROUND

To fully enhance the feature extraction capabilities of deep learning models, so as to accurately diagnose coronavirus disease 2019 (COVID-19) based on chest CT images, a densely connected attention network (DenseANet) was constructed by utilizing the self-attention mechanism in deep learning.

METHODS

During the construction of the DenseANet, we not only densely connected attention features within and between the feature extraction blocks with the same scale, but also densely connected attention features with different scales at the end of the deep model, thereby further enhancing the high-order features. In this way, as the depth of the deep model increases, the spatial attention features generated by different layers can be densely connected and gradually transferred to deeper layers. The DenseANet takes CT images of the lung fields segmented by an improved U-Net as inputs and outputs the probability of the patients suffering from COVID-19.

RESULTS

Compared with exiting attention networks, DenseANet can maximize the utilization of self-attention features at different depths in the model. A five-fold cross-validation experiment was performed on a dataset containing 2993 CT scans of 2121 patients, and experiments showed that the DenseANet can effectively locate the lung lesions of patients infected with SARS-CoV-2, and distinguish COVID-19, common pneumonia and normal controls with an average of 96.06% Acc and 0.989 AUC.

CONCLUSIONS

The DenseANet we proposed can generate strong attention features and achieve the best diagnosis results. In addition, the proposed method of densely connecting attention features can be easily extended to other advanced deep learning methods to improve their performance in related tasks.

Bidirectional cross-modality unsupervised domain adaptation using generative adversarial networks for cardiac image segmentation

BACKGROUND

A novel Generative Adversarial Networks (GAN) based bidirectional cross-modality unsupervised domain adaptation (GBCUDA) framework is developed for cardiac image segmentation, which can effectively tackle the problem of network's segmentation performance degradation when adapting to the target domain without ground truth labels.

METHOD

GBCUDA uses GAN for image alignment, applies adversarial learning to extract image features, and gradually enhances the domain invariance of extracted features. The shared encoder performs an end-to-end learning task in which features that differ between the two domains complement each other. The self-attention mechanism is incorporated to the GAN network, which can generate details based on the prompts of all feature positions. Furthermore, spectrum normalization is implemented to stabilize the training of GAN, and knowledge distillation loss is introduced to process high-level feature-maps in order to better complete the cross-mode segmentation task.

RESULTS

The effectiveness of our proposed unsupervised domain adaptation framework is tested over the Multi-Modality Whole Heart Segmentation (MM-WHS) Challenge 2017 dataset. The proposed method is able to improve the average Dice from 74.1% to 81.5% for the four cardiac substructures, and reduce the average symmetric surface distance (ASD) from 7.0 to 5.8 over CT images. For MRI images, our proposed framework trained on CT images gives the average Dice of 59.2% and reduces the average ASD from 5.7 to 4.9.

CONCLUSIONS

The evaluation results demonstrate our method's effectiveness on domain adaptation and the superiority to the current state-of-the-art domain adaptation methods.

Self-selective attention using correlation between instances for distant supervision relation extraction

Distant supervision relation extraction methods are widely used to extract relational facts in text. The traditional selective attention model regards instances in the bag as independent of each other, which makes insufficient use of correlation information between instances and supervision information of all correctly labeled instances, affecting the performance of relation extractor. Aiming at this problem, a distant supervision relation extraction method with self-selective attention is proposed. The method uses a layer of convolution and self-attention mechanism to encode instances to learn the better semantic vector representation of instances. The correlation between instances in the bag is used to assign a higher weight to all correctly labeled instances, and the weighted summation of instances in the bag is used to obtain a bag vector representation. Experiments on the NYT dataset show that the method can make full use of the information of all correctly labeled instances in the bag. The method can achieve better results as compared with baselines.

SAM-GAN: Self-Attention supporting Multi-stage Generative Adversarial Networks for text-to-image synthesis

Synthesizing photo-realistic images based on text descriptions is a challenging task in the field of computer vision. Although generative adversarial networks have made significant breakthroughs in this task, they still face huge challenges in generating high-quality visually realistic images consistent with the semantics of text. Generally, existing text-to-image methods accomplish this task with two steps, that is, first generating an initial image with a rough outline and color, and then gradually yielding the image within high-resolution from the initial image. However, one drawback of these methods is that, if the quality of the initial image generation is not high, it is hard to generate a satisfactory high-resolution image. In this paper, we propose SAM-GAN, Self-Attention supporting Multi-stage Generative Adversarial Networks, for text-to-image synthesis. With the self-attention mechanism, the model can establish the multi-level dependence of the image and fuse the sentence- and word-level visual-semantic vectors, to improve the quality of the generated image. Furthermore, a multi-stage perceptual loss is introduced to enhance the semantic similarity between the synthesized image and the real image, thus enhancing the visual-semantic consistency between text and images. For the diversity of the generated images, a mode seeking regularization term is integrated into the model. The results of extensive experiments and ablation studies, which were conducted in the Caltech-UCSD Birds and Microsoft Common Objects in Context datasets, show that our model is superior to competitive models in text-to-image synthesis.

hERG-Att: Self-attention-based deep neural network for predicting hERG blockers

A voltage-gated potassium channel encoded by the human ether-à-go-go-related gene (hERG) regulates cardiac action potential, and it is involved in cardiotoxicity with compounds that inhibit its activity. Therefore, the screening of hERG channel blockers is a mandatory step in the drug discovery process. The screening of hERG blockers by using conventional methods is inefficient in terms of cost and efforts. This has led to the development of many in silico hERG blocker prediction models. However, constructing a high-performance predictive model with interpretability on hERG blockage by certain compounds is a major obstacle. In this study, we developed the first, attention-based, interpretable model that predicts hERG blockers and captures important hERG-related compound substructures. To do that, we first collected various datasets, ranging from public databases to publicly available private datasets, to train and test the model. Then, we developed a precise and interpretable hERG blocker prediction model by using deep learning with a self-attention approach that has an appropriate molecular descriptor, Morgan fingerprint. The proposed prediction model was validated, and the validation result showed that the model was well-optimized and had high performance. The test set performance of the proposed model was significantly higher than that of previous fingerprint-based conventional machine learning models. In particular, the proposed model generally had high accuracy and F1 score thereby, representing the model's predictive reliability. Furthermore, we interpreted the calculated attention score vectors obtained from the proposed prediction model and demonstrated the important structural patterns that are represented in hERG blockers. In summary, we have proposed a powerful and interpretable hERG blocker prediction model that can reduce the overall cost of drug discovery by accurately screening for hERG blockers and suggesting hERG-related substructures.

Attention guided capsule networks for chemical-protein interaction extraction

The biomedical literature contains a sufficient number of chemical-protein interactions (CPIs). Automatic extraction of CPI is a crucial task in the biomedical domain, which has excellent benefits for precision medicine, drug discovery and basic biomedical research. In this study, we propose a novel model, BERT-based attention-guided capsule networks (BERT-Att-Capsule), for CPI extraction. Specifically, the approach first employs BERT (Bidirectional Encoder Representations from Transformers) to capture the long-range dependencies and bidirectional contextual information of input tokens. Then, the aggregation is regarded as a routing problem for how to pass messages from source capsule nodes to target capsule nodes. This process enables capsule networks to determine what and how much information need to be transferred, as well as to identify sophisticated and interleaved features. Afterwards, the multi-head attention is applied to guide the model to learn different contribution weights of capsule networks obtained by the dynamic routing. We evaluate our model on the CHEMPROT corpus. Our approach is superior in performance as compared with other state-of-the-art methods. Experimental results show that our approach can adequately capture the long-range dependencies and bidirectional contextual information of input tokens, obtain more fine-grained aggregation information through attention-guided capsule networks, and therefore improve the performance.