HGAT: Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification

Multimodal data integration for oncology in the era of deep neural networks: a review

Cancer research encompasses data across various scales, modalities, and resolutions, from screening and diagnostic imaging to digitized histopathology slides to various types of molecular data and clinical records. The integration of these diverse data types for personalized cancer care and predictive modeling holds the promise of enhancing the accuracy and reliability of cancer screening, diagnosis, and treatment. Traditional analytical methods, which often focus on isolated or unimodal information, fall short of capturing the complex and heterogeneous nature of cancer data. The advent of deep neural networks has spurred the development of sophisticated multimodal data fusion techniques capable of extracting and synthesizing information from disparate sources. Among these, Graph Neural Networks (GNNs) and Transformers have emerged as powerful tools for multimodal learning, demonstrating significant success. This review presents the foundational principles of multimodal learning including oncology data modalities, taxonomy of multimodal learning, and fusion strategies. We delve into the recent advancements in GNNs and Transformers for the fusion of multimodal data in oncology, spotlighting key studies and their pivotal findings. We discuss the unique challenges of multimodal learning, such as data heterogeneity and integration complexities, alongside the opportunities it presents for a more nuanced and comprehensive understanding of cancer. Finally, we present some of the latest comprehensive multimodal pan-cancer data sources. By surveying the landscape of multimodal data integration in oncology, our goal is to underline the transformative potential of multimodal GNNs and Transformers. Through technological advancements and the methodological innovations presented in this review, we aim to chart a course for future research in this promising field. This review may be the first that highlights the current state of multimodal modeling applications in cancer using GNNs and transformers, presents comprehensive multimodal oncology data sources, and sets the stage for multimodal evolution, encouraging further exploration and development in personalized cancer care.

Dual-Path Graph Neural Network with Adaptive Auxiliary Module for Link Prediction

Link prediction, which has important applications in many fields, predicts the possibility of the link between two nodes in a graph. Link prediction based on Graph Neural Network (GNN) obtains node representation and graph structure through GNN, which has attracted a growing amount of attention recently. However, the existing GNN-based link prediction approaches possess some shortcomings. On the one hand, because a graph contains different types of nodes, it leads to a great challenge for aggregating information and learning node representation from its neighbor nodes. On the other hand, the attention mechanism has been an effect instrument for enhancing the link prediction performance. However, the traditional attention mechanism is always monotonic for query nodes, which limits its influence on link prediction. To address these two problems, a Dual-Path Graph Neural Network (DPGNN) for link prediction is proposed in this study. First, we propose a novel Local Random Features Augmentation for Graph Convolution Network as a baseline of one path. Meanwhile, Graph Attention Network version 2 based on dynamic attention mechanism is adopted as a baseline of the other path. And then, we capture more meaningful node representation and more accurate link features by concatenating the information of these two paths. In addition, we propose an adaptive auxiliary module for better balancing the weight of auxiliary tasks, which brings more benefit to link prediction. Finally, extensive experiments verify the effectiveness and superiority of our proposed DPGNN for link prediction.

Interference recommendation for the pump sizing process in progressive cavity pumps using graph neural networks

Pump sizing is the process of dimensional matching of an impeller and stator to provide a satisfactory performance test result and good service life during the operation of progressive cavity pumps. In this process, historical data analysis and dimensional monitoring are done manually, consuming a large number of man-hours and requiring a deep knowledge of progressive cavity pump behavior. This paper proposes the use of graph neural networks in the construction of a prototype to recommend interference during the pump sizing process in a progressive cavity pump. For this, data from different applications is used in addition to individual control spreadsheets to build the database used in the prototype. From the pre-processed data, complex network techniques and the betweenness centrality metric are used to calculate the degree of importance of each order confirmation, as well as to calculate the dimensionality of the rotors. Using the proposed method a mean squared error of 0.28 is obtained for the cases where there are recommendations for order confirmations. Based on the results achieved, it is noticeable that there is a similarity of the dimensions defined by the project engineers during the pump sizing process, and this outcome can be used to validate the new design definitions.

Chinese text classification by combining Chinese-BERTology-wwm and GCN

Text classification is an important and classic application in natural language processing (NLP). Recent studies have shown that graph neural networks (GNNs) are effective in tasks with rich structural relationships and serve as effective transductive learning approaches. Text representation learning methods based on large-scale pretraining can learn implicit but rich semantic information from text. However, few studies have comprehensively utilized the contextual semantic and structural information for Chinese text classification. Moreover, the existing GNN methods for text classification did not consider the applicability of their graph construction methods to long or short texts. In this work, we propose Chinese-BERTology-wwm-GCN, a framework that combines Chinese bidirectional encoder representations from transformers (BERT) series models with whole word masking (Chinese-BERTology-wwm) and the graph convolutional network (GCN) for Chinese text classification. When building text graph, we use documents and words as nodes to construct a heterogeneous graph for the entire corpus. Specifically, we use the term frequency-inverse document frequency (TF-IDF) to construct the word-document edge weights. For long text corpora, we propose an improved pointwise mutual information (PMI*) measure for words according to their word co-occurrence distances to represent the weights of word-word edges. For short text corpora, the co-occurrence information between words is often limited. Therefore, we utilize cosine similarity to represent the word-word edge weights. During the training stage, we effectively combine the cross-entropy and hinge losses and use them to jointly train Chinese-BERTology-wwm and GCN. Experiments show that our proposed framework significantly outperforms the baselines on three Chinese benchmark datasets and achieves good performance even with few labeled training sets.

Multilevel Classification of Users' Needs in Chinese Online Medical and Health Communities: Model Development and Evaluation Based on Graph Convolutional Network

BACKGROUND

Online medical and health communities provide a platform for internet users to share experiences and ask questions about medical and health issues. However, there are problems in these communities, such as the low accuracy of the classification of users' questions and the uneven health literacy of users, which affect the accuracy of user retrieval and the professionalism of the medical personnel answering the question. In this context, it is essential to study more effective classification methods of users' information needs.

OBJECTIVE

Most online medical and health communities tend to provide only disease-type labels, which do not give a comprehensive summary of users' needs. The study aims to construct a multilevel classification framework based on the graph convolutional network (GCN) model for users' needs in online medical and health communities so that users can perform more targeted information retrieval.

METHODS

Using the Chinese online medical and health community "Qiuyi" as an example, we crawled questions posted by users in the "Cardiovascular Disease" section as the data source. First, the disease types involved in the problem data were segmented by manual coding to generate the first-level label. Second, the needs were identified by K-means clustering to generate the users' information needs label as the second-level label. Finally, by constructing a GCN model, users' questions were automatically classified, thus realizing the multilevel classification of users' needs.

RESULTS

Based on the empirical research of questions posted by users in the "Cardiovascular Disease" section of Qiuyi, the hierarchical classification of users' questions (data) was realized. The classification models designed in the study achieved accuracy, precision, recall, and F1-score of 0.6265, 0.6328, 0.5788, and 0.5912, respectively. Compared with the traditional machine learning method naïve Bayes and the deep learning method hierarchical text classification convolutional neural network, our classification model showed better performance. At the same time, we also performed a single-level classification experiment on users' needs, which in comparison with the multilevel classification model exhibited a great improvement.

CONCLUSIONS

A multilevel classification framework has been designed based on the GCN model. The results demonstrated that the method is effective in classifying users' information needs in online medical and health communities. At the same time, users with different diseases have different directions for information needs, which plays an important role in providing diversified and targeted services to the online medical and health community. Our method is also applicable to other similar disease classifications.

A Review Selection Method based on Consumer Decision Phases in E-commerce

A valuable small subset strategically selected from massive online reviews is beneficial to improve consumers’ decision-making efficiency in e-commerce. Existing review selection methods primarily concentrate on the informativeness of reviews and aim to find a subset of reviews that can reflect the informational properties of the original review set. However, changes in consumers’ review diets during the two-phase decision process are not fully considered. In this study, we propose a novel review selection problem of finding a diet-matched review subset with high diversity and representativeness, which can better adapt to consumers’ review-diet conversion from attribute-oriented to experience-oriented reviews between two decision phases. A novel decision-phase-based review selection method named DPRS is further proposed, which involves two steps: review classification and review selection. In the review classification step, the probability of a review being attribute-oriented or experience-oriented is estimated by prior knowledge-aware attentive neural network. In the second step, a novel heuristic algorithm, namely stepwise non-dominated selection with superiority strategy, is introduced to seek the solution to the review selection problem. Extensive experiments on a real-world dataset demonstrate that DPRS outperforms state-of-the-art methods in terms of both review classification and review selection.

A review of semi-supervised learning for text classification

A huge amount of data is generated daily leading to big data challenges. One of them is related to text mining, especially text classification. To perform this task we usually need a large set of labeled data that can be expensive, time-consuming, or difficult to be obtained. Considering this scenario semi-supervised learning (SSL), the branch of machine learning concerned with using labeled and unlabeled data has expanded in volume and scope. Since no recent survey exists to overview how SSL has been used in text classification, we aim to fill this gap and present an up-to-date review of SSL for text classification. We retrieve 1794 works from the last 5 years from IEEE Xplore, ACM Digital Library, Science Direct, and Springer. Then, 157 articles were selected to be included in this review. We present the application domain, datasets, and languages employed in the works. The text representations and machine learning algorithms. We also summarize and organize the works following a recent taxonomy of SSL. We analyze the percentage of labeled data used, the evaluation metrics, and obtained results. Lastly, we present some limitations and future trends in the area. We aim to provide researchers and practitioners with an outline of the area as well as useful information for their current research.

News Stance Discrimination Based on a Heterogeneous Network of Social Background Information Fusion

Media with partisan tendencies publish news articles to support their preferred political parties to guide the direction of public opinion. Therefore, discovering political bias in news texts has important practical significance for national election prediction and public opinion management. Some biased news often has obscure expressions and ambiguous writing styles. By bypassing the language model, the accuracy of methods that rely on news semantic information for position discrimination is low. This manuscript proposes a news standpoint discrimination method based on social background information fusion heterogeneous network. This method expands the judgment ability of creators and topics on news standpoints from external information and fine-grained topics based on news semantics. Multi-attribute features of nodes enrich the feature representation of nodes, and joint representation of heterogeneous networks can reduce the dependence of position discrimination on the news semantic information. To effectively deal with the position discrimination of new news, the design of a multi-attribute fusion heterogeneous network is extended to inductive learning, avoiding the cost of model training caused by recomposition. Based on the Allsides dataset, this manuscript expands the information of its creator's social background and compares the model for discriminating political positions based on news content. In the experiment, the best transductive attribute fusion heterogeneous network achieved an accuracy of 92.24% and a macro F1 value of 92.05%. The effect is improved based purely on semantic information for position discrimination, which proves the effectiveness of the model design.

Multi-Stream Graph Convolutional Networks for Text Classification via Representative-Word Document Mining

Recently, graph convolutional networks (GCNs) for text classification have received considerable attention in natural language processing. However, most current methods just use original documents and words in the corpus to construct the topology of graph which may lose some effective information. In this paper, we propose a Multi-Stream Graph Convolutional Network (MS-GCN) for text classification via Representative-Word Document (RWD) mining, which is implemented in PyTorch. In the proposed method, we first introduce temporary labels and mine the RWDs which are treated as additional documents in the corpus. Then, we build a heterogeneous graph based on relations among a Group of RWDs (GRWDs), words and original documents. Furthermore, we construct the MS-GCN based on multiple heterogeneous graphs according to different GRWDs. Finally, we optimize our MS-GCN model through updated mechanism of GRWDs. We evaluate the proposed approach on six text classification datasets, 20NG, R8, R52, Ohsumed, MR and Pheme. Extensive experiments on these datasets show that our proposed approach outperforms state-of-the-art methods for text classification.

N-GPETS: Neural Attention Graph-Based Pretrained Statistical Model for Extractive Text Summarization

The extractive summarization approach involves selecting the source document's salient sentences to build a summary. One of the most important aspects of extractive summarization is learning and modelling cross-sentence associations. Inspired by the popularity of Transformer-based Bidirectional Encoder Representations (BERT) pretrained linguistic model and graph attention network (GAT) having a sophisticated network that captures intersentence associations, this research work proposes a novel neural model N-GPETS by combining heterogeneous graph attention network with BERT model along with statistical approach using TF-IDF values for extractive summarization task. Apart from sentence nodes, N-GPETS also works with different semantic word nodes of varying granularity levels that serve as a link between sentences, improving intersentence interaction. Furthermore, proposed N-GPETS becomes more improved and feature-rich by integrating graph layer with BERT encoder at graph initialization step rather than employing other neural network encoders such as CNN or LSTM. To the best of our knowledge, this work is the first attempt to combine the BERT encoder and TF-IDF values of the entire document with a heterogeneous attention graph structure for the extractive summarization task. The empirical outcomes on benchmark news data sets CNN/DM show that the proposed model N-GPETS gets favorable results in comparison with other heterogeneous graph structures employing the BERT model and graph structures without the BERT model.

Automatic arrhythmia detection with multi-lead ECG signals based on heterogeneous graph attention networks

Automatic arrhythmia detection is very important for cardiovascular health. It is generally performed by measuring the electrocardiogram (ECG) signals of standard multiple leads. However, the correlations of multiple leads are often ignored. In addition, an extensive and complex feature extraction process is usually needed in most existing studies. Therefore, these challenges will not only lead to the loss of overall lead information, but also cause the detection performance to depend on the quality of features. To solve these challenges, a novel multi-lead arrhythmia detection model based on a heterogeneous graph attention network is proposed in this paper. We have modeled the multi-lead data as a heterogeneous graph to integrate diverse information and construct intra-lead and inter-lead correlations in multi-lead data, providing a reasonable and effective the data model. A heterogeneous graph network with a dual-level attention strategy has been utilized to capture the interactions among diverse information and information types. At the same time, our model does not require any feature extraction process for the ECG signals, which avoids out complex feature engineering. Extensive experimental results show that multi-lead information and complex correlations can be well captured, thus confirming that the proposed model results in significant improvements in multi-lead arrhythmia detection.

Investigating Multi-Level Semantic Extraction with Squash Capsules for Short Text Classification

At present, short text classification is a hot topic in the area of natural language processing. Due to the sparseness and irregularity of short text, the task of short text classification still faces great challenges. In this paper, we propose a new classification model from the aspects of short text representation, global feature extraction and local feature extraction. We use convolutional networks to extract shallow features from short text vectorization, and introduce a multi-level semantic extraction framework. It uses BiLSTM as the encoding layer while the attention mechanism and normalization are used as the interaction layer. Finally, we concatenate the convolution feature vector and semantic results of the semantic framework. After several rounds of feature integration, the framework improves the quality of the feature representation. Combined with the capsule network, we obtain high-level local information by dynamic routing and then squash them. In addition, we explore the optimal depth of semantic feature extraction for short text based on a multi-level semantic framework. We utilized four benchmark datasets to demonstrate that our model provides comparable results. The experimental results show that the accuracy of SUBJ, TREC, MR and ProcCons are 93.8%, 91.94%, 82.81% and 98.43%, respectively, which verifies that our model has greatly improves classification accuracy and model robustness.