GHS-NET a generic hybridized shallow neural network for multi-label biomedical text classification

Detecting mental and physical disorders using multi-task learning equipped with knowledge graph attention network

Mental and physical disorders (MPD) are inextricably linked in many medical cases; psychosomatic diseases can be induced by mental concerns and psychological discomfort can ensue from physiological diseases. However, existing medical informatics studies focus on identifying mental or physical disorders from a unilateral perspective. Consequently, no existing domain knowledge base, corpus, or detection modeling approach considers mental as well as physical aspects concurrently. This paper proposes a joint modeling approach to detect MPD. First, we crawl through online medical consultation records of patients from websites and build an MPD knowledge ontology by extracting the core conceptual features of the text. Based on the ontology, an MPD knowledge graph containing 12,673 nodes and 82,195 relations is obtained using term matching with a domain thesaurus of each concept. Subsequently, an MPD corpus with fine-grained severities (None, Mild, Moderate, Severe, Dangerous) and 8909 records is constructed by formulating MPD classification criteria and a data annotation process under the guidance of domain experts. Taking the knowledge graph and corpus as the dataset, we design a multi-task learning model to detect the MPD severity, in which a knowledge graph attention network (KGAT) is embedded to better extract knowledge features. Experiments are performed to demonstrate the effectiveness of our model. Furthermore, we employ ontology-based and centrality-based methods to discover additional potential inferred knowledge, which can be captured by KGAT so as to improve the prediction performance and interpretability of our model. Our dataset has been made publicly available, so it can be further used as a medical informatics reference in the fields of psychosomatic medicine, psychiatrics, physical co-morbidity, and so on.

Interrelated feature selection from health surveys using domain knowledge graph

Finding patterns among risk factors and chronic illness can suggest similar causes, provide guidance to improve healthy lifestyles, and give clues for possible treatments for outliers. Prior studies have typically isolated data challenges from single-disease datasets. However, the predictive power of multiple diseases is more helpful in establishing a healthy lifestyle than investigating one disease. Most studies typically focus on single-disease datasets; however, to ensure that health advice is generalized and contemporary, the features that predict the likelihood of many diseases can improve health advice effectiveness when considering the patient's point of view. We construct and present a novel knowledge-based qualitative method to remove redundant features from a dataset and redefine the outliers. The results of our trials upon five annual chronic disease health surveys demonstrate that our Knowledge Graph-based feature selection, when applied to many machine learning and deep learning multi-label classifiers, can improve classification performance. Our methodology is compatible with future directions, such as graph neural networks. It provides clinicians with an efficient process to select the most relevant health survey questions and responses regarding single or many human organ systems.

Medical text classification based on the discriminative pre-training model and prompt-tuning

Medical text classification, as a fundamental medical natural language processing task, aims to identify the categories to which a short medical text belongs. Current research has focused on performing the medical text classification task using a pre-training language model through fine-tuning. However, this paradigm introduces additional parameters when training extra classifiers. Recent studies have shown that the "prompt-tuning" paradigm induces better performance in many natural language processing tasks because it bridges the gap between pre-training goals and downstream tasks. The main idea of prompt-tuning is to transform binary or multi-classification tasks into mask prediction tasks by fully exploiting the features learned by pre-training language models. This study explores, for the first time, how to classify medical texts using a discriminative pre-training language model called ERNIE-Health through prompt-tuning. Specifically, we attempt to perform prompt-tuning based on the multi-token selection task, which is a pre-training task of ERNIE-Health. The raw text is wrapped into a new sequence with a template in which the category label is replaced by a [UNK] token. The model is then trained to calculate the probability distribution of the candidate categories. Our method is tested on the KUAKE-Question Intention Classification and CHiP-Clinical Trial Criterion datasets and obtains the accuracy values of 0.866 and 0.861. In addition, the loss values of our model decrease faster throughout the training period compared to the fine-tuning. The experimental results provide valuable insights to the community and suggest that prompt-tuning can be a promising approach to improve the performance of pre-training models in domain-specific tasks.

Ensemble of Networks for Multilabel Classification

Multilabel learning goes beyond standard supervised learning models by associating a sample with more than one class label. Among the many techniques developed in the last decade to handle multilabel learning best approaches are those harnessing the power of ensembles and deep learners. This work proposes merging both methods by combining a set of gated recurrent units, temporal convolutional neural networks, and long short-term memory networks trained with variants of the Adam optimization approach. We examine many Adam variants, each fundamentally based on the difference between present and past gradients, with step size adjusted for each parameter. We also combine Incorporating Multiple Clustering Centers and a bootstrap-aggregated decision trees ensemble, which is shown to further boost classification performance. In addition, we provide an ablation study for assessing the performance improvement that each module of our ensemble produces. Multiple experiments on a large set of datasets representing a wide variety of multilabel tasks demonstrate the robustness of our best ensemble, which is shown to outperform the state-of-the-art.

TextConvoNet: a convolutional neural network based architecture for text classification

This paper presents, TextConvoNet, a novel Convolutional Neural Network (CNN) based architecture for binary and multi-class text classification problems. Most of the existing CNN-based models use one-dimensional convolving filters, where each filter specializes in extracting n-grams features of a particular input word embeddings (Sentence Matrix). These features can be termed as intra-sentence n-gram features. To the best of our knowledge, all the existing CNN models for text classification are based on the aforementioned concept. The presented TextConvoNet not only extracts the intra-sentence n-gram features but also captures the inter-sentence n-gram features in input text data. It uses an alternative approach for input matrix representation and applies a two-dimensional multi-scale convolutional operation on the input. We perform an experimental study on five binary and multi-class classification datasets and evaluate the performance of the TextConvoNet for text classification. The results are evaluated using eight performance measures, accuracy, precision, recall, f1-score, specificity, gmean1, gmean2, and Mathews correlation coefficient (MCC). Furthermore, we extensively compared presented TextConvoNet with machine learning, deep learning, and attention-based models. The experimental results evidenced that the presented TextConvoNet outperformed and yielded better performance than the other used models for text classification purposes.

Consumers' Opinions towards Public Health Effects of Online Games: An Empirical Study Based on Social Media Comments in China

Online game products have fueled the boom in China's digital economy. Meanwhile, its public health concerns have sparked discussion among consumers on social media. However, past research has seldom studied the public health topics caused by online games from the perspective of consumer opinions. This paper attempts to identify consumers' opinions on the health impact of online game products through non-structured text and large-size social media comments. Thus, we designed a natural language processing (NLP) framework based on machine learning, which consists of topic mining, multi-label classification, and sentimental analysis. The hierarchical clustering method-based topic mining procedure determines the compatibility of this study and previous research. Every three topics are identified in "Personal Health Effects" and "Social Health Effects", respectively. Then, the multi-label classification model's results show that 61.62% of 327,505 comments have opinions about the health effects of online games. Topics "Adolescent Education" and "Commercial Morality" occupy the top two places of consumer attention. More than 31% of comments support two or more topics, and the "Adolescent Education" and "Commercial Morality" combination also have the highest co-occurrence. Finally, consumers expressed different emotional preferences for different topics, with an average of 63% of comments expressing negative emotions related to the health attributes of online games. In general, Chinese consumers are most concerned with adolescent education issues and hold the strongest negative emotion towards the commercial morality problems of enterprises. The significance of research results is that it reminds online game-related enterprises to pay attention to the potential harm to public health while bringing about additional profits through online game products. Furthermore, negative consumer emotions may cause damage to brand image, business reputation, and the sustainable development of the enterprises themselves. It also provides the government supervision departments with an advanced analysis method reference for more effective administration to protect public health and promote the development of the digital economy.

A Survey on Text Classification Algorithms: From Text to Predictions

In recent years, the exponential growth of digital documents has been met by rapid progress in text classification techniques. Newly proposed machine learning algorithms leverage the latest advancements in deep learning methods, allowing for the automatic extraction of expressive features. The swift development of these methods has led to a plethora of strategies to encode natural language into machine-interpretable data. The latest language modelling algorithms are used in conjunction with ad hoc preprocessing procedures, of which the description is often omitted in favour of a more detailed explanation of the classification step. This paper offers a concise review of recent text classification models, with emphasis on the flow of data, from raw text to output labels. We highlight the differences between earlier methods and more recent, deep learning-based methods in both their functioning and in how they transform input data. To give a better perspective on the text classification landscape, we provide an overview of datasets for the English language, as well as supplying instructions for the synthesis of two new multilabel datasets, which we found to be particularly scarce in this setting. Finally, we provide an outline of new experimental results and discuss the open research challenges posed by deep learning-based language models.

A New Hybrid Based on Long Short-Term Memory Network with Spotted Hyena Optimization Algorithm for Multi-Label Text Classification

An essential work in natural language processing is the Multi-Label Text Classification (MLTC). The purpose of the MLTC is to assign multiple labels to each document. Traditional text classification methods, such as machine learning usually involve data scattering and failure to discover relationships between data. With the development of deep learning algorithms, many authors have used deep learning in MLTC. In this paper, a novel model called Spotted Hyena Optimizer (SHO)-Long Short-Term Memory (SHO-LSTM) for MLTC based on LSTM network and SHO algorithm is proposed. In the LSTM network, the Skip-gram method is used to embed words into the vector space. The new model uses the SHO algorithm to optimize the initial weight of the LSTM network. Adjusting the weight matrix in LSTM is a major challenge. If the weight of the neurons to be accurate, then the accuracy of the output will be higher. The SHO algorithm is a population-based meta-heuristic algorithm that works based on the mass hunting behavior of spotted hyenas. In this algorithm, each solution of the problem is coded as a hyena. Then the hyenas are approached to the optimal answer by following the hyena of the leader. Four datasets are used (RCV1-v2, EUR-Lex, Reuters-21578, and Bookmarks) to evaluate the proposed model. The assessments demonstrate that the proposed model has a higher accuracy rate than LSTM, Genetic Algorithm-LSTM (GA-LSTM), Particle Swarm Optimization-LSTM (PSO-LSTM), Artificial Bee Colony-LSTM (ABC-LSTM), Harmony Algorithm Search-LSTM (HAS-LSTM), and Differential Evolution-LSTM (DE-LSTM). The improvement of SHO-LSTM model accuracy for four datasets compared to LSTM is 7.52%, 7.12%, 1.92%, and 4.90%, respectively.

Automatic Film Label Acquisition Method Based on Improved Neural Networks Optimized by Mutation Ant Colony Algorithm

Nowadays, with the constant change of public aesthetic standards, a large number of new types and themes of film programs have emerged. For this reason, this paper proposes an improved neural network optimized by mutation ant colony algorithm for automatic acquisition of film labels, which not only overcomes the disadvantages of traditional neural network, such as difficulty in determining weights, slow convergence speed, and easiness to fall into local minimum, but also makes up for the shortcomings faced by using ant colony algorithm alone through the gradient information of quantum genetic algorithm neural network. The results show that the user similarity judgment is added in the process of calculating the user rating deviation between movies, and the neighbor chooses to add the movie tag weight and rating similarity as the basis for the neighbor selection of the target movie in the process of predicting the target movie rating. Experiments show the effectiveness of the algorithm.