Survey of BERT-Base Models for Scientific Text Classification: COVID-19 Case Study

Deep learning-based electricity theft prediction in non-smart grid environments

In developing countries, smart grids are nonexistent, and electricity theft significantly hampers power supply. This research introduces a lightweight deep-learning model using monthly customer readings as input data. By employing careful direct and indirect feature engineering techniques, including Principal Component Analysis (PCA), t-distributed Stochastic Neighbor Embedding (t-SNE), UMAP (Uniform Manifold Approximation and Projection), and resampling methods such as Random-Under-Sampler (RUS), Synthetic Minority Over-sampling Technique (SMOTE), and Random-Over-Sampler (ROS), an effective solution is proposed. Previous studies indicate that models achieve high precision, recall, and F1 score for the non-theft (0) class, but perform poorly, even achieving 0 %, for the theft (1) class. Through parameter tuning and employing Random-Over-Sampler (ROS), significant improvements in accuracy, precision (89 %), recall (94 %), and F1 score (91 %) for the theft (1) class are achieved. The results demonstrate that the proposed model outperforms existing methods, showcasing its efficacy in detecting electricity theft in non-smart grid environments.

Exploring diverse interests of collaborators in smart cities: A topic analysis using LDA and BERT

Smart cities have emerged as a promising solution to the problems associated with urbanization. However, research that holistically considers diverse stakeholders in smart cities is scarce. This study utilizes data from four types of collaborators (academia, public sector, industry, and civil society actors) to identify key topics and suggest research areas for developing smart cities. We used latent Dirichlet allocation and Bidirectional Encoder Representations from Transformers for topic extraction and analysis. The analysis reveals that sustainability and digital platform have received similar levels of interest from academia, industry, and government, whereas governance, resource, and green space are less frequently mentioned than technology-related topics. Hype cycle analysis, which considers public and media expectations, reveals that smart cities experienced rapid growth from 2015 to 2021, but the growth rate has slowed since 2022. This means that a breakthrough improvement in the current situation is required. Accordingly, we propose resolving the unbalanced distribution of topic interests among collaborators, especially in the areas of governance, environment, economy, and healthcare. We expect that our findings will help researchers, policymakers, and industry stakeholders in understanding which topics are underdeveloped in their fields and taking active measures for the future development of smart cities.

An Automated Analysis Framework for Epidemiological Survey on COVID-19

For a long time, the prevention and control of COVID-19 has received significant attention. A crucial aspect of controlling the disease's spread is the epidemiological survey of patients and the subsequent analysis of epidemiological survey reports (case reports). However, current mainstream analysis approaches are all made manually. This manual method is time-consuming and manpower-intensive. This paper designs an automated visual epidemiological survey analysis (AVESA) framework for the epidemiological survey on COVID-19. AVESA designs a deep neural network for information extraction from case reports and automatically constructs an epidemiological knowledge graph based on predefined pattern. Moreover, a multi-dimensional knowledge reasoning model is developed for conducting knowledge reasoning in the complete COVID-19 epidemiological knowledge graph. In the entity extraction sub-task and multi-task extraction sub-task, AVESA achieved F1 scores of 85.12% and 92.29% respectively on the constructed dataset, significantly outperforming the standalone information extraction models. In full-graph computing, all three experiments align closely with manual analysis standards. In the risk analysis experiment, the weighted PageRank algorithm showed an average improvement of 11.21% in Top_Recall_n% over the standard PageRank algorithm. In the community detection experiment, the weighted Louvain algorithm showed a mere 4.34% community difference rate compared to manual analysis.

An interpretable method for automated classification of spoken transcripts and written text

We investigate the differences between spoken language (in the form of radio show transcripts) and written language (Wikipedia articles) in the context of text classification. We present a novel, interpretable method for text classification, involving a linear classifier using a large set of n - gram features, and apply it to a newly generated data set with sentences originating either from spoken transcripts or written text. Our classifier reaches an accuracy less than 0.02 below that of a commonly used classifier (DistilBERT) based on deep neural networks (DNNs). Moreover, our classifier has an integrated measure of confidence, for assessing the reliability of a given classification. An online tool is provided for demonstrating our classifier, particularly its interpretable nature, which is a crucial feature in classification tasks involving high-stakes decision-making. We also study the capability of DistilBERT to carry out fill-in-the-blank tasks in either spoken or written text, and find it to perform similarly in both cases. Our main conclusion is that, with careful improvements, the performance gap between classical methods and DNN-based methods may be reduced significantly, such that the choice of classification method comes down to the need (if any) for interpretability.

Mental Health Analysis in Social Media Posts: A Survey

The surge in internet use to express personal thoughts and beliefs makes it increasingly feasible for the social NLP research community to find and validate associations between social media posts and mental health status. Cross-sectional and longitudinal studies of social media data bring to fore the importance of real-time responsible AI models for mental health analysis. Aiming to classify the research directions for social computing and tracking advances in the development of machine learning (ML) and deep learning (DL) based models, we propose a comprehensive survey on quantifying mental health on social media. We compose a taxonomy for mental healthcare and highlight recent attempts in examining social well-being with personal writings on social media. We define all the possible research directions for mental healthcare and investigate a thread of handling online social media data for stress, depression and suicide detection for this work. The key features of this manuscript are (i) feature extraction and classification, (ii) recent advancements in AI models, (iii) publicly available dataset, (iv) new frontiers and future research directions. We compile this information to introduce young research and academic practitioners with the field of computational intelligence for mental health analysis on social media. In this manuscript, we carry out a quantitative synthesis and a qualitative review with the corpus of over 92 potential research articles. In this context, we release the collection of existing work on suicide detection in an easily accessible and updatable repository:https://github.com/drmuskangarg/mentalhealthcare.

Special Issue on Big Data for eHealth Applications

In the last few years, the rapid growth in available digitised medical data has opened new challenges for the scientific research community in the healthcare informatics field [...]