Monolingual and multilingual topic analysis using LDA and BERT embeddings

Exploring climate change discourse on social media and blogs using a topic modeling analysis

Climate change is one of the most pressing global issues of our time, and understanding public perception and awareness of the topic is crucial for developing effective policies to mitigate its effects. While traditional survey methods have been used to gauge public opinion, advances in natural language processing (NLP) and data visualization techniques offer new opportunities to analyze user-generated content from social media and blog posts. In this study, a new dataset of climate change-related texts was collected from social media sources and various blogs. The dataset was analyzed using BERTopic and LDA to identify and visualize the most important topics related to climate change. The study also used sentence similarity to determine the similarities in the comments written and which topic categories they belonged to. The performance of different techniques for keyword extraction and text representation, including OpenAI, Maximal Marginal Relevance (MMR), and KeyBERT, was compared for topic modeling with BERTopic. It was seen that the best coherence score and topic diversity metric were obtained with OpenAI-based BERTopic. The results provide insights into the public's attitudes and perceptions towards climate change, which can inform policy development and contribute to efforts to reduce activities that cause climate change.

Exploring diverse interests of collaborators in smart cities: A topic analysis using LDA and BERT

Smart cities have emerged as a promising solution to the problems associated with urbanization. However, research that holistically considers diverse stakeholders in smart cities is scarce. This study utilizes data from four types of collaborators (academia, public sector, industry, and civil society actors) to identify key topics and suggest research areas for developing smart cities. We used latent Dirichlet allocation and Bidirectional Encoder Representations from Transformers for topic extraction and analysis. The analysis reveals that sustainability and digital platform have received similar levels of interest from academia, industry, and government, whereas governance, resource, and green space are less frequently mentioned than technology-related topics. Hype cycle analysis, which considers public and media expectations, reveals that smart cities experienced rapid growth from 2015 to 2021, but the growth rate has slowed since 2022. This means that a breakthrough improvement in the current situation is required. Accordingly, we propose resolving the unbalanced distribution of topic interests among collaborators, especially in the areas of governance, environment, economy, and healthcare. We expect that our findings will help researchers, policymakers, and industry stakeholders in understanding which topics are underdeveloped in their fields and taking active measures for the future development of smart cities.

Public Opinion About COVID-19 on a Microblog Platform in China: Topic Modeling and Multidimensional Sentiment Analysis of Social Media

BACKGROUND

The COVID-19 pandemic raised wide concern from all walks of life globally. Social media platforms became an important channel for information dissemination and an effective medium for public sentiment transmission during the COVID-19 pandemic.

OBJECTIVE

Mining and analyzing social media text information can not only reflect the changes in public sentiment characteristics during the COVID-19 pandemic but also help the government understand the trends in public opinion and reasonably control public opinion.

METHODS

First, this study collected microblog comments related to the COVID-19 pandemic as a data set. Second, sentiment analysis was carried out based on the topic modeling method combining latent Dirichlet allocation (LDA) and Bidirectional Encoder Representations from Transformers (BERT). Finally, a machine learning linear regression (ML-LR) model combined with a sparse matrix was proposed to explore the evolutionary trend in public opinion on social media and verify the high accuracy of the model.

RESULTS

The experimental results show that, in different stages, the characteristics of public emotion are different, and the overall trend is from negative to positive.

CONCLUSIONS

The proposed method can effectively reflect the characteristics of the different times and space of public opinion. The results provide theoretical support and practical reference in response to public health and safety events.

An integrated clustering and BERT framework for improved topic modeling

Topic modelling is a machine learning technique that is extensively used in Natural Language Processing (NLP) applications to infer topics within unstructured textual data. Latent Dirichlet Allocation (LDA) is one of the most used topic modeling techniques that can automatically detect topics from a huge collection of text documents. However, the LDA-based topic models alone do not always provide promising results. Clustering is one of the effective unsupervised machine learning algorithms that are extensively used in applications including extracting information from unstructured textual data and topic modeling. A hybrid model of Bidirectional Encoder Representations from Transformers (BERT) and Latent Dirichlet Allocation (LDA) in topic modeling with clustering based on dimensionality reduction have been studied in detail. As the clustering algorithms are computationally complex, the complexity increases with the higher number of features, the PCA, t-SNE and UMAP based dimensionality reduction methods are also performed. Finally, a unified clustering-based framework using BERT and LDA is proposed as part of this study for mining a set of meaningful topics from the massive text corpora. The experiments are conducted to demonstrate the effectiveness of the cluster-informed topic modeling framework using BERT and LDA by simulating user input on benchmark datasets. The experimental results show that clustering with dimensionality reduction would help infer more coherent topics and hence this unified clustering and BERT-LDA based approach can be effectively utilized for building topic modeling applications.

A content-aware corpus-based model for analysis of marine accidents

In the past decades, marine accidents brought the serious loss of life and property and environmental contamination. With the accumulation of marine accident data, especially accident investigation reports, compared with subjective reasoning based on expert experience, data-driven methods for analysis and accident prevention are more comprehensive and objective. This paper aims to develop a content-aware corpus-based model for the analysis of marine accidents to mine the accident semantic features. The general research framework is established to combine accident data, expert prior knowledge, and semi-automated natural language processing (NLP) technology. The NLP models are optimized, fused, and applied to the case study of ship collision accidents. The results show that the proposed model can accurately and quickly extract hazards, accident causes, and scenarios from the accident reports, and perform semantic analysis for the latent relationships between them to extend the accident causation theory. This study can provide a powerful and innovative analysis tool for marine accidents for maritime traffic safety management departments and relevant research institutions.

Research on Topic Evolution Path Recognition Based on LDA2vec Symmetry Model

Topic extraction and evolution analysis became a research hotspot in the academic community due to its ability to reveal the development trend of a certain field and discover the evolution law of topic content in different development stages of the field. However, current research methods still face challenges, such as inaccurate topic recognition and unclear evolution paths, which can seriously compromise the comprehensiveness and accuracy of the analysis. To address the problem, the paper proposes a topic evolution path recognition method based on the LDA2vec symmetry model. Under given conditions, both the LDA and Word2vec used in the model conform to the structural symmetry of their datasets in high-dimensional space, and the fused LDA2vec method improves the accuracy of the analysis results. Firstly, we recognize the topics based on the LDA model, which uses Gibbs symmetric sampling and obeys the symmetric Dirichlet distribution to ensure data convergence. Secondly, Word2vec is used to learn the contextual information of the topic words in the document collection, and the words in the corpus are projected as vectors in the high-dimensional space so that the computed pairs of words with similar semantics have symmetry in the hyperplane of the high-dimensional space. Subsequently, the word vector is used as a weight, and the LDA topic word probability value is weighted to generate a new topic vector. Thirdly, the vector similarity index is employed to calculate the semantic similarity among topics at adjacent stages, and evolution paths that directly reflect the topic relationships are constructed. Finally, an empirical study is conducted in the field of data security to demonstrate the effectiveness of the proposed approach for topic evolution analysis. The results show that the proposed approach can accurately recognize the topic content and construct clear evolution paths, which contribute to the comprehensive and accurate analysis of topic evolution in a specific research field.

Security Analysis of Social Network Topic Mining Using Big Data and Optimized Deep Convolutional Neural Network

This research aims to conduct topic mining and data analysis of social network security using social network big data. At present, the main problem is that users’ behavior on social networks may reveal their private data. The main contribution lies in the establishment of a network security topic detection model combining Convolutional Neural Network (CNN) and social network big data technology. Deep Convolution Neural Network (DCNN) is utilized to complete the analysis and search of social network security issues. The Long Short-Term Memory (LSTM) algorithm is used for the extraction of Weibo topic information in the memory wisdom. Experimental results show that the recognition accuracy of the constructed model can reach 96.17% after 120 iterations, which is at least 5.4% higher than other models. Additionally, the accuracy, recall, and F1 value of the intrusion detection model are 88.57%, 75.22%, and 72.05%, respectively. Compared with other algorithms, the model’s accuracy, recall, and F1 value are at least 3.1% higher than other models. In addition, the training time and testing time of the improved DCNN network security detection model are stabilized at 65.86 s and 27.90 s, respectively. The prediction time of the improved DCNN network security detection model is significantly shortened compared with that of the models proposed by other scholars. The experimental conclusion is that the improved DCNN has the characteristics of lower delay under deep learning. The model shows good performance for network data security transmission.

Study on Influencing Factors of Construction Workers' Unsafe Behavior Based on Text Mining

The unsafe behavior of construction workers is the key cause of safety accidents. The accident investigation report contains rich experience and lessons, which can be used to prevent and reduce the occurrence of safety accidents. In order to draw lessons from the accident and realize knowledge sharing and reuse, this paper uses text mining technology to analyze the data of 500 construction accident investigation reports in Shenzhen, China. Firstly, a Latent Dirichlet Allocation (LDA) topic model is used to identify the unsafe behavior of construction workers and its influencing factors. Then, with the help of Social Network Analysis, the importance of influencing factors and the relationship between them are identified. The results show that weak safety awareness, operating regulations, supervision dereliction of duty, equipment resources, and inadequate supervision of the construction party are the key and important factors. It is also found that there are correlations between weak safety awareness and supervision dereliction of duty, between equipment resources and poor construction environment, between organization and coordination and inadequate supervision of the construction party, and between operating regulations and hidden dangers investigation. This study not only helps to improve the theoretical system in the field of construction workers’ unsafe behavior but also helps managers to find the key control direction of construction safety, so as to effectively curb unsafe behavior of construction workers and improve the level of safety management.

Collaboration of issuing agencies and topic evolution of health informatisation policies in China

Digital transformation in the Chinese healthcare industry has led national government agencies to issue a series of policies to guide the construction of health informatisation. However, little is known about the issuing agencies and the topics of health informatisation policies. This study aimed to explore the collaboration of policies issuing and the evolution of policy topics. In this study, a total of 156 policy documents were identified. Author–Topic model and pre-discretised method based on Latent Dirichlet Allocation model were employed to mine the correlation between the issuing agencies and policy topics, and the evolution of policy contents. Findings suggest that the development of health informatisation policies can be divided into three stages. The number of policies has been increasing constantly, among which the policy of opinion and notification accounts for the vast majority. Many government agencies are involved in formulating policies collaboratively. On the whole, the topics changed constantly over time. From 2003 to 2008, policy topics focused on standards and specifications, with the phenomenon of splitting and development. From 2009 to 2014, policies were predominantly related to the construction of regional health informatisation, with some emerging topics generating. Internet + medical and new information technology gained attention from 2015 to 2020; most topics in this period were inherited, split or merged from the previous period. This study is helpful to research and formulation of the health informatisation-related policies.

Identifying Silver Linings During the Pandemic Through Natural Language Processing

COVID-19 has presented an unprecedented challenge to human welfare. Indeed, we have witnessed people experiencing a rise of depression, acute stress disorder, and worsening levels of subclinical psychological distress. Finding ways to support individuals' mental health has been particularly difficult during this pandemic. An opportunity for intervention to protect individuals' health & well-being is to identify the existing sources of consolation and hope that have helped people persevere through the early days of the pandemic. In this paper, we identified positive aspects, or “silver linings,” that people experienced during the COVID-19 crisis using computational natural language processing methods and qualitative thematic content analysis. These silver linings revealed sources of strength that included finding a sense of community, closeness, gratitude, and a belief that the pandemic may spur positive social change. People's abilities to engage in benefit-finding and leverage protective factors can be bolstered and reinforced by public health policy to improve society's resilience to the distress of this pandemic and potential future health crises.

Evolution and diffusion of information literacy topics

Investigation of the topic of information literacy and its changes can be informative for researchers and provide a better understanding of the corresponding domains. This study conducted a topic model dynamic analysis of the articles on information literacy studies in the Web of Science core collection database that were published from 2005 to 2019. The global topics and their popularities, topical similarities and correlations, along with the evolution of temporal local topics and the diffusion of subject local topics were analyzed and presented. Nine global topics differed in terms of their temporal and subject characteristics, and this study focused on ability, technology, field, people, place and application of information literacy. For the temporal local topics, crossing was the main evolutionary mechanism; hence, the core topic words were relatively stable, but few new research directions have been explored in recent years. For the subject local topics, absorbing with division and absorbing were the main mechanisms, which supported the diffusion progress of information literacy studies among subjects. However, it is necessary to promote the development of future research through the innovative development of multidisciplinary integration. Researchers and practitioners should focus on the impact of information technology, increase the breadth and depth of the research field, and develop innovative evaluation methods that are based on data to promote the comprehensive, sustainable and effective improvement in information literacy.