CPM: A large-scale generative Chinese Pre-trained language model

Emergency entity relationship extraction for water diversion project based on pre-trained model and multi-featured graph convolutional network

Using information technology to extract emergency decision-making knowledge from emergency plan documents is an essential means to enhance the efficiency and capacity of emergency management. To address the problems of numerous terminologies and complex relationships faced by emergency knowledge extraction of water diversion project, a multi-feature graph convolutional network (PTM-MFGCN) based on pre-trained model is proposed. Initially, through the utilization of random masking of domain-specific terminologies during pre-training, the model's comprehension of the meaning and application of such terminologies within specific fields is enhanced, thereby augmenting the network's proficiency in extracting professional terminologies. Furthermore, by introducing a multi-feature adjacency matrix to capture a broader range of neighboring node information, thereby enhancing the network's ability to handle complex relationships. Lastly, we utilize the PTM-MFGCN to achieve the extraction of emergency entity relationships in water diversion project, thus constructing a knowledge graph for water diversion emergency management. The experimental results demonstrate that PTM-MFGCN exhibits improvements of 2.84% in accuracy, 4.87% in recall, and 5.18% in F1 score, compared to the baseline model. Relevant studies can effectively enhance the efficiency and capability of emergency management, mitigating the impact of unforeseen events on engineering safety.

Comparison of pre-trained language models in terms of carbon emissions, time and accuracy in multi-label text classification using AutoML

Since Turkish is an agglutinative language and contains reduplication, idiom, and metaphor words, Turkish texts are sources of information with extremely rich meanings. For this reason, the processing and classification of Turkish texts according to their characteristics is both time-consuming and difficult. In this study, the performances of pre-trained language models for multi-text classification using Autotrain were compared in a 250 K Turkish dataset that we created. The results showed that the BERTurk (uncased, 128 k) language model on the dataset showed higher accuracy performance with a training time of 66 min compared to the other models and the CO2 emission was quite low. The ConvBERTurk mC4 (uncased) model is also the best-performing second language model. As a result of this study, we have provided a deeper understanding of the capabilities of pre-trained language models for Turkish on machine learning.