A hybrid structured deep neural network with Word2Vec for construction accident causes classification

Systematic review of predictive maintenance and digital twin technologies challenges, opportunities, and best practices

Background

Maintaining machines effectively continues to be a challenge for industrial organisations, which frequently employ reactive or premeditated methods. Recent research has begun to shift its attention towards the application of Predictive Maintenance (PdM) and Digital Twins (DT) principles in order to improve maintenance processes. PdM technologies have the capacity to significantly improve profitability, safety, and sustainability in various industries. Significantly, precise equipment estimation, enabled by robust supervised learning techniques, is critical to the efficacy of PdM in conjunction with DT development. This study underscores the application of PdM and DT, exploring its transformative potential across domains demanding real-time monitoring. Specifically, it delves into emerging fields in healthcare, utilities (smart water management), and agriculture (smart farm), aligning with the latest research frontiers in these areas.

Methodology

Employing the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) criteria, this study highlights diverse modeling techniques shaping asset lifetime evaluation within the PdM context from 34 scholarly articles.

Results

The study revealed four important findings: various PdM and DT modelling techniques, their diverse approaches, predictive outcomes, and implementation of maintenance management. These findings align with the ongoing exploration of emerging applications in healthcare, utilities (smart water management), and agriculture (smart farm). In addition, it sheds light on the critical functions of PdM and DT, emphasising their extraordinary ability to drive revolutionary change in dynamic industrial challenges. The results highlight these methodologies' flexibility and application across many industries, providing vital insights into their potential to revolutionise asset management and maintenance practice for real-time monitoring.

Conclusions

Therefore, this systematic review provides a current and essential resource for academics, practitioners, and policymakers to refine PdM strategies and expand the applicability of DT in diverse industrial sectors.

Contextualizing injury severity from occupational accident reports using an optimized deep learning prediction model

Background

This study introduced a novel approach for predicting occupational injury severity by leveraging deep learning-based text classification techniques to analyze unstructured narratives. Unlike conventional methods that rely on structured data, our approach recognizes the richness of information within injury narrative descriptions with the aim of extracting valuable insights for improved occupational injury severity assessment.

Methods

Natural language processing (NLP) techniques were harnessed to preprocess the occupational injury narratives obtained from the US Occupational Safety and Health Administration (OSHA) from January 2015 to June 2023. The methodology involved meticulous preprocessing of textual narratives to standardize text and eliminate noise, followed by the innovative integration of Term Frequency-Inverse Document Frequency (TF-IDF) and Global Vector (GloVe) word embeddings for effective text representation. The proposed predictive model adopts a novel Bidirectional Long Short-Term Memory (Bi-LSTM) architecture and is further refined through model optimization, including random search hyperparameters and in-depth feature importance analysis. The optimized Bi-LSTM model has been compared and validated against other machine learning classifiers which are naïve Bayes, support vector machine, random forest, decision trees, and K-nearest neighbor.

Results

The proposed optimized Bi-LSTM models' superior predictability, boasted an accuracy of 0.95 for hospitalization and 0.98 for amputation cases with faster model processing times. Interestingly, the feature importance analysis revealed predictive keywords related to the causal factors of occupational injuries thereby providing valuable insights to enhance model interpretability.

Conclusion

Our proposed optimized Bi-LSTM model offers safety and health practitioners an effective tool to empower workplace safety proactive measures, thereby contributing to business productivity and sustainability. This study lays the foundation for further exploration of predictive analytics in the occupational safety and health domain.

Predicting occupational injury causal factors using text-based analytics: A systematic review

Workplace accidents can cause a catastrophic loss to the company including human injuries and fatalities. Occupational injury reports may provide a detailed description of how the incidents occurred. Thus, the narrative is a useful information to extract, classify and analyze occupational injury. This study provides a systematic review of text mining and Natural Language Processing (NLP) applications to extract text narratives from occupational injury reports. A systematic search was conducted through multiple databases including Scopus, PubMed, and Science Direct. Only original studies that examined the application of machine and deep learning-based Natural Language Processing models for occupational injury analysis were incorporated in this study. A total of 27, out of 210 articles were reviewed in this study by adopting the Preferred Reporting Items for Systematic Review (PRISMA). This review highlighted that various machine and deep learning-based NLP models such as K-means, Naïve Bayes, Support Vector Machine, Decision Tree, and K-Nearest Neighbors were applied to predict occupational injury. On top of these models, deep neural networks are also included in classifying the type of accidents and identifying the causal factors. However, there is a paucity in using the deep learning models in extracting the occupational injury reports. This is due to these techniques are pretty much very recent and making inroads into decision-making in occupational safety and health as a whole. Despite that, this paper believed that there is a huge and promising potential to explore the application of NLP and text-based analytics in this occupational injury research field. Therefore, the improvement of data balancing techniques and the development of an automated decision-making support system for occupational injury by applying the deep learning-based NLP models are the recommendations given for future research.

Towards Automated Construction Quantity Take-Off: An Integrated Approach to Information Extraction from Work Descriptions

Construction-oriented quantity take-off (QTO) refers to the process of determining the quantities for construction items or work packages in accordance with their descriptions. However, the current construction-oriented QTO practice relies on estimators’ manual interpretation of work descriptions and manual processes to look up proper building objects for quantity calculation. Hence, this research aims to develop natural language processing (NLP) and rule-based algorithms to automate the information extraction (IE) from work descriptions for QTO in building construction. Specifically, several named entity recognition (NER) models, including Hidden Markov Model (HMM), Conditional Random Field (CRF), Bidirectional-Long Short-Term Memory (Bi-LSTM), and Bi-LSTM+CRF, were developed to identify construction activities, material, building component, product features, measurement unit, and additional information (e.g., work scope) from work descriptions. Cost items in the RSMeans database are used to evaluate the developed models in terms of F1 scores. HMM was found to achieve a 5% higher F1 score in the NER than the other three algorithms. Then, labeling rules and active learning strategies were applied along with the HMM model, which improved F1 score by 3% and reduced the labeling efforts by 26%. The results showed that the proposed IE method successfully interprets the desired information from the work description for QTO. This research contributed to the body of knowledge by the NLP-based information extraction model integrating HMM and formalized labeling rules that automatically process work descriptions and lay a foundation for automated QTO and cost estimation.

Relational Graph Convolutional Network for Text-Mining-Based Accident Causal Classification

Accident investigation reports are text documents that systematically review and analyze the cause and process of accidents after accidents have occurred and have been widely used in the fields such as transportation, construction and aerospace. With the aid of accident investigation reports, the cause of the accident can be clearly identified, which provides an important basis for accident prevention and reliability assessment. However, since accident record reports are mostly composed of unstructured data such as text, the analysis of accident causes inevitably relies on a lot of expert experience and statistical analyses also require a lot of manual classification. Although, in recent years, with the development of natural language processing technology, there have been many efforts to automatically analyze and classify text. However, the existing methods either rely on large corpus and data preprocessing methods, which are cumbersome, or extract text information based on bidirectional encoder representation from transformers (BERT), but the computational cost is extremely high. These shortcomings make it still a great challenge to automatically analyze accident investigation reports and extract the information therein. To address the aforementioned problems, this study proposes a text-mining-based accident causal classification method based on a relational graph convolutional network (R-GCN) and pre-trained BERT. On the one hand, the proposed method avoids preprocessing such as stop word removal and word segmentation, which not only preserves the information of accident investigation reports to the greatest extent, but also avoids tedious operations. On the other hand, with the help of R-GCN to process the semantic features obtained by BERT representation, the dependence of BERT retraining on computing resources can be avoided.

Adapting the influences of publishers to perform news event detection

Online news outlets have the power to influence public policy issues. To understand the opinions of the people, many government departments check online news outlets to manually detect events that interest people. This process is time-consuming. To promptly respond to public expectations, this research proposes a framework for detecting news events that may interest government departments. This article proposes a method for finding event trigger words used to represent an event. The news media can be a critical participant in ‘agenda-setting’, which means that more widely discussed news is more attractive and critical than news that is less discussed. However, few studies have considered the influence of news media publishers from the ‘agenda setting’ perspective. Therefore, this study proposes an ‘agenda setting’-based filter to establish a high-impact news event detection model. The proposed framework identifies trigger words and utilises word embedding to find news event–related words. After that, an event detection model is designed to determine the events that are attractive to government departments. The experimental results show that purity increases from 0.666 when no extraction method is used to 0.809 when the extraction method in this study is used. The overall improvement trend shows significant improvement in event detection performance.

MenuNER: Domain-Adapted BERT Based NER Approach for a Domain with Limited Dataset and Its Application to Food Menu Domain

Entity-based information extraction is one of the main applications of Natural Language Processing (NLP). Recently, deep transfer-learning utilizing contextualized word embedding from pre-trained language models has shown remarkable results for many NLP tasks, including Named-entity recognition (NER). BERT (Bidirectional Encoder Representations from Transformers) is gaining prominent attention among various contextualized word embedding models as a state-of-the-art pre-trained language model. It is quite expensive to train a BERT model from scratch for a new application domain since it needs a huge dataset and enormous computing time. In this paper, we focus on menu entity extraction from online user reviews for the restaurant and propose a simple but effective approach for NER task on a new domain where a large dataset is rarely available or difficult to prepare, such as food menu domain, based on domain adaptation technique for word embedding and fine-tuning the popular NER task network model ‘Bi-LSTM+CRF’ with extended feature vectors. The proposed NER approach (named as ‘MenuNER’) consists of two step-processes: (1) Domain adaptation for target domain; further pre-training of the off-the-shelf BERT language model (BERT-base) in semi-supervised fashion on a domain-specific dataset, and (2) Supervised fine-tuning the popular Bi-LSTM+CRF network for downstream task with extended feature vectors obtained by concatenating word embedding from the domain-adapted pre-trained BERT model from the first step, character embedding and POS tag feature information. Experimental results on handcrafted food menu corpus from customers’ review dataset show that our proposed approach for domain-specific NER task, that is: food menu named-entity recognition, performs significantly better than the one based on the baseline off-the-shelf BERT-base model. The proposed approach achieves 92.5% F1 score on the YELP dataset for the MenuNER task.