Evaluation of the HFACS-ADF safety classification system: inter-coder consensus and intra-coder consistency

Identification of critical causes of construction accidents in China using a hybrid HFACS-CN model

Construction safety is of significance since construction accidents can result in loss of property and large numbers of casualties. This research aims to identify the critical causes of construction accidents by introducing a hybrid approach. The hybrid approach is developed to identify the critical causes of construction accidents by combining the human factors analysis and classification system (HFACS) model with complex network (CN) theory. A total of 863 construction accident cases were collected, and 46 causal factors were identified. Subsequently, the accident causal network was established, and six critical causal factors were extracted. The hybrid analysis approach is demonstrated with a real construction accident case, and the results demonstrate that the hybrid approach could better identify the critical causal factors. Consequently, this research enables the enhancement of understanding the HFACS framework and CN theory, as well as a contribution to safety management in the construction industry at different levels.

An Impacting Factors Analysis of Unsafe Acts in Coal Mine Gas Explosion Accidents Based on HFACS-ISM-BN

With the development of intelligent coal mine construction, China’s coal production safety has been greatly improved, but coal mine gas explosion accidents still cannot be completely avoided and the unsafe acts of miners are an important cause of the accidents. Therefore, this study firstly collected 100 coal mine gas explosion cases in China, improved the framework of human factors analysis and classification system (HFACS) and used it to identify the causes of miners’ unsafe acts in detail. A hierarchy of the impacting factors is established. Then, combining with the interpretive structural model (ISM), the correlation between the impacting factors among different levels, especially among non-adjacent levels, is qualitatively analyzed through expert judgment. Then, the correlation among the contributing factors was quantitatively tested by chi-square test and odds ratio (OR) analysis. On this basis, a Bayesian network (BN) is constructed for the impacting factors of miners’ unsafe acts. The results show that the probability of coal mine gas explosion accident is 20% and 52%, respectively. Among the leading factors, the government’s insufficient crackdown on illegal activities had the greatest impact on miners’ violations, with a sensitive value of 13.2%. This study can provide reference for evaluating the unsafe acts of miners in coal mine gas explosion accidents by the probabilistic method.

Natural language processing for aviation safety: extracting knowledge from publicly-available loss of separation reports

BACKGROUND

The air traffic management (ATM) system has historically coped with a global increase in traffic demand ultimately leading to increased operational complexity. When dealing with the impact of this increasing complexity on system safety it is crucial to automatically analyse the losses of separation (LoSs) using tools able to extract meaningful and actionable information from safety reports. Current research in this field mainly exploits natural language processing (NLP) to categorise the reports,with the limitations that the considered categories need to be manually annotated by experts and that general taxonomies are seldom exploited.

METHODS

To address the current gaps,authors propose to perform exploratory data analysis on safety reports combining state-of-the-art techniques like topic modelling and clustering and then to develop an algorithm able to extract the Toolkit for ATM Occurrence Investigation (TOKAI) taxonomy factors from the free-text safety reports based on syntactic analysis. TOKAI is a tool for investigation developed by EUROCONTROL and its taxonomy is intended to become a standard and harmonised approach to future investigations.

RESULTS

Leveraging on the LoS events reported in the public databases of the Comisión de Estudio y Análisis de Notificaciones de Incidentes de Tránsito Aéreo and the United Kingdom Airprox Board,authors show how their proposal is able to automatically extract meaningful and actionable information from safety reports,other than to classify their content according to the TOKAI taxonomy. The quality of the approach is also indirectly validated by checking the connection between the identified factors and the main contributor of the incidents.

CONCLUSIONS

Authors' results are a promising first step toward the full automation of a general analysis of LoS reports supported by results on real-world data coming from two different sources. In the future,authors' proposal could be extended to other taxonomies or tailored to identify factors to be included in the safety taxonomies.

A systematic approach to public health - Novel application of the human factors analysis and classification system to public health and COVID-19

In this article, we argue for a novel adaptation of the Human Factors Analysis and Classification System (HFACS) to proactive incidence prevention in the public health and in particular, during and in response to COVID-19. HFACS is a framework of causal categories of human errors typically applied for systematic retrospective incident analysis in high-risk domains. By leveraging this approach proactively, appropriate, and targeted measures can be quickly identified and established to mitigate potential errors at different levels within the public health system (from tertiary and secondary healthcare workers to primary public health officials, regulators, and policymakers).

The development and validation of a human factors analysis and classification system for the construction industry

Human factors significantly contribute to accidents and vary with the industries in which they exist. However, there are few analytical methods for human factors in the construction industry. Based on the prevalent human factor analysis and classification system (HFACS), the present study proposes a HFACS for the construction industry (HFACS-CI). Compared with the HFACS, the HFACS-CI develops Level 5 with classifications including 'the attitude of owner' and 'the regulation of engineering firm', and adds classifications, i.e., 'management for change' and 'management for subcontractors', to Level 4. Its validation is verified by application to the 2016 platform collapse in Fengcheng, Jiangxi, China. Finally, utilizing the χ² test and Apriori algorithm to explore the causalities among the classifications of the HFACS-CI, 'the attitude of owner', 'the regulation of engineering firm' and 'organizational climate' are identified as the human factors that may create conditions for the occurrence of other human factors.

Evaluation of construct and criterion-referenced validity of a systems-thinking based near miss reporting form

The validity of methods is an ongoing issue in ergonomics. Inconsistent definitions and approaches to evaluation exacerbate this challenge. In this study, the construct and criterion-referenced validity of a new near miss reporting form was evaluated to determine the extent to which it comprehensively captures near miss incidents and is aligned with the systems thinking approach to accident causation. Interview data were used as the reference standard in the evaluation. Using signal detection theory (SDT), a high average hit rate (HR), predictive value (PV) and sensitivity index (SI) were found, with an almost perfect ranking for the index of concordance. The findings show that the reporting form has strong construct and criterion-referenced validity. It is proposed that the approach used in this study could be used by researchers and practitioners when testing the validity of incident data collection tools. Practitioner summary: The validity of methods is a key issue in ergonomics. In this study, we test the validity of a near miss reporting form using interview data as a standard. This approach could be used by practitioners when testing the validity of other ergonomics methods.

A taxonomy of interactions in socio-technical systems: A functional perspective

Although the modelling of interactions has long been at the core of socio-technical systems theory, and is a key for understanding resilience, there is a lack of a holistic taxonomy of interactions. This study introduces a taxonomy of interactions to be used in association with the Functional Resonance Analysis Method (FRAM). The taxonomy has nine criteria: nature of agents, output nature, levelling, waiting time, distance, degree of coupling, visibility, safety and/or security hazards, and parallel replications. For each criterion, two descriptors are proposed: what the interaction looks like; and - when applicable - the variability level of the interaction. The use of the taxonomy is presented for three systems with clearly distinct complexity characteristics: cash withdrawal from an ATM, teaching a university course, and manufacturing operations. These case studies indicate the usefulness of the taxonomy for the identification of leverage points in work system design. They also show the value of modelling the variability of the interactions in FRAM models, in addition to the traditional modelling of the variability of the outputs of functions. Implications of the taxonomy for resilience engineering are discussed.

Incorporating Human Factors Analysis and Classification System (HFACS) Into Analysis of Reported Near Misses and Incidents in Radiation Oncology

PURPOSE

Human factors analysis and classification system (HFACS) is a framework for investigation into causation of human errors. We herein assess whether radiation oncology professionals, with brief training, can conduct HFACS on reported near misses or safety incidents (NMSIs) in a reliable (eg, with a high level of agreement) and practical (eg, timely and with user satisfaction) manner.

METHODS AND MATERIALS

We adapted a classical HFACS framework by selecting and modifying main headings, subheadings, and nano-codes that were most likely to apply to radiation oncology settings. The final modified HFACS included 3 main headings, 8 subheadings, and 20 nano-codes. The modified HFACS was first tested in a simulated trial on 8 NMSI and was analyzed by 5 to 10 radiation oncology professionals, with 2 endpoints: (1) agreement among participants at the main-heading, subheading, and nano-code level, and (2) time to complete the analysis. We then performed a prospective trial integrating this approach into a weekly NMSI review meeting, with 10 NMSIs analyzed by 8 to 13 radiation oncology professionals with the same endpoints, while also collecting survey data on participants' satisfaction.

RESULTS

In the simulated trial, agreement among participants was 85% on the main headings, 73% on the subheadings, and 70% on the nano-codes. Participants needed, on average, 16.4 minutes (standard deviation, 5.7 minutes) to complete an analysis. In the prospective trial, agreement between participants was 81% on the main headings, 75% on the subheadings, and 74% on the nano-codes. Participants needed, on average, 8.3 minutes (standard deviation, 4.7 minutes) to complete an analysis. The average satisfaction with the proposed HFACS approach was 3.9 (standard deviation 1.0) on a scale from 1 to 5.

CONCLUSIONS

This study demonstrates that, after relatively brief training, radiation oncology professionals were able to perform HFACS analysis in a reliable and timely manner and with a relatively high level of satisfaction.

Developing a contributing factor classification scheme for Rasmussen's AcciMap: Reliability and validity evaluation

One factor potentially limiting the uptake of Rasmussen's (1997) Accimap method by practitioners is the lack of a contributing factor classification scheme to guide accident analyses. This article evaluates the intra- and inter-rater reliability and criterion-referenced validity of a classification scheme developed to support the use of Accimap by led outdoor activity (LOA) practitioners. The classification scheme has two levels: the system level describes the actors, artefacts and activity context in terms of 14 codes; the descriptor level breaks the system level codes down into 107 specific contributing factors. The study involved 11 LOA practitioners using the scheme on two separate occasions to code a pre-determined list of contributing factors identified from four incident reports. Criterion-referenced validity was assessed by comparing the codes selected by LOA practitioners to those selected by the method creators. Mean intra-rater reliability scores at the system (M = 83.6%) and descriptor (M = 74%) levels were acceptable. Mean inter-rater reliability scores were not consistently acceptable for both coding attempts at the system level (M_T1 = 68.8%; M_T2 = 73.9%), and were poor at the descriptor level (M_T1 = 58.5%; M_T2 = 64.1%). Mean criterion referenced validity scores at the system level were acceptable (M_T1 = 73.9%; M_T2 = 75.3%). However, they were not consistently acceptable at the descriptor level (M_T1 = 67.6%; M_T2 = 70.8%). Overall, the results indicate that the classification scheme does not currently satisfy reliability and validity requirements, and that further work is required. The implications for the design and development of contributing factors classification schemes are discussed.

Application of classification principles to improve the reliability of incident classification systems: A test case using HFACS-ADF

Accident classification systems are important tools for safety management. Unfortunately, many of the tools available have demonstrated poor reliability of coding, making their validity and usefulness questionable. This paper demonstrates the application of four strategies to improve the reliability of accident and incident classification systems. The strategies include creating a domain-specific system with limitations on system size and careful selection of codes, specifically the reduction of abstract concepts and bias-causing terminology. Using HFACS-ADF as a test case, the system was adapted using the strategies and validated using comprehension and comprehensiveness testing. The new system was then assessed for reliability. The reliability of the system increased by at least 20% at all levels of the classification system following the changes made. The results provide evidence that the application of theoretically and empirically-derived classification principles are effective for improving the reliability of accident and incident classification systems in high hazard industries.

Rasmussen's legacy in the great outdoors: A new incident reporting and learning system for led outdoor activities

Jens Rasmussen's seminal risk management framework and accompanying Accimap method have become highly popular in safety science circles. Despite this, widespread adoption of the model and method in practice has not yet been achieved. This paper describes a project involving the development and implementation of an incident reporting and learning system underpinned by Rasmussen's risk management framework and Accimap method. The system was developed for the led outdoor activity sector in Australia to enable reporting and analysis of injuries and near miss incidents, with the aim of supporting the development of more effective countermeasures. An analysis of the data derived from the first 3 months use of the system by 43 organisations is presented. The outputs provide an in-depth Accimap-based analysis of all incidents reported by participating organisations over the 3 month period. In closing, the importance of developing usable domain specific tools to support translation of Ergonomics theory and methods in practice is discussed.

Application of Human Factors Analysis and Classification System (HFACS) to UK rail safety of the line incidents

Minor safety incidents on the railway cause disruption, and may be indicators of more serious safety risks. The following paper aimed to gain an understanding of the relationship between active and latent factors, and particular causal paths for these types of incidents by using the Human Factors Analysis and Classification System (HFACS) to examine rail industry incident reports investigating such events. 78 reports across 5 types of incident were reviewed by two authors and cross-referenced for interrater reliability using the index of concordance. The results indicate that the reports were strongly focused on active failures, particularly those associated with work-related distraction and environmental factors. Few latent factors were presented in the reports. Different causal pathways emerged for memory failures for events such a failure to call at stations, and attentional failures which were more often associated with signals passed at danger. The study highlights a need for the rail industry to look more closely at latent factors at the supervisory and organisational levels when investigating minor safety of the line incidents. The results also strongly suggest the importance of a new factor - operational environment - that captures unexpected and non-routine operating conditions which have a risk of distracting the driver. Finally, the study provides further demonstration of the utility of HFACS to the rail industry, and of the usefulness of the index of concordance measure of interrater reliability.

Development of safety incident coding systems through improving coding reliability

This paper reviews classification theory sources to develop five research questions concerning factors associated with incident coding system development and use and how these factors affect coding reliability. Firstly, a method was developed to enable the comparison of reliability results obtained using different methods. Second, a statistical and qualitative review of reliability studies was conducted to investigate the influence of the identified factors on the reliability of incident coding systems. As a result several factors were found to have a statistically significant effect on reliability. Four recommendations for system development and use are provided to assist researchers in improving the reliability of incident coding systems in high hazard industries.

Identifying behaviour patterns of construction safety using system archetypes

Construction safety management involves complex issues (e.g., different trades, multi-organizational project structure, constantly changing work environment, and transient workforce). Systems thinking is widely considered as an effective approach to understanding and managing the complexity. This paper aims to better understand dynamic complexity of construction safety management by exploring archetypes of construction safety. To achieve this, this paper adopted the ground theory method (GTM) and 22 interviews were conducted with participants in various positions (government safety inspector, client, health and safety manager, safety consultant, safety auditor, and safety researcher). Eight archetypes were emerged from the collected data: (1) safety regulations, (2) incentive programs, (3) procurement and safety, (4) safety management in small businesses (5) production and safety, (6) workers' conflicting goals, (7) blame on workers, and (8) reactive and proactive learning. These archetypes capture the interactions between a wide range of factors within various hierarchical levels and subsystems. As a free-standing tool, they advance the understanding of dynamic complexity of construction safety management and provide systemic insights into dealing with the complexity. They also can facilitate system dynamics modelling of construction safety process.

Surgical never events and contributing human factors

INTRODUCTION

We report the first prospective analysis of human factors elements contributing to invasive procedural never events by using a validated Human Factors Analysis and Classification System (HFACS).

METHODS

From August 2009 to August 2014, operative and invasive procedural "Never Events" (retained foreign object, wrong site/side procedure, wrong implant, wrong procedure) underwent systematic causation analysis promptly after the event. Contributing human factors were categorized using the 4 levels of error causation described by Reason and 161 HFACS subcategories (nano-codes).

RESULTS

During the study, approximately 1.5 million procedures were performed, during which 69 never events were identified. A total of 628 contributing human factors nano-codes were identified. Action-based errors (n = 260) and preconditions to actions (n = 296) accounted for the majority of the nano-codes across all 4 types of events, with individual cognitive factors contributing one half of the nano-codes. The most common action nano-codes were confirmation bias (n = 36) and failed to understand (n = 36). The most common precondition nano-codes were channeled attention on a single issue (n = 33) and inadequate communication (n = 30).

CONCLUSION

Targeting quality and interventions in system improvement addressing cognitive factors and team resource management as well as perceptual biases may decrease errors and further improve patient safety. These results delineate targets to further decrease never events from our health care system.

Automated UMLS-based comparison of medical forms

Medical forms are very heterogeneous: on a European scale there are thousands of data items in several hundred different systems. To enable data exchange for clinical care and research purposes there is a need to develop interoperable documentation systems with harmonized forms for data capture. A prerequisite in this harmonization process is comparison of forms. So far – to our knowledge – an automated method for comparison of medical forms is not available. A form contains a list of data items with corresponding medical concepts. An automatic comparison needs data types, item names and especially item with these unique concept codes from medical terminologies. The scope of the proposed method is a comparison of these items by comparing their concept codes (coded in UMLS). Each data item is represented by item name, concept code and value domain. Two items are called identical, if item name, concept code and value domain are the same. Two items are called matching, if only concept code and value domain are the same. Two items are called similar, if their concept codes are the same, but the value domains are different. Based on these definitions an open-source implementation for automated comparison of medical forms in ODM format with UMLS-based semantic annotations was developed. It is available as package compareODM from http://cran.r-project.org. To evaluate this method, it was applied to a set of 7 real medical forms with 285 data items from a large public ODM repository with forms for different medical purposes (research, quality management, routine care). Comparison results were visualized with grid images and dendrograms. Automated comparison of semantically annotated medical forms is feasible. Dendrograms allow a view on clustered similar forms. The approach is scalable for a large set of real medical forms.

Validating the Danish adaptation of the World Health Organization's International Classification for Patient Safety classification of patient safety incident types

OBJECTIVE

Validation of a Danish patient safety incident classification adapted from the World Health Organizaton's International Classification for Patient Safety (ICPS-WHO).

DESIGN

Thirty-three hospital safety management experts classified 58 safety incident cases selected to represent all types and subtypes of the Danish adaptation of the ICPS (ICPS-DK).

OUTCOME MEASURES

Two measures of inter-rater agreement: kappa and intra-class correlation (ICC).

RESULTS

An average number of incident types used per case per rater was 2.5. The mean ICC was 0.521 (range: 0.199-0.809) and the mean kappa was 0.513 (range: 0.193-0.804). Kappa and ICC showed high correlation (r = 0.99). An inverse correlation was found between the prevalence of type and inter-rater reliability. Results are discussed according to four factors known to determine the inter-rater agreement: skill and motivation of raters; clarity of case descriptions; clarity of the operational definitions of the types and the instructions guiding the coding process; adequacy of the underlying classification scheme.

CONCLUSIONS

The incident types of the ICPS-DK are adequate, exhaustive and well suited for classifying and structuring incident reports. With a mean kappa a little above 0.5 the inter-rater agreement of the classification system is considered 'fair' to 'good'. The wide variation in the inter-rater reliability and low reliability and poor discrimination among the highly prevalent incident types suggest that for these types, precisely defined incident sub-types may be preferred. This evaluation of the reliability and usability of WHO's ICPS should be useful for healthcare administrations that consider or are in the process of adapting the ICPS.

Reliability studies of incident coding systems in high hazard industries: A narrative review of study methodology

This paper reviews the current literature on incident coding system reliability and discusses the methods applied in the conduct and measurement of reliability. The search strategy targeted three electronic databases using a list of search terms and the results were examined for relevance, including any additional relevant articles from the bibliographies. Twenty five papers met the relevance criteria and their methods are discussed. Disagreements in the selection of methods between reliability researchers are highlighted as are the effects of method selection on the outcome of the trials. The review provides evidence that the meaningfulness of and confidence in results is directly affected by the methodologies employed by the researcher during the preparation, conduct and analysis of the reliability study. Furthermore, the review highlights the heterogeneity of methodologies employed by researchers measuring reliability of incident coding techniques, reducing the ability to critically compare and appraise techniques being considered for the adoption of report coding and trend analysis by client organisations. It is recommended that future research focuses on the standardisation of reliability research and measurement within the incident coding domain.

Predictive models of safety based on audit findings: Part 1: Model development and reliability

This consecutive study was aimed at the quantitative validation of safety audit tools as predictors of safety performance, as we were unable to find prior studies that tested audit validity against safety outcomes. An aviation maintenance domain was chosen for this work as both audits and safety outcomes are currently prescribed and regulated. In Part 1, we developed a Human Factors/Ergonomics classification framework based on HFACS model (Shappell and Wiegmann, 2001a,b), for the human errors detected by audits, because merely counting audit findings did not predict future safety. The framework was tested for measurement reliability using four participants, two of whom classified errors on 1238 audit reports. Kappa values leveled out after about 200 audits at between 0.5 and 0.8 for different tiers of errors categories. This showed sufficient reliability to proceed with prediction validity testing in Part 2.

A systems approach to accident causation in mining: an application of the HFACS method

This project aimed to provide a greater understanding of the systemic factors involved in mining accidents, and to examine those organisational and supervisory failures that are predictive of sub-standard performance at operator level. A sample of 263 significant mining incidents in Australia across 2007-2008 were analysed using the Human Factors Analysis and Classification System (HFACS). Two human factors specialists independently undertook the analysis. Incidents occurred more frequently in operations concerning the use of surface mobile equipment (38%) and working at heights (21%), however injury was more frequently associated with electrical operations and vehicles and machinery. Several HFACS categories appeared frequently: skill-based errors (64%) and violations (57%), issues with the physical environment (56%), and organisational processes (65%). Focussing on the overall system, several factors were found to predict the presence of failures in other parts of the system, including planned inappropriate operations and team resource management; inadequate supervision and team resource management; and organisational climate and inadequate supervision. It is recommended that these associations deserve greater attention in future attempts to develop accident countermeasures, although other significant associations should not be ignored. In accordance with findings from previous HFACS-based analyses of aviation and medical incidents, efforts to reduce the frequency of unsafe acts or operations should be directed to a few critical HFACS categories at the higher levels: organisational climate, planned inadequate operations, and inadequate supervision. While remedial strategies are proposed it is important that future efforts evaluate the utility of the measures proposed in studies of system safety.

Investigations of Human and Organizational Factors in hazardous vapor accidents

This paper presents a model to assess the contribution of Human and Organizational Factor (HOF) to accidents. The proposed model is made up of two phases. The first phase is the qualitative analysis of HOF responsible for accidents, which utilizes Human Factors Analysis and Classification System (HFACS) to seek out latent HOFs. The hierarchy of HOFs identified in the first phase provides inputs for the analysis in the second phase, which is a quantitative analysis using Bayesian Network (BN). BN enhances the ability of HFACS by allowing investigators or domain experts to measure the degree of relationships among the HOFs. In order to estimate the conditional probabilities of BN, fuzzy analytical hierarchy process and decomposition method are applied in the model. Case studies show that the model is capable of seeking out critical latent human and organizational errors and carrying out quantitative analysis of accidents. Thereafter, corresponding safety prevention measures are derived.

Extending hierarchical task analysis to identify cognitive demands and information design requirements

While hierarchical task analysis (HTA) is well established as a general task analysis method, there appears a need to make more explicit both the cognitive elements of a task and design requirements that arise from an analysis. One way of achieving this is to make use of extensions to the standard HTA. The aim of the current study is to evaluate the use of two such extensions--the sub-goal template (SGT) and the skills-rules-knowledge (SRK) framework--to analyse the cognitive activity that takes place during the planning and delivery of anaesthesia. In quantitative terms, the two methods were found to have relatively poor inter-rater reliability; however, qualitative evidence suggests that the two methods were nevertheless of value in generating insights about anaesthetists' information handling and cognitive performance. Implications for the use of an extended HTA to analyse work systems are discussed.

Accident investigation reporting deficiencies related to organizational factors in machinery space fires and explosions

Careful accident investigation provides opportunities to review safety arrangements in socio-technical systems. There is consensus that human intervention is involved in the majority of accidents. Ever cautious of the consequences attributed to such a claim vis-à-vis the apportionment of blame, several authors have highlighted the importance of investigating organizational factors in this respect. Specific regulations to limit what were perceived as unsuitable organizational influences in shipping operations were adopted by the International Maritime Organization (IMO). Guidance is provided for the investigation of human and organizational factors involved in maritime accidents. This paper presents a review of 41 accident investigation reports related to machinery space fires and explosions. The objective was to find out if organizational factors are identified during maritime accident investigations. An adapted version of the Human Factor Analysis and Classification System (HFACS) with minor modifications related to machinery space features was used for this review. The results of the review show that organizational factors were not identified by maritime accident investigators to the extent expected had the IMO guidelines been observed. Instead, contributing factors at the lower end of organizational echelons are over-represented.