A framework for evaluating OSH program effectiveness using leading and trailing metrics

Constructs of leading indicators: A synthesis of safety literature

INTRODUCTION

Leading indicators represent an invaluable tool that offer organizations the capability to: track health and safety performance, not just failures and accidents; measure effectiveness of safety efforts adopted; and focus on undesired precursors, rather than undesired occurred events. Despite these palpable advantages associated with their adoption, leading indicator's definition, application, and function are mostly ambiguous and inconsistent within literature. Therefore, this study systematically reviews pertinent literature to identify the constructs of leading indicators and generates guidance for leading indicator implementation (as a conceptual model).

METHOD

The overarching epistemological design adopted interpretivism and critical realism philosophical stances together with inductive reasoning to analyze 80 articles retrieved from the Scopus database, plus 13 more publications supplemented by the snowballing technique. Analysis of the safety discourse within literature (as secondary data) was undertaken in two stages, namely: (1) a cross-componential analysis identified the main features of leading indicators in comparison to lagging indicators; and (2) content analysis revealed prominent constructs of leading indicators.

RESULTS AND CONCLUSION

Analysis results identify that the definition, types, and development methods represent the main constructs for understanding the concept of leading indicators. The study identifies that ambiguity around the definition and function of leading indicators is due to the lack of differentiation of its types, namely passive leading indicators and active leading indicators.

PRACTICAL APPLICATION

As a practical contribution, the conceptual model, which introduces continuous learning through a perpetual loop of development and application of leading indicators, will help adopters create a knowledge repository of leading indicators and to continuously learn and improve their safety and safety performance. Specifically, the work clarifies their difference in terms of the timeframe passive leading indicators and active leading indicators take to measure different safety aspects, the functions they serve, the target they measure and their stage of development.

Establishment-level occupational safety analytics: Challenges and opportunities

In occupational safety and health, big data and analytics show promise for the prediction and prevention of workplace injuries. Advances in computing power and analytical methods have allowed companies to reveal insights from the "big" data that previously would have gone undetected. Despite the promise, occupational safety has lagged behind other industries, such as supply chain management and healthcare, in terms of exploiting the potential of analytics and much of the data collected by organizations goes unanalyzed. The purpose of the present paper is to argue for the broader application of establishment-level safety analytics. This is accomplished by defining the terms, describing previous research, outlining the necessary components required, and describing knowledge gaps and future directions. The knowledge gaps and future directions for research in establishment-level analytics are categorized into readiness for analytics, analytics methods, technology integration, data culture, and impact of analytics.

Evaluation of Occupational Health and Safety Management of Listed Companies in China’s Energy Industry Based on the Combined Weight-Cloud Model: From the Perspective of FPE Information Disclosure

Protecting labor safety and health and actively carrying out occupational safety and health management (OSHM) is a common need worldwide, and it is also one of the important efforts of Chinese enterprises under the background of promoting the implementation of the Healthy China strategy. Based on in-depth thinking on the current stage of OHSM, this study incorporated “management framework, management process, management effectiveness” (FPE) into an integrated framework and constructed an FPE evaluation system for enterprise OHSM. This study innovatively collected and refined FPE information from the perspective of information disclosure and used the combined weight cloud model to evaluate the occupational health and safety management level (OHSML) of 69 listed companies in China’s energy industry from 2009–2019. The results showed the following. (1) The OHSML of most listed companies in China’s energy industry was still at a low-end level. Among the companies that have issued relevant information reports, only 5.58% (S = 30) of the sample companies’ OHSML were at an acceptable level (Level IV) or declarable level (Level V). The OHSML comprehensive evaluation level of 92.56% (S = 498) of the sample companies was between the transitional level (Level III) and the improved level (Level II). (2) During 2009–2019, although the annual OHSML of listed companies in China’s energy industry showed an upward trend, the growth rate was low, and even the OHSML of some listed companies in the energy industry showed the characteristics of reduced fluctuations. (3) From the perspective of the PFT three-dimensional subsystem level of OHSM, the evaluation level of the governance framework subsystem was the highest, whereas the evaluation level of the management process subsystem and the management effectiveness subsystem were relatively low. Finally, according to the relevant results, some suggestions were proposed to improve the OHSML of listed companies in China’s energy industry. These findings can provide guidance for companies to improve their OSHM performance.

Reliability and validity of an employer-completed safety hazard and management assessment questionnaire

INTRODUCTION

Managing and improving occupational safety and health requires evaluating performance. Organizations are encouraged to use both lagging indicators (such as injury rates and costs) and leading indicators (such as questionnaire-assessed safety hazards and management practices) for this purpose, but the association between types of indicators over time can be complex. Longitudinal data can assist in clarifying these associations and increasing indicator utility.

METHOD

Employer data were used to evaluate the reliability and predictive validity of a safety management questionnaire. Employers' longitudinal questionnaire responses and workers' compensation (WC) claims data were analyzed using a marginal model with time-dependent covariates. Multivariable Poisson and linear regression analyses with claim rate and logarithmic cost, respectively, as dependent variables were carried out after adjusting for industry sector and size. Questionnaire data were used to evaluate questionnaire scaling properties and to assess generalizability of results.

RESULTS

One safety management scale was associated with a better WC outcome as predicted and two scales were unexpectedly associated with poorer WC claim outcomes. Analyses assisted in interpreting the latter results, suggesting that WC outcomes were a stimulus for change in some cases. Twelve hazards assessed on the questionnaire were associated with poorer WC claim outcomes as predicted.

CONCLUSIONS

This study extends leading indicator research using longitudinal questionnaire and WC claims data from employers. Analyses provided insight into associations between leading and lagging indicators, emphasizing the importance of both for safety improvement. Safety management questionnaire scales were predictive of WC claim outcomes, although support for hazard assessments as leading indicators was stronger.

PRACTICAL APPLICATIONS

This study supports the use of employer-completed hazard assessment questionnaires for targeting and prioritizing improvement efforts. Employer-completed safety management scales may be useful for directing improvement efforts, although the conditions under which they are completed, including submission to insurers, require additional consideration.

Advancing safety analytics: A diagnostic framework for assessing system readiness within occupational safety and health

Big data and analytics have shown promise in predicting safety incidents and identifying preventative measures directed towards specific risk variables. However, the safety industry is lagging in big data utilization due to various obstacles, which may include lack of data readiness (e.g., disparate databases, missing data, low validity) and personnel competencies. This paper provides a primer on the application of big data to safety. We then describe a safety analytics readiness assessment framework that highlights system requirements and the challenges that safety professionals may encounter in meeting these requirements. The proposed framework suggests that safety analytics readiness depends on (a) the quality of the data available, (b) organizational norms around data collection, scaling, and nomenclature, (c) foundational infrastructure, including technological platforms and skills required for data collection, storage, and analysis of health and safety metrics, and (d) measurement culture, or the emergent social patterns between employees, data acquisition, and analytic processes. A safety-analytics readiness assessment can assist organizations with understanding current capabilities so measurement systems can be matured to accommodate more advanced analytics for the ultimate purpose of improving decisions that mitigate injury and incidents.

Ergonomics indicators: A proposal for sustainable process performance measurement in ergonomics

Process performance measurement (PPM) has become a challenging task for organisations, which have many various processes, in today's globalised world. This challenging task is also remarkable in ergonomics, and combining sustainability with ergonomics can provide beneficial solutions for assessing risks and providing ergonomically favourable conditions for human well-being. In this paper, new ergonomics indicators (EIs) are proposed, classified, and described in detail to overcome this challenge through a systematic literature review that integrates ergonomics and sustainability. By considering sustainability indicators (SIs) selected from the review, the proposed EIs are presented within a three-dimensional structure. This structure enables measuring the performance on various issues in an organisation such as occupational health and safety, decent work, technology, working conditions, employee empowerment and development, charities, innovations, and recycling. Through systematic measurements using these indicators, more traceable and manageable processes can be achieved. Hence, ergonomists or managers can identify and prevent risky conditions practically. Practitioner Summary: This paper proposes ergonomics indicators and detailed descriptions for ensuring sustainable process performance measurement in ergonomics. The sustainability indicators presented as references for these proposals are utilised through selection and consolidation from literature. The ergonomics indicators are structured within a sustainability framework that considers six sustainability sub-dimensions proposed for ergonomics.

Leading Indicators—A Conceptual IoT-Based Framework to Produce Active Leading Indicators for Construction Safety

Active leading indicators (ALIs) have the potential to identify safety hazards and prompt immediate actions to prevent incidents. Currently, there is a major gap in research that incorporates a fully automated ALI system because implementation has been hindered by a lack of established industry thresholds of measurable performance that would trigger an actionable response. Therefore, this paper addresses this gap by presenting a new method that utilizes the Internet of Things (IoT) to collect quantifiable data which can trigger an actionable response in real time based on established thresholds. This novel method integrates the Construction Industry Institute (CII) active leading indicator framework with a prototype IoT-based system. Significantly, the ALI provides the physical–virtual feedback loop, which is an essential aspect of the IoT system because it provides real-time feedback to both the users and systems. This paper also identifies potential inputs to the ALI framework from emerging IoT-enabled systems. A case study was presented to initially validate the IoT-based ALI framework. Bluetooth-enabled heart rate monitors were issued to workers in a hazardous and critical mining construction site. The ALIs that were recorded included heart rate and body temperature. Thresholds were established that alerted the monitoring safety staff when a worker exhibited potentially unsafe conditions. The results of the study demonstrated the feasibility of the system. Additionally, other results included worker resistance; non-disclosing of medical conditions, and limitations for IoT connectivity.

Metrics to assess injury prevention programs for young workers in high-risk occupations: a scoping review of the literature

INTRODUCTION

Despite legal protections for young workers in Canada, youth aged 15-24 are at high risk of traumatic occupational injury. While many injury prevention initiatives targeting young workers exist, the challenge faced by youth advocates and employers is deciding what aspect(s) of prevention will be the most effective focus for their efforts. A review of the academic and grey literatures was undertaken to compile the metrics-both the indicators being evaluated and the methods of measurement-commonly used to assess injury prevention programs for young workers. Metrics are standards of measurement through which efficiency, performance, progress, or quality of a plan, process, or product can be assessed.

METHODS

A PICO framework was used to develop search terms. Medline, PubMed, OVID, EMBASE, CCOHS, PsychINFO, CINAHL, NIOSHTIC, Google Scholar and the grey literature were searched for articles in English, published between 1975-2015. Two independent reviewers screened the resulting list and categorized the metrics in three domains of injury prevention: Education, Environment and Enforcement.

RESULTS

Of 174 acquired articles meeting the inclusion criteria, 21 both described and assessed an intervention. Half were educational in nature (N=11). Commonly assessed metrics included: knowledge, perceptions, self-reported behaviours or intentions, hazardous exposures, injury claims, and injury counts. One study outlined a method for developing metrics to predict injury rates.

CONCLUSION

Metrics specific to the evaluation of young worker injury prevention programs are needed, as current metrics are insufficient to predict reduced injuries following program implementation. One study, which the review brought to light, could be an appropriate model for future research to develop valid leading metrics specific to young workers, and then apply these metrics to injury prevention programs for youth.

Workers' compensation insurer risk control systems: Opportunities for public health collaborations

INTRODUCTION

Workers' compensation (WC) insurers offer services and programs for prospective client selection and insured client risk control (RC) purposes. Toward these aims, insurers collect employer data that may include information on types of hazards present in the workplace, safety and health programs and controls in place to prevent injury/illness, and return-to-work programs to reduce injury/illness severity. Despite the potential impact of RC systems on workplace safety and health and the use of RC data in guiding prevention efforts, few research studies on the types of RC services provided to employers or the RC data collected have been published in the peer-reviewed literature.

METHODS

Researchers conducted voluntary interviews with nine private and state-fund WC insurers to collect qualitative information on RC data and systems.

RESULTS

Insurers provided information describing their RC data, tools, and practices. Unique practices as well as similarities including those related to RC services, policyholder goals, and databases were identified.

CONCLUSIONS

Insurers collect and store extensive RC data, which have utility for public health research for improving workplace safety and health.

PRACTICAL APPLICATIONS

Increased public health understanding of RC data and systems and an identification of key collaboration opportunities between insurers and researchers will facilitate increased use of RC data for public health purposes.

Association between safety leading indicators and safety climate levels

OBJECTIVES

The aim of this study was to evaluate the association of leading indicators for occupational health and safety, particularly safety inspections and non-compliances, with safety climate levels.

METHODS

Nordic Occupational Safety Climate Assessment Questionnaire was employed to evaluate safety climate in cross-sectional design. The geographically diverse population of the inspection body made it possible to conduct the survey across 10 provinces in Iran. 89 completed questionnaires were obtained with a response rate of 47%. Except for management safety justice, the internal consistency of other six dimensions was found to be acceptable (α≥0.7).

RESULTS

Mean scores of dimensions ranged from 3.50 in trust in the efficacy of safety systems (SD=0.38) to 2.98 in workers' safety priority and risk non-acceptance (SD=0.47). Tukey HSD tests indicated a statistically significant difference of mean scores among groups undergoing different number of safety inspections and those receiving different number of non-compliances (p<0.05), with no significant differences based on safety training man-hours and sessions (p>0.05). Spearman's rank-order correlation showed no relationship between work experience and number of non-compliances (correlation coefficient=-0.04, p>0.05) and between safety training man-hours and number of non-compliances (correlation coefficient=-0.15, p>0.05).

CONCLUSIONS

Our results indicate that safety climate levels are influenced by number of safety inspections and the resultant non-compliances.

PRACTICAL APPLICATIONS

Findings suggest that safety non-compliances detected as a result of conducting safety inspections could be used to monitor the safety climate state. Establishing plans to conduct scheduled safety inspections and recording findings in the form of safety non-compliance and monitoring their trend could be used to monitor levels of safety climate.

Developing leading indicators from OHS management audit data: Determining the measurement properties of audit data from the field

INTRODUCTION

OHS management audits are one means of obtaining data that may serve as leading indicators. The measurement properties of such data are therefore important. This study used data from Workwell audit program in Ontario, a Canadian province. The audit instrument consisted of 122 items related to 17 OHS management elements. The study sought answers regarding (a) the ability of audit-based scores to predict workers' compensation claims outcomes, (b) structural characteristics of the data in relation to the organization of the audit instrument, and (c) internal consistency of items within audit elements.

METHOD

The sample consisted of audit and claims data from 1240 unique firms that had completed one or two OHS management audits during 2007-2010. Predictors derived from the audit results were used in multivariable negative binomial regression modeling of workers' compensation claims outcomes. Confirmatory factor analyses were used to examine the instrument's structural characteristics. Kuder-Richardson coefficients of internal consistency were calculated for each audit element.

RESULTS

The ability of audit scores to predict subsequent claims data could not be established. Factor analysis supported the audit instrument's element-based structure. KR-20 values were high (≥0.83).

CONCLUSIONS

The Workwell audit data display structural validity and high internal consistency, but not, to date, construct validity, since the audit scores are generally not predictive of subsequent firm claim experience. Audit scores should not be treated as leading indicators of workplace OHS performance without supporting empirical data.

PRACTICAL APPLICATIONS

Analyses of the measurement properties of audit data can inform decisionmakers about the operation of an audit program, possible future directions in audit instrument development, and the appropriate use of audit data. In particular, decision-makers should be cautious in their use of audit scores as leading indicators, in the absence of supporting empirical data.

Self-audit of lockout/tagout in manufacturing workplaces: A pilot study

BACKGROUND

Occupational health and safety (OHS) self-auditing is a common practice in industrial workplaces. However, few audit instruments have been tested for inter-rater reliability and accuracy.

METHODS

A lockout/tagout (LOTO) self-audit checklist was developed for use in manufacturing enterprises. It was tested for inter-rater reliability and accuracy using responses of business self-auditors and external auditors.

RESULTS

Inter-rater reliability at ten businesses was excellent (κ = 0.84). Business self-auditors had high (100%) accuracy in identifying elements of LOTO practice that were present as well those that were absent (81% accuracy). Reliability and accuracy increased further when problematic checklist questions were removed from the analysis.

CONCLUSIONS

Results indicate that the LOTO self-audit checklist would be useful in manufacturing firms' efforts to assess and improve their LOTO programs. In addition, a reliable self-audit instrument removes the need for external auditors to visit worksites, thereby expanding capacity for outreach and intervention while minimizing costs.

Leading and lagging indicators of occupational health and safety: The moderating role of safety leadership

In response to the call for empirical evidence of a connection between leading and lagging indicators of occupational health and safety (OHS), the first aim of the current research is to consider the association between leading and lagging indicators of OHS. Our second aim is to investigate the moderating effect of safety leadership on the association between leading and lagging indicators. Data were collected from 3578 employees nested within 66 workplaces. Multi-level modelling was used to test the two hypotheses. The results confirm an association between leading and lagging indicators of OHS as well as the moderating impact of middle management safety leadership on the direct association. The association between leading and lagging indicators provides OHS practitioners with useful information to substantiate efforts within organisations to move away from a traditional focus on lagging indicators towards a preventative focus on leading indicators. The research also highlights the important role played by middle managers and the value of OHS leadership development and investment at the middle management level.