Desktop study of occupational exposure judgments: do education and experience influence accuracy?

Micro-environmental factors impact breathing zone exposures: A simulated petrochemical manufacturing facility task

This study investigates the impact of micro-environmental factors on worker breathing zone exposure levels in petrochemical facilities. A laboratory simulation study evaluated near-field exposure to methane for a typical maintenance task. Individual and combinations of micro-environmental factors significantly affected methane exposure. Airflow direction and speed were significant determinants of exposure concentration reduction. A side airflow direction at medium to high speed produced the lowest gas concentration in the breathing zone. Worker body orientation relative to the methane emission point was also a critical factor affecting gas concentration in the worker's breathing zone. The study provides insights into how variations in airflow and small changes in position impact near-field exposures for petrochemical tasks, guiding industrial hygiene professionals' training on qualitative exposure estimation and providing input for near-field exposure modeling to guide quantitative exposure and risk assessment.

Accuracy of professional judgments for dermal exposure assessment using deterministic models

The accuracy of exposure judgments, particularly for scenarios where only qualitative information is available or a systematic approach is not used, has been evaluated and shown to have a relatively low level of accuracy. This is particularly true for dermal exposures, where less information is generally available compared to inhalation exposures. Relatively few quantitative validation efforts have been performed for scenarios where dermal exposures are of interest. In this study, a series of dermal exposure judgments were collected from 90 volunteer U.S. occupational health practitioners in a workshop format to assess the accuracy of their judgments for three specific scenarios. Accuracy was defined as the ability of the participants to identify the correct reference exposure category, as defined by the quantitative exposure banding categories utilized by the American Industrial Hygiene Association (AIHA^®). The participants received progressively additional information and training regarding dermal exposure assessments and scenario-specific information during the workshop, and the relative accuracy of their category judgments over time was compared. The results of the study indicated that despite substantial education and training in exposure assessment generally, the practitioners had very little experience in performing dermal exposure assessments and a low level of comfort in performing these assessments. Further, contrary to studies of practitioners performing inhalation exposure assessments demonstrating a trend toward underestimating exposures, participants in this study consistently overestimated the potential for dermal exposure without quantitative data specific to the scenario of interest. Finally, it was found that participants were able to identify the reference or "true" category of dermal exposure acceptability when provided with relevant, scenario-specific dermal and/or surface-loading data for use in the assessment process. These results support the need for additional training and education of practitioners in performing dermal exposure assessments. A closer analysis of default loading values used in dermal exposure assessments to evaluate their accuracy relative to real-world or measured dermal loading values, along with consistent improvements in current dermal models, is also needed.

Use of expert elicitation in the field of occupational hygiene: Comparison of expert and observed data distributions

The concept of professional judgement underpins the way in which an occupational hygienist assesses an exposure problem. Despite the importance placed on professional judgement in the discipline, a method of assessment to characterise accuracy has not been available. In this paper, we assess the professional judgement of four occupational hygienists (‘experts’) when completing exposure assessments on a range of airborne contaminants across a number of job roles within a surface mining environment in the Pilbara region of Western Australia. The job roles assessed were project driller, mobile equipment operator, fixed plant maintainer, and drill and blast operator. The contaminants of interest were respirable crystalline silica, respirable dust, and inhalable dust. The novel approach of eliciting exposure estimates focusing on contaminant concentration and attribution of an exposure standard estimate was used. The majority of the elicited values were highly skewed; therefore, a scaled Beta distribution were fitted. These elicited fitted distributions were then compared to measured data distributions, the results of which had been collected as part of an occupational hygiene program assessing full-shift exposures to the same contaminants and job roles assessed by the experts. Our findings suggest that the participating experts within this study tended to overestimate exposures. In addition, the participating experts were more accurate at estimating percentage of an exposure standard than contaminant concentration. We demonstrate that this elicitation approach and the encoding methodology contained within can be applied to assess accuracy of exposure judgements which will impact on worker protection and occupational health outcomes.

The design of a matrix linking work situations to chemical health risk at the workplace

In France, laws require each company to draw up an inventory of the risks that may threaten employees' health in order to prioritize the preventive actions to be implemented. Focusing on chemical risk, databases on hazards or exposures are widely available but they lack information regarding chemical risks resulting from combining the hazards of chemicals with their conditions of use, thus generating exposures. Our objective is to build a matrix of French work situations associated with their chemical risk. Eighty-eight work situations were collected from reports written by professionals from the French public health insurance service. Each work situation is defined by descriptive parameters of the task, the exposure, and the hazard. According to an expert elicitation method (Delphi, n = 21 experts), each work situation was assessed and a chemical risk score defined, taking into account all the descriptive exposure and hazard parameters. Chemical risk scores were expressed as a range of values from 0 to 100, with the size of the range chosen by the experts themselves according to their uncertainty. The experts' assessments were merged to assign one risk score for each work situation, variability, and confidence. The results showed that 50% of the work situations had a risk score between 40 and 60. The average variability and confidence were around 15% and 82%, respectively. This work situation matrix constructed from French data can be used by occupational safety and health managers that have similar work situations in their company (Western European industrial sector). In this context, it may be useful to easily determine the level of risks for similar tasks and prioritize those that are most risky. Moreover, it could be used to compare and define the differences between a risk assessment performed by "expertise" and another defined by a software.

Estimation of Lead Exposure Prevalence in Korean Population through Combining Multiple Experts' Judgment based on Objective Data Sources

Objective

Estimating carcinogen exposure prevalence is important for preventing occupational cancers. To develop the Korean version of CARcinogen EXposure (CAREX), a carcinogen surveillance system used in many countries, we estimated lead exposure prevalence in the Korean working population.

Methods

We used three Korean nationwide data sources to obtain objective database-derived prevalences of lead exposure across industries: airborne lead measurement data from the work environment measurement database (WEMD), blood lead measurement data from the special health examination database (SHED), and lead exposure prevalence computed using data from the work environment condition survey (WECS), which is a nationwide occupational exposure survey. We also asked a panel of 52 experts with ≥20 years of experience in industrial hygiene practice for their judgment about lead exposure prevalence across industries after they reviewed the database-derived prevalences computed from the three exposure databases. We developed and compared various estimation methods for combining the experts' judgments. The 2010 census was used as the reference population to estimate the number of lead-exposed workers in 228 industries by multiplying the exposure prevalence by the number of workers in each industry.

Results

The database-derived prevalences of lead exposure in the 228 industries were calculated using data collected between 2009 and 2011 from the WEMD and SHED and from the 2009 WECS. From the various estimation methods assessed, the median values of experts' responses were selected as our estimates of lead exposure prevalence in each industry. As a result, it was estimated that 129,250 Korean workers were exposed to lead in 2010.

Conclusions

Based on objective databases, we developed a method for estimating exposure prevalence for the CAREX system by combining experts' judgments. This work may offer an unbiased approach to the development process that accounts for the uncertainty in exposure.

Combining Lead Exposure Measurements and Experts' Judgment Through a Bayesian Framework

Objectives

CARcinogen EXposure (CAREX) is a carcinogen-surveillance system employed in many countries. To develop Korean CAREX, the intensity of exposure to lead, as an example, was estimated across industries.

Methods

Airborne-lead measurement records were extracted from the work-environment measurement database (WEMD), which is a nationwide workplace-monitoring database. Lead measurements were log-transformed; then, the log-transformed geometric means (LGMs) and log-transformed geometric standard deviations (LGSDs) were calculated for each industry. However, the data of many industries was limited. To address this shortcoming, experts' judgments of the lead exposure levels across industries were elicited. Experts provided their estimates of lead exposure levels as the boundary of the 5th and 95th percentiles, and it is assumed that these estimates are based on the log-normal distributions of exposure levels. Estimates of LGM and LGSD were extracted from each expert's response and then combined to quantify the experts' prior distribution. Then, the experts' prior distributions for each industry were updated with the corresponding LGMs and LGSDs calculated from the WEMD data through a Bayesian framework, yielding posterior distributions of the LGM and LGSD.

Results

The WEMD contains 83035 airborne-lead measurements that were collected between 2002 and 2007. A total of 17 occupational-hygiene professionals with >20 years of experience provided lead exposure estimates. In industries where measurement data were abundant, the measurement data dominated the posterior exposure estimates. For example, for one industry, 'Manufacture of Accumulator, Primary Cells, and Primary Batteries,' 1152 lead measurements [with a geometric mean (GM) of 14.42 µg m-3 and a geometric standard deviation (GSD) of 3.31] were available and 15 experts' responses (with a GM of 7.06 µg m-3 and a GSD of 4.15) were collected, resulting in a posterior exposure estimate of 14.41µg m-3 as the GM with a GSD of 3.31. For industries with a limited number of measurements available in the WEMD, experts' decisions played a significant role in determining the posterior exposure estimates. For example, for the 'Manufacture of Weapons and Ammunition' industry, 15 lead measurements (with a GM of 6.45 µg m-3 and a GSD of 3.37) were available and seven experts' responses (with a GM of 3.28 µg m-3 and a GSD of 4.54) were obtained, resulting in a posterior exposure estimate of 5.42 µg m-3 as the GM with a GSD of 3.73.

Conclusions

The proposed method for estimating the intensity of exposure to carcinogens may introduce an unbiased approach to the development process by simultaneously utilizing both prior knowledge of experts and measurement data. In addition, it supplies a framework for future updates.

Estimating diesel fuel exposure for a plumber repairing an underground pipe

We estimated the diesel fuel exposure of a plumber repairing an underground water line leak at a truck stop. The repair work was performed over three days during which the plumber spent most of his time in a pit filled with a mixture of water and diesel fuel. Thus, the plumber was exposed via both the inhalation and dermal routes. While previously asymptomatic, he was diagnosed with acute renal failure 35 days after working at this site. No measurements were available for estimating either inhalation or dermal exposures or the cumulative dose and, therefore, two different approaches were used that were based on simple models of the exposure scenario. The first approach used the ideal gas law with the vapor pressure of the diesel fuel mixture to estimate a saturation vapor concentration, while the second one used a mass balance of the petroleum hydrocarbon component of diesel fuel in conjunction with the Henry's Law constant for this mixture. These inhalation exposure estimates were then adjusted to account for the limited ventilation in a confined space. The inhalation exposure concentrations predicted when handling the water layer alone is much lower than that expected from the organic layer. This case study illustrates the large differences in inhalation exposure associated with volatile organic layers and aqueous solution containing these chemicals. The estimate of dermal exposure was negligible compared to the inhalation exposure because the skin presents a much smaller surface area of exposure to the contaminant compared to the lungs. The methodology presented here is useful for situations where little information is available for more formal mathematical exposure modeling, but where adjustments to the worst-case exposures, estimated simply, can provide reasonable exposure estimates.

Wishful Thinking? Inside the Black Box of Exposure Assessment

BACKGROUND

Decision-making processes used by experts when undertaking occupational exposure assessment are relatively unknown, but it is often assumed that there is a common underlying method that experts employ. However, differences in training and experience of assessors make it unlikely that one general method for expert assessment would exist. Therefore, there are concerns about formalizing, validating, and comparing expert estimates within and between studies that are difficult, if not impossible, to characterize. Heuristics on the other hand (the processes involved in decision making) have been extensively studied. Heuristics are deployed by everyone as short-cuts to make the often complex process of decision-making simpler, quicker, and less burdensome. Experts' assessments are often subject to various simplifying heuristics as a way to reach a decision in the absence of sufficient data. Therefore, investigating the underlying heuristics or decision-making processes involved may help to shed light on the 'black box' of exposure assessment.

METHODS

A mixed method study was conducted utilizing both a web-based exposure assessment exercise incorporating quantitative and semiqualitative elements of data collection, and qualitative semi-structured interviews with exposure assessors. Qualitative data were analyzed using thematic analysis.

RESULTS

Twenty-five experts completed the web-based exposure assessment exercise and 8 of these 25 were randomly selected to participate in the follow-up interview. Familiar key themes relating to the exposure assessment exercise emerged; 'intensity'; 'probability'; 'agent'; 'process'; and 'duration' of exposure. However, an important aspect of the detailed follow-up interviews revealed a lack of structure and order with which participants described their decision making. Participants mostly described some form of an iterative process, heavily relying on the anchoring and adjustment heuristic, which differed between experts.

CONCLUSION

In spite of having undertaken comparable training (in occupational hygiene or exposure assessment), experts use different methods to assess exposure. Decision making appears to be an iterative process with heavy reliance on the key heuristic of anchoring and adjustment. Using multiple experts to assess exposure while providing some form of anchoring scenario to build from, and additional training in understanding the impact of simple heuristics on the process of decision making, is likely to produce a more methodical approach to assessment; thereby improving consistency and transparency in expert exposure assessment.

Exposure Estimation and Interpretation of Occupational Risk: Enhanced Information for the Occupational Risk Manager

The fundamental goal of this article is to describe, define, and analyze the components of the risk characterization process for occupational exposures. Current methods are described for the probabilistic characterization of exposure, including newer techniques that have increasing applications for assessing data from occupational exposure scenarios. In addition, since the probability of health effects reflects variability in the exposure estimate as well as the dose-response curve-the integrated considerations of variability surrounding both components of the risk characterization provide greater information to the occupational hygienist. Probabilistic tools provide a more informed view of exposure as compared to use of discrete point estimates for these inputs to the risk characterization process. Active use of such tools for exposure and risk assessment will lead to a scientifically supported worker health protection program. Understanding the bases for an occupational risk assessment, focusing on important sources of variability and uncertainty enables characterizing occupational risk in terms of a probability, rather than a binary decision of acceptable risk or unacceptable risk. A critical review of existing methods highlights several conclusions: (1) exposure estimates and the dose-response are impacted by both variability and uncertainty and a well-developed risk characterization reflects and communicates this consideration; (2) occupational risk is probabilistic in nature and most accurately considered as a distribution, not a point estimate; and (3) occupational hygienists have a variety of tools available to incorporate concepts of risk characterization into occupational health and practice.

Agreement of experts and non-experts in a desktop exercise evaluating exposure to asthmagens in the cotton and textile, and other industries

In the absence of personal exposure measurements, expert assessment, generally on a case-by-case basis, is often used to estimate exposures. However, the decision processes of individual experts when making assessments are unknown, making it difficult to assess the quality of these assessments or to compare different assessments to each other. We conducted a study in primarily the textile and cotton industries, but also in baking, metal work, and agriculture industries in which we assessed agreement between experts assessing intensity and probability of exposure in the absence of exposure measurements to compare how well their performance compares to agreement of non-desktop-based exercises reported in literature. In addition, agreement was compared with that of non-experts undertaking the same exercise, and results were further stratified to assess the impact of factors expected of affected assessments. Intraclass correlation coefficients of absolute agreement (ICC1) and consistency (ICC3) between raters were calculated. Sensitivity and specificity were estimated using a probabilistic simulation methodology developed previously. Fourteen occupational hygienists and exposure assessors with complete data for all 48 job descriptions and 8 non-experts participated. Although confidence intervals about correlation-coefficient differences are not reported, the individual limits were found to be so broad as to suggest that no statistically significant comparisons can be made. Nevertheless, preliminary observations are presented here as suggested by the computed means. Absolute agreement between expert raters was fair-good, but was somewhat better for intensity (ICC1 = 0.61) than for probability (ICC1 = 0.44) of exposure and was better for experts than non-experts. Estimated sensitivity was 0.95 and specificity 0.82 for intensity, and 0.91 and 0.78 for probability of exposure, respectively. Stratification for factors hypothesized to affect agreement did not show statistically significant differences, but consistent patterns of point estimates indicated that agreement between raters (both expert on non-experts) dropped for medium levels of information compared with little or extensive information. Inclusion of a photo or video generally improved agreement between experts but not between non-experts, whereas the year of the job description had no influence on the assessments. These data indicate that the desktop exposure assessment exercise was of similar quality to previously reported levels of agreement. Agreements between experts' assessments were independent of the time period of the job and can be improved by inclusion of visual material. Agreement between experts as well as the non-experts does not increase with the detail of provided job information.

Reliability of the Advanced REACH Tool (ART)

OBJECTIVES

The aim of this study was to assess the reliability of the Advanced REACH Tool (ART) by (i) studying interassessor agreement of the resulting exposure estimates generated by the ART mechanistic model, (ii) studying interassessor agreement per model parameters of the ART mechanistic model, (iii) investigating assessor characteristics resulting in reliable estimates, and (iv) estimating the effect of training on assessor agreement.

METHODS

Prior to the 1-day workshop, participants had to assess four scenarios with the ART. During two 1-day workshops, 54 participants received 3-h training in applying the mechanistic model and the technical aspects of the web tool. Afterward, the participants assessed another four scenarios. The assessments of the participants were compared with gold standard estimates compiled by the workshop instructors. Intraclass correlation coefficients (ICCs) were calculated and per model parameter and the percentage agreement and Cohen kappa statistics were estimated.

RESULTS

The ICCs showed good agreement before and almost perfect agreement after training. However, substantial variability was observed between individual assessors' estimates for an individual scenario. After training, only 42% of the assessments lay within a factor of three of the gold standard estimate. The reliability appeared to be influenced by several factors: (i) information provided by text and video hampered the assessors gaining additional information required to make the assessments, (ii) for some parameters, the guidance documentation implemented in the tool may have been insufficient, and (iii) in some cases, the assessors were not able to implement the information explicitly provided.

CONCLUSIONS

The ART is an expert tool and extensive training is recommended prior to use. Improvements of the guidance documentation, consensus procedures, and improving the training methods could improve the reliability of ART. Nevertheless, considerable variability can be expected between assessors using ART to estimate exposure levels for a given scenario.

Brain cancer in workers employed at a specialty chemical research facility

ABSTRACT This study evaluated unique exposures and their relationship to brain cancer mortality in employees of a specialty chemical research facility. Following an exposure assessment that concerned compounds distinct to this facility, the authors conducted a cohort mortality study of 5,284 workers to assess mortality in reference to the general population and a nested case-control study to evaluate brain cancer risk associated with specific jobs and unique chemical exposures. Four hundred eighty-six deaths, including 14 brain cancer deaths, were identified. Overall mortality was lower than expected. Brain cancer mortality was elevated (standardized mortality ratio [SMR] = 2.02, 95% confidence interval [CI] = 1.11-3.40). Exposures to 5 specific chemical categories were assessed. Exploration of work history and the specific chemical exposures did not explain the brain cancer cases. No clear occupational etiology was identified.