A survey of frameworks for best practices in weight-of-evidence analyses

Incorporating a weight-of-evidence approach into a tiered assessment for chemicals management, with emphasis on program development and applications in developing countries and emerging economies

A flexible approach is described for incorporating a weight-of-evidence (WoE) methodology into a tiered ecological risk assessment (ERA)/management framework for chemicals. The approach is oriented toward informing decisions about chemicals. Communication is regarded as a critical component of the risk assessment process. The paper resulted from insights gained from seven ERA workshops held by SETAC (Society of Environmental Toxicology and Chemistry, www.setac.org) in the Asia-Pacific, African, and Latin American regions. Formal ERA methods are not fully developed or applied in many of these countries and assessments often begin with tables of risk values and test methods from countries where ERA is already implemented. While appropriate and sometimes necessary, workshop participants had questions about the reliability and relevance of using this information for regionally specific ecosystems with different receptors, fate processes, and exposure characteristics. The idea that an assessment of reliability and relevance of available information and the need for additional information was necessary at an early stage of the assessment process was considered. The judgment of reliability and relevance is central to WoE approaches along with the identification of information needs and the integration of such information. The need to engage in WoE considerations early and throughout the assessment process indicates that a tiered approach is appropriate for unifying the evaluation process in a consistent way from early screening-level steps to later more involved evaluations. The approach outlined in this article is complementary to WoE guidance developed by the Organization for Economic Co-operation and Development and many national guidance documents. To link assessments of risk to management decisions, emphasis is given to communications at each tier between the risk assessor (technical side) and the decision-makers (policy and regulatory side). Tools and information sources are suggested for each tier and suggestions are meant to be illustrative and not prescriptive. Integr Environ Assess Manag 2024;00:1-15. © 2024 SETAC.

Causal Criteria in Medical and Biological Disciplines: History, Essence, and Radiation Aspects. Report 4, Part 1: The Post-Hill Criteria and Ecolgoical Criteria

Part 1 of Report 4 is focused on the development and modifications of causal criteria after A.B. Hill (1965). Criteria from B. MacMahon et al. (1970-1996), regarded as the first textbook for modern epidemiology, were considered, and it was found that the named researchers did not offer anything new despite the frequent mention of this source in relation to the theme. A similar situation emerged with the criteria of M. Susser: the three obligatory points of this author, "Association" (or "Probability" of causality), "Time order," and "Direction of effect," are trivial, and two more special criteria, which are the development of "Popperian Epidemiology," i.e., "Surviability" of the hypothesis when it is tested by different methods (included in the refinement in Hill's criterion "Consistency of association") and "Predictive performance" of the hypothesis are more theoretical and hardly applicable for the practice of epidemiology and public health. The same restrictions apply to the similar "Popperian" criteria of D.L. Weed, "Predictability" and "Testability" of the causal hypothesis. Although the universal postulates of A.S. Evans for infectious and noninfectious pathologies can be considered exhaustive, they are not used either in epidemiology or in any other discipline practice, except for the field of infectious pathologies, which is probably explained by the complication of the ten-point complex. The little-known criteria of P. Cole (1997) for medical and forensic practice are the most important. The three parts of Hill's criterion-based approaches are important in that they go from a single epidemiological study through a cycle of studies (coupled with the integration of data from other biomedical disciplines) to re-base Hill's criteria for assessing the individual causality of an effect. These constructs complement the earlier guidance from R.E. Gots (1986) on establishing probabilistic personal causation. The collection of causal criteria and the guidelines for environmental disciplines (ecology of biota, human ecoepidemiology, and human ecotoxicology) were considered. The total dominance of inductive causal criteria, both initial and in modifications and with additions, was revealed for an apparently complete base of sources (1979-2020). Adaptations of all known causal schemes based on guidelines have been found, from Henle-Koch postulates to Hill and Susser, including in the international programs and practice of the U.S. Environmental Protection Agency. The Hill Criteria are used by the WHO and other organizations on chemical safety (IPCS) to assess causality in animal experiments for subsequent extrapolation to humans. Data on the assessment of the causality of effects in ecology, ecoepidemiology, and ecotoxicology, together with the use of Hill's criteria for animal experiments, are of significant relevance not only for radiation ecology, but also for radiobiology.

Lost in the woods: Finding our way back to the scientific method in systematic review

Systematic review has become the preferred approach to addressing causality and informing regulatory and other decision-making processes, including chemical risk assessments. While advocates of systematic reviews acknowledge that they hold great potential for increasing objectivity and transparency in assessments of chemicals and human health risks, standardizing and harmonizing systematic review methods have been challenging. This review provides a brief summary of the development of systematic review methods and some of the frameworks currently in use in the US and Europe. We also provide an in-depth evaluation and comparison of two "competing" US EPA systematic review frameworks, informed by the constructively critical recommendations from the US National Academies of Science, Engineering and Medicine. We conclude with suggestions for moving forward to harmonize systematic review methods, as we believe that further criticism of individual available frameworks likely will be unproductive. Specifically, we issue a call to action for an international collaboration to work toward a blueprint that embraces the most scientifically critical elements common to most systematic review frameworks, while necessarily accommodating adaptations for specific purposes. Despite the array of available systematic review methods, it is clear that there is a shared goal and desire to promote objective assessment and synthesis of scientific evidence informing globally important issues regarding disease causality and human health risk evaluation.

Systematic review of the association between long-term exposure to fine particulate matter and mortality

We used a transparent systematic review framework based on best practices for evaluating study quality and integrating evidence to conduct a review of the available epidemiology studies evaluating associations between long-term exposure to ambient concentrations of PM_2.5 and mortality (all-cause and non-accidental) conducted in North America. We found that while there is some consistency across studies for reporting positive associations, these associations are weak and several important methodological issues have led to uncertainties with regard to the evidence from these studies, including potential confounding by measured and unmeasured factors, exposue measurement error, and model misspecification. These uncertainties provide a plausible, alternative explanation to causality for the weakly positive findings across studies. Using a causality framework that incorporates best practices for making causal determinations, we concluded that the evidence for a causal relationship between long-term exposure to ambient PM_2.5 concentrations and mortality from these studies is inadequate.

Principles of risk decision-making

Risk management decisions in public health require consideration of a number of complex, often conflicting factors. The aim of this review was to propose a set of 10 fundamental principles to guide risk decision-making. Although each of these principles is sound in its own right, the guidance provided by different principles might lead the decision-maker in different directions. For example, where the precautionary principle advocates for preemptive risk management action under situations of scientific uncertainty and potentially catastrophic consequences, the principle of risk-based decision-making encourages decision-makers to focus on established and modifiable risks, where a return on the investment in risk management is all but guaranteed in the near term. To evaluate the applicability of the 10 principles in practice, one needs to consider 10 diverse risk issues of broad concern and explore which of these principles are most appropriate in different contexts. The 10 principles presented here afford substantive insight into the process of risk management decision-making, although decision-makers will ultimately need to exercise judgment in reaching appropriate risk decisions, accounting for all of the scientific and extra-scientific factors relevant to the risk decision at hand.

Moving persistence assessments into the 21st century: A role for weight-of-evidence and overall persistence

Assessing the persistence of chemicals in the environment is a key element in existing regulatory frameworks to protect human health and ecosystems. Persistence in the environment depends on many fate processes, including abiotic and biotic transformations and physical partitioning, which depend on substances' physicochemical properties and environmental conditions. A main challenge in persistence assessment is that existing frameworks rely on simplistic and reductionist evaluation schemes that may lead substances to be falsely assessed as persistent or the other way around-to be falsely assessed as nonpersistent. Those evaluation schemes typically assess persistence against degradation half-lives determined in single-compartment simulation tests or against degradation levels measured in stringent screening tests. Most of the available test methods, however, do not apply to all types of substances, especially substances that are poorly soluble, complex in composition, highly sorptive, or volatile. In addition, the currently applied half-life criteria are derived mainly from a few legacy persistent organic pollutants, which do not represent the large diversity of substances entering the environment. Persistence assessment would undoubtedly benefit from the development of more flexible and holistic evaluation schemes including new concepts and methods. A weight-of-evidence (WoE) approach incorporating multiple influencing factors is needed to account for chemical fate and transformation in the whole environment so as to assess overall persistence. The present paper's aim is to begin to develop an integrated assessment framework that combines multimedia approaches to organize and interpret data using a clear WoE approach to allow for a more consistent, transparent, and thorough assessment of persistence. Integr Environ Assess Manag 2022;18:868-887. © 2021 ExxonMobil Biomedical Sciences, Inc. Integrated Environmental Assessment and Management published by Wiley Periodicals LLC on behalf of Society of Environmental Toxicology & Chemistry (SETAC).

Multi-Strategy Assessment of Different Uses of QSAR under REACH Analysis of Alternatives to Advance Information Transparency

Under the Registration, Evaluation, Authorization, and Restriction of Chemicals (REACH) analysis of alternatives (AoA) process, quantitative structure–activity relationship (QSAR) models play an important role in expanding information gathering and organizing frameworks. Increasingly recognized as an alternative to testing under registration. QSARs have become a relevant tool in bridging data gaps and supporting weight of evidence (WoE) when assessing alternative substances. Additionally, QSARs are growing in importance in integrated testing strategies (ITS). For example, the REACH ITS framework for specific endpoints directs registrants to consider non-testing results, including QSAR predictions, when deciding if further animal testing is needed. Despite the raised profile of QSARs in these frameworks, a gap exists in the evaluation of QSAR use and QSAR documentation under authorization. An assessment of the different uses (e.g., WoE and ITS) in which QSAR predictions play a role in evidence gathering and organizing remains unaddressed for AoA. This study approached the disparity in information for QSAR predictions by conducting a substantive review of 24 AoA through May 2017, which contained higher-tier endpoints under REACH. Understanding the manner in which applicants manage QSAR prediction information in AoA and assessing their potential within ITS will be valuable in promoting regulatory use of QSARs and building out future platforms in the face of rapidly evolving technology while advancing information transparency.

Evaluation of animal toxicity studies with diisocyanates regarding presence of thresholds for induction and elicitation of respiratory allergy by quantitative weight of evidence

Animal toxicity studies on diisocyanates were evaluated using quantitative weight of evidence (QWoE) to test the hypothesis that the dose-response curve shows a threshold for the induction and/or elicitation of respiratory sensitization. A literature search identified 59 references that included at least two concentration groups of the diisocyanate and a vehicle-exposed concurrent control in the study design. These studies were subjected to a QWoE-assessment applying scoring criteria for quality and relevance/strength of effects relevant to the selected endpoint of respiratory sensitization. Overall, the studies assessing dose/concentration-response for diisocyanates with the endpoint, respiratory sensitization, were heterogenous regarding study design, animal models used, endpoints assessed, and quality. Only a limited number of the studies subjected to the QWoE-assessment allowed drawing conclusions about possible thresholds for respiratory sensitization. Highest quality and relevance/strength of effects scores were obtained by a series of studies specifically designed to investigate a potential threshold for elicitation of respiratory sensitization in the Brown Norway (BN) rat. These studies applied an elaborate study design to optimize induction of respiratory sensitization and reduce interference by respiratory tract irritation. In summary, the available studies provided moderate to good support for the existence of a threshold for elicitation and limited to moderate support for a threshold regarding induction of respiratory allergy by diisocyanates in experimental animals. However, a quantitative extrapolation of threshold values established in rodents to humans remains complex.

The World Trade Center Health Program: Petitions for adding qualifying health conditions

The federally mandated World Trade Center Health Program provides limited health benefits for qualifying health conditions related to the 9/11 terrorist attacks. A qualifying health condition is an illness or health condition for which the member's exposure to airborne toxins, any other hazard, or any other adverse condition resulting from the 9/11 terrorist attacks is considered substantially likely to be a significant factor in aggravating, contributing to, or causing the illness or health condition. These qualifying health conditions are listed in federal regulations. The regulations also provide a process for amending this list. This commentary describes the methods developed for adding health conditions to the list of qualifying health conditions and discusses changes to the list that have occurred during the Program's 2011-2020 period.

A Survey of Systematic Evidence Mapping Practice and the Case for Knowledge Graphs in Environmental Health and Toxicology

Systematic evidence mapping offers a robust and transparent methodology for facilitating evidence-based approaches to decision-making in chemicals policy and wider environmental health (EH). Interest in the methodology is growing; however, its application in EH is still novel. To facilitate the production of effective systematic evidence maps for EH use cases, we survey the successful application of evidence mapping in other fields where the methodology is more established. Focusing on issues of “data storage technology,” “data integrity,” “data accessibility,” and “transparency,” we characterize current evidence mapping practice and critically review its potential value for EH contexts. We note that rigid, flat data tables and schema-first approaches dominate current mapping methods and highlight how this practice is ill-suited to the highly connected, heterogeneous, and complex nature of EH data. We propose this challenge is overcome by storing and structuring data as “knowledge graphs.” Knowledge graphs offer a flexible, schemaless, and scalable model for systematically mapping the EH literature. Associated technologies, such as ontologies, are well-suited to the long-term goals of systematic mapping methodology in promoting resource-efficient access to the wider EH evidence base. Several graph storage implementations are readily available, with a variety of proven use cases in other fields. Thus, developing and adapting systematic evidence mapping for EH should utilize these graph-based resources to ensure the production of scalable, interoperable, and robust maps to aid decision-making processes in chemicals policy and wider EH.

Evaluating reliability and risk of bias of in vivo animal data for risk assessment of chemicals - Exploring the use of the SciRAP tool in a systematic review context

Within the field of health risk assessment, it is essential that evaluations of reliability or validity of toxicity data are conducted with structure and transparency. To this end, different tools for evaluating toxicity studies have been developed by different groups and organizations, for different specific purposes. The Science in Risk Assessment and Policy (SciRAP) tool was developed for use in the regulatory health risk assessment of chemicals and to promote structured and transparent evaluation of study reliability within European regulatory frameworks. As such, the SciRAP tool is not specifically tailored for use in a systematic review context. However, in light of the current movement towards applying systematic review in the field of environmental health and chemical assessments and European chemicals regulation, we were interested in exploring how SciRAP could be applied in such a context. To achieve this, the scope of the SciRAP tool was first compared to two tools developed based on systematic review principles at the US Environmental Protection Agency's IRIS program and the National Toxicology Program's Office of Health Assessment and Translation (OHAT). Next, the SciRAP and IRIS tools were both applied in a case study to evaluate the same nine in vivo animal studies and the resulting evaluations were compared. The SciRAP tool was found to address the majority of the elements included for study evaluation in the OHAT and IRIS tools. In the case study, no major differences were found in the conclusions drawn when using SciRAP or IRIS tools. However, future developments to bring the SciRAP tool more in line with systematic review principles were identified and are discussed. Overall, this work illustrates the advantages of applying structured and pre-defined methods for study evaluation and provides a unique case study comparing the impact of using different tools for evaluating animal toxicity studies.

Facilitation of risk assessment with evidence-based methods - A framework for use of systematic mapping and systematic reviews in determining hazard, developing toxicity values, and characterizing uncertainty

Systematic review tools and approaches developed for clinical medicine are often difficult to apply "off the shelf" in order to meet the needs of chemical risk assessments. To address such, we propose an approach that can be used by practitioners for using evidence-based methods to facilitate the risk assessment process. The framework builds on and combines efforts conducted to date by a number of agencies and researchers; the novelty is in combining these efforts with a practical understanding of risk assessment, and translating such into a 'step-by-step' guide. The approach relies on three key components: problem formulation, systematic evidence mapping, and systematic review, applied using a stepwise approach. Unique to this framework is the consideration of exposure in selecting, prioritizing, and evaluating data (e.g., dose-relevance, routes of exposure, etc.). Using the proposed step-by-step process, critical appraisal of individual studies (e.g., formal and structured assessment of both relevance and reliability) and integration efforts are considered in context of specified risk assessment objectives (e.g., mode of action, dose-response) as well as chemical-specific considerations. The resulting framework provides a logical approach of how evidence-based methods can be used to facilitate risk assessment, and elevates the use of systematic methods beyond hazard identification to directly facilitating transparent and objective selection of candidate studies and/or datasets used to quantitatively characterize risk, and to better use the underlying process to inform the approaches used to develop toxicity values.

Systematic Review and Weight of Evidence Are Integral to Ecological and Human Health Assessments: They Need an Integrated Framework

Scientific assessments synthesize the various results of scientific research for policy and decision making. Synthesizing evidence in environmental assessments can involve either or both of 2 systems: systematic review (SR) and weight of evidence (WoE). Systematic review was developed to systematically assemble results of clinical trials to be combined by meta-analysis. Weight-of-evidence approaches have evolved from jurisprudence to make inferences from diverse bodies of evidence in various fields. Our objectives are to describe the similarities and differences between SR and WoE and to suggest how their best practices can be combined into a general framework that is applicable to human health and ecological assessments. Integrating SR and WoE is based on the recognition that 2 processes are required, assembling evidence and making an inference. Systematic review is characterized by methodical literature searching, screening, and data extraction, originally for meta-analysis but now for various inferential methods. Weight of evidence is characterized by systematically relating heterogeneous evidence to considerations appropriate to the inference and making the inference by weighing the evidence. Systematic review enables the unbiased assembly of evidence from literature, but methods for assembling other information must be considered as well. If only 1 type of quantitative study estimates the assessment endpoint, meta-analysis is appropriate for inference. Otherwise, the heterogeneous evidence must be weighed. A framework is presented that integrates best practices into a methodical assembly and weighing of evidence. A glossary of terms for the combined practice and a history of the origins of SR and WoE are provided in Supplemental Data. Integr Environ Assess Manag 2020;16:718-728. Published 2020. This article is a US Government work and is in the public domain in the USA.

A flow scheme for cumulative assessment of pesticides for adverse liver effects

The European Food Safety Authority (EFSA) is developing approaches to cumulative risk assessment by assigning pesticides to cumulative assessment groups (CAGs). For assignment to CAGs, EFSA relies on common toxic effects (CTEs) on the target system. The developed flow scheme for assignment to liver CAGs sequentially assesses the consistency of the CTE, its adversity, its potential to be secondary to other toxicities, its human relevance, and the relation of the NOAEL for the CTE to the overall NOAEL. If the responses to all questions are "yes", allocation to a CAG is supported; "no" stops the process.

A case study applying pathway-oriented thinking to problem formulation for planning a systematic review

The use of evidence-based methods in chemical risk assessment (CRA) is still in its infancy. Novel approaches exploring how to implement Systematic Review (SR) principles and methods for evaluating human health risks from environmental chemical exposures are needed. This paper reports and comments on a conceptual model that was developed as part of a mapping exercise for planning a SR, using aluminium-containing antiperspirants (Al-AP) and female breast cancer risk as a case study. The work explores how knowledge-assembly tools and pathway-oriented thinking developed in systems toxicology can be applied to support problem formulation (PF) in the context of SR. A conceptual model was developed to map out key research questions, working hypotheses, routes of exposure, toxicity pathways and endpoints, and related health outcomes. The model draws on the analytic framework for screening topics of the U.S. Preventive Services Task Force and builds on the concept of a "source-to-outcome continuum", integrating knowledge gained from exposure pathway concepts such as the Aggregate Exposure Pathway and Adverse Outcome Pathways. The model can be used as a central decision and prioritization tool for scoping and framing Population, Exposure, Control, Outcome (PECO) questions in a transparent and iterative manner; as a supporting tool to guide the whole SR process; and to lay down the methodological foundation of a SR on the Al-AP breast cancer topic. Logic modelling can be easily combined with systems or pathway-oriented thinking, and allows for a more structured, objective and transparent approach to PF when applying SR methods to the CRA context.

Recommendations for further revisions to improve the International Agency for Research on Cancer (IARC) Monograph program

In 2019, the International Agency for Research on Cancer (IARC) "Preamble to the IARC Monographs" expanded guidance regarding the scientific approaches that should be employed in its monographs. These amendments to the monograph development process are an improvement but still fall short in several areas. While the revised Preamble lays out broad methods and approaches to evaluate scientific evidence, there is a lack of specificity with regard to how IARC Working Groups will conduct consistent evaluations in a standardized, objective, and transparent manner; document systematic review and evidence integration actions, and substantiate how these actions and decisions inform the ultimate classifications. Furthermore, no guidance is provided to ensure Working Groups consistently incorporate mechanistic evidence in a robust manner using a defined approach in the context of 21st century knowledge of modes of action. Nor are the conclusions of the working groups subjected to outside, independent scientific peer review. Continued improvements and modernization of the procedures for evaluating, presenting, and communicating study quality, and in the methods used to conduct and peer-review evidence-based decision making will benefit the Working Group members, the IARC Monographs Programme overall, and the international regulatory community and public who rely upon the monographs.

Corporations' use and misuse of evidence to influence health policy: a case study of sugar-sweetened beverage taxation

Background

Sugar sweetened beverages (SSB) are a major source of sugar in the diet. Although trends in consumption vary across regions, in many countries, particularly LMICs, their consumption continues to increase. In response, a growing number of governments have introduced a tax on SSBs. SSB manufacturers have opposed such taxes, disputing the role that SSBs play in diet-related diseases and the effectiveness of SSB taxation, and alleging major economic impacts. Given the importance of evidence to effective regulation of products harmful to human health, we scrutinised industry submissions to the South African government’s consultation on a proposed SSB tax and examined their use of evidence.

Results

Corporate submissions were underpinned by several strategies involving the misrepresentation of evidence. First, references were used in a misleading way, providing false support for key claims. Second, raw data, which represented a pliable, alternative evidence base to peer reviewed studies, was misused to dispute both the premise of targeting sugar for special attention and the impact of SSB taxes on SSB consumption. Third, purposively selected evidence was used in conjunction with other techniques, such as selective quoting from studies and omitting important qualifying information, to promote an alternative evidential narrative to that supported by the weight of peer-reviewed research. Fourth, a range of mutually enforcing techniques that inflated the effects of SSB taxation on jobs, public revenue generation, and gross domestic product, was used to exaggerate the economic impact of the tax. This “hyperbolic accounting” included rounding up figures in original sources, double counting, and skipping steps in economic modelling.

Conclusions

Our research raises fundamental questions concerning the bona fides of industry information in the context of government efforts to combat diet-related diseases. The beverage industry’s claims against SSB taxation rest on a complex interplay of techniques, that appear to be grounded in evidence, but which do not observe widely accepted approaches to the use of either scientific or economic evidence. These techniques are similar, but not identical, to those used by tobacco companies and highlight the problems of introducing evidence-based policies aimed at managing the market environment for unhealthful commodities.

Electronic supplementary material

The online version of this article (10.1186/s12992-019-0495-5) contains supplementary material, which is available to authorized users.

A unified approach for protecting listed species and ecosystem services in isolated wetlands using community-level protection goals

The protection of listed species through the Ecological Risk Assessment (ERA) process is encumbered by the number and diversity of species that need protection and the limited data available to inform assessments. Ecological communities within isolated ecosystems often contain a number of biologically diverse endemic, endangered, and threatened species, as well as providing numerous ecosystem services (ES). We propose an approach that develops community-level protection goals using isolated wetlands that includes both listed species and Service Providing Units (SPUs) that drive ES for ecological risk assessments (ERAs). Community-level protection goals are achieved by developing a protection community and weighing lines of evidence to determine a set of focal species within that community upon which to base the assessment. Lines of evidence include chemical mechanism of action, likely routes of exposure, and taxa susceptibility, as well as relationships among species, and other ecological factors. We demonstrate the process using case studies of chlorpyrifos in California vernal pools and coal ash effluent in Carolina bays. In the California vernal pool case study, listed species were the primary SPUs for the ES provided by the critical habitat. The weight of evidence demonstrated the honey bee as the focal species for the terrestrial environment and the vernal pool fairy shrimp as the focal species for the aquatic environment. The protection community within the Carolina bay case study was more taxonomically diverse than vernal pools for both listed species and SPUs, with amphibians identified as the focal species for which to target mitigation goals and hazard levels. The approach presented here will reduce the time and resource investment required for assessment of risk to listed species and adds an ES perspective to demonstrate value of assessments beyond listed species concerns.

Risk Assessment for Tobacco Regulation

Risk assessment is a process that uses a transparent, reproducible and pre-established methodology to evaluate alternatives for managing health-related risks. Although an array of federal agencies regularly use risk assessment to inform regulatory decisions, its application to tobacco regulation is new. By comparing examples of FDA risk assessments for food and tobacco, this paper highlights some of the challenges inherent in applying risk assessment methodologies to tobacco regulation. In doing so, it calls upon researchers to work with the FDA to develop a tobacco-specific approach to risk assessment that reflects the Tobacco Control Act's regulatory framework and the distinctive features of tobacco products and tobacco use.

A review of chemical-based sediment quality assessment methodologies for the marine environment

This review of 19 chemical approaches used in assessing sediment quality are classified into empirical, mechanistic and sediment quality indices (SQI) groups. Empirical sediment quality guidelines (SQGs), based on matching chemical and biological-effects data and the mechanistic techniques, founded on equilibrium partitioning principals (EqP), are well established and most used. Empirical SQGs provide a useful screening tool to initially identify locations and chemicals of most concern, but are not regulatory criteria. The EqP approach is causally linked however, the scheme assumes porewater chemistry largely controls sediment toxicity. SQIs are not based on matching chemical-biological data and combine schemes with multiple narrative intents. The 41 SQGs reviewed show a considerable range in upper and lower guideline values. Grain size and organic content should be included into SQGs, however inclusion of suspended sediment into SQGs raises concerns. SQGs are built into decision-tree schemes with other lines-of-evidence and evaluated in a weight-of-evidence framework.

Weight of Evidence for Hazard Identification: A Critical Review of the Literature

BACKGROUND

Transparency when documenting and assessing weight of evidence (WOE) has been an area of increasing focus for national and international health agencies.

OBJECTIVE

The objective of this work was to conduct a critical review of WOE analysis methods as a basis for developing a practical framework for considering and assessing WOE in hazard identification in areas of application at the French Agency for Food, Environmental and Occupational Health and Safety (ANSES).

METHODS

Based on a review of the literature and directed requests to 63 international and national agencies, 116 relevant articles and guidance documents were selected. The WOE approaches were assessed based on three aspects: the extent of their prescriptive nature, their purpose-specific relevance, and their ease of implementation.

RESULTS

Twenty-four approaches meeting the specified criteria were identified from selected reviewed documents. Most approaches satisfied one or two of the assessed considerations, but not all three. The approaches were grouped within a practical framework comprising the following four stages: (1) planning the assessment, including scoping, formulating the question, and developing the assessment method; (2) establishing lines of evidence (LOEs), including identifying and selecting studies, assessing their quality, and integrating with studies of similar type; (3) integrating the LOEs to evaluate WOE; and (4) presenting conclusions.

DISCUSSION

Based on the review, considerations for selecting methods for a wide range of applications are proposed. Priority areas for further development are identified. https://doi.org/10.1289/EHP3067.

Pyrethroid epidemiology: a quality-based review

Pyrethroids are commonly used around the home and in agricultural production to control insects. Human contact to one or more pyrethroid insecticides is likely. Numerous epidemiology studies have evaluated the association between health outcomes in humans and pyrethroid exposure. The purpose of this review was to identify and evaluate the quality of pyrethroid-related epidemiology studies that addressed chronic health effects, and compare findings with animal toxicology studies. We evaluated the quality of 61 studies published between 2000 and 2016 by using elements of outcome, exposure metric, exposure level, and study design. None of the 61 publications demonstrated strong quality for all elements. A few of the outcome measures were strong, particularly those relying upon medical diagnoses. Most of the pyrethroid epidemiology studies used a poor exposure metric, relying upon a single sample of pyrethroid urinary metabolites, which is subject to misclassification of past exposures. In addition, many studies were a cross-sectional design, preventing an evaluation of the temporality of the exposure-disease association. Furthermore, none of the effects observed in the epidemiological literature was concordant with toxicological effects noted in extensive testing of pyrethroids in animals. In order to provide more robust data on potential health outcomes from low dose exposure to pyrethroid insecticides, future epidemiological studies should fully characterize an adverse outcome, include exposure validation components, and quantify exposure over time.

A weight of evidence framework for environmental assessments: Inferring qualities

The weighing of heterogeneous evidence such as conventional laboratory toxicity tests, field tests, biomarkers, and community surveys is essential to environmental assessments. Evidence synthesis and weighing is needed to determine causes of observed effects, hazards posed by chemicals or other agents, the completeness of remediation, and other environmental qualities. As part of its guidelines for weight of evidence (WoE) in ecological assessments, the US Environmental Protection Agency has developed a generally applicable framework. Its basic steps are these: assemble evidence, weight the evidence, and weigh the body of evidence. Use of the framework can increase the consistency and rigor of WoE practices and provide greater transparency than ad hoc and narrative-based approaches. Integr Environ Assess Manag 2017;13:1038-1044. Published 2017. This article is a US Government work and is in the public domain in the USA.

Using systematic review in occupational safety and health

Evaluation of scientific evidence is critical in developing recommendations to reduce risk. Healthcare was the first scientific field to employ a systematic review approach for synthesizing research findings to support evidence-based decision-making and it is still the largest producer and consumer of systematic reviews. Systematic reviews in the field of occupational safety and health are being conducted, but more widespread use and adoption would strengthen assessments. In 2016, NIOSH asked RAND to develop a framework for applying the traditional systematic review elements to the field of occupational safety and health. This paper describes how essential systematic review elements can be adapted for use in occupational systematic reviews to enhance their scientific quality, objectivity, transparency, reliability, utility, and acceptability.

Framework for the quantitative weight-of-evidence analysis of 'omics data for regulatory purposes

A framework for the quantitative weight-of-evidence (QWoE) analysis of 'omics data for regulatory purposes is presented. The QWoE framework encompasses seven steps to evaluate 'omics data (also together with non-'omics data): (1) Hypothesis formulation, identification and weighting of lines of evidence (LoEs). LoEs conjoin different (types of) studies that are used to critically test the hypothesis. As an essential component of the QWoE framework, step 1 includes the development of templates for scoring sheets that predefine scoring criteria with scores of 0-4 to enable a quantitative determination of study quality and data relevance; (2) literature searches and categorisation of studies into the pre-defined LoEs; (3) and (4) quantitative assessment of study quality and data relevance using the respective pre-defined scoring sheets for each study; (5) evaluation of LoE-specific strength of evidence based upon the study quality and study relevance scores of the studies conjoined in the respective LoE; (6) integration of the strength of evidence from the individual LoEs to determine the overall strength of evidence; (7) characterisation of uncertainties and conclusion on the QWoE. To put the QWoE framework in practice, case studies are recommended to confirm the relevance of its different steps, or to adapt them as necessary.

Guidance on the use of the weight of evidence approach in scientific assessments

EFSA requested the Scientific Committee to develop a guidance document on the use of the weight of evidence approach in scientific assessments for use in all areas under EFSA's remit. The guidance document addresses the use of weight of evidence approaches in scientific assessments using both qualitative and quantitative approaches. Several case studies covering the various areas under EFSA's remit are annexed to the guidance document to illustrate the applicability of the proposed approach. Weight of evidence assessment is defined in this guidance as a process in which evidence is integrated to determine the relative support for possible answers to a question. This document considers the weight of evidence assessment as comprising three basic steps: (1) assembling the evidence into lines of evidence of similar type, (2) weighing the evidence, (3) integrating the evidence. The present document identifies reliability, relevance and consistency as three basic considerations for weighing evidence.

Glyphosate toxicity and carcinogenicity: a review of the scientific basis of the European Union assessment and its differences with IARC

Glyphosate is the most widely used herbicide worldwide. It is a broad spectrum herbicide and its agricultural uses increased considerably after the development of glyphosate-resistant genetically modified (GM) varieties. Since glyphosate was introduced in 1974, all regulatory assessments have established that glyphosate has low hazard potential to mammals, however, the International Agency for Research on Cancer (IARC) concluded in March 2015 that it is probably carcinogenic. The IARC conclusion was not confirmed by the EU assessment or the recent joint WHO/FAO evaluation, both using additional evidence. Glyphosate is not the first topic of disagreement between IARC and regulatory evaluations, but has received greater attention. This review presents the scientific basis of the glyphosate health assessment conducted within the European Union (EU) renewal process, and explains the differences in the carcinogenicity assessment with IARC. Use of different data sets, particularly on long-term toxicity/carcinogenicity in rodents, could partially explain the divergent views; but methodological differences in the evaluation of the available evidence have been identified. The EU assessment did not identify a carcinogenicity hazard, revised the toxicological profile proposing new toxicological reference values, and conducted a risk assessment for some representatives uses. Two complementary exposure assessments, human-biomonitoring and food-residues-monitoring, suggests that actual exposure levels are below these reference values and do not represent a public concern.

Framework for assessing causality of air pollution-related health effects for reviews of the National Ambient Air Quality Standards

To inform regulatory decisions on the risk due to exposure to ambient air pollution, consistent and transparent communication of the scientific evidence is essential. The United States Environmental Protection Agency (U.S. EPA) develops the Integrated Science Assessment (ISA), which contains evaluations of the policy-relevant science on the effects of criteria air pollutants and conveys critical science judgments to inform decisions on the National Ambient Air Quality Standards. This article discusses the approach and causal framework used in the ISAs to evaluate and integrate various lines of scientific evidence and draw conclusions about the causal nature of air pollution-induced health effects. The framework has been applied to diverse pollutants and cancer and noncancer effects. To demonstrate its flexibility, we provide examples of causality judgments on relationships between health effects and pollutant exposures, drawing from recent ISAs for ozone, lead, carbon monoxide, and oxides of nitrogen. U.S. EPA's causal framework has increased transparency by establishing a structured process for evaluating and integrating various lines of evidence and uniform approach for determining causality. The framework brings consistency and specificity to the conclusions in the ISA, and the flexibility of the framework makes it relevant for evaluations of evidence across media and health effects.

Determinants of neurological disease: Synthesis of systematic reviews

Systematic reviews were conducted to identify risk factors associated with the onset and progression of 14 neurological conditions, prioritized as a component of the National Population Health Study of Neurological Conditions. These systematic reviews provided a basis for evaluating the weight of evidence of evidence for risk factors for the onset and progression of the 14 individual neurological conditions considered. A number of risk factors associated with an increased risk of onset for more than one condition, including exposure to pesticides (associated with an increased risk of AD, amyotrophic lateral sclerosis, brain tumours, and PD; smoking (AD, MS); and infection (MS, Tourette syndrome). Coffee and tea intake was associated with a decreased risk of onset of both dystonia and PD. Further understanding of the etiology of priority neurological conditions will be helpful in focusing future research initiatives and in the development of interventions to reduce the burden associated with neurological conditions in Canada and internationally.

Options for basing Dietary Reference Intakes (DRIs) on chronic disease endpoints: report from a joint US-/Canadian-sponsored working group

Dietary Reference Intakes (DRIs) are used in Canada and the United States in planning and assessing diets of apparently healthy individuals and population groups. The approaches used to establish DRIs on the basis of classical nutrient deficiencies and/or toxicities have worked well. However, it has proved to be more challenging to base DRI values on chronic disease endpoints; deviations from the traditional framework were often required, and in some cases, DRI values were not established for intakes that affected chronic disease outcomes despite evidence that supported a relation. The increasing proportions of elderly citizens, the growing prevalence of chronic diseases, and the persistently high prevalence of overweight and obesity, which predispose to chronic disease, highlight the importance of understanding the impact of nutrition on chronic disease prevention and control. A multidisciplinary working group sponsored by the Canadian and US government DRI steering committees met from November 2014 to April 2016 to identify options for addressing key scientific challenges encountered in the use of chronic disease endpoints to establish reference values. The working group focused on 3 key questions: 1) What are the important evidentiary challenges for selecting and using chronic disease endpoints in future DRI reviews, 2) what intake-response models can future DRI committees consider when using chronic disease endpoints, and 3) what are the arguments for and against continuing to include chronic disease endpoints in future DRI reviews? This report outlines the range of options identified by the working group for answering these key questions, as well as the strengths and weaknesses of each option.

A critical review of frameworks used for evaluating reliability and relevance of (eco)toxicity data: Perspectives for an integrated eco-human decision-making framework

Considerable efforts have been invested so far to evaluate and rank the quality and relevance of (eco)toxicity data for their use in regulatory risk assessment to assess chemical hazards. Many frameworks have been developed to improve robustness and transparency in the evaluation of reliability and relevance of individual tests, but these frameworks typically focus on either environmental risk assessment (ERA) or human health risk assessment (HHRA), and there is little cross talk between them. There is a need to develop a common approach that would support a more consistent, transparent and robust evaluation and weighting of the evidence across ERA and HHRA. This paper explores the applicability of existing Data Quality Assessment (DQA) frameworks for integrating environmental toxicity hazard data into human health assessments and vice versa. We performed a comparative analysis of the strengths and weaknesses of eleven frameworks for evaluating reliability and/or relevance of toxicity and ecotoxicity hazard data. We found that a frequent shortcoming is the lack of a clear separation between reliability and relevance criteria. A further gaps and needs analysis revealed that none of the reviewed frameworks satisfy the needs of a common eco-human DQA system. Based on our analysis, some key characteristics, perspectives and recommendations are identified and discussed for building a common DQA system as part of a future integrated eco-human decision-making framework. This work lays the basis for developing a common DQA system to support the further development and promotion of Integrated Risk Assessment.

Comprehensive review of epidemiological and animal studies on the potential carcinogenic effects of nicotine per se

The effects of long-term use of nicotine per se on cancer risk, in the absence of tobacco extract or smoke, are not clearly understood. This review evaluates the strength of published scientific evidence, in both epidemiological and animal studies, for the potential carcinogenic effects of nicotine per se; that is to act as a complete carcinogen or as a modulator of carcinogenesis. For human studies, there appears to be inadequate evidence for an association between nicotine exposure and the presence of or lack of a carcinogenic effect due to the limited information available. In animal studies, limited evidence suggests an association between long-term nicotine exposure and a lack of a complete carcinogenic effect. Conclusive studies using current bioassay guidelines, however, are missing. In studies using chemical/physical carcinogens or transgenic models, there appears to be inadequate evidence for an association between nicotine exposure and the presence of or lack of a modulating (stimulating) effect on carcinogenesis. This is primarily due to the large number of conflicting studies. In contrast, a majority of studies provides sufficient evidence for an association between nicotine exposure and enhanced carcinogenesis of cancer cells inoculated in mice. This modulating effect was especially prominent in immunocompromized mice. Overall, taking the human and animal studies into consideration, there appears to be inadequate evidence to conclude that nicotine per se does or does not cause or modulate carcinogenesis in humans. This conclusion is in agreement with the recent US Surgeon General's 2014 report on the health consequences of nicotine exposure.

Key Elements for Judging the Quality of a Risk Assessment

BACKGROUND

Many reports have been published that contain recommendations for improving the quality, transparency, and usefulness of decision making for risk assessments prepared by agencies of the U.S. federal government. A substantial measure of consensus has emerged regarding the characteristics that high-quality assessments should possess.

OBJECTIVE

The goal was to summarize the key characteristics of a high-quality assessment as identified in the consensus-building process and to integrate them into a guide for use by decision makers, risk assessors, peer reviewers and other interested stakeholders to determine if an assessment meets the criteria for high quality.

DISCUSSION

Most of the features cited in the guide are applicable to any type of assessment, whether it encompasses one, two, or all four phases of the risk-assessment paradigm; whether it is qualitative or quantitative; and whether it is screening level or highly sophisticated and complex. Other features are tailored to specific elements of an assessment. Just as agencies at all levels of government are responsible for determining the effectiveness of their programs, so too should they determine the effectiveness of their assessments used in support of their regulatory decisions. Furthermore, if a nongovernmental entity wishes to have its assessments considered in the governmental regulatory decision-making process, then these assessments should be judged in the same rigorous manner and be held to similar standards.

CONCLUSIONS

The key characteristics of a high-quality assessment can be summarized and integrated into a guide for judging whether an assessment possesses the desired features of high quality, transparency, and usefulness.

CITATION

Fenner-Crisp PA, Dellarco VL. 2016. Key elements for judging the quality of a risk assessment. Environ Health Perspect 124:1127-1135; http://dx.doi.org/10.1289/ehp.1510483.

Systematic reviews of factors associated with the onset and progression of neurological conditions in humans: A methodological overview

As a component of the National Population Health Study of Neurological conditions, systematic reviews were conducted to identify risk factors associated with the onset and progression of 14 priority neurological conditions. Between 2011 and 2013, electronic databases and grey literature sources were searched to identify systematic reviews and primary studies reporting on the onset and progression of each condition. Inclusion was restricted to studies of humans reported in English or French. Additional condition-specific eligibility criteria were also applied. Titles and abstracts were screened by one reviewer with excluded records verified by a second reviewer. Full-text reports were screened independently by two reviewers. Disagreements were resolved by consensus or third party adjudication. Systematic reviews were quality appraised using the AMSTAR criteria, with only moderate and high quality reviews considered for inclusion. Primary studies were also sought to ensure that evidence from existing systematic reviews was supplemented with recent primary study findings (i.e., those published after the most recent systematic review). Evidence from primary studies was also considered if a systematic review was unavailable or of poor quality. Data were extracted using standardized forms. Where feasible, data were extracted independently by two reviewers. Otherwise, data were extracted by a single reviewer and independent data extraction by a second reviewer was conducted for a randomly selected sample of studies. An updated search was conducted in 2016 to identify systematic reviews published since the initial search in 2011-2013. A summary of the methodology used to conduct the systematic reviews is described. Illustrative results are provided for the risk of amyotrophic lateral sclerosis in relation to occupational exposure to lead and other heavy metals.

Weight of evidence evaluation and systematic review in EU chemical risk assessment: Foundation is laid but guidance is needed

The aim of this review was to investigate if and how the application of weight of evidence (WoE) evaluation or systematic review (SR) in chemical risk assessment is promoted within different regulatory frameworks in the European Union. Legislative and relevant guidance documents within nine regulatory frameworks were scrutinized and compared. WoE evaluation or SR is promoted in seven of the investigated frameworks but sufficient guidance for how to perform these processes is generally lacking. None of the investigated frameworks give enough guidance for generating robust and reproducible WoE evaluations or SRs. In conclusion, the foundation for use of WoE evaluation and SR is laid in the majority of the investigated frameworks, but there is a need to provide more structured and detailed guidance. In order to make the process of developing guidance as efficient as possible, and to ensure smooth transfer of risk assessment's between frameworks if a chemical is risk assessed both as, for example, a biocide and an industrial chemical, it is recommended that guidance is developed jointly by the European regulatory agencies.

Does GLP enhance the quality of toxicological evidence for regulatory decisions?

There is debate over whether the requirements of GLP are appropriate standards for evaluating the quality of toxicological data used to formulate regulations. A group promoting the importance of non-monotonic dose responses for endocrine disruptors contend that scoring systems giving primacy to GLP are biased against non-GLP studies from the literature and are merely record-keeping exercises to prevent fraudulent reporting of data from non-published guideline toxicology studies. They argue that guideline studies often employ insensitive species and outdated methods, and ignore the perspectives of subject-matter experts in endocrine disruption, who should be the sole arbiters of data quality. We believe regulatory agencies should use both non-GLP and GLP studies, that GLP requirements assure fundamental tenets of study integrity not typically addressed by journal peer-review, and that use of standardized test guidelines and GLP promotes consistency, reliability, comparability, and harmonization of various types of studies used by regulatory agencies worldwide. This debate suffers two impediments to progress: a conflation of different phases of study interpretation and levels of data validity, and a misleading characterization of many essential components of GLP and regulatory toxicology. Herein we provide clarifications critical for removing those impediments.