The weight of scientific evidence in policy and law

Evidence integration on health damage for humidifier disinfectant exposure and legal presumption of causation

OBJECTIVES

Inhalation exposure to humidifier disinfectants has resulted to various types of health damages in Korea. To determine the epidemiological correlation necessary for presuming the legal causation, we aimed to develop a method to synthesize the entire evidence.

METHODS

Epidemiological and toxicological studies are systematically reviewed. Target health problems are selected by criteria such as frequent complaints of claimants. Relevant epidemiologic studies are reviewed and the risk of bias and confidence level of the total evidence are evaluated. Toxicological literature reviews are conducted on three lines of evidence including hazard information, animal studies, and mechanistic studies, considering the source-to-exposure-to-outcome continuum. The confidence level of the body of evidence is then translated into the toxicological evidence levels for the causality between humidifier disinfectant exposure and health effects. Finally, the levels of epidemiological and toxicological evidence are synthesized.

RESULTS

Under the Special Act revised in 2020, if the history of exposure and the disease occurred/worsened after exposure were approved, and the epidemiological correlation between the exposure and disease was verified, the legal causation is presumed unless the company proves the evidence against it. The epidemiological correlation can be verified through epidemiological investigations, health monitoring, cohort investigations and/or toxicological studies. It is not simply as statistical association as understood in judicial precedents, but a general causation established by the evidence as a whole, i.e., through weight-of-the-evidence approach.

CONCLUSIONS

The weight-of-the-evidence approach differs from the conclusive single study approach and this systematic evidence integration can be used in presumption of causation.

Corporations' use and misuse of evidence to influence health policy: a case study of sugar-sweetened beverage taxation

Background

Sugar sweetened beverages (SSB) are a major source of sugar in the diet. Although trends in consumption vary across regions, in many countries, particularly LMICs, their consumption continues to increase. In response, a growing number of governments have introduced a tax on SSBs. SSB manufacturers have opposed such taxes, disputing the role that SSBs play in diet-related diseases and the effectiveness of SSB taxation, and alleging major economic impacts. Given the importance of evidence to effective regulation of products harmful to human health, we scrutinised industry submissions to the South African government’s consultation on a proposed SSB tax and examined their use of evidence.

Results

Corporate submissions were underpinned by several strategies involving the misrepresentation of evidence. First, references were used in a misleading way, providing false support for key claims. Second, raw data, which represented a pliable, alternative evidence base to peer reviewed studies, was misused to dispute both the premise of targeting sugar for special attention and the impact of SSB taxes on SSB consumption. Third, purposively selected evidence was used in conjunction with other techniques, such as selective quoting from studies and omitting important qualifying information, to promote an alternative evidential narrative to that supported by the weight of peer-reviewed research. Fourth, a range of mutually enforcing techniques that inflated the effects of SSB taxation on jobs, public revenue generation, and gross domestic product, was used to exaggerate the economic impact of the tax. This “hyperbolic accounting” included rounding up figures in original sources, double counting, and skipping steps in economic modelling.

Conclusions

Our research raises fundamental questions concerning the bona fides of industry information in the context of government efforts to combat diet-related diseases. The beverage industry’s claims against SSB taxation rest on a complex interplay of techniques, that appear to be grounded in evidence, but which do not observe widely accepted approaches to the use of either scientific or economic evidence. These techniques are similar, but not identical, to those used by tobacco companies and highlight the problems of introducing evidence-based policies aimed at managing the market environment for unhealthful commodities.

Electronic supplementary material

The online version of this article (10.1186/s12992-019-0495-5) contains supplementary material, which is available to authorized users.

Socioeconomic status and the brain: prospects for neuroscience-informed policy

Socioeconomic status (SES) is associated with health (physical and mental) and cognitive ability. Understanding and ameliorating the problems of low SES have long been goals of economics and sociology; in recent years, these have also become goals of neuroscience. However, opinion varies widely on the relevance of neuroscience to SES-related policy. The present article addresses the question of whether and how neuroscience can contribute to the development of social policy concerning poverty and the social and ethical risks inherent in trying. I argue that the neuroscience approach to SES-related policy has been both prematurely celebrated and peremptorily dismissed and that some of its possible social impacts have been viewed with excessive alarm. Neuroscience has already made modest contributions to SES-related policy, and its potential to have a more effective and beneficial influence can be expected to grow over the coming years.

Using problem formulation to clarify the meaning of weight of evidence and biological relevance in environmental risk assessments for genetically modified crops

Weight of evidence and biological relevance are important concepts for risk assessment and decision-making over the use of GM crops; however, their meanings are not well defined. We use problem formulation to clarify the definition of these concepts and thereby identify data that are relevant for risk assessment. Problem formulation defines criteria for the acceptability of risk and devises rigorous tests of the hypothesis that the criteria are met. Corroboration or falsification of such hypotheses characterize risk and enable predictable and transparent decisions about whether certain risks from using a particular GM crop are acceptable. Decisions based on a weight of evidence approach use a synthesis of several lines of evidence, whereas a "definitive" approach to risk assessment enables some decisions to be based on the results of a single test. Data are biologically relevant for risk assessment only if they test a hypothesis that is useful for decision-making.

A review of chemical-based sediment quality assessment methodologies for the marine environment

This review of 19 chemical approaches used in assessing sediment quality are classified into empirical, mechanistic and sediment quality indices (SQI) groups. Empirical sediment quality guidelines (SQGs), based on matching chemical and biological-effects data and the mechanistic techniques, founded on equilibrium partitioning principals (EqP), are well established and most used. Empirical SQGs provide a useful screening tool to initially identify locations and chemicals of most concern, but are not regulatory criteria. The EqP approach is causally linked however, the scheme assumes porewater chemistry largely controls sediment toxicity. SQIs are not based on matching chemical-biological data and combine schemes with multiple narrative intents. The 41 SQGs reviewed show a considerable range in upper and lower guideline values. Grain size and organic content should be included into SQGs, however inclusion of suspended sediment into SQGs raises concerns. SQGs are built into decision-tree schemes with other lines-of-evidence and evaluated in a weight-of-evidence framework.

Weight of Evidence for Hazard Identification: A Critical Review of the Literature

BACKGROUND

Transparency when documenting and assessing weight of evidence (WOE) has been an area of increasing focus for national and international health agencies.

OBJECTIVE

The objective of this work was to conduct a critical review of WOE analysis methods as a basis for developing a practical framework for considering and assessing WOE in hazard identification in areas of application at the French Agency for Food, Environmental and Occupational Health and Safety (ANSES).

METHODS

Based on a review of the literature and directed requests to 63 international and national agencies, 116 relevant articles and guidance documents were selected. The WOE approaches were assessed based on three aspects: the extent of their prescriptive nature, their purpose-specific relevance, and their ease of implementation.

RESULTS

Twenty-four approaches meeting the specified criteria were identified from selected reviewed documents. Most approaches satisfied one or two of the assessed considerations, but not all three. The approaches were grouped within a practical framework comprising the following four stages: (1) planning the assessment, including scoping, formulating the question, and developing the assessment method; (2) establishing lines of evidence (LOEs), including identifying and selecting studies, assessing their quality, and integrating with studies of similar type; (3) integrating the LOEs to evaluate WOE; and (4) presenting conclusions.

DISCUSSION

Based on the review, considerations for selecting methods for a wide range of applications are proposed. Priority areas for further development are identified. https://doi.org/10.1289/EHP3067.

Using systematic review in occupational safety and health

Evaluation of scientific evidence is critical in developing recommendations to reduce risk. Healthcare was the first scientific field to employ a systematic review approach for synthesizing research findings to support evidence-based decision-making and it is still the largest producer and consumer of systematic reviews. Systematic reviews in the field of occupational safety and health are being conducted, but more widespread use and adoption would strengthen assessments. In 2016, NIOSH asked RAND to develop a framework for applying the traditional systematic review elements to the field of occupational safety and health. This paper describes how essential systematic review elements can be adapted for use in occupational systematic reviews to enhance their scientific quality, objectivity, transparency, reliability, utility, and acceptability.

Inspiring Collaboration: The Legacy of Theo Colborn's Transdisciplinary Research on Fracking

This article describes Dr Theo Colborn's legacy of inspiring complementary and synergistic environmental health research and advocacy. Colborn, a founder of endocrine disruption research, also stimulated study of hydraulic fracturing (fracking). In 2014, the United States led the world in oil and gas production, with fifteen million Americans living within one mile of an oil or gas well. Colborn pioneered efforts to understand and control the impacts of this sea change in energy production. In 2005, her research organization The Endocrine Disruption Exchange (TEDX) developed a database of chemicals used in natural gas extraction and their health effects. This database stimulated novel scientific and social scientific research and informed advocacy by (1) connecting communities' diverse health impacts to chemicals used in natural gas development, (2) inspiring social science research on open-source software and hardware for citizen science, and (3) posing new scientific questions about the endocrine-disrupting properties of fracking chemicals.

A proposed framework for the systematic review and integrated assessment (SYRINA) of endocrine disrupting chemicals

BACKGROUND

The issue of endocrine disrupting chemicals (EDCs) is receiving wide attention from both the scientific and regulatory communities. Recent analyses of the EDC literature have been criticized for failing to use transparent and objective approaches to draw conclusions about the strength of evidence linking EDC exposures to adverse health or environmental outcomes. Systematic review methodologies are ideal for addressing this issue as they provide transparent and consistent approaches to study selection and evaluation. Objective methods are needed for integrating the multiple streams of evidence (epidemiology, wildlife, laboratory animal, in vitro, and in silico data) that are relevant in assessing EDCs.

METHODS

We have developed a framework for the systematic review and integrated assessment (SYRINA) of EDC studies. The framework was designed for use with the International Program on Chemical Safety (IPCS) and World Health Organization (WHO) definition of an EDC, which requires appraisal of evidence regarding 1) association between exposure and an adverse effect, 2) association between exposure and endocrine disrupting activity, and 3) a plausible link between the adverse effect and the endocrine disrupting activity.

RESULTS

Building from existing methodologies for evaluating and synthesizing evidence, the SYRINA framework includes seven steps: 1) Formulate the problem; 2) Develop the review protocol; 3) Identify relevant evidence; 4) Evaluate evidence from individual studies; 5) Summarize and evaluate each stream of evidence; 6) Integrate evidence across all streams; 7) Draw conclusions, make recommendations, and evaluate uncertainties. The proposed method is tailored to the IPCS/WHO definition of an EDC but offers flexibility for use in the context of other definitions of EDCs.

CONCLUSIONS

When using the SYRINA framework, the overall objective is to provide the evidence base needed to support decision making, including any action to avoid/minimise potential adverse effects of exposures. This framework allows for the evaluation and synthesis of evidence from multiple evidence streams. Finally, a decision regarding regulatory action is not only dependent on the strength of evidence, but also the consequences of action/inaction, e.g. limited or weak evidence may be sufficient to justify action if consequences are serious or irreversible.

Weight of evidence evaluation and systematic review in EU chemical risk assessment: Foundation is laid but guidance is needed

The aim of this review was to investigate if and how the application of weight of evidence (WoE) evaluation or systematic review (SR) in chemical risk assessment is promoted within different regulatory frameworks in the European Union. Legislative and relevant guidance documents within nine regulatory frameworks were scrutinized and compared. WoE evaluation or SR is promoted in seven of the investigated frameworks but sufficient guidance for how to perform these processes is generally lacking. None of the investigated frameworks give enough guidance for generating robust and reproducible WoE evaluations or SRs. In conclusion, the foundation for use of WoE evaluation and SR is laid in the majority of the investigated frameworks, but there is a need to provide more structured and detailed guidance. In order to make the process of developing guidance as efficient as possible, and to ensure smooth transfer of risk assessment's between frameworks if a chemical is risk assessed both as, for example, a biocide and an industrial chemical, it is recommended that guidance is developed jointly by the European regulatory agencies.

Using systematic reviews for hazard and risk assessment of endocrine disrupting chemicals

The possibility that endocrine disrupting chemicals (EDCs) in our environment contribute to hormonally related effects and diseases observed in human and wildlife populations has caused concern among decision makers and researchers alike. EDCs challenge principles traditionally applied in chemical risk assessment and the identification and assessment of these compounds has been a much debated topic during the last decade. State of the science reports and risk assessments of potential EDCs have been criticized for not using systematic and transparent approaches in the evaluation of evidence. In the fields of medicine and health care, systematic review methodologies have been developed and used to enable objectivity and transparency in the evaluation of scientific evidence for decision making. Lately, such approaches have also been promoted for use in the environmental health sciences and risk assessment of chemicals. Systematic review approaches could provide a tool for improving the evaluation of evidence for decision making regarding EDCs, e.g. by enabling systematic and transparent use of academic research data in this process. In this review we discuss the advantages and challenges of applying systematic review methodology in the identification and assessment of EDCs.

Evidence-based causation in toxicology

We introduced Evidence-based Toxicology (EBT) in 2005 to address the disparities that exist between the various Weight-of-Evidence (WOE) methods typically applied in the regulatory hazard decision-making arena and urged toxicologists to adopt the evidence-based guidelines long-utilized in medicine (i.e., Evidence-Based Medicine or EBM). This review of the activities leading to the adoption of evidence-based methods and EBT during the last decade demonstrates how fundamental concepts that form EBT, such as the use of systematic reviews to capture and consider all available information, are improving toxicological evaluations performed by various groups and agencies. We reiterate how the EBT framework, a process that provides a method for performing human chemical causation analyses in an objective, transparent and reproducible manner, differs significantly from past and current regulatory WOE approaches. We also discuss why the uncertainties associated with regulatory WOE schemes lead to a definition of the term “risk” that contains unquantifiable uncertainties not present in this term as it is used in epidemiology and medicine. We believe this distinctly different meaning of “risk” should be clearly conveyed to those not familiar with this difference (e.g., the lay public), when theoretical/nomologic risks associated with chemical-induced toxicities are presented outside of regulatory and related scientific parlance.

Endocrine Disruptors: Improving Regulatory Science Policy

Law and science combine in the estimation of risks from endocrine disruptors (EDs) and actions for their regulation. For both, dose–response models are the causal link between exposure and probability (or percentage change) of adverse response. The evidence that leads to either regulations or judicial decrees is affected by uncertainty and limited knowledge, raising difficult policy issues that we enumerate and discuss. In the United States, some courts have dealt with EDs, but causation based on animal studies has been a stumbling block for plaintiffs seeking compensation, principally because those courts opt for epidemiological evidence. The European Union (EU) has several regulatory tools and ongoing research on the risks associated with bisphenol A, under the Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH) Regulation and other regulations or directives. The integration of a vast (in kind and in scope) number of research papers into a statement of causation for either policy or to satisfy legal requirements, in both the United States and the EU, relies on experts. We outline the discursive dilemma and issues that may affect consensus-based results and a Bayesian causal approach that accounts for the evolution of information, yielding both value of information and flexibility associated with public choices.

Effects of atrazine in fish, amphibians, and reptiles: an analysis based on quantitative weight of evidence

A quantitative weight of evidence (WoE) approach was developed to evaluate studies used for regulatory purposes, as well as those in the open literature, that report the effects of the herbicide atrazine on fish, amphibians, and reptiles. The methodology for WoE analysis incorporated a detailed assessment of the relevance of the responses observed to apical endpoints directly related to survival, growth, development, and reproduction, as well as the strength and appropriateness of the experimental methods employed. Numerical scores were assigned for strength and relevance. The means of the scores for relevance and strength were then used to summarize and weigh the evidence for atrazine contributing to ecologically significant responses in the organisms of interest. The summary was presented graphically in a two-dimensional graph which showed the distributions of all the reports for a response. Over 1290 individual responses from studies in 31 species of fish, 32 amphibians, and 8 reptiles were evaluated. Overall, the WoE showed that atrazine might affect biomarker-type responses, such as expression of genes and/or associated proteins, concentrations of hormones, and biochemical processes (e.g. induction of detoxification responses), at concentrations sometimes found in the environment. However, these effects were not translated to adverse outcomes in terms of apical endpoints. The WoE approach provided a quantitative, transparent, reproducible, and robust framework that can be used to assist the decision-making process when assessing environmental chemicals. In addition, the process allowed easy identification of uncertainty and inconsistency in observations, and thus clearly identified areas where future investigations can be best directed.

The Inadmissibility of What We Eat in America and NHANES Dietary Data in Nutrition and Obesity Research and the Scientific Formulation of National Dietary Guidelines

The Scientific Report of the 2015 Dietary Guidelines Advisory Committee was primarily informed by memory-based dietary assessment methods (M-BMs) (eg, interviews and surveys). The reliance on M-BMs to inform dietary policy continues despite decades of unequivocal evidence that M-BM data bear little relation to actual energy and nutrient consumption. Data from M-BMs are defended as valid and valuable despite no empirical support and no examination of the foundational assumptions regarding the validity of human memory and retrospective recall in dietary assessment. We assert that uncritical faith in the validity and value of M-BMs has wasted substantial resources and constitutes the greatest impediment to scientific progress in obesity and nutrition research. Herein, we present evidence that M-BMs are fundamentally and fatally flawed owing to well-established scientific facts and analytic truths. First, the assumption that human memory can provide accurate or precise reproductions of past ingestive behavior is indisputably false. Second, M-BMs require participants to submit to protocols that mimic procedures known to induce false recall. Third, the subjective (ie, not publicly accessible) mental phenomena (ie, memories) from which M-BM data are derived cannot be independently observed, quantified, or falsified; as such, these data are pseudoscientific and inadmissible in scientific research. Fourth, the failure to objectively measure physical activity in analyses renders inferences regarding diet-health relationships equivocal. Given the overwhelming evidence in support of our position, we conclude that M-BM data cannot be used to inform national dietary guidelines and that the continued funding of M-BMs constitutes an unscientific and major misuse of research resources.

Improving weight of evidence approaches to chemical evaluations

Federal and other regulatory agencies often use or claim to use a weight of evidence (WoE) approach in chemical evaluation. Their approaches to the use of WoE, however, differ significantly, rely heavily on subjective professional judgment, and merit improvement. We review uses of WoE approaches in key articles in the peer-reviewed scientific literature, and find significant variations. We find that a hypothesis-based WoE approach, developed by Lorenz Rhomberg et al., can provide a stronger scientific basis for chemical assessment while improving transparency and preserving the appropriate scope of professional judgment. Their approach, while still evolving, relies on the explicit specification of the hypothesized basis for using the information at hand to infer the ability of an agent to cause human health impacts or, more broadly, affect other endpoints of concern. We describe and endorse such a hypothesis-based WoE approach to chemical evaluation.

Commentary: Flawed scientific-evidence standards delay diesel regulations

Of 188 government-monitored air toxics, diesel particulate matter (DPM) causes seven times more cancer than all the other 187 air toxics combined, including benzene, lead, and mercury. Yet, DPM is the only air toxic not regulated more stringently under the Clean Air Act, as a hazardous air pollutant (HAP). One reason is that regulators use flawed standards of scientific evidence. The article argues (1) that DPM meets all six specified evidentiary criteria, any one of which is sufficient for HAP regulation and (2) that regulators' standards of evidence for denying HAP status to DPM (no DPM unit-risk estimate, inadequate dose-response data, alleged weak mechanistic data) err logically and scientifically, set the evidence bar too high, delay regulation, and allow 21,000 avoidable DPM deaths annually in the U.S.

Improving Concordance in Environmental Epidemiology: A Three-Part Proposal

In observational research, evidence is usually derived from multiple studies, and any single result is rarely considered sufficient for public health decision making. Despite more than five decades of research and thousands of studies published, the ability to draw robust conclusions regarding the presence or absence of causal links between specific environmental exposures and human health remains limited. To develop policies that are protective of public health and can withstand scrutiny, agencies need to rely on investigations of satisfactory quality that follow sufficiently concordant protocols in terms of exposure assessment, outcome ascertainment, data analysis, and reporting of results. Absent such concordance, the ability of environmental epidemiology studies to inform decision making is greatly diminished. Systems and tools are proposed here to improve concordance among environmental epidemiology studies. Specifically, working systems in place in other fields of research are critically examined and used as guidelines to develop analogous policies and procedures for environmental epidemiology. A three-part path forward toward more concordant, transparent, and readily accessible environmental epidemiology evidence that parallels ongoing efforts in medical research is proposed. The three parts address methods for improving quality and accessibility of systematic reviews, access to information on ongoing and completed studies, and principles for reporting. The goals are to increase the value of epidemiological research in public health decision making and to stimulate discussions around solutions proposed herein.

Analyzing precautionary regulation: do precaution, science, and innovation go together?

In this article we argue that the precautionary principle, as applied to the regulation of science and technology, cannot be considered in any general manner inconsistent with the norms and methods of scientific knowledge generation and justification. Moreover, it does not necessarily curtail scientific-technological innovation. Our argument flows from a differentiated view of what precaution in regulation means. We first characterize several of the most relevant interpretations given to the precautionary principle in academic debate and regulatory practice. We then use examples of actual precaution-based regulation to show that, even though science can have varying functions in different circumstances and frames, all of those interpretations recur to scientific method and knowledge, and tend to imply innovation in methods, products, and processes. In fact, the interplay of regulation and innovation in precautionary policy, at least in the case of the interpretations of precaution that our analysis takes into account, could be understood as a way of reconciling the two fundamental science and technology policy functions of promotion and control.

A survey of frameworks for best practices in weight-of-evidence analyses

The National Academy of Sciences (NAS) Review of the Environmental Protection Agency's Draft IRIS Assessment of Formaldehyde proposed a "roadmap" for reform and improvement of the Agency's risk assessment process. Specifically, it called for development of a transparent and defensible methodology for weight-of-evidence (WoE) assessments. To facilitate development of an improved process, we developed a white paper that reviewed approximately 50 existing WoE frameworks, seeking insights from their variations and nominating best practices for WoE analyses of causation of chemical risks. Four phases of WoE analysis were identified and evaluated in each framework: (1) defining the causal question and developing criteria for study selection, (2) developing and applying criteria for review of individual studies, (3) evaluating and integrating evidence and (4) drawing conclusions based on inferences. We circulated the draft white paper to stakeholders and then held a facilitated, multi-disciplinary invited stakeholder workshop to broaden and deepen the discussion on methods, rationales, utility and limitations among the surveyed WoE frameworks. The workshop developed recommendations for improving the conduct of WoE evaluations. Based on the analysis of the 50 frameworks and discussions at the workshop, best practices in conducting WoE analyses were identified for each of the four phases. Many of these best practices noted from the analysis and workshop could be implemented immediately, while others may require additional refinement as part of the ongoing discussions for improving the scientific basis of chemical risk assessments.

Contested evidence: Exposure to competing scientific claims and public support for banning bisphenol A

The public controversy surrounding bisphenol A (BPA) revolves around competing claims about what scientific evidence shows regarding the effects of the chemical on human health. This study uses an experiment embedded within a public opinion survey to test the effects of exposure to such claims on public support for banning the use of BPA in products. Exposure to the claim that "there is not enough scientific evidence that BPA harms human health" reduced support, whereas exposure to the claim that there "is enough scientific evidence" failed to increase support. No effect emerged among those simultaneously exposed to both claims. The "not enough evidence" claim influenced less educated respondents and women but not college-educated respondents or men. Aspects of the underlying structure of opinion also differed depending on which claim(s) respondents received. The results illuminate how members of the public respond to competing scientific claims regarding controversial issues.

Representation and misrepresentation of scientific evidence in contemporary tobacco regulation: a review of tobacco industry submissions to the UK Government consultation on standardised packaging

BACKGROUND

Standardised packaging (SP) of tobacco products is an innovative tobacco control measure opposed by transnational tobacco companies (TTCs) whose responses to the UK government's public consultation on SP argued that evidence was inadequate to support implementing the measure. The government's initial decision, announced 11 months after the consultation closed, was to wait for 'more evidence', but four months later a second 'independent review' was launched. In view of the centrality of evidence to debates over SP and TTCs' history of denying harms and manufacturing uncertainty about scientific evidence, we analysed their submissions to examine how they used evidence to oppose SP.

METHODS AND FINDINGS

We purposively selected and analysed two TTC submissions using a verification-oriented cross-documentary method to ascertain how published studies were used and interpretive analysis with a constructivist grounded theory approach to examine the conceptual significance of TTC critiques. The companies' overall argument was that the SP evidence base was seriously flawed and did not warrant the introduction of SP. However, this argument was underpinned by three complementary techniques that misrepresented the evidence base. First, published studies were repeatedly misquoted, distorting the main messages. Second, 'mimicked scientific critique' was used to undermine evidence; this form of critique insisted on methodological perfection, rejected methodological pluralism, adopted a litigation (not scientific) model, and was not rigorous. Third, TTCs engaged in 'evidential landscaping', promoting a parallel evidence base to deflect attention from SP and excluding company-held evidence relevant to SP. The study's sample was limited to sub-sections of two out of four submissions, but leaked industry documents suggest at least one other company used a similar approach.

CONCLUSIONS

The TTCs' claim that SP will not lead to public health benefits is largely without foundation. The tools of Better Regulation, particularly stakeholder consultation, provide an opportunity for highly resourced corporations to slow, weaken, or prevent public health policies.

Facilitating the use of non-standard in vivo studies in health risk assessment of chemicals: a proposal to improve evaluation criteria and reporting

To improve data availability in health risk assessment of chemicals and fill information gaps there is a need to facilitate the use of non-standard toxicity studies, i.e. studies not conducted according to any standardized toxicity test guidelines. The purpose of this work was to propose criteria and guidance for the evaluation of reliability and relevance of non-standard in vivo studies, which could be used to facilitate systematic and transparent evaluation of such studies for health risk assessment. Another aim was to propose user friendly guidance for reporting of non-standard studies intended to promote an improvement in reporting of studies that could be of use in risk assessment. Requirements and recommendations for the design and execution of in vivo toxicity studies were identified from The Organisation for Economic Co-operation and Development (OECD) test guidelines, and served as basis for the data evaluation criteria and reporting guidelines. Feedback was also collected from experts within the field of toxicity testing and risk assessment and used to construct a two-tiered framework for study evaluation, as well as refine the reporting guidelines. The proposed framework emphasizes the importance of study relevance and an important aspect is to not completely dismiss studies from health risk assessment based on very strict criteria for reliability. The suggested reporting guidelines provide researchers with a tool to fulfill reporting requirements as stated by regulatory agencies. Together, these resources provide an approach to include all relevant data that may fill information gaps and reduce scientific uncertainty in health risk assessment conclusions, and subsequently also in chemical policy decisions.

Regulatory decisions on endocrine disrupting chemicals should be based on the principles of endocrinology

For years, scientists from various disciplines have studied the effects of endocrine disrupting chemicals (EDCs) on the health and wellbeing of humans and wildlife. Some studies have specifically focused on the effects of low doses, i.e. those in the range that are thought to be safe for humans and/or animals. Others have focused on the existence of non-monotonic dose-response curves. These concepts challenge the way that chemical risk assessment is performed for EDCs. Continued discussions have clarified exactly what controversies and challenges remain. We address several of these issues, including why the study and regulation of EDCs should incorporate endocrine principles; what level of consensus there is for low dose effects; challenges to our understanding of non-monotonicity; and whether EDCs have been demonstrated to produce adverse effects. This discussion should result in a better understanding of these issues, and allow for additional dialog on their impact on risk assessment.

The quality of nutrition and cancer reviews: a systematic assessment

The methodological quality of published reviews of nutrition and cancer (2008-2009) and of the carcinogenicity of acrylamide (1999-2009) was systematically assessed. Each review was examined with respect to four characteristics: whether the purpose of the review was explicitly stated, whether a methods section (detailing the methods used to "weigh" the evidence) was included, whether "weight of evidence" methods were described elsewhere in the paper (e.g., in the discussion), and finally, whether references to recognized "weight of evidence" methods were included. In this study, ninety per cent of a systematically selected sample of recent reviews on nutrition and cancer published in 2008-2009 and 74% of reviews on acrylamide on cancer published in 1999-2009 were found to be methodologically troublesome or frankly unsound. Failure of peer review and editorial oversight are possible explanations, suggesting a broad lack of concern about this issue in the scientific community. If peer reviewers in the nutrition and cancer community do not require "weight of evidence" methods, then these methods may not appear in the published reviews. Similarly, if journal editors (or editorial policies) do not require methods sections in literature reviews, then these sections may not appear. The prerogative of the author(s) seems the most likely determinant of whether a systematic approach is used or not in nutrition and cancer reviews.

Agency perspectives on food safety for the products of animal biotechnology

Animal biotechnology represents one subset of tools among a larger set of technologies for potential use to meet increasing world demands for food. Assisted reproductive technologies (ART) such as artificial insemination and embryo transfer continue to make positive contributions in food animal production. The US Food and Drug Administration (FDA) performed a comprehensive risk assessment to identify potential food consumption or animal health risks associated with animal cloning, an emerging ART. At that time, FDA concluded that animal cloning posed no unique risks either to animal health or to food consumption, and food from animal clones and their sexually reproduced offspring required no additional federal regulation beyond that applicable to conventionally bred animals of the species examined. At this time, no new information has arisen that would necessitate a change in FDA's conclusions on food from animal clones or their sexually reproduced offspring. Use of recombinant DNA technologies to produce genetically engineered (GE) animals represents another emerging technology with potential to impact food animal production. In its regulation of GE animals, FDA follows a cumulative, risk-based approach to address scientific questions related to the GE animals. FDA evaluates data and information on the safety, effectiveness and stability of the GE event. FDA carries out its review at several levels (e.g. molecular biology, animal safety, food safety, environmental safety and claim validation). GE animal sponsors provide data to address risk questions for each level. This manuscript discusses FDA's role in evaluation of animal cloning and GE animals.

Current status of the epidemiologic evidence linking polychlorinated biphenyls and non-hodgkin lymphoma, and the role of immune dysregulation

BACKGROUND

Although case-control studies conducted to date have largely affirmed the relationship between polychlorinated biphenyls (PCBs) and non-Hodgkin lymphoma (NHL), occupational cohort studies of PCB-exposed workers have been generally interpreted as negative, thereby raising doubts about a potential causal association. A common theme of immune dysregulation unifies many of NHL's strongest risk factors, and several authors have posited that subclinical immune dysregulation may increase NHL risk by decreasing host resistance, reducing control of cellular proliferation and differentiation, and diminishing tumor surveillance mechanisms.

OBJECTIVES

The goals of this review were a) to evaluate the epidemiological research examining the association between PCB exposure and NHL and discuss the contribution to the weight of evidence of case-control studies and occupational cohort studies; and b) to summarize the evidence for immune dysregulation as a means by which PCBs may cause NHL.

METHODS

We performed a literature search using PubMed and seven additional online biomedical and toxicological referencing libraries to identify literature published through August 2011.

DISCUSSION AND CONCLUSIONS

Overall, we conclude that the weight of evidence supports a causal role of PCBs in lymphomagenesis. The strongest epidemiological evidence for the relationship between PCBs and NHL comes from case-control studies conducted among the general population. Epidemiological and toxicological data demonstrating immunosuppressive and inflammatory effects of PCBs further contribute to the weight of evidence by providing a plausible explanation for how PCBs can cause NHL through immune dysregulation.

Incorporating Low-Dose Epidemiology Data in a Chlorpyrifos Risk Assessment

USEPA assessed whether epidemiology data suggest that fetal or early-life chlorpyrifos exposure causes neurodevelopmental effects and, if so, whether they occur at exposures below those causing the current most sensitive endpoint, 10% inhibition of blood acetyl-cholinesterase (AChE). We previously conducted a hypothesis-based weight-of-evidence analysis and found that a proposed causal association between chlorpyrifos exposure and neurodevelopmental effects in the absence of AChE inhibition does not have a substantial basis in existing animal or in vitro studies, and there is no plausible basis for invoking such effects in humans at their far lower exposure levels. The epidemiology studies fail to show consistent patterns; the few associations are likely attributable to alternative explanations. Human data are inappropriate for a dose-response assessment because biomarkers were only measured at one time point, may reflect exposure to other pesticides, and many values are at or below limits of quantification. When considered with pharmacokinetic data, however, these biomarkers provide information on exposure levels relative to those in experimental studies and indicate a margin of exposure of at least 1,000. Because animal data take into account the most sensitive lifestages, the use of AChE inhibition as a regulatory endpoint is protective of adverse effects in sensitive populations.

Incorporating Low-dose Epidemiology Data in a Chlorpyrifos Risk Assessment

USEPA assessed whether epidemiology data suggest that fetal or early-life chlorpyrifos exposure causes neurodevelopmental effects and, if so, whether they occur at exposures below those causing the current most sensitive endpoint, 10% inhibition of blood acetylcholinesterase (AChE). We previously conducted a hypothesis-based weight-of-evidence analysis and found that a proposed causal association between chlorpyrifos exposure and neurodevelopmental effects in the absence of AChE inhibition does not have a substantial basis in existing animal or in vitro studies, and there is no plausible basis for invoking such effects in humans at their far lower exposure levels. The epidemiology studies fail to show consistent patterns; the few associations are likely attributable to alternative explanations. Human data are inappropriate for a dose-response assessment because biomarkers were only measured at one time point, may reflect exposure to other pesticides, and many values are at or below limits of quantification. When considered with pharmacokinetic data, however, these biomarkers provide information on exposure levels relative to those in experimental studies and indicate a margin of exposure of at least 1,000. Because animal data take into account the most sensitive lifestages, the use of AChE inhibition as a regulatory endpoint is protective of adverse effects in sensitive populations.

Managing scientific uncertainty in medical decision making: the case of the advisory committee on immunization practices

This article explores the question of how scientific uncertainty can be managed in medical decision making using the Advisory Committee on Immunization Practices as a case study. It concludes that where a high degree of technical consensus exists about the evidence and data, decision makers act according to a clear decision rule. If a high degree of technical consensus does not exist and uncertainty abounds, the decision will be based on a variety of criteria, including readily available resources, decision-process constraints, and the available knowledge base, among other things. Decision makers employ a variety of heuristic devices and techniques, thereby employing a pragmatic approach to uncertainty in medical decision making. The article concludes with recommendations for managing scientific uncertainty in medical decision making.

Evidence maps: communicating risk assessments in societal controversies: the case of engineered nanoparticles

The transparent and fair characterization of scientific evidence for reporting the results of a hazard assessment is a demanding task. In this article, we present an approach for characterizing evidence--the evidence map approach. The theoretical starting point is to view evidence characterization as a form of argumentation. Thus, evidence maps are designed to depict the evidence base, the pro and con arguments, and the remaining uncertainties, which together lead experts to their conclusions when summarizing and evaluating the scientific evidence about a potential hazard. To illustrate its use, the evidence maps approach is applied to characterizing the health-relevant effects of engineered nanoparticles. Empirical data from an online survey suggests that the use of evidence maps improves the reporting of hazard assessments. Nonexperts prefer to receive the information included in an evidence map in order to come to an informed judgment. Furthermore, the benefits and limitations of evidence maps are discussed in the light of recent literature on risk communication. Finally, the article underlines the need for further research in order to increase quality of evidence reporting.

Expert review for GHS classification of chemicals on health effects

Intoxication as a result of chemical accidents is a major issue in industrial health. The Globally Harmonized System of Classification and Labelling of Chemicals (GHS) provides a framework for hazard communication on chemicals using labelling or safety data sheets. The GHS will be expected to reduce the number of chemical accidents by communicating the hazards posed and prompting safety measures to be taken. One of the issues which may be a barrier to effective implementation of the GHS results from discrepancies in GHS classifications of chemicals across countries/regions. The main reasons are the differences in information sources used and in the expertise of people making the classification (Classifiers). The GHS requests expert judgment in a weight of evidence (WOE) approach in the application of the criteria of classification. A WOE approach is an assessment method that considers all available information bearing on the determination of toxicity. The quality and consistency of the data, study design, mechanism or mode of action, dose-effect relationships and biological relevance should be taken into account. Therefore, expert review should be necessary to classify chemicals accurately. However, the GHS does not provide any information on the required level of expertise of the Classifiers, definition of who qualifies as an expert, evaluation methods of WOE or data quality, and the timing of expert judgment and the need for updating/re-classification as new information becomes available. In this paper, key methods and issues in expert reviews are discussed. Examples of expert reviews and recommendations for harmonized classification are also presented.

Is exposure to formaldehyde in air causally associated with leukemia?--A hypothesis-based weight-of-evidence analysis

Recent scientific debate has focused on the potential for inhaled formaldehyde to cause lymphohematopoietic cancers, particularly leukemias, in humans. The concern stems from certain epidemiology studies reporting an association, although particulars of endpoints and dosimetry are inconsistent across studies and several other studies show no such effects. Animal studies generally report neither hematotoxicity nor leukemia associated with formaldehyde inhalation, and hematotoxicity studies in humans are inconsistent. Formaldehyde's reactivity has been thought to preclude systemic exposure following inhalation, and its apparent inability to reach and affect the target tissues attacked by known leukemogens has, heretofore, led to skepticism regarding its potential to cause human lymphohematopoietic cancers. Recently, however, potential modes of action for formaldehyde leukemogenesis have been hypothesized, and it has been suggested that formaldehyde be identified as a known human leukemogen. In this article, we apply our hypothesis-based weight-of-evidence (HBWoE) approach to evaluate the large body of evidence regarding formaldehyde and leukemogenesis, attending to how human, animal, and mode-of-action results inform one another. We trace the logic of inference within and across all studies, and articulate how one could account for the suite of available observations under the various proposed hypotheses. Upon comparison of alternative proposals regarding what causal processes may have led to the array of observations as we see them, we conclude that the case for a causal association is weak and strains biological plausibility. Instead, apparent association between formaldehyde inhalation and leukemia in some human studies is better interpreted as due to chance or confounding.

Hypothesis-driven weight of evidence framework for evaluating data within the US EPA's Endocrine Disruptor Screening Program

"Weight of Evidence" (WoE) approaches are often used to critically examine, prioritize, and integrate results from different types of studies to reach general conclusions. For assessing hormonally active agents, WoE evaluations are necessary to assess screening assays that identify potential interactions with components of the endocrine system, long-term reproductive and developmental toxicity tests that define adverse effects, mode of action studies aimed at identifying toxicological pathways underlying adverse effects, and toxicity, exposure and pharmacokinetic data to characterize potential risks. We describe a hypothesis-driven WoE approach for hormonally active agents and illustrate the approach by constructing hypotheses for testing the premise that a substance interacts as an agonist or antagonist with components of estrogen, androgen, or thyroid pathways or with components of the aromatase or steroidogenic enzyme systems for evaluating data within the US EPA's Endocrine Disruptor Screening Program. Published recommendations are used to evaluate data validity for testing each hypothesis and quantitative weightings are proposed to reflect two data parameters. Relevance weightings should be derived for each endpoint to reflect the degree to which it probes each specific hypothesis. Response weightings should be derived based on assay results from the test substance compared to the range of responses produced in the assay by the appropriate prototype hormone and positive and negative controls. Overall WoE scores should be derived based on response and relevance weightings and a WoE narrative developed to clearly describe the final determinations.

Why and how to combine evidence in environmental assessments: weighing evidence and building cases

All types of environmental decisions benefit from assessments that assemble and analyze diverse evidence. The diversity of that evidence creates complexities that can be managed using an explicit, well-designed process. We suggest two adaptations from the legal lexicon, weight of evidence and building a case. When weighing evidence, weights are assigned to each piece of evidence, and then the body of evidence is weighed in favor of each hypothesis by amassing the weights. Finally, the total weights of evidence for the alternative hypotheses are compared to determine which alternative has the preponderance of evidence in its favor. When building a case, pieces of evidence are organized to show relationships among multiple hypotheses or complex interactions among agents, events, or processes. We provide processes for weighing evidence and building a case and illustrate both approaches in a case study involving the decline of a kit fox population. The general approach presented here is flexible, transparent, and defensible. During its development, it has been applied to risk assessments for contaminated sites and to causal assessments in aquatic and terrestrial systems. It is intended to balance the need for rigor and discipline with the need for sufficient flexibility to accept all relevant evidence and generate creative solutions to difficult environmental problems.

Correlating neurobehavioral performance with biomarkers of organophosphorous pesticide exposure

There is compelling evidence that adverse neurobehavioral effects are associated with occupational organophosphorous pesticide (OP) exposure in humans. Behavioral studies of pesticide applicators, greenhouse workers, agricultural workers and farm residents exposed repeatedly over months or years to low levels of OPs reveal a relatively consistent pattern of neurobehavioral deficits. However, only two studies have demonstrated a link between neurobehavioral performance and current biomarkers of OP exposure including blood cholinesterase (ChE) activity and urinary levels of OP metabolites. A variety of reasons may explain why so few studies have reported such correlations, including differing individual and group exposure histories, differing methodologies for assessing behavior and exposure, and lack of a reliable index of exposure. Alternatively, these data may suggest that current biomarkers (ChE, urine metabolites) are neither predictive nor diagnostic of the neurobehavioral effects of chronic OP pesticide exposures. This review focuses on the evidence that neurobehavioral performance deficits are associated with occupational OP pesticide exposure and concludes that research needs to return to the basics and rigorously test the relationships between neurobehavioral performance and both current (ChE and urine metabolites) and novel (e.g., inflammation and oxidative stress) biomarkers using human and animal models. The results of such studies are critically important because OP pesticides are widely and extensively used throughout the world, including situations where exposure controls and personal protective equipment are not routinely used.

Better environmental regulation--contributions from risk-based decision-making

Internationally, pressure is being exerted on governments and regulators to develop modern forms of regulation that deliver more for less, and in better ways. We discuss the ways in which one large regulator, the Environment Agency for England and Wales, has responded to such pressures by implementing risk-based approaches to regulation. After exploring the context for key elements of risk-based environmental regulation, we consider the evolving influence of such approaches. We discuss the impacts against the UK Government's principles for better regulation and the key criteria for policy evaluation, before considering some of the challenges: the need to i) understand the best practice and promote consistency in risk-based regulation; ii) develop reliable, responsive forms of risk assessment and monitoring; iii) build capacities for responsive risk regulation; iv) evaluate the influence of different regulatory styles; and v) better understand the potential role of the private sector.

Weight-of-evidence evaluation in environmental assessment: review of qualitative and quantitative approaches

Assessments of human health and ecological risk draw upon multiple types and sources of information, requiring the integration of multiple lines of evidence before conclusions may be reached. Risk assessors often make use of weight-of-evidence (WOE) approaches to perform the integration, whether integrating evidence concerning potential carcinogenicity, toxicity, and exposure from chemicals at a contaminated site, or evaluating processes concerned with habitat loss or modification when managing a natural resource. Historically, assessors have relied upon qualitative WOE approaches, such as professional judgment, or limited quantitative methods, such as direct scoring, to develop conclusions from multiple lines of evidence. Current practice often lacks transparency resulting in risk estimates lacking quantified uncertainty. This paper reviews recent applications of weight of evidence used in human health and ecological risk assessment. Applications are sorted based on whether the approach relies on qualitative and quantitative methods in order to reveal trends in the use of the term weight of evidence, especially as a means to facilitate structured and transparent development of risk conclusions from multiple lines of evidence.

New views on the hypothesis of respiratory cancer risk from soluble nickel exposure; and reconsideration of this risk's historical sources in nickel refineries

INTRODUCTION

While epidemiological methods have grown in sophistication during the 20th century, their application in historical occupational (and environmental) health research has also led to a corresponding growth in uncertainty in the validity and reliability of the attribution of risk in the resulting studies, particularly where study periods extend back in time to the immediate postwar era (1945-70) when exposure measurements were sporadic, unsystematically collected and primitive in technique; and, more so, to the pre-WWII era (when exposure data were essentially non-existent). These uncertainties propagate with animal studies that are designed to confirm the carcinogenicity by inhalation exposure of a chemical putatively responsible for historical workplace cancers since exact exposure conditions were never well characterized. In this report, we present a weight of scientific evidence examination of the human and toxicological evidence to show that soluble nickel is not carcinogenic; and, furthermore, that the carcinogenic potencies previously assigned by regulators to sulphidic and oxidic nickel compounds for the purposes of developing occupational exposure limits have likely been overestimated.

METHODS

Published, file and archival evidence covering the pertinent epidemiology, biostatistics, confounding factors, toxicology, industrial hygiene and exposure factors, and other risky exposures were examined to evaluate the soluble nickel carcinogenicity hypothesis; and the likely contribution of a competing workplace carcinogen (arsenic) on sulphidic and oxidic nickel risk estimates.

FINDINGS

Sharp contrasts in available land area and topography, and consequent intensity of production and refinery process layouts, likely account for differences in nickel species exposures in the Kristiansand (KNR) and Port Colborne (PCNR) refineries. These differences indicate mixed sulphidic and oxidic nickel and arsenic exposures in KNR's historical electrolysis department that were previously overlooked in favour of only soluble nickel exposure; and the absence of comparable insoluble nickel exposures in PCNR's tankhouse, a finding that is consistent with the absence of respiratory cancer risk there. The most recent KNR evidence linking soluble nickel with lung cancer risk arose in a reconfiguration of KNR's historical exposures. But the resulting job exposure matrix lacks an objective, protocol-driven rationale that could provide a valid and reliable basis for analyzing the relationship of KNR lung cancer risk with any nickel species. Evidence of significant arsenic exposure during the processing step in the Clydach refinery's hydrometallurgy department in the 1902-1934 time period likely accounts for most of the elevated respiratory cancer risk observed at that time. An understanding of the mechanism for nickel carcinogenicity remains an elusive goal of toxicological research; as does its capacity to confirm the human health evidence on this subject with animal studies.

CONCLUDING REMARKS

Epidemiological methods have failed to accurately identify the source(s) of observed lung cancer risk in at least one nickel refinery (KNR). This failure, together with the negative long-term animal inhalation studies on soluble nickel and other toxicological evidence, strongly suggest that the designation of soluble nickel as carcinogenic should be reconsidered, and that the true causes of historical lung cancer risk at certain nickel refineries lie in other exposures, including insoluble nickel compounds, arsenic, sulphuric acid mists and smoking.

Better environmental decision making - recent progress and future trends

Recent trends in risk-based decision making are reviewed in relation to novel developments in comparative risk analysis, strategic risk analysis, weight of evidence frameworks, and participative decision making. Delivery of these innovations must take account of organisational capabilities in risk management and the institutional culture that implements decision on risk. We stress the importance of managing risk knowledge within organisations, and emphasise the use of core criteria for effective risk-based decisions by reference to decision process, implementation and the security of strategic added value.

Mesothelioma in patients with nonoccupational asbestos exposure

The specific parameters of nonoccupational asbestos exposures (NOAE) that can distinguish an idiopathic from an asbestos-caused malignant mesothelioma (MM) are controversial. A systematic literature review yielded 1028 cases with this putative association. Only 287 of those reports had a defined single exposure to a household, building occupant, or neighborhood/community asbestos source. The available "evidence" was used to develop semiarbitrary evidence-based causation guideline rules for the assessment of putative associations between MM and NOAE. The rules are classified into class A (tissue burden analysis shows asbestos body counts or fiber counts in lung tissues comparable to MM caused by occupational exposure to asbestos) and classes B to D based on whether certain combinations of NOAE features and MM (evidence) have been described in over 15% (class B), 5% to 15% (class C), and less than 5% (class D) of the patients reviewed. The proposed 4 classes of evidence-based causation guidelines provide a semiarbitrary framework to evaluate the causation of individual MM patients by NOAE based on decreasing levels of currently available evidence. The neoplasms in classes A to C patients are probably caused by NOAE, with decreasing weight of evidence in the 3 groups. There is minimal evidence to support the causation of MM by NOAE in class D patients. There is no evidence or only anecdotal evidence to support a causal association between MM and NOAE in individuals who cannot be classified into any of the 4 classes. Future studies are needed to provide more comprehensive data regarding the association between MM and NOAE.

Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology

Starting with the proposition that "if we want more evidence-based practice, we need more practice-based evidence," this article (a) offers questions and guides that practitioners, program planners, and policy makers can use to determine the applicability of evidence to situations and populations other than those in which the evidence was produced (generalizability), (b) suggests criteria that reviewers can use to evaluate external validity and potential for generalization, and (c) recommends procedures that practitioners and program planners can use to adapt evidence-based interventions and integrate them with evidence on the population and setting characteristics, theory, and experience into locally appropriate programs. The development and application in tandem of such questions, guides, criteria, and procedures can be a step toward increasing the relevance of research for decision making and should support the creation and reporting of more practice-based research having high external validity.