Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations

Applying a modified systematic review and integrated assessment framework (SYRINA) - a case study on triphenyl phosphate

This work presents a case study in applying a systematic review framework (SYRINA) to the identification of chemicals as endocrine disruptors. The suitability and performance of the framework is tested with regard to the widely accepted World Health Organization definition of an endocrine disruptor (ED). The endocrine disrupting potential of triphenyl phosphate (TPP), a well-studied flame retardant reported to exhibit various endocrine related effects was assessed. We followed the 7 steps of the SYRINA framework, articulating the research objective via Populations, Exposures, Comparators, Outcomes (PECO) statements, performed literature search and screening, conducted study evaluation, performed data extraction and summarized and integrated the evidence. Overall, 66 studies, consisting of in vivo, in vitro and epidemiological data, were included. We concluded that triphenyl phosphate could be identified as an ED based on metabolic disruption and reproductive function. We found that the tools used in this case study and the optimizations performed on the framework were suitable to assess properties of EDs. A number of challenges and areas for methodological development in systematic appraisal of evidence relating to endocrine disrupting potential were identified; significant time and effort were needed for the analysis of in vitro mechanistic data in this case study, thus increasing the workload and time needed to perform the systematic review process. Further research and development of this framework with regards to grey literature (non-peer-reviewed literature) search, harmonization of study evaluation methods, more consistent evidence integration approaches and a pre-defined method to assess links between adverse effect and endocrine activity are recommended. It would also be advantageous to conduct more case studies for a chemical with less data than TPP.

Systematic evaluation of the evidence for identification of endocrine disrupting properties of Bisphenol F

Identification of endocrine disruptors (EDs) is a highly prioritized issue in the EU. However, scientific criteria to identify EDs have so far only been implemented for biocidal and plant protection products. The European Commission is working on developing a horizontal approach to the identification of EDs across legislations, based on these scientific criteria. With this study, our aim was to evaluate evidence on endocrine disrupting properties of Bisphenol F (BPF) by applying the process set out for biocidal and plant protection products in Europe. BPF is not registered under REACH and therefore assumed not to be produced in the EU > 1 ton/year, yet the substance has been detected in urine, serum, and breast milk in populations from different countries in Europe. BPF is raising concern since it is an analogue of the known ED and reproductive toxicant Bisphenol A. Relevant evidence on endocrine disrupting properties of BPF from the open literature was collected using systematic review methodology. Pre-defined inclusion criteria were developed to select relevant studies, and data was extracted. The reliability of included studies was evaluated by the Science in Risk Assessment and Policy tool, and results were converted into Klimisch categories to allow for categorization of study reliability. A weight-of-evidence approach was used to analyze the evidence and draw conclusions on endocrine-related activity and/or endocrine adversity. We found that there is sufficient evidence to conclude on an endocrine mechanism, and while evidence for adversity was not considered sufficient, we still conclude that BPF could also cause endocrine-mediated adversity. Two modes of action were postulated based on the collected data for BPF. Challenges of performing the ED assessment for data-poor chemicals and the importance of adequate reporting of studies in the open literature, especially for new approach methods, are discussed.

Assessment of endocrine disruptive properties of PFOS: EFSA/ECHA guidance case study utilising AOP networks and alternative methods

Endocrine disruptors (EDs) are chemical substances that interfere with the endocrine system, adversely affecting human health and environment. Legislation with aim to eliminate and ban EDs have been introduced in EU, but the identification of EDs remains challenging and crucial step towards regulation and risk management. A guidance for ED assessment has been recently established for pesticides and biocides in the EU, which heavily relies on traditional toxicological testing in vivo. Most notably lacking mechanistic methods for some ED modalities and not covering many other modalities that might be affected by EDs. In this project, we focus on the ED assessment according to the valid legislation and explore the possibility to employ alternative methods to bolster the mechanistic understanding of the ED effects and eventually decrease the need for in vivo testing. We selected a well‐studied industrial chemical perfluorooctanesulfonic acid (PFOS) to be a model compound in a case study for ED assessment where the EU criteria were applied in the frame of human health risk assessment with focus on thyroid disruption and developmental neurotoxicity. A systematic literature review has been conducted for these effects (Scopus, Pubmed, Embase), and relevant studies were selected by title/abstract screening (RAYYAN) and full‐text examination. Selected studies were assessed for reliability (SciRAP), and all relevant data were extracted into a database and assessed by Weight of Evidence (WoE) approach. The initial analysis showed potential endocrine adverse effects and endocrine activity, meeting the ED criteria. The use of mechanistic and alternative methods enhanced the outcomes of WoE assessment. Also, the study provides a great hands‐on experience with the most up‐to‐date development in the area of risk assessment and EDs.

Biological plausibility in environmental health systematic reviews: a GRADE concept paper

BACKGROUND

"Biological plausibility" is a concept frequently referred to in environmental and public health when researchers are evaluating how confident they are in the results and inferences of a study or evidence review. Biological plausibility is not, however, a domain of one of the most widely-used approaches for assessing the certainty of evidence (CoE) which underpins the findings of a systematic review, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) CoE Framework. Whether the omission of biological plausibility is a potential limitation of the GRADE CoE Framework is a topic that is regularly discussed, especially in the context of environmental health systematic reviews.

OBJECTIVES

We analyse how the concept of "biological plausibility", as applied in the context of assessing certainty of the evidence that supports the findings of a systematic review, is accommodated under the processes of systematic review and the existing GRADE domains.

RESULTS AND DISCUSSION

We argue that "biological plausibility" is a concept which primarily comes into play when direct evidence about the effects of an exposure on a population of concern (usually humans) is absent, at high risk of bias, is inconsistent, or limited in other ways. In such circumstances, researchers look toward evidence from other study designs in order to draw conclusions. In this respect, we can consider experimental animal and in vitro evidence as "surrogates" for the target populations, exposures, comparators and outcomes of actual interest. Through discussion of 10 examples of experimental surrogates, we propose that the concept of biological plausibility consists of two principal aspects: a "generalisability aspect" and a "mechanistic aspect". The "generalisability aspect" concerns the validity of inferences from experimental models to human scenarios, and asks the same question as does the assessment of external validity or indirectness in systematic reviews. The "mechanistic aspect" concerns certainty in knowledge of biological mechanisms and would inform judgements of indirectness under GRADE, and thus the overall CoE. While both aspects are accommodated under the indirectness domain of the GRADE CoE Framework, further research is needed to determine how to use knowledge of biological mechanisms in the assessment of indirectness of the evidence in systematic reviews.

Assessing the quality of evidence in studies estimating prevalence of exposure to occupational risk factors: The QoE-SPEO approach applied in the systematic reviews from the WHO/ILO Joint Estimates of the Work-related burden of disease and Injury

BACKGROUND

The World Health Organization (WHO) and the International Labour Organization (ILO) have produced the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury (WHO/ILO Joint Estimates). For these, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors have been conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methodology is to assess the quality of evidence across studies. In this article, we present the approach applied in these WHO/ILO systematic reviews for performing such assessments on studies of prevalence of exposure. It is called the Quality of Evidence in Studies estimating Prevalence of Exposure to Occupational risk factors (QoE-SPEO) approach. We describe QoE-SPEO's development to date, demonstrate its feasibility reporting results from pilot testing and case studies, note its strengths and limitations, and suggest how QoE-SPEO should be tested and developed further.

METHODS

Following a comprehensive literature review, and using expert opinion, selected existing quality of evidence assessment approaches used in environmental and occupational health were reviewed and analysed for their relevance to prevalence studies. Relevant steps and components from the existing approaches were adopted or adapted for QoE-SPEO. New steps and components were developed. We elicited feedback from other systematic review methodologists and exposure scientists and reached consensus on the QoE-SPEO approach. Ten individual experts pilot-tested QoE-SPEO. To assess inter-rater agreement, we counted ratings of expected (actual and non-spurious) heterogeneity and quality of evidence and calculated a raw measure of agreement (P_i) between individual raters and rater teams for the downgrade domains. P_i ranged between 0.00 (no two pilot testers selected the same rating) and 1.00 (all pilot testers selected the same rating). Case studies were conducted of experiences of QoE-SPEO's use in two WHO/ILO systematic reviews.

RESULTS

We found no existing quality of evidence assessment approach for occupational exposure prevalence studies. We identified three relevant, existing approaches for environmental and occupational health studies of the effect of exposures. Assessments using QoE-SPEO comprise three steps: (1) judge the level of expected heterogeneity (defined as non-spurious variability that can be expected in exposure prevalence, within or between individual persons, because exposure may change over space and/or time), (2) assess downgrade domains, and (3) reach a final rating on the quality of evidence. Assessments are conducted using the same five downgrade domains as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach: (a) risk of bias, (b) indirectness, (c) inconsistency, (d) imprecision, and (e) publication bias. For downgrade domains (c) and (d), the assessment varies depending on the level of expected heterogeneity. There are no upgrade domains. The QoE-SPEO's ratings are "very low", "low", "moderate", and "high". To arrive at a final decision on the overall quality of evidence, the assessor starts at "high" quality of evidence and for each domain downgrades by one or two levels for serious concerns or very serious concerns, respectively. In pilot tests, there was reasonable agreement in ratings for expected heterogeneity; 70% of raters selected the same rating. Inter-rater agreement ranged considerably between downgrade domains, both for individual rater pairs (range P_i: 0.36-1.00) and rater teams (0.20-1.00). Sparse data prevented rigorous assessment of inter-rater agreement in quality of evidence ratings.

CONCLUSIONS

We present QoE-SPEO as an approach for assessing quality of evidence in prevalence studies of exposure to occupational risk factors. It has been developed to its current version (as presented here), has undergone pilot testing, and was applied in the systematic reviews for the WHO/ILO Joint Estimates. While the approach requires further testing and development, it makes steps towards filling an identified gap, and progress made so far can be used to inform future work in this area.

Biological plausibility in environmental health systematic reviews: a GRADE concept paper

BACKGROUND

"Biological plausibility" is a concept frequently referred to in environmental and public health when researchers are evaluating how confident they are in the results and inferences of a study or evidence review. Biological plausibility is not, however, a domain of one of the most widely-used approaches for assessing the certainty of evidence (CoE) which underpins the findings of a systematic review, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) CoE Framework. Whether the omission of biological plausibility is a potential limitation of the GRADE CoE Framework is a topic that is regularly discussed, especially in the context of environmental health systematic reviews.

OBJECTIVES

We analyse how the concept of "biological plausibility", as applied in the context of assessing certainty of the evidence that supports the findings of a systematic review, is accommodated under the processes of systematic review and the existing GRADE domains.

RESULTS AND DISCUSSION

We argue that "biological plausibility" is a concept which primarily comes into play when direct evidence about the effects of an exposure on a population of concern (usually humans) is absent, at high risk of bias, is inconsistent, or limited in other ways. In such circumstances, researchers look toward evidence from other study designs in order to draw conclusions. In this respect, we can consider experimental animal and in vitro evidence as "surrogates" for the target populations, exposures, comparators and outcomes of actual interest. Through discussion of 10 examples of experimental surrogates, we propose that the concept of biological plausibility consists of two principal aspects: a "generalisability aspect" and a "mechanistic aspect". The "generalisability aspect" concerns the validity of inferences from experimental models to human scenarios, and asks the same question as does the assessment of external validity or indirectness in systematic reviews. The "mechanistic aspect" concerns certainty in knowledge of biological mechanisms and would inform judgements of indirectness under GRADE, and thus the overall CoE. While both aspects are accommodated under the indirectness domain of the GRADE CoE Framework, further research is needed to determine how to use knowledge of biological mechanisms in the assessment of indirectness of the evidence in systematic reviews.

Development of an Adverse Outcome Pathway for Intrahepatic Cholestasis of Pregnancy

•

Adverse outcome pathways are an established method for knowledge synthesis in toxicology.

•

This method has not yet been applied in the context of clinical research.

•

There are opportunities to apply AOPs to the integration and synthesis of biomedical research.

•

The AOP developed in this paper is an example of how the method could be used to investigate clinical questions.

Adverse Outcome Pathways (AOPs) are a research synthesis tool, used primarily by toxicologists for numerous applications including: hypothesis generation, data integration, biomarker determination, and identification of gaps in current knowledge. The AOP model provides a means for evaluating critical interactions between stressors and biological systems which result in adversity, meaning there is significant potential value in using this model in clinical research. However, AOPs have so far not been applied in this context, which may be attributable to the fact that the method is not yet streamlined with established practices in evidence-based medicine, such as systematic review.

Here, we present one approach to developing a clinically focused AOP for intrahepatic cholestasis of pregnancy; aiming to enhance understanding of the mechanistic link between this common, gestational liver disease and its association with preterm birth.

Mechanistic aspects of the disease pathogenesis, and use of AOPs to broaden inclusion and improve integration of in vitro and in vivo data in clinical research are discussed.

We also demonstrate for the first time how central components of systematic review can be integrated into the development of an AOP.

Key Characteristics of Human Hepatotoxicants as a Basis for Identification and Characterization of the Causes of Liver Toxicity

Hazard identification regarding adverse effects on the liver is a critical step in safety evaluations of drugs and other chemicals. Current testing paradigms for hepatotoxicity rely heavily on preclinical studies in animals and human data (epidemiology and clinical trials). Mechanistic understanding of the molecular and cellular pathways that may cause or exacerbate hepatotoxicity is well advanced and holds promise for identification of hepatotoxicants. One of the challenges in translating mechanistic evidence into robust decisions about potential hepatotoxicity is the lack of a systematic approach to integrate these data to help identify liver toxicity hazards. Recently, marked improvements were achieved in the practice of hazard identification of carcinogens, female and male reproductive toxicants, and endocrine disrupting chemicals using the key characteristics approach. Here, we describe the methods by which key characteristics of human hepatotoxicants were identified and provide examples for how they could be used to systematically identify, organize, and use mechanistic data when identifying hepatotoxicants.

Development of the SciRAP Approach for Evaluating the Reliability and Relevance of in vitro Toxicity Data

Efficient and successful integration of data generated from non-animal test methods must rely on reliable and relevant data. It is important therefore to develop tools and criteria that facilitate scientifically sound, structured, and transparent evaluation of reliability and relevance of in vitro toxicity data to efficiently inform regulatory hazard and risk assessment. The Science in Risk Assessment and Policy (SciRAP) initiative aims to promote such overarching goals. We present the work to develop and refine the SciRAP tool for evaluation of reliability and relevance of in vitro studies for incorporation on the SciRAP web-based platform (www.scirap.org). In the SciRAP approach, reliability evaluation is based on criteria for reporting quality and methodological quality, and is explicitly separated from relevance evaluation. The SciRAP in vitro tool (version 1.0) was tested and evaluated during an expert test round (April 2019-September 2020) on three in vitro studies by thirty-one experts from regulatory authorities, industry and academia from different geographical areas and with various degree of experience in in vitro research and/or human health risk assessment. In addition, the experts answered an online survey to collect their feedback about the general features and desired characteristics of the tool for further refinement. The SciRAP in vitro tool (version 2.0) was revised based on the outcome of the expert test round (study evaluation and online survey) and consists of 24 criteria for evaluating “reporting quality” (reliability), 16 criteria for “methodological quality” (reliability), and 4 items for evaluating relevance of in vitro studies. Participants were generally positive about the adequacy, flexibility, and user-friendliness of the tool. The expert test round outlined the need to (i) revise the formulation of certain criteria; (ii) provide new or revised accompanying guidance for reporting quality and methodological quality criteria in the “test compounds and controls,” “test system,” and “data collection and analysis” domains; and (iii) provide revised guidance for relevance items, as general measures to reduce inter-expert variability. The SciRAP in vitro tool allows for a structured and transparent evaluation of in vitro studies for use in regulatory hazard and risk assessment of chemicals.

A Systematic Review to Compare Chemical Hazard Predictions of the Zebrafish Embryotoxicity Test With Mammalian Prenatal Developmental Toxicity

Originally developed to inform the acute toxicity of chemicals on fish, the zebrafish embryotoxicity test (ZET) has also been proposed for assessing the prenatal developmental toxicity of chemicals, potentially replacing mammalian studies. Although extensively evaluated in primary studies, a comprehensive review summarizing the available evidence for the ZET's capacity is lacking. Therefore, we conducted a systematic review of how well the presence or absence of exposure-related findings in the ZET predicts prenatal development toxicity in studies with rats and rabbits. A two-tiered systematic review of the developmental toxicity literature was performed, a review of the ZET literature was followed by one of the mammalian literature. Data were extracted using DistillerSR, and study validity was assessed with an amended SYRCLE's risk-of-bias tool. Extracted data were analyzed for each species and substance, which provided the basis for comparing the 2 test methods. Although limited by the number of 24 included chemicals, our results suggest that the ZET has potential to identify chemicals that are mammalian prenatal developmental toxicants, with a tendency for overprediction. Furthermore, our analysis confirmed the need for further standardization of the ZET. In addition, we identified contextual and methodological challenges in the application of systematic review approaches to toxicological questions. One key to overcoming these challenges is a transition to more comprehensive and transparent planning, conduct and reporting of toxicological studies. The first step toward bringing about this change is to create broad awareness in the toxicological community of the need for and benefits of more evidence-based approaches.

Reviews in environmental health: How systematic are they?

BACKGROUND

Synthesizing environmental health science is crucial to taking action to protect public health. Procedures for evidence evaluation and integration are transitioning from "expert-based narrative" to "systematic" review methods. However, little is known about the methodology being utilized for either type of review.

OBJECTIVES

To appraise the methodological strengths and weaknesses of a sample of "expert-based narrative" and "systematic" reviews in environmental health.

METHODS

We conducted a comprehensive search of multiple databases and identified relevant reviews using pre-specified eligibility criteria. We applied a modified version of the Literature Review Appraisal Toolkit (LRAT) to three environmental health topics that assessed the utility, validity and transparency of reviews.

RESULTS

We identified 29 reviews published between 2003 and 2019, of which 13 (45%) were self-identified as systematic reviews. Across every LRAT domain, systematic reviews received a higher percentage of "satisfactory" ratings compared to non-systematic reviews. In eight of these domains, there was a statistically significant difference observed between the two types of reviews and "satisfactory" ratings. Non-systematic reviews performed poorly with the majority receiving an "unsatisfactory" or "unclear" rating in 11 of the 12 domains. Systematic reviews performed poorly in six of the 12 domains; 10 (77%) did not state the reviews objectives or develop a protocol; eight (62%) did not state the roles and contribution of the authors, or evaluate the internal validity of the included evidence consistently using a valid method; and only seven (54%) stated a pre-defined definition of the evidence bar on which their conclusions were based, or had an author disclosure of interest statement.

DISCUSSION

Systematic reviews produced more useful, valid, and transparent conclusions compared to non-systematic reviews, but poorly conducted systematic reviews were prevalent. Ongoing development and implementation of empirically based systematic review methods are required in environmental health to ensure transparent and timely decision making to protect the public's health.

Towards guidelines for time-trend reviews examining temporal variability in human biomonitoring data of pollutants

In the last few decades, a plethora of studies have focused on human biomonitoring (HBM) of chemical pollutants. Reviewing the copious HBM data reported in these studies is essential for evaluating the effectiveness of pollution management efforts, for example by evaluating time-trends. Nevertheless, guidance to systematically evaluate time trends in published HBM data has never been developed. In this study, we therefore present a proposal for guidelines to conduct "time-trend reviews" (TTRs) that examine time trends in published large HBM datasets of chemical pollutant concentrations. We also demonstrate the applicability of these guidelines through a case study that assesses time-trends in global and regional HBM data on mercury. The recommended TTR guidelines in this study are divided into seven steps: formulating the objective of the TTR, setting up of eligibility criteria, defining search strategy and screening of literature, screening results of search, extracting data, analysing data, and assessing certainty, including the potential for bias in the evidence base. The TTR guidelines proposed in this study are straightforward and less complex than those for conducting systematic reviews assessing datasets on potential human health effects of exposure to pollutants or medical interventions. These proposed guidelines are intended to enable the credible, transparent, and reproducible conduct of TTRs.

Assessment of the endocrine disrupting properties of bisphenol AF: a case study applying the European regulatory criteria and guidance

BACKGROUND

Scientific criteria to identify endocrine disruptors (ED) was recently implemented for plant protection products (PPP) and biocidal products (BP). A guidance document has been published by ECHA and EFSA in the context of ED criteria for PPPs and BPs.

METHODS

In the present work, a case study was performed on Bisphenol AF (BPAF) to explore the application of the EU criteria and EFSA/ECHA guidance document for the ED assessment of a non-pesticide chemical regulated under REACH. A data dossier was built by a systematic literature search (Web of Science, Pubmed, Embase; n = 511), title/abstract screening (n = 124) and full text examination (n = 88). All the information was extracted and systematically reported for 309 parameters (100 for adversity; 209 for endocrine activity). The reliability of studies was assessed (SciRAP tool).

RESULTS

Data were synthesized into 96 lines of evidence for adversity (n = 57), and endocrine activity (n = 39); and assessed by weight of evidence methodology. The initial analysis of the evidence indicated EATS-mediated adversity in mammals, therefore a mode of action (MoA) was postulated for both male and female adult exposure. Female MoA included estrogen receptor activation and altered steroidogenesis leading to ovarian dysfunction, altered estrous cycling and impaired female fertility. Male MoA was initiated by androgen receptor inhibition and altered steroidogenesis leading to dysfunction of male reproductive organs and impaired male fertility.

CONCLUSIONS

The overall conclusion of the ED assessment indicated that BPAF meets the ED criteria for human health. The steps described in the ED guidance document were successfully completed, resulting in a thorough, structured and transparent identification of BPAF as an ED. Advantages and limitations of applying the ED criteria and guidance for a REACH chemical are discussed.

Suggestions for Improving the Characterization of Risk from Exposures to Per and Polyfluorinated Alkyl Substances

Many state and federal environmental and health agencies have developed risk-based criteria for assessing the risk of adverse health effects of per- and polyfluorinated alkyl substances (PFAS) exposure to humans and the environment. However, the criteria that have been developed vary; drinking water criteria developed for perfluorooctanoic acid, for example, can vary by up to 750 fold. This is due to differences and variability in the data and information used, study/endpoint selection, assumptions and magnitude of uncertainty factors used in the absence and extrapolation of critical effect data, differences in underlying approaches to addressing exposure within criteria development, and/or policy decisions on levels of acceptable risk. We have critically evaluated the methods used to develop these criteria while focusing on derivation and application of drinking water criteria and discuss a range of improvements to risk-characterization practice recently presented at a Focused Topic Meeting on PFAS conducted by the Society of Environmental Toxicology and Chemistry in Durham, North Carolina, USA, 12 to 15 August 2019. We propose methods that consider maximizing the use of disparate data streams, seeking patterns, and proposing biologically based approaches to evidence integration toward informed criteria development. Environ Toxicol Chem 2021;40:883-898. © 2020 SETAC.

Evaluating reliability and risk of bias of in vivo animal data for risk assessment of chemicals - Exploring the use of the SciRAP tool in a systematic review context

Within the field of health risk assessment, it is essential that evaluations of reliability or validity of toxicity data are conducted with structure and transparency. To this end, different tools for evaluating toxicity studies have been developed by different groups and organizations, for different specific purposes. The Science in Risk Assessment and Policy (SciRAP) tool was developed for use in the regulatory health risk assessment of chemicals and to promote structured and transparent evaluation of study reliability within European regulatory frameworks. As such, the SciRAP tool is not specifically tailored for use in a systematic review context. However, in light of the current movement towards applying systematic review in the field of environmental health and chemical assessments and European chemicals regulation, we were interested in exploring how SciRAP could be applied in such a context. To achieve this, the scope of the SciRAP tool was first compared to two tools developed based on systematic review principles at the US Environmental Protection Agency's IRIS program and the National Toxicology Program's Office of Health Assessment and Translation (OHAT). Next, the SciRAP and IRIS tools were both applied in a case study to evaluate the same nine in vivo animal studies and the resulting evaluations were compared. The SciRAP tool was found to address the majority of the elements included for study evaluation in the OHAT and IRIS tools. In the case study, no major differences were found in the conclusions drawn when using SciRAP or IRIS tools. However, future developments to bring the SciRAP tool more in line with systematic review principles were identified and are discussed. Overall, this work illustrates the advantages of applying structured and pre-defined methods for study evaluation and provides a unique case study comparing the impact of using different tools for evaluating animal toxicity studies.

Knowledge Organization Systems for Systematic Chemical Assessments

BACKGROUND

Although the implementation of systematic review and evidence mapping methods stands to improve the transparency and accuracy of chemical assessments, they also accentuate the challenges that assessors face in ensuring they have located and included all the evidence that is relevant to evaluating the potential health effects an exposure might be causing. This challenge of information retrieval can be characterized in terms of "semantic" and "conceptual" factors that render chemical assessments vulnerable to the streetlight effect.

OBJECTIVES

This commentary presents how controlled vocabularies, thesauruses, and ontologies contribute to overcoming the streetlight effect in information retrieval, making up the key components of Knowledge Organization Systems (KOSs) that enable more systematic access to assessment-relevant information than is currently achievable. The concept of Adverse Outcome Pathways is used to illustrate what a general KOS for use in chemical assessment could look like.

DISCUSSION

Ontologies are an underexploited element of effective knowledge organization in the environmental health sciences. Agreeing on and implementing ontologies in chemical assessment is a complex but tractable process with four fundamental steps. Successful implementation of ontologies would not only make currently fragmented information about health risks from chemical exposures vastly more accessible, it could ultimately enable computational methods for chemical assessment that can take advantage of the full richness of data described in natural language in primary studies. https://doi.org/10.1289/EHP6994.

Advancing systematic-review methodology in exposure science for environmental health decision making

Systematic review (SR) is a rigorous methodology applied to synthesize and evaluate a body of scientific evidence to answer a research or policy question. Effective use of systematic-review methodology enables use of research evidence by decision makers. In addition, as reliance on systematic reviews increases, the required standards for quality of evidence enhances the policy relevance of research. Authoritative guidance has been developed for use of SR to evaluate evidence in the fields of medicine, social science, environmental epidemiology, toxicology, as well as ecology and evolutionary biology. In these fields, SR is typically used to evaluate a cause-effect relationship, such as the effect of an intervention, procedure, therapy, or exposure on an outcome. However, SR is emerging to be a useful methodology to transparently review and integrate evidence for a wider range of scientifically informed decisions and actions across disciplines. As SR is being used more broadly, there is growing consensus for developing resources, guidelines, ontologies, and technology to make SR more efficient and transparent, especially for handling large amounts of diverse data being generated across multiple scientific disciplines. In this article, we advocate for advancing SR methodology as a best practice in the field of exposure science to synthesize exposure evidence and enhance the value of exposure studies. We discuss available standards and tools that can be applied and extended by exposure scientists and highlight early examples of SRs being developed to address exposure research questions. Finally, we invite the exposure science community to engage in further development of standards and guidance to grow application of SR in this field and expand the opportunities for exposure science to inform environment and public health decision making.

Facilitation of risk assessment with evidence-based methods - A framework for use of systematic mapping and systematic reviews in determining hazard, developing toxicity values, and characterizing uncertainty

Systematic review tools and approaches developed for clinical medicine are often difficult to apply "off the shelf" in order to meet the needs of chemical risk assessments. To address such, we propose an approach that can be used by practitioners for using evidence-based methods to facilitate the risk assessment process. The framework builds on and combines efforts conducted to date by a number of agencies and researchers; the novelty is in combining these efforts with a practical understanding of risk assessment, and translating such into a 'step-by-step' guide. The approach relies on three key components: problem formulation, systematic evidence mapping, and systematic review, applied using a stepwise approach. Unique to this framework is the consideration of exposure in selecting, prioritizing, and evaluating data (e.g., dose-relevance, routes of exposure, etc.). Using the proposed step-by-step process, critical appraisal of individual studies (e.g., formal and structured assessment of both relevance and reliability) and integration efforts are considered in context of specified risk assessment objectives (e.g., mode of action, dose-response) as well as chemical-specific considerations. The resulting framework provides a logical approach of how evidence-based methods can be used to facilitate risk assessment, and elevates the use of systematic methods beyond hazard identification to directly facilitating transparent and objective selection of candidate studies and/or datasets used to quantitatively characterize risk, and to better use the underlying process to inform the approaches used to develop toxicity values.

Recommendations for the conduct of systematic reviews in toxicology and environmental health research (COSTER)

BACKGROUND

There are several standards that offer explicit guidance on good practice in systematic reviews (SRs) for the medical sciences; however, no similarly comprehensive set of recommendations has been published for SRs that focus on human health risks posed by exposure to environmental challenges, chemical or otherwise.

OBJECTIVES

To develop an expert, cross-sector consensus view on a key set of recommended practices for the planning and conduct of SRs in the environmental health sciences.

METHODS

A draft set of recommendations was derived from two existing standards for SRs in biomedicine and developed in a consensus process, which engaged international participation from government, industry, non-government organisations, and academia. The consensus process consisted of a workshop, follow-up webinars, email discussion and bilateral phone calls.

RESULTS

The Conduct of Systematic Reviews in Toxicology and Environmental Health Research (COSTER) recommendations cover 70 SR practices across eight performance domains. Detailed explanations for specific recommendations are made for those identified by the authors as either being novel to SR in general, specific to the environmental health SR context, or potentially controversial to environmental health SR stakeholders.

DISCUSSION

COSTER provides a set of recommendations that should facilitate the production of credible, high-value SRs of environmental health evidence, and advance discussion of a number of controversial aspects of conduct of EH SRs. Key recommendations include the management of conflicts of interest, handling of grey literature, and protocol registration and publication. A process for advancing from COSTER's recommendations to developing a formal standard for EH SRs is also indicated.

A Systematic Literature Review on Safety Research Related to Chemical Industrial Parks

The increasing demand for chemical products has driven the construction and development of chemical industrial areas, or so-called ‘chemical industrial parks’ (CIPs), but this has intrinsically raised the risk of major accidents. Therefore, it is significant and urgent to summarize the state of art and research needs in the field of CIP safety. In this paper, a keyword co-occurrence analysis of 116 scientific articles was conducted to support the classification of research topics in this field, then an overview of those research topics was presented to investigate the evolution of safety research with respect to CIPs. Specifically, the way that safety assessments are conducted, as well as how safety management and safety technology in such areas are classified and investigated, followed by detailed descriptions of representative methods and their contributions to CIP safety, are discussed. An integrated safety framework for CIPs is proposed to organize safety approaches and measures systematically. Based on the classification and analysis of studies on management, assessment, and technology related to CIP safety, the research trends and future directions and challenges are discussed and outlined. Those results are useful for improving theoretical method and industrial strategies, and can advance the safety and sustainability development of CIPs.

A case study applying pathway-oriented thinking to problem formulation for planning a systematic review

The use of evidence-based methods in chemical risk assessment (CRA) is still in its infancy. Novel approaches exploring how to implement Systematic Review (SR) principles and methods for evaluating human health risks from environmental chemical exposures are needed. This paper reports and comments on a conceptual model that was developed as part of a mapping exercise for planning a SR, using aluminium-containing antiperspirants (Al-AP) and female breast cancer risk as a case study. The work explores how knowledge-assembly tools and pathway-oriented thinking developed in systems toxicology can be applied to support problem formulation (PF) in the context of SR. A conceptual model was developed to map out key research questions, working hypotheses, routes of exposure, toxicity pathways and endpoints, and related health outcomes. The model draws on the analytic framework for screening topics of the U.S. Preventive Services Task Force and builds on the concept of a "source-to-outcome continuum", integrating knowledge gained from exposure pathway concepts such as the Aggregate Exposure Pathway and Adverse Outcome Pathways. The model can be used as a central decision and prioritization tool for scoping and framing Population, Exposure, Control, Outcome (PECO) questions in a transparent and iterative manner; as a supporting tool to guide the whole SR process; and to lay down the methodological foundation of a SR on the Al-AP breast cancer topic. Logic modelling can be easily combined with systems or pathway-oriented thinking, and allows for a more structured, objective and transparent approach to PF when applying SR methods to the CRA context.

Development and assessment of an instrument measuring environmental health perception and behavior toward reproductive health of female adolescents

AIM

This study is aimed to develop and test the measurement of environmental health perceptions and behavior concerning female adolescents' reproductive health.

METHODS

The scales constructed through literature review, in -depth interview, and factor analysis based on the protection motivation theory. The sample comprised 384 female adolescents for exploratory factor analysis, aged 18-22 years, and recruited in September-November 2019 in South Korea. The construct was validated using factor analysis, convergent validity, and known-group comparisons. Reliability was examined using Cronbach's alpha for internal consistency and half-split reliability.

RESULTS

The environmental health perception for female adolescents (EHP-FA) comprises "Sensitivity," "Susceptibility," "Response efficacy," and "Self-efficacy." The environmental health behavior for female adolescents (EHB-FA), based on principal component analysis, yielded 19 items with five factors, including "Chemical reduction," "Electromagnetic reduction," "Food selection," "Cosmetic selection," "Dust & Gas reduction," and explained 62.6% of the variance (Cronbach's α = .93).

CONCLUSION

The utility of EHP-FA and EHB-FA recommend assessment of female youth's environmental attention and health behavior in the community. Nursing professionals can use the scales to promote female adolescents' reproductive health.

RoB-SPEO: A tool for assessing risk of bias in studies estimating the prevalence of exposure to occupational risk factors from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury

BACKGROUND

The World Health Organization (WHO) and the International Labour Organization (ILO) are developing joint estimates of the work-related burden of disease and injury (WHO/ILO Joint Estimates). For this, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors will be conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methods is to assess risk of bias (RoB) of individual studies. In this article, we present and describe the development of such a tool, called the Risk of Bias in Studies estimating Prevalence of Exposure to Occupational risk factors (RoB-SPEO) tool; report results from RoB-SPEO's pilot testing; note RoB-SPEO's limitations; and suggest how the tool might be tested and developed further.

METHODS

Selected existing RoB tools used in environmental and occupational health systematic reviews were reviewed and analysed. From existing tools, we identified domains for the new tool and, if necessary, added new domains. For each domain, we then identified and integrated components from the existing tools (i.e. instructions, domains, guiding questions, considerations, ratings and rating criteria), and, if necessary, we developed new components. Finally, we elicited feedback from other systematic review methodologists and exposure scientists and agreed upon RoB-SPEO. Nine experts pilot tested RoB-SPEO, and we calculated a raw measure of inter-rater agreement (Pi) for each of its domain, rating Pi < 0.4 as poor, 0.4 ≤ Pi ≥ 0.8 as substantial and Pi > 0.80 as almost perfect agreement.

RESULTS

Our review found no standard tool for assessing RoB in prevalence studies of exposure to occupational risk factors. We identified six existing tools for environmental and occupational health systematic reviews and found that their components for assessing RoB differ considerably. With the new RoB-SPEO tool, assessors judge RoB for each of eight domains: (1) bias in selection of participants into the study; (2) bias due to a lack of blinding of study personnel; (3) bias due to exposure misclassification; (4) bias due to incomplete exposure data; (5) bias due to conflict of interest; (6) bias due to selective reporting of exposures; (7) bias due to difference in numerator and denominator; and (8) other bias. The RoB-SPEO's ratings are low, probably low, probably high, high or no information. Pilot testing of the RoB-SPEO tool found substantial inter-rater agreement for six domains (range of Pi for these domains: 0.51-0.80), but poor agreement for two domains (i.e. Pi of 0.31 and 0.33 for biases due to incomplete exposure data and in selection of participants into the study, respectively). Limitations of RoB-SPEO include that it has not yet been fully performance-tested.

CONCLUSIONS

We developed the RoB-SPEO tool for assessing RoB in prevalence studies of exposure to occupational risk factors. The tool will be applied and its performance tested in the ongoing systematic reviews for the WHO/ILO Joint Estimates.

Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management

BACKGROUND

While systematic review (SR) methods are gaining traction as a method for providing a reliable summary of existing evidence for health risks posed by exposure to chemical substances, it is becoming clear that their value is restricted to a specific range of risk management scenarios - in particular, those which can be addressed with tightly focused questions and can accommodate the time and resource requirements of a systematic evidence synthesis.

METHODS

The concept of a systematic evidence map (SEM) is defined and contrasted to the function and limitations of systematic review (SR) in the context of risk management decision-making. The potential for SEMs to facilitate evidence-based decision-making are explored using a hypothetical example in risk management priority-setting. The potential role of SEMs in reference to broader risk management workflows is characterised.

RESULTS

SEMs are databases of systematically gathered research which characterise broad features of the evidence base. Although not intended to substitute for the evidence synthesis element of systematic reviews, SEMs provide a comprehensive, queryable summary of a large body of policy relevant research. They provide an evidence-based approach to characterising the extent of available evidence and support forward looking predictions or trendspotting in the chemical risk sciences. In particular, SEMs facilitate the identification of related bodies of decision critical chemical risk information which could be further analysed using SR methods, and highlight gaps in the evidence which could be addressed with additional primary studies to reduce uncertainties in decision-making.

CONCLUSIONS

SEMs have strong and growing potential as a high value tool in resource efficient use of existing research in chemical risk management. They can be used as a critical precursor to efficient deployment of high quality SR methods for characterising chemical health risks. Furthermore, SEMs have potential, at a large scale, to support the sort of evidence summarisation and surveillance methods which would greatly increase the resource efficiency, transparency and effectiveness of regulatory initiatives such as EU REACH and US TSCA.

Scientific integrity issues in Environmental Toxicology and Chemistry: Improving research reproducibility, credibility, and transparency

High-profile reports of detrimental scientific practices leading to retractions in the scientific literature contribute to lack of trust in scientific experts. Although the bulk of these have been in the literature of other disciplines, environmental toxicology and chemistry are not free from problems. While we believe that egregious misconduct such as fraud, fabrication of data, or plagiarism is rare, scientific integrity is much broader than the absence of misconduct. We are more concerned with more commonly encountered and nuanced issues such as poor reliability and bias. We review a range of topics including conflicts of interests, competing interests, some particularly challenging situations, reproducibility, bias, and other attributes of ecotoxicological studies that enhance or detract from scientific credibility. Our vision of scientific integrity encourages a self-correcting culture that promotes scientific rigor, relevant reproducible research, transparency in competing interests, methods and results, and education. Integr Environ Assess Manag 2019;00:000-000. © 2019 SETAC.

A review of methods used for hazard identification and risk assessment of environmental hazards

BACKGROUND

Approximately one quarter of all deaths globally are attributed to living or working in an unhealthy environment, with household and ambient air pollution, along with exposures to ultraviolet radiation and chemicals amongst the leading causes. At present there are no international standards for assessing the risks of these environmental hazards. The use of heterogeneous methods to identify health risks from environmental hazards may reduce the level of confidence the public has in the conclusions that are made.

OBJECTIVES

To describe and compare the processes and methods used by national and international organisations that conduct hazard identification and/or risk assessment (HI/RA) of environmental hazards and to identify knowledge gaps to inform the development of future methods.

METHODS

We searched the websites of 19 organisations (ten national, five international and four World Health Organization (WHO) units) and extracted data from all relevant, publicly available resources which described the processes and methods used in HI/RA of environmental hazards. We contacted each organisation for any additional information.

RESULTS

Five organisations were excluded from further analysis: three made recommendations but did not conduct HI/RA; one used heterogenous methods across their reviews for HI; and one WHO unit did not have any published guidelines. Of the 14 organisations analysed, five (36%) describe the process for establishing the questions to be answered in the assessments. Only one (7%) organisation uses systematic review methods, although five (36%) state that they use such methods. Ten (71%) assess the scientific quality of the included studies, however only three (21%) use explicit criteria. Only three (21%) organisations assess the quality of the body of evidence using explicit criteria. Four (29%) organisations describe the process for making the final HI conclusions and three (38%) the final RA conclusions. Eight (57%) have a conflict of interest policy and seven (50%) organisations describe a process for managing them. The US Office of Health Assessment and Translation and the World Health Organisation meet the most criteria for describing their processes and methods.

CONCLUSIONS

The processes and methods used by organisations conducting HI/RA of environmental hazards are inconsistent. There is a need for empirically based tools and methods to be adopted for the evaluation and synthesis of evidence, and the formulation of conclusions across all organisations that conduct HI or RA. These tools and methods will lead to increased transparency, comparability and validity of the assessments.

Associations between multimorbidity, all-cause mortality and glycaemia in people with type 2 diabetes: A systematic review

Introduction

Type 2 diabetes (T2D) is a major health priority worldwide and the majority of people with diabetes live with multimorbidity (MM) (the co-occurrence of ≥2 chronic conditions). The aim of this systematic review was to explore the association between MM and all-cause mortality and glycaemic outcomes in people with T2D.

Methods

The search strategy centred on: T2D, MM, comorbidity, mortality and glycaemia. Databases searched: MEDLINE, EMBASE, CINAHL Complete, The Cochrane Library, and SCOPUS. Restrictions included: English language, quantitative empirical studies. Two reviewers independently carried out: abstract and full text screening, data extraction, and quality appraisal. Disagreements adjudicated by a third reviewer.

Results

Of the 4882 papers identified; 41 met inclusion criteria. The outcome was all-cause mortality in 16 studies, glycaemia in 24 studies and both outcomes in one study. There were 28 longitudinal cohort studies and 13 cross-sectional studies, with the number of participants ranging from 96–892,223. Included studies were conducted in high or upper-middle-income countries. Fifteen of 17 studies showed a statistically significant association between increasing MM and higher mortality. Ten of 14 studies showed no significant associations between MM and HbA1c. Four of 14 studies found higher levels of MM associated with higher HbA1c. Increasing MM was significantly associated with hypoglycaemia in 9/10 studies. There was no significant association between MM and fasting glucose (one study). No studies explored effects on glycaemic variability.

Conclusions

This review demonstrates that MM in T2D is associated with higher mortality and hypoglycaemia, whilst evidence regarding the association with other measures of glycaemic control is mixed. The current single disease focused approach to management of T2D seems inappropriate. Our findings highlight the need for clinical guidelines to support a holistic approach to the complex care needs of those with T2D and MM, accounting for the various conditions that people with T2D may be living with.

Systematic review registration

International Prospective Register of Systematic Reviews CRD42017079500

Adaptation of the Systematic Review Framework to the Assessment of Toxicological Test Methods: Challenges and Lessons Learned with the Zebrafish Embryotoxicity Test

Systematic review methodology is a means of addressing specific questions through structured, consistent, and transparent examinations of the relevant scientific evidence. This methodology has been used to advantage in clinical medicine, and is being adapted for use in other disciplines. Although some applications to toxicology have been explored, especially for hazard identification, the present preparatory study is, to our knowledge, the first attempt to adapt it to the assessment of toxicological test methods. As our test case, we chose the zebrafish embryotoxicity test (ZET) for developmental toxicity and its mammalian counterpart, the standard mammalian prenatal development toxicity study, focusing the review on how well the ZET predicts the presence or absence of chemical-induced prenatal developmental toxicity observed in mammalian studies. An interdisciplinary team prepared a systematic review protocol and adjusted it throughout this piloting phase, where needed. The final protocol was registered and will guide the main study (systematic review), which will execute the protocol to comprehensively answer the review question. The goal of this preparatory study was to translate systematic review methodology to the assessment of toxicological test method performance. Consequently, it focused on the methodological issues encountered, whereas the main study will report substantive findings. These relate to numerous systematic review steps, but primarily to searching and selecting the evidence. Applying the lessons learned to these challenges can improve not only our main study, but may also be helpful to others seeking to use systematic review methodology to compare toxicological test methods. We conclude with a series of recommendations that, if adopted, would help improve the quality of the published literature, and make conducting systematic reviews of toxicological studies faster and easier over time.

Probabilistic estimates of heterocyclic amines and polycyclic aromatic hydrocarbons concentrations in meats and breads applicable to exposure assessments

Random effect meta-regressions were constructed to estimate concentrations of two heterocyclic amines (HCA) and eight polycyclic aromatic hydrocarbons (PAH) in meat and breads. Eighteen HCA studies and nine PAH studies of food concentration were assembled. Concentration was computed for beef, poultry, pork, and seafood, and bread. Fixed effect predictors included cooking time, form of the food, cooking method, interaction between form and cooking method, temperature at which the food was cooked, fuel of the flame source, percentage of fat, and other elements. Meat type was significant to all HCAs but only three of the PAHs. Cooking method or an interaction between cooking method and food form was significant in all the overall models for each compound, and 80% of models created for stratifications of the data based on meat type. Improvement on compilations such as the Computerized Heterocyclic Amines Resource for Research in Epidemiology of Disease (CHARRED) database comes from inclusion of additional studies, PAH compounds, more generalizable food categories, more cooking methods (such as smoking), and addition of seafood. Meta-regression allows parameters to be estimated with separation of between-study heterogeneity, and extrapolation of exposures to more foods. Resulting uncertainty estimates are useful in a probabilistic exposure assessment.

Role of Risk of Bias in Systematic Review for Chemical Risk Assessment: A Case Study in Understanding the Relationship Between Congenital Heart Defects and Exposures to Trichloroethylene

The National Academy of Science has recommended that a risk of bias (RoB; credibility of the link between exposure and outcome) assessment be conducted on studies that are used as primary data sources for hazard identification and dose-response assessment. Few applications of such have been conducted. Using trichloroethylene and congenital heart defects (CHDs) as a case study, we explore the role of RoB in chemical risk assessment using the National Toxicology Program's Office of Health Assessment and Translation RoB tool. Selected questions were tailored to evaluation of CHD and then applied to 12 experimental animal studies and 9 epidemiological studies. Results demonstrated that the inconsistent findings of a single animal study were likely explained by the limitations in study design assessed via RoB (eg, lack of concurrent controls, unvalidated method for assessing outcome, unreliable statistical methods, etc). Such limitations considered in the context of the body of evidence render the study not sufficiently reliable for the development of toxicity reference values. The case study highlights the utility of RoB as part of a robust risk assessment process and specifically demonstrates the role RoB can play in objectively selecting candidate data sets to develop toxicity values.

Cardiovascular risk from water arsenic exposure in Vietnam: Application of systematic review and meta-regression analysis in chemical health risk assessment

A systematic review (SR) and meta-analysis cannot provide the endpoint answer for a chemical risk assessment (CRA). The objective of this study was to apply SR and meta-regression (MR) analysis to address this limitation using a case study in cardiovascular risk from arsenic exposure in Vietnam. Published studies were searched from PubMed using the keywords of arsenic exposure and cardiovascular diseases (CVD). Random-effects meta-regression was applied to model the linear relationship between arsenic concentration in water and risk of CVD, and then the no-observable-adverse-effect level (NOAEL) were identified from the regression function. The probabilistic risk assessment (PRA) technique was applied to characterize risk of CVD due to arsenic exposure by estimating the overlapping coefficient between dose-response and exposure distribution curves. The risks were evaluated for groundwater, treated and drinking water. A total of 8 high quality studies for dose-response and 12 studies for exposure data were included for final analyses. The results of MR suggested a NOAEL of 50 μg/L and a guideline of 5 μg/L for arsenic in water which valued as a half of NOAEL and guidelines recommended from previous studies and authorities. The results of PRA indicated that the observed exposure level with exceeding CVD risk was 52% for groundwater, 24% for treated water, and 10% for drinking water in Vietnam, respectively. The study found that systematic review and meta-regression can be considered as an ideal method to chemical risk assessment due to its advantages to bring the answer for the endpoint question of a CRA.

A primer on systematic reviews in toxicology

Systematic reviews, pioneered in the clinical field, provide a transparent, methodologically rigorous and reproducible means of summarizing the available evidence on a precisely framed research question. Having matured to a well-established approach in many research fields, systematic reviews are receiving increasing attention as a potential tool for answering toxicological questions. In the larger framework of evidence-based toxicology, the advantages and obstacles of, as well as the approaches for, adapting and adopting systematic reviews to toxicology are still being explored. To provide the toxicology community with a starting point for conducting or understanding systematic reviews, we herein summarized available guidance documents from various fields of application. We have elaborated on the systematic review process by breaking it down into ten steps, starting with planning the project, framing the question, and writing and publishing the protocol, and concluding with interpretation and reporting. In addition, we have identified the specific methodological challenges of toxicological questions and have summarized how these can be addressed. Ultimately, this primer is intended to stimulate scientific discussions of the identified issues to fuel the development of toxicology-specific methodology and to encourage the application of systematic review methodology to toxicological issues.

Toxicity reference values for methylmercury effects on avian reproduction: Critical review and analysis

Effects of mercury (Hg) on birds have been studied extensively and with increasing frequency in recent years. The authors conducted a comprehensive review of methylmercury (MeHg) effects on bird reproduction, evaluating laboratory and field studies in which observed effects could be attributed primarily to Hg. The review focuses on exposures via diet and maternal transfer in which observed effects (or lack thereof) were reported relative to Hg concentrations in diet, eggs, or adult blood. Applicable data were identified for 23 species. From this data set, the authors identified ranges of toxicity reference values suitable for risk-assessment applications. Typical ranges of Hg effect thresholds are approximately 0.2 mg/kg to >1.4 mg/kg in diet, 0.05 mg/kg/d to 0.5 mg/kg/d on a dose basis, 0.6 mg/kg to 2.7 mg/kg in eggs, and 2.1 mg/kg to >6.7 mg/kg in parental blood (all concentrations on a wet wt basis). For Hg in avian blood, the review represents the first broad compilation of relevant toxicity data. For dietary exposures, the current data support TRVs that are greater than older, commonly used TRVs. The older diet-based TRVs incorporate conservative assumptions and uncertainty factors that are no longer justified, although they generally were appropriate when originally derived, because of past data limitations. The egg-based TRVs identified from the review are more similar to other previously derived TRVs but have been updated to incorporate new information from recent studies. While important research needs remain, a key recommendation is that species not yet tested for MeHg toxicity should be evaluated using toxicity data from tested species with similar body weights. Environ Toxicol Chem 2017;36:294-319. © 2016 SETAC.

Systematic Review: A Method at Risk for Being Corrupted

The production of systematic reviews is increasing, but their credibility is under threat. Although systematic reviews are an important tool for policymaking, their influence can be weakened by methodological problems and poor policy relevance. Using Cochrane as an example, I address standards for systematic reviews, the influence of special interests on these reviews, and ways to increase their relevance for policymakers.

Assessing and managing multiple risks in a changing world-The Roskilde recommendations

Roskilde University (Denmark) hosted a November 2015 workshop, Environmental Risk-Assessing and Managing Multiple Risks in a Changing World. This Focus article presents the consensus recommendations of 30 attendees from 9 countries regarding implementation of a common currency (ecosystem services) for holistic environmental risk assessment and management; improvements to risk assessment and management in a complex, human-modified, and changing world; appropriate development of protection goals in a 2-stage process; dealing with societal issues; risk-management information needs; conducting risk assessment of risk management; and development of adaptive and flexible regulatory systems. The authors encourage both cross-disciplinary and interdisciplinary approaches to address their 10 recommendations: 1) adopt ecosystem services as a common currency for risk assessment and management; 2) consider cumulative stressors (chemical and nonchemical) and determine which dominate to best manage and restore ecosystem services; 3) fully integrate risk managers and communities of interest into the risk-assessment process; 4) fully integrate risk assessors and communities of interest into the risk-management process; 5) consider socioeconomics and increased transparency in both risk assessment and risk management; 6) recognize the ethical rights of humans and ecosystems to an adequate level of protection; 7) determine relevant reference conditions and the proper ecological context for assessments in human-modified systems; 8) assess risks and benefits to humans and the ecosystem and consider unintended consequences of management actions; 9) avoid excessive conservatism or possible underprotection resulting from sole reliance on binary, numerical benchmarks; and 10) develop adaptive risk-management and regulatory goals based on ranges of uncertainty. Environ Toxicol Chem 2017;36:7-16. © 2016 SETAC.

Review of reviews on exposures to synthetic organic chemicals and children's neurodevelopment: Methodological and interpretation challenges

Environmental epidemiology data are becoming increasingly important in public health decision making, which commonly incorporates a systematic review of multiple studies. This review addresses two fundamental questions: What is the quality of available reviews on associations between exposure to synthetic organic chemicals and neurodevelopmental outcomes? What is the value (e.g., quality and consistency) of the underlying literature? Published reviews on associations between synthetic organic environmental chemical exposures and neurodevelopmental outcomes in children were systematically evaluated. Seventy-four relevant reviews were identified, and these were evaluated with respect to four methodological characteristics: (1) systematic inclusion/exclusion criteria and reproducible methods for search and retrieval of studies; (2) structured evaluation of underlying data quality; (3) systematic assessment of consistency across specific exposure-outcome associations; and (4) evaluation of reporting/publication bias. None of the 74 reviews fully met the criteria for all four methodological characteristics. Only four reviews met two criteria, and six reviews fulfilled only one criterion. Perhaps more importantly, the higher quality reviews were not able to meet all of the criteria owing to the shortcomings of underlying studies, which lacked comparability in terms of specific research question of interest, overall design, exposure assessment, outcome ascertainment, and analytic methods. Thus, even the most thoughtful and rigorous review may be of limited value if the underlying literature includes investigations that address different hypotheses and are beset by methodological inconsistencies and limitations. Issues identified in this review of reviews illustrate considerable challenges that are facing assessments of epidemiological evidence.

A proposed framework for the systematic review and integrated assessment (SYRINA) of endocrine disrupting chemicals

BACKGROUND

The issue of endocrine disrupting chemicals (EDCs) is receiving wide attention from both the scientific and regulatory communities. Recent analyses of the EDC literature have been criticized for failing to use transparent and objective approaches to draw conclusions about the strength of evidence linking EDC exposures to adverse health or environmental outcomes. Systematic review methodologies are ideal for addressing this issue as they provide transparent and consistent approaches to study selection and evaluation. Objective methods are needed for integrating the multiple streams of evidence (epidemiology, wildlife, laboratory animal, in vitro, and in silico data) that are relevant in assessing EDCs.

METHODS

We have developed a framework for the systematic review and integrated assessment (SYRINA) of EDC studies. The framework was designed for use with the International Program on Chemical Safety (IPCS) and World Health Organization (WHO) definition of an EDC, which requires appraisal of evidence regarding 1) association between exposure and an adverse effect, 2) association between exposure and endocrine disrupting activity, and 3) a plausible link between the adverse effect and the endocrine disrupting activity.

RESULTS

Building from existing methodologies for evaluating and synthesizing evidence, the SYRINA framework includes seven steps: 1) Formulate the problem; 2) Develop the review protocol; 3) Identify relevant evidence; 4) Evaluate evidence from individual studies; 5) Summarize and evaluate each stream of evidence; 6) Integrate evidence across all streams; 7) Draw conclusions, make recommendations, and evaluate uncertainties. The proposed method is tailored to the IPCS/WHO definition of an EDC but offers flexibility for use in the context of other definitions of EDCs.

CONCLUSIONS

When using the SYRINA framework, the overall objective is to provide the evidence base needed to support decision making, including any action to avoid/minimise potential adverse effects of exposures. This framework allows for the evaluation and synthesis of evidence from multiple evidence streams. Finally, a decision regarding regulatory action is not only dependent on the strength of evidence, but also the consequences of action/inaction, e.g. limited or weak evidence may be sufficient to justify action if consequences are serious or irreversible.

Systematic reviews: Separating fact from fiction

The volume of scientific literature continues to expand and decision-makers are faced with increasingly unmanageable volumes of evidence to assess. Systematic reviews (SRs) are powerful tools that aim to provide comprehensive, transparent, reproducible and updateable summaries of evidence. SR methods were developed, and have been employed, in healthcare for more than two decades, and they are now widely used across a broad range of topics, including environmental management and social interventions in crime and justice, education, international development, and social welfare. Despite these successes and the increasing acceptance of SR methods as a 'gold standard' in evidence-informed policy and practice, misconceptions still remain regarding their applicability. The aim of this article is to separate fact from fiction, addressing twelve common misconceptions that can influence the decision as to whether a SR is the most appropriate method for evidence synthesis for a given topic. Through examples, we illustrate the flexibility of SR methods and demonstrate their suitability for addressing issues on environmental health and chemical risk assessment.

Study sensitivity: Evaluating the ability to detect effects in systematic reviews of chemical exposures

A critical step in systematic reviews of potential health hazards is the structured evaluation of the strengths and weaknesses of the included studies; risk of bias is a term often used to represent this process, specifically with respect to the evaluation of systematic errors that can lead to inaccurate (biased) results (i.e. focusing on internal validity). Systematic review methods developed in the clinical medicine arena have been adapted for use in evaluating environmental health hazards; this expansion raises questions about the scope of risk of bias tools and the extent to which they capture the elements that can affect the interpretation of results from environmental and occupational epidemiology studies and in vivo animal toxicology studies, (the studies typically available for assessment of risk of chemicals). One such element, described here as "sensitivity", is a measure of the ability of a study to detect a true effect or hazard. This concept is similar to the concept of the sensitivity of an assay; an insensitive study may fail to show a difference that truly exists, leading to a false conclusion of no effect. Factors relating to study sensitivity should be evaluated in a systematic manner with the same rigor as the evaluation of other elements within a risk of bias framework. We discuss the importance of this component for the interpretation of individual studies, examine approaches proposed or in use to address it, and describe how it relates to other evaluation components. The evaluation domains contained within a risk of bias tool can include, or can be modified to include, some features relating to study sensitivity; the explicit inclusion of these sensitivity criteria with the same rigor and at the same stage of study evaluation as other bias-related criteria can improve the evaluation process. In some cases, these and other features may be better addressed through a separate sensitivity domain. The combined evaluation of risk of bias and sensitivity can be used to identify the most informative studies, to evaluate the confidence of the findings from individual studies and to identify those study elements that may help to explain heterogeneity across the body of literature.

GRADE: Assessing the quality of evidence in environmental and occupational health

There is high demand in environmental health for adoption of a structured process that evaluates and integrates evidence while making decisions and recommendations transparent. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework holds promise to address this demand. For over a decade, GRADE has been applied successfully to areas of clinical medicine, public health, and health policy, but experience with GRADE in environmental and occupational health is just beginning. Environmental and occupational health questions focus on understanding whether an exposure is a potential health hazard or risk, assessing the exposure to understand the extent and magnitude of risk, and exploring interventions to mitigate exposure or risk. Although GRADE offers many advantages, including its flexibility and methodological rigor, there are features of the different sources of evidence used in environmental and occupational health that will require further consideration to assess the need for method refinement. An issue that requires particular attention is the evaluation and integration of evidence from human, animal, in vitro, and in silico (computer modeling) studies when determining whether an environmental factor represents a potential health hazard or risk. Assessment of the hazard of exposures can produce analyses for use in the GRADE evidence-to-decision (EtD) framework to inform risk-management decisions about removing harmful exposures or mitigating risks. The EtD framework allows for grading the strength of the recommendations based on judgments of the certainty in the evidence (also known as quality of the evidence), as well as other factors that inform recommendations such as social values and preferences, resource implications, and benefits. GRADE represents an untapped opportunity for environmental and occupational health to make evidence-based recommendations in a systematic and transparent manner. The objectives of this article are to provide an overview of GRADE, discuss GRADE's applicability to environmental health, and identify priority areas for method assessment and development.