A primer on systematic reviews in toxicology

Cross-mapping of terms used in chemical risk assessment with those used in systematic review: research protocol

The focus on implementation of systematic review (SR) principles in chemical risk assessments (CRAs) is growing as it has the potential to advance the rigour and transparency of the CRAs. However, the SR and CRA communities use their own specific terminologies. Understanding the meaning of core SR and CRA terms and where they overlap is critical for application of SR methods and principles in CRAs. Moreover, it will increase the possibility for cross-sectorial collaboration, avoid misunderstandings, and improve communication among risk assessors, researchers, and policy makers. We present a process for the cross-mapping of core CRA terms and core SR terms. Core terms for study appraisal, evidence synthesis and integration used in the SR and CRA communities will be included. The outcome will be an overview of how core SR terms map onto core CRA terms and vice versa, and a description of the relationship and conceptual overlap between the terms. The cross-mapping is divided in three phases, where in the first phase the core SR and CRA terms will be identified. In the second phase, existing SR and CRA definitions will be mapped. In the third phase, descriptions of the relationship and conceptual overlap between the terms will be derived. The third phase will include weekly one-hour online meetings for SR and CRA experts.

A systematic scoping review of the neurological effects of COVID-19

BACKGROUND

The global coronavirus 2019 (COVID-19) pandemic began in early 2020, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). In mid-2020 the CIAO (Modelling the Pathogenesis of COVID-19 Using the Adverse Outcome Pathway Framework) project was established, bringing together over 75 interdisciplinary scientists worldwide to collaboratively investigate the underlying biological mechanisms of COVID-19 and consolidate the data using the Adverse Outcome Pathway (AOP) Framework. Neurological symptoms such as anosmia and encephalitis have been frequently reported to be associated with infection with SARS-CoV-2.

OBJECTIVE

Within CIAO, a working group was formed to conduct a systematic scoping review of COVID-19 and its related neurological symptoms to determine which key events and modulating factors are most commonly reported and to identify knowledge gaps.

DESIGN

LitCOVID was used to retrieve 86,075 papers of which 10,244 contained relevant keywords. After title and abstract screening, 2,328 remained and their full texts were reviewed based on predefined inclusion and exclusion criteria. 991 studies fulfilled the inclusion criteria and were retrieved to conduct knowledge synthesis.

RESULTS

The majority of publications reported human observational studies. Early key events were less likely to be reported compared to middle and late key events/adverse outcomes. The majority of modulating factors described related to age or sex. Less recognised COVID-19 associated AO or neurological effects of COVID-19 were also identified including multiple sclerosis/demyelination, neurodegeneration/cognitive effects and peripheral neuronal effects.

CONCLUSION

There were many methodological and reporting issues noted in the reviewed studies. In particular, publication abstracts would benefit from clearer reporting of the methods and endpoints used and the key findings, to ensure relevant papers are included when systematic reviews are conducted. The information extracted from the scoping review may be useful in understanding the mechanisms of neurological effects of COVID-19 and to further develop or support existing AOPs linking COVID-19 and its neurological key events and adverse outcomes. Further evaluation of the less recognised COVID-19 effects is needed.

Persistent organic pollutants and heavy metals in Ghanaian environment: a systematic review of food safety implications

Advances in industrial and technological innovations have led to significant socio-economic benefits, but with overwhelming negative impacts on the environment. These impacts include the infiltration of organic contaminants into soil, water, and air, posing a threat to the environment and public health. Polybrominated diphenyl ethers (PBDEs), heavy metals, and polycyclic aromatic hydrocarbons (PAHs) are increasingly released as waste, endangering the environment. In countries like Ghana, where regulations are weakly enforced, industrial waste is released uncontrollably, posing threats to public health, environmental integrity, and food systems. This study systematically evaluated existing literature on PBDEs, heavy metals, PAHs, and organic contaminant exposure in Ghana and proposes a roadmap for achieving food safety and protecting the environment and human health. The research identified high mobility of specific heavy metals and risks associated with PBDEs and PAHs in sediments, dumpsites, and various food items. Unregulated dumping of electronic waste with PBDEs raised environmental concerns. An integrated approach is needed to address the multifaceted impact of organic pollutants on public health and ecosystems. Urgent implementation of effective environmental management strategies and regulatory measures is crucial. The study proposed short- to mid-term priorities emphasising the need to foster collaboration and implementing global measures. The mid- to long-term strategy includes a national information surveillance system, local monitoring capacity development, and integrating land contamination controls with food safety legislation. These measures would mitigate risks, ensure sustainable practices, and improve overall food safety management in Ghana, serving as a model for regions facing similar challenges with diverse pollutants.

Artificial intelligence (AI)-it's the end of the tox as we know it (and I feel fine)

The rapid progress of AI impacts diverse scientific disciplines, including toxicology, and has the potential to transform chemical safety evaluation. Toxicology has evolved from an empirical science focused on observing apical outcomes of chemical exposure, to a data-rich field ripe for AI integration. The volume, variety and velocity of toxicological data from legacy studies, literature, high-throughput assays, sensor technologies and omics approaches create opportunities but also complexities that AI can help address. In particular, machine learning is well suited to handle and integrate large, heterogeneous datasets that are both structured and unstructured-a key challenge in modern toxicology. AI methods like deep neural networks, large language models, and natural language processing have successfully predicted toxicity endpoints, analyzed high-throughput data, extracted facts from literature, and generated synthetic data. Beyond automating data capture, analysis, and prediction, AI techniques show promise for accelerating quantitative risk assessment by providing probabilistic outputs to capture uncertainties. AI also enables explanation methods to unravel mechanisms and increase trust in modeled predictions. However, issues like model interpretability, data biases, and transparency currently limit regulatory endorsement of AI. Multidisciplinary collaboration is needed to ensure development of interpretable, robust, and human-centered AI systems. Rather than just automating human tasks at scale, transformative AI can catalyze innovation in how evidence is gathered, data are generated, hypotheses are formed and tested, and tasks are performed to usher new paradigms in chemical safety assessment. Used judiciously, AI has immense potential to advance toxicology into a more predictive, mechanism-based, and evidence-integrated scientific discipline to better safeguard human and environmental wellbeing across diverse populations.

Understanding target-specific effects of antidepressant drug pollution on molluscs: A systematic review protocol

BACKGROUND

The environmental prevalence of widely prescribed human pharmaceuticals that target key evolutionary conserved biomolecules present across phyla is concerning. Antidepressants, one of the most widely consumed pharmaceuticals globally, have been developed to target biomolecules modulating monoaminergic neurotransmission, thus interfering with the endogenous regulation of multiple key neurophysiological processes. Furthermore, rising prescription and consumption rates of antidepressants caused by the burgeoning incidence of depression is consistent with increasing reports of antidepressant detection in aquatic environments worldwide. Consequently, there are growing concerns that long-term exposure to environmental levels of antidepressants may cause adverse drug target-specific effects on non-target aquatic organisms. While these concerns have resulted in a considerable body of research addressing a range of toxicological endpoints, drug target-specific effects of environmental levels of different classes of antidepressants in non-target aquatic organisms remain to be understood. Interestingly, evidence suggests that molluscs may be more vulnerable to the effects of antidepressants than any other animal phylum, making them invaluable in understanding the effects of antidepressants on wildlife. Here, a protocol for the systematic review of literature to understand drug target-specific effects of environmental levels of different classes of antidepressants on aquatic molluscs is described. The study will provide critical insight needed to understand and characterize effects of antidepressants relevant to regulatory risk assessment decision-making, and/or direct future research efforts.

METHODS

The systematic review will be conducted in line with the guidelines by the Collaboration for Environmental Evidence (CEE). A literature search on Scopus, Web of Science, PubMed, as well as grey literature databases, will be carried out. Using predefined criteria, study selection, critical appraisal and data extraction will be done by multiple reviewers with a web-based evidence synthesis platform. A narrative synthesis of outcomes of selected studies will be presented. The protocol has been registered in the Open Science Framework (OSF) registry with the registration DOI: 10.17605/OSF.IO/P4H8W.

Updated systematic assessment of human, animal and mechanistic evidence demonstrates lack of human carcinogenicity with consumption of aspartame

Aspartame has been studied extensively and evaluated for its safety in foods and beverages yet concerns for its potential carcinogenicity have persisted, driven primarily by animal studies conducted at the Ramazzini Institute (RI). To address this controversy, an updated systematic review of available human, animal, and mechanistic data was conducted leveraging critical assessment tools to consider the quality and reliability of data. The evidence base includes 12 animal studies and >40 epidemiological studies reviewed by the World Health Organization which collectively demonstrate a lack of carcinogenic effect. Assessment of >1360 mechanistic endpoints, including many guideline-based genotoxicity studies, demonstrate a lack of activity associated with endpoints grouped to key characteristics of carcinogens. Other non-specific mechanistic data (e.g., mixed findings of oxidative stress across study models, tissues, and species) do not provide evidence of a biologically plausible carcinogenic pathway associated with aspartame. Taken together, available evidence supports that aspartame consumption is not carcinogenic in humans and that the inconsistent findings of the RI studies may be explained by flaws in study design and conduct (despite additional analyses to address study limitations), as acknowledged by authoritative bodies.

Toxic mechanism of the Mongolian medicine "Hunqile-7" based on metabonomics and the metabolism of intestinal flora

The traditional Mongolian medicine Hunqile-7 (HQL-7), which is mainly used to relieve pain in clinic, has certain toxicity. Therefore, toxicological investigation of HQL-7 is of great significance to its safety assessment. In this study, the toxic mechanism of HQL-7 was explored based on a combination of metabolomics and intestinal flora metabolism. UHPLC-MS was used to analyze the serum, liver and kidney samples of rats after intragastric administration of HQL-7. The decision tree and K Nearest Neighbor (KNN) model were established based on the bootstrap aggregation (bagging) algorithm to classify the omics data. After samples were extracted from rat feces, the high-throughput sequencing platform was used to analyze the 16s rRNA V3-V4 region of bacteria. The experimental results confirm that the bagging algorithm improved the classification accuracy. The toxic dose, toxic intensity, and toxic target organ of HQL-7 were determined in toxicity tests. Seventeen biomarkers were identified and the metabolism dysregulation of these biomarkers may be responsible for the toxicity of HQL-7 in vivo. Several kinds of bacteria was demonstrated to be closely related to the physiological indices of renal and liver function, indicating liver and kidney damage induced by HQL-7 may be related to the disturbance of these intestinal bacteria. Overall, the toxic mechanism of HQL-7 was revealed in vivo, which not only provides a scientific basis for the safe and rational clinical use of HQL-7, but also opens up a new field of research on big data for Mongolian medicine.

An interdisciplinary framework for derivation of occupational exposure limits

Protecting the health and safety of workers in industrial operations is a top priority. One of the resources used in industry to ensure worker safety is the occupational exposure limit (OEL). OELs are derived from the assessment and interpretation of empirical data from animal and/or human studies. There are various guidelines for the derivation and implementation of OELs globally, with a range of stakeholders (including regulatory bodies, governmental agencies, expert groups and others). The purpose of this manuscript is to supplement existing guidance with learnings from a multidisciplinary team approach within an industry setting. The framework we present is similar in construct to other risk assessment frameworks and includes: (1) problem formulation, (2) literature review, (3) weight of evidence considerations, (4) point of departure selection/derivation, (5) application of assessment factors, and the final step, (6) derivation of the OEL. Within each step are descriptions and examples to consider when incorporating data from various disciplines such as toxicology, epidemiology, and exposure science. This manuscript describes a technical framework by which available data relevant for occupational exposures is compiled, analyzed, and utilized to inform safety threshold derivation applicable to OELs.

Evaluating the food safety and risk assessment evidence-base of polyethylene terephthalate oligomers: Protocol for a systematic evidence map

BACKGROUND

Polyethylene terephthalate (PET) oligomers are ubiquitous in PET used in food contact applications. Consumer exposure by migration of PET oligomers into food and beverages is documented. However, no specific risk assessment framework or guidance for the safety evaluating of PET oligomers exist to date.

AIM

The aim of this systematic evidence map (SEM) is to identify and organize existing knowledge clusters and associated gaps in hazard and exposure information of PET oligomers. Research needs will be identified as an input for chemical risk assessment, and to support future toxicity testing strategies of PET oligomers and regulatory decision-making.

SEARCH STRATEGY AND ELIGIBILITY CRITERIA

Multiple bibliographic databases (incl. Embase, Medline, Scopus, and Web of Science Core Collection), chemistry databases (SciFinder-n, Reaxys), and gray literature sources will be searched, and the search results will be supplemented by backward and forward citation tracking on eligible records. The search will be based on a single-concept PET oligomer-focused strategy to ensure sensitive and unbiased coverage of all evidence related to hazard and exposure in a data-poor environment. A scoping exercise conducted during planning identified 34 relevant PET oligomers. Eligible work of any study type must include primary research data on at least one relevant PET oligomer with regard to exposure, health, or toxicological outcomes.

STUDY SELECTION

For indexed scientific literature, title and abstract screening will be performed by one reviewer. Selected studies will be screened in full-text by two independent reviewers. Gray literature will be screened by two independent reviewers for inclusion and exclusion.

STUDY QUALITY ASSESSMENT

Risk of bias analysis will not be conducted as part of this SEM.

DATA EXTRACTION AND CODING

Will be performed by one reviewer and peer-checked by a second reviewer for indexed scientific literature or by two independent reviewers for gray literature.

SYNTHESIS AND VISUALIZATION

The extracted and coded information will be synthesized in different formats, including narrative synthesis, tables, and heat maps.

SYSTEMATIC MAP PROTOCOL REGISTRY AND REGISTRATION NUMBER

Zenodo: https://doi.org/10.5281/zenodo.6224302.

A systematic review on fluoride-induced epigenetic toxicity in mammals

Fluoride, one of the global groundwater contaminants, is ubiquitous in our day-to-day life from various natural and anthropogenic sources. Numerous in vitro, in vivo, and epidemiological studies are conducted to understand the effect of fluoride on biological systems. A low concentration of fluoride is reported to increase oral health, whereas chronic exposure to higher concentrations causes fluoride toxicity (fluorosis). It includes dental fluorosis, skeletal fluorosis, and fluoride toxicity in soft tissues. The mechanism of fluoride toxicity has been reviewed extensively. However, epigenetic regulation in fluoride toxicity has not been reviewed. This systematic review summarizes the current knowledge regarding fluoride-induced epigenetic toxicity in the in vitro, in vivo, and epidemiological studies in mammalian systems. We examined four databases for the association between epigenetics and fluoride exposure. Out of 932 articles (as of 31 March 2022), 39 met our inclusion criteria. Most of the studies focused on different genes, and overall, preliminary evidence for epigenetic regulation of fluoride toxicity was identified. We further highlight the need for epigenome studies rather than candidate genes and provide recommendations for future research. Our results indicate a correlation between fluoride exposure and epigenetic processes. Further studies are warranted to elucidate and confirm the mechanism of epigenetic alterations mediated fluoride toxicity.

The ECOTOXicology Knowledgebase: A Curated Database of Ecologically Relevant Toxicity Tests to Support Environmental Research and Risk Assessment

The need for assembled existing and new toxicity data has accelerated as the amount of chemicals introduced into commerce continues to grow and regulatory mandates require safety assessments for a greater number of chemicals. To address this evolving need, the ECOTOXicology Knowledgebase (ECOTOX) was developed starting in the 1980s and is currently the world's largest compilation of curated ecotoxicity data, providing support for assessments of chemical safety and ecological research through systematic and transparent literature review procedures. The recently released version of ECOTOX (Ver 5, www.epa.gov/ecotox) provides single-chemical ecotoxicity data for over 12,000 chemicals and ecological species with over one million test results from over 50,000 references. Presented is an overview of ECOTOX, detailing the literature review and data curation processes within the context of current systematic review practices and discussing how recent updates improve the accessibility and reusability of data to support the assessment, management, and research of environmental chemicals. Relevant and acceptable toxicity results are identified from studies in the scientific literature, with pertinent methodological details and results extracted following well-established controlled vocabularies and newly extracted toxicity data added quarterly to the public website. Release of ECOTOX, Ver 5, included an entirely redesigned user interface with enhanced data queries and retrieval options, visualizations to aid in data exploration, customizable outputs for export and use in external applications, and interoperability with chemical and toxicity databases and tools. This is a reliable source of curated ecological toxicity data for chemical assessments and research and continues to evolve with accessible and transparent state-of-the-art practices in literature data curation and increased interoperability to other relevant resources. Environ Toxicol Chem 2022;41:1520-1539. © 2022 SETAC. This article has been contributed to by US Government employees and their work is in the public domain in the USA.

Biological plausibility in environmental health systematic reviews: a GRADE concept paper

BACKGROUND

"Biological plausibility" is a concept frequently referred to in environmental and public health when researchers are evaluating how confident they are in the results and inferences of a study or evidence review. Biological plausibility is not, however, a domain of one of the most widely-used approaches for assessing the certainty of evidence (CoE) which underpins the findings of a systematic review, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) CoE Framework. Whether the omission of biological plausibility is a potential limitation of the GRADE CoE Framework is a topic that is regularly discussed, especially in the context of environmental health systematic reviews.

OBJECTIVES

We analyse how the concept of "biological plausibility", as applied in the context of assessing certainty of the evidence that supports the findings of a systematic review, is accommodated under the processes of systematic review and the existing GRADE domains.

RESULTS AND DISCUSSION

We argue that "biological plausibility" is a concept which primarily comes into play when direct evidence about the effects of an exposure on a population of concern (usually humans) is absent, at high risk of bias, is inconsistent, or limited in other ways. In such circumstances, researchers look toward evidence from other study designs in order to draw conclusions. In this respect, we can consider experimental animal and in vitro evidence as "surrogates" for the target populations, exposures, comparators and outcomes of actual interest. Through discussion of 10 examples of experimental surrogates, we propose that the concept of biological plausibility consists of two principal aspects: a "generalisability aspect" and a "mechanistic aspect". The "generalisability aspect" concerns the validity of inferences from experimental models to human scenarios, and asks the same question as does the assessment of external validity or indirectness in systematic reviews. The "mechanistic aspect" concerns certainty in knowledge of biological mechanisms and would inform judgements of indirectness under GRADE, and thus the overall CoE. While both aspects are accommodated under the indirectness domain of the GRADE CoE Framework, further research is needed to determine how to use knowledge of biological mechanisms in the assessment of indirectness of the evidence in systematic reviews.

Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks

The workshop titled “Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks” was co-organized by the Evidence-based Toxicology Collaboration and the European Food Safety Authority (EFSA) and hosted by EFSA at its headquarters in Parma, Italy on October 2 and 3, 2019. The goal was to explore integration of systematic review with mechanistic evidence evaluation. Participants were invited to work on concrete products to advance the exploration of how evidence-based approaches can support the development and application of adverse outcome pathways (AOP) in chemical risk assessment. The workshop discussions were centered around three related themes: 1) assessing certainty in AOPs, 2) literature-based AOP development, and 3) integrating certainty in AOPs and non-animal evidence into decision frameworks. Several challenges, mostly related to methodology, were identified and largely determined the workshop recommendations. The workshop recommendations included the comparison and potential alignment of processes used to develop AOP and systematic review methodology, including the translation of vocabulary of evidence-based methods to AOP and vice versa, the development and improvement of evidence mapping and text mining methods and tools, as well as a call for a fundamental change in chemical risk and uncertainty assessment methodology if to be conducted based on AOPs and new approach methodologies (NAM). The usefulness of evidence-based approaches for mechanism-based chemical risk assessments was stressed, particularly the potential contribution of the rigor and transparency inherent to such approaches in building stakeholders’ trust for implementation of NAM evidence and AOPs into chemical risk assessment.

Biological plausibility in environmental health systematic reviews: a GRADE concept paper

BACKGROUND

"Biological plausibility" is a concept frequently referred to in environmental and public health when researchers are evaluating how confident they are in the results and inferences of a study or evidence review. Biological plausibility is not, however, a domain of one of the most widely-used approaches for assessing the certainty of evidence (CoE) which underpins the findings of a systematic review, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) CoE Framework. Whether the omission of biological plausibility is a potential limitation of the GRADE CoE Framework is a topic that is regularly discussed, especially in the context of environmental health systematic reviews.

OBJECTIVES

We analyse how the concept of "biological plausibility", as applied in the context of assessing certainty of the evidence that supports the findings of a systematic review, is accommodated under the processes of systematic review and the existing GRADE domains.

RESULTS AND DISCUSSION

We argue that "biological plausibility" is a concept which primarily comes into play when direct evidence about the effects of an exposure on a population of concern (usually humans) is absent, at high risk of bias, is inconsistent, or limited in other ways. In such circumstances, researchers look toward evidence from other study designs in order to draw conclusions. In this respect, we can consider experimental animal and in vitro evidence as "surrogates" for the target populations, exposures, comparators and outcomes of actual interest. Through discussion of 10 examples of experimental surrogates, we propose that the concept of biological plausibility consists of two principal aspects: a "generalisability aspect" and a "mechanistic aspect". The "generalisability aspect" concerns the validity of inferences from experimental models to human scenarios, and asks the same question as does the assessment of external validity or indirectness in systematic reviews. The "mechanistic aspect" concerns certainty in knowledge of biological mechanisms and would inform judgements of indirectness under GRADE, and thus the overall CoE. While both aspects are accommodated under the indirectness domain of the GRADE CoE Framework, further research is needed to determine how to use knowledge of biological mechanisms in the assessment of indirectness of the evidence in systematic reviews.

A systematic review on the effects of nanomaterials on gut microbiota

Some nanomaterials (NMs) have been shown to possess antimicrobial activity and cause GM dysbiosis. Since NMs are being used widely, a systematic assessment of the effects of NMs on GM is warranted. In this systematic review, a total of 46 in vivo and 22 in vitro studies were retrieved from databases and search engines including Science-Direct, Pubmed and Google scholar. Criteria for assessment of studies included use of in vitro or in vivo studies, characterization of NMs, use of single or multiple doses as well as consistency of results. GM dysbiosis has been studied most widely on TiO2, Ag, Zn-based NMs. There was moderate evidence for GM dysbiosis caused by Zn- and Cu-based NMs, Cu-loaded chitosan NPs and Ag NMs, and anatase TiO2 NPs, as well as low evidence for SWCNTs, nanocellulose, SiO2, Se, nanoplastics, CeO2, MoO3 and graphene-based NMs. Most studies indicate adverse effects of NMs towards GM. However, more work is required to elucidate the differences on the reported effects of NM by type and sex of organisms, size, shape and surface properties of NMs as well as effects of exposure to mixtures of NMs. For consistency and better agreement among studies on GM dysbiosis, there is need for internationally agreed protocols on, inter alia, characterization of NMs, dosing (amounts, frequency and duration), use of sonication, test systems (both in vitro and in vivo), including oxygen levels for in vitro models.

The effect of radiofrequency electromagnetic fields (RF-EMF) on biomarkers of oxidative stress in vivo and in vitro: A protocol for a systematic review

BACKGROUND

Oxidative stress is conjectured to be related to many diseases. Furthermore, it is hypothesized that radiofrequency fields may induce oxidative stress in various cell types and thereby compromise human and animal health. This systematic review (SR) aims to summarize and evaluate the literature related to this hypothesis.

OBJECTIVES

The main objective of this SR is to evaluate the associations between the exposure to radiofrequency electromagnetic fields and oxidative stress in experimental models (in vivo and in vitro).

METHODS

The SR framework has been developed following the guidelines established in the WHO Handbook for Guideline Development and the Handbook for Conducting a Literature-Based Health Assessment). We will include controlled in vivo and in vitro laboratory studies that assess the effects of an exposure to RF-EMF on valid markers for oxidative stress compared to no or sham exposure. The protocol is registered in PROSPERO. We will search the following databases: PubMed, Embase, Web of Science Core Collection, Scopus, and the EMF-Portal. The reference lists of included studies and retrieved review articles will also be manually searched.

STUDY APPRAISAL AND SYNTHESIS METHOD

Data will be extracted according to a pre-defined set of forms developed in the DistillerSR online software and synthesized in a meta-analysis when studies are judged sufficiently similar to be combined. If a meta-analysis is not possible, we will describe the effects of the exposure in a narrative way.

RISK OF BIAS

The risk of bias will be assessed with the NTP/OHAT risk of bias rating tool for human and animal studies. We will use GRADE to assess the certainty of the conclusions (high, moderate, low, or inadequate) regarding the association between radiofrequency electromagnetic fields and oxidative stress.

FUNDING

This work was funded by the World Health Organization (WHO).

REGISTRATION

The protocol was registered on the PROSPERO webpage on July 8, 2021.

Probabilistic risk assessment - the keystone for the future of toxicology

Safety sciences must cope with uncertainty of models and results as well as information gaps. Acknowledging this uncer-tainty necessitates embracing probabilities and accepting the remaining risk. Every toxicological tool delivers only probable results. Traditionally, this is taken into account by using uncertainty / assessment factors and worst-case / precautionary approaches and thresholds. Probabilistic methods and Bayesian approaches seek to characterize these uncertainties and promise to support better risk assessment and, thereby, improve risk management decisions. Actual assessments of uncertainty can be more realistic than worst-case scenarios and may allow less conservative safety margins. Most importantly, as soon as we agree on uncertainty, this defines room for improvement and allows a transition from traditional to new approach methods as an engineering exercise. The objective nature of these mathematical tools allows to assign each methodology its fair place in evidence integration, whether in the context of risk assessment, sys-tematic reviews, or in the definition of an integrated testing strategy (ITS) / defined approach (DA) / integrated approach to testing and assessment (IATA). This article gives an overview of methods for probabilistic risk assessment and their application for exposure assessment, physiologically-based kinetic modelling, probability of hazard assessment (based on quantitative and read-across based structure-activity relationships, and mechanistic alerts from in vitro studies), indi-vidual susceptibility assessment, and evidence integration. Additional aspects are opportunities for uncertainty analysis of adverse outcome pathways and their relation to thresholds of toxicological concern. In conclusion, probabilistic risk assessment will be key for constructing a new toxicology paradigm - probably!

Development of the SciRAP Approach for Evaluating the Reliability and Relevance of in vitro Toxicity Data

Efficient and successful integration of data generated from non-animal test methods must rely on reliable and relevant data. It is important therefore to develop tools and criteria that facilitate scientifically sound, structured, and transparent evaluation of reliability and relevance of in vitro toxicity data to efficiently inform regulatory hazard and risk assessment. The Science in Risk Assessment and Policy (SciRAP) initiative aims to promote such overarching goals. We present the work to develop and refine the SciRAP tool for evaluation of reliability and relevance of in vitro studies for incorporation on the SciRAP web-based platform (www.scirap.org). In the SciRAP approach, reliability evaluation is based on criteria for reporting quality and methodological quality, and is explicitly separated from relevance evaluation. The SciRAP in vitro tool (version 1.0) was tested and evaluated during an expert test round (April 2019-September 2020) on three in vitro studies by thirty-one experts from regulatory authorities, industry and academia from different geographical areas and with various degree of experience in in vitro research and/or human health risk assessment. In addition, the experts answered an online survey to collect their feedback about the general features and desired characteristics of the tool for further refinement. The SciRAP in vitro tool (version 2.0) was revised based on the outcome of the expert test round (study evaluation and online survey) and consists of 24 criteria for evaluating "reporting quality" (reliability), 16 criteria for "methodological quality" (reliability), and 4 items for evaluating relevance of in vitro studies. Participants were generally positive about the adequacy, flexibility, and user-friendliness of the tool. The expert test round outlined the need to (i) revise the formulation of certain criteria; (ii) provide new or revised accompanying guidance for reporting quality and methodological quality criteria in the "test compounds and controls," "test system," and "data collection and analysis" domains; and (iii) provide revised guidance for relevance items, as general measures to reduce inter-expert variability. The SciRAP in vitro tool allows for a structured and transparent evaluation of in vitro studies for use in regulatory hazard and risk assessment of chemicals.

Systematic Review Methodologies and Endocrine Disrupting Chemicals: Improving Evaluations of the Plastic Monomer Bisphenol A

BACKGROUND

Endocrine disrupting chemicals (EDCs) are found in plastics, personal care products, household items, and other consumer goods. Risk assessments are intended to characterize a chemical's hazards, identify the doses at which adverse outcomes are observed, quantify exposure levels, and then compare these doses to determine the likelihood of risk in a given population. There are many problems with risk assessments for EDCs, allowing people to be exposed to levels that are later associated with serious health outcomes in epidemiology studies.

OBJECTIVE

In this review, we examine issues that affect the evaluation of EDCs in risk assessments (e.g., use of insensitive rodent strains and absence of disease-oriented outcomes in hazard assessments; inadequate exposure assessments). We then review one well-studied chemical, Bisphenol A (BPA; CAS #80-05-7) an EDC found in plastics, food packaging, and other consumer products. More than one hundred epidemiology studies suggest associations between BPA exposures and adverse health outcomes in environmentally exposed human populations.

FINDINGS

We present support for the use of systematic review methodologies in the evaluation of BPA and other EDCs. Systematic reviews would allow studies to be evaluated for their reliability and risk of bias. They would also allow all data to be used in risk assessments, which is a requirement for some regulatory agencies.

CONCLUSION

Systematic review methodologies can be used to improve evaluations of BPA and other EDCs. Their use could help to restore faith in risk assessments and ensure that all data are utilized in decision-making. Regulatory agencies are urged to conduct transparent, well-documented and proper systematic reviews for BPA and other EDCs.

A Systematic Review to Compare Chemical Hazard Predictions of the Zebrafish Embryotoxicity Test With Mammalian Prenatal Developmental Toxicity

Originally developed to inform the acute toxicity of chemicals on fish, the zebrafish embryotoxicity test (ZET) has also been proposed for assessing the prenatal developmental toxicity of chemicals, potentially replacing mammalian studies. Although extensively evaluated in primary studies, a comprehensive review summarizing the available evidence for the ZET's capacity is lacking. Therefore, we conducted a systematic review of how well the presence or absence of exposure-related findings in the ZET predicts prenatal development toxicity in studies with rats and rabbits. A two-tiered systematic review of the developmental toxicity literature was performed, a review of the ZET literature was followed by one of the mammalian literature. Data were extracted using DistillerSR, and study validity was assessed with an amended SYRCLE's risk-of-bias tool. Extracted data were analyzed for each species and substance, which provided the basis for comparing the 2 test methods. Although limited by the number of 24 included chemicals, our results suggest that the ZET has potential to identify chemicals that are mammalian prenatal developmental toxicants, with a tendency for overprediction. Furthermore, our analysis confirmed the need for further standardization of the ZET. In addition, we identified contextual and methodological challenges in the application of systematic review approaches to toxicological questions. One key to overcoming these challenges is a transition to more comprehensive and transparent planning, conduct and reporting of toxicological studies. The first step toward bringing about this change is to create broad awareness in the toxicological community of the need for and benefits of more evidence-based approaches.

Does paraquat cause Parkinson's disease? A review of reviews

To examine the extent to which a consensus exists in the scientific community regarding the relationship between exposure to paraquat and Parkinson's disease, a critical review of reviews was undertaken focusing on reviews published between 2006 and the present that offered opinions on the issue of causation. Systematic searches were undertaken of scientific databases along with searches of published bibliographies to identify English language reviews on the topic of paraquat and Parkinson's disease including those on the broader topic of environmental and occupational risk factors for Parkinson's disease. Of the 269 publications identified in the searches, there were twelve reviews, some with meta-analyses, that met the inclusion criteria. Information on methods used by the reviewers, if any, and source of funding was collected; the quality of the reviews was considered. No author of any published review stated that it has been established that exposure to paraquat causes Parkinson's disease, regardless of methods used and independent of funding source. A consensus exists in the scientific community that the available evidence does not warrant a claim that paraquat causes Parkinson's disease. Future research on this topic should focus on improving the quality of epidemiological studies including better exposure measures and identifying specific mechanisms of action. Future reviews of emerging evidence should be structured as systematic narrative reviews with meta-analysis if appropriate.

Developing a database of systematic reviews of animal studies

Systematic reviews (SRs) are common practice in clinical and public health research, but less common in non-human animal research. Systematic reviews of animal studies can be valuable to inform clinical research, to evaluate the need for further animal experiments on a given topic, and to assess the hazard of an environmental exposure in the evaluation of toxicological studies. In the last 10 years, there has been an increase in the number of SRs of animal research, as well as several publications with detailed guidance on how to perform high-quality systematic reviews of experimental animal studies. In order to evaluate current analytical approaches used in SRs of animal studies, easily identify all systematic reviews on a specific topic, and subsequently the original animal studies and their results and promote awareness and understanding of these emerging approaches, we compiled a database of SRs of animal studies. The database was developed using a rigorous, systematic approach and covers a broad range of research fields: preclinical research, toxicology, environmental health, and veterinary medicine. The database currently includes 3113 SRs of animal studies (search date June 2019). In addition to bibliographical information, data on whether or not a risk of bias assessment and meta-analysis were conducted were extracted. For future users, the search features of the database provide users with a platform to identify and select SRs with a particular characteristic for export to Microsoft Word or Microsoft Excel. From there, users may perform additional data extraction to meet their research needs. The database is freely available at www.Mendeley.com (link). The database provides methodologists a comprehensive source that can be used to explore and advance the current methodology applied to SRs of animal studies, and can help researchers to easily identify all systematic reviews on a specific topic, and subsequently the original animal studies and their results and avoid duplication and unnecessary animal research.

Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do

Systematic reviews are fast increasing in prevalence in the toxicology and environmental health literature. However, how well these complex research projects are being conducted and reported is unclear. Since editors have an essential role in ensuring the scientific quality of manuscripts being published in their journals, a workshop was convened where editors, systematic review practitioners, and research quality control experts could discuss what editors can do to ensure the systematic reviews they publish are of sufficient scientific quality. Interventions were explored along four themes: setting standards; reviewing protocols; optimizing editorial workflows; and measuring the effectiveness of editorial interventions. In total, 58 editorial interventions were proposed. Of these, 26 were shortlisted for being potentially effective, and 5 were prioritized as short-term actions that editors could relatively easily take to improve the quality of published systematic reviews. Recent progress in improving systematic reviews is summarized, and outstanding challenges to further progress are highlighted.

Avoiding Regrettable Substitutions: Green Toxicology for Sustainable Chemistry

Green chemistry seeks to design less hazardous chemicals, but many of the efforts to replace chemicals have resulted in so-called "Regrettable Substitutions", when a chemical with an unknown or unforeseen hazard is used to replace a chemical identified as problematic. Here, we discuss the literature on regrettable substitution and focus on an oft-mentioned case, Bisphenol A, which was replaced with Bisphenol S-and the lessons that can be learned from this history. In particular, we focus on how Green Toxicology can offer a way to make better substitutions.

Large scale meta-analysis of preclinical toxicity data for target characterisation and hypotheses generation

Recent technological advances in the field of big data have increased our capabilities to query large databases and combine information from different domains and disciplines. In the area of preclinical studies, initiatives like SEND (Standard for Exchange of Nonclinical Data) will also contribute to collect and present nonclinical data in a consistent manner and increase analytical possibilities. With facilitated access to preclinical data and improvements in analytical algorithms there will surely be an expectation for organisations to ensure all the historical data available to them is leveraged to build new hypotheses. These kinds of analyses may soon become as important as the animal studies themselves, in addition to being critical components to achieve objectives aligned with 3Rs. This article proposes the application of meta-analyses at large scale in corporate databases as a tool to exploit data from both preclinical studies and in vitro pharmacological activity assays to identify associations between targets and tissues that can be used as seeds for the development of causal hypotheses to characterise of targets. A total of 833 in-house preclinical toxicity studies relating to 416 compounds reported to be active (pXC50 ≥ 5.5) against a panel of 96 selected targets of interest for potential off-target non desired effects were meta-analysed, aggregating the data in tissue-target pairs. The primary outcome was the odds ratio (OR) of the number of animals with observed events (any morphology, any severity) in treated and control groups in the tissue analysed. This led to a total of 2139 meta-analyses producing a total of 364 statistically significant associations (random effects model), 121 after adjusting by multiple comparison bias. The results show the utility of the proposed approach to leverage historical corporate data and may offer a vehicle for researchers to share, aggregate and analyse their preclinical toxicological data in precompetitive environments.

Towards guidelines for time-trend reviews examining temporal variability in human biomonitoring data of pollutants

In the last few decades, a plethora of studies have focused on human biomonitoring (HBM) of chemical pollutants. Reviewing the copious HBM data reported in these studies is essential for evaluating the effectiveness of pollution management efforts, for example by evaluating time-trends. Nevertheless, guidance to systematically evaluate time trends in published HBM data has never been developed. In this study, we therefore present a proposal for guidelines to conduct "time-trend reviews" (TTRs) that examine time trends in published large HBM datasets of chemical pollutant concentrations. We also demonstrate the applicability of these guidelines through a case study that assesses time-trends in global and regional HBM data on mercury. The recommended TTR guidelines in this study are divided into seven steps: formulating the objective of the TTR, setting up of eligibility criteria, defining search strategy and screening of literature, screening results of search, extracting data, analysing data, and assessing certainty, including the potential for bias in the evidence base. The TTR guidelines proposed in this study are straightforward and less complex than those for conducting systematic reviews assessing datasets on potential human health effects of exposure to pollutants or medical interventions. These proposed guidelines are intended to enable the credible, transparent, and reproducible conduct of TTRs.

A Survey of Systematic Evidence Mapping Practice and the Case for Knowledge Graphs in Environmental Health and Toxicology

Systematic evidence mapping offers a robust and transparent methodology for facilitating evidence-based approaches to decision-making in chemicals policy and wider environmental health (EH). Interest in the methodology is growing; however, its application in EH is still novel. To facilitate the production of effective systematic evidence maps for EH use cases, we survey the successful application of evidence mapping in other fields where the methodology is more established. Focusing on issues of “data storage technology,” “data integrity,” “data accessibility,” and “transparency,” we characterize current evidence mapping practice and critically review its potential value for EH contexts. We note that rigid, flat data tables and schema-first approaches dominate current mapping methods and highlight how this practice is ill-suited to the highly connected, heterogeneous, and complex nature of EH data. We propose this challenge is overcome by storing and structuring data as “knowledge graphs.” Knowledge graphs offer a flexible, schemaless, and scalable model for systematically mapping the EH literature. Associated technologies, such as ontologies, are well-suited to the long-term goals of systematic mapping methodology in promoting resource-efficient access to the wider EH evidence base. Several graph storage implementations are readily available, with a variety of proven use cases in other fields. Thus, developing and adapting systematic evidence mapping for EH should utilize these graph-based resources to ensure the production of scalable, interoperable, and robust maps to aid decision-making processes in chemicals policy and wider EH.

Management of bias and conflict of interest among occupational exposure limit expert groups

Our aim was to evaluate policies and procedures for management of conflict of interest (CoI) and other sources of bias, implemented in Occupational Exposure Limit (OEL) expert groups. First, we compiled procedural criteria applicable to OEL-setting, based on literature on CoI and systematic reviews. Second, we identified 58 global OEL-sources and sought the underlying expert groups and operating procedures. We identified eleven active groups, of which five have documented CoI policies. In all five, CoI management is based on declarations of interests (DoIs) and removal of experts from decisions in which they have an interest. Notable differences include publication of DoIs (three of five groups), limitation of DoI to current interests (two groups), quantitative limits for financial interests (none specified to ≥€10,000 per interest), control procedures for undisclosed CoI (one group), and procedures in case of discovery of undisclosed CoI (three groups). Methods to evaluate study quality are described by three groups, while reproducible and comprehensive strategies to identify and select data receive less attention. We conclude that procedures to manage CoI and bias are not broadly implemented, or at least not openly and transparently communicated. This lack of visible procedures is remarkable, considering OEL's impact on health and economy.

Applying evidence-based methods to the development and use of adverse outcome pathways

The workshop “Application of evidence-based methods to construct mechanistic frameworks for the development and use of non-animal toxicity tests” was organized by the Evidence-based Toxicology Collaboration and hosted by the Grading of Recommendations Assessment, Development and Evaluation Working Group on June 12, 2019. The purpose of the workshop was to bring together international regulatory bodies, risk assessors, academic scientists, and industry to explore how systematic review methods and the adverse outcome pathway framework could be combined to develop and use mechanistic test methods for predicting the toxicity of chemical substances in an evidence-based manner. The meeting covered the history of biological frameworks, the way adverse outcome pathways are currently developed, the basic principles of systematic methodology, including systematic reviews and evidence maps, and assessment of cer­tainty in models, and adverse outcome pathways in particular. Specific topics were discussed via case studies in small break-out groups. The group concluded that adverse outcome pathways provide an important framework to support mechanism-based assessment in environmental health. The process of their development has a few challenges that could be addressed with systematic methods and automation tools. Addressing these challenges will increase the transparency of the evidence behind adverse outcome pathways and the consistency with which they are defined; this in turn will increase their value for supporting public health decisions. It was suggested to explore the details of applying systematic methods to adverse outcome pathway development in a series of case studies and workshops.

Reviewing the animal literature: how to describe and choose between different types of literature reviews

Before starting any (animal) research project, review of the existing literature is good practice. From both the scientific and the ethical perspective, high-quality literature reviews are essential. Literature reviews have many potential advantages besides synthesising the evidence for a research question. First, they can show if a proposed study has already been performed, preventing redundant research. Second, when planning new experiments, reviews can inform the experimental design, thereby increasing the reliability, relevance and efficiency of the study. Third, reviews may even answer research questions using already available data. Multiple definitions of the term literature review co-exist. In this paper, we describe the different steps in the review process, and the risks and benefits of using various methodologies in each step. We then suggest common terminology for different review types: narrative reviews, mapping reviews, scoping reviews, rapid reviews, systematic reviews and umbrella reviews. We recommend which review to select, depending on the research question and available resources. We believe that improved understanding of review methods and terminology will prevent ambiguity and increase appropriate interpretation of the conclusions of reviews.

Performance of preclinical models in predicting drug-induced liver injury in humans: a systematic review

Drug-induced liver injury (DILI) causes one in three market withdrawals due to adverse drug reactions, causing preventable human suffering and massive financial loss. We applied evidence-based methods to investigate the role of preclinical studies in predicting human DILI using two anti-diabetic drugs from the same class, but with different toxicological profiles: troglitazone (withdrawn from US market due to DILI) and rosiglitazone (remains on US market). Evidence Stream 1: A systematic literature review of in vivo studies on rosiglitazone or troglitazone was conducted (PROSPERO registration CRD42018112353). Evidence Stream 2: in vitro data on troglitazone and rosiglitazone were retrieved from the US EPA ToxCast database. Evidence Stream 3: troglitazone- and rosiglitazone-related DILI cases were retrieved from WHO Vigibase. All three evidence stream analyses were conducted according to evidence-based methodologies and performed according to pre-registered protocols. Evidence Stream 1: 9288 references were identified, with 42 studies included in analysis. No reported biomarker for either drug indicated a strong hazard signal in either preclinical animal or human studies. All included studies had substantial limitations, resulting in "low" or "very low" certainty in findings. Evidence Stream 2: Troglitazone was active in twice as many in vitro assays (129) as rosiglitazone (60), indicating a strong signal for more off-target effects. Evidence Stream 3: We observed a fivefold difference in both all adverse events and liver-related adverse events reported, and an eightfold difference in fatalities for troglitazone, compared to rosiglitazone. In summary, published animal and human trials failed to predict troglitazone's potential to cause severe liver injury in a wider patient population, while in vitro data showed marked differences in the two drugs' off-target activities, offering a new paradigm for reducing drug attrition in late development and in the market. This investigation concludes that death and disability due to adverse drug reactions may be prevented if mechanistic information is deployed at early stages of drug development by pharmaceutical companies and is considered by regulators as a part of regulatory submissions.

Mapping Chemical Respiratory Sensitization: How Useful Are Our Current Computational Tools?

Chemical respiratory sensitization is an immunological process that manifests clinically mostly as occupational asthma and is responsible for 1 in 6 cases of adult asthma, although this may be an underestimate of the prevalence, as it is under-diagnosed. Occupational asthma results in unemployment for roughly one-third of those affected due to severe health issues. Despite its high prevalence, chemical respiratory sensitization is difficult to predict, as there are currently no validated models and the mechanisms are not entirely understood, creating a significant challenge for regulatory bodies and industry alike. The Adverse Outcome Pathway (AOP) for respiratory sensitization is currently incomplete. However, some key events have been identified, and there is overlap with the comparatively well-characterized AOP for dermal sensitization. Because of this, and the fact that dermal sensitization is often assessed by in vivo, in chemico, or in silico methods, regulatory bodies are defaulting to the dermal sensitization status of chemicals as a proxy for respiratory sensitization status when evaluating chemical safety. We identified a data set of known human respiratory sensitizers, which we used to investigate the accuracy of a structural alert model, Toxtree, designed for skin sensitization and the Centre for Occupational and Environmental Health (COEH)'s model, a model developed specifically for occupational asthma. While both models had a reasonable level of accuracy, the COEH model achieved the highest balanced accuracy at 76%; when the models agreed, the overall accuracy was 87%. There were important differences between the models: Toxtree had superior performance for some structural alerts and some categories of well-characterized skin sensitizers, while the COEH model had high accuracy in identifying sensitizers that lacked identified skin sensitization reactivity domains. Overall, both models achieved respectable accuracy. However, neither model addresses potency, which, along with data quality, remains a hurdle, and the field must prioritize these issues to move forward.

Knowledge Organization Systems for Systematic Chemical Assessments

BACKGROUND

Although the implementation of systematic review and evidence mapping methods stands to improve the transparency and accuracy of chemical assessments, they also accentuate the challenges that assessors face in ensuring they have located and included all the evidence that is relevant to evaluating the potential health effects an exposure might be causing. This challenge of information retrieval can be characterized in terms of "semantic" and "conceptual" factors that render chemical assessments vulnerable to the streetlight effect.

OBJECTIVES

This commentary presents how controlled vocabularies, thesauruses, and ontologies contribute to overcoming the streetlight effect in information retrieval, making up the key components of Knowledge Organization Systems (KOSs) that enable more systematic access to assessment-relevant information than is currently achievable. The concept of Adverse Outcome Pathways is used to illustrate what a general KOS for use in chemical assessment could look like.

DISCUSSION

Ontologies are an underexploited element of effective knowledge organization in the environmental health sciences. Agreeing on and implementing ontologies in chemical assessment is a complex but tractable process with four fundamental steps. Successful implementation of ontologies would not only make currently fragmented information about health risks from chemical exposures vastly more accessible, it could ultimately enable computational methods for chemical assessment that can take advantage of the full richness of data described in natural language in primary studies. https://doi.org/10.1289/EHP6994.

Using benchmark dose modeling for the quantitative risk assessment: Carbon nanotubes, asbestos, glyphosate

Benchmark dose method is one of the most famous quantitative approaches available for toxicological risks prediction. However, it is not fully clear how occupational health professionals can use it for specific workplace scenarios requiring carcinogen risk assessment. The paper explores the hypothesis that benchmark dose method allows to effectively approximate dose-response data on carcinogenic response, providing reasonable estimations of risks in the situations when a choice between more complex models is not warranted for practical purposes. Three case studies were analyzed for the agents with different levels of scientific confidence in human carcinogenicity: carbon nanotubes, amosite asbestos, and glyphosate. For each agent, a critical study was determined, and a dose-response slope factor was quantified, based on the weighted average lower bound benchmark dose. The linear slope factors of 0.111 lifetime excess cases of lung carcinoma per mg/m³ of MWCNT-7 (in rats exposure equivalent), 0.009 cases of mesothelioma per f/cc-years of cumulative exposure to amosite asbestos, and 0.000094 cases of malignant lymphoma per mg/kg/day of glyphosate (in mice equivalent) were determined. The correlations between the proposed linear predictive models and observed data points were R = 0.96 (R² = 0.92) for carbon nanotubes, R = 0.97 (R² = 0.95) for amosite asbestos, and R = 0.89 (R² = 0.79) for glyphosate. In all three cases, the linear extrapolation yielded comparable level of risk estimations with the "best fit" nonlinear model; for nanoparticles and amosite asbestos, linear estimations were more conservative. By performing a simulation study, it was demonstrated that a weighted average benchmark dose expressed the highest correlation with multistage and quantal-linear models.

Facilitation of risk assessment with evidence-based methods - A framework for use of systematic mapping and systematic reviews in determining hazard, developing toxicity values, and characterizing uncertainty

Systematic review tools and approaches developed for clinical medicine are often difficult to apply "off the shelf" in order to meet the needs of chemical risk assessments. To address such, we propose an approach that can be used by practitioners for using evidence-based methods to facilitate the risk assessment process. The framework builds on and combines efforts conducted to date by a number of agencies and researchers; the novelty is in combining these efforts with a practical understanding of risk assessment, and translating such into a 'step-by-step' guide. The approach relies on three key components: problem formulation, systematic evidence mapping, and systematic review, applied using a stepwise approach. Unique to this framework is the consideration of exposure in selecting, prioritizing, and evaluating data (e.g., dose-relevance, routes of exposure, etc.). Using the proposed step-by-step process, critical appraisal of individual studies (e.g., formal and structured assessment of both relevance and reliability) and integration efforts are considered in context of specified risk assessment objectives (e.g., mode of action, dose-response) as well as chemical-specific considerations. The resulting framework provides a logical approach of how evidence-based methods can be used to facilitate risk assessment, and elevates the use of systematic methods beyond hazard identification to directly facilitating transparent and objective selection of candidate studies and/or datasets used to quantitatively characterize risk, and to better use the underlying process to inform the approaches used to develop toxicity values.

A case study applying pathway-oriented thinking to problem formulation for planning a systematic review

The use of evidence-based methods in chemical risk assessment (CRA) is still in its infancy. Novel approaches exploring how to implement Systematic Review (SR) principles and methods for evaluating human health risks from environmental chemical exposures are needed. This paper reports and comments on a conceptual model that was developed as part of a mapping exercise for planning a SR, using aluminium-containing antiperspirants (Al-AP) and female breast cancer risk as a case study. The work explores how knowledge-assembly tools and pathway-oriented thinking developed in systems toxicology can be applied to support problem formulation (PF) in the context of SR. A conceptual model was developed to map out key research questions, working hypotheses, routes of exposure, toxicity pathways and endpoints, and related health outcomes. The model draws on the analytic framework for screening topics of the U.S. Preventive Services Task Force and builds on the concept of a "source-to-outcome continuum", integrating knowledge gained from exposure pathway concepts such as the Aggregate Exposure Pathway and Adverse Outcome Pathways. The model can be used as a central decision and prioritization tool for scoping and framing Population, Exposure, Control, Outcome (PECO) questions in a transparent and iterative manner; as a supporting tool to guide the whole SR process; and to lay down the methodological foundation of a SR on the Al-AP breast cancer topic. Logic modelling can be easily combined with systems or pathway-oriented thinking, and allows for a more structured, objective and transparent approach to PF when applying SR methods to the CRA context.

Systematic evaluation of mechanistic data in assessing in utero exposures to trichloroethylene and development of congenital heart defects

The hypothesis that in utero exposures to low levels of trichloroethylene (TCE) may increase the risk of congenital heart defects (CHDs) in offspring remains a subject of substantial controversy within the scientific community due primarily to the reliance on an inconsistent and unreproducible experimental study in rats. To build on previous assessments that have primarily focused on epidemiological and experimental animal studies in developing conclusions, the objective of the current study is to conduct a systematic evaluation of mechanistic data related to in utero exposures to TCE and the development of CHDs. The evidence base was heterogeneous; 79 mechanistic datasets were identified, characterizing endpoints which ranged from molecular to organismal responses in seven species, involving both in vivo and in vitro study designs in mammalian and non-mammalian models. Of these, 24 datasets were considered reliable following critical appraisal using a study quality tool that employs metrics specific to the study type. Subsequent synthesis and integration demonstrated that the available mechanistic data: 1) did not support the potential for CHD hazard in humans, 2) did not support the biological plausibility of a response in humans based on organization via a putative adverse outcome pathway for valvulo-septal cardiac defects, and 3) were not suitable for serving as candidate studies in risk assessment. Findings supportive of an association were generally limited to in ovo chicken studies, in which TCE was administered in high concentration solutions via direct injection. Results of these in ovo studies were difficult to interpret for human health risk assessment given the lack of generalizability of the study models (including dose relevance, species-specific biological differences, variations in the construct of the study design, etc.). When the mechanistic data are integrated with findings from previous evaluations of human and animal evidence streams, the totality of evidence does not support CHDs as a critical effect in TCE human health risk assessment.

Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management

BACKGROUND

While systematic review (SR) methods are gaining traction as a method for providing a reliable summary of existing evidence for health risks posed by exposure to chemical substances, it is becoming clear that their value is restricted to a specific range of risk management scenarios - in particular, those which can be addressed with tightly focused questions and can accommodate the time and resource requirements of a systematic evidence synthesis.

METHODS

The concept of a systematic evidence map (SEM) is defined and contrasted to the function and limitations of systematic review (SR) in the context of risk management decision-making. The potential for SEMs to facilitate evidence-based decision-making are explored using a hypothetical example in risk management priority-setting. The potential role of SEMs in reference to broader risk management workflows is characterised.

RESULTS

SEMs are databases of systematically gathered research which characterise broad features of the evidence base. Although not intended to substitute for the evidence synthesis element of systematic reviews, SEMs provide a comprehensive, queryable summary of a large body of policy relevant research. They provide an evidence-based approach to characterising the extent of available evidence and support forward looking predictions or trendspotting in the chemical risk sciences. In particular, SEMs facilitate the identification of related bodies of decision critical chemical risk information which could be further analysed using SR methods, and highlight gaps in the evidence which could be addressed with additional primary studies to reduce uncertainties in decision-making.

CONCLUSIONS

SEMs have strong and growing potential as a high value tool in resource efficient use of existing research in chemical risk management. They can be used as a critical precursor to efficient deployment of high quality SR methods for characterising chemical health risks. Furthermore, SEMs have potential, at a large scale, to support the sort of evidence summarisation and surveillance methods which would greatly increase the resource efficiency, transparency and effectiveness of regulatory initiatives such as EU REACH and US TSCA.

Evaluating Chemicals for Thyroid Disruption: Opportunities and Challenges with in Vitro Testing and Adverse Outcome Pathway Approaches

BACKGROUND

Extensive clinical and experimental research documents the potential for chemical disruption of thyroid hormone (TH) signaling through multiple molecular targets. Perturbation of TH signaling can lead to abnormal brain development, cognitive impairments, and other adverse outcomes in humans and wildlife. To increase chemical safety screening efficiency and reduce vertebrate animal testing, in vitro assays that identify chemical interactions with molecular targets of the thyroid system have been developed and implemented.

OBJECTIVES

We present an adverse outcome pathway (AOP) network to link data derived from in vitro assays that measure chemical interactions with thyroid molecular targets to downstream events and adverse outcomes traditionally derived from in vivo testing. We examine the role of new in vitro technologies, in the context of the AOP network, in facilitating consideration of several important regulatory and biological challenges in characterizing chemicals that exert effects through a thyroid mechanism.

DISCUSSION

There is a substantial body of knowledge describing chemical effects on molecular and physiological regulation of TH signaling and associated adverse outcomes. Until recently, few alternative nonanimal assays were available to interrogate chemical effects on TH signaling. With the development of these new tools, screening large libraries of chemicals for interactions with molecular targets of the thyroid is now possible. Measuring early chemical interactions with targets in the thyroid pathway provides a means of linking adverse outcomes, which may be influenced by many biological processes, to a thyroid mechanism. However, the use of in vitro assays beyond chemical screening is complicated by continuing limits in our knowledge of TH signaling in important life stages and tissues, such as during fetal brain development. Nonetheless, the thyroid AOP network provides an ideal tool for defining causal linkages of a chemical exerting thyroid-dependent effects and identifying research needs to quantify these effects in support of regulatory decision making. https://doi.org/10.1289/EHP5297.

A question of trust: can we build an evidence base to gain trust in systematic review automation technologies?

BACKGROUND

Although many aspects of systematic reviews use computational tools, systematic reviewers have been reluctant to adopt machine learning tools.

DISCUSSION

We discuss that the potential reason for the slow adoption of machine learning tools into systematic reviews is multifactorial. We focus on the current absence of trust in automation and set-up challenges as major barriers to adoption. It is important that reviews produced using automation tools are considered non-inferior or superior to current practice. However, this standard will likely not be sufficient to lead to widespread adoption. As with many technologies, it is important that reviewers see "others" in the review community using automation tools. Adoption will also be slow if the automation tools are not compatible with workflows and tasks currently used to produce reviews. Many automation tools being developed for systematic reviews mimic classification problems. Therefore, the evidence that these automation tools are non-inferior or superior can be presented using methods similar to diagnostic test evaluations, i.e., precision and recall compared to a human reviewer. However, the assessment of automation tools does present unique challenges for investigators and systematic reviewers, including the need to clarify which metrics are of interest to the systematic review community and the unique documentation challenges for reproducible software experiments.

CONCLUSION

We discuss adoption barriers with the goal of providing tool developers with guidance as to how to design and report such evaluations and for end users to assess their validity. Further, we discuss approaches to formatting and announcing publicly available datasets suitable for assessment of automation technologies and tools. Making these resources available will increase trust that tools are non-inferior or superior to current practice. Finally, we identify that, even with evidence that automation tools are non-inferior or superior to current practice, substantial set-up challenges remain for main stream integration of automation into the systematic review process.

Language Consideration and Methodological Transparency in "Systematic" Reviews of Animal Toxicity Studies

This study evaluated the use of non-English literature (NEL) in systematic reviews (SRs) or meta-analyses (MAs) of animal-based toxicity or communicable disease (CD) studies. A secondary goal was to assess how grant funding, country of primary authorship, or study quality reporting influenced the use of NEL in these reviews. Inclusion criteria and data extraction forms were based on a pilot evaluation of a 10% random sample of reviews that were identified from a PubMed search (2006 to May 2017). This search yielded 111 animal toxicity and 69 CD reviews. Reviews (33 animal toxicity and 32 CD studies) were included when the authors identified their work as an SR or MA, described a literature search strategy, and provided defined inclusion criteria. Extracted data included PubMed indexing of publication type, author affiliations, and grant funding. Language use was mentioned in the methods in 55% of the toxicity SRs and 69% of CD SRs, of which 44% (n = 8) and 41% (n = 9) were limited to English, respectively. Neither the study type, grant funding, nor first author country of affiliation was associated with an increased consideration of NEL. Study quality reporting was more common in SRs that considered multiple languages. Despite guidelines that encourage the use of NEL in SRs and translation tools, SR/MA authors often fail to report language inclusion or focus on English publications. Librarian involvement in SR can promote awareness of relevant NEL and collaborative and technological strategies to improve their incorporation into the SR process.

Overview of known plastic packaging-associated chemicals and their hazards

Global plastics production has reached 380 million metric tons in 2015, with around 40% used for packaging. Plastic packaging is diverse and made of multiple polymers and numerous additives, along with other components, such as adhesives or coatings. Further, packaging can contain residues from substances used during manufacturing, such as solvents, along with non-intentionally added substances (NIAS), such as impurities, oligomers, or degradation products. To characterize risks from chemicals potentially released during manufacturing, use, disposal, and/or recycling of packaging, comprehensive information on all chemicals involved is needed. Here, we present a database of Chemicals associated with Plastic Packaging (CPPdb), which includes chemicals used during manufacturing and/or present in final packaging articles. The CPPdb lists 906 chemicals likely associated with plastic packaging and 3377 substances that are possibly associated. Of the 906 chemicals likely associated with plastic packaging, 63 rank highest for human health hazards and 68 for environmental hazards according to the harmonized hazard classifications assigned by the European Chemicals Agency within the Classification, Labeling and Packaging (CLP) regulation implementing the United Nations' Globally Harmonized System (GHS). Further, 7 of the 906 substances are classified in the European Union as persistent, bioaccumulative, and toxic (PBT), or very persistent, very bioaccumulative (vPvB), and 15 as endocrine disrupting chemicals (EDC). Thirty-four of the 906 chemicals are also recognized as EDC or potential EDC in the recent EDC report by the United Nations Environment Programme. The identified hazardous chemicals are used in plastics as monomers, intermediates, solvents, surfactants, plasticizers, stabilizers, biocides, flame retardants, accelerators, and colorants, among other functions. Our work was challenged by a lack of transparency and incompleteness of publicly available information on both the use and toxicity of numerous substances. The most hazardous chemicals identified here should be assessed in detail as potential candidates for substitution.

COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology

BACKGROUND

To our knowledge, no publication providing overarching guidance on the conduct of systematic reviews of observational studies of etiology exists.

METHODS AND FINDINGS

Conducting Systematic Reviews and Meta-Analyses of Observational Studies of Etiology (COSMOS-E) provides guidance on all steps in systematic reviews of observational studies of etiology, from shaping the research question, defining exposure and outcomes, to assessing the risk of bias and statistical analysis. The writing group included researchers experienced in meta-analyses and observational studies of etiology. Standard peer-review was performed. While the structure of systematic reviews of observational studies on etiology may be similar to that for systematic reviews of randomised controlled trials, there are specific tasks within each component that differ. Examples include assessment for confounding, selection bias, and information bias. In systematic reviews of observational studies of etiology, combining studies in meta-analysis may lead to more precise estimates, but such greater precision does not automatically remedy potential bias. Thorough exploration of sources of heterogeneity is key when assessing the validity of estimates and causality.

CONCLUSION

As many reviews of observational studies on etiology are being performed, this document may provide researchers with guidance on how to conduct and analyse such reviews.

Perspectives on In Vitro to In Vivo Extrapolations

Quantitative in vitro to in vivo extrapolation (QIVIVE) is broadly considered a prerequisite bridge from in vitro findings to a dose paradigm. Quality and relevance of cell systems are the first prerequisite for QIVIVE. Information-rich and mechanistic endpoints (biomarkers) improve extrapolations, but a sophisticated endpoint does not make a bad cell model a good one. The next need is reverse toxicokinetics (TK), which estimates the dose necessary to reach a tissue concentration that is active in vitro. The Johns Hopkins Center for Alternatives to Animal Testing (CAAT) has created a roadmap for animal-free systemic toxicity testing, in which the needs and opportunities for TK are elaborated, in the context of different systemic toxicities. The report was discussed at two stakeholder forums in Brussels in 2012 and in Washington in 2013; the key recommendations are summarized herein. Contrary to common belief and the Paracelsus paradigm of everything is toxic, the majority of industrial chemicals do not exhibit toxicity. Strengthening the credibility of negative results of alternative approaches for hazard identification, therefore, avoids the need for QIVIVE. Here, especially the combination of methods in integrated testing strategies is most promising. Two further but very different approaches aim to overcome the problem of modeling in vivo complexity: The human-on-a-chip movement aims to reproduce large parts of living organism's complexity via microphysiological systems, that is, organ equivalents combined by microfluidics. At the same time, the Toxicity Testing in the 21st Century (Tox-21c) movement aims for mechanistic approaches (adverse outcome pathways as promoted by Organisation for Economic Co-operation and Development (OECD) or pathways of toxicity in the Human Toxome Project) for high-throughput screening, biological phenotyping, and ultimately a systems toxicology approach through integration with computer modeling. These 21st century approaches also require 21st century validation, for example, by evidence-based toxicology. Ultimately, QIVIVE is a prerequisite for extrapolating Tox-21c such approaches to human risk assessment.

Big-data and machine learning to revamp computational toxicology and its use in risk assessment

The creation of large toxicological databases and advances in machine-learning techniques have empowered computational approaches in toxicology. Work with these large databases based on regulatory data has allowed reproducibility assessment of animal models, which highlight weaknesses in traditional in vivo methods. This should lower the bars for the introduction of new approaches and represents a benchmark that is achievable for any alternative method validated against these methods. Quantitative Structure Activity Relationships (QSAR) models for skin sensitization, eye irritation, and other human health hazards based on these big databases, however, also have made apparent some of the challenges facing computational modeling, including validation challenges, model interpretation issues, and model selection issues. A first implementation of machine learning-based predictions termed REACHacross achieved unprecedented sensitivities of >80% with specificities >70% in predicting the six most common acute and topical hazards covering about two thirds of the chemical universe. While this is awaiting formal validation, it demonstrates the new quality introduced by big data and modern data-mining technologies. The rapid increase in the diversity and number of computational models, as well as the data they are based on, create challenges and opportunities for the use of computational methods.