Applying a modified systematic review and integrated assessment framework (SYRINA) - a case study on triphenyl phosphate

This work presents a case study in applying a systematic review framework (SYRINA) to the identification of chemicals as endocrine disruptors. The suitability and performance of the framework is tested with regard to the widely accepted World Health Organization definition of an endocrine disruptor (ED). The endocrine disrupting potential of triphenyl phosphate (TPP), a well-studied flame retardant reported to exhibit various endocrine related effects was assessed. We followed the 7 steps of the SYRINA framework, articulating the research objective via Populations, Exposures, Comparators, Outcomes (PECO) statements, performed literature search and screening, conducted study evaluation, performed data extraction and summarized and integrated the evidence. Overall, 66 studies, consisting of in vivo, in vitro and epidemiological data, were included. We concluded that triphenyl phosphate could be identified as an ED based on metabolic disruption and reproductive function. We found that the tools used in this case study and the optimizations performed on the framework were suitable to assess properties of EDs. A number of challenges and areas for methodological development in systematic appraisal of evidence relating to endocrine disrupting potential were identified; significant time and effort were needed for the analysis of in vitro mechanistic data in this case study, thus increasing the workload and time needed to perform the systematic review process. Further research and development of this framework with regards to grey literature (non-peer-reviewed literature) search, harmonization of study evaluation methods, more consistent evidence integration approaches and a pre-defined method to assess links between adverse effect and endocrine activity are recommended. It would also be advantageous to conduct more case studies for a chemical with less data than TPP.

Reconciling chemical flame retardant exposure and fire risk in domestic furniture

INTRODUCTION

Evidence suggests that standards for resistance of furniture to ignition may lead to an increase in use of chemical flame retardants (CFRs). This is motivating the development of new approaches that maintain high levels of fire safety while facilitating a reduction in use of CFRs. However, reconciling potential fire risk with use of CFRs in relation to specific policy objectives is challenging.

OBJECTIVES

To inform the development of a new policy in the UK for the fire safety of furniture, we developed for domestic furniture quantitative models of fire risk and potential for CFR exposure. We then combined the models to determine if any lower fire risk, higher CFR exposure categories of furniture were identifiable.

METHODS

We applied a novel mixed-methods approach to modelling furniture fire risk and CFR exposure in a data-poor environment, using literature-based concept mapping, qualitative research, and data visualisation methods to generate fire risk and CFR exposure models and derive furniture product rankings.

RESULTS

Our analysis suggests there exists a cluster of furniture types including baby and infant products and pillows that have comparable overall properties in terms of lower fire risk and higher potential for CFR exposure.

DISCUSSION

There are multiple obstacles to reconciling fire risk and CFR use in furniture. In particular, these include a lack of empirical data that would allow absolute fire risk and exposure levels to be quantified. Nonetheless, it seems that our modelling method can potentially yield meaningful product clusters, providing a basis for further research.

Protocol for designing INVITES-IN, a tool for assessing the internal validity of in vitro studies

This protocol describes the design and development of a tool for evaluation of the internal validity of in vitro studies, which is needed to include the data as evidence in systematic reviews and chemical risk assessments. The tool will be designed specifically to be applied to cell culture studies, including, but not restricted to, studies meeting the new approach methodology (NAM) definition. The tool is called INVITES-IN (IN VITro Experimental Studies INternal validity). In this protocol, three of the four studies that will be performed to create the release version of INVITES-IN are described. In the first study, evaluation of existing assessment tools will be combined with focus group discussions to identify how characteristics of the design or conduct of an in vitro study can affect its internal validity. Bias domains and items considered to be of relevance for in vitro studies will be identified. In the second study, group agreement on internal validity domains and items of importance for in vitro studies will be identified via a modified Delphi methodology. In the third study, the draft version of the tool will be created, based on the data on relevance and importance of bias domains and items collected in Studies 1 and 2. A separate protocol will be prepared for the fourth study, which includes the user testing and validation of the tool, and collection of users' experience.

A new consensus on reconciling fire safety with environmental & health impacts of chemical flame retardants

Flame retardants are chemical substances that are intended to mitigate fire safety risks posed by a range of goods including furniture, electronics, and building insulation. There are growing concerns about their effectiveness in ensuring fire safety and the potential harms they pose to human health and the environment. In response to these concerns, on 13 June 2022, a roundtable of experts was convened by the UKRI Six Clean Air Strategic Priorities Fund programme 7. The meeting produced a Consensus Statement that summarises the issues around the use of flame retardants, laying out a series of policy recommendations that should lead to more effective fire safety measures and reduce the human and environmental health risks posed by these potentially toxic chemicals.

How we promote rigour in systematic reviews and evidence maps at Environment International

In 2016, Environment International became the first environmental health journal to adopt specialist policies for handling systematic review (SR) submissions. This included the appointment of a dedicated editor of SRs, the use of the CREST_Triage tool for transparent and consistent enforcement of editorial standards for SRs, the acceptance of SR protocols as full manuscripts, and the extension of SR handling policies to systematic evidence maps as a novel evidence synthesis methodology. Our data on triage decisions for SR submissions, gathered via CREST_Triage, indicates several ways in which researchers are challenged by SR methods, including problem formulation, critical appraisal methods, and certainty assessment. We recommend that author teams invest in developing protocols as a means to de-risk SR projects, arguing that the benefits outweigh the potential increase in time it may take to complete the research project. Finally, we present evidence that reliance among environmental health journals on informal peer-review and editorial checks for standards compliance and quality control is insufficient for ensuring the rigour of SR publications. This emphasises the importance of specialist editors using triage instruments for the effective enforcement of standards. Observing that Environment International appears to be one of few journals implementing effective quality control measures for SR publications, we suggest that adoption of our SR policies by other journals may be beneficial to the field at large.

Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks

The workshop titled “Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks” was co-organized by the Evidence-based Toxicology Collaboration and the European Food Safety Authority (EFSA) and hosted by EFSA at its headquarters in Parma, Italy on October 2 and 3, 2019. The goal was to explore integration of systematic review with mechanistic evidence evaluation. Participants were invited to work on concrete products to advance the exploration of how evidence-based approaches can support the development and application of adverse outcome pathways (AOP) in chemical risk assessment. The workshop discussions were centered around three related themes: 1) assessing certainty in AOPs, 2) literature-based AOP development, and 3) integrating certainty in AOPs and non-animal evidence into decision frameworks. Several challenges, mostly related to methodology, were identified and largely determined the workshop recommendations. The workshop recommendations included the comparison and potential alignment of processes used to develop AOP and systematic review methodology, including the translation of vocabulary of evidence-based methods to AOP and vice versa, the development and improvement of evidence mapping and text mining methods and tools, as well as a call for a fundamental change in chemical risk and uncertainty assessment methodology if to be conducted based on AOPs and new approach methodologies (NAM). The usefulness of evidence-based approaches for mechanism-based chemical risk assessments was stressed, particularly the potential contribution of the rigor and transparency inherent to such approaches in building stakeholders’ trust for implementation of NAM evidence and AOPs into chemical risk assessment.

Towards a framework for systematic reviews of the prevalence of exposure to environmental and occupational risk factors

Exposure prevalence studies (as here defined) record the prevalence of exposure to environmental and occupational risk factors to human health. Applying systematic review methods to the synthesis of these studies would improve the rigour and transparency of normative products produced based on this evidence (e.g., exposure prevalence estimates). However, a dedicated framework, including standard methods and tools, for systematically reviewing exposure prevalence studies has yet to be created. We describe the need for this framework and progress made towards it through a series of such systematic reviews that the World Health Organization and the International Labour Organization conducted for their WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury (WHO/ILO Joint Estimates).We explain that existing systematic review frameworks for environmental and occupational health cannot be directly applied for the generation of exposure prevalence estimates because they seek to synthesise different types of evidence (e.g., intervention or exposure effects on health) for different purposes (e.g., identify intervention effectiveness or exposure toxicity or carcinogenicity). Concepts unique to exposure prevalence studies (e.g., "expected heterogeneity": the real, non-spurious variability in exposure prevalence due to exposure changes over space and/or time) also require new assessment methods. A framework for systematic reviews of prevalence of environmental and occupational exposures requires adaptation of existing methods (e.g., a standard protocol) and development of new tools or approaches (e.g., for assessing risk of bias and certainty of a body of evidence, including exploration of expected heterogeneity).As part of the series of systematic reviews for the WHO/ILO Joint Estimates, the World Health Organization collaborating with partners has created a preliminary framework for systematic reviews of prevalence studies of exposures to occupational risk factors. This included development of protocol templates, data extraction templates, a risk of bias assessment tool, and an approach for assessing certainty of evidence in these studies. Further attention and efforts are warranted from scientific and policy communities, especially exposure scientists and policy makers, to establish a standard framework for comprehensive and transparent systematic reviews of studies estimating prevalence of exposure to environmental and occupational risk factors, to improve estimates, risk assessments and guidelines.

Biological plausibility in environmental health systematic reviews: a GRADE concept paper

BACKGROUND

"Biological plausibility" is a concept frequently referred to in environmental and public health when researchers are evaluating how confident they are in the results and inferences of a study or evidence review. Biological plausibility is not, however, a domain of one of the most widely-used approaches for assessing the certainty of evidence (CoE) which underpins the findings of a systematic review, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) CoE Framework. Whether the omission of biological plausibility is a potential limitation of the GRADE CoE Framework is a topic that is regularly discussed, especially in the context of environmental health systematic reviews.

OBJECTIVES

We analyse how the concept of "biological plausibility", as applied in the context of assessing certainty of the evidence that supports the findings of a systematic review, is accommodated under the processes of systematic review and the existing GRADE domains.

RESULTS AND DISCUSSION

We argue that "biological plausibility" is a concept which primarily comes into play when direct evidence about the effects of an exposure on a population of concern (usually humans) is absent, at high risk of bias, is inconsistent, or limited in other ways. In such circumstances, researchers look toward evidence from other study designs in order to draw conclusions. In this respect, we can consider experimental animal and in vitro evidence as "surrogates" for the target populations, exposures, comparators and outcomes of actual interest. Through discussion of 10 examples of experimental surrogates, we propose that the concept of biological plausibility consists of two principal aspects: a "generalisability aspect" and a "mechanistic aspect". The "generalisability aspect" concerns the validity of inferences from experimental models to human scenarios, and asks the same question as does the assessment of external validity or indirectness in systematic reviews. The "mechanistic aspect" concerns certainty in knowledge of biological mechanisms and would inform judgements of indirectness under GRADE, and thus the overall CoE. While both aspects are accommodated under the indirectness domain of the GRADE CoE Framework, further research is needed to determine how to use knowledge of biological mechanisms in the assessment of indirectness of the evidence in systematic reviews.

Principles and framework for assessing the risk of bias for studies included in comparative quantitative environmental systematic reviews

The internal validity of conclusions about effectiveness or impact in systematic reviews, and of decisions based on them, depends on risk of bias assessments being conducted appropriately. However, a random sample of 50 recently-published articles claiming to be quantitative environmental systematic reviews found 64% did not include any risk of bias assessment, whilst nearly all that did omitted key sources of bias. Other limitations included lack of transparency, conflation of quality constructs, and incomplete application of risk of bias assessments to the data synthesis. This paper addresses deficiencies in risk of bias assessments by highlighting core principles that are required for risk of bias assessments to be fit-for-purpose, and presenting a framework based on these principles to guide review teams on conducting risk of bias assessments appropriately and consistently. The core principles require that risk of bias assessments be Focused, Extensive, Applied and Transparent (FEAT). These principles support risk of bias assessments, appraisal of risk of bias tools, and the development of new tools. The framework follows a Plan-Conduct-Apply-Report approach covering all stages of risk of bias assessment. The scope of this paper is comparative quantitative environmental systematic reviews which address PICO or PECO-type questions including, but not limited to, topic areas such as environmental management, conservation, ecosystem restoration, and analyses of environmental interventions, exposures, impacts and risks.

Assessing author willingness to enter study information into structured data templates as part of the manuscript submission process: A pilot study

Background

Environmental health and other researchers can benefit from automated or semi-automated summaries of data within published studies as summarizing study methods and results is time and resource intensive. Automated summaries can be designed to identify and extract details of interest pertaining to the study design, population, testing agent/intervention, or outcome (etc.). Much of the data reported across existing publications lack unified structure, standardization and machine-readable formats or may be presented in complex tables which serve as barriers that impede the development of automated data extraction methodologies.

As full automation of data extraction seems unlikely soon, encouraging investigators to submit structured summaries of methods and results in standardized formats with meta-data tagging of content may be of value during the publication process. This would produce machine-readable content to facilitate automated data extraction, establish sharable data repositories, help make research data FAIR, and could improve reporting quality.

Objectives

A pilot study was conducted to assess the feasibility of asking participants to summarize study methods and results using a structured, web-based data extraction model as a potential workflow that could be implemented during the manuscript submission process.

Methods

Eight participants entered study details and data into the Health Assessment Workplace Collaborative (HAWC). Participants were surveyed after the extraction exercise to ascertain 1) whether this extraction exercise will impact their conducting and reporting of future research, 2) the ease of data extraction, including which fields were easiest and relatively more problematic to extract and 3) the amount of time taken to perform data extractions and other related tasks. Investigators then presented participants the potential benefits of providing structured data in the format they were extracting. After this, participants were surveyed about 1) their willingness to provide structured data during the publication process and 2) whether they felt the potential application of structured data entry approaches and their implementation during the journal submission process should continue to be further explored.

Conclusions

Routine provision of structured data that summarizes key information from research studies could reduce the amount of effort required for reusing that data in the future, such as in systematic reviews or agency scientific assessments. Our pilot study suggests that directly asking authors to provide that data, via structured templates, may be a viable approach to achieving this: participants were willing to do so, and the overall process was not prohibitively arduous. We also found some support for the hypothesis that use of study templates may have halo benefits in improving the conduct and completeness of reporting of future research. While limitations in the generalizability of our findings mean that the conditions of success of templates cannot be assumed, further research into how such templates might be designed and implemented does seem to have enough chance of success that it ought to be undertaken.

How evidence-based methodologies can help identify and reduce uncertainty in chemical risk assessment

Evidence-based methodology, in particular systematic review, is increasingly being applied in environmental, public, and occupational health to increase the transparency, comprehensiveness, and objectivity of the processes by which existing evidence is gathered, assessed, and synthesized in answering research questions. This development is also changing risk assessment practices and will impact the assessment of uncertainties in the evidence for risks to human health that are posed by exposure to chemicals. The potential of evidence-based methodology for characterizing uncertainties in risk assessment has been widely recognized, while its contribution to uncertainty reduction is yet to be fully elucidated. We therefore present some key aspects of the evidence-based approach to risk assessment, showing how they can contribute to the identification and the assessment of uncertainties. We focus on the pre-specification of an assessment method­ology in a protocol, comprehensive search strategies, study selection using predefined eligibility criteria, critical appraisal of individual studies, and an evidence integration and uncertainty characterization process based on certainty of evi­dence frameworks that are well-established in health care research. We also provide examples of uncertainty in risk assessment and discuss how evidence-based methodology could address those. This perspective, which neither claims to be comprehensive nor complete, is intended to stimulate discussion of the topic and to motivate detailed exploration of how evidence-based methodology contributes to characterization of uncertainties, and how it will lead to uncertainty reduction in the conduct of health risk assessment.

An approach to quantifying the potential importance of residual confounding in systematic reviews of observational studies: A GRADE concept paper

Small relative effect sizes are common in observational studies of exposure in environmental and public health. However, such effects can still have considerable policy importance when the baseline rate of the health outcome is high, and many persons are exposed. Assessing the certainty of the evidence based on these effect sizes is challenging because they can be prone to residual confounding due to the non-randomized nature of the evidence. When applying GRADE, a precise relative risk >2.0 increases the certainty in an existing effect because residual confounding is unlikely to explain the association. GRADE also suggests rating up when opposing plausible residual confounding exists for other effect sizes. In this concept paper, we propose using the E-value, defined as the smallest effect size of a confounder that still can reduce an observed RR to the null value, and a reference confounder to assess the likelihood of residual confounding. We propose a 4-step approach. 1. Assess the association of interest for relevant exposure levels. 2. Calculate the E-value for this observed association. 3. Choose a reference confounder with sufficient strength and information and assess its effect on the observed association using the E-value. 4. Assess how likely it is that residual confounding will still bias the observed RR. We present three case studies and discuss the feasibility of the approach.

Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do

Systematic reviews are fast increasing in prevalence in the toxicology and environmental health literature. However, how well these complex research projects are being conducted and reported is unclear. Since editors have an essential role in ensuring the scientific quality of manuscripts being published in their journals, a workshop was convened where editors, systematic review practitioners, and research quality control experts could discuss what editors can do to ensure the systematic reviews they publish are of sufficient scientific quality. Interventions were explored along four themes: setting standards; reviewing protocols; optimizing editorial workflows; and measuring the effectiveness of editorial interventions. In total, 58 editorial interventions were proposed. Of these, 26 were shortlisted for being potentially effective, and 5 were prioritized as short-term actions that editors could relatively easily take to improve the quality of published systematic reviews. Recent progress in improving systematic reviews is summarized, and outstanding challenges to further progress are highlighted.

Towards guidelines for time-trend reviews examining temporal variability in human biomonitoring data of pollutants

In the last few decades, a plethora of studies have focused on human biomonitoring (HBM) of chemical pollutants. Reviewing the copious HBM data reported in these studies is essential for evaluating the effectiveness of pollution management efforts, for example by evaluating time-trends. Nevertheless, guidance to systematically evaluate time trends in published HBM data has never been developed. In this study, we therefore present a proposal for guidelines to conduct "time-trend reviews" (TTRs) that examine time trends in published large HBM datasets of chemical pollutant concentrations. We also demonstrate the applicability of these guidelines through a case study that assesses time-trends in global and regional HBM data on mercury. The recommended TTR guidelines in this study are divided into seven steps: formulating the objective of the TTR, setting up of eligibility criteria, defining search strategy and screening of literature, screening results of search, extracting data, analysing data, and assessing certainty, including the potential for bias in the evidence base. The TTR guidelines proposed in this study are straightforward and less complex than those for conducting systematic reviews assessing datasets on potential human health effects of exposure to pollutants or medical interventions. These proposed guidelines are intended to enable the credible, transparent, and reproducible conduct of TTRs.

A Survey of Systematic Evidence Mapping Practice and the Case for Knowledge Graphs in Environmental Health and Toxicology

Systematic evidence mapping offers a robust and transparent methodology for facilitating evidence-based approaches to decision-making in chemicals policy and wider environmental health (EH). Interest in the methodology is growing; however, its application in EH is still novel. To facilitate the production of effective systematic evidence maps for EH use cases, we survey the successful application of evidence mapping in other fields where the methodology is more established. Focusing on issues of “data storage technology,” “data integrity,” “data accessibility,” and “transparency,” we characterize current evidence mapping practice and critically review its potential value for EH contexts. We note that rigid, flat data tables and schema-first approaches dominate current mapping methods and highlight how this practice is ill-suited to the highly connected, heterogeneous, and complex nature of EH data. We propose this challenge is overcome by storing and structuring data as “knowledge graphs.” Knowledge graphs offer a flexible, schemaless, and scalable model for systematically mapping the EH literature. Associated technologies, such as ontologies, are well-suited to the long-term goals of systematic mapping methodology in promoting resource-efficient access to the wider EH evidence base. Several graph storage implementations are readily available, with a variety of proven use cases in other fields. Thus, developing and adapting systematic evidence mapping for EH should utilize these graph-based resources to ensure the production of scalable, interoperable, and robust maps to aid decision-making processes in chemicals policy and wider EH.

Applying evidence-based methods to the development and use of adverse outcome pathways

The workshop “Application of evidence-based methods to construct mechanistic frameworks for the development and use of non-animal toxicity tests” was organized by the Evidence-based Toxicology Collaboration and hosted by the Grading of Recommendations Assessment, Development and Evaluation Working Group on June 12, 2019. The purpose of the workshop was to bring together international regulatory bodies, risk assessors, academic scientists, and industry to explore how systematic review methods and the adverse outcome pathway framework could be combined to develop and use mechanistic test methods for predicting the toxicity of chemical substances in an evidence-based manner. The meeting covered the history of biological frameworks, the way adverse outcome pathways are currently developed, the basic principles of systematic methodology, including systematic reviews and evidence maps, and assessment of cer­tainty in models, and adverse outcome pathways in particular. Specific topics were discussed via case studies in small break-out groups. The group concluded that adverse outcome pathways provide an important framework to support mechanism-based assessment in environmental health. The process of their development has a few challenges that could be addressed with systematic methods and automation tools. Addressing these challenges will increase the transparency of the evidence behind adverse outcome pathways and the consistency with which they are defined; this in turn will increase their value for supporting public health decisions. It was suggested to explore the details of applying systematic methods to adverse outcome pathway development in a series of case studies and workshops.

Knowledge Organization Systems for Systematic Chemical Assessments

BACKGROUND

Although the implementation of systematic review and evidence mapping methods stands to improve the transparency and accuracy of chemical assessments, they also accentuate the challenges that assessors face in ensuring they have located and included all the evidence that is relevant to evaluating the potential health effects an exposure might be causing. This challenge of information retrieval can be characterized in terms of "semantic" and "conceptual" factors that render chemical assessments vulnerable to the streetlight effect.

OBJECTIVES

This commentary presents how controlled vocabularies, thesauruses, and ontologies contribute to overcoming the streetlight effect in information retrieval, making up the key components of Knowledge Organization Systems (KOSs) that enable more systematic access to assessment-relevant information than is currently achievable. The concept of Adverse Outcome Pathways is used to illustrate what a general KOS for use in chemical assessment could look like.

DISCUSSION

Ontologies are an underexploited element of effective knowledge organization in the environmental health sciences. Agreeing on and implementing ontologies in chemical assessment is a complex but tractable process with four fundamental steps. Successful implementation of ontologies would not only make currently fragmented information about health risks from chemical exposures vastly more accessible, it could ultimately enable computational methods for chemical assessment that can take advantage of the full richness of data described in natural language in primary studies. https://doi.org/10.1289/EHP6994.

Recommendations for the conduct of systematic reviews in toxicology and environmental health research (COSTER)

BACKGROUND

There are several standards that offer explicit guidance on good practice in systematic reviews (SRs) for the medical sciences; however, no similarly comprehensive set of recommendations has been published for SRs that focus on human health risks posed by exposure to environmental challenges, chemical or otherwise.

OBJECTIVES

To develop an expert, cross-sector consensus view on a key set of recommended practices for the planning and conduct of SRs in the environmental health sciences.

METHODS

A draft set of recommendations was derived from two existing standards for SRs in biomedicine and developed in a consensus process, which engaged international participation from government, industry, non-government organisations, and academia. The consensus process consisted of a workshop, follow-up webinars, email discussion and bilateral phone calls.

RESULTS

The Conduct of Systematic Reviews in Toxicology and Environmental Health Research (COSTER) recommendations cover 70 SR practices across eight performance domains. Detailed explanations for specific recommendations are made for those identified by the authors as either being novel to SR in general, specific to the environmental health SR context, or potentially controversial to environmental health SR stakeholders.

DISCUSSION

COSTER provides a set of recommendations that should facilitate the production of credible, high-value SRs of environmental health evidence, and advance discussion of a number of controversial aspects of conduct of EH SRs. Key recommendations include the management of conflicts of interest, handling of grey literature, and protocol registration and publication. A process for advancing from COSTER's recommendations to developing a formal standard for EH SRs is also indicated.

Artificial Intelligence for chemical risk assessment

As the basis for managing the risks of chemical exposure, the Chemical Risk Assessment (CRA) process can impact a substantial part of the economy, the health of hundreds of millions of people, and the condition of the environment. However, the number of properly assessed chemicals falls short of societal needs due to a lack of experts for evaluation, interference of third party interests, and the sheer volume of potentially relevant information on the chemicals from disparate sources. In order to explore ways in which computational methods may help overcome this discrepancy between the number of chemical risk assessments required on the one hand and the number and adequateness of assessments actually being conducted on the other, the European Commission's Joint Research Centre organised a workshop on Artificial Intelligence for Chemical Risk Assessment (AI4CRA). The workshop identified a number of areas where Artificial Intelligence could potentially increase the number and quality of regulatory risk management decisions based on CRA, involving process simulation, supporting evaluation, identifying problems, facilitating collaboration, finding experts, evidence gathering, systematic review, knowledge discovery, and building cognitive models. Although these are interconnected, they are organised and discussed under two main themes: scientific-technical process and social aspects and the decision making process.

Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management

BACKGROUND

While systematic review (SR) methods are gaining traction as a method for providing a reliable summary of existing evidence for health risks posed by exposure to chemical substances, it is becoming clear that their value is restricted to a specific range of risk management scenarios - in particular, those which can be addressed with tightly focused questions and can accommodate the time and resource requirements of a systematic evidence synthesis.

METHODS

The concept of a systematic evidence map (SEM) is defined and contrasted to the function and limitations of systematic review (SR) in the context of risk management decision-making. The potential for SEMs to facilitate evidence-based decision-making are explored using a hypothetical example in risk management priority-setting. The potential role of SEMs in reference to broader risk management workflows is characterised.

RESULTS

SEMs are databases of systematically gathered research which characterise broad features of the evidence base. Although not intended to substitute for the evidence synthesis element of systematic reviews, SEMs provide a comprehensive, queryable summary of a large body of policy relevant research. They provide an evidence-based approach to characterising the extent of available evidence and support forward looking predictions or trendspotting in the chemical risk sciences. In particular, SEMs facilitate the identification of related bodies of decision critical chemical risk information which could be further analysed using SR methods, and highlight gaps in the evidence which could be addressed with additional primary studies to reduce uncertainties in decision-making.

CONCLUSIONS

SEMs have strong and growing potential as a high value tool in resource efficient use of existing research in chemical risk management. They can be used as a critical precursor to efficient deployment of high quality SR methods for characterising chemical health risks. Furthermore, SEMs have potential, at a large scale, to support the sort of evidence summarisation and surveillance methods which would greatly increase the resource efficiency, transparency and effectiveness of regulatory initiatives such as EU REACH and US TSCA.

Toward Good In Vitro Reporting Standards

A good experiment reported badly is worthless. Meaningful contributions to the body of science are made by sharing the full methodology and results so that they can be evaluated and reproduced by peers. Erroneous and incomplete reporting does not do justice to the resources spent on conducting the experiment and the time peers spend reading the article. In theory peer-review should ensure adequate reporting - in practice it does not. Many areas have developed reporting standards and checklists to support the adequate reporting of scientific efforts, but in vitro research still has no generally accepted criteria. It is characterized by a "Wild West" or "anything goes" attitude. Such a culture may undermine trust in the reproducibility of animal-free methods, and thus parallel the "reproducibility crisis" discussed for other life science fields. The increasing data retrieval needs of computational approaches (in extreme as "big data" and artificial intelligence) makes reporting quality even more important so that the scientific community can take full advantage of the results. The first priority of reporting standards is to ensure the completeness and transparency of information provided (data focus). The second tier is a quality of data display that makes information digestible and easy to grasp, compare and further analyze (information focus). This article summarizes a series of initiatives geared towards improving the quality of in vitro work and its reporting. This shall ultimately lead to Good In Vitro Reporting Standards (GIVReSt).

A risk of bias instrument for non-randomized studies of exposures: A users' guide to its application in the context of GRADE

The objective of this paper is to explain how to apply, interpret, and present the results of a new instrument to assess the risk of bias (RoB) in non-randomized studies (NRS) dealing with effects of environmental exposures on health outcomes. This instrument is modeled on the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) instrument. The RoB instrument for NRS of exposures assesses RoB along a standardized comparison to a randomized target experiment, instead of the study-design directed RoB approach. We provide specific guidance for the integral steps of developing a research question and target experiment, distinguishing issues of indirectness from RoB, making individual-study judgments, and performing and interpreting sensitivity analyses for RoB judgments across a body of evidence. Also, we present an approach for integrating the RoB assessments within the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework to assess the certainty of the evidence in the systematic review. Finally, we guide the reader through an overall assessment to support the rating of all domains that determine the certainty of a body of evidence using the GRADE approach.

A proposed framework for the systematic review and integrated assessment (SYRINA) of endocrine disrupting chemicals

BACKGROUND

The issue of endocrine disrupting chemicals (EDCs) is receiving wide attention from both the scientific and regulatory communities. Recent analyses of the EDC literature have been criticized for failing to use transparent and objective approaches to draw conclusions about the strength of evidence linking EDC exposures to adverse health or environmental outcomes. Systematic review methodologies are ideal for addressing this issue as they provide transparent and consistent approaches to study selection and evaluation. Objective methods are needed for integrating the multiple streams of evidence (epidemiology, wildlife, laboratory animal, in vitro, and in silico data) that are relevant in assessing EDCs.

METHODS

We have developed a framework for the systematic review and integrated assessment (SYRINA) of EDC studies. The framework was designed for use with the International Program on Chemical Safety (IPCS) and World Health Organization (WHO) definition of an EDC, which requires appraisal of evidence regarding 1) association between exposure and an adverse effect, 2) association between exposure and endocrine disrupting activity, and 3) a plausible link between the adverse effect and the endocrine disrupting activity.

RESULTS

Building from existing methodologies for evaluating and synthesizing evidence, the SYRINA framework includes seven steps: 1) Formulate the problem; 2) Develop the review protocol; 3) Identify relevant evidence; 4) Evaluate evidence from individual studies; 5) Summarize and evaluate each stream of evidence; 6) Integrate evidence across all streams; 7) Draw conclusions, make recommendations, and evaluate uncertainties. The proposed method is tailored to the IPCS/WHO definition of an EDC but offers flexibility for use in the context of other definitions of EDCs.

CONCLUSIONS

When using the SYRINA framework, the overall objective is to provide the evidence base needed to support decision making, including any action to avoid/minimise potential adverse effects of exposures. This framework allows for the evaluation and synthesis of evidence from multiple evidence streams. Finally, a decision regarding regulatory action is not only dependent on the strength of evidence, but also the consequences of action/inaction, e.g. limited or weak evidence may be sufficient to justify action if consequences are serious or irreversible.

Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations

Systematic review (SR) is a rigorous, protocol-driven approach designed to minimise error and bias when summarising the body of research evidence relevant to a specific scientific question. Taking as a comparator the use of SR in synthesising research in healthcare, we argue that SR methods could also pave the way for a "step change" in the transparency, objectivity and communication of chemical risk assessments (CRA) in Europe and elsewhere. We suggest that current controversies around the safety of certain chemicals are partly due to limitations in current CRA procedures which have contributed to ambiguity about the health risks posed by these substances. We present an overview of how SR methods can be applied to the assessment of risks from chemicals, and indicate how challenges in adapting SR methods from healthcare research to the CRA context might be overcome. Regarding the latter, we report the outcomes from a workshop exploring how to increase uptake of SR methods, attended by experts representing a wide range of fields related to chemical toxicology, risk analysis and SR. Priorities which were identified include: the conduct of CRA-focused prototype SRs; the development of a recognised standard of reporting and conduct for SRs in toxicology and CRA; and establishing a network to facilitate research, communication and training in SR methods. We see this paper as a milestone in the creation of a research climate that fosters communication between experts in CRA and SR and facilitates wider uptake of SR methods into CRA.

Stability Assessment of 10 Active Pharmaceutical Ingredients Compounded in SyrSpend SF

The stability of 10 active pharmaceutical ingredients was studied in SyrSpend SF PH4 or SyrSpend SF Alka at room and/or refrigerated temperature (2°C to 8°C). An oral suspension of each active pharmaceutical ingredient was compounded in low actinic plastic bottles at a specific concentration in SyrSpend SF PH4 or SyrSpend SF Alka. Samples were assessed for stability immediately after preparation (day 0) followed by storage at room temperature and/or at refrigerated temperature. At set time points, the samples were removed from storage and assayed using a high-performance liquid chromatographic stability- indicating method. The active pharmaceutical ingredient was considered stable if the suspension retained 90% to 110% of the initial concentration. Furosemide was stable for at least 14 days in SyrSpend SF Alka at refrigerated conditions. Prednisolone sodium phosphate in SyrSpend SF PH4 was stable for at least 30 days at room temperature and refrigerated conditions. Ranitidine hydrochloride suspensions in SyrSpend SF PH4 at room temperature and refrigerated conditions were stable for at least 30 days and 58 days, respectively. Hydrocortisone hemisuccinate and sodium phosphate retained greater than 90% for at least 60 days at both room temperature and refrigerated samples in SyrSpend SF PH4. Amiodarone hydrochloride and nifedipine suspensions at both room temperature and refrigerated conditions retained greater than 90% of the initial concentrations for at least 90 days in SyrSpend SF PH4. Refrigerated samples of simvastatin in SyrSpend SF PH4 were stable for at least 90 days. Spironolactone in SyrSpend SF PH4 at room temperature retained more than 90% of the initial concentration for at least 90 days. Phenobarbital in SyrSpend SF PH4 retained above 90% of initial concentration for at least 154 days at room temperature. This study demonstrated the stability of a wide range of frequently used active pharmaceutical ingredients, tested in SyrSpend SF PH4 and SyrSpend SF Alka at different storage conditions.

Stability of rifampin in SyrSpend SF

Rifampin is a bactericidal antibiotic drug of the rifamycin group. It is a semisynthetic drug produced from the bacterium Streptomyces mediterranei. Rifampin is commonly manufactured in capsule, tablet, and syrup dosage solutions containing alcohol or sorbitol. The objective of this study was to determine the stability of rifampin in SyrSpend SF. The studied samples were compounded into 25-mg/mL suspensions and stored in low-actinic bottles at room temperature and refrigerated conditions. Samples were assayed at each time point out to 60 days by a stability-indicating high-performance liquid chromatography method. The method was validated for its specificity through forced-degradation studies. The sample remained within 90% to 110% of the initial concentration throughout the course of the study. Based on data collected, the beyond-use date of the preparation is at least 60 days when refrigerated or stored at room temperature and protected from light.

Processing of cues for stop consonant voicing by young hearing-impaired listeners

To assess whether young hearing-impaired listeners are as sensitive as normal-hearing children to the cues for stop consonant voicing, we presented stimuli from along VOT continua to young normal-hearing listeners and to listeners with mild, moderate, severe, and profound hearing impairments. The response measures were the location of the phonetic boundaries, the change in boundaries with changes in place of articulation, and response variability. The listeners with normal hearing sensitivity and those with mild and moderate hearing impairments did not differ in performance on any response measure. The listeners with severe impairments did not show the expected change in VOT boundary with changes in place of articulation. Moreover, stimulus uncertainty (i.e., the number of possible choices in the response set) affected their response variability. One listener with profound impairment was able to process the cues for voicing in a normal fashion under conditions of minimum stimulus uncertainty. We infer from these results that the cochlear damage which underlies mild and moderate hearing impairment does not significantly alter the auditory representation of VOT. However, the cochlear damage underlying severe impairment, possibly interacting with high signal presentation levels, does alter the auditory representation of VOT.

Identification and discrimination of a synthesized voicing contrast by normal and sensorineural hearing-impaired children

The identification and discrimination of a stop-consonant voicing contrast (/da/--/ta/) was assessed in children and adolescents who had moderate, severe, and profound sensorineural hearing losses. The location of the perceptual boundary between /da/ and /ta/ did not differ between normal listeners and listeners with moderate losses. Of the ten listeners with severe losses, five evidenced normal boundaries, three evidenced longer-than-normal boundaries, and two could not identify the stimuli at all. Of the three listeners with profound hearing losses, one could identify normally, and two could not identify at all. For the most part, discrimination data mirrored identification data. However, in some instances listeners were able to discriminate between stimuli they could not differentially identify. These subjects appeared to have the auditory capacity to resolve differences in voice-onset-time but could not use this capacity to make phonetic identification.