Assessing toxicological data quality: basic principles, existing schemes and current limitations

Semi-automated harmonization and selection of chemical data for risk and impact assessment

Chemical data for thousands of substances are available for safety, risk, life cycle and substitution assessments, as submitted for example under the European Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH) Regulation. However, to widely disseminate reported physicochemical properties as well as human and ecological exposure and toxicological data for use in various science and policy fields, systematic methods for data harmonization and selection are necessary. In response to this need, we developed a semi-automated method for deriving appropriate substance property values as input for various assessment frameworks with different requirements for resolution and data quality. Starting with data reported for a given substance and property, we propose a set of aligned data selection and harmonization criteria to obtain a representative mean value and related confidence intervals per chemical-property combination. The proposed method was tested on a set of octanol-water partition coefficients (K_ow) for an illustrative set of 20 substances, reported under the REACH regulation as example data source. Our method is generally applicable to any set of substances, and can assess specific distributions in quality and variability across reported data. Further research can likely extend our method for mining information from text fields and adapt it to available data reported or collected from other sources and other substance properties to improve the reliability of input data for risk and impact assessments.

In Silico Models for Repeated-Dose Toxicity (RDT): Prediction of the No Observed Adverse Effect Level (NOAEL) and Lowest Observed Adverse Effect Level (LOAEL) for Drugs

Many regulatory contexts require the evaluation of repeated-dose toxicity (RDT) studies conducted in laboratory animals. The main outcome of RDT studies is the identification of the no observed adverse effect level (NOAEL) and the lowest observed adverse effect level (LOAEL) that are normally used as point of departure for the establishment of health-based guidance values. Since in vivo RDT studies are expensive and time-consuming, in silico approaches could offer a valuable alternative. However, NOAEL and LOAEL modeling suffer some limitations since they do not refer to a single end point but to several different effects, and the doses used in experimental studies strongly influence the results. Few attempts to model NOAEL and LOAEL have been reported. The available database and models for the prediction of NOAEL and LOAEL are reviewed here.

Zeta potentials (ζ) of metal oxide nanoparticles: A meta-analysis of experimental data and a predictive neural networks modeling

Zeta potential is usually measured to estimate the surface charge and the stability of nanomaterials, as changes in these characteristics directly influence the biological activity of a given nanoparticle. Nowadays, theoretical methods are commonly used for a pre-screening safety assessments of nanomaterials. At the same time, the consistency of data on zeta potential measurements in the context of environmental impact is an important challenge. The inconsistency of data measurements leads to inaccuracies in predictive modeling. In this article, we report a new curated dataset of zeta potentials measured for 208 silica- and metal oxide nanoparticles in different media. We discuss the data curation framework for zeta potentials designed to assess the quality and usefulness of the literature data for further computational modeling. We also provide an analysis of specific trends for the datapoints harvested from different literature sources. In addition to that, we present for the first time a structure-property relationship model for nanoparticles (nano-SPR) that predicts values of zeta potential values measured in different environmental conditions (i.e., biological media and pH).

Functional Role of circRNAs in the Regulation of Fetal Development, Muscle Development, and Lactation in Livestock

circRNAs are a class of endogenous noncoding RNA molecules with closed loop structures. They are mainly responsible for regulating gene expression in eukaryotic cells. With the emergence of high-throughput RNA sequencing (RNA-Seq) and new types of bioinformatics tools, thousands of circRNAs have been discovered, making circRNA one of the research hotspots. Recent studies have shown that circRNAs play an important regulatory role in the growth, reproduction, and formation of livestock products. They can not only regulate mammalian fetal growth and development but also have important regulatory effects on livestock muscle development and lactation. In this review, we briefly introduce the putative biogenic pathways and regulatory functions of circRNA and highlight our understanding of circRNA and its latest advances in fetal development, muscle development, and lactation biogenesis as well as expression in livestock. This review will provide a theoretical basis for the research and development of related industries.

A Critical Review of Morphologic Findings and Data From 14 Toxicological Studies Involving Fish Exposures to Diclofenac

A number of studies have investigated the potential toxicity of the analgesic agent diclofenac (DCF) in various fish species under a diverse array of experimental conditions. Reported evidence of toxicity in these investigations is often strongly reliant on morphologic end points such as histopathology, immunohistochemistry, and transmission electron microscopy. However, it may be challenging for scientists who perform environmental hazard or risk determination to fully appreciate the intricacies of these specialized endpoints. Therefore, the purpose of the current review was to critically assess the quality of morphologic data in 14 papers that described the experimental exposure of fish to DCF. Areas of focus during this review included study design, diagnostic accuracy, magnitude of reported changes, data interpretation and presentation, and the credibility of individual reported findings. Positive attributes of some studies included robust experimental designs, accurate diagnoses, and straightforward and transparent data reporting. Issues identified in certain articles included diagnostic errors, failure to account for sampling and/or observer bias, failure to evaluate findings according to sex, exaggeration of lesion severity, interstudy inconsistencies, unexplained phenomena, and incomplete or ambiguous data presentation. It is hoped that the outcome of this review will be of value for personnel involved in regulatory decision-making.

Unlocking the potential of in silico chemical safety assessment - A report on a cross-sector symposium on current opportunities and future challenges

In silico chemical safety assessment can support the evaluation of hazard and risk following potential exposure to a substance. A symposium identified a number of opportunities and challenges to implement in silico methods, such as quantitative structure-activity relationships (QSARs) and read-across, to assess the potential harm of a substance in a variety of exposure scenarios, e.g. pharmaceuticals, personal care products, and industrial chemicals. To initiate the process of in silico safety assessment, clear and unambiguous problem formulation is required to provide the context for these methods. These approaches must be built on data of defined quality, while acknowledging the possibility of novel data resources tapping into on-going progress with data sharing. Models need to be developed that cover appropriate toxicity and kinetic endpoints, and that are documented appropriately with defined uncertainties. The application and implementation of in silico models in chemical safety requires a flexible technological framework that enables the integration of multiple strands of data and evidence. The findings of the symposium allowed for the identification of priorities to progress in silico chemical safety assessment towards the animal-free assessment of chemicals.

A critical review of adverse effects to the kidney: mechanisms, data sources, and in silico tools to assist prediction

INTRODUCTION

The kidney is a major target for toxicity elicited by pharmaceuticals and environmental pollutants. Standard testing which often does not investigate underlying mechanisms has proven not to be an adequate hazard assessment approach. As such, there is an opportunity for the application of computational approaches that utilize multiscale data based on the Adverse Outcome Pathway (AOP) paradigm, coupled with an understanding of the chemistry underpinning the molecular initiating event (MIE) to provide a deep understanding of how structural fragments of molecules relate to specific mechanisms of nephrotoxicity. Aims covered: The aim of this investigation was to review the current scientific landscape related to computational methods, including mechanistic data, AOPs, publicly available knowledge bases and current in silico models, for the assessment of pharmaceuticals and other chemicals with regard to their potential to elicit nephrotoxicity. A list of over 250 nephrotoxicants enriched with, where possible, mechanistic and AOP-derived understanding was compiled. Expert opinion: Whilst little mechanistic evidence has been translated into AOPs, this review identified a number of data sources of in vitro, in vivo, and human data that may assist in the development of in silico models which in turn may shed light on the interrelationships between nephrotoxicity mechanisms.

Perspectives from the NanoSafety Modelling Cluster on the validation criteria for (Q)SAR models used in nanotechnology

Nanotechnology and the production of nanomaterials have been expanding rapidly in recent years. Since many types of engineered nanoparticles are suspected to be toxic to living organisms and to have a negative impact on the environment, the process of designing new nanoparticles and their applications must be accompanied by a thorough risk analysis. (Quantitative) Structure-Activity Relationship ([Q]SAR) modelling creates promising options among the available methods for the risk assessment. These in silico models can be used to predict a variety of properties, including the toxicity of newly designed nanoparticles. However, (Q)SAR models must be appropriately validated to ensure the clarity, consistency and reliability of predictions. This paper is a joint initiative from recently completed European research projects focused on developing (Q)SAR methodology for nanomaterials. The aim was to interpret and expand the guidance for the well-known "OECD Principles for the Validation, for Regulatory Purposes, of (Q)SAR Models", with reference to nano-(Q)SAR, and present our opinions on the criteria to be fulfilled for models developed for nanoparticles.

Advancing alternatives analysis: The role of predictive toxicology in selecting safer chemical products and processes

Alternatives analysis (AA) is a method used in regulation and product design to identify, assess, and evaluate the safety and viability of potential substitutes for hazardous chemicals. It requires toxicological data for the existing chemical and potential alternatives. Predictive toxicology uses in silico and in vitro approaches, computational models, and other tools to expedite toxicological data generation in a more cost-effective manner than traditional approaches. The present article briefly reviews the challenges associated with using predictive toxicology in regulatory AA, then presents 4 recommendations for its advancement. It recommends using case studies to advance the integration of predictive toxicology into AA, adopting a stepwise process to employing predictive toxicology in AA beginning with prioritization of chemicals of concern, leveraging existing resources to advance the integration of predictive toxicology into the practice of AA, and supporting transdisciplinary efforts. The further incorporation of predictive toxicology into AA would advance the ability of companies and regulators to select alternatives to harmful ingredients, and potentially increase the use of predictive toxicology in regulation more broadly. Integr Environ Assess Manag 2017;13:915-925. © 2017 SETAC.

Comparison of the Predictive Performance and Interpretability of Random Forest and Linear Models on Benchmark Data Sets

The ability to interpret the predictions made by quantitative structure-activity relationships (QSARs) offers a number of advantages. While QSARs built using nonlinear modeling approaches, such as the popular Random Forest algorithm, might sometimes be more predictive than those built using linear modeling approaches, their predictions have been perceived as difficult to interpret. However, a growing number of approaches have been proposed for interpreting nonlinear QSAR models in general and Random Forest in particular. In the current work, we compare the performance of Random Forest to those of two widely used linear modeling approaches: linear Support Vector Machines (SVMs) (or Support Vector Regression (SVR)) and partial least-squares (PLS). We compare their performance in terms of their predictivity as well as the chemical interpretability of the predictions using novel scoring schemes for assessing heat map images of substructural contributions. We critically assess different approaches for interpreting Random Forest models as well as for obtaining predictions from the forest. We assess the models on a large number of widely employed public-domain benchmark data sets corresponding to regression and binary classification problems of relevance to hit identification and toxicology. We conclude that Random Forest typically yields comparable or possibly better predictive performance than the linear modeling approaches and that its predictions may also be interpreted in a chemically and biologically meaningful way. In contrast to earlier work looking at interpretation of nonlinear QSAR models, we directly compare two methodologically distinct approaches for interpreting Random Forest models. The approaches for interpreting Random Forest assessed in our article were implemented using open-source programs that we have made available to the community. These programs are the rfFC package ( https://r-forge.r-project.org/R/?group_id=1725 ) for the R statistical programming language and the Python program HeatMapWrapper [ https://doi.org/10.5281/zenodo.495163 ] for heat map generation.

A critical review of frameworks used for evaluating reliability and relevance of (eco)toxicity data: Perspectives for an integrated eco-human decision-making framework

Considerable efforts have been invested so far to evaluate and rank the quality and relevance of (eco)toxicity data for their use in regulatory risk assessment to assess chemical hazards. Many frameworks have been developed to improve robustness and transparency in the evaluation of reliability and relevance of individual tests, but these frameworks typically focus on either environmental risk assessment (ERA) or human health risk assessment (HHRA), and there is little cross talk between them. There is a need to develop a common approach that would support a more consistent, transparent and robust evaluation and weighting of the evidence across ERA and HHRA. This paper explores the applicability of existing Data Quality Assessment (DQA) frameworks for integrating environmental toxicity hazard data into human health assessments and vice versa. We performed a comparative analysis of the strengths and weaknesses of eleven frameworks for evaluating reliability and/or relevance of toxicity and ecotoxicity hazard data. We found that a frequent shortcoming is the lack of a clear separation between reliability and relevance criteria. A further gaps and needs analysis revealed that none of the reviewed frameworks satisfy the needs of a common eco-human DQA system. Based on our analysis, some key characteristics, perspectives and recommendations are identified and discussed for building a common DQA system as part of a future integrated eco-human decision-making framework. This work lays the basis for developing a common DQA system to support the further development and promotion of Integrated Risk Assessment.

How should the completeness and quality of curated nanomaterial data be evaluated?

Nanotechnology is of increasing significance. Curation of nanomaterial data into electronic databases offers opportunities to better understand and predict nanomaterials' behaviour. This supports innovation in, and regulation of, nanotechnology. It is commonly understood that curated data need to be sufficiently complete and of sufficient quality to serve their intended purpose. However, assessing data completeness and quality is non-trivial in general and is arguably especially difficult in the nanoscience area, given its highly multidisciplinary nature. The current article, part of the Nanomaterial Data Curation Initiative series, addresses how to assess the completeness and quality of (curated) nanomaterial data. In order to address this key challenge, a variety of related issues are discussed: the meaning and importance of data completeness and quality, existing approaches to their assessment and the key challenges associated with evaluating the completeness and quality of curated nanomaterial data. Considerations which are specific to the nanoscience area and lessons which can be learned from other relevant scientific disciplines are considered. Hence, the scope of this discussion ranges from physicochemical characterisation requirements for nanomaterials and interference of nanomaterials with nanotoxicology assays to broader issues such as minimum information checklists, toxicology data quality schemes and computational approaches that facilitate evaluation of the completeness and quality of (curated) data. This discussion is informed by a literature review and a survey of key nanomaterial data curation stakeholders. Finally, drawing upon this discussion, recommendations are presented concerning the central question: how should the completeness and quality of curated nanomaterial data be evaluated?

Mind the Gap! A Journey towards Computational Toxicology

Computational methods have advanced toxicology towards the development of target-specific models based on a clear cause-effect rationale. However, the predictive potential of these models presents strengths and weaknesses. On the good side, in silico models are valuable cheap alternatives to in vitro and in vivo experiments. On the other, the unconscious use of in silico methods can mislead end-users with elusive results. The focus of this review is on the basic scientific and regulatory recommendations in the derivation and application of computational models. Attention is paid to examine the interplay between computational toxicology and drug discovery and development. Avoiding the easy temptation of an overoptimistic future, we report our view on what can, or cannot, realistically be done. Indeed, studies of safety/toxicity represent a key element of chemical prioritization programs carried out by chemical industries, and primarily by pharmaceutical companies.

In Silico Models for Repeated-Dose Toxicity (RDT): Prediction of the No Observed Adverse Effect Level (NOAEL) and Lowest Observed Adverse Effect Level (LOAEL) for Drugs

The preclinical stage in drug development requires the determination of repeated-dose toxicity (RDT) in animal models. The main outcome of RDT studies is the determination of the no observed adverse effect level (NOAEL) and the lowest observed adverse effect level (LOAEL). NOAEL is important since it serves to calculate the maximum recommended starting dose (MRSD) which is the safe starting dose for clinical studies in human beings. Since in vivo RDT studies are expensive and time-consuming, in silico approaches could offer a valuable alternative. However, NOAEL and LOAEL modeling suffer some limitations since they do not refer to a single end point but to several different effects and the doses used in experimental studies strongly influence the final results. Few attempts to model NOAEL and LOAEL have been reported. The available database and models for the prediction of NOAEL and LOAEL are reviewed here.

Summary and Analysis of the Currently Existing Literature Data on Metal-based Nanoparticles Published for Selected Aquatic Organisms: Applicability for Toxicity Prediction by (Q)SARs

This review establishes an inventory of existing toxicity data on nanoparticles (NPs) with the purpose of developing (Quantitative) Structure–Activity Relationships for NPs (nano-[Q]SARs), and also of maximising the use of scientific sources for NP risk assessment. From a data search carried out on 27 February 2014, a total of 910 publications were retrieved from the Web of Science™ Core Collection, and a database comprising 886 records of toxicity endpoints, based on these publications, was built. The test organisms mainly comprised bacteria, algae, yeast, protozoa, nematoda, crustacea and fish. The NPs consisted mostly of metals, metal oxides, nanocomposites and quantum dots. The data were analysed further, in order to: a) categorise each toxicity endpoint and the biological effects triggered by the NPs; b) survey the characterisation of the NPs used; and c) assess whether the data were suitable for nano-(Q)SAR development. Despite the efforts of numerous scientific programmes on nanomaterial safety and design, our study concluded that lack of data consistency prevents the use of experimental data in developing and validating nano-(Q)SARs. Finally, an outlook on the future of nano-(Q)SAR development is provided.

Ensuring confidence in predictions: A scheme to assess the scientific validity of in silico models

The use of in silico tools within the drug development process to predict a wide range of properties including absorption, distribution, metabolism, elimination and toxicity has become increasingly important due to changes in legislation and both ethical and economic drivers to reduce animal testing. Whilst in silico tools have been used for decades there remains reluctance to accept predictions based on these methods particularly in regulatory settings. This apprehension arises in part due to lack of confidence in the reliability, robustness and applicability of the models. To address this issue we propose a scheme for the verification of in silico models that enables end users and modellers to assess the scientific validity of models in accordance with the principles of good computer modelling practice. We report here the implementation of the scheme within the Innovative Medicines Initiative project "eTOX" (electronic toxicity) and its application to the in silico models developed within the frame of this project.

Stepwise Information-Filtering Tool (SIFT): A method for using risk assessment metadata in a nontraditional way

Tools exist to evaluate large ecotoxicity databases for risk assessment purposes, but these tools are less useful for alternative analytical purposes. In the present study, the authors developed the Stepwise Information-Filtering Tool (SIFT), a strategic method to select relevant, reliable data from a large ecotoxicity database; demonstrated utility in a case study of chronic toxicity data for statistical endpoint comparison purposes; and evaluated SIFT by comparison with 2 existing data evaluation methods.

Evaluation of the ToxRTool's ability to rate the reliability of toxicological data for human health hazard assessments

Regulatory agencies often utilize results from peer reviewed publications for hazard assessments. A problem in doing so is the lack of well-accepted tools to objectively, efficiently and systematically assess the quality of published toxicological studies. Herein, we evaluated the publicly available software-based ToxRTool (Toxicological data Reliability assessment Tool) for use in human health hazard assessments. The ToxRTool was developed by the European Commission's Joint Research Center in 2009. It builds on Klimisch categories, a rating system established in 1997, by providing additional criteria and guidance for assessing the reliability of toxicological studies. It also transparently documents the study-selection process. Eight scientists used the ToxRTool to rate the same 20 journal articles on thyroid toxicants. Results were then compared using the Finn coefficient and "AC1" to determine inter-rater consistency. Ratings were most consistent for high-quality journal articles, but less consistent as study quality decreased. Primary reasons for inconsistencies were that some criteria were subjective and some were not clearly described. It was concluded, however, that the ToxRTool has potential and, with refinement, could provide a more objective approach for screening published toxicology studies for use in health risk evaluations, although the ToxRTool ratings are primarily based on study reporting quality.

Methods for assigning confidence to toxicity data with multiple values--Identifying experimental outliers

The assessment of data quality is a crucial element in many disciplines such as predictive toxicology and risk assessment. Currently, the reliability of toxicity data is assessed on the basis of testing information alone (adherence to Good Laboratory Practice (GLP), detailed testing protocols, etc.). Common practice is to take one toxicity data point per compound - usually the one with the apparently highest reliability. All other toxicity data points (for the same experiment and compound) from other sources are neglected. To show the benefits of incorporating the "less reliable" data, a simple, independent, statistical approach to assess data quality and reliability on a mathematical basis was developed. A large data set of toxicity values to Aliivibrio fischeri was assessed. The data set contained 1813 data points for 1227 different compounds, including 203 identified as non-polar narcotic. Log KOW values were calculated and non-polar narcosis quantitative structure-activity relationship (QSAR) models were built. A statistical approach to data quality assessment, which is based on data outlier omission and confidence scoring, improved the linear QSARs. The results indicate that a beneficial method for using large data sets containing multiple data values per compound and highly variable study data has been developed. Furthermore this statistical approach can help to develop novel QSARs and support risk assessment by obtaining more reliable values for biological endpoints.

Sources of Chemical Information, Toxicity Data and Assessment of Their Quality

This chapter identifies a range of sources that provide toxicity data that may be of use in category formation and read­across. Data in this context relate to both the chemical identity and characteristics of molecules in addition to biological (toxicological) information. Different methods of representing chemicals are given and caveats associated with the use of certain representations are also indicated. A glossary of key terms relating to assessment of data quality is provided along with guidance on methods to perform data quality assessment.

From data point timelines to a well curated data set, data mining of experimental data and chemical structure data from scientific articles, problems and possible solutions

The scientific literature is important source of experimental and chemical structure data. Very often this data has been harvested into smaller or bigger data collections leaving the data quality and curation issues on shoulders of users. The current research presents a systematic and reproducible workflow for collecting series of data points from scientific literature and assembling a database that is suitable for the purposes of high quality modelling and decision support. The quality assurance aspect of the workflow is concerned with the curation of both chemical structures and associated toxicity values at (1) single data point level and (2) collection of data points level. The assembly of a database employs a novel "timeline" approach. The workflow is implemented as a software solution and its applicability is demonstrated on the example of the Tetrahymena pyriformis acute aquatic toxicity endpoint. A literature collection of 86 primary publications for T. pyriformis was found to contain 2,072 chemical compounds and 2,498 unique toxicity values, which divide into 2,440 numerical and 58 textual values. Every chemical compound was assigned to a preferred toxicity value. Examples for most common chemical and toxicological data curation scenarios are discussed.

Evaluation criteria for the quality of published experimental data on nanomaterials and their usefulness for QSAR modelling

Nowadays nanotechnology is one of the most promising areas of science. The number and quantity of synthesized nanomaterials increase exponentially, therefore it is reasonable to expect that comprehensive risk assessment based only on empirical testing of all novel engineered nanoparticles (NPs) will very soon become impossible. Hence, the development of computational methods complementary to experimentation is very important. Quantitative structure-property relationship (QSPR) and quantitative structure-activity relationship (QSAR) models widely used in pharmaceutical chemistry and environmental science can also be modified and adopted for nanotechnology to predict physico-chemical properties and toxicity of empirically untested nanomaterials. All QSPR/QSAR modelling activities are based on experimentally derived data. It is important that, within a given data set, all values should be consistent, of high quality and measured according to a standardized protocol. Unfortunately, the amount of such data available for engineered nanoparticles in various data sources (i.e. databases and the literature) is very limited and seldom measured with a standardized protocol. Therefore, we have proposed a framework for collecting and evaluating the existing data, with the focus on possible applications for computational evaluation of properties and biological activities of nanomaterials.