The eTRANSAFE Project on Translational Safety Assessment through Integrative Knowledge Management: Achievements and Perspectives

Cross study analyses of SEND data: toxicity profile classification

A SEND toxicology data transformation, harmonization, and analysis platform were created to improve the identification of unique findings related to the intended target, species, and duration of dosing using data from multiple studies. The lack of a standardized digital format for data analysis had impeded large-scale analysis of in vivo toxicology studies. The CDISC SEND standard enables the analysis of data from multiple studies performed by different laboratories. This work describes methods to analyze data and automate cross-study analysis of toxicology studies. Cross-study analysis can be used to understand a single compound's toxicity profile across all studies performed and/or to evaluate on-target versus off-target toxicity for multiple compounds intended for the same pharmacological target. This work involved development of data harmonization/transformation strategies to enable cross-study analysis of both numerical and categorical SEND data. Four de-identified SEND datasets from the BioCelerate database were used for the analyses. Toxicity profiles for key organ systems were developed for liver, kidney, male reproductive tract, endocrine system, and hematopoietic system using SEND domains. A cross-study analysis dashboard with a built-in user-defined scoring system was created for custom analyses, including visualizations to evaluate data at the organ system level and drill down into individual animal data. This data analysis provides the tools for scientists to compare toxicity profiles across multiple studies using SEND. A cross-study analysis of 2 different compounds intended for the same pharmacological target is described and the analyses indicate potential on-target effects to liver, kidney, and hematopoietic systems.

Points to consider regarding the use and implementation of virtual controls in nonclinical general toxicology studies

The replacement of a proportion of concurrent controls by virtual controls in nonclinical safety studies has gained traction over the last few years. This is supported by foundational work, encouraged by regulators, and aligned with societal expectations regarding the use of animals in research. This paper provides an overview of the points to consider for any institution on the verge of implementing this concept, with emphasis given on database creation, risks, and discipline-specific perspectives.

Mining Real-World Big Data to Characterize Adverse Drug Reaction Quantitatively: Mixed Methods Study

BACKGROUND

Adverse drug reactions (ADRs), which are the phenotypic manifestations of clinical drug toxicity in humans, are a major concern in precision clinical medicine. A comprehensive evaluation of ADRs is helpful for unbiased supervision of marketed drugs and for discovering new drugs with high success rates.

OBJECTIVE

In current practice, drug safety evaluation is often oversimplified to the occurrence or nonoccurrence of ADRs. Given the limitations of current qualitative methods, there is an urgent need for a quantitative evaluation model to improve pharmacovigilance and the accurate assessment of drug safety.

METHODS

In this study, we developed a mathematical model, namely the Adverse Drug Reaction Classification System (ADReCS) severity-grading model, for the quantitative characterization of ADR severity, a crucial feature for evaluating the impact of ADRs on human health. The model was constructed by mining millions of real-world historical adverse drug event reports. A new parameter called Severity_score was introduced to measure the severity of ADRs, and upper and lower score boundaries were determined for 5 severity grades.

RESULTS

The ADReCS severity-grading model exhibited excellent consistency (99.22%) with the expert-grading system, the Common Terminology Criteria for Adverse Events. Hence, we graded the severity of 6277 standard ADRs for 129,407 drug-ADR pairs. Moreover, we calculated the occurrence rates of 6272 distinct ADRs for 127,763 drug-ADR pairs in large patient populations by mining real-world medication prescriptions. With the quantitative features, we demonstrated example applications in systematically elucidating ADR mechanisms and thereby discovered a list of drugs with improper dosages.

CONCLUSIONS

In summary, this study represents the first comprehensive determination of both ADR severity grades and ADR frequencies. This endeavor establishes a strong foundation for future artificial intelligence applications in discovering new drugs with high efficacy and low toxicity. It also heralds a paradigm shift in clinical toxicity research, moving from qualitative description to quantitative evaluation.

Drug review process advancement and required manufacturer and contract research oraganization responses

The United States Senate passed the "FDA Modernization Act 2.0." on September 29, 2022. Although the effectiveness of this Bill, which aims to eliminate the mandatory use of laboratory animals in new drug development, is limited, it represents a significant trend that will change the shape of drug applications in the United States and other countries. However, pharmaceutical companies have not taken major steps towards the complete elimination of animal testing from the standpoint of product safety, where they prioritize patient safety. Nonetheless, society is becoming increasingly opposed to animal testing, and efforts will be made to use fewer animals and conduct fewer animal tests as a natural and reasonable response. These changes eventually alter the shape of new drug applications. Based on the assumption that fewer animal tests will be conducted or fewer animals will be used in testing, this study explored bioinformatics and new technologies as alternatives to compensate for reduced information and provide a picture of how future new drug applications may look. The authors also discuss the directions that pharmaceutical companies and nonclinical contract research organizations should adopt to promote the replacement, reduction, and refinement of animals used in research, teaching, testing, and exhibitions.

ELIXIR and Toxicology: a community in development

Toxicology has been an active research field for many decades, with academic, industrial and government involvement. Modern omics and computational approaches are changing the field, from merely disease-specific observational models into target-specific predictive models. Traditionally, toxicology has strong links with other fields such as biology, chemistry, pharmacology and medicine. With the rise of synthetic and new engineered materials, alongside ongoing prioritisation needs in chemical risk assessment for existing chemicals, early predictive evaluations are becoming of utmost importance to both scientific and regulatory purposes. ELIXIR is an intergovernmental organisation that brings together life science resources from across Europe. To coordinate the linkage of various life science efforts around modern predictive toxicology, the establishment of a new ELIXIR Community is seen as instrumental. In the past few years, joint efforts, building on incidental overlap, have been piloted in the context of ELIXIR. For example, the EU-ToxRisk, diXa, HeCaToS, transQST, and the nanotoxicology community have worked with the ELIXIR TeSS, Bioschemas, and Compute Platforms and activities. In 2018, a core group of interested parties wrote a proposal, outlining a sketch of what this new ELIXIR Toxicology Community would look like. A recent workshop (held September 30th to October 1st, 2020) extended this into an ELIXIR Toxicology roadmap and a shortlist of limited investment-high gain collaborations to give body to this new community. This Whitepaper outlines the results of these efforts and defines our vision of the ELIXIR Toxicology Community and how it complements other ELIXIR activities.

CSL-Tox: an open-source analytical framework for the comparison of short-term and long-term toxicity end points and assessing the need of chronic studies in drug development

In-vivo toxicity assessment is an important step prior to clinical development and is still the main source of data for overall risk assessment of a new molecular entity (NCE). All in-vivo studies are performed according to regulatory requirements and many efforts have been exerted to minimize these studies in accordance with the (Replacement, Reduction and Refinement) 3Rs principle. Many aspects of in-vivo toxicology packages can be optimized to reduce animal use, including the number of studies performed as well as study durations, which is the main focus of this analysis. We performed a statistical comparison of adverse findings observed in 116 short-term versus 78 long-term in-house or in-house sponsored Contract Research Organizations (CRO) studies, in order to explore the possibility of using only short-term studies as a prediction tool for the longer-term effects. All the data analyzed in this study was manually extracted from the toxicology reports (in PDF formats) to construct the dataset. Annotation of treatment related findings was one of the challenges faced during this work. A specific focus was therefore put on the summary and conclusion sections of the reports since they contain expert assessments on whether the findings were considered adverse or were attributed to other reasons. Our analysis showed a general good concordance between short-term and long-term toxicity findings for large molecules and the majority of small molecules. Less concordance was seen for certain body organs, which can be named as "target organ systems' findings". While this work supports the minimization of long-term studies, a larger-scale effort would be needed to provide more evidence. We therefore present the steps performed in this study as an open-source R workflow for the Comparison of Short-term and Long-term Toxicity studies (CSL-Tox). The dataset used in the work is provided to allow researchers to reproduce such analysis, re-evaluate the statistical tools used and promote large-scale application of this study. Important aspects of animal research reproducibility are highlighted in this work, specifically, the necessity of a reproducible adverse effects reporting system and utilization of the controlled terminologies in-vivo toxicology reports and finally the importance of open-source analytical workflows that can be assessed by other scientists in the field of preclinical toxicology.

Identifying Differences in the Performance of Machine Learning Models for Off-Targets Trained on Publicly Available and Proprietary Data Sets

Each year, publicly available databases are updated with new compounds from different research institutions. Positive experimental outcomes are more likely to be reported; therefore, they account for a considerable fraction of these entries. Established publicly available databases such as ChEMBL allow researchers to use information without constrictions and create predictive tools for a broad spectrum of applications in the field of toxicology. Therefore, we investigated the distribution of positive and nonpositive entries within ChEMBL for a set of off-targets and its impact on the performance of classification models when applied to pharmaceutical industry data sets. Results indicate that models trained on publicly available data tend to overpredict positives, and models based on industry data sets predict negatives more often than those built using publicly available data sets. This is strengthened even further by the visualization of the prediction space for a set of 10,000 compounds, which makes it possible to identify regions in the chemical space where predictions converge. Finally, we highlight the utilization of these models for consensus modeling for potential adverse events prediction.

Perspectives of data science in preclinical safety assessment

The data landscape in preclinical safety assessment is fundamentally changing because of not only emerging new data types, such as human systems biology, or real-world data (RWD) from clinical trials, but also technological advancements in data-processing software and analytical tools based on deep learning approaches. The recent developments of data science are illustrated with use cases for the three factors: predictive safety (new in silico tools), insight generation (new data for outstanding questions); and reverse translation (extrapolating from clinical experience to resolve preclinical questions). Further advances in this field can be expected if companies focus on overcoming identified challenges related to a lack of platforms and data silos and assuring appropriate training of data scientists within the preclinical safety teams.

Making in silico predictive models for toxicology FAIR

In silico predictive models for toxicology include quantitative structure-activity relationship (QSAR) and physiologically based kinetic (PBK) approaches to predict physico-chemical and ADME properties, toxicological effects and internal exposure. Such models are used to fill data gaps as part of chemical risk assessment. There is a growing need to ensure in silico predictive models for toxicology are available for use and reproducible. This paper describes how the FAIR (Findable, Accessible, Interoperable, Reusable) principles, developed for data sharing, have been applied to in silico predictive models. In particular, this investigation has focussed on how the FAIR principles could be applied to improved regulatory acceptance of predictions from such models. Eighteen principles have been developed that cover all aspects of FAIR. It is intended that FAIRification of in silico predictive models for toxicology will increase their use and acceptance.

The evolving role of investigative toxicology in the pharmaceutical industry

For decades, preclinical toxicology was essentially a descriptive discipline in which treatment-related effects were carefully reported and used as a basis to calculate safety margins for drug candidates. In recent years, however, technological advances have increasingly enabled researchers to gain insights into toxicity mechanisms, supporting greater understanding of species relevance and translatability to humans, prediction of safety events, mitigation of side effects and development of safety biomarkers. Consequently, investigative (or mechanistic) toxicology has been gaining momentum and is now a key capability in the pharmaceutical industry. Here, we provide an overview of the current status of the field using case studies and discuss the potential impact of ongoing technological developments, based on a survey of investigative toxicologists from 14 European-based medium-sized to large pharmaceutical companies.

Retrospective analysis of the potential use of virtual control groups in preclinical toxicity assessment using the eTOX database

Virtual Control Groups (VCGs) based on Historical Control Data (HCD) in preclinical toxicity testing have the potential to reduce animal usage. As a case study we retrospectively analyzed the impact of replacing Concurrent Control Groups (CCGs) with VCGs on the treatment-relatedness of 28 selected histopathological findings reported in either rat or dog in the eTOX database. We developed a novel methodology whereby statistical predictions of treatment-relatedness using either CCGs or VCGs of varying covariate similarity to CCGs were compared to designations from original toxicologist reports; and changes in agreement were used to quantify changes in study outcomes. Generally, the best agreement was achieved when CCGs were replaced with VCGs with the highest level of similarity; the same species, strain, sex, administration route, and vehicle. For example, balanced accuracies for rat findings were 0.704 (predictions based on CCGs) vs. 0.702 (predictions based on VCGs). Moreover, we identified covariates which resulted in poorer identification of treatment-relatedness. This was related to an increasing incidence rate divergence in HCD relative to CCGs. Future databases which collect data at the individual animal level including study details such as animal age and testing facility are required to build adequate VCGs to accurately identify treatment-related effects.

Hurdles and signposts on the road to virtual control groups-A case study illustrating the influence of anesthesia protocols on electrolyte levels in rats

Introduction: Virtual Control Groups (VCGs) represent the concept of using historical control data from legacy animal studies to replace concurrent control group (CCG) animals. Based on the data curation and sharing activities of the Innovative Medicine Initiatives project eTRANSAFE (enhancing TRANSlational SAFEty Assessment through Integrative Knowledge Management) the ViCoG working group was established with the objectives of i) collecting suitable historical control data sets from preclinical toxicity studies, ii) evaluating statistical methodologies for building adequate and regulatory acceptable VCGs from historical control data, and iii) sharing those control-group data across multiple pharmaceutical companies. During the qualification process of VCGs a particular focus was put on the identification of hidden confounders in the data sets, which might impair the adequate matching of VCGs with the CCG. Methods: During our analyses we identified such a hidden confounder, namely, the choice of the anesthetic procedure used in animal experiments before blood withdrawal. Anesthesia using CO₂ may elevate the levels of some electrolytes such as calcium in blood, while the use of isoflurane is known to lower these values. Identification of such hidden confounders is particularly important if the underlying experimental information (e.g., on the anesthetic procedure) is not routinely recorded in the standard raw data files, such as SEND (Standard for Exchange of Non-clinical Data). We therefore analyzed how the replacement of CCGs with VCGs would affect the reproducibility of treatment-related findings regarding electrolyte values (potassium, calcium, sodium, and phosphate). The analyses were performed using a legacy rat systemic toxicity study consisting of a control and three treatment groups conducted according to pertinent OECD guidelines. In the report of this study treatment-related hypercalcemia was reported. The rats in this study were anesthetized with isoflurane. Results: Replacing the CCGs with VCGs derived from studies comprising both anesthetics resulted in a shift of control electrolyte parameters. Instead of the originally reported hypercalcemia the use of VCG led to fallacious conclusions of no observed effect or hypocalcemia. Discussion: Our study highlights the importance of a rigorous statistical analysis including the detection and elimination of hidden confounders prior to the implementation of the VCG concept.

Pharmacokinetic Tools and Applications

Drug toxicity, as well as therapeutic activity, is contingent upon the parent drug, or a derivative thereof, reaching the relevant site of action in the body, at sufficient concentration, over a given period of time. Thus, the potential to truly elicit an effect is governed by both the intrinsic activity/toxicity of the drug (or its transformation products) and its pharmacokinetic profile. As the pharmaceutical industry has become increasingly aware of the role of pharmacokinetics in determining drug activity and toxicity, the range of software, both freely available and commercial, to predict relevant properties has proliferated. Such tools can be considered on three different levels, applicable at different stages within the drug development process and providing increasing detail and relevance of information. Level (i) is the prediction of fundamental physicochemical properties that can be used to screen vast virtual libraries of potential candidates. Level (ii), predicting the individual absorption, distribution, metabolism, and excretion (ADME) characteristics of potential drugs, can also be applied to many compounds simultaneously. Level (iii), predicting the concentration-time profile of a drug in blood or specific tissues/organs for individuals or a population, is the most sophisticated level of prediction, applied to fewer candidates. In this chapter, in silico tools for predicting ADME-relevant properties, across these three levels, and the applications of this information, are described using exemplar, freely available resources. Further resources are signposted but not all are considered in detail as the purpose here is more to provide an introduction to the capabilities and practicalities of the tools, rather than to provide an exhaustive review of all the tools available.

Increasing the Value of Data Within a Large Pharmaceutical Company Through In Silico Models

The present contribution describes how in silico models and methods are applied at different stages of the drug discovery process in the pharmaceutical industry. A description of the most relevant computational methods and tools is given along with an evaluation of their performance in the assessment of potential genotoxic impurities and the prediction of off-target in vitro pharmacology. The challenges of predicting the outcome of highly complex in vivo studies are discussed followed by considerations on how novel ways to manage, store, exchange, and analyze data may advance knowledge and facilitate modeling efforts. In this context, the current status of broad data sharing initiatives, namely, eTOX and eTransafe, will be described along with related projects that could significantly reduce the use of animals in drug discovery in the future.

Development of In Silico Methods for Toxicity Prediction in Collaboration Between Academia and the Pharmaceutical Industry

The pharmaceutical industry would benefit from the collaboration with academic groups in the development of predictive safety models using the newest computational technologies. However, this collaboration is sometimes hampered by the handling of confidential proprietary information and different working practices in both environments. In this manuscript, we propose a strategy for facilitating this collaboration, based on the use of modeling frameworks developed for facilitating the use of sensitive data, as well as the development, interchange, hosting, and use of predictive models in production. The strategy is illustrated with a real example in which we used Flame, an open-source modeling framework developed in our group, for the development of an in silico eye irritation model. The model was based on bibliographic data, refined during the company-academic group collaboration, and enriched with the incorporation of confidential data, yielding a useful model that was validated experimentally.