Tiered chemical testing: a value of information approach

A quantitative AOP of mitochondrial toxicity based on data from three cell lines

Adverse Outcome Pathways (AOPs) are increasingly used to support the integration of in vitro data in hazard assessment for chemicals. Quantitative AOPs (qAOPs) use mathematical models to describe the relationship between key events (KEs). In this paper, data obtained in three cell lines, LHUMES, HepG2 and RPTEC/TERT1, using similar experimental protocols, was used to calibrate a qAOP of mitochondrial toxicity for two chemicals, rotenone and deguelin. The objectives were to determine whether the same qAOP could be used for the three cell types, and to test chemical-independence by cross-validation with a dataset obtained on eight other chemicals in LHUMES cells. Repeating the calibration approach for both chemicals in three cell lines highlighted various practical difficulties. Even when the same readouts of KEs are measured, the mathematical functions used to describe the key event relationships may not be the same. Cross-validation in LHUMES cells was attempted by estimating chemical-specific potency at the molecular initiating events and using the rest of the calibrated qAOP to predict downstream KEs: toxicity of azoxystrobin, carboxine, mepronil and thifluzamide was underestimated. Selection of most relevant readouts and accurate characterization of the molecular initiating event for cross-validation are critical when designing in vitro experiments targeted at calibrating qAOPs.

The Pebble Remains in the Master's Hand: Two Careers Spent Learning (Still) from John Evans

In this article, we discuss four vexing problems in risk-based decision making that John Evans has addressed over the last nearly 40 years and has perennially challenged the two of us and others to think about. We tackle the role in decision making of potential thresholds in dose-response functions, how the lack of health reference values for many chemicals may distort risk management, the challenge of model uncertainty for risk characterization, and the yet-untapped potential for value-of-information analysis to enhance public health decision making. Our theme is that work remains to be done on each of these, but that some of that work would merely involve listening to ideas that John has already offered.

Evaluation of Non-animal Methods for Assessing Skin Sensitisation Hazard: A Bayesian Value-of-Information Analysis

This paper offers a Bayesian Value-of-Information (VOI) analysis for guiding the development of non-animal testing strategies, balancing information gains from testing with the expected social gains and costs from the adoption of regulatory decisions. Testing is assumed to have value, if, and only if, the information revealed from testing triggers a welfare-improving decision on the use (or non-use) of a substance. As an illustration, our VOI model is applied to a set of five individual non-animal prediction methods used for skin sensitisation hazard assessment, seven battery combinations of these methods, and 236 sequential 2-test and 3-test strategies. Their expected values are quantified and compared to the expected value of the local lymph node assay (LLNA) as the animal method. We find that battery and sequential combinations of non-animal prediction methods reveal a significantly higher expected value than the LLNA. This holds for the entire range of prior beliefs. Furthermore, our results illustrate that the testing strategy with the highest expected value does not necessarily have to follow the order of key events in the sensitisation adverse outcome pathway (AOP).

A decision analytic approach to exposure-based chemical prioritization

The manufacture of novel synthetic chemicals has increased in volume and variety, but often the environmental and health risks are not fully understood in terms of toxicity and, in particular, exposure. While efforts to assess risks have generally been effective when sufficient data are available, the hazard and exposure data necessary to assess risks adequately are unavailable for the vast majority of chemicals in commerce. The US Environmental Protection Agency has initiated the ExpoCast Program to develop tools for rapid chemical evaluation based on potential for exposure. In this context, a model is presented in which chemicals are evaluated based on inherent chemical properties and behaviorally-based usage characteristics over the chemical's life cycle. These criteria are assessed and integrated within a decision analytic framework, facilitating rapid assessment and prioritization for future targeted testing and systems modeling. A case study outlines the prioritization process using 51 chemicals. The results show a preliminary relative ranking of chemicals based on exposure potential. The strength of this approach is the ability to integrate relevant statistical and mechanistic data with expert judgment, allowing for an initial tier assessment that can further inform targeted testing and risk management strategies.

Bayesian inference

This chapter provides an overview of the Bayesian approach to data analysis, modeling, and statistical decision making. The topics covered go from basic concepts and definitions (random variables, Bayes' rule, prior distributions) to various models of general use in biology (hierarchical models, in particular) and ways to calibrate and use them (MCMC methods, model checking, inference, and decision). The second half of this Bayesian primer develops an example of model setup, calibration, and inference for a physiologically based analysis of 1,3-butadiene toxicokinetics in humans.

Data Quality Assessment for In Silico Methods: A Survey of Approaches and Needs

As indicated in Chapter 3, there are a large number of potential sources of data now available for modelling purposes. These range from historical literature references for a few compounds to highly curated databases of hundreds of thousands of compounds, available via the internet. Before including any data in an in silico model, the question of data quality must be addressed. Although it is difficult to define the quality of data in absolute terms, it is possible to assess the suitability of data for a given purpose. There are many reasons for variability within data and the degree of error that is acceptable for one model may not be the same as for another. For example generating a global model intended to pre-screen large numbers of compounds does not require the same degree of accuracy as performing an individual risk assessment for a chemical of interest. In this chapter, sources of data variability and error will be discussed and formal methods to score data quality, such as use of the Klimisch criteria, will be described. Examples of data quality issues will be given for specific endpoints relating to both environmental and human health effects. Mathematical approaches (Dempster-Schafer theory and Bayesian networks) demonstrating how this information relating to confidence in the data can be incorporated into in silico models is also discussed.

Toxicity testing in the 21st century: a vision and a strategy

With the release of the landmark report Toxicity Testing in the 21st Century: A Vision and a Strategy, the U.S. National Academy of Sciences, in 2007, precipitated a major change in the way toxicity testing is conducted. It envisions increased efficiency in toxicity testing and decreased animal usage by transitioning from current expensive and lengthy in vivo testing with qualitative endpoints to in vitro toxicity pathway assays on human cells or cell lines using robotic high-throughput screening with mechanistic quantitative parameters. Risk assessment in the exposed human population would focus on avoiding significant perturbations in these toxicity pathways. Computational systems biology models would be implemented to determine the dose-response models of perturbations of pathway function. Extrapolation of in vitro results to in vivo human blood and tissue concentrations would be based on pharmacokinetic models for the given exposure condition. This practice would enhance human relevance of test results, and would cover several test agents, compared to traditional toxicological testing strategies. As all the tools that are necessary to implement the vision are currently available or in an advanced stage of development, the key prerequisites to achieving this paradigm shift are a commitment to change in the scientific community, which could be facilitated by a broad discussion of the vision, and obtaining necessary resources to enhance current knowledge of pathway perturbations and pathway assays in humans and to implement computational systems biology models. Implementation of these strategies would result in a new toxicity testing paradigm firmly based on human biology.

Separation of uncertainty and interindividual variability in human exposure modeling

The NORMTOX model predicts the lifetime-averaged exposure to contaminants through multiple environmental media, that is, food, air, soil, drinking and surface water. The model was developed to test the coherence of Dutch environmental quality objectives (EQOs). A set of EQOs is called coherent if simultaneous exposure to different environmental media that are all polluted up to their respective EQOs does not result in exceeding the acceptable or tolerable daily intake (ADI or TDI). Aim of the present study is to separate the impact of uncertainty and interindividual variability in coherence predictions with the NORMTOX model. The method is illustrated in a case study for chlorfenvinphos, mercury and nitrate. First, ANOVA was used to calculate interindividual variability in input parameters. Second, nested Monte Carlo simulation was used to propagate uncertainty and interindividual variability separately. Lifetime-averaged exposure to chlorfenvinphos, mercury and nitrate was modeled for the Dutch population. Output distributions specified the population fraction at risk, due to a particular exposure, and the reliability of this risk. From the case study, it was obtained that at lifelong exposure to all media polluted up to their standard, 100% of the Dutch population exceeds the ADI for chlorfenvinphos, 15% for mercury and 0% for nitrate. Variance in exposure to chlorfenvinphos, mercury and nitrate is mostly caused by interindividual variability instead of true uncertainty. It is concluded that the likelihood that ADIs of chlorfenvinphos and mercury will be exceeded should be further explored. If exceeding is likely, decision makers should focus on identification of high-risk subpopulations, rather than on additional research to obtain more accurate estimates for particular parameters.

Expected value of perfect information: an empirical example of reducing decision uncertainty by conducting additional research

OBJECTIVE

Value of information (VOI) analysis informs decision-makers about the expected value of conducting more research to support a decision. This expected value of (partial) perfect information (EV(P)PI) can be estimated by simultaneously eliminating uncertainty on all (or some) parameters involved in model-based decision-making. This study aimed to calculate the EVPPI, before and after collecting additional information on the parameter of a probabilistic Markov model with the highest EVPPI.

METHODS

The model assessed the 5-year costs per quality-adjusted life year (QALY) of three bronchodilators in chronic obstructive pulmonary disease (COPD). It had identified tiotropium as the bronchodilator with the highest expected net benefit. Total EVPI was estimated plus the EVPPIs for four groups of parameters: 1) transition probabilities between COPD severity stages; 2) exacerbation probabilities; 3) utility weights; and 4) costs. Partial EVPI analyses were performed using one-level and two-level sampling algorithms.

RESULTS

Before additional research, the total EVPI was Euro 1985 per patient at a threshold value of Euro 20,000 per QALY. EVPPIs were Euro 1081 for utilities, Euro 724 for transition probabilities, and relatively small for exacerbation probabilities and costs. A large study was performed to obtain more precise EQ-5D utilities by COPD severity stages. After using posterior utilities, the EVPPI for utilities decreased to almost zero. The total EVPI for the updated model was reduced to Euro 1037. With an EVPPI of Euro 856, transition probabilities were now the single most important parameter contributing to the EVPI.

CONCLUSIONS

This VOI analysis clearly identified parameters for which additional research is most worthwhile. After conducting additional research on the most important parameter, i.e., the utilities, total EVPI was substantially reduced.

VCCEP pilot: progress on evaluating children's risks and data needs

The Voluntary Children's Chemical Evaluation Program (VCCEP) is designed to provide information to the public on children's potential health risks associated with chemical exposures. The key question of the VCCEP is whether the potential hazards, exposures, and risks to children have been adequately characterized, and, if not, what additional data are necessary. To answer this question, manufacturers or importers of 23 chemicals were asked by the U. S. Environmental Protection Agency (U.S. EPA) to sponsor their chemicals in the first tier of a pilot program. These chemicals were selected for evaluation because they have been found as contaminants in human tissue or fluids (adipose tissue, blood, breath, breast milk, or urine); food and water children may eat and drink; or air children may breathe (including residential or school air). Under the VCCEP framework, sponsoring companies agree to prepare Tier 1 hazard, exposure, and risk assessments on the individual chemicals, and identify the need for additional data. These assessment documents are submitted to the U.S. EPA and subsequently undergo review by experts in an independent peer consultation meeting that is open to the public. Following this peer consultation process, the U.S. EPA reviews each submission and makes a data-needs determination, which may include requesting further data collection or generation by the sponsor. Sponsoring companies then decide whether to volunteer for the next tier and collect or generate the requested data. The purpose of this article is to describe the VCCEP process and to review and present the key findings from the first set of chemicals that have been fully or partially evaluated under the pilot program (vinylidene chloride, decabromodiphenyl ether, pentabromodiphenyl ether, octabromodiphenyl ether, acetone, methyl ethyl ketone, decane, undecane, and dodecane). Specifically, we provide a brief summary of the sponsors' submissions, the peer consultation panels' discussions, and the U.S. EPA's data-needs decisions. Although we do not attempt to conduct independent analyses of the underlying data, we do identify a number of common themes that have emerged during implementation of the pilot program and discuss several key issues that could become important in the future. The information presented here should be useful for various parties interested in the progress of the VCCEP and the results of the initial (Tier 1) children's assessments.

The application of special technologies in diagnostic anatomic pathology: is it consistent with the principles of evidence-based medicine?

Proponents of evidence-based medicine (EBM) have emphasized the need to consider the quality of different sources of medical information and have proposed various methods to integrate available "best evidence" into rules, guidelines and other diagnostic, therapeutic and prognostic models. The various factors that can affect the internal validity of studies in anatomic pathology, such as interobserver variability, use of retrospective rather than prospective data and others, are reviewed. The need for testing for the external validity of the results of anatomic pathology studies is introduced, using "test sets" of cases that have not been used to generate the classification or prognostic models. This methodology has been seldom used in anatomic pathology to validate the generalizability of various "entities," usefulness of diagnostic tests under different conditions and other information. Basic concepts of meta-analysis for research synthesis are introduced; these methods have been seldom used in anatomic pathology to integrate information from different studies using quantitative techniques rather than summary tables that merely list the results of various publications. The potential use of decision analysis and value of information analysis for the adoption of new tests is briefly discussed.