An optimal experimental design criterion for discriminating between non-normal models

A Review of Degradation Models and Remaining Useful Life Prediction for Testing Design and Predictive Maintenance of Lithium-Ion Batteries

We present a novel decision-making framework for accelerated degradation tests and predictive maintenance that exploits prior knowledge and experimental data on the system's state. As a framework for sequential decision making in these areas, dynamic programming and reinforcement learning are considered, along with data-driven degradation learning when necessary. Furthermore, we illustrate both stochastic and machine learning degradation models, which are integrated in the framework, using data-driven methods. These methods are presented as a valuable tool for designing life-testing experiments and for maintaining lithium-ion batteries.

Discrimination between Gaussian process models: active learning and static constructions

The paper covers the design and analysis of experiments to discriminate between two Gaussian process models with different covariance kernels, such as those widely used in computer experiments, kriging, sensor location and machine learning. Two frameworks are considered. First, we study sequential constructions, where successive design (observation) points are selected, either as additional points to an existing design or from the beginning of observation. The selection relies on the maximisation of the difference between the symmetric Kullback Leibler divergences for the two models, which depends on the observations, or on the mean squared error of both models, which does not. Then, we consider static criteria, such as the familiar log-likelihood ratios and the Fréchet distance between the covariance functions of the two models. Other distance-based criteria, simpler to compute than previous ones, are also introduced, for which, considering the framework of approximate design, a necessary condition for the optimality of a design measure is provided. The paper includes a study of the mathematical links between different criteria and numerical illustrations are provided.

Optimal designs for health risk assessments using fractional polynomial models

Fractional polynomials (FP) have been shown to be more flexible than polynomial models for fitting data from an univariate regression model with a continuous outcome but design issues for FP models have lagged. We focus on FPs with a single variable and construct D-optimal designs for estimating model parameters and I-optimal designs for prediction over a user-specified region of the design space. Some analytic results are given, along with a discussion on model uncertainty. In addition, we provide an applet to facilitate users find tailor made optimal designs for their problems. As applications, we construct optimal designs for three studies that used FPs to model risk assessments of (a) testosterone levels from magnesium accumulation in certain areas of the brains in songbirds, (b) rats subject to exposure of different chemicals, and (c) hormetic effects due to small toxic exposure. In each case, we elaborate the benefits of having an optimal design in terms of cost and quality of the statistical inference.

Hybrid algorithms for generating optimal designs for discriminating multiple nonlinear models under various error distributional assumptions

Finding a model-based optimal design that can optimally discriminate among a class of plausible models is a difficult task because the design criterion is non-differentiable and requires 2 or more layers of nested optimization. We propose hybrid algorithms based on particle swarm optimization (PSO) to solve such optimization problems, including cases when the optimal design is singular, the mean response of some models are not fully specified and problems that involve 4 layers of nested optimization. Using several classical examples, we show that the proposed PSO-based algorithms are not models or criteria specific, and with a few repeated runs, can produce either an optimal design or a highly efficient design. They are also generally faster than the current algorithms, which are generally slow and work for only specific models or discriminating criteria. As an application, we apply our techniques to find optimal discriminating designs for a dose-response study in toxicology with 5 possible models and compare their performances with traditional and a recently proposed algorithm. In the supplementary material, we provide a R package to generate different types of discriminating designs and evaluate efficiencies of competing designs so that the user can implement an informed design.

An Optimal Design Criterion for Within-Individual Covariance Matrices Discrimination and Parameter Estimation in Nonlinear Mixed Effects Models

In this paper, we consider the problem of nding optimal populationdesigns for within-individual covariance matrices discrimination andparameter estimation in nonlinear mixed eects models. A compound optimality criterion is provided, which combines an estimation criterion and a discrimination criterion. We used the D-optimality criterion for parameter estimation, which maximizes the determinant of the Fisher information matrix. For discrimination, we propose a generalization of the T-optimality criterion for xed-eects models. Equivalence theorems are provided for these criteria. We illustrated the application of compound criteria with an example in a pharmacokinetic experiment.

Optimal designs for frequentist model averaging

We consider the problem of designing experiments for estimating a target parameter in regression analysis when there is uncertainty about the parametric form of the regression function. A new optimality criterion is proposed that chooses the experimental design to minimize the asymptotic mean squared error of the frequentist model averaging estimate. Necessary conditions for the optimal solution of a locally and Bayesian optimal design problem are established. The results are illustrated in several examples, and it is demonstrated that Bayesian optimal designs can yield a reduction of the mean squared error of the model averaging estimator by up to 45%.

A web-based tool for designing experimental studies to detect hormesis and estimate the threshold dose

Hormesis has been widely observed and debated in a variety of context in biomedicine and toxicological sciences. Detecting its presence can be an important problem with wide ranging implications. However, there is little work on constructing an efficient experiment to detect its existence or estimate the threshold dose. We use optimal design theory to develop a variety of locally optimal designs to detect hormesis, estimate the threshold dose and the zero-equivalent point (ZEP) for commonly used models in toxicology and risk assessment. To facilitate use of more efficient designs to detect hormesis, estimate threshold dose and estimate the ZEP in practice, we implement computer algorithms and create a user-friendly web site to help the biomedical researcher generate different types of optimal designs. The online tool facilitates the user to evaluate robustness properties of a selected design to various model assumptions and compare designs before implementation.

[Formula: see text]-optimality in copula models

Optimum experimental design theory has recently been extended for parameter estimation in copula models. The use of these models allows one to gain in flexibility by considering the model parameter set split into marginal and dependence parameters. However, this separation also leads to the natural issue of estimating only a subset of all model parameters. In this work, we treat this problem with the application of the [Formula: see text]-optimality to copula models. First, we provide an extension of the corresponding equivalence theory. Then, we analyze a wide range of flexible copula models to highlight the usefulness of [Formula: see text]-optimality in many possible scenarios. Finally, we discuss how the usage of the introduced design criterion also relates to the more general issue of copula selection and optimal design for model discrimination.

Bayesian T-optimal discriminating designs

The problem of constructing Bayesian optimal discriminating designs for a class of regression models with respect to the T-optimality criterion introduced by Atkinson and Fedorov (1975a) is considered. It is demonstrated that the discretization of the integral with respect to the prior distribution leads to locally T-optimal discriminating design problems with a large number of model comparisons. Current methodology for the numerical construction of discrimination designs can only deal with a few comparisons, but the discretization of the Bayesian prior easily yields to discrimination design problems for more than 100 competing models. A new efficient method is developed to deal with problems of this type. It combines some features of the classical exchange type algorithm with the gradient methods. Convergence is proved and it is demonstrated that the new method can find Bayesian optimal discriminating designs in situations where all currently available procedures fail.

A Semi-Infinite Programming based algorithm for determining T-optimum designs for model discrimination

T-optimum designs for model discrimination are notoriously difficult to find because of the computational difficulty involved in solving an optimization problem that involves two layers of optimization. Only a handful of analytical T-optimal designs are available for the simplest problems; the rest in the literature are found using specialized numerical procedures for a specific problem. We propose a potentially more systematic and general way for finding T-optimal designs using a Semi-Infinite Programming (SIP) approach. The strategy requires that we first reformulate the original minimax or maximin optimization problem into an equivalent semi-infinite program and solve it using an exchange-based method where lower and upper bounds produced by solving the outer and the inner programs, are iterated to convergence. A global Nonlinear Programming (NLP) solver is used to handle the subproblems, thus finding the optimal design and the least favorable parametric configuration that minimizes the residual sum of squares from the alternative or test models. We also use a nonlinear program to check the global optimality of the SIP-generated design and automate the construction of globally optimal designs. The algorithm is successfully used to produce results that coincide with several T-optimal designs reported in the literature for various types of model discrimination problems with normally distributed errors. However, our method is more general, merely requiring that the parameters of the model be estimated by a numerical optimization.

Optimizing disease progression study designs for drug effect discrimination

Investigate the possibility to directly optimize a clinical trial design for statistical power to detect a drug effect and compare to optimal designs that focus on parameter precision. An improved statistic derived from the general formulation of the Wald approximation was used to predict the statistical power for given trial designs of a disease progression study. The predicted value was compared, together with the classical Wald statistic, to a type I error-corrected model-based power determined via clinical trial simulations. In a second step, a study design for maximal power was determined by directly maximizing the new statistic. The resulting power-optimal designs and their corresponding performance based on empirical power calculations were compared to designs focusing on parameter precision. Comparisons of empirically determined power and the newly developed statistic, showed excellent agreement across all scenarios investigated. This was in contrast to the classical Wald statistic, which consistently over-predicted the reference power with deviations of up to 90 %. Designs maximized using the proposed metric differed from traditional optimal designs and showed equal or up to 20 % higher power in the subsequent clinical trial simulations. Furthermore, the proposed method was used to minimize the number of individuals required to achieve 80 % power through a simultaneous optimization of study size and study design. The targeted power of 80 % was confirmed in subsequent simulation study. A new statistic was developed, allowing for the explicit optimization of a clinical trial design with respect to statistical power.

Minimal plus one point designs for testing lack of fit for some sigmoid curve models

D-optimal designs for nonlinear models are often minimally supported. They have been frequently criticized for their inability to test for lack of fit. We construct alternative designs to address this issue for some commonly used sigmoid curves, including logistic, probit, and Gompertz models with two, three, or four parameters. For each model, we compare five nonminimally supported designs in terms of their efficiency, and propose designs that are both statistically efficient and practically convenient for practitioners.

Mechanistic analysis of challenge-response experiments

We present an application of mechanistic modeling and nonlinear longitudinal regression in the context of biomedical response-to-challenge experiments, a field where these methods are underutilized. In this type of experiment, a system is studied by imposing an experimental challenge, and then observing its response. The combination of mechanistic modeling and nonlinear longitudinal regression has brought new insight, and revealed an unexpected opportunity for optimal design. Specifically, the mechanistic aspect of our approach enables the optimal design of experimental challenge characteristics (e.g., intensity, duration). This article lays some groundwork for this approach. We consider a series of experiments wherein an isolated rabbit heart is challenged with intermittent anoxia. The heart responds to the challenge onset, and recovers when the challenge ends. The mean response is modeled by a system of differential equations that describe a candidate mechanism for cardiac response to anoxia challenge. The cardiac system behaves more variably when challenged than when at rest. Hence, observations arising from this experiment exhibit complex heteroscedasticity and sharp changes in central tendency. We present evidence that an asymptotic statistical inference strategy may fail to adequately account for statistical uncertainty. Two alternative methods are critiqued qualitatively (i.e., for utility in the current context), and quantitatively using an innovative Monte-Carlo method. We conclude with a discussion of the exciting opportunities in optimal design of response-to-challenge experiments.

Robust optimal design of experiments for model discrimination using an interactive software tool

Mathematical modeling of biochemical processes significantly contributes to a better understanding of biological functionality and underlying dynamic mechanisms. To support time consuming and costly lab experiments, kinetic reaction equations can be formulated as a set of ordinary differential equations, which in turn allows to simulate and compare hypothetical models in silico. To identify new experimental designs that are able to discriminate between investigated models, the approach used in this work solves a semi-infinite constrained nonlinear optimization problem using derivative based numerical algorithms. The method takes into account parameter variabilities such that new experimental designs are robust against parameter changes while maintaining the optimal potential to discriminate between hypothetical models. In this contribution we present a newly developed software tool that offers a convenient graphical user interface for model discrimination. We demonstrate the beneficial operation of the discrimination approach and the usefulness of the software tool by analyzing a realistic benchmark experiment from literature. New robust optimal designs that allow to discriminate between the investigated model hypotheses of the benchmark experiment are successfully calculated and yield promising results. The involved robustification approach provides maximally discriminating experiments for the worst parameter configurations, which can be used to estimate the meaningfulness of upcoming experiments. A major benefit of the graphical user interface is the ability to interactively investigate the model behavior and the clear arrangement of numerous variables. In addition to a brief theoretical overview of the discrimination method and the functionality of the software tool, the importance of robustness of experimental designs against parameter variability is demonstrated on a biochemical benchmark problem. The software is licensed under the GNU General Public License and freely available at http://sourceforge.net/projects/mdtgui/.

An optimal experimental design approach to model discrimination in dynamic biochemical systems

MOTIVATION

Finding suitable models of dynamic biochemical systems is an important task in systems biology approaches to the biosciences. On the one hand, a correct model helps to understand the underlying mechanisms and on the other hand, one can use the model to predict the behavior of a biological system under various circumstances. Typically, before the correct model of a biochemical system is found, different hypothetical models might be reasonable and consistent with previous knowledge and available data. The main goal now is to find the best suited model out of different hypotheses. The process of falsifying inappropriate candidate models is called model discrimination.

RESULTS

We have developed a new computational tool to compute optimal experiments for biochemical kinetic systems with underlying ordinary differential equation (ODE) models for the purpose of model discrimination. We were inspired by the demands of biological experimentalists which perform one run measurement where perturbations to the system are possible. We provide a criterion which calculates the number and location of time points of optimal measurements as well as optimal initial conditions and optimal perturbations to the system.

AVAILABILITY

The model discrimination algorithm described here is implemented in C++ in the package ModelDiscriminationToolkit. The source code can be downloaded from http://omnibus.unifreiburg.de/~ds500/_software.html.