D-Optimality for Regression Designs: A Review

Being Uncertain in Chromatographic Calibration-Some Unobvious Details in Experimental Design

This is an introductory tutorial and review about the uncertainty problem in chromatographic calibration. It emphasizes some unobvious, but important details influencing errors in the calibration curve estimation, uncertainty in prediction, as well as the connections and dependences between them, all from various perspectives of uncertainty measurement. Nonuniform D-optimal designs coming from Fedorov theorem are computed and presented. As an example, all possible designs of 24 calibration samples (3–8, 4–6, 6–4, 8–3 and 12–2, both uniform and D-optimal) are compared in context of many optimality criteria. It can be concluded that there are only two independent (orthogonal, but slightly complex) trends in optimality of these designs. The conclusions are important, as the uniform designs with many concentrations are not the best choices, contrary to some intuitive perception. Nonuniform designs are visibly better alternative in most calibration cases.

Optimal allocation in stratified cluster-based outcome-dependent sampling designs

In public health research, finite resources often require that decisions be made at the study design stage regarding which individuals to sample for detailed data collection. At the same time, when study units are naturally clustered, as patients are in clinics, it may be preferable to sample clusters rather than the study units, especially when the costs associated with travel between clusters are high. In this setting, aggregated data on the outcome and select covariates are sometimes routinely available through, for example, a country's Health Management Information System. If used wisely, this information can be used to guide decisions regarding which clusters to sample, and potentially obtain gains in efficiency over simple random sampling. In this article, we derive a series of formulas for optimal allocation of resources when a single-stage stratified cluster-based outcome-dependent sampling design is to be used and a marginal mean model is specified to answer the question of interest. Specifically, we consider two settings: (i) when a particular parameter in the mean model is of primary interest; and, (ii) when multiple parameters are of interest. We investigate the finite population performance of the optimal allocation framework through a comprehensive simulation study. Our results show that there are trade-offs that must be considered at the design stage: optimizing for one parameter yields efficiency gains over balanced and simple random sampling, while resulting in losses for the other parameters in the model. Optimizing for all parameters simultaneously yields smaller gains in efficiency, but mitigates the losses for the other parameters in the model.

On optimum dynamic temperature profiles for thermal inactivation kinetics determination

Determination of inactivation kinetics, associated with thermal processing of foods and obtained from dynamic temperature experiments, requires carefully designed experiments, the primary element being the selection of the appropriate temperature profile along with a carefully planned sampling schedule. In the present work, a number of different dynamic temperature profiles were investigated in terms of their ability to generate accurate kinetic parameters with low confidence intervals (CIs). Although alternative models have been also tested, our work was concentrated on thermal inactivation kinetics that could be described by the classical D-z values. A pair of D and z values was assumed, and for each temperature profile tested, concentration data at different processing times were generated through the appropriate models. Next, an error (up to ±2.5% or ±5%) was introduced on these theoretical values to generate pseudo-experimental data, and the back-calculation of the assumed kinetic parameters by non-linear regression was performed. The accuracy and the 95% CIs of the estimated kinetic parameters were evaluated; joint confidence regions were also constructed to investigate parameters correlation. The effect of temperature profile pattern, level of error, number of experimental points, and reference temperature was assessed. A stepwise increasing and a single triangle-pattern temperature profile were the best profiles among those tested. As a general observation, based on different kinetic models investigated, temperature profiles and sampling intervals that result in concentration versus time diagrams having shapes as suggested by the primary model used when isothermally applied are not considered appropriate for parameter estimation.

About Model Validation in Bioprocessing

In bioprocess engineering the Qualtiy by Design (QbD) initiative encourages the use of models to define design spaces. However, clear guidelines on how models for QbD are validated are still missing. In this review we provide a comprehensive overview of the validation methods, mathematical approaches, and metrics currently applied in bioprocess modeling. The methods cover analytics for data used for modeling, model training and selection, measures for predictiveness, and model uncertainties. We point out the general issues in model validation and calibration for different types of models and put this into the context of existing health authority recommendations. This review provides a starting point for developing a guide for model validation approaches. There is no one-fits-all approach, but this review should help to identify the best fitting validation method, or combination of methods, for the specific task and the type of bioprocess model that is being developed.

Optimal Estimation of Neural Recruitment Curves Using Fisher Information: Application to Transcranial Magnetic Stimulation

This paper presents a novel method for fast and optimal determination of recruitment (input-output, IO) curve parameters in neural stimulation. A sequential parameter estimation (SPE) method was developed based on the Fisher information matrix (FIM), with a stopping rule based on successively satisfying a specified estimation tolerance. Simulated motor responses evoked by transcranial magnetic stimulation (TMS) were used as a test bed. Performance of FIM-SPE was characterized in 10 177 simulation runs for various IO parameter values corresponding to different virtual subjects, compared with uniform sampling. Unlike uniform sampling, FIM-SPE identifies and samples the areas of the IO curve that contain maximum information about the curve parameters. For the most relaxed stopping rule, the median number of samples required for convergence was only 17 for FIM-SPE versus 294 for uniform sampling. For the highest reliability stopping rule, more than 92% of the FIM-SPE runs converged, with a median of 88 samples, whereas all uniform sampling runs reached 1000 samples without converging. Compared to uniform sampling, FIM-SPE reduced estimation errors up to two-fold and required ten times fewer stimuli. FIM-SPE could improve the speed and accuracy of determination of IO curves for neural stimulation. A software implementation of the algorithm is provided online.

Bayesian adaptive design: improving the effectiveness of monitoring of the Great Barrier Reef

Monitoring programs are essential for understanding patterns, trends, and threats in ecological and environmental systems. However, such programs are costly in terms of dollars, human resources, and technology, and complex in terms of balancing short- and long-term requirements. In this work, We develop new statistical methods for implementing cost-effective adaptive sampling and monitoring schemes for coral reef that can better utilize existing information and resources, and which can incorporate available prior information. Our research was motivated by developing efficient monitoring practices for Australia's Great Barrier Reef. We develop and implement two types of adaptive sampling schemes, static and sequential, and show that they can be more informative and cost-effective than an existing (nonadaptive) monitoring program. Our methods are developed in a Bayesian framework with a range of utility functions relevant to environmental monitoring. Our results demonstrate the considerable potential for adaptive design to support improved management outcomes in comparison to set-and-forget styles of surveillance monitoring.

Benefits of statistical molecular design, covariance analysis, and reference models in QSAR: a case study on acetylcholinesterase

Scientific disciplines such as medicinal- and environmental chemistry, pharmacology, and toxicology deal with the questions related to the effects small organic compounds exhort on biological targets and the compounds' physicochemical properties responsible for these effects. A common strategy in this endeavor is to establish structure-activity relationships (SARs). The aim of this work was to illustrate benefits of performing a statistical molecular design (SMD) and proper statistical analysis of the molecules' properties before SAR and quantitative structure-activity relationship (QSAR) analysis. Our SMD followed by synthesis yielded a set of inhibitors of the enzyme acetylcholinesterase (AChE) that had very few inherent dependencies between the substructures in the molecules. If such dependencies exist, they cause severe errors in SAR interpretation and predictions by QSAR-models, and leave a set of molecules less suitable for future decision-making. In our study, SAR- and QSAR models could show which molecular sub-structures and physicochemical features that were advantageous for the AChE inhibition. Finally, the QSAR model was used for the prediction of the inhibition of AChE by an external prediction set of molecules. The accuracy of these predictions was asserted by statistical significance tests and by comparisons to simple but relevant reference models.

Application of optimal designs to item calibration

In computerized adaptive testing (CAT), examinees are presented with various sets of items chosen from a precalibrated item pool. Consequently, the attrition speed of the items is extremely fast, and replenishing the item pool is essential. Therefore, item calibration has become a crucial concern in maintaining item banks. In this study, a two-parameter logistic model is used. We applied optimal designs and adaptive sequential analysis to solve this item calibration problem. The results indicated that the proposed optimal designs are cost effective and time efficient.

Enantioselective aza-Michael addition of imides by using an integrated strategy involving the synthesis of a family of multifunctional catalysts, usage of multiple catalysis, and rational design of experiment

A challenging asymmetric reaction (aza-Michael addition of imides to enones) has been optimized through an integrated approach involving the synthesis of a family of organocatalysts, multiple catalysis (usage of additives), and finally with rational exploration of the chemical space by the application of the experiment design.

Mapping pilicide anti-virulence effect in Escherichia coli, a comprehensive structure-activity study

Pilicides prevent pili formation and thereby the development of bacterial biofilms in Escherichia coli. We have performed a comprehensive structure activity relationship (SAR) study of the dihydrothiazolo ring-fused 2-pyridone pilicide central fragment by varying all open positions. Orthogonal projections to latent structures discriminant analysis (OPLS-DA) was used to distinguish active from inactive compounds in which polarity proved to be the most important factor for discrimination. A quantitative SAR (QSAR) partial least squares (PLS) model was calculated on the active compounds for prediction of biofilm inhibition activity. In this model, compounds with high inhibitory activity were generally larger, more lipophilic, more flexible and had a lower HOMO. Overall, this resulted in both highly valuable SAR information and potent inhibitors of type 1 pili dependent biofilm formation. The most potent biofilm inhibitor had an EC(50) of 400 nM.

Design of glycopeptides used to investigate class II MHC binding and T-cell responses associated with autoimmune arthritis

The glycopeptide fragment CII259–273 from type II collagen (CII) binds to the murine A^q and human DR4 class II Major Histocompatibility Complex (MHC II) proteins, which are associated with development of murine collagen-induced arthritis (CIA) and rheumatoid arthritis (RA), respectively. It has been shown that CII259–273 can be used in therapeutic vaccination of CIA. This glycopeptide also elicits responses from T-cells obtained from RA patients, which indicates that it has an important role in RA as well. We now present a methodology for studies of (glyco)peptide-receptor interactions based on a combination of structure-based virtual screening, ligand-based statistical molecular design and biological evaluations. This methodology included the design of a CII259–273 glycopeptide library in which two anchor positions crucial for binding in pockets of A^q and DR4 were varied. Synthesis and biological evaluation of the designed glycopeptides provided novel structure-activity relationship (SAR) understanding of binding to A^q and DR4. Glycopeptides that retained high affinities for these MHC II proteins and induced strong responses in panels of T-cell hybridomas were also identified. An analysis of all the responses revealed groups of glycopeptides with different response patterns that are of high interest for vaccination studies in CIA. Moreover, the SAR understanding obtained in this study provides a platform for the design of second-generation glycopeptides with tuned MHC affinities and T-cell responses.

Determinant Efficiencies in Ill-Conditioned Models

Thecanonical correlationsbetween subsets ofOLSestimators are identified withdesign linkage parametersbetween their regressors. Knowncollinearity indicesare extended to encompass angles between each regressor vector and remaining vectors. One such angle quantifies the collinearity of regressors with the intercept, of concern in the corruption of all estimates due to ill-conditioning. Matrix identities factorize a determinant in terms of principal subdeterminants and the canonicalVector Alienation Coefficientsbetween subset estimators—by duality, theAlienation Coefficientsbetween subsets of regressors. These identities figure in the study ofDand as determinant efficiencies for estimators and their subsets, specifically, -efficiencies for the constant, linear, pure quadratic, and interactive coefficients in eight known small second-order designs. Studies onD- and -efficiencies confirm that designs are seldom efficient for both. Determinant identities demonstrate the propensity for -inefficient subsets to be masked through near collinearities in overallD-efficient designs.

Selecting optimal experiments for multiple output multilayer perceptrons

Where should a researcher conduct experiments to provide training data for a multilayer perceptron? This question is investigated, and a statistical method for selecting optimal experimental design points for multiple output multilayer perceptrons is introduced. Multiple class discrimination problems are examined using a framework in which the multilayer perceptron is viewed as a multivariate nonlinear regression model. Following a Bayesian formulation for the case where the variance-covariance matrix of the responses is unknown, a selection criterion is developed. This criterion is based on the volume of the joint confidence ellipsoid for the weights in a multilayer perceptron. An example is used to demonstrate the superiority of optimally selected design points over randomly chosen points, as well as points chosen in a grid pattern. Simplification of the basic criterion is offered through the use of Hadamard matrices to produce uncorrelated outputs.

Bayesian design criteria: computation, comparison, and application to a pharmacokinetic and a pharmacodynamic model

In this paper 3 criteria to design experiments for Bayesian estimation of the parameters of nonlinear models with respect to their parameters, when a prior distribution is available, are presented: the determinant of the Bayesian information matrix, the determinant of the pre-posterior covariance matrix, and the expected information provided by an experiment. A procedure to simplify the computation of these criteria is proposed in the case of continuous prior distributions and is compared with the criterion obtained from a linearization of the model about the mean of the prior distribution for the parameters. This procedure is applied to two models commonly encountered in the area of pharmacokinetics and pharmacodynamics: the one-compartment open model with bolus intravenous single-dose injection and the Emax model. They both involve two parameters. Additive as well as multiplicative gaussian measurement errors are considered with normal prior distributions. Various combinations of the variances of the prior distribution and of the measurement error are studied. Our attention is restricted to designs with limited numbers of measurements (1 or 2 measurements). This situation often occurs in practice when Bayesian estimation is performed. The optimal Bayesian designs that result vary with the variances of the parameter distribution and with the measurement error. The two-point optimal designs sometimes differ from the D-optimal designs for the mean of the prior distribution and may consist of replicating measurements. For the studied cases, the determinant of the Bayesian information matrix and its linearized form lead to the same optimal designs. In some cases, the pre-posterior covariance matrix can be far from its lower bound, namely, the inverse of the Bayesian information matrix, especially for the Emax model and a multiplicative measurement error. The expected information provided by the experiment and the determinant of the pre-posterior covariance matrix generally lead to the same designs except for the Emax model and the multiplicative measurement error. Results show that these criteria can be easily computed and that they could be incorporated in modules for designing experiments.

Designing an optimal experiment for Bayesian estimation: application to the kinetics of iodine thyroid uptake

We consider the problem of designing an optimal experiment for Bayesian estimation of the parameters of a non-linear model. When their distribution is known, the Bayesian approach allows individual estimation from a small number of measurements; the design determines the accuracy of the estimates. We propose to optimize this design by maximizing a general criterion: the expectation of the information supplied by the experiment. This approach is applied to optimize the two sampling times for Bayesian estimation of the kinetics of radioiodine thyroid uptake from an estimated non-parametric prior distribution.