Estimating bias in population parameters for some models for repeated measures ordinal data using NONMEM and NLMIXED

Evaluation of bias, precision, robustness and runtime for estimation methods in NONMEM 7

NONMEM is the most widely used software for population pharmacokinetic (PK)-pharmacodynamic (PD) analyses. The latest version, NONMEM 7 (NM7), includes several sampling-based estimation methods in addition to the classical methods. In this study, performance of the estimation methods available in NM7 was investigated with respect to bias, precision, robustness and runtime for a diverse set of PD models. Simulations of 500 data sets from each PD model were reanalyzed with the available estimation methods to investigate bias and precision. Simulations of 100 data sets were used to investigate robustness by comparing final estimates obtained after estimations starting from the true parameter values and initial estimates randomly generated using the CHAIN feature in NM7. Average estimation time for each algorithm and each model was calculated from the runtimes reported by NM7. The method giving the lowest bias and highest precision across models was importance sampling, closely followed by FOCE/LAPLACE and stochastic approximation expectation-maximization. The methods relative robustness differed between models and no method showed clear superior performance. FOCE/LAPLACE was the method with the shortest runtime for all models, followed by iterative two-stage. The Bayesian Markov Chain Monte Carlo method, used in this study for point estimation, performed worst in all tested metrics.

Performance comparison of various maximum likelihood nonlinear mixed-effects estimation methods for dose-response models

Estimation methods for nonlinear mixed-effects modelling have considerably improved over the last decades. Nowadays, several algorithms implemented in different software are used. The present study aimed at comparing their performance for dose-response models. Eight scenarios were considered using a sigmoid E(max) model, with varying sigmoidicity and residual error models. One hundred simulated datasets for each scenario were generated. One hundred individuals with observations at four doses constituted the rich design and at two doses, the sparse design. Nine parametric approaches for maximum likelihood estimation were studied: first-order conditional estimation (FOCE) in NONMEM and R, LAPLACE in NONMEM and SAS, adaptive Gaussian quadrature (AGQ) in SAS, and stochastic approximation expectation maximization (SAEM) in NONMEM and MONOLIX (both SAEM approaches with default and modified settings). All approaches started first from initial estimates set to the true values and second, using altered values. Results were examined through relative root mean squared error (RRMSE) of the estimates. With true initial conditions, full completion rate was obtained with all approaches except FOCE in R. Runtimes were shortest with FOCE and LAPLACE and longest with AGQ. Under the rich design, all approaches performed well except FOCE in R. When starting from altered initial conditions, AGQ, and then FOCE in NONMEM, LAPLACE in SAS, and SAEM in NONMEM and MONOLIX with tuned settings, consistently displayed lower RRMSE than the other approaches. For standard dose-response models analyzed through mixed-effects models, differences were identified in the performance of estimation methods available in current software, giving material to modellers to identify suitable approaches based on an accuracy-versus-runtime trade-off.

Empirical Bayes estimation of random effects of a mixed-effects proportional odds Markov model for ordinal data

The objective of this work was to investigate the factors influencing the quality of empirical Bayes estimates (EBEs) of individual random effects of a mixed-effects Markov model for ordered categorical data. It was motivated by an attempt to develop a model-based dose adaptation tool for clinical use in colorectal cancer patients receiving capecitabine, which induces severe hand-and-foot syndrome (HFS) toxicity in more than a half of the patients. This simulation-based study employed a published mixed-effects model for HFS. The quality of EBEs was assessed in terms of accuracy and precision, as well as shrinkage. Three optimization algorithms were compared: simplex, quasi-Newton and adaptive random search. The investigated factors were amount of data per patient, distribution of categories within patients, magnitude of the inter-individual variability, and values of the effect model parameters. The main factors affecting the quality of EBEs were the values of parameters governing the dose-response relationship and the within-subject distribution of categories. For the chosen HFS toxicity model, the accuracy and precision of EBEs were rather low, and therefore the feasibility of their use for individual model-based dose adaptation seemed limited.

Multinomial logistic functions in markov chain models of sleep architecture: internal and external validation and covariate analysis

Mixed-effect Markov chain models have been recently proposed to characterize the time course of transition probabilities between sleep stages in insomniac patients. The most recent one, based on multinomial logistic functions, was used as a base to develop a final model combining the strengths of the existing ones. This final model was validated on placebo data applying also new diagnostic methods and then used for the inclusion of potential age, gender, and BMI effects. Internal validation was performed through simplified posterior predictive check (sPPC), visual predictive check (VPC) for categorical data, and new visual methods based on stochastic simulation and estimation and called visual estimation check (VEC). External validation mainly relied on the evaluation of the objective function value and sPPC. Covariate effects were identified through stepwise covariate modeling within NONMEM VI. New model features were introduced in the model, providing significant sPPC improvements. Outcomes from VPC, VEC, and external validation were generally very good. Age, gender, and BMI were found to be statistically significant covariates, but their inclusion did not improve substantially the model's predictive performance. In summary, an improved model for sleep internal architecture has been developed and suitably validated in insomniac patients treated with placebo. Thereafter, covariate effects have been included into the final model.

Performance and robustness of the Monte Carlo importance sampling algorithm using parallelized S-ADAPT for basic and complex mechanistic models

The Monte Carlo Parametric Expectation Maximization (MC-PEM) algorithm can approximate the true log-likelihood as precisely as needed and is efficiently parallelizable. Our objectives were to evaluate an importance sampling version of the MC-PEM algorithm for mechanistic models and to qualify the default estimation settings in SADAPT-TRAN. We assessed bias, imprecision and robustness of this algorithm in S-ADAPT for mechanistic models with up to 45 simultaneously estimated structural parameters, 14 differential equations, and 10 dependent variables (one drug concentration and nine pharmacodynamic effects). Simpler models comprising 15 parameters were estimated using three of the ten dependent variables. We set initial estimates to 0.1 or 10 times the true value and evaluated 30 bootstrap replicates with frequent or sparse sampling. Datasets comprised three dose levels with 16 subjects each. For simultaneous estimation of the full model, the ratio of estimated to true values for structural model parameters (median [5-95% percentile] over 45 parameters) was 1.01 [0.94-1.13] for means and 0.99 [0.68-1.39] for between-subject variances for frequent sampling and 1.02 [0.81-1.47] for means and 1.02 [0.47-2.56] for variances for sparse sampling. Imprecision was ≤25% for 43 of 45 means for frequent sampling. Bias and imprecision was well comparable for the full and simpler models. Parallelized estimation was 23-fold (6.9-fold) faster using 48 threads (eight threads) relative to one thread. The MC-PEM algorithm was robust and provided unbiased and adequately precise means and variances during simultaneous estimation of complex, mechanistic models in a 45 dimensional parameter space with rich or sparse data using poor initial estimates.

Performance of three estimation methods in repeated time-to-event modeling

It is not uncommon that the outcome measurements, symptoms or side effects, of a clinical trial belong to the family of event type data, e.g., bleeding episodes or emesis events. Event data is often low in information content and the mixed-effects modeling software NONMEM has previously been shown to perform poorly with low information ordered categorical data. The aim of this investigation was to assess the performance of the Laplace method, the stochastic approximation expectation-maximization (SAEM) method, and the importance sampling method when modeling repeated time-to-event data. The Laplace method already existed, whereas the two latter methods have recently become available in NONMEM 7. A stochastic simulation and estimation study was performed to assess the performance of the three estimation methods when applied to a repeated time-to-event model with a constant hazard associated with an exponential interindividual variability. Various conditions were investigated, ranging from rare to frequent events and from low to high interindividual variability. The method performance was assessed by parameter bias and precision. Due to the lack of information content under conditions where very few events were observed, all three methods exhibit parameter bias and imprecision, however most pronounced by the Laplace method. The performance of the SAEM and importance sampling were generally higher than Laplace when the frequency of individuals with events was less than 43%, while at frequencies above that all methods were equal in performance.

Implementation and evaluation of the SAEM algorithm for longitudinal ordered categorical data with an illustration in pharmacokinetics-pharmacodynamics

Analysis of longitudinal ordered categorical efficacy or safety data in clinical trials using mixed models is increasingly performed. However, algorithms available for maximum likelihood estimation using an approximation of the likelihood integral, including LAPLACE approach, may give rise to biased parameter estimates. The SAEM algorithm is an efficient and powerful tool in the analysis of continuous/count mixed models. The aim of this study was to implement and investigate the performance of the SAEM algorithm for longitudinal categorical data. The SAEM algorithm is extended for parameter estimation in ordered categorical mixed models together with an estimation of the Fisher information matrix and the likelihood. We used Monte Carlo simulations using previously published scenarios evaluated with NONMEM. Accuracy and precision in parameter estimation and standard error estimates were assessed in terms of relative bias and root mean square error. This algorithm was illustrated on the simultaneous analysis of pharmacokinetic and discretized efficacy data obtained after a single dose of warfarin in healthy volunteers. The new SAEM algorithm is implemented in MONOLIX 3.1 for discrete mixed models. The analyses show that for parameter estimation, the relative bias is low for both fixed effects and variance components in all models studied. Estimated and empirical standard errors are similar. The warfarin example illustrates how simple and rapid it is to analyze simultaneously continuous and discrete data with MONOLIX 3.1. The SAEM algorithm is extended for analysis of longitudinal categorical data. It provides accurate estimates parameters and standard errors. The estimation is fast and stable.

Sample size/power calculations for repeated ordinal measurements in population pharmacodynamic experiments

Population pharmacodynamic experiments sometime involve repeated measurements of ordinal random variables at specific time points. Such longitudinal data presents a challenge during modelling due to correlation between measurements within an individual and often mixed-effects modelling approach may be used for the analysis. It is important that these studies are adequately powered by including an adequate number of subjects in order to detect a significant treatment effect. This paper describes a method for calculating sample size for repeated ordinal measurements in population pharmacodynamic experiments based on analysis by a mixed-effects modelling approach. The Wald test is used for testing the significance of treatment effects. This method is fast, simple and efficient. It can also be extended to account for differential allocation of subjects to the groups and unbalanced sampling designs between and within groups. The results obtained from two simulation studies using nonlinear mixed-effects modelling software (NONMEM) showed good agreement between the power obtained from simulation and nominal power used for sample size calculations.

Performance in population models for count data, part II: a new SAEM algorithm

Analysis of count data from clinical trials using mixed effect analysis has recently become widely used. However, algorithms available for the parameter estimation, including LAPLACE and Gaussian quadrature (GQ), are associated with certain limitations, including bias in parameter estimates and the long analysis runtime. The stochastic approximation expectation maximization (SAEM) algorithm has proven to be a very efficient and powerful tool in the analysis of continuous data. The aim of this study was to implement and investigate the performance of a new SAEM algorithm for application to count data. A new SAEM algorithm was implemented in MATLAB for estimation of both, parameters and the Fisher information matrix. Stochastic Monte Carlo simulations followed by re-estimation were performed according to scenarios used in previous studies (part I) to investigate properties of alternative algorithms (Plan et al., 2008, Abstr 1372 [ http://wwwpage-meetingorg/?abstract=1372 ]). A single scenario was used to explore six probability distribution models. For parameter estimation, the relative bias was less than 0.92% and 4.13% for fixed and random effects, for all models studied including ones accounting for over- or under-dispersion. Empirical and estimated relative standard errors were similar, with distance between them being <1.7% for all explored scenarios. The longest CPU time was 95 s for parameter estimation and 56 s for SE estimation. The SAEM algorithm was extended for analysis of count data. It provides accurate estimates of both, parameters and standard errors. The estimation is significantly faster compared to LAPLACE and GQ. The algorithm is implemented in Monolix 3.1, (beta-version available in July 2009).

A survey of population analysis methods and software for complex pharmacokinetic and pharmacodynamic models with examples

An overview is provided of the present population analysis methods and an assessment of which software packages are most appropriate for various PK/PD modeling problems. Four PK/PD example problems were solved using the programs NONMEM VI beta version, PDx-MCPEM, S-ADAPT, MONOLIX, and WinBUGS, informally assessed for reasonable accuracy and stability in analyzing these problems. Also, for each program we describe their general interface, ease of use, and abilities. We conclude with discussing which algorithms and software are most suitable for which types of PK/PD problems. NONMEM FO method is accurate and fast with 2-compartment models, if intra-individual and interindividual variances are small. The NONMEM FOCE method is slower than FO, but gives accurate population values regardless of size of intra- and interindividual errors. However, if data are very sparse, the NONMEM FOCE method can lead to inaccurate values, while the Laplace method can provide more accurate results. The exact EM methods (performed using S-ADAPT, PDx-MCPEM, and MONOLIX) have greater stability in analyzing complex PK/PD models, and can provide accurate results with sparse or rich data. MCPEM methods perform more slowly than NONMEM FOCE for simple models, but perform more quickly and stably than NONMEM FOCE for complex models. WinBUGS provides accurate assessments of the population parameters, standard errors and 95% confidence intervals for all examples. Like the MCPEM methods, WinBUGS's efficiency increases relative to NONMEM when solving the complex PK/PD models.

Population pharmacokinetics/pharmacodynamics of anesthetics

In this article we review how population pharmacokinetic/pharmacodynamic (PD) modeling has evolved in the specialty of anesthesiology, how anesthesiology benefited from the mixed-effects approach, and which features of modeling need careful attention. Key articles from the anesthesiology literature are selected to discuss the modeling of typical anesthesiological PD end points, such as level of consciousness and analgesia, interactions between hypnotics and analgesics, estimation with poor and sometimes rich data sets from populations of various sizes, covariate detection, covariances between random effects, and Bayesian forecasting.