Model-Based Residual Post-Processing for Residual Model Identification

Simulation-based evaluation of the Pharmpy Automatic Model Development tool for population pharmacokinetic modeling in early clinical drug development

The Pharmpy Automatic Model Development (AMD) tool automates the building of population pharmacokinetic (popPK) models by utilizing a systematic stepwise process. In this study, the performance of the AMD tool was assessed using simulated datasets. Ten true models mimicking classical popPK models were created. From each true model, dataset replicates were simulated assuming a typical phase I study design-single and multiple ascending doses with/without dichotomous food effect, with rich PK sampling. For every dataset replicate, the AMD tool automatically built an AMD model utilizing NONMEM for parameter estimation. The AMD models were compared to the true and reference models (true model fitted to simulated datasets) based on their model components, predicted population and individual secondary PK parameters (SP) (AUC0-24, cmax, ctrough), and model quality metrics (e.g., model convergence, parameter relative standard errors (RSEs), Bayesian Information Criterion (BIC)). The models selected by the AMD tool closely resembled the true models, particularly in terms of distribution and elimination, although differences were observed in absorption and inter-individual variability components. Bias associated with the derived SP was low. In general, discrepancies between AMD and true SP were also observed for reference models and therefore were attributed to the inherent stochasticity in simulations. In summary, the AMD tool was found to be a valuable asset in automating repetitive modeling tasks, yielding reliable PK models in the scenarios assessed. This tool has the potential to save time during early clinical drug development that can be invested in more complex modeling activities within model-informed drug development.

Simulating realistic patient profiles from pharmacokinetic models by a machine learning postprocessing correction of residual variability

We address the problem of model misspecification in population pharmacokinetics (PopPK), by modeling residual unexplained variability (RUV) by machine learning (ML) methods in a postprocessing step after conventional model building. The practical purpose of the method is the generation of realistic virtual patient profiles and the quantification of the extent of model misspecification, by introducing an appropriate metric, to be used as an additional diagnostic of model quality. The proposed methodology consists of the following steps: After developing a PopPK model, the individual residual errors IRES = DV-IPRED, are computed, where DV are the observations and IPRED the individual predictions and are modeled by ML to obtain IRESML. Correction of the IPREDs can then be carried out as DVML = IPRED + IRESML. The methodology was tested in a PK study of ropinirole, for which a PopPK model was developed while a second deliberately misspecified model was also considered. Various supervised ML algorithms were tested, among which Random Forest gave the best results. The ML model was able to correct individual predictions as inspected in diagnostic plots and most importantly it simulated realistic profiles that resembled the real data and canceled out the artifacts introduced by the elevated RUV, even in the case of the heavily misspecified model. Furthermore, a metric to quantify the extent of model misspecification was introduced based on the R2 between IRES and IRESML, following the rationale that the greater the extent of variability explained by the ML model, the higher the degree of model misspecification present in the original model.

Evaluation of covariate effects in item response theory models

Item response theory (IRT) models are usually the best way to analyze composite or rating scale data. Standard methods to evaluate covariate or treatment effects in IRT models do not allow to identify item-specific effects. Finding subgroups of patients who respond differently to certain items could be very important when designing inclusion or exclusion criteria for clinical trials, and aid in understanding different treatment responses in varying disease manifestations. We present a new method to investigate item-specific effects in IRT models, which is based on inspection of residuals. The method was investigated in a simulation exercise with a model for the Epworth Sleepiness Scale. We also provide a detailed discussion as a guidance on how to build a robust covariate IRT model.

Predictive Value of C-Reactive Protein and Albumin for Temporal Within-Individual Pharmacokinetic Variability of Voriconazole in Pediatric Hematopoietic Cell Transplant Patients

Voriconazole is a widely used antifungal agent in immunocompromised patients, but its utility is limited by its variable exposure and narrow therapeutic index. Population pharmacokinetic (PK) models have been used to characterize voriconazole PK and derive individualized dosing regimens. However, determinants of temporal within-patient variability of voriconazole PK were not well-established. We aimed to characterize temporal variability of voriconazole PK within individuals and identify predictive clinical factors. This study was conducted as a part of a single-institution, phase I study of intravenous voriconazole in children undergoing HCT (NCT02227797). We analyzed voriconazole PK study data collected at week 1 and again at week 2 after the start of voriconazole therapy in 59 pediatric HCT patients (age <21 years). Population PK analysis using nonlinear mixed effect modeling was performed to analyze temporal within-individual variability of voriconazole PK by incorporating a between-occasion variability term in the model. A two-compartment linear elimination model incorporating body weight and CYP2C19 phenotype described the data. Ratio of individual voriconazole clearance between weeks 1 to 2 ranged from 0.11 to 3.3 (-9.1 to +3.3-fold change). Incorporation of covariate effects by serum C-reactive protein (CRP) and albumin levels decreased between-occasion variability of clearance (coefficient of variation: from 59.5% to 41.2%) and improved the model fit (p<0.05). As significant covariates on voriconazole PK, CRP and albumin concentrations may potentially serve as useful biomarkers as part of therapeutic drug monitoring. This article is protected by copyright. All rights reserved.

Development of visual predictive checks accounting for multimodal parameter distributions in mixture models

The assumption of interindividual variability being unimodally distributed in nonlinear mixed effects models does not hold when the population under study displays multimodal parameter distributions. Mixture models allow the identification of parameters characteristic to a subpopulation by describing these multimodalities. Visual predictive check (VPC) is a standard simulation based diagnostic tool, but not yet adapted to account for multimodal parameter distributions. Mixture model analysis provides the probability for an individual to belong to a subpopulation (IPmix) and the most likely subpopulation for an individual to belong to (MIXEST). Using simulated data examples, two implementation strategies were followed to split the data into subpopulations for the development of mixture model specific VPCs. The first strategy splits the observed and simulated data according to the MIXEST assignment. A shortcoming of the MIXEST-based allocation strategy was a biased allocation towards the dominating subpopulation. This shortcoming was avoided by splitting observed and simulated data according to the IPmix assignment. For illustration purpose, the approaches were also applied to an irinotecan mixture model demonstrating 36% lower clearance of irinotecan metabolite (SN-38) in individuals with UGT1A1 homo/heterozygote versus wild-type genotype. VPCs with segregated subpopulations were helpful in identifying model misspecifications which were not evident with standard VPCs. The new tool provides an enhanced power of evaluation of mixture models.

Variability Attribution for Automated Model Building

We investigated the possible advantages of using linearization to evaluate models of residual unexplained variability (RUV) for automated model building in a similar fashion to the recently developed method “residual modeling.” Residual modeling, although fast and easy to automate, cannot identify the impact of implementing the needed RUV model on the imprecision of the rest of model parameters. We used six RUV models to be tested with 12 real data examples. Each example was first linearized; then, we assessed the agreement in improvement of fit between the base model and its extended models for linearization and conventional analysis, in comparison to residual modeling performance. Afterward, we compared the estimates of parameters’ variabilities and their uncertainties obtained by linearization to conventional analysis. Linearization accurately identified and quantified the nature and magnitude of RUV model misspecification similar to residual modeling. In addition, linearization identified the direction of change and quantified the magnitude of this change in variability parameters and their uncertainties. This method is implemented in the software package PsN for automated model building/evaluation with continuous data.

Model-Based Conditional Weighted Residuals Analysis for Structural Model Assessment

Nonlinear mixed effects models are widely used to describe longitudinal data to improve the efficiency of drug development process or increase the understanding of the studied disease. In such settings, the appropriateness of the modeling assumptions is critical in order to draw correct conclusions and must be carefully assessed for any substantial violations. Here, we propose a new method for structure model assessment, based on assessment of bias in conditional weighted residuals (CWRES). We illustrate this method by assessing prediction bias in two integrated models for glucose homeostasis, the integrated glucose-insulin (IGI) model, and the integrated minimal model (IMM). One dataset was simulated from each model then analyzed with the two models. CWRES outputted from each model fitting were modeled to capture systematic trends in CWRES as well as the magnitude of structural model misspecifications in terms of difference in objective function values (ΔOFV_Bias). The estimates of CWRES bias were used to calculate the corresponding bias in conditional predictions by the inversion of first-order conditional estimation method’s covariance equation. Time, glucose, and insulin concentration predictions were the investigated independent variables. The new method identified correctly the bias in glucose sub-model of the integrated minimal model (IMM), when this bias occurred, and calculated the absolute and proportional magnitude of the resulting bias. CWRES bias versus the independent variables agreed well with the true trends of misspecification. This method is fast easily automated diagnostic tool for model development/evaluation process, and it is already implemented as part of the Perl-speaks-NONMEM software.