1
|
Chen X, Nyberg HB, Donnelly M, Zhao L, Fang L, Karlsson MO, Hooker AC. Development and comparison of model-integrated evidence approaches for bioequivalence studies with pharmacokinetic end points. CPT Pharmacometrics Syst Pharmacol 2024. [PMID: 39177211 DOI: 10.1002/psp4.13216] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2023] [Revised: 06/26/2024] [Accepted: 07/19/2024] [Indexed: 08/24/2024] Open
Abstract
By applying nonlinear mixed-effect (NLME) models, model-integrated evidence (MIE) approaches are able to analyze bioequivalence (BE) data with pharmacokinetic end points that have sparse sampling, which is problematic for non-compartmental analysis (NCA). However, MIE approaches may suffer from inflation of type I error due to underestimation of parameter uncertainty and to the assumption of asymptotic normality. In this study, we developed a MIE BE analysis method that is based on a pre-defined model and consists of several steps including model fitting, uncertainty assessment, simulation, and BE determination. The presented MIE approach has several improvements compared with the previously reported model-integrated methods: (1) treatment, sequence, and period effects are only added to absorption parameters (such as relative bioavailability and rate of absorption) instead of all PK parameters; (2) a simulation step is performed to generate confidence intervals of the pharmacokinetic metrics for BE assessment; and (3) in an effort to maintain type I error, two more advanced parameter uncertainty evaluation approaches are explored, a nonparametric (case resampling) bootstrap, and sampling importance resampling (SIR). To evaluate the developed method and compare the uncertainty assessment methods, simulation experiments were performed for BE studies using a two-way crossover design with different amounts of information (sparse to rich designs) and levels of variability. Based on the simulation results, the method using SIR for parameter uncertainty quantification controls type I error at the nominal level of 0.05 (i.e., the significance level set for BE evaluation) even for studies with small sample size and/or sparse sampling. As expected, our MIE approach for BE assessment exhibited higher power than the NCA-based method, especially as the data becomes sparser and/or more variable.
Collapse
Affiliation(s)
- Xiaomei Chen
- Department of Pharmacy, Uppsala University, Uppsala, Sweden
| | | | - Mark Donnelly
- Division of Quantitative Methods and Modeling, Office of Research Standards, Office of Generic Drugs, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, USA
| | - Liang Zhao
- Division of Quantitative Methods and Modeling, Office of Research Standards, Office of Generic Drugs, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, USA
| | - Lanyan Fang
- Division of Quantitative Methods and Modeling, Office of Research Standards, Office of Generic Drugs, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, USA
| | | | | |
Collapse
|
2
|
Fayette L, Leroux R, Mentré F, Seurat J. Robust and Adaptive Two-stage Designs in Nonlinear Mixed Effect Models. AAPS J 2023; 25:71. [PMID: 37466809 DOI: 10.1208/s12248-023-00810-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2023] [Accepted: 04/06/2023] [Indexed: 07/20/2023] Open
Abstract
To get informative studies for nonlinear mixed effect models (NLMEM), design optimization can be performed based on Fisher Information Matrix (FIM) using the D-criterion. Its computation requires knowledge about models and parameters, which are often prior guesses. Thus, adaptive designs composed of several stages may be used. Robust approach can also be used to account for various candidate models. In the estimation step of a given stage, model selection (MS) or model averaging (MA) can be performed. In this work we propose a new two-stage adaptive design strategy, based on the robust expected FIM and MA over several candidate models. The methodology is applied to a clinical trial simulation in ophthalmology to optimize doses and time measurements. A set of dose-response candidate models is defined, and one-stage designs are compared to two-stage 50/50 designs (i.e., each stage performed with half of the available subjects), using either local optimal design or robust design, and performing analysis with one model, MS or MA. Performing a two-stage design with MS at the interim analysis can correct the choice of a wrong model for designing the first stage. Overall, starting from a robust design (1- or 2-stage) is valuable and leads to reasonable bias and precision. The proposed robust adaptive design strategy is a new tool to design longitudinal studies that could be used in different therapeutic areas.
Collapse
Affiliation(s)
- Lucie Fayette
- Inserm, IAME, Université Paris Cité and Université Sorbonne Paris Nord, F-75018, Paris, France
- École des Ponts, UGE, Champs-sur-Marne, France
| | - Romain Leroux
- Inserm, IAME, Université Paris Cité and Université Sorbonne Paris Nord, F-75018, Paris, France
| | - France Mentré
- Inserm, IAME, Université Paris Cité and Université Sorbonne Paris Nord, F-75018, Paris, France
| | - Jérémy Seurat
- Inserm, IAME, Université Paris Cité and Université Sorbonne Paris Nord, F-75018, Paris, France.
| |
Collapse
|
3
|
Chasseloup E, Karlsson MO. Comparison of Seven Non-Linear Mixed Effect Model-Based Approaches to Test for Treatment Effect. Pharmaceutics 2023; 15:460. [PMID: 36839782 PMCID: PMC9959233 DOI: 10.3390/pharmaceutics15020460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Revised: 01/04/2023] [Accepted: 01/09/2023] [Indexed: 01/31/2023] Open
Abstract
Analyses of longitudinal data with non-linear mixed-effects models (NLMEM) are typically associated with high power, but sometimes at the cost of inflated type I error. Approaches to overcome this problem were published recently, such as model-averaging across drug models (MAD), individual model-averaging (IMA), and combined Likelihood Ratio Test (cLRT). This work aimed to assess seven NLMEM approaches in the same framework: treatment effect assessment in balanced two-armed designs using real natural history data with or without the addition of simulated treatment effect. The approaches are MAD, IMA, cLRT, standard model selection (STDs), structural similarity selection (SSs), randomized cLRT (rcLRT), and model-averaging across placebo and drug models (MAPD). The assessment included type I error, using Alzheimer's Disease Assessment Scale-cognitive (ADAS-cog) scores from 817 untreated patients and power and accuracy in the treatment effect estimates after the addition of simulated treatment effects. The model selection and averaging among a set of pre-selected candidate models were driven by the Akaike information criteria (AIC). The type I error rate was controlled only for IMA and rcLRT; the inflation observed otherwise was explained by the placebo model misspecification and selection bias. Both IMA and rcLRT had reasonable power and accuracy except under a low typical treatment effect.
Collapse
|
4
|
A parallel sampling framework for model averaging: Application to dose response studies. Contemp Clin Trials 2022; 123:106957. [PMID: 36228983 DOI: 10.1016/j.cct.2022.106957] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Revised: 09/23/2022] [Accepted: 10/01/2022] [Indexed: 01/27/2023]
|
5
|
Kunina H, Al‐Mashat A, Chien JY, Garhyan P, Kjellsson MC. Optimization of trial duration to predict long-term HbA1c change with therapy: A pharmacometrics simulation-based evaluation. CPT Pharmacometrics Syst Pharmacol 2022; 11:1443-1457. [PMID: 35899461 PMCID: PMC9662199 DOI: 10.1002/psp4.12854] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Revised: 07/10/2022] [Accepted: 07/24/2022] [Indexed: 11/30/2022] Open
Abstract
Glycated hemoglobin (HbA1c) is the main biomarker of diabetes drug development. However, because of its delayed turnover, trial duration is rarely shorter than 12 weeks, and being able to predict long-term HbA1c with precision using data from shorter studies would be beneficial. The feasibility of reducing study duration was therefore investigated in this study, assuming a model-based analysis. The aim was to investigate the predictive performance of 24- and 52-week extrapolations using data from up to 4, 6, 8 or 12 weeks, with six previously published pharmacometric models of HbA1c. Predictive performance was assessed through simulation-based dose-response predictions and model averaging (MA) with two hypothetical drugs. Results were consistent across the methods of assessment, with MA supporting the results derived from the model-based framework. The models using mean plasma glucose (MPG) or nonlinear fasting plasma glucose (FPG) effect, driving the HbA1c formation, showed good predictive performance despite a reduced study duration. The models, using the linear effect of FPG to drive the HbA1c formation, were sensitive to the limited amount of data in the shorter studies. The MA with bootstrap demonstrated strongly that a 4-week study duration is insufficient for precise predictions of all models. Our findings suggest that if data are analyzed with a pharmacometric model with MPG or FPG with a nonlinear effect to drive HbA1c formation, a study duration of 8 weeks is sufficient with maintained accuracy and precision of dose-response predictions.
Collapse
Affiliation(s)
- Hanna Kunina
- Pharmacometrics Research Group, Department of PharmacyUppsala UniversityUppsalaSweden
| | - Alex Al‐Mashat
- Pharmacometrics Research Group, Department of PharmacyUppsala UniversityUppsalaSweden
| | - Jenny Y. Chien
- Global Pharmacokinetics/Pharmacodynamics and Pharmacometrics, Lilly Research LaboratoriesLilly Corporate CenterIndianapolisIndianaUSA
| | - Parag Garhyan
- Global Pharmacokinetics/Pharmacodynamics and Pharmacometrics, Lilly Research LaboratoriesLilly Corporate CenterIndianapolisIndianaUSA
| | - Maria C. Kjellsson
- Pharmacometrics Research Group, Department of PharmacyUppsala UniversityUppsalaSweden
| |
Collapse
|
6
|
Guhl M, Mercier F, Hofmann C, Sharan S, Donnelly M, Feng K, Sun W, Sun G, Grosser S, Zhao L, Fang L, Mentré F, Comets E, Bertrand J. Impact of model misspecification on model-based tests in PK studies with parallel design: real case and simulation studies. J Pharmacokinet Pharmacodyn 2022; 49:557-577. [PMID: 36112338 PMCID: PMC9483500 DOI: 10.1007/s10928-022-09821-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2022] [Accepted: 08/11/2022] [Indexed: 11/26/2022]
Abstract
This article evaluates the performance of pharmacokinetic (PK) equivalence testing between two formulations of a drug through the Two-One Sided Tests (TOST) by a model-based approach (MB-TOST), as an alternative to the classical non-compartmental approach (NCA-TOST), for a sparse design with a few time points per subject. We focused on the impact of model misspecification and the relevance of model selection for the reference data. We first analysed PK data from phase I studies of gantenerumab, a monoclonal antibody for the treatment of Alzheimer’s disease. Using the original rich sample data, we compared MB-TOST to NCA-TOST for validation. Then, the analysis was repeated on a sparse subset of the original data with MB-TOST. This analysis inspired a simulation study with rich and sparse designs. With rich designs, we compared NCA-TOST and MB-TOST in terms of type I error and study power. With both designs, we explored the impact of misspecifying the model on the performance of MB-TOST and adding a model selection step. Using the observed data, the results of both approaches were in general concordance. MB-TOST results were robust with sparse designs when the underlying PK structural model was correctly specified. Using the simulated data with a rich design, the type I error of NCA-TOST was close to the nominal level. When using the simulated model, the type I error of MB-TOST was controlled on rich and sparse designs, but using a misspecified model led to inflated type I errors. Adding a model selection step on the reference data reduced the inflation. MB-TOST appears as a robust alternative to NCA-TOST, provided that the PK model is correctly specified and the test drug has the same PK structural model as the reference drug.
Collapse
|
7
|
Llanos-Paez C, Ambery C, Yang S, Beerahee M, Plan EL, Karlsson MO. Improved Confidence in a Confirmatory Stage by Application of Item-Based Pharmacometrics Model: Illustration with a Phase III Active Comparator-Controlled Trial in COPD Patients. Pharm Res 2022; 39:1779-1787. [PMID: 35233731 PMCID: PMC9314306 DOI: 10.1007/s11095-022-03194-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2021] [Accepted: 02/09/2022] [Indexed: 11/25/2022]
Abstract
PURPOSE The current study aimed to illustrate how a non-linear mixed effect (NLME) model-based analysis may improve confidence in a Phase III trial through more precise estimates of the drug effect. METHODS The FULFIL clinical trial was a Phase III study that compared 24 weeks of once daily inhaled triple therapy with twice daily inhaled dual therapy in patients with chronic obstructive pulmonary disease (COPD). Patient reported outcome data, obtained by using The Evaluating Respiratory Symptoms in COPD (E-RS:COPD) questionnaire, from the FULFIL study were analyzed using an NLME item-based response theory model (IRT). The change from baseline (CFB) in E-RS:COPD total score over 4-week intervals for each treatment arm was obtained using the IRT and compared with published results obtained with a mixed model repeated measures (MMRM) analysis. RESULTS The IRT included a graded response model characterizing item parameters and a Weibull function combined with an offset function to describe the COPD symptoms-time course in patients receiving either triple therapy (n = 907) or dual therapy (n = 894). The IRT improved precision of the estimated drug effect compared to MMRM, resulting in a sample size of at least 3.64 times larger for the MMRM analysis to achieve the IRT precision in the CFB estimate. CONCLUSION This study shows the advantage of IRT over MMRM with a direct comparison of the same primary endpoint for the two analyses using the same observed clinical trial data, resulting in an increased confidence in Phase III.
Collapse
Affiliation(s)
- Carolina Llanos-Paez
- Department of Pharmacy, Uppsala University, BMC, Box 580, 751 23, Uppsala, Sweden
| | - Claire Ambery
- Clinical Pharmacology Modelling and Simulation, GlaxoSmithKline plc, London, UK
| | - Shuying Yang
- Clinical Pharmacology Modelling and Simulation, GlaxoSmithKline plc, London, UK
| | - Misba Beerahee
- Clinical Pharmacology Modelling and Simulation, GlaxoSmithKline plc, London, UK
| | - Elodie L Plan
- Department of Pharmacy, Uppsala University, BMC, Box 580, 751 23, Uppsala, Sweden
| | - Mats O Karlsson
- Department of Pharmacy, Uppsala University, BMC, Box 580, 751 23, Uppsala, Sweden.
| |
Collapse
|
8
|
Prague M, Lavielle M. SAMBA: A novel method for fast automatic model building in nonlinear mixed-effects models. CPT Pharmacometrics Syst Pharmacol 2022; 11:161-172. [PMID: 35104058 PMCID: PMC8846636 DOI: 10.1002/psp4.12742] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2021] [Revised: 10/15/2021] [Accepted: 10/28/2021] [Indexed: 11/30/2022] Open
Abstract
The success of correctly identifying all the components of a nonlinear mixed-effects model is far from straightforward: it is a question of finding the best structural model, determining the type of relationship between covariates and individual parameters, detecting possible correlations between random effects, or also modeling residual errors. We present the Stochastic Approximation for Model Building Algorithm (SAMBA) procedure and show how this algorithm can be used to speed up this process of model building by identifying at each step how best to improve some of the model components. The principle of this algorithm basically consists in "learning something" about the "best model," even when a "poor model" is used to fit the data. A comparison study of the SAMBA procedure with Stepwise Covariate Modeling (SCM) and COnditional Sampling use for Stepwise Approach (COSSAC) show similar performances on several real data examples but with a much reduced computing time. This algorithm is now implemented in Monolix and in the R package Rsmlx.
Collapse
Affiliation(s)
- Mélanie Prague
- Inria Bordeaux Sud‐Ouest, Inserm, Bordeaux Population Health Research CenterSISTM TeamUMR 1219University of BordeauxBordeauxFrance
- Vaccine Research InstituteCréteilFrance
| | - Marc Lavielle
- Inria & CMAP, Ecole PolytechniqueCNRSInstitut Polytechnique de ParisParisFrance
| |
Collapse
|
9
|
Llanos-Paez C, Ambery C, Yang S, Tabberer M, Beerahee M, Plan EL, Karlsson MO. Improved Decision-Making Confidence Using Item-Based Pharmacometric Model: Illustration with a Phase II Placebo-Controlled Trial. AAPS JOURNAL 2021; 23:79. [PMID: 34080077 PMCID: PMC8172506 DOI: 10.1208/s12248-021-00600-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/09/2021] [Accepted: 04/20/2021] [Indexed: 02/02/2023]
Abstract
This study aimed to illustrate how a new methodology to assess clinical trial outcome measures using a longitudinal item response theory–based model (IRM) could serve as an alternative to mixed model repeated measures (MMRM). Data from the EXACT (Exacerbation of chronic pulmonary disease tool) which is used to capture frequency, severity, and duration of exacerbations in COPD were analyzed using an IRM. The IRM included a graded response model characterizing item parameters and functions describing symptom-time course. Total scores were simulated (month 12) using uncertainty in parameter estimates. The 50th (2.5th, 97.5th) percentiles of the resulting simulated differences in average total score (drug minus placebo) represented the estimated drug effect (95%CI), which was compared with published MMRM results. Furthermore, differences in sample size, sensitivity, specificity, and type I and II errors between approaches were explored. Patients received either oral danirixin 75 mg twice daily (n = 45) or placebo (n = 48) on top of standard of care over 52 weeks. A step function best described the COPD symptoms-time course in both trial arms. The IRM improved precision of the estimated drug effect compared to MMRM, resulting in a sample size of 2.5 times larger for the MMRM analysis to achieve the IRM precision. The IRM showed a higher probability of a positive predictive value (34%) than MMRM (22%). An item model–based analysis data gave more precise estimates of drug effect than MMRM analysis for the same endpoint in this one case study.
Collapse
Affiliation(s)
| | - Claire Ambery
- Clinical Pharmacology Modelling and Simulation, GlaxoSmithKline plc, London, UK
| | - Shuying Yang
- Clinical Pharmacology Modelling and Simulation, GlaxoSmithKline plc, London, UK
| | - Maggie Tabberer
- Patient Centred Outcomes: Value Evidence and Outcomes, GlaxoSmithKline plc, Brentford, Middlesex, UK
| | - Misba Beerahee
- Clinical Pharmacology Modelling and Simulation, GlaxoSmithKline plc, London, UK
| | - Elodie L Plan
- Department of Pharmacy, Uppsala University, Box 580, 751 23, Uppsala, Sweden
| | - Mats O Karlsson
- Department of Pharmacy, Uppsala University, Box 580, 751 23, Uppsala, Sweden.
| |
Collapse
|
10
|
Comets E, Rodrigues C, Jullien V, Ursino M. Conditional Non-parametric Bootstrap for Non-linear Mixed Effect Models. Pharm Res 2021; 38:1057-1066. [PMID: 34075519 DOI: 10.1007/s11095-021-03052-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Accepted: 05/03/2021] [Indexed: 11/26/2022]
Abstract
PURPOSE Non-linear mixed effect models are widely used and increasingly integrated into decision-making processes. Propagating uncertainty is an important element of this process, and while standard errors (SE) on pa- rameters are most often computed using asymptotic approaches, alternative methods such as the bootstrap are also available. In this article, we propose a modified residual parametric bootstrap taking into account the different levels of variability involved in these models. METHODS The proposed approach uses samples from the individual conditional distribution, and was implemented in R using the saemix algorithm. We performed a simulation study to assess its performance in different scenarios, comparing it to the asymptotic approximation and to standard bootstraps in terms of coverage, also looking at bias in the parameters and their SE. RESULTS Simulations with an Emax model with different designs and sigmoidicity factors showed a similar coverage rate to the parametric bootstrap, while requiring less hypotheses. Bootstrap improved coverage in several scenarios compared to the asymptotic method especially for the variance param-eters. However, all bootstraps were sensitive to estimation bias in the original datasets. CONCLUSIONS The conditional bootstrap provided better coverage rate than the traditional residual bootstrap, while preserving the structure of the data generating process.
Collapse
Affiliation(s)
- Emmanuelle Comets
- Universit́e de Paris, INSERM IAME; INSERM, CIC 1414; Rennes-1 University, France 16 rue Henri Huchard, 75018, Paris, France.
| | | | - Vincent Jullien
- UF Pharmacologie, GH Paris Seine Saint-Denis, Universit́e Paris, 13, Paris, France
| | - Moreno Ursino
- Unit of Clinical Epidemiology, Assistance Publique-Hôpitaux de Paris, CHU Robert Debré, Université de Paris, Sorbonne Paris-Cité, Inserm U1123 and CIC-EC 1426, Paris, F-75019, France
- Inserm, Centre de Recherche des Cordeliers, Sorbonne Université, Université de Paris, Paris, F-75006, France
- Inria, HeKA, F-75006, Paris, France
| |
Collapse
|
11
|
Chasseloup E, Tessier A, Karlsson MO. Assessing Treatment Effects with Pharmacometric Models: A New Method that Addresses Problems with Standard Assessments. AAPS JOURNAL 2021; 23:63. [PMID: 33942179 PMCID: PMC8093168 DOI: 10.1208/s12248-021-00596-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/17/2021] [Accepted: 04/13/2021] [Indexed: 12/02/2022]
Abstract
Longitudinal pharmacometric models offer many advantages in the analysis of clinical trial data, but potentially inflated type I error and biased drug effect estimates, as a consequence of model misspecifications and multiple testing, are main drawbacks. In this work, we used real data to compare these aspects for a standard approach (STD) and a new one using mixture models, called individual model averaging (IMA). Placebo arm data sets were obtained from three clinical studies assessing ADAS-Cog scores, Likert pain scores, and seizure frequency. By randomly (1:1) assigning patients in the above data sets to “treatment” or “placebo,” we created data sets where any significant drug effect was known to be a false positive. Repeating the process of random assignment and analysis for significant drug effect many times (N = 1000) for each of the 40 to 66 placebo-drug model combinations, statistics of the type I error and drug effect bias were obtained. Across all models and the three data types, the type I error was (5th, 25th, 50th, 75th, 95th percentiles) 4.1, 11.4, 40.6, 100.0, 100.0 for STD, and 1.6, 3.5, 4.3, 5.0, 6.0 for IMA. IMA showed no bias in the drug effect estimates, whereas in STD bias was frequently present. In conclusion, STD is associated with inflated type I error and risk of biased drug effect estimates. IMA demonstrated controlled type I error and no bias.
Collapse
Affiliation(s)
| | - Adrien Tessier
- Division of Quantitative Pharmacology, Institut de Recherches Internationales Servier, Suresnes, France
| | - Mats O Karlsson
- Department of Pharmacy, Uppsala University, Uppsala, Sweden.
| |
Collapse
|
12
|
Buatois S, Ueckert S, Frey N, Retout S, Mentré F. cLRT-Mod: An efficient methodology for pharmacometric model-based analysis of longitudinal phase II dose finding studies under model uncertainty. Stat Med 2021; 40:2435-2451. [PMID: 33650148 DOI: 10.1002/sim.8913] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2019] [Revised: 12/14/2020] [Accepted: 02/01/2021] [Indexed: 11/07/2022]
Abstract
Within the challenging context of phase II dose-finding trials, longitudinal analyses may increase drug effect detection power compared to an end-of-treatment analysis. This work proposes cLRT-Mod, a pharmacometric adaptation of the MCP-Mod methodology, which allows the use of nonlinear mixed effect models to first detect a dose-response signal and then identify the doses for the confirmatory phase while accounting for model structure uncertainty. The method was evaluated through extensive clinical trial simulations of a hypothetical phase II dose-finding trial using different scenarios and comparing different methods such as MCP-Mod. The results show an increase in power using cLRT with longitudinal data compared to an EOT multiple contrast tests for scenarios with small sample size and weak drug effect while maintaining pre-specifiability of the models prior to data analysis and the nominal type I error. This work shows how model averaging provides better coverage probability of the drug effect in the prediction step, and avoids under-estimation of the size of the confidence interval. Finally, for illustration purpose cLRT-Mod was applied to the analysis of a real phase II dose-finding trial.
Collapse
Affiliation(s)
- Simon Buatois
- IAME, UMR 1137, INSERM, University Paris Diderot, Paris, France.,Roche Pharma Research and Early Development, Pharmaceutical Sciences, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd, Basel, Switzerland
| | - Sebastian Ueckert
- Department of Pharmaceutical Biosciences, Uppsala University, Uppsala, Sweden
| | - Nicolas Frey
- Roche Pharma Research and Early Development, Pharmaceutical Sciences, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd, Basel, Switzerland
| | - Sylvie Retout
- Roche Pharma Research and Early Development, Pharmaceutical Sciences, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd, Basel, Switzerland
| | - France Mentré
- IAME, UMR 1137, INSERM, University Paris Diderot, Paris, France
| |
Collapse
|
13
|
Uster DW, Stocker SL, Carland JE, Brett J, Marriott DJE, Day RO, Wicha SG. A Model Averaging/Selection Approach Improves the Predictive Performance of Model-Informed Precision Dosing: Vancomycin as a Case Study. Clin Pharmacol Ther 2020; 109:175-183. [PMID: 32996120 DOI: 10.1002/cpt.2065] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2020] [Accepted: 09/12/2020] [Indexed: 11/10/2022]
Abstract
Many important drugs exhibit substantial variability in pharmacokinetics and pharmacodynamics leading to a loss of the desired clinical outcomes or significant adverse effects. Forecasting drug exposures using pharmacometric models can improve individual target attainment when compared with conventional therapeutic drug monitoring (TDM). However, selecting the "correct" model for this model-informed precision dosing (MIPD) is challenging. We derived and evaluated a model selection algorithm (MSA) and a model averaging algorithm (MAA), which automates model selection and finds the best model or combination of models for each patient using vancomycin as a case study, and implemented both algorithms in the MIPD software "TDMx." The predictive performance (based on accuracy and precision) of the two algorithms was assessed in (i) a simulation study of six distinct populations and (ii) a clinical dataset of 180 patients undergoing TDM during vancomycin treatment and compared with the performance obtained using a single model. Throughout the six virtual populations the MSA and MAA (imprecision: 9.9-24.2%, inaccuracy: less than ± 8.2%) displayed more accurate predictions than the single models (imprecision: 8.9-51.1%; inaccuracy: up to 28.9%). In the clinical dataset, the predictive performance of the single models applying at least one plasma concentration varied substantially (imprecision: 28-62%, inaccuracy: -16 to 25%), whereas the MSA or MAA utilizing these models simultaneously resulted in unbiased and precise predictions (imprecision: 29% and 30%, inaccuracy: -5% and 0%, respectively). MSA and MAA approaches implemented in TDMx might thereby lower the burden of fit-for-purpose validation of individual models and streamline MIPD.
Collapse
Affiliation(s)
- David W Uster
- Department of Clinical Pharmacy, Institute of Pharmacy, University of Hamburg, Hamburg, Germany
| | - Sophie L Stocker
- Department of Clinical Pharmacology and Toxicology, St. Vincent's Hospital, Sydney, New South Wales, Australia.,St. Vincent's Clinical School, University of New South Wales, Sydney, New South Wales, Australia
| | - Jane E Carland
- Department of Clinical Pharmacology and Toxicology, St. Vincent's Hospital, Sydney, New South Wales, Australia.,St. Vincent's Clinical School, University of New South Wales, Sydney, New South Wales, Australia
| | - Jonathan Brett
- Department of Clinical Pharmacology and Toxicology, St. Vincent's Hospital, Sydney, New South Wales, Australia.,St. Vincent's Clinical School, University of New South Wales, Sydney, New South Wales, Australia
| | - Deborah J E Marriott
- St. Vincent's Clinical School, University of New South Wales, Sydney, New South Wales, Australia.,Department of Clinical Microbiology and Infectious Diseases, St. Vincent's Hospital, Sydney, New South Wales, Australia
| | - Richard O Day
- Department of Clinical Pharmacology and Toxicology, St. Vincent's Hospital, Sydney, New South Wales, Australia.,St. Vincent's Clinical School, University of New South Wales, Sydney, New South Wales, Australia
| | - Sebastian G Wicha
- Department of Clinical Pharmacy, Institute of Pharmacy, University of Hamburg, Hamburg, Germany
| |
Collapse
|
14
|
Loingeville F, Bertrand J, Nguyen TT, Sharan S, Feng K, Sun W, Han J, Grosser S, Zhao L, Fang L, Möllenhoff K, Dette H, Mentré F. New Model-Based Bioequivalence Statistical Approaches for Pharmacokinetic Studies with Sparse Sampling. AAPS JOURNAL 2020; 22:141. [PMID: 33125589 DOI: 10.1208/s12248-020-00507-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/22/2020] [Accepted: 09/01/2020] [Indexed: 11/30/2022]
Abstract
In traditional pharmacokinetic (PK) bioequivalence analysis, two one-sided tests (TOST) are conducted on the area under the concentration-time curve and the maximal concentration derived using a non-compartmental approach. When rich sampling is unfeasible, a model-based (MB) approach, using nonlinear mixed effect models (NLMEM) is possible. However, MB-TOST using asymptotic standard errors (SE) presents increased type I error when asymptotic conditions do not hold. In this work, we propose three alternative calculations of the SE based on (i) an adaptation to NLMEM of the correction proposed by Gallant, (ii) the a posteriori distribution of the treatment coefficient using the Hamiltonian Monte Carlo algorithm, and (iii) parametric random effects and residual errors bootstrap. We evaluate these approaches by simulations, for two-arms parallel and two-period, two-sequence cross-over design with rich (n = 10) and sparse (n = 3) sampling under the null and the alternative hypotheses, with MB-TOST. All new approaches correct for the inflation of MB-TOST type I error in PK studies with sparse designs. The approach based on the a posteriori distribution appears to be the best compromise between controlled type I errors and computing times. MB-TOST using non-asymptotic SE controls type I error rate better than when using asymptotic SE estimates for bioequivalence on PK studies with sparse sampling.
Collapse
Affiliation(s)
- Florence Loingeville
- University of Paris, IAME INSERM, UMR 1137, 75018, Paris, France. .,University of Lille, CHU Lille, ULR 2694 - METRICS : Evaluation of Health Technologies and Medical Practices, F-59000, Lille, France. .,Laboratoire de Biomathématiques, Faculté de Pharmacie, 3 Rue du Professeur Laguesse, 59 000, Lille, France.
| | - Julie Bertrand
- University of Paris, IAME INSERM, UMR 1137, 75018, Paris, France
| | - Thu Thuy Nguyen
- University of Paris, IAME INSERM, UMR 1137, 75018, Paris, France
| | - Satish Sharan
- Division of Quantitative Methods and Modeling, Office of Research Standards, Office of Generic Drugs, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, 20993, USA
| | - Kairui Feng
- Division of Quantitative Methods and Modeling, Office of Research Standards, Office of Generic Drugs, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, 20993, USA
| | - Wanjie Sun
- Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, 20993, USA
| | - Jing Han
- Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, 20993, USA
| | - Stella Grosser
- Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, 20993, USA
| | - Liang Zhao
- Division of Quantitative Methods and Modeling, Office of Research Standards, Office of Generic Drugs, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, 20993, USA
| | - Lanyan Fang
- Division of Quantitative Methods and Modeling, Office of Research Standards, Office of Generic Drugs, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, 20993, USA
| | - Kathrin Möllenhoff
- Department of Mathematics, Ruhr-Universitat Bochum, Bochum, Germany.,Institute of Medical Statistics and Computational Biology, Faculty of Medicine, University of Cologne, Cologne, Germany
| | - Holger Dette
- Department of Mathematics, Ruhr-Universitat Bochum, Bochum, Germany
| | - France Mentré
- University of Paris, IAME INSERM, UMR 1137, 75018, Paris, France
| |
Collapse
|
15
|
Möllenhoff K, Loingeville F, Bertrand J, Nguyen TT, Sharan S, Zhao L, Fang L, Sun G, Grosser S, Mentré F, Dette H. Efficient model-based bioequivalence testing. Biostatistics 2020; 23:314-327. [DOI: 10.1093/biostatistics/kxaa026] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2020] [Revised: 06/18/2020] [Accepted: 06/21/2020] [Indexed: 11/14/2022] Open
Abstract
SummaryThe classical approach to analyze pharmacokinetic (PK) data in bioequivalence studies aiming to compare two different formulations is to perform noncompartmental analysis (NCA) followed by two one-sided tests (TOST). In this regard, the PK parameters area under the curve (AUC) and $C_{\max}$ are obtained for both treatment groups and their geometric mean ratios are considered. According to current guidelines by the U.S. Food and Drug Administration and the European Medicines Agency, the formulations are declared to be sufficiently similar if the $90\%$ confidence interval for these ratios falls between $0.8$ and $1.25 $. As NCA is not a reliable approach in case of sparse designs, a model-based alternative has already been proposed for the estimation of $\rm AUC$ and $C_{\max}$ using nonlinear mixed effects models. Here we propose another, more powerful test than the TOST and demonstrate its superiority through a simulation study both for NCA and model-based approaches. For products with high variability on PK parameters, this method appears to have closer type I errors to the conventionally accepted significance level of $0.05$, suggesting its potential use in situations where conventional bioequivalence analysis is not applicable.
Collapse
Affiliation(s)
- Kathrin Möllenhoff
- Department of Mathematics, Ruhr-Universität Bochum and Institute of Medical Statistics and Computational Biology, Faculty of Medicine, University of Cologne, Cologne, Germany
| | - Florence Loingeville
- Faculty of Pharmacy, University of Lille, EA 2694: Public health: Epidemiology and Healthcare Quality, 59000 Lille, France
| | | | | | - Satish Sharan
- Division of Quantitative Methods and Modeling, Office of Research Standards, Office of Generic Drugs, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD 20993, USA
| | - Liang Zhao
- Division of Quantitative Methods and Modeling, Office of Research Standards, Office of Generic Drugs, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD 20993, USA
| | - Lanyan Fang
- Division of Quantitative Methods and Modeling, Office of Research Standards, Office of Generic Drugs, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD 20993, USA
| | - Guoying Sun
- Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD 20993, USA
| | - Stella Grosser
- Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD 20993, USA
| | - France Mentré
- IAME INSERM, Université de Paris, 75018 Paris, France
| | - Holger Dette
- Department of Mathematics, Ruhr-Universität Bochum, Germany
| |
Collapse
|
16
|
Gonçalves A, Mentré F, Lemenuel-Diot A, Guedj J. Model Averaging in Viral Dynamic Models. AAPS JOURNAL 2020; 22:48. [PMID: 32060662 DOI: 10.1208/s12248-020-0426-7] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/04/2019] [Accepted: 01/16/2020] [Indexed: 12/24/2022]
Abstract
The paucity of experimental data makes both inference and prediction particularly challenging in viral dynamic models. In the presence of several candidate models, a common strategy is model selection (MS), in which models are fitted to the data but only results obtained with the "best model" are presented. However, this approach ignores model uncertainty, which may lead to inaccurate predictions. When several models provide a good fit to the data, another approach is model averaging (MA) that weights the predictions of each model according to its consistency to the data. Here, we evaluated by simulations in a nonlinear mixed-effect model framework the performances of MS and MA in two realistic cases of acute viral infection, i.e., (1) inference in the presence of poorly identifiable parameters, namely, initial viral inoculum and eclipse phase duration, (2) uncertainty on the mechanisms of action of the immune response. MS was associated in some scenarios with a large rate of false selection. This led to a coverage rate lower than the nominal coverage rate of 0.95 in the majority of cases and below 0.50 in some scenarios. In contrast, MA provided better estimation of parameter uncertainty, with coverage rates ranging from 0.72 to 0.98 and mostly comprised within the nominal coverage rate. Finally, MA provided similar predictions than those obtained with MS. In conclusion, parameter estimates obtained with MS should be taken with caution, especially when several models well describe the data. In this situation, MA has better performances and could be performed to account for model uncertainty.
Collapse
Affiliation(s)
- Antonio Gonçalves
- Université de Paris, IAME, INSERM, Henri Huchard, F-75018, Paris, France.
| | - France Mentré
- Université de Paris, IAME, INSERM, Henri Huchard, F-75018, Paris, France
| | - Annabelle Lemenuel-Diot
- Roche Pharmaceutical Research and Early Development, Pharmaceutical Sciences, Roche Innovation Center, Basel, Switzerland
| | - Jérémie Guedj
- Université de Paris, IAME, INSERM, Henri Huchard, F-75018, Paris, France
| |
Collapse
|
17
|
Alhorn K, Schorning K, Dette H. Optimal designs for frequentist model averaging. Biometrika 2019; 106:665-682. [PMID: 31427825 DOI: 10.1093/biomet/asz036] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2018] [Indexed: 11/14/2022] Open
Abstract
We consider the problem of designing experiments for estimating a target parameter in regression analysis when there is uncertainty about the parametric form of the regression function. A new optimality criterion is proposed that chooses the experimental design to minimize the asymptotic mean squared error of the frequentist model averaging estimate. Necessary conditions for the optimal solution of a locally and Bayesian optimal design problem are established. The results are illustrated in several examples, and it is demonstrated that Bayesian optimal designs can yield a reduction of the mean squared error of the model averaging estimator by up to 45%.
Collapse
Affiliation(s)
- K Alhorn
- Fakultät Statistik, Technische Universität Dortmund, Dortmund, Germany
| | - K Schorning
- Fakultät für Mathematik, Ruhr-Universität Bochum, Bochum, Germany
| | - H Dette
- Fakultät für Mathematik, Ruhr-Universität Bochum, Bochum, Germany
| |
Collapse
|
18
|
Seurat J, Nguyen TT, Mentré F. Robust designs accounting for model uncertainty in longitudinal studies with binary outcomes. Stat Methods Med Res 2019; 29:934-952. [DOI: 10.1177/0962280219850588] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
To optimize designs for longitudinal studies analyzed by mixed-effect models with binary outcomes, the Fisher information matrix can be used. Optimal design approaches, however, require a priori knowledge of the model. We aim to propose, for the first time, a robust design approach accounting for model uncertainty in longitudinal trials with two treatment groups, assuming mixed-effect logistic models. To optimize designs given one model, we compute several optimality criteria based on Fisher information matrix evaluated by the new approach based on Monte-Carlo/Hamiltonian Monte-Carlo. We propose to use the DDS-optimality criterion, as it ensures a compromise between the precision of estimation of the parameters, and hence the Wald test power, and the overall precision of parameter estimation. To account for model uncertainty, we assume candidate models with their respective weights. We compute robust design across these models using compound DDS-optimality. Using the Fisher information matrix, we propose to predict the average power over these models. Evaluating this approach by clinical trial simulations, we show that the robust design is efficient across all models, allowing one to achieve good power of test. The proposed design strategy is a new and relevant approach to design longitudinal studies with binary outcomes, accounting for model uncertainty.
Collapse
Affiliation(s)
- Jérémy Seurat
- IAME, UMR 1137, INSERM, Université Paris Diderot, Paris, France
| | - Thu Thuy Nguyen
- IAME, UMR 1137, INSERM, Université Paris Diderot, Paris, France
| | - France Mentré
- IAME, UMR 1137, INSERM, Université Paris Diderot, Paris, France
| |
Collapse
|
19
|
Loingeville F, Nguyen TT, Riviere MK, Mentré F. Robust designs in longitudinal studies accounting for parameter and model uncertainties – application to count data. J Biopharm Stat 2019; 30:31-45. [DOI: 10.1080/10543406.2019.1607367] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Florence Loingeville
- INSERM, IAME, UMR 1137, F-75018, Paris, France
- University Paris Diderot, Sorbonne Paris Cité, F-75018, Paris, France
- Faculty of Pharmacy, University of Lille, EA 2694, Public Health: Epidemiology and Healthcare quality, F-59000 Lille, France
| | - Thu Thuy Nguyen
- INSERM, IAME, UMR 1137, F-75018, Paris, France
- University Paris Diderot, Sorbonne Paris Cité, F-75018, Paris, France
| | - Marie-Karelle Riviere
- Statistical Methodology Group, Biostatistics and Programming department, Sanofi-Aventis R&D, Chilly-Mazarin, France
| | - France Mentré
- INSERM, IAME, UMR 1137, F-75018, Paris, France
- University Paris Diderot, Sorbonne Paris Cité, F-75018, Paris, France
| |
Collapse
|