Methods for non-proportional hazards in clinical trials: A systematic review

For the analysis of time-to-event data, frequently used methods such as the log-rank test or the Cox proportional hazards model are based on the proportional hazards assumption, which is often debatable. Although a wide range of parametric and non-parametric methods for non-proportional hazards has been proposed, there is no consensus on the best approaches. To close this gap, we conducted a systematic literature search to identify statistical methods and software appropriate under non-proportional hazard. Our literature search identified 907 abstracts, out of which we included 211 articles, mostly methodological ones. Review articles and applications were less frequently identified. The articles discuss effect measures, effect estimation and regression approaches, hypothesis tests, and sample size calculation approaches, which are often tailored to specific non-proportional hazard situations. Using a unified notation, we provide an overview of methods available. Furthermore, we derive some guidance from the identified articles.

Item performance of the scale for the assessment and rating of ataxia in rare and ultra-rare genetic ataxias

The Scale for the Assessment and Rating of Ataxia (SARA) is widely used for assessing the severity and progression of genetic cerebellar ataxias. SARA is now considered a primary end point in several ataxia treatment trials, but its underlying composite item measurement model has not yet been tested. This work aimed to evaluate the composite properties of SARA and its items using item response theory (IRT) and to demonstrate its applicability across even ultra-rare genetic ataxias. Leveraging SARA subscores data from 1932 visits from 990 patients of the Autosomal Recessive Cerebellar Ataxias (ARCA) registry, we assessed the performance of SARA using IRT methodology. The item characteristics were evaluated over the ataxia severity range of the entire ataxia population as well as the assessment validity across 115 genetic ARCA subpopulations. A unidimensional IRT model was able to describe SARA item data, indicating that SARA captures one single latent variable. All items had high discrimination values (1.5-2.9) indicating the effectiveness of the SARA in differentiating between subjects with different disease statuses. Each item contributed between 7% and 28% of the total assessment informativeness. There was no evidence for differences between the 115 genetic ARCA subpopulations in SARA applicability. These results show the good discrimination ability of SARA with all of its items adding informational value. The IRT framework provides a thorough description of SARA on the item level, and facilitates its utilization as a clinical outcome assessment in upcoming longitudinal natural history or treatment trials, across a large number of ataxias, including ultra-rare ones.

The Role of Model Master Files for Sharing, Acceptance, and Communication with FDA

With the evolving role of Model Integrated Evidence (MIE) in generic drug development and regulatory applications, the need for improving Model Sharing, Acceptance, and Communication with the FDA is warranted. Model Master File (MMF) refers to a quantitative model or a modeling platform that has undergone sufficient model Verification & Validation to be recognized as sharable intellectual property that is acceptable for regulatory purposes. MMF provides a framework for regulatorily acceptable modeling practice, which can be used with confidence to support MIE by both the industry and the U.S. Food and Drug Administration (FDA). In 2022, the FDA and the Center for Research on Complex Generics (CRCG) hosted a virtual public workshop to discuss the best practices for utilizing modeling approaches to support generic product development. This report summarizes the presentations and panel discussions of the workshop symposium entitled "Model Sharing, Acceptance, and Communication with the FDA". The symposium and this report serve as a kick-off discussion for further utilities of MMF and best practices of utilizing MMF in drug development and regulatory submissions. The potential advantages of MMFs have garnered acknowledgment from model developers, industries, and the FDA throughout the workshop. To foster a unified comprehension of MMFs and establish best practices for their application, further dialogue and cooperation among stakeholders are imperative. To this end, a subsequent workshop is scheduled for May 2-3, 2024, in Rockville, Maryland, aiming to delve into the practical facets and best practices of MMFs pertinent to regulatory submissions involving modeling and simulation methodologies.

Optimizing the design of a pharmacokinetic trial to evaluate the dosing scheme of a novel tuberculosis drug in children living with or without HIV

Pharmacokinetic (PK) studies in children are usually small and have ethical constraints due to the medical complexities of drawing blood in this special population. Often, population PK models for the drug(s) of interest are available in adults, and these models can be extended to incorporate the expected deviations seen in children. As a consequence, there is increasing interest in the use of optimal design methodology to design PK sampling schemes in children that maximize information using a small sample size and limited number of sampling times per dosing period. As a case study, we use the novel tuberculosis drug delamanid, and show how applications of optimal design methodology can result in highly efficient and model-robust designs in children for estimating PK parameters using a limited number of sampling measurements. Using developed population PK models based on available data from adults living with and without HIV, and limited data on children without HIV, competing designs for children living with HIV were derived and assessed based on robustness to model uncertainty.

Statistical recommendations for count, binary, and ordinal data in rare disease cross-over trials

BACKGROUND

Recommendations for statistical methods in rare disease trials are scarce, especially for cross-over designs. As a result various state-of-the-art methodologies were compared as neutrally as possible using an illustrative data set from epidermolysis bullosa research to build recommendations for count, binary, and ordinal outcome variables. For this purpose, parametric (model averaging), semiparametric (generalized estimating equations type [GEE-like]) and nonparametric (generalized pairwise comparisons [GPC] and a marginal model implemented in the R package nparLD) methods were chosen by an international consortium of statisticians.

RESULTS

It was found that there is no uniformly best method for the aforementioned types of outcome variables, but in particular situations, there are methods that perform better than others. Especially if maximizing power is the primary goal, the prioritized unmatched GPC method was able to achieve particularly good results, besides being appropriate for prioritizing clinically relevant time points. Model averaging led to favorable results in some scenarios especially within the binary outcome setting and, like the GEE-like semiparametric method, also allows for considering period and carry-over effects properly. Inference based on the nonparametric marginal model was able to achieve high power, especially in the ordinal outcome scenario, despite small sample sizes due to separate testing of treatment periods, and is suitable when longitudinal and interaction effects have to be considered.

CONCLUSION

Overall, a balance has to be found between achieving high power, accounting for cross-over, period, or carry-over effects, and prioritizing clinically relevant time points.

Generation and application of avatars in pharmacometric modelling

Simulations from population models have critical applications in drug discovery and development. Avatars or digital twins, defined as individual simulations matching clinical criteria of interest compared to observations from a real subject within a predefined margin of accuracy, may be a better option for simulations performed to inform future drug development stages in cases where an adequate model is not achievable. The aim of this work was to (1) investigate methods for generating avatars with pharmacometric models, and (2) explore the properties of the generated avatars to assess the impact of the different selection settings on the number of avatars per subject, their closeness to the individual observations, and the properties of the selected samples subset from the theoretical model parameters probability density function. Avatars were generated using different combinations of nature and number of clinical criteria, accuracy of agreement, and/or number of simulations for two examples models previously published (hemato-toxicity and integrated glucose-insulin model). The avatar distribution could be used to assess the appropriateness of the models assumed parameter distribution. Similarly it could be used to assess the models ability to properly describe the trajectories of the observations. Avatars can give nuanced information regarding the ability of a model to simulate data similar to the observations both at the population and at the individual level. Further potential applications for avatars may be as a diagnostic tool, an alternative to simulations with insurance to replicate key clinical features, and as an individual measure of model fit.

Establishing the suitability of model-integrated evidence to demonstrate bioequivalence for long-acting injectable and implantable drug products: Summary of workshop

On November 30, 2021, the US Food and Drug administration (FDA) and the Center for Research on Complex Generics (CRCG) hosted a virtual public workshop titled "Establishing the Suitability of Model-Integrated Evidence (MIE) to Demonstrate Bioequivalence for Long-Acting Injectable and Implantable (LAI) Drug Products." This workshop brought relevant parties from the industry, academia, and the FDA in the field of modeling and simulation to explore, identify, and recommend best practices on utilizing MIE for bioequivalence (BE) assessment of LAI products. This report summerized presentations and panel discussions for topics including challenges and opportunities in development and assessment of generic LAI products, current status of utilizing MIE, recent research progress of utilizing MIE in generic LAI products, alternative designs for BE studies of LAI products, and model validation/verification strategies associated with different types of MIE approaches.

Metaheuristics for pharmacometrics

Metaheuristics is a powerful optimization tool that is increasingly used across disciplines to tackle general purpose optimization problems. Nature-inspired metaheuristic algorithms is a subclass of metaheuristic algorithms and have been shown to be particularly flexible and useful in solving complicated optimization problems in computer science and engineering. A common practice with metaheuristics is to hybridize it with another suitably chosen algorithm for enhanced performance. This paper reviews metaheuristic algorithms and demonstrates some of its utility in tackling pharmacometric problems. Specifically, we provide three applications using one of its most celebrated members, particle swarm optimization (PSO), and show that PSO can effectively estimate parameters in complicated nonlinear mixed-effects models and to gain insights into statistical identifiability issues in a complex compartment model. In the third application, we demonstrate how to hybridize PSO with sparse grid, which is an often-used technique to evaluate high dimensional integrals, to search for D -efficient designs for estimating parameters in nonlinear mixed-effects models with a count outcome. We also show the proposed hybrid algorithm outperforms its competitors when sparse grid is replaced by its competitor, adaptive gaussian quadrature to approximate the integral, or when PSO is replaced by three notable nature-inspired metaheuristic algorithms.

Tutorial for $DESIGN in NONMEM: Clinical trial evaluation and optimization

This NONMEM tutorial shows how to evaluate and optimize clinical trial designs, using algorithms developed in design software, such as PopED and PFIM 4.0. Parameter precision and model parameter estimability is obtained by assessing the Fisher Information Matrix (FIM), providing expected model parameter uncertainty. Model parameter identifiability may be uncovered by very large standard errors or inability to invert an FIM. Because evaluation of FIM is more efficient than clinical trial simulation, more designs can be investigated, and the design of a clinical trial can be optimized. This tutorial provides simple and complex pharmacokinetic/pharmacodynamic examples on obtaining optimal sample times, doses, or best division of subjects among design groups. Robust design techniques accounting for likely variability among subjects are also shown. A design evaluator and optimizer within NONMEM allows any control stream first developed for trial design exploration to be subsequently used for estimation of parameters of simulated or clinical data, without transferring the model to another software. Conversely, a model developed in NONMEM could be used for design optimization. In addition, the $DESIGN feature can be used on any model file and dataset combination to retrospectively evaluate the model parameter uncertainty one would expect given that the model generated the data, particularly if outliers of the actual data prevent a reasonable assessment of the variance‐covariance. The NONMEM trial design feature is suitable for standard continuous data, whereas more elaborate trial designs or with noncontinuous data‐types can still be accomplished in optimal design dedicated software like PopED and PFIM.

Pharmacometrics meets statistics-A synergy for modern drug development

Modern drug development problems are very complex and require integration of various scientific fields. Traditionally, statistical methods have been the primary tool for design and analysis of clinical trials. Increasingly, pharmacometric approaches using physiology-based drug and disease models are applied in this context. In this paper, we show that statistics and pharmacometrics have more in common than what keeps them apart, and collectively, the synergy from these two quantitative disciplines can provide greater advances in clinical research and development, resulting in novel and more effective medicines to patients with medical need.

Bounded Integer Modeling of Symptom Scales Specific to Lower Urinary Tract Symptoms Secondary to Benign Prostatic Hyperplasia

The International Prostate Symptom Score (IPSS), the quality of life (QoL) score, and the benign prostatic hyperplasia impact index (BII) are three different scales commonly used to assess the severity of lower urinary tract symptoms associated with benign prostatic hyperplasia (BPH-LUTS). Based on a phase II clinical trial including 403 patients with moderate to severe BPH-LUTS, the objectives of this study were to (i) develop traditional pharmacometric and bounded integer (BI) models for the IPSS, QoL score, and BII endpoints, respectively; (ii) compare the power and type I error in detecting drug effects of BI modeling with traditional methods through simulation; and (iii) obtain quantitative translation between scores on the three abovementioned scales using a BI modeling framework. All developed models described the data adequately. Pharmacometric modeling using a continuous variable (CV) approach was overall found to be the most robust in terms of type I error and power to detect a drug effect. In most cases, BI modeling showed similar performance to the CV approach, yet severely inflated type I error was generally observed when inter-individual variability (IIV) was incorporated in the BI variance function (g()). BI modeling without IIV in g() showed greater type I error control compared to the ordered categorical approach. Lastly, a multiple-scale BI model was developed and estimated the relationship between scores on the three BPH-LUTS scales with overall low uncertainty. The current study yields greater understanding of the operating characteristics of the novel BI modeling approach and highlights areas potentially requiring further improvement.

Public Workshop Summary Report on Fiscal Year 2021 Generic Drug Regulatory Science Initiatives: Data Analysis and Model-Based Bioequivalence

On May 4, 2020, the US Food and Drug Administration (FDA) hosted an online public workshop titled "FY 2020 Generic Drug Regulatory Science Initiatives Public Workshop" to provide an overview of the status of the science and research priorities and to solicit input on the development of Generic Drug User Fee Amendments fiscal year 2021 priorities. This report summarizes the podium presentations and the outcome of discussions along with innovative ways to overcome challenges and significant opportunities related to model-based approaches in bioequivalence assessment for breakout session 4 titled, "Data analysis and model-based bioequivalence (BE)." This session focused on the application of model-based approaches in the generic drug development, with a vision of accelerating regulatory decision making for abbreviated new drug application assessments. The session included both podium presentations and panel discussions with three topics of interest: (i) in vitro study evaluation methods and their clinical relevance, (ii) challenges in model-based BE, (iii) emerging expertise and tools in implementing new BE approaches.

Item Response Theory Modeling of the International Prostate Symptom Score in Patients with Lower Urinary Tract Symptoms Associated with Benign Prostatic Hyperplasia

Item response theory (IRT) was used to characterize the time course of lower urinary tract symptoms due to benign prostatic hyperplasia (BPH-LUTS) measured by item-level International Prostate Symptom Scores (IPSS). The Fisher information content of IPSS items was determined and the power to detect a drug effect using the IRT approach was examined. Data from 403 patients with moderate-to-severe BPH-LUTS in a placebo-controlled phase II trial studying the effect of degarelix over 6 months were used for modeling. Three pharmacometric models were developed: a model for total IPSS, a unidimensional IRT model, and a bidimensional IRT model, the latter separating voiding and storage items. The population-level time course of BPH-LUTS in all models was described by initial improvement followed by worsening. In the unidimensional IRT model, the combined information content of IPSS voiding items represented 72% of the total information content, indicating that the voiding subscore may be more sensitive to changes in BPH-LUTS compared with the storage subscore. The pharmacometric models showed considerably higher power to detect a drug effect compared with a cross-sectional and while-on-treatment analysis of covariance, respectively. Compared with the sample size required to detect a drug effect at 80% power with the total IPSS model, a reduction of 5.9% and 11.7% was obtained with the unidimensional and bidimensional IPSS IRT model, respectively. Pharmacometric IRT analysis of the IPSS within BPH-LUTS may increase the precision and efficiency of treatment effect assessment, albeit to a more limited extent compared with applications in other therapeutic areas.

Integrated Item Response Theory Modeling of Multiple Patient-Reported Outcomes Assessing Lower Urinary Tract Symptoms Associated with Benign Prostatic Hyperplasia

In clinical trials within lower urinary tract symptoms due to benign prostatic hyperplasia (BPH-LUTS), the International Prostate Symptom Score (IPSS) is commonly the primary efficacy outcome while the Quality of Life (QoL) score and the BPH Impact Index (BII) are common secondary efficacy markers. The current study aimed to characterize BPH-LUTS progression using responses to the IPSS, the QoL, and the BII in an integrated item response theory (IRT) framework and assess the Fisher information of each scale. The power of this approach to detect a drug effect was compared with an IRT approach considering only IPSS responses. A unidimensional and a bidimensional pharmacometric IRT model, based on item-level IPSS responses in a clinical trial with 403 patients, were extended by incorporating patients’ QoL and summary BII scores over the 6-month trial period. In the developed unidimensional integrated model, the QoL score was found to be the most informative, representing 17% of the total Fisher information, while the combined information content of the seven IPSS items represented 70.6%. In the bidimensional model, “storage” and both storage and “voiding” disability drove QoL and summary BII responses, respectively. Sample size reduction of 16% to detect a drug effect at 80% power was obtained with the unidimensional integrated IRT model compared with its counterpart IPSS IRT model. This study shows that utilizing the information content across the IPSS, QoL, and BII scales in an integrated IRT framework results in a modest but meaningful increase in power to detect a drug effect.

Saddle-Reset for Robust Parameter Estimation and Identifiability Analysis of Nonlinear Mixed Effects Models

Parameter estimation of a nonlinear model based on maximizing the likelihood using gradient-based numerical optimization methods can often fail due to premature termination of the optimization algorithm. One reason for such failure is that these numerical optimization methods cannot distinguish between the minimum, maximum, and a saddle point; hence, the parameters found by these optimization algorithms can possibly be in any of these three stationary points on the likelihood surface. We have found that for maximization of the likelihood for nonlinear mixed effects models used in pharmaceutical development, the optimization algorithm Broyden–Fletcher–Goldfarb–Shanno (BFGS) often terminates in saddle points, and we propose an algorithm, saddle-reset, to avoid the termination at saddle points, based on the second partial derivative test. In this algorithm, we use the approximated Hessian matrix at the point where BFGS terminates, perturb the point in the direction of the eigenvector associated with the lowest eigenvalue, and restart the BFGS algorithm. We have implemented this algorithm in industry standard software for nonlinear mixed effects modeling (NONMEM, version 7.4 and up) and showed that it can be used to avoid termination of parameter estimation at saddle points, as well as unveil practical parameter non-identifiability. We demonstrate this using four published pharmacometric models and two models specifically designed to be practically non-identifiable.

A Population Pharmacokinetic-Pharmacodynamic Model of Pegfilgrastim

Neutropenia and febrile neutropenia (FN) are serious side effects of cytotoxic chemotherapy which may be alleviated with the administration of recombinant granulocyte colony-stimulating factor (GCSF) derivatives, such as pegfilgrastim (PG) which increases absolute neutrophil count (ANC). In this work, a population pharmacokinetic-pharmacodynamic (PKPD) model was developed based on data obtained from healthy volunteers receiving multiple administrations of PG. The developed model was a bidirectional PKPD model, where PG stimulated the proliferation, maturation, and margination of neutrophils and where circulating neutrophils in turn increased the elimination of PG. Simulations from the developed model show disproportionate changes in response with changes in dose. A dose increase of 10% from the 6 mg therapeutic dose taken as a reference leads to area under the curve (AUC) increases of ~50 and ~5% for PK and PD, respectively. A full random effects covariate model showed that little of the parameter variability could be explained by sex, age, body size, and race. As a consequence, little of the secondary parameter variability (C_max and AUC of PG and ANC) could be explained by these covariates.

Implementing Optimal Designs for Dose-Response Studies Through Adaptive Randomization for a Small Population Group

In dose-response studies with censored time-to-event outcomes, D-optimal designs depend on the true model and the amount of censored data. In practice, such designs can be implemented adaptively, by performing dose assignments according to updated knowledge of the dose-response curve at interim analysis. It is also essential that treatment allocation involves randomization-to mitigate various experimental biases and enable valid statistical inference at the end of the trial. In this work, we perform a comparison of several adaptive randomization procedures that can be used for implementing D-optimal designs for dose-response studies with time-to-event outcomes with small to moderate sample sizes. We consider single-stage, two-stage, and multi-stage adaptive designs. We also explore robustness of the designs to experimental (chronological and selection) biases. Simulation studies provide evidence that both the choice of an allocation design and a randomization procedure to implement the target allocation impact the quality of dose-response estimation, especially for small samples. For best performance, a multi-stage adaptive design with small cohort sizes should be implemented using a randomization procedure that closely attains the targeted D-optimal design at each stage. The results of the current work should help clinical investigators select an appropriate randomization procedure for their dose-response study.

Reduced and optimized trial designs for drugs described by a target mediated drug disposition model

Monoclonal antibodies against soluble targets are often rich and include the sampling of multiple analytes over a lengthy period of time. Predictive models built on data obtained in such studies can be useful in all drug development phases. If adequate model predictions can be maintained with a reduced design (e.g. fewer samples or shorter duration) the use of such designs may be advocated. The effect of reducing and optimizing a rich design based on a published study for Omalizumab (OMA) was evaluated as an example. OMA pharmacokinetics were characterized using a target-mediated drug disposition model considering the binding of OMA to free IgE and the subsequent formation of an OMA–IgE complex. The performance of the reduced and optimized designs was evaluated with respect to: efficiency, parameter uncertainty and predictions of free target. It was possible to reduce the number of samples in the study by 30% while still maintaining an efficiency of almost 90%. A reduction in sampling duration by two-thirds resulted in an efficiency of 75%. Omission of any analyte measurement or a reduction of the number of dose levels was detrimental to the efficiency of the designs (efficiency ≤ 51%). However, other metrics were, in some cases, relatively unaffected, showing that multiple metrics may be needed to obtain balanced assessments of design performance.

Feasibility of Exposure-Response Analyses for Clinical Dose-Ranging Studies of Drug Combinations

The exposure-response relationship of combinatory drug effects can be quantitatively described using pharmacodynamic interaction models, which can be used for the selection of optimal dose combinations. The aim of this simulation study was to evaluate the reliability of parameter estimates and the probability for accurate dose identification for various underlying exposure-response profiles, under a number of different phase II designs. An efficacy variable driven by the combined exposure of two theoretical compounds was simulated and model parameters were estimated using two different models, one estimating all parameters and one assuming that adequate previous knowledge for one drug is readily available. Estimation of all pharmacodynamic parameters under a realistic, in terms of sample size and study design, phase II trial, proved to be challenging. Inaccurate estimates were found in all exposure-response scenarios, except for situations where no pharmacodynamic interaction was present, with the drug potency and interaction parameters being the hardest to estimate. When previous knowledge of the exposure-response relationship of one of the monocomponents is available, such information should be utilized, as it enabled relevant improvements in parameter estimation and in correct dose identification. No general trends for classification of the performance of the tested study designs across different scenarios could be identified. This study shows that pharmacodynamic interactions models can be used for the exposure-response analysis of clinical endpoints especially when accompanied by appropriate dose selection in regard to the expected drug potencies and appropriate trial size and if information regarding the exposure-response profile of one monocomponent is available.

Correction to: Model-Based Adaptive Optimal Design (MBAOD) Improves Combination Dose Finding Designs: an Example in Oncology

The middle initial in the fifth author's name is incorrect in the original article. "Lena F. Friberg" should be "Lena E. Friberg". The original article was corrected.

Adaptive Optimal Designs for Dose-Finding Studies with Time-to-Event Outcomes

We consider optimal design problems for dose-finding studies with censored Weibull time-to-event outcomes. Locally D-optimal designs are investigated for a quadratic dose-response model for log-transformed data subject to right censoring. Two-stage adaptive D-optimal designs using maximum likelihood estimation (MLE) model updating are explored through simulation for a range of different dose-response scenarios and different amounts of censoring in the model. The adaptive optimal designs are found to be nearly as efficient as the locally D-optimal designs. A popular equal allocation design can be highly inefficient when the amount of censored data is high and when the Weibull model hazard is increasing. The issues of sample size planning/early stopping for an adaptive trial are investigated as well. The adaptive D-optimal design with early stopping can potentially reduce study size while achieving similar estimation precision as the fixed allocation design.