Interim analysis of continuous long-term endpoints in clinical trials with longitudinal outcomes

Using an early outcome as the sole source of information of interim decisions regarding treatment effect on a long-term endpoint: The non-Gaussian case

In randomized clinical trials that use a long-term efficacy endpoint, the follow-up time necessary to observe the endpoint may be substantial. In such trials, an attractive option is to consider an interim analysis based solely on an early outcome that could be used to expedite the evaluation of treatment's efficacy. Garcia Barrado et al. (Pharm Stat. 2022; 21: 209-219) developed a methodology that allows introducing such an early interim analysis for the case when both the early outcome and the long-term endpoint are normally-distributed, continuous variables. We extend the methodology to any combination of the early-outcome and long-term-endpoint types. As an example, we consider the case of a binary outcome and a time-to-event endpoint. We further evaluate the potential gain in operating characteristics (power, expected trial duration, and expected sample size) of a trial with such an interim analysis in function of the properties of the early outcome as a surrogate for the long-term endpoint.

Bayesian model averaging of longitudinal dose-response models

Selecting a safe and clinically beneficial dose can be difficult in drug development. Dose justification often relies on dose-response modeling where parametric assumptions are made in advance which may not adequately fit the data. This is especially problematic in longitudinal dose-response models, where additional parametric assumptions must be made. This paper proposes a class of longitudinal dose-response models to be used in the Bayesian model averaging paradigm which improve trial operating characteristics while maintaining flexibility a priori. A new longitudinal model for non-monotonic longitudinal profiles is proposed. The benefits and trade-offs of the proposed approach are demonstrated through a case study and simulation.

Group sequential designs for pragmatic clinical trials with early outcomes: methods and guidance for planning and implementation

BACKGROUND

Group sequential designs are one of the most widely used methodologies for adaptive design in randomized clinical trials. In settings where early outcomes are available, they offer large gains in efficiency compared to a fixed design. However, such designs are underused and used predominantly in therapeutic areas where there is expertise and experience in implementation. One barrier to their greater use is the requirement to undertake simulation studies at the planning stage that require considerable knowledge, coding experience and additional costs. Based on some modest assumptions about the likely patterns of recruitment and the covariance structure of the outcomes, some simple analytic expressions are presented that negate the need to undertake simulations.

METHODS

A model for longitudinal outcomes with an assumed approximate multivariate normal distribution and three contrasting simple recruitment models are described, based on fixed, increasing and decreasing rates. For assumed uniform and exponential correlation models, analytic expressions for the variance of the treatment effect and the effects of the early outcomes on reducing this variance at the primary outcome time-point are presented. Expressions for the minimum and maximum values show how the correlations and timing of the early outcomes affect design efficiency.

RESULTS

Simulations showed how patterns of information accrual varied between correlation and recruitment models, and consequentially to some general guidance for planning a trial. Using a previously reported group sequential trial as an exemplar, it is shown how the analytic expressions given here could have been used as a quick and flexible planning tool, avoiding the need for extensive simulation studies based on individual participant data.

CONCLUSIONS

The analytic expressions described can be routinely used at the planning stage of a putative trial, based on some modest assumptions about the likely number of outcomes and when they might occur and the expected recruitment patterns. Numerical simulations showed that these models behaved sensibly and allowed a range of design options to be explored in a way that would have been difficult and time-consuming if the previously described method of simulating individual trial participant data had been used.

Improving the mixed model for repeated measures to robustly increase precision in randomized trials

In randomized trials, repeated measures of the outcome are routinely collected. The mixed model for repeated measures (MMRM) leverages the information from these repeated outcome measures, and is often used for the primary analysis to estimate the average treatment effect at the primary endpoint. MMRM, however, can suffer from bias and precision loss when it models intermediate outcomes incorrectly, and hence fails to use the post-randomization information harmlessly. This paper proposes an extension of the commonly used MMRM, called IMMRM, that improves the robustness and optimizes the precision gain from covariate adjustment, stratified randomization, and adjustment for intermediate outcome measures. Under regularity conditions and missing completely at random, we prove that the IMMRM estimator for the average treatment effect is robust to arbitrary model misspecification and is asymptotically equal or more precise than the analysis of covariance (ANCOVA) estimator and the MMRM estimator. Under missing at random, IMMRM is less likely to be misspecified than MMRM, and we demonstrate via simulation studies that IMMRM continues to have less bias and smaller variance. Our results are further supported by a re-analysis of a randomized trial for the treatment of diabetes.

Design, implementation, and inferential issues associated with clinical trials that rely on data in electronic medical records: a narrative review

Real world evidence is now accepted by authorities charged with assessing the benefits and harms of new therapies. Clinical trials based on real world evidence are much less expensive than randomized clinical trials that do not rely on "real world evidence" such as contained in electronic health records (EHR). Consequently, we can expect an increase in the number of reports of these types of trials, which we identify here as 'EHR-sourced trials.' 'In this selected literature review, we discuss the various designs and the ethical issues they raise. EHR-sourced trials have the potential to improve/increase common data elements and other aspects of the EHR and related systems. Caution is advised, however, in drawing causal inferences about the relationships among EHR variables. Nevertheless, we anticipate that EHR-CTs will play a central role in answering research and regulatory questions.

Surrogacy of Pathologic Complete Response in Trials of Neoadjuvant Therapy for Early Breast Cancer: Critical Analysis of Strengths, Weaknesses, and Misinterpretations

Importance

The pathologic complete response (pCR) is supported by regulatory agencies as a surrogate end point for long-term patients' clinical outcomes in the accelerated approval process of new drugs tested in neoadjuvant randomized clinical trials (RCTs) for early breast cancer (BC). However, a meaningful association between pCR and patients' survival has been proven only at the patient level (ie, significantly better survival of patients who achieved pCR compared with those who did not), but not at trial level (ie, poor association between degree of improvement in pCR rate and survival reported across trials).

Observations

We critically discuss the potential reasons of such discrepancy between pCR surrogacy value at the patient and trial level, as well as the relevant implications for both clinical research and drug regulatory policy. We also describe alternative surrogate end points, including combined end points that jointly analyzed pathological response and event-free survival data, or the assessment of circulating tumor DNA (ctDNA). Such proposed surrogate end points could overcome limits of pCR and provide a reasonable trade-off between the 2 conflicting needs to have access to effective therapies rapidly, and to reliably assess patients' clinical benefit.

Conclusions and Relevance

Using surrogate end points to grant drug approvals is justified only when they can provide accurate prediction of a drug's effect on the long-term patient outcomes. Evidence currently available does not support pCR used alone as a reliable surrogate end point in regulatory neoadjuvant RCTs for BC. The surrogacy value at trial level of potentially more robust surrogate end points needs to be urgently tested.

Predicting outcomes of phase III oncology trials with Bayesian mediation modeling of tumor response

Pivotal cancer trials often fail to yield evidence in support of new therapies thought to offer promising alternatives to standards-of-care. Conducting randomized controlled trials in oncology tends to be considerably more expensive than studies of other diseases with comparable sample size. Moreover, phase III trial design often takes place with a paucity of survival data for experimental therapies. Experts have explained the failures on the basis of design flaws which produce studies with unrealistic expectations. This article presents a framework for predicting outcomes of phase III oncology trials using Bayesian mediation models. Predictions, which arise from interim analyses, derive from multivariate modeling of the relationships among treatment, tumor response, and their conjoint effects on survival. Acting as a safeguard against inaccurate pre-trial design assumptions, the methodology may better facilitate rapid closure of negative studies. Additionally the models can be used to inform re-estimations of sample size for under-powered trials that demonstrate survival benefit via tumor response mediation. The methods are applied to predict the outcomes of two colorectal cancer studies. Simulation is used to evaluate and compare models in the absence versus presence of reliable surrogate markers of survival.

Using an interim analysis based exclusively on an early outcome in a randomized clinical trial with a long-term clinical endpoint

In RCTs with an interest in a long-term efficacy endpoint, the follow-up time necessary to observe the endpoint may be substantial. In order to reduce the expected duration of such trials, early-outcome data may be collected to enrich an interim analysis aimed at stopping the trial early for efficacy. We propose to extend such a design with an additional interim analysis using solely early-outcome data in order to expedite the evaluation of treatment's efficacy. We evaluate the potential gain in operating characteristics (power, expected trial duration, and expected sample size) when introducing such an early interim analysis, in function of the properties of the early outcome as a surrogate for the long-term endpoint. In the context of a longitudinal age-related macular degeneration (ARMD) ophthalmology trial, results show potentially substantial gains in both the expected trial duration and the expected sample size. A prerequisite, though, is that the treatment effect on the early outcome has to be strongly correlated with the treatment effect on the long-term endpoint, that is, that the early outcome is a validated surrogate for the long-term endpoint.

Combined assessment of early and late-phase outcomes in orphan drug development

In drug development programs, proof‐of‐concept Phase II clinical trials typically have a biomarker as a primary outcome, or an outcome that can be observed with relatively short follow‐up. Subsequently, the Phase III clinical trials aim to demonstrate the treatment effect based on a clinical outcome that often needs a longer follow‐up to be assessed. Early‐phase outcomes or biomarkers are typically associated with late‐phase outcomes and they are often included in Phase III trials. The decision to proceed to Phase III development is based on analysis of the early‐Phase II outcome data. In rare diseases, it is likely that only one Phase II trial and one Phase III trial are available. In such cases and before drug marketing authorization requests, positive results of the early‐phase outcome of Phase II trials are then likely seen as supporting (or even replicating) positive Phase III results on the late‐phase outcome, without a formal retrospective combined assessment and without accounting for between‐study differences. We used double‐regression modeling applied to the Phase II and Phase III results to numerically mimic this informal retrospective assessment. We provide an analytical solution for the bias and mean square error of the overall effect that leads to a corrected double‐regression. We further propose a flexible Bayesian double‐regression approach that minimizes the bias by accounting for between‐study differences via discounting the Phase II early‐phase outcome when they are not in line with the Phase III biomarker outcome results. We illustrate all methods with an orphan drug example for Fabry disease.

Clinical Design and Analysis Strategies for the Development of Gene Therapies: Considerations for Quantitative Drug Development in the Age of Genetic Medicine

Cell and gene therapies have shown enormous promise across a range of diseases in recent years. Numerous adoptive cell therapy modalities as well as systemic and direct-to-target tissue gene transfer administrations are currently in clinical development. The clinical trial design, development, reporting, and analysis of novel cell and gene therapies can differ significantly from established practices for small molecule drugs and biologics. Here, we discuss important quantitative considerations and key competencies for drug developers in preclinical requirements, trial design, and lifecycle planning for gene therapies. We argue that the unique development path of gene therapies requires practicing quantitative drug developers-statisticians, pharmacometricians, pharmacokineticists, epidemiologists, and medical and translational science leads-to exercise active collaboration and cross-functional learning across development stages.

Clinical Trials Impacted by the COVID-19 Pandemic: Adaptive Designs to the Rescue?

Very recently the new pathogen severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was identified and the coronavirus disease 2019 (COVID-19) declared a pandemic by the World Health Organization. The pandemic has a number of consequences for ongoing clinical trials in non-COVID-19 conditions. Motivated by four current clinical trials in a variety of disease areas we illustrate the challenges faced by the pandemic and sketch out possible solutions including adaptive designs. Guidance is provided on (i) where blinded adaptations can help; (ii) how to achieve Type I error rate control, if required; (iii) how to deal with potential treatment effect heterogeneity; (iv) how to use early read-outs; and (v) how to use Bayesian techniques. In more detail approaches to resizing a trial affected by the pandemic are developed including considerations to stop a trial early, the use of group-sequential designs or sample size adjustment. All methods considered are implemented in a freely available R shiny app. Furthermore, regulatory and operational issues including the role of data monitoring committees are discussed.

An adaptive two-arm clinical trial using early endpoints to inform decision making: design for a study of sub-acromial spacers for repair of rotator cuff tendon tears

Background

There is widespread concern across the clinical and research communities that clinical trials, powered for patient-reported outcomes, testing new surgical procedures are often expensive and time-consuming, particularly when the new intervention is shown to be no better than the standard. Conventional (non-adaptive) randomised controlled trials (RCTs) are perceived as being particularly inefficient in this setting. Therefore, we have developed an adaptive group sequential design that allows early endpoints to inform decision making and show, through simulations and a worked example, that these designs are feasible and often preferable to conventional non-adaptive designs. The methodology is motivated by an ongoing clinical trial investigating a saline-filled balloon, inserted above the main joint of the shoulder at the end of arthroscopic debridement, for treatment of tears of rotor cuff tendons. This research question and setting is typical of many studies undertaken to assess new surgical procedures.

Methods

Test statistics are presented based on the setting of two early outcomes, and methods for estimation of sequential stopping boundaries are described. A framework for the implementation of simulations to evaluate design characteristics is also described.

Results

Simulations show that designs with one, two and three early looks are feasible and, with appropriately chosen futility stopping boundaries, have appealing design characteristics. A number of possible design options are described that have good power and a high probability of stopping for futility if there is no evidence of a treatment effect at early looks. A worked example, with code in R, provides a practical demonstration of how the design might work in a real study.

Conclusions

In summary, we show that adaptive designs are feasible and could work in practice. We describe the operating characteristics of the designs and provide guidelines for appropriate values for the stopping boundaries for the START:REACTS (Sub-acromial spacer for Tears Affecting Rotator cuff Tendons: a Randomised, Efficient, Adaptive Clinical Trial in Surgery) study.

Trial registration

ISRCTN Registry, ISRCTN17825590. Registered on 5 March 2018.

Evaluating futility of a binary clinical endpoint using early read-outs

Interim analyses are routinely used to monitor accumulating data in clinical trials. When the objective of the interim analysis is to stop the trial if the trial is deemed futile, it must ideally be conducted as early as possible. In trials where the clinical endpoint of interest is only observed after a long follow-up, many enrolled patients may therefore have no information on the primary endpoint available at the time of the interim analysis. To facilitate earlier decision-making, one may incorporate early response data that are predictive for the primary endpoint (eg, an assessment of the primary endpoint at an earlier time) in the interim analysis. Most attention so far has been given to the development of interim test statistics that include such short-term endpoints, but not to decision procedures. Existing tests moreover perform poorly when the information is scarce, eg, due to rare events, when the cohort of patients with observed primary endpoint data is small, or when the short-term endpoint is a strong but imperfect predictor. In view of this, we develop an interim decision procedure based on the conditional power approach that utilizes the short-term and long-term binary endpoints in a framework that is expected to provide reliable inferences, even when the primary endpoint is only available for a few patients, and has the added advantage that it allows the use of historical information. The operational characteristics of the proposed procedure are evaluated for the phase III clinical trial that motivated this approach, using simulation studies.

Interim analysis of binary outcome data in clinical trials: a comparison of five estimators

In clinical trials, where the outcome of interest is the occurrence of an event over a fixed time period, estimation of the event proportion at interim analysis can form a basis for decision-making such as early trial termination, sample size re-estimation, and/or dropping inferior treatment arms. In addition to derivation of mean squared error under an exponential time-to-event distribution, we performed a simulation study to examine the performance of five estimators of the event proportion when time to the event is assessable. The simulation results showed advantages of the Kaplan-Meier estimator over others in terms of robustness, and the bias and variability of the event proportion estimate. An example was given to illustrate how the estimators affect dropping treatment arms in a multi-arm multi-stage adaptive trial. We recommended the use of the Kaplan-Meier estimator and discourage the use of other estimators that discard the inherent time-to-event information.

Predictive probability methods for interim monitoring in clinical trials with longitudinal outcomes

In clinical research and development, interim monitoring is critical for better decision-making and minimizing the risk of exposing patients to possible ineffective therapies. For interim futility or efficacy monitoring, predictive probability methods are widely adopted in practice. Those methods have been well studied for univariate variables. However, for longitudinal studies, predictive probability methods using univariate information from only completers may not be most efficient, and data from on-going subjects can be utilized to improve efficiency. On the other hand, leveraging information from on-going subjects could allow an interim analysis to be potentially conducted once a sufficient number of subjects reach an earlier time point. For longitudinal outcomes, we derive closed-form formulas for predictive probabilities, including Bayesian predictive probability, predictive power, and conditional power and also give closed-form solutions for predictive probability of success in a future trial and the predictive probability of success of the best dose. When predictive probabilities are used for interim monitoring, we study their distributions and discuss their analytical cutoff values or stopping boundaries that have desired operating characteristics. We show that predictive probabilities utilizing all longitudinal information are more efficient for interim monitoring than that using information from completers only. To illustrate their practical application for longitudinal data, we analyze 2 real data examples from clinical trials.

Blinded versus unblinded estimation of a correlation coefficient to inform interim design adaptations

Regulatory authorities require that the sample size of a confirmatory trial is calculated prior to the start of the trial. However, the sample size quite often depends on parameters that might not be known in advance of the study. Misspecification of these parameters can lead to under- or overestimation of the sample size. Both situations are unfavourable as the first one decreases the power and the latter one leads to a waste of resources. Hence, designs have been suggested that allow a re-assessment of the sample size in an ongoing trial. These methods usually focus on estimating the variance. However, for some methods the performance depends not only on the variance but also on the correlation between measurements. We develop and compare different methods for blinded estimation of the correlation coefficient that are less likely to introduce operational bias when the blinding is maintained. Their performance with respect to bias and standard error is compared to the unblinded estimator. We simulated two different settings: one assuming that all group means are the same and one assuming that different groups have different means. Simulation results show that the naïve (one-sample) estimator is only slightly biased and has a standard error comparable to that of the unblinded estimator. However, if the group means differ, other estimators have better performance depending on the sample size per group and the number of groups.

Extrapolation of efficacy and other data to support the development of new medicines for children: A systematic review of methods

OBJECTIVE

When developing new medicines for children, the potential to extrapolate from adult data to reduce the experimental burden in children is well recognised. However, significant assumptions about the similarity of adults and children are needed for extrapolations to be biologically plausible. We reviewed the literature to identify statistical methods that could be used to optimise extrapolations in paediatric drug development programmes.

METHODS

Web of Science was used to identify papers proposing methods relevant for using data from a 'source population' to support inferences for a 'target population'. Four key areas of methods development were targeted: paediatric clinical trials, trials extrapolating efficacy across ethnic groups or geographic regions, the use of historical data in contemporary clinical trials and using short-term endpoints to support inferences about long-term outcomes.

RESULTS

Searches identified 626 papers of which 52 met our inclusion criteria. From these we identified 102 methods comprising 58 Bayesian and 44 frequentist approaches. Most Bayesian methods (n = 54) sought to use existing data in the source population to create an informative prior distribution for a future clinical trial. Of these, 46 allowed the source data to be down-weighted to account for potential differences between populations. Bayesian and frequentist versions of methods were found for assessing whether key parameters of source and target populations are commensurate (n = 34). Fourteen frequentist methods synthesised data from different populations using a joint model or a weighted test statistic.

CONCLUSIONS

Several methods were identified as potentially applicable to paediatric drug development. Methods which can accommodate a heterogeneous target population and which allow data from a source population to be down-weighted are preferred. Methods assessing the commensurability of parameters may be used to determine whether it is appropriate to pool data across age groups to estimate treatment effects.

A comparison of methods for treatment selection in seamless phase II/III clinical trials incorporating information on short-term endpoints

In an adaptive seamless phase II/III clinical trial interim analysis, data are used for treatment selection, enabling resources to be focused on comparison of more effective treatment(s) with a control. In this paper, we compare two methods recently proposed to enable use of short-term endpoint data for decision-making at the interim analysis. The comparison focuses on the power and the probability of correctly identifying the most promising treatment. We show that the choice of method depends on how well short-term data predict the best treatment, which may be measured by the correlation between treatment effects on short- and long-term endpoints.

Data-driven treatment selection for seamless phase II/III trials incorporating early-outcome data

Seamless phase II/III clinical trials are conducted in two stages with treatment selection at the first stage. In the first stage, patients are randomized to a control or one of k > 1 experimental treatments. At the end of this stage, interim data are analysed, and a decision is made concerning which experimental treatment should continue to the second stage. If the primary endpoint is observable only after some period of follow-up, at the interim analysis data may be available on some early outcome on a larger number of patients than those for whom the primary endpoint is available. These early endpoint data can thus be used for treatment selection. For two previously proposed approaches, the power has been shown to be greater for one or other method depending on the true treatment effects and correlations. We propose a new approach that builds on the previously proposed approaches and uses data available at the interim analysis to estimate these parameters and then, on the basis of these estimates, chooses the treatment selection method with the highest probability of correctly selecting the most effective treatment. This method is shown to perform well compared with the two previously described methods for a wide range of true parameter values. In most cases, the performance of the new method is either similar to or, in some cases, better than either of the two previously proposed methods.

Dose-Response Determination in Multistage Endpoint Clinical Trials

Improper dose selection remains one of the key drivers of the large attrition rates observed in confirmatory studies in clinical drug development. Many factors contribute to this problem, such as insufficient resources allocated to dose-ranging studies and the use of statistical methods better suited for phase 3 studies than for dose selection. This paper describes a model-based dose-finding method that leverages all longitudinal data collected in the trial to estimate the dose-response relationship at any desired visit, using it to estimate target doses of interest, such as the minimum dose producing a desired clinical benefit. The approach uses a Markov chain model to account for correlation in the repeated measures obtained on the same patient. An actual phase 2 study and simulations are used to illustrate the methodology.

Advancing clinical trial design in pulmonary hypertension

In pulmonary hypertension, as in many other diseases, there is a need for a smarter approach to evaluating new treatments. The traditional randomized controlled trial has served medical science well, but constrains the development of treatments for rare diseases. A workshop was established to consider alternative clinical trial designs in pulmonary hypertension and here discusses their merits, limitations and challenges to implementation of novel approaches.

A confirmatory seamless phase II/III clinical trial design incorporating short-term endpoint information

Seamless phase II/III designs allow strong control of the familywise type I error rate when the most promising of a number of experimental treatments is selected at an interim analysis to continue along with the control treatment. If the primary endpoint is observed only after long-term follow-up it may be desirable to use correlated short-term endpoint data available at the interim analysis to inform the treatment selection. If short-term data are available for some patients for whom the primary endpoint is not available, basing treatment selection on these data may, however, lead to inflation of the type I error rate. This paper proposes a method for the adjustment of the usual group-sequential boundaries to maintain strong control of the familywise error rate even when short-term endpoint data are used for the treatment selection at the first interim analysis. This method allows the use of the short-term data, leading to an increase in power when these data are correlated with the primary endpoint data.

Sample Size Recalculation in Internal Pilot Study Designs: A Review

The adequacy of sample size is important to clinical trials. In the planning phase of a trial, however, the investigators are often quite uncertain about the sizes of parameters which are needed for sample size calculations. A solution to this problem is mid-course recalculation of the sample size during the ongoing trial. In internal pilot study designs, nuisance parameters are estimated on the basis of interim data and the sample size is adjusted accordingly. This review attempts to give an overview on the available methods. It is written not only for biometricians who are already familar with the the topic and wish to update their knowledge but also for users new to the subject.

Group sequential clinical trials for longitudinal data with analyses using summary statistics

Longitudinal endpoints are used in clinical trials, and the analysis of the results is often conducted using within-individual summary statistics. When these trials are monitored, interim analyses that include subjects with incomplete follow-up can give incorrect decisions due to bias by non-linearity in the true time trajectory of the treatment effect. Linear mixed-effects models can be used to remove this bias, but there is a lack of software to support both the design and implementation of monitoring plans in this setting. This paper considers a clinical trial in which the measurement time schedule is fixed (at least for pre-trial design), and the scientific question is parameterized by a contrast across these measurement times. This setting assures generalizable inference in the presence of non-linear time trajectories. The distribution of the treatment effect estimate at the interim analyses using the longitudinal outcome measurements is given, and software to calculate the amount of information at each interim analysis is provided. The interim information specifies the analysis timing thereby allowing standard group sequential design software packages to be used for trials with longitudinal outcomes. The practical issues with implementation of these designs are described; in particular, methods are presented for consistent estimation of treatment effects at the interim analyses when outcomes are not measured according to the pre-trial schedule. Splus/R functions implementing this inference using appropriate linear mixed-effects models are provided. These designs are illustrated using a clinical trial of statin treatment for the symptoms of peripheral arterial disease.