"Super-covariates": Using predicted control group outcome as a covariate in randomized clinical trials

The power of randomized controlled clinical trials to demonstrate the efficacy of a drug compared with a control group depends not just on how efficacious the drug is, but also on the variation in patients' outcomes. Adjusting for prognostic covariates during trial analysis can reduce this variation. For this reason, the primary statistical analysis of a clinical trial is often based on regression models that besides terms for treatment and some further terms (e.g., stratification factors used in the randomization scheme of the trial) also includes a baseline (pre-treatment) assessment of the primary outcome. We suggest to include a "super-covariate"-that is, a patient-specific prediction of the control group outcome-as a further covariate (but not as an offset). We train a prognostic model or ensembles of such models on the individual patient (or aggregate) data of other studies in similar patients, but not the new trial under analysis. This has the potential to use historical data to increase the power of clinical trials and avoids the concern of type I error inflation with Bayesian approaches, but in contrast to them has a greater benefit for larger sample sizes. It is important for prognostic models behind "super-covariates" to generalize well across different patient populations in order to similarly reduce unexplained variability whether the trial(s) to develop the model are identical to the new trial or not. In an example in neovascular age-related macular degeneration we saw efficiency gains from the use of a "super-covariate".

Improving the assessment of the probability of success in late stage drug development

There are several steps to confirming the safety and efficacy of a new medicine. A sequence of trials, each with its own objectives, is usually required. Quantitative risk metrics can be useful for informing decisions about whether a medicine should transition from one stage of development to the next. To obtain an estimate of the probability of regulatory approval, pharmaceutical companies may start with industry-wide success rates and then apply to these subjective adjustments to reflect program-specific information. However, this approach lacks transparency and fails to make full use of data from previous clinical trials. We describe a quantitative Bayesian approach for calculating the probability of success (PoS) at the end of phase II which incorporates internal clinical data from one or more phase IIb studies, industry-wide success rates, and expert opinion or external data if needed. Using an example, we illustrate how PoS can be calculated accounting for differences between the phase II data and future phase III trials, and discuss how the methods can be extended to accommodate accelerated drug development pathways.

A New Comprehensive Approach to Assess the Probability of Success of Development Programs Before Pivotal Trials

The point at which clinical development programs transition from early phase to pivotal trials is a critical milestone. Substantial uncertainty about the outcome of pivotal trials may remain even after seeing positive early phase data, and companies may need to make difficult prioritization decisions for their portfolio. The probability of success (PoS) of a program, a single number expressed as a percentage reflecting the multitude of risks that may influence the final program outcome, is a key decision-making tool. Despite its importance, companies often rely on crude industry benchmarks that may be "adjusted" by experts based on undocumented criteria and which are typically misaligned with the definition of success used to drive commercial forecasts, leading to overly optimistic expected net present value calculations. We developed a new framework to assess the PoS of a program before pivotal trials begin. Our definition of success encompasses the successful outcome of pivotal trials, regulatory approval and meeting the requirements for market access as outlined in the target product profile. The proposed approach is organized in four steps and uses an innovative Bayesian approach to synthesize all relevant evidence. The new PoS framework is systematic and transparent. It will help organizations to make more informed decisions. In this paper, we outline the rationale and elaborate on the structure of the proposed framework, provide examples, and discuss the benefits and challenges associated with its adoption.

Predicting drug approvals: The Novartis data science and artificial intelligence challenge

We describe a novel collaboration between academia and industry, an in-house data science and artificial intelligence challenge held by Novartis to develop machine-learning models for predicting drug-development outcomes, building upon research at MIT using data from Informa as the starting point. With over 50 cross-functional teams from 25 Novartis offices around the world participating in the challenge, the domain expertise of these Novartis researchers was leveraged to create predictive models with greater sophistication. Ultimately, two winning teams developed models that outperformed the baseline MIT model-areas under the curve of 0.88 and 0.84 versus 0.78, respectively-through state-of-the-art machine-learning algorithms and the use of newly incorporated features and data. In addition to validating the variables shown to be associated with drug approval in the earlier MIT study, the challenge also provided new insights into the drivers of drug-development success and failure.

Maximizing Adherence and Gaining New Information For Your Chronic Obstructive Pulmonary Disease (MAGNIFY COPD): Study Protocol for the Pragmatic, Cluster Randomized Trial Evaluating the Impact of Dual Bronchodilator with Add-On Sensor and Electronic Monitoring on Clinical Outcomes

Background

Poor treatment adherence in COPD patients is associated with poor clinical outcomes and increased healthcare burden. Personalized approaches for adherence management, supported with technology-based interventions, may offer benefits to patients and providers but are currently unproven in terms of clinical outcomes as opposed to adherence outcomes.

Methods

Maximizing Adherence and Gaining New Information For Your COPD (MAGNIFY COPD study), a pragmatic cluster randomized trial, aims to evaluate the impact of an adherence technology package (interventional package), comprising an adherence review, ongoing provision of a dual bronchodilator but with an add-on inhaler sensor device and a connected mobile application. This will compare time to treatment failure and other clinical outcomes in patients identified at high risk of exacerbations with historic poor treatment adherence as measured by prescription collection to mono/dual therapy over one year (1312 patients) versus usual care. Treatment failure is defined as the first occurrence of one of the following: (1) moderate/severe COPD exacerbation, (2) prescription of triple therapy (inhaled corticosteroid/long-acting β₂-agonist/long-acting muscarinic antagonist [ICS/LABA/LAMA]), (3) prescription of additional chronic therapy for COPD, or (4) respiratory-related death. Adherence, moderate/severe exacerbations, respiratory-related healthcare resource utilization and costs, and intervention package acceptance rate will also be assessed. Eligible primary care practices (N=176) participating in the Optimum Patient Care Quality Improvement Program will be randomized (1:1) to either adherence support cluster arm (suitable patients already receiving or initiated Ultibro^® Breezhaler^® [indacaterol/glycopyrronium] will be offered interventional package) or the control cluster arm (suitable patients continue to receive usual clinical care). Patients will be identified and outcomes collected from anonymized electronic medical records within the Optimum Patient Care Research Database. On study completion, electronic medical record data will be re-extracted to analyze outcomes in both study groups.

Registration Number

ISRCTN10567920.

Conclusion

MAGNIFY will explore patient benefits of technology-based interventions for electronic adherence monitoring.

Predicting Hospitalization Due to COPD Exacerbations in Swedish Primary Care Patients Using Machine Learning - Based on the ARCTIC Study

Purpose

Chronic obstructive pulmonary disease (COPD) exacerbations can negatively impact disease severity, progression, mortality and lead to hospitalizations. We aimed to develop a model that predicts a patient's risk of hospitalization due to severe exacerbations (defined as COPD-related hospitalizations) of COPD, using Swedish patient level data.

Patients and Methods

Patient level data for 7823 Swedish patients with COPD was collected from electronic medical records (EMRs) and national registries covering healthcare contacts, diagnoses, prescriptions, lab tests, hospitalizations and socioeconomic factors between 2000 and 2013. Models were created using machine-learning methods to predict risk of imminent exacerbation causing patient hospitalization due to COPD within the next 10 days. Exacerbations occurring within this period were considered as one event. Model performance was assessed using the Area under the Precision-Recall Curve (AUPRC). To compare performance with previous similar studies, the Area Under Receiver Operating Curve (AUROC) was also reported. The model with the highest mean cross validation AUPRC was selected as the final model and was in a final step trained on the entire training dataset.

Results

The most important factors for predicting severe exacerbations were exacerbations in the previous six months and in whole history, number of COPD-related healthcare contacts and comorbidity burden. Validation on test data yielded an AUROC of 0.86 and AUPRC of 0.08, which was high in comparison to previously published attempts to predict COPD exacerbation.

Conclusion

Our work suggests that clinically available information on patient history collected via automated retrieval from EMRs and national registries or directly during patient consultation can form the basis for future clinical tools to predict risk of severe COPD exacerbations.

Cardiovascular outcomes at recommended blood pressure targets in middle-aged and elderly patients with type 2 diabetes mellitus compared to all middle-aged and elderly hypertensive study patients with high cardiovascular risk

PURPOSE

Event-based clinical outcome trials have shown limited evidence to support guidelines recommendations to lower blood pressure (BP) to <130/80 mmHg in middle-aged and elderly hypertensive patients with diabetes mellitus or with general high cardiovascular (CV) risk. We addressed this issue by post-hoc analysing the risk of CV events in patients who participated in the Valsartan Antihypertensive Long-term Use Evaluation (VALUE) trial and compared the hypertensive patients with type 2 diabetes mellitus with all high-risk hypertensive patients.

MATERIALS AND METHODS

Patients were divided into 4 groups according to the proportion of on-treatment visits before the occurrence of an event (<25% to ≥75%) in which BP was reduced to <140/90 or <130/80 mmHg. Patients with diabetes mellitus (n = 5250) were compared with the entire VALUE population with high CV risk (n = 15,245).

RESULTS

After adjustments for baseline differences between groups, a reduction in the proportion of visits in which BP was reduced to <140/90 mmHg, but not to <130/80 mmHg, was accompanied by a progressive increase in the risk of CV morbidity and mortality as well as stroke, myocardial infarction and heart failure in both diabetes mellitus and in all high-risk patients. Target BP <130/80 mmHg reduced stroke risk in the main population but not in the diabetes mellitus patients. Patients with diabetes mellitus had higher event rates for the primary cardiac endpoint and all-cause mortality driven by a higher rate of heart failure.

CONCLUSION

In the high-risk hypertensive patients of the VALUE trial achieving more frequently BP <140/90 mmHg, but not <130/80 mmHg, showed principally the same protective effect on overall and cause-specific cardiovascular outcomes in patients with diabetes mellitus and in the general high-risk hypertensive population.

Cardiovascular outcomes at recommended blood pressure targets in middle-aged and elderly patients with type 2 diabetes mellitus and hypertension

PURPOSE

Available data of event-based clinical outcomes trials show that little evidence supports the guidelines recommendations to lower blood pressure (BP) to <130/80 mmHg in middle-aged and elderly people with type 2 diabetes mellitus and hypertension. We addressed this issue by post-hoc analysing the risk of cardiovascular (CV) events in mostly elderly high-risk hypertensive patients with type 2 diabetes mellitus participating in the Valsartan Antihypertensive Long-term Use Evaluation (VALUE) trial.

MATERIAL AND METHODS

Patients (n = 5250) were divided into 4 groups according to the proportion of on-treatment visits before the occurrence of an event (<25% to ≥ 75%) in which BP was reduced to <140/90 or <130/80 mmHg.

RESULTS

After adjustment for baseline demographic differences between groups, a reduction in the proportion of visits in which BP achieved <140/90 mmHg accompanied a progressive increase in the risk of CV mortality and morbidity as well as of cause-specific events such as stroke, myocardial infarction and heart failure. A progressive reduction in the proportion of visits in which BP was reduced <130/80 mmHg did not have any effect on CV risks.

CONCLUSION

In mostly elderly high-risk hypertensive patients with type 2 diabetes mellitus participating in the VALUE trial, achieving more frequently BP <140/90 mmHg showed a marked protective effect on overall and all cause-specific cardiovascular outcomes. This was not the case for a more frequent achievement of the more intensive BP target, i.e. <130/80 mmHg.

Evidence synthesis from aggregate recurrent event data for clinical trial design and analysis

Information from historical trials is important for the design, interim monitoring, analysis, and interpretation of clinical trials. Meta-analytic models can be used to synthesize the evidence from historical data, which are often only available in aggregate form. We consider evidence synthesis methods for trials with recurrent event endpoints, which are common in many therapeutic areas. Such endpoints are typically analyzed by negative binomial regression. However, the individual patient data necessary to fit such a model are usually unavailable for historical trials reported in the medical literature. We describe approaches for back-calculating model parameter estimates and their standard errors from available summary statistics with various techniques, including approximate Bayesian computation. We propose to use a quadratic approximation to the log-likelihood for each historical trial based on 2 independent terms for the log mean rate and the log of the dispersion parameter. A Bayesian hierarchical meta-analysis model then provides the posterior predictive distribution for these parameters. Simulations show this approach with back-calculated parameter estimates results in very similar inference as using parameter estimates from individual patient data as an input. We illustrate how to design and analyze a new randomized placebo-controlled exacerbation trial in severe eosinophilic asthma using data from 11 historical trials.

Fevipiprant, an oral prostaglandin DP2 receptor (CRTh2) antagonist, in allergic asthma uncontrolled on low-dose inhaled corticosteroids

Dose-related efficacy and safety of fevipiprant (QAW039), an oral DP₂ (CRTh2) receptor antagonist, was assessed in patients with allergic asthma uncontrolled by low-dose inhaled corticosteroids (ICS).Adult patients were randomised to 12 weeks' treatment with once-daily (1, 3, 10, 30, 50, 75, 150, 300 or 450 mg q.d) or twice-daily (2, 25, 75 or 150 mg b.i.d) fevipiprant (n=782), montelukast 10 mg q.d (n=139) or placebo (n=137). All patients received inhaled budesonide 200 μg b.i.dFevipiprant produced a statistically significant improvement in the primary end-point of change in pre-dose forced expiratory volume in 1 s at week 12 (p=0.0035) with a maximum model-averaged difference to placebo of 0.112 L. The most favourable pairwise comparisons to placebo were for the fevipiprant 150 mg q.d and 75 mg b.i.d groups, with no clinically meaningful differences between q.d and b.i.d Montelukast also demonstrated a significant improvement in this end-point. No impact on other efficacy end-points was observed. Adverse events were generally mild/moderate in severity, and were evenly distributed across doses and treatments.Fevipiprant appears to be efficacious and well-tolerated in this patient population, with an optimum total daily dose of 150 mg. Further investigations into the clinical role of fevipiprant in suitably designed phase III clinical trials are warranted.

Effect of valsartan on kidney outcomes in people with impaired glucose tolerance

AIMS

To examine the effect of valsartan on kidney outcomes in patients with impaired glucose tolerance (IGT).

METHODS

In a double-blind randomized trial, 9306 patients with IGT were assigned to valsartan (160 mg daily) or placebo. The co-primary endpoints were the development of diabetes and two composite cardiovascular outcomes. Prespecified renal endpoints included: the composite of renal death, end-stage renal disease (ESRD) or doubling of serum creatinine; estimated glomerular filtration rate (eGFR) ≤30 mL/min/1.73 m² ; hospitalization for renal failure; and progression from normoalbuminuria to microalbuminuria, microalbuminuria to macroalbuminuria, and normoalbuminuria to macroalbuminuria. The median follow-up was 6.2 years.

RESULTS

Valsartan reduced the incidence of diabetes but not cardiovascular events. In the valsartan group, 25/4631 patients (0.5%), vs 26/4675 (0.6%) patients in the placebo group, developed ESRD or experienced doubling of serum creatinine (hazard ratio [HR] 0.96, 95% confidence interval [CI] 0.55-1.66; P  = .87). Few patients in either group developed an eGFR of ≤30 mL/min/1.73 m² or had a renal hospitalization. Fewer patients on valsartan (237/4084 [5.8%]) than on placebo (342/4092 [8.4%]) developed microalbuminuria (HR 0.68, 95% CI 0.57-0.80; P  < .0001), and fewer valsartan-treated patients developed macroalbuminuria. Overall, urinary albumin-to-creatinine ratio (UACR) was 11% lower with valsartan (95% CI 8-13; P  < .0001) and 9% lower (95% CI 6-11; P  < .0001) after adjusting for both glucose and blood pressure.

CONCLUSIONS

The effect of valsartan on UACR was not wholly explained by change in blood pressure or glucose. Valsartan reduced the incidence of microalbuminuria in IGT without increasing the incidence of hyperkalaemia or renal dysfunction compared with placebo.

Predictors of cardiac morbidity in diabetic, new-onset diabetic and non-diabetic high-risk hypertensive patients: The Valsartan Antihypertensive Long-term Use Evaluation (VALUE) trial

Diabetic and new-onset diabetic patients with hypertension have higher cardiac morbidity than patients without diabetes. We aimed to investigate whether baseline predictors of cardiac morbidity, the major constituent of the primary endpoint in the Valsartan Antihypertensive Long-term Use Evaluation (VALUE) trial, were different in patients with diabetes and new-onset diabetes compared to patients without diabetes. In total, 15,245 high-risk hypertensive patients in the VALUE trial were followed for an average of 4.2 years. At baseline, 5250 patients were diabetic by the 1999 World Health Organization criteria, 1298 patients developed new-onset diabetes and 8697 patients stayed non-diabetic during follow-up. Cardiac morbidity was defined as a composite of myocardial infarction and heart failure requiring hospitalization, and baseline predictors were identified by univariate and multivariate stepwise Cox regression analyses. History of coronary heart disease (CHD) and age were the most important predictors of cardiac morbidity in both diabetic and non-diabetic patients. History of CHD, history of stroke and age were the only significant predictors of cardiac morbidity in patients with new-onset diabetes. Predictors of cardiac morbidity, in particular history of CHD and age, were essentially the same in high-risk hypertensive patients with diabetes, new-onset diabetes and without diabetes who participated in the VALUE trial.

Choice of estimand and analysis methods in diabetes trials with rescue medication

The analysis of clinical trials aiming to show symptomatic benefits is often complicated by the ethical requirement for rescue medication when the disease state of patients worsens. In type 2 diabetes trials, patients receive glucose-lowering rescue medications continuously for the remaining trial duration, if one of several markers of glycemic control exceeds pre-specified thresholds. This may mask differences in glycemic values between treatment groups, because it will occur more frequently in less effective treatment groups. Traditionally, the last pre-rescue medication value was carried forward and analyzed as the end-of-trial value. The deficits of such simplistic single imputation approaches are increasingly recognized by regulatory authorities and trialists. We discuss alternative approaches and evaluate them through a simulation study. When the estimand of interest is the effect attributable to the treatments initially assigned at randomization, then our recommendation for estimation and hypothesis testing is to treat data after meeting rescue criteria as deterministically 'missing' at random, because initiation of rescue medication is determined by observed in-trial values. An appropriate imputation of values after meeting rescue criteria is then possible either directly through multiple imputation or implicitly with a repeated measures model. Crucially, one needs to jointly impute or model all markers of glycemic control that can lead to the initiation of rescue medication. An alternative for hypothesis testing only are rank tests with outcomes from patients 'requiring rescue medication' ranked worst, and non-rescued patients ranked according to final visit values. However, an appropriate ranking of not observed values may be controversial.

Heart rate as a predictor of stroke in high-risk, hypertensive patients with previous stroke or transient ischemic attack

BACKGROUND

Risk factors for first stroke are well established, but less is known about risk factors for recurrent stroke. In the present analysis, we aimed to assess the effect of heart rate and other possible predictors of stroke in a hypertensive population with previous stroke or transient ischemic attack (TIA).

METHODS

The Valsartan Antihypertensive Long-Term Use Evaluation trial was a multicentre, double-masked, randomized controlled, parallel group trial comparing the effects of an angiotensin receptor blocker (valsartan) and a calcium channel blocker (amlodipine) in patients with hypertension and high cardiovascular risk. We used Cox proportional hazard models to investigate the effect of baseline variables on the risk of stroke. Quadratic terms of the continuous variables were entered in the models to test for linearity.

RESULTS

Of 15,245 patients included in the trial, 3014 had a previous stroke or TIA at baseline and were included in the present analysis. Stroke recurrence occurred in 239 patients (7.9%) during a median of 4.5 years of follow-up. Resting heart rate (per 10 beats per minute; hazard ratio [HR], 2.78; 95% confidence interval [CI], 1.18-6.58) and diabetes mellitus at baseline (HR, 1.47; 95% CI, 1.03-2.10) were significantly associated with an increased risk of stroke recurrence in the multivariable analysis.

CONCLUSIONS

In high-risk, hypertensive patients with previous stroke or TIA, resting heart rate was the strongest predictor of recurrent stroke.

Effect of valsartan on the incidence of diabetes and cardiovascular events

BACKGROUND

It is not known whether drugs that block the renin-angiotensin system reduce the risk of diabetes and cardiovascular events in patients with impaired glucose tolerance.

METHODS

In this double-blind, randomized clinical trial with a 2-by-2 factorial design, we assigned 9306 patients with impaired glucose tolerance and established cardiovascular disease or cardiovascular risk factors to receive valsartan (up to 160 mg daily) or placebo (and nateglinide or placebo) in addition to lifestyle modification. We then followed the patients for a median of 5.0 years for the development of diabetes (6.5 years for vital status). We studied the effects of valsartan on the occurrence of three coprimary outcomes: the development of diabetes; an extended composite outcome of death from cardiovascular causes, nonfatal myocardial infarction, nonfatal stroke, hospitalization for heart failure, arterial revascularization, or hospitalization for unstable angina; and a core composite outcome that excluded unstable angina and revascularization.

RESULTS

The cumulative incidence of diabetes was 33.1% in the valsartan group, as compared with 36.8% in the placebo group (hazard ratio in the valsartan group, 0.86; 95% confidence interval [CI], 0.80 to 0.92; P<0.001). Valsartan, as compared with placebo, did not significantly reduce the incidence of either the extended cardiovascular outcome (14.5% vs. 14.8%; hazard ratio, 0.96; 95% CI, 0.86 to 1.07; P=0.43) or the core cardiovascular outcome (8.1% vs. 8.1%; hazard ratio, 0.99; 95% CI, 0.86 to 1.14; P=0.85).

CONCLUSIONS

Among patients with impaired glucose tolerance and cardiovascular disease or risk factors, the use of valsartan for 5 years, along with lifestyle modification, led to a relative reduction of 14% in the incidence of diabetes but did not reduce the rate of cardiovascular events. (ClinicalTrials.gov number, NCT00097786.)

Effect of nateglinide on the incidence of diabetes and cardiovascular events

BACKGROUND

The ability of short-acting insulin secretagogues to reduce the risk of diabetes or cardiovascular events in people with impaired glucose tolerance is unknown.

METHODS

In a double-blind, randomized clinical trial, we assigned 9306 participants with impaired glucose tolerance and either cardiovascular disease or cardiovascular risk factors to receive nateglinide (up to 60 mg three times daily) or placebo, in a 2-by-2 factorial design with valsartan or placebo, in addition to participation in a lifestyle modification program. We followed the participants for a median of 5.0 years for incident diabetes (and a median of 6.5 years for vital status). We evaluated the effect of nateglinide on the occurrence of three coprimary outcomes: the development of diabetes; a core cardiovascular outcome that was a composite of death from cardiovascular causes, nonfatal myocardial infarction, nonfatal stroke, or hospitalization for heart failure; and an extended cardiovascular outcome that was a composite of the individual components of the core composite cardiovascular outcome, hospitalization for unstable angina, or arterial revascularization.

RESULTS

After adjustment for multiple testing, nateglinide, as compared with placebo, did not significantly reduce the cumulative incidence of diabetes (36% and 34%, respectively; hazard ratio, 1.07; 95% confidence interval [CI], 1.00 to 1.15; P=0.05), the core composite cardiovascular outcome (7.9% and 8.3%, respectively; hazard ratio, 0.94, 95% CI, 0.82 to 1.09; P=0.43), or the extended composite cardiovascular outcome (14.2% and 15.2%, respectively; hazard ratio, 0.93, 95% CI, 0.83 to 1.03; P=0.16). Nateglinide did, however, increase the risk of hypoglycemia.

CONCLUSIONS

Among persons with impaired glucose tolerance and established cardiovascular disease or cardiovascular risk factors, assignment to nateglinide for 5 years did not reduce the incidence of diabetes or the coprimary composite cardiovascular outcomes. (ClinicalTrials.gov number, NCT00097786.)