The impact of high-risk patients on the results of clinical trials

Re-Analyses of 8 Historical Trials in Cardiovascular Medicine Assessing Multimorbidity Burden and Its Association with Treatment Response

OBJECTIVE

The purpose of this study was to examine the multimorbidity burden of clinical trial participants and assess its association with treatment response.

METHODS

We conducted a reanalysis of patient level data. There were 29,954 participants from 8 clinical trials containing 11 comparisons between an intervention and control condition. Patients were classified by Charlson Comorbidity Index (CCI) score. The primary outcomes were the primary study endpoints as originally specified for each trial. A Cox model that included the CCI score groups, the randomized group, and their interaction, was used to compare the primary outcome between randomized groups. The interaction term between randomized group and comorbidity index allowed the treatment effect to differ by level of comorbidity index and comprised the primary effect of interest. Hazard ratios and risk differences were reported for all comparisons.

RESULTS

The mean CCI scores of trial populations ranged from 2.1 to 3.9 points, and the percentage of patients with scores ≥5 from 3% to 39%. Tests of interaction terms in models yielded P values ≤ .10 for 4/11 comparisons and ≤ .05 for 2/11 comparisons. In 3 additional comparisons, potentially important treatment variation on an absolute scale was observed despite interaction tests with P values > .10 on the relative scale.

CONCLUSIONS

These trials were mainly composed of patient populations with CCI scores ≤4. Despite this, biologically plausible treatment interactions were commonly suggested. These results are hypothesis generating; confirmation of results would require larger studies or studies targeted specifically toward patients with higher levels of multimorbidity.

Comparison of causal forest and regression-based approaches to evaluate treatment effect heterogeneity: an application for type 2 diabetes precision medicine

OBJECTIVE

Precision medicine requires reliable identification of variation in patient-level outcomes with different available treatments, often termed treatment effect heterogeneity. We aimed to evaluate the comparative utility of individualized treatment selection strategies based on predicted individual-level treatment effects from a causal forest machine learning algorithm and a penalized regression model.

METHODS

Cohort study characterizing individual-level glucose-lowering response (6 month reduction in HbA1c) in people with type 2 diabetes initiating SGLT2-inhibitor or DPP4-inhibitor therapy. Model development set comprised 1,428 participants in the CANTATA-D and CANTATA-D2 randomised clinical trials of SGLT2-inhibitors versus DPP4-inhibitors. For external validation, calibration of observed versus predicted differences in HbA1c in patient strata defined by size of predicted HbA1c benefit was evaluated in 18,741 patients in UK primary care (Clinical Practice Research Datalink).

RESULTS

Heterogeneity in treatment effects was detected in clinical trial participants with both approaches (proportion predicted to have a benefit on SGLT2-inhibitor therapy over DPP4-inhibitor therapy: causal forest: 98.6%; penalized regression: 81.7%). In validation, calibration was good with penalized regression but sub-optimal with causal forest. A strata with an HbA1c benefit > 10 mmol/mol with SGLT2-inhibitors (3.7% of patients, observed benefit 11.0 mmol/mol [95%CI 8.0-14.0]) was identified using penalized regression but not causal forest, and a much larger strata with an HbA1c benefit 5-10 mmol with SGLT2-inhibitors was identified with penalized regression (regression: 20.9% of patients, observed benefit 7.8 mmol/mol (95%CI 6.7-8.9); causal forest 11.6%, observed benefit 8.7 mmol/mol (95%CI 7.4-10.1).

CONCLUSIONS

Consistent with recent results for outcome prediction with clinical data, when evaluating treatment effect heterogeneity researchers should not rely on causal forest or other similar machine learning algorithms alone, and must compare outputs with standard regression, which in this evaluation was superior.

Valuing the Diversity of Research Methods to Advance Nutrition Science

The ASN Board of Directors appointed the Nutrition Research Task Force to develop a report on scientific methods used in nutrition science to advance discovery, interpretation, and application of knowledge in the field. The genesis of this report was growing concern about the tone of discourse among nutrition professionals and the implications of acrimony on the productive study and translation of nutrition science. Too often, honest differences of opinion are cast as conflicts instead of areas of needed collaboration. Recognition of the value (and limitations) of contributions from well-executed nutrition science derived from the various approaches used in the discipline, as well as appreciation of how their layering will yield the strongest evidence base, will provide a basis for greater productivity and impact. Greater collaborative efforts within the field of nutrition science will require an understanding that each method or approach has a place and function that should be valued and used together to create the nutrition evidence base. Precision nutrition was identified as an important emerging nutrition topic by the preponderance of task force members, and this theme was adopted for the report because it lent itself to integration of many approaches in nutrition science. Although the primary audience for this report is nutrition researchers and other nutrition professionals, a secondary aim is to develop a document useful for the various audiences that translate nutrition research, including journalists, clinicians, and policymakers. The intent is to promote accurate, transparent, verifiable evidence-based communication about nutrition science. This will facilitate reasoned interpretation and application of emerging findings and, thereby, improve understanding and trust in nutrition science and appropriate characterization, development, and adoption of recommendations.

Effect of intramuscular midazolam premedication on patient satisfaction in women undergoing general anaesthesia: a randomised control trial

OBJECTIVE

To determine the effect of premedication with intramuscular midazolam on patient satisfaction in women undergoing general anaesthesia.

TRIAL DESIGN, SETTING AND PARTICIPANTS

Double-blind, parallel randomised control trial at a tertiary care medical centre in South Korea. Initially, 140 women aged 20-65 years who underwent general anaesthesia and had an American Society of Anesthesiology physical status classification of I or II were randomly assigned to the intervention group or the control group, and 134 patients (intervention n=65; control n=69) completed the study.

INTERVENTION

Intramuscular administration of midazolam (0.05 mg/kg) or placebo (normal saline 0.01 mL/kg) on arrival at the preoperative holding area.

MAIN OUTCOMES

The primary outcome was the patient's overall satisfaction with the anaesthesia experience as determined by questionnaire responses on the day after surgery. Satisfaction was defined as a response of 3 or 4 on a five-point scale (0-4). The secondary outcomes included blood pressure, heart rate, oxygen desaturation, recovery duration and postoperative pain.

RESULTS

Patients who received midazolam were more satisfied than those who received placebo (percentage difference: 21.0%, OR 3.56, 95% CI 1.46 to 8.70). A subgroup analysis revealed that this difference was greater in patients with anxiety, defined as those whose Amsterdam Preoperative Anxiety and Information Scale anxiety score was ≥11, than that for the whole sample population (percentage difference: 24.0%, OR 4.33, 95% CI 1.25 to 14.96). Both groups had similar heart rates, blood pressure and oxygen desaturation.

CONCLUSION

Intramuscular administration of midazolam in women before general anaesthesia in the preoperative holding area improved self-reported satisfaction with the anaesthesia experience, with an acceptable safety profile.

TRIAL REGISTRATION NUMBER

KCT0006002.

Effect of Esophageal Pressure-Guided Positive End-Expiratory Pressure on Survival from Acute Respiratory Distress Syndrome: A Risk-Based and Mechanistic Reanalysis of the EPVent-2 Trial

RATIONALE

In acute respiratory distress syndrome (ARDS), the effect of positive end-expiratory pressure (PEEP) may depend on the extent to which multiorgan dysfunction contributes to risk of death, and the precision with which PEEP is titrated to attenuate atelectrauma without exacerbating overdistension.

OBJECTIVE

To evaluate whether multiorgan dysfunction and lung mechanics modified treatment effect in EPVent-2, a multicenter trial of esophageal pressure (P_ES)-guided PEEP versus empirical high PEEP in moderate-to-severe ARDS.

METHODS

This post-hoc reanalysis of EPVent-2 evaluated for heterogeneity of treatment effect on mortality by baseline multiorgan dysfunction, determined via Acute Physiology and Chronic Health Evaluation-II (APACHE-II). It also evaluated whether PEEP titrated to end-expiratory transpulmonary pressure near 0 cmH₂O was associated with survival.

MEASUREMENTS AND MAIN RESULTS

All 200 trial participants were included. Treatment effect on 60-day mortality differed by multiorgan dysfunction severity (p=0.03 for interaction). P_ES-guided PEEP was associated with lower mortality among patients with lower APACHE-II (HR 0.43, 95% CI 0.20-0.92 for APACHE-II less than median) and may have had the opposite effect in patients with higher APACHE-II (HR 1.69; 95% CI 0.93-3.05). Independent of treatment group or multiorgan dysfunction severity, mortality was lowest when PEEP titration achieved end-expiratory transpulmonary pressure near 0 cmH₂O.

CONCLUSIONS

The effect on survival of P_ES-guided PEEP, compared to empirical high PEEP, differed by multiorgan dysfunction severity. Independent of multiorgan dysfunction, PEEP titrated to end-expiratory transpulmonary pressure closer to 0 cmH₂O was associated with greater survival than more positive or negative values. These findings warrant prospective testing in a future trial.

Experimental Designs to Optimize Treatments for Individuals: Personalized N-of-1 Trials

Conventional randomized clinical trials (RCTs) compare treatment effectiveness to provide support for evidence-based treatments that can be generalized to the average patient. However, the information obtained from RCTs may not always be useful for selecting the best treatment for individual patients. This article presents a complementary approach to identifying optimized treatments using experimental designs that focus on individuals. Personalized, or N-of-1, designs provide both a comparative analysis of treatments and a functional analysis demonstrating that changes in patient symptoms are likely because of the treatment implemented. This approach contributes to the zeitgeist of personalized medicine and provides clinicians with a paradigm for investigating optimal treatments for rare diseases for which RCTs are not always feasible, identifying personally effective treatments for patients with comorbidities who have historically been excluded from most RCTs, handling clinical situations in which patients respond idiosyncratically (either positively or negatively) to treatment, and shortening the time lag between identification and implementation of an evidence-based treatment. These designs merge experimental analysis of behavior methods used for decades in psychology with new methodological and statistical advances to assess significance levels of changes in individual patients, and they can be generalized to larger populations for meta-analytic purposes. This article presents a case for why these models are needed, an overview of how to apply personalized designs for different types of clinical scenarios, and a brief discussion of challenges associated with interpretation and implementation of personalized designs. The goal is to empower pediatricians to take personalized trial designs into clinical practice to identify optimal treatments for their patients.

Quantifying dementia prevention potential in the FINGER randomized controlled trial using the LIBRA prevention index

INTRODUCTION

Individuals in early dementia prevention trials may differ in how much they benefit from interventions depending on their initial risk level. Additionally, modifiable dementia risk scores might be used as surrogate/intermediate outcomes.

METHODS

In the Finnish Geriatric Intervention Study to Prevent Cognitive Impairment and Disability (FINGER), we investigated in post hoc analyses (N = 1207) whether the cognitive benefits of the 2-year multi-domain lifestyle intervention differed by baseline dementia risk measured with the "LIfestyle for BRAin Health" (LIBRA) score. We also investigated intervention effects on change in LIBRA score over time.

RESULTS

Overall, higher baseline LIBRA was related to less cognitive improvement over time. This association did not differ between the intervention and control groups. The intervention was effective in decreasing LIBRA scores over time, regardless of baseline demographics or cognition.

DISCUSSION

The cognitive benefit of the FINGER intervention was similar across individuals with different LIBRA scores at baseline. Furthermore, LIBRA may be useful as a surrogate/intermediate endpoint and surveillance tool to monitor intervention success during trial execution.

Computational medicine, present and the future: obstetrics and gynecology perspective

Medicine is, in its essence, decision making under uncertainty; the decisions are made about tests to be performed and treatments to be administered. Traditionally, the uncertainty in decision making was handled using expertise collected by individual providers and, more recently, systematic appraisal of research in the form of evidence-based medicine. The traditional approach has been used successfully in medicine for a very long time. However, it has substantial limitations because of the complexity of the system of the human body and healthcare. The complex systems are a network of highly coupled components intensely interacting with each other. These interactions give those systems redundancy and thus robustness to failure and, at the same time, equifinality, that is, many different causative pathways leading to the same outcome. The equifinality of the complex systems of the human body and healthcare system demand the individualization of medical care, medicine, and medical decision making. Computational models excel in modeling complex systems and, consequently, enabling individualization of medical decision making and medicine. Computational models are theory- or knowledge-based models, data-driven models, or models that combine both approaches. Data are essential, although to a different degree, for computational models to successfully represent complex systems. The individualized decision making, made possible by the computational modeling of complex systems, has the potential to revolutionize the entire spectrum of medicine from individual patient care to policymaking. This approach allows applying tests and treatments to individuals who receive a net benefit from them, for whom benefits outweigh the risk, rather than treating all individuals in a population because, on average, the population benefits. Thus, the computational modeling-enabled individualization of medical decision making has the potential to both improve health outcomes and decrease the costs of healthcare.

Preterm neonates benefit from low prophylactic platelet transfusion threshold despite varying risk of bleeding or death

The Platelets for Neonatal Thrombocytopenia (PlaNeT-2) trial reported an unexpected overall benefit of a prophylactic platelet transfusion threshold of 25 × 109/L compared with 50 × 109/L for major bleeding and/or mortality in preterm neonates (7% absolute-risk reduction). However, some neonates in the trial may have experienced little benefit or even harm from the 25 × 109/L threshold. We wanted to assess this heterogeneity of treatment effect in the PlaNet-2 trial, to investigate whether all preterm neonates benefit from the low threshold. We developed a multivariate logistic regression model in the PlaNet-2 data to predict baseline risk of major bleeding and/or mortality for all 653 neonates. We then ranked the neonates based on their predicted baseline risk and categorized them into 4 risk quartiles. Within these quartiles, we assessed absolute-risk difference between the 50 × 109/L- and 25 × 109/L-threshold groups. A total of 146 neonates died or developed major bleeding. The internally validated C-statistic of the model was 0.63 (95% confidence interval, 0.58-0.68). The 25 × 109/L threshold was associated with absolute-risk reduction in all risk groups, varying from 4.9% in the lowest risk group to 12.3% in the highest risk group. These results suggest that a 25 × 109/L prophylactic platelet count threshold can be adopted in all preterm neonates, irrespective of predicted baseline outcome risk. Future studies are needed to improve the predictive accuracy of the baseline risk model. This trial was registered at www.isrctn.com as #ISRCTN87736839.

The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement: Explanation and Elaboration

The PATH (Predictive Approaches to Treatment effect Heterogeneity) Statement was developed to promote the conduct of, and provide guidance for, predictive analyses of heterogeneity of treatment effects (HTE) in clinical trials. The goal of predictive HTE analysis is to provide patient-centered estimates of outcome risk with versus without the intervention, taking into account all relevant patient attributes simultaneously, to support more personalized clinical decision making than can be made on the basis of only an overall average treatment effect. The authors distinguished 2 categories of predictive HTE approaches (a "risk-modeling" and an "effect-modeling" approach) and developed 4 sets of guidance statements: criteria to determine when risk-modeling approaches are likely to identify clinically meaningful HTE, methodological aspects of risk-modeling methods, considerations for translation to clinical practice, and considerations and caveats in the use of effect-modeling approaches. They discuss limitations of these methods and enumerate research priorities for advancing methods designed to generate more personalized evidence. This explanation and elaboration document describes the intent and rationale of each recommendation and discusses related analytic considerations, caveats, and reservations.

Pharmacotherapy for hypertension in adults 60 years or older

BACKGROUND

This is the second substantive update of this review. It was originally published in 1998 and was previously updated in 2009. Elevated blood pressure (known as 'hypertension') increases with age - most rapidly over age 60. Systolic hypertension is more strongly associated with cardiovascular disease than is diastolic hypertension, and it occurs more commonly in older people. It is important to know the benefits and harms of antihypertensive treatment for hypertension in this age group, as well as separately for people 60 to 79 years old and people 80 years or older.

OBJECTIVES

Primary objective• To quantify the effects of antihypertensive drug treatment as compared with placebo or no treatment on all-cause mortality in people 60 years and older with mild to moderate systolic or diastolic hypertensionSecondary objectives• To quantify the effects of antihypertensive drug treatment as compared with placebo or no treatment on cardiovascular-specific morbidity and mortality in people 60 years and older with mild to moderate systolic or diastolic hypertension• To quantify the rate of withdrawal due to adverse effects of antihypertensive drug treatment as compared with placebo or no treatment in people 60 years and older with mild to moderate systolic or diastolic hypertension SEARCH METHODS: The Cochrane Hypertension Information Specialist searched the following databases for randomised controlled trials up to 24 November 2017: the Cochrane Hypertension Specialised Register, the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE Ovid (from 1946), Embase (from 1974), the World Health Organization International Clinical Trials Registry Platform, and ClinicalTrials.gov. We contacted authors of relevant papers regarding further published and unpublished work.

SELECTION CRITERIA

Randomised controlled trials of at least one year's duration comparing antihypertensive drug therapy versus placebo or no treatment and providing morbidity and mortality data for adult patients (≥ 60 years old) with hypertension defined as blood pressure greater than 140/90 mmHg.

DATA COLLECTION AND ANALYSIS

Outcomes assessed were all-cause mortality; cardiovascular morbidity and mortality; cerebrovascular morbidity and mortality; coronary heart disease morbidity and mortality; and withdrawal due to adverse effects. We modified the definition of cardiovascular mortality and morbidity to exclude transient ischaemic attacks when possible.

MAIN RESULTS

This update includes one additional trial (MRC-TMH 1985). Sixteen trials (N = 26,795) in healthy ambulatory adults 60 years or older (mean age 73.4 years) from western industrialised countries with moderate to severe systolic and/or diastolic hypertension (average 182/95 mmHg) met the inclusion criteria. Most of these trials evaluated first-line thiazide diuretic therapy for a mean treatment duration of 3.8 years.Antihypertensive drug treatment reduced all-cause mortality (high-certainty evidence; 11% with control vs 10.0% with treatment; risk ratio (RR) 0.91, 95% confidence interval (CI) 0.85 to 0.97; cardiovascular morbidity and mortality (moderate-certainty evidence; 13.6% with control vs 9.8% with treatment; RR 0.72, 95% CI 0.68 to 0.77; cerebrovascular mortality and morbidity (moderate-certainty evidence; 5.2% with control vs 3.4% with treatment; RR 0.66, 95% CI 0.59 to 0.74; and coronary heart disease mortality and morbidity (moderate-certainty evidence; 4.8% with control vs 3.7% with treatment; RR 0.78, 95% CI 0.69 to 0.88. Withdrawals due to adverse effects were increased with treatment (low-certainty evidence; 5.4% with control vs 15.7% with treatment; RR 2.91, 95% CI 2.56 to 3.30. In the three trials restricted to persons with isolated systolic hypertension, reported benefits were similar.This comprehensive systematic review provides additional evidence that the reduction in mortality observed was due mostly to reduction in the 60- to 79-year-old patient subgroup (high-certainty evidence; RR 0.86, 95% CI 0.79 to 0.95). Although cardiovascular mortality and morbidity was significantly reduced in both subgroups 60 to 79 years old (moderate-certainty evidence; RR 0.71, 95% CI 0.65 to 0.77) and 80 years or older (moderate-certainty evidence; RR 0.75, 95% CI 0.65 to 0.87), the magnitude of absolute risk reduction was probably higher among 60- to 79-year-old patients (3.8% vs 2.9%). The reduction in cardiovascular mortality and morbidity was primarily due to a reduction in cerebrovascular mortality and morbidity.

AUTHORS' CONCLUSIONS

Treating healthy adults 60 years or older with moderate to severe systolic and/or diastolic hypertension with antihypertensive drug therapy reduced all-cause mortality, cardiovascular mortality and morbidity, cerebrovascular mortality and morbidity, and coronary heart disease mortality and morbidity. Most evidence of benefit pertains to a primary prevention population using a thiazide as first-line treatment.

Risk related therapy in meta-analyses of critical care interventions: Bayesian meta-regression analysis

PURPOSE

The relationship between treatment efficacy and patient risk is explored in a series of meta-analyses from the critical care domain, focusing on mortality outcome.

METHODS

Systematic reviews of randomized controlled trials were identified by electronic search over the period 2002 to July 2018. A Bayesian meta-regression model was employed, using the risk difference metric to estimate the relationship between mortality difference and control arm risk, and estimate the mortality difference with and without adjusting for control arm risk.

RESULTS

Of 780 initially identified published systematic reviews, 113 had appropriate mortality data comprising 123 analysable groups. The 123 meta-analyses were pharmaceutical therapeutic (59.3%), non-pharmaceutical therapeutic (24.4%) and nutritional (16.3%), with a 25% overall average control arm mortality. In 25/123 (20%) analyses, meta-regression indicated significant baseline risk (Bayesian 95% credible intervals excluding zero). In all analyses, the relationship between risk-difference and control arm risk was negative indicating a positive treatment effect with increasing control arm risk. Adjusted estimates identified six studies with significant positive treatment effects, not evident until after adjustment for control arm risk.

CONCLUSION

Underlying risk-related therapy is apparent in meta-analyses of the critically-ill and identification is of importance to both the conduct and interpretation of these meta-analyses.

Perspective: Limiting Dependence on Nonrandomized Studies and Improving Randomized Trials in Human Nutrition Research: Why and How

A large majority of human nutrition research uses nonrandomized observational designs, but this has led to little reliable progress. This is mostly due to many epistemologic problems, the most important of which are as follows: difficulty detecting small (or even tiny) effect sizes reliably for nutritional risk factors and nutrition-related interventions; difficulty properly accounting for massive confounding among many nutrients, clinical outcomes, and other variables; difficulty measuring diet accurately; and suboptimal research reporting. Tiny effect sizes and massive confounding are largely unfixable problems that narrowly confine the scenarios in which nonrandomized observational research is useful. Although nonrandomized studies and randomized trials have different priorities (assessment of long-term causality compared with assessment of treatment effects), the odds for obtaining reliable information with the former are limited. Randomized study designs should therefore largely replace nonrandomized studies in human nutrition research going forward. To achieve this, many of the limitations that have traditionally plagued most randomized trials in nutrition, such as small sample size, short length of follow-up, high cost, and selective reporting, among others, must be overcome. Pivotal megatrials with tens of thousands of participants and lifelong follow-up are possible in nutrition science with proper streamlining of operational costs. Fixable problems that have undermined observational research, such as dietary measurement error and selective reporting, need to be addressed in randomized trials. For focused questions in which dietary adherence is important to maximize, trials with direct observation of participants in experimental in-house settings may offer clean answers on short-term metabolic outcomes. Other study designs of randomized trials to consider in nutrition include registry-based designs and "N-of-1" designs. Mendelian randomization designs may also offer some more reliable leads for testing interventions in trials. Collectively, an improved randomized agenda may clarify many things in nutrition science that might never be answered credibly with nonrandomized observational designs.

Using group data to treat individuals: understanding heterogeneous treatment effects in the age of precision medicine and patient-centred evidence

Although often conflated, determining the best treatment for an individual (the task of a doctor) is fundamentally different from determining the average effect of treatment in a population (the purpose of a trial). In this paper, we review concepts of heterogeneity of treatment effects (HTE) essential in providing the evidence base for precision medicine and patient-centred care, and explore some inherent limitations of using group data (e.g. from a randomized trial) to guide treatment decisions for individuals. We distinguish between person-level HTE (i.e. that individuals experience different effects from a treatment) and group-level HTE (i.e. that subgroups have different average treatment effects), and discuss the reference class problem, engendered by the large number of potentially informative subgroupings of a study population (each of which may lead to applying a different estimated effect to the same patient), and the scale dependence of group-level HTE. We also review the limitations of conventional 'one-variable-at-a-time' subgroup analyses and discuss the potential benefits of using more comprehensive subgrouping schemes that incorporate information on multiple variables, such as those based on predicted outcome risk. Understanding the conceptual underpinnings of HTE is critical for understanding how studies can be designed, analysed, and interpreted to better inform individualized clinical decisions.

Cultural primer for cardiometabolic health: health disparities, structural factors, community, pathways to improvement, and clinical applications

The quest to optimize cardiometabolic health has created great interest in nonmedical health variables in the population, community-based research and coordination, and addressing social, ethnic, and cultural barriers. All of these may be of equal or even greater importance than classical health care delivery in achieving individual well-being. One dominant issue is health disparity - causes, methods of reduction, and community versus other levels of solutions. This communication summarizes some major views regarding social structures, followed by amplification and synthesis of central ideas in the literature. The role of community involvement, tools, and partnerships is also presented in this Primer. Recent views of how these approaches could be incorporated into cardiometabolic initiatives and strategies follow, with implications for research. Two examples comparing selected aspects of community leverage and interventions in relation to individual approaches to health care equity are examined in depth: overall performance in reducing cardiovascular risk and mortality, and the recent National Diabetes Prevention Program, both touching upon healthy diets and adherence. Finally, the potential that precision medicine offers, and possible effects on disparities are also discussed.

Clinical Trials in Veterinary Medicine: A New Era Brings New Challenges

Randomized clinical trials (RCTs) are among the most rigorous ways to determine the causal relationship between an intervention and important clinical outcome. Their use in veterinary medicine has become increasingly common, and as is often the case, with progress comes new challenges. Randomized clinical trials yield important answers, but results from these studies can be unhelpful or even misleading unless the study design and reporting are carried out with care. Herein, we offer some perspective on several emerging challenges associated with RCTs, including use of composite endpoints, the reporting of different forms of risk, analysis in the presence of missing data, and issues of reporting and safety assessment. These topics are explored in the context of previously reported veterinary internal medicine studies as well as through illustrative examples with hypothetical data sets. Moreover, many insights germane to RCTs in veterinary internal medicine can be drawn from the wealth of experience with RCTs in the human medical field. A better understanding of the issues presented here can help improve the design, interpretation, and reporting of veterinary RCTs.

Individualizing endpoints in randomized clinical trials to better inform individual patient care: the TARGET proposal

In practice, critical care practitioners individualize treatments and goals of care for each patient in light of that patient’s acute and chronic pathophysiology, as well as their beliefs and values. Yet critical care researchers routinely measure one endpoint for all patients during randomized clinical trials (RCTs), eschewing any such individualization. More recent methodology work has explored the possibility that enrollment criteria in RCTs can be individualized, as can data analysis plans. Here we propose that the specific endpoints of a RCT can be individualized—that is, different patients within a single RCT might have different secondary endpoints measured. If done rigorously and objectively, based on pre-randomization data, such individualization of endpoints may improve the bedside usefulness of information obtained during a RCT, while perhaps also improving the power and efficiency of any RCT. We discuss the theoretical underpinnings of this proposal in light of related innovations in RCT design such as sliding dichotomies. We discuss what a full elaboration of such individualization would require, and outline a pragmatic initial step towards the use of “individualized secondary endpoints” in a large RCT evaluating optimal enteral nutrition targets in the critically ill.

Implications of Heterogeneity of Treatment Effect for Reporting and Analysis of Randomized Trials in Critical Care

Randomized clinical trials (RCTs) are conducted to guide clinicians' selection of therapies for individual patients. Currently, RCTs in critical care often report an overall mean effect and selected individual subgroups. Yet work in other fields suggests that such reporting practices can be improved. Specifically, this Critical Care Perspective reviews recent work on so-called "heterogeneity of treatment effect" (HTE) by baseline risk and extends that work to examine its applicability to trials of acute respiratory failure and severe sepsis. Because patients in RCTs in critical care medicine-and patients in intensive care units-have wide variability in their risk of death, these patients will have wide variability in the absolute benefit that they can derive from a given therapy. If the side effects of the therapy are not perfectly collinear with the treatment benefits, this will result in HTE, where different patients experience quite different expected benefits of a therapy. We use simulations of RCTs to demonstrate that such HTE could result in apparent paradoxes, including: (1) positive trials of therapies that are beneficial overall but consistently harm or have little benefit to low-risk patients who met enrollment criteria, and (2) overall negative trials of therapies that still consistently benefit high-risk patients. We further show that these results persist even in the presence of causes of death unmodified by the treatment under study. These results have implications for reporting and analyzing RCT data, both to better understand how our therapies work and to improve the bedside applicability of RCTs. We suggest a plan for measurement in future RCTs in the critically ill.

Welfare implications of learning through solicitation versus diversification in health care

Using Roy's model of sorting behavior, I study welfare implications of learning about medical care quality through the current health care data production infrastructure that relies on solicitation of research subjects. Due to severe adverse-selection issues, I show that such learning could be biased and welfare decreasing. Direct diversification of treatment receipt may solve these issues but is infeasible. Unifying Manski's work on diversified treatment choice under ambiguity and Heckman's work on estimating heterogeneous treatment effects, I propose a new infrastructure based on temporary diversification of access that resolves the prior issues and can identify nuanced effect heterogeneity.

Improving diabetes prevention with benefit based tailored treatment: risk based reanalysis of Diabetes Prevention Program

OBJECTIVE

To determine whether some participants in the Diabetes Prevention Program were more or less likely to benefit from metformin or a structured lifestyle modification program.

DESIGN

Post hoc analysis of the Diabetes Prevention Program, a randomized controlled trial.

SETTING

Ambulatory care patients.

PARTICIPANTS

3060 people without diabetes but with evidence of impaired glucose metabolism.

INTERVENTION

Intervention groups received metformin or a lifestyle modification program with the goals of weight loss and physical activity.

MAIN OUTCOME MEASURE

Development of diabetes, stratified by the risk of developing diabetes according to a diabetes risk prediction model.

RESULTS

Of the 3081 participants with impaired glucose metabolism at baseline, 655 (21%) progressed to diabetes over a median 2.8 years' follow-up. The diabetes risk model had good discrimination (C statistic=0.73) and calibration. Although the lifestyle intervention provided a sixfold greater absolute risk reduction in the highest risk quarter than in the lowest risk quarter, patients in the lowest risk quarter still received substantial benefit (three year absolute risk reduction 4.9% v 28.3% in highest risk quarter; numbers needed to treat of 20.4 and 3.5, respectively). The benefit of metformin, however, was seen almost entirely in patients in the top quarter of risk of diabetes. No benefit was seen in the lowest risk quarter. Participants in the highest risk quarter averaged a 21.4% three year absolute risk reduction (number needed to treat 4.6).

CONCLUSIONS

Patients at high risk of diabetes have substantial variation in their likelihood of receiving benefit from diabetes prevention treatments. Using this knowledge could decrease overtreatment and make prevention of diabetes far more efficient, effective, and patient centered, provided that decision making is based on an accurate risk prediction tool.

Systemic safety of bevacizumab versus ranibizumab for neovascular age-related macular degeneration

BACKGROUND

Neovascular age-related macular degeneration (AMD) is the leading cause of legal blindness in elderly populations of industrialised countries. Bevacizumab (Avastin®) and ranibizumab (Lucentis®) are targeted biological drugs (a monoclonal antibody) that inhibit vascular endothelial growth factor, an angiogenic cytokine that promotes vascular leakage and growth, thereby preventing its pathological angiogenesis. Ranibizumab is approved for intravitreal use to treat neovascular AMD, while bevacizumab is approved for intravenous use as a cancer therapy. However, due to the biological similarity of the two drugs, bevacizumab is widely used off-label to treat neovascular AMD.

OBJECTIVES

To assess the systemic safety of intravitreal bevacizumab (brand name Avastin®; Genentech/Roche) compared with intravitreal ranibizumab (brand name Lucentis®; Novartis/Genentech) in people with neovascular AMD. Primary outcomes were death and All serious systemic adverse events (All SSAEs), the latter as a composite outcome in accordance with the International Conference on Harmonisation Good Clinical Practice. Secondary outcomes examined specific SSAEs: fatal and non-fatal myocardial infarctions, strokes, arteriothrombotic events, serious infections, and events grouped in some Medical Dictionary for Regulatory Activities System Organ Classes (MedDRA SOC). We assessed the safety at the longest available follow-up to a maximum of two years.

SEARCH METHODS

We searched CENTRAL, MEDLINE, EMBASE and other online databases up to 27 March 2014. We also searched abstracts and clinical study presentations at meetings, trial registries, and contacted authors of included studies when we had questions.

SELECTION CRITERIA

Randomised controlled trials (RCTs) directly comparing intravitreal bevacizumab (1.25 mg) and ranibizumab (0.5 mg) in people with neovascular AMD, regardless of publication status, drug dose, treatment regimen, or follow-up length, and whether the SSAEs of interest were reported in the trial report.

DATA COLLECTION AND ANALYSIS

Two authors independently selected studies and assessed the risk of bias for each study. Three authors independently extracted data.We conducted random-effects meta-analyses for the primary and secondary outcomes. We planned a pre-specified analysis to explore deaths and All SSAEs at the one-year follow-up.

MAIN RESULTS

We included data from nine studies (3665 participants), including six published (2745 participants) and three unpublished (920 participants) RCTs, none supported by industry. Three studies excluded participants at high cardiovascular risk, increasing clinical heterogeneity among studies. The studies were well designed, and we did not downgrade the quality of the evidence for any of the outcomes due to risk of bias. Although the estimated effects of bevacizumab and ranibizumab on our outcomes were similar, we downgraded the quality of the evidence due to imprecision.At the maximum follow-up (one or two years), the estimated risk ratio (RR) of death with bevacizumab compared with ranibizumab was 1.10 (95% confidence interval (CI) 0.78 to 1.57, P value = 0.59; eight studies, 3338 participants; moderate quality evidence). Based on the event rates in the studies, this gives a risk of death with ranibizumab of 3.4% and with bevacizumab of 3.7% (95% CI 2.7% to 5.3%).For All SSAEs, the estimated RR was 1.08 (95% CI 0.90 to 1.31, P value = 0.41; nine studies, 3665 participants; low quality evidence). Based on the event rates in the studies, this gives a risk of SSAEs of 22.2% with ranibizumab and with bevacizumab of 24% (95% CI 20% to 29.1%).For the secondary outcomes, we could not detect any difference between bevacizumab and ranibizumab, with the exception of gastrointestinal disorders MedDRA SOC where there was a higher risk with bevacizumab (RR 1.82; 95% CI 1.04 to 3.19, P value = 0.04; six studies, 3190 participants).Pre-specified analyses of deaths and All SSAEs at one-year follow-up did not substantially alter the findings of our review.Fixed-effect analysis for deaths did not substantially alter the findings of our review, but fixed-effect analysis of All SSAEs showed an increased risk for bevacizumab (RR 1.12; 95% CI 1.00 to 1.26, P value = 0.04; nine studies, 3665 participants): the meta-analysis was dominated by a single study (weight = 46.9%).The available evidence was sensitive to the exclusion of CATT or unpublished results. For All SSAEs, the exclusion of CATT moved the overall estimate towards no difference (RR 1.01; 95% CI 0.82 to 1.25, P value = 0.92), while the exclusion of LUCAS yielded a larger RR, with more SSAEs in the bevacizumab group, largely driven by CATT (RR 1.19; 95% CI 1.06 to 1.34, P value = 0.004). The exclusion of all unpublished studies produced a RR of 1.12 for death (95% CI 0.78 to 1.62, P value = 0.53) and a RR of 1.21 for SSAEs (95% CI 1.06 to 1.37, P value = 0.004), indicating a higher risk of SSAEs in those assigned to bevacizumab than ranibizumab.

AUTHORS' CONCLUSIONS

This systematic review of non-industry sponsored RCTs could not determine a difference between intravitreal bevacizumab and ranibizumab for deaths, All SSAEs, or specific subsets of SSAEs in the first two years of treatment, with the exception of gastrointestinal disorders. The current evidence is imprecise and might vary across levels of patient risks, but overall suggests that if a difference exists, it is likely to be small. Health policies for the utilisation of ranibizumab instead of bevacizumab as a routine intervention for neovascular AMD for reasons of systemic safety are not sustained by evidence. The main results and quality of evidence should be verified once all trials are fully published.

Tiotropium might improve survival in subjects with COPD at high risk of mortality

Background

Inhaled therapies reduce risk of chronic obstructive pulmonary disease (COPD) exacerbations, but their effect on mortality is less well established. We hypothesized that heterogeneity in baseline mortality risk influenced the results of drug trials assessing mortality in COPD.

Methods

The 5706 patients with COPD from the Understanding Potential Long-term Impacts on Function with Tiotropium (UPLIFT®) study that had complete clinical information for variables associated with mortality (age, forced expiratory volume in 1 s, St George’s Respiratory Questionnaire, pack-years and body mass index) were classified by cluster analysis. Baseline risk of mortality between clusters, and impact of tiotropium were evaluated during the 4-yr follow up.

Results

Four clusters were identified, including low-risk (low mortality rate) patients (n = 2339; 41%; cluster 2), and high-risk patients (n = 1022; 18%; cluster 3), who had a 2.6- and a six-fold increase in all-cause and respiratory mortality compared with cluster 2, respectively. Tiotropium reduced exacerbations in all clusters, and reduced hospitalizations in high-risk patients (p < 0.05). The beneficial effect of tiotropium on all-cause mortality in the overall population (hazard ratio, 0.87; 95% confidence interval, 0.75–1.00, p = 0.054) was explained by a 21% reduction in cluster 3 (p = 0.07), with no effect in other clusters.

Conclusions

Large variations in baseline risks of mortality existed among patients in the UPLIFT® study. Inclusion of numerous low-risk patients may have reduced the ability to show beneficial effect on mortality. Future clinical trials should consider selective inclusion of high-risk patients.

Effects of interventions on survival in acute respiratory distress syndrome: an umbrella review of 159 published randomized trials and 29 meta-analyses

PURPOSE

Multiple interventions have been tested in acute respiratory distress syndrome (ARDS). We examined the entire agenda of published randomized controlled trials (RCTs) in ARDS that reported on mortality and of respective meta-analyses.

METHODS

We searched PubMed, the Cochrane Library, and Web of Knowledge until July 2013. We included RCTs in ARDS published in English. We excluded trials of newborns and children; and those on short-term interventions, ARDS prevention, or post-traumatic lung injury. We also reviewed all meta-analyses of RCTs in this field that addressed mortality. Treatment modalities were grouped in five categories: mechanical ventilation strategies and respiratory care, enteral or parenteral therapies, inhaled/intratracheal medications, nutritional support, and hemodynamic monitoring.

RESULTS

We identified 159 published RCTs of which 93 had overall mortality reported (n = 20,671 patients)--44 trials (14,426 patients) reported mortality as a primary outcome. A statistically significant survival benefit was observed in eight trials (seven interventions) and two trials reported an adverse effect on survival. Among RCTs with more than 50 deaths in at least one treatment arm (n = 21), two showed a statistically significant mortality benefit of the intervention (lower tidal volumes and prone positioning), one showed a statistically significant mortality benefit only in adjusted analyses (cisatracurium), and one (high-frequency oscillatory ventilation) showed a significant detrimental effect. Across 29 meta-analyses, the most consistent evidence was seen for low tidal volumes and prone positioning in severe ARDS.

CONCLUSIONS

There is limited supportive evidence that specific interventions can decrease mortality in ARDS. While low tidal volumes and prone positioning in severe ARDS seem effective, most sporadic findings of interventions suggesting reduced mortality are not corroborated consistently in large-scale evidence including meta-analyses.

Comparative effect sizes in randomised trials from less developed and more developed countries: meta-epidemiological assessment

OBJECTIVE

To compare treatment effects from randomised trials conducted in more developed versus less developed countries.

DESIGN

Meta-epidemiological study.

DATA SOURCES

Cochrane Database of Systematic Reviews (August 2012).

DATA EXTRACTION

Meta-analyses with mortality outcomes including data from at least one randomised trial conducted in a less developed country and one in a more developed country. Relative risk estimates of more versus less developed countries were compared by calculating the relative relative risks for each topic and the summary relative relative risks across all topics. Similar analyses were performed for the primary binary outcome of each topic.

RESULTS

139 meta-analyses with mortality outcomes were eligible. No nominally significant differences between more developed and less developed countries were found for 128 (92%) meta-analyses. However, differences were beyond chance in 11 (8%) cases, always showing more favourable treatment effects in trials from less developed countries. The summary relative relative risk was 1.12 (95% confidence interval 1.06 to 1.18; P<0.001; I(2)=0%), suggesting significantly more favourable mortality effects in trials from less developed countries. Results were similar for meta-analyses with nominally significant treatment effects for mortality (1.15), meta-analyses with recent trials (1.14), and when excluding trials from less developed countries that subsequently became more developed (1.12). For the primary binary outcomes (127 meta-analyses), 20 topics had differences in treatment effects beyond chance (more favourable in less developed countries in 15/20 cases).

CONCLUSIONS

Trials from less developed countries in a few cases show significantly more favourable treatment effects than trials in more developed countries and, on average, treatment effects are more favourable in less developed countries. These discrepancies may reflect biases in reporting or study design as well as genuine differences in baseline risk or treatment implementation and should be considers when generalising evidence across different settings.

Individualized cost-effectiveness analysis

John Ioannidis and Alan Garber discuss how to use incremental cost-effectiveness ratios (ICER) and related metrics so they can be useful for decision-making at the individual level, whether used by clinicians or individual patients.

Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects

OBJECTIVE

To assess whether nominally statistically significant effects in meta-analyses of clinical trials are true and whether their magnitude is inflated.

STUDY DESIGN AND SETTING

Data from the Cochrane Database of Systematic Reviews 2005 (issue 4) and 2010 (issue 1) were used. We considered meta-analyses with binary outcomes and four or more trials in 2005 with P<0.05 for the random-effects odds ratio (OR). We examined whether any of these meta-analyses had updated counterparts in 2010. We estimated the credibility (true-positive probability) under different prior assumptions and inflation in OR estimates in 2005.

RESULTS

Four hundred sixty-one meta-analyses in 2005 were eligible, and 80 had additional trials included by 2010. The effect sizes (ORs) were smaller in the updating data (2005-2010) than in the respective meta-analyses in 2005 (median 0.85-fold, interquartile range [IQR]: 0.66-1.06), even more prominently for meta-analyses with less than 300 events in 2005 (median 0.67-fold, IQR: 0.54-0.96). Mean credibility of the 461 meta-analyses in 2005 was 63-84% depending on the assumptions made. Credibility estimates changed >20% in 19-31 (24-39%) of the 80 updated meta-analyses.

CONCLUSIONS

Most meta-analyses with nominally significant results pertain to truly nonnull effects, but exceptions are not uncommon. The magnitude of observed effects, especially in meta-analyses with limited evidence, is often inflated.

Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal

Mounting evidence suggests that there is frequently considerable variation in the risk of the outcome of interest in clinical trial populations. These differences in risk will often cause clinically important heterogeneity in treatment effects (HTE) across the trial population, such that the balance between treatment risks and benefits may differ substantially between large identifiable patient subgroups; the "average" benefit observed in the summary result may even be non-representative of the treatment effect for a typical patient in the trial. Conventional subgroup analyses, which examine whether specific patient characteristics modify the effects of treatment, are usually unable to detect even large variations in treatment benefit (and harm) across risk groups because they do not account for the fact that patients have multiple characteristics simultaneously that affect the likelihood of treatment benefit. Based upon recent evidence on optimal statistical approaches to assessing HTE, we propose a framework that prioritizes the analysis and reporting of multivariate risk-based HTE and suggests that other subgroup analyses should be explicitly labeled either as primary subgroup analyses (well-motivated by prior evidence and intended to produce clinically actionable results) or secondary (exploratory) subgroup analyses (performed to inform future research). A standardized and transparent approach to HTE assessment and reporting could substantially improve clinical trial utility and interpretability.

Variation in the net benefit of aggressive cardiovascular risk factor control across the US population of patients with diabetes mellitus

BACKGROUND

Lowering low-density lipoprotein cholesterol (LDL-C) and blood pressure (BP) in patients with diabetes mellitus (DM) can significantly reduce the risk of cardiovascular disease (CVD). However, to our knowledge, previous studies have not assessed variability in both the benefit and harm from pursuing LDL-C and BP target levels.

METHODS

Our sample comprised individuals 30 to 75 years old with DM participating in the National Health and Nutrition Examination Survey III. We used Monte Carlo methods to simulate a treat-to-target strategy, in which patients underwent treatment intensification with the goal of achieving LDL-C and BP target levels of 100 mg/dL and 130/80 mm Hg, respectively. Patients received up to 5 titrations of statin therapy and 8 titrations of antihypertensive therapy. Treatment adverse effects and polypharmacy risks and burdens were incorporated using disutilities. Health outcomes were simulated using a Markov model.

RESULTS

Treating to targets resulted in gains of 1.50 (for LDL-C) and 1.35 (for BP) quality-adjusted life-years (QALYs) of lifetime treatment-related benefit, which declined to 1.42 and 1.16 QALYs after accounting for treatment-related harms. Most of the total benefit was limited to the first few steps of medication intensification or to tight control for a limited group of very high-risk patients. However, because of treatment-related disutility, intensifying beyond the first step (LDL-C) or third step (BP) resulted in either limited benefit or net harm for patients with below-average risk.

CONCLUSION

The benefits and harms from aggressive risk factor modification vary widely across the US population of individuals with DM, depending on a patient's underlying CVD risk, suggesting that a personalized approach could maximize a patient's net benefit from treatment.

Pharmacotherapy for hypertension in the elderly

BACKGROUND

Elevated blood pressure (known as hypertension) increases with age, and most rapidly over age 60. Systolic hypertension is more strongly associated with cardiovascular disease than diastolic hypertension, and occurs more commonly in older people. It is important to know the benefits and harms of antihypertensive treatment of hypertension in this age group.

OBJECTIVES

To quantify antihypertensive drug effect on overall mortality, cardiovascular mortality and morbidity and withdrawal due to adverse effects in people 60 years and older with mild to moderate systolic or diastolic hypertension.

SEARCH STRATEGY

Updated search of electronic database of EMBASE, CENTRAL, MEDLINE until Dec 2008; previous search of two Japanese databases (1973-1995) and WHO-ISH Collaboration register (August 1997); references from reviews, trials and previously published meta-analyses; and experts.

SELECTION CRITERIA

Randomized controlled trials of at least one year duration in hypertensive elders (at least 60 years old) comparing antihypertensive drug therapy with placebo or no treatment and providing morbidity and mortality data.

DATA COLLECTION AND ANALYSIS

Outcomes assessed were total mortality (including cardiovascular, coronary heart disease and cerebrovascular mortality); total cardiovascular morbidity and mortality (representing combined coronary heart disease and cerebrovascular morbidity and mortality); and withdrawal due to adverse events.

MAIN RESULTS

Fifteen trials (24,055 subjects >/= 60 years) with moderate to severe hypertension were identified. These trials mostly evaluated first-line thiazide diuretic therapy for a mean duration of treatment of 4.5 years. Treatment reduced total mortality, RR 0.90 (0.84, 0.97); event rates per 1000 participants reduced from 116 to 104. Treatment also reduced total cardiovascular morbidity and mortality, RR 0.72 (0.68, 0.77); event rates per 1000 participants reduced from 149 to 106. In the three trials restricted to persons with isolated systolic hypertension the benefit was similar. In very elderly patients >/= 80 years the reduction in total cardiovascular mortality and morbidity was similar RR 0.75 [0.65, 0.87] however, there was no reduction in total mortality, RR 1.01 [0.90, 1.13]. Withdrawals due to adverse effects were increased with treatment, RR 1.71 [1.45, 2.00].

AUTHORS' CONCLUSIONS

Treating healthy persons (60 years or older) with moderate to severe systolic and/or diastolic hypertension reduces all cause mortality and cardiovascular morbidity and mortality. The decrease in all cause mortality was limited to persons 60 to 80 years of age.

Revisiting the relationship between baseline risk and risk under treatment

BACKGROUND

In medical practice, it is generally accepted that the 'effect model' describing the relationship between baseline risk and risk under treatment is linear, i.e. 'relative risk' is constant. Absolute benefit is then proportional to a patient's baseline risk and the treatment is most effective among high-risk patients. Alternatively, the 'effect model' becomes curvilinear when 'odds ratio' is considered to be constant. However these two models are based on purely empirical considerations, and there is still no theoretical approach to support either the linear or the non-linear relation.

PRESENTATION OF THE HYPOTHESIS

From logistic and sigmoidal Emax (Hill) models, we derived a phenomenological model which includes the possibility of integrating both beneficial and harmful effects. Instead of a linear relation, our model suggests that the relationship is curvilinear i.e. the moderate-risk patients gain most from the treatment in opposition to those with low or high risk.

TESTING THE HYPOTHESIS

Two approaches can be proposed to investigate in practice such a model. The retrospective one is to perform a meta-analysis of clinical trials with subgroups of patients including a great range of baseline risks. The prospective one is to perform a large clinical trial in which patients are recruited according to several prestratified diverse and high risk groups.

IMPLICATIONS OF THE HYPOTHESIS

For the quantification of the treatment effect and considering such a model, the discrepancy between odds ratio and relative risk may be related not only to the level of risk under control conditions, but also to the characteristics of the dose-effect relation and the amount of dose administered. In the proposed approach, OR may be considered as constant in the whole range of Rc, and depending only on the intrinsic characteristics of the treatment. Therefore, OR should be preferred rather than RR to summarize information on treatment efficacy.

Interpretation of tests of heterogeneity and bias in meta-analysis

Statistical tests of heterogeneity and bias, in particular publication bias, are very popular in meta-analyses. These tests use statistical approaches whose limitations are often not recognized. Moreover, it is often implied with inappropriate confidence that these tests can provide reliable answers to questions that in essence are not of statistical nature. Statistical heterogeneity is only a correlate of clinical and pragmatic heterogeneity and the correlation may sometimes be weak. Similarly, statistical signals may hint to bias, but seen in isolation they cannot fully prove or disprove bias in general, let alone specific causes of bias, such as publication bias in particular. Both false-positive and false-negative signals of heterogeneity and bias can be common and their prevalence may be anticipated based on some rational considerations. Here I discuss the major common challenges and flaws that emerge in using and interpreting statistical tests of heterogeneity and bias in meta-analyses. I discuss misinterpretations that can occur at the level of statistical inference, clinical/pragmatic inference and specific cause attribution. Suggestions are made on how to avoid these flaws, use these tests properly and learn from them.

Competing risk and heterogeneity of treatment effect in clinical trials

It has been demonstrated that patients enrolled in clinical trials frequently have a large degree of variation in their baseline risk for the outcome of interest. Thus, some have suggested that clinical trial results should routinely be stratified by outcome risk using risk models, since the summary results may otherwise be misleading. However, variation in competing risk is another dimension of risk heterogeneity that may also underlie treatment effect heterogeneity. Understanding the effects of competing risk heterogeneity may be especially important for pragmatic comparative effectiveness trials, which seek to include traditionally excluded patients, such as the elderly or complex patients with multiple comorbidities. Indeed, the observed effect of an intervention is dependent on the ratio of outcome risk to competing risk, and these risks - which may or may not be correlated - may vary considerably in patients enrolled in a trial. Further, the effects of competing risk on treatment effect heterogeneity can be amplified by even a small degree of treatment related harm. Stratification of trial results along both the competing and the outcome risk dimensions may be necessary if pragmatic comparative effectiveness trials are to provide the clinically useful information their advocates intend.

Molecular evidence-based medicine: evolution and integration of information in the genomic era

Evidence-based medicine and molecular medicine have both been influential in biomedical research in the last 15 years. Despite following largely parallel routes to date, the goals and principles of evidence-based and molecular medicine are complementary and they should be converging. I define molecular evidence-based medicine as the study of medical information that makes sense of the advances of molecular biological disciplines and where errors and biases are properly appreciated and placed in context. Biomedical measurement capacity improves very rapidly. The exponentially growing mass of hypotheses being tested requires a new approach to both statistical and biological inference. Multidimensional biology requires careful exact replication of research findings, but indirect corroboration is often all that is achieved at best. Besides random error, bias remains a major threat. It is often difficult to separate bias from the spirit of scientific inquiry to force data into coherent and 'significant' biological stories. Transparency and public availability of protocols, data, analyses and results may be crucial to make sense of the complex biology of human disease and avoid being flooded by spurious research findings. Research efforts should be integrated across teams in an open, sharing environment. Most research in the future may be designed, performed, and integrated in the public cyberspace.

Static magnetic therapy does not decrease pain or opioid requirements: a randomized double-blind trial

A growing multibillion dollar industry markets magnetic necklaces, bracelets, bands, insoles, back braces, mattresses, etc., for pain relief, although there is little evidence for their efficacy. We sought to evaluate the effect of magnetic therapy on pain intensity and opioid requirements in patients with postoperative pain. We designed a randomized, double-blind, controlled trial. One-hundred-sixty-five patients older than 12 yr of age were randomized to magnetic (n = 81) or sham therapy (n = 84) upon reporting moderate-to-severe pain in the postanesthesia care unit. Devices were placed over the surgical incision and left in place for 2 h. Patients rated their pain intensity on a 0-10 scale every 10 min and received incremental doses of morphine until pain intensity was < or =4 of 10. Pain intensity levels were similar in both groups. The magnet group had on average 0.04 U more pain intensity (95% confidence interval, -0.4 to 0.5) than the sham group. Opioid requirements also were similar in both groups. The active magnet group required 1.5 mg more morphine (95% confidence interval, -1.8 to 4.0) than the sham magnet group. Magnetic therapy lacks efficacy in controlling acute postoperative pain intensity levels or opioid requirements and should not be recommended for pain relief in this setting.

Evolution and translation of research findings: from bench to where?

The credibility and replication of research findings evolve over time, as data accumulate. However, translation of postulated research promises to real-life biomedical applications is uncommon. In some fields of research, we may observe diminishing effects for the strength of research findings and rapid alternations of exaggerated claims and extreme contradictions--the "Proteus Phenomenon." While these phenomena are probably more prominent in the basic sciences, similar manifestations have been documented even in clinical trials and they may undermine the credibility of clinical research. Significance-chasing bias may be in part responsible, but the greatest threat may come from the poor relevance and scientific rationale and thus low pre-study odds of success of research efforts. Given that we currently have too many research findings, often with low credibility, replication and rigorous evaluation become as important as or even more important than discovery. Credibility, replication, and translation are all desirable properties of research findings, but are only modestly correlated. In this essay, I discuss some of the evidence (or lack thereof) for the process of evolution and translation of research findings, with emphasis on the biomedical sciences.

Multivariable risk prediction can greatly enhance the statistical power of clinical trial subgroup analysis

Background

When subgroup analyses of a positive clinical trial are unrevealing, such findings are commonly used to argue that the treatment's benefits apply to the entire study population; however, such analyses are often limited by poor statistical power. Multivariable risk-stratified analysis has been proposed as an important advance in investigating heterogeneity in treatment benefits, yet no one has conducted a systematic statistical examination of circumstances influencing the relative merits of this approach vs. conventional subgroup analysis.

Methods

Using simulated clinical trials in which the probability of outcomes in individual patients was stochastically determined by the presence of risk factors and the effects of treatment, we examined the relative merits of a conventional vs. a "risk-stratified" subgroup analysis under a variety of circumstances in which there is a small amount of uniformly distributed treatment-related harm. The statistical power to detect treatment-effect heterogeneity was calculated for risk-stratified and conventional subgroup analysis while varying: 1) the number, prevalence and odds ratios of individual risk factors for risk in the absence of treatment, 2) the predictiveness of the multivariable risk model (including the accuracy of its weights), 3) the degree of treatment-related harm, and 5) the average untreated risk of the study population.

Results

Conventional subgroup analysis (in which single patient attributes are evaluated "one-at-a-time") had at best moderate statistical power (30% to 45%) to detect variation in a treatment's net relative risk reduction resulting from treatment-related harm, even under optimal circumstances (overall statistical power of the study was good and treatment-effect heterogeneity was evaluated across a major risk factor [OR = 3]). In some instances a multi-variable risk-stratified approach also had low to moderate statistical power (especially when the multivariable risk prediction tool had low discrimination). However, a multivariable risk-stratified approach can have excellent statistical power to detect heterogeneity in net treatment benefit under a wide variety of circumstances, instances under which conventional subgroup analysis has poor statistical power.

Conclusion

These results suggest that under many likely scenarios, a multivariable risk-stratified approach will have substantially greater statistical power than conventional subgroup analysis for detecting heterogeneity in treatment benefits and safety related to previously unidentified treatment-related harm. Subgroup analyses must always be well-justified and interpreted with care, and conventional subgroup analyses can be useful under some circumstances; however, clinical trial reporting should include a multivariable risk-stratified analysis when an adequate externally-developed risk prediction tool is available.

The Avoiding Cardiovascular events through COMbination therapy in Patients Living with Systolic Hypertension (ACCOMPLISH) trial: a comparison of first-line combination therapies

Although multidrug therapy is required in order to achieve good blood pressure control in many hypertensives, there are no studies directly comparing fixed-dose combinations as initial therapy. The Avoiding Cardiovascular events through COMbination therapy in Patients Living with Systolic Hypertension (ACCOMPLISH) trial compares regimens of benazepril plus amlodipine versus benazepril plus hydrochlorothiazide, force-titrated to 40/10 and 40/25mg, respectively. A total of 12,600 high-risk hypertensives have been randomised and will be followed for 3 - 5years, during which cardiovascular events will be monitored. The investigators hypothesise that the benazepril plus amlodipine regimen will decrease cardiovascular events by 15% compared with benazepril plus hydrochlorothiazide. Recruitment began in 2003, and the trial is expected to end in 2008. The ACCOMPLISH trial shares important limitations with many other recent trials that will make it difficult to apply the results in clinical practice. These include the focus on high-risk hypertensive patients, in whom significant reductions in relative risk will translate into meaningful reductions in absolute risk: in lower-risk hypertensives with a low absolute risk, similar relative risk reductions may not be of great impact on the population disease burden. In ACCOMPLISH, as in most industry-sponsored clinical trials, the main goal appears to be market-driven: doses of drugs tested are not those available for clinical practice. The question asked, whether the combination of benazepril with either diuretic or dihydropyridine calcium channel blocker is more efficacious, is not a clinically compelling one. Finally, the univariate subgroup analyses proposed are unlikely to lead to an understanding of whether either combination has specific advantages for patients encountered clinically, most of whom have multiple risk factors. Thus, it appears that ACCOMPLISH, as with many recent pharmacological trials, will not greatly impact the treatment of hypertension.

Treating individuals 2. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation

Large pragmatic trials provide the most reliable data about the effects of treatments, but should be designed, analysed, and reported to enable the most effective use of treatments in routine practice. Subgroup analyses are important if there are potentially large differences between groups in the risk of a poor outcome with or without treatment, if there is potential heterogeneity of treatment effect in relation to pathophysiology, if there are practical questions about when to treat, or if there are doubts about benefit in specific groups, such as elderly people, which are leading to potentially inappropriate undertreatment. Analyses must be predefined, carefully justified, and limited to a few clinically important questions, and post-hoc observations should be treated with scepticism irrespective of their statistical significance. If important subgroup effects are anticipated, trials should either be powered to detect them reliably or pooled analyses of several trials should be undertaken. Formal rules for the planning, analysis, and reporting of subgroup analyses are proposed.

Treatment outcomes after highly active antiretroviral therapy: a meta-analysis of randomised controlled trials

This systematic review summarises the evidence for treatment efficacy and tolerability of highly active antiretroviral therapies containing two nucleoside reverse transcriptase inhibitors (NRTI) with a protease inhibitor (PI), compared with two NRTIs alone for the treatment of HIV-1 infection in randomised controlled trials. Three electronic databases (Medline, Embase, and the Cochrane Library) were searched up to December 2003. 16 randomised controlled trials met the inclusion criteria and were included in the analysis from 328 articles screened. The pooled analysis indicated that treatment with two NRTIs with a PI is more effective in achieving viral suppression than two NRTIs alone (relative risk [RR] 3.44, 95% confidence interval [CI] 2.43-4.87). However, the RR for discontinuation of treatment due to adverse events of treatment with two NRTIs with a PI compared with two NRTIs alone was 1.81 (95% CI 1.17-2.79). The benefits of treatment with two NRTIs and a PI are substantial among those who can tolerate the regimen in comparison with treatment with two NRTIs alone.

The relationships of vigorous exercise, alcohol, and adiposity to low and high high-density lipoprotein-cholesterol levels

While vigorous exercise, alcohol, and weight loss are all known to increase high-density lipoprotein-cholesterol (HDL-C), it is not known whether these interventions increase low HDL as effectively as has been demonstrated for normal HDL. This report tests the hypothesis that there may be differences in the calculated response of men and women with low versus high HDL-C to exercise, alcohol, and weight loss across the spectrum of HDL-C levels. Physican-supplied medical data from 7,288 men and 2,326 women were divided into deciles of self-reported vigorous exercise, alcohol intake, body mass index (BMI), or body circumferences. Within each decile we determined the percentiles of the HDL distributions and average running distance, alcohol intake, BMI, or body circumference. Simple least-squares regression analysis was then used to estimate the slope for kth HDL percentile (k = 5%, 6%, ...,95%) versus running distance, alcohol intake, BMI, or body circumference across deciles. Bootstrap resampling was used to estimate standard errors and statistical significance for the regression lines. In both sexes, the increase in HDL-C per unit alcohol intake was at least twice as great at the 95th as at the 5th percentile of the HDL distribution. There was also a significant graded increase from the 5th to the 95th HDL percentile for the slopes relating HDL to exercise (km run) and alcohol intake. Men's HDL-C declined in association with fatness (BMI, waist, and chest circumference) more sharply at the 95th than at the 5th percentile of the HDL distribution. The results of this study suggest that the effects of physical activity, alcohol, and weight reduction on HDL-C levels may be, to a large extent, dependent on the initial level with the greatest improvement achieved in subjects with high HDL and the least improvement in those having low HDL-C levels.

Vigorous exercise and the population distribution of body weight

BACKGROUND

While the benefits of vigorous exercise on body weight and regional adiposity are well established, whether these benefits affect equally the highest and lowest portions of the weight distribution have not been previously reported. The impact of exercise on the more extreme body weights and body circumferences is clinically important because these values represent individuals at greatest health risk.

METHOD

Self-reported weights and body circumferences from a cross-sectional sample of 7288 male and 2326 female runners were divided into five strata, according to the distances run per week and within each stratum the 5th, 10th, 25th, 50th, 75th, 90th and 95th percentiles were determined. Least-squares regression was then employed at each percentile to determine the dose-response relationship between running distance and adiposity as determined by body mass index (BMI) and self-reported circumferences of the waist, hip and chest.

RESULTS

Per kilometer run per week, the associated decline for BMI was three-fold greater at the 95th than at the 5th percentile in men, and six-fold greater at the 95th than the 5th percentile in women (all P<0.001). Reported waist circumference also declined more sharply at the 95th percentile than at the 5th percentile in men (-0.13 +/- 0.02 vs -0.06 +/- 0.01 cm per km/week) and women (-0.18 +/- 0.04 vs -0.05 +/- 0.01 cm per km/week). In women, both hip and chest circumferences declined more sharply per kilometer run at the 95th percentile than at the 5th percentile.

CONCLUSIONS

These results are consistent with the hypothesis that running promotes the greatest weight loss specifically in those individuals who have the most to gain from losing weight. Comparisons based on average BMI or average body circumferences are likely to underestimate the health benefits of running because of the J-shaped relationship between adiposity and mortality. Whether the observed cross-sectional associations are primarily due to exercise-induced weight loss or self-selection remains to be determined.

Randomised trials comparing chemotherapy regimens for advanced non-small cell lung cancer: biases and evolution over time

We systematically evaluated the evidence from randomised trials comparing various chemotherapy regimens for advanced non-small cell lung cancer. Across 254 eligible trials (42661 patients), no regimens were compared in >6 studies. Twenty-six trials (10%) found statistically significant differences in survival between the compared arms. Only five reported the randomisation mode, and four reported adequate allocation concealment; nine performed unaccounted interim analyses. Statistical significance was more common in larger (P=0.003), more recent studies (P=0.031), and trials from countries with only one published eligible study (P=0.008). Increased reported median survival was independently associated with platinum and/or taxane and combination regimens, but also with the year of publication, smaller sample size, and larger representation of non-stage IV patients and patients with a better performance status. The proportion of enrolled patients with a performance status of 2 or worse decreased significantly over time (12.9% per decade, P<0.001). Randomised evidence in this field is fragmented and subject to considerable selection biases.