Inconsistent reporting of surrogate outcomes in randomised clinical trials: cohort study

Comparison of Publications on Heated Tobacco Products With Conventional Cigarettes and Implied Desirability of the Products According to Tobacco Industry Affiliation: A Systematic Review

INTRODUCTION

Heated tobacco products (HTPs) have been advertised as "reduced-harm" tobacco products compared to conventional cigarettes (CC); however, no direct evidence supporting HTPs being desirable for human health exists. A previous systematic review reported that evidence on HTPs published in 2017 or earlier was primarily drawn from industry-related papers. We aimed to investigate whether tobacco industry-affiliated studies are more likely to conclude that HTPs are more desirable than CC.

METHODS

PubMed and Ichushi-Web were searched up to March 15, 2022, for studies on HTPs published in 2017 or after. We selected studies that assessed any measures of HTPs and CC, including secondary analyses using gray literature in English or Japanese. We excluded review articles except for a meta-analysis that met the aforementioned criteria. Data on the authors' affiliations, grant, conflict of interest, category of research subjects, and interpretation were extracted. Research members in two groups independently assessed the papers; discrepancies were solved by discussion between the groups.

RESULTS

Overall, 134 studies met the criteria. Eighty-seven (64.9%) of them were affiliated with the tobacco industry. Of the 134 studies, 56.3% (49/87) of the industry-affiliated studies versus 19.1% (9/47) of nonindustry-affiliated studies concluded that HTPs were more desirable than CC (p < .01). No study investigated clinically relevant outcomes, such as disease occurrence.

CONCLUSIONS

Publications on HTPs in the biomedical literature from January 2017 to March 2022 were dominated by tobacco industry-affiliated studies. More than half of them concluded that HTPs were more desirable than CC compared to independent studies.

IMPLICATIONS

Tobacco industry advertises HTPs as "reduced-harm" tobacco products compared to CC. HTP users tend to consider HTPs as alternative tobacco products less harmful than CC (ie, products for "harm reduction"). Our results demonstrated that papers written by tobacco industry-affiliated authors concluded that HTPs were more desirable than CC compared to papers by independent authors. However, all their judgments were based on surrogate outcomes. Surrogate outcomes are not necessarily linked to clinically relevant outcomes such as disease occurrence. Further studies on HTPs using clinically relevant outcomes are warranted by independent authors from tobacco industry.

A framework for the definition and interpretation of the use of surrogate endpoints in interventional trials

Background

Interventional trials that evaluate treatment effects using surrogate endpoints have become increasingly common. This paper describes four linked empirical studies and the development of a framework for defining, interpreting and reporting surrogate endpoints in trials.

Methods

As part of developing the CONSORT (Consolidated Standards of Reporting Trials) and SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) extensions for randomised trials reporting surrogate endpoints, we undertook a scoping review, e-Delphi study, consensus meeting, and a web survey to examine current definitions and stakeholder (including clinicians, trial investigators, patients and public partners, journal editors, and health technology experts) interpretations of surrogate endpoints as primary outcome measures in trials.

Findings

Current surrogate endpoint definitional frameworks are inconsistent and unclear. Surrogate endpoints are used in trials as a substitute of the treatment effects of an intervention on the target outcome(s) of ultimate interest, events measuring how patients feel, function, or survive. Traditionally the consideration of surrogate endpoints in trials has focused on biomarkers (e.g., HDL cholesterol, blood pressure, tumour response), especially in the medical product regulatory setting. Nevertheless, the concept of surrogacy in trials is potentially broader. Intermediate outcomes that include a measure of function or symptoms (e.g., angina frequency, exercise tolerance) can also be used as substitute for target outcomes (e.g., all-cause mortality)-thereby acting as surrogate endpoints. However, we found a lack of consensus among stakeholders on accepting and interpreting intermediate outcomes in trials as surrogate endpoints or target outcomes. In our assessment, patients and health technology assessment experts appeared more likely to consider intermediate outcomes to be surrogate endpoints than clinicians and regulators.

Interpretation

There is an urgent need for better understanding and reporting on the use of surrogate endpoints, especially in the setting of interventional trials. We provide a framework for the definition of surrogate endpoints (biomarkers and intermediate outcomes) and target outcomes in trials to improve future reporting and aid stakeholders' interpretation and use of trial surrogate endpoint evidence.

Funding

SPIRIT-SURROGATE/CONSORT-SURROGATE project is Medical Research Council Better Research Better Health (MR/V038400/1) funded.

Definitions, acceptability, limitations, and guidance in the use and reporting of surrogate end points in trials: a scoping review

OBJECTIVE

To synthesize the current literature on the use of surrogate end points, including definitions, acceptability, and limitations of surrogate end points and guidance for their design/reporting, into trial reporting items.

STUDY DESIGN AND SETTING

Literature was identified through searching bibliographic databases (until March 1, 2022) and gray literature sources (until May 27, 2022). Data were thematically analyzed into four categories: (1) definitions, (2) acceptability, (3) limitations and challenges, and (4) guidance, and synthesized into reporting guidance items.

RESULTS

After screening, 90 documents were included: 79% (n = 71) had data on definitions, 77% (n = 69) on acceptability, 72% (n = 65) on limitations and challenges, and 61% (n = 55) on guidance. Data were synthesized into 17 potential trial reporting items: explicit statements on the use of surrogate end point(s) and justification for their use (items 1-6); methodological considerations, including whether sample size calculations were informed by surrogate validity (items 7-9); reporting of results for composite outcomes containing a surrogate end point (item 10); discussion and interpretation of findings (items 11-14); plans for confirmatory studies, collecting data on the surrogate end point and target outcome, and data sharing (items 15-16); and informing trial participants about using surrogate end points (item 17).

CONCLUSION

The review identified and synthesized items on the use of surrogate end points in trials; these will inform the development of the Standard Protocol Items: Recommendations for Interventional Trials-SURROGATE and Consolidated Standards of Reporting Trials-SURROGATE extensions.

Surrogate endpoints in trials: a call for better reporting

Using a surrogate endpoint as a substitute for a patient-relevant final outcome enables randomised controlled trials (RCTs) to be conducted more efficiently. However, the use of surrogates remains controversial and there is currently no guideline for the reporting of RCTs using surrogate endpoints; therefore, we seek to develop SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) extensions to improve the reporting of these trials. We would like to invite interested individuals (trial methodologists, journal editors, healthcare industry, regulators and payers, and patient/public representative groups), particularly those with experience in the use of surrogate endpoints in trials.

Scoping and targeted reviews to support development of SPIRIT and CONSORT extensions for randomised controlled trials with surrogate primary endpoints: protocol

INTRODUCTION

Using a surrogate endpoint as a substitute for a primary patient-relevant outcome enables randomised controlled trials (RCTs) to be conducted more efficiently, that is, with shorter time, smaller sample size and lower cost. However, there is currently no consensus-driven guideline for the reporting of RCTs using a surrogate endpoint as a primary outcome; therefore, we seek to develop SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) extensions to improve the design and reporting of these trials. As an initial step, scoping and targeted reviews will identify potential items for inclusion in the extensions and participants to contribute to a Delphi consensus process.

METHODS AND ANALYSIS

The scoping review will search and include literature reporting on the current understanding, limitations and guidance on using surrogate endpoints in trials. Relevant literature will be identified through: (1) bibliographic databases; (2) grey literature; (3) handsearching of reference lists and (4) solicitation from experts. Data from eligible records will be thematically analysed into potential items for inclusion in extensions. The targeted review will search for RCT reports and protocols published from 2017 to 2021 in six high impact general medical journals. Trial corresponding author contacts will be listed as potential participants for the Delphi exercise.

ETHICS AND DISSEMINATION

Ethical approval is not required. The reviews will support the development of SPIRIT and CONSORT extensions for reporting surrogate primary endpoints (surrogate endpoint as the primary outcome). The findings will be published in open-access publications.This review has been prospectively registered in the OSF Registration DOI: 10.17605/OSF.IO/WP3QH.

Protocol for the development of SPIRIT and CONSORT extensions for randomised controlled trials with surrogate primary endpoints: SPIRIT-SURROGATE and CONSORT-SURROGATE

INTRODUCTION

Randomised controlled trials (RCTs) may use surrogate endpoints as substitutes and predictors of patient-relevant/participant-relevant final outcomes (eg, survival, health-related quality of life). Translation of effects measured on a surrogate endpoint into health benefits for patients/participants is dependent on the validity of the surrogate; hence, more accurate and transparent reporting on surrogate endpoints is needed to limit misleading interpretation of trial findings. However, there is currently no explicit guidance for the reporting of such trials. Therefore, we aim to develop extensions to the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines to improve the design and completeness of reporting of RCTs and their protocols using a surrogate endpoint as a primary outcome.

METHODS AND ANALYSIS

The project will have four phases: phase 1 (literature reviews) to identify candidate reporting items to be rated in a Delphi study; phase 2 (Delphi study) to rate the importance of items identified in phase 1 and receive suggestions for additional items; phase 3 (consensus meeting) to agree on final set of items for inclusion in the extensions and phase 4 (knowledge translation) to engage stakeholders and disseminate the project outputs through various strategies including peer-reviewed publications. Patient and public involvement will be embedded into all project phases.

ETHICS AND DISSEMINATION

The study has received ethical approval from the University of Glasgow College of Medical, Veterinary and Life Sciences Ethics Committee (project no: 200210051). The findings will be published in open-access peer-reviewed publications and presented in conferences, meetings and relevant forums.

Getting it wrong most of the time? Comparing trialists' choice of primary outcome with what patients and health professionals want

Background

Randomised trials support improved decision-making through the data they collect. One important piece of data is the primary outcome — so called because it is what the investigators decide is the most important. Secondary outcomes provide additional information to support decision-making. We were interested in knowing how important patients and healthcare professionals consider the outcomes (especially the primary outcome) measured in a selection of published trials.

Methods

The work had three stages: (1) We identified a body of late-stage trials in two clinical areas, breast cancer management and nephrology. (2) We identified the primary and secondary outcomes for these trials. (3) We randomly ordered these outcomes and presented them to patients and healthcare professionals (with experience of the clinical area), and we asked them to rank the importance of the outcomes. They were not told which outcomes trial authors considered primary and secondary.

Results

In our sample of 44 trials with 46 primary outcomes, 29 patients, one patient representative and 12 healthcare professionals together ranked the primary outcome as the most important outcome 13/46 times or 28%. Breast cancer patients and healthcare professionals considered the primary outcome to be the most important outcome for 8/21 primary outcomes chosen by trialists. For nephrology, the equivalent figure was 5/25. The primary outcome appeared in a respondent’s top 5 ranked outcomes 151/178 (85%) times for breast cancer and 225/259 (87%) times for nephrology even if the primary was not considered the most important outcome.

Conclusions

The primary outcome in a trial is the most important piece of data collected. It is used to determine how many participants are required, and it is the main piece of information used to judge whether the intervention is effective or not. In our study, patients and healthcare professionals agreed with the choice of the primary outcome made by trial teams doing late-stage trials in breast cancer management and nephrology 28% of the time.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-022-06348-z.

Effects of Septal Reduction Therapy on Acute Cardiovascular Events and All-Cause Mortality in Patients with Hypertrophic Cardiomyopathy

Septal reduction therapy (SRT) -i.e. septal myectomy and alcohol septal ablation-has been performed to treat medically refractory hypertrophic cardiomyopathy (HCM) for decades. However, it is largely unknown whether SRT prevents HCM-related cardiovascular events or death. The objective was to examine the effects of SRT on acute cardiovascular events and all-cause mortality in HCM. We performed a propensity score (PS) -matched study using databases that capture all hospitalizations and outpatient visits in New York state. We identified patients with HCM who underwent SRT between 2007 and 2014 (i.e. the SRT group) and those who had never had SRT but had at least one hospitalization for HCM during the same period (i.e. the control group). We performed PS matching at a 1:1 ratio. The primary outcome was a composite of acute cardiovascular events and all-cause mortality during 0-180 days and 181-360 days. The secondary outcome was 180- and 360-day all-cause mortality. We included 846 patients with HCM (423 PS-matched pairs). Patients who underwent SRT had a lower risk of the primary outcome event (0-180 days: odds ratio [OR], 0.54; 95% confidence intervals (CI), 0.37-0.80; P = 0.002 and 181-360 days: OR, 0.33; 95% CI, 0.22-0.51; P < 0.0001). Furthermore, the risk of all-cause mortality was lower at 180 days (OR, 0.37; 95% CI, 0.22-0.63; P = 0.0003) and 360 days post-SRT (OR, 0.32; 95% CI, 0.20-0.51; P < 0.0001). In conclusion, our PS-matched study using population-based datasets demonstrated that SRT was associated with a reduced risk of a composite of acute cardiovascular events and all-cause mortality in HCM during the first post-SRT year.

South Australian Medicines Evaluation Panel in review: providing evidence-based guidance on the use of high-cost medicines in the South Australian public health system

Objective The South Australian Medicines Evaluation Panel (SAMEP) was established in 2011 to make evidence-based recommendations on the funding of high-cost medicines in South Australian public hospitals via a high-cost medicines formulary. SAMEP represents one component of South Australia's process for state-based health technology assessment (HTA). The aim of this study was to describe the experience of SAMEP in the context of Australia's complex governance model for hospital-based care. Methods A retrospective review was conducted of the SAMEP process and outcomes of medicine evaluations. Decision summaries and meeting minutes were reviewed and reflected upon by the authors to explore the views of the SAMEP membership regarding the function of the committee and state-based HTA more broadly. Results SAMEP has reviewed 29 applications, with 14 (48%) listed on the high-cost medicines formulary. Three applications have been the subject of outcome review and confirm expectations of patient benefit. Conclusion Retrospective review of the committee experience suggests that state-based HTA as operationalised by SAMEP is feasible, provides greater equity of access to high-cost medicines in the South Australian public hospital system and allows for access with evidence development. What is known about the topic? State-based hospital funders often need to make decisions on the provision of high-cost medicines for which there is no national guidance or subsidy. Little published information exists about state-based approaches to medicines evaluation and reimbursement within public hospitals in Australia. What does this paper add? The South Australian experience demonstrates a method for states and territories to tackle the challenges of providing evidence-based access to high-cost medicines in Australian public hospitals. What are the implications for practitioners? This paper provides information for other jurisdictions considering state-based approaches to medicines evaluation and contributes to the broader literature about state-based HTA in Australia.

Surrogate Endpoints in Oncology: Overview of Systematic Reviews and Their Use for Health Decision Making in Mexico

OBJECTIVES

The use of surrogate endpoints (SEs) for cancer drug approval in health systems is common. The objectives of this study were to identify systematic reviews (SRs) that evaluated the correlation of SEs with overall survival (OS) in cancer drugs to analyze the applications of approved cancer drugs with SEs in Mexico and to apply the validation framework proposed by the Institute for Quality and Efficiency in Health Care (IQWiG).

METHODS

An overview of SRs was conducted according to Cochrane Collaboration methodology. Applications for approved cancer drugs with SEs in Mexico were analyzed. The IQWiG validation framework was applied to evaluate the SEs identified in the overview and in the applications in Mexico.

RESULTS

A total of 85 SRs that assessed 192 SEs for different types of cancer were selected. According to the IQWiG model, only 2.5% of the SEs analyzed in the overview and only one of the applications in Mexico could be used as surrogates for OS because the reliability (methodological quality) of the SRs and the strength of the correlation of SEs with OS was mostly low (92%) and low (correlation coefficient r ≤ 0.7; 50.5%), respectively. Of the total number of cancer drugs approved in Mexico, 19.4% used SEs.

CONCLUSIONS

Most SEs for different types of cancer could not be used as surrogates for OS according to the IQWiG model, and their use for the approval of cancer drugs in Mexico is generally not justified.

Outcomes, Interventions and Funding in Randomised Research Published in High-Impact Journals

Background

Randomised clinical trials are pivotal in guiding clinical practice. Trials with surrogate outcomes and industry sponsorship might be less reliable than those with hard outcomes and independent sponsorship. The types of interventions evaluated in randomised clinical trials might not reflect the diversity of those employed in clinical practice.

Methods

We assessed the types of primary outcome, types of intervention and sponsorship of 360 randomised clinical trials evaluating 416 interventions, published in seven major general medical journals and 10 speciality medical journals in five clinical disciplines.

Results

Primary outcomes were surrogate in 223/360 (62%) trials. Neither type of journal nor source of sponsorship was associated with type of primary outcome. Among the interventions evaluated, 233/416 (56%) were drugs, 17/416 (4%) devices and 49/416 (12%) procedures. The majority of trials were non-industry funded (220/360, 61%). Trials of drug interventions and those with industry sponsorship were more common in specialty than general journals (68% vs 48% and 55% vs 25%, respectively). Industry sponsorship was not associated with results for the primary outcome but was strongly associated with trials of drugs and devices.

Within the groups of both general and speciality journals, there were wide ranges in the prevalence of industry funding (7–63% and 37–70%, respectively), but in all cases the prevalence of hard primary outcomes was <40%.

Conclusions

Most randomised clinical trials published in influential journals reported surrogate primary outcomes and assessed drug interventions. Trials that evaluated devices and procedures were infrequently published, despite the prominence of each type of intervention in clinical practice. Industry funding was more common for trials published in specialty than general journals but was not associated with more positive results for primary outcomes or with a greater preponderance of surrogate outcomes.

Factors associated with clinical trials that fail and opportunities for improving the likelihood of success: A review

Clinical trials are time consuming, expensive, and often burdensome on patients. Clinical trials can fail for many reasons. This survey reviews many of these reasons and offers insights on opportunities for improving the likelihood of creating and executing successful clinical trials. Literature from the past 30 years was reviewed for relevant data. Common patterns in reported successful trials are identified, including factors regarding the study site, study coordinator/investigator, and the effects on participating patients. Specific instances where artificial intelligence can help improve clinical trials are identified.

The assessment and appraisal of regenerative medicines and cell therapy products: an exploration of methods for review, economic evaluation and appraisal

BACKGROUND

The National Institute for Health and Care Excellence (NICE) commissioned a 'mock technology appraisal' to assess whether changes to its methods and processes are needed. This report presents the findings of independent research commissioned to inform this appraisal and the deliberations of a panel convened by NICE to evaluate the mock appraisal.

METHODS

Our research included reviews to identify issues, analysis methods and conceptual differences and the relevance of alternative decision frameworks, alongside the development of an exemplar case study of chimeric antigen receptor (CAR) T-cell therapy for treating acute lymphoblastic leukaemia.

RESULTS

An assessment of previous evaluations of regenerative medicines found that, although there were a number of evidential challenges, none was unique to regenerative medicines or was beyond the scope of existing methods used to conceptualise decision uncertainty. Regarding the clinical evidence for regenerative medicines, the issues were those associated with a limited evidence base but were not unique to regenerative medicines: small non-randomised studies, high variation in response and the intervention subject to continuing development. The relative treatment effects generated from single-arm trials are likely to be optimistic unless it is certain that the historical data have accurately estimated the efficacy of the control agent. Pivotal trials may use surrogate end points, which, on average, overestimate treatment effects. To reduce overall uncertainty, multivariate meta-analysis of all available data should be considered. Incorporating indirectly relevant but more reliable (more mature) data into the analysis can also be considered; such data may become available as a result of the evolving regulatory pathways being developed by the European Medicines Agency. For the exemplar case of CAR T-cell therapy, target product profiles (TPPs) were developed, which considered the 'curative' and 'bridging to stem-cell transplantation' treatment approaches separately. Within each TPP, three 'hypothetical' evidence sets (minimum, intermediate and mature) were generated to simulate the impact of alternative levels of precision and maturity in the clinical evidence. Subsequent assessments of cost-effectiveness were undertaken, employing the existing NICE reference case alongside additional analyses suggested within alternative frameworks. The additional exploratory analyses were undertaken to demonstrate how assessments of cost-effectiveness and uncertainty could be impacted by alternative managed entry agreements (MEAs), including price discounts, performance-related schemes and technology leasing. The panel deliberated on the range of TPPs, evidence sets and MEAs, commenting on the likely recommendations for each scenario. The panel discussed the challenges associated with the exemplar and regenerative medicines more broadly, focusing on the need for a robust quantification of the level of uncertainty in the cost-effective estimates and the potential value of MEAs in limiting the exposure of the NHS to high upfront costs and loss associated with a wrong decision.

CONCLUSIONS

It is to be expected that there will be a significant level of uncertainty in determining the clinical effectiveness of regenerative medicines and their long-term costs and benefits, but the existing methods available to estimate the implications of this uncertainty are sufficient. The use of risk sharing and MEAs between the NHS and manufacturers of regenerative medicines should be investigated further.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Intervention studies to foster resilience - A systematic review and proposal for a resilience framework in future intervention studies

Psychological resilience refers to the phenomenon that many people are able to adapt to the challenges of life and maintain mental health despite exposure to adversity. This has stimulated research on training programs to foster psychological resilience. We evaluated concepts, methods and designs of 43 randomized controlled trials published between 1979 and 2014 which assessed the efficacy of such training programs and propose standards for future intervention research based on recent developments in the field. We found that concepts, methods and designs in current resilience intervention studies are of limited use to properly assess efficacy of interventions to foster resilience. Major problems are the use of definitions of resilience as trait or a composite of resilience factors, the use of unsuited assessment instruments, and inappropriate study designs. To overcome these challenges, we propose 1) an outcome-oriented definition of resilience, 2) an outcome-oriented assessment of resilience as change in mental health in relation to stressor load, and 3) methodological standards for suitable study designs of future intervention studies. Our proposals may contribute to an improved quality of resilience intervention studies and may stimulate further progress in this growing research field.

Scientific Opinion of the PPR Panel on the follow-up of the findings of the External Scientific Report 'Literature review of epidemiological studies linking exposure to pesticides and health effects'

In 2013, EFSA published a comprehensive systematic review of epidemiological studies published from 2006 to 2012 investigating the association between pesticide exposure and many health outcomes. Despite the considerable amount of epidemiological information available, the quality of much of this evidence was rather low and many limitations likely affect the results so firm conclusions cannot be drawn. Studies that do not meet the 'recognised standards' mentioned in the Regulation (EU) No 1107/2009 are thus not suited for risk assessment. In this Scientific Opinion, the EFSA Panel on Plant Protection Products and their residues (PPR Panel) was requested to assess the methodological limitations of pesticide epidemiology studies and found that poor exposure characterisation primarily defined the major limitation. Frequent use of case-control studies as opposed to prospective studies was considered another limitation. Inadequate definition or deficiencies in health outcomes need to be avoided and reporting of findings could be improved in some cases. The PPR Panel proposed recommendations on how to improve the quality and reliability of pesticide epidemiology studies to overcome these limitations and to facilitate an appropriate use for risk assessment. The Panel recommended the conduct of systematic reviews and meta-analysis, where appropriate, of pesticide observational studies as useful methodology to understand the potential hazards of pesticides, exposure scenarios and methods for assessing exposure, exposure-response characterisation and risk characterisation. Finally, the PPR Panel proposed a methodological approach to integrate and weight multiple lines of evidence, including epidemiological data, for pesticide risk assessment. Biological plausibility can contribute to establishing causation.

Outcome switching in randomized controlled oncology trials reporting on surrogate endpoints: a cross-sectional analysis

Inconsistent reporting of clinical trials is well-known in the literature. Despite this, factors associated with poor practice such as outcome switching in clinical trials are poorly understood. We performed a cross-sectional analysis to evaluate the prevalence of, and the factors associated with outcome switching. PubMed and Embase were searched for pharmaceutical randomized controlled trials (RCTs) in oncology reporting on a surrogate primary outcome published in 2015. Outcome switching was present in 18% (39/216). First-author male sex was significantly more likely associated with outcome switching compared to female sex with an OR of 3.05 (95% CI 1.07-8.64, p = 0.04) after multivariable adjustment. For-profit funded RCTs were less likely associated with outcome switching compared to non-profit funded research with an OR of 0.22 (95% CI 0.07-0.74, p = 0.01). First author male sex was more likely associated with outcome switching compared to female sex in drug oncology RCTs reporting on a primary surrogate endpoint. For-profit funded research was less likely associated with outcome switching compared to research funded by non-profit organizations. Furthermore, 18 percent of drug oncology trials reporting on a surrogate endpoint could have a higher risk of false positive results due to primary outcome switching.

Self-declared stock ownership and association with positive trial outcome in randomized controlled trials with binary outcomes published in general medical journals: a cross-sectional study

Background

Describe the prevalence and types of conflicts of interest (COI) in published randomized controlled trials (RCTs) in general medical journals with a binary primary outcome and assess the association between conflicts of interest and favorable outcome.

Methods

Parallel-group RCTs with a binary primary outcome published in three general medical journals during 2013–2015 were identified. COI type, funding source, and outcome were extracted. Binomial logistic regression model was performed to assess association between COI and funding source with outcome.

Results

A total of 509 consecutive parallel-group RCTs were included in the study. COI was reported in 74% in mixed funded RCTs and in 99% in for-profit funded RCTs. Stock ownership was reported in none of the non-profit RCTs, in 7% of mixed funded RCTs, and in 50% of for-profit funded RCTs. Mixed-funded RCTs had employees from the funding company in 11% and for-profit RCTs in 76%. Multivariable logistic regression revealed that stock ownership in the funding company among any of the authors was associated with a favorable outcome (odds ratio = 3.53; 95% confidence interval = 1.59–7.86; p < 0.01).

Conclusion

COI in for-profit funded RCTs is extensive, because the factors related to COI are not fully independent, a multivariable analysis should be cautiously interpreted. However, after multivariable adjustment only stock ownership from the funding company among authors is associated with a favorable outcome.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-2108-z) contains supplementary material, which is available to authorized users.

Two Decades of Cardiovascular Trials With Primary Surrogate Endpoints: 1990-2011

Background

Surrogate endpoint trials test strategies more efficiently but are accompanied by uncertainty about the relationship between changes in surrogate markers and clinical outcomes.

Methods and Results

We identified cardiovascular trials with primary surrogate endpoints published in the New England Journal of Medicine, Lancet, and JAMA: Journal of the American Medical Association from 1990 to 2011 and determined the trends in publication of surrogate endpoint trials and the success of the trials in meeting their primary endpoints. We tracked for publication of clinical outcome trials on the interventions tested in surrogate trials. We screened 3016 articles and identified 220 surrogate endpoint trials. From the total of 220 surrogate trials, 157 (71.4%) were positive for their primary endpoint. Only 59 (26.8%) surrogate trials had a subsequent clinical outcomes trial. Among these 59 trials, 24 outcomes trial results validated the positive surrogates, whereas 20 subsequent outcome trials were negative following positive results on a surrogate. We identified only 3 examples in which the surrogate trial was negative but a subsequent outcomes trial was conducted and showed benefit. Findings were consistent in a sample cohort of 383 screened articles inclusive of 37 surrogate endpoint trials from 6 other high‐impact journals.

Conclusions

Although cardiovascular surrogate outcomes trials frequently show superiority of the tested intervention, they are infrequently followed by a prominent outcomes trial. When there was a high‐profile clinical outcomes study, nearly half of the positive surrogate trials were not validated. Cardiovascular surrogate outcome trials may be more appropriate for excluding benefit from the patient perspective than for identifying it.

Why clinical trial outcomes fail to translate into benefits for patients

Clinical research should ultimately improve patient care. For this to be possible, trials must evaluate outcomes that genuinely reflect real-world settings and concerns. However, many trials continue to measure and report outcomes that fall short of this clear requirement. We highlight problems with trial outcomes that make evidence difficult or impossible to interpret and that undermine the translation of research into practice and policy. These complex issues include the use of surrogate, composite and subjective endpoints; a failure to take account of patients' perspectives when designing research outcomes; publication and other outcome reporting biases, including the under-reporting of adverse events; the reporting of relative measures at the expense of more informative absolute outcomes; misleading reporting; multiplicity of outcomes; and a lack of core outcome sets. Trial outcomes can be developed with patients in mind, however, and can be reported completely, transparently and competently. Clinicians, patients, researchers and those who pay for health services are entitled to demand reliable evidence demonstrating whether interventions improve patient-relevant clinical outcomes.

Why clinical trial outcomes fail to translate into benefits for patients

Clinical research should ultimately improve patient care. For this to be possible, trials must evaluate outcomes that genuinely reflect real-world settings and concerns. However, many trials continue to measure and report outcomes that fall short of this clear requirement. We highlight problems with trial outcomes that make evidence difficult or impossible to interpret and that undermine the translation of research into practice and policy. These complex issues include the use of surrogate, composite and subjective endpoints; a failure to take account of patients' perspectives when designing research outcomes; publication and other outcome reporting biases, including the under-reporting of adverse events; the reporting of relative measures at the expense of more informative absolute outcomes; misleading reporting; multiplicity of outcomes; and a lack of core outcome sets. Trial outcomes can be developed with patients in mind, however, and can be reported completely, transparently and competently. Clinicians, patients, researchers and those who pay for health services are entitled to demand reliable evidence demonstrating whether interventions improve patient-relevant clinical outcomes.

Time to Review the Role of Surrogate End Points in Health Policy: State of the Art and the Way Forward

The efficacy of medicines, medical devices, and other health technologies should be proved in trials that assess final patient-relevant outcomes such as survival or morbidity. Market access and coverage decisions are, however, often based on surrogate end points, biomarkers, or intermediate end points, which aim to substitute and predict patient-relevant outcomes that are unavailable because of methodological, financial, or practical constraints. We provide a summary of the present use of surrogate end points in health care policy, discussing the case for and against their adoption and reviewing validation methods. We introduce a three-step framework for policymakers to handle surrogates, which involves establishing the level of evidence, assessing the strength of the association, and quantifying relations between surrogates and final outcomes. Although the use of surrogates can be problematic, they can, when selected and validated appropriately, offer important opportunities for more efficient clinical trials and faster access to new health technologies that benefit patients and health care systems.

Trends in Utilization of Surrogate Endpoints in Contemporary Cardiovascular Clinical Trials

Surrogate endpoints facilitate trial efficiency but are variably linked to clinical outcomes, and limited data are available exploring their utilization in cardiovascular clinical trials over time. We abstracted data regarding primary clinical, intermediate, and surrogate endpoints from all phase II to IV cardiovascular clinical trials from 2001 to 2012 published in the 8 highest Web of Science impact factor journals. Two investigators independently classified the type of primary endpoint. Of the 1,224 trials evaluated, 677 (55.3%) primary endpoints were clinical, 165 (13.5%) intermediate, and 382 (31.2%) surrogate. The relative proportions of these endpoints remained constant over time (p = 0.98). Trials using surrogate endpoints were smaller (187 vs 1,028 patients) and enrolled patients more expeditiously (1.4 vs 0.9 patients per site per month) compared with trials using clinical endpoints (p <0.001 for both comparisons). Surrogate endpoint trials were independently more likely to meet their primary endpoint compared to trials with clinical endpoints (adjusted odds ratio 1.56, 95% CI 1.05 to 2.34; p = 0.03). Rates of positive results in clinical endpoint trials have decreased over time from 66.1% in 2001 to 2003 to 47.2% in 2010 to 2012 (p = 0.001), whereas these rates have remained stable over the same period for surrogate (72.0% to 69.3%, p = 0.27) and intermediate endpoints (74.4% to 71.4%, p = 0.98). In conclusion, approximately a third of contemporary cardiovascular trials use surrogate endpoints. These trials are completed more expeditiously and are more likely to meet their primary outcomes. The overall scientific contribution of these surrogate endpoint trials requires further attention given their variable association with definitive outcomes.

Biomarkers in Pharmaceutical Research

BACKGROUND

Biomarkers are important tools in drug development and are used throughout pharmaceutical research.

CONTENT

This review focuses on molecular biomarkers in drug development. It contains sections on how biomarkers are used to assess target engagement, pharmacodynamics, safety, and proof-of-concept. It also covers the use of biomarkers as surrogate end points and patient selection/companion diagnostics and provides insights into clinical biomarker discovery and biomarker development/validation with regulatory implications. To survey biomarkers used in drug development--acknowledging that many pharmaceutical development biomarkers are not published--we performed a focused PubMed search employing "biomarker" and the names of the largest pharmaceutical companies as keywords and filtering on clinical trials and publications in the last 10 years. This yielded almost 500 entries, the majority of which included disease-related (approximately 60%) or prognostic/predictive (approximately 20%) biomarkers. A notable portion (approximately 8%) included HER2 (human epidermal growth factor receptor 2) testing, highlighting the utility of biomarkers for patient selection. The remaining publications included target engagement, safety, and drug metabolism biomarkers. Oncology, cardiovascular disease, and osteoporosis were the areas with the most citations, followed by diabetes and Alzheimer disease.

SUMMARY

Judicious biomarker use can improve pharmaceutical development efficiency by helping to select patients most appropriate for treatment using a given mechanism, optimize dose selection, and provide earlier confidence in accelerating or discontinuing compounds in clinical development. Optimal application of biomarker technology requires understanding of candidate drug pharmacology, detailed modeling of biomarker readouts relative to pharmacokinetics, rigorous validation and qualification of biomarker assays, and creative application of these elements to drug development problems.

Point-of-care testing: where is the evidence? A systematic survey

Point-of-care testing (POCT) has had rapid technological development and their use is widespread in clinical laboratories to assure reduction of turn-around-time and rapid patient management in some clinical settings where it is important to make quick decisions. Until now the papers published about the POCT have focused on the reliability of the technology used and their analytical accuracy. We aim to perform a systematic survey of the evidence of POCT efficacy focused on clinical outcomes, selecting POCT denoted special analytes characterized by possible high clinical impact. We searched in Medline and Embase. Two independent reviewers assessed the eligibility, extracted study details and assessed the methodological quality of studies. We analyzed 84 studies for five POCT instruments: neonatal bilirubin, procalcitonin, intra-operative parathyroid hormone, troponin and blood gas analysis. Studies were at high risk of bias. Most of the papers (50%) were studies of correlation between the results obtained by using POCT instruments and those obtained by using laboratory instruments. These data showed a satisfactory correlation between methods when similar analytical reactions were used. Only 13% of the studies evaluated the impact of POCT on clinical practice. POCT decreases the time elapsed for making decisions on patient management but the clinical outcomes have never been adequately evaluated. Our work shows that, although POCT has the potential to provide beneficial patient outcome, further studies may be required, especially for defining its real utility on clinical decision making.

Comparison of treatment effect sizes associated with surrogate and final patient relevant outcomes in randomised controlled trials: meta-epidemiological study

OBJECTIVE

To quantify and compare the treatment effect and risk of bias of trials reporting biomarkers or intermediate outcomes (surrogate outcomes) versus trials using final patient relevant primary outcomes.

DESIGN

Meta-epidemiological study.

DATA SOURCES

All randomised clinical trials published in 2005 and 2006 in six high impact medical journals: Annals of Internal Medicine, BMJ, Journal of the American Medical Association, Lancet, New England Journal of Medicine, and PLoS Medicine.

STUDY SELECTION

Two independent reviewers selected trials.

DATA EXTRACTION

Trial characteristics, risk of bias, and outcomes were recorded according to a predefined form. Two reviewers independently checked data extraction. The ratio of odds ratios was used to quantify the degree of difference in treatment effects between the trials using surrogate outcomes and those using patient relevant outcomes, also adjusted for trial characteristics. A ratio of odds ratios >1.0 implies that trials with surrogate outcomes report larger intervention effects than trials with patient relevant outcomes.

RESULTS

84 trials using surrogate outcomes and 101 using patient relevant outcomes were considered for analyses. Study characteristics of trials using surrogate outcomes and those using patient relevant outcomes were well balanced, except for median sample size (371 v 741) and single centre status (23% v 9%). Their risk of bias did not differ. Primary analysis showed trials reporting surrogate endpoints to have larger treatment effects (odds ratio 0.51, 95% confidence interval 0.42 to 0.60) than trials reporting patient relevant outcomes (0.76, 0.70 to 0.82), with an unadjusted ratio of odds ratios of 1.47 (1.07 to 2.01) and adjusted ratio of odds ratios of 1.46 (1.05 to 2.04). This result was consistent across sensitivity and secondary analyses.

CONCLUSIONS

Trials reporting surrogate primary outcomes are more likely to report larger treatment effects than trials reporting final patient relevant primary outcomes. This finding was not explained by differences in the risk of bias or characteristics of the two groups of trials.

Does intravascular ultrasound provide clinical benefits for percutaneous coronary intervention with bare-metal stent implantation? A meta-analysis of randomized controlled trials

BACKGROUND

The role of intravascular ultrasound (IVUS) in percutaneous coronary interventions (PCI) is still controversial despite several previously published meta-analyses. A meta-analysis to evaluate the controversial role of IVUS-guided PCI with bare-metal stenting was performed and a previous published meta-analysis was re-evaluated in order to clarify the discrepancy between results of these studies.

METHODS

A systematic review was performed by an electronic search of the PubMed, Embase and Web of Knowledge databases and by a manual search of reference lists for randomized controlled trials published until April 2011, with clinical outcomes and, at least, six months of clinical follow-up. A meta-analysis based on the intention to treat was performed with the selected studies.

RESULTS

Five studies and 1,754 patients were included. There were no differences in death (OR = 1.86; 95% CI = 0.88-3.95; p = 0.10), non-fatal myocardial infarction (OR = 0.65; 95% CI = 0.27-1.58; p = 0.35) and major adverse cardiac events (OR = 0.74; 95% CI = 0.49-1.13; p = 0.16). An analysis of the previous published meta-analysis strongly suggested the presence of publication bias.

CONCLUSIONS

There is no evidence to recommend routine IVUS-guided PCI with bare-metal stent implantation. This may be explained by the paucity and heterogeneity of the studies published so far.

Endpoint selection and relative (versus absolute) risk reporting in published medication trials

BACKGROUND

The use of surrogate and composite endpoints, disease-specific mortality as an endpoint, and relative (rather than absolute) risk reporting in clinical trials may produce results that are misleading or difficult to interpret.

OBJECTIVE

To describe the prevalence of these endpoints and of relative risk reporting in medication trials. DESIGN AND MAIN MEASURES: We analyzed all randomized medication trials published in the six highest impact general medicine journals between June 1, 2008 and September 30, 2010 and determined the percentage using these endpoints and the percentage reporting results in the abstract exclusively in relative terms.

KEY RESULTS

We identified 316 medication trials, of which 116 (37%) used a surrogate primary endpoint and 106 (34%) used a composite primary endpoint. Among 118 trials in which the primary endpoint involved mortality, 32 (27%) used disease-specific mortality rather than all-cause mortality. Among 157 trials with positive results, 69 (44%) reported these results in the abstract exclusively in relative terms. Trials using surrogate endpoints and disease-specific mortality as an endpoint were more likely to be exclusively commercially funded (45% vs. 29%, difference 15% [95% CI 5%-26%], P = 0.004, and 39% vs. 16%, difference 22% [95% CI 6%-37%], P = 0.007, respectively). Trials using surrogate endpoints were more likely to report positive results (66% vs. 49%, difference 17% [95% CI 5%-28%], P = 0.006) while those using mortality endpoints were less likely to be positive (46% vs. 62%, difference -16% [95% CI -27%--4%], P = 0.01).

CONCLUSIONS

The use of surrogate and composite endpoints, endpoints involving disease-specific mortality, and relative risk reporting is common. Articles should highlight the limitations of these endpoints and should report results in absolute terms.