Outcomes for patients with the same disease treated inside and outside of randomized trials: a systematic review and meta-analysis

Survival Benefit Associated With Participation in Clinical Trials of Anticancer Drugs: A Systematic Review and Meta-analysis

Importance

Many cancer clinical investigators view clinical trials as offering better care for patients than routine clinical care. However, definitive evidence of clinical benefit from trial participation (hereafter referred to as the participation effect) has yet to emerge.

Objective

To conduct a systematic review and meta-analysis of the evidence examining whether patient participation in cancer trials was associated with greater survival benefit compared with routine care.

Data Sources

Studies were found through PubMed and Embase (January 1, 2000, until August 31, 2022), as well as backward and forward citation searching.

Study Selection

Studies were included that compared overall survival of trial participants and routine care patients.

Data Extraction and Synthesis

Data extraction and methodological quality assessment were completed by 2 independent coders using Covidence software. Data were pooled using a random-effects model and analyzed based on the quality of the comparison between trial participants and routine care patients (ie, extent to which studies controlled for bias and confounders).

Main Outcomes and Measures

The hazard ratio (HR) for overall survival of trial participants vs routine care patients.

Results

Thirty-nine publications were included, comprising 85 comparisons of trial participants and routine care patients. The meta-analysis revealed a statistically significant overall survival benefit for trial participants (HR, 0.76 [95% CI, 0.69-0.82]) when all studies were pooled, regardless of design or quality. However, survival benefits diminished in study subsets that matched trial participants and routine care patients for eligibility criteria (HR, 0.85 [95% CI, 0.75-0.97]) and disappeared when only high-quality studies were pooled (HR, 0.91 [95% CI, 0.80-1.05]). They also disappeared when estimates were adjusted for potential publication bias (HR, 0.94 [95% CI, 0.86-1.03]).

Conclusions and Relevance

Many studies suggest a survival benefit for cancer trial participants. However, these benefits were not detected in studies using designs addressing important sources of bias and confounding. Pooled results of high-quality studies are not consistent with a beneficial effect of trial participation on its own.

The placebo response rate and nocebo events in obesity pharmacological trials. A systematic review and meta-analysis

BACKGROUND

There is a growing number of trials examining the effectiveness of pharmacotherapies for obesity, however, little is known about placebo and nocebo effect in these trials. Hence, we sought to examine the effect of placebo in obesity trials, to better understand the potential factors affecting clinical endpoints in them.

METHODS

Medline, Embase, and Cochrane CENTRAL were searched for articles examining weight-loss RCTs examining patients with overweight or obesity in placebo-controlled arms from inception till 25 June 2022. This paper was registered online with PROSPERO (CRD42022302482). A single arm meta-analysis of proportions was used to estimate the primary outcomes, ≥5%, ≥10%, and ≥15% total weight loss - and the adverse effects that patients experienced during the trial. A meta-analysis of means was used to estimate the pooled mean differences of the secondary outcomes including, body weight measurements, lipid levels, glycemic indices, and blood pressure over time.

FINDINGS

A total of 63 papers involving 20,454 patients and 69 trials were included. The proportion of patients that had ≥5%, ≥10%, and ≥15% weight loss was 20·4% (CI:16·1% to 25·0%), 8·3% (CI:6·1% to 10·9%), and 6·2% (CI:3·8% to 9·7%), respectively. Analysis by duration of trials showed stepwise increase in proportion of patients with ≥5% and ≥10% weight loss with increasing duration of study. Analysis of secondary outcomes found modest improvement in all analyses. The pooled average rate of overall AEs, serious AEs, and discontinuation was 73·7% (CI:68·0% to 79·0%), 3·4% (CI:2·4% to 4·5%), and 5·2% (CI:4·0% to 6·5%), respectively. In psychiatric complications, the pooled rates of anxiety and depression were 2·7% (CI:1·8% to 3·7%) and 2·5 (CI:1·7% to 3·3%).

INTERPRETATION

Our meta-analysis of placebo-treated participants in weight-loss RCTs indicate a significant placebo and nocebo effect. These findings are important to quantify their effect and may inform the design of future RCTs.

FUNDING

This research did not receive additional support from organizations beyond the authors' academic institutions.

Benefits of Participation in Clinical Trials: An Umbrella Review

Participation in randomised clinical trials (RCTs) entails taking part in the discovery of effects of health care interventions. The question of whether participants' outcomes are different to those of non-participants remains controversial. This umbrella review was aimed at assessing whether there are health benefits of participation in RCTs, compared to non-participation. After prospective registration (PROSPERO CRD42021287812), we searched the Medline, Scopus, Web of Science and Cochrane Library databases from inception to June 2022 to identify relevant systematic reviews with or without meta-analyses. Data extraction and study quality assessment (AMSTAR-2) were performed by two independent reviewers. Of 914 records, six systematic reviews summarising 380 comparisons of RCT participants with non-participants met the inclusion criteria. In two reviews, the majority of comparisons were in favour of participation in RCTs. Of the total of comparisons, 69 (18.7%) were in favour of participation, reporting statistically significant better outcomes for patients treated within RCTs, 264 (71.7%) comparisons were not statistically significant, and 35 (9.5%) comparisons were in favour of non-participation. None of the reviews found a harmful effect of participation in RCTs. Our findings suggest that taking part in RCTs may be beneficial compared to non-participation.

Participation in obstetrical studies is associated with improved pregnancy outcomes

BACKGROUND

The association between pregnant patients participating in obstetrical studies and pregnancy outcomes is understudied.

OBJECTIVE

This study aimed to evaluate the association between participation in obstetrical studies and maternal and neonatal outcomes.

STUDY DESIGN

This was a retrospective cohort study of all patients delivering at a single center from 2013 to 2018. Patients with pregnancy loss at <13 weeks of gestation or major fetal anomalies were excluded. Patients who enrolled in one or more obstetrical studies were categorized as "study participants" and were compared with patients who did not enroll in an obstetrical study, that is, "study nonparticipants." The primary outcome was a composite of maternal morbidity diagnosed up to 6 weeks after delivery. The secondary outcomes included composite neonatal morbidity and other perinatal outcomes. Bivariate analyses compared baseline demographics and outcomes between groups. Multivariable logistic regression was used to estimate adjusted odds ratios with 95% confidence intervals. Subgroup analyses by study design (trial or observational) were planned.

RESULTS

Of 19,569 patients included in this analysis, 3848 (19.7%) were study participants, and 15,721 (80.3%) were study nonparticipants. Among study participants, 3023 (78.6%) enrolled in a trial, and 825 (21.4%) enrolled in an observational study. The study participants had higher body mass index and were more likely to be younger, non-Hispanic Black, publicly insured, nulliparous, and undergo cesarean delivery than study nonparticipants. Compared with study nonparticipants, the study participants had significantly lower odds of composite maternal morbidity (9.2% vs 8.7%; adjusted odds ratio, 0.83; 95% confidence interval, 0.73-0.95) and composite neonatal morbidity (27.5% vs 18.6%; adjusted odds ratio, 0.53; 95% confidence interval, 0.48-0.58). In addition, the odds of fetal death, 5-minute Apgar score of <5, neonatal death, maternal and neonatal intensive care unit admissions, and lengths of stay were lower for study participants than for study nonparticipants. In stratified analyses, maternal morbidity was only significantly decreased among trial participants; however, there was a significantly reduced odds of neonatal morbidity, regardless of study design (trial or observational vs no study).

CONCLUSION

Participation in obstetrical studies was associated with decreased maternal and neonatal morbidities after adjusting for potential confounders. This underscored the importance of pregnant patients enrolling in obstetrical clinical studies and potentially benefiting from the additional surveillance. Further study of how study participation exerts this effect on pregnancy outcomes is warranted.

A systematic review and meta-analysis of weight loss in control group participants of lifestyle randomized trials

Randomized clinical trials (RCTs) of lifestyle modification have reported beneficial effects of interventions, compared to control. Whether participation in the control group has benefits is unknown. To determine whether control group participants experience weight loss during the course of RCTs. After prospective registration (PROSPERO CRD42021233070), we conducted searches in Medline, Scopus, Web of Science, Cochrane library and Clinicaltrials.gov databases from inception to May 2021 without language restriction to capture RCTs on dietary advice or physical activity interventions in adults with overweight, obesity or metabolic syndrome. Data extraction and study quality assessment was performed by two independent reviewers. Weight loss in the control group, i.e., the difference between baseline and post-intervention, was pooled using random effects model generating mean difference and 95% confidence interval (CI). Heterogeneity was assessed using the I² statistical test. Subgroup meta-analysis was performed stratifying by follow-up period, type of control group protocols and high-quality studies. Among the 22 included studies (4032 participants), the risk of bias was low in 9 (40%) studies. Overall, the controls groups experienced weight loss of − 0.41 kg (95% CI − 0.53 to − 0.28; I² = 73.5% p < 0.001). To identify a result that is an outlier, we inspected the forest plot for spread of the point estimates and the confidence intervals. The magnitude of the benefit was related to the duration of follow-up (− 0.51 kg, 95% CI − 0.68, − 0.3, for 1–4 months follow-up; − 0.32 kg, 95% CI − 0.58, − 0.07, 5–12 months; − 0.20 kg, 95% CI − 0.49, 0.10, ≥ 12 months). In high-quality studies we found an overall weight loss mean difference of − 0.16 (95% CI − 0.39, 0.09) with a considerable heterogeneity (I² = 74%; p < 0.000). Among studies including control group in waiting lists and combining standard care, advice and material, no heterogeneity was found (I² = 0%, p = 0.589) and (I² = 0%, p = 0.438); and the mean difference was − 0.84 kg (95% CI − 2.47, 0.80) and − 0.65 kg (95% CI − 1.03, − 0.27) respectively. Participation in control groups of RCTs of lifestyle interventions had a benefit in terms of weight loss in meta-analysis with heterogeneity. These results should be used to interpret the benefits observed with respect to intervention effect in trials. That control groups accrue benefits should be included in patient information sheets to encourage participation in future trials among patients with overweight and obesity.

Enhancing Trial Delivery in Parkinson’s Disease: Qualitative Insights from PD STAT

Background:

Recruitment and retention of participants in clinical trials for Parkinson’s disease (PD) is challenging. A qualitative study embedded in the PD STAT multi-centre randomised controlled trial of simvastatin for neuroprotection in PD explored the motivators, barriers and challenges of participants, care partners and research staff.

Objective:

To outline a set of considerations informing a patient-centred approach to trial recruitment, retention, and delivery.

Method:

We performed semi-structured interviews and focus groups with a subset of trial participants and their care partners. Quantitative and qualitative data were obtained through surveys circulated among the 235 participants across 23 UK sites at the beginning, middle and end of the 2-year trial. We also interviewed and surveyed research staff at trial closure.

Results:

Twenty-seven people with PD, 6 care partners and 9 researchers participated in interviews and focus groups. A total of 463 trial participant survey datasets were obtained across three timepoints, and 53 staff survey datasets at trial closure. Trial participants discussed the physical and psychological challenges they faced, especially in the context of OFF state assessments, relationships, and communication with research staff. Care partners shared their insights into OFF state challenges, and the value of being heard by research teams. Research staff echoed many concerns with suggestions on flexible, person-centred approaches to maximising convenience, comfort, and privacy.

Conclusion:

These considerations, in favour of person-centred research protocols informed by the variable needs of participants, care partners and staff, could be developed into a set of recommendations for future trials.

The ethical justification for inclusion of neonates in pragmatic randomized clinical trials for emergency newborn care

BACKGROUND

Research guidelines generally recognize vulnerable populations to include neonates with the aim of enhancing protections from harm. In practice, such guidance results in limiting participation in randomized clinical trials (RCTs). Yet while medical care of neonates should be based on best research evidence to ensure that safe, efficacious treatment or procedures are used, this seldom happens in contemporary practice.

DISCUSSION

The compelling need to generate information on effectiveness and safety of procedures and medications that are already in use during neonatal care has led to increase in calls for pragmatic randomized clinical trials (PCTs). This raises ethical concerns as to whether exclusion of the vulnerable populations from research participations constitutes harm. First, neonates are denied access to both potentially beneficial research outputs and an opportunity to generate data on how interventions or medications perform in diverse clinical settings and inform clinical decision-making. Secondly, risks and harms in PCTs may differ from traditional RCTs, and can be reduced by modifications in study designs. The latter may involve assessment of effectiveness of comparable medication, devices or practices (whose safety data is available), randomization at the group level rather than at the individual level, avoidance of invasive and innovative study procedures, reliance on locally available data on relevant patient outcomes, and employment of procedures that tend to meet the criteria of minimal risk for human subject research. Thirdly, informed consent procedures should be modified from those of traditional RCTs, as neonates in traditional RCTs may be vulnerable to different extents in PCTs. Lastly, regulatory and oversight procedures designed for traditional RCT settings need modification, as they may not be translatable, feasible, appropriate or even ethical to apply in PCTs.

CONCLUSION

The principle of justice, commonly interpreted as preventing an inequitable burden of research, should also allow fair access to potential benefits from PCTs for neonates and other vulnerable populations. Under certain conditions, prospective randomized trials involving neonates should be ethically permissible to allow inclusion of neonates in research. This may require modification of the research design, consent procedures or regulations for research oversight.

Waived Consent in Perinatal/Neonatal Research-When Is It Appropriate?

Informed consent is a process ensuring that subjects enrolled in research are appropriately informed of the risks and benefits. While this process is well-defined when it is possible and practical to obtain consent prior to the research intervention, it can be less clear in cases of deferred or waived consent. Defining minimal risk, such as when research is attempting to determine which of two currently practiced interventions is safest and/or most effective, is critical to moving forward in establishing appropriate care in newborns. For perinatal/neonatal research the challenge lies between the ethical justification for approaching women in labor or under medication vs. the scientific integrity of excluding a number of subjects that may potentially benefit the most from an intervention. Researchers must work with their IRBs as well as families who have participated in trials to determine the most appropriate method for obtaining informed consent from expectant parents. Clinical researchers and IRBs ultimately need to find a middle ground for the appropriate use of deferred or waived consent.

Limitations of the interpretation and extrapolation of clinical trial data in kidney transplant recipients

OBJECTIVE

The risks and benefits of the participation of kidney transplant recipients in randomized clinical trials (RCTs) investigating new immunosuppressive therapies are unknown.

DESIGN AND SETTING

We included patients from 12 prospective phase II/III RCTs randomized to the experimental (G1, n=319) or standard-of-care internal control group (G2, n=118). We constructed two additional external control groups with (G3, n=319) or without (G4, n=319) matching inclusion/exclusion criteria based on transplant date. The primary outcome analysis was the composite clinical efficacy failure, defined as biopsy-proven acute rejection (BPAR), graft loss, death, or loss to follow-up 12 months after kidney transplantation.

RESULTS

Survival free of composite clinical efficacy failure was higher among participants in RCT, without difference between experimental or standard-of-care therapy (80∙3 vs 78∙0 vs 69∙9 vs 66∙1%, P<.001), respectively. Patient (98.1 vs 99.2 vs 96.9 vs 91.8 P<.001) and graft (94.0 vs 98.3 vs 90.9 vs 82.4) survivals were also higher in G1 compared to G4, but no differences in survival free of BPAR were observed (85.3 vs 78.8 vs 82.8 vs 81.2 P>.05), respectively.

CONCLUSION

These findings suggested that new treatments investigated in kidney transplant recipients are not associated with detectable harm compared to standard of care.

Improved in-hospital outcomes and care for patients in stroke research: An observational study

OBJECTIVE

To describe stroke research activity in Australian acute public hospitals and determine if participation in research provides better quality of care and outcomes for patients with stroke.

METHODS

This was an observational study using data from hospitals that participated in the National Stroke Foundation (Australia) acute services audit program in 2009, 2011, and 2013. This included self-reported organizational features and a retrospective clinical audit of up to 40 medical records of patients with stroke from each hospital. Multilevel random effects logistic regression with level defined as hospital and adjustments for hospital, demographic, clinical, and stroke severity factors were undertaken.

RESULTS

A total of 240 hospitals submitted organizational data. Hospitals with a stroke unit (70% vs 7%, p < 0.001) and >200 stroke admissions per year (80% vs 17%, p < 0.001) reported greater involvement in research studies. Of 9,537 patients audited at 129 hospitals, 469 (5%) consented to participate in research. Patients who participated in research compared to nonparticipants were likely to be younger (median age 73 years; 25th percentile [Q1]: 63, 75th percentile [Q3]: 80, vs median age 76 years Q1: 64, Q3: 83; p < 0.001) and receive important clinical practices such as a swallow screen/assessment prior to oral intake (62% vs 56%; p < 0.01). An independent association with reduced in-hospital mortality (adjusted odds ratio 0.30, 95% confidence interval 0.12, 0.76) was evident if participating in research regardless of access to stroke unit care.

CONCLUSIONS

Patients who participate in stroke research receive better in-hospital care and are more likely to survive compared to nonresearch participants.

CLASSIFICATION OF EVIDENCE

This study provides Class III evidence that patients with stroke who participate in research receive better quality of care and have reduced in-hospital mortality.

Clinical Trials Infrastructure as a Quality Improvement Intervention in Low- and Middle-Income Countries

Mounting evidence suggests that participation in clinical trials confers neither advantage nor disadvantage on those enrolled. Narrow focus on the question of a "trial effect," however, distracts from a broader mechanism by which patients may benefit from ongoing clinical research. We hypothesize that the existence of clinical trials infrastructure-the organizational culture, systems, and expertise that develop as a product of sustained participation in cooperative clinical trials research-may function as a quality improvement lever, improving the quality of care and outcomes of all patients within an institution or region independent of their individual participation in trials. We further contend that this "infrastructure effect" can yield particular benefits for patients in low- and middle-income countries (LMICs). The hypothesis of an infrastructure effect as a quality improvement intervention, if correct, justifies enhanced research capacity in LMIC as a pillar of health system development.

Design and conduct of a large obstetric or neonatal randomized controlled trial

As event rates fall, if mortality and disability are to improve further there is increasing need for large, well-designed trials. These should enroll more patients, more rapidly and at lower cost, with better representation of infants at highest risk and greater integration with routine care. This may require simpler datasets, linkage with routinely collected data, and international collaboration. It may be helpful to draw attention to recent evidence that participation in Phase III randomized controlled trials (RCTs) is at least as safe as receiving established care. Nationally coordinated clinical research networks employing local research staff may be the single most effective strategy to integrate clinical trials into routine practice. Other goals are: international standardization of outcomes; consensus on composite endpoints, biomarkers, surrogates and measures of disability; greater efficiency through randomized factorial designs and cluster or cross-over cluster RCTs; and equipping parents as partners in all aspects of the conduct of RCTs and in implementing their results.

Considerations in the evaluation and determination of minimal risk in pragmatic clinical trials

Institutional review boards, which are charged with overseeing research, must classify the riskiness of proposed research according to a federal regulation known as the Common Rule (45 CFR 46, Subpart A) and by regulations governing the US Food and Drug Administration codified in 21 CFR 50. If an institutional review board determines that a clinical trial constitutes "minimal risk," there are important practical implications: the institutional review board may then allow a waiver or alteration of the informed consent process; the study may be carried out in certain vulnerable populations; or the study may be reviewed by institutional review boards using an expedited process. However, it is unclear how institutional review boards should assess the risk levels of pragmatic clinical trials. Such trials typically compare existing, widely used medical therapies or interventions in the setting of routine clinical practice. Some of the therapies may be considered risky of themselves but the study comparing them may or may not add to that pre-existing level of risk. In this article, we examine the common interpretations of research regulations regarding minimal-risk classifications and suggest that they are marked by a high degree of variability and confusion, which in turn may ultimately harm patients by delaying or hindering potentially beneficial research. We advocate for a clear differentiation between the risks associated with a given therapy and the incremental risk incurred during research evaluating those therapies as a basic principle for evaluating the risk of a pragmatic clinical trial. We then examine two pragmatic clinical trials and consider how various factors including clinical equipoise, practice variation, research methods such as cluster randomization, and patients' perspectives may contribute to current and evolving concepts of minimal-risk determinations, and how this understanding in turn affects the design and conduct of pragmatic clinical trials.

Reducing the confusion and controversies around pragmatic trials: using the Cardiovascular Health Awareness Program (CHAP) trial as an illustrative example

Abstract

Knowledge translation (KT) involves implementation of evidence-based strategies and guidelines into practice to improve the process of care and health outcomes for patients. Findings from pragmatic trials may be used in KT to provide patients, healthcare providers and policymakers with information to optimize healthcare decisions based on how a given strategy or intervention performs under the real world conditions. However, pragmatic trials have been criticized for having the following problems: i) high rates of loss to follow-up; ii) nonadherence to study intervention; iii) unblinded treatment and patient self-assessment, which can potentially create bias; iv) being less perfect experiments than efficacy trials; v) sacrificing internal validity to achieve generalizability; and vi) often requiring large sample sizes to detect small treatment effects in heterogeneous populations. In this paper, we discuss whether these criticisms hold merit, or if they are simply driven by confusion about the purpose of pragmatic trials. We use the Cardiovascular Health Awareness Program (CHAP) trial - a community randomized pragmatic trial designed to assess whether offering a highly organized, community-based CHAP intervention compared to usual care can reduce cardiovascular disease-related outcomes - to address these specific criticisms and illustrate how to reduce this confusion.

Trial registration

Current controlled trials ISRCTN50550004 (9 May 2007).

Research without Informed Patient Consent in Incompetent Patients

Most patients needing intensive care cannot give informed consent to participation in research. This includes the most acutely and severely ill, with the highest mortality and morbidity where research has the greatest potential to improve patient outcomes. In these circumstances consent is usually sought from a substitute decision maker, but while survivors of intensive care believe substitute decision makers will look after their interests, evidence suggests substitute decision makers are poorly equipped for this task. Various models have been suggested for research without patient informed consent when intervention is urgent and cannot wait until first person consent is possible, including a waiver of consent if conditions are met. A nationally consistent model is proposed for Australia with a robust process for initial waiver of consent followed by first person consent to further research-related procedures or ongoing follow-up when this can be competently provided.