Pooling of cohort studies and RCTs affects GRADE certainty of evidence in nutrition research

Evaluating agreement between individual nutrition randomised controlled trials and cohort studies - a meta-epidemiological study

BACKGROUND

In nutrition research, randomised controlled trials (RCTs) and cohort studies provide complementary evidence. This meta-epidemiological study aims to evaluate the agreement of effect estimates from individual nutrition RCTs and cohort studies investigating a highly similar research question and to investigate determinants of disagreement.

METHODS

MEDLINE, Epistemonikos, and the Cochrane Database of Systematic Reviews were searched from January 2010 to September 2021. We matched individual RCTs to cohort studies based on population, intervention/exposure, comparator, and outcome (PI/ECO) characteristics. Two reviewers independently extracted study characteristics and effect estimates and rated the risk of bias using RoB2 and ROBINS-E. Agreement of matched RCTs/cohort studies was analysed by pooling ratio of risk ratios (RRR) and difference of (standardised) mean differences (DSMD).

RESULTS

We included 64 RCT/cohort study pairs with 4,136,837 participants. Regarding PI/ECO similarity, 20.3% pairs were "more or less identical", 71.9% "similar but not identical" and 7.8% "broadly similar". Most RCTs were classified as "low risk of bias" (26.6%) or with "some concerns" (65.6%); cohort studies were mostly rated with "some concerns" (46.6%) or "high risk of bias" (47.9%), driven by inadequate control of important confounding factors. Effect estimates across RCTs and cohort studies were in high agreement (RRR 1.00 (95% CI 0.91-1.10, n = 54); and DSMD - 0.26 (95% CI - 0.87-0.35, n = 7)). In meta-regression analyses exploring determinants of disagreements, risk-of-bias judgements tend to have had more influence on the effect estimate than "PI/ECO similarity" degree.

CONCLUSIONS

Effect estimates of nutrition RCTs and cohort studies were generally similar. Careful consideration and evaluation of PI/ECO characteristics and risk of bias is crucial for a trustworthy utilisation of evidence from RCTs and cohort studies.

Quality of Life in Breast Cancer Patients Treated With Mistletoe Extracts: A Systematic Review and Meta-Analysis

BACKGROUND

Breast cancer is the most common cancer type in women and quality of life an essential part of patients' well-being. Although the treatment with mistletoe extracts is covered by multiple cancer guidelines and reviews, it is uncertain whether mistletoe extracts can improve the quality of life in breast cancer patients. We therefore performed a systematic review and meta-analysis on this topic.

METHODS

This systematic review included randomized clinical trials (RCTs) and non-randomized studies of intervention (NRSIs) comparing the quality of life in breast cancer patients treated with mistletoe extracts as add-on therapy to control groups treated conventionally. We searched previous systematic reviews and multiple databases until January 2023. We conducted a meta-analysis and assessed the risk of bias according to the Cochrane Handbook via RoB 2 and ROBINS-I and the certainty of evidence via GRADE, respectively.

RESULTS

Nine RCTs and 7 NRSIs with 833 and 2831 participants, respectively, were included. The pre-post changes for the quality of life resulted in a pooled standardized mean difference for RCTs of SMD = 0.61 (95% CI 0.47-0.75; P < .0001) and for retrospective NRSIs of SMD = 0.46 (95% CI 0.10-0.82; P = .01). The risk of bias was low to high for the RCTs and serious for all NRSIs. The certainty of evidence was moderate for RCTs and very low for NRSIs.

DISCUSSION

Our results indicate a clinically relevant, medium-sized effect of mistletoe extracts on the quality of life in breast cancer patients which may be based on the immunomodulating effects of mistletoe extracts during chemotherapy. The limitations of evidence include the risk of bias which is mainly caused by the difficulty of blinding. Further RCTs and real-world evidence need to confirm this result, especially in the setting of neoadjuvant chemotherapy and in breast cancer survivors.

An empirical evaluation of the impact scenario of pooling bodies of evidence from randomized controlled trials and cohort studies in medical research

BACKGROUND

Randomized controlled trials (RCTs) and cohort studies are the most common study design types used to assess treatment effects of medical interventions. We aimed to hypothetically pool bodies of evidence (BoE) from RCTs with matched BoE from cohort studies included in the same systematic review.

METHODS

BoE derived from systematic reviews of RCTs and cohort studies published in the 13 medical journals with the highest impact factor were considered. We re-analyzed effect estimates of the included systematic reviews by pooling BoE from RCTs with BoE from cohort studies using random and common effects models. We evaluated statistical heterogeneity, 95% prediction intervals, weight of BoE from RCTs to the pooled estimate, and whether integration of BoE from cohort studies modified the conclusion from BoE of RCTs.

RESULTS

Overall, 118 BoE-pairs based on 653 RCTs and 804 cohort studies were pooled. By pooling BoE from RCTs and cohort studies with a random effects model, for 61 (51.7%) out of 118 BoE-pairs, the 95% confidence interval (CI) excludes no effect. By pooling BoE from RCTs and cohort studies, the median I² was 48%, and the median contributed percentage weight of RCTs to the pooled estimates was 40%. The direction of effect between BoE from RCTs and pooled effect estimates was mainly concordant (79.7%). The integration of BoE from cohort studies modified the conclusion (by examining the 95% CI) from BoE of RCTs in 32 (27%) of the 118 BoE-pairs, but the direction of effect was mainly concordant (88%).

CONCLUSIONS

Our findings provide insights for the potential impact of pooling both BoE in systematic reviews. In medical research, it is often important to rely on both evidence of RCTs and cohort studies to get a whole picture of an investigated intervention-disease association. A decision for or against pooling different study designs should also always take into account, for example, PI/ECO similarity, risk of bias, coherence of effect estimates, and also the trustworthiness of the evidence. Overall, there is a need for more research on the influence of those issues on potential pooling.

Surgery for the treatment of obesity in children and adolescents

BACKGROUND

Child and adolescent overweight and obesity have increased globally and are associated with significant short- and long-term health consequences.

OBJECTIVES

To assess the effects of surgery for treating obesity in childhood and adolescence.

SEARCH METHODS

For this update, we searched Cochrane Central Register of Controlled Trials, MEDLINE, Latin American and Caribbean Health Science Information database (LILACS), World Health Organization International Clinical Trials Registry Platform (ICTRP)and ClinicalTrials.gov on 20 August 2021 (date of the last search for all databases). We did not apply language restrictions. We checked references of identified studies and systematic reviews.

SELECTION CRITERIA

We selected randomised controlled trials (RCTs) of surgical interventions for treating obesity in children and adolescents (age < 18 years) with a minimum of six months of follow-up. We excluded interventions that specifically dealt with the treatment of eating disorders or type 2 diabetes, or which included participants with a secondary or syndromic cause of obesity, or who were pregnant.

DATA COLLECTION AND ANALYSIS

We used standard methodological procedures expected by Cochrane. Two review authors independently extracted data and assessed the risk of bias using the Cochrane Risk of Bias 2.0 tool. Where necessary, we contacted authors for additional information.

MAIN RESULTS

With this update, we did not find any new RCTs. Therefore, this updated review still includes a single RCT (a total of 50 participants, 25 in both the intervention and comparator groups). The intervention focused on laparoscopic adjustable gastric banding surgery, which was compared to a control group receiving a multi-component lifestyle programme. The participating population consisted of Australian adolescents (a higher proportion of girls than boys) aged 14 to 18 years, with a mean age of 16.5 and 16.6 years in the gastric banding and lifestyle groups, respectively. The trial was conducted in a private hospital, receiving funding from the gastric banding manufacturer. For most of the outcomes, we identified a high risk of bias, mainly due to bias due to missing outcome data. Laparoscopic gastric banding surgery may reduce BMI by a mean difference (MD) of -11.40 kg/m2 (95% CI -13.22 to -9.58) and weight by -31.60 kg (95% CI -36.66 to -26.54) compared to a multi-component lifestyle programme at two years follow-up. The evidence is very uncertain due to serious imprecision and a high risk of bias. Adverse events were reported in 12/25 (48%) participants in the intervention group compared to 11/25 (44%) in the control group. A total of 28% of the adolescents undergoing gastric banding required revisional surgery. The evidence is very uncertain due to serious imprecision and a high risk of bias. At two years of follow-up, laparoscopic gastric banding surgery may increase health-related quality of life in the physical functioning scores by an MD of 16.30 (95% CI 4.90 to 27.70) and change in health scores by an MD of 0.82 (95% CI 0.18 to 1.46) compared to the lifestyle group. The evidence is very uncertain due to serious imprecision and a high risk of bias. No data were reported for all-cause mortality, behaviour change, participants' views of the intervention and socioeconomic effects. Finally, we have identified three ongoing RCTs that are evaluating the efficacy and safety of metabolic and bariatric surgery in children and adolescents.

AUTHORS' CONCLUSIONS

Laparoscopic gastric banding led to greater body weight loss compared to a multi-component lifestyle program in one small study with 50 participants. These results have very limited application, primarily due to more recent recommendations derived from observation studies to avoid the use of banding in youth due to long-term reoperation rates. This systematic review update still highlights the lack of RCTs in this field. The authors are concerned that there may be ethical barriers to RTCs in this field, despite the lack of other effective therapies for severe obesity in children and adolescents and the significant morbidity and premature mortality caused by childhood obesity. Nevertheless, future studies, whether pre-registered and planned non-randomised or pragmatic randomised trials, should assess the impact of the surgical procedure and post-operative care to minimise adverse events, including the need for post-operative adjustments and revisional surgery. Long-term follow-up is also critical to comprehensively assess the impact of surgery as participants enter adulthood.