The effect of sodium-glucose transport protein 2 inhibitors on mortality and heart failure in randomized trials versus observational studies

Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study

BACKGROUND

Researchers and decision-makers often use evidence from randomised controlled trials (RCTs) to determine the efficacy or effectiveness of a treatment or intervention. Studies with observational designs are often used to measure the effectiveness of an intervention in 'real world' scenarios. Numerous study designs and their modifications (including both randomised and observational designs) are used for comparative effectiveness research in an attempt to give an unbiased estimate of whether one treatment is more effective or safer than another for a particular population. An up-to-date systematic analysis is needed to identify differences in effect estimates from RCTs and observational studies. This updated review summarises the results of methodological reviews that compared the effect estimates of observational studies with RCTs from evidence syntheses that addressed the same health research question.

OBJECTIVES

To assess and compare synthesised effect estimates by study type, contrasting RCTs with observational studies. To explore factors that might explain differences in synthesised effect estimates from RCTs versus observational studies (e.g. heterogeneity, type of observational study design, type of intervention, and use of propensity score adjustment). To identify gaps in the existing research comparing effect estimates across different study types.

SEARCH METHODS

We searched MEDLINE, the Cochrane Database of Systematic Reviews, Web of Science databases, and Epistemonikos to May 2022. We checked references, conducted citation searches, and contacted review authors to identify additional reviews.

SELECTION CRITERIA

We included systematic methodological reviews that compared quantitative effect estimates measuring the efficacy or effectiveness of interventions tested in RCTs versus in observational studies. The included reviews compared RCTs to observational studies (including retrospective and prospective cohort, case-control and cross-sectional designs). Reviews were not eligible if they compared RCTs with studies that had used some form of concurrent allocation.

DATA COLLECTION AND ANALYSIS

Using results from observational studies as the reference group, we examined the relative summary effect estimates (risk ratios (RRs), odds ratios (ORs), hazard ratios (HRs), mean differences (MDs), and standardised mean differences (SMDs)) to evaluate whether there was a relatively larger or smaller effect in the ratio of odds ratios (ROR) or ratio of risk ratios (RRR), ratio of hazard ratios (RHR), and difference in (standardised) mean differences (D(S)MD). If an included review did not provide an estimate comparing results from RCTs with observational studies, we generated one by pooling the estimates for observational studies and RCTs, respectively. Across all reviews, we synthesised these ratios to produce a pooled ratio of ratios comparing effect estimates from RCTs with those from observational studies. In overviews of reviews, we estimated the ROR or RRR for each overview using observational studies as the reference category. We appraised the risk of bias in the included reviews (using nine criteria in total). To receive an overall low risk of bias rating, an included review needed: explicit criteria for study selection, a complete sample of studies, and to have controlled for study methodological differences and study heterogeneity. We assessed reviews/overviews not meeting these four criteria as having an overall high risk of bias. We assessed the certainty of the evidence, consisting of multiple evidence syntheses, with the GRADE approach.

MAIN RESULTS

We included 39 systematic reviews and eight overviews of reviews, for a total of 47. Thirty-four of these contributed data to our primary analysis. Based on the available data, we found that the reviews/overviews included 2869 RCTs involving 3,882,115 participants, and 3924 observational studies with 19,499,970 participants. We rated 11 reviews/overviews as having an overall low risk of bias, and 36 as having an unclear or high risk of bias. Our main concerns with the included reviews/overviews were that some did not assess the quality of their included studies, and some failed to account appropriately for differences between study designs - for example, they conducted aggregate analyses of all observational studies rather than separate analyses of cohort and case-control studies. When pooling RORs and RRRs, the ratio of ratios indicated no difference or a very small difference between the effect estimates from RCTs versus from observational studies (ratio of ratios 1.08, 95% confidence interval (CI) 1.01 to 1.15). We rated the certainty of the evidence as low. Twenty-three of 34 reviews reported effect estimates of RCTs and observational studies that were on average in agreement. In a number of subgroup analyses, small differences in the effect estimates were detected: - pharmaceutical interventions only (ratio of ratios 1.12, 95% CI 1.04 to 1.21); - RCTs and observational studies with substantial or high heterogeneity; that is, I2 ≥ 50% (ratio of ratios 1.11, 95% CI 1.04 to 1.18); - no use (ratio of ratios 1.07, 95% CI 1.03 to 1.11) or unclear use (ratio of ratios 1.13, 95% CI 1.03 to 1.25) of propensity score adjustment in observational studies; and - observational studies without further specification of the study design (ratio of ratios 1.06, 95% CI 0.96 to 1.18). We detected no clear difference in other subgroup analyses.

AUTHORS' CONCLUSIONS

We found no difference or a very small difference between effect estimates from RCTs and observational studies. These findings are largely consistent with findings from recently published research. Factors other than study design need to be considered when exploring reasons for a lack of agreement between results of RCTs and observational studies, such as differences in the population, intervention, comparator, and outcomes investigated in the respective studies. Our results underscore that it is important for review authors to consider not only study design, but the level of heterogeneity in meta-analyses of RCTs or observational studies. A better understanding is needed of how these factors might yield estimates reflective of true effectiveness.

Sodium-glucose co-transporter-2 inhibitors in type 2 diabetes: Are clinical trial benefits for heart failure reflected in real-world clinical practice? A systematic review and meta-analysis of observational studies

AIM

To determine the absolute risk reduction (ARR) of heart failure events in people treated with sodium-glucose co-transporter-2 (SGLT2) inhibitors.

MATERIALS AND METHODS

We searched PubMed, EMBASE, CINAHL and ISI Web of Science for observational studies published to 9 May 2022 that explored the association between SGLT2 inhibitors and any indication for heart failure (including new diagnosis or hospitalization for heart failure) in type 2 diabetes. Identified studies were independently screened by two reviewers and assessed for bias using the Newcastle-Ottawa scale. Eligible studies with comparable outcome data were pooled for meta-analysis using random-effects models, reporting hazard ratios (HRs) with 95% confidence intervals (CIs). The ARR per 100 person-years was determined overall, and in subgroups with and without baseline cardiovascular disease (CVD).

RESULTS

From 43 eligible studies, with a total of 4 818 242 participants from 17 countries, 21 were included for meta-analysis. SGLT2 inhibitors were associated with a reduced risk of hospitalization for heart failure (HR 0.65, 95% CI 0.59-0.72) overall and both in those with CVD (HR 0.78, 95% CI 0.68-0.89) and without CVD (HR 0.53, 95% CI 0.39-0.71). Risk reduction for hospitalization for heart failure in people with a history of CVD (ARR 1.17, 95% CI 0.78-1.55) was significantly greater than for those without CVD (ARR 0.39, 95% CI 0.32-0.47). The number-needed-to-treat to prevent one event of hospitalization for heart failure was 86 (95% CI 65-128) person-years of treatment for the CVD group and 256 (95% CI 215-316) person-years for those without CVD.

CONCLUSIONS

Real-world SGLT2 inhibitor use supports randomized trial data for the size effect of reduced hospitalization for heart failure in type 2 diabetes, although with a much lower ARR in people without CVD.

A read-world study on SGLT2 inhibitors and diabetic kidney disease progression

Background

Randomized controlled trials have demonstrated the benefits of sodium–glucose cotransporter 2 inhibitors (SGLT2is) in people with type 2 diabetes mellitus (T2DM) and chronic kidney disease (CKD). However, real-world data on CKD progression and the development of end-stage kidney disease (ESKD) remains scarce. Our aim was to study renal outcomes of people with diabetic kidney disease (DKD) using SGLT2is in a highly prevalent DKD population.

Methods

Between 2016 and 2019 we recruited T2DM patients in the renal and diabetic clinics in a regional hospital in Singapore. Patients prescribed SGLT2is were compared with those on standard anti-diabetic and renoprotective treatment. The outcome measures were CKD progression [a ≥25% decrease from baseline and worsening of estimated glomerular filtration rate (eGFR) categories according to the Kidney Disease: Improving Global Outcomes guidelines] and ESKD (eGFR <15 mL/min/1.73 m²).

Results

We analysed a total of 4446 subjects; 1598 were on SGLT2is. There was a significant reduction in CKD progression {hazard ratio [HR] 0.60 [95% confidence interval (CI) 0.49–0.74]} with SGLT2is. The HR for eGFR ≥45 mL/min/1.73 m² and 15–44 mL/min/1.73 m² was 0.60 (95% CI 0.47–0.76) and 0.43 (95% CI 0.23–0.66), respectively. There was also a reduction in risk for developing ESKD for the entire cohort [HR 0.33 (95% CI 0.17–0.65)] and eGFR 15–44 mL/min/1.73 m² [HR 0.24 (95% CI 0.09–0.66)]. Compared with canagliflozin and dapagliflozin, empagliflozin showed a sustained risk reduction of renal outcomes across CKD stages 1–4.

Conclusions

This real-world study demonstrates the benefits of SGLT2is on CKD progression and ESKD. The effect is more pronounced in moderate to advanced CKD patients.