A systematic literature review on the efficacy-effectiveness gap: comparison of randomized controlled trials and observational studies of glucose-lowering drugs

Combining randomized and non-randomized data to predict heterogeneous effects of competing treatments

Some patients benefit from a treatment while others may do so less or do not benefit at all. We have previously developed a two-stage network meta-regression prediction model that synthesized randomized trials and evaluates how treatment effects vary across patient characteristics. In this article, we extended this model to combine different sources of types in different formats: aggregate data (AD) and individual participant data (IPD) from randomized and non-randomized evidence. In the first stage, a prognostic model is developed to predict the baseline risk of the outcome using a large cohort study. In the second stage, we recalibrated this prognostic model to improve our predictions for patients enrolled in randomized trials. In the third stage, we used the baseline risk as effect modifier in a network meta-regression model combining AD, IPD randomized clinical trial to estimate heterogeneous treatment effects. We illustrated the approach in the re-analysis of a network of studies comparing three drugs for relapsing-remitting multiple sclerosis. Several patient characteristics influence the baseline risk of relapse, which in turn modifies the effect of the drugs. The proposed model makes personalized predictions for health outcomes under several treatment options and encompasses all relevant randomized and non-randomized evidence.

The efficacy and real-world effectiveness of a diet low in fermentable oligo-, di-, monosaccharides and polyols in irritable bowel syndrome: A systematic review and meta-analysis

BACKGROUND & AIMS

A diet low in fermentable oligo-, di-, monosaccharides, and polyols (LFD) has been shown to effectively reduce irritable bowel syndrome (IBS) symptoms. Effects resulting from real-world studies may differ from those seen in efficacy studies because of the diversity of patients in real-world settings. This systematic review and meta-analysis aimed to compare the effect of the LFD on reducing IBS symptoms and improving the quality of life (QoL) in efficacy trials and real-world studies.

METHODS

Major databases, trial registries, dissertations, and journals were systematically searched for studies on the LFD in adults with IBS. Meta-analysis was conducted using a random effects model with standardized mean differences (SMD) and 95% confidence intervals (CI). Outcomes of interest were all patient-reported: stool consistency, stool frequency, abdominal pain, overall symptoms, adequate symptom relief, IBS-specific QoL and adherence to the LFD.

RESULTS

Eleven efficacy and 19 real-world studies were reviewed. The meta-analysis results for abdominal pain (SMD 0.35, 95% CI 0.16 to 0.54) and QoL (SMD 0.23, 95% CI -0.05 to 0.50) showed the LFD was beneficial in efficacy studies with no statistically significant results for stool frequency (SMD 0.71, 95% CI 0.34 to 1.07). Real-world studies found improvements in abdominal pain and QoL. Due to heterogeneity, no meta-analysis was done for stool consistency and overall symptoms. In these outcomes, results were mostly supportive of the LFD, but they were not always statistically significant.

CONCLUSIONS

The results of this systematic review and meta-analysis suggest the LFD improves outcomes compared to a control diet (efficacy studies) or baseline data (real-world studies). Because of diverse study designs and heterogeneity of results, a clear superiority of the LFD over control diets could not be concluded. There are no indications of an efficacy-effectiveness gap for the LFD in adults with IBS.

Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study

BACKGROUND

Researchers and decision-makers often use evidence from randomised controlled trials (RCTs) to determine the efficacy or effectiveness of a treatment or intervention. Studies with observational designs are often used to measure the effectiveness of an intervention in 'real world' scenarios. Numerous study designs and their modifications (including both randomised and observational designs) are used for comparative effectiveness research in an attempt to give an unbiased estimate of whether one treatment is more effective or safer than another for a particular population. An up-to-date systematic analysis is needed to identify differences in effect estimates from RCTs and observational studies. This updated review summarises the results of methodological reviews that compared the effect estimates of observational studies with RCTs from evidence syntheses that addressed the same health research question.

OBJECTIVES

To assess and compare synthesised effect estimates by study type, contrasting RCTs with observational studies. To explore factors that might explain differences in synthesised effect estimates from RCTs versus observational studies (e.g. heterogeneity, type of observational study design, type of intervention, and use of propensity score adjustment). To identify gaps in the existing research comparing effect estimates across different study types.

SEARCH METHODS

We searched MEDLINE, the Cochrane Database of Systematic Reviews, Web of Science databases, and Epistemonikos to May 2022. We checked references, conducted citation searches, and contacted review authors to identify additional reviews.

SELECTION CRITERIA

We included systematic methodological reviews that compared quantitative effect estimates measuring the efficacy or effectiveness of interventions tested in RCTs versus in observational studies. The included reviews compared RCTs to observational studies (including retrospective and prospective cohort, case-control and cross-sectional designs). Reviews were not eligible if they compared RCTs with studies that had used some form of concurrent allocation.

DATA COLLECTION AND ANALYSIS

Using results from observational studies as the reference group, we examined the relative summary effect estimates (risk ratios (RRs), odds ratios (ORs), hazard ratios (HRs), mean differences (MDs), and standardised mean differences (SMDs)) to evaluate whether there was a relatively larger or smaller effect in the ratio of odds ratios (ROR) or ratio of risk ratios (RRR), ratio of hazard ratios (RHR), and difference in (standardised) mean differences (D(S)MD). If an included review did not provide an estimate comparing results from RCTs with observational studies, we generated one by pooling the estimates for observational studies and RCTs, respectively. Across all reviews, we synthesised these ratios to produce a pooled ratio of ratios comparing effect estimates from RCTs with those from observational studies. In overviews of reviews, we estimated the ROR or RRR for each overview using observational studies as the reference category. We appraised the risk of bias in the included reviews (using nine criteria in total). To receive an overall low risk of bias rating, an included review needed: explicit criteria for study selection, a complete sample of studies, and to have controlled for study methodological differences and study heterogeneity. We assessed reviews/overviews not meeting these four criteria as having an overall high risk of bias. We assessed the certainty of the evidence, consisting of multiple evidence syntheses, with the GRADE approach.

MAIN RESULTS

We included 39 systematic reviews and eight overviews of reviews, for a total of 47. Thirty-four of these contributed data to our primary analysis. Based on the available data, we found that the reviews/overviews included 2869 RCTs involving 3,882,115 participants, and 3924 observational studies with 19,499,970 participants. We rated 11 reviews/overviews as having an overall low risk of bias, and 36 as having an unclear or high risk of bias. Our main concerns with the included reviews/overviews were that some did not assess the quality of their included studies, and some failed to account appropriately for differences between study designs - for example, they conducted aggregate analyses of all observational studies rather than separate analyses of cohort and case-control studies. When pooling RORs and RRRs, the ratio of ratios indicated no difference or a very small difference between the effect estimates from RCTs versus from observational studies (ratio of ratios 1.08, 95% confidence interval (CI) 1.01 to 1.15). We rated the certainty of the evidence as low. Twenty-three of 34 reviews reported effect estimates of RCTs and observational studies that were on average in agreement. In a number of subgroup analyses, small differences in the effect estimates were detected: - pharmaceutical interventions only (ratio of ratios 1.12, 95% CI 1.04 to 1.21); - RCTs and observational studies with substantial or high heterogeneity; that is, I2 ≥ 50% (ratio of ratios 1.11, 95% CI 1.04 to 1.18); - no use (ratio of ratios 1.07, 95% CI 1.03 to 1.11) or unclear use (ratio of ratios 1.13, 95% CI 1.03 to 1.25) of propensity score adjustment in observational studies; and - observational studies without further specification of the study design (ratio of ratios 1.06, 95% CI 0.96 to 1.18). We detected no clear difference in other subgroup analyses.

AUTHORS' CONCLUSIONS

We found no difference or a very small difference between effect estimates from RCTs and observational studies. These findings are largely consistent with findings from recently published research. Factors other than study design need to be considered when exploring reasons for a lack of agreement between results of RCTs and observational studies, such as differences in the population, intervention, comparator, and outcomes investigated in the respective studies. Our results underscore that it is important for review authors to consider not only study design, but the level of heterogeneity in meta-analyses of RCTs or observational studies. A better understanding is needed of how these factors might yield estimates reflective of true effectiveness.

Design, implementation, and inferential issues associated with clinical trials that rely on data in electronic medical records: a narrative review

Real world evidence is now accepted by authorities charged with assessing the benefits and harms of new therapies. Clinical trials based on real world evidence are much less expensive than randomized clinical trials that do not rely on "real world evidence" such as contained in electronic health records (EHR). Consequently, we can expect an increase in the number of reports of these types of trials, which we identify here as 'EHR-sourced trials.' 'In this selected literature review, we discuss the various designs and the ethical issues they raise. EHR-sourced trials have the potential to improve/increase common data elements and other aspects of the EHR and related systems. Caution is advised, however, in drawing causal inferences about the relationships among EHR variables. Nevertheless, we anticipate that EHR-CTs will play a central role in answering research and regulatory questions.

The Efficacy of a Diet Low in Fermentable Oligo-, Di-, Monosaccharides, and Polyols in Irritable Bowel Syndrome Compared to Its "Real-world" Effectiveness: Protocol for a Systematic Review

BACKGROUND

Irritable bowel syndrome (IBS) is associated with various gastrointestinal and nongastrointestinal symptoms and reduced quality of life. A diet low in fermentable oligo-, di-, monosaccharides, and polyols (FODMAPs) is one therapeutic option for IBS. Although the efficacy of the low FODMAP diet has been reported in several systematic reviews, the efficacy-effectiveness gap of the low FODMAP diet has not yet been assessed.

OBJECTIVE

This systematic review aims to compare the efficacy of the low FODMAP diet from efficacy randomized controlled trials (RCTs) with the effectiveness of studies conducted in "real-world" settings.

METHODS

RCTs, prospective and retrospective cohort studies, and retrospective audits assessing the low FODMAP diet in adults with IBS will be searched in 4 databases: Embase, MEDLINE, CENTRAL, and CINAHL. Two independent reviewers will perform study selection, data extraction, and risk of bias assessment and assess selected quality aspects from the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) protocol. Outcomes assessed are stool frequency, stool consistency, abdominal pain, overall symptom scores, adequate symptom relief, IBS-specific quality of life, and diet adherence. Data will be summarized with forest plots without summary statistics, tables, and narrative descriptions.

RESULTS

The search, title and abstract screening, and full-text screening were completed in March 2021, and an updated search was done in May 2022. As of May 2023, data analysis is almost finished, and manuscript writing is in progress. Submission of the manuscript is expected by July 2023.

CONCLUSIONS

The findings of this systematic review will compare the efficacy of the low FODMAP diet for IBS found in RCTs to the diet's real-world effectiveness.

TRIAL REGISTRATION

PROSPERO CRD42021278952; https://tinyurl.com/32jk43ev.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/41399.

Uncovering the Relationship Between Statins and Muscle Problems in the ELSA-Brasil MSK Cohort

PURPOSE

To investigate the association between statins and muscle problems in a highly diverse sample of Brazilian civil servants.

METHODS

We conducted a cross-sectional data analysis at baseline of the ELSA-Brasil MSK cohort. Pain was identified through self-reported symptoms in large muscle groups (lower back and/or hips/thighs). Muscle strength was assessed using the five-times-sit-to-stand (FTSTS) and handgrip tests, with weakness defined as the lowest and highest quintiles of age- and sex-stratified handgrip strength and FTSTS performance time, respectively. Multivariable logistic regression analyses were conducted to investigate the association between statin use and muscle pain and weakness. Secondary analyses explored the impact of different types of statins and their duration of use on the response variables.

RESULTS

A total of 2156 participants (mean age 55.6 ± SD 8.9, 52.8% women) were included, of whom 21.1% were taking statins and 25.1% reported muscle pain. We found no significant association between statin use and muscle problems. Secondary analysis on different types of statins revealed an association between atorvastatin and muscle weakness, as measured by the five-times-sit-to-stand test (OR 1.94, 95% CI 1.12-3.37), but not by the handgrip test (OR 0.75, 95% CI 0.29-1.42). No evidence was found to support a link between the duration of statin treatment and muscle problems.

CONCLUSIONS

This study challenges previous claims of an efficacy-effectiveness gap between experimental and observational literature on statins. The findings indicate that statin use does not contribute to muscular problems.

Health economic evidence for adjuvant chemotherapy in stage II and III colon cancer: a systematic review

OBJECTIVE

The aims of this study was to appraise the health economic evidence for adjuvant chemotherapy (AC) strategies in stage II and III colon cancer (CC) and identify gaps in the available evidence that might inform further research.

METHOD

A systematic review of published economic evaluations was undertaken. Four databases were searched and full-text publications in English were screened for inclusion. A narrative synthesis was performed to summarise the evidence.

RESULTS

Thirty-eight studies were identified and stratified by cancer stage and AC strategy. The majority (89%) were full economic evaluations considering both health outcomes, usually measured as quality-adjusted life years (QALYs), and costs. AC was found to be cost-effective compared to no AC for both stage II and III CC. Oral and oxaliplatin-based AC was cost-effective for stage III. Three months of CAPOX was cost-effective compared to 6-month in high-risk stage II and stage III CC. Preliminary evidence suggests that biomarker approaches to AC selection in stage II can reduce costs and improve health outcomes. Notably, assessment of QALYs were predominantly reliant on a small number of non-contemporary health-utility studies. Only 32% of studies considered societal costs such as travel and time off work.

CONCLUSIONS

Published economic evaluations consistently supported the use of AC in stage II and III colon cancer. Biomarker-driven approaches to patient selection have great potential to be cost-effective, but more robust clinical and economic evidence is warranted. Patient surveys embedded into clinical trials may address critical knowledge gaps regarding accurate assessment of QALYs and societal costs in the modern era.

Real-world evidence for the long-term effect of allergen immunotherapy: Current status on database-derived European studies

Randomized controlled trials (RCTs) are the gold-standard for benefit-risk assessments during drug approval processes. Real-word data (RWD) and the resulting real-world evidence (RWE) are becoming increasingly important for assessing the effectiveness of drug products after marketing authorization showing how RCT results are transferred into real life care. The effectiveness of allergen immunotherapy (AIT) has been assessed in several RWE studies based on large prescription databases. We performed a literature search for retrospective cohort assessments of prescription databases in Europe to provide an overview on the methodology, long-term effectiveness outcomes, and adherence to AIT. Thirteen respective publications were selected. AIT was more effective in reducing the progression of allergic rhinitis (AR) compared to a non-AIT control group receiving only symptomatic treatment for AR for up to 6 years. The development and progression of asthma were hampered for most endpoints in patients treated with most preparations compared to the non-AIT group, receiving only anti-asthmatic medication. The results for "time to onset" of asthma were inconsistent. Adherence to AIT decreased during the recommended 3-year treatment period, however, in most studies higher adherence to subcutaneous than to sublingual AIT was shown. The analysis of long-term effectiveness outcomes of the RWE studies based on prescription databases confirms the long-term efficacy of AIT demonstrated in RCTs. Progression of rhinitis and asthma symptoms as well as delayed onset of asthma triggered by different allergens, real life adherence to the treatment shows differences in particular application routes.

Real-world evidence for the long-term effect of allergen immunotherapy: Current status on database-derived European studies

Randomized controlled trials (RCTs) are the gold-standard for benefit-risk assessments during drug approval processes. Real-word data (RWD) and the resulting real-world evidence (RWE) are becoming increasingly important for assessing the effectiveness of drug products after marketing authorization showing how RCT results are transferred into real life care. The effectiveness of allergen immunotherapy (AIT) has been assessed in several RWE studies based on large prescription databases. We performed a literature search for retrospective cohort assessments of prescription databases in Europe to provide an overview on the methodology, long-term effectiveness outcomes, and adherence to AIT. Thirteen respective publications were selected. AIT was more effective in reducing the progression of allergic rhinitis (AR) compared to a non-AIT control group receiving only symptomatic treatment for AR for up to 6 years. The development and progression of asthma were hampered for most endpoints in patients treated with most preparations compared to the non-AIT group, receiving only anti-asthmatic medication. The results for "time to onset" of asthma were inconsistent. Adherence to AIT decreased during the recommended 3-year treatment period, however, in most studies higher adherence to subcutaneous than to sublingual AIT was shown. The analysis of long-term effectiveness outcomes of the RWE studies based on prescription databases confirms the long-term efficacy of AIT demonstrated in RCTs. Progression of rhinitis and asthma symptoms as well as delayed onset of asthma triggered by different allergens, real life adherence to the treatment shows differences in particular application routes.

Adverse drug reactions of GLP-1 agonists: A systematic review of case reports

BACKGROUND AND AIM

The importance of glucagon-like peptide-1 (GLP-1) agonists is increasing because of its blood sugar controlling and weight loss properties. The data regarding safety of GLP-1 agonists are limited. This study aims to review case reports and case series on adverse drug reactions (ADRs) of GLP-1 agonist.

METHODOLOGY

A comprehensive search was performed in PubMed/Medline, Scopus and Embase to identify literatures. Bibliographic search and open search in Google, Google Scholar, SpringerLink and ResearchGate was performed to identify additional studies. Case reports and case series published the ADRs by the use of GLP-1 agonists in type 2 diabetes patients were included in the study. Reviews, experimental studies, observational studies, grey literature and non English studies were excluded.

RESULTS

The study identified 120 cases of GLP-1 agonists associated ADRs (liraglutide - 46, exenatide - 46, dulaglutide - 20, semaglutide - 4, albiglutide - 2, lixisenatide - 2). The major ADRs reported was gastrointestinal disorders (n = 40) followed by renal (n = 23), dermatologic (n = 14), hepatic (n = 10), immunologic (n = 13), endocrine/metabolic (n = 7), hematologic (n = 3), angioedema (n = 3), neurologic (n = 2), cardiovascular (n = 2) and 1 from each of psychiatric, reproductive, generalized edema problems.

CONCLUSION

Gastrointestinal problems, particularly pancreatitis was the more frequently reported adverse drug reaction associated with GLP-1 agonist. The most adverse drug reactions were observed with liraglutide and exenatide.

Real World Data in Health Technology Assessment of Complex Health Technologies

The available evidence on relative effectiveness and risks of new health technologies is often limited at the time of health technology assessment (HTA). Additionally, a wide variety in real-world data (RWD) policies exist among HTA organizations. This study assessed which challenges, related to the increasingly complex nature of new health technologies, make the acceptance of RWD most likely. A questionnaire was disseminated among 33 EUnetHTA member HTA organizations. The questions focused on accepted data sources, circumstances that allowed for RWD acceptance and barriers to acceptance. The questionnaire was validated and tested for reliability by an expert panel, and pilot-tested before dissemination via LimeSurvey. Twenty-two HTA organizations completed the questionnaire (67%). All reported accepting randomized clinical trials. The most accepted RWD source were patient registries (19/22, 86%), the least accepted were editorials and expert opinions (8/22, 36%). With orphan treatments or companion diagnostics, organizations tended to be most likely to accept RWD sources, 4.3-3.2 on a 5-point Likert scale, respectively. Additional circumstances were reported to accept RWD (e.g., a high disease burden). The two most important barriers to accepting RWD were lacking necessary RWD sources and existing policy structures. European HTA organizations seem positive toward the (wider) use of RWD in HTA of complex therapies. Expanding the use of patient registries could be potentially useful, as a large share of the organizations already accepts this source. However, many barriers still exist to the widespread use of RWD. Our results can be used to prioritize circumstances in which RWD might be accepted.

Outcome Prediction at Patient Level Derived from Pre-Treatment 18F-FDG PET Due to Machine Learning in Metastatic Melanoma Treated with Anti-PD1 Treatment

(1) Background: As outcome of patients with metastatic melanoma treated with anti-PD1 immunotherapy can vary in success, predictors are needed. We aimed to predict at the patients’ levels, overall survival (OS) and progression-free survival (PFS) after one year of immunotherapy, based on their pre-treatment 18F-FDG PET; (2) Methods: Fifty-six metastatic melanoma patients—without prior systemic treatment—were retrospectively included. Forty-five 18F-FDG PET-based radiomic features were computed and the top five features associated with the patient’s outcome were selected. The analyzed machine learning classifiers were random forest (RF), neural network, naive Bayes, logistic regression and support vector machine. The receiver operating characteristic curve was used to compare model performances, which were validated by cross-validation; (3) Results: The RF model obtained the best performance after validation to predict OS and PFS and presented AUC, sensitivities and specificities (IC95%) of 0.87 ± 0.1, 0.79 ± 0.11 and 0.95 ± 0.06 for OS and 0.9 ± 0.07, 0.88 ± 0.09 and 0.91 ± 0.08 for PFS, respectively. (4) Conclusion: A RF classifier, based on pretreatment 18F-FDG PET radiomic features may be useful for predicting the survival status for melanoma patients, after one year of a first line systemic treatment by immunotherapy.

GLP-1 receptor agonists and cardiovascular outcomes in patients with type 2 diabetes: Clinical evidence and best practice

ABSTRACT

Cardiovascular disease (CVD) is a major cause of death and disability among people with type 2 diabetes (T2D), presenting a significant impact on longevity, patient quality of life, and health care costs. In the United States, attainment of recommended glycemic targets is low and T2D-related cardiovascular complications remain a significant burden. Many glucose-lowering treatment options are available, but glucagon-like peptide-1 receptor agonists (GLP-1RAs) and sodium-glucose cotransporter-2 (SGLT-2) inhibitors are recommended in recent guidelines as the preferred add-on therapy to metformin to improve glycemic control. This is particularly the case for patients with T2D and established atherosclerotic CVD, at high risk of atherosclerotic CVD, and/or with chronic kidney disease. Recommendations were based on GLP-1RA and SGLT-2 inhibitor cardiovascular outcomes trials (CVOTs), which consistently showed that these agents pose no additional cardiovascular risk compared with placebo. Three GLP-1RAs (liraglutide, dulaglutide, and subcutaneous semaglutide) demonstrated significantly lower major adverse cardiovascular events versus placebo and are now approved for this indication. However, to realize improvement in outcomes in the clinical setting, organized, systematic, and coordinated approaches to patient management are also needed. For example, nurse-led diabetes self-management education and support programs have been shown to be effective. This article explores T2D management with emphasis on cardiovascular risk and CVOTs performed to date and reviews the clinical experience with GLP-1RAs for managing hyperglycemia and their impact on cardiovascular risk. In addition, practical guidance is given for key health care providers involved in the care of patients with T2D with cardiovascular risk outside of diabetes clinics/endocrinology centers.

Patient-Level Effectiveness Prediction Modeling for Glioblastoma Using Classification Trees

Objectives

Little research has been done in pharmacoepidemiology on the use of machine learning for exploring medicinal treatment effectiveness in oncology. Therefore, the aim of this study was to explore the added value of machine learning methods to investigate individual treatment responses for glioblastoma patients treated with temozolomide.

Methods

Based on a retrospective observational registry covering 3090 patients with glioblastoma treated with temozolomide, we proposed the use of a two-step iterative exploratory learning process consisting of an initialization phase and a machine learning phase. For initialization, we defined a binary response variable as the target label using one-by-one nearest neighbor propensity score matching. Secondly, a classification tree algorithm was trained and validated for dividing individual patients into treatment response and non-response groups. Theorizing about treatment response was then done by evaluating the tree performance.

Results

The classification tree model has an area under the curve (AUC) classification performance of 67% corresponding to a sensitivity of 0.69 and a specificity of 0.51. This result in predicting patient-level response was slightly better than the logistic regression model featuring an AUC of 64% (0.63 sensitivity and 0.54 specificity). The tree confirms confounding by age and discovers further age-related stratification with chemotherapy-treatment dependency, both not revealed in preceding clinical studies. The model lacked genetic information confounding treatment response.

Conclusions

A classification tree was found to be suitable for understanding patient-level effectiveness for this glioblastoma–temozolomide case because of its high interpretability and capability to deal with covariate interdependencies, essential in a real-world environment. Possible improvements in the model’s classification can be achieved by including genetic information and collecting primary data on treatment response. The model can be valuable in clinical practice for predicting personal treatment pathways.

Efficacy gap between phase II and subsequent phase III studies in oncology

Aims

There is a trend for more flexibility in timing of evidence generation in relation to marketing authorization, including the option to complete phase III trials after authorization or not at all. This paper investigated the relation between phase II and III clinical trial efficacy in oncology.

Methods

All oncology drugs approved by the European Medicines Agency (2007–2016) were included. Phase II and phase III trials were matched based on indication and treatment and patient characteristics. Reported objective response rates (ORR), median progression‐free survival (PFS) and median overall survival (OS) were analysed through weighted mixed‐effects regression with previous treatment, treatment regimen, blinding, randomization, marketing authorization type and cancer type as covariates.

Results

A total of 81 phase II‐III matches were identified including 252 trials. Mean (standard deviation) weighted difference (phase III minus II) was −4.2% (17.4) for ORR, 2.1 (6.7) months for PFS and −0.3 (5.1) months for OS, indicating very small average differences between phases. Differences varied substantially between individual indications: from −46.6% to 47.3% for ORR, from −5.3 to 35.9 months for PFS and from −13.3 to 10.8 months for OS. All covariates except blinding were associated with differences in effect sizes for at least 1 outcome.

Conclusions

The lack of marked average differences between phases may encourage decision‐makers to regard the quality of design and total body of evidence instead of differentiating between phases of clinical development. The large variability emphasizes that replication of study findings remains essential to confirm efficacy of oncology drugs and discern variables associated with demonstrated effects.

Retrospective Cohort Study: Ledipasvir-Sofosbuvir With/Without Ribavirin for Chronic Hepatitis C Post–Liver Transplant in a Real-World Population

Background: Liver damage caused by hepatitis C virus (HCV) is the number one indication for liver transplantation in the United States and Europe. Patients with a detectable HCV level at time of transplant will universally develop a recurrent infection, which can lead to increased morbidity and mortality. Objective: To assess the sustained virologic response rate post end-of-treatment (SVR) in HCV-infected, post–liver transplant patients at the University of Washington Medical Center (UWMC) treated with ledipasvir-sofosbuvir (LDV/SOF). Methods: This retrospective, cohort study of HCV-positive, genotype 1 or 4 infected, post–liver transplant patients treated with LDV/SOF was conducted at a large academic medical center affiliated clinic. Patients treated with 12 weeks of LDV/SOF with or without ribavirin were included in the 12-week group, and patients treated with 24 weeks of LDV/SOF without ribavirin were included in the 24-week group. Results: Twenty-nine patients with recurrent HCV post–liver transplant receiving 12 weeks of LDV/SOF with or without ribavirin and 32 patients receiving 24 weeks of LDV/SOF alone were assessed. SVR was achieved by 100% (29/29) of patients in the 12-week group and 100% (32/32) of patients in the 24-week group. Conclusion: Post–liver transplant patients at UWMC treated with LDV/SOF for recurrent HCV achieved high rates of SVR.

The inclusion of real world evidence in clinical development planning

BACKGROUND

When designing studies it is common to search the literature to investigate variability estimates to use in sample size calculations. Proprietary data of previously designed trials in a particular indication are also used to obtain estimates of variability. Estimates of treatment effects are typically obtained from randomised controlled clinical trials (RCTs). Based on the observed estimates of treatment effect, variability and the minimum clinical relevant difference to detect, the sample size for a subsequent trial is estimated. However, data from real world evidence (RWE) studies, such as observational studies and other interventional studies in patients in routine clinical practice, are not widely used in a systematic manner when designing studies. In this paper, we propose a framework for inclusion of RWE in planning of a clinical development programme.

METHODS

In our proposed approach, all evidence, from both RCTs and RWE (i.e. from studies in routine clinical practice), available at the time of designing of a new clinical trial is combined in a Bayesian network meta-analysis (NMA). The results can be used to inform the design of the next clinical trial in the programme. The NMA was performed at key milestones, such as at the end of the phase II trial and prior to the design of key phase III studies. To illustrate the methods, we designed an alternative clinical development programme in multiple sclerosis using RWE through clinical trial simulations.

RESULTS

Inclusion of RWE in the NMA and the resulting trial simulations demonstrated that 284 patients per arm were needed to achieve 90% power to detect effects of predetermined size in the TRANSFORMS study. For the FREEDOMS and FREEDOMS II clinical trials, 189 patients per arm were required. Overall there was a reduction in sample size of at least 40% across the three phase III studies, which translated to a time savings of at least 6 months for the undertaking of the fingolimod phase III programme.

CONCLUSION

The use of RWE resulted in a reduced sample size of the pivotal phase III studies, which led to substantial time savings compared to the approach of sample size calculations without RWE.

Discontinuation of non-anti-TNF drugs for rheumatoid arthritis in interventional versus observational studies: a systematic review and meta-analysis

PURPOSE

Although randomized controlled trials (RCTs) are the gold standard for the assessment of clinical outcomes, long-term extension trials (LTEs) and observational cohorts may help generate evidence. Our goal was to compare the discontinuation rates of abatacept, rituximab, and tocilizumab in rheumatoid arthritis (RA) reported in different study designs.

METHODS

A systematic review was conducted with searches in PubMed, Scopus, and the Cochrane Library, plus a manual search, for RCTs, LTEs, and observational cohorts reporting discontinuation rates by any of three causes (all-cause, inefficacy, adverse events). Meta-analyses with sensitivity analyses and meta-regressions were conducted.

RESULTS

Of the 111 studies included, 74 were RCTs (n = 55) or LTEs (n = 17) reporting data on abatacept (n = 33), rituximab (n = 10), and tocilizumab (n = 31) and 37 were observational cohort studies (abatacept = 11, rituximab = 8, tocilizumab = 18). The follow-up duration did not differ among the study designs. Discontinuation rates were similar among the drugs but varied among the study designs. Discontinuation rates were significantly higher in cohort studies than those in interventional studies for the three drugs. Sensitivity analyses could not identify patient characteristics associated with these differences. Meta-regression analyses demonstrated no correlation between study follow-up duration and discontinuation rates.

CONCLUSIONS

The discontinuation rates reported for non-anti-TNF drugs varied relative to the study design in which they were investigated. Regulatory agencies, price-setting entities, and evidence-gathering researchers should consider the effect of the real-life environment in their decisions and conclusions.

Short and medium-term efficacy of sodium glucose co-transporter-2 (SGLT-2) inhibitors: A meta-analysis of randomized clinical trials

AIMS

Sodium glucose co-transport-2 (SGLT-2) inhibitors reduce tubular glucose reabsorption, producing a reduction of blood glucose without stimulating insulin release. The aim of this meta-analysis was the systematic collection of available data from randomized trials, in order to establish the durability of the efficacy of SGLT-2 inhibitors on glycaemic control and body mass index.

METHODS

A meta-analysis was performed, including all trials with a duration of at least 12 weeks, comparing SGLT-2 inhibitors with non-SGLT-2 inhibitor agents in type 2 diabetes. The principal outcome was the effect of SGLT-2 inhibitors on hemoglobin A1c (HbA1c) at 12, 24, 52 and 104 weeks. Data on body mass index at the same time points were also collected.

RESULTS

Among 66 randomized trials, HbA1c reduction at 12, 24, 52 and 104 weeks was 0.63% (0.57; 0.68, 0.63% (0.57; 0.70), 0.66% (0.57; 0.74) and 0.60% (0.40; 0.81), respectively. SGLT-2 inhibitors showed a greater efficacy than dipeptidyl-peptidase-4 inhibitors (DPP-4i). Sulfonylureas appeared to be superior to SGLT-2 inhibitors at 12 weeks, but not at 24 and 52 weeks; SGLT-2 inhibitors produced a greater reduction in HbA1c than did sulfonylureas at 104 weeks. SGLT-2 inhibitor-induced weight loss in placebo-controlled trials appeared to increase progressively with the duration of treatment.

CONCLUSIONS

SGLT-2 inhibitors showed a good persistence of efficacy, at least up to 2 years, with a small but significant superiority over DPP-4i. Sulfonylureas are more effective in the very short term, but less effective in the longer term.

Use of the PRECIS-II instrument to categorize reports along the efficacy-effectiveness spectrum in an hepatitis C virus care continuum systematic review and meta-analysis

There is increasing recognition of the importance of the distinction between efficacy and effectiveness research in the design, conduct, and evaluation of interventions and program outcomes. There is a concurrent increase in the application of systematic reviews and meta-analyses. These two lines of inquiry are only beginning to meet. There is an emerging need for systematic reviews and meta-analyses to account for differences in degrees to which included studies reflect either efficacy or effectiveness design. Based on ongoing work on a formal systematic review of the hepatitis C virus care continuum, this paper describes and discusses the rationale for, and how the PRECIS-II instrument can be used on, and modestly adapted to, studies included in the systematic review examining the extent to which studies include elements of efficacy or effectiveness or a combination of the two. We also highlight that use of such an instrument may have general applicability to and value in the conduct of systematic reviews and meta-analysis.

The use of random-effects models to identify health care center-related characteristics modifying the effect of antipsychotic drugs

Purpose

A case study was conducted, exploring methods to identify drugs effects modifiers, at a health care center level.

Patients and methods

Data were drawn from the Schizophrenia Outpatient Health Outcome cohort, including hierarchical information on 6641 patients, recruited from 899 health care centers from across ten European countries. Center-level characteristics included the following: psychiatrist’s gender, age, length of practice experience, practice setting and type, countries’ Healthcare System Efficiency score, and psychiatrist density in the country. Mixed multivariable linear regression models were used: 1) to estimate antipsychotic drugs’ effectiveness (defined as the association between patients’ outcome at 3 months – dependent variable, continuous – and antipsychotic drug initiation at baseline – drug A vs other antipsychotic drug); 2) to estimate the similarity between clustered data (using the intra-cluster correlation coefficient); and 3) to explore antipsychotic drug effects modification by center-related characteristics (using the addition of an interaction term).

Results

About 23% of the variance found for patients’ outcome was explained by unmeasured confounding at a center level. Psychiatrists’ practice experience was found to be associated with patient outcomes (p=0.04) and modified the relative effect of “drug A” (p<0.001), independent of center- or patient-related characteristics.

Conclusion

Mixed models may be useful to explore how center-related characteristics modify drugs’ effect estimates, but require numerous assumptions.