Replication of Randomized, Controlled Trials Using Real-World Data: What Could Go Wrong?

A prototype protocol for evaluating the real-world data set using a structured electronic health record in glaucoma

PURPOSE

As the first step in monitoring and evaluating day-to-day glaucoma care, this study reports all real-world data recorded during the first full year after the implementation of a prototype for glaucoma-specific structured electronic healthcare record (EHR).

METHODS

In 2019, 4618 patients visited Tays Medical Glaucoma Clinic at Tays Eye Centre, Tampere University Hospital, Finland, that serves a population of 0.53 M. Patient data were entered into a glaucoma-specific EHR by trained nurses to be checked by glaucoma specialists. Tays Eye Centre follows the Finnish Current Care Guideline for glaucoma in which glaucoma is defined using a '2 out of 3' rule, that is, ≥2 findings evaluated as glaucomatous in optic nerve head (ONH), retinal nerve fibre layer (RNFL) and visual field (VF).

RESULTS

The clinical evaluations of ONH, RNFL and VF were recorded in 95%-100% of all eyes. ONH was evaluated as glaucomatous more often (44%) than RNFL (33%) and VF tests (30%). Progressive changes in any of the three tests were recorded in 35% of the '≥2/3 glaucoma group' compared to 2%-9% in the other groups. The mean IOP at visit was 15 mmHg. The mean target IOP was 17 mmHg, and it was recorded in 94% of eyes.

CONCLUSION

The developed structured data presentation enables comparisons between different population-based real-world glaucoma data sets and glaucoma clinics. Compared to a data set from the UK, the proportion of glaucoma suspicion-related visits was smaller in Tays Eye Centre and test intervals were longer.

Comparing Survival Outcomes between Hemodialysis and Hemodiafiltration Using Real-World Data from Brazil

The CONVINCE trial demonstrates that high-dose hemodiafiltration offers a survival advantage for patients in the high-flux hemodiafiltration group compared to hemodialysis. We compared the outcomes of hemodialysis and hemodiafiltration using real-world data. We conducted an analysis on a cohort of patients who underwent hemodiafiltration therapy (HDF) at a single center, NefroStar Clinics. The results obtained were then compared with data from patients receiving hemodialysis (HD) therapy within the Brazilian Public Health System (SUS). The primary outcome was mortality from any cause. Results: A total of 85 patients undergoing hemodiafiltration were compared with 149,372 patients receiving hemodialysis through the Brazilian Public Health System (SUS). Using a 2:1 propensity score, we compared the 170 best-match HD patients with 85 HDF patients. In the Cox analysis, HDF therapy showed a reduced risk of mortality with an HR of 0.29 [0.11-0.77]. The propensity score analysis showed a HR of 0.32 [95% CI: 0.11-0.91]. This analysis was adjusted for age, type of access, KT/v, hemoglobin, and phosphorus. The Kaplan-Meier analysis showed respective survival rates for HDF and HD at the end of one year, 92.1% and 79.9%, p < 0.001. These results suggest high-flux hemodiafiltration has survival advantages over hemodialysis in a real-world scenario.

A propensity score matched comparison of blood pressure lowering in essential hypertension patients treated with antihypertensive Chinese herbal Medicine: comparing the real-world registry data vs. randomized controlled trial

BACKGROUND

Randomized controlled trials have demonstrated that Songling Xuemaikang capsule (SXC) is effective in blood pressure (BP) lowering for essential hypertension. However, the effectiveness of SXC in real-world clinical practice remains unknown. We aimed to investigate whether the BP-lowering effectiveness of SXC in the real-world practice setting is comparable to the efficacy of the intervention in a randomized controlled trial.

METHODS

We included 1325 patients treated with SXC monotherapy from a real-world registry and 300 from the SXC-BP trial. A propensity score matching (PSM) approach was used to select participants from the two cohorts. The primary outcome was a change in the office of BP from baseline to 8 weeks.

RESULTS

After PSM, there were 552 patients for the comparative analysis. Clinically meaningful BP reductions were observed both in the real world and in the RCT cohorts after 8-week SXC treatment. The 8-week systolic/diastolic BP was 129.50/81.33 mm Hg vs. 134.97/84.14 mm Hg in the real-world population and the RCT population, respectively. The changes in systolic BP (15.82 ± 10.71 vs. 10.48 ± 10.24; P < .001), and diastolic BP (10.01 ± 7.73 vs. 7.75 ± 8.14; P = .001) from baseline to 8 weeks were significantly greater in the real-world population.

CONCLUSION

The current comparison demonstrated that SXC monotherapy is at least as effective in real-world settings as within the randomized controlled trial for BP lowering in patients with grade 1 hypertension.

The relationships between pain-associated psychological distress, pain intensity, patient expectations, and physical function in individuals with musculoskeletal pain: A retrospective cohort study

INTRODUCTION

The presence of pain-associated psychological distress (PAPD) in musculoskeletal disorders, including negative mood, fear-avoidance, and lack of positive affect/coping, is associated with prolonged disability. The importance of considering psychological influence on pain is well known, but practical ways of addressing it are not as straightforward. Identifying relationships between PAPD and pain intensity, patient expectations, and physical function may guide the development of future studies that assess causality and inform clinical practice.

OBJECTIVE

To assess the relationship between PAPD measured by the Optimal Screening for Prediction of Referral and Outcome-Yellow Flag tool, and baseline pain intensity, expectations of treatment effectiveness, and self-reported physical function at discharge.

DESIGN

Retrospective cohort study.

SETTING

Hospital-based outpatient physical therapy.

PARTICIPANTS

Patients 18 to 90 years old with spinal pain or lower extremity osteoarthritis.

MAIN OUTCOME MEASURES

Pain intensity and patient expectations of treatment effectiveness at intake, and self-reported physical function at discharge.

RESULTS

A total of 534 patients, 56.2% female, median (interquartile range [IQR]) age 61 (21) years with an episode of care between November 2019 and January 2021 were included. A multiple linear regression showed a significant association between PAPD and pain intensity with 6.4% (p < .001) of the variance explained. PAPD explained 3.3% (p < .001) of the variance in patient expectations. One additional yellow flag present resulted in a 0.17-point increase in pain intensity and 1.3% decrease in patient expectations. PAPD was also associated with physical function with 3.2% (p < .001) of the variance explained. PAPD explained 9.1% (p < .001) of the variance in physical function at discharge in the low back pain cohort only when assessed independently by body region.

CONCLUSION

These findings support the theory that the pain experience is complex and multiple factors should be considered when evaluating a patient with musculoskeletal pain. Clinicians who have identified PAPD may consider these relationships when planning or modifying interventions and pursuing multidisciplinary collaboration.

Impact of Baricitinib on Patients' Quality of Life after One Year of Treatment for Atopic Dermatitis in Real-World Practice: Results of the Observatory of Chronic Inflammatory Skin Diseases Registry

The efficacy and safety of baricitinib for treatment of atopic dermatitis have been demonstrated in clinical trials; however, very few real-life studies have been published to date. The Observatory of Chronic Inflammatory Skin Diseases (OMCCI) registry was initiated to prospectively determine the long-term impairment caused by chronic inflammatory dermatoses on patients' lives. The study included 88 patients starting baricitinib for treatment of atopic dermatitis. Clinical evaluation and patient-reported outcomes were recorded at baseline and after 6 and 12 months. After 6 months and 1 year of follow-up, 65 and 47 patients, respectively, were still being treated with baricitinib. Treatment failure was the main reason for discontinuation. Only 1 patient stopped baricitinib because of a side-effect. After 1 year of follow-up, the mean Eczema Area and Severity Index score decreased significantly from 20.7 to 6.4; the percentage of patients with severe atopic dermatitis decreased from 42.9% to 6.5% and a significant improvement in most patient-reported outcomes was noted. There was no difference in terms of efficacy whether or not patients were previously treated with dupilumab. The results remained stable after 6 and 12 months of treatment, which suggests a sustained efficacy of the treatment in patients who initially responded well.

Using public clinical trial reports to probe non-experimental causal inference methods

BACKGROUND

Non-experimental studies (also known as observational studies) are valuable for estimating the effects of various medical interventions, but are notoriously difficult to evaluate because the methods used in non-experimental studies require untestable assumptions. This lack of intrinsic verifiability makes it difficult both to compare different non-experimental study methods and to trust the results of any particular non-experimental study.

METHODS

We introduce TrialProbe, a data resource and statistical framework for the evaluation of non-experimental methods. We first collect a dataset of pseudo "ground truths" about the relative effects of drugs by using empirical Bayesian techniques to analyze adverse events recorded in public clinical trial reports. We then develop a framework for evaluating non-experimental methods against that ground truth by measuring concordance between the non-experimental effect estimates and the estimates derived from clinical trials. As a demonstration of our approach, we also perform an example methods evaluation between propensity score matching, inverse propensity score weighting, and an unadjusted approach on a large national insurance claims dataset.

RESULTS

From the 33,701 clinical trial records in our version of the ClinicalTrials.gov dataset, we are able to extract 12,967 unique drug/drug adverse event comparisons to form a ground truth set. During our corresponding methods evaluation, we are able to use that reference set to demonstrate that both propensity score matching and inverse propensity score weighting can produce estimates that have high concordance with clinical trial results and substantially outperform an unadjusted baseline.

CONCLUSIONS

We find that TrialProbe is an effective approach for probing non-experimental study methods, being able to generate large ground truth sets that are able to distinguish how well non-experimental methods perform in real world observational data.

Do SGLT2 inhibitors and GLP-1 receptor agonists modulate differently the risk of stroke ? Discordance between randomised controlled trials and observational studies

Stroke represents a major burden in patients with type 2 diabetes, yet this cerebrovascular complication has been less carefully investigated than the risk of cardiovascular mortality, heart failure and renal disease. Some data suggested that glucagon-like peptide-1 receptor agonists (GLP-1RAs) exert a better protection against stroke than sodium-glucose cotransporter 2 inhibitors (SGLT2is). However, this conclusion was derived from indirect comparisons in absence of any head-to-head randomised controlled trial (RCT). The present comprehensive review compares the effects of SGLT2is versus GLP-1RAs on nonfatal and fatal/nonfatal strokes in network meta-analyses of RCTs (mostly cardiovascular outcome trials) versus placebo, on the one hand, and in real-life observational cohort studies, on the other hand. Whereas network meta-analyses of placebo-controlled RCTs confirm a slight but significant (in 11 out of 13 meta-analyses) higher incidence of stroke in patients treated with SGLT2is compared with those treated with GLP-1RAs, a large majority of retrospective observational cohort studies (19 out of 21) failed to find any significant difference in the risk of stroke between the two pharmacological classes. Available, yet limited, findings suggest that SGLT2is may be more efficacious against haemorrhagic than ischaemic strokes, in patients at risk for atrial fibrillation and in patients with chronic kidney disease.

Evaluation of reporting quality of cohort studies using real-world data based on RECORD: systematic review

OBJECTIVE

Real-world data (RWD) and real-world evidence (RWE) have been paid more and more attention in recent years. We aimed to evaluate the reporting quality of cohort studies using real-world data (RWD) published between 2013 and 2021 and analyze the possible factors.

METHODS

We conducted a comprehensive search in Medline and Embase through the OVID interface for cohort studies published from 2013 to 2021 on April 29, 2022. Studies aimed at comparing the effectiveness or safety of exposure factors in the real-world setting were included. The evaluation was based on the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. Agreement for inclusion and evaluation was calculated using Cohen's kappa. Pearson chi-square test or Fisher's exact test and Mann-Whitney U test were used to analyze the possible factors, including the release of RECORD, journal IFs, and article citations. Bonferroni's correction was conducted for multiple comparisons. Interrupted time series analysis was performed to display the changes in report quality over time.

RESULTS

187 articles were finally included. The mean ± SD of the percentage of adequately reported items in the 187 articles was 44.7 ± 14.3 with a range of 11.1-87%. Of 23 items, the adequate reporting rate of 10 items reached 50%, and the reporting rate of some vital items was inadequate. After Bonferroni's correction, the reporting of only one item significantly improved after the release of RECORD and there was no significant improvement in the overall report quality. For interrupted time series analysis, there were no significant changes in the slope (p = 0.42) and level (p = 0.12) of adequate reporting rate. The journal IFs and citations were respectively related to 2 areas and the former significantly higher in high-reporting quality articles.

CONCLUSION

The endorsement of the RECORD cheklist was generally inadequate in cohort studies using RWD and has not improved in recent years. We encourage researchers to endorse relevant guidelines when utilizing RWD for research.

An Attempt to Replicate Randomized Trials of Diabetes Treatments Using a Japanese Administrative Claims and Health Checkup Database: A Feasibility Study

BACKGROUND

Use of real-world evidence (RWE) has been limited for evaluating effectiveness because of the lack of confidence in its reliability. Examining whether a rigorously designed observational study using real-world data (RWD) can reproduce the results of a randomized controlled trial (RCT) will provide insights into the implementation of high-quality RWE studies that can produce valid conclusions.

OBJECTIVE

We aimed to replicate published RCTs using a Japanese claims and health checkup database and examine whether the emulated RWE studies' results agree with those of the original RCTs.

METHODS

We selected three RCTs on diabetes medications for replication in patients with type 2 diabetes. The study outcome was either the change or percentage change in HbA1c levels from baseline. We designed three observational studies using the RWD to mimic the critical study elements of the respective RCTs as closely as possible. We performed 1:1 propensity score nearest-neighbor matching to balance the groups for potential confounders. The differences in outcomes between the groups and their 95% confidence intervals (CIs) were calculated in each RWE study, and the results were compared with those of the RCT.

RESULTS

Patient characteristics, such as age, sex, and duration of diabetes, differed between the RWE studies and RCTs. In Trial 1 emulation, the percentage changes in HbA1c levels were larger in the treatment group than in the comparator group (difference -6.21, 95% confidence interval (CI) -11.01 to -1.40). In Trial 2, the change in HbA1c level was larger in the treatment group (difference -0.01; 95% CI -0.25 to 0.23), and in Trial 3, it was smaller in the treatment group (difference 0.46; 95% CI -0.01 to 0.94). These results did not show regulatory or estimate agreement with the RCTs.

CONCLUSIONS

None of the three emulated RWE studies using this claims and health checkup database reproduced the same conclusions as the RCTs. These discrepancies could largely be attributed to design differences between RWE studies and RCTs, primarily due to the lack of necessary data in the database. This particular RWD source may not be the best fit for evaluating treatment effects using laboratory data as the study outcome.

Emulating VIEW 1 and VIEW 2 Clinical Trial Outcome Data Using the American Academy of Ophthalmology IRIS Registry

BACKGROUND AND OBJECTIVE

A retrospective, noninterventional cohort study of the American Academy of Ophthalmology IRIS Registry, an electronic health record (EHR)-based comprehensive eye disease and condition registry, intended to assess whether the IRIS® Registry (Intelligent Research in Sight) could emulate the VIEW randomized clinical trials (VIEW RCTs) eligibility criteria, treatment protocol regimen, and primary endpoint.

PATIENTS AND METHODS

Deidentified patients having an anti-VEGF injection of aflibercept or ranibizumab between January 1, 2013, and December 31, 2018, from the IRIS Registry. Patients were treated in accordance with one of three treatment regimens from the VIEW RCT: monthly intravitreal aflibercept injection (IAI 2Q4), intravitreal aflibercept every 2 months after 3 initial monthly doses (IAI 2Q8), or monthly ranibizumab (RQ4) injection. The main outcome measures are the number and proportion of patients meeting VIEW RCT eligibility and treatment group criteria, demographic, and clinical differences between IRIS Registry treatment groups, mean change in best documented visual acuity at one year, and evaluation of the primary endpoint of the VIEW RCT: difference in the proportion of patients maintaining vision.

RESULTS

Among the 90,900 patients who met VIEW RCT eligibility criteria, 4,457 (4.85%) met treatment group criteria. The percentage of patients maintaining vision at one year was over 90%. No statistically significant difference was observed when comparing the proportion of patients maintaining vision among the RQ4 treatment group to the IAI 2Q4 or IAI 2Q8 treatment group.

CONCLUSIONS

A small percentage of real-world patients met VIEW RCT study eligibility criteria and treatment protocol regimen. Among patients meeting all available criteria, the primary endpoint interpretation yielded by an observational EHR-based dataset suggested comparable results to the VIEW RCT. [Ophthalmic Surg Lasers Imaging Retina 2023;54:6-14.].

Generating real-world evidence compatible with evidence from randomized controlled trials: a novel observational study design applicable to surgical transfusion research

BACKGROUND

Numerous observational studies have revealed an increased risk of death and complications with transfusion, but this observation has not been confirmed in randomized controlled trials (RCTs). The "transfusion kills patients" paradox persists in real-world observational studies despite application of analytic methods such as propensity-score matching. We propose a new design to address this long-term existing issue, which if left unresolved, will be deleterious to the healthy generation of evidence that supports optimized transfusion practice.

METHODS

In the new design, we stress three aspects for reconciling observational studies and RCTs on transfusion safety: (1) re-definition of the study population according to a stable hemoglobin range (gray zone of transfusion decision; 7.5-9.5 g/dL in this study); (2) selection of comparison groups according to a trigger value (last hemoglobin measurement before transfusion; nadir during hospital stay for control); (3) dealing with patient heterogeneity according to standardized mean difference (SMD) values. We applied the new design to hospitalized older patients (aged ≥60 years) undergoing general surgery at four academic/teaching hospitals. Four datasets were analyzed: a base population before (Base Match-) and after (Base Match+) propensity-score matching to simulate previous observational studies; a study population before (Study Match-) and after (Study Match+) propensity-score matching to demonstrate effects of our design.

RESULTS

Of 6141 older patients, 662 (10.78%) were transfused and showed high heterogeneity compared with those not receiving transfusion, particularly regarding preoperative hemoglobin (mean: 11.0 vs. 13.5 g/dL) and intraoperative bleeding (≥500 mL: 37.9% vs. 2.1%). Patient heterogeneity was reduced with the new design; SMD of the two variables was reduced from approximately 100% (Base Match-) to 0% (Study Match+). Transfusion was related to a higher risk of death and complications in Base Match- (odds ratio [OR], 95% confidence interval [CI]: 2.68, 1.86-3.86) and Base Match+ (2.24, 1.43-3.49), but not in Study Match- (0.77, 0.32-1.86) or Study Match+ (0.66, 0.23-1.89).

CONCLUSIONS

We show how choice of study population and analysis could affect real-world study findings. Our results following the new design are in accordance with relevant RCTs, highlighting its value in accelerating the pace of transfusion evidence generation and generalization.

Causal analyses with target trial emulation for real-world evidence removed large self-inflicted biases: systematic bias assessment of ovarian cancer treatment effectiveness

BACKGROUND AND OBJECTIVES

Drawing causal conclusions from real-world data (RWD) poses methodological challenges and risk of bias. We aimed to systematically assess the type and impact of potential biases that may occur when analyzing RWD using the case of progressive ovarian cancer.

METHODS

We retrospectively compared overall survival with and without second-line chemotherapy (LOT2) using electronic medical records. Potential biases were determined using directed acyclic graphs. We followed a stepwise analytic approach ranging from crude analysis and multivariable-adjusted Cox model up to a full causal analysis using a marginal structural Cox model with replicates emulating a reference randomized controlled trial (RCT). To assess biases, we compared effect estimates (hazard ratios [HRs]) of each approach to the HR of the reference trial.

RESULTS

The reference trial showed an HR for second line vs. delayed therapy of 1.01 (95% confidence interval [95% CI]: 0.82-1.25). The corresponding HRs from the RWD analysis ranged from 0.51 for simple baseline adjustments to 1.41 (95% CI: 1.22-1.64) accounting for immortal time bias with time-varying covariates. Causal trial emulation yielded an HR of 1.12 (95% CI: 0.96-1.28).

CONCLUSION

Our study, using ovarian cancer as an example, shows the importance of a thorough causal design and analysis if one is expecting RWD to emulate clinical trial results.

For insights into the real world, consider real-world data

Randomized control trials (RCTs) are required before drug and device approvals and have contributed to patient safety, but they have also increased the cost and time of regulatory assessments. We propose that using real-world evidence to complement or, in some settings, to replace RCTs will accelerate delivery of new drugs to patients.

How real can we get in generating real world evidence? Exploring the opportunities of routinely collected administrative data for evaluation of medical devices

Real-world data are considered a potentially valuable source of evidence for assessing medical technologies in clinical practice, but their widespread use is hampered by numerous challenges. Using the case of coronary stents in Italy, we investigate the potential of administrative databases for estimating costs and health outcomes associated with the use of medical devices in real world conditions. An administrative dataset was created ad hoc by merging hospital records from patients admitted between 2013 and 2019 for stent implantations with ambulatory records, pharmaceutical use data and vital statistics. Health outcomes were multifold: all-cause and cardiac mortality and myocardial infarction, within 30 days, 1, 2, 5 years. Costs were estimated from the National Health System perspective. We used multivariable Cox models and propensity score (PS) methods (PS matching; stratification on PS; inverse probability of treatment weighting using PS; PS adjustment). 257,907 coronary stents were implanted in 113,912 patients. For all health outcomes and follow-up times, and across all methods, patients receiving drug-eluting stents (DES) presented lower risk. For all-cause mortality, the DES patient advantage over bare-metal stent (BMS) patients declined over time but remained significant even at 5 years. For myocardial infarction, results remained quite stable. The DES group presented lower cumulative total costs (ranging from 3264 to 2363 Euros less depending on methods). Our results confirm the consolidated evidence of the benefits of DES compared to BMS. The consistency of results across methods suggests internal validity of the study, while highlighting strengths and limitations of each depending on research context. Administrative data yield great potential to perform comparative effectiveness and cost-effectiveness analysis of medical devices provided certain conditions are met.

Therapeutic indications and metabolic effects of metreleptin in patients with lipodystrophy syndromes: Real-life experience from a national reference network

AIM

To describe baseline characteristics and follow-up data in patients with lipodystrophy syndromes treated with metreleptin in a national reference network, in a real-life setting.

PATIENTS AND METHODS

Clinical and metabolic data from patients receiving metreleptin in France were retrospectively collected, at baseline, at 1 year and at the latest follow-up during treatment.

RESULTS

Forty-seven patients with lipodystrophy including generalized lipodystrophy (GLD; n = 28) and partial lipodystrophy (PLD; n = 19) received metreleptin over the last decade. At baseline, the median (interquartile range [IQR]) patient age was 29.3 (16.6-47.6) years, body mass index was 23.8 (21.2-25.7) kg/m² and serum leptin was 3.2 (1.0-4.9) ng/mL, 94% of patients had diabetes (66% insulin-treated), 53% had hypertension and 87% had dyslipidaemia. Metreleptin therapy, administered for a median (IQR) of 31.7 (14.2-76.0) months, was ongoing in 77% of patients at the latest follow-up. In patients with GLD, glycated haemoglobin (HbA1c) and fasting triglyceride levels significantly decreased from baseline to 1 year of metreleptin treatment, from 8.4 (6.5-9.9)% [68 (48-85) mmol/mol] to 6.8 (5.6-7.4)% [51(38-57) mmol/mol], and 3.6 (1.7-8.5) mmol/L to 2.2 (1.1-3.7) mmol/L, respectively (P < 0.001), with sustained efficacy thereafter. In patients with PLD, HbA1c was not significantly modified (7.7 [7.1-9.1]% [61 (54-76) mmol/mol] at baseline vs. 7.7 [7.4-9.5]% [61(57-80) mmol/mol] at 1 year), and the decrease in fasting triglycerides (from 3.3 [1.9-9.9] mmol/L to 2.5 [1.6-5.3] mmol/L; P < 0.01) was not confirmed at the latest assessment (5.2 [2.2-11.3] mmol/L). However, among PLD patients, at 1 year, 61% were responders regarding glucose homeostasis, with lower baseline leptin levels compared to nonresponders, and 61% were responders regarding triglyceridaemia. Liver enzymes significantly decreased only in the GLD group.

CONCLUSIONS

In this real-life setting study, metabolic outcomes are improved by metreleptin therapy in patients with GLD. The therapeutic indication for metreleptin needs to be clarified in patients with PLD.

A comprehensive model for measuring real-life cost-effectiveness in eyecare: automation in care and evaluation of system (aces-rwm™)

This paper describes a holistic, yet simple and comprehensible, ecosystem model to deal with multiple and complex challenges in eyecare. It aims at producing the best possible wellbeing and eyesight with the available resources. When targeting to improve the real-world cost-effectiveness, what gets done in everyday practice needs be measured routinely, efficiently and unselectively. Collection of all real-world data of all patients will enable evaluation and comparison of eyecare systems and departments between themselves nationally and internationally. The concept advocates a strategy to optimize real-life effectiveness, sustainability and outcomes of the service delivery in ophthalmology. The model consists of three components: (1) resource-governing principles (i.e., to deal with increasing demand and limited resources), (2) real-world monitoring (i.e., to collect structured real-world data utilizing automation and visualization of clinical parameters, health-related quality of life and costs), and (3) digital innovation strategy (i.e., to evaluate and benchmark real-world outcomes and cost-effectiveness). The core value and strength of the model lies in the consensus and collaboration of all Finnish university eye clinics to collect and evaluate the uniformly structured real-world outcomes data. In addition to ophthalmology, the approach is adaptable to any medical discipline to efficiently generate real-world insights and resilience in health systems.

Comparative effectiveness and safety of pharmaceuticals assessed in observational studies compared with randomized controlled trials

BACKGROUND

There have been ongoing efforts to understand when and how data from observational studies can be applied to clinical and regulatory decision making. The objective of this review was to assess the comparability of relative treatment effects of pharmaceuticals from observational studies and randomized controlled trials (RCTs).

METHODS

We searched PubMed and Embase for systematic literature reviews published between January 1, 1990, and January 31, 2020, that reported relative treatment effects of pharmaceuticals from both observational studies and RCTs. We extracted pooled relative effect estimates from observational studies and RCTs for each outcome, intervention-comparator, or indication assessed in the reviews. We calculated the ratio of the relative effect estimate from observational studies over that from RCTs, along with the corresponding 95% confidence interval (CI) for each pair of pooled RCT and observational study estimates, and we evaluated the consistency in relative treatment effects.

RESULTS

Thirty systematic reviews across 7 therapeutic areas were identified from the literature. We analyzed 74 pairs of pooled relative effect estimates from RCTs and observational studies from 29 reviews. There was no statistically significant difference (based on the 95% CI) in relative effect estimates between RCTs and observational studies in 79.7% of pairs. There was an extreme difference (ratio < 0.7 or > 1.43) in 43.2% of pairs, and, in 17.6% of pairs, there was a significant difference and the estimates pointed in opposite directions.

CONCLUSIONS

Overall, our review shows that while there is no significant difference in the relative risk ratios between the majority of RCTs and observational studies compared, there is significant variation in about 20% of comparisons. The source of this variation should be the subject of further inquiry to elucidate how much of the variation is due to differences in patient populations versus biased estimates arising from issues with study design or analytical/statistical methods.

Leveraging external data in the design and analysis of clinical trials in neuro-oncology

Integration of external control data, with patient-level information, in clinical trials has the potential to accelerate the development of new treatments in neuro-oncology by contextualising single-arm studies and improving decision making (eg, early stopping decisions). Based on a series of presentations at the 2020 Clinical Trials Think Tank hosted by the Society of Neuro-Oncology, we provide an overview on the use of external control data representative of the standard of care in the design and analysis of clinical trials. High-quality patient-level records, rigorous methods, and validation analyses are necessary to effectively leverage external data. We review study designs, statistical methods, risks, and potential distortions in using external data from completed trials and real-world data, as well as data sources, data sharing models, ongoing work, and applications in glioblastoma.

Renal recipients' knowledge and self-efficacy during first year after implementing an evidence based educational intervention as routine care at the transplantation clinic

BACKGROUND

Following an implementation plan based on dynamic dialogue between researchers and clinicians, this study implemented an evidence-based patient education program (tested in an RCT) into routine care at a clinical transplant center. The aim of this study was to investigate renal recipients' knowledge and self-efficacy during first year the after the intervention was provided in an everyday life setting.

METHODS

The study has a longitudinal design. The sample consisted of 196 renal recipients. Measurement points were 5 days (baseline), 2 months (T1), 6 months (T2), and one-year post transplantation (T3). Outcome measures were post-transplant knowledge, self-efficacy, and self-perceived general health.

RESULTS

No statistically significant changes were found from baseline to T1, T2, and T3. Participants' levels of knowledge and self-efficacy were high prior to the education program and did not change throughout the first year post transplantation.

CONCLUSION

Renal recipients self-efficacy and insight in post-transplant aspects seem to be more robust when admitted to the hospital for transplantation compared to baseline observations in the RCT study. This may explain why the implemented educational intervention did not lead to the same positive increase in outcome measures as in the RCT. This study supports that replicating clinical interventions in real-life settings may provide different results compared to results from RCT's. In order to gain a complete picture of the impacts of an implemented intervention, it is vital also to evaluate results after implementing findings from RCT-studies into everyday practice.