Emulation of Randomized Clinical Trials With Nonrandomized Database Analyses: Results of 32 Clinical Trials

Exploratory subgroup identification in the heterogeneous Cox model: A relatively simple procedure

For survival analysis applications we propose a novel procedure for identifying subgroups with large treatment effects, with focus on subgroups where treatment is potentially detrimental. The approach, termed forest search, is relatively simple and flexible. All-possible subgroups are screened and selected based on hazard ratio thresholds indicative of harm with assessment according to the standard Cox model. By reversing the role of treatment one can seek to identify substantial benefit. We apply a splitting consistency criteria to identify a subgroup considered "maximally consistent with harm." The type-1 error and power for subgroup identification can be quickly approximated by numerical integration. To aid inference we describe a bootstrap bias-corrected Cox model estimator with variance estimated by a Jacknife approximation. We provide a detailed evaluation of operating characteristics in simulations and compare to virtual twins and generalized random forests where we find the proposal to have favorable performance. In particular, in our simulation setting, we find the proposed approach favorably controls the type-1 error for falsely identifying heterogeneity with higher power and classification accuracy for substantial heterogeneous effects. Two real data applications are provided for publicly available datasets from a clinical trial in oncology, and HIV.

Two Weeks Versus One Week of Maximal Patient-Intensivist Continuity for Adult Medical Intensive Care Patients: A Two-Center Target Trial Emulation

OBJECTIVES

To compare outcomes for 2 weeks vs. 1 week of maximal patient-intensivist continuity in the ICU.

DESIGN

Retrospective cohort study.

SETTING

Two U.S. urban, teaching, medical ICUs where intensivists were scheduled for 2-week service blocks: site A was in the Midwest and site B was in the Northeast.

PATIENTS

Patients 18 years old or older admitted to a study ICU between March 1, 2017, and February 28, 2020.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We applied target trial emulation to compare admission during an intensivist's first week (as a proxy for 2 wk of maximal continuity) vs. admission during their second week (as a proxy for 1 wk of maximal continuity). Outcomes included hospital mortality, ICU length of stay, and, for mechanically ventilated patients, duration of ventilation. Exploratory outcomes included imaging, echocardiogram, and consultation orders. We used inverse probability weighting to adjust for baseline differences and random-effects meta-analysis to calculate overall effect estimates. Among 2571 patients, 1254 were admitted during an intensivist's first week and 1317 were admitted during a second week. At sites A and B, hospital mortality rates were 25.8% and 24.2%, median ICU length of stay were 4 and 2 days, and median mechanical ventilation durations were 3 and 3 days, respectively. There were no differences in adjusted mortality (odds ratio [OR], 1.01 [95% CI, 0.96-1.06]) or ICU length of stay (-0.25 d [-0.82 d to +0.32 d]) for 2 weeks vs. 1 week of maximal continuity. Among mechanically ventilated patients, there were no differences in adjusted mortality (OR, 1.00 [0.87-1.16]), ICU length of stay (+0.06 d [-0.78 d to +0.91 d]), or duration of mechanical ventilation (+0.37 d [-0.46 d to +1.21 d]) for 2 weeks vs. 1 week of maximal continuity.

CONCLUSIONS

Two weeks of maximal patient-intensivist continuity was not associated with differences in clinical outcomes compared with 1 week in two medical ICUs.

Frailty and Clinical Outcomes of Ticagrelor Versus Clopidogrel in Older Adults With Acute Myocardial Infarction

BACKGROUND

Ticagrelor is recommended over clopidogrel in acute coronary syndrome based on the results of the PLATO (Study of Platelet Inhibition and Patient Outcomes) trial. We aimed to emulate PLATO in older adults with and without frailty and with acute coronary syndrome treated with percutaneous coronary intervention.

METHODS AND RESULTS

We created a new-user cohort of Medicare fee-for-service beneficiaries aged ≥65 years hospitalized for acute coronary syndrome from 2014 to 2018 and initiated ticagrelor or clopidogrel following percutaneous coronary intervention. Frailty was defined using a validated claims-based frailty index ≥0.25. Coprimary outcomes were major adverse cardiovascular events and major bleeding. Follow-up began on the date of first outpatient prescription for ticagrelor or clopidogrel and ended on the earliest date for an outcome event, death, discontinuation of the index drug, or disenrollment from Medicare. The study included 42 843 older adults; 23% were frail. After propensity score matching, the rates of major adverse cardiovascular events per 100 person-years comparing ticagrelor versus clopidogrel groups were 7.8 and 7.3 in the frail cohort (hazard ratio [HR], 1.07 [95% CI, 0.84-1.36]) and 3.7 and 4.2 in the nonfrail cohort (HR, 0.87 [95% CI, 0.75-1.02]). The corresponding rates of major bleeding were 4.3 and 3.8 in the frail cohort (HR, 1.12 95% CI, [0.80-1.56]) and 2.2 and 1.8 in the nonfrail cohort (HR, 1.22 [95% CI, 0.98-1.51]).

CONCLUSIONS

There was a trend toward a modest reduction in risk of major adverse cardiovascular events and a trend toward a modest increase in risk of major bleeding with ticagrelor compared with clopidogrel in the nonfrail cohort. There was insufficient evidence for the benefit of ticagrelor in frail older adults.

Why Evidence Generation Should Matter to Payers and How They Can Help

Importance

The US leads the world in bringing new medical products to market, but the ability to generate evidence to inform clinical practice in postmarket settings needs improvement. Although a diverse group of stakeholders is working to improve postmarket evidence generation, the role of private payers has been underappreciated.

Observations

Payers are crucial allies in improving evidence generation because better data would better inform coverage decisions, their policies and practices influence the conduct of care and research, and their claims data are a source of real-world evidence used in medical product evaluation. In addition, payers have a stake in improving evidence generation because the kinds of evidence needed to inform health care and coverage decisions are often not available when a product enters the market and may not be generated without their involvement. Here, we describe several key steps payers could take to improve evidence generation, including participating in efforts to reduce administrative and financial barriers to the conduct of clinical trials, directly incentivizing evidence generation on high-priority questions by funding potential cost-saving trials, increasing engagement with the medical products industry on evidentiary needs for coverage decisions, and improving usability of claims data by reducing data lags and routinely recording unique device identifiers. Broad payer engagement with US Food and Drug Administration recommendations regarding evidence generation will ensure that the opportunities to participate in clinical research are extended to all communities and that evidence needed to inform care is generated in trials and surveillance systems that reflect the clinical reality across the US.

Conclusions and Relevance

Increasing payer involvement in evidence generation can benefit all participants in the medical innovation ecosystem. The importance of payers in these efforts will continue to grow in response to imperatives to increase integration of care and research, engage a diverse set of communities in clinical research, and move toward alternative payment models.

Examining the Effect of Missing Data and Unmeasured Confounding on External Comparator Studies: Case Studies and Simulations

BACKGROUND AND OBJECTIVE

Missing data and unmeasured confounding are key challenges for external comparator studies. This work evaluates bias and other performance characteristics depending on missingness and unmeasured confounding by means of two case studies and simulations.

METHODS

Two case studies were constructed by taking the treatment arms from two randomised controlled trials and an external real-world data source that exhibited substantial missingness. The indications of the randomised controlled trials were multiple myeloma and metastatic hormone-sensitive prostate cancer. Overall survival was taken as the main endpoint. The effects of missing data and unmeasured confounding were assessed for the case studies by reporting estimated external comparator versus randomised controlled trial treatment effects. Based on the two case studies, simulations were performed broadening the settings by varying the underlying hazard ratio, the sample size, the sample size ratio between the experimental arm and the external comparator, the number of missing covariates and the percentage of missingness. Thereby, bias and other performance metrics could be quantified dependent on these factors.

RESULTS

For the multiple myeloma external comparator study, results were in line with the randomised controlled trial, despite missingness and potential unmeasured confounding, while for the metastatic hormone-sensitive prostate cancer case study missing data led to a low sample size, leading overall to inconclusive results. Furthermore, for the metastatic hormone-sensitive prostate cancer study, missing data in important eligibility criteria led to further limitations. Simulations were successfully applied to gain a quantitative understanding of the effects of missing data and unmeasured confounding.

CONCLUSIONS

This exploratory study confirmed external comparator strengths and limitations by quantifying the impact of missing data and unmeasured confounding using case studies and simulations. In particular, missing data in key eligibility criteria were seen to limit the ability to derive the external comparator target analysis population accurately, while simulations demonstrated the magnitude of bias to expect for various settings.

RWE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 16

In this update, we discuss recent US FDA guidance offering more specific guidelines on appropriate study design and analysis to support causal inference for non-interventional studies and the launch of the European Medicines Agency (EMA) and the Heads of Medicines Agencies (HMA) public electronic catalogues. We also highlight an article recommending assessing data quality and suitability prior to protocol finalization and a Journal of the American Medical Association-endorsed framework for using causal language when publishing real-world evidence studies. Finally, we explore the potential of large language models to automate the development of health economic models.

Comparative Effectiveness and Safety of Apixaban, Rivaroxaban, and Warfarin in Patients With Cirrhosis and Atrial Fibrillation : A Nationwide Cohort Study

BACKGROUND

Apixaban, rivaroxaban, and warfarin have shown benefit for preventing major ischemic events, albeit with increased bleeding risk, among patients in the general population with atrial fibrillation (AF). However, data are scarce in patients with cirrhosis and AF.

OBJECTIVE

To compare the effectiveness and safety of apixaban versus rivaroxaban and versus warfarin in patients with cirrhosis and AF.

DESIGN

Population-based cohort study.

SETTING

Two U.S. claims data sets (Medicare and Optum's de-identified Clinformatics Data Mart Database [2013 to 2022]).

PARTICIPANTS

1:1 propensity score (PS)-matched patients with cirrhosis and nonvalvular AF initiating use of apixaban, rivaroxaban, or warfarin.

MEASUREMENTS

Primary outcomes included ischemic stroke or systemic embolism and major hemorrhage (intracranial hemorrhage or major gastrointestinal bleeding). Database-specific and pooled PS-matched rate differences (RDs) per 1000 person-years (PY) and Cox proportional hazard ratios (HRs) with 95% CIs were estimated, controlling for 104 preexposure covariates.

RESULTS

Rivaroxaban initiators had significantly higher rates of major hemorrhagic events than apixaban initiators (RD, 33.1 per 1000 PY [95% CI, 12.9 to 53.2 per 1000 PY]; HR, 1.47 [CI, 1.11 to 1.94]) but no significant differences in rates of ischemic events or death. Consistently higher rates of major hemorrhage were found with rivaroxaban across subgroup and sensitivity analyses. Warfarin initiators also had significantly higher rates of major hemorrhage than apixaban initiators (RD, 26.1 per 1000 PY [CI, 6.8 to 45.3 per 1000 PY]; HR, 1.38 [CI, 1.03 to 1.84]), particularly hemorrhagic stroke (RD, 9.7 per 1000 PY [CI, 2.2 to 17.2 per 1000 PY]; HR, 2.85 [CI, 1.24 to 6.59]).

LIMITATION

Nonrandomized treatment selection.

CONCLUSION

Among patients with cirrhosis and nonvalvular AF, initiators of rivaroxaban versus apixaban had significantly higher rates of major hemorrhage and similar rates of ischemic events and death. Initiation of warfarin versus apixaban also contributed to significantly higher rates of major hemorrhagic events, including hemorrhagic stroke.

PRIMARY FUNDING SOURCE

National Institutes of Health.

Rise in First-Time ERCP for Benign Indications >1 Year After Cholecystectomy Is Associated With Worse Outcomes

BACKGROUND & AIMS

Greater availability of less invasive biliary imaging to rule out choledocholithiasis should reduce the need for diagnostic endoscopic retrograde cholangiopancreatography (ERCP) in patients who have a remote history of cholecystectomy. The primary aims were to determine the incidence, characteristics, and outcomes of individuals who undergo first-time ERCP >1 year after cholecystectomy (late-ERCP).

METHODS

Data from a commercial insurance claim database (Optum Clinformatics) identified 583,712 adults who underwent cholecystectomy, 4274 of whom underwent late-ERCP, defined as first-time ERCP for nonmalignant indications >1 year after cholecystectomy. Outcomes were exposure and temporal trends in late-ERCP, biliary imaging utilization, and post-ERCP outcomes. Multivariable logistic regression was used to examine patient characteristics associated with undergoing late-ERCP.

RESULTS

Despite a temporal increase in the use of noninvasive biliary imaging (35.9% in 2004 to 65.6% in 2021; P < .001), the rate of late-ERCP increased 8-fold (0.5-4.2/1000 person-years from 2005 to 2021; P < .001). Although only 44% of patients who underwent late-ERCP had gallstone removal, there were high rates of post-ERCP pancreatitis (7.1%), hospitalization (13.1%), and new chronic opioid use (9.7%). Factors associated with late-ERCP included concomitant disorder of gut-brain interaction (odds ratio [OR], 6.48; 95% confidence interval [CI], 5.88-6.91) and metabolic dysfunction steatotic liver disease (OR, 3.27; 95% CI, 2.79-3.55) along with use of anxiolytic (OR, 3.45; 95% CI, 3.19-3.58), antispasmodic (OR, 1.60; 95% CI, 1.53-1.72), and chronic opioids (OR, 6.24; 95% CI, 5.79-6.52).

CONCLUSIONS

The rate of late-ERCP postcholecystectomy is increasing significantly, particularly in patients with comorbidities associated with disorder of gut-brain interaction and mimickers of choledocholithiasis. Late-ERCPs are associated with disproportionately higher rates of adverse events, including initiation of chronic opioid use.

A Comprehensive Framework for Evaluating the Value Created by Real-World Evidence for Diverse Stakeholders: The Case for Coordinated Registry Networks

OBJECTIVES

This manuscript presents a comprehensive framework for the assessment of the value of real-world evidence (RWE) in healthcare decision-making. While RWE has been proposed to overcome some limitations of traditional, one-off studies, no systematic framework exists to measure if RWE actually lowers the burden. This framework aims to fill that gap by providing conceptual approaches for evaluating the time and cost efficiencies of RWE, thus guiding strategic investments in RWE infrastructure.

METHODS

The framework consists of four components: (114th Congress. 21st Century Cures Act.; 2015. https://www.congress.gov/114/plaws/publ255/PLAW-114publ255.pdf .) identification of stakeholders using and producing RWE, (National Health Council. Glossary of Patient Engagement Terms. Published 2019. Accessed May 18. 2021. https://nationalhealthcouncil.org/glossary-of-patient-engagement-terms/ .) understanding value propositions on how RWE can benefit stakeholders, (Center for Drug Evaluation and Research. CDER Patient-Focused Drug Development. U.S. Food & Drug Administration.) defining key performance indicators (KPIs), and (U.S. Department of Health and Human Services - Food and Drug Administration: Center for Devices and Radiological Health and Center for Biologics Evaluation and Research. Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices - Guidance for Industry and Food and Drug Administration Staff. 2017. http://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/Guida .) establishing metrics and case studies to assess value. KPIs are categorized as 'better, faster, or cheaper" as an indicator of value: better focusing on high-quality actionable evidence; 'faster,' denoting time-saving in evidence generation, and 'cheaper,' emphasizing cost-efficiency decision compared to methodologies that do not involve data routinely collected in clinical practice. Metrics and relevant case studies are tailored based on stakeholder value propositions and selected KPIs that can be used to assess what value has been created by using RWE compared to traditional evidence-generation approaches and comparing different RWE sources.

RESULTS

Operationalized through metrics and case studies drawn from the literature, the value of RWE is documented as improving treatment effect heterogeneity evaluation, expanding medical product labels, and expediting post-market compliance. RWE is also shown to reduce the cost and time required to produce evidence compared to traditional one-off approaches. An original example of a metric that measures the time saved by RWE methods to detect a signal of a product failure was presented based on analysis of the National Cardiovascular Disease Registry.

CONCLUSIONS

The framework presented in this manuscript offers a comprehensive approach for evaluating the value of RWE, applicable to all stakeholders engaged in leveraging RWE for healthcare decision-making. Through the proposed metrics and illustrated case studies, valuable insights are provided into the heightened efficiency, cost-effectiveness, and improved decision-making within clinical and regulatory domains facilitated by RWE. While this framework is primarily focused on medical devices, it could potentially inform the determination of RWE value in other medical products. By discerning the variations in cost, time, and data utility among various evidence-generation methods, stakeholders are empowered to invest strategically in RWE infrastructure and shape future research endeavors.

The evidence base of the 10 most prescribed drugs in England, France, and the United States: a scoping review

OBJECTIVES

To evaluate whether there is evidence of efficacy of the most commonly used medications in their primary indications.

STUDY DESIGN AND SETTING

This scoping review was executed using the Cochrane Library and MEDLINE databases up to May 2023. The 10 most prescribed medications in England, France, and the United States were identified using country-specific public databases. Up to 3 common indications in primary care were defined for each medication, based on a survey of general practitioners. The outcomes were determined by the authors to be patient-important outcomes, with placebo as the comparator. Two investigators independently conducted searches, following a predefined algorithm, to identify randomized controlled trials or meta-analyses of randomized controlled trials assessing the efficacy of these medications for each indication. The risk of bias was assessed using the ROBIS or ROB 2.0 tools.

RESULTS

We identified 21 drugs, covering 56 indications and 114 outcomes. Sixty-seven percent of the evaluated medications demonstrated efficacy for at least one outcome in at least one of the sought indications. Overall, evidence of efficacy was found for 48% of the indications. There was no study evaluating the efficacy of amoxicillin and salbutamol. For other drugs such as phloroglucinol or cholecalciferol, available studies suggested an absence of efficacy in the most common indications.

CONCLUSION

This study underscores the lack of data regarding the level of evidence for the most prescribed medications. Limitations include the choice of outcomes, and the understanding that the absence of evidence is not synonymous with the absence of efficacy.

Why Should the FDA Focus on Pragmatic Clinical Research?

This Viewpoint from the FDA discusses how pragmatic clinical research—assessment that uses real-world data, often in combination with research data, after initial marketing approval—can help in evaluation of new technologies, benefit research sites in underresourced settings, and better inform regulatory decisions and clinical practice.

Target trial framework for determining the effect of changes in training load on injury risk using observational data: a methodological commentary

In recent years, a large focus has been placed on managing training load for injury prevention. To minimise injuries, training recommendations should be based on research that examines causal relationships between load and injury risk. While observational studies can be used to estimate causal effects, conventional methods to study the relationship between load and injury are prone to bias. The target trial framework is a valuable tool that requires researchers to emulate a hypothetical randomised trial using observational data. This framework helps to explicitly define research questions and design studies in a way that estimates causal effects. This article provides an overview of the components of the target trial framework as applied to studies on load and injury and describes various considerations that should be made in study design and analyses to minimise bias.

Enhancing External Control Arm Analyses through Data Calibration and Hybrid Designs

Almost all external control arm analyses to contextualize findings of a single arm trial struggle with two key issues: the lack of baseline randomization, and equally important, the difference in data collection between the experimental arm with its primary data collection, and the external control arm using secondary data. We illustrate the data calibration design to remedy issues arising from differential measurements in the two arms, and discuss the hybrid design that expands an underpowered randomized internal control arm with real-world data to mitigate the lack of randomization of the external control arm. We show how the two approaches fit into an evidence-development strategy that naturally builds on the incremental insights gained.

Selection for transcatheter versus surgical aortic valve replacement and mid-term survival: results of the AUTHEARTVISIT study

OBJECTIVES

Limited data are available from randomized trials comparing outcomes between transcatheter aortic valve replacement (TAVR) and surgery in patients with different risks and with follow-up of at least 4 years or longer. In this large, population-based cohort study, long-term mortality and morbidity were investigated in patients undergoing aortic valve replacement (AVR) for severe aortic stenosis using a surgically implanted bioprosthesis (surgical/biological aortic valve replacement; sB-AVR) or TAVR.

METHODS

Individual data from the Austrian Insurance Funds from 2010 through 2020 were analysed. The primary outcome was all-cause mortality, assessed in the overall and propensity score-matched populations. Secondary outcomes included reoperation and cardiovascular events.

RESULTS

From January 2010 through December 2020, a total of 18 882 patients underwent sB-AVR (n = 11 749; 62.2%) or TAVR (n = 7133; 37.8%); median follow-up was 5.8 (95% CI 5.7-5.9) years (maximum 12.3 years). The risk of all-cause mortality was higher with TAVR compared with sB-AVR: hazard ratio 1.552, 95% confidence interval (CI) 1.469-1.640, P < 0.001; propensity score-matched hazard ratio 1.510, 1.403-1.625, P < 0.001. Estimated median survival was 8.8 years (95% CI 8.6-9.1) with sB-AVR versus 5 years (4.9-5.2) with TAVR. Estimated 5-year survival probability was 0.664 (0.664-0.686) with sB-AVR versus 0.409 (0.378-0.444) with TAVR overall, and 0.690 (0.674-0.707) and 0.560 (0.540-0.582), respectively, with propensity score matching. Separate subgroup analyses for patients aged 65-75 years and >75 years indicated a significant survival benefit in patients selected for sB-AVR in both groups. Other predictors of mortality were age, sex, previous heart failure, diabetes and chronic kidney disease.

CONCLUSIONS

In this retrospective national population-based study, selection for TAVR was significantly associated with higher all-cause mortality compared with sB-AVR in patients ≥65 years with severe, symptomatic aortic stenosis in the >2-year follow-up.

Effect of surgery versus chemotherapy in pancreatic cancer patients: a target trial emulation

BACKGROUND

To estimate the causal effect of surgery vs chemotherapy on survival in patients with T1-3NxM0 pancreatic cancer in a rigorous framework addressing selection bias and immortal time bias.

METHODS

We used population-based Danish health-care registries to conduct a cohort study emulating a hypothetical randomized trial to estimate the absolute difference in survival, comparing surgery with chemotherapy. We included pancreatic cancer patients diagnosed during 2008-2021. Exposure was surgery or chemotherapy initiated within a 16-week grace period after diagnosis. At the time of diagnosis, data of each patient were duplicated; one copy was assigned to the surgery protocol, and one copy to the chemotherapy protocol of the hypothetical trial. Copies were censored when the assigned treatment deviated from the observed treatment. To account for informative censoring, uncensored patients were weighted according to confounders. For comparison, we also applied a more conventional analysis using propensity score-based inverse probability weighting.

RESULTS

We included 1744 patients with a median age of 68 years: 73.6% underwent surgery, and 18.6% had chemotherapy without surgery; 7.8% received no treatment. The 3-year survival was 39.7% (95% confidence interval [CI] = 36.7% to 42.6%) after surgery and 22.7% (95% CI = 17.7% to 28.4%) after chemotherapy, corresponding to an absolute difference of 17.0% (95% CI = 10.8% to 23.1%). In the conventional survival analysis, this difference was 23.0% (95% CI = 17.0% to 29.0%).

CONCLUSION

Surgery was superior to chemotherapy in achieving long-term survival for pancreatic cancer. The difference comparing surgery and chemotherapy was substantially smaller when using the clone-censor-weight approach than conventional survival analysis.

Target trial emulation for comparative effectiveness research with observational data: Promise and challenges for studying medications for opioid use disorder

Medications for opioid use disorder (MOUD) increase retention in care and decrease mortality during active treatment; however, information about the comparative effectiveness of different forms of MOUD is sparse. Observational comparative effectiveness studies are subject to many types of bias; a robust framework to minimize bias would improve the quality of comparative effectiveness evidence. This paper discusses the use of target trial emulation as a framework to conduct comparative effectiveness studies of MOUD with administrative data. Using examples from our planned research project comparing buprenorphine-naloxone and extended-release naltrexone with respect to the rates of MOUD discontinuation, we provide a primer on the challenges and approaches to employing target trial emulation in the study of MOUD.

Effectiveness of COVID-19 vaccines against severe COVID-19 among patients with cancer in Catalonia, Spain

Patients with cancer were excluded from pivotal randomized clinical trials of COVID-19 vaccine products, and available observational evidence on vaccine effectiveness (VE) focused mostly on mild, and not severe COVID-19, which is the ultimate goal of vaccination for high-risk groups. Here, using primary care electronic health records from Catalonia, Spain (SIDIAP), we built two large cohorts of vaccinated and matched control cancer patients with a primary vaccination scheme (n = 184,744) and a booster (n = 108,534). Most patients received a mRNA-based product in primary (76.2%) and booster vaccination (99.9%). Patients had 51.8% (95% CI 40.3%-61.1%) and 58.4% (95% CI 29.3%-75.5%) protection against COVID-19 hospitalization and COVID-19 death respectively after full vaccination (two-doses) and 77.9% (95% CI 69.2%-84.2%) and 80.2% (95% CI 63.0%-89.4%) after booster. Compared to primary vaccination, the booster dose provided higher peak protection during follow-up. Calibration of VE estimates with negative outcomes, and sensitivity analyses with slight different population and COVID-19 outcomes definitions provided similar results. Our results confirm the role of primary and booster COVID-19 vaccination in preventing COVID-19 severe events in patients with cancer and highlight the need for the additional dose in this population.

Emulating randomised clinical trials in relapsing-remitting multiple sclerosis with non-randomised real-world evidence: an application using data from the MSBase Registry

BACKGROUND

To mimic as closely as possible a randomised controlled trial (RCT) and calibrate the real-world evidence (RWE) studies against a known treatment effect would be helpful to understand if RWE can support causal conclusions in selected circumstances. The aim was to emulate the TRANSFORMS trial comparing Fingolimod (FTY) versus intramuscular interferon β-1a (IFN) using observational data.

METHODS

We extracted from the MSBase registry all the patients with relapsing-remitting multiple sclerosis (RRMS) collected in the period 2011-2021 who received IFN or FTY (0.5 mg) and with the same inclusion and exclusion criteria of the TRANSFORMS RCT. The primary endpoint was the annualised relapse rate (ARR) over 12 months. Patients were 1:1 propensity-score (PS) matched. Relapse-rate ratio (RR) was calculated by mean of a negative binomial regression.

RESULTS

A total of 4376 patients with RRMS (1140 in IFN and 3236 in FTY) were selected. After PS, 856 patients in each group were matched. The ARR was 0.45 in IFN and 0.25 in FTY with a significant difference between the two groups (RR: 0.55, 95% CI: 0.45 to 0.68; p<0.001). The result of the emulation was very similar and fell within the 95% CI of that observed in the RCT (RR: 0.49, 95% CI: 0.37 to 0.64; p<0.001) with a standardised difference of 0.66 (p=0.51).

CONCLUSIONS

By applying the same inclusion and exclusion criteria used in the RCT and employing appropriate methodology, we successfully replicated the RCT results with only minor discrepancies. Also, even if the confounding bias cannot be fully eliminated, conducting a rigorous target trial emulation could still yield valuable insights for comparative effectiveness research.

Use of an emulated trial to investigate the association between use of nitrogen-based bisphosphonates and risk of epithelial ovarian cancer

BACKGROUND

Epithelial ovarian cancer (EOC) is the eighth most common cancer in women, with poor survival outcomes. Observational evidence suggests that nitrogen-based bisphosphonate (NBB) use may be associated with reduced risk of EOC, particularly the endometrioid and serous histotypes; however, confounding by indication is a concern. An alternative approach to investigate the chemo-preventive potential of NBBs is to emulate a target trial by identifying all women who initiate use of NBBs and investigate the risk of EOC for continued users compared with discontinued users.

METHODS

Using population-based linked data, we identified all Australian women aged over 50 years who first used NBBs over 2004-12. We used the year after first use to define treatment for each woman as either continued or discontinued use. We emulated randomization using stabilized inverse probability weights to balance the treatment groups using covariates including age, comorbidities and socioeconomic status. We followed women from treatment assignment until EOC diagnosis, death or 31 December 2013. We assessed the risk of EOC (overall and by histotype) using flexible parametric time-to-event models allowing for time-varying effects, and produced time-varying coefficients.

RESULTS

Of the 313 383 women in the study, 472 were diagnosed with EOC during follow-up (261 serous EOC), with an average age at diagnosis of 72 years. Continued use of NBBs was associated with reduced risk of EOC overall (HR = 0.87, 95% CI: 0.69, 1.10), and serous EOC (HR = 0.71, 95% CI: 0.53, 0.96), compared with discontinued treatment, with estimates remaining constant over the 9-year follow-up.

CONCLUSIONS

Results from our emulated trial suggest that in women who initiated NBB treatment, those who continued use had 13% and 29% lower hazards of being diagnosed with EOC overall and serous EOC, respectively, compared with women who discontinued use.

Target Trial Emulation: Improving the Quality of Observational Studies in Inflammatory Bowel Disease Using the Principles of Randomized Trials

The past decade has seen a substantial increase in the number of randomized controlled trials (RCTs) conducted in inflammatory bowel disease (IBD). Randomized controlled trials are the gold standard method for generating robust evidence of drug safety and efficacy but are expensive, time-consuming, and may have ethical implications. Observational studies in IBD are often used to fill the gaps in evidence but are typically hindered by significant bias. There are several approaches for making statistical inferences from observational data with some that focus on study design and others on statistical techniques. Target trial emulation is an emerging methodological process that aims to bridge this gap and improve the quality of observational studies by applying the principles of an ideal, or "target," randomized trial to routinely collected clinical data. There has been a rapid expansion of observational studies that have emulated trials over the past 5 years in other medical fields, but this has yet to be adopted in gastroenterology and IBD. The wealth of nonrandomized clinical data available through electronic health records, patient registries, and administrative health databases afford innumerable hypothesis-generating opportunities for IBD research. This review outlines the principles of target trial emulation, discusses the merits to IBD observational studies in reducing the most common biases and improving confidence in causality, and details the caveats of using this approach.

Lazertinib versus Platinum-Based Chemotherapy with Epidermal Growth Factor Receptor (EGFR)-Positive Non-Small-Cell Lung Cancer after Failing EGFR-Tyrosine Kinase Inhibitor: A Real-World External Comparator Study

BACKGROUND

Lazertinib is a third-generation tyrosine kinase inhibitor of epidermal growth factor receptor (EGFR-TKI) that selectively inhibit common EGFR mutation and T790M mutation in non-small-cell lung cancer (NSCLC) patients. No previous studies have compared lazertinib to platinum-based chemotherapy. We have compared lazertinib with platinum-based chemotherapy in EGFR-mutated NSCLC patients after previous EGFR-TKI therapy.

METHODS

We retrospectively compared 200 patients from LASER201, LASER301, and LASER-PMS studies to 334 patients who were treated with platinum-based chemotherapy after previous EGFR-TKI from the Samsung Medical Center. After propensity score matching (PSM), we selected 156 patients from each group. The primary outcome was progression-free survival (PFS), with overall survival (OS), objective response rate (ORR), and time to treatment discontinuation (TTD) as secondary outcomes.

RESULTS

The median follow-up of PFS was 15.61 months in the lazertinib group and 21.67 months in the external control group. The PFS was significantly longer in patients who were treated with lazertinib than those treated with platinum-based chemotherapy (10.97 months vs. 5.10 months; adjusted hazard ratio (HR) 0.40; 95% confidence interval (CI), 0.29-0.55; p < 0.01) after PSM. Lazertinib showed superior OS (32.23 months vs. 18.73 months; adjusted HR 0.45; 95% CI, 0.29-0.69; p < 0.001), ORR (64.1% vs. 47.4%), and TTD (11.66 months vs. 6.73 months; adjusted HR 0.54; 95% CI, 0.39-0.75; p < 0.001) compared to platinum-based chemotherapy.

CONCLUSION

Based on this retrospective, external control study, lazertinib has demonstrated significantly better efficacy compared with platinum-based chemotherapy. The external controls provide important context to evaluate efficacy in single-arm studies.

When Will Real-World Data Fulfill Its Promise to Provide Timely Insights in Oncology?

Randomized trials provide high-quality, internally consistent data on selected clinical questions, but lack generalizability for the aging population who are most often diagnosed with cancer and have comorbid conditions that may affect the interpretation of treatment benefit. The need for high-quality, relevant, and timely data is greater than ever. Promising solutions lie in the collection and analysis of real-world data (RWD), which can potentially provide timely insights about the patient's course during and after initial treatment and the outcomes of important subgroups such as the elderly, rural populations, children, and patients with greater social health needs. However, to inform practice and policy, real-world evidence must be created from trustworthy and comprehensive sources of RWD; these may include pragmatic clinical trials, registries, prospective observational studies, electronic health records (EHRs), administrative claims, and digital technologies. There are unique challenges in oncology since key parameters (eg, cancer stage, biomarker status, genomic assays, imaging response, side effects, quality of life) are not recorded, siloed in inaccessible documents, or available only as free text or unstructured reports in the EHR. Advances in analytics, such as artificial intelligence, may greatly enhance the ability to obtain more granular information from EHRs and support integrated diagnostics; however, they will need to be validated purpose by purpose. We recommend a commitment to standardizing data across sources and building infrastructures that can produce fit-for-purpose RWD that will provide timely understanding of the effectiveness of individual interventions.

Effectiveness of nirmatrelvir/ritonavir and molnupiravir in non-hospitalized adults with COVID-19: systematic review and meta-analysis of observational studies

OBJECTIVE

To determine the effectiveness of nirmatrelvir/ritonavir and molnupiravir among vaccinated and unvaccinated non-hospitalized adults with COVID-19.

METHODS

Observational studies of nirmatrelvir/ritonavir or molnupiravir compared to no antiviral drug treatment for COVID-19 in non-hospitalized adults with data on vaccination status were included. We searched MEDLINE, EMBASE, Scopus, Web of Science, WHO COVID-19 Research Database and medRxiv for reports published between 1 January 2022 and 8 November 2023. The primary outcome was a composite of hospitalization or mortality up to 35 days after COVID-19 diagnosis. Risk of bias was assessed with ROBINS-I. Risk ratios (RR), hazard ratios (HR) and risk differences (RD) were separately estimated using random-effects models.

RESULTS

We included 30 cohort studies on adults treated with nirmatrelvir/ritonavir (n = 462 279) and molnupiravir (n = 48 008). Nirmatrelvir/ritonavir probably reduced the composite outcome (RR 0.62, 95%CI 0.55-0.70; I2 = 0%; moderate certainty) with no evidence of effect modification by vaccination status (RR Psubgroup = 0.47). In five studies, RD estimates against the composite outcome for nirmatrelvir/ritonavir were 1.21% (95%CI 0.57% to 1.84%) in vaccinated and 1.72% (95%CI 0.59% to 2.85%) in unvaccinated subgroups.Molnupiravir may slightly reduce the composite outcome (RR 0.75, 95%CI 0.67-0.85; I2 = 32%; low certainty). Evidence of effect modification by vaccination status was inconsistent among studies reporting different effect measures (RR Psubgroup = 0.78; HR Psubgroup = 0.08). In two studies, RD against the composite outcome for molnupiravir were -0.01% (95%CI -1.13% to 1.10%) in vaccinated and 1.73% (95%CI -2.08% to 5.53%) in unvaccinated subgroups.

CONCLUSIONS

Among cohort studies of non-hospitalized adults with COVID-19, nirmatrelvir/ritonavir is effective against the composite outcome of severe COVID-19 independent of vaccination status. Further research and a reassessment of molnupiravir use among vaccinated adults are warranted.

REGISTRATION

PROSPERO CRD42023429232.

Use of the French National Health Data System (SNDS) in pharmacoepidemiology: A systematic review in its maturation phase

AIM OF THE STUDY

The French National Health Data System (SNDS) comprises healthcare data that cover 99% of the population (over 67 million individuals) in France. The aim of this study was to present an overview of published pharmacoepidemiological studies using the SNDS in its maturation phase.

METHODS

We conducted a systematic literature review of original research articles in the Pubmed and EMBASE databases from January 2012 until August 2018.

RESULTS

A total of 316 full-text articles were included, with an annual increase over the study period. Only 16 records were excluded after screening because they did not involve the SNDS but other French healthcare databases. The study design was clearly reported in only 66% of studies of which 57% were retrospective cohorts and 22% cross-sectional studies. The reported study objectives were drug utilization (65%), safety (22%) and effectiveness (9%). Almost all ATC groups were studied but the most frequent ones concerned the nervous system in 149 studies (49%), cardiovascular system drugs in 104 studies (34%) and anti-infectives for systemic use in 50 studies (16%).

CONCLUSION

The SNDS is of growing interest for studies on drug use and safety, which could be conducted more in specific populations, including children, pregnant women and the elderly, as these populations are often not included in clinical trials.

Computational Phenomapping of Randomized Clinical Trials to Enable Assessment of their Real-world Representativeness and Personalized Inference

Importance

Randomized clinical trials (RCTs) are the standard for defining an evidence-based approach to managing disease, but their generalizability to real-world patients remains challenging to quantify.

Objective

To develop a multidimensional patient variable mapping algorithm to quantify the similarity and representation of electronic health record (EHR) patients corresponding to an RCT and estimate the putative treatment effects in real-world settings based on individual treatment effects observed in an RCT.

Design

A retrospective analysis of the Treatment of Preserved Cardiac Function Heart Failure with an Aldosterone Antagonist Trial (TOPCAT; 2006-2012) and a multi-hospital patient cohort from the electronic health record (EHR) in the Yale New Haven Hospital System (YNHHS; 2015-2023).

Setting

A multicenter international RCT (TOPCAT) and multi-hospital patient cohort (YNHHS).

Participants

All TOPCAT participants and patients with heart failure with preserved ejection fraction (HFpEF) and ≥1 hospitalization within YNHHS.

Exposures

63 pre-randomization characteristics measured across the TOPCAT and YNNHS cohorts.

Main Outcomes and Measures

Real-world generalizability of the RCT TOPCAT using a multidimensional phenotypic distance metric between TOPCAT and YNHHS cohorts. Estimation of the individualized treatment effect of spironolactone use on all-cause mortality within the YNHHS cohort based on phenotypic distance from the TOPCAT cohort.

Results

There were 3,445 patients in TOPCAT and 11,712 HFpEF patients across five hospital sites. Across the 63 TOPCAT variables mapped by clinicians to the EHR, there were larger differences between TOPCAT and each of the 5 EHR sites (median SMD 0.200, IQR 0.037-0.410) than between the 5 EHR sites (median SMD 0.062, IQR 0.010-0.130). The synthesis of these differences across covariates using our multidimensional similarity score also suggested substantial phenotypic dissimilarity between the TOPCAT and EHR cohorts. By phenotypic distance, a majority (55%) of TOPCAT participants were closer to each other than any individual EHR patient. Using a TOPCAT-derived model of individualized treatment benefit from spironolactone, those predicted to derive benefit and receiving spironolactone in the EHR cohorts had substantially better outcomes compared with predicted benefit and not receiving the medication (HR 0.74, 95% CI 0.62-0.89).

Conclusions and Relevance

We propose a novel approach to evaluating the real-world representativeness of RCT participants against corresponding patients in the EHR across the full multidimensional spectrum of the represented phenotypes. This enables the evaluation of the implications of RCTs for real-world patients.

KEY POINTS

Question: How can we examine the multi-dimensional generalizability of randomized clinical trials (RCT) to real-world patient populations?Findings: We demonstrate a novel phenotypic distance metric comparing an RCT to real-world populations in a large multicenter RCT of heart failure patients and the corresponding patients in multisite electronic health records (EHRs). Across 63 pre-randomization characteristics, pairwise assessments of members of the RCT and EHR cohorts were more discordant from each other than between members of the EHR cohort (median standardized mean difference 0.200 [0.037-0.410] vs 0.062 [0.010-0.130]), with a majority (55%) of RCT participants closer to each other than any individual EHR patient. The approach also enabled the quantification of expected real world outcomes based on effects observed in the RCT.Meaning: A multidimensional phenotypic distance metric quantifies the generalizability of RCTs to a given population while also offering an avenue to examine expected real-world patient outcomes based on treatment effects observed in the RCT.

Comparative effectiveness of second line oral antidiabetic treatments among people with type 2 diabetes mellitus: emulation of a target trial using routinely collected health data

OBJECTIVE

To compare the effectiveness of three commonly prescribed oral antidiabetic drugs added to metformin for people with type 2 diabetes mellitus requiring second line treatment in routine clinical practice.

DESIGN

Cohort study emulating a comparative effectiveness trial (target trial).

SETTING

Linked primary care, hospital, and death data in England, 2015-21.

PARTICIPANTS

75 739 adults with type 2 diabetes mellitus who initiated second line oral antidiabetic treatment with a sulfonylurea, DPP-4 inhibitor, or SGLT-2 inhibitor added to metformin.

MAIN OUTCOME MEASURES

Primary outcome was absolute change in glycated haemoglobin A1c (HbA1c) between baseline and one year follow-up. Secondary outcomes were change in body mass index (BMI), systolic blood pressure, and estimated glomerular filtration rate (eGFR) at one year and two years, change in HbA1c at two years, and time to ≥40% decline in eGFR, major adverse kidney event, hospital admission for heart failure, major adverse cardiovascular event (MACE), and all cause mortality. Instrumental variable analysis was used to reduce the risk of confounding due to unobserved baseline measures.

RESULTS

75 739 people initiated second line oral antidiabetic treatment with sulfonylureas (n=25 693, 33.9%), DPP-4 inhibitors (n=34 464 ,45.5%), or SGLT-2 inhibitors (n=15 582, 20.6%). SGLT-2 inhibitors were more effective than DPP-4 inhibitors or sulfonylureas in reducing mean HbA1c values between baseline and one year. After the instrumental variable analysis, the mean differences in HbA1c change between baseline and one year were -2.5 mmol/mol (95% confidence interval (CI) -3.7 to -1.3) for SGLT-2 inhibitors versus sulfonylureas and -3.2 mmol/mol (-4.6 to -1.8) for SGLT-2 inhibitors versus DPP-4 inhibitors. SGLT-2 inhibitors were more effective than sulfonylureas or DPP-4 inhibitors in reducing BMI and systolic blood pressure. For some secondary endpoints, evidence for SGLT-2 inhibitors being more effective was lacking-the hazard ratio for MACE, for example, was 0.99 (95% CI 0.61 to 1.62) versus sulfonylureas and 0.91 (0.51 to 1.63) versus DPP-4 inhibitors. SGLT-2 inhibitors had reduced hazards of hospital admission for heart failure compared with DPP-4 inhibitors (0.32, 0.12 to 0.90) and sulfonylureas (0.46, 0.20 to 1.05). The hazard ratio for a ≥40% decline in eGFR indicated a protective effect versus sulfonylureas (0.42, 0.22 to 0.82), with high uncertainty in the estimated hazard ratio versus DPP-4 inhibitors (0.64, 0.29 to 1.43).

CONCLUSIONS

This emulation study of a target trial found that SGLT-2 inhibitors were more effective than sulfonylureas or DPP-4 inhibitors in lowering mean HbA1c, BMI, and systolic blood pressure and in reducing the hazards of hospital admission for heart failure (v DPP-4 inhibitors) and kidney disease progression (v sulfonylureas), with no evidence of differences in other clinical endpoints.

Data Checks Before Registering Study Protocols for Health Care Database Analyses

This Viewpoint discusses the challenges involved with secondary health care data collection vs primary data collection and provides a list of suggested data checks before registration of a study protocol using secondary data.

Emulation of randomized trials of direct oral anticoagulants with claims data and implications for new Factor XI inhibitors

Direct oral anticoagulants (DOACs) revolutionized the management of thromboembolic disorders. Clinical care may be further improved as Factor XIs undergo large-scale outcome trials. What role can non-randomized database studies play in expediting understanding of these drugs in clinical practice? The RCT-DUPLICATIVE Initiative emulated the design of eight DOAC randomized clinical trials (RCT) using non-randomized claims database studies. RCT study design parameters and measurements were closely emulated by the database studies and produced highly concordant results. The results of the single database study that did not meet all agreement metrics with the specific RCT it was emulating were aligned with a meta-analysis of six trials studying similar questions, suggesting the trial result was an outlier. Well-designed database studies using fit-for-purpose data came to the same conclusions as DOAC trials, illustrating how database studies could complement RCTs for Factor XI inhibitors-by accelerating insights in underrepresented populations, demonstrating effectiveness and safety in clinical practice, and testing broader indications.

The Next Horizon of Drug Development: External Control Arms and Innovative Tools to Enrich Clinical Trial Data

Conducting clinical trials (CTs) has become increasingly costly and complex in terms of designing and operationalizing. These challenges exist in running CTs on novel therapies, particularly in oncology and rare diseases, where CTs increasingly target narrower patient groups. In this study, we describe external control arms (ECA) and other relevant tools, such as virtualization and decentralized clinical trials (DCTs), and the ability to follow the clinical trial subjects in the real world using tokenization. ECAs are typically constructed by identifying appropriate external sources of data, then by cleaning and standardizing it to create an analysis-ready data file, and finally, by matching subjects in the external data with the subjects in the CT of interest. In addition, ECA tools also include subject-level meta-analysis and simulated subjects' data for analyses. By implementing the recent advances in digital health technologies and devices, virtualization, and DCTs, realigning of CTs from site-centric designs to virtual, decentralized, and patient-centric designs can be done, which reduces the patient burden to participate in the CTs and encourages diversity. Tokenization technology allows linking the CT data with real-world data (RWD), creating more comprehensive and longitudinal outcome measures. These tools provide robust ways to enrich the CT data for informed decision-making, reduce the burden on subjects and costs of trial operations, and augment the insights gained for the CT data.

Reverse total shoulder replacement versus anatomical total shoulder replacement for osteoarthritis: population based cohort study using data from the National Joint Registry and Hospital Episode Statistics for England

OBJECTIVES

To answer a national research priority by comparing the risk-benefit and costs associated with reverse total shoulder replacement (RTSR) and anatomical total shoulder replacement (TSR) in patients having elective primary shoulder replacement for osteoarthritis.

DESIGN

Population based cohort study using data from the National Joint Registry and Hospital Episode Statistics for England.

SETTING

Public hospitals and publicly funded procedures at private hospitals in England, 2012-20.

PARTICIPANTS

Adults aged 60 years or older who underwent RTSR or TSR for osteoarthritis with intact rotator cuff tendons. Patients were identified from the National Joint Registry and linked to NHS Hospital Episode Statistics and civil registration mortality data. Propensity score matching and inverse probability of treatment weighting were used to balance the study groups.

MAIN OUTCOME MEASURES

The main outcome measure was revision surgery. Secondary outcome measures included serious adverse events within 90 days, reoperations within 12 months, prolonged hospital stay (more than three nights), change in Oxford Shoulder Score (preoperative to six month postoperative), and lifetime costs to the healthcare service.

RESULTS

The propensity score matched population comprised 7124 RTSR or TSR procedures (126 were revised), and the inverse probability of treatment weighted population comprised 12 968 procedures (294 were revised) with a maximum follow-up of 8.75 years. RTSR had a reduced hazard ratio of revision in the first three years (hazard ratio local minimum 0.33, 95% confidence interval 0.18 to 0.59) with no clinically important difference in revision-free restricted mean survival time, and a reduced relative risk of reoperations at 12 months (odds ratio 0.45, 95% confidence interval 0.25 to 0.83) with an absolute risk difference of -0.51% (95% confidence interval -0.89 to -0.13). Serious adverse events and prolonged hospital stay risks, change in Oxford Shoulder Score, and modelled mean lifetime costs were similar. Outcomes remained consistent after weighting.

CONCLUSIONS

This study's findings provide reassurance that RTSR is an acceptable alternative to TSR for patients aged 60 years or older with osteoarthritis and intact rotator cuff tendons. Despite a significant difference in the risk profiles of revision surgery over time, no statistically significant and clinically important differences between RTSR and TSR were found in terms of long term revision surgery, serious adverse events, reoperations, prolonged hospital stay, or lifetime healthcare costs.

The role of COVID-19 vaccines in preventing post-COVID-19 thromboembolic and cardiovascular complications

OBJECTIVE

To study the association between COVID-19 vaccination and the risk of post-COVID-19 cardiac and thromboembolic complications.

METHODS

We conducted a staggered cohort study based on national vaccination campaigns using electronic health records from the UK, Spain and Estonia. Vaccine rollout was grouped into four stages with predefined enrolment periods. Each stage included all individuals eligible for vaccination, with no previous SARS-CoV-2 infection or COVID-19 vaccine at the start date. Vaccination status was used as a time-varying exposure. Outcomes included heart failure (HF), venous thromboembolism (VTE) and arterial thrombosis/thromboembolism (ATE) recorded in four time windows after SARS-CoV-2 infection: 0-30, 31-90, 91-180 and 181-365 days. Propensity score overlap weighting and empirical calibration were used to minimise observed and unobserved confounding, respectively.Fine-Gray models estimated subdistribution hazard ratios (sHR). Random effect meta-analyses were conducted across staggered cohorts and databases.

RESULTS

The study included 10.17 million vaccinated and 10.39 million unvaccinated people. Vaccination was associated with reduced risks of acute (30-day) and post-acute COVID-19 VTE, ATE and HF: for example, meta-analytic sHR of 0.22 (95% CI 0.17 to 0.29), 0.53 (0.44 to 0.63) and 0.45 (0.38 to 0.53), respectively, for 0-30 days after SARS-CoV-2 infection, while in the 91-180 days sHR were 0.53 (0.40 to 0.70), 0.72 (0.58 to 0.88) and 0.61 (0.51 to 0.73), respectively.

CONCLUSIONS

COVID-19 vaccination reduced the risk of post-COVID-19 cardiac and thromboembolic outcomes. These effects were more pronounced for acute COVID-19 outcomes, consistent with known reductions in disease severity following breakthrough versus unvaccinated SARS-CoV-2 infection.

Randomized controlled trials: not always the "gold standard" for evidence in obstetrics and gynecology

Randomized controlled trials are considered the "gold standard" for therapeutic interventions, and it is not uncommon for sweeping changes in medical practice to follow positive results from such trials. However, randomized controlled trials are not without their limitations. Physicians frequently view randomized controlled trials as infallible, whereas they tend to dismiss evidence derived from sources other than randomized controlled trials as less credible or reliable. In several situations in obstetrics and gynecology, there are no randomized controlled trials to help guide the clinician. In these circumstances, it is important to evaluate the entire body of evidence including observational studies, rather than dismiss interventions altogether simply because no randomized controlled trials exist. Randomized controlled trials and observational studies should be viewed as complementary rather than at odds with each other. Some reversals in widely adopted clinical practice have recently been implemented following subsequent studies that contradicted the outcomes of major randomized controlled trials. The most notable of these was the withdrawal from the market of 17-hydroxyprogesterone caproate for preterm birth prevention. Such reversals could potentially have been averted if the inherent limitations of randomized controlled trials were carefully considered before implementing these universal practice changes. This Clinical Opinion underscores the limitations of an exclusive reliance on randomized controlled trials while disregarding other evidence in determining how best to care for patients. Solutions are proposed that advocate that clinicians adopt a more balanced perspective that considers the entirety of the available medical evidence and the individual patient characteristics, needs, and wishes.

Observational studies to emulate randomized trials: Some real-world barriers

The randomized controlled trial (RCT) forms the basis for drug approval by regulatory agencies. Observational studies using existing data from healthcare databases now also provide real-world evidence (RWE) in regulatory decision-making. Several initiatives are assessing the value of RWE by conducting observational studies that emulate published RCTs. While many RCTs are straightforward to emulate, others are challenging. We describe three RCT design aspects that pose challenges for observational studies. First are trials that enrol already treated subjects who must discontinue these treatments at the time of randomization, which can distort the comparison with observational studies. Second is the inclusion of a run-in phase, especially to exclude non-compliant subjects from the trial. Third are trials that evaluate the effect of weaning off treatment. In conclusion, future randomized trials that aim to be emulated by observational studies could consider study designs that allow emulation and thus provide valid and complementary RWE.

Advancing the utilization of real-world data and real-world evidence in clinical pharmacology and translational research-Proceedings from the ASCPT 2023 preconference workshop

Real-world data (RWD) and real-world evidence (RWE) are now being routinely used in epidemiology, clinical practice, and post-approval regulatory decisions. Despite the increasing utility of the methodology and new regulatory guidelines in recent years, there remains a lack of awareness of how this approach can be applied in clinical pharmacology and translational research settings. Therefore, the American Society of Clinical Pharmacology & Therapeutics (ASCPT) held a workshop on March 21st, 2023 entitled "Advancing the Utilization of Real-World Data (RWD) and Real-World Evidence (RWE) in Clinical Pharmacology and Translational Research." The work described herein is a summary of the workshop proceedings.

An evaluation of the replicability of analyses using synthetic health data

Synthetic data generation is being increasingly used as a privacy preserving approach for sharing health data. In addition to protecting privacy, it is important to ensure that generated data has high utility. A common way to assess utility is the ability of synthetic data to replicate results from the real data. Replicability has been defined using two criteria: (a) replicate the results of the analyses on real data, and (b) ensure valid population inferences from the synthetic data. A simulation study using three heterogeneous real-world datasets evaluated the replicability of logistic regression workloads. Eight replicability metrics were evaluated: decision agreement, estimate agreement, standardized difference, confidence interval overlap, bias, confidence interval coverage, statistical power, and precision (empirical SE). The analysis of synthetic data used a multiple imputation approach whereby up to 20 datasets were generated and the fitted logistic regression models were combined using combining rules for fully synthetic datasets. The effects of synthetic data amplification were evaluated, and two types of generative models were used: sequential synthesis using boosted decision trees and a generative adversarial network (GAN). Privacy risk was evaluated using a membership disclosure metric. For sequential synthesis, adjusted model parameters after combining at least ten synthetic datasets gave high decision and estimate agreement, low standardized difference, as well as high confidence interval overlap, low bias, the confidence interval had nominal coverage, and power close to the nominal level. Amplification had only a marginal benefit. Confidence interval coverage from a single synthetic dataset without applying combining rules were erroneous, and statistical power, as expected, was artificially inflated when amplification was used. Sequential synthesis performed considerably better than the GAN across multiple datasets. Membership disclosure risk was low for all datasets and models. For replicable results, the statistical analysis of fully synthetic data should be based on at least ten generated datasets of the same size as the original whose analyses results are combined. Analysis results from synthetic data without applying combining rules can be misleading. Replicability results are dependent on the type of generative model used, with our study suggesting that sequential synthesis has good replicability characteristics for common health research workloads.

Glucocorticoid use in acute respiratory failure from pulmonary causes and association with early changes in the systemic host immune response

BACKGROUND

Glucocorticoids are commonly used in patients with or at-risk for acute respiratory distress syndrome (ARDS), but optimal use remains unclear despite well-conducted clinical trials. We performed a secondary analysis in patients previously enrolled in the Acute Lung Injury and Biospecimen Repository at the University of Pittsburgh. The primary aim of our study was to investigate early changes in host response biomarkers in response to real-world use of glucocorticoids in patients with acute respiratory failure due to ARDS or at-risk due to a pulmonary insult. Participants had baseline plasma samples obtained on study enrollment and on follow-up 3 to 5 days later to measure markers of innate immunity (IL-6, IL-8, IL-10, TNFr1, ST2, fractalkine), epithelial injury (sRAGE), endothelial injury (angiopoietin-2), and host response to bacterial infections (procalcitonin, pentraxin-3). In our primary analyses, we investigated the effect of receiving glucocorticoids between baseline and follow-up samples on host response biomarkers measured at follow-up by doubly robust inverse probability weighting analysis. In exploratory analyses, we examined associations between glucocorticoid use and previously characterized host response subphenotypes (hyperinflammatory and hypoinflammatory).

RESULTS

67 of 148 participants (45%) received glucocorticoids between baseline and follow-up samples. Dose and type of glucocorticoids varied. Regimens that used hydrocortisone alone were most common (37%), and median daily dose was equivalent to 40 mg methylprednisolone (interquartile range: 21, 67). Participants who received glucocorticoids were more likely to be female, to be on immunosuppressive therapy at baseline, and to have higher baseline levels of ST-2, fractalkine, IL-10, pentraxin-3, sRAGE, and TNFr1. Glucocorticoid use was associated with decreases in IL-6 and increases in fractalkine. In exploratory analyses, glucocorticoid use was more frequent in participants in the hyperinflammatory subphenotype (58% vs 40%, p = 0.05), and was not associated with subphenotype classification at the follow-up time point (p = 0.16).

CONCLUSIONS

Glucocorticoid use varied in a cohort of patients with or at-risk for ARDS and was associated with early changes in the systemic host immune response.

The Association Between Metformin Use and New-Onset ICD Coding of Geographic Atrophy

Purpose

Metformin has been suggested to protect against the development of age-related macular degeneration (AMD) in multiple observational studies. However, the association between metformin and geographic atrophy (GA), a debilitating subtype of AMD, has not been analyzed.

Methods

We conducted a case-control study of patients ages 60 years and older with new-onset International Classification of Diseases (ICD) coding of GA in the Merative MarketScan Commercial and Medicare Databases between 2017 and 2021. Cases were matched with propensity scores estimated by age, region, hypertension, and Charlson Comorbidity Index to a control without GA of the same year. Exposure to metformin was assessed for cases and controls in the year prior to their index visit. Conditional multivariable logistic regression, adjusting for AMD risk factors, was used to calculate odd ratios and 95% confidence intervals (CIs). This study design and analysis were repeated in a sample of patients without diabetes.

Results

In the full sample, we identified 10,505 cases with GA and 10,502 matched controls without GA. In total, 1149 (10.9%) cases and 1277 (12.2%) controls were exposed to metformin, and in multivariable regression, metformin decreased the odds of new-onset ICD coding of GA by 12% (95% CI, 0.79-0.99). In the sample of patients without diabetes, we identified 7611 cases with GA and 7608 matched controls without GA. Twenty-nine (0.4%) cases and 63 (0.8%) controls were exposed to metformin, and in multivariable regression, metformin decreased the odds of new-onset ICD coding of GA by 47% (95% CI, 0.33-0.83).

Conclusions

Metformin may hold promise as a noninvasive, alternative agent to prevent the development of GA. This finding is notable due to shortcomings in recently approved therapeutics for GA and metformin's overall ease of use and few adverse effects. Additional studies are required to explore our findings further and motivate a clinical trial.

Staging and clean room: Constructs designed to facilitate transparency and reduce bias in comparative analyses of real-world data

PURPOSE

We describe constructs designed to protect the integrity of the results from comparative analyses using real-world data (RWD): staging and clean room.

METHODS

Staging involves performing sequential preliminary analyses and evaluating the population size available and potential bias before conducting comparative analyses. A clean room involves restricted access to data and preliminary results, policies governing exploratory analyses and protocol deviations, and audit trail. These constructs are intended to allow decisions about protocol deviations, such as changes to design or model specification, to be made without knowledge of how they might affect subsequent analyses. We describe an example for implementing staging with a clean room.

RESULTS

Stage 1 may involve selecting a data source, developing and registering a protocol, establishing a clean room, and applying inclusion/exclusion criteria. Stage 2 may involve attempting to achieve covariate balance, often through propensity score models. Stage 3 may involve evaluating the presence of residual confounding using negative control outcomes. After each stage, check points may be implemented when a team of statisticians, epidemiologists and clinicians masked to how their decisions may affect study outcomes, reviews the results. This review team may be tasked with making recommendations for protocol deviations to address study precision or bias. They may recommend proceeding to the next stage, conducting additional analyses to address bias, or terminating the study. Stage 4 may involve conducting the comparative analyses.

CONCLUSIONS

The staging and clean room constructs are intended to protect the integrity and enhance confidence in the results of analyses of RWD.

Relevance of Protein Intake for Weaning in the Mechanically Ventilated Critically Ill: Analysis of a Large International Database

OBJECTIVES

The association between protein intake and the need for mechanical ventilation (MV) is controversial. We aimed to investigate the associations between protein intake and outcomes in ventilated critically ill patients.

DESIGN

Analysis of a subset of a large international point prevalence survey of nutritional practice in ICUs.

SETTING

A total of 785 international ICUs.

PATIENTS

A total of 12,930 patients had been in the ICU for at least 96 hours and required MV by the fourth day after ICU admission at the latest.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We modeled associations between the adjusted hazard rate (aHR) of death in patients requiring MV and successful weaning (competing risks), and three categories of protein intake (low: < 0.8 g/kg/d, standard: 0.8-1.2 g/kg/d, high: > 1.2 g/kg/d). We compared five different hypothetical protein diets (an exclusively low protein intake, a standard protein intake given early (days 1-4) or late (days 5-11) after ICU admission, and an early or late high protein intake). There was no evidence that the level of protein intake was associated with time to weaning. However, compared with an exclusively low protein intake, a standard protein intake was associated with a lower hazard of death in MV: minimum aHR 0.60 (95% CI, 0.45-0.80). With an early high intake, there was a trend to a higher risk of death in patients requiring MV: maximum aHR 1.35 (95% CI, 0.99-1.85) compared with a standard diet.

CONCLUSIONS

The duration of MV does not appear to depend on protein intake, whereas mortality in patients requiring MV may be improved by a standard protein intake. Adverse effects of a high protein intake cannot be excluded.

Adapting Health Technology Assessment agency standards for surrogate outcomes in early stage cancer trials: what needs to happen?

INTRODUCTION

An avalanche of early stage cancer clinical trials is coming. The majority of these solely use surrogate outcomes that have not been validated against a target outcome of interest (e.g. overall survival). Current HTA guidance on surrogate outcome validation are not methodologically or practically conducive to this scenario.

AREAS COVERED

We provide a high-level overview of methods, approaches, and conceptual thinking for making better use of limited evidence within early stage cancer HTA submissions. We outline regulatory and HTA issues and emphasize how evidence transitions from one to another, what major gaps currently exist, and how these may be bridged. We summarize current methodologies and practices, their pros and cons. We outline how complementary measurements strengthen evaluations and address fallacies and biases of conventional statistical methods for surrogate outcomes validation. The value of real-world data to support some of the necessary validity components is discussed. Lastly, we address the importance of the patient voice for better understanding which surrogate outcomes may appropriately inform HTA.

EXPERT OPINION

Conventional surrogate outcome validation represents a fraught and sub-optimal framework for HTA purposes, particularly for early stage cancer. Tools for optimizing use of limited evidence exist. Education of stakeholders is highly needed.