Estimation of Population Average Treatment Effects in the FIRST Trial: Application of a Propensity Score-Based Stratification Approach

Can propensity score matching replace randomized controlled trials?

Randomized controlled trials (RCTs) have long been recognized as the gold standard for establishing causal relationships in clinical research. Despite that, various limitations of RCTs prevent its widespread implementation, ranging from the ethicality of withholding potentially-lifesaving treatment from a group to relatively poor external validity due to stringent inclusion criteria, amongst others. However, with the introduction of propensity score matching (PSM) as a retrospective statistical tool, new frontiers in establishing causation in clinical research were opened up. PSM predicts treatment effects using observational data from existing sources such as registries or electronic health records, to create a matched sample of participants who received or did not receive the intervention based on their propensity scores, which takes into account characteristics such as age, gender and comorbidities. Given its retrospective nature and its use of observational data from existing sources, PSM circumvents the aforementioned ethical issues faced by RCTs. Majority of RCTs exclude elderly, pregnant women and young children; thus, evidence of therapy efficacy is rarely proven by robust clinical research for this population. On the other hand, by matching study patient characteristics to that of the population of interest, including the elderly, pregnant women and young children, PSM allows for generalization of results to the wider population and hence greatly increases the external validity. Instead of replacing RCTs with PSM, the synergistic integration of PSM into RCTs stands to provide better research outcomes with both methods complementing each other. For example, in an RCT investigating the impact of mannitol on outcomes among participants of the Intensive Blood Pressure Reduction in Acute Cerebral Hemorrhage Trial, the baseline characteristics of comorbidities and current medications between treatment and control arms were significantly different despite the randomization protocol. Therefore, PSM was incorporated in its analysis to create samples from the treatment and control arms that were matched in terms of these baseline characteristics, thus providing a fairer comparison for the impact of mannitol. This literature review reports the applications, advantages, and considerations of using PSM with RCTs, illustrating its utility in refining randomization, improving external validity, and accounting for non-compliance to protocol. Future research should consider integrating the use of PSM in RCTs to better generalize outcomes to target populations for clinical practice and thereby benefit a wider range of patients, while maintaining the robustness of randomization offered by RCTs.

Applicability of Transcatheter Aortic Valve Replacement Trials to Real-World Clinical Practice: Findings From EXTEND-CoreValve

OBJECTIVES

The aim of this study was to examine the applicability of pivotal transcatheter aortic valve replacement (TAVR) trials to the real-world population of Medicare patients undergoing TAVR.

BACKGROUND

It is unclear whether randomized controlled trial results of novel cardiovascular devices apply to patients encountered in clinical practice.

METHODS

Characteristics of patients enrolled in the U.S. CoreValve pivotal trials were compared with those of the population of Medicare beneficiaries who underwent TAVR in U.S. clinical practice between November 2, 2011, and December 31, 2017. Inverse probability weighting was used to reweight the trial cohort on the basis of Medicare patient characteristics, and a "real-world" treatment effect was estimated.

RESULTS

A total of 2,026 patients underwent TAVR in the U.S. CoreValve pivotal trials, and 135,112 patients underwent TAVR in the Medicare cohort. Trial patients were mostly similar to real-world patients at baseline, though trial patients were more likely to have hypertension (50% vs 39%) and coagulopathy (25% vs 17%), whereas real-world patients were more likely to have congestive heart failure (75% vs 68%) and frailty. The estimated real-world treatment effect of TAVR was an 11.4% absolute reduction in death or stroke (95% CI: 7.50%-14.92%) and an 8.7% absolute reduction in death (95% CI: 5.20%-12.32%) at 1 year with TAVR compared with conventional therapy (surgical aortic valve replacement for intermediate- and high-risk patients and medical therapy for extreme-risk patients).

CONCLUSIONS

The trial and real-world populations were mostly similar, with some notable differences. Nevertheless, the extrapolated real-world treatment effect was at least as high as the observed trial treatment effect, suggesting that the absolute benefit of TAVR in clinical trials is similar to the benefit of TAVR in the U.S. real-world setting.

A National Mixed-Methods Evaluation of Preparedness for General Surgery Residency and the Association With Resident Burnout

Importance

Differences in medical school experiences may affect how prepared residents feel themselves to be as they enter general surgery residency and may contribute to resident burnout.

Objectives

To assess preparedness for surgical residency, to identify factors associated with preparedness, to examine the association between preparedness and burnout, and to explore resident and faculty perspectives on resident preparedness.

Design, Setting, and Participants

This cross-sectional study used convergent mixed-methods analysis of data from a survey of US general surgery residents delivered at the time of the 2017 American Board of Surgery In-Training Examination (January 26 to 31, 2017) in conjunction with qualitative interviews of residents and program directors conducted as part of the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) trial. A total of 262 Accreditation Council for Graduate Medical Education-approved US general surgery residency programs participated. Survey data were collected from 3693 postgraduate year (PGY) 1 and PGY2 surgical residents (response rate, 99%) and 98 interviews were conducted with residents and faculty from September 1 to December 15, 2018. Data were analyzed from June 1, 2017, to February 15, 2018.

Main Outcomes and Measures

Hierarchical regression models were developed to examine factors associated with preparedness and to assess the association between preparedness and resident burnout. Qualitative interviews were conducted to identify themes associated with preparation for residency.

Results

Of the 3693 PGY1 and PGY2 residents who participated (2258 male [61.1%]), 1775 (48.1%) reported feeling unprepared for residency. Approximately half of surgery residents took overnight call infrequently (≤2 per month) during their core medical student clerkship (1904 [51.6%]) or their subinternship (1600 [43.3%]); 524 (14.2%) took no call during their core clerkship. In multivariable analysis, residents were more likely to report feeling unprepared for residency if they were female (odds ratio [OR], 1.34; 95% CI, 1.15-1.57) or did not take call as a medical student (OR for 0 vs >4 calls, 2.72; 95% CI, 2.10-3.52). Residents who did not complete a subinternship were less likely to report feeling prepared for residency (OR, 0.68; 95% CI, 0.48-0.96). Feeling adequately prepared for residency was associated with a nearly 2-fold lower risk of experiencing burnout symptoms (OR, 0.57; 95% CI, 0.48-0.68). In interviews, the dominant themes associated with preparedness included the following: (1) various regulations limit the medical school experience, (2) overnight call facilitates preparation and selection of a specialty compatible with their preferences, and (3) adequate perceptions of residency improve expectations, resulting in improved preparedness, lower burnout rates, and lower risk of attrition.

Conclusions and Relevance

In this cross-sectional study, the perception of feeling unprepared was associated with inadequate exposure to resident responsibilities while in medical school. These findings suggest that effective preparation of medical students for residency may result in lower rates of subsequent burnout.