Large, Sparse Optimal Matching with Refined Covariate Balance in an Observational Study of the Health Outcomes Produced by New Surgeons

The use and costs of same-day surgery versus overnight admission for total thyroidectomy: A multi-state, all-payer analysis

BACKGROUND

Outpatient thyroidectomy is increasingly favored, given evidence of safety and convenience for selected patients. However, the prevalence of same-day discharge is unclear. We aimed to evaluate temporal trends, hospital characteristics, and costs associated with same-day discharge after total thyroidectomy in an all-payer, multi-state cohort.

METHODS

We included patients aged ≥18 years who underwent a total thyroidectomy (2013-2019) using Healthcare Cost and Utilization Project data. Admission type was defined as same-day, overnight, or inpatient based on length of stay. Same-day patients were propensity-score matched 1:1 with overnight patients. Hospital characteristics and costs were compared in the matched cohort.

RESULTS

Among 86,187 patients who underwent total thyroidectomy, 16,743 (19.4%) cases were same-day, 59,778 (69.4%) were overnight, and 9,666 (11.2%) were inpatient. The proportion of patients who underwent same-day thyroidectomy increased from 14.8% to 20.8% over the study period (P < .001), whereas overnight admissions decreased from 72.9% to 68.8% (P < .001). In total, 9,571 same-day patients were matched to 9,571 overnight patients. Same-day patients had higher odds of treatment at a certified cancer center (odds ratio 1.77; 95% confidence interval 1.65-1.90), Accreditation Council for Graduate Medical Education-accredited teaching hospital (odds ratio 1.72; 95% confidence interval 1.61-1.85), and high-volume hospital (odds ratio 1.53; 95% confidence interval 1.42-1.65). Pairwise cost differences showed median savings of $974 (interquartile range -1,610 to 3,491) for same-day relative to overnight admission (P < .001).

CONCLUSION

Although over two-thirds of patients are admitted overnight, same-day total thyroidectomy is increasingly performed. Same-day thyroidectomy may be a lower-cost option for selected patients, particularly in specialty centers with experience in thyroidectomy.

How well can fine balance work for covariate balancing

Fine balance is a matching technique to improve covariate balance in observational studies. It constrains a match to have identical distributions for some covariates without restricting who is matched to whom. However, despite its wide application and excellent performance in practice, there is very little theory indicating when the method is likely to succeed or fail and to what extent it can remove covariate imbalance. In order to answer these questions, this paper studies the limits of what is possible for covariate balancing using fine balance and near-fine balance. The investigations suggest that given the distributions of the treated and control groups, in large samples, the maximum achievable balance by using fine balance only depends on the matching ratio (ie, the ratio of the sample size of the control group to that of the treated group). In addition, the results indicate how to estimate this matching ratio threshold without knowledge of the true distributions in finite samples. The findings are also illustrated by numerical studies in this paper.

Best (but oft forgotten) statistical practices: Measuring real-world intervention effectiveness using electronic health data

The evidence base supporting the use of most interventions consists primarily of data from randomized controlled trials (RCTs), but how and to whom interventions are delivered in clinical practice may differ substantially from these foundational RCTs. With the increasing availability of electronic health data, it is now feasible to study the "real-world" effectiveness of a wide range of interventions. However, real-world intervention effectiveness studies using electronic health data face many challenges including data quality, selection bias, confounding by indication, and lack of generalizability. In this article, we describe the key barriers to generating high-quality evidence from real-world intervention effectiveness studies and suggest statistical best practices for addressing them.

The Minnesota attributable risk of kidney donation (MARKD) study: a retrospective cohort study of long-term (> 50 year) outcomes after kidney donation compared to well-matched healthy controls

BACKGROUND

There is uncertainty about the long-term risks of living kidney donation. Well-designed studies with controls well-matched on risk factors for kidney disease are needed to understand the attributable risks of kidney donation.

METHODS

The goal of the Minnesota Attributable Risk of Kidney Donation (MARKD) study is to compare the long-term (> 50 years) outcomes of living donors (LDs) to contemporary and geographically similar controls that are well-matched on health status. University of Minnesota (n = 4022; 1st transplant: 1963) and Mayo Clinic LDs (n = 3035; 1st transplant: 1963) will be matched to Rochester Epidemiology Project (REP) controls (approximately 4 controls to 1 donor) on the basis of age, sex, and race/ethnicity. The REP controls are a well-defined population, with detailed medical record data linked between all providers in Olmsted and surrounding counties, that come from the same geographic region and era (early 1960s to present) as the donors. Controls will be carefully selected to have health status acceptable for donation on the index date (date their matched donor donated). Further refinement of the control group will include confirmed kidney health (e.g., normal serum creatinine and/or no proteinuria) and matching (on index date) of body mass index, smoking history, family history of chronic kidney disease, and blood pressure. Outcomes will be ascertained from national registries (National Death Index and United States Renal Data System) and a new survey administered to both donors and controls; the data will be supplemented by prior surveys and medical record review of donors and REP controls. The outcomes to be compared are all-cause mortality, end-stage kidney disease, cardiovascular disease and mortality, estimated glomerular filtration rate (eGFR) trajectory and chronic kidney disease, pregnancy risks, and development of diseases that frequently lead to chronic kidney disease (e.g. hypertension, diabetes, and obesity). We will additionally evaluate whether the risk of donation differs based on baseline characteristics.

DISCUSSION

Our study will provide a comprehensive assessment of long-term living donor risk to inform candidate living donors, and to inform the follow-up and care of current living donors.

Testing Biased Randomization Assumptions and Quantifying Imperfect Matching and Residual Confounding in Matched Observational Studies

One central goal of design of observational studies is to embed non-experimental data into an approximate randomized controlled trial using statistical matching. Despite empirical researchers' best intention and effort to create high-quality matched samples, residual imbalance due to observed covariates not being well matched often persists. Although statistical tests have been developed to test the randomization assumption and its implications, few provide a means to quantify the level of residual confounding due to observed covariates not being well matched in matched samples. In this article, we develop two generic classes of exact statistical tests for a biased randomization assumption. One important by-product of our testing framework is a quantity called residual sensitivity value (RSV), which provides a means to quantify the level of residual confounding due to imperfect matching of observed covariates in a matched sample. We advocate taking into account RSV in the downstream primary analysis. The proposed methodology is illustrated by re-examining a famous observational study concerning the effect of right heart catheterization (RHC) in the initial care of critically ill patients. Code implementing the method can be found in the supplementary materials.

Profile Matching for the Generalization and Personalization of Causal Inferences

We introduce profile matching, a multivariate matching method for randomized experiments and observational studies that finds the largest possible unweighted samples across multiple treatment groups that are balanced relative to a covariate profile. This covariate profile can represent a specific population or a target individual, facilitating the generalization and personalization of causal inferences. For generalization, because the profile often amounts to summary statistics for a target population, profile matching does not always require accessing individual-level data, which may be unavailable for confidentiality reasons. For personalization, the profile comprises the characteristics of a single individual. Profile matching achieves covariate balance by construction, but unlike existing approaches to matching, it does not require specifying a matching ratio, as this is implicitly optimized for the data. The method can also be used for the selection of units for study follow-up, and it readily applies to multivalued treatments with many treatment categories. We evaluate the performance of profile matching in a simulation study of the generalization of a randomized trial to a target population. We further illustrate this method in an exploratory observational study of the relationship between opioid use and mental health outcomes. We analyze these relationships for three covariate profiles representing: (i) sexual minorities, (ii) the Appalachian United States, and (iii) the characteristics of a hypothetical vulnerable patient. The method can be implemented via the new function profmatch in the designmatch package for R, for which we provide a step-by-step tutorial.

Matching One Sample According to Two Criteria in Observational Studies

Multivariate matching has two goals: (i) to construct treated and control groups that have similar distributions of observed covariates, and (ii) to produce matched pairs or sets that are homogeneous in a few key covariates. When there are only a few binary covariates, both goals may be achieved by matching exactly for these few covariates. Commonly, however, there are many covariates, so goals (i) and (ii) come apart, and must be achieved by different means. As is also true in a randomized experiment, similar distributions can be achieved for a high-dimensional covariate, but close pairs can be achieved for only a few covariates. We introduce a new polynomial-time method for achieving both goals that substantially generalizes several existing methods; in particular, it can minimize the earthmover distance between two marginal distributions. The method involves minimum cost flow optimization in a network built around a tripartite graph, unlike the usual network built around a bipartite graph. In the tripartite graph, treated subjects appear twice, on the far left and the far right, with controls sandwiched between them, and efforts to balance covariates are represented on the right, while efforts to find close individual pairs are represented on the left. In this way, the two efforts may be pursued simultaneously without conflict. The method is applied to our on-going study in the Medicare population of the relationship between superior nursing and sepsis mortality. The match2C package in R implements the method.

Scientific prizes and the extraordinary growth of scientific topics

Fast growing scientific topics have famously been key harbingers of the new frontiers of science, yet, large-scale analyses of their genesis and impact are rare. We investigated one possible factor connected with a topic’s extraordinary growth: scientific prizes. Our longitudinal analysis of nearly all recognized prizes worldwide and over 11,000 scientific topics from 19 disciplines indicates that topics associated with a scientific prize experience extraordinary growth in productivity, impact, and new entrants. Relative to matched non-prizewinning topics, prizewinning topics produce 40% more papers and 33% more citations, retain 55% more scientists, and gain 37 and 47% more new entrants and star scientists, respectively, in the first five-to-ten years after the prize. Funding do not account for a prizewinning topic’s growth. Rather, growth is positively related to the degree to which the prize is discipline-specific, conferred for recent research, or has prize money. These findings reveal new dynamics behind scientific innovation and investment.

Scientific revolutions have famously inspired scientists and innovation but large-scale analyses of scientific revolutions in modern science are rare. Here, the authors investigate one possible factor connected with a topic’s extraordinary growth—scientific prizes.

The impact of a population-based prevention program on cardiovascular events: Findings from the heart of new Ulm project

BACKGROUND

Cardiovascular disease (CVD) and its associated risk factors are the principal drivers of mortality and healthcare costs in the United States with rural residents experiencing higher CVD death rates than their urban counterparts.

METHODS

The purpose of this study was to examine incidence of major CVD events over 9 years of implementation of the Heart of New Ulm (HONU) Project, a rural population-based CVD prevention initiative. HONU interventions were delivered at individual, organizational, and community levels addressing clinical risk factors, lifestyle behaviors and environmental changes. The sample included 4,056 residents of New Ulm matched with 4,056 residents from a different community served by the same health system. The primary outcome was a composite of major CVD events (myocardial infarction, ischemic stroke, percutaneous coronary intervention (PCI), coronary artery bypass graft (CABG), and CVD-related death). Secondary outcomes were the individual CVD events and procedures.

RESULTS

The proportion of residents in New Ulm with a major CVD event (7.79%) was not significantly different than the comparison community (8.43%, P = .290). However, the total number of events did differ by community with fewer events in New Ulm than the comparison community (447 vs 530, P = .005), with 48 fewer strokes (84 vs 132, P = .001) and 42 fewer PCI procedures (147 vs 189, P = 0.019) in New Ulm. Incidence of ischemic stroke was lower in the New Ulm community (1.85 vs 2.61, P = .020) than in the comparison community. Other specific CVD events did not have significantly different incidence or frequencies between the 2 communities.

CONCLUSION

In HONU, the proportion of residents experiencing a CVD event was not significantly lower than a match comparison community. However, there was a significant reduction in the total number of CVD events in New Ulm, driven primarily by lower stroke, PCI, and CABG events in the intervention community.

Relationship between changing malaria burden and low birth weight in sub-Saharan Africa: A difference-in-differences study via a pair-of-pairs approach

Background

According to the World Health Organization (WHO), in 2018, an estimated 228 million malaria cases occurred worldwide with most cases occurring in sub-Saharan Africa. Scale-up of vector control tools coupled with increased access to diagnosis and effective treatment has resulted in a large decline in malaria prevalence in some areas, but other areas have seen little change. Although interventional studies demonstrate that preventing malaria during pregnancy can reduce the rate of low birth weight (i.e. child's birth weight <2500 g), it remains unknown whether natural changes in parasite transmission and malaria burden can improve birth outcomes.

Methods

We conducted an observational study of the effect of changing malaria burden on low birth weight using data from 18,112 births in 19 countries in sub-Saharan African countries during the years 2000-2015. Specifically, we conducted a difference-in-differences study via a pair-of-pairs matching approach using the fact that some sub-Saharan areas experienced sharp drops in malaria prevalence and some experienced little change.

Results

A malaria prevalence decline from a high rate (Plasmodium falciparum parasite rate in children aged 2-up-to-10 (i.e. PfPR2-10) > 0.4) to a low rate (PfPR2-10 < 0.2) is estimated to reduce the rate of low birth weight by 1.48 percentage points (95% confidence interval: 3.70 percentage points reduction, 0.74 percentage points increase), which is a 17% reduction in the low birth weight rate compared to the average (8.6%) in our study population with observed birth weight records (1.48/8.6 ≈ 17%). When focusing on first pregnancies, a decline in malaria prevalence from high to low is estimated to have a greater impact on the low birth weight rate than for all births: 3.73 percentage points (95% confidence interval: 9.11 percentage points reduction, 1.64 percentage points increase).

Conclusions

Although the confidence intervals cannot rule out the possibility of no effect at the 95% confidence level, the concurrence between our primary analysis, secondary analyses, and sensitivity analyses, and the magnitude of the effect size, contribute to the weight of the evidence suggesting that declining malaria burden can potentially substantially reduce the low birth weight rate at the community level in sub-Saharan Africa, particularly among firstborns. The novel statistical methodology developed in this article-a pair-of-pairs approach to a difference-in-differences study-could be useful for many settings in which different units are observed at different times.

Funding

Ryan A. Simmons is supported by National Center for Advancing Translational Sciences (UL1TR002553). The funder had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Comparing the Performance of Statistical Adjustment Methods by Recovering the Experimental Benchmark from the REFLUX Trial

Much evidence in comparative effectiveness research is based on observational studies. Researchers who conduct observational studies typically assume that there are no unobservable differences between the treatment groups under comparison. Treatment effectiveness is estimated after adjusting for observed differences between comparison groups. However, estimates of treatment effectiveness may be biased because of misspecification of the statistical model. That is, if the method of treatment effect estimation imposes unduly strong functional form assumptions, treatment effect estimates may be inaccurate, leading to inappropriate recommendations about treatment decisions. We compare the performance of a wide variety of treatment effect estimation methods for the average treatment effect. We do so within the context of the REFLUX study from the United Kingdom. In REFLUX, participants were enrolled in either an randomized controlled trial (RCT) or an observational study arm. In the RCT, patients were randomly assigned to either surgery or medical management. In the patient preference arm, participants selected to either have surgery or medical management. We attempt to recover the treatment effect estimate from the RCT using the data from the patient preference arms of the study. We vary the method of treatment effect estimation and record which methods are successful and which are not. We apply more than 20 different methods, including standard regression models as well as advanced machine learning methods. We find that simple propensity score matching methods provide the least accurate estimates versus the RCT benchmark. We find variation in performance across the other methods, with some, but not all recovering the experimental benchmark. We conclude that future studies should use multiple methods of estimation to fully represent uncertainty according to the choice of estimation approach.

Use of Electronic Cigarettes to Aid Long-Term Smoking Cessation in the United States: Prospective Evidence From the PATH Cohort Study

Electronic cigarettes (e-cigarettes) are the preferred smoking-cessation aid in the United States; however, there is little evidence regarding long-term effectiveness among those who use them. We used the Population Assessment of Tobacco and Health Study to compare long-term abstinence between matched US smokers who tried to quit with and without use of e-cigarettes as a cessation aid. We identified a nationally representative cohort of 2,535 adult US smokers in 2014-2015 (baseline assessment), who, in 2015-2016 (exposure assessment), reported a past-year attempt to quit and the cessation aids used, and reported smoking status in 2016-2017 (outcome assessment; self-reported ≥12 months continuous abstinence). We used propensity-score methods to match each e-cigarette user with similar nonusers. Among US smokers who used e-cigarettes to help quit, 12.9% (95% confidence interval (CI): 9.1%, 16.7%) successfully attained long-term abstinence. However, there was no difference compared with matched non-e-cigarette users (cigarette abstinence difference: 2%; 95% CI: -3%, 7%). Furthermore, fewer e-cigarette users were abstinent from nicotine products in the long term (nicotine abstinence difference: -4%; 95% CI: -7%, -1%); approximately two-thirds of e-cigarette users who successfully quit smoking continued to use e-cigarettes. These results suggest e-cigarettes may not be an effective cessation aid for adult smokers and, instead, may contribute to continuing nicotine dependence.

Impact of Dependent Coverage Provision of the Affordable Care Act on Insurance Continuity for Adolescents and Young Adults With Cancer

PURPOSE

The 2010 Dependent Coverage Provision (DCP) of the Affordable Care Act (ACA) allowed enrollees to remain on their parents' health insurance until 26 years of age. We compared rates of insurance disenrollment among patients with cancer who were DCP-eligible at age 19 to those who were not eligible at age 19.

METHODS

Using OptumLabs Data Warehouse, which contains longitudinal, real-world, de-identified administrative claims for commercial enrollees, we examined patients born between 1982 and 1993 and diagnosed with cancer between 2000 and 2015. In the recent cohort, patients who turned 19 in 2010-2012 (DCP-eligible to stay on parents' insurance) were matched to patients who turned 19 in 2007-2009 (not DCP-eligible when turning 19). In an earlier control cohort, patients who turned 19 between 2004 and 2006 (not DCP-eligible) were matched to patients who turned 19 between 2001 and 2003 (not DCP-eligible). Patients were matched on cancer type, diagnosis date, demographics, and treatment characteristics. The time to loss of coverage was estimated using Cox models. Difference-in-difference between the recent and earlier cohorts was also evaluated.

RESULTS

A total of 2,829 patients who turned 19 years of age in 2010-2012 were matched to patients who turned 19 in 2007-2009. Median time to disenrollment was 26 months for younger patients versus 22 months for older patients (hazard ratio [HR], 0.85; 95% CI, 0.80 to 0.90; P = .001). In 8,978 patients who turned 19 between 2001 and 2006, median time to disenrollment was 20 months among both younger and older patients (HR, 0.99; 95% CI, 0.94 to 1.03; P = .59). The difference between the recent cohort and the earlier control cohort was a 15% greater reduction in coverage loss (P < .0001), favoring those turning 19 after the DCP went into effect.

CONCLUSION

In the vulnerable population of adolescent and young adult cancer survivors, the ACA may have lowered the insurance dropout rate.

The uniform general signed rank test and its design sensitivity

A sensitivity analysis in an observational study tests whether the qualitative conclusions of an analysis would change if we were to allow for the possibility of limited bias due to confounding. The design sensitivity of a hypothesis test quantifies the asymptotic performance of the test in a sensitivity analysis against a particular alternative. We propose a new, nonasymptotic, distribution-free test, the uniform general signed rank test, for observational studies with paired data, and examine its performance under Rosenbaum’s sensitivity analysis model. Our test can be viewed as adaptively choosing from among a large underlying family of signed rank tests, and we show that the uniform test achieves design sensitivity equal to the maximum design sensitivity over the underlying family of signed rank tests. Our test thus achieves superior design sensitivity, indicating it will perform well in sensitivity analyses on large samples. We support this conclusion with simulations and a data example, showing that the advantages of our test extend to moderate sample sizes as well.

Evaluating and improving a matched comparison of antidepressants and bone density

Matching is a common approach to covariate adjustment in estimating causal effects in observational studies. It is important to assess covariate balance of the matched samples. This is usually done informally, in ways that have a number of limitations. First, there are many diagnostics, even if covariates are assessed one at a time, which raises multiplicity issues. In addition, joint distributions of covariates, even bivariate distributions, are often ignored. Further, it is an open question whether diagnostics identify the major problems. To address these issues, a formal assessment of covariate balance is developed in the current paper. Unlike the common informal diagnostics, the proposed method compares both marginal distributions and joint distributions of the matched sample with those of the benchmark, complete randomizations. The method controls the probability of falsely identifying a covariate imbalance among many comparisons, yet it has a high probability of correctly detecting and identifying a major problem. An R package met implementing the proposed method is available on CRAN.

To use or not to use propensity score matching?

Propensity score matching (PSM) has been widely used to reduce confounding biases in observational studies. Its properties for statistical inference have also been investigated and well documented. However, some recent publications showed concern of using PSM, especially on increasing postmatching covariate imbalance, leading to discussion on whether PSM should be used or not. We review empirical and theoretical evidence for and against its use in practice and revisit the property of equal percent bias reduction and adapt it to more practical situations, showing that PSM has some additional desirable properties. With a small simulation, we explore the impact of caliper width on biases due to mismatching in matched samples and due to the difference between matched and target populations and show some issue of PSM may be due to inadequate caliper selection. In summary, we argue that the right question should be when and how to use PSM rather than to use or not to use it and give suggestions accordingly.

Patients With a History of a Cardiac Implantable Electronic Device Have a Higher Likelihood of 90-Day Cardiac Events After Total Joint Arthroplasty: A Matched Cohort Study

INTRODUCTION

We sought to identify the incidence of new 90-day cardiac events, 90-day mortality, 90-day unplanned readmissions, and 30-day emergency department (ED) visits after total joint arthroplasty (TJA) in patients with a history of a cardiac implantable electronic device (CIED) and compare these outcomes in TJA patients without a CIED.

METHODS

Kaiser Permanente's Cardiac Device and Total Joint Replacement Registries were used to identify elective primary TJA performed for osteoarthritis. TJA with a CIED was matched with TJA without a CIED (n = 365 pairs) on patient characteristics, demographics, and procedure type. A McNemar test was used to evaluate categorical outcomes.

RESULTS

Of the TJA with a CIED, there were 24 cardiac events (6.6%), 1 mortality (0.3%), 30 readmissions (8.2%), and 39 ED visits (10.7%). TJA patients with a CIED had a higher likelihood of cardiac events (odds ratio [OR] = 3.14, 95% confidence interval [CI] = 1.28 to 8.08). No difference was observed in mortality (OR = 0.50, 95% CI = 0.02 to 6.98), readmissions (OR = 1.26, 95% CI = 0.71 to 2.25), or ED visits (OR = 1.15, 95% CI = 0.71 to 1.88).

CONCLUSION

In our matched cohort study, TJA patients with a history of a CIED had a higher likelihood of incident 90-day cardiac events when compared with patients without a CIED without a difference observed for 90-day mortality, unplanned readmission, and 30-day ED visit.

LEVEL OF EVIDENCE

Level III.

Developing a template matching algorithm for benchmarking hospital performance in a diverse, integrated healthcare system

Template matching is a proposed approach for hospital benchmarking, which measures performance based on matching a subset of comparable patient hospitalizations from each hospital. We assessed the ability to create the required matched samples and thus the feasibility of template matching to benchmark hospital performance in a diverse healthcare system.Nationwide Veterans Affairs (VA) hospitals, 2017.Observational cohort study.We used administrative and clinical data from 668,592 hospitalizations at 134 VA hospitals in 2017. A standardized template of 300 hospitalizations was selected, and then 300 hospitalizations were matched to the template from each hospital.There was substantial case-mix variation across VA hospitals, which persisted after excluding small hospitals, hospitals with primarily psychiatric admissions, and hospitalizations for rare diagnoses. Median age ranged from 57 to 75 years across hospitals; percent surgical admissions ranged from 0.0% to 21.0%; percent of admissions through the emergency department, 0.1% to 98.7%; and percent Hispanic patients, 0.2% to 93.3%. Characteristics for which there was substantial variation across hospitals could not be balanced with any matching algorithm tested. Although most other variables could be balanced, we were unable to identify a matching algorithm that balanced more than ∼20 variables simultaneously.We were unable to identify a template matching approach that could balance hospitals on all measured characteristics potentially important to benchmarking. Given the magnitude of case-mix variation across VA hospitals, a single template is likely not feasible for general hospital benchmarking.

Propensity score methods for time-dependent cluster confounding

In observational studies, subjects are often nested within clusters. In medical studies, patients are often treated by doctors and therefore patients are regarded as nested or clustered within doctors. A concern that arises with clustered data is that cluster-level characteristics (e.g., characteristics of the doctor) are associated with both treatment selection and patient outcomes, resulting in cluster-level confounding. Measuring and modeling cluster attributes can be difficult and statistical methods exist to control for all unmeasured cluster characteristics. An assumption of these methods however is that characteristics of the cluster and the effects of those characteristics on the outcome (as well as probability of treatment assignment when using covariate balancing methods) are constant over time. In this paper, we consider methods that relax this assumption and allow for estimation of treatment effects in the presence of unmeasured time-dependent cluster confounding. The methods are based on matching with the propensity score and incorporate unmeasured time-specific cluster effects by performing matching within clusters or using fixed- or random-cluster effects in the propensity score model. The methods are illustrated using data to compare the effectiveness of two total hip devices with respect to survival of the device and a simulation study is performed that compares the proposed methods. One method that was found to perform well is matching within surgeon clusters partitioned by time. Considerations in implementing the proposed methods are discussed.

Comparing Outcomes and Costs of Surgical Patients Treated at Major Teaching and Nonteaching Hospitals: A National Matched Analysis

OBJECTIVE

To compare outcomes and costs between major teaching and nonteaching hospitals on a national scale by closely matching on patient procedures and characteristics.

BACKGROUND

Teaching hospitals have been shown to often have better quality than nonteaching hospitals, but cost and value associated with teaching hospitals remains unclear.

METHODS

A study of Medicare patients at 340 teaching hospitals (resident-to-bed ratios ≥ 0.25) and matched patient controls from 2444 nonteaching hospitals (resident-to-bed ratios < 0.05).We studied 86,751 pairs admitted for general surgery (GS), 214,302 pairs of patients admitted for orthopedic surgery, and 52,025 pairs of patients admitted for vascular surgery.

RESULTS

In GS, mortality was 4.62% in teaching hospitals versus 5.57%, (a difference of -0.95%, <0.0001), and overall paired cost difference = $915 (P < 0.0001). For the GS quintile of pairs with highest risk on admission, mortality differences were larger (15.94% versus 18.18%, difference = -2.24%, P < 0.0001), and paired cost difference = $3773 (P < 0.0001), yielding $1682 per 1% mortality improvement at 30 days. Patterns for vascular surgery outcomes resembled general surgery; however, orthopedics outcomes did not show significant differences in mortality across teaching and nonteaching environments, though costs were higher at teaching hospitals.

CONCLUSIONS

Among Medicare patients, as admission risk of mortality increased, the absolute mortality benefit of treatment at teaching hospitals also increased, though accompanied by marginally higher cost. Major teaching hospitals appear to return good value for the extra resources used in general surgery, and to some extent vascular surgery, but this was not apparent in orthopedic surgery.

Comparing Outcomes and Costs of Medical Patients Treated at Major Teaching and Non-teaching Hospitals: A National Matched Analysis

BACKGROUND

Teaching hospitals typically pioneer investment in new technology and cultivate workforce characteristics generally associated with better quality, but the value of this extra investment is unclear.

OBJECTIVE

Compare outcomes and costs between major teaching and non-teaching hospitals by closely matching on patient characteristics.

DESIGN

Medicare patients at 339 major teaching hospitals (resident-to-bed (RTB) ratios ≥ 0.25); matched patient controls from 2439 non-teaching hospitals (RTB ratios < 0.05).

PARTICIPANTS

Forty-three thousand nine hundred ninety pairs of patients (one from a major teaching hospital and one from a non-teaching hospital) admitted for acute myocardial infarction (AMI), 84,985 pairs admitted for heart failure (HF), and 74,947 pairs admitted for pneumonia (PNA).

EXPOSURE

Treatment at major teaching hospitals versus non-teaching hospitals.

MAIN MEASURES

Thirty-day all-cause mortality, readmissions, ICU utilization, costs, payments, and value expressed as extra cost for a 1% improvement in survival.

KEY RESULTS

Thirty-day mortality was lower in teaching than non-teaching hospitals (10.7% versus 12.0%, difference = - 1.3%, P < 0.0001). The paired cost difference (teaching - non-teaching) was $273 (P < 0.0001), yielding $211 per 1% mortality improvement. For the quintile of pairs with highest risk on admission, mortality differences were larger (24.6% versus 27.6%, difference = - 3.0%, P < 0.0001), and paired cost difference = $1289 (P < 0.0001), yielding $427 per 1% mortality improvement at 30 days. Readmissions and ICU utilization were lower in teaching hospitals (both P < 0.0001), but length of stay was longer (5.5 versus 5.1 days, P < 0.0001). Finally, individual results for AMI, HF, and PNA showed similar findings as in the combined results.

CONCLUSIONS AND RELEVANCE

Among Medicare patients admitted for common medical conditions, as admission risk of mortality increased, the absolute mortality benefit of treatment at teaching hospitals also increased, though accompanied by marginally higher cost. Major teaching hospitals appear to return good value for the extra resources used.

The Association Between Tibial Slope and Revision Anterior Cruciate Ligament Reconstruction in Patients ≤21 Years Old: A Matched Case-Control Study Including 317 Revisions

BACKGROUND

There is evidence that tibial slope may play a role in revision risk after anterior cruciate ligament reconstruction (ACLR); however, prior studies are inconsistent.

PURPOSE

To determine (1) whether there is a difference in lateral tibial posterior slope (LTPS) or medial tibial posterior slope (MTPS) between patients undergoing revised ACLR and those not requiring revision and (2) whether the medial-to-lateral slope difference is different between these 2 groups.

STUDY DESIGN

Case-control study; Level of evidence, 3.

METHODS

We conducted a matched case-control study (2006-2015). Cases were patients aged ≤21 years who underwent revision surgery after primary unilateral ACLR; controls were patients aged ≤21 years without revision who were identified from the same source population. Controls were matched to cases by age, sex, body mass index, race, graft type, femoral fixation device, and post-ACLR follow-up time. Tibial slope measurements were made by a single blinded reviewer using magnetic resonance imaging. The Wilcoxon signed rank test and McNemar test were used for continuous and categorical variables, respectively.

RESULTS

No difference was observed between revised and nonrevised ACLR groups for LTPS (median: 6° vs 6°, P = .973) or MTPS (median: 4° vs 5°, P = .281). Furthermore, no difference was found for medial-to-lateral slope difference (median: -1 vs -1, P = .289). A greater proportion of patients with revised ACLR had an LTPS ≥12° (7.6% vs 3.8%) and ≥13° (4.7% vs 1.3%); however, this was not statistically significant after accounting for multiple testing.

CONCLUSION

We failed to observe an association between revision ACLR surgery and LTPS, MTPS, or medial-to-lateral slope difference. However, there was a greater proportion of patients in the revision ACLR group with an LTPS ≥12°, suggesting that a minority of patients who have more extreme values of LTPS have a higher revision risk after primary ACLR. A future cohort study evaluating the angle that best differentiates patients at highest risk for revision is needed.

Comparing Resource Use in Medical Admissions of Children With Complex Chronic Conditions

BACKGROUND

Children with complex chronic conditions (CCCs) utilize a disproportionate share of hospital resources.

OBJECTIVE

We asked whether some hospitals display a significantly different pattern of resource utilization than others when caring for similar children with CCCs admitted for medical diagnoses.

RESEARCH DESIGN

Using Pediatric Health Information System data from 2009 to 2013, we constructed an inpatient Template of 300 children with CCCs, matching these to 300 patients at each hospital, thereby performing a type of direct standardization.

SUBJECTS

Children with CCCs were drawn from a list of the 40 most common medical principal diagnoses, then matched to patients across 40 Children's Hospitals.

MEASURES

Rate of intensive care unit admission, length of stay, resource cost.

RESULTS

For the Template-matched patients, when comparing resource use at the lower 12.5-percentile and upper 87.5-percentile of hospitals, we found: intensive care unit utilization was 111% higher (6.6% vs. 13.9%, P<0.001); hospital length of stay was 25% higher (2.4 vs. 3.0 d/admission, P<0.001); and finally, total cost per patient varied by 47% ($6856 vs. $10,047, P<0.001). Furthermore, some hospitals, compared with their peers, were more efficient with low-risk patients and less efficient with high-risk patients, whereas other hospitals displayed the opposite pattern.

CONCLUSIONS

Hospitals treating similar patients with CCCs admitted for similar medical diagnoses, varied greatly in resource utilization. Template Matching can aid chief quality officers benchmarking their hospitals to peer institutions and can help determine types of their patients having the most aberrant outcomes, facilitating quality initiatives to target these patients.

A method to reduce imbalance for site-level randomized stepped wedge implementation trial designs

BACKGROUND

Controlled implementation trials often randomize the intervention at the site level, enrolling relatively few sites (e.g., 6-20) compared to trials that randomize by subject. Trials with few sites carry a substantial risk of an imbalance between intervened (cases) and non-intervened (control) sites in important site characteristics, thereby threatening the internal validity of the primary comparison. A stepped wedge design (SWD) staggers the intervention at sites over a sequence of times or time waves until all sites eventually receive the intervention. We propose a new randomization method, sequential balance, to control time trend in site allocation by minimizing sequential imbalance across multiple characteristics. We illustrate the new method by applying it to a SWD implementation trial.

METHODS

The trial investigated the impact of blended internal-external facilitation on the establishment of evidence-based teams in general mental health clinics in nine US Department of Veterans Affairs medical centers. Prior to randomization to start time, an expert panel of implementation researchers and health system program leaders identified by consensus a series of eight facility-level characteristics judged relevant to the success of implementation. We characterized each of the nine sites according to these consensus features. Using a weighted sum of these characteristics, we calculated imbalance scores for each of 1680 possible site assignments to identify the most sequentially balanced assignment schemes.

RESULTS

From 1680 possible site assignments, we identified 34 assignments with minimal imbalance scores, and then randomly selected one assignment by which to randomize start time. Initially, the mean imbalance score was 3.10, but restricted to the 34 assignments, it declined to 0.99.

CONCLUSIONS

Sequential balancing of site characteristics across groups of sites in the time waves of a SWD strengthens the internal validity of study conclusions by minimizing potential confounding.

TRIAL REGISTRATION

Registered at ClinicalTrials.gov as clinical trials # NCT02543840 ; entered 9/4/2015.

Defining Multimorbidity in Older Surgical Patients

BACKGROUND

There are numerous definitions of multimorbidity (MM). None systematically examines specific comorbidity combinations accounting for multiple testing when exploring large datasets.

OBJECTIVES

Develop and validate a list of all single, double, and triple comorbidity combinations, with each individual qualifying comorbidity set (QCS) more than doubling the odds of mortality versus its reference population. Patients with at least 1 QCS were defined as having MM.

RESEARCH DESIGN

Cohort-based study with a matching validation study.

SUBJECTS

All fee-for-service Medicare patients between age 65 and 85 without dementia or metastatic solid tumors undergoing general surgery in 2009-2010, and an additional 2011-2013 dataset.

MEASURES

30-day all-location mortality.

RESULTS

There were 576 QCSs (2 singles, 63 doubles, and 511 triples), each set more than doubling the odds of dying. In 2011, 36% of eligible patients had MM. As a group, multimorbid patients (mortality rate=7.0%) had a mortality Mantel-Haenszel odds ratio=1.90 (1.77-2.04) versus a reference that included both multimorbid and nonmultimorbid patients (mortality rate=3.3%), and Mantel-Haenszel odds ratio=3.72 (3.51-3.94) versus only nonmultimorbid patients (mortality rate=1.6%). When matching 3151 pairs of multimorbid patients from low-volume hospitals to similar patients in high-volume hospitals, the mortality rates were 6.7% versus 5.2%, respectively (P=0.006).

CONCLUSIONS

A list of QCSs identified a third of older patients undergoing general surgery that had greatly elevated mortality. These sets can be used to identify vulnerable patients and the specific combinations of comorbidities that make them susceptible to poor outcomes.

A review and empirical comparison of causal inference methods for clustered observational data with application to the evaluation of the effectiveness of medical devices

Observational studies are commonplace in medicine. A frequent concern is confounding bias due to differences in patient characteristics across treatment groups, but other important issues include dependency among observations nested within clusters (e.g. patients clustered within physicians or surgeons) and confounding due to cluster characteristics (e.g. physician or surgeon experience or training). Given the frequency with which these issues arise in medical research, as well as their relative complexity, methods for the analysis of clustered observational data are reviewed. We argue for estimating causal treatment effects using marginal models that either match or weight observations using a suitable distance metric (e.g. the propensity score). Simulation results demonstrated that methods incorporating clustering into calculation of the variance were generally more accurate than those that did not. Moreover, methods that account for cluster confounding when estimating the treatment effect were least biased and most accurate. Throughout the paper we illustrate the proposed methods in a medical device setting that compares the effectiveness of femoral heads used in total hip replacements. Whenever possible the clustered aspect of the data should be considered in the design of the study when constructing the distance measure or in the matching process, as well as in the analysis when estimating the variance of the treatment effect.

Assessing the impact of the heart of New Ulm Project on cardiovascular disease risk factors: A population-based program to reduce cardiovascular disease

The Heart of New Ulm Project (HONU), is a population-based project designed to reduce modifiable cardiovascular disease (CVD) risk factors in the rural community of New Ulm, MN. HONU interventions address multiple levels of the social-ecological model. The community is served by one health system, enabling the use of electronic health record (EHR) data for surveillance. The purpose of this study was to assess if trends in CVD risk factors and healthcare utilization differed between a cohort of New Ulm residents age 40-79 and matched controls selected from a similar community, using EHR data from baseline (2008-2009) through three follow up time periods (2010-2011, 2012-2013, 2014-2015). Matching, using covariate balance sparse technique, yielded a sample of 4077 New Ulm residents and 4077 controls. We used mixed effects longitudinal models to examine trends over time between the two groups. Blood pressure, total cholesterol, low-density lipoprotein-cholesterol, and triglycerides showed better management in New Ulm over time compared to the controls. The proportion of residents in New Ulm with controlled blood pressure increased by 6.2 percentage points compared to an increase of 2 points in controls (p < 0.0001). As the cohort aged, 10-year ASCVD risk scores increased less in New Ulm (5.1) than the comparison community (5.9). The intervention and control community did not differ with regard to inpatient stays, smoking, or glucose. Findings suggest efficacy for the HONU project interventions for some outcomes.