Optimal Matching of an Optimally Chosen Subset in Observational Studies

Mortality Among Older Medical Patients at Flagship Hospitals and Their Affiliates

BACKGROUND

We define a "flagship hospital" as the largest academic hospital within a hospital referral region and a "flagship system" as a system that contains a flagship hospital and its affiliates. It is not known if patients admitted to an affiliate hospital, and not to its main flagship hospital, have better outcomes than those admitted to a hospital outside the flagship system but within the same hospital referral region.

OBJECTIVE

To compare mortality at flagship hospitals and their affiliates to matched control patients not in the flagship system but within the same hospital referral region.

DESIGN

A matched cohort study PARTICIPANTS: The study used hospitalizations for common medical conditions between 2018-2019 among older patients age ≥ 66 years. We analyzed 118,321 matched pairs of Medicare patients admitted with pneumonia (N=57,775), heart failure (N=42,531), or acute myocardial infarction (N=18,015) in 35 flagship hospitals, 124 affiliates, and 793 control hospitals.

MAIN MEASURES

30-day (primary) and 90-day (secondary) all-cause mortality.

KEY RESULTS

30-day mortality was lower among patients in flagship systems versus control hospitals that are not part of the flagship system but within the same hospital referral region (difference= -0.62%, 95% CI [-0.88%, -0.37%], P<0.001). This difference was smaller in affiliates versus controls (-0.43%, [-0.75%, -0.11%], P=0.008) than in flagship hospitals versus controls (-1.02%, [-1.46%, -0.58%], P<0.001; difference-in-difference -0.59%, [-1.13%, -0.05%], P=0.033). Similar results were found for 90-day mortality.

LIMITATIONS

The study used claims-based data.

CONCLUSIONS

In aggregate, within a hospital referral region, patients treated at the flagship hospital, at affiliates of the flagship hospital, and in the flagship system as a whole, all had lower mortality rates than matched controls outside the flagship system. However, the mortality advantage was larger for flagship hospitals than for their affiliates.

Impact of Hospital Affiliation With a Flagship Hospital System on Surgical Outcomes

OBJECTIVE

To compare general surgery outcomes at flagship systems, flagship hospitals, and flagship hospital affiliates versus matched controls.

SUMMARY BACKGROUND DATA

It is unknown whether flagship hospitals perform better than flagship hospital affiliates for surgical patients.

METHODS

Using Medicare claims for 2018 to 2019, we matched patients undergoing inpatient general surgery in flagship system hospitals to controls who underwent the same procedure at hospitals outside the system but within the same region. We defined a "flagship hospital" within each region as the major teaching hospital with the highest patient volume that is also part of a hospital system; its system was labeled a "flagship system." We performed 4 main comparisons: patients treated at any flagship system hospital versus hospitals outside the flagship system; flagship hospitals versus hospitals outside the flagship system; flagship hospital affiliates versus hospitals outside the flagship system; and flagship hospitals versus affiliate hospitals. Our primary outcome was 30-day mortality.

RESULTS

We formed 32,228 closely matched pairs across 35 regions. Patients at flagship system hospitals (32,228 pairs) had lower 30-day mortality than matched control patients [3.79% vs. 4.36%, difference=-0.57% (-0.86%, -0.28%), P<0.001]. Similarly, patients at flagship hospitals (15,571/32,228 pairs) had lower mortality than control patients. However, patients at flagship hospital affiliates (16,657/32,228 pairs) had similar mortality to matched controls. Flagship hospitals had lower mortality than affiliate hospitals [difference-in-differences=-1.05% (-1.62%, -0.47%), P<0.001].

CONCLUSIONS

Patients treated at flagship hospitals had significantly lower mortality rates than those treated at flagship hospital affiliates. Hence, flagship system affiliation does not alone imply better surgical outcomes.

Prioritizing Variables for Observational Study Design using the Joint Variable Importance Plot

Observational studies of treatment effects require adjustment for confounding variables. However, causal inference methods typically cannot deliver perfect adjustment on all measured baseline variables, and there is often ambiguity about which variables should be prioritized. Standard prioritization methods based on treatment imbalance alone neglect variables' relationships with the outcome. We propose the joint variable importance plot to guide variable prioritization for observational studies. Since not all variables are equally relevant to the outcome, the plot adds outcome associations to quantify the potential confounding jointly with the standardized mean difference. To enhance comparisons on the plot between variables with different confounding relationships, we also derive and plot bias curves. Variable prioritization using the plot can produce recommended values for tuning parameters in many existing matching and weighting methods. We showcase the use of the joint variable importance plots in the design of a balance-constrained matched study to evaluate whether taking an antidiabetic medication, glyburide, increases the incidence of C-section delivery among pregnant individuals with gestational diabetes.

Using Balancing Weights to Target the Treatment Effect on the Treated when Overlap is Poor

Inverse probability weights are commonly used in epidemiology to estimate causal effects in observational studies. Researchers often focus on either the average treatment effect or the average treatment effect on the treated with inverse probability weighting estimators. However, poor overlap in the baseline covariates between the treated and control groups can produce extreme weights that can result in biased treatment effect estimates. One alternative to inverse probability weights are overlap weights, which target the population with the most overlap on observed covariates. Although estimates based on overlap weights produce less bias in such contexts, the causal estimand can be difficult to interpret. An alternative to model-based inverse probability weights are balancing weights, which directly target imbalances during the estimation process, rather than model fit. Here, we explore whether balancing weights allow analysts to target the average treatment effect on the treated in cases where inverse probability weights lead to biased estimates due to poor overlap. We conduct three simulation studies and an empirical application. We find that balancing weights often allow the analyst to still target the average treatment effect on the treated even when overlap is poor. We show that although overlap weights remain a key tool, more familiar estimands can sometimes be targeted by using balancing weights.

Flexible template matching for observational study design

Matching is a popular design for inferring causal effect with observational data. Unlike model-based approaches, it is a nonparametric method to group treated and control subjects with similar characteristics together, hence to re-create a randomization-like scenario. The application of matched design for real world data may be limited by: (1) the causal estimand of interest; (2) the sample size of different treatment arms. We propose a flexible design of matching, based on the idea of template matching, to overcome these challenges. It first identifies the template group which is representative of the target population, then match subjects from the original data to this template group and make inference. We provide theoretical justification on how it unbiasedly estimates the average treatment effect using matched pairs and the average treatment effect on the treated when the treatment group has a bigger sample size. We also propose using the triplet matching algorithm to improve matching quality and devise a practical strategy to select the template size. One major advantage of matched design is that it allows both randomization-based or model-based inference, with the former being more robust. For the commonly used binary outcome in medical research, we adopt a randomization inference framework of attributable effects in matched data, which allows heterogeneous effects and can incorporate sensitivity analysis for unmeasured confounding. We apply our design and analytical strategy to a trauma care evaluation study.

Addressing positivity violations in causal effect estimation using Gaussian process priors

In observational studies, causal inference relies on several key identifying assumptions. One identifiability condition is the positivity assumption, which requires the probability of treatment be bounded away from 0 and 1. That is, for every covariate combination, it should be possible to observe both treated and control subjects the covariate distributions should overlap between treatment arms. If the positivity assumption is violated, population-level causal inference necessarily involves some extrapolation. Ideally, a greater amount of uncertainty about the causal effect estimate should be reflected in such situations. With that goal in mind, we construct a Gaussian process model for estimating treatment effects in the presence of practical violations of positivity. Advantages of our method include minimal distributional assumptions, a cohesive model for estimating treatment effects, and more uncertainty associated with areas in the covariate space where there is less overlap. We assess the performance of our approach with respect to bias and efficiency using simulation studies. The method is then applied to a study of critically ill female patients to examine the effect of undergoing right heart catheterization.

Addressing Disparities in the Propensity Score Distributions for Treatment Comparisons from Observational Studies

Propensity score (PS) based methods, such as matching, stratification, regression adjustment, simple and augmented inverse probability weighting, are popular for controlling for observed confounders in observational studies of causal effects. More recently, we proposed penalized spline of propensity prediction (PENCOMP), which multiply-imputes outcomes for unassigned treatments using a regression model that includes a penalized spline of the estimated selection probability and other covariates. For PS methods to work reliably, there should be sufficient overlap in the propensity score distributions between treatment groups. Limited overlap can result in fewer subjects being matched or in extreme weights causing numerical instability and bias in causal estimation. The problem of limited overlap suggests (a) defining alternative estimands that restrict inferences to subpopulations where all treatments have the potential to be assigned, and (b) excluding or down-weighting sample cases where the propensity to receive one of the compared treatments is close to zero. We compared PENCOMP and other PS methods for estimation of alternative causal estimands when limited overlap occurs. Simulations suggest that, when there are extreme weights, PENCOMP tends to outperform the weighted estimators for ATE and performs similarly to the weighted estimators for alternative estimands. We illustrate PENCOMP in two applications: the effect of antiretroviral treatments on CD4 counts using the Multicenter AIDS cohort study (MACS) and whether right heart catheterization (RHC) is a beneficial treatment in treating critically ill patients.

Penalized Spline-Involved Tree-based (PenSIT) Learning for estimating an optimal dynamic treatment regime using observational data

Dynamic treatment regimes are a set of time-adaptive decision rules that can be used to personalize treatment across multiple stages of care. Grounded in causal inference methods, dynamic treatment regimes identify variables that differentiate the treatment effect and may be used to tailor treatments across individuals based on the patient's own characteristics - thereby representing an important step toward personalized medicine. In this manuscript we introduce Penalized Spline-Involved Tree-based Learning, which seeks to improve upon existing tree-based approaches to estimating an optimal dynamic treatment regime. Instead of using weights determined from the estimated propensity scores, which may result in unstable estimates when weights are highly variable, we predict missing counterfactual outcomes using regression models that incorporate a penalized spline of the propensity score and other covariates predictive of the outcome. We further develop a novel purity measure applied within a decision tree framework to produce a flexible yet interpretable method for estimating an optimal multi-stage multi-treatment dynamic treatment regime. In simulation experiments we demonstrate good performance of Penalized Spline-Involved Tree-based Learning relative to competing methods and, in particular, we show that Penalized Spline-Involved Tree-based Learning may be advantageous when the sample size is small and/or when the level of confounding of the outcome is high. We apply Penalized Spline-Involved Tree-based Learning to the retrospectively-collected Medical Information Mart for Intensive Care dataset to identify variables that may be used to tailor early fluid resuscitation strategies in septic patients.

Testing Biased Randomization Assumptions and Quantifying Imperfect Matching and Residual Confounding in Matched Observational Studies

One central goal of design of observational studies is to embed non-experimental data into an approximate randomized controlled trial using statistical matching. Despite empirical researchers' best intention and effort to create high-quality matched samples, residual imbalance due to observed covariates not being well matched often persists. Although statistical tests have been developed to test the randomization assumption and its implications, few provide a means to quantify the level of residual confounding due to observed covariates not being well matched in matched samples. In this article, we develop two generic classes of exact statistical tests for a biased randomization assumption. One important by-product of our testing framework is a quantity called residual sensitivity value (RSV), which provides a means to quantify the level of residual confounding due to imperfect matching of observed covariates in a matched sample. We advocate taking into account RSV in the downstream primary analysis. The proposed methodology is illustrated by re-examining a famous observational study concerning the effect of right heart catheterization (RHC) in the initial care of critically ill patients. Code implementing the method can be found in the supplementary materials.

A generalizability score for aggregate causal effect

Scientists frequently generalize population level causal quantities such as average treatment effect from a source population to a target population. When the causal effects are heterogeneous, differences in subject characteristics between the source and target populations may make such a generalization difficult and unreliable. Reweighting or regression can be used to adjust for such differences when generalizing. However, these methods typically suffer from large variance if there is limited covariate distribution overlap between the two populations. We propose a generalizability score to address this issue. The score can be used as a yardstick to select target subpopulations for generalization. A simplified version of the score avoids using any outcome information and thus can prevent deliberate biases associated with inadvertent access to such information. Both simulation studies and real data analysis demonstrate convincing results for such selection.

Core concepts in pharmacoepidemiology: Violations of the positivity assumption in the causal analysis of observational data: Consequences and statistical approaches

In the causal analysis of observational data, the positivity assumption requires that all treatments of interest be observed in every patient subgroup. Violations of this assumption are indicated by nonoverlap in the data in the sense that patients with certain covariate combinations are not observed to receive a treatment of interest, which may arise from contraindications to treatment or small sample size. In this paper, we emphasize the importance and implications of this often-overlooked assumption. Further, we elaborate on the challenges nonoverlap poses to estimation and inference and discuss previously proposed methods. We distinguish between structural and practical violations and provide insight into which methods are appropriate for each. To demonstrate alternative approaches and relevant considerations (including how overlap is defined and the target population to which results may be generalized) when addressing positivity violations, we employ an electronic health record-derived data set to assess the effects of metformin on colon cancer recurrence among diabetic patients.

Propensity score methods for time-dependent cluster confounding

In observational studies, subjects are often nested within clusters. In medical studies, patients are often treated by doctors and therefore patients are regarded as nested or clustered within doctors. A concern that arises with clustered data is that cluster-level characteristics (e.g., characteristics of the doctor) are associated with both treatment selection and patient outcomes, resulting in cluster-level confounding. Measuring and modeling cluster attributes can be difficult and statistical methods exist to control for all unmeasured cluster characteristics. An assumption of these methods however is that characteristics of the cluster and the effects of those characteristics on the outcome (as well as probability of treatment assignment when using covariate balancing methods) are constant over time. In this paper, we consider methods that relax this assumption and allow for estimation of treatment effects in the presence of unmeasured time-dependent cluster confounding. The methods are based on matching with the propensity score and incorporate unmeasured time-specific cluster effects by performing matching within clusters or using fixed- or random-cluster effects in the propensity score model. The methods are illustrated using data to compare the effectiveness of two total hip devices with respect to survival of the device and a simulation study is performed that compares the proposed methods. One method that was found to perform well is matching within surgeon clusters partitioned by time. Considerations in implementing the proposed methods are discussed.

Estimating treatment effects with machine learning

OBJECTIVE

To demonstrate the performance of methodologies that include machine learning (ML) algorithms to estimate average treatment effects under the assumption of exogeneity (selection on observables).

DATA SOURCES

Simulated data and observational data on hospitalized adults.

STUDY DESIGN

We assessed the performance of several ML-based estimators, including Targeted Maximum Likelihood Estimation, Bayesian Additive Regression Trees, Causal Random Forests, Double Machine Learning, and Bayesian Causal Forests, applying these methods to simulated data as well as data on the effects of right heart catheterization.

PRINCIPAL FINDINGS

In Monte Carlo studies, ML-based estimators generated estimates with smaller bias than traditional regression approaches, demonstrating substantial (69 percent-98 percent) bias reduction in some scenarios. Bayesian Causal Forests and Double Machine Learning were top performers, although all were sensitive to high dimensional (>150) sets of covariates.

CONCLUSIONS

ML-based methods are promising methods for estimating treatment effects, allowing for the inclusion of many covariates and automating the search for nonlinearities and interactions among variables. We provide guidance and sample code for researchers interested in implementing these tools in their own empirical work.

Comparison of alternative approaches to trim subjects in the tails of the propensity score distribution

PURPOSE

In nonexperimental comparative effectiveness research, restricting analysis to subjects with better overlap of covariate distributions, hence greater treatment equipoise, helps balance the groups compared and can improve validity. Three alternative approaches, derived from different perspectives, implement restriction by trimming observations in the tails of the propensity score (PS). Across approaches, we compared the relationships between the overlap in treatment-specific PS distributions and the size of the balanced study population after trimming.

METHODS

The three trimming approaches considered were absolute trimming to the range 0.1

RESULTS

The magnitude of the C-statistic strongly predicted (R2 ≥.95) the percent of the balanced study population remaining. The balanced study population was largest under trimming at absolute PS levels, unless the target treatment was uncommon. Fewer than half of original study subjects remained after preference score trimming if C≥.80 and after asymmetric trimming if C≥.85. In examples, trimming improved the precision of estimated risk differences and identified apparent treatment effect heterogeneity in the PS tails where covariate balance was limited. Relative amounts of trimming in examples reflected the simulation results.

CONCLUSIONS

Study populations with high PS C-statistics include only small percentages of subjects in whom valid treatment effects are confidently expected.

Collapse

Key Words

• comparative effectiveness
• confounding bias
• equipoise
• methods
• pharmacoepidemiology
• propensity score
• statistical

Collapse

MESH Headings

• Bias
• Comparative Effectiveness Research/methods
• Computer Simulation
• Data Interpretation, Statistical
• Models, Statistical
• Propensity Score
• Sample Size

Collapse

Grants

• AG056479 NIA NIH HHS
• R21 HD080214 NICHD NIH HHS
• UL1 TR002489 NCATS NIH HHS
• R01 AG056479 NIA NIH HHS
• R01 CA174453 NCI NIH HHS
• R01 HL118255 NHLBI NIH HHS

Collapse

Affiliation(s)

Robert J Glynn Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts
Mark Lunt The Arthritis Research UK Epidemiology Unit, University of Manchester, Manchester, UK
Kenneth J Rothman RTI Health Solutions, and the Department of Epidemiology, Boston University, Boston, Massachusetts
Charles Poole Department of Epidemiology, University of North Carolina, Chapel Hill, North Carolina
Sebastian Schneeweiss Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts
Til Stürmer Department of Epidemiology, University of North Carolina, Chapel Hill, North Carolina

Collapse

Causal Modeling in Environmental Health

The field of environmental health has been dominated by modeling associations, especially by regressing an observed outcome on a linear or nonlinear function of observed covariates. Readers interested in advances in policies for improving environmental health are, however, expecting to be informed about health effects resulting from, or more explicitly caused by, environmental exposures. The quantification of health impacts resulting from the removal of environmental exposures involves causal statements. Therefore, when possible, causal inference frameworks should be considered for analyzing the effects of environmental exposures on health outcomes.

Practice Style Variation in Medicaid and Non-Medicaid Children With Complex Chronic Conditions Undergoing Surgery

OBJECTIVES

With differential payment between Medicaid and Non-Medicaid services, we asked whether style-of-practice differs between similar Medicaid and Non-Medicaid children with complex chronic conditions (CCCs) undergoing surgery.

SUMMARY OF BACKGROUND DATA

Surgery in children with CCCs accounts for a disproportionately large percentage of resource utilization at major children's hospitals.

METHODS

A matched cohort design, studying 23,582 pairs of children with CCCs undergoing surgery (Medicaid matched to Non-Medicaid within the same hospital) from 2009 to 2013 in 41 Children's Hospitals. Patients were matched on age, sex, principal procedure, CCCs, and other characteristics.

RESULTS

Median cost in Medicaid patients was $21,547 versus $20,527 in Non-Medicaid patients (5.0% higher, P < 0.001). Median paired difference in cost (Medicaid minus Non-Medicaid) was $320 [95% confidence interval (CI): $208, $445], (1.6% higher, P < 0.001). 90th percentile costs were $133,640 versus $127,523, (4.8% higher, P < 0.001). Mean paired difference in length of stay (LOS) was 0.50 days (95% CI: 0.36, 0.65), (P < 0.001). ICU utilization was 2.8% higher (36.7% vs 35.7%, P < 0.001). Finally, in-hospital mortality pooled across all pairs was higher in Medicaid patients (0.38% vs 0.22%, P = 0.002). After adjusting for multiple testing, no individual hospital displayed significant differences in cost between groups, only 1 hospital displayed significant differences in LOS and 1 in ICU utilization.

CONCLUSIONS

Treatment style differences between Medicaid and Non-Medicaid children were small, suggesting little disparity with in-hospital surgical care for patients with CCCs operated on within Children's Hospitals. However, in-hospital mortality, although rare, was slightly higher in Medicaid patients and merits further investigation.

Outcomes, ICU Use, and Length of Stay in Chronically Ill Black and White Children on Medicaid and Hospitalized for Surgery

BACKGROUND

With increasing Medicaid coverage, it has become especially important to determine whether racial differences exist within the Medicaid system. We asked whether disparities exist in hospital practice and patient outcomes between matched black and white Medicaid children with chronic conditions undergoing surgery.

STUDY DESIGN

We conducted a matched cohort study, matching 6,398 pairs within states on detailed patient characteristics using data from 25 states contributing adequate Medicaid Analytic eXtract claims for admissions of children with chronic conditions undergoing the same surgical procedures between January 1, 2009 and November 30, 2010 for ages 1 to 18 years.

RESULTS

The black patient 30-day revisit rate was 19.3% vs 19.8% in matched white patients (p = 0.61), 30-day readmission rates were 7.0% vs 6.9% (p = 0.43), and 30-day mortality rates were 0.38% vs 0.19% (p = 0.06), respectively. A higher percentage of black patients exceeded their own state's individual median length of stay (44.0% vs 39.6%; p < 0.001) and median ICU length of stay (25.9% vs 23.8%; p < 0.001). Intensive care unit use was higher in black patients (25.9% vs 23.8%; p < 0.001). After adjusting for multiple testing, only 2 states were found to differ significantly by race (New York for length of stay and New Jersey for ICU use).

CONCLUSIONS

We did not observe disparities in 30-day revisits and readmissions for chronically ill children in Medicaid undergoing surgery, and only slight differences in length of stay, ICU length of stay, and use of the ICU, where blacks displayed somewhat elevated rates compared with white controls.

Racial Disparities in Medicaid Asthma Hospitalizations

BACKGROUND AND OBJECTIVES

Black children with asthma comprise one-third of all asthma patients in Medicaid. With increasing Medicaid coverage, it has become especially important to monitor Medicaid for differences in hospital practice and patient outcomes by race.

METHODS

A multivariate matched cohort design, studying 11 079 matched pairs of children in Medicaid (black versus white matched pairs from inside the same state) admitted for asthma between January 1, 2009 and November 30, 2010 in 33 states contributing adequate Medicaid Analytic eXtract claims.

RESULTS

Ten-day revisit rates were 3.8% in black patients versus 4.2% in white patients (P = .12); 30-day revisit and readmission rates were also not significantly different by race (10.5% in black patients versus 10.8% in white patients; P = .49). Length of stay (LOS) was also similar; both groups had a median stay of 2.0 days, with a slightly lower percentage of black patients exceeding their own state's median LOS (30.2% in black patients versus 31.8% in white patients; P = .01). The mean paired difference in LOS was 0.00 days (95% confidence interval, -0.08 to 0.08). However, ICU use was higher in black patients than white patients (22.2% versus 17.5%; P < .001). After adjusting for multiple testing, only 4 states were found to differ significantly, but only in ICU use, where blacks had higher rates of use.

CONCLUSIONS

For closely matched black and white patients, racial disparities concerning asthma admission outcomes and style of practice are small and generally nonsignificant, except for ICU use, where we observed higher rates in black patients.

Effect of prophylactic CPAP in very low birth weight infants in South America

OBJECTIVE

The objective of this study was to examine the effect of prophylactic continuous positive airway pressure (CPAP) on infants born in 25 South American neonatal intensive care units affiliated with the Neocosur Neonatal Network using novel multivariate matching methods.

STUDY DESIGN

A prospective cohort was constructed of infants with a birth weight 500 to 1500 g born between 2005 and 2011 who clinically were eligible for prophylactic CPAP. Patients who received prophylactic CPAP were matched to those who did not on 23 clinical and sociodemographic variables (N=1268). Outcomes were analyzed using the McNemar's test.

RESULTS

Infants not receiving prophylactic CPAP had higher mortality rates (odds ratio (OR)=1.69, 95% confidence interval (CI) 1.17, 2.46), need for any mechanical ventilation (OR=1.68, 95% CI 1.33, 2.14) and death or bronchopulmonary dysplasia (BPD) (OR=1.47, 95% CI 1.09, 1.98). The benefit of prophylactic CPAP varied by birth weight and gender.

CONCLUSIONS

The implementation of this process was associated with a significant improvement in survival and survival free of BPD.

Practice Patterns in Medicaid and Non-Medicaid Asthma Admissions

BACKGROUND AND OBJECTIVES

With American children experiencing increased Medicaid coverage, it has become especially important to determine if practice patterns differ between Medicaid and non-Medicaid patients. Auditing such potential differences must carefully compare like patients to avoid falsely identifying suspicious practice patterns. We asked if we could observe differences in practice patterns between Medicaid and non-Medicaid patients admitted for asthma inside major children's hospitals.

METHODS

A matched cohort design, studying 17 739 matched pairs of children (Medicaid to non-Medicaid) admitted for asthma in the same hospital between April 1, 2011 and March 31, 2014 in 40 Children's Hospital Association hospitals contributing data to the Pediatric Hospital Information System database. Patients were matched on age, sex, asthma severity, and other patient characteristics.

RESULTS

Medicaid patient median cost was $4263 versus $4160 for non-Medicaid patients (P < .001). Additionally, the median cost difference (Medicaid minus non-Medicaid) between individual pairs was only $84 (95% confidence interval: 44 to 124), and the mean cost difference was only $49 (95% confidence interval: -72 to 170). The 90th percentile costs were also similar between groups ($10 710 vs $10 948; P < .07). Length of stay (LOS) was also very similar; both groups had a median stay of 1 day, with a similar percentage of patients exceeding the 90th percentile of individual hospital LOS (7.1% vs 6.7%; P = .14). ICU use was also similar (10.1% vs 10.6%; P = .12).

CONCLUSIONS

For closely matched patients within the same hospital, Medicaid status did not importantly influence costs, LOS, or ICU use.

Evaluation of subset matching methods and forms of covariate balance

This paper conducts a Monte Carlo simulation study to evaluate the performance of multivariate matching methods that select a subset of treatment and control observations. The matching methods studied are the widely used nearest neighbor matching with propensity score calipers and the more recently proposed methods, optimal matching of an optimally chosen subset and optimal cardinality matching. The main findings are: (i) covariate balance, as measured by differences in means, variance ratios, Kolmogorov-Smirnov distances, and cross-match test statistics, is better with cardinality matching because by construction it satisfies balance requirements; (ii) for given levels of covariate balance, the matched samples are larger with cardinality matching than with the other methods; (iii) in terms of covariate distances, optimal subset matching performs best; (iv) treatment effect estimates from cardinality matching have lower root-mean-square errors, provided strong requirements for balance, specifically, fine balance, or strength-k balance, plus close mean balance. In standard practice, a matched sample is considered to be balanced if the absolute differences in means of the covariates across treatment groups are smaller than 0.1 standard deviations. However, the simulation results suggest that stronger forms of balance should be pursued in order to remove systematic biases due to observed covariates when a difference in means treatment effect estimator is used. In particular, if the true outcome model is additive, then marginal distributions should be balanced, and if the true outcome model is additive with interactions, then low-dimensional joints should be balanced. Copyright © 2016 John Wiley & Sons, Ltd.

Examining Causes of Racial Disparities in General Surgical Mortality: Hospital Quality Versus Patient Risk

BACKGROUND

Racial disparities in general surgical outcomes are known to exist but not well understood.

OBJECTIVES

To determine if black-white disparities in general surgery mortality for Medicare patients are attributable to poorer health status among blacks on admission or differences in the quality of care provided by the admitting hospitals.

RESEARCH DESIGN

Matched cohort study using Tapered Multivariate Matching.

SUBJECTS

All black elderly Medicare general surgical patients (N=18,861) and white-matched controls within the same 6 states or within the same 838 hospitals.

MEASURES

Thirty-day mortality (primary); others include in-hospital mortality, failure-to-rescue, complications, length of stay, and readmissions.

RESULTS

Matching on age, sex, year, state, and the exact same procedure, blacks had higher 30-day mortality (4.0% vs. 3.5%, P<0.01), in-hospital mortality (3.9% vs. 2.9%, P<0.0001), in-hospital complications (64.3% vs. 56.8% P<0.0001), and failure-to-rescue rates (6.1% vs. 5.1%, P<0.001), longer length of stay (7.2 vs. 5.8 d, P<0.0001), and more 30-day readmissions (15.0% vs. 12.5%, P<0.0001). Adding preoperative risk factors to the above match, there was no significant difference in mortality or failure-to-rescue, and all other outcome differences were small. Blacks matched to whites in the same hospital displayed no significant differences in mortality, failure-to-rescue, or readmissions.

CONCLUSIONS

Black and white Medicare patients undergoing the same procedures with closely matched risk factors displayed similar mortality, suggesting that racial disparities in general surgical mortality are not because of differences in hospital quality. To reduce the observed disparities in surgical outcomes, the poorer health of blacks on presentation for surgery must be addressed.

Reweighted Mahalanobis distance matching for cluster-randomized trials with missing data

PURPOSE

This paper introduces an improved tool for designing matched-pairs randomized trials. The tool allows the incorporation of clinical and other knowledge regarding the relative importance of variables used in matching and allows for multiple types of missing data. The method is illustrated in the context of a cluster-randomized trial. A Web application and an R package are introduced to implement the method and incorporate recent advances in the area.

METHODS

Reweighted Mahalanobis distance (RMD) matching incorporates user-specified weights and imputed values for missing data. Weight may be assigned to missingness indicators to match on missingness patterns. Three examples are presented, using real data from a cohort of 90 Veterans Health Administration sites that had at least 100 incident metformin users in 2007. Matching is utilized to balance seven factors aggregated at the site level. Covariate balance is assessed for 10,000 randomizations under each strategy: simple randomization, matched randomization using the Mahalanobis distance, and matched randomization using the RMD.

RESULTS

The RMD matching achieved better balance than simple randomization or MD randomization. In the first example, simple and MD randomization resulted in a 10% chance of seeing an absolute mean difference of greater than 26% in the percent of nonwhite patients per site; the RMD dramatically reduced that to 6%. The RMD achieved significant improvement over simple randomization even with as much as 20% of the data missing.

CONCLUSIONS

Reweighted Mahalanobis distance matching provides an easy-to-use tool that incorporates user knowledge and missing data.

Contrasting Evidence Within and Between Institutions That Provide Treatment in an Observational Study of Alternate Forms of Anesthesia

In a randomized trial, subjects are assigned to treatment or control by the flip of a fair coin. In many nonrandomized or observational studies, subjects find their way to treatment or control in two steps, either or both of which may lead to biased comparisons. By a vague process perhaps affected by proximity or sociodemographic issues, subjects find their way to institutions that provide treatment. Once at such an institution, a second process, perhaps thoughtful and deliberate, assigns individuals to treatment or control. In the current paper, the institutions are hospitals, and the treatment under study is the use of general anesthesia alone versus some use of regional anesthesia during surgery. For a specific operation, the use of regional anesthesia may be typical in one hospital and atypical in another. A new matched design is proposed for studies of this sort, one that creates two types of nonoverlapping matched pairs. Using a new extension of optimal matching with fine balance, pairs of the first type exactly balance treatment assignment across institutions, so each institution appears in the treated group with the same frequency that it appears in the control group; hence, differences between institutions that affect everyone in the same way cannot bias this comparison. Pairs of the second type compare institutions that assign most subjects to treatment and other institutions that assign most subjects to control, so each institution is represented in the treated group if it typically assigns subjects to treatment or alternatively in the control group if it typically assigns subjects to control, and no institution appears in both groups. By and large, in the second type of matched pair, subjects became treated subjects or controls by choosing an institution, not by a thoughtful and deliberate process of selecting subjects for treatment within institutions. The design provides two evidence factors, that is, two tests of the null hypothesis of no treatment effect that are independent when the null hypothesis is true, where each factor is largely unaffected by certain unmeasured biases that could readily invalidate the other factor. The two factors permit separate and combined sensitivity analyses, where the magnitude of bias affecting the two factors may differ. The case of knee surgery in the study of regional versus general anesthesia is considered in detail.