Template matching for auditing hospital cost and quality

Prognostic score-based methods for estimating center effects based on survival probability: Application to post-kidney transplant survival

In evaluating the performance of different facilities or centers on survival outcomes, the standardized mortality ratio (SMR), which compares the observed to expected mortality has been widely used, particularly in the evaluation of kidney transplant centers. Despite its utility, the SMR may exaggerate center effects in settings where survival probability is relatively high. An example is one-year graft survival among U.S. kidney transplant recipients. We propose a novel approach to estimate center effects in terms of differences in survival probability (ie, each center versus a reference population). An essential component of the method is a prognostic score weighting technique, which permits accurately evaluating centers without necessarily specifying a correct survival model. Advantages of our approach over existing facility-profiling methods include a metric based on survival probability (greater clinical relevance than ratios of counts/rates); direct standardization (valid to compare between centers, unlike indirect standardization based methods, such as the SMR); and less reliance on correct model specification (since the assumed model is used to generate risk classes as opposed to fitted-value based 'expected' counts). We establish the asymptotic properties of the proposed weighted estimator and evaluate its finite-sample performance under a diverse set of simulation settings. The method is then applied to evaluate U.S. kidney transplant centers with respect to graft survival probability.

Efficient algorithms for building representative matched pairs with enhanced generalizability

Many recent efforts center on assessing the ability of real-world evidence (RWE) generated from non-randomized, observational data to produce results compatible with those from randomized controlled trials (RCTs). One noticeable endeavor is the RCT DUPLICATE initiative. To better reconcile findings from an observational study and an RCT, or two observational studies based on different databases, it is desirable to eliminate differences between study populations. We outline an efficient, network-flow-based statistical matching algorithm that designs well-matched pairs from observational data that resemble the covariate distributions of a target population, for instance, the target-RCT-eligible population in the RCT DUPLICATE initiative studies or a generic population of scientific interest. We demonstrate the usefulness of the method by revisiting the inconsistency regarding a cardioprotective effect of the hormone replacement therapy (HRT) in the Women's Health Initiative (WHI) clinical trial and corresponding observational study. We found that the discrepancy between the trial and observational study persisted in a design that adjusted for the difference in study populations' cardiovascular risk profile, but seemed to disappear in a study design that further adjusted for the difference in HRT initiation age and previous estrogen-plus-progestin use. The proposed method is integrated into the R package match2C.

Entropy balancing for causal generalization with target sample summary information

In this paper, we focus on estimating the average treatment effect (ATE) of a target population when individual-level data from a source population and summary-level data (e.g., first or second moments of certain covariates) from the target population are available. In the presence of the heterogeneous treatment effect, the ATE of the target population can be different from that of the source population when distributions of treatment effect modifiers are dissimilar in these two populations, a phenomenon also known as covariate shift. Many methods have been developed to adjust for covariate shift, but most require individual covariates from a representative target sample. We develop a weighting approach based on the summary-level information from the target sample to adjust for possible covariate shift in effect modifiers. In particular, weights of the treated and control groups within a source sample are calibrated by the summary-level information of the target sample. Our approach also seeks additional covariate balance between the treated and control groups in the source sample. We study the asymptotic behavior of the corresponding weighted estimator for the target population ATE under a wide range of conditions. The theoretical implications are confirmed in simulation studies and a real-data application.

Reformulating provider profiling by grouping providers treating similar patients prior to evaluating performance

Standard approaches to comparing health providers' performance rely on hierarchical logistic regression models that adjust for patient characteristics at admission. Estimates from these models may be misleading when providers treat different patient populations and the models are misspecified. To address this limitation, we propose a novel profiling approach that identifies groups of providers treating similar populations of patients and then evaluates providers' performance within each group. The groups of providers are identified using a Bayesian multilevel finite mixture of general location models. To compare the performance of our proposed profiling approach to standard methods, we use patient-level data from 119 skilled nursing facilities in Massachusetts. We use simulated and observed outcome data to explore the performance of these profiling methods in different settings. In simulations, our proposed method classifies providers to groups with similar patients' admission characteristics. In addition, in the presence of limited overlap in patient characteristics across providers and misspecifications of the outcome model, the provider-level estimates obtained using our approach identified providers that under- and overperformed compared to the standard regression-based approaches more accurately.

Flexible template matching for observational study design

Matching is a popular design for inferring causal effect with observational data. Unlike model-based approaches, it is a nonparametric method to group treated and control subjects with similar characteristics together, hence to re-create a randomization-like scenario. The application of matched design for real world data may be limited by: (1) the causal estimand of interest; (2) the sample size of different treatment arms. We propose a flexible design of matching, based on the idea of template matching, to overcome these challenges. It first identifies the template group which is representative of the target population, then match subjects from the original data to this template group and make inference. We provide theoretical justification on how it unbiasedly estimates the average treatment effect using matched pairs and the average treatment effect on the treated when the treatment group has a bigger sample size. We also propose using the triplet matching algorithm to improve matching quality and devise a practical strategy to select the template size. One major advantage of matched design is that it allows both randomization-based or model-based inference, with the former being more robust. For the commonly used binary outcome in medical research, we adopt a randomization inference framework of attributable effects in matched data, which allows heterogeneous effects and can incorporate sensitivity analysis for unmeasured confounding. We apply our design and analytical strategy to a trauma care evaluation study.

Alzheimer's Dementia After Exposure to Anesthesia and Surgery in the Elderly: A Matched Natural Experiment Using Appendicitis

OBJECTIVE

The aim of this study was to determine whether surgery and anesthesia in the elderly may promote Alzheimer disease and related dementias (ADRD).

BACKGROUND

There is a substantial conflicting literature concerning the hypothesis that surgery and anesthesia promotes ADRD. Much of the literature is confounded by indications for surgery or has small sample size. This study examines elderly patients with appendicitis, a common condition that strikes mostly at random after controlling for some known associations.

METHODS

A matched natural experiment of patients undergoing appendectomy for appendicitis versus control patients without appendicitis using Medicare data from 2002 to 2017, examining 54,996 patients without previous diagnoses of ADRD, cognitive impairment, or neurological degeneration, who developed appendicitis between ages 68 through 77 years and underwent an appendectomy (the ''Appendectomy'' treated group), matching them 5:1 to 274,980 controls, examining the subsequent hazard for developing ADRD.

RESULTS

The hazard ratio (HR) for developing ADRD or death was lower in the Appendectomy group than controls: HR = 0.96 [95% confidence interval (CI) 0.94-0.98], P < 0.0001, (28.2% in Appendectomy vs 29.1% in controls, at 7.5 years). The HR for death was 0.97 (95% CI 0.95-0.99), P = 0.002, (22.7% vs 23.1% at 7.5 years). The HR for developing ADRD alone was 0.89 (95% CI 0.86-0.92), P < 0.0001, (7.6% in Appendectomy vs 8.6% in controls, at 7.5 years). No subgroup analyses found significantly elevated rates of ADRD in the Appendectomy group.

CONCLUSION

In this natural experiment involving 329,976 elderly patients, exposure to appendectomy surgery and anesthesia did not increase the subsequent rate of ADRD.

Emerging approaches to multiple chronic condition assessment

Older adults experience a higher prevalence of multiple chronic conditions (MCCs). Establishing the presence and pattern of MCCs in individuals or populations is important for healthcare delivery, research, and policy. This report describes four emerging approaches and discusses their potential applications for enhancing assessment, treatment, and policy for the aging population. The National Institutes of Health convened a 2-day panel workshop of experts in 2018. Four emerging models were identified by the panel, including classification and regression tree (CART), qualifying comorbidity sets (QCS), the multimorbidity index (MMI), and the application of omics to network medicine. Future research into models of multiple chronic condition assessment may improve understanding of the epidemiology, diagnosis, and treatment of older persons.

Profile Matching for the Generalization and Personalization of Causal Inferences

We introduce profile matching, a multivariate matching method for randomized experiments and observational studies that finds the largest possible unweighted samples across multiple treatment groups that are balanced relative to a covariate profile. This covariate profile can represent a specific population or a target individual, facilitating the generalization and personalization of causal inferences. For generalization, because the profile often amounts to summary statistics for a target population, profile matching does not always require accessing individual-level data, which may be unavailable for confidentiality reasons. For personalization, the profile comprises the characteristics of a single individual. Profile matching achieves covariate balance by construction, but unlike existing approaches to matching, it does not require specifying a matching ratio, as this is implicitly optimized for the data. The method can also be used for the selection of units for study follow-up, and it readily applies to multivalued treatments with many treatment categories. We evaluate the performance of profile matching in a simulation study of the generalization of a randomized trial to a target population. We further illustrate this method in an exploratory observational study of the relationship between opioid use and mental health outcomes. We analyze these relationships for three covariate profiles representing: (i) sexual minorities, (ii) the Appalachian United States, and (iii) the characteristics of a hypothetical vulnerable patient. The method can be implemented via the new function profmatch in the designmatch package for R, for which we provide a step-by-step tutorial.

Interpretability, credibility, and usability of hospital-specific template matching versus regression-based hospital performance assessments; a multiple methods study

BACKGROUND

Hospital-specific template matching (HS-TM) is a newer method of hospital performance assessment.

OBJECTIVE

To assess the interpretability, credibility, and usability of HS-TM-based vs. regression-based performance assessments.

RESEARCH DESIGN

We surveyed hospital leaders (January-May 2021) and completed follow-up semi-structured interviews. Surveys included four hypothetical performance assessment vignettes, with method (HS-TM, regression) and hospital mortality randomized.

SUBJECTS

Nationwide Veterans Affairs Chiefs of Staff, Medicine, and Hospital Medicine.

MEASURES

Correct interpretation; self-rated confidence in interpretation; and self-rated trust in assessment (via survey). Concerns about credibility and main uses (via thematic analysis of interview transcripts).

RESULTS

In total, 84 participants completed 295 survey vignettes. Respondents correctly interpreted 81.8% HS-TM vs. 56.5% regression assessments, p < 0.001. Respondents "trusted the results" for 70.9% HS-TM vs. 58.2% regression assessments, p = 0.03. Nine concerns about credibility were identified: inadequate capture of case-mix and/or illness severity; inability to account for specialized programs (e.g., transplant center); comparison to geographically disparate hospitals; equating mortality with quality; lack of criterion standards; low power; comparison to dissimilar hospitals; generation of rankings; and lack of transparency. Five concerns were equally relevant to both methods, one more pertinent to HS-TM, and three more pertinent to regression. Assessments were mainly used to trigger further quality evaluation (a "check oil light") and motivate behavior change.

CONCLUSIONS

HS-TM-based performance assessments were more interpretable and more credible to VA hospital leaders than regression-based assessments. However, leaders had a similar set of concerns related to credibility for both methods and felt both were best used as a screen for further evaluation.

Matching on poset-based average rank for multiple treatments to compare many unbalanced groups

In this article, we propose an original matching procedure for multiple treatment frameworks based on partially ordered set theory (poset). In our proposal, called matching on poset‐based average rank for multiple treatments (MARMoT), poset theory is used to summarize individuals' confounders and the relative average rank is used to balance confounders and match individuals in different treatment groups. This approach proves to be particularly useful for balancing confounders when the number of treatments considered is high. We apply our approach to the estimation of neighborhood effect on the fractures among older people in Turin (a city in northern Italy).

Facility profiling under competing risks using multivariate prognostic scores: Application to kidneytransplant centers

The performance of health care facilities (e.g. hospitals, transplant centers, etc.) is often evaluated through time-to-event outcomes. In this paper, we consider the case where, for each subject, the failure event is due to one of several mutually exclusive causes (competing risks). Since the distribution of patient characteristics may differ greatly by the center, some form of covariate adjustment is generally necessary in order for center-specific outcomes to be accurately compared (to each other or to an overall average). We propose a weighting method for comparing facility-specific cumulative incidence functions to an overall average. The method directly standardizes each facility's non-parametric cumulative incidence function through a weight function constructed from a multivariate prognostic score. We formally define the center effects and derive large-sample properties of the proposed estimator. We evaluate the finite sample performance of the estimator through simulation. The proposed method is applied to the end-stage renal disease setting to evaluate the center-specific pre-transplant mortality and transplant cumulative incidence functions from the Scientific Registry of Transplant Recipients.

Hospital-specific Template Matching for Benchmarking Performance in a Diverse Multihospital System

BACKGROUND

Hospital-specific template matching is a newer method of hospital performance measurement that may be fairer than regression-based benchmarking. However, it has been tested in only limited research settings.

OBJECTIVE

The objective of this study was to test the feasibility of hospital-specific template matching assessments in the Veterans Affairs (VA) health care system and determine power to detect greater-than-expected 30-day mortality.

RESEARCH DESIGN

Observational cohort study with hospital-specific template matching assessment. For each VA hospital, the 30-day mortality of a representative subset of hospitalizations was compared with the pooled mortality from matched hospitalizations at a set of comparison VA hospitals treating sufficiently similar patients. The simulation was used to determine power to detect greater-than-expected mortality.

SUBJECTS

A total of 556,266 hospitalizations at 122 VA hospitals in 2017.

MEASURES

A number of comparison hospitals identified per hospital; 30-day mortality.

RESULTS

Each hospital had a median of 38 comparison hospitals (interquartile range: 33, 44) identified, and 116 (95.1%) had at least 20 comparison hospitals. In total, 8 hospitals (6.6%) had a significantly lower 30-day mortality than their benchmark, 5 hospitals (4.1%) had a significantly higher 30-day mortality, and the remaining 109 hospitals (89.3%) were similar to their benchmark. Power to detect a standardized mortality ratio of 2.0 ranged from 72.5% to 79.4% for a hospital with the fewest (6) versus most (64) comparison hospitals.

CONCLUSIONS

Hospital-specific template matching may be feasible for assessing hospital performance in the diverse VA health care system, but further refinements are needed to optimize the approach before operational use. Our findings are likely applicable to other large and diverse multihospital systems.

Evaluation of Reliability and Correlations of Quality Measures in Cancer Care

IMPORTANCE

Measurement of the quality of care is important for alternative payment models in oncology, yet the ability to distinguish high-quality from low-quality care across oncology practices remains uncertain.

OBJECTIVE

To assess the reliability of cancer care quality measures across oncology practices using registry and claims-based measures of process, utilization, end-of-life (EOL) care, and survival, and to assess the correlations of practice-level performance across measure and cancer types.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study used the Surveillance, Epidemiology, and End Results (SEER) Program registry linked to Medicare administrative data to identify individuals with lung cancer, breast cancer, or colorectal cancer (CRC) that was newly diagnosed between January 1, 2011, and December 31, 2015, and who were treated in oncology practices with 20 or more patients. Data were analyzed from January 2018 to December 2020.

MAIN OUTCOMES AND MEASURES

Receipt of guideline-recommended treatment and surveillance, hospitalizations or emergency department visits during 6-month chemotherapy episodes, care intensity in the last month of life, and 12-month survival were measured. Summary measures for each domain in each cohort were calculated. Practice-level rates for each measure were estimated from hierarchical linear models with practice-level random effects; practice-level reliability (reproducibility) for each measure based on the between-measure variance, within-measure variance, and distribution of patients treated in each practice; and correlations of measures across measure and cancer types.

RESULTS

In this study of SEER registry data linked to Medicare administrative data from 49 715 patients with lung cancer treated in 502 oncology practices, 21 692 with CRC treated in 347 practices, and 52 901 with breast cancer treated in 492 practices, few practices had 20 or more patients who were eligible for most process measures during the 5-year study period. Patients were 65 years or older; approximately 50% of the patients with lung cancer and CRC and all of the patients with breast cancer were women. Most measures had limited variability across practices. Among process measures, 0 of 6 for lung cancer, 0 of 6 for CRC, and 3 of 11 for breast cancer had a practice-level reliability of 0.75 or higher for the median-sized practice. No utilization, EOL care, or survival measure had reliability across practices of 0.75 or higher. Correlations across measure types were low (r ≤ 0.20 for all) except for a correlation between the CRC process and 1-year survival summary measures (r = 0.35; P < .001). Summary process measures had limited or no correlation across lung cancer, breast cancer, and CRC (r ≤ 0.16 for all).

CONCLUSIONS AND RELEVANCE

This study found that quality measures were limited by the small numbers of Medicare patients with newly diagnosed cancer treated in oncology practices, even after pooling 5 years of data. Measures had low reliability and had limited to no correlation across measure and cancer types, suggesting the need for research to identify reliable quality measures for practice-level quality assessments.

Developing a template matching algorithm for benchmarking hospital performance in a diverse, integrated healthcare system

Template matching is a proposed approach for hospital benchmarking, which measures performance based on matching a subset of comparable patient hospitalizations from each hospital. We assessed the ability to create the required matched samples and thus the feasibility of template matching to benchmark hospital performance in a diverse healthcare system.Nationwide Veterans Affairs (VA) hospitals, 2017.Observational cohort study.We used administrative and clinical data from 668,592 hospitalizations at 134 VA hospitals in 2017. A standardized template of 300 hospitalizations was selected, and then 300 hospitalizations were matched to the template from each hospital.There was substantial case-mix variation across VA hospitals, which persisted after excluding small hospitals, hospitals with primarily psychiatric admissions, and hospitalizations for rare diagnoses. Median age ranged from 57 to 75 years across hospitals; percent surgical admissions ranged from 0.0% to 21.0%; percent of admissions through the emergency department, 0.1% to 98.7%; and percent Hispanic patients, 0.2% to 93.3%. Characteristics for which there was substantial variation across hospitals could not be balanced with any matching algorithm tested. Although most other variables could be balanced, we were unable to identify a matching algorithm that balanced more than ∼20 variables simultaneously.We were unable to identify a template matching approach that could balance hospitals on all measured characteristics potentially important to benchmarking. Given the magnitude of case-mix variation across VA hospitals, a single template is likely not feasible for general hospital benchmarking.

Comparing Outcomes and Costs of Surgical Patients Treated at Major Teaching and Nonteaching Hospitals: A National Matched Analysis

OBJECTIVE

To compare outcomes and costs between major teaching and nonteaching hospitals on a national scale by closely matching on patient procedures and characteristics.

BACKGROUND

Teaching hospitals have been shown to often have better quality than nonteaching hospitals, but cost and value associated with teaching hospitals remains unclear.

METHODS

A study of Medicare patients at 340 teaching hospitals (resident-to-bed ratios ≥ 0.25) and matched patient controls from 2444 nonteaching hospitals (resident-to-bed ratios < 0.05).We studied 86,751 pairs admitted for general surgery (GS), 214,302 pairs of patients admitted for orthopedic surgery, and 52,025 pairs of patients admitted for vascular surgery.

RESULTS

In GS, mortality was 4.62% in teaching hospitals versus 5.57%, (a difference of -0.95%, <0.0001), and overall paired cost difference = $915 (P < 0.0001). For the GS quintile of pairs with highest risk on admission, mortality differences were larger (15.94% versus 18.18%, difference = -2.24%, P < 0.0001), and paired cost difference = $3773 (P < 0.0001), yielding $1682 per 1% mortality improvement at 30 days. Patterns for vascular surgery outcomes resembled general surgery; however, orthopedics outcomes did not show significant differences in mortality across teaching and nonteaching environments, though costs were higher at teaching hospitals.

CONCLUSIONS

Among Medicare patients, as admission risk of mortality increased, the absolute mortality benefit of treatment at teaching hospitals also increased, though accompanied by marginally higher cost. Major teaching hospitals appear to return good value for the extra resources used in general surgery, and to some extent vascular surgery, but this was not apparent in orthopedic surgery.

Comparing Resource Use in Medical Admissions of Children With Complex Chronic Conditions

BACKGROUND

Children with complex chronic conditions (CCCs) utilize a disproportionate share of hospital resources.

OBJECTIVE

We asked whether some hospitals display a significantly different pattern of resource utilization than others when caring for similar children with CCCs admitted for medical diagnoses.

RESEARCH DESIGN

Using Pediatric Health Information System data from 2009 to 2013, we constructed an inpatient Template of 300 children with CCCs, matching these to 300 patients at each hospital, thereby performing a type of direct standardization.

SUBJECTS

Children with CCCs were drawn from a list of the 40 most common medical principal diagnoses, then matched to patients across 40 Children's Hospitals.

MEASURES

Rate of intensive care unit admission, length of stay, resource cost.

RESULTS

For the Template-matched patients, when comparing resource use at the lower 12.5-percentile and upper 87.5-percentile of hospitals, we found: intensive care unit utilization was 111% higher (6.6% vs. 13.9%, P<0.001); hospital length of stay was 25% higher (2.4 vs. 3.0 d/admission, P<0.001); and finally, total cost per patient varied by 47% ($6856 vs. $10,047, P<0.001). Furthermore, some hospitals, compared with their peers, were more efficient with low-risk patients and less efficient with high-risk patients, whereas other hospitals displayed the opposite pattern.

CONCLUSIONS

Hospitals treating similar patients with CCCs admitted for similar medical diagnoses, varied greatly in resource utilization. Template Matching can aid chief quality officers benchmarking their hospitals to peer institutions and can help determine types of their patients having the most aberrant outcomes, facilitating quality initiatives to target these patients.

A method to reduce imbalance for site-level randomized stepped wedge implementation trial designs

BACKGROUND

Controlled implementation trials often randomize the intervention at the site level, enrolling relatively few sites (e.g., 6-20) compared to trials that randomize by subject. Trials with few sites carry a substantial risk of an imbalance between intervened (cases) and non-intervened (control) sites in important site characteristics, thereby threatening the internal validity of the primary comparison. A stepped wedge design (SWD) staggers the intervention at sites over a sequence of times or time waves until all sites eventually receive the intervention. We propose a new randomization method, sequential balance, to control time trend in site allocation by minimizing sequential imbalance across multiple characteristics. We illustrate the new method by applying it to a SWD implementation trial.

METHODS

The trial investigated the impact of blended internal-external facilitation on the establishment of evidence-based teams in general mental health clinics in nine US Department of Veterans Affairs medical centers. Prior to randomization to start time, an expert panel of implementation researchers and health system program leaders identified by consensus a series of eight facility-level characteristics judged relevant to the success of implementation. We characterized each of the nine sites according to these consensus features. Using a weighted sum of these characteristics, we calculated imbalance scores for each of 1680 possible site assignments to identify the most sequentially balanced assignment schemes.

RESULTS

From 1680 possible site assignments, we identified 34 assignments with minimal imbalance scores, and then randomly selected one assignment by which to randomize start time. Initially, the mean imbalance score was 3.10, but restricted to the 34 assignments, it declined to 0.99.

CONCLUSIONS

Sequential balancing of site characteristics across groups of sites in the time waves of a SWD strengthens the internal validity of study conclusions by minimizing potential confounding.

TRIAL REGISTRATION

Registered at ClinicalTrials.gov as clinical trials # NCT02543840 ; entered 9/4/2015.

Template matching for benchmarking hospital performance in the veterans affairs healthcare system

Comparing hospital performance in a health system is traditionally done with multilevel regression models that adjust for differences in hospitals' patient case-mix. In contrast, "template matching" compares outcomes of similar patients at different hospitals but has been used only in limited patient settings.Our objective was to test a basic template matching approach in the nationwide Veterans Affairs healthcare system (VA), compared with a more standard regression approach.We performed various simulations using observational data from VA electronic health records whereby we randomly assigned patients to "pseudo hospitals," eliminating true hospital level effects. We randomly selected a representative template of 240 patients and matched 240 patients on demographic and physiological factors from each pseudo hospital to the template. We varied hospital performance for different simulations such that some pseudo hospitals negatively impacted patient mortality.Electronic health record data of 460,213 hospitalizations at 111 VA hospitals across the United States in 2015.We assessed 30-day mortality at each pseudo hospital and identified lowest quintile hospitals by template matching and regression. The regression model adjusted for predicted 30-day mortality (as a measure of illness severity).Regression identified the lowest quintile hospitals with 100% accuracy compared with 80.3% to 82.0% for template matching when systematic differences in 30-day mortality existed.The current standard practice of risk-adjusted regression incorporating patient-level illness severity was better able to identify lower-performing hospitals than the simplistic template matching algorithm.

Practice Style Variation in Medicaid and Non-Medicaid Children With Complex Chronic Conditions Undergoing Surgery

OBJECTIVES

With differential payment between Medicaid and Non-Medicaid services, we asked whether style-of-practice differs between similar Medicaid and Non-Medicaid children with complex chronic conditions (CCCs) undergoing surgery.

SUMMARY OF BACKGROUND DATA

Surgery in children with CCCs accounts for a disproportionately large percentage of resource utilization at major children's hospitals.

METHODS

A matched cohort design, studying 23,582 pairs of children with CCCs undergoing surgery (Medicaid matched to Non-Medicaid within the same hospital) from 2009 to 2013 in 41 Children's Hospitals. Patients were matched on age, sex, principal procedure, CCCs, and other characteristics.

RESULTS

Median cost in Medicaid patients was $21,547 versus $20,527 in Non-Medicaid patients (5.0% higher, P < 0.001). Median paired difference in cost (Medicaid minus Non-Medicaid) was $320 [95% confidence interval (CI): $208, $445], (1.6% higher, P < 0.001). 90th percentile costs were $133,640 versus $127,523, (4.8% higher, P < 0.001). Mean paired difference in length of stay (LOS) was 0.50 days (95% CI: 0.36, 0.65), (P < 0.001). ICU utilization was 2.8% higher (36.7% vs 35.7%, P < 0.001). Finally, in-hospital mortality pooled across all pairs was higher in Medicaid patients (0.38% vs 0.22%, P = 0.002). After adjusting for multiple testing, no individual hospital displayed significant differences in cost between groups, only 1 hospital displayed significant differences in LOS and 1 in ICU utilization.

CONCLUSIONS

Treatment style differences between Medicaid and Non-Medicaid children were small, suggesting little disparity with in-hospital surgical care for patients with CCCs operated on within Children's Hospitals. However, in-hospital mortality, although rare, was slightly higher in Medicaid patients and merits further investigation.

A Mixed Methods Study of Tele-ICU Nursing Interventions to Prevent Failure to Rescue of Patients in Critical Care

Background: Failure to rescue (FTR) is a benchmark of quality care. Limited evidence exists examining the influence of telemedicine intensive care units (tele-ICU) nursing interventions in preventing FTR. The purpose of this study was to characterize tele-ICU nursing interventions and to determine which combination of documented tele-ICU nursing interventions (DTNI) best predicts prevention of FTR in ICU patients with hospital-acquired conditions (HACs). Materials and Methods: We used convergent parallel mixed methods design to conduct qualitative interviews with a purposive sample of tele-ICU nurses (n = 19) from 11 US tele-ICU centers. Quantitative data, including demographics, DTNIs, severity of illness scores, and video assessment times from January 2016 to December 2016 were retrieved for ICU patients discharged from a multihospital health system with a tele-ICU center (n = 861). Findings from both qualitative and quantitative analyses were merged, compared, and contrasted. Results: FTR patients had higher severity of illness, longer video assessment by tele-ICU nurses, and were more likely to have DTNIs related to hemodynamic instability. Four themes emerged from qualitative analysis: fundamental tele-ICU nurse attributes, proactive clinical practice, effective collaborative relationships, and strategic use of advanced technology. Mixed methods analysis revealed convergence between DTNIs and tele-ICU nurses' characterizations of their practice. Conclusions: Tele-ICU nurses' characterizations of their practice closely align with DTNIs. Tele-ICU nursing practice to prevent FTR involves systems thinking and integration of many complex factors. Tele-ICU nurses can reduce the odds of FTR with focus on support and clinical coordination interventions that avoid hemodynamic instability in ICU patients with a diagnosed HAC.

Incorporating Longitudinal Comorbidity and Acute Physiology Data in Template Matching for Assessing Hospital Quality: An Exploratory Study in an Integrated Health Care Delivery System

OBJECTIVE

We sought to build on the template-matching methodology by incorporating longitudinal comorbidities and acute physiology to audit hospital quality.

STUDY SETTING

Patients admitted for sepsis and pneumonia, congestive heart failure, hip fracture, and cancer between January 2010 and November 2011 at 18 Kaiser Permanente Northern California hospitals.

STUDY DESIGN

We generated a representative template of 250 patients in 4 diagnosis groups. We then matched between 1 and 5 patients at each hospital to this template using varying levels of patient information.

DATA COLLECTION

Data were collected retrospectively from inpatient and outpatient electronic records.

PRINCIPAL FINDINGS

Matching on both present-on-admission comorbidity history and physiological data significantly reduced the variation across hospitals in patient severity of illness levels compared with matching on administrative data only. After adjustment for longitudinal comorbidity and acute physiology, hospital rankings on 30-day mortality and estimates of length of stay were statistically different from rankings based on administrative data.

CONCLUSIONS

Template matching-based approaches to hospital quality assessment can be enhanced using more granular electronic medical record data.

Comparison of the Value of Nursing Work Environments in Hospitals Across Different Levels of Patient Risk

IMPORTANCE

The literature suggests that hospitals with better nursing work environments provide better quality of care. Less is known about value (cost vs quality).

OBJECTIVES

To test whether hospitals with better nursing work environments displayed better value than those with worse nursing environments and to determine patient risk groups associated with the greatest value.

DESIGN, SETTING, AND PARTICIPANTS

A retrospective matched-cohort design, comparing the outcomes and cost of patients at focal hospitals recognized nationally as having good nurse working environments and nurse-to-bed ratios of 1 or greater with patients at control group hospitals without such recognition and with nurse-to-bed ratios less than 1. This study included 25 752 elderly Medicare general surgery patients treated at focal hospitals and 62 882 patients treated at control hospitals during 2004-2006 in Illinois, New York, and Texas. The study was conducted between January 1, 2004, and November 30, 2006; this analysis was conducted from April to August 2015.

EXPOSURES

Focal vs control hospitals (better vs worse nursing environment).

MAIN OUTCOMES AND MEASURES

Thirty-day mortality and costs reflecting resource utilization.

RESULTS

This study was conducted at 35 focal hospitals (mean nurse-to-bed ratio, 1.51) and 293 control hospitals (mean nurse-to-bed ratio, 0.69). Focal hospitals were larger and more teaching and technology intensive than control hospitals. Thirty-day mortality in focal hospitals was 4.8% vs 5.8% in control hospitals (P < .001), while the cost per patient was similar: the focal-control was -$163 (95% CI = -$542 to $215; P = .40), suggesting better value in the focal group. For the focal vs control hospitals, the greatest mortality benefit (17.3% vs 19.9%; P < .001) occurred in patients in the highest risk quintile, with a nonsignificant cost difference of $941 per patient ($53 701 vs $52 760; P = .25). The greatest difference in value between focal and control hospitals appeared in patients in the second-highest risk quintile, with mortality of 4.2% vs 5.8% (P < .001), with a nonsignificant cost difference of -$862 ($33 513 vs $34 375; P = .12).

CONCLUSIONS AND RELEVANCE

Hospitals with better nursing environments and above-average staffing levels were associated with better value (lower mortality with similar costs) compared with hospitals without nursing environment recognition and with below-average staffing, especially for higher-risk patients. These results do not suggest that improving any specific hospital's nursing environment will necessarily improve its value, but they do show that patients undergoing general surgery at hospitals with better nursing environments generally receive care of higher value.

Comparing Outcomes between Antibody Induction Therapies in Kidney Transplantation

Kidney transplant recipients often receive antibody induction. Previous studies of induction therapy were often limited by short follow-up and/or absence of information about complications. After linking Organ Procurement and Transplantation Network data with Medicare claims, we compared outcomes between three induction therapies for kidney recipients. Using novel matching techniques developed on the basis of 15 clinical and demographic characteristics, we generated 1:1 pairs of alemtuzumab-rabbit antithymocyte globulin (rATG) (5330 pairs) and basiliximab-rATG (9378 pairs) recipients. We used paired Cox regression to analyze the primary outcomes of death and death or allograft failure. Secondary outcomes included death or sepsis, death or lymphoma, death or melanoma, and healthcare resource utilization within 1 year. Compared with rATG recipients, alemtuzumab recipients had higher risk of death (hazard ratio [HR], 1.14; 95% confidence interval [95% CI], 1.03 to 1.26; P<0.01) and death or allograft failure (HR, 1.18; 95% CI, 1.09 to 1.28; P<0.001). Results for death as well as death or allograft failure were generally consistent among elderly and nonelderly subgroups and among pairs receiving oral prednisone. Compared with rATG recipients, basiliximab recipients had higher risk of death (HR, 1.08; 95% CI, 1.01 to 1.16; P=0.03) and death or lymphoma (HR, 1.12; 95% CI, 1.01 to 1.23; P=0.03), although these differences were not confirmed in subgroup analyses. One-year resource utilization was slightly lower among alemtuzumab recipients than among rATG recipients, but did not differ between basiliximab and rATG recipients. This observational evidence indicates that, compared with alemtuzumab and basiliximab, rATG associates with lower risk of adverse outcomes, including mortality.

Outcomes, ICU Use, and Length of Stay in Chronically Ill Black and White Children on Medicaid and Hospitalized for Surgery

BACKGROUND

With increasing Medicaid coverage, it has become especially important to determine whether racial differences exist within the Medicaid system. We asked whether disparities exist in hospital practice and patient outcomes between matched black and white Medicaid children with chronic conditions undergoing surgery.

STUDY DESIGN

We conducted a matched cohort study, matching 6,398 pairs within states on detailed patient characteristics using data from 25 states contributing adequate Medicaid Analytic eXtract claims for admissions of children with chronic conditions undergoing the same surgical procedures between January 1, 2009 and November 30, 2010 for ages 1 to 18 years.

RESULTS

The black patient 30-day revisit rate was 19.3% vs 19.8% in matched white patients (p = 0.61), 30-day readmission rates were 7.0% vs 6.9% (p = 0.43), and 30-day mortality rates were 0.38% vs 0.19% (p = 0.06), respectively. A higher percentage of black patients exceeded their own state's individual median length of stay (44.0% vs 39.6%; p < 0.001) and median ICU length of stay (25.9% vs 23.8%; p < 0.001). Intensive care unit use was higher in black patients (25.9% vs 23.8%; p < 0.001). After adjusting for multiple testing, only 2 states were found to differ significantly by race (New York for length of stay and New Jersey for ICU use).

CONCLUSIONS

We did not observe disparities in 30-day revisits and readmissions for chronically ill children in Medicaid undergoing surgery, and only slight differences in length of stay, ICU length of stay, and use of the ICU, where blacks displayed somewhat elevated rates compared with white controls.

Racial Disparities in Medicaid Asthma Hospitalizations

BACKGROUND AND OBJECTIVES

Black children with asthma comprise one-third of all asthma patients in Medicaid. With increasing Medicaid coverage, it has become especially important to monitor Medicaid for differences in hospital practice and patient outcomes by race.

METHODS

A multivariate matched cohort design, studying 11 079 matched pairs of children in Medicaid (black versus white matched pairs from inside the same state) admitted for asthma between January 1, 2009 and November 30, 2010 in 33 states contributing adequate Medicaid Analytic eXtract claims.

RESULTS

Ten-day revisit rates were 3.8% in black patients versus 4.2% in white patients (P = .12); 30-day revisit and readmission rates were also not significantly different by race (10.5% in black patients versus 10.8% in white patients; P = .49). Length of stay (LOS) was also similar; both groups had a median stay of 2.0 days, with a slightly lower percentage of black patients exceeding their own state's median LOS (30.2% in black patients versus 31.8% in white patients; P = .01). The mean paired difference in LOS was 0.00 days (95% confidence interval, -0.08 to 0.08). However, ICU use was higher in black patients than white patients (22.2% versus 17.5%; P < .001). After adjusting for multiple testing, only 4 states were found to differ significantly, but only in ICU use, where blacks had higher rates of use.

CONCLUSIONS

For closely matched black and white patients, racial disparities concerning asthma admission outcomes and style of practice are small and generally nonsignificant, except for ICU use, where we observed higher rates in black patients.

Mental Health Conditions and Medical and Surgical Hospital Utilization

OBJECTIVE

Mental health conditions are prevalent among children hospitalized for medical conditions and surgical procedures, but little is known about their influence on hospital resource use. The objectives of this study were to examine how hospitalization characteristics vary by presence of a comorbid mental health condition and estimate the association of a comorbid mental health condition with hospital length of stay (LOS) and costs.

METHODS

Using the 2012 Kids' Inpatient Database, we conducted a retrospective, nationally representative, cross-sectional study of 670 161 hospitalizations for 10 common medical and 10 common surgical conditions among 3- to 20-year-old patients. Associations between mental health conditions and hospital LOS were examined using adjusted generalized linear models. Costs of additional hospital days associated with mental health conditions were estimated using hospital cost-to-charge ratios.

RESULTS

A comorbid mental health condition was present in 13.2% of hospitalizations. A comorbid mental health condition was associated with a LOS increase of 8.8% (from 2.5 to 2.7 days, P < .001) for medical hospitalizations and a 16.9% increase (from 3.6 to 4.2 days, P < .001) for surgical hospitalizations. For hospitalizations in this sample, comorbid mental health conditions were associated with an additional 31 729 (95% confidence interval: 29 085 to 33 492) hospital days and $90 million (95% confidence interval: $81 to $101 million) in hospital costs.

CONCLUSIONS

Medical and surgical hospitalizations with comorbid mental health conditions were associated with longer hospital stay and higher hospital costs. Knowledge about the influence of mental health conditions on pediatric hospital utilization can inform clinical innovation and case-mix adjustment.

Practice Patterns in Medicaid and Non-Medicaid Asthma Admissions

BACKGROUND AND OBJECTIVES

With American children experiencing increased Medicaid coverage, it has become especially important to determine if practice patterns differ between Medicaid and non-Medicaid patients. Auditing such potential differences must carefully compare like patients to avoid falsely identifying suspicious practice patterns. We asked if we could observe differences in practice patterns between Medicaid and non-Medicaid patients admitted for asthma inside major children's hospitals.

METHODS

A matched cohort design, studying 17 739 matched pairs of children (Medicaid to non-Medicaid) admitted for asthma in the same hospital between April 1, 2011 and March 31, 2014 in 40 Children's Hospital Association hospitals contributing data to the Pediatric Hospital Information System database. Patients were matched on age, sex, asthma severity, and other patient characteristics.

RESULTS

Medicaid patient median cost was $4263 versus $4160 for non-Medicaid patients (P < .001). Additionally, the median cost difference (Medicaid minus non-Medicaid) between individual pairs was only $84 (95% confidence interval: 44 to 124), and the mean cost difference was only $49 (95% confidence interval: -72 to 170). The 90th percentile costs were also similar between groups ($10 710 vs $10 948; P < .07). Length of stay (LOS) was also very similar; both groups had a median stay of 1 day, with a similar percentage of patients exceeding the 90th percentile of individual hospital LOS (7.1% vs 6.7%; P = .14). ICU use was also similar (10.1% vs 10.6%; P = .12).

CONCLUSIONS

For closely matched patients within the same hospital, Medicaid status did not importantly influence costs, LOS, or ICU use.

Improving Medicare's Hospital Compare Mortality Model

OBJECTIVE

To improve the predictions provided by Medicare's Hospital Compare (HC) to facilitate better informed decisions regarding hospital choice by the public.

DATA SOURCES/SETTING

Medicare claims on all patients admitted for Acute Myocardial Infarction between 2009 through 2011.

STUDY DESIGN

Cohort analysis using a Bayesian approach, comparing the present assumptions of HC (using a constant mean and constant variance for all hospital random effects), versus an expanded model that allows for the inclusion of hospital characteristics to permit the data to determine whether they vary with attributes of hospitals, such as volume, capabilities, and staffing. Hospital predictions are then created using directly standardized estimates to facilitate comparisons between hospitals.

DATA COLLECTION/EXTRACTION METHODS

Medicare fee-for-service claims.

PRINCIPAL FINDINGS

Our model that included hospital characteristics produces very different predictions from the current HC model, with higher predicted mortality rates at hospitals with lower volume and worse characteristics. Using Chicago as an example, the expanded model would advise patients against seeking treatment at the smallest hospitals with worse technology and staffing.

CONCLUSION

To aid patients when selecting between hospitals, the Centers for Medicare and Medicaid Services (CMS) should improve the HC model by permitting its predictions to vary systematically with hospital attributes such as volume, capabilities, and staffing.

Indirect Standardization Matching: Assessing Specific Advantage and Risk Synergy

OBJECTIVE

To develop a method to allow a hospital to compare its performance using its entire patient population to the outcomes of very similar patients treated elsewhere.

DATA SOURCES/SETTING

Medicare claims in orthopedics and common general, gynecologic, and urologic surgery from Illinois, New York, and Texas from 2004 to 2006.

STUDY DESIGN

Using two example "focal" hospitals, each hospital's patients were matched to 10 very similar patients selected from 619 other hospitals.

DATA COLLECTION/EXTRACTION METHODS

All patients were used at each focal hospital, and we found the 10 closest matched patients from control hospitals with exactly the same principal procedure as each focal patient.

PRINCIPAL FINDINGS

We achieved exact matches on all procedures and very close matches for other patient characteristics for both hospitals. There were few to no differences between each hospital's patients and their matched control patients on most patient characteristics, yet large and significant differences were observed for mortality, failure-to-rescue, and cost.

CONCLUSION

Indirect standardization matching can produce fair audits of quality and cost, allowing for a comprehensive, transparent, and relevant assessment of all patients at a focal hospital. With this approach, hospitals will be better able to benchmark their performance and determine where quality improvement is most needed.

Examining Causes of Racial Disparities in General Surgical Mortality: Hospital Quality Versus Patient Risk

BACKGROUND

Racial disparities in general surgical outcomes are known to exist but not well understood.

OBJECTIVES

To determine if black-white disparities in general surgery mortality for Medicare patients are attributable to poorer health status among blacks on admission or differences in the quality of care provided by the admitting hospitals.

RESEARCH DESIGN

Matched cohort study using Tapered Multivariate Matching.

SUBJECTS

All black elderly Medicare general surgical patients (N=18,861) and white-matched controls within the same 6 states or within the same 838 hospitals.

MEASURES

Thirty-day mortality (primary); others include in-hospital mortality, failure-to-rescue, complications, length of stay, and readmissions.

RESULTS

Matching on age, sex, year, state, and the exact same procedure, blacks had higher 30-day mortality (4.0% vs. 3.5%, P<0.01), in-hospital mortality (3.9% vs. 2.9%, P<0.0001), in-hospital complications (64.3% vs. 56.8% P<0.0001), and failure-to-rescue rates (6.1% vs. 5.1%, P<0.001), longer length of stay (7.2 vs. 5.8 d, P<0.0001), and more 30-day readmissions (15.0% vs. 12.5%, P<0.0001). Adding preoperative risk factors to the above match, there was no significant difference in mortality or failure-to-rescue, and all other outcome differences were small. Blacks matched to whites in the same hospital displayed no significant differences in mortality, failure-to-rescue, or readmissions.

CONCLUSIONS

Black and white Medicare patients undergoing the same procedures with closely matched risk factors displayed similar mortality, suggesting that racial disparities in general surgical mortality are not because of differences in hospital quality. To reduce the observed disparities in surgical outcomes, the poorer health of blacks on presentation for surgery must be addressed.

A hospital-specific template for benchmarking its cost and quality

OBJECTIVE

Develop an improved method for auditing hospital cost and quality tailored to a specific hospital's patient population.

DATA SOURCES/SETTING

Medicare claims in general, gynecologic and urologic surgery, and orthopedics from Illinois, New York, and Texas between 2004 and 2006.

STUDY DESIGN

A template of 300 representative patients from a single index hospital was constructed and used to match 300 patients at 43 hospitals that had a minimum of 500 patients over a 3-year study period.

DATA COLLECTION/EXTRACTION METHODS

From each of 43 hospitals we chose 300 patients most resembling the template using multivariate matching.

PRINCIPAL FINDINGS

We found close matches on procedures and patient characteristics, far more balanced than would be expected in a randomized trial. There were little to no differences between the index hospital's template and the 43 hospitals on most patient characteristics yet large and significant differences in mortality, failure-to-rescue, and cost.

CONCLUSION

Matching can produce fair, directly standardized audits. From the perspective of the index hospital, "hospital-specific" template matching provides the fairness of direct standardization with the specific institutional relevance of indirect standardization. Using this approach, hospitals will be better able to examine their performance, and better determine why they are achieving the results they observe.

Concept and measurement of pediatric value

In the new health care marketplace, families will be making important decisions concerning choice of health plan, health provider, and even accountable care organizations. Ideally, they would make these decisions using information on health care value, which comprises the relationships between patient/family-centered outcomes (the outputs of health care services) and costs of providing care to achieve these outcomes. Providing information on pediatric value will require new investments in data collection systems that include outcomes that matter to children and families and costs measured at the level of the child. The analysis of these data must account for the perspective of the user of the information. In the case of families, direct standardization should be used to contrast care in one health care system with another according to the unique characteristics of each family and child.

Ensemble of trees approaches to risk adjustment for evaluating a hospital's performance

A commonly used method for evaluating a hospital's performance on an outcome is to compare the hospital's observed outcome rate to the hospital's expected outcome rate given its patient (case) mix and service. The process of calculating the hospital's expected outcome rate given its patient mix and service is called risk adjustment (Iezzoni 1997). Risk adjustment is critical for accurately evaluating and comparing hospitals' performances since we would not want to unfairly penalize a hospital just because it treats sicker patients. The key to risk adjustment is accurately estimating the probability of an Outcome given patient characteristics. For cases with binary outcomes, the method that is commonly used in risk adjustment is logistic regression. In this paper, we consider ensemble of trees methods as alternatives for risk adjustment, including random forests and Bayesian additive regression trees (BART). Both random forests and BART are modern machine learning methods that have been shown recently to have excellent performance for prediction of outcomes in many settings. We apply these methods to carry out risk adjustment for the performance of neonatal intensive care units (NICU). We show that these ensemble of trees methods outperform logistic regression in predicting mortality among babies treated in NICU, and provide a superior method of risk adjustment compared to logistic regression.

Template matching for auditing hospital cost and quality

OBJECTIVE

Develop an improved method for auditing hospital cost and quality.

DATA SOURCES/SETTING

Medicare claims in general, gynecologic and urologic surgery, and orthopedics from Illinois, Texas, and New York between 2004 and 2006.

STUDY DESIGN

A template of 300 representative patients was constructed and then used to match 300 patients at hospitals that had a minimum of 500 patients over a 3-year study period.

DATA COLLECTION/EXTRACTION METHODS

From each of 217 hospitals we chose 300 patients most resembling the template using multivariate matching.

PRINCIPAL FINDINGS

The matching algorithm found close matches on procedures and patient characteristics, far more balanced than measured covariates would be in a randomized clinical trial. These matched samples displayed little to no differences across hospitals in common patient characteristics yet found large and statistically significant hospital variation in mortality, complications, failure-to-rescue, readmissions, length of stay, ICU days, cost, and surgical procedure length. Similar patients at different hospitals had substantially different outcomes.

CONCLUSION

The template-matched sample can produce fair, directly standardized audits that evaluate hospitals on patients with similar characteristics, thereby making benchmarking more believable. Through examining matched samples of individual patients, administrators can better detect poor performance at their hospitals and better understand why these problems are occurring.