Spatial risk adjustment between health insurances: using GWR in risk adjustment models to conserve incentives for service optimisation and reduce MAUP

This paper presents a new approach to deal with spatial inequalities in risk adjustment between health insurances. The shortcomings of non-spatial and spatial fixed effects in risk adjustment models are analysed and opposed against spatial kernel estimators. Theoretical and empirical evidence suggests that a reasonable choice of the spatial kernel could limit the spatial uncertainty of the modifiable area unit problem under heavy-tailed claims data, leading to more precise predictions and economically positive incentives on the healthcare market. A case study of the German risk adjustment shows a spatial risk spread of 86 Euro p.c., leading to incentives for spatial risk selection. The proposed estimator eliminates this issue and conserves incentives for services optimisation.

Expenditure variations analysis using residuals for identifying high health care utilizers in a state Medicaid program

Background

High utilizers receive great attention in health care research because they have a largely disproportionate spending. Existing analyses usually identify high utilizers with an empirical threshold on the number of health care visits or associated expenditures. However, such count-and-cost based criteria might not be best for identifying impactable high utilizers.

Methods

We propose an approach to identify impactable high utilizers using residuals from regression-based health care utilization risk adjustment models to analyze the variations in health care expenditures. We develop linear and tree-based models to best adjust per-member per-month health care cost by clinical and socioeconomic risk factors using a large administrative claims dataset from a state public insurance program.

Results

The risk adjustment models identify a group of patients with high residuals whose demographics and categorization of comorbidities are similar to other patients but who have a significant amount of unexplained health care utilization. Deeper analysis of the essential hypertension cohort and chronic kidney disease cohort shows these variations in expenditures could be within individual ICD-9-CM codes and from different mixtures of ICD-9-CM codes. Additionally, correlation analysis with 3M™ Potentially Preventable Events (PPE) software shows that a portion of this utilization may be preventable. In addition, the high utilizers persist from year to year.

Conclusions

After risk adjustment, patients with higher than expected expenditures (high residuals) are associated with more potentially preventable events. These residuals are temporally consistent and hence may be useful in identifying and intervening impactable high utilizers.

Data transformations to improve the performance of health plan payment methods

The conventional method for developing health care plan payment systems uses observed data to study alternative algorithms and set incentives for the health care system. In this paper, we take a different approach and transform the input data rather than the algorithm, so that the data used reflect the desired spending levels rather than the observed spending levels. We present a general economic model that incorporates the previously overlooked two-way relationship between health plan payment and insurer actions. We then demonstrate our systematic approach for data transformations in two Medicare applications: underprovision of care for individuals with chronic illnesses and health care disparities by geographic income levels. Empirically comparing our method to two other common approaches shows that the "side effects" of these approaches vary by context, and that data transformation is an effective tool for addressing misallocations in individual health insurance markets.

Control of MSSA and MRSA in the United States: protocols, policies, risk adjustment and excuses

Data released by the U.S. Centers for Disease Control and Prevention (CDC) on March 5, 2019 showed that Staph aureus infections are a major problem in the United States, with 119,000 infections and almost 20,000 deaths in 2017. Rates of decline for hospital-onset MRSA have slowed since 2012 and the United States is not on track for meeting the 2015 U.S. Dept. of Health and Human Services’ goal of a 50% reduction by 2020. There is a need for improved standards for control of dangerous pathogens. Currently, the World Health Organization’s recommendation of preoperatively screening patients for Staph aureus has not become a standard of care in the United States.

The U.S. Veterans Health Administration also released data which found a much larger decrease in hospital-onset MRSA infections as opposed to hospital-onset MSSA using various infectious disease bundles that all included universal MRSA surveillance and isolation for MRSA carriers. These results mirror the results obtained by the United Kingdom’s National Health Service. These findings support the contention that the marked decline in hospital-onset MRSA infections observed in these studies is due to interventions which are specifically targeted towards MRSA.

A case is made that concerns with the integrity of healthcare policy research, along with industrial conflicts-of-interest have inhibited effective formulation of infectious disease policy in the United States. Because MRSA has become endemic in the general U.S. population (approximately 2%), the author advocates that universal facility-wide screening of MRSA on admission be included in infection prevention bundles used at U.S. hospital.

Establishing risk-adjusted quality indicators in surgery using administrative data-an example from neurosurgery

BACKGROUND

The current draft of the German Hospital Structure Law requires remuneration to incorporate quality indicators. For neurosurgery, several quality indicators have been discussed, such as 30-day readmission, reoperation, or mortality rates; the rates of infections; or the length of stay. When comparing neurosurgical departments regarding these indicators, very heterogeneous patient spectrums complicate benchmarking due to the lack of risk adjustment.

OBJECTIVE

In this study, we performed an analysis of quality indicators and possible risk adjustment, based only on administrative data.

METHODS

All adult patients that were treated as inpatients for a brain or spinal tumour at our neurosurgical department between 2013 and 2017 were assessed for the abovementioned quality indicators. DRG-related data such as relative weight, PCCL (patient clinical complexity level), ICD-10 major diagnosis category, secondary diagnoses, age and sex were obtained. The age-adjusted Charlson Comorbidity Index (CCI) was calculated. Logistic regression analyses were performed in order to correlate quality indicators with administrative data.

RESULTS

Overall, 2623 cases were enrolled into the study. Most patients were treated for glioma (n = 1055, 40.2%). The CCI did not correlate with the quality indicators, whereas PCCL showed a positive correlation with 30-day readmission and reoperation, SSI and nosocomial infection rates.

CONCLUSION

All previously discussed quality indicators are easily derived from administrative data. Administrative data alone might not be sufficient for adequate risk adjustment as they do not reflect the endogenous risk of the patient and are influenced by certain complications during inpatient stay. Appropriate concepts for risk adjustment should be compiled on the basis of prospectively designed registry studies.

[Quality indicators for colon cancer surgery : Evidence-based development of a set of indicators for the outcome quality]

BACKGROUND

Quality assessment in surgery is gaining in importance. Although sporadic recommendations for quality indicators (QI) in oncological colon surgery can be found in the literature, these are usually not systematically derived from a solid evidence base. Moreover, reference ranges for QI are unknown.

OBJECTIVE

The aim of this initiative was the development of evidence-based QI for oncological colon resections by an expert panel invited by the German Society of General and Visceral Surgery (DGAV). Reference ranges from the literature and reference values from the Study, Documentation, and Quality Center (StuDoQ)|Colon Cancer Register were compared in order to deduce recommendations which are tailored to the German healthcare system.

RESULTS

Based on the most recent scientific evidence and agreed by expert consensus, five QI for oncological colon surgery were defined and evaluated according to the QUALIFY tool. Mortality, MTL30 (mortality, transfer to another acute care hospital, or length of stay ≥30 days), anastomotic leakage requiring reintervention, surgical site infections necessitating reopening of the wound and ≥12 lymph nodes in the specimen qualified as QI owing to their relevance, scientific nature, and practicability. Based on the results of the systematic literature search and the statistical analysis of the StuDoQ|Colon Cancer Register, preliminary reference values are proposed for each QI.

CONCLUSION

The presented set of QI seems appropriate for quality assessment of oncological colon surgery in the context of the German healthcare system. The validity of the QI and the reference values must be reviewed within the framework of their implementation. The StuDoQ|Colon Cancer Register provides a suitable infrastructure for collecting clinical data for quality assessment and risk adjustment.

Use of a medication-based risk adjustment index to predict mortality among veterans dually-enrolled in VA and medicare

BACKGROUND

There is systemic undercoding of medical comorbidities within administrative claims in the Department of Veterans Affairs (VA). This leads to bias when applying claims-based risk adjustment indices to compare outcomes between VA and non-VA settings. Our objective was to compare the accuracy of a medication-based risk adjustment index (RxRisk-VM) to diagnostic claims-based indices for predicting mortality.

METHODS

We modified the RxRisk-V index (RxRisk-VM) by incorporating VA and Medicare pharmacy and durable medical equipment claims in Veterans dually-enrolled in VA and Medicare in 2012. Using the concordance (C) statistic, we compared its accuracy in predicting 1 and 3-year all-cause mortality to the following models: demographics only, demographics plus prescription count, or demographics plus a diagnostic claims-based risk index (e.g., Charlson, Elixhauser, or Gagne). We also compared models containing demographics, RxRisk-VM, and a claims-based index.

RESULTS

In our cohort of 271,184 dually-enrolled Veterans (mean age = 70.5 years, 96.1% male, 81.7% non-Hispanic white), RxRisk-VM (C = 0.773) exhibited greater accuracy in predicting 1-year mortality than demographics only (C = 0.716) or prescription counts (C = 0.744), but was less accurate than the Charlson (C = 0.794), Elixhauser (C = 0.80), or Gagne (C = 0.810) indices (all P < 0.001). Combining RxRisk-VM with claims-based indices enhanced its accuracy over each index alone (all models C ≥ 0.81). Relative model performance was similar for 3-year mortality.

CONCLUSIONS

The RxRisk-VM index exhibited a high level of, but slightly less, accuracy in predicting mortality in comparison to claims-based risk indices.

IMPLICATIONS

Its application may enhance the accuracy of studies examining VA and non-VA care and enable risk adjustment when diagnostic claims are not available or biased.

LEVEL OF EVIDENCE

Level 3.

Surveillance for Hepatocellular Carcinoma: Does the Place Where Ultrasound Is Performed Impact Its Effectiveness?

BACKGROUND

Biannual ultrasound (US) is recommended as the clinical screening tool for hepatocellular carcinoma (HCC). The effectiveness of surveillance according to the place where US is performed has not been previously reported.

AIMS

To compare the effectiveness of US performed in the center responsible for follow-up as opposed to US proceeding from centers other than that of follow-up.

METHODS

This is a multicenter cohort study from Argentina. The last US was categorized as done in the same center or done in a different center from the institution of the patient's follow-up. Surveillance failure was defined as HCC diagnosis not meeting Barcelona Clinic Liver Cancer (BCLC) stages 0-A or when no nodules were observed at HCC diagnosis.

RESULTS

From 533 patients with HCC, 62.4% were under routine surveillance with a surveillance failure of 38.8%. After adjusting for a propensity score matching, BCLC stage and lead-time survival bias, surveillance was associated with a significant survival benefit [HR of 0.51 (CI 0.38; 0.69)]. Among patients under routine surveillance (n = 345), last US was performed in the same center in 51.6% and in a different center in 48.4%. Similar rates of surveillance failure were observed between US done in the same or in a different center (32% vs. 26.3%; P = 0.25). Survival was not significantly different between both surveillance modalities [HR 0.79 (CI 0.53; 1.20)].

CONCLUSIONS

Routine surveillance for HCC in the daily practice improved survival either when performed in the same center or in a different center from that of patient's follow-up.

Measuring individual physician clinical productivity in an era of consolidated group practices

BACKGROUND

As physician groups consolidate and value-based payment replaces traditional fee-for-service systems, physician practices have greater need to accurately measure individual physician clinical productivity within team-based systems. We compared methodologies to measure individual physician outpatient clinical productivity after adjustment for shared practice resources.

METHODS

For cardiologists at our hospital between January 2015 and June 2016, we assessed productivity by examining completed patient visits per clinical session per week. Using mixed-effects models, we sequentially accounted for shared practice resources and underlying baseline characteristics. We compared mixed-effects and Generalized Estimating Equations (GEE) models using K-fold cross validation, and compared mixed-effect, GEE, and Data Envelopment Analysis (DEA) models based on ranking of physicians by productivity.

RESULTS

A mixed-effects model adjusting for shared practice resources reduced variation in productivity among providers by 63% compared to an unadjusted model. Mixed-effects productivity rankings correlated strongly with GEE rankings (Spearman 0.99), but outperformed GEE on K-fold cross validation (root mean squared error 2.66 vs 3.02; mean absolute error 1.89 vs 2.20, respectively). Mixed-effects model rankings had moderate correlation with DEA model rankings (Spearman 0.692), though this improved upon exclusion of outliers (Spearman 0.755).

CONCLUSIONS

Mixed-effects modeling accounts for significant variation in productivity secondary to shared practice resources, outperforms GEE in predictive power, and is less vulnerable to outliers than DEA.

IMPLICATIONS

With mixed-effects regression analysis using otherwise easily accessible administrative data, practices can evaluate physician clinical productivity more fairly and make more informed management decisions on physician compensation and resource allocation.

Emergency department charges may be associated with mortality in patients with severe sepsis and septic shock: a cohort study

Background

Sepsis severity of illness is challenging to measure using claims, which makes sepsis difficult to study using administrative data. We hypothesized that emergency department (ED) charges may be associated with hospital mortality, and could be a surrogate marker of severity of illness for research purposes. The objective of this study was to measure concordance between ED charges and mortality in admitted patients with severe sepsis or septic shock.

Methods

Cohort study of all adult patients presenting to a 60,000-visit Midwestern academic ED with severe sepsis or septic shock (by ICD-9 codes) between July 1, 2008 and June 30, 2010. Data on demographics, admission APACHE-II score, and disposition was extracted from the medical record, and comorbidities were identified from diagnosis codes using the Elixhauser methodology. Summary statistics were reported and bivariate concordance was tested using Pearson correlation. Logistic regression models for 28-day mortality were developed to measure the independent association with mortality.

Results

We included a total of 294 patients in the analysis. We found that ED charges were inversely related to mortality (adjusted OR 0.829 per $1000 increase in total ED charges, 95%CI 0.702–0.980). ED charges were also independently associated with 28-day hospital-free and ICU-free days (0.74 days increase per $1000 additional ED charges, 95%CI 0.06–1.41 and 0.81 days increase per $1000 additional ED charges, 95%CI 0.05–1.56, respectively). ED charges were also associated with APACHE-II score ($34 total ED charges per point increase in APACHE-II score, 95%CI $6–62).

Conclusions

ED charges in administrative data sets are associated with in-hospital mortality and health care utilization, likely related to both illness severity and intensity of early sepsis resuscitation. ED charges may have a role in risk adjustment models using administrative data for acute care research.

Sample Selection for Medicare Risk Adjustment Due to Systematically Missing Data

OBJECTIVE

To assess the issue of nonrepresentative sampling in Medicare Advantage (MA) risk adjustment.

DATA SOURCES

Medicare enrollment and claims data from 2008 to 2011.

DATA EXTRACTION

Risk adjustment predictor variables were created from 2008 to 2010 Part A and B claims and the Medicare Beneficiary Summary File. Spending is based on 2009-2011 Part A and B, Durable Medical Equipment, and Home Health Agency claims files.

STUDY DESIGN

A propensity-score matched sample of Traditional Medicare (TM) beneficiaries who resembled MA enrollees was created. Risk adjustment formulas were estimated using multiple techniques, and performance was evaluated based on R2 , predictive ratios, and formula coefficients in the matched sample and a random sample of TM beneficiaries.

PRINCIPAL FINDINGS

Matching improved balance on observables, but performance metrics were similar when comparing risk adjustment formula results fit on and evaluated in the matched sample versus fit on the random sample and evaluated in the matched sample.

CONCLUSIONS

Fitting MA risk adjustment formulas on a random sample versus a matched sample yields little difference in MA plan payments. This does not rule out potential improvements via the matching method should reliable MA encounter data and additional variables become available for risk adjustment.

Weekend admissions may be associated with poorer recording of long-term comorbidities: a prospective study of emergency admissions using administrative data

BACKGROUND

Many studies have investigated the presence of a 'weekend effect' in mortality following hospital admission, and these frequently use diagnostic codes from administrative data for information on comorbidities for risk adjustment. However, it is possible that coding practice differs between week and weekend. We assess patients with a confirmed history of certain long-term health conditions and investigate how well these are recorded in subsequent week and weekend admissions.

METHODS

We selected six long-term conditions that are commonly assessed when risk-adjusting mortality rates, via the Charlson and Elixhauser indices. Using Hospital Episode Statistics data from England for the period April 2009 to March 2011, we identified patients with the condition recorded at least twice, on separate emergency admissions. Then we assessed how often each condition was recorded on subsequent emergency admissions between April 2011 and March 2013. We then compared coding between week and weekend admissions using the Cochran-Mantel-Haenszel test, stratifying by hospital.

RESULTS

We studied 111,457 patients with chronic pulmonary disease, 106,432 with diabetes, 36,447 with congestive heart failure, 30,996 with dementia, 7808 with hemiplegia or paraplegia and 5877 with metastatic cancer. Across the entire week, between April 2011 and March 2013, coding completeness ranged from 89% for diabetes to 43% for hemiplegia/paraplegia. Compared with weekday admissions, congestive heart failure was less likely to be recorded as a secondary diagnosis at the weekend (odds ratio 0.92, 95% CI, 0.88 to 0.97), with smaller but statistically significant differences also detected for chronic pulmonary disease (odds ratio 0.96, 95% CI, 0.93 to 0.99) and diabetes (odds ratio 0.95, 95% CI 0.91 to 0.99). There was no statistically significant difference in recording between week and weekend admissions for dementia (odds ratio 1.04, 95% CI 0.97 to 1.11), hemiplegia/paraplegia (odds ratio 0.99, 95% CI 0.89 to 1.10) or metastatic cancer (odds ratio 1.04, 95% CI 0.90 to 1.20).

CONCLUSIONS

Long-term conditions are often not recorded on administrative data and the lack of recording may be worse for weekend admissions. Studies of the weekend effect that rely on administrative data might have underestimated the health burden of patients, particularly if admitted at the weekend.

On measuring the quality of hospitals

PURPOSE

Researchers, policymakers and hospital managers often encounter numerous quality measures when assessing hospital quality. The purpose of this paper is to address the challenge of summarising, interpreting and comparing multiple quality measures across different quality dimensions by proposing a simple method of constructing a composite quality index. The method is applied to hospital administrative data to demonstrate its use in analysing hospital performance.

DESIGN/METHODOLOGY/APPROACH

Logistic and fixed effects regression analyses are applied to secondary admitted patient data from all hospitals in the state of Victoria, Australia for the period 2000/2001-2011/2012.

FINDINGS

The derived composite quality index was used to rank hospital performance and to assess changes in state-wide average hospital quality over time. Further regression analyses found private hospitals, day hospitals and non-acute hospitals were associated with higher composite quality, while small hospitals were associated with lower quality.

PRACTICAL IMPLICATIONS

The method will enable policymakers and hospital managers to better monitor the performance of hospitals. It allows quality to be related to other attributes of hospitals such as size and volume, and enables policymakers and managers to focus on hospitals with relevant characteristics such that quantity and quality changes can be better understood, monitored and acted upon.

ORIGINALITY/VALUE

A simple method of constructing a composite quality is an indispensable practical tool in tracking the quality of hospitals when numerous measures are used to capture different aspects of quality. The derived composite quality can be used to summarise hospital performance and to identify factors associated with quality via regression analyses.

Comorbidity Type and Health Care Costs in Type 2 Diabetes: A Retrospective Claims Database Analysis

INTRODUCTION

Previous studies suggest that the type and combination of comorbidities may impact diabetes care, but their cost implications are less clear. This study characterized how diabetes patients' health care utilization and costs may vary according to comorbidity type classified on the basis of the Piette and Kerr framework.

METHODS

We conducted a retrospective observational study of privately insured US adults newly diagnosed with type 2 diabetes (n = 138,466) using the 2014-2016 Optum Clinformatics^® Data Mart. Diabetes patients were classified into five mutually exclusive comorbidity groups: concordant only, discordant only, both concordant and discordant, any dominant, and none. We estimated average health care costs of each comorbidity group by using generalized linear models, adjusting for patient demographics, region, insurance type, and prior-year costs.

RESULTS

Most type 2 diabetes patients had discordant conditions only (27%), dominant conditions (25%), or both concordant and discordant conditions (24%); 7% had concordant conditions only. In adjusted analyses, comorbidities were significantly associated with higher health care costs (p < 0.0001) and the magnitude of the association varied with comorbidity type. Diabetes patients with dominant comorbidities incurred substantially higher costs ($38,168) compared with individuals with both concordant and discordant conditions ($20,401), discordant conditions only ($9173), concordant conditions only ($9000), and no comorbidities ($3365). More than half of the total costs in our sample (53%) were attributable to 25% of diabetes patients who had dominant comorbidities.

CONCLUSIONS

Diabetes patients with both concordant and discordant conditions and with clinically dominant conditions incurred substantially higher health costs than other diabetes patients. Our findings suggest that diabetes management programs must explicitly address concordant, discordant, and dominant conditions because patients may have distinctly different health care needs and utilization patterns depending on their comorbidity profiles. The Piette and Kerr framework may serve as a screening tool to identify high-need, high-cost diabetes patients and suggest targets for tailored interventions.

FUNDING

Sanofi.

Do Autopsies Still Matter? The Influence of Autopsy Data on Final Injury Severity Score Calculations

BACKGROUND

Despite a proven record of identifying injuries missed during clinical evaluation, the effect of autopsy on injury severity score (ISS) calculation is unknown. We hypothesized that autopsy data would alter final ISS and improve the accuracy of outcome data analyses.

MATERIALS AND METHODS

All trauma deaths from January 2010 through June 2014 were reviewed. Trauma registrars calculated Abbreviated Injury Scale and ISS from clinical documentation alone. The most detailed available autopsy report then was reviewed, and AIS/ISS recalculated. Predictors of ISS change were identified using multivariate logistic regression.

RESULTS

Seven hundred thirty-nine deaths occurred, of which 682 (92.3%) underwent autopsy (31% view-only, 3% with preliminary report, and 66% with full report). Patients undergoing full autopsy had a lower median age (39 versus 74 years, P < 0.01), a higher rate of penetrating injury (41.7% versus 0%, P < 0.01), and a higher emergency department mortality rate (30.8% versus 0%, P < 0.01) than those receiving view-only autopsy. Incorporating autopsy findings increased mean ISS (21.3 to 29.6, P < 0.001) and the percentage of patients with ISS ≥ 25 (49.9% to 69.2%, P < 0.001). Multivariate analysis identified length of stay, death in the emergency department, full rather than view-only autopsy, and presenting heart rate as variables associated with ISS increase.

CONCLUSIONS

Autopsy data significantly increased ISS values for trauma deaths. This effect was greatest in patients who died early in their course. Targeting this group, rather than all trauma patients, for full autopsy may improve risk-adjustment accuracy while minimizing costs.

Deriving risk adjustment payment weights to maximize efficiency of health insurance markets

Risk-adjustment is critical to the functioning of regulated health insurance markets. To date, estimation and evaluation of a risk-adjustment model has been based on statistical rather than economic objective functions. We develop a framework where the objective of risk-adjustment is to minimize the efficiency loss from service-level distortions due to adverse selection, and we use the framework to develop a welfare-grounded method for estimating risk-adjustment weights. We show that when the number of risk adjustor variables exceeds the number of decisions plans make about service allocations, incentives for service-level distortion can always be eliminated via a constrained least-squares regression. When the number of plan service-level allocation decisions exceeds the number of risk-adjusters, the optimal weights can be found by an OLS regression on a straightforward transformation of the data. We illustrate this method with the data used to estimate risk-adjustment payment weights in the Netherlands (N = 16.5 million).

Exploring an optimal risk adjustment model for public reporting of cesarean section surgical site infections

BACKGROUND

Public report of surgical site infections (SSI) rates has been an important component of SSI reduction strategies, and risk adjustment is needed before SSI rates are publicly reported. Improving the risk adjustment model facilitates meaningful comparison in the public reporting of SSIs. This research aimed to explore an optimal risk adjustment model for the public reporting of cesarean section (CS) SSI.

METHODS

Information on 2506 cases of CS performed at T hospital, a tertiary general hospital located in the W City of H Province in China, from 01 January 2013 to 31 December 2014 was collected. The data were used to construct the multivariate risk adjustment models of CS SSI through logistic and Poisson stepwise regression. The c-index was used to compare the predictive power between the new logistic regression and the National Nosocomial Infections Surveillance (NNIS) risk index model. Pearson goodness-of-fit was determined to compare the goodness-of-fit between the new Poisson regression and the NNIS risk index model. The two new regression models were also compared.

RESULTS

The logistic and Poisson regression models included two patient-related risk factors, namely, BMI (OR=1.085, P=0.006; RR=1.081, P=0.006) and ASA score (OR=1.522, P=0.044; RR=1.501, P=0.047). The c-index of the logistic regression model (0.628) was higher than that of the NNIS risk index model (0.600). The goodness-of-fit of the Poisson regression model (0.946) was better than that of the NNIS risk index model (0.851).

CONCLUSIONS

The logistic and Poisson regression risk models are better than the NNIS risk index model, implying that a multifactorial risk adjustment model is needed for the public reporting of CS SSI. The advantage of logistic regression model is that the predictive power of model can be evaluated by c-index, however, Poisson regression may offer more advantages on model accuracy than logistic regression does when the infection rate decreases.

Outlier classification performance of risk adjustment methods when profiling multiple providers

BACKGROUND

When profiling multiple health care providers, adjustment for case-mix is essential to accurately classify the quality of providers. Unfortunately, misclassification of provider performance is not uncommon and can have grave implications. Propensity score (PS) methods have been proposed as viable alternatives to conventional multivariable regression. The objective was to assess the outlier classification performance of risk adjustment methods when profiling multiple providers.

METHODS

In a simulation study based on empirical data, the classification performance of logistic regression (fixed and random effects), PS adjustment, and three PS weighting methods was evaluated when varying parameters such as the number of providers, the average incidence of the outcome, and the percentage of outliers. Traditional classification accuracy measures were considered, including sensitivity and specificity.

RESULTS

Fixed effects logistic regression consistently had the highest sensitivity and negative predictive value, yet a low specificity and positive predictive value. Of the random effects methods, PS adjustment and random effects logistic regression performed equally well or better than all the remaining PS methods for all classification accuracy measures across the studied scenarios.

CONCLUSIONS

Of the evaluated PS methods, only PS adjustment can be considered a viable alternative to random effects logistic regression when profiling multiple providers in different scenarios.

Challenges in Outcome Reporting

Although measuring outcomes is an integral part of medical quality improvement, large-scale outcome reporting efforts face several challenges. Among these are difficulties in establishing consensus definitions for outcome measurement; classifying gray outcomes, such as postoperative respiratory failure; and adequately adjusting for patient comorbidities and severity of illness. Unintended consequences of outcome reporting can also distort care in undesirable ways, and clinician reluctance to care for high-risk patients may occur with reporting programs. Ultimately, clinicians need not compare outcomes to improve and should recognize that even outcomes that cannot be precisely quantitated can still be improved.

Validation of the All Patient Refined Diagnosis Related Group (APR-DRG) Risk of Mortality and Severity of Illness Modifiers as a Measure of Perioperative Risk

The All Patient Refined Diagnosis Related Group (APR-DRG) is an inpatient visit classification system that assigns a diagnostic related group, a Risk of Mortality (ROM) subclass and a Severity of Illness (SOI) subclass. While extensively used for cost adjustment, no study has compared the APR-DRG subclass modifiers to the popular Charlson Comorbidity Index as a measure of comorbidity severity in models for perioperative in-hospital mortality. In this study we attempt to validate the use of these subclasses to predict mortality in a cohort of surgical patients. We analyzed all adult (age over 18 years) inpatient non-cardiac surgery at our institution between December 2005 and July 2013. After exclusions, we split the cohort into training and validation sets. We created prediction models of inpatient mortality using the Charlson Comorbidity Index, ROM only, SOI only, and ROM with SOI. Models were compared by receiver-operator characteristic (ROC) curve, area under the ROC curve (AUC), and Brier score. After exclusions, we analyzed 63,681 patient-visits. Overall in-hospital mortality was 1.3%. The median number of ICD-9-CM diagnosis codes was 6 (Q1-Q3 4-10). The median Charlson Comorbidity Index was 0 (Q1-Q3 0-2). When the model was applied to the validation set, the c-statistic for Charlson was 0.865, c-statistic for ROM was 0.975, and for ROM and SOI combined the c-statistic was 0.977. The scaled Brier score for Charlson was 0.044, Brier for ROM only was 0.230, and Brier for ROM and SOI was 0.257. The APR-DRG ROM or SOI subclasses are better predictors than the Charlson Comorbidity Index of in-hospital mortality among surgical patients.

Robust Machine Learning Variable Importance Analyses of Medical Conditions for Health Care Spending

OBJECTIVE

To propose nonparametric double robust machine learning in variable importance analyses of medical conditions for health spending.

DATA SOURCES

2011-2012 Truven MarketScan database.

STUDY DESIGN

I evaluate how much more, on average, commercially insured enrollees with each of 26 of the most prevalent medical conditions cost per year after controlling for demographics and other medical conditions. This is accomplished within the nonparametric targeted learning framework, which incorporates ensemble machine learning. Previous literature studying the impact of medical conditions on health care spending has almost exclusively focused on parametric risk adjustment; thus, I compare my approach to parametric regression.

PRINCIPAL FINDINGS

My results demonstrate that multiple sclerosis, congestive heart failure, severe cancers, major depression and bipolar disorders, and chronic hepatitis are the most costly medical conditions on average per individual. These findings differed from those obtained using parametric regression.

CONCLUSIONS

The literature may be underestimating the spending contributions of several medical conditions, which is a potentially critical oversight. If current methods are not capturing the true incremental effect of medical conditions, undesirable incentives related to care may remain. Further work is needed to directly study these issues in the context of federal formulas.

The German Quality Network Sepsis: study protocol for the evaluation of a quality collaborative on decreasing sepsis-related mortality in a quasi-experimental difference-in-differences design

BACKGROUND

While sepsis-related mortality decreased substantially in other developed countries, mortality of severe sepsis remained as high as 44% in Germany. A recent German cluster randomized trial was not able to improve guideline adherence and decrease sepsis-related mortality within the participating hospitals, partly based on lacking support by hospital management and lacking resources for documentation of prospective data. Thus, more pragmatic approaches are needed to improve quality of sepsis care in Germany. The primary objective of the study is to decrease sepsis-related hospital mortality within a quality collaborative relying on claims data.

METHOD

The German Quality Network Sepsis (GQNS) is a quality collaborative involving 75 hospitals. This study protocol describes the conduction and evaluation of the start-up period of the GQNS running from March 2016 to August 2018. Democratic structures assure participatory action, a study coordination bureau provides central support and resources, and local interdisciplinary quality improvement teams implement changes within the participating hospitals. Quarterly quality reports focusing on risk-adjusted hospital mortality in cases with sepsis based on claims data are provided. Hospitals committed to publish their individual risk-adjusted mortality compared to the German average. A complex risk-model is used to control for differences in patient-related risk factors. Hospitals are encouraged to implement a bundle of interventions, e.g., interdisciplinary case analyses, external peer-reviews, hospital-wide staff education, and implementation of rapid response teams. The effectiveness of the GQNS is evaluated in a quasi-experimental difference-in-differences design by comparing the change of hospital mortality of cases with sepsis with organ dysfunction from a retrospective baseline period (January 2014 to December 2015) and the intervention period (April 2016 to March 2018) between the participating hospitals and all other German hospitals. Structural and process quality indicators of sepsis care as well as efforts for quality improvement are monitored regularly.

DISCUSSION

The GQNS is a large-scale quality collaborative using a pragmatic approach based on claims data. A complex risk-adjustment model allows valid quality comparisons between hospitals and with the German average. If this study finds the approach to be useful for improving quality of sepsis care, it may also be applied to other diseases.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02820675.

Mental Health Risk Adjustment with Clinical Categories and Machine Learning

OBJECTIVE

To propose nonparametric ensemble machine learning for mental health and substance use disorders (MHSUD) spending risk adjustment formulas, including considering Clinical Classification Software (CCS) categories as diagnostic covariates over the commonly used Hierarchical Condition Category (HCC) system.

DATA SOURCES

2012-2013 Truven MarketScan database.

STUDY DESIGN

We implement 21 algorithms to predict MHSUD spending, as well as a weighted combination of these algorithms called super learning. The algorithm collection included seven unique algorithms that were supplied with three differing sets of MHSUD-related predictors alongside demographic covariates: HCC, CCS, and HCC + CCS diagnostic variables. Performance was evaluated based on cross-validated R² and predictive ratios.

PRINCIPAL FINDINGS

Results show that super learning had the best performance based on both metrics. The top single algorithm was random forests, which improved on ordinary least squares regression by 10 percent with respect to relative efficiency. CCS categories-based formulas were generally more predictive of MHSUD spending compared to HCC-based formulas.

CONCLUSIONS

Literature supports the potential benefit of implementing a separate MHSUD spending risk adjustment formula. Our results suggest there is an incentive to explore machine learning for MHSUD-specific risk adjustment, as well as considering CCS categories over HCCs.

Risk adjustment with an outside option

Much of the risk adjustment literature has focused on how persons should be classified and given weights. It has given less attention to the amount of funds in the risk adjustment pool. If, however, there is an outside option, as there is in the principal American risk adjustment systems, there can be favorable or adverse selection in the risk pool. To address any such selection requires that the risk adjustment system not be zero sum; the main American risk adjustment systems differ in this respect.

Matching and Imputation Methods for Risk Adjustment in the Health Insurance Marketplaces

New state-level health insurance markets, denoted Marketplaces, created under the Affordable Care Act, use risk-adjusted plan payment formulas derived from a population ineligible to participate in the Marketplaces. We develop methodology to derive a sample from the target population and to assemble information to generate improved risk-adjusted payment formulas using data from the Medical Expenditure Panel Survey and Truven MarketScan databases. Our approach requires multi-stage data selection and imputation procedures because both data sources have systemic missing data on crucial variables and arise from different populations. We present matching and imputation methods adapted to this setting. The long-term goal is to improve risk-adjustment estimation utilizing information found in Truven MarketScan data supplemented with imputed Medical Expenditure Panel Survey values.

Plan responses to diagnosis-based payment: Evidence from Germany's morbidity-based risk adjustment

Many competitive health insurance markets adjust payments to participating health plans according to their enrollees' risk - including based on diagnostic information. We investigate responses of German health plans to the introduction of morbidity-based risk adjustment in the Statutory Health Insurance in 2009, which triggers payments based on "validated" diagnoses by providers. Using the regulator's data from office-based physicians, we estimate a difference-in-difference analysis of the change in the share and number of validated diagnoses for ICD codes that are inside or outside the risk adjustment but are otherwise similar. We find a differential increase in the share of validated diagnoses of 2.6 and 3.6 percentage points (3-4%) between 2008 and 2013. This increase appears to originate from both a shift from not-validated toward validated diagnoses and an increase in the number of such diagnoses. Overall, our results indicate that plans were successful in influencing physicians' coding practices in a way that could lead to higher payments.

Measuring efficiency of health plan payment systems in managed competition health insurance markets

Adverse selection in health insurance markets leads to two types of inefficiency. On the demand side, adverse selection leads to plan price distortions resulting in inefficient sorting of consumers across health plans. On the supply side, adverse selection creates incentives for plans to inefficiently distort benefits to attract profitable enrollees. Reinsurance, risk adjustment, and premium categories address these problems. Building on prior research on health plan payment system evaluation, we develop measures of the efficiency consequences of price and benefit distortions under a given payment system. Our measures are based on explicit economic models of insurer behavior under adverse selection, incorporate multiple features of plan payment systems, and can be calculated prior to observing actual insurer and consumer behavior. We illustrate the use of these measures with data from a simulated market for individual health insurance.

Demand elasticities and service selection incentives among competing private health plans

We examine selection incentives by health plans while refining the selection index of McGuire et al. (2014) to reflect not only service predictability and predictiveness but also variation in cost sharing, risk-adjusted profits, profit margins, and newly-refined demand elasticities across 26 disaggregated types of service. We contrast selection incentives, measured by service selection elasticities, across six plan types using privately-insured claims data from 73 large employers from 2008 to 2014. Compared to flat capitation, concurrent risk adjustment reduces the elasticity by 47%, prospective risk adjustment by 43%, simple reinsurance system by 32%, and combined concurrent risk adjustment with reinsurance by 60%. Reinsurance significantly reduces the variability of individual-level profits, but increases the correlation of expected spending with profits, which strengthens selection incentives.

Imperfect risk adjustment, risk preferences, and sorting in competitive health insurance markets

I develop a model of insurer price-setting and consumer welfare under risk-adjustment, a policy commonly used to combat inefficient sorting due to adverse selection in health insurance markets. I use the model to illustrate graphically that risk-adjustment causes health plan prices to be based on costs not predicted by the risk-adjustment model ("residual costs") rather than total costs, either weakening or exacerbating selection problems depending on the correlation between demand and costs predicted by the risk-adjustment model. I then use a structural model to estimate the welfare consequences of risk-adjustment, finding a welfare gain of over $600 per person-year.

Introducing risk adjustment and free health plan choice in employer-based health insurance: Evidence from Germany

To equalize differences in health plan premiums due to differences in risk pools, the German legislature introduced a simple Risk Adjustment Scheme (RAS) based on age, gender and disability status in 1994. In addition, effective 1996, consumers gained the freedom to choose among hundreds of existing health plans, across employers and state-borders. This paper (a) estimates RAS pass-through rates on premiums, financial reserves, and expenditures and assesses the overall RAS impact on market price dispersion. Moreover, it (b) characterizes health plan switchers and investigates their annual and cumulative switching rates over time. Our main findings are based on representative enrollee panel data linked to administrative RAS and health plan data. We show that sickness funds with bad risk pools and high pre-RAS premiums lowered their total premiums by 42 cents per additional euro allocated by the RAS. Consequently, post-RAS, health plan prices converged but not fully. Because switchers are more likely to be white collar, young and healthy, the new consumer choice resulted in more risk segregation and the amount of money redistributed by the RAS increased over time.

The new measuring multimorbidity index predicted mortality better than Charlson and Elixhauser indices among the general population

OBJECTIVES

The aim of the study was to develop and validate an updated morbidity index for short-term mortality risk, using chronic conditions identified from routine hospital admission ICD-10 data.

STUDY DESIGN AND SETTING

Retrospective cohort study of all adult New Zealand (NZ) residents at January 1, 2012. Adult NZ residents aged 18 years and over, defined by enrollment with a Primary Healthcare Organisation or accessing public health care in preceding year. Data were split into two data sets for index development (70%, n = 2,331,645) and validation (30%, n = 1,000,166).

RESULTS

The M3 index was constructed using log hazard ratios for 1-year mortality modeled from presence of 61 chronic conditions. Validation results were improved for the M3 index for predicting 1-year mortality compared to Charlson and Elixhauser on the c-statistic (M3: 0.931, Charlson: 0.921, Elixhauser: 0.922; difference M3 vs. Charlson = 0.010, 95% confidence interval [CI]: 0.008, 0.012; M3 vs. Elixhauser = 0.009, 95% CI: 0.007, 0.012) and integrated discriminative improvement (M3 vs. Charlson = 0.024, 95% CI: 0.021, 0.026; M3 vs. Elixhauser = 0.024, 95% CI: 0.022, 0.027).

CONCLUSION

The M3 index had improved predictive performance for 1-year mortality risk over Charlson and Elixhauser indices, allowing better adjustment for mortality risk from chronic conditions. This provides an important tool for population-level analyses of health outcomes.

Diabetes Complications Severity Index (DCSI)-Update and ICD-10 translation

AIMS

The Diabetes Complications Severity Index (DCSI) converts diagnostic codes and laboratory results into a 14-level metric quantifying the long-term effects of diabetes on seven body systems. Adoption of the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) necessitates translation from ICD-9-CM and creates refinement opportunities.

METHODS

ICD-9 codes for secondary and primary diabetes plus all five ICD-10 diabetes categories were incorporated into an updated tool. Additional modifications were made to improve the accuracy of severity assignments.

SUBJECTS

The tools were tested in a Medicare Advantage population.

RESULTS

In the type 2 subpopulation, prevalence steadily declined with increasing score according to the updated DCSI tool, whereas the original tool resulted in an aberrant local prevalence peak at DCSI = 2. In the type 1 subpopulation, score prevalence was greater in type 1 versus type 2 subpopulations (3 versus 0) according to both instruments. Both instruments predicted current-year inpatient admissions risk and near-future mortality, using either purely ICD-9 data or a mix of ICD-9 and ICD-10 data.

DISCUSSION

While the performance of the tool with purely ICD-10 data has yet to be evaluated, this updated tool makes assessment of diabetes patient severity and complications possible in the interim.

Service-level variation, patient-level factors, and treatment outcome in those seen by child mental health services

Service comparison is a policy priority but is not without controversy. This paper aims to investigate the amount of service-level variation in outcomes in child mental health, whether it differed when examining outcomes unadjusted vs. adjusted for expected change over time, and which patient-level characteristics were associated with the difference observed between services. Multilevel regressions were used on N = 3256 young people (53% male, mean age 11.33 years) from 13 child mental health services. Outcome was measured using the parent-reported Strengths and Difficulties Questionnaire. The results showed there was 4-5% service-level variation in outcomes. Findings were broadly consistent across unadjusted vs. adjusted outcomes. Young people with autism or infrequent case characteristics (e.g., substance misuse) had greater risk of poor outcomes. Comparison of services with high proportions of young people with autism or infrequent case characteristics requiring specialist input needs particular caution as these young people may be at greater risk of poor outcomes.

Explaining regional variation in home care use by demand and supply variables

In the Netherlands, home care services like district nursing and personal assistance are provided by private service provider organizations and covered by private health insurance companies which bear legal responsibility for purchasing these services. To improve value for money, their procurement increasingly replaces fee-for-service payments with population based budgets. Setting appropriate population budgets requires adaptation to the legitimate needs of the population, whereas historical costs are likely to be influenced by supply factors as well, not all of which are necessarily legitimate. Our purpose is to explain home care costs in terms of demand and supply factors. This allows for adjusting historical cost patterns when setting population based budgets. Using expenses claims of 60 Dutch municipalities, we analyze eight demand variables and five supply variables with a multiple regression model to explain variance in the number of clients per inhabitant, costs per client and costs per inhabitant. Our models explain 69% of variation in the number of clients per inhabitant, 28% of costs per client and 56% of costs per inhabitant using demand factors. Moreover, we find that supply factors explain an additional 17-23% of variation. Predictors of higher utilization are home care organizations that are integrated with intramural nursing homes, higher competition levels among home care organizations and the availability of complementary services.

Risk Adjustment and Performance Measurement for Esophageal Cancer Resection

Achieving high-quality care for all patients undergoing esophageal cancer requires identifying and modifying risk factors associated with poor outcomes. These factors occur at different time points from the preoperative to the postoperative periods. A straightforward model for predicting outcomes has proved difficult to identify. This article reviews the current studies addressing risk adjustment and performance measurement for esophageal cancer resection.

Comparison of three different methods for risk adjustment in neonatal medicine

BACKGROUND

Quality improvement in health care requires identification of areas in need of improvement by comparing processes and patient outcomes within and between health care providers. It is critical to adjust for different case-mix and outcome risks of patient populations but it is currently unclear which approach has higher validity and how limitations need to be dealt with. Our aim was to compare 3 approaches towards risk adjustment for 7 different major quality indicators in neonatal intensive care (21 models).

METHODS

We compared an indirect standardization, logistic regression and multilevel approach. Parameters for risk adjustment were chosen according to literature and the condition that they may not depend on processes performed by treating clinics. Predictive validity was tested using the mean Brier Score and by comparing area under curve (AUC) using high quality population based data separated into training and validation sets. Changes in attributional validity were analysed by comparing the effect of the models on the observed-to-expected ratios of the clinics in standardized mortality/morbidity ratio charts.

RESULTS

Risk adjustment based on indirect standardization revealed inferior c-statistics but superior Brier scores for 3 of 7 outcomes. Logistic regression and multilevel modelling were equivalent to one another. C-statistics revealed that predictive validity was high for 8 and acceptable for 11 of the 21 models. Yet, the effect of all forms of risk adjustment on any clinic's comparison with the standard was small, even though there was clear risk heterogeneity between clinics.

CONCLUSIONS

All three approaches to risk adjustment revealed comparable results. The limited effect of risk adjustment on clinic comparisons indicates a small case-mix influence on observed outcomes, but also a limited ability to isolate quality improvement potential based on risk-adjustment models. Rather than relying on methodological approaches, we instead recommend that clinics build small collaboratives and compare their indicators both in risk-adjusted and unadjusted form together. This allows qualitatively investigating and discussing the residual risk-differences within networks. The predictive validity should be quantified and reported and stratification into risk groups should be more widely used to correct for confounding.

Improving Hospital Performance Rankings Using Discrete Patient Diagnoses for Risk Adjustment of Outcomes

OBJECTIVE

To assess the changes in patient outcome prediction and hospital performance ranking when incorporating diagnoses as risk adjusters rather than comorbidity indices.

DATA SOURCES

Healthcare Cost and Utilization Project State Inpatient Databases for New York State, 2005-2009.

STUDY DESIGN

Conducted tree-based classification for mortality and readmission by incorporating discrete patient diagnoses as predictors, comparing with traditional comorbidity indices such as those used for Centers for Medicare and Medicaid Services (CMS) outcome models.

PRINCIPAL FINDINGS

Diagnosis codes as predictors increased predictive accuracy 5.6 percent (95% CI: 4.5-6.9 percent) relative to CMS condition categories for heart failure 30-day mortality. Most other outcomes exhibited statistically significant accuracy gains and facility ranking shifts. Sensitivity analysis showed improvements even when predictors were limited to only the diagnoses included in CMS models.

CONCLUSIONS

Discretizing patient severity information beyond the levels of traditional comorbidity indices improves patient outcome predictions and substantially shifts facility rankings.

Risk-adjustment methods for all-payer comparative performance reporting in Vermont

Background

As the emphasis in health reform shifts to value-based payments, especially through multi-payer initiatives supported by the U.S. Center for Medicare & Medicaid Innovation, and with the increasing availability of statewide all-payer claims databases, the need for an all-payer, “whole-population” approach to facilitate the reporting of utilization, cost, and quality measures has grown. However, given the disparities between the different populations served by Medicare, Medicaid, and commercial payers, risk-adjustment methods for addressing these differences in a single measure have been a challenge.

Methods

This study evaluated different levels of risk adjustment for primary care practice populations – from basic adjustments for age and gender to a more comprehensive “full model” risk-adjustment method that included additional demographic, payer, and health status factors. It applied risk adjustment to populations attributed to patient-centered medical homes (283,153 adult patients and 78,162 pediatric patients) in the state of Vermont that are part of the Blueprint for Health program. Risk-adjusted expenditure and utilization outcomes for calendar year 2014 were reported in 102 adult and 56 pediatric primary-care comparative practice profiles.

Results

Using total expenditures as the dependent variable for the adult population, the r² for the model adjusted for age and gender was 0.028. It increased to 0.265 with the additional adjustment for 3M Clinical Risk Groups and to 0.293 with the full model. For the adult population at the practice level, the no-adjustment model had the highest variation as measured by the coefficient of variation (18.5) compared to the age and gender model (14.8); the age, gender, and CRG model (13.0); and the full model (11.7). Similar results were found for the pediatric population practices.

Conclusions

Results indicate that more comprehensive risk-adjustment models are effective for comparing cost, utilization, and quality measures across multi-payer populations. Such evaluations will become more important for practices, many of which do not distinguish their patients by payer type, and for the implementation of incentive-based or alternative payment systems that depend on “whole-population” outcomes. In Vermont, providers, accountable care organizations, policymakers, and consumers have used Blueprint profiles to identify priorities and opportunities for improving care in their communities.

Comparison of risk adjustment methods in patients with liver disease using electronic medical record data

Background

Risk adjustment is essential for valid comparison of patients’ health outcomes or performances of health care providers. Several risk adjustment methods for liver diseases are commonly used but the optimal approach is unknown. This study aimed to compare the common risk adjustment methods for predicting in-hospital mortality in cirrhosis patients using electronic medical record (EMR) data.

Methods

The sample was derived from Beijing YouAn hospital between 2010 and 2014. Previously validated EMR extraction methods were applied to define liver disease conditions, Charlson comorbidity index (CCI), Elixhauser comorbidity index (ECI), Child-Turcotte-Pugh (CTP), model for end-stage liver disease (MELD), MELD sodium (MELDNa), and five-variable MELD (5vMELD). The performance of the common risk adjustment models as well as models combining disease severity and comorbidity indexes for predicting in-hospital mortality was compared using c-statistic.

Results

Of 11,121 cirrhotic patients, 69.9% were males and 15.8% age 65 or older. The c-statistics across compared models ranged from 0.785 to 0.887. All models significantly outperformed the baseline model with age, sex, and admission status (c-statistic: 0.628). The c-statistics for the CCI, ECI, MELDNa, and CTP were 0.808, 0.825, 0.849, and 0.851, respectively. The c-statistic was 0.887 for combination of CTP and ECI, and 0.882 for combination of MELDNa score and ECI.

Conclusions

The liver disease severity indexes (i.e., CTP and MELDNa score) outperformed the CCI and ECI for predicting in-hospital mortality among cirrhosis patients using Chinese EMRs. Combining liver disease severity and comorbidities indexes could improve the discrimination power of predicting in-hospital mortality.

Surgeon Characteristics Supersede Hospital Characteristics in Mortality After Urgent Colectomy

BACKGROUND

Urgent colectomy is a common procedure with a high mortality rate that is performed by a variety of surgeons and hospitals. We investigated patient, surgeon, and hospital characteristics that predicted mortality after urgent colectomy.

METHODS

The University HealthSystem Consortium was queried for adults undergoing urgent or emergent colectomy between 2009 and 2013 (n = 50,707). Hospitals were grouped into quartiles according to risk-adjusted observed-to-expected (O/E) mortality ratios and compared using the 2013 American Hospital Association Annual Survey. Multiple logistic regression was used to determine patient and provider characteristics associated with in-hospital mortality.

RESULTS

The overall mortality rate after urgent colectomy was 9 %. Mortality rates were higher for patients with extreme severity of illness (27.6 %), lowest socioeconomic status (10.6 %), weekend admissions (10.7 %), and open (10.5 %) and total (15.8 %) colectomies. Hospitals with the lowest O/E ratios were smaller and had lower volume and less teaching intensity, but there were no significant trends with regard to financial (expenses, payroll, capital expenditures per bed) or personnel characteristics (physicians, nurses, technicians per bed). On multivariate analysis, mortality was associated with patient age (10 years: OR 1.31, p < 0.01), severity of illness (extreme: OR 34.68, p < 0.01), insurance status (Medicaid: OR 1.24, p < 0.01; uninsured: OR 1.40, p < 0.01), and weekend admission (OR 1.09, p = 0.04). Surgeon volume was associated with reduced mortality (per 10 cases: OR 0.99, p < 0.01), but hospital volume was not (per case: OR 1.00, p = 0.84).

CONCLUSIONS

Mortality is common after urgent colectomy and is associated with patient characteristics. Surgeon volume and practice patterns predicted differences in mortality, whereas hospital factors did not. These data suggest that policies focusing solely on hospital volume ignore other more important predictors of patient outcomes.

A Machine Learning Framework for Plan Payment Risk Adjustment

OBJECTIVE

To introduce cross-validation and a nonparametric machine learning framework for plan payment risk adjustment and then assess whether they have the potential to improve risk adjustment.

DATA SOURCES

2011-2012 Truven MarketScan database.

STUDY DESIGN

We compare the performance of multiple statistical approaches within a broad machine learning framework for estimation of risk adjustment formulas. Total annual expenditure was predicted using age, sex, geography, inpatient diagnoses, and hierarchical condition category variables. The methods included regression, penalized regression, decision trees, neural networks, and an ensemble super learner, all in concert with screening algorithms that reduce the set of variables considered. The performance of these methods was compared based on cross-validated R2 .

PRINCIPAL FINDINGS

Our results indicate that a simplified risk adjustment formula selected via this nonparametric framework maintains much of the efficiency of a traditional larger formula. The ensemble approach also outperformed classical regression and all other algorithms studied.

CONCLUSIONS

The implementation of cross-validated machine learning techniques provides novel insight into risk adjustment estimation, possibly allowing for a simplified formula, thereby reducing incentives for increased coding intensity as well as the ability of insurers to "game" the system with aggressive diagnostic upcoding.

Development and validation of the new ICNARC model for prediction of acute hospital mortality in adult critical care

PURPOSE

To develop and validate an improved risk model to predict acute hospital mortality for admissions to adult critical care units in the UK.

MATERIALS AND METHODS

155,239 admissions to 232 adult critical care units in England, Wales and Northern Ireland between January and December 2012 were used to develop a risk model from a set of 38 candidate predictors. The model was validated using 90,017 admissions between January and September 2013.

RESULTS

The final model incorporated 15 physiological predictors (modelled with continuous nonlinear models), age, dependency prior to hospital admission, chronic liver disease, metastatic disease, haematological malignancy, CPR prior to admission, location prior to admission/urgency of admission, primary reason for admission and interaction terms. The model was well calibrated and outperformed the current ICNARC model on measures of discrimination (area under the receiver operating characteristic curve 0.885 versus 0.869) and model fit (Brier's score 0.108 versus 0.115). On average, the new model reclassified patients into more appropriate risk categories (net reclassification improvement 19.9; P<0.0001). The model performed well across patient subgroups and in specialist critical care units.

CONCLUSIONS

The risk model developed in this study showed excellent discrimination and calibration and when validated on a different period of time and across different types of critical care unit. This in turn allows improved accuracy of comparisons between UK critical care providers.

Neonatal risk mortality scores as predictors for health-related quality of life of infants treated in NICU: a prospective cross-sectional study

PURPOSE

To determine the relationship of Apgar scores, gestational age and neonatal risk mortality scores to health-related quality of life (HRQoL) for infants at the age of 8 months treated after birth in neonatal intensive care unit (NICU).

METHODS

All surviving infants treated in two-third level NICUs in Rijeka, Croatia (from August 2013 to August 2014) were included in this prospective, cross-sectional study. For all neonates, the Score for Neonatal Acute Physiology (SNAP), SNAP with Perinatal Extension (SNAP-PE) and their simplified modifications (SNAP II and SNAP-PE II) were calculated. At the corrected age of 8 months, the Pediatric Quality of Life Questionnaire (PedsQL)-infant scale-was completed by parents of surviving infants. Multiple regression analysis was performed in order to assess the value of neonatal risk mortality scores, Apgar scores and gestational age as possible predictors of HRQoL, measured by questionnaire score.

RESULTS

A strong correlation has been found between SNAP and 5-min Apgar scores to HRQoL. A positive correlation was also found between gestational age and HRQoL.

CONCLUSION

SNAP and 5-min Apgar scores are important outcome indicators, can aid clinicians' and parents' decision making on the benefits and burdens of acute medical interventions and help determine quantities of medical treatment. Educated medical staff, effective and efficient medical treatment and a high quality of care which prevent adverse events in the first minute of life should be a priority in efforts to improve the future quality of life.

Combining quick Sequential Organ Failure Assessment with plasma lactate concentration is comparable to standard Sequential Organ Failure Assessment score in predicting mortality of patients with and without suspected infection

PURPOSE

We sought to determine whether quick Sequential Organ Failure Assessment (qSOFA) score can be used to predict mortality of patients without suspected infection.

MATERIALS AND METHODS

Using prospectively collected data within the first hour of intensive care unit admission, the predictive ability of qSOFA was compared with the Simplified Acute Physiology Score III, Admission Mortality Prediction Model III, Acute Physiology and Chronic Health Evaluation II model, and standard (full-version) SOFA score using area under the receiver operating characteristic (AUROC) curve and Brier score.

RESULTS

Of the 2322 patients included, 279 (12.0%) died after intensive care unit admission. The qSOFA score had a modest ability to predict mortality of all critically ill patients (AUROC, 0.672; 95% confidence interval [CI], 0.638-0.707; Brier score 0.099) including the noninfected patients (AUROC, 0.685; 95% CI, 0.637-0.732; Brier score 0.081). The overall predictive ability and calibration of the qSOFA was comparable to other prognostic scores. Combining qSOFA score with lactate concentrations further enhanced its predictive ability (AUROC, 0.730; 95% CI, 0.694-0.765; Brier score 0.097), comparable to the standard SOFA score.

CONCLUSIONS

The qSOFA score had a modest ability to predict mortality of both septic and nonseptic patients; combining qSOFA with plasma lactate had a predictive ability comparable to the standard SOFA score.

Performance Measurement to Demonstrate Value

Anesthesiologists are obligated to demonstrate the value of the care they provide. The Centers for Medicare and Medicaid Services has multiple performance-based payment programs to drive high-value care and motivate integrated care for surgical patients and hospitalized patients. These programs rely on diverse arrays of performance measures and complex reporting rules. Among all specialties, anesthesiology has tremendous potential to effect wide-ranging change on diverse measures. Performance measures deserve scrutiny by anesthesiologists as tools to improve care, the means by which payment is determined, and as a means to demonstrate the value of care to surgeons, hospitals, and patients.

Effect of change in coding rules on recording diabetes in hospital administrative datasets

AIM

During 2008-2011 Australian Coding Standards mandated a causal relationship between diabetes and inpatient care as a criterion for recording diabetes as a comorbidity in hospital administrative datasets. We aim to measure the effect of the causality mandate on recorded diabetes and associated inter-hospital variations.

METHOD

For patients with diabetes, all admissions between 2004 and 2013 to all New South Wales acute public hospitals were investigated. Poisson mixed models were employed to derive adjusted rates and variations.

RESULTS

The non-recorded diabetes incidence rate was 20.7%. The causality mandate increased the incidence rate four fold during the change period, 2008-2011, compared to the pre- or post-change periods (32.5% vs 8.4% and 6.9%). The inter-hospital variation was also higher, with twice the difference in the non-recorded rate between hospitals with the highest and lowest rates (50% vs 24% and 27% risk gap). The variation decreased during the change period (29%), while the rate continued to rise (53%). Admission characteristics accounted for over 44% of the variation compared with at most two per cent attributable to patient or hospital characteristics. Contributing characteristics explained less of the variation within the change period compared to pre- or post-change (46% vs 58% and 53%). Hospital relative performance was not constant over time.

CONCLUSION

The causality mandate substantially increased the non-recorded diabetes rate and associated inter-hospital variation. Longitudinal accumulation of clinical information at the patient level, and the development of appropriate adoption protocols to achieve comprehensive and timely implementation of coding changes are essential to supporting the integrity of hospital administrative datasets.

The magnitude and mechanisms of the weekend effect in hospital admissions: A protocol for a mixed methods review incorporating a systematic review and framework synthesis

BACKGROUND

Growing literature has demonstrated that patients admitted to hospital during weekends tend to have less favourable outcomes, including increased mortality, compared with similar patients admitted during weekdays. Major policy interventions such as the 7-day services programme in the UK NHS have been initiated to reduce this weekend effect, although the mechanisms behind the effect are unclear. Here, we propose a mixed methods review to systematically examine the literature surrounding the magnitude and mechanisms of the weekend effect.

METHODS

MEDLINE, CINAHL, HMIC, EMBASE, EthOS, CPCI and the Cochrane Library were searched from Jan 2000 to April 2015 using terms related to 'weekends or out-of-hours' and 'hospital admissions'. The 5404 retrieved records were screened by the review team, and will feed into two component reviews: a systematic review of the magnitude of the weekend effect and a framework synthesis of the mechanisms of the weekend effect. A repeat search of MEDLINE will be conducted mid-2016 to update both component reviews. The systematic review will include quantitative studies of non-specific hospital admissions. The primary outcome is the weekend effect on mortality, which will be estimated using a Bayesian random effects meta-analysis. Weekend effects on adverse events, length of hospital stay and patient experience will also be examined. The development of the framework synthesis has been informed by the initial scoping of the literature and focus group discussions. The synthesis will examine both quantitative and qualitative studies that have compared the processes and quality of care between weekends and weekdays, and explicate the underlying mechanisms of the weekend effect.

DISCUSSION

The weekend effect is a complex phenomenon that has major implications for the organisation of health services. Its magnitude and underlying mechanisms have been subject to heated debate. Published literature reviews have adopted restricted scopes or methods and mainly focused on quantitative evidence. This proposed review intends to provide a comprehensive and in-depth synthesis of diverse evidence to inform future policy and research aiming to address the weekend effect.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO 2016: CRD42016036487.

Tradeoffs in the design of health plan payment systems: Fit, power and balance

In many markets, including the new U.S. Marketplaces, health insurance plans are paid by risk-adjusted capitation, sometimes combined with reinsurance and other payment mechanisms. This paper proposes a framework for evaluating the de facto insurer incentives embedded in these complex payment systems. We discuss fit, power and balance, each of which addresses a distinct market failure in health insurance. We implement empirical metrics of fit, power, and balance in a study of Marketplace payment systems. Using data similar to that used to develop the Marketplace risk adjustment scheme, we quantify tradeoffs among the three classes of incentives. We show that an essential tradeoff arises between the goals of limiting costs and limiting cream skimming because risk adjustment, which is aimed at discouraging cream-skimming, weakens cost control incentives in practice. A simple reinsurance system scores better on our measures of fit, power and balance than the risk adjustment scheme in use in the Marketplaces.

Development and validation of the pediatric risk estimate score for children using extracorporeal respiratory support (Ped-RESCUERS)

PURPOSE

To develop and validate the Pediatric Risk Estimation Score for Children Using Extracorporeal Respiratory Support (Ped-RESCUERS). Ped-RESCUERS is designed to estimate the in-hospital mortality risk for children prior to receiving respiratory extracorporeal membrane oxygenation (ECMO) support.

METHODS

This study used data from an international registry of patients aged 29 days to less than 18 years who received ECMO support from 2009 to 2014. We divided the registry into development and validation datasets by calendar date. Candidate variables were selected for model inclusion if the variable independently changed the mortality risk by at least 2 % in a Bayesian logistic regression model with in-hospital mortality as the outcome. We characterized the model's ability to discriminate mortality with the area under curve (AUC) of the receiver operating characteristic.

RESULTS

From 2009 to 2014, 2458 non-neonatal children received ECMO for respiratory support, with a mortality rate of 39.8 %. The development dataset contained 1611 children receiving ECMO support from 2009 to 2012. The model included the following variables: pre-ECMO pH, pre-ECMO arterial partial pressure of carbon dioxide, hours of intubation prior to ECMO support, hours of admission at ECMO center prior to ECMO support, ventilator type, mean airway pressure, pre-ECMO use of milrinone, and a diagnosis of pertussis, asthma, bronchiolitis, or malignancy. The validation dataset included 438 children receiving ECMO support from 2013 to 2014. The Ped-RESCUERS model from the development dataset had an AUC of 0.690, and the validation dataset had an AUC of 0.634.

CONCLUSIONS

Ped-RESCUERS provides a novel measure of pre-ECMO mortality risk. Future studies should seek external validation and improved discrimination of this mortality prediction tool.

Do insurers respond to risk adjustment? A long-term, nationwide analysis from Switzerland

Community rating in social health insurance calls for risk adjustment in order to eliminate incentives for risk selection. Swiss risk adjustment is known to be insufficient, and substantial risk selection incentives remain. This study develops five indicators to monitor residual risk selection. Three indicators target activities of conglomerates of insurers (with the same ownership), which steer enrollees into specific carriers based on applicants' risk profiles. As a proxy for their market power, those indicators estimate the amount of premium-, health care cost-, and risk-adjustment transfer variability that is attributable to conglomerates. Two additional indicators, derived from linear regression, describe the amount of residual cost differences between insurers that are not covered by risk adjustment. All indicators measuring conglomerate-based risk selection activities showed increases between 1996 and 2009, paralleling the establishment of new conglomerates. At their maxima in 2009, the indicator values imply that 56% of the net risk adjustment volume, 34% of premium variability, and 51% cost variability in the market were attributable to conglomerates. From 2010 onwards, all indicators decreased, coinciding with a pre-announced risk adjustment reform implemented in 2012. Likewise, the regression-based indicators suggest that the volume and variance of residual cost differences between insurers that are not equaled out by risk adjustment have decreased markedly since 2009 as a result of the latest reform. Our analysis demonstrates that risk-selection, especially by conglomerates, is a real phenomenon in Switzerland. However, insurers seem to have reduced risk selection activities to optimize their losses and gains from the latest risk adjustment reform.