Impact of Present-on-admission Indicators on Risk-adjusted Hospital Mortality Measurement

Convergent Validity of 2 Widely Used Methodologies for Calculating the Hospital Standardized Mortality Ratio in Flanders, Belgium

OBJECTIVES

To assess their construct validity, we compared results from 2 models used for estimating hospital standardized mortality ratios (HSMRs) in Belgium. The method of the Flemish Hospital Network (FHN) is based on a logistic regression for each of the 64 All Patient Refined Diagnosis-Related Groups that explain 80% of mortality and uses the Elixhauser score to correct for comorbidities. (H)SMRs published on the 3M-Benchmark-Portal are calculated by a simpler indirect standardization for All Patient Refined Diagnosis-Related Groups and risk of mortality (ROM) at discharge.

METHODS

We used administrative data from all eligible hospital admissions in 22 Flemish hospitals between 2016 and 2019 (FHN, n = 682,935; 3M, n = 2,122,305). We evaluated model discrimination and accuracy and assessed agreement in estimated HSMRs between methods.

RESULTS

The Spearman correlation between HSMRs generated by the FHN model and the standard 3M model was 0.79. Although 2 of 22 hospitals showed opposite classification results, that is, an HSMR significantly <1 according to the FHN method but significantly >1 according to the 3M model, classification agreement between methods was significant (agreement for 59.1% of hospitals, κ = 0.45). The 3M model ( c statistic = 0.96, adjusted Brier score = 26%) outperformed the FHN model (0.87, 17%). However, using ROM at admission instead of at discharge in the 3M model significantly reduced model performance ( c statistic = 0.94, adjusted Brier score = 21%), but yielded similar HSMR estimates and eliminated part of the discrepancy with FHN results.

CONCLUSIONS

Results of both models agreed relatively well, supporting convergent validity. Whereas the FHN method only adjusts for disease severity at admission, the ROM indicator of the 3M model includes diagnoses not present on admission. Although diagnosis codes generated by complications during hospitalization have the tendency to increase the predictive performance of a model, these should not be included in risk adjustment procedures.

Elucidating the association between regional variation in diagnostic frequency with risk-adjusted mortality through analysis of claims data of medicare inpatients: a cross-sectional study

OBJECTIVE

The validity of risk-adjustment methods based on administrative data has been questioned because hospital referral regions with higher diagnosis frequencies report lower case-fatality rates, implying that diagnoses do not track the underlying health risk. The objective of this study is to test the hypothesis that regional variation of diagnostic frequency in inpatient records is not associated with different coding practices but a reflection of the underlying health risks.

DESIGN

We applied two stratification methods to Medicare Analysis and Provider Review data from 2009 through 2014: (1) the number of chronic conditions; and, (2) quartiles of Risk Stratification Index (RSI)-defined risk. After sorting hospital referral regions into quintiles of diagnostic frequency, we examined all-cause mortality.

SETTING

Medicare Analysis and Provider Review administrative database.

PARTICIPANTS

18 126 301 hospitalised Medicare fee-for-service beneficiaries aged 65 or older who had at least one hospital-based procedure between 2009 and 2014.

EXPOSURE

Coding frequency and baseline regional population risk factors by hospital referral region.

PRIMARY AND SECONDARY OUTCOMES AND MEASURES

One year all-cause mortality in patients having the same number of chronic conditions or within the same RSI score quartile across US health referral regions, grouped by diagnostic frequency.

RESULTS

No consistent relationship between diagnostic frequency and mortality in the risk stratum defined by number of chronic conditions was detected. In the strata defined by RSI quartile, there was no decrease in mortality as a function of diagnostic frequency. Instead, adjusted mortality was positively correlated with socioeconomic risk factors.

CONCLUSIONS

Using present-on-admission codes only, diagnostic frequency among inpatients with at least one hospital-based procedure appears to be consequent to differences in baseline population health status, rather than diagnostic coding practices. In this population, claims-based risk-adjustment using RSI appears to be useful for assessing hospital outcomes and performance.

Incorporating Present-on-Admission Indicators in Medicare Claims to Inform Hospital Quality Measure Risk Adjustment Models

Importance

Present-on-admission (POA) indicators in administrative claims data allow researchers to distinguish between preexisting conditions and those acquired during a hospital stay. The impact of adding POA information to claims-based measures of hospital quality has not yet been investigated to better understand patient underlying risk factors in the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision setting.

Objective

To assess POA indicator use on Medicare claims and to assess the hospital- and patient-level outcomes associated with incorporating POA indicators in identifying risk factors for publicly reported outcome measures used by the Centers for Medicare & Medicaid Services (CMS).

Design, Setting, and Participants

This comparative effectiveness study used national CMS claims data between July 1, 2015, and June 30, 2018. Six hospital quality measures assessing readmission and mortality outcomes were modified to include POA indicators in risk adjustment models. The models using POA were then compared with models using the existing complications-of-care algorithm to evaluate changes in risk model performance. Patient claims data were included for all Medicare fee-for-service and Veterans Administration beneficiaries aged 65 years or older with inpatient hospitalizations for acute myocardial infarction, heart failure, or pneumonia within the measurement period. Data were analyzed between September 2019 and March 2020.

Main Outcomes and Measures

Changes in patient-level (C statistics) and hospital-level (quintile shifts in risk-standardized outcome rates) model performance after including POA indicators in risk adjustment.

Results

Data from a total of 6 027 988 index admissions were included for analysis, ranging from 491 366 admissions (269 209 [54.8%] men; mean [SD] age, 78.2 [8.3] years) for the acute myocardial infarction mortality outcome measure to 1 395 870 admissions (677 158 [48.5%] men; mean [SD] age, 80.3 [8.7] years) for the pneumonia readmission measure. Use of POA indicators was associated with improvements in risk adjustment model performance, particularly for mortality measures (eg, the C statistic increased from 0.728 [95% CI, 0.726-0.730] to 0.774 [95% CI, 0.773-0.776] when incorporating POA indicators into the acute myocardial infarction mortality measure).

Conclusions and Relevance

The findings of this quality improvement study suggest that leveraging POA indicators in the risk adjustment methodology for hospital quality outcome measures may help to more fully capture patients' risk factors and improve overall model performance. Incorporating POA indicators does not require extra effort on the part of hospitals and would be easy to implement in publicly reported quality outcome measures.

Development and Testing of Improved Models to Predict Payment Using Centers for Medicare & Medicaid Services Claims Data

Importance

Predicting payments for particular conditions or populations is essential for research, benchmarking, public reporting, and calculations for population-based programs. Centers for Medicare & Medicaid Services (CMS) models often group codes into disease categories, but using single, rather than grouped, diagnostic codes and leveraging present on admission (POA) codes may enhance these models.

Objective

To determine whether changes to the candidate variables in CMS models would improve risk models predicting patient total payment within 30 days of hospitalization for acute myocardial infarction (AMI), heart failure (HF), and pneumonia.

Design, Setting, and Participants

This comparative effectiveness research study used data from Medicare fee-for-service hospitalizations for AMI, HF, and pneumonia at acute care hospitals from July 1, 2013, through September 30, 2015. Payments across multiple care settings, services, and supplies were included and adjusted for geographic and policy variations, corrected for inflation, and winsorized. The same data source was used but varied for the candidate variables and their selection, and the method used by CMS for public reporting that used grouped codes was compared with variations that used POA codes and single diagnostic codes. Combinations of use of POA codes, separation of index admission diagnoses from those in the previous 12 months, and use of individual International Classification of Diseases, Ninth Revision, Clinical Modification codes instead of grouped diagnostic categories were tested. Data analysis was performed from December 4, 2017, to June 10, 2019.

Main Outcomes and Measures

The models' goodness of fit was compared using root mean square error (RMSE) and the McFadden pseudo R2.

Results

Among the 1 943 049 total hospitalizations of the study participants, 343 116 admissions were for AMI (52.5% male; 37.4% aged ≤74 years), 677 044 for HF (45.5% male; 25.9% aged ≤74 years), and 922 889 for pneumonia (46.4% male; 28.2% aged ≤74 years). The mean (SD) 30-day payment was $23 103 ($18 221) for AMI, $16 365 ($12 527) for HF, and $17 097 ($12 087) for pneumonia. Each incremental model change improved the pseudo R2 and RMSE. Incorporating all 3 changes improved the pseudo R2 of the patient-level models from 0.077 to 0.129 for AMI, from 0.042 to 0.129 for HF, and from 0.114 to 0.237 for pneumonia. Parallel improvements in RMSE were found for all 3 conditions.

Conclusions and Relevance

Leveraging POA codes, separating index from previous diagnoses, and using single diagnostic codes improved payment models. Better models can potentially improve research, benchmarking, public reporting, and calculations for population-based programs.

Evaluation of the Present-on-Admission Indicator among Hospitalized Fee-for-Service Medicare Patients with a Pressure Ulcer Diagnosis: Coding Patterns and Impact on Hospital-Acquired Pressure Ulcer Rates

OBJECTIVES

To evaluate national present-on-admission (POA) reporting for hospital-acquired pressure ulcers (HAPUs) and examine the impact of quality measure exclusion criteria on HAPU rates.

DATA SOURCES/STUDY SETTING

Medicare inpatient, outpatient, and nursing facility data as well as independent provider claims (2010-2011).

STUDY DESIGN

Retrospective cross-sectional study.

DATA COLLECTION/EXTRACTION METHODS

We evaluated acute inpatient hospital admissions among Medicare fee-for-service (FFS) beneficiaries in 2011. Admissions were categorized as follows: (1) no pressure ulcer diagnosis, (2) new pressure ulcer diagnosis, and (3) previously documented pressure ulcer diagnosis. HAPU rates were calculated by varying patient exclusion criteria.

PRINCIPAL FINDINGS

Among admissions with a pressure ulcer diagnosis, we observed a large discrepancy in the proportion of admissions with a HAPU based on hospital-reported POA data (5.2 percent) and the proportion with a new pressure ulcer diagnosis based on patient history in billing claims (49.7 percent). Applying quality measure exclusion criteria resulted in removal of 91.2 percent of admissions with a pressure injury diagnosis from HAPU rate calculations.

CONCLUSIONS

As payers and health care organizations expand the use of quality measures, it is important to consider how the measures are implemented, coding revisions to improve measure validity, and the impact of patient exclusion criteria on provider performance evaluation.

How to better identify patients at high risk of postoperative complications?

PURPOSE OF REVIEW

Preoperative risk assessment and perioperative factors may help identify patients at increased risk of postoperative complications and allow postoperative management strategies that improve patient outcomes. This review summarizes historical and more recent scoring systems for predicting patients with increased morbidity and mortality in the postoperative period.

RECENT FINDINGS

Most prediction scores predict postoperative mortality with, at best, moderate accuracy. Scores that incorporate surgery-specific and intraoperative covariates may improve the accuracy of traditional scores. Traditional risk factors including increased ASA physical status score, emergent surgery, intraoperative blood loss and hemodynamic instability are consistently associated with increased mortality using most scoring systems.

SUMMARY

Preoperative clinical risk indices and risk calculators estimate surgical risk with moderate accuracy. Surgery-specific risk calculators are helpful in identifying patients at increased risk of 30-day mortality. Particular attention should be paid to intraoperative hemodynamic instability, blood loss, extent of surgical excision and volume of resection.

Period-dependent Associations between Hypotension during and for Four Days after Noncardiac Surgery and a Composite of Myocardial Infarction and Death

Background

The relative contributions of intraoperative and postoperative hypotension to perioperative morbidity remain unclear. We determined the association between hypotension and a composite of 30-day myocardial infarction and death over three periods: (1) intraoperative, (2) remaining day of surgery, and (3) during the initial four postoperative days.

Methods

This was a substudy of POISE-2, a 10,010-patient factorial-randomized trial of aspirin and clonidine for prevention of myocardial infarction. Clinically important hypotension was defined as systolic blood pressure less than 90 mmHg requiring treatment. Minutes of hypotension was the exposure variable intraoperatively and for the remaining day of surgery, whereas hypotension status was treated as binary variable for postoperative days 1 to 4. We estimated the average relative effect of hypotension across components of the composite using a distinct effect generalized estimating model, adjusting for hypotension during earlier periods.

Results

Among 9,765 patients, 42% experienced hypotension, 590 (6.0%) had an infarction, and 116 (1.2%) died within 30 days of surgery. Intraoperatively, the estimated average relative effect across myocardial infarction and mortality was 1.08 (98.3% CI, 1.03, 1.12; P &lt; 0.001) per 10-min increase in hypotension duration. For the remaining day of surgery, the odds ratio was 1.03 (98.3% CI, 1.01, 1.05; P &lt; 0.001) per 10-min increase in hypotension duration. The average relative effect odds ratio was 2.83 (98.3% CI, 1.26, 6.35; P = 0.002) in patients with hypotension during the subsequent four days of hospitalization.

Conclusions

Clinically important hypotension—a potentially modifiable exposure—was significantly associated with a composite of myocardial infarction and death during each of three perioperative periods, even after adjustment for previous hypotension.

Comparison of an Updated Risk Stratification Index to Hierarchical Condition Categories

Background

The Risk Stratification Index and the Hierarchical Condition Categories model baseline risk using comorbidities and procedures. The Hierarchical Condition categories are rederived yearly, whereas the Risk Stratification Index has not been rederived since 2010. The two models have yet to be directly compared. The authors thus rederived the Risk Stratification Index using recent data and compared their results to contemporaneous Hierarchical Condition Categories.

Methods

The authors reimplemented procedures used to derive the original Risk Stratification Index derivation using the 2007 to 2011 Medicare Analysis and Provider review file. The Hierarchical Condition Categories were constructed on the entire data set using software provided by the Center for Medicare and Medicaid Services. C-Statistics were used to compare discrimination between the models. After calibration, accuracy for each model was evaluated by plotting observed against predicted event rates.

Results

Discrimination of the Risk Stratification Index improved after rederivation. The Risk Stratification Index discriminated considerably better than the Hierarchical Condition Categories for in-hospital, 30-day, and 1-yr mortality and for hospital length-of-stay. Calibration plots for both models demonstrated linear predictive accuracy, but the Risk Stratification Index predictions had less variance.

Conclusions

Risk Stratification discrimination and minimum-variance predictions make it superior to Hierarchical Condition Categories. The Risk Stratification Index provides a solid basis for care-quality metrics and for provider comparisons.

Writing Research Reports

Clear writing makes manuscripts easier to understand. Clear writing enhances research reports, increasing clinical adoption and scientific impact. We discuss styles and organization to help junior investigators present their findings and avoid common errors.

Validation and Calibration of the Risk Stratification Index

BACKGROUND

The Risk Stratification Index was developed from 35 million Medicare hospitalizations from 2001 to 2006 but has yet to be externally validated on an independent large national data set, nor has it been calibrated. Finally, the Medicare Analysis and Provider Review file now allows 25 rather than 9 diagnostic codes and 25 rather than 6 procedure codes and includes present-on-admission flags. The authors sought to validate the index on new data, test the impact of present-on-admission codes, test the impact of the expansion to 25 diagnostic and procedure codes, and calibrate the model.

METHODS

The authors applied the original index coefficients to 39,753,036 records from the 2007-2012 Medicare Analysis data set and calibrated the model. The authors compared their results with 25 diagnostic and 25 procedure codes, with results after restricting the model to the first 9 diagnostic and 6 procedure codes and to codes present on admission.

RESULTS

The original coefficients applied to the 2007-2012 data set yielded C statistics of 0.83 for 1-yr mortality, 0.84 for 30-day mortality, 0.94 for in-hospital mortality, and 0.86 for median length of stay-values nearly identical to those originally reported. Calibration equations performed well against observed outcomes. The 2007-2012 model discriminated similarly when codes were restricted to nine diagnostic and six procedure codes. Present-on-admission models were about 10% less predictive for in-hospital mortality and hospital length of stay but were comparably predictive for 30-day and 1-yr mortality.

CONCLUSIONS

Risk stratification performance was largely unchanged by additional diagnostic and procedure codes and only slightly worsened by restricting analysis to codes present on admission. The Risk Stratification Index, after calibration, thus provides excellent discrimination and calibration for important health services outcomes and thus appears to be a good basis for making hospital comparisons.

Differences in Use of High-quality and Low-quality Hospitals Among Working-age Individuals by Insurance Type

BACKGROUND

Research suggests that individuals with Medicaid or no insurance receive fewer evidence-based treatments and have worse outcomes than those with private insurance for a broad range of conditions. These differences may be due to patients' receiving care in hospitals of different quality.

RESEARCH DESIGN

We used the Healthcare Cost and Utilization Project State Inpatient Databases 2009-2010 data to identify patients aged 18-64 years with private insurance, Medicaid, or no insurance who were hospitalized with acute myocardial infarction, heart failure, pneumonia, stroke, or gastrointestinal hemorrhage. Multinomial logit regressions estimated the probability of admissions to hospitals classified as high, medium, or low quality on the basis of risk-adjusted, in-hospital mortality.

RESULTS

Compared with patients who have private insurance, those with Medicaid or no insurance were more likely to be minorities and to reside in areas with low-socioeconomic status. The probability of admission to high-quality hospitals was similar for patients with Medicaid (23.3%) and private insurance (23.0%) but was significantly lower for patients without insurance (19.8%, P<0.01) compared with the other 2 insurance groups. Accounting for demographic, socioeconomic, and clinical characteristics did not influence the results.

CONCLUSIONS

Previously noted disparities in hospital quality of care for Medicaid recipients are not explained by differences in the quality of hospitals they use. Patients without insurance have lower use of high-quality hospitals, a finding that needs exploration with data after 2013 in light of the Affordable Care Act, which is designed to improve access to medical care for patients without insurance.

Attempted Development of a Tool to Predict Anesthesia Preparation Time From Patient-Related and Procedure-Related Characteristics

BACKGROUND

Operating room (OR) utilization generally ranges from 50% to 75%. Inefficiencies can arise from various factors, including prolonged anesthesia preparation time, defined as the period from induction of anesthesia until patients are considered ready for surgery. Our goal was to use patient-related and procedure-related factors to develop a model predicting anesthesia preparation time.

METHODS

From the electronic medical records of adults who had noncardiac surgery at the Cleveland Clinic Main Campus, we developed a model that used a dozen preoperative factors to predict anesthesia preparation time. The model was based on multivariable regression with "Least Absolute Shrinkage and Selection Operator" and 10-fold cross-validation. The overall performance of the final model was measured by R, which describes the proportion of the variance in anesthesia preparation time that is explained by the model.

RESULTS

A total of 43,941 cases met inclusion and exclusion criteria. Our final model had only moderate discriminative ability. The estimated adjusted R for prediction model was 0.34 for the training data set and 0.27 for the testing data set.

CONCLUSIONS

Using preoperative factors, we could explain only about a quarter of the variance in anesthesia preparation time-an amount that is probably of limited clinical value.

Admission Laboratory Results to Enhance Prediction Models of Postdischarge Outcomes in Cardiac Care

Predictive modeling for postdischarge outcomes of inpatient care has been suboptimal. This study evaluated whether admission numerical laboratory data added to administrative models from New York and Minnesota hospitals would enhance the prediction accuracy for 90-day postdischarge deaths without readmission (PD-90) and 90-day readmissions (RA-90) following inpatient care for cardiac patients. Risk-adjustment models for the prediction of PD-90 and RA-90 were designed for acute myocardial infarction, percutaneous cardiac intervention, coronary artery bypass grafting, and congestive heart failure. Models were derived from hospital claims data and were then enhanced with admission laboratory predictive results. Case-level discrimination, goodness of fit, and calibration were used to compare administrative models (ADM) and laboratory predictive models (LAB). LAB models for the prediction of PD-90 were modestly enhanced over ADM, but negligible benefit was seen for RA-90. A consistent predictor of PD-90 and RA-90 was prolonged length of stay outliers from the index hospitalization.

Preoperative Score to Predict Postoperative Mortality (POSPOM)

Background

An accurate risk score able to predict in-hospital mortality in patients undergoing surgery may improve both risk communication and clinical decision making. The aim of the study was to develop and validate a surgical risk score based solely on preoperative information, for predicting in-hospital mortality.

Methods

From January 1, 2010, to December 31, 2010, data related to all surgeries requiring anesthesia were collected from all centers (single hospital or hospitals group) in France performing more than 500 operations in the year on patients aged 18 yr or older (n = 5,507,834). International Statistical Classification of Diseases, 10th revision codes were used to summarize the medical history of patients. From these data, the authors developed a risk score by examining 29 preoperative factors (age, comorbidities, and surgery type) in 2,717,902 patients, and then validated the risk score in a separate cohort of 2,789,932 patients.

Results

In the derivation cohort, there were 12,786 in-hospital deaths (0.47%; 95% CI, 0.46 to 0.48%), whereas in the validation cohort there were 14,933 in-hospital deaths (0.54%; 95% CI, 0.53 to 0.55%). Seventeen predictors were identified and included in the PreOperative Score to predict PostOperative Mortality (POSPOM). POSPOM showed good calibration and excellent discrimination for in-hospital mortality, with a c-statistic of 0.944 (95% CI, 0.943 to 0.945) in the development cohort and 0.929 (95% CI, 0.928 to 0.931) in the validation cohort.

Conclusion

The authors have developed and validated POSPOM, a simple risk score for the prediction of in-hospital mortality in surgical patients.

Preoperative Surgical Risk Predictions Are Not Meaningfully Improved by Including the Surgical Apgar Score: An Analysis of the Risk Quantification Index and Present-On-Admission Risk Models

BACKGROUND

Estimating surgical risk is critical for perioperative decision making and risk stratification. Current risk-adjustment measures do not integrate dynamic clinical parameters along with baseline patient characteristics, which may allow a more accurate prediction of surgical risk. The goal of this study was to determine whether the preoperative Risk Quantification Index (RQI) and Present-On-Admission Risk (POARisk) models would be improved by including the intraoperative Surgical Apgar Score (SAS).

METHODS

The authors identified adult patients admitted after noncardiac surgery. The RQI and POARisk were calculated using published methodologies, and model performance was compared with and without the SAS. Relative quality was measured using Akaike and Bayesian information criteria. Calibration was compared by the Brier score. Discrimination was compared by the area under the receiver operating curves (AUROCs) using a bootstrapping procedure for bias correction.

RESULTS

SAS alone was a statistically significant predictor of both 30-day mortality and in-hospital mortality (P < 0.0001). The RQI had excellent discrimination with an AUROC of 0.8433, which increased to 0.8529 with the addition of the SAS. The POARisk had excellent discrimination with an AUROC of 0.8608, which increased to 0.8645 by including the SAS. Similarly, overall performance and relative quality increased.

CONCLUSIONS

While AUROC values increased, the RQI and POARisk preoperative risk models were not meaningfully improved by adding intraoperative risk using the SAS. In addition to the estimated blood loss, lowest heart rate, and lowest mean arterial pressure, other dynamic clinical parameters from the patient's intraoperative course may need to be combined with procedural risk estimate models to improve risk stratification.

Lack of Association between Blood Pressure Management by Anesthesia Residents and Competence Committee Evaluations or In-training Exam Performance: A Cohort Analysis

BACKGROUND

Prompt treatment of severe blood pressure instability requires both cognitive and technical skill. The ability to anticipate and respond to episodes of hemodynamic instability should improve with training. The authors tested the hypothesis that the duration of severe hypotension during anesthesia administered by residents correlates with concurrent adjusted overall performance evaluations by the Clinical Competence Committee and subsequent in-training exam scores.

METHODS

The authors obtained data on 70 first- and second-year anesthesia residents at the Cleveland Clinic. Analysis was restricted to adults having noncardiac surgery with general anesthesia. Outcome variables were in-training exam scores and subjective evaluations of resident performance ranked in quintiles. The primary predictor was cumulative systolic arterial pressure less than 70 mmHg. Secondary predictors were administration of vasopressors, frequency of hypotension, average duration of hypotensive episodes, and blood pressure variability.

RESULTS

The primary statistical approach was mixed-effects modeling, adjusted for potential confounders. The authors considered 15,216 anesthesia care episodes. A total of 1,807 hypotensive episodes were observed, lasting an average of 32 ± 20 min (SD) per 100 h of anesthesia, with 68% being followed by vasopressor administration. The duration of severe hypotension (systolic pressure less than 70 mmHg) was associated with neither Competence Committee evaluations nor in-training exam scores. There was also no association between secondary blood pressure predictors and either Competence Committee evaluations or in-training exam results.

CONCLUSIONS

There was no association between any of the five blood pressure management characteristics and either in-training exam scores or clinical competence evaluations. However, it remains possible that the measures of physiologic control, as assessed from electronic anesthesia records, evaluate useful but different aspects of anesthesiologist performance.

Enhancing Clinical Content and Race/Ethnicity Data in Statewide Hospital Administrative Databases: Obstacles Encountered, Strategies Adopted, and Lessons Learned

OBJECTIVES

Eight grant teams used Agency for Healthcare Research and Quality infrastructure development research grants to enhance the clinical content of and improve race/ethnicity identifiers in statewide all-payer hospital administrative databases.

PRINCIPAL FINDINGS

Grantees faced common challenges, including recruiting data partners and ensuring their continued effective participation, acquiring and validating the accuracy and utility of new data elements, and linking data from multiple sources to create internally consistent enhanced administrative databases. Successful strategies to overcome these challenges included aggressively engaging with providers of critical sources of data, emphasizing potential benefits to participants, revising requirements to lessen burdens associated with participation, maintaining continuous communication with participants, being flexible when responding to participants' difficulties in meeting program requirements, and paying scrupulous attention to preparing data specifications and creating and implementing protocols for data auditing, validation, cleaning, editing, and linking. In addition to common challenges, grantees also had to contend with unique challenges from local environmental factors that shaped the strategies they adopted.

CONCLUSIONS

The creation of enhanced administrative databases to support comparative effectiveness research is difficult, particularly in the face of numerous challenges with recruiting data partners such as competing demands on information technology resources. Excellent communication, flexibility, and attention to detail are essential ingredients in accomplishing this task. Additional research is needed to develop strategies for maintaining these databases when initial funding is exhausted.

Quality-Cost Relationship in Congenital Heart Surgery

BACKGROUND

There is an increasing focus on optimizing health care quality and reducing costs. The care of children undergoing heart surgery requires significant investment of resources, and it remains unclear how costs of care relate to quality. We evaluated this relationship across a multicenter cohort.

METHODS

Clinical data from The Society of Thoracic Surgeons Database were merged with cost data from the Pediatric Health Information Systems Database for children undergoing heart surgery (2006 to 2010). Hospital-level costs were modeled using Bayesian hierarchical methods adjusting for case-mix, and hospitals were categorized into cost tertiles. The primary quality metric evaluated was in-hospital mortality.

RESULTS

Overall, 27 hospitals (30,670 patients) were included. Median adjusted cost per case was $82,360 and varied fivefold across hospitals, while median adjusted mortality was 3.4% and ranged from 2.4% to 5.0% across hospitals. Overall, hospitals in the lowest cost tertile had significantly lower adjusted mortality rates compared with the middle and high cost tertiles (2.5% vs 3.8% and 3.5%, respectively, both p < 0.001). When assessed at the individual hospital level, most (75%) but not all hospitals in the lowest cost tertile were also in the lowest mortality tertile. Similar relationships were seen across the spectrum of surgical complexity. Lower cost hospitals also had shorter length of stay and trends toward fewer major complications.

CONCLUSIONS

Lowest cost hospitals generally deliver the highest quality care for children undergoing heart surgery, although there is some variation in this relationship. This information is important in the design of initiatives aiming to optimize health care value in this population.

Capturing diagnosis-timing in ICD-coded hospital data: recommendations from the WHO ICD-11 topic advisory group on quality and safety

PURPOSE

To develop a consensus opinion regarding capturing diagnosis-timing in coded hospital data.

METHODS

As part of the World Health Organization International Classification of Diseases-11th Revision initiative, the Quality and Safety Topic Advisory Group is charged with enhancing the capture of quality and patient safety information in morbidity data sets. One such feature is a diagnosis-timing flag. The Group has undertaken a narrative literature review, scanned national experiences focusing on countries currently using timing flags, and held a series of meetings to derive formal recommendations regarding diagnosis-timing reporting.

RESULTS

The completeness of diagnosis-timing reporting continues to improve with experience and use; studies indicate that it enhances risk-adjustment and may have a substantial impact on hospital performance estimates, especially for conditions/procedures that involve acutely ill patients. However, studies suggest that its reliability varies, is better for surgical than medical patients (kappa in hip fracture patients of 0.7-1.0 versus kappa in pneumonia of 0.2-0.6) and is dependent on coder training and setting. It may allow simpler and more precise specification of quality indicators.

CONCLUSIONS

As the evidence indicates that a diagnosis-timing flag improves the ability of routinely collected, coded hospital data to support outcomes research and the development of quality and safety indicators, the Group recommends that a classification of 'arising after admission' (yes/no), with permitted designations of 'unknown or clinically undetermined', will facilitate coding while providing flexibility when there is uncertainty. Clear coding standards and guidelines with ongoing coder education will be necessary to ensure reliability of the diagnosis-timing flag.

Empirical Treatment Effectiveness Models for Binary Outcomes

Randomized trials provide strong evidence regarding efficacy of interventions but are limited in their capacity to address potential heterogeneity in effectiveness within broad clinical populations. For example, a treatment that on average is superior may be distinctly worse in certain patients. We propose a technique for using large electronic health registries to develop and validate decision models that measure-for distinct combinations of covariate values-the difference in predicted outcomes among 2 alternative treatments. We demonstrate the methodology in a prototype analysis of in-hospital mortality under alternative revascularization treatments. First, we developed prediction models for a binary outcome of interest for each treatment. Decision criteria were then defined based on the treatment-specific model predictions. Patients were then classified as receiving concordant or discordant care (in relation to the model recommendation), and the association between discordance and outcomes was evaluated. We then present alternative decision criteria and validation methodologies, as well as sensitivity analyses that investigate 1) the imbalance between treatments on observed covariates and 2) the aggregate impact of unobserved covariates. Our methodology supplements population-average clinical trial results by modeling heterogeneity in outcomes according to specific covariate values. It thus allows for assessment of current practice, from which cogent hypotheses for improved care can be derived. Newly emerging large population registries will allow for accurate predictions of outcome risk under competing treatments, as complex functions of predictor variables. Whether or not the models might be used to inform decision making depends on the extent to which important predictors are available. Further work is needed to understand the strengths and limitations of this approach, particularly in relation to those based on randomized trials.

Effect of Present-on-Admission (POA) Reporting Accuracy on Hospital Performance Assessments Using Risk-Adjusted Mortality

OBJECTIVE

To evaluate how the accuracy of present-on-admission (POA) reporting affects hospital 30-day acute myocardial infarction (AMI) mortality assessments.

DATA SOURCES

A total of 2005 California patient discharge data (PDD) and vital statistics death files.

STUDY DESIGN

We compared hospital performance rankings using an established model assessing hospital performance for AMI with (1) a model incorporating POA indicators of whether a secondary condition was a comorbidity or a complication of care, and (2) a simulation analysis that factored POA indicator accuracy into the hospital performance assessment. For each simulation, we changed POA indicators for six major acute risk factors of AMI mortality. The probability of POA being changed depended on patient and hospital characteristics.

PRINCIPAL FINDINGS

Comparing the performance rankings of 268 hospitals using the established model with that using the POA indicator, 67 hospitals' (25 percent) rank differed by ≥10 percent. POA reporting inaccuracy due to overreporting and underreporting had little additional impact; POA overreporting contributed to 4 percent of hospitals' difference in rank compared to the POA model and POA underreporting contributed to <1 percent difference.

CONCLUSION

Incorporating POA indicators into risk-adjusted models of AMI care has a substantial impact on hospital rankings of performance that is not primarily attributable to inaccuracy in POA hospital reporting.

Economic trends from 2003 to 2010 for perioperative myocardial infarction: a retrospective, cohort study

BACKGROUND

Perioperative myocardial infarction (PMI) is a major surgical complication that is costly and causes much morbidity and mortality. Diagnosis and treatment of PMIs have evolved over time. Many treatments are expensive but may reduce ancillary expenses including the duration of hospital stay. The time-dependent economic impact of novel treatments for PMI remains unexplored. The authors thus evaluated absolute and incremental costs of PMI over time and discharge patterns.

METHODS

Approximately 31 million inpatient discharges were analyzed between 2003 and 2010 from the California State Inpatient Database. PMI was defined using International Classification of Diseases, Ninth Revision, Clinical Modification codes. Propensity matching generated 21,637 pairs of comparable patients. Quantile regression modeled incremental charges as the response variable and year of discharge as the main predictor. Time trends of incremental charges adjusted to 2012 dollars, mortality, and discharge destination was evaluated.

RESULTS

Median incremental charges decreased annually by $1,940 (95% CI, $620 to $3,250); P < 0.001. Compared with non-PMI patients, the median length of stay of patients who experienced PMI decreased significantly over time: yearly decrease was 0.16 (0.10 to 0.23) days; P < 0.001. No mortality differences were seen; but over time, PMI patients were increasingly likely to be transferred to another facility.

CONCLUSIONS

Reduced incremental cost and unchanged mortality may reflect improving efficiency in the standard management of PMI. An increasing fraction of discharges to skilled nursing facilities seems likely a result from hospitals striving to reduce readmissions. It remains unclear whether this trend represents a transfer of cost and risk or improves patient care.

Use of present-on-admission indicators for complications after total knee arthroplasty: an analysis of Medicare administrative data

Administrative data are commonly used to evaluate total joint arthroplasty, but analyses have historically been limited by the inability to capture which conditions were present-on-admission (POA). In 2007 Medicare began allowing hospitals to submit POA information. We used Medicare Part A data from 2008 to 2009 to examine POA coding for three common complications (pulmonary embolism [PE], hemorrhage/hematoma, and infection) for primary and revision total knee arthroplasty (TKA). POA information was complete for 60%-75% of complications. There was no evidence that higher TKA volume hospitals or major teaching hospitals were more likely to accurately code POA data. The percentage of complications coded as POA ranged from 6.4% (PE during index admission for primary TKA) to 68.8% (infection during index admission for revision TKA). Early experience suggests that POA coding can significantly enhance the value of Medicare data for evaluating TKA outcomes.