Using Automated Clinical Data for Risk Adjustment

Exploratory analysis of novel electronic health record variables for quantification of healthcare delivery strain, prediction of mortality, and prediction of imminent discharge

OBJECTIVE

To explore the relationship between novel, time-varying predictors for healthcare delivery strain (eg, counts of patient orders per hour) and imminent discharge and in-hospital mortality.

MATERIALS AND METHODS

We conducted a retrospective cohort study using data from adults hospitalized at 21 Kaiser Permanente Northern California hospitals between November 1, 2015 and October 31, 2020 and the nurses caring for them. Patient data extracted included demographics, diagnoses, severity measures, occupancy metrics, and process of care metrics (eg, counts of intravenous drip orders per hour). We linked these data to individual registered nurse records and created multiple dynamic, time-varying predictors (eg, mean acute severity of illness for all patients cared for by a nurse during a given hour). All analyses were stratified by patients' initial hospital unit (ward, stepdown unit, or intensive care unit). We used discrete-time hazard regression to assess the association between each novel time-varying predictor and the outcomes of discharge and mortality, separately.

RESULTS

Our dataset consisted of 84 162 161 hourly records from 954 477 hospitalizations. Many novel time-varying predictors had strong associations with the 2 study outcomes. However, most of the predictors did not merely track patients' severity of illness; instead, many of them only had weak correlations with severity, often with complex relationships over time.

DISCUSSION

Increasing availability of process of care data from automated electronic health records will permit better quantification of healthcare delivery strain. This could result in enhanced prediction of adverse outcomes and service delays.

CONCLUSION

New conceptual models will be needed to use these new data elements.

Development and validation of a 30-day death nomogram in patients with spontaneous cerebral hemorrhage: a retrospective cohort study

The purpose of this study was to establish and validate a nomogram to estimate the 30-day probability of death in patients with spontaneous cerebral hemorrhage. From January 2015 to December 2017, a cohort of 450 patients with clinically diagnosed cerebral hemorrhage was collected for model development. The minimum absolute contraction and the selection operator (lasso) regression model were used to select the strongest prediction of patients with cerebral hemorrhage. Discrimination and calibration were used to evaluate the performance of the resulting nomogram. After internal validation, the nomogram was further assessed in a different cohort containing 148 consecutive subjects examined between January 2018 and December 2018. The nomogram included five predictors from the lasso regression analysis, including: Glasgow coma scale (GCS), hematoma location, hematoma volume, white blood cells, and D-dimer. Internal verification showed that the model had good discrimination, (the area under the curve is 0.955), and good calibration [unreliability (U) statistic, p = 0.739]. The nomogram still showed good discrimination (area under the curve = 0.888) and good calibration [U statistic, p = 0.926] in the verification cohort data. Decision curve analysis showed that the prediction nomogram was clinically useful. The current study delineates a predictive nomogram combining clinical and imaging features, which can help identify patients who may die of cerebral hemorrhage.

Disease evolution and risk-based disease trajectories in congestive heart failure patients

Congestive Heart Failure (CHF) is among the most prevalent chronic diseases worldwide, and is commonly associated with comorbidities and complex health conditions. Consequently, CHF patients are typically hospitalized frequently, and are at a high risk of premature death. Early detection of an envisaged patient disease trajectory is crucial for precision medicine. However, despite the abundance of patient-level data, cardiologists currently struggle to identify disease trajectories and track the evolution patterns of the disease over time, especially in small groups of patients with specific disease subtypes. The present study proposed a five-step method that allows clustering CHF patients, detecting cluster similarity, and identifying disease trajectories, and promises to overcome the existing difficulties. This work is based on a rich dataset of patients' records spanning ten years of hospital visits. The dataset contains all the health information documented in the hospital during each visit, including diagnoses, lab results, clinical data, and demographics. It utilizes an innovative Cluster Evolution Analysis (CEA) method to analyze the complex CHF population where each subject is potentially associated with numerous variables. We have defined sub-groups for mortality risk levels, which we used to characterize patients' disease evolution by refined data clustering in three points in time over ten years, and generating patients' migration patterns across periods. The results elicited 18, 23, and 25 clusters respective to the first, second, and third visits, uncovering clinically interesting small sub-groups of patients. In the following post-processing stage, we identified meaningful patterns. The analysis yielded fine-grained patient clusters divided into several finite risk levels, including several small-sized groups of high-risk patients. Significantly, the analysis also yielded longitudinal patterns where patients' risk levels changed over time. Four types of disease trajectories were identified: decline, preserved state, improvement, and mixed-progress. This stage is a unique contribution of the work. The resulting fine partitioning and longitudinal insights promise to significantly assist cardiologists in tailoring personalized interventions to improve care quality. Cardiologists could utilize these results to glean previously undetected relationships between symptoms and disease evolution that would allow a more informed clinical decision-making and effective interventions.

A critical appraisal of the prognostic predictive models for patients with sepsis: Which model can be applied in clinical practice?

BACKGROUND

Sepsis is associated with high mortality and predictive models can help in clinical decision-making. The objective of this study was to carry out a systematic review of these models.

METHODS

In 2019, we conducted a systematic review in MEDLINE and EMBASE (CDR42018111121:PROSPERO) of articles that developed predictive models for mortality in septic patients (inclusion criteria). We followed the CHARMS recommendations (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies), extracting the information from its 11 domains (Source of data, Participants, etc). We determined the risk of bias and applicability (participants, outcome, predictors and analysis) through PROBAST (Prediction model Risk Of Bias ASsessment Tool).

RESULTS

A total of 14 studies were included. In the CHARMS extraction, the models found showed great variability in its 11 domains. Regarding the PROBAST checklist, only one article had an unclear risk of bias as it did not indicate how missing data were handled while the others all had a high risk of bias. This was mainly due to the statistical analysis (inadequate sample size, handling of continuous predictors, missing data and selection of predictors), since 13 studies had a high risk of bias. Applicability was satisfactory in six articles. Most of the models integrate predictors from routine clinical practice. Discrimination and calibration were assessed for almost all the models, with the area under the ROC curve ranging from 0.59 to 0.955 and no lack of calibration. Only three models were externally validated and their maximum discrimination values in the derivation were from 0.712 and 0.84. One of them (Osborn) had undergone multiple validation studies.

DISCUSSION

Despite most of the studies showing a high risk of bias, we very cautiously recommend applying the Osborn model, as this has been externally validated various times.

Effects of Neighborhood-level Data on Performance and Algorithmic Equity of a Model That Predicts 30-day Heart Failure Readmissions at an Urban Academic Medical Center

BACKGROUND

Socioeconomic data may improve predictions of clinical events. However, owing to structural racism, algorithms may not perform equitably across racial subgroups. Therefore, we sought to compare the predictive performance overall, and by racial subgroup, of commonly used predictor variables for heart failure readmission with and without the area deprivation index (ADI), a neighborhood-level socioeconomic measure.

METHODS AND RESULTS

We conducted a retrospective cohort study of 1316 Philadelphia residents discharged with a primary diagnosis of congestive heart failure from the University of Pennsylvania Health System between April 1, 2015, and March 31, 2017. We trained a regression model to predict the probability of a 30-day readmission using clinical and demographic variables. A second model also included the ADI as a predictor variable. We measured predictive performance with the Brier Score (BS) in a held-out test set. The baseline model had moderate performance overall (BS 0.13, 95% CI 0.13-0.14), and among White (BS 0.12, 95% CI 0.12-0.13) and non-White (BS 0.13, 95% CI 0.13-0.14) patients. Neither performance nor algorithmic equity were significantly changed with the addition of the ADI.

CONCLUSIONS

The inclusion of neighborhood-level data may not reliably improve performance or algorithmic equity.

Assessment of Value of Neighborhood Socioeconomic Status in Models That Use Electronic Health Record Data to Predict Health Care Use Rates and Mortality

Importance

Prediction models are widely used in health care as a way of risk stratifying populations for targeted intervention. Most risk stratification has been done using a small number of predictors from insurance claims. However, the utility of diverse nonclinical predictors, such as neighborhood socioeconomic contexts, remains unknown.

Objective

To assess the value of using neighborhood socioeconomic predictors in the context of 1-year risk prediction for mortality and 6 different health care use outcomes in a large integrated care system.

Design, Setting, and Participants

Diagnostic study using data from all adults age 18 years or older who had Kaiser Foundation Health Plan membership and/or use in the Kaiser Permantente Northern California: a multisite, integrated health care delivery system between January 1, 2013, and June 30, 2014. Data were recorded before the index date for each patient to predict their use and mortality in a 1-year post period using a test-train split for model training and evaluation. Analyses were conducted in fall of 2019.

Main Outcomes and Measures

One-year encounter counts (doctor office, virtual, emergency department, elective hospitalizations, and nonelective), total costs, and mortality.

Results

A total of 2 951 588 patients met inclusion criteria (mean [SD] age, 47.2 [17.4] years; 47.8% were female). The mean (SD) Neighborhood Deprivation Index was -0.32 (0.84). The areas under the receiver operator curve ranged from 0.71 for emergency department use (using the LASSO method and electronic health record predictors) to 0.94 for mortality (using the random forest method and electronic health record predictors). Neighborhood socioeconomic status predictors did not meaningfully increase the predictive performance of the models for any outcome.

Conclusions and Relevance

In this study, neighborhood socioeconomic predictors did not improve risk estimates compared with what is obtainable using standard claims data regardless of model used.

Processes of Care Associated With Risk of Mortality and Recurrent Stroke Among Patients With Transient Ischemic Attack and Nonsevere Ischemic Stroke

IMPORTANCE

Early evaluation and management of patients with transient ischemic attack (TIA) and nonsevere ischemic stroke improves outcomes.

OBJECTIVE

To identify processes of care associated with reduced risk of death or recurrent stroke among patients with TIA or nonsevere ischemic stroke.

DESIGN, SETTING, AND PARTICIPANTS

This cohort study included all patients with TIA or nonsevere ischemic stroke at Department of Veterans Affairs emergency department or inpatient settings from October 2010 to September 2011. Multivariable logistic regression was used to model associations of processes of care and without-fail care, defined as receiving all guideline-concordant processes of care for which patients are eligible, with risk of death and recurrent stroke. Data were analyzed from March 2018 to April 2019.

MAIN OUTCOMES AND MEASURES

Risk of all-cause mortality and recurrent ischemic stroke at 90 days and 1 year was calculated. Overall, 28 processes of care were examined. Without-fail care was assessed for 6 processes: brain imaging, carotid artery imaging, hypertension medication intensification, high- or moderate-potency statin therapy, antithrombotics, and anticoagulation for atrial fibrillation.

RESULTS

Among 8076 patients, the mean (SD) age was 67.8 (11.6) years, 7752 patients (96.0%) were men, 5929 (73.4%) were white, 474 (6.1%) had a recurrent ischemic stroke within 90 days, 793 (10.7%) had a recurrent ischemic stroke within 1 year, 320 (4.0%) died within 90 days, and 814 (10.1%) died within 1 year. Overall, 9 processes were independently associated with lower odds of both 90-day and 1-year mortality after adjustment for multiple comparisons: carotid artery imaging (90-day adjusted odds ratio [aOR], 0.49; 95% CI, 0.38-0.63; 1-year aOR, 0.61; 95% CI, 0.52-0.72), antihypertensive medication class (90-day aOR, 0.58; 95% CI, 0.45-0.74; 1-year aOR, 0.70; 95% CI, 0.60-0.83), lipid measurement (90-day aOR, 0.68; 95% CI, 0.51-0.90; 1-year aOR, 0.64; 95% CI, 0.53-0.78), lipid management (90-day aOR, 0.46; 95% CI, 0.33-0.65; 1-year aOR, 0.67; 95% CI, 0.53-0.85), discharged receiving statin medication (90-day aOR, 0.51; 95% CI, 0.36-0.73; 1-year aOR, 0.70; 95% CI, 0.55-0.88), cholesterol-lowering medication intensification (90-day aOR, 0.47; 95% CI, 0.26-0.83; 1-year aOR, 0.56; 95% CI, 0.41-0.77), antithrombotics by day 2 (90-day aOR, 0.56; 95% CI, 0.40-0.79; 1-year aOR, 0.69; 95% CI, 0.55-0.87) or at discharge (90-day aOR, 0.59; 95% CI, 0.41-0.86; 1-year aOR, 0.69; 95% CI, 0.54-0.88), and neurology consultation (90-day aOR, 0.67; 95% CI, 0.52-0.87; 1-year aOR, 0.74; 95% CI, 0.63-0.87). Anticoagulation for atrial fibrillation was associated with lower odds of 1-year mortality only (aOR, 0.59; 95% CI, 0.40-0.85). No processes were associated with reduced risk of recurrent stroke after adjustment for multiple comparisons. The rate of without-fail care was 15.3%; 1216 patients received all guideline-concordant processes of care for which they were eligible. Without-fail care was associated with a 31.2% lower odds of 1-year mortality (aOR, 0.69; 95% CI, 0.55-0.87) but was not independently associated with stroke risk.

CONCLUSIONS AND RELEVANCE

Patients who received 6 readily available processes of care had lower adjusted mortality 1 year after TIA or nonsevere ischemic stroke. Clinicians caring for patients with TIA and nonsevere ischemic stroke should seek to ensure that patients receive all guideline-concordant processes of care for which they are eligible.

Development and validation of risk-adjustment models for elective, single-level posterior lumbar spinal fusions

Background

There is a paucity of literature examining the development and subsequent validation of risk-adjustment models that inform the trade-off between adequate risk-adjustment and data collection burden. We aimed to evaluate patient risk stratification by surgeons with the development and validation of risk-adjustment models for elective, single-level, posterior lumbar spinal fusions (PLSFs).

Methods

Patients undergoing PLSF from 2011-2014 were identified in the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). The derivation cohort included patients from 2011-2013, while the validation cohort included patients from 2014. Outcomes of interest were severe adverse events (SAEs) and unplanned readmission. Bivariate analysis of risk factors followed by a stepwise logistic regression model was used. Limited risk-adjustment models were created and analyzed by sequentially adding variables until the full model was reached.

Results

A total of 7,192 and 4,182 patients were included in our derivation and validation cohorts, respectively. Full model performance was similar for the derivation and validation cohorts in both 30-day SAEs (C-statistic =0.66 vs. 0.69) and 30-day unplanned readmission (C-statistic =0.62 vs. 0.65). All models demonstrated good calibration and fit (P≥0.58). Intraoperative variables, laboratory values, and comorbid conditions explained >75% of the variation in 30-day SAEs; ASA class, laboratory values, and comorbid conditions accounted for >80% of model risk prediction for 30-day unplanned readmission. Four variables for the 30-day SAE models (age, gender, ASA ≥3, operative time) and 3 variables for the 30-day unplanned readmission models (age, ASA ≥3, operative time) were sufficient to achieve a C-statistic within four percentage points of the full model.

Conclusions

Risk-adjustment models for PLSF demonstrated acceptable calibration and discrimination using variables commonly found in health records and demonstrated only a limited set of variables were required to achieve an appropriate level of risk prediction.

Improving palliative care with deep learning

BACKGROUND

Access to palliative care is a key quality metric which most healthcare organizations strive to improve. The primary challenges to increasing palliative care access are a combination of physicians over-estimating patient prognoses, and a shortage of palliative staff in general. This, in combination with treatment inertia can result in a mismatch between patient wishes, and their actual care towards the end of life.

METHODS

In this work, we address this problem, with Institutional Review Board approval, using machine learning and Electronic Health Record (EHR) data of patients. We train a Deep Neural Network model on the EHR data of patients from previous years, to predict mortality of patients within the next 3-12 month period. This prediction is used as a proxy decision for identifying patients who could benefit from palliative care.

RESULTS

The EHR data of all admitted patients are evaluated every night by this algorithm, and the palliative care team is automatically notified of the list of patients with a positive prediction. In addition, we present a novel technique for decision interpretation, using which we provide explanations for the model's predictions.

CONCLUSION

The automatic screening and notification saves the palliative care team the burden of time consuming chart reviews of all patients, and allows them to take a proactive approach in reaching out to such patients rather then relying on referrals from the treating physicians.

Prognostic models for intracerebral hemorrhage: systematic review and meta-analysis

Background

Prognostic tools for intracerebral hemorrhage (ICH) patients are potentially useful for ascertaining prognosis and recommended in guidelines to facilitate streamline assessment and communication between providers. In this systematic review with meta-analysis we identified and characterized all existing prognostic tools for this population, performed a methodological evaluation of the conducting and reporting of such studies and compared different methods of prognostic tool derivation in terms of discrimination for mortality and functional outcome prediction.

Methods

PubMed, ISI, Scopus and CENTRAL were searched up to 15th September 2016, with additional studies identified using reference check. Two reviewers independently extracted data regarding the population studied, process of tool derivation, included predictors and discrimination (c statistic) using a predesignated spreadsheet based in the CHARMS checklist. Disagreements were solved by consensus. C statistics were pooled using robust variance estimation and meta-regression was applied for group comparisons using random effect models.

Results

Fifty nine studies were retrieved, including 48,133 patients and reporting on the derivation of 72 prognostic tools. Data on discrimination (c statistic) was available for 53 tools, 38 focusing on mortality and 15 focusing on functional outcome. Discrimination was high for both outcomes, with a pooled c statistic of 0.88 for mortality and 0.87 for functional outcome. Forty three tools were regression based and nine tools were derived using machine learning algorithms, with no differences found between the two methods in terms of discrimination (p = 0.490). Several methodological issues however were identified, relating to handling of missing data, low number of events per variable, insufficient length of follow-up, absence of blinding, infrequent use of internal validation, and underreporting of important model performance measures.

Conclusions

Prognostic tools for ICH discriminated well for mortality and functional outcome in derivation studies but methodological issues require confirmation of these findings in validation studies. Logistic regression based risk scores are particularly promising given their good performance and ease of application.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0613-8) contains supplementary material, which is available to authorized users.

Comparing Hospital Processes and Outcomes in California Medicare Beneficiaries: Simulation Prompts Reconsideration

INTRODUCTION

This article is not a traditional research report. It describes how conducting a specific set of benchmarking analyses led us to broader reflections on hospital benchmarking. We reexamined an issue that has received far less attention from researchers than in the past: How variations in the hospital admission threshold might affect hospital rankings. Considering this threshold made us reconsider what benchmarking is and what future benchmarking studies might be like. Although we recognize that some of our assertions are speculative, they are based on our reading of the literature and previous and ongoing data analyses being conducted in our research unit. We describe the benchmarking analyses that led to these reflections.

OBJECTIVES

The Centers for Medicare and Medicaid Services' Hospital Compare Web site includes data on fee-for-service Medicare beneficiaries but does not control for severity of illness, which requires physiologic data now available in most electronic medical records.To address this limitation, we compared hospital processes and outcomes among Kaiser Permanente Northern California's (KPNC) Medicare Advantage beneficiaries and non-KPNC California Medicare beneficiaries between 2009 and 2010.

METHODS

We assigned a simulated severity of illness measure to each record and explored the effect of having the additional information on outcomes.

RESULTS

We found that if the admission severity of illness in non-KPNC hospitals increased, KPNC hospitals' mortality performance would appear worse; conversely, if admission severity at non-KPNC hospitals' decreased, KPNC hospitals' performance would appear better.

CONCLUSION

Future hospital benchmarking should consider the impact of variation in admission thresholds.

Development of a Risk Tool to Support Discussions of Care for Older Adults Admitted to the ICU With Pneumonia

BACKGROUND

Early, data-driven discussion surrounding palliative care can improve care delivery and patient experience.

OBJECTIVE

To develop a 30-day mortality prediction tool for older patients in intensive care unit (ICU) with pneumonia that will initiate palliative care earlier in hospital course.

DESIGN

Retrospective Electronic Health Record (EHR) review.

SETTING

Four urban and suburban hospitals in a Western New York hospital system.

PARTICIPANTS

A total of 1237 consecutive patients (>75 years) admitted to the ICU with pneumonia from July 2011 to December 2014.

MEASUREMENTS

Data abstracted included demographics, insurance type, comorbidities, and clinical factors. Thirty-day mortality was also determined. Logistic regression identified predictors of 30-day mortality. Area under the receiver operating curve (ROC) was calculated to quantify the degree to which the model accurately classified participants. Using the coordinates of the ROC, a predicted probability was identified to indicate high risk.

RESULTS

A total of 1237 patients were included with 30-day mortality data available for 100% of patients. The mortality rate equaled 14.3%. Age >85 years, having active cancer, Congestive Heart Failure (CHF), Chronic Obstructive Pulmonary Disease (COPD), sepsis, and being on a vasopressor all predicted mortality. Using the derived index, with a predicted probability of mortality >0.146 as a cutoff, sensitivity equaled 70.6% and specificity equaled 65.6%. The area under the ROC was 0.735.

CONCLUSION

Our risk tool can help care teams make more informed decisions among care options by identifying a patient group for whom a careful review of goals of care is indicated both during and after hospitalization. External validation and further refinement of the index with a larger sample will improve prognostic value.

Predictive modeling of inpatient mortality in departments of internal medicine

Despite overwhelming data on predictors of inpatient mortality, it is unclear which variables are the most instructive in predicting mortality of patients in departments of internal medicine. This study aims to identify the most informative predictors of inpatient mortality, and builds a prediction model on an individual level, given a constellation of patient characteristics. We use a penalized method for developing the prediction model by applying the least-absolute-shrinkage and selection-operator regression. We utilize a cohort of adult patients admitted to any of 5 departments of internal medicine during 3.5 years. We integrated data from electronic health records that included clinical, epidemiological, administrative, and laboratory variables. The prediction model was evaluated using the validation sample. Of 10,788 patients hospitalized during the study period, 874 (8.1%) died during admission. We find that the strongest predictors of inpatient mortality are prior admission within 3 months, malignant morbidity, serum creatinine levels, and hypoalbuminemia at hospital admission, and an admitting diagnosis of sepsis, pneumonia, malignant neoplastic disease, or cerebrovascular disease. The C-statistic of the risk prediction model is 89.4% (95% CI 88.4-90.4%). The predictive performance of this model is better than a multivariate stepwise logistic regression model. By utilizing the prediction model, the AUC for the independent (validation) data set is 85.7% (95% CI 84.1-87.3%). Using penalized regression, this prediction model identifies the most informative predictors of inpatient mortality. The model illustrates the potential value and feasibility of a tool that can aid physicians in decision-making.

Derivation and Validation of an In-Hospital Mortality Prediction Model Suitable for Profiling Hospital Performance in Heart Failure

BACKGROUND

Comparing heart failure (HF) outcomes across hospitals requires adequate risk adjustment. We aimed to develop and validate a model that can be used to compare quality of HF care across hospitals.

METHODS AND RESULTS

We included patients with HF aged ≥18 years admitted to one of 433 hospitals that participated in the Premier Inc Data Warehouse. This model (Premier) contained patient demographics, comorbidities, and acute conditions present on admission, derived from administrative and billing records. In a separate data set derived from electronic health records, we validated the Premier model by comparing hospital risk-standardized mortality rates calculated with the Premier model to those calculated with a validated clinical model containing laboratory data (LAPS [Laboratory-Based Acute Physiology Score]). Among the 200 832 admissions in the Premier Inc Data Warehouse, inpatient mortality was 4.0%. The model showed acceptable discrimination in the warehouse data (C statistic 0.75; 95% confidence interval, 0.74-0.76). In the validation data set, both the Premier model and the LAPS models showed acceptable discrimination (C statistic: Premier: 0.76 [95% confidence interval, 0.74-0.77]; LAPS: 0.78 [95% confidence interval, 0.76-0.80]). Risk-standardized mortality rates for both models ranged from 2% to 7%. A linear regression equation describing the association between Premier- and LAPS-specific mortality rates revealed a regression line with a slope of 0.71 (SE: 0.07). The correlation coefficient of the standardized mortality rates from the 2 models was 0.82.

CONCLUSIONS

Compared with a validated model derived from clinical data, an HF mortality model derived from administrative data showed highly correlated risk-standardized mortality rate estimates, suggesting it could be used to identify high- and low-performing hospitals for HF care.

A self-assessment predictive model for type 2 diabetes or impaired fasting glycaemia derived from a population-based survey

AIMS

There is no cure for diabetes and its prevention is interesting for both people and health policy makers. The aim of this study was to construct a simple scoring system to predict diabetes and suggest a self assessment predictive model for type 2 diabetes in Iran.

METHODS

This study was a part of a comprehensive population based survey performed in Ilam province during 2011-2012, including 2158 cases≥25years. All demographic and laboratory results were entered into the prepared sheets and were analysed using SPSS 16. By identification of relative risks of diabetes and IFG, a predictive model was constructed and proposed for these abnormalities.

RESULTS

Totally, 2158 people comprising 72% female, 60% from urban regions, mean age of 45.5±14years were investigated and the average height, weight, FBS and waist of participants were as follows respectively: 164±8.9cm, 68.4±12.3kg, 5.7±2.8mmol/l (102.6±49.9mg/dl) and 82.3±14.3cm. The prevalence of IFG, diabetes and hyperglycaemia among all participants were 7.8%, 11.8% and 19.6% respectively. Regression analysis revealed familial history of diabetes, place of life, age, hypertension, daily exercise, marital status, gender, waist size, smoking, and BMI as the most relevant risk factors for diabetes and hyperglycemia.

CONCLUSION

A self-assessment predictive model was constructed for general population living in the west of Iran. This is the first self-assessment predictive model for diabetes in Iran.

Does adding clinical data to administrative data improve agreement among hospital quality measures?

BACKGROUND

Hospital performance measures based on patient mortality and readmission have indicated modest rates of agreement. We examined if combining clinical data on laboratory tests and vital signs with administrative data leads to improved agreement with each other, and with other measures of hospital performance in the nation's largest integrated health care system.

METHODS

We used patient-level administrative and clinical data, and hospital-level data on quality indicators, for 2007-2010 from the Veterans Health Administration (VA). For patients admitted for acute myocardial infarction (AMI), heart failure (HF) and pneumonia we examined changes in hospital performance on 30-d mortality and 30-d readmission rates as a result of adding clinical data to administrative data. We evaluated whether this enhancement yielded improved measures of hospital quality, based on concordance with other hospital quality indicators.

RESULTS

For 30-d mortality, data enhancement improved model performance, and significantly changed hospital performance profiles; for 30-d readmission, the impact was modest. Concordance between enhanced measures of both outcomes, and with other hospital quality measures - including Joint Commission process measures, VA Surgical Quality Improvement Program (VASQIP) mortality and morbidity, and case volume - remained poor.

CONCLUSIONS

Adding laboratory tests and vital signs to measure hospital performance on mortality and readmission did not improve the poor rates of agreement across hospital quality indicators in the VA.

INTERPRETATION

Efforts to improve risk adjustment models should continue; however, evidence of validation should precede their use as reliable measures of quality.

Validation and Comparison of Seven Mortality Prediction Models for Hospitalized Patients With Acute Decompensated Heart Failure

BACKGROUND

Heart failure (HF) inpatient mortality prediction models can help clinicians make treatment decisions and researchers conduct observational studies; however, published models have not been validated in external populations.

METHODS AND RESULTS

We compared the performance of 7 models that predict inpatient mortality in patients hospitalized with acute decompensated heart failure: 4 HF-specific mortality prediction models developed from 3 clinical databases (ADHERE [Acute Decompensated Heart Failure National Registry], EFFECT study [Enhanced Feedback for Effective Cardiac Treatment], and GWTG-HF registry [Get With the Guidelines-Heart Failure]); 2 administrative HF mortality prediction models (Premier, Premier+); and a model that uses clinical data but is not specific for HF (Laboratory-Based Acute Physiology Score [LAPS2]). Using a multihospital, electronic health record-derived data set (HealthFacts [Cerner Corp], 2010-2012), we identified patients ≥18 years admitted with HF. Of 13 163 eligible patients, median age was 74 years; half were women; and 27% were black. In-hospital mortality was 4.3%. Model-predicted mortality ranges varied: Premier+ (0.8%-23.1%), LAPS2 (0.7%-19.0%), ADHERE (1.2%-17.4%), EFFECT (1.0%-12.8%), GWTG-Eapen (1.2%-13.8%), and GWTG-Peterson (1.1%-12.8%). The LAPS2 and Premier models outperformed the clinical models (C statistics: LAPS2 0.80 [95% confidence interval 0.78-0.82], Premier models 0.81 [95% confidence interval 0.79-0.83] and 0.76 [95% confidence interval 0.74-0.78], and clinical models 0.68 to 0.70).

CONCLUSIONS

Four clinically derived, inpatient, HF mortality models exhibited similar performance, with C statistics near 0.70. Three other models, 1 developed in electronic health record data and 2 developed in administrative data, also were predictive, with C statistics from 0.76 to 0.80. Because every model performed acceptably, the decision to use a given model should depend on practical concerns and intended use.

Predicting In-Hospital Mortality in Patients With Acute Myocardial Infarction

BACKGROUND

As a foundation for quality improvement, assessing clinical outcomes across hospitals requires appropriate risk adjustment to account for differences in patient case mix, including presentation after cardiac arrest.

OBJECTIVES

The aim of this study was to develop and validate a parsimonious patient-level clinical risk model of in-hospital mortality for contemporary patients with acute myocardial infarction.

METHODS

Patient characteristics at the time of presentation in the ACTION (Acute Coronary Treatment and Intervention Outcomes Network) Registry-GWTG (Get With the Guidelines) database from January 2012 through December 2013 were used to develop a multivariate hierarchical logistic regression model predicting in-hospital mortality. The population (243,440 patients from 655 hospitals) was divided into a 60% sample for model derivation, with the remaining 40% used for model validation. A simplified risk score was created to enable prospective risk stratification in clinical care.

RESULTS

The in-hospital mortality rate was 4.6%. Age, heart rate, systolic blood pressure, presentation after cardiac arrest, presentation in cardiogenic shock, presentation in heart failure, presentation with ST-segment elevation myocardial infarction, creatinine clearance, and troponin ratio were all independently associated with in-hospital mortality. The C statistic was 0.88, with good calibration. The model performed well in subgroups based on age; sex; race; transfer status; and the presence of diabetes mellitus, renal dysfunction, cardiac arrest, cardiogenic shock, and ST-segment elevation myocardial infarction. Observed mortality rates varied substantially across risk groups, ranging from 0.4% in the lowest risk group (score <30) to 49.5% in the highest risk group (score >59).

CONCLUSIONS

This parsimonious risk model for in-hospital mortality is a valid instrument for risk adjustment and risk stratification in contemporary patients with acute myocardial infarction.

The derivation and validation of a simple model for predicting in-hospital mortality of acutely admitted patients to internal medicine wards

Limited information is available about clinical predictors of in-hospital mortality in acute unselected medical admissions. Such information could assist medical decision-making.To develop a clinical model for predicting in-hospital mortality in unselected acute medical admissions and to test the impact of secondary conditions on hospital mortality.This is an analysis of the medical records of patients admitted to internal medicine wards at one university-affiliated hospital. Data obtained from the years 2013 to 2014 were used as a derivation dataset for creating a prediction model, while data from 2015 was used as a validation dataset to test the performance of the model. For each admission, a set of clinical and epidemiological variables was obtained. The main diagnosis at hospitalization was recorded, and all additional or secondary conditions that coexisted at hospital admission or that developed during hospital stay were considered secondary conditions.The derivation and validation datasets included 7268 and 7843 patients, respectively. The in-hospital mortality rate averaged 7.2%. The following variables entered the final model; age, body mass index, mean arterial pressure on admission, prior admission within 3 months, background morbidity of heart failure and active malignancy, and chronic use of statins and antiplatelet agents. The c-statistic (ROC-AUC) of the prediction model was 80.5% without adjustment for main or secondary conditions, 84.5%, with adjustment for the main diagnosis, and 89.5% with adjustment for the main diagnosis and secondary conditions. The accuracy of the predictive model reached 81% on the validation dataset.A prediction model based on clinical data with adjustment for secondary conditions exhibited a high degree of prediction accuracy. We provide a proof of concept that there is an added value for incorporating secondary conditions while predicting probabilities of in-hospital mortality. Further improvement of the model performance and validation in other cohorts are needed to aid hospitalists in predicting health outcomes.

Hospital Readmission and Social Risk Factors Identified from Physician Notes

OBJECTIVE

To evaluate the prevalence of seven social factors using physician notes as compared to claims and structured electronic health records (EHRs) data and the resulting association with 30-day readmissions.

STUDY SETTING

A multihospital academic health system in southeastern Massachusetts.

STUDY DESIGN

An observational study of 49,319 patients with cardiovascular disease admitted from January 1, 2011, to December 31, 2013, using multivariable logistic regression to adjust for patient characteristics.

DATA COLLECTION/EXTRACTION METHODS

All-payer claims, EHR data, and physician notes extracted from a centralized clinical registry.

PRINCIPAL FINDINGS

All seven social characteristics were identified at the highest rates in physician notes. For example, we identified 14,872 patient admissions with poor social support in physician notes, increasing the prevalence from 0.4 percent using ICD-9 codes and structured EHR data to 16.0 percent. Compared to an 18.6 percent baseline readmission rate, risk-adjusted analysis showed higher readmission risk for patients with housing instability (readmission rate 24.5 percent; p < .001), depression (20.6 percent; p < .001), drug abuse (20.2 percent; p = .01), and poor social support (20.0 percent; p = .01).

CONCLUSIONS

The seven social risk factors studied are substantially more prevalent than represented in administrative data. Automated methods for analyzing physician notes may enable better identification of patients with social needs.

Predicting Readmission at Early Hospitalization Using Electronic Clinical Data: An Early Readmission Risk Score

BACKGROUND

Identifying patients at high risk for readmission early during hospitalization may aid efforts in reducing readmissions. We sought to develop an early readmission risk predictive model using automated clinical data available at hospital admission.

METHODS

We developed an early readmission risk model using a derivation cohort and validated the model with a validation cohort. We used a published Acute Laboratory Risk of Mortality Score as an aggregated measure of clinical severity at admission and the number of hospital discharges in the previous 90 days as a measure of disease progression. We then evaluated the administrative data-enhanced model by adding principal and secondary diagnoses and other variables. We examined the c-statistic change when additional variables were added to the model.

RESULTS

There were 1,195,640 adult discharges from 70 hospitals with 39.8% male and the median age of 63 years (first and third quartile: 43, 78). The 30-day readmission rate was 11.9% (n=142,211). The early readmission model yielded a graded relationship of readmission and the Acute Laboratory Risk of Mortality Score and the number of previous discharges within 90 days. The model c-statistic was 0.697 with good calibration. When administrative variables were added to the model, the c-statistic increased to 0.722.

CONCLUSIONS

Automated clinical data can generate a readmission risk score early at hospitalization with fair discrimination. It may have applied value to aid early care transition. Adding administrative data increases predictive accuracy. The administrative data-enhanced model may be used for hospital comparison and outcome research.

Risk-adjustment models for heart failure patients' 30-day mortality and readmission rates: the incremental value of clinical data abstracted from medical charts beyond hospital discharge record

Background

Hospital discharge records (HDRs) are routinely used to assess outcomes of care and to compare hospital performance for heart failure. The advantages of using clinical data from medical charts to improve risk-adjustment models remain controversial. The aim of the present study was to evaluate the additional contribution of clinical variables to HDR-based 30-day mortality and readmission models in patients with heart failure.

Methods

This retrospective observational study included all patients residing in the Local Healthcare Authority of Bologna (about 1 million inhabitants) who were discharged in 2012 from one of three hospitals in the area with a diagnosis of heart failure. For each study outcome, we compared the discrimination of the two risk-adjustment models (i.e., HDR-only model and HDR-clinical model) through the area under the ROC curve (AUC).

Results

A total of 1145 and 1025 patients were included in the mortality and readmission analyses, respectively. Adding clinical data significantly improved the discrimination of the mortality model (AUC = 0.84 vs. 0.73, p < 0.001), but not the discrimination of the readmission model (AUC = 0.65 vs. 0.63, p = 0.08).

Conclusions

We identified clinical variables that significantly improved the discrimination of the HDR-only model for 30-day mortality following heart failure. By contrast, clinical variables made little contribution to the discrimination of the HDR-only model for 30-day readmission.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-016-1731-9) contains supplementary material, which is available to authorized users.

Development and validation of a predictive model for detection of colorectal cancer in primary care by analysis of complete blood counts: a binational retrospective study

OBJECTIVE

The use of risk prediction models grows as electronic medical records become widely available. Here, we develop and validate a model to identify individuals at increased risk for colorectal cancer (CRC) by analyzing blood counts, age, and sex, then determine the model's value when used to supplement conventional screening.

MATERIALS AND METHODS

Primary care data were collected from a cohort of 606 403 Israelis (of whom 3135 were diagnosed with CRC) and a case control UK dataset of 5061 CRC cases and 25 613 controls. The model was developed on 80% of the Israeli dataset and validated using the remaining Israeli and UK datasets. Performance was evaluated according to the area under the curve, specificity, and odds ratio at several working points.

RESULTS

Using blood counts obtained 3-6 months before diagnosis, the area under the curve for detecting CRC was 0.82 ± 0.01 for the Israeli validation set. The specificity was 88 ± 2% in the Israeli validation set and 94 ± 1% in the UK dataset. Detecting 50% of CRC cases, the odds ratio was 26 ± 5 and 40 ± 6, respectively, for a false-positive rate of 0.5%. Specificity for 50% detection was 87 ± 2% a year before diagnosis and 85 ± 2% for localized cancers. When used in addition to the fecal occult blood test, our model enabled more than a 2-fold increase in CRC detection.

DISCUSSION

Comparable results in 2 unrelated populations suggest that the model should generally apply to the detection of CRC in other groups. The model's performance is superior to current iron deficiency anemia management guidelines, and may help physicians to identify individuals requiring additional clinical evaluation.

CONCLUSIONS

Our model may help to detect CRC earlier in clinical practice.

A Focus Group Exploration of Automated Case-Finders to Identify High-Risk Heart Failure Patients Within an Urban Safety Net Hospital

Background:

Leveraging “big data” as a means of informing cost-effective care holds potential in triaging high-risk heart failure (HF) patients for interventions within hospitals seeking to reduce 30-day readmissions.

Objective:

Explore provider’s beliefs and perceptions about using an electronic health record (EHR)-based tool that uses unstructured clinical notes to risk-stratify high-risk heart failure patients.

Methods:

Six providers from an inpatient HF clinic within an urban safety net hospital were recruited to participate in a semistructured focus group. A facilitator led a discussion on the feasibility and value of using an EHR tool driven by unstructured clinical notes to help identify high-risk patients. Data collected from transcripts were analyzed using a thematic analysis that facilitated drawing conclusions clustered around categories and themes.

Results:

From six categories emerged two themes: (1) challenges of finding valid and accurate results, and (2) strategies used to overcome these challenges. Although employing a tool that uses electronic medical record (EMR) unstructured text as the benchmark by which to identify high-risk patients is efficient, choosing appropriate benchmark groups could be challenging given the multiple causes of readmission. Strategies to mitigate these challenges include establishing clear selection criteria to guide benchmark group composition, and quality outcome goals for the hospital.

Conclusion:

Prior to implementing into practice an innovative EMR-based case-finder driven by unstructured clinical notes, providers are advised to do the following: (1) define patient quality outcome goals, (2) establish criteria by which to guide benchmark selection, and (3) verify the tool’s validity and reliability. Achieving consensus on these issues would be necessary for this innovative EHR-based tool to effectively improve clinical decision-making and in turn, decrease readmissions for high-risk patients.

Accuracy and Calibration of Computational Approaches for Inpatient Mortality Predictive Modeling

Electronic Health Record (EHR) data can be a key resource for decision-making support in clinical practice in the "big data" era. The complete database from early 2012 to late 2015 involving hospital admissions to Inselspital Bern, the largest Swiss University Hospital, was used in this study, involving over 100,000 admissions. Age, sex, and initial laboratory test results were the features/variables of interest for each admission, the outcome being inpatient mortality. Computational decision support systems were utilized for the calculation of the risk of inpatient mortality. We assessed the recently proposed Acute Laboratory Risk of Mortality Score (ALaRMS) model, and further built generalized linear models, generalized estimating equations, artificial neural networks, and decision tree systems for the predictive modeling of the risk of inpatient mortality. The Area Under the ROC Curve (AUC) for ALaRMS marginally corresponded to the anticipated accuracy (AUC = 0.858). Penalized logistic regression methodology provided a better result (AUC = 0.872). Decision tree and neural network-based methodology provided even higher predictive performance (up to AUC = 0.912 and 0.906, respectively). Additionally, decision tree-based methods can efficiently handle Electronic Health Record (EHR) data that have a significant amount of missing records (in up to >50% of the studied features) eliminating the need for imputation in order to have complete data. In conclusion, we show that statistical learning methodology can provide superior predictive performance in comparison to existing methods and can also be production ready. Statistical modeling procedures provided unbiased, well-calibrated models that can be efficient decision support tools for predicting inpatient mortality and assigning preventive measures.

A Simple and Powerful Risk-Adjustment Tool for 30-day Mortality Among Inpatients

BACKGROUND

Risk adjustment for mortality is increasingly important in an era when hospitals and health care systems are being compared with respect to health outcomes and quality. A powerful predictive model has been developed to risk-adjust for 30-day mortality among inpatients, but it is complex and not widely used.

OBJECTIVE

To develop and validate a simpler model, with predictive power similar to more complex models.

RESEARCH DESIGN

This was a retrospective split-validation study. In a derivation cohort, a predictive model for 30-day mortality was developed using logistic regression with the Charlson comorbidity score, Laboratory-Based Acute Physiology Score, and age as the predictor variables. In the validation cohort, the performance and calibration of the model to predict 30-day mortality was examined.

SUBJECTS

All admissions to the medical service of 2 urban university-based teaching hospitals located in Bronx, New York, between July 1, 2002, and April 30, 2008.

MEASURES

All-cause mortality was taken from the social security death registry. Predictor variables were constructed from demographic characteristics, laboratory and billing data extracted from a clinical data repository.

RESULTS

The study sample included 147 991 admissions and overall 30-day mortality was 5.4%. The model had excellent discrimination, with a c-statistics of 0.8585 in the derivation cohort and 0.8484 in the validation cohort. The model accurately predicts 30-day mortality in all risk deciles.

CONCLUSIONS

This simple and powerful predictive model can be used by hospitals and health care systems as a risk-adjustment tool for quality and research purposes.

Direct healthcare costs for patients hospitalized with Crimean-Congo haemorrhagic fever can be predicted by a clinical illness severity scoring system

Crimean-Congo hemorrhagic fever (CCHF) is endemic in Turkey, with peak incidence of hospital admissions in the summer months. The aim of this pilot study was to evaluate the role of the severity grading score (SGS) in predicting length of hospital stay, laboratory usage, need for blood products, and hence total costs of patients. Thirty-five patients admitted to one specialist center in Turkey in 2013 and 2014 with PCR-proven CCHF. The mean (SD) age was 55 (±14) and 63% of the patients were male, with 8 (22.9%) mortality. Patients were classified by SGS into three groups with mortality as follows: low risk (0/19); intermediate (6/14); and high (2/2). The direct hospital cost of these admissions was at least $41 740 with median (range) of $1210 ($97-$13 054) per patient. There was a significant difference between low-risk and combined (intermediate-high) risk groups as 635 (97-1500) and 2264.5 (154-13 054), respectively (p = 0.012). In conclusion, a clinical grading score can be used to predict illness severity and to predict associated health care costs.

Nonelective Rehospitalizations and Postdischarge Mortality: Predictive Models Suitable for Use in Real Time

BACKGROUND

Hospital discharge planning has been hampered by the lack of predictive models.

OBJECTIVE

To develop predictive models for nonelective rehospitalization and postdischarge mortality suitable for use in commercially available electronic medical records (EMRs).

DESIGN

Retrospective cohort study using split validation.

SETTING

Integrated health care delivery system serving 3.9 million members.

PARTICIPANTS

A total of 360,036 surviving adults who experienced 609,393 overnight hospitalizations at 21 hospitals between June 1, 2010 and December 31, 2013.

MAIN OUTCOME MEASURE

A composite outcome (nonelective rehospitalization and/or death within 7 or 30 days of discharge).

RESULTS

Nonelective rehospitalization rates at 7 and 30 days were 5.8% and 12.4%; mortality rates were 1.3% and 3.7%; and composite outcome rates were 6.3% and 14.9%, respectively. Using data from a comprehensive EMR, we developed 4 models that can generate risk estimates for risk of the combined outcome within 7 or 30 days, either at the time of admission or at 8 AM on the day of discharge. The best was the 30-day discharge day model, which had a c-statistic of 0.756 (95% confidence interval, 0.754-0.756) and a Nagelkerke pseudo-R of 0.174 (0.171-0.178) in the validation dataset. The most important predictors-a composite acute physiology score and end of life care directives-accounted for 54% of the predictive ability of the 30-day model. Incorporation of diagnoses (not reliably available for real-time use) did not improve model performance.

CONCLUSIONS

It is possible to develop robust predictive models, suitable for use in real time with commercially available EMRs, for nonelective rehospitalization and postdischarge mortality.

Using Administrative Mental Health Indicators in Heart Failure Outcomes Research: Comparison of Clinical Records and International Classification of Disease Coding

BACKGROUND

Use of mental indication in health outcomes research is of growing interest to researchers. This study, as part of a larger research program, quantified agreement between administrative International Classification of Disease (ICD-9) coding for, and "gold standard" clinician documentation of, mental health issues (MHIs) in hospitalized heart failure (HF) patients to determine the validity of mental health administrative data for use in HF outcomes research.

METHODS

A 13% random sample (n = 504) was selected from all unique patients (n = 3,769) hospitalized with a primary HF diagnosis at 4 San Diego County community hospitals during 2009-2012. MHI was defined as ICD-9 discharge diagnostic coding 290-319. Records were audited for clinician documentation of MHI.

RESULTS

A total of 43% (n = 216) had mental health clinician documentation; 33% (n = 164) had ICD-9 coding for MHI. ICD-9 code bundle 290-319 had 0.70 sensitivity, 0.97 specificity, and kappa 0.69 (95% confidence interval 0.61-0.79). More specific ICD-9 MHI code bundles had kappas ranging from 0.44 to 0.82 and sensitivities ranging from 42% to 82%.

CONCLUSIONS

Agreement between ICD-9 coding and clinician documentation for a broadly defined MHI is substantial, and can validly "rule in" MHI for hospitalized patients with heart failure. More specific MHI code bundles had fair to almost perfect agreement, with a wide range of sensitivities for identifying patients with an MHI.

Statewide Hospital Discharge Data: Collection, Use, Limitations, and Improvements

OBJECTIVES

To provide an overview of statewide hospital discharge databases (HDD), including their uses in health services research and limitations, and to describe Agency for Healthcare Research and Quality (AHRQ) Enhanced State Data grants to address clinical and race-ethnicity data limitations.

PRINCIPAL FINDINGS

Almost all states have statewide HDD collected by public or private data organizations. Statewide HDD, based on the hospital claim with state variations, contain useful core variables and require minimal collection burden. AHRQ's Healthcare Cost and Utilization Project builds uniform state and national research files using statewide HDD. States, hospitals, and researchers use statewide HDD for many purposes. Illustrating researchers' use, during 2012-2014, HSR published 26 HDD-based articles on health policy, access, quality, clinical aspects of care, race-ethnicity and insurance impacts, economics, financing, and research methods. HDD have limitations affecting their use. Five AHRQ grants focused on enhancing clinical data and three grants aimed at improving race-ethnicity data.

CONCLUSION

ICD-10 implementation will significantly affect the HDD. The AHRQ grants, information technology advances, payment policy changes, and the need for outpatient information may stimulate other statewide HDD changes. To remain a mainstay of health services research, statewide HDD need to keep pace with changing user needs while minimizing collection burdens.

Risk-Adjusted In-Hospital Mortality Models for Congestive Heart Failure and Acute Myocardial Infarction: Value of Clinical Laboratory Data and Race/Ethnicity

OBJECTIVE

To examine the impact of key laboratory and race/ethnicity data on the prediction of in-hospital mortality for congestive heart failure (CHF) and acute myocardial infarction (AMI).

DATA SOURCES

Hawaii adult hospitalizations database between 2009 and 2011, linked to laboratory database.

STUDY DESIGN

Cross-sectional design was employed to develop risk-adjusted in-hospital mortality models among patients with CHF (n = 5,718) and AMI (n = 5,703).

DATA COLLECTION/EXTRACTION METHODS

Results of 25 selected laboratory tests were requested from hospitals and laboratories across the state and mapped according to Logical Observation Identifiers Names and Codes standards. The laboratory data were linked to administrative data for each discharge of interest from an all-payer database, and a Master Patient Identifier was used to link patient-level encounter data across hospitals statewide.

PRINCIPAL FINDINGS

Adding a simple three-level summary measure based on the number of abnormal laboratory data observed to hospital administrative claims data significantly improved the model prediction for inpatient mortality compared with a baseline risk model using administrative data that adjusted only for age, gender, and risk of mortality (determined using 3M's All Patient Refined Diagnosis Related Groups classification). The addition of race/ethnicity also improved the model.

CONCLUSIONS

The results of this study support the incorporation of a simple summary measure of laboratory data and race/ethnicity information to improve predictions of in-hospital mortality from CHF and AMI. Laboratory data provide objective evidence of a patient's condition and therefore are accurate determinants of a patient's risk of mortality. Adding race/ethnicity information helps further explain the differences in in-hospital mortality.

Procedure-based severity index for inpatients: development and validation using administrative database

Background

Risk adjustment is important in studies using administrative databases. Although utilization of diagnostic and therapeutic procedures can represent patient severity, the usability of procedure records in risk adjustment is not well-documented. Therefore, we aimed to develop and validate a severity index calculable from procedure records.

Methods

Using the Japanese nationwide Diagnosis Procedure Combination database of acute-care hospitals, we identified patients discharged between 1 April 2012 and 31 March 2013 with an admission-precipitating diagnosis of acute myocardial infarction, congestive heart failure, acute cerebrovascular disease, gastrointestinal hemorrhage, pneumonia, or septicemia. Subjects were randomly assigned to the derivation cohort or the validation cohort. In the derivation cohort, we used multivariable logistic regression analysis to identify procedures performed on admission day which were significantly associated with in-hospital death, and a point corresponding to regression coefficient was assigned to each procedure. An index was then calculated in the validation cohort as sum of points for performed procedures, and performance of mortality-predicting model using the index and other patient characteristics was evaluated.

Results

Of the 539 385 hospitalizations included, 270 054 and 269 331 were assigned to the derivation and validation cohorts, respectively. Nineteen significant procedures were identified from the derivation cohort with points ranging from −3 to 23, producing a severity index with possible range of −13 to 69. In the validation cohort, c-statistic of mortality-predicting model was 0.767 (95 % confidence interval: 0.764–0.770). The ω-statistic representing contribution of the index relative to other variables was 1.09 (95 % confidence interval: 1.03–1.17).

Conclusions

Procedure-based severity index predicted mortality well, suggesting that procedure records in administrative database are useful for risk adjustment.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-015-0889-x) contains supplementary material, which is available to authorized users.

Predicting the risk for hospital-onset Clostridium difficile infection (HO-CDI) at the time of inpatient admission: HO-CDI risk score

OBJECTIVE

To predict the likelihood of hospital-onset Clostridium difficile infection (HO-CDI) based on patient clinical presentations at admission

DESIGN

Retrospective data analysis

SETTING

Six US acute care hospitals

PATIENTS

Adult inpatients

METHODS

We used clinical data collected at the time of admission in electronic health record (EHR) systems to develop and validate a HO-CDI predictive model. The outcome measure was HO-CDI cases identified by a nonduplicate positive C. difficile toxin assay result with stool specimens collected >48 hours after inpatient admission. We fit a logistic regression model to predict the risk of HO-CDI. We validated the model using 1,000 bootstrap simulations.

RESULTS

Among 78,080 adult admissions, 323 HO-CDI cases were identified (ie, a rate of 4.1 per 1,000 admissions). The logistic regression model yielded 14 independent predictors, including hospital community onset CDI pressure, patient age ≥65, previous healthcare exposures, CDI in previous admission, admission to the intensive care unit, albumin ≤3 g/dL, creatinine >2.0 mg/dL, bands >32%, platelets ≤150 or >420 109/L, and white blood cell count >11,000 mm3. The model had a c-statistic of 0.78 (95% confidence interval [CI], 0.76-0.81) with good calibration. Among 79% of patients with risk scores of 0-7, 19 HO-CDIs occurred per 10,000 admissions; for patients with risk scores >20, 623 HO-CDIs occurred per 10,000 admissions (P<.0001).

CONCLUSION

Using clinical parameters available at the time of admission, this HO-CDI model demonstrated good predictive ability, and it may have utility as an early risk identification tool for HO-CDI preventive interventions and outcome comparisons.

The value of adding laboratory data to coronary artery bypass grafting registry data to improve models for risk-adjusting provider mortality rates

BACKGROUND

Clinical databases are currently being used for calculating provider risk-adjusted mortality rates for coronary artery bypass grafting (CABG) in a few states and by the Society for Thoracic Surgeons. These databases contain very few laboratory data for purposes of risk adjustment.

METHODS

For 15 hospitals, New York's CABG registry data from 2008 to 2010 were linked to laboratory data to develop statistical models comparing risk-adjusted mortality rates with and without supplementary laboratory data. Differences between these two models in discrimination, calibration, and outlier status were compared, and correlations in hospital risk-adjusted mortality rates were examined.

RESULTS

The discrimination of the statistical models was very similar (c = 0.785 for the registry model and 0.797 for the registry/laboratory model, p =0.63). The correlation between hospital risk-adjusted mortality rates by use of the two models was 0.90. The registry/laboratory model contained three additional laboratory variables: alkaline phosphatase (ALKP), aspartate aminotransferase (AST), and prothrombin time (PT). The registry model yielded one hospital with significantly higher mortality than the statewide average, and the registry/laboratory model yielded no outliers.

CONCLUSIONS

The clinical models with and without laboratory data had similar discrimination. Hospital risk-adjusted mortality rates were essentially unchanged, and hospital outlier status was identical. However, three laboratory variables, ALKP, AST, and PT, were significant independent predictors of mortality, and they deserve consideration of addition to CABG clinical databases.

Strategies to Reduce 30-Day Readmissions in Older Patients Hospitalized with Heart Failure and Acute Myocardial Infarction

Readmission within 30 days after hospital discharge for common cardiovascular conditions such as heart failure and acute myocardial infarction is extremely common among older persons. To incentivize investment in reducing preventable rehospitalizations, the United States federal government has directed increasing financial penalties to hospitals with higher-than-expected 30-day readmission rates. Uncertainty exists, however, regarding the best approaches to reducing these adverse outcomes. In this review, we summarize the literature on predictors of 30-day readmission, the utility of risk prediction models, and strategies to reduce short-term readmission after hospitalization for heart failure and acute myocardial infarction. We report that few variables have been found to consistently predict the occurrence of 30-day readmission and that risk prediction models lack strong discriminative ability. We additionally report that the literature on interventions to reduce 30-day rehospitalization has significant limitations due to heterogeneity, susceptibility to bias, and lack of reporting on important contextual factors and details of program implementation. New information is characterizing the period after hospitalization as a time of high generalized risk, which has been termed the post-hospital syndrome. This framework for characterizing inherent post-discharge instability suggests new approaches to reducing readmissions.

Factors influencing the predictive power of models for predicting mortality and/or heart failure hospitalization in patients with heart failure

The present paper systematically reviews and compares existing prediction models in order to establish the strongest variables, models, and model characteristics in patients with heart failure predicting outcome. To improve decision making accurately predicting mortality and heart-failure hospitalization in patients with heart failure can be important for selecting patients with a poorer prognosis or nonresponders to current therapy, to improve decision making. MEDLINE/PubMed was searched for papers dealing with heart failure prediction models. To identify similar models on the basis of their variables hierarchical cluster analysis was performed. Meta-analysis was used to estimate the mean predictive value of the variables and models; meta-regression was used to find characteristics that explain variation in discriminating values between models. We identified 117 models in 55 papers. These models used 249 different variables. The strongest predictors were blood urea nitrogen and sodium. Four subgroups of models were identified. Mortality was most accurately predicted by prospective registry-type studies using a large number of clinical predictor variables. Mean C-statistic of all models was 0.66 ± 0.0005, with 0.71 ± 0.001, 0.68 ± 0.001 and 0.63 ± 0.001 for models predicting mortality, heart failure hospitalization, or both, respectively. There was no significant difference in discriminating value of models between patients with chronic and acute heart failure. Prediction of mortality and in particular heart failure hospitalization in patients with heart failure remains only moderately successful. The strongest predictors were blood urea nitrogen and sodium. The highest C-statistic values were achieved in a clinical setting, predicting short-term mortality with the use of models derived from prospective cohort/registry studies with a large number of predictor variables.

A "reverse july effect": association between timing of admission, medical team workload, and 30-day readmission rate

BACKGROUND

High teaching team workload has been associated with poor supervision and worse patient outcomes, yet it is unclear whether this association is more pronounced during the early months of the academic year when the residents are less experienced.

OBJECTIVE

We examined the associations between teaching team workload, timing of admission, and the 30-day readmission rate.

METHODS

In this retrospective observational study, all admissions to an urban internal medicine teaching service over a 16-month period were divided into 2 groups based on admission date: early in the academic year (July-September) or late (October-June) and further defined as being admitted to "busy" versus "less busy" teams based on number of monthly admissions. The primary outcome was 30-day readmission rate. Multivariate logistic regression was used to determine the independent association between teaching team workload and readmission rates, stratified by time of year of admission after adjustment for demographic and clinical characteristics.

RESULTS

Of 12 118 admissions examined, 2352 (19.4%) were admitted early in the year, and 9766 (80.6%) were admitted later. After multivariate adjustment, we found that patients admitted to busy versus less busy teams in the first quarter had similar 30-day readmission rate (odds ratio [OR]adj  =  1.03 [0.82-1.30]). Later year admission to a busy team was associated with increased risk of readmission after adjustment (ORadj  =  1.16 [1.03-1.30]).

CONCLUSIONS

Admission to busy teams early in the year was not associated with increased odds of 30-day readmission, whereas admission later in the year to busy teams was associated with 16% increased odds of readmission.

Using highly detailed administrative data to predict pneumonia mortality

Background

Mortality prediction models generally require clinical data or are derived from information coded at discharge, limiting adjustment for presenting severity of illness in observational studies using administrative data.

Objectives

To develop and validate a mortality prediction model using administrative data available in the first 2 hospital days.

Research Design

After dividing the dataset into derivation and validation sets, we created a hierarchical generalized linear mortality model that included patient demographics, comorbidities, medications, therapies, and diagnostic tests administered in the first 2 hospital days. We then applied the model to the validation set.

Subjects

Patients aged ≥18 years admitted with pneumonia between July 2007 and June 2010 to 347 hospitals in Premier, Inc.’s Perspective database.

Measures

In hospital mortality.

Results

The derivation cohort included 200,870 patients and the validation cohort had 50,037. Mortality was 7.2%. In the multivariable model, 3 demographic factors, 25 comorbidities, 41 medications, 7 diagnostic tests, and 9 treatments were associated with mortality. Factors that were most strongly associated with mortality included receipt of vasopressors, non-invasive ventilation, and bicarbonate. The model had a c-statistic of 0.85 in both cohorts. In the validation cohort, deciles of predicted risk ranged from 0.3% to 34.3% with observed risk over the same deciles from 0.1% to 33.7%.

Conclusions

A mortality model based on detailed administrative data available in the first 2 hospital days had good discrimination and calibration. The model compares favorably to clinically based prediction models and may be useful in observational studies when clinical data are not available.

Risk adjustment of ischemic stroke outcomes for comparing hospital performance: a statement for healthcare professionals from the American Heart Association/American Stroke Association

BACKGROUND AND PURPOSE

Stroke is the fourth-leading cause of death and a leading cause of long-term major disability in the United States. Measuring outcomes after stroke has important policy implications. The primary goals of this consensus statement are to (1) review statistical considerations when evaluating models that define hospital performance in providing stroke care; (2) discuss the benefits, limitations, and potential unintended consequences of using various outcome measures when evaluating the quality of ischemic stroke care at the hospital level; (3) summarize the evidence on the role of specific clinical and administrative variables, including patient preferences, in risk-adjusted models of ischemic stroke outcomes; (4) provide recommendations on the minimum list of variables that should be included in risk adjustment of ischemic stroke outcomes for comparisons of quality at the hospital level; and (5) provide recommendations for further research.

METHODS AND RESULTS

This statement gives an overview of statistical considerations for the evaluation of hospital-level outcomes after stroke and provides a systematic review of the literature for the following outcome measures for ischemic stroke at 30 days: functional outcomes, mortality, and readmissions. Data on outcomes after stroke have primarily involved studies conducted at an individual patient level rather than a hospital level. On the basis of the available information, the following factors should be included in all hospital-level risk-adjustment models: age, sex, stroke severity, comorbid conditions, and vascular risk factors. Because stroke severity is the most important prognostic factor for individual patients and appears to be a significant predictor of hospital-level performance for 30-day mortality, inclusion of a stroke severity measure in risk-adjustment models for 30-day outcome measures is recommended. Risk-adjustment models that do not include stroke severity or other recommended variables must provide comparable classification of hospital performance as models that include these variables. Stroke severity and other variables that are included in risk-adjustment models should be standardized across sites, so that their reliability and accuracy are equivalent. There is a pressing need for research in multiple areas to better identify methods and metrics to evaluate outcomes of stroke care.

CONCLUSIONS

There are a number of important methodological challenges in undertaking risk-adjusted outcome comparisons to assess the quality of stroke care in different hospitals. It is important for stakeholders to recognize these challenges and for there to be a concerted approach to improving the methods for quality assessment and improvement.

Using electronic health record data to develop inpatient mortality predictive model: Acute Laboratory Risk of Mortality Score (ALaRMS)

Objective

Using numeric laboratory data and administrative data from hospital electronic health record (EHR) systems, to develop an inpatient mortality predictive model.

Methods

Using EHR data of 1 428 824 adult discharges from 70 hospitals in 2006–2007, we developed the Acute Laboratory Risk of Mortality Score (ALaRMS) using age, gender, and initial laboratory values on admission as candidate variables. We then added administrative variables using the Agency for Healthcare Research and Quality (AHRQ)'s clinical classification software (CCS) and comorbidity software (CS) as disease classification tools. We validated the model using 770 523 discharges in 2008.

Results

Mortality predictors with ORs >2.00 included age, deranged albumin, arterial pH, bands, blood urea nitrogen, oxygen partial pressure, platelets, pro-brain natriuretic peptide, troponin I, and white blood cell counts. The ALaRMS model c-statistic was 0.87. Adding the CCS and CS variables increased the c-statistic to 0.91. The relative contributions were 69% (ALaRMS), 25% (CCS), and 6% (CS). Furthermore, the integrated discrimination improvement statistic demonstrated a 127% (95% CI 122% to 133%) overall improvement when ALaRMS was added to CCS and CS variables. In contrast, only a 22% (CI 19% to 25%) improvement was seen when CCS and CS variables were added to ALaRMS.

Conclusions

EHR data can generate clinically plausible mortality predictive models with excellent discrimination. ALaRMS uses automated laboratory data widely available on admission, providing opportunities to aid real-time decision support. Models that incorporate laboratory and AHRQ's CCS and CS variables have utility for risk adjustment in retrospective outcome studies.

Risk adjustment for health care financing in chronic disease: what are we missing by failing to account for disease severity?

BACKGROUND

Adjustment for differing risks among patients is usually incorporated into newer payment approaches, and current risk models rely on age, sex, and diagnosis codes. It is unknown the extent to which controlling additionally for disease severity improves cost prediction. Failure to adjust for within-disease variation may create incentives to avoid sicker patients. We address this issue among patients with chronic obstructive pulmonary disease (COPD).

METHODS

Cost and clinical data were collected prospectively from 1202 COPD patients at Kaiser Permanente. Baseline analysis included age, sex, and diagnosis codes (using the Diagnostic Cost Group Relative Risk Score) in a general linear model predicting total medical costs in the following year. We determined whether adding COPD severity measures-forced expiratory volume in 1 second, 6-Minute Walk Test, dyspnea score, body mass index, and BODE Index (composite of the other 4 measures)-improved predictions. Separately, we examined household income as a cost predictor.

RESULTS

Mean costs were $12,334/y. Controlling for Relative Risk Score, each ½ SD worsening in COPD severity factor was associated with $629 to $1135 in increased annual costs (all P<0.01). The lowest stratum of forced expiratory volume in 1 second (<30% normal) predicted $4098 (95% confidence interval, $576-$8773) additional costs. Household income predicted excess costs when added to the baseline model (P=0.038), but this became nonsignificant when also incorporating the BODE Index.

CONCLUSIONS

Disease severity measures explain significant cost variations beyond current risk models, and adding them to such models appears important to fairly compensate organizations that accept responsibility for sicker COPD patients. Appropriately controlling for disease severity also accounts for costs otherwise associated with lower socioeconomic status.

Allocating scarce resources in real-time to reduce heart failure readmissions: a prospective, controlled study

Objective

To test a multidisciplinary approach to reduce heart failure (HF) readmissions that tailors the intensity of care transition intervention to the risk of the patient using a suite of electronic medical record (EMR)-enabled programmes.

Methods

A prospective controlled before and after study of adult inpatients admitted with HF and two concurrent control conditions (acute myocardial infarction (AMI) and pneumonia (PNA)) was performed between 1 December 2008 and 1 December 2010 at a large urban public teaching hospital. An EMR-based software platform stratified all patients admitted with HF on a daily basis by their 30-day readmission risk using a published electronic predictive model. Patients at highest risk received an intensive set of evidence-based interventions designed to reduce readmission using existing resources. The main outcome measure was readmission for any cause and to any hospital within 30 days of discharge.

Results

There were 834 HF admissions in the pre-intervention period and 913 in the post-intervention period. The unadjusted readmission rate declined from 26.2% in the pre-intervention period to 21.2% in the post-intervention period (p=0.01), a decline that persisted in adjusted analyses (adjusted OR (AOR)=0.73; 95% CI 0.58 to 0.93, p=0.01). In contrast, there was no significant change in the unadjusted and adjusted readmission rates for PNA and AMI over the same period. There were 45 fewer readmissions with 913 patients enrolled and 228 patients receiving intervention, resulting in a number needed to treat (NNT) ratio of 20.

Conclusions

An EMR-enabled strategy that targeted scarce care transition resources to high-risk HF patients significantly reduced the risk-adjusted odds of readmission.

Risk-adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system

OBJECTIVE

Using a comprehensive inpatient electronic medical record, we sought to develop a risk-adjustment methodology applicable to all hospitalized patients. Further, we assessed the impact of specific data elements on model discrimination, explanatory power, calibration, integrated discrimination improvement, net reclassification improvement, performance across different hospital units, and hospital rankings.

DESIGN

Retrospective cohort study using logistic regression with split validation.

PARTICIPANTS

A total of 248,383 patients who experienced 391,584 hospitalizations between January 1, 2008 and August 31, 2011.

SETTING

Twenty-one hospitals in an integrated health care delivery system in Northern California.

RESULTS

Inpatient and 30-day mortality rates were 3.02% and 5.09%, respectively. In the validation dataset, the greatest improvement in discrimination (increase in c statistic) occurred with the introduction of laboratory data; however, subsequent addition of vital signs and end-of-life care directive data had significant effects on integrated discrimination improvement, net reclassification improvement, and hospital rankings. Use of longitudinally captured comorbidities did not improve model performance when compared with present-on-admission coding. Our final model for inpatient mortality, which included laboratory test results, vital signs, and care directives, had a c statistic of 0.883 and a pseudo-R of 0.295. Results for inpatient and 30-day mortality were virtually identical.

CONCLUSIONS

Risk-adjustment of hospital mortality using comprehensive electronic medical records is feasible and permits one to develop statistical models that better reflect actual clinician experience. In addition, such models can be used to assess hospital performance across specific subpopulations, including patients admitted to intensive care.

Using enriched observational data to develop and validate age-specific mortality risk adjustment models for hospitalized pediatric patients

BACKGROUND

Growth and development in early childhood are associated with rapid physiological changes. We sought to develop and validate age-specific mortality risk adjustment models for hospitalized pediatric patients using objective physiological variables on admission in addition to administrative variables.

METHODS

Age-specific laboratory and vital sign variables were crafted for neonates (up to 30 d old), infants/toddlers (1-23 mo), and children (2-17 y). We fit 3 logistic regression models, 1 for each age group, using a derivation cohort comprising admissions from 2000-2001 in 215 hospitals. We validated the models with a separate validation cohort comprising admissions from 2002-2007 in 62 hospitals. We used the c statistic to assess model fit.

RESULTS

The derivation cohort comprised 93,011 neonates (0.55% mortality), 46,152 infants/toddlers (0.37% mortality), and 104,010 children (0.40% mortality). The corresponding numbers of admissions (mortality rates) for the validation cohort were 162,131 (0.50%), 33,818 (0.09%), and 73,362 (0.20%), respectively. The c statistics for the 3 models were 0.94, 0.91, and 0.92, respectively, for the derivation cohort and 0.91, 0.86, and 0.93, respectively, for the validation cohort. The relative contributions of physiological versus administrative variables to the model fit were 52% versus 48% (neonates), 93% versus 7% (infants/toddlers), and 82% versus 18% (children).

CONCLUSIONS

The thresholds for physiological determinants varied by age. Common physiological variables assessed on admission contributed significantly to predicting mortality for hospitalized pediatric patients. These models may have practical utility in risk adjustment for pediatric outcomes and comparative effectiveness research when physiological data are captured through the electronic medical record.

Global comparators project: international comparison of hospital outcomes using administrative data

OBJECTIVE

To produce comparable risk-adjusted outcome rates for an international sample of hospitals in a collaborative project to share outcomes and learning.

DATA SOURCES

Administrative data varying in scope, format, and coding systems were pooled from each participating hospital for the years 2005-2010.

STUDY DESIGN

Following reconciliation of the different coding systems in the various countries, in-hospital mortality, unplanned readmission within 30 days, and "prolonged" hospital stay (>75th percentile) were risk-adjusted via logistic regression. A web-based interface was created to facilitate outcomes analysis for individual medical centers and enable peer comparisons. Small groups of clinicians are now exploring the potential reasons for variations in outcomes in their specialty.

PRINCIPAL FINDINGS

There were 6,737,211 inpatient records, including 214,622 in-hospital deaths. Although diagnostic coding depth varied appreciably by country, comorbidity weights were broadly comparable. U.S. hospitals generally had the lowest mortality rates, shortest stays, and highest readmission rates.

CONCLUSIONS

Intercountry differences in outcomes may result from differences in the quality of care or in practice patterns driven by socio-economic factors. Carefully managed administrative data can be an effective resource for initiating dialog between hospitals within and across countries. Inclusion of important outcomes beyond hospital discharge would increase the value of these analyses.

Placing clinical variables on a common linear scale of empirically based risk as a step towards construction of a general patient acuity score from the electronic health record: a modelling study

OBJECTIVE

To explore the hypothesis that placing clinical variables of differing metrics on a common linear scale of all-cause postdischarge mortality provides risk functions that are directly correlated with in-hospital mortality risk.

DESIGN

Modelling study.

SETTING

An 805-bed community hospital in the southeastern USA.

PARTICIPANTS

42302 inpatients admitted for any reason, excluding obstetrics, paediatric and psychiatric patients.

OUTCOME MEASURES

All-cause in-hospital and postdischarge mortalities, and associated correlations.

RESULTS

Pearson correlation coefficients comparing in-hospital risks with postdischarge risks for creatinine, heart rate and a set of 12 nursing assessments are 0.920, 0.922 and 0.892, respectively. Correlation between postdischarge risk heart rate and the Modified Early Warning System (MEWS) component for heart rate is 0.855. The minimal excess risk values for creatinine and heart rate roughly correspond to the normal reference ranges. We also provide the risks for values outside that range, independent of expert opinion or a regression model. By summing risk functions, a first-approximation patient risk score is created, which correctly ranks 6 discharge categories by average mortality with p<0.001 for differences in category means, and Tukey's Honestly Significant Difference Test confirmed that the means were all different at the 95% confidence level.

CONCLUSIONS

Quantitative or categorical clinical variables can be transformed into risk functions that correlate well with in-hospital risk. This methodology provides an empirical way to assess inpatient risk from data available in the Electronic Health Record. With just the variables in this paper, we achieve a risk score that correlates with discharge disposition. This is the first step towards creation of a universal measure of patient condition that reflects a generally applicable set of health-related risks. More importantly, we believe that our approach opens the door to a way of exploring and resolving many issues in patient assessment.

Mortality predictions on admission as a context for organizing care activities

BACKGROUND

Favorable health outcomes are more likely to occur when the clinical team recognizes patients at risk and intervenes in consort. Prediction rules can identify high-risk subsets, but the availability of multiple rules for various conditions present implementation and assimilation challenges.

METHODS

A prediction rule for 30-day mortality at the beginning of the hospitalization was derived in a retrospective cohort of adult inpatients from a community hospital in the Midwestern United States from 2008 to 2009, using clinical laboratory values, past medical history, and diagnoses present on admission. It was validated using 2010 data from the same and from a different hospital. The calculated mortality risk was then used to predict unplanned transfers to intensive care units, resuscitation attempts for cardiopulmonary arrests, a condition not present on admission (complications), intensive care unit utilization, palliative care status, in-hospital death, rehospitalizations within 30 days, and 180-day mortality.

RESULTS

The predictions of 30-day mortality for the derivation and validation datasets had areas under the receiver operating characteristic curve of 0.88. The 30-day mortality risk was in turn a strong predictor for in-hospital death, palliative care status, 180-day mortality; a modest predictor for unplanned transfers and cardiopulmonary arrests; and a weaker predictor for the other events of interest.

CONCLUSIONS

The probability of 30-day mortality provides health systems with an array of prognostic information that may provide a common reference point for organizing the clinical activities of the many health professionals involved in the care of the patient.