Mortality prediction in intensive care units including premorbid functional status improved performance and internal validity

OBJECTIVE

Prognostic models are key for benchmarking intensive care units (ICUs). They require up-to-date predictors and should report transportability properties for reliable predictions. We developed and validated an in-hospital mortality risk prediction model to facilitate benchmarking, quality assurance, and health economics evaluation.

STUDY DESIGN AND SETTING

We retrieved data from the database of an international (Finland, Estonia, Switzerland) multicenter ICU cohort study from 2015 to 2017. We used a hierarchical logistic regression model that included age, a modified Simplified Acute Physiology Score-II, admission type, premorbid functional status, and diagnosis as grouping variable. We used pooled and meta-analytic cross-validation approaches to assess temporal and geographical transportability.

RESULTS

We included 61,224 patients treated in the ICU (hospital mortality 10.6%). The developed prediction model had an area under the receiver operating characteristic curve 0.886, 95% confidence interval (CI) 0.882-0.890; a calibration slope 1.01, 95% CI (0.99-1.03); a mean calibration -0.004, 95% CI (-0.035 to 0.027). Although the model showed very good internal validity and geographic discrimination transportability, we found substantial heterogeneity of performance measures between ICUs (I-squared: 53.4-84.7%).

CONCLUSION

A novel framework evaluating the performance of our prediction model provided key information to judge the validity of our model and its adaptation for future use.

In the eye of the storm: impact of COVID-19 pandemic on admission patterns to paediatric intensive care units in the UK and Eire

BACKGROUND

The coronavirus disease-19 (COVID-19) pandemic had a relatively minimal direct impact on critical illness in children compared to adults. However, children and paediatric intensive care units (PICUs) were affected indirectly. We analysed the impact of the pandemic on PICU admission patterns and patient characteristics in the UK and Ireland.

METHODS

We performed a retrospective cohort study of all admissions to PICUs in children < 18 years during Jan-Dec 2020, using data collected from 32 PICUs via a central database (PICANet). Admission patterns, case-mix, resource use, and outcomes were compared with the four preceding years (2016-2019) based on the date of admission.

RESULTS

There were 16,941 admissions in 2020 compared to an annual average of 20,643 (range 20,340-20,868) from 2016 to 2019. During 2020, there was a reduction in all PICU admissions (18%), unplanned admissions (20%), planned admissions (15%), and bed days (25%). There was a 41% reduction in respiratory admissions, and a 60% reduction in children admitted with bronchiolitis but an 84% increase in admissions for diabetic ketoacidosis during 2020 compared to the previous years. There were 420 admissions (2.4%) with either PIMS-TS or COVID-19 during 2020. Age and sex adjusted prevalence of unplanned PICU admission reduced from 79.7 (2016-2019) to 63.1 per 100,000 in 2020. Median probability of death [1.2 (0.5-3.4) vs. 1.2 (0.5-3.4) %], length of stay [2.3 (1.0-5.5) vs. 2.4 (1.0-5.7) days] and mortality rates [3.4 vs. 3.6%, (risk-adjusted OR 1.00 [0.91-1.11, p = 0.93])] were similar between 2016-2019 and 2020. There were 106 fewer in-PICU deaths in 2020 (n = 605) compared with 2016-2019 (n = 711).

CONCLUSIONS

The use of a high-quality international database allowed robust comparisons between admission data prior to and during the COVID-19 pandemic. A significant reduction in prevalence of unplanned admissions, respiratory diseases, and fewer child deaths in PICU observed may be related to the targeted COVID-19 public health interventions during the pandemic. However, analysis of wider and longer-term societal impact of the pandemic and public health interventions on physical and mental health of children is required.

Evaluating quality in clinical care

Defining and maintaining quality is essential to surgical practice. It is only through structured approaches to assessing outcomes that we can ensure that optimal care is delivered. This article will define quality in healthcare and discuss assessment models with reference to pertinent surgical literature. National initiatives are discussed with a critical appraisal of their role and effectiveness. We discuss the aim of quality improvement initiatives and comment on reporting of outcomes. The difficult question of how to maintain quality during a crisis, such as an infectious disease pandemic, is addressed.

Effect of Hospital Choice on the Risk of Caesarean Delivery

OBJECTIVE

This study aimed to evaluate the variation in Caesarean delivery rate (CDR) among hospitals across the United States, its effect on maternal and neonatal outcomes, and whether differences in pregnancy and hospital characteristics can explain the higher CDRs seen in certain hospitals.

METHODS

This retrospective population-based cohort study was conducted using the 2014 Healthcare and Utilization Project Nationwide Inpatient Sample. The investigators identified all hospitals with birth admissions and compared hospitals with high CDRs with hospitals with low/mid CDRs, in terms of hospital characteristics, maternal characteristics, and maternal and neonatal outcomes. Regression analyses within multiple hospital and patient characteristic strata were used to evaluate the adjusted independent effect of the hospital on the risk of Caesarean delivery (Canadian Task Force Classification II-2).

RESULTS

In this study population, 96% of U.S. hospitals had a CDR above 20%, and 5% had a CDR >40%. High-CDR hospitals (>40%) were more often privately owned, non-teaching hospitals with an older patient population. When adjusting for baseline obstetrical and hospital characteristics, high-CDR hospitals remained independently associated with an elevated risk of Caesarean delivery. These findings persisted in stratified analyses of each hospital and patient-level characteristic. Obstetrical and neonatal outcomes were comparable in all hospitals irrespective of CDR.

CONCLUSION

Hospital characteristics and case mix do not account for the significant variation in CDRs across U.S. hospitals. Individual hospitals are in themselves independent risk factors for Caesarean delivery. Choosing to give birth in a certain hospital may put women at an increased risk of having a Caesarean delivery, without maternal or neonatal benefit.

Positive predictive value of ICD-10 codes for acute myocardial infarction in Japan: a validation study at a single center

Background

In Japan, several large healthcare databases have become available for research since the early 2000’s. However, validation studies to examine the accuracy of these databases remain scarce. We conducted a validation study in order to estimate the positive predictive value (PPV) of local or ICD-10 codes for acute myocardial infarction (AMI) in Japanese claims. In particular, we examined whether the PPV differs between claims in the Diagnosis Procedure Combination case mix scheme (DPC claims) and in non-DPC claims.

Methods

We selected a random sample of 200 patients from all patients hospitalized at a large tertiary-care university hospital between January 1, 2009 and December 31, 2011 who had an inpatient claim assigned a local or ICD-10 code for AMI. We used a standardized data abstraction form to collect the relevant information from an electronic medical records system. Abstracted information was then categorized by a single cardiologist as being either definite or not having AMI.

Results

In a random sample of 200 patients, the average age was 67.7 years and the proportion of males was 78.0%. The PPV of the local or ICD-10 code for AMI was 82.5% in this sample of 200 patients. Further, of 178 patients who had an ICD-10 code for AMI based on any of the 7 types of condition codes in the DPC claims, the PPV was 89.3%, whereas of the 161 patients who had an ICD-10 code for AMI based on any of 3 major types of condition codes in the DPC claims, the PPV was 93.8%.

Conclusion

The PPV of the local or ICD-10 code for AMI was high for inpatient claims in Japan. The PPV was even higher for the ICD-10 code for AMI for those patients who received AMI care through the DPC case mix scheme. The current study was conducted in a single center, suggesting that a multi-center study involving different types of hospitals is needed in the future. The accuracy of condition codes for DPC claims in Japan may also be worth examining for conditions other than AMI such as stroke.

A case-mix classification system for explaining healthcare costs using administrative data in Italy

BACKGROUND

The Italian National Health Service (NHS) provides universal coverage to all citizens, granting primary and hospital care with a copayment system for outpatient and drug services. Financing of Local Health Trusts (LHTs) is based on a capitation system adjusted only for age, gender and area of residence. We applied a risk-adjustment system (Johns Hopkins Adjusted Clinical Groups System, ACG® System) in order to explain health care costs using routinely collected administrative data in the Veneto Region (North-eastern Italy).

METHODS

All residents in the Veneto Region were included in the study. The ACG system was applied to classify the regional population based on the following information sources for the year 2015: Hospital Discharges, Emergency Room visits, Chronic disease registry for copayment exemptions, ambulatory visits, medications, the Home care database, and drug prescriptions. Simple linear regressions were used to contrast an age-gender model to models incorporating more comprehensive risk measures aimed at predicting health care costs.

RESULTS

A simple age-gender model explained only 8% of the variance of 2015 total costs. Adding diagnoses-related variables provided a 23% increase, while pharmacy based variables provided an additional 17% increase in explained variance. The adjusted R-squared of the comprehensive model was 6 times that of the simple age-gender model.

CONCLUSIONS

ACG System provides substantial improvement in predicting health care costs when compared to simple age-gender adjustments. Aging itself is not the main determinant of the increase of health care costs, which is better explained by the accumulation of chronic conditions and the resulting multimorbidity.

Validation of the All Patient Refined Diagnosis Related Group (APR-DRG) Risk of Mortality and Severity of Illness Modifiers as a Measure of Perioperative Risk

The All Patient Refined Diagnosis Related Group (APR-DRG) is an inpatient visit classification system that assigns a diagnostic related group, a Risk of Mortality (ROM) subclass and a Severity of Illness (SOI) subclass. While extensively used for cost adjustment, no study has compared the APR-DRG subclass modifiers to the popular Charlson Comorbidity Index as a measure of comorbidity severity in models for perioperative in-hospital mortality. In this study we attempt to validate the use of these subclasses to predict mortality in a cohort of surgical patients. We analyzed all adult (age over 18 years) inpatient non-cardiac surgery at our institution between December 2005 and July 2013. After exclusions, we split the cohort into training and validation sets. We created prediction models of inpatient mortality using the Charlson Comorbidity Index, ROM only, SOI only, and ROM with SOI. Models were compared by receiver-operator characteristic (ROC) curve, area under the ROC curve (AUC), and Brier score. After exclusions, we analyzed 63,681 patient-visits. Overall in-hospital mortality was 1.3%. The median number of ICD-9-CM diagnosis codes was 6 (Q1-Q3 4-10). The median Charlson Comorbidity Index was 0 (Q1-Q3 0-2). When the model was applied to the validation set, the c-statistic for Charlson was 0.865, c-statistic for ROM was 0.975, and for ROM and SOI combined the c-statistic was 0.977. The scaled Brier score for Charlson was 0.044, Brier for ROM only was 0.230, and Brier for ROM and SOI was 0.257. The APR-DRG ROM or SOI subclasses are better predictors than the Charlson Comorbidity Index of in-hospital mortality among surgical patients.

Impact of State Psychiatric Hospital Waitlists on Monthly Admissions

OBJECTIVE

This study estimated the effects of a waitlist policy on the monthly number and case mix of admissions to state psychiatric hospitals (SPHs) in North Carolina (NC).

METHODS

Descriptive analyses compared pre/postwaitlist differences in the monthly number and case mix of nonforensic adult admissions (N=72,035) to NC's four SPHs by using data from the three years before and the three years after the waitlist announcement. Hospital-level fixed-effects regression models further evaluated the waitlist policy's impact on the number and case mix of admissions.

RESULTS

Regression results confirmed that the waitlist policy was associated with both fewer admissions and changes to the case mix of admissions, including a 4.2% decrease in the percentage of monthly admissions by patients with diagnoses of substance abuse disorders (p=.002) across all months postwaitlist (partially offset by an increase of patients with diagnoses of severe mental illness alone).

CONCLUSIONS

Waitlists led to reduced monthly admissions and altered case mix following implementation at NC SPHs.

Do prices reflect the costs of cardiac surgery in the elderly?

INTRODUCTION

Payment for cardiac surgery in Portugal is based on a contract agreement between hospitals and the health ministry. Our aim was to compare the prices paid according to this contract agreement with calculated costs in a population of patients aged ≥65 years undergoing cardiac surgery in one hospital department.

METHODS

Data on 250 patients operated between September 2011 and September 2012 were prospectively collected. The procedures studied were coronary artery bypass graft surgery (CABG) (n=67), valve surgery (n=156) and combined CABG and valve surgery (n=27). Costs were calculated by two methods: micro-costing when feasible and mean length of stay otherwise. Price information was provided by the hospital administration and calculated using the hospital's mean case-mix.

RESULTS

Thirty-day mortality was 3.2%. Mean EuroSCORE I was 5.97 (standard deviation [SD] 4.5%), significantly lower for CABG (p<0.01). Mean intensive care unit stay was 3.27 days (SD 4.7) and mean hospital stay was 9.92 days (SD 6.30), both significantly shorter for CABG. Calculated costs for CABG were €6539.17 (SD 3990.26), for valve surgery €8289.72 (SD 3319.93) and for combined CABG and valve surgery €11 498.24 (SD 10 470.57). The payment for each patient was €4732.38 in 2011 and €4678.66 in 2012 based on the case-mix index of the hospital group, which was 2.06 in 2011 and 2.21 in 2012; however, the case-mix in our sample was 6.48 in 2011 and 6.26 in 2012.

CONCLUSION

The price paid for each patient was lower than the calculated costs. Prices would be higher than costs if the case-mix of the sample had been used. Costs were significantly lower for CABG.

[Cases and duration of mechanical ventilation in German hospitals : An analysis of DRG incentives and developments in respiratory medicine]

BACKGROUND

Diagnosis-related groups (DRGs) have been used to reimburse hospitals services in Germany since 2003/04. Like any other reimbursement system, DRGs offer specific incentives for hospitals that may lead to unintended consequences for patients. In the German context, specific procedures and their documentation are suspected to be primarily performed to increase hospital revenues. Mechanical ventilation of patients and particularly the duration of ventilation, which is an important variable for the DRG-classification, are often discussed to be among these procedures.

OBJECTIVES

The aim of this study was to examine incentives created by the German DRG-based payment system with regard to mechanical ventilation and to identify factors that explain the considerable increase of mechanically ventilated patients in recent years. Moreover, the assumption that hospitals perform mechanical ventilation in order to gain economic benefits was examined.

MATERIAL AND METHODS

In order to gain insights on the development of the number of mechanically ventilated patients, patient-level data provided by the German Federal Statistical Office and the German Institute for the Hospital Remuneration System were analyzed. The type of performed ventilation, the total number of ventilation hours, the age distribution, mortality and the DRG distribution for mechanical ventilation were calculated, using methods of descriptive and inferential statistics. Furthermore, changes in DRG-definitions and changes in respiratory medicine were compared for the years 2005-2012.

RESULTS

Since the introduction of the DRG-based payment system in Germany, the hours of ventilation and the number of mechanically ventilated patients have substantially increased, while mortality has decreased. During the same period there has been a switch to less invasive ventilation methods. The age distribution has shifted to higher age-groups. A ventilation duration determined by DRG definitions could not be found.

CONCLUSION

Due to advances in respiratory medicine, new ventilation methods have been introduced that are less prone to complications. This development has simultaneously improved survival rates. There was no evidence supporting the assumption that the duration of mechanical ventilation is influenced by the time intervals relevant for DRG grouping. However, presumably operational routines such as staff availability within early and late shifts of the hospital have a significant impact on the termination of mechanical ventilation.

Pediatric surgery - a changing field: national trends in pediatric surgical practice

BACKGROUND

Over the last decade, our institution has experienced a relative increase in the number of mundane cases, such as appendectomy and incision and drainage of abscess, versus index (complex) cases. We sought to determine if this trend is present at the national level.

METHODS

A retrospective review of surgical case volume at 36 freestanding children's hospitals was performed between January 2004 and December 2013 using the Pediatric Health Information System (PHIS) database. Procedures were classified as "mundane" or "index", and 10 procedures of each type were selected for analysis. Results were reported as a percentage of total cases. Statistical analysis of linear trends was performed with the Mann-Kendall test.

RESULTS

Overall, index procedures had a significant downward trend (p<0.01), whereas mundane procedures had a significant upward trend (p<0.01). Individually, 5 mundane procedures had significant upward trends, and 3 had downward trends. Five index procedures had significant downward trends, and none had an upward trend.

CONCLUSION

The field of pediatric surgery is undergoing change with mundane procedures constituting an increasing proportion of the surgical caseload, while complex procedures are proportionately decreasing. These trends may be useful to inform decisions regarding future pediatric surgery workforce planning.

Paying for primary care: a cross-sectional analysis of cost and morbidity distributions across primary care payment models in Ontario Canada

Policy-makers desire an optimal balance of financial incentives to improve productivity and encourage improved quality in primary care, while also avoiding issues of risk-selection inherent to capitation-based payment. In this paper we analyze risk-selection in capitation-based payment by using administrative data for patients (n = 11,600,911) who were rostered (i.e., signed an enrollment form, or received a majority of care) with a primary care physician (n = 8621) in Ontario, Canada in 2010/11. We analyze this data using a relative distribution approach and compare distributions of patient costs and morbidity across primary care payment models. Our results suggest a relationship between being in a capitation-based payment scheme and having low cost patients (and presumably healthy patients) compared to fee-for-service physicians. However, we do not have evidence that physicians in capitation-based models are reducing the care they provide to sick and high cost patients. These findings suggest there is a relationship between payment type and risk-selection, particularly for low-cost and healthy patients.

A new framework to enhance the interpretation of external validation studies of clinical prediction models

OBJECTIVES

It is widely acknowledged that the performance of diagnostic and prognostic prediction models should be assessed in external validation studies with independent data from "different but related" samples as compared with that of the development sample. We developed a framework of methodological steps and statistical methods for analyzing and enhancing the interpretation of results from external validation studies of prediction models.

STUDY DESIGN AND SETTING

We propose to quantify the degree of relatedness between development and validation samples on a scale ranging from reproducibility to transportability by evaluating their corresponding case-mix differences. We subsequently assess the models' performance in the validation sample and interpret the performance in view of the case-mix differences. Finally, we may adjust the model to the validation setting.

RESULTS

We illustrate this three-step framework with a prediction model for diagnosing deep venous thrombosis using three validation samples with varying case mix. While one external validation sample merely assessed the model's reproducibility, two other samples rather assessed model transportability. The performance in all validation samples was adequate, and the model did not require extensive updating to correct for miscalibration or poor fit to the validation settings.

CONCLUSION

The proposed framework enhances the interpretation of findings at external validation of prediction models.

Analysis of trends in emergency department attendances, hospital admissions and medical staffing in a Hong Kong university hospital: 5-year study

Background

The workload of emergency departments (ED) continually changes in response to presentations, overcrowding and availability of expertise and investigations.

Aims

To investigate changes in ED presentations and care processes, and the relationship of patient demand and ED staff resources to waiting times and processing times.

Methods

Retrospective analysis of prospectively collected administrative data from January 1999 to April 2005 in an emergency department in a university teaching hospital in Hong Kong. All patients attending the emergency department during the study period were included. Monthly attendance data were retrieved and analysed to determine both qualitative and quantitative changes in the patterns of presentation to the ED using prospectively gathered data.

Results

Total ED attendances decreased by 25% during the study with little seasonal variation. The admission rate and the use of ambulances increased steadily and significantly. Medical patients are increasing proportionately, but trauma patients are decreased in number.

Conclusion

There have been major changes in the patterns of ED attendances and ED waiting times over the study period in this teaching hospital ED. Decreasing overall ED numbers are offset by an increasingly elderly population and a more complex case mix. Reducing clinical staff numbers appears to reduce the ED’s capacity to provide timely assessments and care and to function as hospital gatekeepers. Restoring staff numbers to previous levels may improve the quality and timeliness of ED services. It is necessary to refine measures of ED complexity and workload to determine appropriate staffing levels in the future.

A Relationship of Care Time with Functional Status and Patients Characteristics among Patients in Long-term Care Hospitals

OBJECTIVES

The aim of this study was to investigate the functional status variables related to the care time of health professionals for patients in long-term care facilities.

METHODS

The functional stati of 1001 patients in 8 longterm care hospitals were examined by the Resident Assessment Instrument for Long-term Care Facility Version 2.0. The care time of health professionals for patients was calculated using data from a self-reported task survey by nurses, auxiliary nurses, private aides, doctors, physiotherapists and social workers.

RESULTS

The average care time per diem was 240.6 minutes. The care time by doctors, nurses and private aides were 11.0, 71.0 and 139.5 minutes, respectively. The lower the function of activities of daily living (ADL) and the greater the symptoms of extensive services, special care and clinical complexity, the more care time was served. On the contrary, the greater the symptoms of nursing rehabilitation, depression, cognitive disorder, behavior problem and psychiatry/mood disorder, the less care time was served. Age and gender were not significantly related to the care time.

CONCLUSIONS

Developing a case mix classification system for elderly long term care patients may be helpful for both of patients and health care providers. The ADL, extensive services, special care and clinical complexity of variables should be considered in the development of a case mix system for the long term care of patients in Korea.