Classifying hospitals as mortality outliers: logistic versus hierarchical logistic models

Do Common Risk Adjustment Methods Do Their Job Well if Center Effects are Correlated With the Center-Specific Mean Values of Patient Characteristics?

BACKGROUND

Direct and indirect standardization are well-established approaches to performing risk adjustment when comparing outcomes between healthcare providers. However, it is an open question whether they work well when there is an association between the center effects and the distributions of the patient characteristics in these centers.

OBJECTIVES AND METHODS

We try to shed further light on the impact of such an association. We construct an artificial case study with a single covariate, in which centers can be classified as performing above, on, or below average, and the center effects correlate with center-specific mean values of a patient characteristic, as a consequence of differential quality improvement. Based on this case study, direct standardization and indirect standardization-based on marginal as well as conditional models-are compared with respect to systematic differences between their results.

RESULTS

Systematic differences between the methods were observed. All methods produced results that partially reflect differences in mean age across the centers. This may mask the classification as above, on, or below average. The differences could be explained by an inspection of the parameter estimates in the models fitted.

CONCLUSIONS

In case of correlations of center effects with center-specific mean values of a covariate, different risk adjustment methods can produce systematically differing results. This suggests the routine use of sensitivity analyses. Center effects in a conditional model need not reflect the position of a center above or below average, questioning its use in defining the truth. Further empirical investigations are necessary to judge the practical relevance of these findings.

Forecasting emergency department overcrowding: A deep learning framework

As the demand for medical cares has considerably expanded, the issue of managing patient flow in hospitals and especially in emergency departments (EDs) is certainly a key issue to be carefully mitigated. This can lead to overcrowding and the degradation of the quality of the provided medical services. Thus, the accurate modeling and forecasting of ED visits are critical for efficiently managing the overcrowding problems and enable appropriate optimization of the available resources. This paper proposed an effective method to forecast daily and hourly visits at an ED using Variational AutoEncoder (VAE) algorithm. Indeed, the VAE model as a deep learning-based model has gained special attention in features extraction and modeling due to its distribution-free assumptions and superior nonlinear approximation. Two types of forecasting were conducted: one- and multi-step-ahead forecasting. To the best of our knowledge, this is the first time that the VAE is investigated to improve forecasting of patient arrivals time-series data. Data sets from the pediatric emergency department at Lille regional hospital center, France, are employed to evaluate the forecasting performance of the introduced method. The VAE model was evaluated and compared with seven methods namely Recurrent Neural Network (RNN), Long short-term memory (LSTM), Bidirectional LSTM (BiLSTM), Convolutional LSTM Network (ConvLSTM), restricted Boltzmann machine (RBM), Gated recurrent units (GRUs), and convolutional neural network (CNN). The results clearly show the promising performance of these deep learning models in forecasting ED visits and emphasize the better performance of the VAE in comparison to the other models.

Association between job-related stress and experience of presenteeism among Korean workers stratified on the presence of depression

Background

Presenteeism refers to the phenomenon of working while sick. Its development can be attributed to not only somatic symptoms but also underlying social agreements and workplace atmosphere. In this study, we analyzed presenteeism among workers from various industries, focusing on job-related stress with stratification on the presence of depression.

Methods

We conducted the study with data from questionnaires filled in by different enterprises enrolled in the Federation of Korean Trade Unions. Workers' depressive symptoms were investigated using the Patient Health Questionnaire-2, while questions on job-related stress and presenteeism were derived from the short form of the Korean Occupational Stress Scale and the official Korean version of the Work-Productivity and Activity Impairment Questionnaire-General Health, respectively. Multilevel logistic analysis was conducted to determine the statistical differences derived from the differences between companies.

Results

In total, 930 participants (753 men and 177 women) from 59 enterprises participated in the research. We conducted multilevel logistic regression to determine the association between the variables and presenteeism, with stratification by the presence of depression. Higher job demands and higher interpersonal conflict showed significantly elevated odds ratios (ORs) in univariate models and in the multivariate multilevel model. In the final model of total population, fully adjusted by general and work-related characteristics, higher job demands (OR: 3.29, 95% confidence interval [CI]: 2.08-5.21) and interpersonal conflict (OR: 1.87, 95% CI: 1.29-2.71) had significantly higher ORs-a tendency that remained in participants without depression.

Conclusions

This study reflected the factors associated with presenteeism among workers from various enterprises. The findings revealed that job-related stress was closely related to presenteeism in both the total population and in the population without depression. Thus, it emphasized interventions for managing job stress among workers to reduce presenteeism in general workers' population.

Machine Learning for Emergency Department Management

Nowadays, emergency department services are confronted to an increasing demand. This situation causes emergency department overcrowding which often increases the length of stay of patients and leads to strain situations. To overcome this issue, emergency department managers must predict the length of stay. In this work, the researchers propose to use machine learning techniques to set up a methodology that supports the management of emergency departments (EDs). The target of this work is to predict the length of stay of patients in the ED in order to prevent strain situations. The experiments were carried out on a real database collected from the pediatric emergency department (PED) in Lille regional hospital center, France. Different machine learning techniques have been used to build the best prediction models. The results seem better with Naive Bayes, C4.5 and SVM methods. In addition, the models based on a subset of attributes proved to be more efficient than models based on the set of attributes.

The impact of disease severity adjustment on hospital standardised mortality ratios: Results from a service-wide analysis of ischaemic stroke admissions using linked pre-hospital, admissions and mortality data

Background

Administrative data are used to examine variation in thirty-day mortality across health services in several jurisdictions. Hospital performance measurement may be error-prone as information about disease severity is not typically available in routinely collected data to incorporate into case-mix adjusted analyses. Using ischaemic stroke as a case study, we tested the extent to which accounting for disease severity impacts on hospital performance assessment.

Methods

We linked all recorded ischaemic stroke admissions between July, 2011 and June, 2014 to death registrations and a measure of stroke severity obtained at first point of patient contact with health services, across New South Wales, Australia’s largest health service jurisdiction. Thirty-day hospital standardised mortality ratios were adjusted for either comorbidities, as is typically done, or for both comorbidities and stroke severity. The impact of stroke severity adjustment on mortality ratios was determined using 95% and 99% control limits applied to funnel plots and by calculating the change in rank order of hospital risk adjusted mortality rates.

Results

The performance of the stroke severity adjusted model was superior to incorporating comorbidity burden alone (c-statistic = 0.82 versus 0.75; N = 17,700 patients, 176 hospitals). Concordance in outlier classification was 89% and 97% when applying 95% or 99% control limits to funnel plots, respectively. The sensitivity rates of outlier detection using comorbidity adjustment compared with gold-standard severity and comorbidity adjustment was 74% and 83% with 95% and 99% control limits, respectively. Corresponding positive predictive values were 74% and 91%. Hospital rank order of risk adjusted mortality rates shifted between 0 to 22 places with severity adjustment (Median = 4.0, Inter-quartile Range = 2–7).

Conclusions

Rankings of mortality rates varied widely depending on whether stroke severity was taken into account. Funnel plots yielded largely concordant results irrespective of severity adjustment and may be sufficiently accurate as a screening tool for assessing hospital performance.

Outlier classification performance of risk adjustment methods when profiling multiple providers

BACKGROUND

When profiling multiple health care providers, adjustment for case-mix is essential to accurately classify the quality of providers. Unfortunately, misclassification of provider performance is not uncommon and can have grave implications. Propensity score (PS) methods have been proposed as viable alternatives to conventional multivariable regression. The objective was to assess the outlier classification performance of risk adjustment methods when profiling multiple providers.

METHODS

In a simulation study based on empirical data, the classification performance of logistic regression (fixed and random effects), PS adjustment, and three PS weighting methods was evaluated when varying parameters such as the number of providers, the average incidence of the outcome, and the percentage of outliers. Traditional classification accuracy measures were considered, including sensitivity and specificity.

RESULTS

Fixed effects logistic regression consistently had the highest sensitivity and negative predictive value, yet a low specificity and positive predictive value. Of the random effects methods, PS adjustment and random effects logistic regression performed equally well or better than all the remaining PS methods for all classification accuracy measures across the studied scenarios.

CONCLUSIONS

Of the evaluated PS methods, only PS adjustment can be considered a viable alternative to random effects logistic regression when profiling multiple providers in different scenarios.

Observed to expected or logistic regression to identify hospitals with high or low 30-day mortality?

INTRODUCTION

A common quality indicator for monitoring and comparing hospitals is based on death within 30 days of admission. An important use is to determine whether a hospital has higher or lower mortality than other hospitals. Thus, the ability to identify such outliers correctly is essential. Two approaches for detection are: 1) calculating the ratio of observed to expected number of deaths (OE) per hospital and 2) including all hospitals in a logistic regression (LR) comparing each hospital to a form of average over all hospitals. The aim of this study was to compare OE and LR with respect to correctly identifying 30-day mortality outliers. Modifications of the methods, i.e., variance corrected approach of OE (OE-Faris), bias corrected LR (LR-Firth), and trimmed mean variants of LR and LR-Firth were also studied.

MATERIALS AND METHODS

To study the properties of OE and LR and their variants, we performed a simulation study by generating patient data from hospitals with known outlier status (low mortality, high mortality, non-outlier). Data from simulated scenarios with varying number of hospitals, hospital volume, and mortality outlier status, were analysed by the different methods and compared by level of significance (ability to falsely claim an outlier) and power (ability to reveal an outlier). Moreover, administrative data for patients with acute myocardial infarction (AMI), stroke, and hip fracture from Norwegian hospitals for 2012-2014 were analysed.

RESULTS

None of the methods achieved the nominal (test) level of significance for both low and high mortality outliers. For low mortality outliers, the levels of significance were increased four- to fivefold for OE and OE-Faris. For high mortality outliers, OE and OE-Faris, LR 25% trimmed and LR-Firth 10% and 25% trimmed maintained approximately the nominal level. The methods agreed with respect to outlier status for 94.1% of the AMI hospitals, 98.0% of the stroke, and 97.8% of the hip fracture hospitals.

CONCLUSION

We recommend, on the balance, LR-Firth 10% or 25% trimmed for detection of both low and high mortality outliers.

Should Emergency Department Attendances be Used With or Instead of Readmission Rates as a Performance Metric?: Comparison of Statistical Properties Using National Data

BACKGROUND

Hospital readmissions are common and are viewed as unfavorable. They are commonly used as a measure of quality of care and, in the United States and England, are associated with financial penalties. Readmissions are not the only possible return-to-acute-care metric; patients may also attend emergency departments (EDs).

OBJECTIVE

To assess hospital-level return-to-acute-care metrics using statistical criteria.

RESEARCH DESIGN

Patient readmissions and/or ED attendances were aggregated to produce risk-standardized hospital rates. Return-to-acute-care rates at 7, 30, 90, and 365 days were assessed using key statistical properties: (i) variability between hospitals; (ii) the relative contribution of patient and nonpatient factors to variation; and (iii) the statistical power to detect performance differences.

SUBJECTS

We had pseudonymized administrative data on all inpatient hospital admissions and ED attendances in National Health Service hospitals in England between April 2009 and March 2011. Patients with an inpatient stay for chronic obstructive pulmonary disorder or heart failure were eligible for inclusion.

MEASURES

ED attendances and readmissions for patients discharged from an inpatient stay for chronic obstructive pulmonary disorder or heart failure.

RESULTS

Interhospital variation was greatest for ED attendance; in addition, readmission was more strongly determined by patient characteristics than was ED attendance or both combined. Because of smaller numbers, the statistical power to detect differences in rates at 7 days for any indicator was limited.

CONCLUSIONS

Despite the current emphasis on readmissions, we found that ED attendance within 30 days has more desirable statistical properties and therefore the potential to be a useful metric when comparing hospitals.

Time series modelling and forecasting of emergency department overcrowding

Efficient management of patient flow (demand) in emergency departments (EDs) has become an urgent issue for many hospital administrations. Today, more and more attention is being paid to hospital management systems to optimally manage patient flow and to improve management strategies, efficiency and safety in such establishments. To this end, EDs require significant human and material resources, but unfortunately these are limited. Within such a framework, the ability to accurately forecast demand in emergency departments has considerable implications for hospitals to improve resource allocation and strategic planning. The aim of this study was to develop models for forecasting daily attendances at the hospital emergency department in Lille, France. The study demonstrates how time-series analysis can be used to forecast, at least in the short term, demand for emergency services in a hospital emergency department. The forecasts were based on daily patient attendances at the paediatric emergency department in Lille regional hospital centre, France, from January 2012 to December 2012. An autoregressive integrated moving average (ARIMA) method was applied separately to each of the two GEMSA categories and total patient attendances. Time-series analysis was shown to provide a useful, readily available tool for forecasting emergency department demand.