1
|
Vach W, Wehberg S, Luta G. Do Common Risk Adjustment Methods Do Their Job Well if Center Effects are Correlated With the Center-Specific Mean Values of Patient Characteristics? Med Care 2024:00005650-990000000-00237. [PMID: 38833716 DOI: 10.1097/mlr.0000000000002008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/06/2024]
Abstract
BACKGROUND Direct and indirect standardization are well-established approaches to performing risk adjustment when comparing outcomes between healthcare providers. However, it is an open question whether they work well when there is an association between the center effects and the distributions of the patient characteristics in these centers. OBJECTIVES AND METHODS We try to shed further light on the impact of such an association. We construct an artificial case study with a single covariate, in which centers can be classified as performing above, on, or below average, and the center effects correlate with center-specific mean values of a patient characteristic, as a consequence of differential quality improvement. Based on this case study, direct standardization and indirect standardization-based on marginal as well as conditional models-are compared with respect to systematic differences between their results. RESULTS Systematic differences between the methods were observed. All methods produced results that partially reflect differences in mean age across the centers. This may mask the classification as above, on, or below average. The differences could be explained by an inspection of the parameter estimates in the models fitted. CONCLUSIONS In case of correlations of center effects with center-specific mean values of a covariate, different risk adjustment methods can produce systematically differing results. This suggests the routine use of sensitivity analyses. Center effects in a conditional model need not reflect the position of a center above or below average, questioning its use in defining the truth. Further empirical investigations are necessary to judge the practical relevance of these findings.
Collapse
Affiliation(s)
- Werner Vach
- Basel Academy for Quality and Research in Medicine, Basel, Switzerland
- Department of Environmental Sciences, University of Basel, Basel, Switzerland
| | - Sonja Wehberg
- The Research Unit of General Practice, Department of Public Health, University of Southern Denmark, Odense, Denmark
| | - George Luta
- Department of Biostatistics, Bioinformatics and Biomathematics, Georgetown University, Washington, DC
- Clinical Research Unit, The Parker Institute, Copenhagen University Hospital, Bispebjerg and Frederiksberg, Nordre Fasanvej, Frederiksberg, Denmark
- Department of Clinical Epidemiology, Aarhus University, Olof Palmes Allé, Aarhus, Denmark
| |
Collapse
|
2
|
Harrou F, Dairi A, Kadri F, Sun Y. Forecasting emergency department overcrowding: A deep learning framework. CHAOS, SOLITONS, AND FRACTALS 2020; 139:110247. [PMID: 32982079 PMCID: PMC7505132 DOI: 10.1016/j.chaos.2020.110247] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/09/2020] [Accepted: 08/23/2020] [Indexed: 05/03/2023]
Abstract
As the demand for medical cares has considerably expanded, the issue of managing patient flow in hospitals and especially in emergency departments (EDs) is certainly a key issue to be carefully mitigated. This can lead to overcrowding and the degradation of the quality of the provided medical services. Thus, the accurate modeling and forecasting of ED visits are critical for efficiently managing the overcrowding problems and enable appropriate optimization of the available resources. This paper proposed an effective method to forecast daily and hourly visits at an ED using Variational AutoEncoder (VAE) algorithm. Indeed, the VAE model as a deep learning-based model has gained special attention in features extraction and modeling due to its distribution-free assumptions and superior nonlinear approximation. Two types of forecasting were conducted: one- and multi-step-ahead forecasting. To the best of our knowledge, this is the first time that the VAE is investigated to improve forecasting of patient arrivals time-series data. Data sets from the pediatric emergency department at Lille regional hospital center, France, are employed to evaluate the forecasting performance of the introduced method. The VAE model was evaluated and compared with seven methods namely Recurrent Neural Network (RNN), Long short-term memory (LSTM), Bidirectional LSTM (BiLSTM), Convolutional LSTM Network (ConvLSTM), restricted Boltzmann machine (RBM), Gated recurrent units (GRUs), and convolutional neural network (CNN). The results clearly show the promising performance of these deep learning models in forecasting ED visits and emphasize the better performance of the VAE in comparison to the other models.
Collapse
Affiliation(s)
- Fouzi Harrou
- King Abdullah University of Science and Technology (KAUST) Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, Thuwal 23955-6900, Saudi Arabia
| | - Abdelkader Dairi
- University of Science and Technology of Oran-Mohamed Boudiaf (USTO-MB), Computer Science department Signal, Image and Speech Laboratory (SIMPA) laboratory, El Mnaouar, BP 1505, Bir El Djir 31000, Oran, Algeria
| | - Farid Kadri
- Aeroline and Customer Services, Agence 1024, Sopra Steria Group, 31770, Colomiers, France
| | - Ying Sun
- King Abdullah University of Science and Technology (KAUST) Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, Thuwal 23955-6900, Saudi Arabia
| |
Collapse
|
3
|
Association between job-related stress and experience of presenteeism among Korean workers stratified on the presence of depression. Ann Occup Environ Med 2019; 31:e26. [PMID: 31620303 PMCID: PMC6792004 DOI: 10.35371/aoem.2019.31.e26] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2019] [Accepted: 09/09/2019] [Indexed: 01/01/2023] Open
Abstract
Background Presenteeism refers to the phenomenon of working while sick. Its development can be attributed to not only somatic symptoms but also underlying social agreements and workplace atmosphere. In this study, we analyzed presenteeism among workers from various industries, focusing on job-related stress with stratification on the presence of depression. Methods We conducted the study with data from questionnaires filled in by different enterprises enrolled in the Federation of Korean Trade Unions. Workers' depressive symptoms were investigated using the Patient Health Questionnaire-2, while questions on job-related stress and presenteeism were derived from the short form of the Korean Occupational Stress Scale and the official Korean version of the Work-Productivity and Activity Impairment Questionnaire-General Health, respectively. Multilevel logistic analysis was conducted to determine the statistical differences derived from the differences between companies. Results In total, 930 participants (753 men and 177 women) from 59 enterprises participated in the research. We conducted multilevel logistic regression to determine the association between the variables and presenteeism, with stratification by the presence of depression. Higher job demands and higher interpersonal conflict showed significantly elevated odds ratios (ORs) in univariate models and in the multivariate multilevel model. In the final model of total population, fully adjusted by general and work-related characteristics, higher job demands (OR: 3.29, 95% confidence interval [CI]: 2.08-5.21) and interpersonal conflict (OR: 1.87, 95% CI: 1.29-2.71) had significantly higher ORs-a tendency that remained in participants without depression. Conclusions This study reflected the factors associated with presenteeism among workers from various enterprises. The findings revealed that job-related stress was closely related to presenteeism in both the total population and in the population without depression. Thus, it emphasized interventions for managing job stress among workers to reduce presenteeism in general workers' population.
Collapse
|
4
|
Benbelkacem S, Kadri F, Atmani B, Chaabane S. Machine Learning for Emergency Department Management. INTERNATIONAL JOURNAL OF INFORMATION SYSTEMS IN THE SERVICE SECTOR 2019. [DOI: 10.4018/ijisss.2019070102] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Nowadays, emergency department services are confronted to an increasing demand. This situation causes emergency department overcrowding which often increases the length of stay of patients and leads to strain situations. To overcome this issue, emergency department managers must predict the length of stay. In this work, the researchers propose to use machine learning techniques to set up a methodology that supports the management of emergency departments (EDs). The target of this work is to predict the length of stay of patients in the ED in order to prevent strain situations. The experiments were carried out on a real database collected from the pediatric emergency department (PED) in Lille regional hospital center, France. Different machine learning techniques have been used to build the best prediction models. The results seem better with Naive Bayes, C4.5 and SVM methods. In addition, the models based on a subset of attributes proved to be more efficient than models based on the set of attributes.
Collapse
Affiliation(s)
- Sofia Benbelkacem
- Laboratoire d'Informatique d'Oran (LIO), University of Oran 1 Ahmed Ben Bella, Algeria
| | - Farid Kadri
- Big Data & Analytics Services, Institut d'Optique Graduate School, Talence, France
| | - Baghdad Atmani
- Laboratoire d'Informatique d'Oran (LIO), University of Oran 1 Ahmed Ben Bella, Algeria
| | - Sondès Chaabane
- University Polytechnique Hauts-de-France, CNRS, UMR 8201 – LAMIH, Laboratoire d'Automatique de Mécanique et d'Informatique Industrielles et Humaines, F-59313 Valenciennes, France
| |
Collapse
|
5
|
Gattellari M, Goumas C, Jalaludin B, Worthington J. The impact of disease severity adjustment on hospital standardised mortality ratios: Results from a service-wide analysis of ischaemic stroke admissions using linked pre-hospital, admissions and mortality data. PLoS One 2019; 14:e0216325. [PMID: 31112556 PMCID: PMC6528964 DOI: 10.1371/journal.pone.0216325] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2018] [Accepted: 04/18/2019] [Indexed: 11/19/2022] Open
Abstract
Background Administrative data are used to examine variation in thirty-day mortality across health services in several jurisdictions. Hospital performance measurement may be error-prone as information about disease severity is not typically available in routinely collected data to incorporate into case-mix adjusted analyses. Using ischaemic stroke as a case study, we tested the extent to which accounting for disease severity impacts on hospital performance assessment. Methods We linked all recorded ischaemic stroke admissions between July, 2011 and June, 2014 to death registrations and a measure of stroke severity obtained at first point of patient contact with health services, across New South Wales, Australia’s largest health service jurisdiction. Thirty-day hospital standardised mortality ratios were adjusted for either comorbidities, as is typically done, or for both comorbidities and stroke severity. The impact of stroke severity adjustment on mortality ratios was determined using 95% and 99% control limits applied to funnel plots and by calculating the change in rank order of hospital risk adjusted mortality rates. Results The performance of the stroke severity adjusted model was superior to incorporating comorbidity burden alone (c-statistic = 0.82 versus 0.75; N = 17,700 patients, 176 hospitals). Concordance in outlier classification was 89% and 97% when applying 95% or 99% control limits to funnel plots, respectively. The sensitivity rates of outlier detection using comorbidity adjustment compared with gold-standard severity and comorbidity adjustment was 74% and 83% with 95% and 99% control limits, respectively. Corresponding positive predictive values were 74% and 91%. Hospital rank order of risk adjusted mortality rates shifted between 0 to 22 places with severity adjustment (Median = 4.0, Inter-quartile Range = 2–7). Conclusions Rankings of mortality rates varied widely depending on whether stroke severity was taken into account. Funnel plots yielded largely concordant results irrespective of severity adjustment and may be sufficiently accurate as a screening tool for assessing hospital performance.
Collapse
Affiliation(s)
- Melina Gattellari
- Heart and Brain Collaboration, Ingham Institute for Applied Medical Research, Liverpool, Sydney, New South Wales, Australia
- Institute of Clinical Neurosciences, Royal Prince Alfred Hospital, Camperdown, Sydney, New South Wales, Australia
| | - Chris Goumas
- Heart and Brain Collaboration, Ingham Institute for Applied Medical Research, Liverpool, Sydney, New South Wales, Australia
| | - Bin Jalaludin
- Population Health Intelligence, Healthy People and Places Unit; South Western Sydney Local Health District, Liverpool, Sydney, New South Wales, Australia
- School of Public Health, The University of New South Wales, Kensington, Sydney, New South Wales, Australia
| | - John Worthington
- Institute of Clinical Neurosciences, Royal Prince Alfred Hospital, Camperdown, Sydney, New South Wales, Australia
- South Western Sydney Clinical School, The University of New South Wales, Liverpool, Sydney, New South Wales, Australia
- * E-mail:
| |
Collapse
|
6
|
Brakenhoff TB, Roes KCB, Moons KGM, Groenwold RHH. Outlier classification performance of risk adjustment methods when profiling multiple providers. BMC Med Res Methodol 2018; 18:54. [PMID: 29902975 PMCID: PMC6003201 DOI: 10.1186/s12874-018-0510-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2017] [Accepted: 05/15/2018] [Indexed: 12/25/2022] Open
Abstract
BACKGROUND When profiling multiple health care providers, adjustment for case-mix is essential to accurately classify the quality of providers. Unfortunately, misclassification of provider performance is not uncommon and can have grave implications. Propensity score (PS) methods have been proposed as viable alternatives to conventional multivariable regression. The objective was to assess the outlier classification performance of risk adjustment methods when profiling multiple providers. METHODS In a simulation study based on empirical data, the classification performance of logistic regression (fixed and random effects), PS adjustment, and three PS weighting methods was evaluated when varying parameters such as the number of providers, the average incidence of the outcome, and the percentage of outliers. Traditional classification accuracy measures were considered, including sensitivity and specificity. RESULTS Fixed effects logistic regression consistently had the highest sensitivity and negative predictive value, yet a low specificity and positive predictive value. Of the random effects methods, PS adjustment and random effects logistic regression performed equally well or better than all the remaining PS methods for all classification accuracy measures across the studied scenarios. CONCLUSIONS Of the evaluated PS methods, only PS adjustment can be considered a viable alternative to random effects logistic regression when profiling multiple providers in different scenarios.
Collapse
Affiliation(s)
- Timo B. Brakenhoff
- Julius Center for Health Sciences and Primary CareUniversity Medical Center Utrecht, PO Box 85500, Utrecht, 3508 GA the Netherlands
| | - Kit C. B. Roes
- Julius Center for Health Sciences and Primary CareUniversity Medical Center Utrecht, PO Box 85500, Utrecht, 3508 GA the Netherlands
| | - Karel G. M. Moons
- Julius Center for Health Sciences and Primary CareUniversity Medical Center Utrecht, PO Box 85500, Utrecht, 3508 GA the Netherlands
| | - Rolf H. H. Groenwold
- Julius Center for Health Sciences and Primary CareUniversity Medical Center Utrecht, PO Box 85500, Utrecht, 3508 GA the Netherlands
| |
Collapse
|
7
|
Kristoffersen DT, Helgeland J, Clench-Aas J, Laake P, Veierød MB. Observed to expected or logistic regression to identify hospitals with high or low 30-day mortality? PLoS One 2018; 13:e0195248. [PMID: 29652941 PMCID: PMC5898724 DOI: 10.1371/journal.pone.0195248] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2017] [Accepted: 03/19/2018] [Indexed: 11/19/2022] Open
Abstract
INTRODUCTION A common quality indicator for monitoring and comparing hospitals is based on death within 30 days of admission. An important use is to determine whether a hospital has higher or lower mortality than other hospitals. Thus, the ability to identify such outliers correctly is essential. Two approaches for detection are: 1) calculating the ratio of observed to expected number of deaths (OE) per hospital and 2) including all hospitals in a logistic regression (LR) comparing each hospital to a form of average over all hospitals. The aim of this study was to compare OE and LR with respect to correctly identifying 30-day mortality outliers. Modifications of the methods, i.e., variance corrected approach of OE (OE-Faris), bias corrected LR (LR-Firth), and trimmed mean variants of LR and LR-Firth were also studied. MATERIALS AND METHODS To study the properties of OE and LR and their variants, we performed a simulation study by generating patient data from hospitals with known outlier status (low mortality, high mortality, non-outlier). Data from simulated scenarios with varying number of hospitals, hospital volume, and mortality outlier status, were analysed by the different methods and compared by level of significance (ability to falsely claim an outlier) and power (ability to reveal an outlier). Moreover, administrative data for patients with acute myocardial infarction (AMI), stroke, and hip fracture from Norwegian hospitals for 2012-2014 were analysed. RESULTS None of the methods achieved the nominal (test) level of significance for both low and high mortality outliers. For low mortality outliers, the levels of significance were increased four- to fivefold for OE and OE-Faris. For high mortality outliers, OE and OE-Faris, LR 25% trimmed and LR-Firth 10% and 25% trimmed maintained approximately the nominal level. The methods agreed with respect to outlier status for 94.1% of the AMI hospitals, 98.0% of the stroke, and 97.8% of the hip fracture hospitals. CONCLUSION We recommend, on the balance, LR-Firth 10% or 25% trimmed for detection of both low and high mortality outliers.
Collapse
Affiliation(s)
| | - Jon Helgeland
- Division for Health Services, Norwegian Institute of Public Health, Oslo, Norway
| | - Jocelyne Clench-Aas
- Division for Physical and Mental Health, Norwegian Institute of Public Health, Oslo, Norway
| | - Petter Laake
- Oslo Centre for Biostatistics and Epidemiology, Department of Biostatistics, Institute of Basic Medical Sciences, University of Oslo, Oslo, Norway
| | - Marit B. Veierød
- Oslo Centre for Biostatistics and Epidemiology, Department of Biostatistics, Institute of Basic Medical Sciences, University of Oslo, Oslo, Norway
| |
Collapse
|
8
|
Should Emergency Department Attendances be Used With or Instead of Readmission Rates as a Performance Metric?: Comparison of Statistical Properties Using National Data. Med Care 2018; 57:e1-e8. [PMID: 29601401 DOI: 10.1097/mlr.0000000000000899] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
Abstract
BACKGROUND Hospital readmissions are common and are viewed as unfavorable. They are commonly used as a measure of quality of care and, in the United States and England, are associated with financial penalties. Readmissions are not the only possible return-to-acute-care metric; patients may also attend emergency departments (EDs). OBJECTIVE To assess hospital-level return-to-acute-care metrics using statistical criteria. RESEARCH DESIGN Patient readmissions and/or ED attendances were aggregated to produce risk-standardized hospital rates. Return-to-acute-care rates at 7, 30, 90, and 365 days were assessed using key statistical properties: (i) variability between hospitals; (ii) the relative contribution of patient and nonpatient factors to variation; and (iii) the statistical power to detect performance differences. SUBJECTS We had pseudonymized administrative data on all inpatient hospital admissions and ED attendances in National Health Service hospitals in England between April 2009 and March 2011. Patients with an inpatient stay for chronic obstructive pulmonary disorder or heart failure were eligible for inclusion. MEASURES ED attendances and readmissions for patients discharged from an inpatient stay for chronic obstructive pulmonary disorder or heart failure. RESULTS Interhospital variation was greatest for ED attendance; in addition, readmission was more strongly determined by patient characteristics than was ED attendance or both combined. Because of smaller numbers, the statistical power to detect differences in rates at 7 days for any indicator was limited. CONCLUSIONS Despite the current emphasis on readmissions, we found that ED attendance within 30 days has more desirable statistical properties and therefore the potential to be a useful metric when comparing hospitals.
Collapse
|
9
|
Kadri F, Harrou F, Chaabane S, Tahon C. Time series modelling and forecasting of emergency department overcrowding. J Med Syst 2014; 38:107. [PMID: 25053208 DOI: 10.1007/s10916-014-0107-0] [Citation(s) in RCA: 48] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2014] [Accepted: 07/07/2014] [Indexed: 10/25/2022]
Abstract
Efficient management of patient flow (demand) in emergency departments (EDs) has become an urgent issue for many hospital administrations. Today, more and more attention is being paid to hospital management systems to optimally manage patient flow and to improve management strategies, efficiency and safety in such establishments. To this end, EDs require significant human and material resources, but unfortunately these are limited. Within such a framework, the ability to accurately forecast demand in emergency departments has considerable implications for hospitals to improve resource allocation and strategic planning. The aim of this study was to develop models for forecasting daily attendances at the hospital emergency department in Lille, France. The study demonstrates how time-series analysis can be used to forecast, at least in the short term, demand for emergency services in a hospital emergency department. The forecasts were based on daily patient attendances at the paediatric emergency department in Lille regional hospital centre, France, from January 2012 to December 2012. An autoregressive integrated moving average (ARIMA) method was applied separately to each of the two GEMSA categories and total patient attendances. Time-series analysis was shown to provide a useful, readily available tool for forecasting emergency department demand.
Collapse
Affiliation(s)
- Farid Kadri
- Univ. Lille Nord de France, 59000, Lille, France,
| | | | | | | |
Collapse
|