Structure and process associated with the efficiency of intensive care units in low-resource settings: An analysis of the CHECKLIST-ICU trial database

Comparing causal random forest and linear regression to estimate the independent association of organisational factors with ICU efficiency

PURPOSE

Parametric regression models have been the main statistical method for identifying average treatment effects. Causal machine learning models showed promising results in estimating heterogeneous treatment effects in causal inference. Here we aimed to compare the application of causal random forest (CRF) and linear regression modelling (LRM) to estimate the effects of organisational factors on ICU efficiency.

METHODS

A retrospective analysis of 277,459 patients admitted to 128 Brazilian and Uruguayan ICUs over three years. ICU efficiency was assessed using the average standardised efficiency ratio (ASER), measured as the average of the standardised mortality ratio (SMR) and the standardised resource use (SRU) according to the SAPS-3 score. Using a causal inference framework, we estimated and compared the conditional average treatment effect (CATE) of seven common structural and organisational factors on ICU efficiency using LRM with interaction terms and CRF.

RESULTS

The hospital mortality was 14 %; median ICU and hospital lengths of stay were 2 and 7 days, respectively. Overall median SMR was 0.97 [IQR: 0.76,1.21], median SRU was 1.06 [IQR: 0.79,1.30] and median ASER was 0.99 [IQR: 0.82,1.21]. Both CRF and LRM showed that the average number of nurses per ten beds was independently associated with ICU efficiency (CATE [95 %CI]: -0.13 [-0.24, -0.01] and -0.09 [-0.17,-0.01], respectively). Finally, CRF identified some specific ICUs with a significant CATE in exposures that did not present a significant average effect.

CONCLUSION

In general, both methods were comparable to identify organisational factors significantly associated with CATE on ICU efficiency. CRF however identified specific ICUs with significant effects, even when the average effect was nonsignificant. This can assist healthcare managers in further in-dept evaluation of process interventions to improve ICU efficiency.

Hospital performance evaluation indicators: a scoping review

BACKGROUND

Hospitals are the biggest consumers of health system budgets and hence measuring hospital performance by quantitative or qualitative accessible and reliable indicators is crucial. This review aimed to categorize and present a set of indicators for evaluating overall hospital performance.

METHODS

We conducted a literature search across three databases, i.e., PubMed, Scopus, and Web of Science, using possible keyword combinations. We included studies that explored hospital performance evaluation indicators from different dimensions.

RESULTS

We included 91 English language studies published in the past 10 years. In total, 1161 indicators were extracted from the included studies. We classified the extracted indicators into 3 categories, 14 subcategories, 21 performance dimensions, and 110 main indicators. Finally, we presented a comprehensive set of indicators with regard to different performance dimensions and classified them based on what they indicate in the production process, i.e., input, process, output, outcome and impact.

CONCLUSION

The findings provide a comprehensive set of indicators at different levels that can be used for hospital performance evaluation. Future studies can be conducted to validate and apply these indicators in different contexts. It seems that, depending on the specific conditions of each country, an appropriate set of indicators can be selected from this comprehensive list of indicators for use in the performance evaluation of hospitals in different settings.

The Association Between Prepandemic ICU Performance and Mortality Variation in COVID-19: A Multicenter Cohort Study of 35,619 Critically Ill Patients

BACKGROUND

During the COVID-19 pandemic, ICUs remained under stress and observed elevated mortality rates and high variations of outcomes. A knowledge gap exists regarding whether an ICU performing best during nonpandemic times would still perform better when under high pressure compared with the least performing ICUs.

RESEARCH QUESTION

Does prepandemic ICU performance explain the risk-adjusted mortality variability for critically ill patients with COVID-19?

STUDY DESIGN AND METHODS

This study examined a cohort of adults with real-time polymerase chain reaction-confirmed COVID-19 admitted to 156 ICUs in 35 hospitals from February 16, 2020, through December 31, 2021, in Brazil. We evaluated crude and adjusted in-hospital mortality variability of patients with COVID-19 in the ICU during the pandemic. Association of baseline (prepandemic) ICU performance and in-hospital mortality was examined using a variable life-adjusted display (VLAD) during the pandemic and a multivariable mixed regression model adjusted by clinical characteristics, interaction of performance with the year of admission, and mechanical ventilation at admission.

RESULTS

Thirty-five thousand six hundred nineteen patients with confirmed COVID-19 were evaluated. The median age was 52 years, median Simplified Acute Physiology Score 3 was 42, and 18% underwent invasive mechanical ventilation. In-hospital mortality was 13% and 54% for those receiving invasive mechanical ventilation. Adjusted in-hospital mortality ranged from 3.6% to 63.2%. VLAD in the most efficient ICUs was higher than the overall median in 18% of weeks, whereas VLAD was 62% and 84% in the underachieving and least efficient groups, respectively. The least efficient baseline ICU performance group was associated independently with increased mortality (OR, 2.30; 95% CI, 1.45-3.62) after adjusting for patient characteristics, disease severity, and pandemic surge.

INTERPRETATION

ICUs caring for patients with COVID-19 presented substantial variation in risk-adjusted mortality. ICUs with better baseline (prepandemic) performance showed reduced mortality and less variability. Our findings suggest that achieving ICU efficiency by targeting improvement in organizational aspects of ICUs may impact outcomes, and therefore should be a part of the preparedness for future pandemics.

Effective Response to Hospital Congestion Scenarios: Simulation-Based Evaluation of Decongestion Interventions

Hospital overcrowding is becoming a major concern in the modern era due to the increasing demand for hospital services. This study seeks to identify effective and efficient ways to resolve the serious problem of congestion in hospitals by testing a range of decongestion strategies with simulated scenarios. In order to determine more efficient solutions, interventions with smaller changes were consistently tested at the beginning through a simulation platform. In addition, the implementation patterns were investigated, which are important to hospital managers with respect to the decisions made to control hospital congestion. The results indicated that diverting a small number of ambulances seems to be more effective and efficient in congestion reduction compared to other approaches. Furthermore, instead of implementing an isolated approach continuously, combining one approach with other strategies is recommended as a method for dealing with hospital overcrowding.

Intensive care nurses' knowledge and use of a nursing checklist: A cross-sectional survey

AIM

This descriptive study was conducted to determine the awareness of nurses working in intensive care units about the daily patient checklist.

BACKGROUND

Checklists are used in the daily follow-up and evaluation of patients admitted to the intensive care unit.

METHOD

The research was carried out with 180 nurses through the social media account of the Turkish Intensive Care Specialists Association Nurse Commission between July 2021 and March 2022. Data were collected with a descriptive information form and intensive care unit daily patient checklist. Necessary permissions were obtained before the study.

RESULTS

Among nurses, 45.0% of them got 15 full points from the Checklist. Moreover, 81.1% of the nurses stated that they knew that a checklist should be used to help eliminate the deficiencies of daily care and treatment in the intensive care unit, while 66.7% stated that they used a checklist. It was determined that nurses knew the most about parameters 'Check the daily infection parameters', 'Glycaemic control', 'Therapy', and the least about parameters 'Thromboprophylaxis', 'Ulcer prevention', 'Hypo-hyper delirium' and 'Use a daily checklist'.

CONCLUSION

It was determined that the level of awareness of intensive care unit nurses about some parameters that should be followed daily for patient care was low.

IMPLICATIONS FOR NURSING MANAGEMENT

Checklists can be used as a guide for health care professionals in the routine daily evaluation of intensive care unit patients. It is thought that these reminder abbreviations will provide efficiency in preventing the disruption of applications, reducing medical errors, reducing mortality and morbidity, and cost.

Using data envelopment analysis to perform benchmarking in intensive care units

BACKGROUND

Studies using Data Envelopment Analysis to benchmark Intensive Care Units (ICUs) are scarce. Previous studies have focused on comparing efficiency using only performance metrics, without accounting for resources. Hence, we aimed to perform a benchmarking analysis of ICUs using data envelopment analysis.

METHODS

We performed a retrospective analysis on observational data of patients admitted to ICUs in Brazil (ORCHESTRA Study). The outputs in our data envelopment analysis model were the performance metrics: Standardized Mortality Ratio (SMR) and Standardized Resource Use (SRU); whereas the inputs consisted of three groups of variables that represented staffing patterns, structure, and strain, thus resulting in three models. We compared efficient and non-efficient units for each model. In addition, we compared our results to the efficiency matrix method and presented targets to each non-efficient unit.

RESULTS

We performed benchmarking in 93 ICUs and 129,680 patients. The median age was 64 years old, and mortality was 12%. Median SMR was 1.00 [interquartile range (IQR): 0.79-1.21] and SRU was 1.15 [IQR: 0.95-1.56]. Efficient units presented lower median physicians per bed ratio (1.44 [IQR: 1.18-1.88] vs. 1.7 [IQR: 1.36-2.00]) and nursing workload (168 hours [IQR: 168-291] vs 396 hours [IQR: 336-672]) but higher nurses per bed ratio (2.02 [1.16-2.48] vs. 1.71 [1.43-2.36]) compared to non-efficient units. Units from for-profit hospitals and specialized ICUs presented the best efficiency scores. Our results were mostly in line with the efficiency matrix method: the efficiency units in our models were mostly in the "most efficient" quadrant.

CONCLUSION

Data envelopment analysis provides managers the information needed to identify not only the outcomes to be achieved but what are the levels of resources needed to provide efficient care. Different perspectives can be achieved depending on the chosen variables. Its use jointly with the efficiency matrix can provide deeper understanding of ICU performance and efficiency.

Characterisation of the first 250,000 hospital admissions for COVID-19 in Brazil: a retrospective analysis of nationwide data

Background

Most low-income and middle-income countries (LMICs) have little or no data integrated into a national surveillance system to identify characteristics or outcomes of COVID-19 hospital admissions and the impact of the COVID-19 pandemic on their national health systems. We aimed to analyse characteristics of patients admitted to hospital with COVID-19 in Brazil, and to examine the impact of COVID-19 on health-care resources and in-hospital mortality.

Methods

We did a retrospective analysis of all patients aged 20 years or older with quantitative RT-PCR (RT-qPCR)-confirmed COVID-19 who were admitted to hospital and registered in SIVEP-Gripe, a nationwide surveillance database in Brazil, between Feb 16 and Aug 15, 2020 (epidemiological weeks 8–33). We also examined the progression of the COVID-19 pandemic across three 4-week periods within this timeframe (epidemiological weeks 8–12, 19–22, and 27–30). The primary outcome was in-hospital mortality. We compared the regional burden of hospital admissions stratified by age, intensive care unit (ICU) admission, and respiratory support. We analysed data from the whole country and its five regions: North, Northeast, Central-West, Southeast, and South.

Findings

Between Feb 16 and Aug 15, 2020, 254 288 patients with RT-qPCR-confirmed COVID-19 were admitted to hospital and registered in SIVEP-Gripe. The mean age of patients was 60 (SD 17) years, 119 657 (47%) of 254 288 were aged younger than 60 years, 143 521 (56%) of 254 243 were male, and 14 979 (16%) of 90 829 had no comorbidities. Case numbers increased across the three 4-week periods studied: by epidemiological weeks 19–22, cases were concentrated in the North, Northeast, and Southeast; by weeks 27–30, cases had spread to the Central-West and South regions. 232 036 (91%) of 254 288 patients had a defined hospital outcome when the data were exported; in-hospital mortality was 38% (87 515 of 232 036 patients) overall, 59% (47 002 of 79 687) among patients admitted to the ICU, and 80% (36 046 of 45 205) among those who were mechanically ventilated. The overall burden of ICU admissions per ICU beds was more pronounced in the North, Southeast, and Northeast, than in the Central-West and South. In the Northeast, 1545 (16%) of 9960 patients received invasive mechanical ventilation outside the ICU compared with 431 (8%) of 5388 in the South. In-hospital mortality among patients younger than 60 years was 31% (4204 of 13 468) in the Northeast versus 15% (1694 of 11 196) in the South.

Interpretation

We observed a widespread distribution of COVID-19 across all regions in Brazil, resulting in a high overall disease burden. In-hospital mortality was high, even in patients younger than 60 years, and worsened by existing regional disparities within the health system. The COVID-19 pandemic highlights the need to improve access to high-quality care for critically ill patients admitted to hospital with COVID-19, particularly in LMICs.

Funding

National Council for Scientific and Technological Development (CNPq), Coordinating Agency for Advanced Training of Graduate Personnel (CAPES), Carlos Chagas Filho Foundation for Research Support of the State of Rio de Janeiro (FAPERJ), and Instituto de Salud Carlos III.