A New Method for Evaluating Trauma Centre Outcome Performance: TRAM-Adjusted Mortality Estimates

Are trauma centers penalized for improved prehospital resuscitation?: The effect of prehospital transfusion on arrival vitals and predicted mortality

BACKGROUND

Prediction models for survival in trauma rely on arrival vital signs to generate survival probabilities. Hospitals are benchmarked on expected and observed outcomes. Prehospital blood (PB) transfusion has been shown to improve mortality, which may affect survival prediction modeling. We hypothesize that the use of PB increases the predicted survival derived from probability models compared with non-blood-based resuscitation.

METHODS

All adult trauma patients presenting to a level 1 trauma center requiring emergency release blood transfusion from January 2017 to December 2021 were reviewed. Patients were grouped into those receiving PB or those who did not (no PB). Prehospital Trauma and Injury Severity Score (TRISS) and shock index were compared with those at presentation to hospital. Univariate and multivariate regressions were performed to identify factors associated with changes in survival probability at presentation.

RESULTS

In total, 2117 patients were reviewed (PB, 1,011; no PB, 1,106). Patients receiving PB were younger (35 vs. 40 years, p < 0.001), more likely to have blunt mechanism (71% vs. 65%, p = 0.002), and more severely injured (Injury Severity Score, 27 vs. 25; p < 0.001) and had higher rates of prehospital hypotension (44% vs. 19%, p < 0.001) and shock index (1.10 vs. 0.87, p < 0.001). Upon arrival, PB patients had lower rates of ED hypotension (34% vs. 39%, p = 0.01), and significant improvements in arrival TRISS scores (+0.09 vs. -0.02, p < 0.001) and shock index (+0.10 vs. -0.07, p < 0.001) compared with prehospital. On multivariate analysis, PB was associated with a threefold increase in unexpected survivors (odds ratio, 3.28; 95% confidence interval, 2.23-4.60).

CONCLUSION

The use of PB was associated with improved probability of survival and an increase in unexpected survivors. Applying TRISS and shock index at hospital arrival does not account for en route hemostatic resuscitation, causing patients to arrive with improved vitals despite severity of injury. Caution should be used when implementing survival probability calculations using arrival vitals in centers with prehospital transfusion capability.

LEVEL OF EVIDENCE

Therapeutic/Care Management; Level IV.

Inter-rater reliability of the Abbreviated Injury Scale scores in patients with severe head injury shows good inter-rater agreement but variability between countries. An inter-country comparison study

INTRODUCTION

Substantial difference in mortality following severe traumatic brain injury (TBI) across international trauma centers has previously been demonstrated. This could be partly attributed to variability in the severity coding of the injuries. This study evaluated the inter-rater and intra-rater reliability of Abbreviated Injury Scale (AIS) scores of patients with severe TBI across three international level I trauma centers.

METHODS

A total 150 patients (50 per center) were randomly selected from each respective trauma registry: University Medical Center Utrecht (UMCU), the Netherlands; John Hunter Hospital (JHH), Australia; and Harborview Medical Center (HMC), the United States. Reliability between coders and trauma centers was measured with the intraclass correlation coefficient (ICC).

RESULTS

The reliability between the coders and the original trauma registry scores was 0.50, 0.50, and 0.41 in, respectively, UMCU, JHH, and HMC. The AIS coders at UMCU scored the most AIS codes of ≥ 4. Reliability within the trauma centers was substantial in UMCU (ICC = 0.62) and HMC (ICC = 0.78) and almost perfect in JHH (ICC = 0.85). Reliability between trauma centers was 0.70 between UMCU and JHH, 0.70 between JHH and HMC, and 0.59 between UMCU and HMC.

CONCLUSION

The results of this study demonstrated a substantial and almost perfect reliability of the AIS coders within the same trauma center, but variability across trauma centers. This indicates a need to improve inter-rater reliability in AIS coders and quality assessments of trauma registry data, specifically for patients with head injuries. Future research should study the effect of differences in AIS scoring on outcome predictions.

Do patient outcomes differ when the trauma team leader is a surgeon or non-surgeon? A multicentre cohort study

PURPOSE

Trauma team leaders (TTLs) have traditionally been general surgeons; however, some trauma centres use a mixed model of care where both surgeons and non-surgeons (primarily emergency physicians) perform this role. The objective of this multicentre study was to provide a well-powered study to determine if TTL specialty is associated with mortality among major trauma patients.

METHODS

Data were collected from provincial trauma registries at six level 1 trauma centres across Canada over a 10-year period. We included adult trauma patients (age ≥ 18 yrs) who triggered the highest-level trauma activation. The primary outcome was the difference in risk-adjusted in-hospital mortality for trauma patients receiving initial care from a surgeon versus a non-surgeon TTL.

RESULTS

Overall, 12,961 major trauma patients were included in the analysis. Initial treatment was provided by a surgeon TTL in 57.8% (n = 7513) of cases, while 42.2% (n = 5448) of patients were treated by a non-surgeon TTL. Unadjusted mortality occurred in 11.6% of patients in the surgeon TTL group and 12.7% of patients in the non-surgeon TTL group (OR 0.87, 95% CI 0.78-0.98, p = 0.02). Risk-adjusted mortality was not significantly different between patients cared for by surgeon and non-surgeon TTLs (OR 0.92, 95% CI 0.80-1.06, p = 0.23). Furthermore, we did not observe differences in risk-adjusted mortality for any of the subgroups evaluated.

CONCLUSIONS

After risk adjustment, there was no difference in mortality between trauma patients treated by surgeon or non-surgeon TTLs. Our study supports emergency physicians performing the role of TTL at level 1 trauma centres.

Effects of the establishment of trauma centres on the mortality rate among seriously injured patients: a propensity score matching retrospective study

BACKGROUND

Little evidence suggests that trauma centres are associated with a lower risk of mortality in severely injured patients (Injury Severity Score (ISS) ≥16) with multiple injuries in China. The objective of this study was to determine the association between the establishment of trauma centres and mortality among severely injured patients with multiple injuries and to identify some risk factors associated with mortality.

METHODS

A retrospective single-centre study was performed including trauma patients admitted to the First Affiliated Hospital of Nanchang University (FAHNU) between January 2016 and December 2021. To determine whether the establishment of a trauma centre was an independent predictor of mortality, logistic regression analysis and propensity score matching (PSM) were performed.

RESULTS

Among 431 trauma patients, 172 were enrolled before the trauma centre was built, while 259 were included after the trauma centre was built. A higher frequency of older age and traffic accident injury was found in patients diagnosed after the trauma centre was built. The times for the completion of CT examinations, emergency operations and blood transfusions in the "after trauma centre" group were shorter than those in the "before trauma centre" group. However, the total expenditure of patients was increased. In the overall group, univariate and multivariate logistic regression analyses showed that a higher ISS was an independent predictor for worse mortality (OR = 17.859, 95% CI, 8.207-38.86, P < 0.001), while the establishment of a trauma centre was favourable for patient survival (OR = 0.492), which was also demonstrated by PSM. After determining the cut-off value of time for the completion of CT examination, emergency operation and blood transfusion, we found that the values were within the "golden one hour", and it was better for patients when the time was less than the cut-off value.

CONCLUSION

Our study showed that for severely injured patients, the establishment of a trauma centre was favourable for a lower mortality rate. Furthermore, the completion of a CT examination, emergency surgery and blood transfusion in a timely manner and a lower ISS were associated with a decreased mortality rate.

Model for Predicting In-Hospital Mortality of Physical Trauma Patients Using Artificial Intelligence Techniques: Nationwide Population-Based Study in Korea

BACKGROUND

Physical trauma-related mortality places a heavy burden on society. Estimating the mortality risk in physical trauma patients is crucial to enhance treatment efficiency and reduce this burden. The most popular and accurate model is the Injury Severity Score (ISS), which is based on the Abbreviated Injury Scale (AIS), an anatomical injury severity scoring system. However, the AIS requires specialists to code the injury scale by reviewing a patient's medical record; therefore, applying the model to every hospital is impossible.

OBJECTIVE

We aimed to develop an artificial intelligence (AI) model to predict in-hospital mortality in physical trauma patients using the International Classification of Disease 10th Revision (ICD-10), triage scale, procedure codes, and other clinical features.

METHODS

We used the Korean National Emergency Department Information System (NEDIS) data set (N=778,111) compiled from over 400 hospitals between 2016 and 2019. To predict in-hospital mortality, we used the following as input features: ICD-10, patient age, gender, intentionality, injury mechanism, and emergent symptom, Alert/Verbal/Painful/Unresponsive (AVPU) scale, Korean Triage and Acuity Scale (KTAS), and procedure codes. We proposed the ensemble of deep neural networks (EDNN) via 5-fold cross-validation and compared them with other state-of-the-art machine learning models, including traditional prediction models. We further investigated the effect of the features.

RESULTS

Our proposed EDNN with all features provided the highest area under the receiver operating characteristic (AUROC) curve of 0.9507, outperforming other state-of-the-art models, including the following traditional prediction models: Adaptive Boosting (AdaBoost; AUROC of 0.9433), Extreme Gradient Boosting (XGBoost; AUROC of 0.9331), ICD-based ISS (AUROC of 0.8699 for an inclusive model and AUROC of 0.8224 for an exclusive model), and KTAS (AUROC of 0.1841). In addition, using all features yielded a higher AUROC than any other partial features, namely, EDNN with the features of ICD-10 only (AUROC of 0.8964) and EDNN with the features excluding ICD-10 (AUROC of 0.9383).

CONCLUSIONS

Our proposed EDNN with all features outperforms other state-of-the-art models, including the traditional diagnostic code-based prediction model and triage scale.

Validation of Quality Indicators Targeting Low-Value Trauma Care

Importance

Reducing low-value care has the potential to improve patient experiences and outcomes and free up health care resources. Sixteen quality indicators were recently developed targeting reductions in low-value trauma care based on a synthesis of the best available evidence, expert consensus, and patient preferences.

Objective

To assess the validity of quality indicators on low-value trauma care using trauma registry data.

Design, Setting, and Participants

Data from an inclusive Canadian provincial trauma system were used in this analysis. Included were all admissions for injury to any of the 57 provincial adult trauma centers between April 1, 2013, and March 31, 2020. Metrics for quality indicators were developed iteratively with clinical experts.

Main Outcomes and Measures

Validity was assessed using a priori criteria based on 5 parameters: frequency (incidence and case volume), discrimination (interhospital variation), construct validity (correlation with quality indicators on high-value care), predictive validity (correlation with quality indicators on risk-adjusted outcomes), and forecasting (correlation over time).

Results

The study sample included 136 783 patient admissions (mean [SD] age, 63 [22] years; 68 428 men [50%]). Metrics were developed for 12 of the 16 quality indicators. Six quality indicators showed moderate or high validity on all measurable parameters: initial head, cervical spine, or whole-body computed tomography for low-risk patients; posttransfer repeated computed tomography; neurosurgical consultation for mild complicated traumatic brain injury; and spine service consultation for isolated thoracolumbar process fractures. Red blood cell transfusion in low-risk patients had low frequency but had moderate or high validity on all other parameters. Five quality indicators had low validity on at least 2 parameters: repeated head CT and intensive care unit admission for mild complicated traumatic brain injury, hospital admission for minor blunt abdominal trauma, orthosis for thoracolumbar burst fractures, and surgical exploration in penetrating neck injury without hard signs.

Conclusions and Relevance

This cohort study shows the feasibility of assessing low-value trauma care using routinely collected data. It provided data on quality indicators properties that can be used to decide which quality indicators are most appropriate in a given system. Results suggest that 6 quality indicators have moderate to high validity. Their implementation now needs to be tested.

Artificial intelligence to predict in-hospital mortality using novel anatomical injury score

The aim of the study is to develop artificial intelligence (AI) algorithm based on a deep learning model to predict mortality using abbreviate injury score (AIS). The performance of the conventional anatomic injury severity score (ISS) system in predicting in-hospital mortality is still limited. AIS data of 42,933 patients registered in the Korean trauma data bank from four Korean regional trauma centers were enrolled. After excluding patients who were younger than 19 years old and those who died within six hours from arrival, we included 37,762 patients, of which 36,493 (96.6%) survived and 1269 (3.4%) deceased. To enhance the AI model performance, we reduced the AIS codes to 46 input values by organizing them according to the organ location (Region-46). The total AIS and six categories of the anatomic region in the ISS system (Region-6) were used to compare the input features. The AI models were compared with the conventional ISS and new ISS (NISS) systems. We evaluated the performance pertaining to the 12 combinations of the features and models. The highest accuracy (85.05%) corresponded to Region-46 with DNN, followed by that of Region-6 with DNN (83.62%), AIS with DNN (81.27%), ISS-16 (80.50%), NISS-16 (79.18%), NISS-25 (77.09%), and ISS-25 (70.82%). The highest AUROC (0.9084) corresponded to Region-46 with DNN, followed by that of Region-6 with DNN (0.9013), AIS with DNN (0.8819), ISS (0.8709), and NISS (0.8681). The proposed deep learning scheme with feature combination exhibited high accuracy metrics such as the balanced accuracy and AUROC than the conventional ISS and NISS systems. We expect that our trial would be a cornerstone of more complex combination model.

Development and validation of a risk-adjustment model for mortality and hospital length of stay for trauma patients: a prospective registry-based study in Australia

OBJECTIVES

Adequate risk adjustment for factors beyond the control of the healthcare system contributes to the process of transparent and equitable benchmarking of trauma outcomes. Current risk adjustment models are not optimal in terms of the number and nature of predictor variables included in the model and the treatment of missing data. We propose a statistically robust and parsimonious risk adjustment model for the purpose of benchmarking.

SETTING

This study analysed data from the multicentre Australia New Zealand Trauma Registry from 1 July 2016 to 30 June 2018 consisting of 31 trauma centres.

OUTCOME MEASURES

The primary endpoints were inpatient mortality and length of hospital stay. Firth logistic regression and robust linear regression models were used to study the endpoints, respectively. Restricted cubic splines were used to model non-linear relationships with age. Model validation was performed on a subset of the dataset.

RESULTS

Of the 9509 patients in the model development cohort, 72% were male and approximately half (51%) aged over 50 years . For mortality, cubic splines in age, injury cause, arrival Glasgow Coma Scale motor score, highest and second-highest Abbreviated Injury Scale scores and shock index were significant predictors. The model performed well in the validation sample with an area under the curve of 0.93. For length of stay, the identified predictor variables were similar. Compared with low falls, motor vehicle occupants stayed on average 2.6 days longer (95% CI: 2.0 to 3.1), p<0.001. Sensitivity analyses did not demonstrate any marked differences in the performance of the models.

CONCLUSION

Our risk adjustment model of six variables is efficient and can be reliably collected from registries to enhance the process of benchmarking.

Trauma system accreditation and patient outcomes in British Columbia: an interrupted time series analysis

OBJECTIVE

We aim to assess the impact of several accreditation cycles of trauma centers on patient outcomes, specifically in-hospital mortality, complications and hospital length of stay.

DESIGN

Interrupted time series.

SETTING

British Columbia, Canada.

PARTICIPANTS

Trauma patients admitted to all level I and level II trauma centers between January 2008 and March 2018.

EXPOSURE

Accreditation.

MAIN OUTCOMES AND MEASURES

We first computed quarterly estimates of the proportions of in-hospital mortality, complications and survival to discharge standardized for change in patient case-mix using prognostic scores and the Aalen-Johansen estimator of the cumulative incidence function. Piecewise regressions were then used to estimate the change in levels and trends for patient outcomes following accreditation.

RESULTS

For in-hospital mortality and major complications, the impact of accreditation seems to be associated with short- and long-term reductions after the first cycle and only short-term reductions for subsequent cycles. However, the 95% confidence intervals for these estimates were wide, and we lacked the precision to consistently conclude that accreditation is beneficial.

CONCLUSIONS

Applying a quasi-experimental design to time series accounting for changes in patient case-mix, our results suggest that accreditation might reduce in-hospital mortality and major complications. However, there was uncertainty around the estimates of accreditation. Further studies looking at clinical processes of care and other outcomes such as patient or health staff satisfaction are needed.

Impact of trauma centre accreditation on mortality and complications in a Canadian trauma system: an interrupted time series analysis

BACKGROUND

Periodic external accreditation visits aiming to determine whether trauma centres are fulfilling the criteria for optimal care are part of most trauma systems. However, despite the growing trend towards accreditation of trauma centres, its impact on patient outcomes remains unclear. In addition, a recent systematic review found inconsistent results on the association between accreditation and patient outcomes, mostly due to the lack of robust controls. We aim to address these gaps by assessing the impact of trauma centre accreditation on patient outcomes, specifically in-hospital mortality and complications, using an interrupted time series (ITS) design.

METHODS

We included all major trauma admissions to five level I and four level II trauma centres in Quebec, Canada between 2008 and 2017. In order to perform ITS, we first obtained monthly and quarterly estimates of the proportions of in-hospital mortality and complications, respectively, for level I and level II centres. Prognostic scores were used to standardise these proportions to account for changes in patient case mix and segmented regressions with autocorrelated errors were used to estimate changes in levels and trends in both outcomes following accreditation.

RESULTS

There were 51 035 admissions, including 20 165 for major trauma during the study period. After accounting for changes in patient case mix and secular trend in studied outcomes, we globally did not observe an association between accreditation and patient outcomes. However, associations were heterogeneous across centres. For example, in a level II centre with worsening preaccreditation outcomes, accreditation led to -9.08 (95% CI -13.29 to -4.87) and -9.60 (95% CI -15.77 to -3.43) percentage point reductions in mortality and complications, respectively.

CONCLUSION

Accreditation seemed to be beneficial for centres that were experiencing a decrease in performance preceding accreditation.

High-Performance Trauma Centers in a Single-State Trauma System : Big Saves or Marginal Gains?

BACKGROUND

Trauma centers with low observed:expected (O:E) mortality ratios are considered high performers; however, it is unknown whether improvements in this ratio are due to a small number of unexpected survivors with high mortality risk (big saves) or a larger number of unexpected survivors with moderate mortality risk (marginal gains). We hypothesized that the highest-performing centers achieve that status via larger numbers of unexpected survivors with moderate mortality risk.

METHODS

We calculated O:E ratios for trauma centers in Pennsylvania for 2016 using a risk-adjusted mortality model. We identified high and low performers as centers whose 95% CIs did not cross 1. We visualized differences between these centers by plotting patient-level observed and expected mortality; we then examined differences in a subset of patients with a predicted mortality of ≥10% using the chi-squared test.

RESULTS

One high performer and 1 low performer were identified. The high performer managed a population with more blunt injuries (97.2% vs 93.6%, P < .001) and a higher median Injury Severity Score (14 vs 11, P < .001). There was no difference in survival between these centers in patients with an expected mortality of <10% (98.0% vs 96.7%, P = .11) or ≥70% (23.5% vs 10.8%, P = .22), but there was a difference in the subset with an expected mortality of ≥10% (77.5% vs 43.1%, P < .001).

CONCLUSIONS

Though patients with very low predicted mortality do equally well in high-performing and low-performing centers, the fact that performance seems determined by outcomes of patients with moderate predicted mortality favors a "marginal gains" theory.

Trends of clinical outcomes in patients with a Traumatic Brain Injury (TBI) in Canada between 2006 and 2012

BACKGROUND

Traumatic brain injuries (TBI) are the leading cause of death for people aged <40 years. In Canada, the structure of trauma care has evolved independently across provinces during the last decade. However, little is known about the evolution of clinical outcomes. We aimed to compare trends in hospital mortality, unplanned readmission, hospital length of stay (LOS) and intensive care unit (ICU) LOS for TBI between 2006 and 2012 across Canadian provinces.

METHODS

We conducted a retrospective multicentre cohort study based on TBI admissions across Canadian level I and II trauma centres. Data were extracted from the National Trauma Registry linked to hospital discharge databases. All adults with an injury severity score ≥12 were included. Multilevel generalized linear models were used to evaluate trends in clinical outcomes.

RESULTS

Between 2006 and 2012, we observed a decrease in mortality in Canada (odd ratio [OR] = 0.95; 95% confidence intervals [CI] = 0.92-0.98) mostly driven by Ontario (OR = 0.95; 95% CI = 0.93-0.98). We observed a significant decrease in hospital length of stay in Canada (hazard ratio [HR]: hazard of being discharged alive from hospital = 1.02; 95% CI = 1.01-1.02) mostly driven by a decrease in Quebec (HR = 1.03; 95% CI = 1.01-1.04). We observed a decrease in ICU Length of stay only in Alberta (HR = 1.05; 95% CI = 1.01-1.09). No trend was observed for hospital readmissions.

CONCLUSION

We observed significant decreases in mortality, hospital and ICU length of stay for TBI in Canada between 2006 and 2012 but only in certain provinces. This study may represent the first step towards a better understanding of the influence of trauma system configuration on the burden of injuries in Canada.

Improvement of the performance of survival prediction in the ageing blunt trauma population: A cohort study

Introduction

The overestimation of survival predictions in the ageing trauma population results in negative benchmark numbers in hospitals that mainly treat elderly patients. The aim of this study was to develop and validate a modified Trauma and Injury Severity Score (TRISS) for accurate survival prediction in the ageing blunt trauma population.

Methods

This retrospective study was conducted with data from two Dutch Trauma regions. Missing values were imputed. New prediction models were created in the development set, including age (continuous or categorical) and Anesthesiologists Physical Status (ASA). The models were externally validated. Subsets were created based on age (≥75 years) and the presence of hip fracture. Model performance was assessed by proportion explained variance (Nagelkerke R²), discrimination (Area Under the curve of the Receiver Operating Characteristic, AUROC) and visually with calibration plots. A final model was created based on both datasets.

Results

No differences were found between the baseline characteristics of the development dataset (n = 15,530) and the validation set (n = 15,504). The inclusion of ASA in the prediction models showed significant improved discriminative abilities in the two subsets (e.g. AUROC of 0.52 [95% CI: 0.46, 0.58] vs. 0.74 [95% CI: 0.69, 0.78] for elderly patients with hip fracture) and an increase in the proportion explained variance (R² = 0.32 to R² = 0.35 in the total cohort). The final model showed high agreement between observed and predicted survival in the calibration plot, also in the subsets.

Conclusions

Including ASA and age (continuous) in survival prediction is a simple adjustment of the TRISS methodology to improve survival predictions in the ageing blunt trauma population. A new model is presented, through which even patients with isolated hip fractures could be included in the evaluation of trauma care.

Hospital and Intensive Care Unit Length of Stay for Injury Admissions: A Pan-Canadian Cohort Study

OBJECTIVE

To assess the variation in hospital and intensive care unit (ICU) length of stay (LOS) for injury admissions across Canadian provinces and to evaluate the relative contribution of patient case mix and treatment-related factors (intensity of care, complications, and discharge delays) to explaining observed variations.

BACKGROUND

Identifying unjustified interprovider variations in resource use and the determinants of such variations is an important step towards optimizing health care.

METHODS

We conducted a multicenter, retrospective cohort study on admissions for major trauma (injury severity score >12) to level I and II trauma centers across Canada (2006-2012). We used data from the Canadian National Trauma Registry linked to hospital discharge data to compare risk-adjusted hospital and ICU LOS across provinces.

RESULTS

Risk-adjusted hospital LOS was shortest in Ontario (10.0 days) and longest in Newfoundland and Labrador (16.1 days; P < 0.001). Risk-adjusted ICU LOS was shortest in Québec (4.4 days) and longest in Alberta (6.1 days; P < 0.001). Patient case-mix explained 32% and 8% of interhospital variations in hospital and ICU LOS, respectively, whereas treatment-related factors explained 63% and 22%.

CONCLUSIONS

We observed significant variation in risk-adjusted hospital and ICU LOS across trauma systems in Canada. Provider ranks on hospital LOS were not related to those observed for ICU LOS. Treatment-related factors explained more interhospital variation in LOS than patient case-mix. Results suggest that interventions targeting reductions in low-value procedures, prevention of adverse events, and better discharge planning may be most effective for optimizing LOS for injury admissions.

Canadian benchmarks for acute injury care

BACKGROUND

Acute care injury outcomes vary substantially across Canadian provinces and trauma centres. Our aim was to develop Canadian benchmarks to monitor mortality and hospital length of stay (LOS) for injury admissions.

METHODS

Benchmarks were derived using data from the Canadian National Trauma Registry on patients with major trauma admitted to any level I or II trauma centre in Canada and from the following patient subgroups: isolated traumatic brain injury (TBI), isolated thoracoabdominal injury, multisystem blunt injury, age 65 years or older. We assessed predictive validity using measures of discrimination and calibration, and performed sensitivity analyses to assess the impact of replacing analytically complex methods (multiple imputation, shrinkage estimates and flexible modelling) with simple models that can be implemented locally.

RESULTS

The mortality risk adjustment model had excellent discrimination and calibration (area under the receiver operating characteristic curve 0.886, Hosmer-Lemeshow 36). The LOS risk-adjustment model predicted 29% of the variation in LOS. Overall, observed:expected ratios of mortality and mean LOS generated by an analytically simple model correlated strongly with those generated by analytically complex models (r > 0.95, κ on outliers > 0.90).

CONCLUSION

We propose Canadian benchmarks that can be used to monitor quality of care in Canadian trauma centres using Excel (see the appendices, available at canjsurg.ca). The program can be implemented using local trauma registries, providing that at least 100 patients are available for analysis.

Mortality prediction models in the general trauma population: A systematic review

BACKGROUND

Trauma is the leading cause of death in individuals younger than 40 years. There are many different models for predicting patient outcome following trauma. To our knowledge, no comprehensive review has been performed on prognostic models for the general trauma population. Therefore, this review aimed to describe (1) existing mortality prediction models for the general trauma population, (2) the methodological quality and (3) which variables are most relevant for the model prediction of mortality in the general trauma population.

METHODS

An online search was conducted in June 2015 using Embase, Medline, Web of Science, Cinahl, Cochrane, Google Scholar and PubMed. Relevant English peer-reviewed articles that developed, validated or updated mortality prediction models in a general trauma population were included.

RESULTS

A total of 90 articles were included. The cohort sizes ranged from 100 to 1,115,389 patients, with overall mortality rates that ranged from 0.6% to 35%. The Trauma and Injury Severity Score (TRISS) was the most commonly used model. A total of 258 models were described in the articles, of which only 103 models (40%) were externally validated. Cases with missing values were often excluded and discrimination of the different prediction models ranged widely (AUROC between 0.59 and 0.98). The predictors were often included as dichotomized or categorical variables, while continuous variables showed better performance.

CONCLUSION

Researchers are still searching for a better mortality prediction model in the general trauma population. Models should 1) be developed and/or validated using an adequate sample size with sufficient events per predictor variable, 2) use multiple imputation models to address missing values, 3) use the continuous variant of the predictor if available and 4) incorporate all different types of readily available predictors (i.e., physiological variables, anatomical variables, injury cause/mechanism, and demographic variables). Furthermore, while mortality rates are decreasing, it is important to develop models that predict physical, cognitive status, or quality of life to measure quality of care.

Donabedian's structure-process-outcome quality of care model: Validation in an integrated trauma system

BACKGROUND

According to Donabedian's health care quality model, improvements in the structure of care should lead to improvements in clinical processes that should in turn improve patient outcome. This model has been widely adopted by the trauma community but has not yet been validated in a trauma system. The objective of this study was to assess the performance of an integrated trauma system in terms of structure, process, and outcome and evaluate the correlation between quality domains.

METHODS

Quality of care was evaluated for patients treated in a Canadian provincial trauma system (2005-2010; 57 centers, n = 63,971) using quality indicators (QIs) developed and validated previously. Structural performance was measured by transposing on-site accreditation visit reports onto an evaluation grid according to American College of Surgeons criteria. The composite process QI was calculated as the average sum of proportions of conformity to 15 process QIs derived from literature review and expert opinion. Outcome performance was measured using risk-adjusted rates of mortality, complications, and readmission as well as hospital length of stay (LOS). Correlation was assessed with Pearson's correlation coefficients.

RESULTS

Statistically significant correlations were observed between structure and process QIs (r = 0.33), and process and outcome QIs (r = -0.33 for readmission, r = -0.27 for LOS). Significant positive correlations were also observed between outcome QIs (r = 0.37 for mortality-readmission; r = 0.39 for mortality-LOS and readmission-LOS; r = 0.45 for mortality-complications; r = 0.34 for readmission-complications; 0.63 for complications-LOS).

CONCLUSION

Significant correlations between quality domains observed in this study suggest that Donabedian's structure-process-outcome model is a valid model for evaluating trauma care. Trauma centers that perform well in terms of structure also tend to perform well in terms of clinical processes, which in turn has a favorable influence on patient outcomes.

LEVEL OF EVIDENCE

Prognostic study, level III.

Reduced population burden of road transport-related major trauma after introduction of an inclusive trauma system

This population-based study found that since the introduction of an inclusive, regionalized trauma system in Victoria, Australia, the burden of road transport–related serious injury has decreased. Hospitalized major trauma cases increased, but disability burden per case declined. Increased survival did not result in an overall increase in nonfatal injury burden.

Objective:

To describe the burden of road transport–related serious injury in Victoria, Australia, over a 10-year period, after the introduction of an integrated trauma system.

Background:

Road traffic injury is a leading cause of death and disability worldwide. Efforts to improve care of the injured are important for reducing burden, but the impact of trauma care systems on burden and cost of road traffic injury has not been evaluated.

Methods:

All road transport–related deaths and major trauma (injury severity score >12) cases were extracted from population-based coroner and trauma registry data sets for July 2001 to June 2011. Modeling was used to assess changes in population incidence rates and odds of in-hospital mortality. Disability-adjusted life years, combining years of life lost and years lived with disability, were calculated. Cost of health loss was calculated from estimates of the value of a disability-adjusted life year.

Results:

Incidence of road transport–related deaths decreased (incidence rate ratio 0.95, 95% confidence interval: 0.94–0.96), whereas the incidence of hospitalized major trauma increased (incidence rate ratio 1.03, 95% confidence interval: 1.02–1.04). Years of life lost decreased by 43%, and years lived with disability increased by 32%, with an overall 28% reduction in disability-adjusted life years over the decade. There was a cost saving per case of A$633,446 in 2010–2011 compared with the 2001–2002 financial year.

Conclusions:

Since introduction of the trauma system in Victoria, Australia, the burden of road transport–related serious injury has decreased. Hospitalized major trauma cases increased, whereas disability burden per case declined. Increased survival does not necessarily result in an overall increase in nonfatal injury burden.

Evaluating hospital performance based on excess cause-specific incidence

Formal evaluation of hospital performance in specific types of care is becoming an indispensable tool for quality assurance in the health care system. When the prime concern lies in reducing the risk of a cause-specific event, we propose to evaluate performance in terms of an average excess cumulative incidence, referring to the center's observed patient mix. Its intuitive interpretation helps give meaning to the evaluation results and facilitates the determination of important benchmarks for hospital performance. We apply it to the evaluation of cerebrovascular deaths after stroke in Swedish stroke centers, using data from Riksstroke, the Swedish stroke registry. © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Derivation and validation of a quality indicator of acute care length of stay to evaluate trauma care

OBJECTIVE

To derive and internally validate a quality indicator (QI) for acute care length of stay (LOS) after admission for injury.

BACKGROUND

Unnecessary hospital days represent an estimated 20% of total LOS implying an important waste of resources as well as increased patient exposure to hospital-acquired infections and functional decline.

METHODS

This study is based on a multicenter, retrospective cohort from a Canadian provincial trauma system (2005-2010; 57 trauma centers; n = 57,524). Data were abstracted from the provincial trauma registry and the hospital discharge database. Candidate risk factors were identified by expert consensus and selected for model derivation using bootstrap resampling. The validity of the QI was evaluated in terms of interhospital discrimination, construct validity, and forecasting.

RESULTS

The risk adjustment model explains 37% of the variation in LOS. The QI discriminates well across trauma centers (coefficient of variation = 0.02, 95% confidence interval: 0.011-0.028) and is correlated with the QI on processes of care (r = -0.32), complications (r = 0.66), unplanned readmissions (r = 0.38), and mortality (r = 0.35). Performance in 2005 to 2007 was predictive of performance in 2008 to 2010 (r = 0.80).

CONCLUSIONS

We have developed a QI on the basis of risk-adjusted LOS to evaluate trauma care that can be implemented with routinely collected data. The QI is based on a robust risk adjustment model with good internal and temporal validity, and demonstrates good properties in terms of discrimination, construct validity, and forecasting. This QI can be used to target interventions to reduce LOS, which will lead to more efficient resource use and may improve patient outcomes after injury.

Establishing consensus on the definition of an isolated hip fracture for trauma system performance evaluation: A systematic review

Background:

Risk-adjusted mortality is widely used to benchmark trauma center care. Patients presenting with isolated hip fractures (IHFs) are usually excluded from these evaluations. However, there is no standardized definition of an IHF. We aimed to evaluate whether there is consensus on the definition of an IHF used as an exclusion criterion in studies evaluating the performance of trauma centers in terms of mortality.

Materials and Methods:

We conducted a systematic review of observational studies. We searched the electronic databases MEDLINE, EMBASE, BIOSIS, The Cochrane Library, CINAHL, TRIP Database, and PROQUEST for cohort studies that presented data on mortality to assess the performance of trauma centers and excluded IHF. A standardized, piloted data abstraction form was used to extract data on study settings, IHF definitions and methodological quality of included studies. Consensus was considered to be reached if more than 50% of studies used the same definition of IHF.

Results:

We identified 8,506 studies of which 11 were eligible for inclusion. Only two studies (18%) used the same definition of an IHF. Three (27%) used a definition based on Abbreviated Injury Scale (AIS) Codes and five (45%) on International Classification of Diseases (ICD) codes. Four (36%) studies had inclusion criteria based on age, five (45%) on secondary injuries, and four (36%) on the mechanism of injury. Eight studies (73%) had good overall methodological quality.

Conclusions:

We observed important heterogeneity in the definition of an IHF used as an exclusion criterion in studies evaluating the performance of trauma centers. Consensus on a standardized definition is needed to improve the validity of evaluations of the quality of trauma care.

Trauma center performance indicators for nonfatal outcomes: a scoping review of the literature

BACKGROUND

According to Donabedian's framework, outcomes covering the following six domains should be used to evaluate health care quality: death, adverse events, readmissions to hospital, resource use, quality of life, and ability to function in daily activities. The objective of this study was to identify the nonfatal outcomes that have been used to evaluate the performance of trauma hospitals. Secondary objectives were to describe definitions and methodological quality.

METHODS

We performed a scoping literature review of studies using at least one nonfatal outcome to evaluate the performance of acute care hospitals for the treatment of general trauma populations. We searched MEDLINE, EMBASE, Cochrane central, CINAHL, BIOSIS, TRIP and ProQuest databases. Methodological quality was evaluated using elements of the STROBE statement and the Downs and Black tool.

RESULTS

Of 14,521 citations, 40 were eligible for inclusion. We identified 14 nonfatal outcomes as follows: (i) adverse events including complications (used in 35 evaluations), missed injuries (n = 4), reintubation (n = 2), unplanned intensive care unit admissions (n = 2), and unplanned surgeries (n = 4); (ii) resource use including hospital (n = 19), intensive care unit (n = 15), and ventilator (n = 4) length of stay, inappropriate hospital stay (n = 1), and potentially unnecessary care (n = 1); (iii) hospital readmissions (n = 4); and (iv) ability to function in daily activities including functional capacity (n = 2), and discharge destination (n = 3). No measures of quality of life were identified. There was high heterogeneity in the definitions used. Only 18% of studies had high methodological quality.

CONCLUSION

Among recommended domains of nonfatal outcomes, adverse events and resource use were frequently used to evaluate trauma care, readmissions and function in daily activities were rarely used, and quality of life was never used. In addition, definitions of nonfatal outcomes were variable, and methodological quality was low. There is a need to develop valid and reliable performance indicators based on each domain of Donabedian's framework to evaluate trauma care.

Evaluating trauma center structural performance: The experience of a Canadian provincial trauma system

BACKGROUND

Indicators of structure, process, and outcome are required to evaluate the performance of trauma centers to improve the quality and efficiency of care. While periodic external accreditation visits are part of most trauma systems, a quantitative indicator of structural performance has yet to be proposed. The objective of this study was to develop and validate a trauma center structural performance indicator using accreditation report data.

MATERIALS AND METHODS

Analyses were based on accreditation reports completed during on-site visits in the Quebec trauma system (1994-2005). Qualitative report data was retrospectively transposed onto an evaluation grid and the weighted average of grid items was used to quantify performance. The indicator of structural performance was evaluated in terms of test-retest reliability (kappa statistic), discrimination between centers (coefficient of variation), content validity (correlation with accreditation decision, designation level, and patient volume) and forecasting (correlation between visits performed in 1994-1999 and 1998-2005).

RESULTS

Kappa statistics were >0.8 for 66 of the 73 (90%) grid items. Mean structural performance score over 59 trauma centers was 47.4 (95% CI: 43.6-51.1). Two centers were flagged as outliers and the coefficient of variation was 31.2% (95% CI: 25.5% to 37.6%), showing good discrimination. Correlation coefficients of associations with accreditation decision, designation level, and volume were all statistically significant (r = 0.61, -0.40, and 0.24, respectively). No correlation was observed over time (r = 0.03).

CONCLUSION

This study demonstrates the feasibility of quantifying trauma center structural performance using accreditation reports. The proposed performance indicator shows good test-retest reliability, between-center discrimination, and construct validity. The observed variability in structural performance across centers and over-time underlines the importance of evaluating structural performance in trauma systems at regular intervals to drive quality improvement efforts.

Comparing regression-adjusted mortality to standardized mortality ratios for trauma center profiling

Background:

Trauma center profiling is commonly performed with Standardized Mortality Ratios (SMRs). However, comparison of SMRs across trauma centers with different case mix can induce confounding leading to biased trauma center ranks. We hypothesized that Regression-Adjusted Mortality (RAM) estimates would provide a more valid measure of trauma center performance than SMRs.

Objective:

Compare trauma center ranks generated by RAM estimates to those generated by SMRs.

Materials and Methods:

The study was based on data from a provincial Trauma Registry (1999-2006; n = 88,235). SMRs were derived as the ratio of observed to expected deaths using: (1) the study population as an internal standard, (2) the US National Trauma Data Bank as an external standard. The expected death count was calculated as the sum of mortality probabilities for all patients treated in a hospital conditional on the injury severity score, the revised trauma score, and age. RAM estimates were obtained directly from a hierarchical logistic regression model.

Results:

Crude mortality was 5.4% and varied between 1.3% and 13.5% across the 59 trauma centers. When trauma center ranks from internal SMRs and RAM were compared, 49 out of 59 centers changed rank and six centers changed by more than five ranks. When trauma center ranks from external SMRs and RAM were compared, 55 centers changed rank and 17 changed by more than five ranks.

Conclusions:

The results of this study suggest that the use of SMRs to rank trauma centers in terms of mortality may be misleading. RAM estimates represent a potentially more valid method of trauma center profiling.

Hazard regression models of early mortality in trauma centers

BACKGROUND

Factors affecting early hospital deaths after trauma can be different from factors affecting later hospital deaths, and the distribution of short and long prehospital times can vary among hospitals. Hazard regression (HR) models might therefore be more useful than logistic regression (LR) models for analysis of trauma mortality, especially when treatment effects at different time points are of interest.

STUDY DESIGN

We obtained data for trauma center patients from the 2008-2009 National Trauma Data Bank. Patients were included if they had complete data for prehospital times, hospital times, survival outcomes, age, vital signs, and severity scores. Patients were excluded if pulseless on admission, transferred in or out, or had an Injury Severity Score <9. Using covariates proposed for the Trauma Quality Improvement Program and an indicator for each hospital, we compared LR models predicting survival at 8 hours after injury with HR models with survival censored at 8 hours. Hazard regression models were then modified to allow time-varying hospital effects.

RESULTS

A total of 85,327 patients in 161 hospitals met inclusion criteria. Crude hazards peaked initially and then declined steadily. When hazard ratios were assumed constant in HR models, they were similar to odds ratios in LR models associating increased mortality with increased age, firearm mechanism, increased severity, more deranged physiology, and estimated hospital-specific effects. However, when hospital effects were allowed to vary by time, HR models demonstrated that hospital outliers were not the same at different times after injury.

CONCLUSIONS

Hazard regression models with time-varying hazard ratios reveal inconsistencies in treatment effects, data quality, and/or timing of early death among trauma centers. Hazard regression models are generally more flexible than LR models, can be adapted for censored data, and potentially offer a better tool for analysis of factors affecting early death after injury.

Trauma centre outcome performance: a comparison of young adults and geriatric patients in an inclusive trauma system

BACKGROUND

Elderly trauma patients represent a unique clientele requiring specialised care but they rarely benefit from standardised care strategies within trauma systems. We aimed to evaluate whether trauma centres with lower/higher than expected mortality amongst patients <65 years of age have similar results for geriatric patients. A secondary objective was to compare transfer to level I/II trauma centres across age groups.

METHODS

The study was based on data from a Canadian provincial trauma registry (1999-2006). Outcome performance was evaluated with estimates of risk-adjusted 30-day mortality generated for each of the system's 57 adult trauma centres. Agreement in performance results was evaluated with correlation coefficients.

RESULTS

The study sample comprised 55,283 young adults (3.5% mortality) and 30,960 geriatric patients (8.2% mortality). The two age groups only had one out of six outliers in common. Hospital ranks amongst young adults were not correlated to those assigned amongst geriatric patients (r = 0.01, 95%CI -0.25;0.27). Correlation was also low for patients with major trauma (r = 0.20, 95%CI -0.06;0.44). Amongst patients with severe head injuries initially received in a level III/IV centre, 81% of young adults versus 71% of geriatric patients were transferred to a level I/II centre (p<0.0001).

CONCLUSIONS

Trauma centres that have low risk-adjusted mortality for young adults do not necessarily do so for geriatric patients. In addition, geriatric patients with severe head injuries are less likely to be treated in neurosurgical trauma centres. Further research is needed to identify determinants of inter-hospital variation in outcome for geriatric trauma patients.

Definition of mortality for trauma center performance evaluation: a comparative study

OBJECTIVE

Mortality is widely used as a performance indicator to evaluate the quality of trauma care, but there is no consensus on the most appropriate definition. Our objective was to evaluate the influence of the definition of mortality in terms of the place (in-hospital or postdischarge) and time (30 days and 3, 6, and 12 months) of death on the results of trauma center performance evaluations according to the patients' ages.

DESIGN

Multicenter retrospective cohort study.

SETTING

Inclusive Canadian provincial trauma system.

PATIENTS

Adults admitted between 1999 and 2006 with a maximum abbreviated injury severity score≥3 (n=47,261).

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Trauma registry data were linked to vital statistics data to obtain mortality up to 12 months postadmission. Observed mortality was compared to that expected according to provincial population mortality rates. Trauma center performance was evaluated with risk-adjusted mortality estimates. Agreement between performance results based on different definitions of mortality was evaluated with correlation coefficients; >.9 was considered acceptable. Analyses were stratified by predefined age categories (16-64, 65-84, and ≥85 yrs). A total of 3,338 patients (7%) died in-hospital, and 1,794 patients (4%) died postdischarge. Among patients 16-64 yrs old, 30-day hospital mortality represented 83% of all deaths and correlation coefficients across all definitions of mortality were >.9. In patients 65-84 yrs old, 30-day hospital mortality represented 52% of all deaths, observed mortality reached expected rates at around 6 months, and agreement across mortality definitions was low.

CONCLUSIONS

We observed an important variation in performance evaluation results across definitions of mortality, specifically in patients aged≥65 yrs. Half of the deaths among elders occurred later than 30 days following admission, including a significant number postdischarge. Results suggest that if performance evaluations include elderly patients, data on postdischarge mortality up to 6 months following admission are required.

Mortality associated with withdrawal of life-sustaining therapy for patients with severe traumatic brain injury: a Canadian multicentre cohort study

BACKGROUND

Severe traumatic brain injury often leads to death from withdrawal of life-sustaining therapy, although prognosis is difficult to determine.

METHODS

To evaluate variation in mortality following the withdrawal of life-sustaining therapy and hospital mortality in patients with critical illness and severe traumatic brain injury, we conducted a two-year multicentre retrospective cohort study in six Canadian level-one trauma centres. The effect of centre on hospital mortality and withdrawal of life-sustaining therapy was evaluated using multivariable logistic regression adjusted for baseline patient-level covariates (sex, age, pupillary reactivity and score on the Glasgow coma scale).

RESULTS

We randomly selected 720 patients with traumatic brain injury for our study. The overall hospital mortality among these patients was 228/720 (31.7%, 95% confidence interval [CI] 28.4%-35.2%) and ranged from 10.8% to 44.2% across centres (χ(2) test for overall difference, p < 0.001). Most deaths (70.2% [160/228], 95% CI 63.9%-75.7%) were associated with withdrawal of life-sustaining therapy, ranging from 45.0% (18/40) to 86.8% (46/53) (χ(2) test for overall difference, p < 0.001) across centres. Adjusted odd ratios (ORs) for the effect of centre on hospital mortality ranged from 0.61 to 1.55 (p < 0.001). The incidence of withdrawal of life-sustaining therapy varied by centre, with ORs ranging from 0.42 to 2.40 (p = 0.001). About one half of deaths that occurred following the withdrawal of life-sustaining therapies happened within the first three days of care.

INTERPRETATION

We observed significant variation in mortality across centres. This may be explained in part by regional variations in physician, family or community approaches to the withdrawal of life-sustaining therapy. Considering the high proportion of early deaths associated with the withdrawal of life-sustaining therapy and the limited accuracy of current prognostic indicators, caution should be used regarding early withdrawal of life-sustaining therapy following severe traumatic brain injury.

Influence of socioeconomic status on trauma center performance evaluations in a Canadian trauma system

BACKGROUND

Trauma center performance evaluations generally include adjustment for injury severity, age, and comorbidity. However, disparities across trauma centers may be due to other differences in source populations that are not accounted for, such as socioeconomic status (SES). We aimed to evaluate whether SES influences trauma center performance evaluations in an inclusive trauma system with universal access to health care.

STUDY DESIGN

The study was based on data collected between 1999 and 2006 in a Canadian trauma system. Patient SES was quantified using an ecologic index of social and material deprivation. Performance evaluations were based on mortality adjusted using the Trauma Risk Adjustment Model. Agreement between performance results with and without additional adjustment for SES was evaluated with correlation coefficients.

RESULTS

The study sample comprised a total of 71,784 patients from 48 trauma centers, including 3,828 deaths within 30 days (4.5%) and 5,549 deaths within 6 months (7.7%). The proportion of patients in the highest quintile of social and material deprivation varied from 3% to 43% and from 11% to 90% across hospitals, respectively. The correlation between performance results with or without adjustment for SES was almost perfect (r = 0.997; 95% CI 0.995-0.998) and the same hospital outliers were identified.

CONCLUSIONS

We observed an important variation in SES across trauma centers but no change in risk-adjusted mortality estimates when SES was added to adjustment models. Results suggest that after adjustment for injury severity, age, comorbidity, and transfer status, disparities in SES across trauma center source populations do not influence trauma center performance evaluations in a system offering universal health coverage.