Development and validation of a novel computer-aided score to predict the risk of in-hospital mortality for acutely ill medical admissions in two acute hospitals using their first electronically recorded blood test results and vital signs: a cross-sectional study

External validation of the computer aided risk scoring system in predicting in-hospital mortality following emergency medical admissions

BACKGROUND

Clinical prediction models have the potential to improve the quality of care and enhance patient safety outcomes. A Computer-aided Risk Scoring system (CARSS) was previously developed to predict in-hospital mortality following emergency admissions based on routinely collected blood tests and vitals. We aimed to externally validate the CARSS model.

METHODS

In this retrospective external validation study, we considered all adult (≥18 years) emergency medical admissions discharged between 11/11/2020 and 11/11/2022 from The Rotherham Foundation Trust (TRFT), UK. We assessed the predictive performance of the CARSS model based on its discriminative (c-statistic) and calibration characteristics (calibration slope and calibration plots).

RESULTS

Out of 32,774 admissions, 20,422 (62.3 %) admissions were included. The TRFT sample had similar demographic characteristics to the development sample but had higher mortality (6.1 % versus 5.7 %). The CARSS model demonstrated good discrimination (c-statistic 0.87 [95 % CI 0.86-0.88]) and good calibration to the TRFT dataset (slope = 1.03 [95 % CI 0.98-1.08] intercept = 0 [95 % CI -0.06-0.07]) after re-calibrating for differences in baseline mortality (intercept = 0.96 [95 % CI 0.90-1.03] before re-calibration).

CONCLUSION

In summary, the CARSS model is externally validated after correcting the baseline risk of death between development and validation datasets. External validation of the CARSS model showed that it under-predicted in-hospital mortality. Re-calibration of this model showed adequate performance in the TRFT dataset.

Early Warning Systems for Critical Illness Outside the Intensive Care Unit

Early warning systems (EWSs) are designed and deployed to create a rapid assessment and response for patients with clinical deterioration outside the intensive care unit (ICU). These models incorporate patient-level data such as vital signs and laboratory values to detect or prevent adverse clinical events, such as vital signs and laboratories to allow detection and prevention of adverse clinical events such as cardiac arrest, intensive care transfer, or sepsis. The applicability, development, clinical utility, and general perception of EWS in clinical practice vary widely. Here, we review the field as it has grown from early vital sign-based scoring systems to contemporary multidimensional algorithms and predictive technologies for clinical decompensation outside the ICU.

Accuracy of automated computer-aided risk scoring systems to estimate the risk of COVID-19: a retrospective cohort study

BACKGROUND

In the UK National Health Service (NHS), the patient's vital signs are monitored and summarised into a National Early Warning Score (NEWS) score. A set of computer-aided risk scoring systems (CARSS) was developed and validated for predicting in-hospital mortality and sepsis in unplanned admission to hospital using NEWS and routine blood tests results. We sought to assess the accuracy of these models to predict the risk of COVID-19 in unplanned admissions during the first phase of the pandemic.

METHODS

Adult ( > = 18 years) non-elective admissions discharged (alive/deceased) between 11-March-2020 to 13-June-2020 from two acute hospitals with an index NEWS electronically recorded within ± 24 h of admission. We identified COVID-19 admission based on ICD-10 code 'U071' which was determined by COVID-19 swab test results (hospital or community). We assessed the performance of CARSS (CARS_N, CARS_NB, CARM_N, CARM_NB) for predicting the risk of COVID-19 in terms of discrimination (c-statistic) and calibration (graphically).

RESULTS

The risk of in-hospital mortality following emergency medical admission was 8.4% (500/6444) and 9.6% (620/6444) had a diagnosis of COVID-19. For predicting COVID-19 admissions, the CARS_N model had the highest discrimination 0.73 (0.71 to 0.75) and calibration slope 0.81 (0.72 to 0.89) compared to other CARSS models: CARM_N (discrimination:0.68 (0.66 to 0.70) and calibration slope 0.47 (0.41 to 0.54)), CARM_NB (discrimination:0.68 (0.65 to 0.70) and calibration slope 0.37 (0.31 to 0.43)), and CARS_NB (discrimination:0.68 (0.66 to 0.70) and calibration slope 0.56 (0.47 to 0.64)).

CONCLUSIONS

The CARS_N model is reasonably accurate for predicting the risk of COVID-19. It may be clinically useful as an early warning system at the time of admission especially to triage large numbers of unplanned admissions because it requires no additional data collection and is readily automated.

The Norton scale is an important predictor of in-hospital mortality in internal medicine patients

BACKGROUND

The Norton scale, a marker of patient frailty used to predict the risk of pressure ulcers, but the predictive value of the Norton scale for in-hospital mortality after adjustment for a wide range of demographic, and abnormal admission laboratory test results shown in themselves to have a high predictive value for in-hospital mortality is unclear.

AIM

The study aims to determine the value of the Norton scale and the presence of a urinary catheter in predicting in hospital mortality.

METHODS

The study population included all acutely admitted adult patients in 2020 through October 2021 to one of three internal medicine departments at the Laniado Hospital, a regional hospital with 400 beds in Israel. The main objective was to (a) identify the variables associated with the Norton Scale and (b) determine whether it predicts in-hospital mortality after adjustment for these variables.

RESULTS

The Norton scale was associated with an older age, female gender, presence of a urinary catheter, and abnormal laboratory tests. The odds of in-hospital mortality in those with intermediate, high, and very high Norton scale risk groups were 3.10 (2.23-3.56), 6.48 (4.02-10.46), and 12.27 (7.37-20.44), respectively, after adjustment for the remaining predictors. Adding the Norton scale and the presence of a urinary catheter to the prediction logistic regression model that included age, gender, and abnormal laboratory test results increased the c-statistic from 0.870 (0.864-0.876) to 0.908 (0.902-0.913).

CONCLUSIONS

The Norton scale and presence of a urinary catheter are important predictors of in-hospital mortality in acutely hospitalized adults in internal medicine departments.

Implementation and clinical utility of a Computer-Aided Risk Score for Mortality (CARM): a qualitative study

OBJECTIVES

The Computer-Aided Risk Score for Mortality (CARM) estimates the risk of in-hospital mortality following acute admission to the hospital by automatically amalgamating physiological measures, blood tests, gender, age and COVID-19 status. Our aims were to implement the score with a small group of practitioners and understand their first-hand experience of interacting with the score in situ.

DESIGN

Pilot implementation evaluation study involving qualitative interviews.

SETTING

This study was conducted in one of the two National Health Service hospital trusts in the North of England in which the score was developed.

PARTICIPANTS

Medical, older person and ICU/anaesthetic consultants and specialist grade registrars (n=116) and critical outreach nurses (n=7) were given access to CARM. Nine interviews were conducted in total, with eight doctors and one critical care outreach nurse.

INTERVENTIONS

Participants were given access to the CARM score, visible after login to the patients' electronic record, along with information about the development and intended use of the score.

RESULTS

Four themes and 14 subthemes emerged from reflexive thematic analysis: (1) current use (including support or challenge clinical judgement and decision making, communicating risk of mortality and professional curiosity); (2) barriers and facilitators to use (including litigation, resource needs, perception of the evidence base, strengths and limitations), (3) implementation support needs (including roll-out and integration, access, training and education); and (4) recommendations for development (including presentation and functionality and potential additional data). Barriers and facilitators to use, and recommendations for development featured highly across most interviews.

CONCLUSION

Our in situ evaluation of the pilot implementation of CARM demonstrated its scope in supporting clinical decision making and communicating risk of mortality between clinical colleagues and with service users. It suggested to us barriers to implementation of the score. Our findings may support those seeking to develop, implement or improve the adoption of risk scores.

The presence of a urinary catheter is an important predictor of in-hospital mortality in internal medicine patients

RATIONALE AND OBJECTIVE

Mortality rates are used to assess the quality of hospital care after appropriate adjustment for case-mix. Urinary catheters are frequent in hospitalized adults and might be a marker of patient frailty and illness severity. However, we know of no attempts to estimate the predictive value of indwelling catheters for specific patient outcomes. The objective of the present study was to (a) identify the variables associated with the presence of a urinary catheter and (b) determine whether it predicts in-hospital mortality after adjustment for these variables.

METHODS

The study population included all acutely admitted adult patients in 2020 (exploratory cohort) and January-October 2021 (validation cohort) to internal medicine, cardiology and intensive care departments at the Laniado Hospital, a regional hospital with 400 beds in Israel. There were no exclusion criteria. The predictor variables were the presence of a urinary catheter on admission, age, gender, comorbidities and admission laboratory test results. We used bivariate and multivariate logistic regression to test the associations between the presence of a urinary catheter and mortality after adjustment for the remaining independent variables on admission.

RESULTS

The presence of a urinary catheter was associated with other independent variables. In 2020, the odds of in-hospital mortality in patients with a urinary catheter before and after adjustment for the remaining predictors were 14.3 (11.6-17.7) and 6.05 (4.78-7.65), respectively. Adding the presence of a urinary catheter to the prediction logistic regression model increased its c-statistic from 0.887 (0.880-0.894) to 0.907 (0.901-0.913). The results of the validation cohort reduplicated those of the exploratory cohort.

CONCLUSIONS

The presence of a urinary catheter on admission is an important and independent predictor of in-hospital mortality in acutely hospitalized adults in internal medicine departments.

Identification of Prognostic Fatty Acid Metabolism lncRNAs and Potential Molecular Targeting Drugs in Uveal Melanoma

Background

The aim of this study was to identify prognostic fatty acid metabolism lncRNAs and potential molecular targeting drugs in uveal melanoma through integrated bioinformatics analysis.

Methods

In the present study, we obtained the expression matrix of 309 FAM-mRNAs and identified 225 FAM-lncRNAs by coexpression network analysis. We then performed univariate Cox analysis, LASSO regression analysis, and cross-validation and finally obtained an optimized UVM prognosis prediction model composed of four PFAM-lncRNAs (AC104129.1, SOS1-IT1, IDI2-AS1, and DLGAP1-AS2).

Results

The survival curves showed that the survival time of UVM patients in the high-risk group was significantly lower than that in the low-risk group in the train cohort, test cohort, and all patients in the prognostic prediction model (P < 0.05). We further performed risk prognostic assessment, and the results showed that the risk scores of the high-risk group in the train cohort, test cohort, and all patients were significantly higher than those of the low-risk group (P < 0.05), patient survival decreased and the number of deaths increased with increasing risk scores, and AC104129.1, SOS1-IT1, and DLGAP1-AS2 were high-risk PFAM-lncRNAs, while IDI2-AS1 were low-risk PFAM-lncRNAs. Afterwards, we further verified the accuracy and the prognostic value of our model in predicting prognosis by PCA analysis and ROC curves.

Conclusion

We identified 24 potential molecularly targeted drugs with significant sensitivity differences between high- and low-risk UVM patients, of which 13 may be potential targeted drugs for high-risk patients. Our findings have important implications for early prediction and early clinical intervention in high-risk UVM patients.

Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study

OBJECTIVES

There are no established mortality risk equations specifically for unplanned emergency medical admissions which include patients with SARS-19 (COVID-19). We aim to develop and validate a computer-aided risk score (CARMc19) for predicting mortality risk by combining COVID-19 status, the first electronically recorded blood test results and the National Early Warning Score (NEWS2).

DESIGN

Logistic regression model development and validation study.

SETTING

Two acute hospitals (York Hospital-model development data; Scarborough Hospital-external validation data).

PARTICIPANTS

Adult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic NEWS and blood test results recorded on admission. We used logistic regression modelling to predict the risk of in-hospital mortality using two models: (1) CARMc19_N: age+sex+NEWS2 including subcomponents+COVID19; (2) CARMc19_NB: CARMc19_N in conjunction with seven blood test results and acute kidney injury score. Model performance was evaluated according to discrimination (c-statistic), calibration (graphically) and clinical usefulness at NEWS2 thresholds of 4+, 5+, 6+.

RESULTS

The risk of in-hospital mortality following emergency medical admission was similar in development and validation datasets (8.4% vs 8.2%). The c-statistics for predicting mortality for CARMc19_NB is better than CARMc19_N in the validation dataset (CARMc19_NB=0.88 (95% CI 0.86 to 0.90) vs CARMc19_N=0.86 (95% CI 0.83 to 0.88)). Both models had good calibration (CARMc19_NB=1.01 (95% CI 0.88 to 1.14) and CARMc19_N:0.95 (95% CI 0.83 to 1.06)). At all NEWS2 thresholds (4+, 5+, 6+) model, CARMc19_NB had better sensitivity and similar specificity.

CONCLUSIONS

We have developed a validated CARMc19 scores with good performance characteristics for predicting the risk of in-hospital mortality. Since the CARMc19 scores place no additional data collection burden on clinicians, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

Parameters of the complete blood count predict in hospital mortality

INTRODUCTION

Mortality rates are used to evaluate the quality of hospital care after adjusting for disease severity and, commonly also, for age, comorbidity, and laboratory data with only few parameters of the complete blood count (CBC).

OBJECTIVE

To identify the parameters of the CBC that predict independently in-hospital mortality of acutely admitted patients.

POPULATION

All patients were admitted to internal medicine, cardiology, and intensive care departments at the Laniado Hospital in Israel in 2018 and 2019.

VARIABLES

Independent variables were patients' age, sex, and parameters of the CBC. The outcome variable was in-hospital mortality.

ANALYSIS

Logistic regression. In 2018, we identified the variables that were associated with in-hospital mortality and validated this association in the 2019 cohort.

RESULTS

In the validation cohort, a model consisting of nine parameters that are commonly available in modern analyzers had a c-statistics (area under the receiver operator curve) of 0.86 and a 10%-90% risk gradient of 0%-21.4%. After including the proportions of large unstained cells, hypochromic, and macrocytic red cells, the c-statistic increased to 0.89, and the risk gradient to 0.1%-29.5%.

CONCLUSION

The commonly available parameters of the CBC predict in-hospital mortality. Addition of the proportions of hypochromic red cells, macrocytic red cells, and large unstained cells may improve the predictive value of the CBC.

Association of the All-Patient Refined Diagnosis-Related Groups Severity of Illness and Risk of Mortality Classification with Outcomes

OBJECTIVES

Diagnosis-related groups (DRGs) is a patient classification system used to characterize the types of patients that the hospital manages and to compare the resources needed during hospitalization. The DRG classification is based on International Classification of Diseases diagnoses, procedures, demographics, discharge status, and complications or comorbidities and compares hospital resources and outcomes used to determine how much Medicare pays the hospital for each "product/medical condition." The All-Patient Refined DRG (APR-DRG) incorporated severity of illness (SOI) and risk of mortality (ROM) into the DRG system to adjust for patient complexity to compare resource utilization, complication rates, and lengths of stay.

METHODS

This study included 18,478 adult patients admitted to a tertiary care center in Lubbock, Texas during a 1-year period. We recorded the APR-DRG SOI and ROM and some clinical information on these patients, including age, sex, admission shock index, admission glucose and lactate levels, diagnoses based on International Classification of Diseases, Tenth Revision discharge coding, length of stay, and mortality. We compared the levels of SOI and ROM across this clinical information.

RESULTS

As the levels of SOI and ROM increase (which indicates increased disease severity and risk of mortality), age, glucose levels, lactate levels, shock index, length of stay, and mortality increased significantly (P < 0.001). Multiple logistic regression analysis demonstrated that each unit increase in ROM and SOI level was significantly associated with an 11.45 and a 10.37 times increase in the odds of in-hospital mortality, respectively. The C-statistics for the corresponding models are 0.947 and 0.929, respectively. When both ROM and SOI were included in the model, the magnitudes of increase in odds of in-hospital mortality were 5.61 and 1.17 times for ROM and SOI, respectively. The C-statistic is 0.949.

CONCLUSIONS

This study indicates that the APR-DRG SOI and ROM scores provide a classification system that is associated with mortality and correlates with other clinical variables, such as the shock index and lactate levels, which are available on admission.

A simple index predicting mortality in acutely hospitalized patients

BACKGROUND

Mortality rates used to evaluate and improve the quality of hospital care are adjusted for comorbidity and disease severity. Comorbidity, measured by International Classification of Diseases codes, do not reflect the severity of the medical condition, that requires clinical assessments not available in electronic databases, and/or laboratory data with clinically relevant ranges to permit extrapolation from one setting to the next.

AIM

To propose a simple index predicting mortality in acutely hospitalized patients.

DESIGN

Retrospective cohort study with internal and external validation.

METHODS

The study populations were all acutely admitted patients in 2015-16, and in January 2019-November 2019 to internal medicine, cardiology and intensive care departments at the Laniado Hospital in Israel, and in 2002-19, at St. James Hospital, Ireland. Predictor variables were age and admission laboratory tests. The outcome variable was in-hospital mortality. Using logistic regression of the data in the 2015-16 Israeli cohort, we derived an index that included age groups and significant laboratory data.

RESULTS

In the Israeli 2015-16 cohort, the index predicted mortality rates from 0.2% to 32.0% with a c-statistic (area under the receiver operator characteristic curve) of 0.86. In the Israeli 2019 validation cohort, the index predicted mortality rates from 0.3% to 38.9% with a c-statistic of 0.87. An abbreviated index performed similarly in the Irish 2002-19 cohort.

CONCLUSIONS

Hospital mortality can be predicted by age and selected admission laboratory data without acquiring information from the patient's medical records. This permits an inexpensive comparison of performance of hospital departments.

Early warning score validation methodologies and performance metrics: a systematic review

Background

Early warning scores (EWS) have been developed as clinical prognostication tools to identify acutely deteriorating patients. In the past few years, there has been a proliferation of studies that describe the development and validation of novel machine learning-based EWS. Systematic reviews of published studies which focus on evaluating performance of both well-established and novel EWS have shown conflicting conclusions. A possible reason is the heterogeneity in validation methods applied. In this review, we aim to examine the methodologies and metrics used in studies which perform EWS validation.

Methods

A systematic review of all eligible studies from the MEDLINE database and other sources, was performed. Studies were eligible if they performed validation on at least one EWS and reported associations between EWS scores and inpatient mortality, intensive care unit (ICU) transfers, or cardiac arrest (CA) of adults. Two reviewers independently did a full-text review and performed data abstraction by using standardized data-worksheet based on the TRIPOD (Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) checklist. Meta-analysis was not performed due to heterogeneity.

Results

The key differences in validation methodologies identified were (1) validation dataset used, (2) outcomes of interest, (3) case definition, time of EWS use and aggregation methods, and (4) handling of missing values. In terms of case definition, among the 48 eligible studies, 34 used the patient episode case definition while 12 used the observation set case definition, and 2 did the validation using both case definitions. Of those that used the patient episode case definition, 18 studies validated the EWS at a single point of time, mostly using the first recorded observation. The review also found more than 10 different performance metrics reported among the studies.

Conclusions

Methodologies and performance metrics used in studies performing validation on EWS were heterogeneous hence making it difficult to interpret and compare EWS performance. Standardizing EWS validation methodology and reporting can potentially address this issue.

Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology

OBJECTIVE

To provide an overview and critical appraisal of early warning scores for adult hospital patients.

DESIGN

Systematic review.

DATA SOURCES

Medline, CINAHL, PsycInfo, and Embase until June 2019.

ELIGIBILITY CRITERIA FOR STUDY SELECTION

Studies describing the development or external validation of an early warning score for adult hospital inpatients.

RESULTS

13 171 references were screened and 95 articles were included in the review. 11 studies were development only, 23 were development and external validation, and 61 were external validation only. Most early warning scores were developed for use in the United States (n=13/34, 38%) and the United Kingdom (n=10/34, 29%). Death was the most frequent prediction outcome for development studies (n=10/23, 44%) and validation studies (n=66/84, 79%), with different time horizons (the most frequent was 24 hours). The most common predictors were respiratory rate (n=30/34, 88%), heart rate (n=28/34, 83%), oxygen saturation, temperature, and systolic blood pressure (all n=24/34, 71%). Age (n=13/34, 38%) and sex (n=3/34, 9%) were less frequently included. Key details of the analysis populations were often not reported in development studies (n=12/29, 41%) or validation studies (n=33/84, 39%). Small sample sizes and insufficient numbers of event patients were common in model development and external validation studies. Missing data were often discarded, with just one study using multiple imputation. Only nine of the early warning scores that were developed were presented in sufficient detail to allow individualised risk prediction. Internal validation was carried out in 19 studies, but recommended approaches such as bootstrapping or cross validation were rarely used (n=4/19, 22%). Model performance was frequently assessed using discrimination (development n=18/22, 82%; validation n=69/84, 82%), while calibration was seldom assessed (validation n=13/84, 15%). All included studies were rated at high risk of bias.

CONCLUSIONS

Early warning scores are widely used prediction models that are often mandated in daily clinical practice to identify early clinical deterioration in hospital patients. However, many early warning scores in clinical use were found to have methodological weaknesses. Early warning scores might not perform as well as expected and therefore they could have a detrimental effect on patient care. Future work should focus on following recommended approaches for developing and evaluating early warning scores, and investigating the impact and safety of using these scores in clinical practice.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42017053324.

Development and validation of early warning score system: A systematic literature review

OBJECTIVES

This review aims to: 1) evaluate the quality of model reporting, 2) provide an overview of methodology for developing and validating Early Warning Score Systems (EWSs) for adult patients in acute care settings, and 3) highlight the strengths and limitations of the methodologies, as well as identify future directions for EWS derivation and validation studies.

METHODOLOGY

A systematic search was conducted in PubMed, Cochrane Library, and CINAHL. Only peer reviewed articles and clinical guidelines regarding developing and validating EWSs for adult patients in acute care settings were included. 615 articles were extracted and reviewed by five of the authors. Selected studies were evaluated based on the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist. The studies were analyzed according to their study design, predictor selection, outcome measurement, methodology of modeling, and validation strategy.

RESULTS

A total of 29 articles were included in the final analysis. Twenty-six articles reported on the development and validation of a new EWS, while three reported on validation and model modification. Only eight studies met more than 75% of the items in the TRIPOD checklist. Three major techniques were utilized among the studies to inform their predictive algorithms: 1) clinical-consensus models (n = 6), 2) regression models (n = 15), and 3) tree models (n = 5). The number of predictors included in the EWSs varied from 3 to 72 with a median of seven. Twenty-eight models included vital signs, while 11 included lab data. Pulse oximetry, mental status, and other variables extracted from electronic health records (EHRs) were among other frequently used predictors. In-hospital mortality, unplanned transfer to the intensive care unit (ICU), and cardiac arrest were commonly used clinical outcomes. Twenty-eight studies conducted a form of model validation either within the study or against other widely-used EWSs. Only three studies validated their model using an external database separate from the derived database.

CONCLUSION

This literature review demonstrates that the characteristics of the cohort, predictors, and outcome selection, as well as the metrics for model validation, vary greatly across EWS studies. There is no consensus on the optimal strategy for developing such algorithms since data-driven models with acceptable predictive accuracy are often site-specific. A standardized checklist for clinical prediction model reporting exists, but few studies have included reporting aligned with it in their publications. Data-driven models are subjected to biases in the use of EHR data, thus it is particularly important to provide detailed study protocols and acknowledge, leverage, or reduce potential biases of the data used for EWS development to improve transparency and generalizability.

A pragmatic method to compare hospital bed provision between countries and regions: Beds in the States of Australia

A simple method is presented to evaluate bed numbers between countries using a logarithmic relationship between beds per 1000 deaths and deaths per 1000 population, both of which are readily available. The method relies on the importance of the nearness to death effect. This method was tested using data from Australian States. Beds per 1000 deaths varied considerably between States. This variation reduced after adjusting for the ratio of deaths per 1000 population which is a measure of population age structure. After this adjustment, most Australian States roughly approximate to the international average for developed countries while Tasmania was shown to have a chronic bed shortage, as has been recognized for many years. The Northern Territory and the Australian Capital Territory, both of which have the youngest populations, have more beds relative to the other States. The nearness to death effect must be incorporated into capacity planning models in order to give robust estimates of future bed demand and to evaluate differences between countries and health care systems.

Performance of externally validated enhanced computer-aided versions of the National Early Warning Score in predicting mortality following an emergency admission to hospital in England: a cross-sectional study

OBJECTIVES

In the English National Health Service, the patient's vital signs are monitored and summarised into a National Early Warning Score (NEWS) to support clinical decision making, but it does not provide an estimate of the patient's risk of death. We examine the extent to which the accuracy of NEWS for predicting mortality could be improved by enhanced computer versions of NEWS (cNEWS).

DESIGN

Logistic regression model development and external validation study.

SETTING

Two acute hospitals (YH-York Hospital for model development; NH-Northern Lincolnshire and Goole Hospital for external model validation).

PARTICIPANTS

Adult (≥16 years) medical admissions discharged over a 24-month period with electronic NEWS (eNEWS) recorded on admission are used to predict mortality at four time points (in-hospital, 24 hours, 48 hours and 72 hours) using the first electronically recorded NEWS (model M0) versus a cNEWS model which included age+sex (model M1) +subcomponents of NEWS (including diastolic blood pressure) (model M2).

RESULTS

The risk of dying in-hospital following emergency medical admission was 5.8% (YH: 2080/35 807) and 5.4% (NH: 1900/35 161). The c-statistics for model M2 in YH for predicting mortality (in-hospital=0.82, 24 hours=0.91, 48 hours=0.88 and 72 hours=0.88) was higher than model M0 (in-hospital=0.74, 24 hours=0.89, 48 hours=0.86 and 72 hours=0.85) with higher Positive Predictive Value (PPVs) for in-hospital mortality (M2 19.3% and M0 16.6%). Similar findings were seen in NH. Model M2 performed better than M0 in almost all major disease subgroups.

CONCLUSIONS

An externally validated enhanced computer-aided NEWS model (cNEWS) incrementally improves on the performance of a NEWS only model. Since cNEWS places no additional data collection burden on clinicians and is readily automated, it may now be carefully introduced and evaluated to determine if it can improve care in hospitals that have eNEWS systems.

A prospective study of consecutive emergency medical admissions to compare a novel automated computer-aided mortality risk score and clinical judgement of patient mortality risk

OBJECTIVES

To compare the performance of a validated automatic computer-aided risk of mortality (CARM) score versus medical judgement in predicting the risk of in-hospital mortality for patients following emergency medical admission.

DESIGN

A prospective study.

SETTING

Consecutive emergency medical admissions in York hospital.

PARTICIPANTS

Elderly medical admissions in one ward were assigned a risk of death at the first post-take ward round by consultant staff over a 2-week period. The consultant medical staff used the same variables to assign a risk of death to the patient as the CARM (age, sex, National Early Warning Score and blood test results) but also had access to the clinical history, examination findings and any immediately available investigations such as ECGs. The performance of the CARM versus consultant medical judgement was compared using the c-statistic and the positive predictive value (PPV).

RESULTS

The in-hospital mortality was 31.8% (130/409). For patients with complete blood test results, the c-statistic for CARM was 0.75 (95% CI: 0.69 to 0.81) versus 0.72 (95% CI: 0.66 to 0.78) for medical judgements (p=0.28). For patients with at least one missing blood test result, the c-statistics were similar (medical judgements 0.70 (95% CI: 0.60 to 0.81) vs CARM 0.70 (95% CI: 0.59 to 0.80)). At a 10% mortality risk, the PPV for CARM was higher than medical judgements in patients with complete blood test results, 62.0% (95% CI: 51.2 to 71.9) versus 49.2% (95% CI: 39.8 to 58.5) but not when blood test results were missing, 50.0% (95% CI: 24.7 to 75.3) versus 53.3% (95% CI: 34.3 to 71.7).

CONCLUSIONS

CARM is comparable with medical judgements in discriminating in-hospital mortality following emergency admission to an elderly care ward. CARM may have a promising role in supporting medical judgements in determining the patient's risk of death in hospital. Further evaluation of CARM in routine practice is required.

Understanding and applying practitioner and patient views on the implementation of a novel automated Computer-Aided Risk Score (CARS) predicting the risk of death following emergency medical admission to hospital: qualitative study

OBJECTIVES

The Computer-Aided Risk Score (CARS) estimates the risk of death following emergency admission to medical wards using routinely collected vital signs and blood test data. Our aim was to elicit the views of healthcare practitioners (staff) and service users and carers (SU/C) on (1) the potential value, unintended consequences and concerns associated with CARS and practitioner views on (2) the issues to consider before embedding CARS into routine practice.

SETTING

This study was conducted in two National Health Service (NHS) hospital trusts in the North of England. Both had in-house information technology (IT) development teams, mature IT infrastructure with electronic National Early Warning Score (NEWS) and were capable of integrating NEWS with blood test results. The study focused on emergency medical and elderly admissions units. There were 60 and 39 acute medical/elderly admissions beds at the two NHS hospital trusts.

PARTICIPANTS

We conducted eight focus groups with 45 healthcare practitioners and two with 11 SU/Cs in two NHS acute hospitals.

RESULTS

Staff and SU/Cs recognised the potential of CARS but were clear that the score should not replace or undermine clinical judgments. Staff recognised that CARS could enhance clinical decision-making/judgments and aid communication with patients. They wanted to understand the components of CARS and be reassured about its accuracy but were concerned about the impact on intensive care and blood tests.

CONCLUSION

Risk scores are widely used in healthcare, but their development and implementation do not usually involve input from practitioners and SU/Cs. We contributed to the development of CARS by eliciting views of staff and SU/Cs who provided important, often complex, insights to support the development and implementation of CARS to ensure successful implementation in routine clinical practice.