A prospective study of consecutive emergency medical admissions to compare a novel automated computer-aided mortality risk score and clinical judgement of patient mortality risk

Implementation and clinical utility of a Computer-Aided Risk Score for Mortality (CARM): a qualitative study

OBJECTIVES

The Computer-Aided Risk Score for Mortality (CARM) estimates the risk of in-hospital mortality following acute admission to the hospital by automatically amalgamating physiological measures, blood tests, gender, age and COVID-19 status. Our aims were to implement the score with a small group of practitioners and understand their first-hand experience of interacting with the score in situ.

DESIGN

Pilot implementation evaluation study involving qualitative interviews.

SETTING

This study was conducted in one of the two National Health Service hospital trusts in the North of England in which the score was developed.

PARTICIPANTS

Medical, older person and ICU/anaesthetic consultants and specialist grade registrars (n=116) and critical outreach nurses (n=7) were given access to CARM. Nine interviews were conducted in total, with eight doctors and one critical care outreach nurse.

INTERVENTIONS

Participants were given access to the CARM score, visible after login to the patients' electronic record, along with information about the development and intended use of the score.

RESULTS

Four themes and 14 subthemes emerged from reflexive thematic analysis: (1) current use (including support or challenge clinical judgement and decision making, communicating risk of mortality and professional curiosity); (2) barriers and facilitators to use (including litigation, resource needs, perception of the evidence base, strengths and limitations), (3) implementation support needs (including roll-out and integration, access, training and education); and (4) recommendations for development (including presentation and functionality and potential additional data). Barriers and facilitators to use, and recommendations for development featured highly across most interviews.

CONCLUSION

Our in situ evaluation of the pilot implementation of CARM demonstrated its scope in supporting clinical decision making and communicating risk of mortality between clinical colleagues and with service users. It suggested to us barriers to implementation of the score. Our findings may support those seeking to develop, implement or improve the adoption of risk scores.

Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study

OBJECTIVES

There are no established mortality risk equations specifically for unplanned emergency medical admissions which include patients with SARS-19 (COVID-19). We aim to develop and validate a computer-aided risk score (CARMc19) for predicting mortality risk by combining COVID-19 status, the first electronically recorded blood test results and the National Early Warning Score (NEWS2).

DESIGN

Logistic regression model development and validation study.

SETTING

Two acute hospitals (York Hospital-model development data; Scarborough Hospital-external validation data).

PARTICIPANTS

Adult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic NEWS and blood test results recorded on admission. We used logistic regression modelling to predict the risk of in-hospital mortality using two models: (1) CARMc19_N: age+sex+NEWS2 including subcomponents+COVID19; (2) CARMc19_NB: CARMc19_N in conjunction with seven blood test results and acute kidney injury score. Model performance was evaluated according to discrimination (c-statistic), calibration (graphically) and clinical usefulness at NEWS2 thresholds of 4+, 5+, 6+.

RESULTS

The risk of in-hospital mortality following emergency medical admission was similar in development and validation datasets (8.4% vs 8.2%). The c-statistics for predicting mortality for CARMc19_NB is better than CARMc19_N in the validation dataset (CARMc19_NB=0.88 (95% CI 0.86 to 0.90) vs CARMc19_N=0.86 (95% CI 0.83 to 0.88)). Both models had good calibration (CARMc19_NB=1.01 (95% CI 0.88 to 1.14) and CARMc19_N:0.95 (95% CI 0.83 to 1.06)). At all NEWS2 thresholds (4+, 5+, 6+) model, CARMc19_NB had better sensitivity and similar specificity.

CONCLUSIONS

We have developed a validated CARMc19 scores with good performance characteristics for predicting the risk of in-hospital mortality. Since the CARMc19 scores place no additional data collection burden on clinicians, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.