The predictive value of variables measurable in the ambulance and the development of the Predict Sepsis screening tools: a prospective cohort study

Validation and comparison of triage-based screening strategies for sepsis

OBJECTIVE

This study sought to externally validate and compare proposed methods for stratifying sepsis risk at emergency department (ED) triage.

METHODS

This nested case/control study enrolled ED patients from four hospitals in Utah and evaluated the performance of previously-published sepsis risk scores amenable to use at ED triage based on their area under the precision-recall curve (AUPRC, which balances positive predictive value and sensitivity) and area under the receiver operator characteristic curve (AUROC, which balances sensitivity and specificity). Score performance for predicting whether patients met Sepsis-3 criteria in the ED was compared to patients' assigned ED triage score (Canadian Triage Acuity Score [CTAS]) with adjustment for multiple comparisons.

RESULTS

Among 2000 case/control patients, 981 met Sepsis-3 criteria on final adjudication. The best performing sepsis risk scores were the Predict Sepsis version #3 (AUPRC 0.183, 95 % CI 0.148-0.256; AUROC 0.859, 95 % CI 0.843-0.875) and Borelli scores (AUPRC 0.127, 95 % CI 0.107-0.160, AUROC 0.845, 95 % CI 0.829-0.862), which significantly outperformed CTAS (AUPRC 0.038, 95 % CI 0.035-0.042, AUROC 0.650, 95 % CI 0.628-0.671, p < 0.001 for all AUPRC and AUROC comparisons). The Predict Sepsis and Borelli scores exhibited sensitivity of 0.670 and 0.678 and specificity of 0.902 and 0.834, respectively, at their recommended cutoff values and outperformed Systemic Inflammatory Response Syndrome (SIRS) criteria (AUPRC 0.083, 95 % CI 0.070-0.102, p = 0.052 and p = 0.078, respectively; AUROC 0.775, 95 % CI 0.756-0.795, p < 0.001 for both scores).

CONCLUSIONS

The Predict Sepsis and Borelli scores exhibited improved performance including increased specificity and positive predictive values for sepsis identification at ED triage compared to CTAS and SIRS criteria.

Prehospital Lactate Levels Obtained in the Ambulance and Prediction of 2-Day In-Hospital Mortality in Patients With Traumatic Brain Injury

BACKGROUND AND OBJECTIVES

To analyze the ability of prehospital lactate levels to predict 2-day in-hospital mortality in patients with traumatic brain injury (TBI), severe TBI (Glasgow Coma Scale (GCS) ≤ 8 points), and mild or moderate TBI (GCS ≥ 9 points). Second, 90-day mortality was also explored.

METHODS

This was a prospective, multicenter, emergency medical services (EMSs) delivery, ambulance-based, derivation-validation cohort study developed in 5 tertiary hospitals (Spain), from November 1, 2019, to July 31, 2022. Patients were recruited from among all phone requests for emergency assistance among adults who were later evacuated to referral hospitals with acute TBI. The exclusion criteria were minors, pregnancy, trauma patients without TBI, delayed presentations, patients were discharged in situ, participants with cardiac arrest, and unavailability to obtain a blood sample. The primary outcome was all-cause 2-day in-hospital mortality and 90-day mortality in patients with moderate or mild TBI compared with patients with severe TBI. Clinical and analytical parameters (lactate and glucose) were collected. The discriminative power (area under the receiver operating characteristic curve [AUC]) and calibration curve were calculated for 2 geographically separated cohorts.

RESULTS

A total of 509 patients were ultimately included. The median age was 58 years (interquartile range: 43-75), and 167 patients were female (32.8%). The primary outcome occurred in 9 (2.2%) of 415 patients with moderate or mild TBI and in 42 (44.7%) of 94 patients with severe TBI. The predictive capacity of the lactate concentration was globally validated in our cohort, for which the AUC was 0.874 (95% CI 0.805-0.942) for the validation cohort. The ability of the GCS score to predict lactate concentration was greater in patients with a GCS score ≥9 points, with an AUC of 0.925 (95% CI 0.808-1.000) and a negative predictive value of 99.09 (95% CI 98.55-99.64) in the validation cohort.

CONCLUSION

Our results show the benefit of using lactate in all patients with TBI, particularly in those with a GCS ≥9 points. Routine incorporation of lactate in the screening of patients with TBI could presumably reduce mortality and deterioration rates because of quicker and better identification of patients at risk.

Added predictive value of prehospital measurement of point-of-care lactate in an adult general EMS population in Sweden: a multi-centre observational study

BACKGROUND

Emergency medical services (EMS) personnel must rapidly assess and transport patients with time-sensitive conditions to optimise patient outcomes. Serum lactate, a valuable in-hospital biomarker, has become more accessible in EMS settings through point-of-care (POC) testing. Although POC lactate levels are valuable in specific patient groups, its broader application in EMS remains unclear. This study assessed the additional predictive value of POC lactate levels in a general adult EMS population.

METHODS

This prospective observational study (March 2018 to September 2019) involved two EMS organisations in Västra Götaland, Sweden. Patients were triaged using the Rapid Triage and Treatment System (RETTS). POC lactate levels were measured using StatStrip Xpress devices. Non-consecutive patients who received EMS and were aged 18 years and above were available for inclusion if triaged into RETTS levels: red, orange, yellow, or green if respiratory rate of ≥ 22 breaths/min. Outcomes were adverse outcomes, including a time-sensitive diagnosis, sequential organ failure assessment (SOFA) score ≥ 2, and 30-day mortality. Statistical analyses included descriptive statistics, imputation, and regression models to assess the impact of the addition of POC lactate levels to a base model (comprising patient age, sex, presence of past medical conditions, vital signs, pain, EMS response time, assessed triage condition, and triage level) and a RETTS triage model.

RESULTS

Of 4,546 patients (median age 75 [57, 84] years; 49% male), 32.4% had time-sensitive conditions, 12.5% met the SOFA criteria, and 7.4% experienced 30-day mortality. The median POC lactate level was 1.7 (1.2, 2.5) mmol/L. Patients with time-sensitive conditions had higher lactate levels (1.9 mmol/L) than those with non-time-sensitive conditions (1.6 mmol/L). The probability of a time-sensitive condition increased with increasing lactate level. The addition of POC lactate marginally enhanced the predictive models, with a 1.5% and 4% increase for the base and RETTS triage models, respectively. POC lactate level as a sole predictor showed chance-only level predictive performance.

CONCLUSIONS

Prehospital POC lactate assessment provided limited additional predictive value in a general adult EMS population. However, it may be beneficial in specific patient subgroups, emphasizing the need for its judicious use in prehospital settings.

Predicting sepsis using a combination of clinical information and molecular immune markers sampled in the ambulance

Sepsis is a time dependent condition. Screening tools based on clinical parameters have been shown to increase the identification of sepsis. The aim of current study was to evaluate the additional predictive value of immunological molecular markers to our previously developed prehospital screening tools. This is a prospective cohort study of 551 adult patients with suspected infection in the ambulance setting of Stockholm, Sweden between 2017 and 2018. Initially, 74 molecules and 15 genes related to inflammation were evaluated in a screening cohort of 46 patients with outcome sepsis and 50 patients with outcome infection no sepsis. Next, 12 selected molecules, as potentially synergistic predictors, were evaluated in combination with our previously developed screening tools based on clinical parameters in a prediction cohort (n = 455). Seven different algorithms with nested cross-validation were used in the machine learning of the prediction models. Model performances were compared using posterior distributions of average area under the receiver operating characteristic (ROC) curve (AUC) and difference in AUCs. Model variable importance was assessed by permutation of variable values, scoring loss of classification as metric and with model-specific weights when applicable. When comparing the screening tools with and without added molecular variables, and their interactions, the molecules per se did not increase the predictive values. Prediction models based on the molecular variables alone showed a performance in terms of AUCs between 0.65 and 0.70. Among the molecular variables, IL-1Ra, IL-17A, CCL19, CX3CL1 and TNF were significantly higher in septic patients compared to the infection non-sepsis group. Combing immunological molecular markers with clinical parameters did not increase the predictive values of the screening tools, most likely due to the high multicollinearity of temperature and some of the markers. A group of sepsis patients was consistently miss-classified in our prediction models, due to milder symptoms as well as lower expression levels of the investigated immune mediators. This indicates a need of stratifying septic patients with a priori knowledge of certain clinical and molecular parameters in order to improve prediction for early sepsis diagnosis.Trial registration: NCT03249597. Registered 15 August 2017.

Prediction of the Risk of Sepsis by Using Analysis of Plasma Glucose and Serum Lactate in Ambulance Services: A Prospective Study

INTRODUCTION

The early recognition of patients with sepsis is difficult and the initial assessment outside of hospitals is challenging for ambulance clinicians (ACs). Indicators that ACs can use to recognize sepsis early are beneficial for patient outcomes. Research suggests that elevated point-of-care (POC) plasma glucose and serum lactate levels may help to predict sepsis in the ambulance service (AS) setting.

STUDY OBJECTIVE

The aim of this study was to test the hypothesis that the elevation of POC plasma glucose and serum lactate levels may help to predict Sepsis-3 in the AS.

METHODS

A prospective observational study was performed in the AS setting of Gothenburg in Sweden from the beginning of March 2018 through the end of September 2019. The criteria for sampling POC plasma glucose and serum lactate levels in the AS setting were high or intermediate risk according to the Rapid Emergency Triage and Treatment System (RETTS), as red, orange, yellow, and green if the respiratory rate was >22 breaths/minutes. Sepsis-3 were identified retrospectively. A primary and secondary analyses were carried out. The primary analysis included patients cared for in the AS and emergency department (ED) and were hospitalized. In the secondary analysis, patients who were only cared for in the AS and ED without being hospitalized were also included. To evaluate the predictive ability of these biomarkers, the area under the curve (AUC), sensitivity, specificity, and predictive values were used.

RESULTS

A total of 1,057 patients were included in the primary analysis and 1,841 patients were included in the secondary analysis. In total, 253 patients met the Sepsis-3 criteria (in both analyses). The AUC for POC plasma glucose and serum lactate levels showed low accuracy in predicting Sepsis-3 in both the primary and secondary analyses. Among all hospitalized patients, regardless of Sepsis-3, more than two-thirds had elevated plasma glucose and nearly one-half had elevated serum lactate when measured in the AS.

CONCLUSIONS

As individual biomarkers, an elevated POC plasma glucose and serum lactate were not associated with an increased likelihood of Sepsis-3 when measured in the AS in this study. However, the high rate of elevation of these biomarkers before arrival in hospital highlights that their role in clinical decision making at this early stage needs further evaluation, including other endpoints than Sepsis-3.

Association between variables measured in the ambulance and in-hospital mortality among adult patients with and without infection: a prospective cohort study

BACKGROUND

Patients presenting with infection to the ambulance are common, but risk factors for poor outcome are not known. The primary aim of the current study was to study the association between variables measured in the ambulance and mortality among adult patients with and without infection. The secondary aim was to study the association between these variables and mortality in a subgroup of patients who developed sepsis within 36 h.

METHODS

Prospective cohort study of 553 ambulance patients with, and 318 patients without infection, performed in Stockholm during 2017-2018. The association between 21 variables (8 keywords related to medical history, 6 vital signs, 4 blood tests, and age, gender, comorbidity) and in-hospital mortality was analysed using logistic regression.

RESULTS

Among patients with infection, inability of the patient to answer questions relating to certain symptoms such as pain and gastrointestinal symptoms was significantly associated with mortality in univariable analysis, in addition to oxygen saturation < 94%, heart rate > 110 /min, Glasgow Coma Scale (GCS) < 15, soluble urokinase Plasminogen Activator Receptor (suPAR) 4.0-7.9 ng/mL, suPAR ≥ 8.0 ng/mL and a Charlson comorbidity score ≥ 5. suPAR ≥ 8.0 ng/mL remained significant in multivariable analysis (OR 25.4; 95% CI, 3.2-199.8). Among patients without infection, suPAR ≥ 8.0 ng/mL and a Charlson comorbidity score ≥ 5 were significantly associated with mortality in univariable analysis, while suPAR ≥ 8.0 ng/mL remained significant in multivariable analysis (OR 56.1; 95% CI, 4.5-700.0). Among patients who developed sepsis, inability to answer questions relating to pain remained significant in multivariable analysis (OR 13.2; 95% CI, 2.2-78.9), in addition to suPAR ≥ 8.0 ng/mL (OR 16.1; 95% CI, 2.0-128.6).

CONCLUSIONS

suPAR ≥ 8.0 ng/mL was associated with mortality in patients presenting to the ambulance both with and without infection and in those who developed sepsis. Furthermore, the inability of the ambulance patient with an infection to answer questions relating to specific symptoms was associated with a surprisingly high mortality. These results suggest that suPAR and medical history are valuable tools with which to identify patients at risk of poor outcome in the ambulance and could potentially signal the need of enhanced attention.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03249597. Registered 15 August 2017-Retrospectively registered, https://clinicaltrials.gov/ct2/show/NCT03249597 .

New clinical prediction model for early recognition of sepsis in adult primary care patients: a prospective diagnostic cohort study of development and external validation

BACKGROUND

Recognising patients who need immediate hospital treatment for sepsis while simultaneously limiting unnecessary referrals is challenging for GPs.

AIM

To develop and validate a sepsis prediction model for adult patients in primary care.

DESIGN AND SETTING

This was a prospective cohort study in four out-of-hours primary care services in the Netherlands, conducted between June 2018 and March 2020.

METHOD

Adult patients who were acutely ill and received home visits were included. A total of nine clinical variables were selected as candidate predictors, next to the biomarkers C-reactive protein, procalcitonin, and lactate. The primary endpoint was sepsis within 72 hours of inclusion, as established by an expert panel. Multivariable logistic regression with backwards selection was used to design an optimal model with continuous clinical variables. The added value of the biomarkers was evaluated. Subsequently, a simple model using single cut-off points of continuous variables was developed and externally validated in two emergency department populations.

RESULTS

A total of 357 patients were included with a median age of 80 years (interquartile range 71-86), of which 151 (42%) were diagnosed with sepsis. A model based on a simple count of one point for each of six variables (aged >65 years; temperature >38°C; systolic blood pressure ≤110 mmHg; heart rate >110/min; saturation ≤95%; and altered mental status) had good discrimination and calibration (C-statistic of 0.80 [95% confidence interval = 0.75 to 0.84]; Brier score 0.175). Biomarkers did not improve the performance of the model and were therefore not included. The model was robust during external validation.

CONCLUSION

Based on this study's GP out-of-hours population, a simple model can accurately predict sepsis in acutely ill adult patients using readily available clinical parameters.

Prehospital fluid therapy in patients with suspected infection: a survey of ambulance personnel's practice

Background

Fluid therapy in patients with suspected infection is controversial, and it is not known whether fluid treatment administered in the prehospital setting is beneficial. In the absence of evidence-based guidelines for prehospital fluid therapy for patients with suspected infection, Emergency Medical Services (EMS) personnel are challenged on when and how to initiate such therapy.

This study aimed to assess EMS personnel’s decision-making in prehospital fluid therapy, including triggers for initiating fluid and fluid volumes, as well as the need for education and evidence-based guidelines on prehospital fluid therapy in patients with suspected infection.

Methods

An online survey concerning fluid administration in prehospital patients with suspected infection was distributed to all EMS personnel in the Central Denmark Region, including ambulance clinicians and prehospital critical care anaesthesiologists (PCCA). The survey consisted of sections concerning academic knowledge, statements about fluid administration, triggers to evaluate patient needs for intravenous fluid, and clinical scenarios.

Results

In total, 468/807 (58%) ambulance clinicians and 106/151 (70%) PCCA responded to the survey. Of the respondents, 73% (n = 341) of the ambulance clinicians and 100% (n = 106) of the PCCA felt confident about administering fluids to prehospital patients with infections. However, both groups primarily based their fluid-related decisions on “clinical intuition”. Ambulance clinicians named the most frequently faced challenges in fluid therapy as “Unsure whether the patient needs fluid” and “Unsure about the volume of fluid the patient needs”. The five most frequently used triggers for evaluating fluid needs were blood pressure, history taking, skin turgor, capillary refill time, and shock index, the last of which only applied to ambulance clinicians. In the scenarios, the majority administered 500 ml to a normotensive woman with suspected sepsis and 1000 ml to a woman with suspected sepsis-related hypotension. Moreover, 97% (n = 250) of the ambulance clinicians strongly agreed or agreed that they were interested in more education about fluid therapy in patients with suspected infection.

Conclusion

The majority of ambulance clinicians and PCCA based their fluid administration on “clinical intuition”. They faced challenges deciding on fluid volumes and individual fluid needs. Thus, they were eager to learn more and requested research and evidence-based guidelines.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13049-022-01025-1.

Novel Prehospital Phenotypes and Outcomes in Adult-Patients with Acute Disease

An early identification of prehospital phenotypes may allow health care workers to speed up and improve patients’ treatment. To determine emergency phenotypes by exclusively using prehospital clinical data, a multicenter, prospective, and observational ambulance-based study was conducted with a cohort of 3,853 adult patients treated consecutively and transferred with high priority from the scene to the hospital emergency department. Cluster analysis determined three clusters with highly different outcome scores and pathological characteristics. The first cluster presented a 30-day mortality after the index event of 45.9%. The second cluster presented a mortality of 26.3%, while mortality of the third cluster was 5.1%. This study supports the detection of three phenotypes with different risk stages and with different clinical, therapeutic, and prognostic considerations. This evidence could allow adapting treatment to each phenotype thereby helping in the decision-making process.

Combination of Prehospital NT-proBNP with qSOFA and NEWS to Predict Sepsis and Sepsis-Related Mortality

Background. The aim of this study was to assess the role of prehospital point-of-care N-terminal probrain natriuretic peptide to predict sepsis, septic shock, or in-hospital sepsis-related mortality. Methods. A prospective, emergency medical service-delivered, prognostic, cohort study of adults evacuated by ambulance and admitted to emergency department between January 2020 and May 2021. The discriminative power of the predictive variable was assessed through a prediction model trained using the derivation cohort and evaluated by the area under the curve of the receiver operating characteristic on the validation cohort. Results. A total of 1,360 patients were enrolled with medical disease in the study. The occurrence of sepsis, septic shock, and in-hospital sepsis-related mortality was 6.4% (67 cases), 4.2% (44 cases), and 6.1% (64 cases). Prehospital National Early Warning Score 2 had superior predictive validity than quick Sequential Organ Failure Assessment and N-terminal probrain natriuretic peptide for detecting sepsis and septic shock, but N-terminal probrain natriuretic peptide outperformed both scores in in-hospital sepsis-related mortality estimation. Application of N-terminal probrain natriuretic peptide to subgroups of the other two scores improved the identification of sepsis, septic shock, and sepsis-related mortality in the group of patients with low-risk scoring. Conclusions. The incorporation of N-terminal probrain natriuretic peptide in prehospital care combined with already existing scores could improve the identification of sepsis, septic shock, and sepsis-related mortality.

Exploring Features Contributing to the Early Prediction of Sepsis Using Machine Learning

The increasing availability of electronic health records and administrative data and the adoption of computer-based technologies in healthcare have significantly focused on medical informatics. Sepsis is a time-critical condition with high mortality, yet it is often not identified in a timely fashion. The early detection and diagnosis of sepsis can increase the likelihood of survival and improve long-term outcomes for patients. In this paper, we use SHapley Additive exPlanations (SHAP) analysis to explore the variables most highly associated with developing sepsis in patients and evaluating different supervised learning models for classification. To develop our predictive models, we used the data collected after the first and the fifth hour of admission and evaluated the contribution of different features to the prediction results for both time intervals. The results of our study show that, while there is a high level of missing data during the early stages of admission, this data can be effectively utilized for the early prediction of sepsis. We also found a high level of inconsistency between the contributing features at different stages of admission, which should be considered when developing machine learning models.

Performance of NEWS2, RETTS, clinical judgment and the Predict Sepsis screening tools with respect to identification of sepsis among ambulance patients with suspected infection: a prospective cohort study

BACKGROUND

There is little evidence of which sepsis screening tool to use in the ambulance setting. The primary aim of the current study was to compare the performance of NEWS2 (National Early Warning score 2) and RETTS (Rapid Emergency Triage and Treatment System) with respect to identification of sepsis among ambulance patients with clinically suspected infection. The secondary aim was to compare the performance of the novel Predict Sepsis screening tools with that of NEWS2, RETTS and clinical judgment.

METHODS

Prospective cohort study of 323 adult ambulance patients with clinically suspected infection, transported to hospitals in Stockholm, during 2017/2018. The sensitivity, specificity, and AUC (Area Under the receiver operating Curve) were calculated and compared by using McNemar´s test and DeLong's test.

RESULTS

The prevalence of sepsis in the current study population was 44.6% (144 of 323 patients). No significant difference in AUC was demonstrated between NEWS2 ≥ 5 and RETTS ≥ orange. NEWS2 ≥ 7 demonstrated a significantly greater AUC than RETTS red. The Predict Sepsis screening tools ≥ 2 demonstrated the highest sensitivity (range 0.87-0.91), along with RETTS ≥ orange (0.83), but the lowest specificity (range 0.39-0.49). The AUC of NEWS2 (0.73) and the Predict Sepsis screening tools (range 0.75-0.77) was similar.

CONCLUSIONS

The results indicate that NEWS2 could be the better alternative for sepsis identification in the ambulance, as compared to RETTS. The Predict Sepsis screening tools demonstrated a high sensitivity and AUCs similar to that of NEWS2. However, these results need to be interpreted with caution as the Predict Sepsis screening tools require external validation.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03249597. Registered 15 August 2017-Retrospectively registered, https://clinicaltrials.gov/ct2/show/NCT03249597 .