Impact of an emergency department electronic sepsis surveillance system on patient mortality and length of stay

Views and Uses of Sepsis Digital Alerts in National Health Service Trusts in England: Qualitative Study With Health Care Professionals

BACKGROUND

Sepsis is a common cause of serious illness and death. Sepsis management remains challenging and suboptimal. To support rapid sepsis diagnosis and treatment, screening tools have been embedded into hospital digital systems to appear as digital alerts. The implementation of digital alerts to improve the management of sepsis and deterioration is a complex intervention that has to fit with team workflow and the views and practices of hospital staff. Despite the importance of human decision-making and behavior in optimal implementation, there are limited qualitative studies that explore the views and experiences of health care professionals regarding digital alerts as sepsis or deterioration computerized clinician decision support systems (CCDSSs).

OBJECTIVE

This study aims to explore the views and experiences of health care professionals on the use of sepsis or deterioration CCDSSs and to identify barriers and facilitators to their implementation and use in National Health Service (NHS) hospitals.

METHODS

We conducted a qualitative, multisite study with unstructured observations and semistructured interviews with health care professionals from emergency departments, outreach teams, and intensive or acute units in 3 NHS hospital trusts in England. Data from both interviews and observations were analyzed together inductively using thematic analysis.

RESULTS

A total of 22 health care professionals were interviewed, and 12 observation sessions were undertaken. A total of four themes regarding digital alerts were identified: (1) support decision-making as nested in electronic health records, but never substitute professionals' knowledge and experience; (2) remind to take action according to the context, such as the hospital unit and the job role; (3) improve the alerts and their introduction, by making them more accessible, easy to use, not intrusive, more accurate, as well as integrated across the whole health care system; and (4) contextual factors affecting views and use of alerts in the NHS trusts. Digital alerts are more optimally used in general hospital units with a lower senior decision maker:patient ratio and by health care professionals with experience of a similar technology. Better use of the alerts was associated with quality improvement initiatives and continuous sepsis training. The trusts' features, such as the presence of a 24/7 emergency outreach team, good technological resources, and staffing and teamwork, favored a more optimal use.

CONCLUSIONS

Trust implementation of sepsis or deterioration CCDSSs requires support on multiple levels and at all phases of the intervention, starting from a prego-live analysis addressing organizational needs and readiness. Advancements toward minimally disruptive and smart digital alerts as sepsis or deterioration CCDSSs, which are more accurate and specific but at the same time scalable and accessible, require policy changes and investments in multidisciplinary research.

Sepsis Alert Systems, Mortality, and Adherence in Emergency Departments: A Systematic Review and Meta-Analysis

Importance

Early detection and management of sepsis are crucial for patient survival. Emergency departments (EDs) play a key role in sepsis management but face challenges in timely response due to high patient volumes. Sepsis alert systems are proposed to expedite diagnosis and treatment initiation per the Surviving Sepsis Campaign guidelines.

Objective

To review and analyze the association of sepsis alert systems in EDs with patient outcomes.

Data Sources

A thorough search was conducted in PubMed, EMBASE, Web of Science, and the Cochrane Library from January 1, 2004, to November 19, 2023.

Study Selection

Studies that evaluated sepsis alert systems specifically designed for adult ED patients were evaluated. Inclusion criteria focused on peer-reviewed, full-text articles in English that reported on mortality, ICU admissions, hospital stay duration, and sepsis management adherence. Exclusion criteria included studies that lacked a control group or quantitative reports.

Data Extraction and Synthesis

The review followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline. Two independent reviewers conducted the data extraction using a standardized form. Any disagreements were resolved through discussion. The data were synthesized using a random-effects model due to the expected heterogeneity among the included studies.

Main Outcomes and Measures

Key outcomes included mortality, intensive care unit admissions, hospital stay duration, and adherence to the sepsis bundle.

Results

Of 3281 initially identified studies, 22 (0.67%) met inclusion criteria, encompassing 19 580 patients. Sepsis alert systems were associated with reduced mortality risk (risk ratio [RR], 0.81; 95% CI, 0.71 to 0.91) and length of hospital stay (standardized mean difference [SMD], -0.15; 95% CI, -0.20 to -0.11). These systems were also associated with better adherence to sepsis bundle elements, notably in terms of shorter time to fluid administration (SMD, -0.42; 95% CI, -0.52 to -0.32), blood culture (SMD, -0.31; 95% CI, -0.40 to -0.21), antibiotic administration (SMD, -0.34; 95% CI, -0.39 to -0.29), and lactate measurement (SMD, -0.15; 95% CI, -0.22 to -0.08). Electronic alerts were particularly associated with reduced mortality (RR, 0.78; 95% CI, 0.67 to 0.92) and adherence with blood culture guidelines (RR, 1.14; 95% CI, 1.03 to 1.27).

Conclusions and Relevance

These findings suggest that sepsis alert systems in EDs were associated with better patient outcomes along with better adherence to sepsis management protocols. These systems hold promise for enhancing ED responses to sepsis, potentially leading to better patient outcomes.

The role of information systems in emergency department decision-making-a literature review

OBJECTIVES

Healthcare providers employ heuristic and analytical decision-making to navigate the high-stakes environment of the emergency department (ED). Despite the increasing integration of information systems (ISs), research on their efficacy is conflicting. Drawing on related fields, we investigate how timing and mode of delivery influence IS effectiveness. Our objective is to reconcile previous contradictory findings, shedding light on optimal IS design in the ED.

MATERIALS AND METHODS

We conducted a systematic review following PRISMA across PubMed, Scopus, and Web of Science. We coded the ISs' timing as heuristic or analytical, their mode of delivery as active for automatic alerts and passive when requiring user-initiated information retrieval, and their effect on process, economic, and clinical outcomes.

RESULTS

Our analysis included 83 studies. During early heuristic decision-making, most active interventions were ineffective, while passive interventions generally improved outcomes. In the analytical phase, the effects were reversed. Passive interventions that facilitate information extraction consistently improved outcomes.

DISCUSSION

Our findings suggest that the effectiveness of active interventions negatively correlates with the amount of information received during delivery. During early heuristic decision-making, when information overload is high, physicians are unresponsive to alerts and proactively consult passive resources. In the later analytical phases, physicians show increased receptivity to alerts due to decreased diagnostic uncertainty and information quantity. Interventions that limit information lead to positive outcomes, supporting our interpretation.

CONCLUSION

We synthesize our findings into an integrated model that reveals the underlying reasons for conflicting findings from previous reviews and can guide practitioners in designing ISs in the ED.

Trend of mortality and length of stay in the emergency department following implementation of a centralized sepsis alert system

Introduction

Sepsis alerts based on laboratory and vital sign criteria were found insufficient to improve patient outcomes. While most early sepsis alerts were implemented into smaller scale operating systems, a centralized new approach may provide more benefits, overcoming alert fatigue, improving deployment of staff and resources, and optimizing the overall management of sepsis. The objective of the study was to assess mortality and length of stay (LOS) trends in emergency department (ED) patients, following the implementation of a centralized and automated sepsis alert system.

Methods

The automated sepsis alert system was implemented in 2021 as part of a hospital-wide command and control center. Administrative data from the years 2018 to 2021 were collected. Data included ED visits, in-hospital mortality, triage levels, LOS, and the Canadian Triage and Acuity Scale (CTAS).

Results

Mortality rate for patients classified as CTAS I triage level was the lowest in 2021, after the implementation of the automated sepsis alert system, compared to 2020, 2019, and 2018 (p < 0.001). The Kaplan-Meier survival curve revealed that for patients classified as CTAS I triage level, the probability of survival was the highest in 2021, after implementation of the sepsis alert algorithm, compared to previous years (Log Rank, Mantel-Cox, χ²=29.742, p < 0.001). No significant differences in survival rate were observed for other triage levels.

Conclusion

Implementing an automated sepsis alert system as part of a command center operation significantly improves mortality rate associated with LOS in the ED for patients in the highest triage level. These findings suggest that a centralized early sepsis alert system has the potential to improve patient outcomes.

Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review

OBJECTIVE

Surveillance algorithms that predict patient decompensation are increasingly integrated with clinical workflows to help identify patients at risk of in-hospital deterioration. This scoping review aimed to identify the design features of the information displays, the types of algorithm that drive the display, and the effect of these displays on process and patient outcomes.

MATERIALS AND METHODS

The scoping review followed Arksey and O'Malley's framework. Five databases were searched with dates between January 1, 2009 and January 26, 2022. Inclusion criteria were: participants-clinicians in inpatient settings; concepts-intervention as deterioration information displays that leveraged automated AI algorithms; comparison as usual care or alternative displays; outcomes as clinical, workflow process, and usability outcomes; and context as simulated or real-world in-hospital settings in any country. Screening, full-text review, and data extraction were reviewed independently by 2 researchers in each step. Display categories were identified inductively through consensus.

RESULTS

Of 14 575 articles, 64 were included in the review, describing 61 unique displays. Forty-one displays were designed for specific deteriorations (eg, sepsis), 24 provided simple alerts (ie, text-based prompts without relevant patient data), 48 leveraged well-accepted score-based algorithms, and 47 included nurses as the target users. Only 1 out of the 10 randomized controlled trials reported a significant effect on the primary outcome.

CONCLUSIONS

Despite significant advancements in surveillance algorithms, most information displays continue to leverage well-understood, well-accepted score-based algorithms. Users' trust, algorithmic transparency, and workflow integration are significant hurdles to adopting new algorithms into effective decision support tools.

Evaluation of evolving sepsis screening criteria in discriminating suspected sepsis and mortality among adult patients admitted to the intensive care unit

BACKGROUND

Institutions struggle with successful use of sepsis alerts within electronic health records.

OBJECTIVE

Test the association of sepsis screening measurement criteria in discrimination of mortality and detection of sepsis in a large dataset.

DESIGN

Retrospective, cohort study using a large United States (U.S.) intensive care database. The Institutional Review Board exempt status was obtained from Kansas University Medical Center Human Research Protection Program (10-1-2015).

SETTING

334 U.S. hospitals participating in the eICU Research Institute.

PARTICIPANTS

Nine hundred twelve thousand five hundred and nine adult intensive care admissions from 183 hospitals.

METHODS

Exposures included: systemic inflammatory response syndrome criteria ≥ 2 (Sepsis-1); systemic inflammatory response syndrome criteria with organ failure criteria ≥ 3.5 points (Sepsis-2); and sepsis-related organ failure assessment score ≥ 2 and quick score ≥ 2 (Sepsis-3). Discrimination of outcomes was determined with/without (adjusted/unadjusted) baseline risk exposure to a model. The receiver operating characteristic curve (AUROC) and odds ratios (ORs) for each decile of baseline risk of sepsis or death were assessed.

RESULTS

Within the eligible cohort of 912,509, a total of 86,219 (9.4 %) patients did not survive their hospital stay and 186,870 (20.5 %) met the definition of suspected sepsis. For suspected sepsis discrimination, Sepsis-2 (unadjusted AUROC 0.67, 99 % CI: 0.66-0.67 and adjusted AUROC 0.77, 99 % CI: 0.77-0.77) outperformed Sepsis-3 (SOFA unadjusted AUROC 0.61, 99 % CI: 0.61-0.61 and adjusted AUROC 0.74, 99 % CI: 0.74-0.74) (qSOFA unadjusted AUROC 0.59, 99 % CI: 0.59-0.60 and adjusted AUROC 0.73, 99 % CI: 0.73-0.73). Sepsis-2 also outperformed Sepsis-1 (unadjusted AUROC 0.58, 99 % CI: 0.58-0.58 and adjusted AUROC 0.73, 99 % CI: 0.73-0.73). In between differences of AUROCs were statistically significantly different. Sepsis-2 ORs were higher for the outcome of suspected sepsis when considering deciles of risk than the other measurement systems.

CONCLUSIONS AND RELEVANCE

Sepsis-2 outperformed other systems in suspected sepsis detection and was comparable to SOFA in prognostic accuracy of mortality in adult intensive care patients.

Epic Sepsis Model Inpatient Predictive Analytic Tool: A Validation Study

Earlier treatment of sepsis leads to decreased mortality. Epic is an electronic medical record providing a predictive alert system for sepsis, the Epic Sepsis Model (ESM) Inpatient Predictive Analytic Tool. External validation of this system is lacking. This study aims to evaluate the ESM as a sepsis screening tool and determine whether an association exists between ESM alert system implementation and subsequent sepsis-related mortality.

DESIGN

Before-and-after study comparing baseline and intervention period.

SETTING

Urban 746-bed academic level 1 trauma center.

PATIENTS

Adult acute care inpatients discharged between January 12, 2018, and July 31, 2019.

INTERVENTIONS

During the before period, ESM was turned on in the background, but nurses and providers were not alerted of results. The system was then activated to alert providers of scores greater than or equal to 5, a set point determined using receiver operating characteristic curve analysis (area under the curve, 0.834; p < 0.001).

MEASUREMENTS AND MAIN RESULTS

Primary outcome was mortality during hospitalization; secondary outcomes were sepsis order set utilization, length of stay, and timing of administration of sepsis-appropriate antibiotics. Of the 11,512 inpatient encounters assessed by ESM, 10.2% (1,171) had sepsis based on diagnosis codes. As a screening test, the ESM had sensitivity, specificity, positive predictive value, and negative predictive value rates of 86.0%, 80.8%, 33.8%, and 98.11%, respectively. After ESM implementation, unadjusted mortality rates in patients with ESM score greater than or equal to 5 and who had not yet received sepsis-appropriate antibiotics declined from 24.3% to 15.9%; multivariable analysis yielded an odds ratio of sepsis-related mortality (95% CI) of 0.56 (0.39-0.80).

CONCLUSIONS

In this single-center before-and-after study, utilization of the ESM score as a screening test was associated with a 44% reduction in the odds of sepsis-related mortality. Due to wide utilization of Epic, this is a potentially promising tool to improve sepsis mortality in the United States. This study is hypothesis generating, and further work with more rigorous study design is needed.

Meaningful time-related aspects of alerts in Clinical Decision Support Systems. A unified framework

Alerts are a common functionality of clinical decision support systems (CDSSs). Although they have proven to be useful in clinical practice, the alert burden can lead to alert fatigue and significantly reduce their usability and acceptance. Based on a literature review, we propose a unified framework consisting of a set of meaningful timestamps that allows the use of state-of-the-art measures for alert burden, such as alert dwell time, alert think time, and response time. In addition, it can be used to investigate other measures that could be relevant as regards dealing with this problem. Furthermore, we provide a case study concerning three different types of alerts to which the framework was successfully applied. We consider that our framework can easily be adapted to other CDSSs and that it could be useful for dealing with alert burden measurement thus contributing to its appropriate management.

Effectiveness of automated alerting system compared to usual care for the management of sepsis

There is a large body of evidence showing that delayed initiation of sepsis bundle is associated with adverse clinical outcomes in patients with sepsis. However, it is controversial whether electronic automated alerts can help improve clinical outcomes of sepsis. Electronic databases are searched from inception to December 2021 for comparative effectiveness studies comparing automated alerts versus usual care for the management of sepsis. A total of 36 studies are eligible for analysis, including 6 randomized controlled trials and 30 non-randomized studies. There is significant heterogeneity in these studies concerning the study setting, design, and alerting methods. The Bayesian meta-analysis by using pooled effects of non-randomized studies as priors shows a beneficial effect of the alerting system (relative risk [RR]: 0.71; 95% credible interval: 0.62 to 0.81) in reducing mortality. The automated alerting system shows less beneficial effects in the intensive care unit (RR: 0.90; 95% CI: 0.73–1.11) than that in the emergency department (RR: 0.68; 95% CI: 0.51–0.90) and ward (RR: 0.71; 95% CI: 0.61–0.82). Furthermore, machine learning-based prediction methods can reduce mortality by a larger magnitude (RR: 0.56; 95% CI: 0.39–0.80) than rule-based methods (RR: 0.73; 95% CI: 0.63–0.85). The study shows a statistically significant beneficial effect of using the automated alerting system in the management of sepsis. Interestingly, machine learning monitoring systems coupled with better early interventions show promise, especially for patients outside of the intensive care unit.

The Ramathibodi early warning score as a sepsis screening tool does not reduce the timing of antibiotic administration

Background

Administration of antibiotics to septic patients within 1 h was recommended in 2018 by the Surviving Sepsis Campaign (SSC) as a strategy to improve survival outcomes. The use of sepsis screening tools in emergency departments (EDs) is important for early diagnosis and initiation of sepsis care. This study aimed to assess the impact of the Ramathibodi early warning score (REWs) on the administration of antibiotics within 1 h of presentation.

Methods

This was an observational retrospective cohort study with propensity score matching between the sepsis-3 criteria (pre-period) and the REWs (post-period) as screening tools in adult patients with sepsis in EDs. The primary outcome was the proportion of receiving antibiotics within 1 h of presentation in the pre- and post-periods.

Results

A total of 476 patients were analyzed without propensity matching. The proportion of antibiotic administration within 1 h was higher in patients screened using the REWs compared with standard of care in the total study population (79.5% vs. 61.4%, p < 0.001). After propensity score matching, 153 patients were included in both groups. The proportion of antibiotic administration within 1 h was similar in patients screened using the REWs and those receiving standard of care (79.7% vs. 80.4%, p = 0.886). However, time to intensive care unit (ICU) admission was faster in patients screened using the REWs. Delays in receiving antibiotics of longer than 3 h were associated with increased mortality (adjusted hazard ratio 7.04, 95% confidence interval 1.45 to 34.11, p = 0.015).

Conclusions

Implementing the REWs as a tool in sepsis screening protocols in EDs did not improve rates of antibiotic administration within 1 h as recommended by the SSC. However, time to ICU admission was improved after implementation of the REWs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12245-022-00420-w.

Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Adult Inpatients: Scoping Review

Background

Sepsis is a significant cause of morbidity and mortality worldwide. Early detection of sepsis followed promptly by treatment initiation improves patient outcomes and saves lives. Hospitals are increasingly using computerized clinical decision support (CCDS) systems for the rapid identification of adult patients with sepsis.

Objective

This scoping review aims to systematically describe studies reporting on the use and evaluation of CCDS systems for the early detection of adult inpatients with sepsis.

Methods

The protocol for this scoping review was previously published. A total of 10 electronic databases (MEDLINE, Embase, CINAHL, the Cochrane database, LILACS [Latin American and Caribbean Health Sciences Literature], Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and PQDT [ProQuest Dissertations and Theses]) were comprehensively searched using terms for sepsis, CCDS, and detection to identify relevant studies. Title, abstract, and full-text screening were performed by 2 independent reviewers using predefined eligibility criteria. Data charting was performed by 1 reviewer with a second reviewer checking a random sample of studies. Any disagreements were discussed with input from a third reviewer. In this review, we present the results for adult inpatients, including studies that do not specify patient age.

Results

A search of the electronic databases retrieved 12,139 studies following duplicate removal. We identified 124 studies for inclusion after title, abstract, full-text screening, and hand searching were complete. Nearly all studies (121/124, 97.6%) were published after 2009. Half of the studies were journal articles (65/124, 52.4%), and the remainder were conference abstracts (54/124, 43.5%) and theses (5/124, 4%). Most studies used a single cohort (54/124, 43.5%) or before-after (42/124, 33.9%) approach. Across all 124 included studies, patient outcomes were the most frequently reported outcomes (107/124, 86.3%), followed by sepsis treatment and management (75/124, 60.5%), CCDS usability (14/124, 11.3%), and cost outcomes (9/124, 7.3%). For sepsis identification, the systemic inflammatory response syndrome criteria were the most commonly used, alone (50/124, 40.3%), combined with organ dysfunction (28/124, 22.6%), or combined with other criteria (23/124, 18.5%). Over half of the CCDS systems (68/124, 54.8%) were implemented alongside other sepsis-related interventions.

Conclusions

The current body of literature investigating the implementation of CCDS systems for the early detection of adult inpatients with sepsis is extremely diverse. There is substantial variability in study design, CCDS criteria and characteristics, and outcomes measured across the identified literature. Future research on CCDS system usability, cost, and impact on sepsis morbidity is needed.

International Registered Report Identifier (IRRID)

RR2-10.2196/24899

Pediatric sepsis screening in US hospitals

Sepsis is a major cause of morbidity and mortality in children. While adverse outcomes can be reduced through prompt initiation of sepsis protocols including fluid resuscitation and antibiotics, provision of these therapies relies on clinician recognition of sepsis. Recognition is challenging in children because early signs of shock such as tachycardia and tachypnea have low specificity while hypotension often does not occur until late in the clinical course. This narrative review highlights the important context that has led to the rapid growth of pediatric sepsis screening in the United States. In this review, we (1) describe different screening tools used in US emergency department, inpatient, and intensive care unit settings; (2) highlight details of the design, implementation, and evaluation of specific tools; (3) review the available data on the process of integrating sepsis screening into an overall sepsis quality improvement program and on the effect of these screening tools on patient outcomes; (4) discuss potential harms of sepsis screening including alarm fatigue; and (5) highlight several future directions in sepsis screening, such as novel tools that incorporate artificial intelligence and machine learning methods to augment sepsis identification with the ultimate goal of precision-based approaches to sepsis recognition and treatment. IMPACT: This narrative review highlights the context that has led to the rapid growth of pediatric sepsis screening nationally. Screening tools used in US emergency department, inpatient, and intensive care unit settings are described in terms of their design, implementation, and clinical performance. Limitations and potential harms of these tools are highlighted, as well as future directions that may lead to a more precision-based approach to sepsis recognition and treatment.

The impact of recency and adequacy of historical information on sepsis predictions using machine learning

Sepsis is a major public and global health concern. Every hour of delay in detecting sepsis significantly increases the risk of death, highlighting the importance of accurately predicting sepsis in a timely manner. A growing body of literature has examined developing new or improving the existing machine learning (ML) approaches for timely and accurate predictions of sepsis. This study contributes to this literature by providing clear insights regarding the role of the recency and adequacy of historical information in predicting sepsis using ML. To this end, we implemented a deep learning model using a bidirectional long short-term memory (BiLSTM) algorithm and compared it with six other ML algorithms based on numerous combinations of the prediction horizons (to capture information recency) and observation windows (to capture information adequacy) using different measures of predictive performance. Our results indicated that the BiLSTM algorithm outperforms all other ML algorithms and provides a great separability of the predicted risk of sepsis among septic versus non-septic patients. Moreover, decreasing the prediction horizon (in favor of information recency) always boosts the predictive performance; however, the impact of expanding the observation window (in favor of information adequacy) depends on the prediction horizon and the purpose of prediction. More specifically, when the prediction is responsive to the positive label (i.e., Sepsis), increasing historical data improves the predictive performance when the prediction horizon is short-moderate.

The appropriateness of clinical decision support systems alerts in supporting clinical workflows: A systematic review

A CDSS generates a high number of inappropriate alerts that interrupt the clinical workflow. As a result, clinicians silence, disable, or ignore alerts, thereby undermining patient safety. Therefore, the effectiveness and appropriateness of CDSS alerts need to be evaluated. A systematic review was carried out to identify the factors that affect CDSS alert appropriateness in supporting clinical workflow. Seven electronic databases (PubMed, Scopus, ACM, Science Direct, IEEE, Ovid Medline, and Ebscohost) were searched for English language articles published between 1997 and 2018. Seventy six papers met the inclusion criteria, of which 26, 24, 15, and 11 papers are retrospective cohort, qualitative, quantitative, and mixed-method studies, respectively. The review highlights various factors influencing the appropriateness and efficiencies of CDSS alerts. These factors are categorized into technology, human, organization, and process aspects using a combination of approaches, including socio-technical framework, five rights of CDSS, and Lean. Most CDSS alerts were not properly designed based on human factor methods and principles, explaining high alert overrides in clinical practices. The identified factors and recommendations from the review may offer valuable insights into how CDSS alerts can be designed appropriately to support clinical workflow.

Outcomes of Patients with Sepsis in a Pediatric Emergency Department after Automated Sepsis Screening

OBJECTIVE

To determine the effect of an automated sepsis screening tool on treatment and outcomes of severe sepsis in a pediatric emergency department (ED).

STUDY DESIGN

Retrospective cohort study of encounters of patients with severe sepsis in a pediatric ED with a high volume of pediatric sepsis cases over a 2-year period. The automated sepsis screening algorithm replaced a manual screen 1 year into the study. The primary outcome was the proportion of patients treated for sepsis while in the ED. Secondary outcomes were time from ED arrival to first intravenous (IV) antibiotic and first IV fluid bolus, volume of fluid administered in the ED, 30-day mortality, intensive care unit-free days, and hospital-free days.

RESULTS

In year 1 of the study, 8910 of 61 026 (14.6%) of encounters had a manual sepsis screen; 137 patients met criteria for severe sepsis. In year 2, 100% of 61 195 encounters had an automated sepsis screen and there were 136 cases of severe sepsis. There was a higher proportion of patients with severe sepsis who had an active malignancy and indwelling central venous catheter in year 2. There were no differences in the proportion of patients treated for sepsis in the ED, time to first IV antibiotic or first IV fluid bolus, fluid volume delivered in the ED, hospital-free days, intensive care unit-free days, or 30-day mortality after implementation of the automated screening algorithm.

CONCLUSIONS

An automated sepsis screening algorithm introduced into an academic pediatric ED with a high volume of sepsis cases did not lead to improvements in treatment or outcomes of severe sepsis in this study.

Applying A/B Testing to Clinical Decision Support: Rapid Randomized Controlled Trials

BACKGROUND

Clinical decision support (CDS) is a valuable feature of electronic health records (EHRs) designed to improve quality and safety. However, due to the complexities of system design and inconsistent results, CDS tools may inadvertently increase alert fatigue and contribute to physician burnout. A/B testing, or rapid-cycle randomized tests, is a useful method that can be applied to the EHR in order to rapidly understand and iteratively improve design choices embedded within CDS tools.

OBJECTIVE

This paper describes how rapid randomized controlled trials (RCTs) embedded within EHRs can be used to quickly ascertain the superiority of potential CDS design changes to improve their usability, reduce alert fatigue, and promote quality of care.

METHODS

A multistep process combining tools from user-centered design, A/B testing, and implementation science was used to understand, ideate, prototype, test, analyze, and improve each candidate CDS. CDS engagement metrics (alert views, acceptance rates) were used to evaluate which CDS version is superior.

RESULTS

To demonstrate the impact of the process, 2 experiments are highlighted. First, after multiple rounds of usability testing, a revised CDS influenza alert was tested against usual care CDS in a rapid (~6 weeks) RCT. The new alert text resulted in minimal impact on reducing firings per patients per day, but this failure triggered another round of review that identified key technical improvements (ie, removal of dismissal button and firings in procedural areas) that led to a dramatic decrease in firings per patient per day (23.1 to 7.3). In the second experiment, the process was used to test 3 versions (financial, quality, regulatory) of text supporting tobacco cessation alerts as well as 3 supporting images. Based on 3 rounds of RCTs, there was no significant difference in acceptance rates based on the framing of the messages or addition of images.

CONCLUSIONS

These experiments support the potential for this new process to rapidly develop, deploy, and rigorously evaluate CDS within an EHR. We also identified important considerations in applying these methods. This approach may be an important tool for improving the impact of and experience with CDS.

TRIAL REGISTRATION

Flu alert trial: ClinicalTrials.gov NCT03415425; https://clinicaltrials.gov/ct2/show/NCT03415425. Tobacco alert trial: ClinicalTrials.gov NCT03714191; https://clinicaltrials.gov/ct2/show/NCT03714191.

Evaluating a digital sepsis alert in a London multisite hospital network: a natural experiment using electronic health record data

Objective

The study sought to determine the impact of a digital sepsis alert on patient outcomes in a UK multisite hospital network.

Materials and Methods

A natural experiment utilizing the phased introduction (without randomization) of a digital sepsis alert into a multisite hospital network. Sepsis alerts were either visible to clinicians (patients in the intervention group) or running silently and not visible (the control group). Inverse probability of treatment-weighted multivariable logistic regression was used to estimate the effect of the intervention on individual patient outcomes.

Outcomes

In-hospital 30-day mortality (all inpatients), prolonged hospital stay (≥7 days) and timely antibiotics (≤60 minutes of the alert) for patients who alerted in the emergency department.

Results

The introduction of the alert was associated with lower odds of death (odds ratio, 0.76; 95% confidence interval [CI], 0.70-0.84; n = 21 183), lower odds of prolonged hospital stay ≥7 days (OR, 0.93; 95% CI, 0.88-0.99; n = 9988), and in patients who required antibiotics, an increased odds of receiving timely antibiotics (OR, 1.71; 95% CI, 1.57-1.87; n = 4622).

Discussion

Current evidence that digital sepsis alerts are effective is mixed. In this large UK study, a digital sepsis alert has been shown to be associated with improved outcomes, including timely antibiotics. It is not known whether the presence of alerting is responsible for improved outcomes or whether the alert acted as a useful driver for quality improvement initiatives.

Conclusions

These findings strongly suggest that the introduction of a network-wide digital sepsis alert is associated with improvements in patient outcomes, demonstrating that digital based interventions can be successfully introduced and readily evaluated.

Comparison of Manual and Automated Sepsis Screening Tools in a Pediatric Emergency Department

OBJECTIVES

To compare the performance and test characteristics of an automated sepsis screening tool with that of a manual sepsis screen in patients presenting to a pediatric emergency department (ED).

METHODS

We conducted a retrospective cohort study of encounters in a pediatric ED over a 2-year period. The automated sepsis screening algorithm replaced the manual sepsis screen 1 year into the study. A positive case was defined as development of severe sepsis or septic shock within 24 hours of disposition from the ED. We calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and positive and negative likelihood ratios with 95% confidence intervals (CIs) for each.

RESULTS

There were 122 221 ED encounters during the study period and 273 cases of severe sepsis. During year 1 of the study, the manual screen was performed in 8910 of 61 026 (14.6%) encounters, resulting in the following test characteristics: sensitivity of 64.6% (95% CI 54.2%-74.1%), specificity of 91.1% (95% CI 90.5%-91.7%), PPV of 7.3% (95% CI 6.3%-8.5%), and NPV of 99.6% (95% CI 99.5%-99.7%). During year 2 of the study, the automated screen was performed in 100% of 61 195 encounters, resulting in the following test characteristics: sensitivity of 84.6% (95% CI 77.4%-90.2%), specificity of 95.1% (95% CI 94.9%-95.2%), PPV of 3.7% (95% CI 3.4%-4%), and NPV of 99.9% (95% CI 99.9%-100%).

CONCLUSIONS

An automated sepsis screening algorithm had higher sensitivity and specificity than a widely used manual sepsis screen and was performed on 100% of patients in the ED, ensuring continuous sepsis surveillance throughout the ED stay.

Implementation of the Surviving Sepsis Campaign one-hour bundle in a short stay unit: A quality improvement project

OBJECTIVE

To improve timely sepsis care by implementing the 2018 Surviving Sepsis Campaign one-hour interventions.

DESIGN

Ten-month prospective quality improvement project.

SETTING

A 38-bed short stay unit within an 800-bed hospital in New York City.

PARTICIPANTS

Patients admitted to the short stay unit who screened positive for sepsis.

INTERVENTION

A sepsis implementation tool was created from the 2018 Surviving Sepsis Campaign guidelines. Sepsis champions delivered education on sepsis recognition, treatment, and management, and the sepsis implementation tool to the healthcare staff.

PROCESS AND OUTCOME MEASURES

Time to first lactate, blood cultures × 2, antibiotic administration, length of stay and mortality were tracked weekly for five months.

RESULTS

From May 6, 2019 to October 1, 2019, 32 patients were diagnosed with sepsis. Initial lactate and blood cultures were completed on every patient within 1one-hour of sepsis diagnosis. Administration of antibiotics within one-hour reached 100% after week four and was sustained.

CONCLUSION

Use of a registered nurse-initiated sepsis implementation tool in a short stay unit led to the completion of blood cultures, initial lactate, and antibiotic administration within one-hour. Key factors to support this practice improvement were increasing registered nurse, physician and physician assistant sepsis knowledge, registered nurse and physician/physician assistant early collaboration, increased staffing and intravenous access equipment.

Impact of Electronic Medical Record Alerts on Emergency Physician Workflow and Medical Management

BACKGROUND

Electronic medical record (EMR) alerts are automated messages that notify the physician of important information. However, little is known about how EMR alerts affect the workflow and decision-making of emergency physicians (EPs).

STUDY OBJECTIVES

This study aimed to quantify the number of EMR alerts EPs receive, the time required to resolve alerts, the types of alerts EPs receive, and the impact of alerts on patient management.

METHODS

We performed a prospective observational study at a tertiary care ED with 130,000 visits annually. Research assistants observed EPs on shift from May to December 2018. They recorded the number of EMR alerts received, time spent addressing the alerts, the types of alerts received, and queried the EP to determine if the alert impacted patient management.

RESULTS

Seven residents and six attending physicians were observed on a total of 17 shifts and 153 patient encounters; 78% (119) of patient encounters involved alerts. These 119 patients triggered 530 EMR alerts. EPs spent a mean of 7.06 s addressing each alert and addressed 3.46 alerts per total patient seen. In total, EPs spent approximately 24 s per patient resolving alerts. Only 12 alerts (2.26%) changed clinical management.

CONCLUSION

EPs frequently receive EMR alerts, however, most alerts were not perceived to impact patient care. These alerts contribute to the high volume of interruptions EPs must contend with in the clinical environment of the ED.

Sepsis Alerts in Emergency Departments: A Systematic Review of Accuracy and Quality Measure Impact

Introduction

For early detection of sepsis, automated systems within the electronic health record have evolved to alert emergency department (ED) personnel to the possibility of sepsis, and in some cases link them to suggested care pathways. We conducted a systematic review of automated sepsis-alert detection systems in the ED.

Methods

We searched multiple health literature databases from the earliest available dates to August 2018. Articles were screened based on abstract, again via manuscript, and further narrowed with set inclusion criteria: 1) adult patients in the ED diagnosed with sepsis, severe sepsis, or septic shock; 2) an electronic system that alerts a healthcare provider of sepsis in real or near-real time; and 3) measures of diagnostic accuracy or quality of sepsis alerts. The final, detailed review was guided by QUADAS-2 and GRADE criteria. We tracked all articles using an online tool (Covidence), and the review was registered with PROSPERO registry of reviews. A two-author consensus was reached at the article choice stage and final review stage. Due to the variation in alert criteria and methods of sepsis diagnosis confirmation, the data were not combined for meta-analysis.

Results

We screened 693 articles by title and abstract and 20 by full text; we then selected 10 for the study. The articles were published between 2009–2018. Two studies had algorithm-based alert systems, while eight had rule-based alert systems. All systems used different criteria based on systemic inflammatory response syndrome (SIRS) to define sepsis. Sensitivities ranged from 10–100%, specificities from 78–99%, and positive predictive value from 5.8–54%. Negative predictive value was consistently high at 99–100%. Studies showed some evidence for improved process-of-care markers, including improved time to antibiotics. Length of stay improved in two studies. One low quality study showed improved mortality.

Conclusion

The limited evidence available suggests that sepsis alerts in the ED setting can be set to high sensitivity. No high-quality studies showed a difference in mortality, but evidence exists for improvements in process of care. Significant further work is needed to understand the consequences of alert fatigue and sensitivity set points.

Clinical Decision Support and Implications for the Clinician Burnout Crisis

OBJECTIVES

This survey aimed to review aspects of clinical decision support (CDS) that contribute to burnout and identify key themes for improving the acceptability of CDS to clinicians, with the goal of decreasing said burnout.

METHODS

We performed a survey of relevant articles from 2018-2019 addressing CDS and aspects of clinician burnout from PubMed and Web of Science™. Themes were manually extracted from publications that met inclusion criteria.

RESULTS

Eighty-nine articles met inclusion criteria, including 12 review articles. Review articles were either prescriptive, describing how CDS should work, or analytic, describing how current CDS tools are deployed. The non-review articles largely demonstrated poor relevance and acceptability of current tools, and few studies showed benefits in terms of efficiency or patient outcomes from implemented CDS. Encouragingly, multiple studies highlighted steps that succeeded in improving both acceptability and relevance of CDS.

CONCLUSIONS

CDS can contribute to clinician frustration and burnout. Using the techniques of improving relevance, soliciting feedback, customization, measurement of outcomes and metrics, and iteration, the effects of CDS on burnout can be ameliorated.

Effect of a sepsis prediction algorithm on patient mortality, length of stay and readmission: a prospective multicentre clinical outcomes evaluation of real-world patient data from US hospitals

BACKGROUND

Severe sepsis and septic shock are among the leading causes of death in the USA. While early prediction of severe sepsis can reduce adverse patient outcomes, sepsis remains one of the most expensive conditions to diagnose and treat.

OBJECTIVE

The purpose of this study was to evaluate the effect of a machine learning algorithm for severe sepsis prediction on in-hospital mortality, hospital length of stay and 30-day readmission.

DESIGN

Prospective clinical outcomes evaluation.

SETTING

Evaluation was performed on a multiyear, multicentre clinical data set of real-world data containing 75 147 patient encounters from nine hospitals across the continental USA, ranging from community hospitals to large academic medical centres.

PARTICIPANTS

Analyses were performed for 17 758 adult patients who met two or more systemic inflammatory response syndrome criteria at any point during their stay ('sepsis-related' patients).

INTERVENTIONS

Machine learning algorithm for severe sepsis prediction.

OUTCOME MEASURES

In-hospital mortality, length of stay and 30-day readmission rates.

RESULTS

Hospitals saw an average 39.5% reduction of in-hospital mortality, a 32.3% reduction in hospital length of stay and a 22.7% reduction in 30-day readmission rate for sepsis-related patient stays when using the machine learning algorithm in clinical outcomes analysis.

CONCLUSIONS

Reductions of in-hospital mortality, hospital length of stay and 30-day readmissions were observed in real-world clinical use of the machine learning-based algorithm. The predictive algorithm may be successfully used to improve sepsis-related outcomes in live clinical settings.

TRIAL REGISTRATION NUMBER

NCT03960203.

Interventions for rapid recognition and treatment of sepsis in the emergency department: a narrative review

BACKGROUND

Sepsis is a major cause of morbidity and mortality worldwide. Early recognition and treatment of sepsis is associated with improved outcome. The emergency department (ED) is the department where patients with sepsis seek care. However, recognition of sepsis in the ED remains difficult. Different alert and triage systems, screening scores and intervention strategies have been developed to assist clinicians in early recognition of sepsis and to optimize management.

OBJECTIVES

This narrative review describes currently applied interventions or interventions we can start using today, such as screening scores, (automated) triage systems, sepsis teams and clinical pathways in sepsis care; and it summarizes evidence for the effect of implementation of these interventions in the ED on patient management and outcomes.

SOURCES

A systematic literature search was conducted in PubMed, resulting in 39 eligible studies.

CONTENT

The main sepsis interventions in the ED are (automated) triage systems, sepsis teams and clinical pathways, the most integrative being a clinical pathway. Implementation of any of these interventions in sepsis care will generally lead to increased protocol adherence. Presumably increased adherence to sepsis guidelines and bundles will lead to better patient outcomes, but the level of evidence to support this improvement is low, whereas implementation of interventions is often complex and costly. No studies comparing different interventions were identified. Two essential factors for success of interventions in the ED are obtaining the support from all professionals and providing ongoing education. The vulnerability of these interventions lies in the lack of accurate tools to identify sepsis; diagnosing sepsis ultimately still relies on clinical assessments. A lack of specificity or sepsis alerts may lead to alert fatigue and/or overtreatment.

IMPLICATIONS

The severity and poor outcome of sepsis as well as the frequency of its presentation in EDs make a structured, protocol-based approach towards these patients essential, preferably as part of a clinical pathway.

Sepsis surveillance: an examination of parameter sensitivity and alert reliability

Objective

To examine performance of a sepsis surveillance system in a simulated environment where modifications to parameters and settings for identification of at-risk patients can be explored in-depth.

Materials and Methods

This was a multiple center observational cohort study. The study population comprised 14 917 adults hospitalized in 2016. An expert-driven rules algorithm was applied against 15.1 million data points to simulate a system with binary notification of sepsis events. Three system scenarios were examined: a scenario as derived from the second version of the Consensus Definitions for Sepsis and Septic Shock (SEP-2), the same scenario but without systolic blood pressure (SBP) decrease criteria (near SEP-2), and a conservative scenario with limited parameters. Patients identified by scenarios as being at-risk for sepsis were assessed for suspected infection. Multivariate binary logistic regression models estimated mortality risk among patients with suspected infection.

Results

First, the SEP-2-based scenario had a hyperactive, unreliable parameter SBP decrease >40 mm Hg from baseline. Second, the near SEP-2 scenario demonstrated adequate reliability and sensitivity. Third, the conservative scenario had modestly higher reliability, but sensitivity degraded quickly. Parameters differed in predicting mortality risk and represented a substitution effect between scenarios.

Discussion

Configuration of parameters and alert criteria have implications for patient identification and predicted outcomes.

Conclusion

Performance of scenarios was associated with scenario design. A single hyperactive, unreliable parameter may negatively influence adoption of the system. A trade-off between modest improvements in alert reliability corresponded to a steep decline in condition sensitivity in scenarios explored.

Digital Alerting and Outcomes in Patients With Sepsis: Systematic Review and Meta-Analysis

BACKGROUND

The diagnosis and management of sepsis remain a global health care challenge. Digital technologies have the potential to improve sepsis care.

OBJECTIVE

The aim of this paper was to systematically review the evidence on the impact of digital alerting systems on sepsis related outcomes.

METHODS

The following databases were searched for studies published from April 1964 to February 12, 2019, with no language restriction: EMBASE, MEDLINE, HMIC, PsycINFO, and Cochrane. All full-text reports of studies identified as potentially eligible after title and abstract reviews were obtained for further review. The search was limited to adult inpatients. Relevant articles were hand searched for other studies. Only studies with clear pre- and postalerting phases were included. Primary outcomes were hospital length of stay (LOS) and intensive care LOS, whereas secondary outcomes were time to antibiotics and mortality. Studies based solely on intensive care, case reports, narrative reviews, editorials, and commentaries were excluded. All other trial designs were included. A qualitative assessment and meta-analysis were performed.

RESULTS

This review identified 72 full-text articles. From these, 16 studies met the inclusion criteria and were included in the final analysis. Of these, 8 studies reviewed hospital LOS, 12 reviewed mortality outcomes, 5 studies explored time to antibiotics, and 5 studies investigated intensive care unit (ICU) LOS. Both quantitative and qualitative assessments of the studies were performed. There was evidence of a significant benefit of digital alerting in hospital LOS, which reduced by 1.31 days (P=.014), and ICU LOS, which reduced by 0.766 days (P=.007). There was no significant association between digital alerts and mortality (mean decrease 11.4%; P=.77) or time to antibiotics (mean decrease 126 min; P=.13).

CONCLUSIONS

This review highlights that digital alerts can considerably reduce hospital and ICU stay for patients with sepsis. Further studies including randomized controlled trials are necessary to confirm these findings and identify the choice of alerting system according to the patient status and pathological cohort.

Antibiotic- and Fluid-Focused Bundles Potentially Improve Sepsis Management, but High-Quality Evidence Is Lacking for the Specificity Required in the Centers for Medicare and Medicaid Service's Sepsis Bundle (SEP-1)

OBJECTIVE

To address three controversial components in the Centers for Medicare and Medicaid Service's sepsis bundle for performance measure (SEP-1): antibiotics within 3 hours, a 30 mL/kg fluid infusion for all hypotensive patients, and repeat lactate measurements within 6 hours if initially elevated. We hypothesized that antibiotic- and fluid-focused bundles like SEP-1 would probably show benefit, but evidence supporting specific antibiotic timing, fluid dosing, or serial lactate requirements would not be concordant. Therefore, we performed a meta-analysis of studies of sepsis bundles like SEP-1.

DATA SOURCES

PubMed, Embase, ClinicalTrials.gov through March 15, 2018.

STUDY SELECTION

Studies comparing survival in septic adults receiving versus not receiving antibiotic- and fluid-focused bundles.

DATA EXTRACTION

Two investigators (D.J.P., P.Q.E.).

DATA SYNTHESIS

Seventeen observational studies (11,303 controls and 4,977 bundle subjects) met inclusion criteria. Bundles were associated with increased odds ratios of survival (odds ratio [95% CI]) in 15 studies with substantial heterogeneity (I = 61%; p < 0.01). Survival benefits were consistent in the five largest (1,697-12,486 patients per study) (1.20 [1.11-1.30]; I = 0%) and six medium-sized studies (167-1,029) (2.03 [1.52-2.71]; I = 8%) but not the six smallest (64-137) (1.25 [0.42-3.66]; I = 57%). Bundles were associated with similarly increased survival benefits whether requiring antibiotics within 1 hour (n = 7 studies) versus 3 hours (n = 8) versus no specified time (n = 2); or 30 mL/kg fluid (n = 7) versus another volume (≥ 2 L, n = 1; ≥ 20 mL/kg, n = 2; 1.5-2 L or 500 mL, n = 1 each; none specified, n = 4) (p = 0.19 for each comparison). In the only study employing serial lactate measurements, survival was not increased versus others. No study had a low risk of bias or assessed potential adverse bundle effects.

CONCLUSIONS

Available studies support the notion that antibiotic- and fluid-focused sepsis bundles like SEP-1 improve survival but do not demonstrate the superiority of any specific antibiotic time or fluid volume or of serial lactate measurements. Until strong reproducible evidence demonstrates the safety and benefit of any fixed requirement for these interventions, the present findings support the revision of SEP-1 to allow flexibility in treatment according to physician judgment.

Artificial Intelligence for Surveillance in Public Health

OBJECTIVES

To introduce and summarize current research in the field of Public Health and Epidemiology Informatics.

METHODS

The 2018 literature concerning public health and epidemiology informatics was searched in PubMed and Web of Science, and the returned references were reviewed by the two section editors to select 15 candidate best papers. These papers were then peer-reviewed by external reviewers to give the editorial team an enlightened selection of the best papers.

RESULTS

Among the 805 references retrieved from PubMed and Web of Science, three were finally selected as best papers. All three papers are about surveillance using digital tools. One study is about the surveillance of flu, another about emerging animal infectious diseases and the last one is about foodborne illness. The sources of information are Google news, Twitter, and Yelp restaurant reviews. Machine learning approaches are most often used to detect signals.

CONCLUSIONS

Surveillance is a central topic in public health informatics with the growing use of machine learning approaches in regards of the size and complexity of data. The evaluation of the approaches developed remains a serious challenge.

CADDIE2-evaluation of a clinical decision-support system for early detection of systemic inflammatory response syndrome in paediatric intensive care: study protocol for a diagnostic study

INTRODUCTION

Systemic inflammatory response syndrome (SIRS) is one of the most critical indicators determining the clinical outcome of paediatric intensive care patients. Clinical decision support systems (CDSS) can be designed to support clinicians in detection and treatment. However, the use of such systems is highly discussed as they are often associated with accuracy problems and 'alert fatigue'. We designed a CDSS for detection of paediatric SIRS and hypothesise that a high diagnostic accuracy together with an adequate alerting will accelerate the use. Our study will (1) determine the diagnostic accuracy of the CDSS compared with gold standard decisions created by two blinded, experienced paediatricians, and (2) compare the system's diagnostic accuracy with that of routine clinical care decisions compared with the same gold standard.

METHODS AND ANALYSIS

CADDIE2 is a prospective diagnostic accuracy study taking place at the Department of Pediatric Cardiology and Intensive Care Medicine at the Hannover Medical School; it represents the second step towards our vision of cross-institutional and data-driven decision-support for intensive care environments (CADDIE). The study comprises (1) recruitment of up to 300 patients (start date 1 August 2018), (2) creation of gold standard decisions (start date 1 May 2019), (3) routine SIRS assessments by physicians (starts with recruitment), (4) SIRS assessments by a CDSS (start date 1 May 2019), and (5) statistical analysis with a modified approach for determining sensitivity and specificity and comparing the accuracy results of the different diagnostic approaches (planned start date 1 July 2019).

ETHICS AND DISSEMINATION

Ethics approval was obtained at the study centre (Ethics Committee of Hannover Medical School). Results of the main study will be communicated via publication in a peer-reviewed journal.

TRIAL REGISTRATION NUMBER

ClinicalTrials.gov NCT03661450; Pre-results.

Interruptive Versus Noninterruptive Clinical Decision Support: Usability Study

Background

Clinical decision support (CDS) has been shown to improve compliance with evidence-based care, but its impact is often diminished because of issues such as poor usability, insufficient integration into workflow, and alert fatigue. Noninterruptive CDS may be less subject to alert fatigue, but there has been little assessment of its usability.

Objective

This study aimed to study the usability of interruptive and noninterruptive versions of a CDS.

Methods

We conducted a usability study of a CDS tool that recommended prescribing an angiotensin-converting enzyme inhibitor for inpatients with heart failure. We developed 2 versions of the CDS: an interruptive alert triggered at order entry and a noninterruptive alert listed in the sidebar of the electronic health record screen. Inpatient providers were recruited and randomly assigned to use the interruptive alert followed by the noninterruptive alert or vice versa in a laboratory setting. We asked providers to “think aloud” while using the CDS and then conducted a brief semistructured interview about usability. We used a constant comparative analysis informed by the CDS Five Rights framework to analyze usability testing.

Results

A total of 12 providers participated in usability testing. Providers noted that the interruptive alert was readily noticed but generally impeded workflow. The noninterruptive alert was felt to be less annoying but had lower visibility, which might reduce engagement. Provider role seemed to influence preferences; for instance, some providers who had more global responsibility for patients seemed to prefer the noninterruptive alert, whereas more task-oriented providers generally preferred the interruptive alert.

Conclusions

Providers expressed trade-offs between impeding workflow and improving visibility with interruptive and noninterruptive versions of a CDS. In addition, 2 potential approaches to effective CDS may include targeting alerts by provider role or supplementing a noninterruptive alert with an occasional, well-timed interruptive alert.

Scope and Influence of Electronic Health Record-Integrated Clinical Decision Support in the Emergency Department: A Systematic Review

STUDY OBJECTIVE

As electronic health records evolve, integration of computerized clinical decision support offers the promise of sorting, collecting, and presenting this information to improve patient care. We conducted a systematic review to examine the scope and influence of electronic health record-integrated clinical decision support technologies implemented in the emergency department (ED).

METHODS

A literature search was conducted in 4 databases from their inception through January 18, 2018: PubMed, Scopus, the Cumulative Index of Nursing and Allied Health, and Cochrane Central. Studies were included if they examined the effect of a decision support intervention that was implemented in a comprehensive electronic health record in the ED setting. Standardized data collection forms were developed and used to abstract study information and assess risk of bias.

RESULTS

A total of 2,558 potential studies were identified after removal of duplicates. Of these, 42 met inclusion criteria. Common targets for clinical decision support intervention included medication and radiology ordering practices, as well as more comprehensive systems supporting diagnosis and treatment for specific disease entities. The majority of studies (83%) reported positive effects on outcomes studied. Most studies (76%) used a pre-post experimental design, with only 3 (7%) randomized controlled trials.

CONCLUSION

Numerous studies suggest that clinical decision support interventions are effective in changing physician practice with respect to process outcomes such as guideline adherence; however, many studies are small and poorly controlled. Future studies should consider the inclusion of more specific information in regard to design choices, attempt to improve on uncontrolled before-after designs, and focus on clinically relevant outcomes wherever possible.

Desiderata for sharable computable biomedical knowledge for learning health systems

In this commentary, we work out the specific desired functions required for sharing knowledge objects (based on statistical models) presumably to be used for clinical decision support derived from a learning health system, and, in so doing, discuss the implications for novel knowledge architectures. We will demonstrate how decision models, implemented as influence diagrams, satisfy the desiderata. The desiderata include locally validate discrimination, locally validate calibration, locally recalculate thresholds by incorporating local preferences, provide explanation, enable monitoring, enable debiasing, account for generalizability, account for semantic uncertainty, shall be findable, and others as necessary and proper. We demonstrate how formal decision models, especially when implemented as influence diagrams based on Bayesian networks, support both the knowledge artifact itself (the "primary decision") and the "meta-decision" of whether to deploy the knowledge artifact. We close with a research and development agenda to put this framework into place.

An investigation of sepsis surveillance and emergency treatment on patient mortality outcomes: An observational cohort study

Objective

To determine the prevalence of initiating the sepsis 3-h bundle of care and estimate effects of bundle completion on risk-adjusted mortality among emergency department (ED) patients screened-in by electronic surveillance.

Materials and Methods

This was a multiple center observational cohort study conducted in 2016. The study population was comprised of patients screened-in by St. John Sepsis Surveillance Agent within 4 h of ED arrival, had a sepsis bundle initiated, and admitted to hospital. We built multivariable logistic regression models to estimate impact of a 3-h bundle completed within 3 h of arrival on mortality outcomes.

Results

Approximately 3% ED patients were screened-in by electronic surveillance within 4 h of arrival and admitted to hospital. Nearly 7 in 10 (69%) patients had a bundle initiated, with most bundles completed within 3 h of arrival. The fully-adjusted risk model achieved good discrimination on mortality outcomes [area under the receiver operating characteristic 0.82, 95% confidence interval (CI) 0.79-0.85] and estimated 34% reduced mortality risk among patients with a bundle completed within 3 h of arrival compared to non-completers.

Discussion

The sepsis bundle is an effective intervention for many vulnerable patients, and likely to be completed within 3 h after arrival when electronic surveillance with reliable alert notifications are integrated into clinical workflow. Beginning at triage, the platform and sepsis program enables identification and management of patients with greater precision, and increases the odds of good outcomes.

Conclusion

Sepsis surveillance and clinical decision support accelerate accurate recognition and stratification of patients, and facilitate timely delivery of health care.

Pentraxin-3, procalcitonin and lactate as prognostic markers in patients with sepsis and septic shock

The purpose of this study was to confirm the prognostic value of pentraxin-3 (PTX3), procalcitonin (PCT) and lactate in patients with severe infections requiring ICU management and to develop and validate a model to enhance mortality prediction by combining severity scores with biomarkers. We included 141 patients with the diagnosis of sepsis/septic shock. The levels of PTX3, PCT and lactate were measured on day 0, 3, 7 of hospitalization and Sequential Organ Failure Assessment (SOFA) and Acute Physiology and Chronic Health Evaluation II (APACHE II) scores were also evaluated. The influence of these variables on 28-day mortality was evaluated. The 28-day mortality rate in this study was 28.8%. The baseline levels of PTX3, PCT and lactate in the non-survival group were higher than in the survival group (P < 0.05 for all). Pearson's correlation found that PTX3, PCT and lactate were all positively correlated with SOFA and APACHE II scores (P <0.01 for all). Univariate and multivariate Cox regression revealed that PTX3, PCT and lactate were independently associated with 28-day mortality. The models combining above three biomarkers performed better predictive property than each individual one as determined by receiver operating characteristic (ROC) analysis. In summary, our results suggest that PTX3, PCT and lactate could serve as clinically informative biomarkers of disease severity and patient outcome in sepsis/septic shock. A model combining PTX3, PCT and lactate improves mortality prediction in these patients.