An Electronic Tool for the Evaluation and Treatment of Sepsis in the ICU: A Randomized Controlled Trial

Improving Sepsis Outcomes in the Era of Pay-for-Performance and Electronic Quality Measures: A Joint IDSA/ACEP/PIDS/SHEA/SHM/SIDP Position Paper

The Centers for Medicare & Medicaid Services (CMS) introduced the Severe Sepsis/Septic Shock Management Bundle (SEP-1) as a pay-for-reporting measure in 2015 and is now planning to make it a pay-for-performance measure by incorporating it into the Hospital Value-Based Purchasing Program. This joint IDSA/ACEP/PIDS/SHEA/SHM/SIPD position paper highlights concerns with this change. Multiple studies indicate that SEP-1 implementation was associated with increased broad-spectrum antibiotic use, lactate measurements, and aggressive fluid resuscitation for patients with suspected sepsis but not with decreased mortality rates. Increased focus on SEP-1 risks further diverting attention and resources from more effective measures and comprehensive sepsis care. We recommend retiring SEP-1 rather than using it in a payment model and shifting instead to new sepsis metrics that focus on patient outcomes. CMS is developing a community-onset sepsis 30-day mortality electronic clinical quality measure (eCQM) that is an important step in this direction. The eCQM preliminarily identifies sepsis using systemic inflammatory response syndrome (SIRS) criteria, antibiotic administrations or diagnosis codes for infection or sepsis, and clinical indicators of acute organ dysfunction. We support the eCQM but recommend removing SIRS criteria and diagnosis codes to streamline implementation, decrease variability between hospitals, maintain vigilance for patients with sepsis but without SIRS, and avoid promoting antibiotic use in uninfected patients with SIRS. We further advocate for CMS to harmonize the eCQM with the Centers for Disease Control and Prevention's (CDC) Adult Sepsis Event surveillance metric to promote unity in federal measures, decrease reporting burden for hospitals, and facilitate shared prevention initiatives. These steps will result in a more robust measure that will encourage hospitals to pay more attention to the full breadth of sepsis care, stimulate new innovations in diagnosis and treatment, and ultimately bring us closer to our shared goal of improving outcomes for patients.

Chinese experts' consensus on the application of intensive care big data

The development of intensive care medicine is inseparable from the diversified monitoring data. Intensive care medicine has been closely integrated with data since its birth. Critical care research requires an integrative approach that embraces the complexity of critical illness and the computational technology and algorithms that can make it possible. Considering the need of standardization of application of big data in intensive care, Intensive Care Medicine Branch of China Health Information and Health Care Big Data Society, Standard Committee has convened expert group, secretary group and the external audit expert group to formulate Chinese Experts' Consensus on the Application of Intensive Care Big Data (2022). This consensus makes 29 recommendations on the following five parts: Concept of intensive care big data, Important scientific issues, Standards and principles of database, Methodology in solving big data problems, Clinical application and safety consideration of intensive care big data. The consensus group believes this consensus is the starting step of application big data in the field of intensive care. More explorations and big data based retrospective research should be carried out in order to enhance safety and reliability of big data based models of critical care field.

Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review

OBJECTIVE

Surveillance algorithms that predict patient decompensation are increasingly integrated with clinical workflows to help identify patients at risk of in-hospital deterioration. This scoping review aimed to identify the design features of the information displays, the types of algorithm that drive the display, and the effect of these displays on process and patient outcomes.

MATERIALS AND METHODS

The scoping review followed Arksey and O'Malley's framework. Five databases were searched with dates between January 1, 2009 and January 26, 2022. Inclusion criteria were: participants-clinicians in inpatient settings; concepts-intervention as deterioration information displays that leveraged automated AI algorithms; comparison as usual care or alternative displays; outcomes as clinical, workflow process, and usability outcomes; and context as simulated or real-world in-hospital settings in any country. Screening, full-text review, and data extraction were reviewed independently by 2 researchers in each step. Display categories were identified inductively through consensus.

RESULTS

Of 14 575 articles, 64 were included in the review, describing 61 unique displays. Forty-one displays were designed for specific deteriorations (eg, sepsis), 24 provided simple alerts (ie, text-based prompts without relevant patient data), 48 leveraged well-accepted score-based algorithms, and 47 included nurses as the target users. Only 1 out of the 10 randomized controlled trials reported a significant effect on the primary outcome.

CONCLUSIONS

Despite significant advancements in surveillance algorithms, most information displays continue to leverage well-understood, well-accepted score-based algorithms. Users' trust, algorithmic transparency, and workflow integration are significant hurdles to adopting new algorithms into effective decision support tools.

A sepsis treatment algorithm to improve early antibiotic de-escalation while maintaining adequacy of coverage (Early-IDEAS): A prospective observational study

BACKGROUND

Empiric antibiotic treatment selection should provide adequate coverage for potential pathogens while minimizing unnecessary broad-spectrum antibiotic use. We sought to pilot a sepsis treatment algorithm to individualize antibiotic recommendations, and thereby improve early antibiotic de-escalation while maintaining adequacy of coverage (Early-IDEAS).

METHODS

In this observational study, the Early-IDEAS decision support algorithm was derived from previous Gram- negative and Gram-positive prediction rules and models along with local guidelines, and then applied to prospectively identified consecutive adults within 24 hours of suspected sepsis. The primary outcome was the proportion of patients for whom de-escalation of the primary antibiotic regimen was recommended by the algorithm. Secondary outcomes included: (1) proportion of patients for whom escalation was recommended; (2) number of recommended de-escalation steps along a pre-specified antibiotic cascade; and (3) adequacy of therapy in patients with culture-confirmed infection.

RESULTS

We screened 578 patients, of whom 107 eligible patients were included. The Early-IDEAS treatment recommendation was informed by Gram-negative models in 76 (71%) patients, Gram-positive rules in 64 (59.8%), and local guidelines in 27 (25.2%). Antibiotic de-escalation was recommended in almost half of all patients (n = 52, 48.6%), with a median of 2 steps down the a priori antibiotic treatment cascade. No treatment change was recommended in 45 patients (42.1%), and escalation was recommended in 10 (9.3%). Among the 17 patients with positive blood cultures, both the clinician prescribed regimen and the algorithm recommendation provided adequate coverage for the isolated pathogen in 12 patients (70.6%), (p = 1). Among the 25 patients with positive relevant, non-blood cultures, both the clinician prescribed regimen and the algorithm recommendation provided adequate coverage in 20 (80%), (p = 1).

CONCLUSION

An individualized decision support algorithm in early sepsis could lead to substantial antibiotic de-escalation without compromising adequate antibiotic coverage.

Patient Information Summarization in Clinical Settings: Scoping Review

BACKGROUND

Information overflow, a common problem in the present clinical environment, can be mitigated by summarizing clinical data. Although there are several solutions for clinical summarization, there is a lack of a complete overview of the research relevant to this field.

OBJECTIVE

This study aims to identify state-of-the-art solutions for clinical summarization, to analyze their capabilities, and to identify their properties.

METHODS

A scoping review of articles published between 2005 and 2022 was conducted. With a clinical focus, PubMed and Web of Science were queried to find an initial set of reports, later extended by articles found through a chain of citations. The included reports were analyzed to answer the questions of where, what, and how medical information is summarized; whether summarization conserves temporality, uncertainty, and medical pertinence; and how the propositions are evaluated and deployed. To answer how information is summarized, methods were compared through a new framework "collect-synthesize-communicate" referring to information gathering from data, its synthesis, and communication to the end user.

RESULTS

Overall, 128 articles were included, representing various medical fields. Exclusively structured data were used as input in 46.1% (59/128) of papers, text in 41.4% (53/128) of articles, and both in 10.2% (13/128) of papers. Using the proposed framework, 42.2% (54/128) of the records contributed to information collection, 27.3% (35/128) contributed to information synthesis, and 46.1% (59/128) presented solutions for summary communication. Numerous summarization approaches have been presented, including extractive (n=13) and abstractive summarization (n=19); topic modeling (n=5); summary specification (n=11); concept and relation extraction (n=30); visual design considerations (n=59); and complete pipelines (n=7) using information extraction, synthesis, and communication. Graphical displays (n=53), short texts (n=41), static reports (n=7), and problem-oriented views (n=7) were the most common types in terms of summary communication. Although temporality and uncertainty information were usually not conserved in most studies (74/128, 57.8% and 113/128, 88.3%, respectively), some studies presented solutions to treat this information. Overall, 115 (89.8%) articles showed results of an evaluation, and methods included evaluations with human participants (median 15, IQR 24 participants): measurements in experiments with human participants (n=31), real situations (n=8), and usability studies (n=28). Methods without human involvement included intrinsic evaluation (n=24), performance on a proxy (n=10), or domain-specific tasks (n=11). Overall, 11 (8.6%) reports described a system deployed in clinical settings.

CONCLUSIONS

The scientific literature contains many propositions for summarizing patient information but reports very few comparisons of these proposals. This work proposes to compare these algorithms through how they conserve essential aspects of clinical information and through the "collect-synthesize-communicate" framework. We found that current propositions usually address these 3 steps only partially. Moreover, they conserve and use temporality, uncertainty, and pertinent medical aspects to varying extents, and solutions are often preliminary.

Machine learning-based prediction of low-value care for hospitalized patients

Objective

Low-value care (i.e., costly health care treatments that provide little or no benefit) is an ongoing problem in United States hospitals. Traditional strategies for reducing low-value care are only moderately successful. Informed by behavioral science principles, we sought to use machine learning to inform a targeted prompting system that suggests preferred alternative treatments at the point of care but before clinicians have made a decision.

Methods

We used intravenous administration of albumin for fluid resuscitation in intensive care unit (ICU) patients as an exemplar of low-value care practice, identified using the electronic health record of a multi-hospital health system. We divided all ICU episodes into 4-h periods and defined a set of relevant clinical features at the period level. We then developed two machine learning models: a single-stage model that directly predicts if a patient will receive albumin in the next period; and a two-stage model that first predicts if any resuscitation fluid will be administered and then predicts albumin only among the patients with a high probability of fluid use.

Results

We examined 87,489 ICU episodes divided into approximately 1.5 million 4-h periods. The area under the receiver operating characteristic curve was 0.86 for both prediction models. The positive predictive value was 0.21 (95% confidence interval: 0.20, 0.23) for the single-stage model and 0.22 (0.20, 0.23) for the two-stage model. Applying either model in a targeted prompting system could prevent 10% of albumin administrations, with an attending physician receiving one prompt every 4.2 days of ICU service.

Conclusion

Prediction of low-value care is feasible and could enable a point-of-care, targeted prompting system that offers suggestions ahead of the moment of need before clinicians have already decided. A two-stage approach does not improve performance but does interject new levers for the calibration of such a system.

The Role of Data Science in Closing the Implementation Gap

Data science has the potential to greatly enhance efforts to translate evidence into practice in critical care. The intensive care unit is a data-rich environment enabling insight into both patient-level care patterns and clinician-level treatment patterns. By applying artificial intelligence to these novel data sources, implementation strategies can be tailored to individual patients, individual clinicians, and individual situations, revealing when evidence-based practices are missed and facilitating context-sensitive clinical decision support. To achieve these goals, technology developers should work closely with clinicians to create unbiased applications that are integrated into the clinical workflow.

The Learning Electronic Health Record

Electronic medical records (EMRs) constitute the electronic version of all medical information included in a patient's paper chart. The electronic health record (EHR) technology has witnessed massive expansion in developed countries and to a lesser extent in underresourced countries during the last 2 decades. We will review factors leading to this expansion, how the emergence of EHRs is affecting several health-care stakeholders; some of the growing pains associated with EHRs with a particular emphasis on the delivery of care to the critically ill; and ongoing developments on the path to improve the quality of research, health-care delivery, and stakeholder satisfaction.

Effectiveness of automated alerting system compared to usual care for the management of sepsis

There is a large body of evidence showing that delayed initiation of sepsis bundle is associated with adverse clinical outcomes in patients with sepsis. However, it is controversial whether electronic automated alerts can help improve clinical outcomes of sepsis. Electronic databases are searched from inception to December 2021 for comparative effectiveness studies comparing automated alerts versus usual care for the management of sepsis. A total of 36 studies are eligible for analysis, including 6 randomized controlled trials and 30 non-randomized studies. There is significant heterogeneity in these studies concerning the study setting, design, and alerting methods. The Bayesian meta-analysis by using pooled effects of non-randomized studies as priors shows a beneficial effect of the alerting system (relative risk [RR]: 0.71; 95% credible interval: 0.62 to 0.81) in reducing mortality. The automated alerting system shows less beneficial effects in the intensive care unit (RR: 0.90; 95% CI: 0.73–1.11) than that in the emergency department (RR: 0.68; 95% CI: 0.51–0.90) and ward (RR: 0.71; 95% CI: 0.61–0.82). Furthermore, machine learning-based prediction methods can reduce mortality by a larger magnitude (RR: 0.56; 95% CI: 0.39–0.80) than rule-based methods (RR: 0.73; 95% CI: 0.63–0.85). The study shows a statistically significant beneficial effect of using the automated alerting system in the management of sepsis. Interestingly, machine learning monitoring systems coupled with better early interventions show promise, especially for patients outside of the intensive care unit.

Effectiveness of clinical dashboards as audit and feedback or clinical decision support tools on medication use and test ordering: a systematic review of randomized controlled trials

BACKGROUND

Clinical dashboards used as audit and feedback (A&F) or clinical decision support systems (CDSS) are increasingly adopted in healthcare. However, their effectiveness in changing the behavior of clinicians or patients is still unclear. This systematic review aims to investigate the effectiveness of clinical dashboards used as CDSS or A&F tools (as a standalone intervention or part of a multifaceted intervention) in primary care or hospital settings on medication prescription/adherence and test ordering.

METHODS

Seven major databases were searched for relevant studies, from inception to August 2021. Two authors independently extracted data, assessed the risk of bias using the Cochrane RoB II scale, and evaluated the certainty of evidence using GRADE. Data on trial characteristics and intervention effect sizes were extracted. A narrative synthesis was performed to summarize the findings of the included trials.

RESULTS

Eleven randomized trials were included. Eight trials evaluated clinical dashboards as standalone interventions and provided conflicting evidence on changes in antibiotic prescribing and no effects on statin prescribing compared to usual care. Dashboards increased medication adherence in patients with inflammatory arthritis but not in kidney transplant recipients. Three trials investigated dashboards as part of multicomponent interventions revealing decreased use of opioids for low back pain, increased proportion of patients receiving cardiovascular risk screening, and reduced antibiotic prescribing for upper respiratory tract infections.

CONCLUSION

There is limited evidence that dashboards integrated into electronic medical record systems and used as feedback or decision support tools may be associated with improvements in medication use and test ordering.

The impact of clinical decision support systems on provider behavior in the inpatient setting: A systematic review and meta-analysis

BACKGROUND

Clinical decision support systems (CDSS) are used to improve processes of care. CDSS proliferation may have unintended consequences impacting effectiveness.

OBJECTIVE

To evaluate the effectiveness of CDSS in altering clinician behavior.

DESIGN

Electronic searches were performed in EMBASE, PubMed, and Cochrane Central Register of Control Trials for randomized controlled trials testing the impacted of CDSS on clinician behavior from 2000-2021. Extracted data included study design, CDSS attributed and outcomes, user characteristics, settings, and risk of bias. Eligible studies were analyzed qualitatively to describe CDSS types. Studies with sufficient outcome data were included in the meta-analysis.

SETTING AND PARTICIPANTS

Adult inpatients in the United States.

INTERVENTION

Clinical decision support system versus non-clinical decision support system.

MAIN OUTCOME AND MEASURE

A random-effects model measured the pooled risk difference (RD) and odds ratio of clinicians' adherence to CDSS; subgroup analyses tested differences in CDSS effectiveness over time and by CDSS type.

RESULTS

Qualitative synthesis included 22 studies. Eleven studies reported sufficient outcome data for inclusion in the meta-analysis. CDSS did not result in a statistically significant increase in clinician adoption of desired practicies (RD = 0.04 [95% confidence interval {CI} 0.00, 0.07]). CDSS from 2010-2015 (n = 5) did not increase clinician adoption of desired practice [RD -0.01, (95% CI -0.04, 0.02)].CDSS from 2016-2021 (n = 6) were associated with an increase in targeted practices [RD 0.07 (95% CI0.03, 0.12)], pInteraction = 0.004. EHR [RD 0.04 (95% CI 0.00, 0.08)] vs. non-EHR [RD 0.01 (95% CI -0.01, 0.04)] based CDSS interventions did not result in different adoption of desired practices (pInteraction = 0.27). The meta-analysis did not find an overall positive impact of CDSS on clinician behavior in the inpatient setting.

Implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: a qualitative study

Objective

Many options are currently available for sepsis surveillance clinical decision support (CDS) from electronic medical record (EMR) vendors, third party, and homegrown models drawing on rule-based (RB) and machine learning (ML) algorithms. This study explores sepsis CDS implementation from the perspective of implementation leads by describing the motivations, tool choices, and implementation experiences of a diverse group of implementers.

Materials and Methods

Semi-structured interviews were conducted with and a questionnaire was administered to 21 hospital leaders overseeing CDS implementation at 15 US medical centers. Participants were recruited via convenience sampling. Responses were coded by 2 coders with consensus approach and inductively analyzed for themes.

Results

Use of sepsis CDS is motivated in part by quality metrics for sepsis patients. Choice of tool is driven by ease of integration, customization capability, and perceived predictive potential. Implementation processes for these CDS tools are complex, time-consuming, interdisciplinary undertakings resulting in heterogeneous choice of tools and workflow integration. To improve clinician acceptance, implementers addressed both optimization of the alerts as well as clinician understanding and buy in. More distrust and confusion was reported for ML models, as compared to RB models. Respondents described a variety of approaches to overcome implementation barriers; these approaches related to alert firing, content, integration, and buy-in.

Discussion

While there are shared socio-technical challenges of implementing CDS for both RB and ML models, attention to user education, support, expectation management, and dissemination of effective practices may improve feasibility and effectiveness of ML models in quality improvement efforts.

Conclusion

Further implementation science research is needed to determine real world efficacy of these tools. Clinician acceptance is a significant barrier to sepsis CDS implementation. Successful implementation of less clinically intuitive ML models may require additional attention to user confusion and distrust.

Sepsis is a life-threatening illness. Improving sepsis care is a growing priority for many hospitals. Patients at risk of developing sepsis can be identified before they get very sick using tools that analyze data from computerized medical records systems. A variety of options are available from different sources. Some tools are programmed using established sepsis screening criteria used in clinical practice. Others rely on machine learning, where computer algorithms identify patterns in the available data without being pre-programmed by a human being. In this study, we interviewed 21 individuals at 15 US medical centers who oversaw hospital level implementations of these tools. Teams were motivated by wanting to improve quality of care for patients with sepsis. One major challenge was making the tools identify as many patients truly at risk for sepsis as possible while limiting false identification of patients not actually at risk. Many interviewees also described lack of trust in the tools from the nurses and doctors using the tools. There was more distrust and confusion reported by implementers of tools that relied on machine learning than tools that programmed human logic. Strategies emphasizing user education, user support, and expectation management were reported to be helpful.

A qualitative study of clinician perspectives and desired characteristics of a clinical decision support system for early sepsis recognition (Preprint)

Background

Sepsis is a major burden for health care systems in the United States, with over 750,000 cases annually and a total cost of approximately US $20 billion. The hallmark of sepsis treatment is early and appropriate initiation of antibiotic therapy. Although sepsis clinical decision support (CDS) systems can provide clinicians with early predictions of suspected sepsis or imminent clinical decline, such systems have not reliably demonstrated improvements in clinical outcomes or care processes. Growing evidence suggests that the challenges of integrating sepsis CDS systems into clinical workflows, gaining the trust of clinicians, and making sepsis CDS systems clinically relevant at the bedside are all obstacles to successful deployment. However, there are significant knowledge gaps regarding the achievement of these implementation and deployment goals.

Objective

We aimed to identify perceptions of predictive information in sepsis CDS systems based on clinicians’ past experiences, explore clinicians’ perceptions of a hypothetical sepsis CDS system, and identify the characteristics of a CDS system that would be helpful in promoting timely recognition and management of suspected sepsis in a multidisciplinary, team-based clinical setting.

Methods

We conducted semistructured interviews with practicing bedside nurses, advanced practice providers, and physicians at a large academic medical center between September 2020 and March 2021. We used modified human factor methods (contextual interview and cognitive walkthrough performed over video calls because of the COVID-19 pandemic) and conducted a thematic analysis using an abductive approach for coding to identify important patterns and concepts in the interview transcripts.

Results

We interviewed 6 bedside nurses and 9 clinicians responsible for ordering antibiotics (advanced practice providers or physicians) who had a median of 4 (IQR 4-6.5) years of experience working in an inpatient setting. We then synthesized critical content from the thematic analysis of the data into four domains: clinician perceptions of prediction models and alerts; previous experiences of clinician encounters with predictive information and risk scores; desired characteristics of a CDS system build, including predictions, supporting information, and delivery methods for a potential alert; and the clinical relevance and potential utility of a CDS system. These 4 domains were strongly linked to clinicians’ perceptions of the likelihood of adoption and the impact on clinical workflows when diagnosing and managing patients with suspected sepsis. Ultimately, clinicians desired a trusted and actionable CDS system to improve sepsis care.

Conclusions

Building a trusted and actionable sepsis CDS alert is paramount to achieving acceptability and use among clinicians. These findings can inform the development, implementation, and deployment strategies for CDS systems that support the early detection and treatment of sepsis. This study also highlights several key opportunities when eliciting clinician input before the development and deployment of prediction models.

Applying artificial neural network for early detection of sepsis with intentionally preserved highly missing real-world data for simulating clinical situation

Purpose

Some predictive systems using machine learning models have been developed to predict sepsis; however, they were mostly built with a low percent of missing values, which does not correspond with the actual clinical situation. In this study, we developed a machine learning model with a high rate of missing and erroneous data to enable prediction under missing, noisy, and erroneous inputs, as in the actual clinical situation.

Materials and methods

The proposed artificial neural network model was implemented using the MATLAB ANN toolbox, based on stochastic gradient descent. The dataset was collected over the past decade with approval from the appropriate institutional review boards, and the sepsis status was identified and labeled using Sepsis-3 clinical criteria. The imputation method was built by last observation carried forward and mean value, aimed to simulate clinical situation.

Results

The mean area under the receiver operating characteristic (ROC) curve (AUC) of classifying sepsis and nonsepsis patients was 0.82 and 0.786 at 0 h and 40 h prior to onset, respectively. The highest model performance was found for one-hourly data, demonstrating that our ANN model can perform adequately with limited hourly data provided.

Conclusions

Our model has the moderate ability to predict sepsis up to 40 h in advance under simulated clinical situation with real-world data.

Electronic early notification of sepsis in hospitalized ward patients: a study protocol for a stepped-wedge cluster randomized controlled trial

Background

To evaluate the effect of screening for sepsis using an electronic sepsis alert vs. no alert in hospitalized ward patients on 90-day in-hospital mortality.

Methods

The SCREEN trial is designed as a stepped-wedge cluster randomized controlled trial. Hospital wards (total of 45 wards, constituting clusters in this design) are randomized to have active alert vs. masked alert, 5 wards at a time, with each 5 wards constituting a sequence. The study consists of ten 2-month periods with a phased introduction of the intervention. In the first period, all wards have a masked alert for 2 months. Afterwards the intervention (alert system) is implemented in a new sequence every 2-month period until the intervention is implemented in all sequences. The intervention includes the implementation of an electronic alert system developed in the hospital electronic medical records based on the quick sequential organ failure assessment (qSOFA). The alert system sends notifications of “possible sepsis alert” to the bedside nurse, charge nurse, and primary medical team and requires an acknowledgment in the health information system from the bedside nurse and physician. The calculated sample size is 65,250. The primary endpoint is in-hospital mortality by 90 days.

Discussion

The trial started on October 1, 2019, and is expected to complete patient follow-up by the end of October 2021.

Trial registration

ClinicalTrials.gov NCT04078594. Registered on September 6, 2019

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05562-5.

External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients

IMPORTANCE

The Epic Sepsis Model (ESM), a proprietary sepsis prediction model, is implemented at hundreds of US hospitals. The ESM's ability to identify patients with sepsis has not been adequately evaluated despite widespread use.

OBJECTIVE

To externally validate the ESM in the prediction of sepsis and evaluate its potential clinical value compared with usual care.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study was conducted among 27 697 patients aged 18 years or older admitted to Michigan Medicine, the academic health system of the University of Michigan, Ann Arbor, with 38 455 hospitalizations between December 6, 2018, and October 20, 2019.

EXPOSURE

The ESM score, calculated every 15 minutes.

MAIN OUTCOMES AND MEASURES

Sepsis, as defined by a composite of (1) the Centers for Disease Control and Prevention surveillance criteria and (2) International Statistical Classification of Diseases and Related Health Problems, Tenth Revision diagnostic codes accompanied by 2 systemic inflammatory response syndrome criteria and 1 organ dysfunction criterion within 6 hours of one another. Model discrimination was assessed using the area under the receiver operating characteristic curve at the hospitalization level and with prediction horizons of 4, 8, 12, and 24 hours. Model calibration was evaluated with calibration plots. The potential clinical benefit associated with the ESM was assessed by evaluating the added benefit of the ESM score compared with contemporary clinical practice (based on timely administration of antibiotics). Alert fatigue was evaluated by comparing the clinical value of different alerting strategies.

RESULTS

We identified 27 697 patients who had 38 455 hospitalizations (21 904 women [57%]; median age, 56 years [interquartile range, 35-69 years]) meeting inclusion criteria, of whom sepsis occurred in 2552 (7%). The ESM had a hospitalization-level area under the receiver operating characteristic curve of 0.63 (95% CI, 0.62-0.64). The ESM identified 183 of 2552 patients with sepsis (7%) who did not receive timely administration of antibiotics, highlighting the low sensitivity of the ESM in comparison with contemporary clinical practice. The ESM also did not identify 1709 patients with sepsis (67%) despite generating alerts for an ESM score of 6 or higher for 6971 of all 38 455 hospitalized patients (18%), thus creating a large burden of alert fatigue.

CONCLUSIONS AND RELEVANCE

This external validation cohort study suggests that the ESM has poor discrimination and calibration in predicting the onset of sepsis. The widespread adoption of the ESM despite its poor performance raises fundamental concerns about sepsis management on a national level.

The number needed to benefit: estimating the value of predictive analytics in healthcare

Predictive analytics in health care has generated increasing enthusiasm recently, as reflected in a rapidly growing body of predictive models reported in literature and in real-time embedded models using electronic health record data. However, estimating the benefit of applying any single model to a specific clinical problem remains challenging today. Developing a shared framework for estimating model value is therefore critical to facilitate the effective, safe, and sustainable use of predictive tools into the future. We highlight key concepts within the prediction-action dyad that together are expected to impact model benefit. These include factors relevant to model prediction (including the number needed to screen) as well as those relevant to the subsequent action (number needed to treat). In the simplest terms, a number needed to benefit contextualizes the numbers needed to screen and treat, offering an opportunity to estimate the value of a clinical predictive model in action.

Development and Validation of a Model to Predict Pediatric Septic Shock Using Data Known 2 Hours After Hospital Arrival

Objective:

To use Electronic Health Record (EHR) data from the first two hours of care to derive and validate a model to predict hypotensive septic shock in children with infection.

Design:

Derivation-validation study using an existing registry

Setting:

Six emergency care sites within a regional pediatric healthcare system. Three datasets of unique visits were designated:

Patients:

Patients in whom clinicians were concerned about serious infection from 60 days-17 years were included; those with septic shock in the first two hours were excluded. There were 2318 included visits; 197 developed septic shock (8.5%).

Interventions:

Lasso with tenfold cross-validation was used for variable selection; logistic regression was then used to construct a model from those variables in the training set. Variables were derived from EHR data known in the first two hours, including vital signs, medical history, demographics, laboratory information. Test characteristics at two thresholds were evaluated: 1) optimizing sensitivity and specificity, 2) set to 90% sensitivity.

Measurements and Main Results:

Septic shock was defined as systolic hypotension and vasoactive use or ≥30 ml/kg isotonic crystalloid administration in the first 24 hours. A model was created using twenty predictors, with an area under the receiver operating curve in the training set of 0.85 (0.82-0.88); 0.83 [0.78-0.89] in the temporal test set; 0.83 [0.60-1.00] in the geographic test set. Sensitivity and specificity varied based on cutpoint; when sensitivity in the training set was set to 90% (83%, 94%), specificity was 62% (60%, 65%).

Conclusions:

This model predicted risk of septic shock in children with suspected infection 2 hours after arrival, a critical timepoint for emergent treatment and transfer decisions. Varied cutpoints could be used to customize sensitivity to clinical context.

Coronavirus Disease 2019 Calls for Predictive Analytics Monitoring-A New Kind of Illness Scoring System

Coronavirus disease 2019 can lead to sudden and severe respiratory failure that mandates endotracheal intubation, a procedure much more safely performed under elective rather than emergency conditions. Early warning of rising risk of this event could benefit both patients and healthcare providers by reducing the high risk of emergency intubation. Current illness severity scoring systems, which usually update only when clinicians measure vital signs or laboratory values, are poorly suited for early detection of this kind of rapid clinical deterioration. We propose that continuous predictive analytics monitoring, a new approach to bedside management, is more useful. The principles of this new practice anchor in analysis of continuous bedside monitoring data, training models on diagnosis-specific paths of deterioration using clinician-identified events, and continuous display of trends in risks rather than alerts when arbitrary thresholds are exceeded.

Effects of Computerized Decision Support Systems on Practitioner Performance and Patient Outcomes: Systematic Review

Background

Computerized decision support systems (CDSSs) are software programs that support the decision making of practitioners and other staff. Other reviews have analyzed the relationship between CDSSs, practitioner performance, and patient outcomes. These reviews reported positive practitioner performance in over half the articles analyzed, but very little information was found for patient outcomes.

Objective

The purpose of this review was to analyze the relationship between CDSSs, practitioner performance, and patient medical outcomes. PubMed, CINAHL, Embase, Web of Science, and Cochrane databases were queried.

Methods

Articles were chosen based on year published (last 10 years), high quality, peer-reviewed sources, and discussion of the relationship between the use of CDSS as an intervention and links to practitioner performance or patient outcomes. Reviewers used an Excel spreadsheet (Microsoft Corporation) to collect information on the relationship between CDSSs and practitioner performance or patient outcomes. Reviewers also collected observations of participants, intervention, comparison with control group, outcomes, and study design (PICOS) along with those showing implicit bias. Articles were analyzed by multiple reviewers following the Kruse protocol for systematic reviews. Data were organized into multiple tables for analysis and reporting.

Results

Themes were identified for both practitioner performance (n=38) and medical outcomes (n=36). A total of 66% (25/38) of articles had occurrences of positive practitioner performance, 13% (5/38) found no difference in practitioner performance, and 21% (8/38) did not report or discuss practitioner performance. Zero articles reported negative practitioner performance. A total of 61% (22/36) of articles had occurrences of positive patient medical outcomes, 8% (3/36) found no statistically significant difference in medical outcomes between intervention and control groups, and 31% (11/36) did not report or discuss medical outcomes. Zero articles found negative patient medical outcomes attributed to using CDSSs.

Conclusions

Results of this review are commensurate with previous reviews with similar objectives, but unlike these reviews we found a high level of reporting of positive effects on patient medical outcomes.

Design, Implementation, and Validation of a Pediatric ICU Sepsis Prediction Tool as Clinical Decision Support

BACKGROUND

Sepsis is an uncontrolled inflammatory reaction caused by infection. Clinicians in the pediatric intensive care unit (PICU) developed a paper-based tool to identify patients at risk of sepsis. To improve the utilization of the tool, the PICU team integrated the paper-based tool as a real-time clinical decision support (CDS) intervention in the electronic health record (EHR).

OBJECTIVE

This study aimed to improve identification of PICU patients with sepsis through an automated EHR-based CDS intervention.

METHODS

A prospective cohort study of all patients admitted to the PICU from May 2017 to May 2019. A CDS intervention was implemented in May 2018. The CDS intervention screened patients for nonspecific sepsis criteria, temperature dysregulation and a blood culture within 6 hours. Following the screening, an interruptive alert prompted nursing staff to complete a perfusion screen to assess for clinical signs of sepsis. The primary alert performance outcomes included sensitivity, specificity, and positive and negative predictive value. The secondary clinical outcome was completion of sepsis management tasks.

RESULTS

During the 1-year post implementation period, there were 45.0 sepsis events per 1,000 patient days over 10,805 patient days. The sepsis alert identified 392 of the 436 sepsis episodes accurately with sensitivity of 92.5%, specificity of 95.6%, positive predictive value of 46.0%, and negative predictive value of 99.7%. Examining only patients with severe sepsis confirmed by chart review, test characteristics fell to a sensitivity of 73.3%, a specificity of 92.5%. Prior to the initiation of the alert, 18.6% (13/70) of severe sepsis patients received recommended sepsis interventions. Following the implementation, 34% (27/80) received these interventions in the time recommended, p = 0.04.

CONCLUSION

An EHR CDS intervention demonstrated strong performance characteristics and improved completion of recommended sepsis interventions.

Identifying Clinical Research Priorities in Adult Pulmonary and Critical Care: NHLBI Working Group Report

Preventing, treating, and promoting recovery from critical illness due to pulmonary disease are foundational goals of the critical care community and the National Heart, Lung, and Blood Institute. Decades of clinical research in acute respiratory distress syndrome, acute respiratory failure, pneumonia, and sepsis have yielded improvements in supportive care, which have translated into improved patient outcomes. Novel therapeutics have largely failed to translate from promising pre-clinical findings into improved patient outcomes in late-phase clinical trials. Recent advances in personalized medicine, "big data", causal inference using observational data, novel clinical trial designs, pre-clinical disease modeling, and understanding recovery from acute illness promise to transform the methods of pulmonary and critical care clinical research. To assess the current state, research priorities, and future directions for adult pulmonary and critical care research, the NHLBI assembled a multidisciplinary working group of investigators. This working group identified recommendations for future research, including: (1) focusing on understanding the clinical, physiological, and biological underpinnings of heterogeneity in syndromes, diseases, and treatment-response with the goal of developing targeted, personalized interventions; (2) optimizing pre-clinical models by incorporating comorbidities, co-interventions, and organ support; (3) developing and applying novel clinical trial designs; and (4) advancing mechanistic understanding of injury and recovery in order to develop and test interventions targeted at achieving long-term improvements in the lives of patients and families. Specific areas of research are highlighted as especially promising for making advances in pneumonia, acute hypoxemic respiratory failure, and acute respiratory distress syndrome.

Novel displays of patient information in critical care settings: a systematic review

OBJECTIVE

Clinician information overload is prevalent in critical care settings. Improved visualization of patient information may help clinicians cope with information overload, increase efficiency, and improve quality. We compared the effect of information display interventions with usual care on patient care outcomes.

MATERIALS AND METHODS

We conducted a systematic review including experimental and quasi-experimental studies of information display interventions conducted in critical care and anesthesiology settings. Citations from January 1990 to June 2018 were searched in PubMed and IEEE Xplore. Reviewers worked independently to screen articles, evaluate quality, and abstract primary outcomes and display features.

RESULTS

Of 6742 studies identified, 22 studies evaluating 17 information displays met the study inclusion criteria. Information display categories included comprehensive integrated displays (3 displays), multipatient dashboards (7 displays), physiologic and laboratory monitoring (5 displays), and expert systems (2 displays). Significant improvement on primary outcomes over usual care was reported in 12 studies for 9 unique displays. Improvement was found mostly with comprehensive integrated displays (4 of 6 studies) and multipatient dashboards (5 of 7 studies). Only 1 of 5 randomized controlled trials had a positive effect in the primary outcome.

CONCLUSION

We found weak evidence suggesting comprehensive integrated displays improve provider efficiency and process outcomes, and multipatient dashboards improve compliance with care protocols and patient outcomes. Randomized controlled trials of physiologic and laboratory monitoring displays did not show improvement in primary outcomes, despite positive results in simulated settings. Important research translation gaps from laboratory to actual critical care settings exist.

A Data-Driven Approach to Predicting Septic Shock in the Intensive Care Unit

Early diagnosis of sepsis and septic shock has been unambiguously linked to lower mortality and better patient outcomes. Despite this, there is a strong unmet need for a reliable clinical tool that can be used for large-scale automated screening to identify high-risk patients. We addressed the following questions: Can a novel algorithm to identify patients at high risk of septic shock 24 hours before diagnosis be discovered using available clinical data? What are performance characteristics of this predictive algorithm? Can current metrics for evaluation of sepsis be improved using novel algorithm? Publicly available data from the intensive care unit setting was used to build septic shock and control patient cohorts. Using Bayesian networks, causal relationships between diagnosis groups, procedure groups, laboratory results, and demographic data were inferred. Predictive model for septic shock 24 hours prior to digital diagnosis was built based on inferred causal networks. Sepsis risk scores were augmented by de novo inferred model and performance was evaluated. A novel predictive model to identify high-risk patients 24 hours ahead of time, with area under curve of 0.81, negative predictive value of 0.87, and a positive predictive value as high as 0.65 was built. The specificity of quick sequential organ failure assessment, systemic inflammatory response syndrome, and modified early warning score was improved when augmented with the novel model, whereas no improvements were made to the sequential organ failure assessment score. We used a data-driven, expert knowledge agnostic method to build a screening algorithm for early detection of septic shock. The model demonstrates strong performance in the data set used and provides a basis for expanding this work toward building an algorithm that is used to screen patients based on electronic medical record data in real time.

The Effect of the Intelligent Sepsis Management System on Outcomes among Patients with Sepsis and Septic Shock Diagnosed According to the Sepsis-3 Definition in the Emergency Department

We developed a novel computer program, the Intelligent Sepsis Management System, based on Sepsis-3 definitions and 2016 Surviving Sepsis Campaign guidelines and performed a quasi-experimental pre-post study to assess its effect on compliance with the Surviving Sepsis Campaign guidelines and outcomes in patients with sepsis and septic shock. During the pre-period, patients were managed with usual care. During the post-period, patients were managed using the Intelligent Sepsis Management System upon arrival at the emergency department. A total of 631 patients were enrolled (pre-period, 316; post-period, 315). The overall compliance with the Surviving Sepsis Campaign guidelines’ bundle improved (pre-period 10.8% vs. post-period 54.6%; p < 0.001). The post-period showed significantly lower 30-day mortality than the pre-period (pre-period 37.3% vs. post-period 29.5%; p = 0.037), but was not a protective factor for 30-day mortality, with an adjusted hazard ratio (95% confidence interval) of 0.75 (0.55–1.04) (p = 0.151). The associated factors for 30-day mortality were age, sequential organ failure assessment score, overall compliance, and lactate levels. The 30-day mortality was significantly lower in the compliance group than in the non-compliance group (27.2% vs. 36.5%; p = 0.002). After implementation of the Intelligent Sepsis Management System, overall compliance with the Surviving Sepsis Campaign guidelines improved and was associated with reduced 30-day mortality. However, we could not verify the causal effect of this system on 30-day mortality.

Delay Within the 3-Hour Surviving Sepsis Campaign Guideline on Mortality for Patients With Severe Sepsis and Septic Shock

OBJECTIVES

To specify when delays of specific 3-hour bundle Surviving Sepsis Campaign guideline recommendations applied to severe sepsis or septic shock become harmful and impact mortality.

DESIGN

Retrospective cohort study.

SETTING

One health system composed of six hospitals and 45 clinics in a Midwest state from January 01, 2011, to July 31, 2015.

PATIENTS

All adult patients hospitalized with billing diagnosis of severe sepsis or septic shock.

INTERVENTIONS

Four 3-hour Surviving Sepsis Campaign guideline recommendations: 1) obtain blood culture before antibiotics, 2) obtain lactate level, 3) administer broad-spectrum antibiotics, and 4) administer 30 mL/kg of crystalloid fluid for hypotension (defined as "mean arterial pressure" < 65) or lactate (> 4).

MEASUREMENTS AND MAIN RESULTS

To determine the effect of t minutes of delay in carrying out each intervention, propensity score matching of "baseline" characteristics compensated for differences in health status. The average treatment effect in the treated computed as the average difference in outcomes between those treated after shorter versus longer delay. To estimate the uncertainty associated with the average treatment effect in the treated metric and to construct 95% CIs, bootstrap estimation with 1,000 replications was performed. From 5,072 patients with severe sepsis or septic shock, 1,412 (27.8%) had in-hospital mortality. The majority of patients had the four 3-hour bundle recommendations initiated within 3 hours. The statistically significant time in minutes after which a delay increased the risk of death for each recommendation was as follows: lactate, 20.0 minutes; blood culture, 50.0 minutes; crystalloids, 100.0 minutes; and antibiotic therapy, 125.0 minutes.

CONCLUSIONS

The guideline recommendations showed that shorter delays indicates better outcomes. There was no evidence that 3 hours is safe; even very short delays adversely impact outcomes. Findings demonstrated a new approach to incorporate time t when analyzing the impact on outcomes and provide new evidence for clinical practice and research.

Implementation and Evaluation of a Web-Based Distribution System For Anesthesia Department Guidelines and Standard Operating Procedures: Qualitative Study and Content Analysis

BACKGROUND

Digitization is spreading exponentially in medical care, with improved availability of electronic devices. Guidelines and standard operating procedures (SOPs) form an important part of daily clinical routine, and adherence is associated with improved outcomes.

OBJECTIVE

This study aimed to evaluate a digital solution for the maintenance and distribution of SOPs and guidelines in 2 different anesthesiology departments in Switzerland.

METHODS

A content management system (CMS), WordPress, was set up in 2 tertiary-level hospitals within 1 year: the Department of Anesthesiology and Pain Medicine at the Kantonsspital Lucerne in Lucerne, Switzerland, as an open-access system, followed by a similar system for internal usage in the Department of Anaesthesiology and Pain Medicine of the Inselspital, Bern University Hospital, in Bern, Switzerland. We analyzed the requirements and implementation processes needed to successfully set up these systems, and we evaluated the systems' impact by analyzing content and usage.

RESULTS

The systems' generated exportable metadata, such as traffic and content. Analysis of the exported metadata showed that the Lucerne website had 269 pages managed by 44 users, with 88,124 visits per month (worldwide access possible), and the Bern website had 341 pages managed by 35 users, with 1765 visits per month (access only possible from within the institution). Creation of an open-access system resulted in third-party interest in the published guidelines and SOPs. The implementation process can be performed over the course of 1 year and setup and maintenance costs are low.

CONCLUSIONS

A CMS, such as WordPress, is a suitable solution for distributing and managing guidelines and SOPs. Content is easily accessible and is accessed frequently. Metadata from the system allow live monitoring of usage and suggest that the system be accepted and appreciated by the users. In the future, Web-based solutions could be an important tool to handle guidelines and SOPs, but further studies are needed to assess the effect of these systems.

Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs

OBJECTIVE

Sepsis remains a costly and prevalent syndrome in hospitals; however, machine learning systems can increase timely sepsis detection using electronic health records. This study validates a gradient boosted ensemble machine learning tool for sepsis detection and prediction, and compares its performance to existing methods.

MATERIALS AND METHODS

Retrospective data was drawn from databases at the University of California, San Francisco (UCSF) Medical Center and the Beth Israel Deaconess Medical Center (BIDMC). Adult patient encounters without sepsis on admission, and with at least one recording of each of six vital signs (SpO₂, heart rate, respiratory rate, temperature, systolic and diastolic blood pressure) were included. We compared the performance of the machine learning algorithm (MLA) to that of commonly used scoring systems. Area under the receiver operating characteristic (AUROC) curve was our primary measure of accuracy. MLA performance was measured at sepsis onset, and at 24 and 48 h prior to sepsis onset.

RESULTS

The MLA achieved an AUROC of 0.88, 0.84, and 0.83 for sepsis onset and 24 and 48 h prior to onset, respectively. These values were superior to those of SIRS (0.66), MEWS (0.61), SOFA (0.72), and qSOFA (0.60) at time of onset. When trained on UCSF data and tested on BIDMC data, sepsis onset AUROC was 0.89.

DISCUSSION AND CONCLUSION

The MLA predicts sepsis up to 48 h in advance and identifies sepsis onset more accurately than commonly used tools, maintaining high performance for sepsis detection when trained and tested on separate datasets.

Intensive Care Unit Telemedicine in the Era of Big Data, Artificial Intelligence, and Computer Clinical Decision Support Systems

This article examines the history of the telemedicine intensive care unit (tele-ICU), the current state of clinical decision support systems (CDSS) in the tele-ICU, applications of machine learning (ML) algorithms to critical care, and opportunities to integrate ML with tele-ICU CDSS. The enormous quantities of data generated by tele-ICU systems is a major driver in the development of the large, comprehensive, heterogeneous, and granular data sets necessary to develop generalizable ML CDSS algorithms, and deidentification of these data sets expands opportunities for ML CDSS research.

Electronic health record-based clinical decision support alert for severe sepsis: a randomised evaluation

Background

Sepsis remains the top cause of morbidity and mortality of hospitalised patients despite concerted efforts. Clinical decision support for sepsis has shown mixed results reflecting heterogeneous populations, methodologies and interventions.

Objectives

To determine whether the addition of a real-time electronic health record (EHR)-based clinical decision support alert improves adherence to treatment guidelines and clinical outcomes in hospitalised patients with suspected severe sepsis.

Design

Patient-level randomisation, single blinded.

Setting

Medical and surgical inpatient units of an academic, tertiary care medical centre.

Patients

1123 adults over the age of 18 admitted to inpatient wards (intensive care units (ICU) excluded) at an academic teaching hospital between November 2014 and March 2015.

Interventions

Patients were randomised to either usual care or the addition of an EHR-generated alert in response to a set of modified severe sepsis criteria that included vital signs, laboratory values and physician orders.

Measurements and main results

There was no significant difference between the intervention and control groups in primary outcome of the percentage of patients with new antibiotic orders at 3 hours after the alert (35% vs 37%, p=0.53). There was no difference in secondary outcomes of in-hospital mortality at 30 days, length of stay greater than 72 hours, rate of transfer to ICU within 48 hours of alert, or proportion of patients receiving at least 30 mL/kg of intravenous fluids.

Conclusions

An EHR-based severe sepsis alert did not result in a statistically significant improvement in several sepsis treatment performance measures.

Impact of Electronic Physician Order-Set on Antibiotic Ordering Time in Septic Patients in the Emergency Department

BACKGROUND

Sepsis is a serious medical condition that can lead to organ dysfunction and death. Research shows that each hour delay in antibiotic administration increases mortality. The Surviving Sepsis Campaign Bundles created standards to assist in the timely treatment of patients with suspected sepsis to improve outcomes and reduce mortality.

OBJECTIVE

This article determines if the use of an electronic physician order-set decreases time to antibiotic ordering for patients with sepsis in the emergency department (ED).

METHODS

A retrospective chart review was performed on adult patients who presented to the ED of four community hospitals from May to July 2016. Patients with severe sepsis and/or septic shock were included. Primary outcome was the difference in time to antibiotic ordering in patients whose physicians utilized the order-set versus those whose physicians did not. Secondary outcomes included differences in time to antibiotic administration, time to lactate test, hospital length of stay, and posthospitalization disposition. The institution's Quality Improvement Committee approved the project.

RESULTS

Forty-five of 123 patients (36.6%) with sepsis had physicians who used the order-set. Order-set utilization reduced the mean time to ordering antibiotics by 20 minutes (99 minutes, 95% confidence interval [CI]: 69-128 vs. 119 minutes, 95% CI: 91-147), but this finding was not statistically significant. Mean time to antibiotic administration (145 minutes, 95% CI: 108-181 vs. 182 minutes, 95% CI: 125-239) and median time to lactate tests (12 minutes, 95% CI: 0-20 vs. 19 minutes, 95% CI: 8-34), although in the direction of the hypotheses, were not significantly different.

CONCLUSION

Utilization of the order-set was associated with a potentially clinically significant, but not statistically significant, reduced time to antibiotic ordering in patients with sepsis. Electronic order-sets are a promising tool to assist hospitals with meeting the Centers for Medicare and Medicaid Services core measure.

An electronic warning system helps reduce the time to diagnosis of sepsis

OBJECTIVE

To describe the improvements of an early warning system for the identification of septic patients on the time to diagnosis, antibiotic delivery, and mortality.

METHODS

This was an observational cohort study that describes the successive improvements made over a period of 10 years using an early warning system to detect sepsis, including systematic active manual surveillance, electronic alerts via a telephonist, and alerts sent directly to the mobile devices of nurses. For all periods, after an alert was triggered, early treatment was instituted according to the institutional sepsis guidelines.

RESULTS

In total, 637 patients with sepsis were detected over the study period. The median triage-to-diagnosis time was reduced from 19:20 (9:10 - 38:15) hours to 12:40 (2:50 - 23:45) hours when the manual surveillance method was used (p = 0.14), to 2:10 (1:25 - 2:20) hours when the alert was sent automatically to the hospital telephone service (p = 0.014), and to 1:00 (0:30 - 1:10) hour when the alert was sent directly to the nurse's mobile phone (p = 0.016). The diagnosis-to-antibiotic time was reduced to 1:00 (0:55 - 1:30) hours when the alert was sent to the telephonist and to 0:45 (0:30 - 1:00) minutes when the alert was sent directly to the nurse's mobile phone (p = 0.02), with the maintenance of similar values over the following years. There was no difference in the time of treatment between survivors and non-survivors.

CONCLUSION

Electronic systems help reduce the triage-to-diagnosis time and diagnosis-to-antibiotic time in patients with sepsis.

Impact of predictive analytics based on continuous cardiorespiratory monitoring in a surgical and trauma intensive care unit

Predictive analytics monitoring, the use of patient data to provide continuous risk estimation of deterioration, is a promising new application of big data analytical techniques to the care of individual patients. We tested the hypothesis that continuous display of novel electronic risk visualization of respiratory and cardiovascular events would impact intensive care unit (ICU) patient outcomes. In an adult tertiary care surgical trauma ICU, we displayed risk estimation visualizations on a large monitor, but in the medical ICU in the same institution we did not. The risk estimates were based solely on analysis of continuous cardiorespiratory monitoring. We examined 4275 individual patient records within a 7 month time period preceding and following data display. We determined cases of septic shock, emergency intubation, hemorrhage, and death to compare rates per patient care pre-and post-implementation. Following implementation, the incidence of septic shock fell by half (p < 0.01 in a multivariate model that included age and APACHE) in the surgical trauma ICU, where the data were continuously on display, but by only 10% (p = NS) in the control Medical ICU. There were no significant changes in the other outcomes. Display of a predictive analytics monitor based on continuous cardiorespiratory monitoring was followed by a reduction in the rate of septic shock, even when controlling for age and APACHE score.

A Critical Care Clinician Survey Comparing Attitudes and Perceived Barriers to Low Tidal Volume Ventilation with Actual Practice

RATIONALE

Low-Vt ventilation lowers mortality in patients with acute respiratory distress syndrome (ARDS) but is underused. Little is known about clinician attitudes toward and perceived barriers to low-Vt ventilation use and their association with actual low-Vt ventilation use.

OBJECTIVES

The objectives of this study were to assess clinicians' attitudes toward and perceived barriers to low-Vt ventilation (Vt <6.5 ml/kg predicted body weight) in patients with ARDS, to identify differences in attitudes and perceived barriers among clinician types, and to compare attitudes toward and perceived barriers to actual low-Vt ventilation use in patients with ARDS.

METHODS

We conducted a survey of critical care physicians, nurses, and respiratory therapists at four non-ARDS Network hospitals in the Chicago region. We compared survey responses with performance in a cohort of 362 patients with ARDS.

RESULTS

Survey responses included clinician attitudes toward and perceived barriers to low-Vt ventilation use. We also measured low-Vt ventilation initiation by these clinicians in 347 patients with ARDS initiated after ARDS onset as well as correlation with clinician attitudes and perceived barriers. Of 674 clinicians surveyed, 467 (69.3%) responded. Clinicians had positive attitudes toward and perceived few process barriers to ARDS diagnosis or initiation of low-Vt ventilation. Physicians had more positive attitudes and perceived fewer barriers than nurses or respiratory therapists. However, use of low-Vt ventilation by all three clinician groups was low. For example, whereas physicians believed that 92.5% of their patients with ARDS warranted treatment with low-Vt ventilation, they initiated low-Vt ventilation for a median (interquartile range) of 7.4% (0 to 14.3%) of their eligible patients with ARDS. Clinician attitudes and perceived barriers were not correlated with low-Vt ventilation initiation.

CONCLUSIONS

Clinicians had positive attitudes toward low-Vt ventilation and perceived few barriers to using it, but attitudes and perceived process barriers were not correlated with actual low-Vt ventilation use, which was low. Implementation strategies should be focused on examining other issues, such as ARDS recognition and process solutions, to improve low-Vt ventilation use.

The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support

Background

Computerised decision support (CDS) based on trustworthy clinical guidelines is a key component of a learning healthcare system. Research shows that the effectiveness of CDS is mixed.

Multifaceted context, system, recommendation and implementation factors may potentially affect the success of CDS interventions. This paper describes the development of a checklist that is intended to support professionals to implement CDS successfully.

Methods

We developed the checklist through an iterative process that involved a systematic review of evidence and frameworks, a synthesis of the success factors identified in the review, feedback from an international expert panel that evaluated the checklist in relation to a list of desirable framework attributes, consultations with patients and healthcare consumers and pilot testing of the checklist.

Results

We screened 5347 papers and selected 71 papers with relevant information on success factors for guideline-based CDS. From the selected papers, we developed a 16-factor checklist that is divided in four domains, i.e. the CDS context, content, system and implementation domains. The panel of experts evaluated the checklist positively as an instrument that could support people implementing guideline-based CDS across a wide range of settings globally. Patients and healthcare consumers identified guideline-based CDS as an important quality improvement intervention and perceived the GUIDES checklist as a suitable and useful strategy.

Conclusions

The GUIDES checklist can support professionals in considering the factors that affect the success of CDS interventions. It may facilitate a deeper and more accurate understanding of the factors shaping CDS effectiveness. Relying on a structured approach may prevent that important factors are missed.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0772-3) contains supplementary material, which is available to authorized users.

Automated monitoring compared to standard care for the early detection of sepsis in critically ill patients

BACKGROUND

Sepsis is a life-threatening condition that is usually diagnosed when a patient has a suspected or documented infection, and meets two or more criteria for systemic inflammatory response syndrome (SIRS). The incidence of sepsis is higher among people admitted to critical care settings such as the intensive care unit (ICU) than among people in other settings. If left untreated sepsis can quickly worsen; severe sepsis has a mortality rate of 40% or higher, depending on definition. Recognition of sepsis can be challenging as it usually requires patient data to be combined from multiple unconnected sources, and interpreted correctly, which can be complex and time consuming to do. Electronic systems that are designed to connect information sources together, and automatically collate, analyse, and continuously monitor the information, as well as alerting healthcare staff when pre-determined diagnostic thresholds are met, may offer benefits by facilitating earlier recognition of sepsis and faster initiation of treatment, such as antimicrobial therapy, fluid resuscitation, inotropes, and vasopressors if appropriate. However, there is the possibility that electronic, automated systems do not offer benefits, or even cause harm. This might happen if the systems are unable to correctly detect sepsis (meaning that treatment is not started when it should be, or it is started when it shouldn't be), or healthcare staff may not respond to alerts quickly enough, or get 'alarm fatigue' especially if the alarms go off frequently or give too many false alarms.

OBJECTIVES

To evaluate whether automated systems for the early detection of sepsis can reduce the time to appropriate treatment (such as initiation of antibiotics, fluids, inotropes, and vasopressors) and improve clinical outcomes in critically ill patients in the ICU.

SEARCH METHODS

We searched CENTRAL; MEDLINE; Embase; CINAHL; ISI Web of science; and LILACS, clinicaltrials.gov, and the World Health Organization trials portal. We searched all databases from their date of inception to 18 September 2017, with no restriction on country or language of publication.

SELECTION CRITERIA

We included randomized controlled trials (RCTs) that compared automated sepsis-monitoring systems to standard care (such as paper-based systems) in participants of any age admitted to intensive or critical care units for critical illness. We defined an automated system as any process capable of screening patient records or data (one or more systems) automatically at intervals for markers or characteristics that are indicative of sepsis. We defined critical illness as including, but not limited to postsurgery, trauma, stroke, myocardial infarction, arrhythmia, burns, and hypovolaemic or haemorrhagic shock. We excluded non-randomized studies, quasi-randomized studies, and cross-over studies . We also excluded studies including people already diagnosed with sepsis.

DATA COLLECTION AND ANALYSIS

We used the standard methodological procedures expected by Cochrane. Our primary outcomes were: time to initiation of antimicrobial therapy; time to initiation of fluid resuscitation; and 30-day mortality. Secondary outcomes included: length of stay in ICU; failed detection of sepsis; and quality of life. We used GRADE to assess the quality of evidence for each outcome.

MAIN RESULTS

We included three RCTs in this review. It was unclear if the RCTs were three separate studies involving 1199 participants in total, or if they were reports from the same study involving fewer participants. We decided to treat the studies separately, as we were unable to make contact with the study authors to clarify.All three RCTs are of very low study quality because of issues with unclear randomization methods, allocation concealment and uncertainty of effect size. Some of the studies were reported as abstracts only and contained limited data, which prevented meaningful analysis and assessment of potential biases.The studies included participants who all received automated electronic monitoring during their hospital stay. Participants were randomized to an intervention group (automated alerts sent from the system) or to usual care (no automated alerts sent from the system).Evidence from all three studies reported 'Time to initiation of antimicrobial therapy'. We were unable to pool the data, but the largest study involving 680 participants reported median time to initiation of antimicrobial therapy in the intervention group of 5.6 hours (interquartile range (IQR) 2.3 to 19.7) in the intervention group (n = not stated) and 7.8 hours (IQR 2.5 to 33.1) in the control group (n = not stated).No studies reported 'Time to initiation of fluid resuscitation' or the adverse event 'Mortality at 30 days'. However very low-quality evidence was available where mortality was reported at other time points. One study involving 77 participants reported 14-day mortality of 20% in the intervention group and 21% in the control group (numerator and denominator not stated). One study involving 442 participants reported mortality at 28 days, or discharge was 14% in the intervention group and 10% in the control group (numerator and denominator not reported). Sample sizes were not reported adequately for these outcomes and so we could not estimate confidence intervals.Very low-quality evidence from one study involving 442 participants reported 'Length of stay in ICU'. Median length of stay was 3.0 days in the intervention group (IQR = 2.0 to 5.0), and 3.0 days (IQR 2.0 to 4.0 in the control).Very low-quality evidence from one study involving at least 442 participants reported the adverse effect 'Failed detection of sepsis'. Data were only reported for failed detection of sepsis in two participants and it wasn't clear which group(s) this outcome occurred in.No studies reported 'Quality of life'.

AUTHORS' CONCLUSIONS

It is unclear what effect automated systems for monitoring sepsis have on any of the outcomes included in this review. Very low-quality evidence is only available on automated alerts, which is only one component of automated monitoring systems. It is uncertain whether such systems can replace regular, careful review of the patient's condition by experienced healthcare staff.

Emerging Technologies for Molecular Diagnosis of Sepsis

Rapid and accurate profiling of infection-causing pathogens remains a significant challenge in modern health care. Despite advances in molecular diagnostic techniques, blood culture analysis remains the gold standard for diagnosing sepsis. However, this method is too slow and cumbersome to significantly influence the initial management of patients. The swift initiation of precise and targeted antibiotic therapies depends on the ability of a sepsis diagnostic test to capture clinically relevant organisms along with antimicrobial resistance within 1 to 3 h. The administration of appropriate, narrow-spectrum antibiotics demands that such a test be extremely sensitive with a high negative predictive value. In addition, it should utilize small sample volumes and detect polymicrobial infections and contaminants. All of this must be accomplished with a platform that is easily integrated into the clinical workflow. In this review, we outline the limitations of routine blood culture testing and discuss how emerging sepsis technologies are converging on the characteristics of the ideal sepsis diagnostic test. We include seven molecular technologies that have been validated on clinical blood specimens or mock samples using human blood. In addition, we discuss advances in machine learning technologies that use electronic medical record data to provide contextual evaluation support for clinical decision-making.

Analysis of a new best-practice advisory on time to initiation of antibiotics in surgical intensive care unit patients with septic shock

Background

Early administration of antibiotics in septic shock is associated with decreased mortality. Promptly identifying sepsis and eliciting a response are necessary to reduce time to antibiotic administration.

Methods

A best-practice advisory was introduced in the surgical intensive care unit to identify patients with septic shock and promote timely action. The best-practice advisory is triggered by blood culture orders and vasopressor administration within 24 h. The nurse or provider who triggers the alert may send an automatic notification to the intensive care unit resident, clinical pharmacist, and charge nurse, prompting bedside response and closer evaluation. Patients who met best-practice advisory criteria in the surgical intensive care unit from May 2016 through March 2017 were included. Outcomes included changes in antibiotics within 24 h, response to best-practice advisory, and time-to-antibiotics. Time-to-antibiotics was compared between a retrospective pre-intervention period and a six-month prospective post-intervention period defined by launch of the new best-practice advisory in September 2016. Data were analyzed by chi square, Mann-Whitney U, and Kruskal-Wallis.

Results

During the first six months of best-practice advisory implementation, 191 alerts were triggered by 97 unique patients. Alert notification was transmitted in 79 best-practice advisories (41%), with pharmacist bedside response in 53 (67%). New antibiotics were started within 24 h following 83 best-practice advisories (43%). There was a trend toward decreased time-to-antibiotics following implementation of the best-practice advisory (7.4 vs. 4.2 h, p = 0.057). Compared to the entire cohort, time-to-antibiotics was shorter when the team was notified and when a pharmacist responded to the bedside (4.2 vs. 1.6 vs. 1.2 hours).

Conclusions

A new best-practice advisory has been effective at eliciting a rapid response and reducing the time-to-antibiotics in surgical intensive care unit patients with septic shock. Team notification and pharmacist response are associated with decreased time-to-antibiotics.

Health Information Technology Continues to Show Positive Effect on Medical Outcomes: Systematic Review

Background

Health information technology (HIT) has been introduced into the health care industry since the 1960s when mainframes assisted with financial transactions, but questions remained about HIT’s contribution to medical outcomes. Several systematic reviews since the 1990s have focused on this relationship. This review updates the literature.

Objective

The purpose of this review was to analyze the current literature for the impact of HIT on medical outcomes. We hypothesized that there is a positive association between the adoption of HIT and medical outcomes.

Methods

We queried the Cumulative Index of Nursing and Allied Health Literature (CINAHL) and Medical Literature Analysis and Retrieval System Online (MEDLINE) by PubMed databases for peer-reviewed publications in the last 5 years that defined an HIT intervention and an effect on medical outcomes in terms of efficiency or effectiveness. We structured the review from the Primary Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), and we conducted the review in accordance with the Assessment for Multiple Systematic Reviews (AMSTAR).

Results

We narrowed our search from 3636 papers to 37 for final analysis. At least one improved medical outcome as a result of HIT adoption was identified in 81% (25/37) of research studies that met inclusion criteria, thus strongly supporting our hypothesis. No statistical difference in outcomes was identified as a result of HIT in 19% of included studies. Twelve categories of HIT and three categories of outcomes occurred 38 and 65 times, respectively.

Conclusions

A strong majority of the literature shows positive effects of HIT on the effectiveness of medical outcomes, which positively supports efforts that prepare for stage 3 of meaningful use. This aligns with previous reviews in other time frames.

Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial

Introduction

Several methods have been developed to electronically monitor patients for severe sepsis, but few provide predictive capabilities to enable early intervention; furthermore, no severe sepsis prediction systems have been previously validated in a randomised study. We tested the use of a machine learning-based severe sepsis prediction system for reductions in average length of stay and in-hospital mortality rate.

Methods

We conducted a randomised controlled clinical trial at two medical-surgical intensive care units at the University of California, San Francisco Medical Center, evaluating the primary outcome of average length of stay, and secondary outcome of in-hospital mortality rate from December 2016 to February 2017. Adult patients (18+) admitted to participating units were eligible for this factorial, open-label study. Enrolled patients were assigned to a trial arm by a random allocation sequence. In the control group, only the current severe sepsis detector was used; in the experimental group, the machine learning algorithm (MLA) was also used. On receiving an alert, the care team evaluated the patient and initiated the severe sepsis bundle, if appropriate. Although participants were randomly assigned to a trial arm, group assignments were automatically revealed for any patients who received MLA alerts.

Results

Outcomes from 75 patients in the control and 67 patients in the experimental group were analysed. Average length of stay decreased from 13.0 days in the control to 10.3 days in the experimental group (p=0.042). In-hospital mortality decreased by 12.4 percentage points when using the MLA (p=0.018), a relative reduction of 58.0%. No adverse events were reported during this trial.

Conclusion

The MLA was associated with improved patient outcomes. This is the first randomised controlled trial of a sepsis surveillance system to demonstrate statistically significant differences in length of stay and in-hospital mortality.

Trial registration

NCT03015454.

Impact of an electronic sepsis initiative on antibiotic use and health care facility-onset Clostridium difficile infection rates

BACKGROUND

Although integrated, electronic sepsis screening and treatment protocols are thought to improve patient outcomes, less is known about their unintended consequences. We aimed to determine if the introduction of a sepsis initiative coincided with increases in broad-spectrum antibiotic use and health care facility-onset (HCFO) Clostridium difficile infection (CDI) rates.

METHODS

We used interrupted time series data from a large, tertiary, urban academic medical center including all adult inpatients on 4 medicine wards (June 2011-July 2014). The main exposure was implementation of the sepsis screening program; the main outcomes were the use of broad-spectrum antibiotics (including 3 that were part of an order set designed for the sepsis initiative) and HCFO CDI rates. Segmented regression analyses compared outcomes in 3 time segments: before (11 months), during (14 months), and after (12 months) implementation of a sepsis initiative.

RESULTS

Antibiotic use and HFCO CDI rates increased during the period of implementation and the period after implementation compared with baseline; these increases were highest in the period after implementation (level change, 50.4 days of therapy per 1,000 patient days for overall antibiotic use and 10.8 HCFO CDIs per 10,000 patient days; P < .05). Remarkably, the main drivers of overall antibiotic use were not those included in the sepsis order set.

CONCLUSIONS

The implementation of an electronic sepsis screening and treatment protocol coincided with increased broad-spectrum antibiotic use and HCFO CDIs. Because these protocols are increasingly used, further study of their unintended consequences is warranted.

A sirs-based automated alarm system for the diagnosis of sepsis after burn injury

The present study examined the effectiveness of an automated systemic inflammatory response syndrome (SIRS)-based alarm system for the early detection of sepsis in adult burn patients. In addition, the relevance of the sepsis criteria from the American Burn Association (ABA) and the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) for this patient group was determined. In this prospective observational study, we included 41 consecutive patients who were admitted to our burn ICU within a one-year period. SIRS alarms were calculated for these patients according to predefined criteria using the PREDEC ALARM automated alarm system and alarms generated according to ABA criteria and daily Sequential Organ Failure Assessment (SOFA) results. The alarms were not visible to the treating doctors and nurses. The alarms identified were related to episodes of sepsis with positive blood cultures. Twenty-one sepsis events were recorded in 41 patients. The receiver operating characteristic (ROC) curve revealed an area under the curve (AUC) of 0.72 for alarms using 2 SIRS criteria, 0.77 for 3 SIRS criteria, and 0.61 for 4 SIRS criteria. AUC values of 0.53 and 0.59 were calculated for the ABA criteria and Sepsis-3 definition, respectively. An alarm system using 3 SIRS criteria proved to be suitable for the early detection of sepsis in burn patients. The frequent number of false alarms requires further refinement of the alarm system. Based on the results of our study, the ABA criteria and the Sepsis-3 definition provide no advantages over the SIRS criteria in the early diagnosis of sepsis after burn injury.